T-SQL (SS2K8) :: Scrub Data In 5 Minute Intervals

Nov 2, 2015

I have a situation where I have table with over a billion records and needs to be scrubbed. Table does have a field with date time timestamp. I have been deleting rows from the table using the script below which basically provides me delete statements by date for records older than 90 days.

But now on each day row count is over 30 million rows and it takes forever to delete by date and transaction log becomes humongous.

So I would like to scrub it in 5 minute intervals instead of daily for records older than 90 days. Even in 5 minute intervals the record count tends to be around a million. This will keep the delete slice small enough to not a gigantic transaction log.

declare @startdate Datetime
declare @enddate Datetime
set @startdate = getdate()-480
set @enddate = getdate()-90

--set @vStart = select convert(varchar,@startdate, 102)

print @startdate
print @enddate

WHILE (@startdate < @enddate)
BEGIN
print 'delete from vending where DetectedDate < ''' + CONVERT(varchar(10), @startdate, 101) +''''
set @startdate = @startdate+1
END

I am hoping to modify the script above to produce a script with statements like this for a window between last 90 and 120 days:

delete from vending where DetectedDate <'6/15/2015 8:25:00 PM'
go
delete from vending where DetectedDate <'6/15/2015 8:30:00 PM'
go
delete from vending where DetectedDate <'6/15/2015 8:35:00 PM'
go

View 2 Replies


ADVERTISEMENT

Aggregating Financial Data To 1 Minute Intervals

Aug 19, 2006

I'm trying to create a query to return Open, Close, Max and Min Price for each 1 minute interval. Source data has two fields - Price, and Datestamp at 5 second intervals.

I can calculate the Max and Min (below) and set the datestamp to the middle of the interval, but get stuck on how to also return the Open and Close price for each interval.

SELECT MAX(price) AS MaxPrice, MIN(price) AS MinPrice,
DATEADD(ss, 30, DATEADD(n,DATEDIFF n, '1/1/2006', DateStamp),'1/1/2006')) AS DateStamp
FROM MasterData
GROUP BY DATEDIFF(n, '1/1/2006',DateStamp)

Any ideas?

thanks in advance.

View 1 Replies View Related

SQL Server 2012 :: Group Data Into 15 Minute Intervals

Apr 8, 2015

I want to group my data into 15 minute interval . Below is my sample data and desired result set.

Create TABLE #HalfHourlyIntervals
(
OrderDate DATETIME,
IRevenue FLOAT,
TRevenue FLOAT
)

[Code] ....

View 9 Replies View Related

Going From 15 Minute Intervals To 1 Min Intervals?

Jun 3, 2015

I have two fields ID and Log data and log data is a 96 character long string of numbers representing 15 minute intervals from midnight to midnight.

I need to convert these 96 characters to a full 1440 characters which would mean taking each of the 96 characters one by one and making 1 character into 15.

I had Vb macro to do the conversion but now it's broken and I can't fix it. Getting it done in SQL would solve a lot of problems.

I then go from the 1440 fields and do log analysis like total time doing a specific activity but my query is dependent on having all 1440 characters.

View 3 Replies View Related

Generate 5 Minute Intervals

Jan 28, 2008

Hello
Probably a very simple problem, but im stumped. I have a table which gives the start-time and end-time of an employees work day. I want to create a view which contains a line of data for each 5 minute period worked. Please help.

View 3 Replies View Related

Need To Only Return Rows In 5 Minute Intervals

Mar 6, 2012

I have collected perfmon data that is in every 15 seconds. I need to run a query that will only retrun rows that are 5 minutes from the last row starting at a specific date/time.

Here is the current query

Select
DisplayString,
MachineName,
ObjectName,
CounterName,
InstanceName,

[Code] ......

"CounterDateTime" is in every 15 seconds. So starting from '2012-03-02 11:59:00' I need only rows for every 5 minutes after that.

View 14 Replies View Related

T-SQL (SS2K8) :: Merging Intervals With Identical Data

Jul 10, 2014

I'm having issues building a cte sql statement for merging intervals. I have a table with data as follows:

declare @table table
(
startpoint int,
stoppoint int,
value int
[Code] ....

The resulting query returns the rows in the table, sorted by startpoint:

startpoint stoppoint value
----------- ----------- -----------
0 10 1
10 15 1
15 25 2
25 30 2
30 40 2
40 55 3
55 60 3
60 80 2

I'm looking for a merge cte that returns consecutive intervals with the same value, as follows:

startpoint stoppoint value
----------- ----------- -----------
0 15 1
15 40 2
40 60 3
60 80 2

View 3 Replies View Related

T-SQL (SS2K8) :: Populating Data By Date Intervals

Mar 6, 2015

Table Name: EmployeeDetails
Columns: EMpID - Date - WorkedHours

For each day I get details of number of hours worked by each employee in this table.

Now my HR wants a report with such columns

empid - Week - Month - Qtr

So, week will have Sum of hours worked by employee in that week
Month will have Sum of hours worked by employee in that Month
Qtr will have Sum of hours worked by employee in that Qtr

View 6 Replies View Related

T-SQL (SS2K8) :: Difference In Seconds To Round To Minute?

May 20, 2014

I'm trying to get the difference between the two times in minutes with the seconds rounded up or down

As an example the difference in the times:

1900-01-01 09:27:49.000, 1900-01-01 09:32:28.000 is 279 seconds which is 4.65 minutes,

using

select
cast(round(datediff(second, convert(datetime, '1900-01-01 09:27:49.000'), convert(datetime, '1900-01-01 09:32:28.000')) / 60, 2) as numeric(18,2))

I am receiving the value 4.00 instead of 5 which I would like 4.65 to round to. How can I get the difference in seconds to round to the minute ?

View 5 Replies View Related

T-SQL (SS2K8) :: Table With 3 Million Plus Records Taking Half A Minute?

Aug 6, 2015

I have a table that I need to do some computations on all the data but first I need to remove the duplicate records and insert the results into a destination table. Here's the example below. My table has 3.1 million rows. I have tried using the DISTINCT and the GROUP BY but both ways to select the data takes about half a minute to run. I'm wondering if there is a way to increase performance. Users are ok with this time since the process runs overnight but improving it won't hurt. I do have a clustered index on these fields but that doesn't seem to improve any.

SELECTDateYear ,
DateMonth ,
Nbr ,
Nbr1 ,
Nbr2 ,
Datafield1 ,
Datafield2,

[code].....

View 7 Replies View Related

Creating Stored Procs That Need To Continusiouly Append To A New Table (this Is To Scrub Data That Is Imported Into DB).

Jan 9, 2005

I have 1 table with a huge amount of data that I recive from someone else in a flat file format. I want to be able to filter through that data and scrub it and find out the good data and bad data from it.

I'm scrubbing the data using different stored procs that i've created and through a web interface that the user can pick which records they wish to create.

If I were to create a new table for clean records, what is the syntax to keep Appending to that table through the data that i'm obtainig via the stored procs that i've created.

Any thoughts or suggestions are greatly appriciated in advance

Thanks again in advance
RB

View 1 Replies View Related

How To Exract SQL Minute Data To Houly Data?

Aug 1, 2006

Hi

I have a SQL table that has data filled with million records and the date is in minutes it looks like :-








RowDateTime
Meter 1
Meter 2
Meter 3

25/05/2006 02:49:00
1220
450
489

25/05/2006 02:50:00
1223
470
500

25/05/2006 02:51:00
1227
490
511

25/05/2006 02:52:00
1230
510
522

25/05/2006 02:53:00
1233.5
530
533

25/05/2006 02:54:00
1236.9
550
544

25/05/2006 02:55:00
1240.3
570
555

25/05/2006 02:56:00
1243.7
590
566

I want to make a query to the above table and convert the data to houlry by summing Meter1,Meter 2 and Meter 3 to be the average. I want to import all the hourly data to a new table that will look like :-








RowDateTime
Meter 1
Meter 2
Meter 3

25/05/2006 03:00:00
9854.4
4160
527.5

25/05/2006 04:00:00
8634.4
3710
533

25/05/2006 05:00:00
7411.4
3240
538.5

Your help will be highly appreciated.

View 3 Replies View Related

Return Data Only If Time Is Equal By A Minute

Apr 9, 2015

Just wondering what is the best time to ensure that we only return data when the datetime field is the same when compared between two datetimes within a minute difference.

As in the following should return the data:

'2015-04-09 09:00:20' compared to '2015-04-09 09:00:50'

And the following should not return the data:

'2015-04-09 09:01:20' compared to '2015-04-09 09:00:50'

The problem, is that I'm merging data from three different result sets, which they all have data for every minute, however, the timestamp can be different by seconds or milliseconds.

So, I'm only interested to return the data when the two fields that I'm comparing are equal within a minute. I need to ignore seconds and milliseconds.

View 4 Replies View Related

How Do You Validate And Scrub Using DTS?

Feb 11, 2004

When I use the DTS GUI and insert a "Bulk Insert Task" the main tab says:

"Import text files into SQL Server. You cannot validate, scrub, or transform data using this task".

So my question is, what shoud you use to validate and scrub?

In particular I have fixed-format text file with some occasional bad records (e.g. wrong length, empty record). What should I be using? If you suggest vbscript could you show me some examples? I'm new to vbscript.

Thanks!

View 2 Replies View Related

SQL Server Scrub Learning Oracle

Jun 11, 2004

Anybody with SQL Server/Oracle experience able to recommend a good book for learning Oracle?

View 4 Replies View Related

Date Intervals

Jan 30, 2006

Is that like some interval function for dates, like i want to group my data in intervals of 15 minutes and 30 minutes. Is there such a function in T-SQL

View 4 Replies View Related

X Axis Intervals

May 4, 2008



Hi All,


I am working on a line chart. The variable on the X- axis is DateTime. The requirement is to have the values displayed with an interval of 4 hours (or mabe 5 or 6 hours - basically every regular intervals). I am using a list control to Subreports to show around 10 reports.

HOW can I do this? The Major and MInor intervals only help in separating which samples will be displayed. But we cannot configure for regular intervals.
Please help.

Thanks in advance,


Manoj Deshpande.

View 1 Replies View Related

How To Get Predicion For Monthly Intervals?

Jan 14, 2007

hi,

I am using timeseries algorithm.I am training my model like the following

Date StudId Perf

5/1/2005 001 99

5/10/005 001 97.6

6/1/2005 001 94

6/10/2005 001 99

6/30/005 001 96

10/1/2005 001 100

Like that.

I need prediction Output like following

Date StudId Perf

10/1/2005 001 99

11/10/005 001 97.6

12/1/2005 001 94

1/10/2006 001 99

... how to write prediction query for this.

Thanks

Karthik.

View 6 Replies View Related

Calculate Time Worked By 15 Min Intervals.

Dec 19, 2007

Ok, I know that there is a very smart programmer out there that can resovle my issue.

I am trying to calculate time worked by 15 minute intervals.

Example:
Emp 1 started work at 13:00:00 and worked 183 minutes
Emp 2 started work at 17:15:00 and worked 150 minutes
Emp 3 started work at 08:30:00 and worked 17 minutes

I need to show the following results:

time employee #of_min_worked
----------------------------------------------
08:30:00 3 15
08:45:00 3 2
09:00:00
08:30:00
08:45:00
09:00:00
09:15:00
09:30:00
09:45:00
10:00:00
10:15:00
10:30:00
10:45:00
11:00:00
11:15:00
11:30:00
11:45:00
12:00:00
12:15:00
12:30:00
12:45:00
13:00:00 1 15
13:15:00 1 15
13:30:00 1 15
13:45:00 1 15
14:00:00 1 15
14:15:00 1 15
14:30:00 1 15
14:45:00 1 15
15:00:00 1 15
15:15:00 1 15
15:30:00 1 15
15:45:00 1 15
16:00:00 1 3
16:15:00
16:30:00
16:45:00
17:00:00
17:15:00 2 15
17:30:00 2 15
17:45:00 2 15
18:00:00 2 15
18:15:00 2 15
18:30:00 2 15
18:45:00 2 15
19:00:00 2 15
19:15:00 2 15
19:30:00 2 15
19:45:00
20:00:00
20:15:00
20:30:00
20:45:00
21:00:00
21:15:00
21:30:00
21:45:00
22:00:00
22:15:00
22:30:00
22:45:00
23:00:00
23:15:00
23:30:00
23:45:00

View 1 Replies View Related

Subscribing Report For Time Intervals

Mar 5, 2008

Hi there,

Is there anyway of subscribing to a report to run for every 1 hour between the times off 9am to 5pm only ?

I cant seem to find the ability to set time intervals anywhere

View 1 Replies View Related

Chart Y-axis Intervals Not Using Formula?

Jul 17, 2007

i put the formula day(Parameters!Parameter1.Value) into my y-axis's major gridlines interval value, and it doesn't obey the value i put in.

is this a bug or another "feature"?



when i just type in "31" is works as expected.



is there a work around for this?

View 4 Replies View Related

Convert To Irregular Time Intervals

Feb 9, 2008

I have a client that collects data from a manufacturing facility a one minute intervals. I already have sql statments to convert the 1 minute data to other timeframes (e.g. 30 min, 60 min, daily). However, now the client wants to look at data converted to irregular time intervals. For example, instead of looking at the first, second, third, etc. 60 minutes of a day, they wish to see data grouped irregularly: first 30 minutes, next 1 hr & 45 min, next 2 hours, next 1 hr & 30 min, etc. These irregular intervals could change; they may later want to look at the first hr, next 2 1/2 hrs, next 1/2 hour, etc.; or whatever strikes their fancy.

So far, all I've come up with is run one query for each desired time session and then do a join on all the resulting tables. Anybody have a better idea on how to do this?

View 1 Replies View Related

Want To Run A Stored Procedure At Regular Intervals Automatically

Nov 8, 2006

Hi,
I want to run a stored procedure automatically at regular intervals.The time duration is given by me as input details.for example i will
give the date and i should send email to some recipients just one day before the given date.I want this to be done in sql stored procedures.
 
Can anyone help?
Thanks in advance,
Nithya

View 7 Replies View Related

SQL Server Admin 2014 :: How To Schedule A Job To Run At Different Intervals

Jul 31, 2014

Is there a way to schedule a sql job to run at different intervals

For eg:
The job should run at
7:00 Am
8:00 AM
and then at 10:00 Am

View 3 Replies View Related

SQL Server 2008 :: Time Differences Between Different Intervals?

Apr 30, 2015

how can I get the time differences between them.Let's say , a person who click on break @ 12:00 PM and he is back and select I am back option @ 12:15 the total break time is 15 minutes. However, can I display this difference of break time.

View 4 Replies View Related

Adding Half Hour Intervals To Time

Oct 6, 2014

I have data that looks like the following.

ID Date Time Length Interval_Num
1 10/11/2014 9:00 420 14

Basically, length represents the # of minutes a person is scheduled for. We have 30 minute intervals. Interval_Num = (Length/30). I need the data to show each interval. For the above example it should only be 14 intervals.For the above example the solution should look like....

ID Date Time Length
1 10/11/2014 9:00 30
1 10/11/2014 9:30 30
1 10/11/2014 10:00 30
1 10/11/2014 10:30 30

for 14 intervals up until 16:00

View 4 Replies View Related

Finding Minutes In Datetime Intervals [SQL Server2005]

May 29, 2008

[SQL Server 2005]


In a report there are following in parameters: from date, to date, from time and to time.

I need to match these search criteria€™s against start time and stop time in assignments.
The trick: How to write SQL that meets all different cases that can arise.

The query need's to traverse through every assignment,
and see if its start and stop time is in the interval given by the user.

It should then return the minutes from the assignment that is within the interval.



I'm thinking of packaging the minute "extracting" logic in a Scalar-valued function.

It should then take the in parameters from the user, and start time and stop time from the assignment as arguments.

To consider: Crossing midnight.



E.g. the user puts in

From date: 2008-05-24

From time: 22:00:00

To date: 2008-05-26

To time: 02:00:00



Since the user wants to evaluate a time span that crosses midnight,

I'll have to look in to time spans like:

2008-05-24 22:00:00 - 2008-05-24 23:59:59

2008-05-25 00:00:00 - 2008-05-25 02:00:00

2008-05-25 22:00:00 - 2008-05-25 23:59:59

2008-05-26 00:00:00 - 2008-05-26 02:00:00



Furthermore the start- and stop time can be partially in the above interval.

E.g. an assignment starts 2008-05-24 21:46:11 and stops 2008-05-24 22:36:05.

This would then yield from the time spans above 36 minutes.

The combinations seem never-ending€¦

View 9 Replies View Related

MsAccess To Sqlserver Transfering At Regular Intervals ?

Nov 15, 2006

I am having Access database in my local system . Values in the access database will be updated for every 5 min .

I am also having Sqlserver database online with the same structure database of Access in my local system .

What i want is my Access database values must be updated in my online sqlserver at every 5 min automatically .

View 1 Replies View Related

Caching &&< 1 Minute

Mar 7, 2007

Is it possible expire a report cache after less than one minute? I'm looking for a way to only have a report hit the database once every 10 seconds, no matter how many people are hitting it. Thanks.

View 1 Replies View Related

Getting Every Other 5-minute Average Value...

Feb 1, 2008

Hi,

Here is a part of result set.
It is of every minute value.

How can I get every other 5-minute average value?

id datetime value
------------------- ----------------------------- --------

0xC00302FD 2008-01-31 18:36:00 0.104
0xC00302FD 2008-01-31 18:37:00 0.104
0xC00302FD 2008-01-31 18:38:00 0.104
0xC00302FD 2008-01-31 18:39:00 0.104
0xC00302FD 2008-01-31 18:40:00 0.104
0xC00302FD 2008-01-31 18:41:00 0.104
0xC00302FD 2008-01-31 18:42:00 0.104
...

...
...

View 1 Replies View Related

Find Average Marks At Various Intervals Of Serial Number

Nov 24, 2013

I have this table of Marks as shown below. All I need is to find the average Marks at various intervals of S.no. That is I need averages at every 3rd S.No. as shown.

S.No. Marks
1 ------ 5
2 ------ 5
3 ------ 6 1st Average Value here (16/3)
4 ------ 5
5 ------ 6
6 ------ 7 2nd Average Value here (18/3)
7 ------ 7
8 ------ 7
9 ------ 8 3rd Average Value here (22/3)
10 ----- 8
11 ----- 9
12 ----- 8 4th Average Value here (26/3)

So basically I need a new table which will have 4 average values for the table above. Of-course the table can be much bigger and the average values can be at any nth value of S.No.

View 12 Replies View Related

How To Calculate Transactions Per Minute

Jan 17, 2002

How do i calculate the Transactions Per Minute (TPM). Do i need to use the Performance Monitor or Profiler. Let me know How do i calcualte.

I would like to have 24,000 inserts in One minute Per data migration. Is 24,000 will be Transaction Per Minute.

Thanks in Advance.

SS

View 1 Replies View Related

Minute Count Query

Aug 22, 2007

I need a query that gives me the sum of every rows (time column) with lower 'rownr'

the result:
rownrtimetimesum
1100
21010
31020
41030
51040
61050
71060
81070

current table looks like this:
rownrtime
110
210
310
410
510
610
710
810

and i want the 'timesum' column to be in format hhhh:mm
current format is rownr=int, time=datetime

thx for all help

//Mr

View 14 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved