Slow UPDATE - Running For 16 Hours And Counting...
Jun 7, 2004
The following basic UPDATE SQL statement has been running for 16 hours and counting. I need to get this done ASAP.
UPDATE Recipients SET UndeliverableTime = getdate()
FROM Recipients
INNER JOIN Domains ON (Recipients.DomainID = Domains.ID)
INNER JOIN Undeliverables ON (
Recipients.UserName + '@' + Domains.Domain =
Undeliverables.EmailAddress)
Is there any way I can see how far this has gone and how long it will take to finish? Will this take another hour to finish or another week?
Both tables (Recipients and Undeliverables) have approximately 80 million records
I did a nearly identical operation with another table that had only 7 million records and it took 10.5 hours. I hope this doesn't scale linearly to 115 hours.
I am tempted to cancel, retune, and rerun but that may be trigger a really expensive rollback operation that could take days. Any ideas?
I'm having what you might call an optimisation issue - but I'm also not sure if my approach to this problem is right. I've spent the whole day trying various methods but none seem to be performing as admirably as I'd hoped.
Basically, here's the scenario:
1. Log files are being inserted into a SQL table via Log Parser 2.2. 2. I have a lookup table of ip address ranges to countries and other geographic data. 3. Once the log row has been inserted from Log Parser, I want to update the same log table with data from the lookup table.
The main problem seems to be the updating (the initial insert from Log Parser is lightning quick).
I initially wrote a trigger on AFTER INSERT on the log table that converted the actual user's IP address into IPNumber format using a simple algorithm, then pumped the IPNumber into a quick select query which retrieved the geolocation data. Then, in the same trigger, there was an update statement which basically said:
update dbo.Logs set [Country] = @Country where [IpNumber] = @IpNumber and [Country] = Null
Therein lies the problem. The update statement slows everything down to almost a standstill and also seems to obtain a lock on the table.
Critical factors:
1. The log table must remain readable. 2. The query must execute in seconds -- not over half hour :)
I've experimented with various combinations of indexing, so far to no avail.
am using FOR UPDATE triggers to audit a table that has 67 fields. Myproblem is that this slows down the system significantly. I havenarrowed down the problem to the size (Lines of code) that need to becompiled after the trigger has been fired. There is about 67 IFUpdate(fieldName) inside the trigger and a not very complex selectstatement inside the if followed by an insert to the audit table. WhenI leave only a few IF-s in the trigger and comment the rest of thecode performance increased dramatically. It seems like it is checkingevery single UPdate() statement. Assuming that this was slowing downdue to doing a select for every update i tried to do to seperateselects in the beginning from Deleted and Inserted and assigningcolumns name to specific variables and instead of doingif Update(fieldName) i didif @DelFieldName <> @InsFieldNamebeginINSERT INTO AUDITSELECT WHAT I NEEDENDThis did not improve performance. If you have any ideas on how to getaround this issue please let me know.Below is an example of what my triggers look like.------------------------------------Trigger 1 -- this was my original designCREATE trigger1 on TableFOR UPDATEASif update(field1)begininsert into AuditSELECT What I needENDif update(field2)begininsert into AuditSELECT What I needEND...... Repeated about 65 more timesif update(field67)insert into AuditSELECT What I needEND---------------------------------------------------------------------------Trigger 2 -- this is what i tried but did not improve performanceCREATE trigger2 on TableFOR UPDATEASDeclare @DelField1 varcharDeclare @DelField2 varchar....Declare @DelField67 varcharSelect@DelField1 = Field1,@DelField2 = Field2,....@DelField67 = Field67From DeletedDeclare @InsField1 varcharDeclare @InsField2 varchar....Declare @InsField67 varcharSelect@insField1 = Field1,@insField2 = Field2,....@InsField67 = Field67From Inserted-- I do not do if Update() but instead compare variablesif @DelField1 <> InsField1beginInsert into AUDITSELECT what I needendif @DelField2 <> InsField2beginInsert into AUDITSELECT what I needend............if @DelField67 <> InsField67beginInsert into AUDITSELECT what I needend----------------------------------------------IF you have any idea how to optimize this please let me know. Anyinput is greatly appreciated. I do not have a problem with triggersdoing what they are supposed to, they are very slow this is myconcern. The reason I gave you two examples is because i suspect ithas something to do with the enormouse amount of code inside thetrigger. both examples perform about the same whether i use the twohuge selects from the Inserted and Deleted or not.Thanks,Gent
I'm currently running a reorganize on a large table of ~60 gig. I started the reorganize when I noticed the fragmentation was 97.95%!!!
Well the reorganize has been running for 25 hours now and apparently bulk inserts cannot happen during this time since my SSIS package just bombed trying to prepare for bulk insert.
Anyway, my question is can I cancel this reorg? I didn't start the reorg through the query analyzer. I saw this cute little reorganize button when I right clicked my indexes, properties, fragmentation in the SQL 2005 management studio. I clicked it and then clicked ok.
I know I should have done an alter rebuild but I wasn't comfortable with the process and did the one click solution that is now killing me.
What happens if I go into task manager and shut down the process? Am I risking a serious side effect of corruption or will SQL just stop so I can rebuild the index properly?
I'm connecting to my LOCAL MSSQL 2005 database using Management Studio. I find that after a few hours of use, Management Studio's connection to the database becomes extremely slow. For example, if I right click on a table and choose Open Table, it will take at least 30 secs to open the table. Then if I make changes to a record and move the cursor to the next row, it will take another 30 secs to update the record.
I don't think it's anything to do with indexes, cos my table is very small (less than 20 records).
I'm not running any batch jobs on my local database. In fact, my SQL Agent is turned off.
I notice this does not happen when I connect to my REMOTE database, which is hosted on a commercial web host.
What could be wrong here? Has anyone experienced this before?
I have an update query running which to just now has been running for 22 hours running on two tables 1 a lookuptable that has just been created within the batch the other a denormalised table for doing data analysis on
the query thats causing teh problem is
--//////////////////////////////////// this is the one thats running
Print 'Update Provider 04-05 EmAdmsCount12mths : ' + CAST(GETDATE() AS varchar) GO Update Provider_APC_2004_05 set EmAdmsCount12mths = (Select COUNT(*)-1 from Combined_Admissions where ((Combined_Admissions.NHSNumber = Provider_APC_2004_05.NHSNumber) or (Combined_Admissions.PASNUMBER = Provider_APC_2004_05.PDDISTNO)) and (Combined_Admissions.AdmDate BETWEEN DateAdd(yyyy,-1,Provider_APC_2004_05.AdmDate) AND Provider_APC_2004_05.AdmDate) AND Combined_Admissions.AdmMethod like 'Emergency%')-- and -- CA.NHSorPrivate = 'NHS')) FROM Provider_APC_2004_05, Combined_Admissions
any help in improving speed would be most welcome as there are 3 more of these updates to run right after this one and the analysis tables are almost double the size of this one
I went to View Synchronization status and see that the sychronization has been running for two hours already. Is that normal? I thought it would just find the one change I made and update that one change?
hi, my Backup job, took a backup at 3.00am , backup went well, but this job is still running , now the time is 10.00am. I tried to kill this job but it is still running. Why my process still running after taking a backup also. Can anyone suggest me, how i have to resolve this.
Hi Can somebody suggest optimization on this query. ITs running for hours,I have checked on mutiple Servers but dont know where the problem is.It starts when i add the not exists piece if the query. Pls help
SELECT distinct isnull(P.id,'') PersonID , s.status , isnull(replace(Ltrim(Rtrim(P.Firstname)),',',''),' ') FirstName , isnull(replace(Ltrim(Rtrim(P.MiddleName)),',',''), '') MiddleName , isnull(replace(Ltrim(Rtrim(P.lastname)),',',''),'' ) LastName , isnull(replace(Ltrim(Rtrim(P.Suffix)),',',''),'') Suffix , isnull(replace(Ltrim(Rtrim(p.company)),',',''),'') Company , isnull(replace(Ltrim(Rtrim(PA.AddressLine1)),',',' '),'') AddressLine1 , isnull(replace(Ltrim(Rtrim(PA.AddressLine2)),',',' '),'') AddressLine2 , isnull(replace(Ltrim(Rtrim(PA.City)),',',''),'') City , State = isnull(replace(Ltrim(Rtrim(PA.State)),',',''),'') , ZipCode = ISNULL(CASEWHEN LEN((LTRIM(RTRIM(PA.Zipcode)))) = 3 THEN ('00' + (PA.Zipcode)) WHEN LEN((LTRIM(RTRIM(PA.Zipcode)))) = 4 THEN ('0' + (PA.Zipcode)) WHEN SUBSTRING(LTRIM(RTRIM(PA.Zipcode)),6,1) = '-' THEN ISNULL(PA.Zipcode,'') WHEN SUBSTRING(PA.Zipcode,6,1) = ' ' THEN ISNULL(PA.Zipcode,'') ELSE substring(PA.Zipcode,1,5)+'-'+ substring(PA.Zipcode,6,4) END ,'') , b.code , CommitteeTermID = '' , RevenueCategoryid = '' FROM vwpersons p with (nolock) join vwsubscriptions s on p.id = s.recipientid join vwcompanies c WITH (NOLOCK) on c.id = p.companyid join vwbusinesscodes b WITH (NOLOCK) on b.id = c.primarybusinesscode join vwRevenueCategorys rc WITH (NOLOCK)on rc.id = c.RevenueCategoryid join vwPersonPrefAddr pa WITH (NOLOCK) on p.id = pa.id WHERE NOT EXISTS (SELECT s1.recipientid FROM vwsubscriptions S1 WHERE S1.SHIPTOID =S.SHIPTOID AND S.PRODUCTID = S1.PRODUCTID AND S1.STATUSID IN (1,2,3,11,12)) and p.id in (select recordid from vwTopicCodeLinks tcl Where tcl.entityid = 457 And tcl.topiccodeid = 332 And tcl.value = 'Yes') and ((p.PreferredAddress = 'Business Address' and p.BadBusinessAddress = 0) or (p.PreferredAddress = 'PO Box Address' and p.BadPOBoxAddress = 0) or(p.PreferredAddress = 'Home Address' and p.BadHomeAddress = 0)) and b.code in ( 'M2','V2','V5','W1','W2','W3','W4' ,'W5','W6','W7','W8','W9','WZ','X2','X3','X4') and p.status <> 5 and S.statusID IN (4,5,6,7,8,9,10,13) and S.productid in (23) group by b.code ,p.id , p.firstname,p.lastname,p.MiddleName,p.suffix, p.company,pa.addressline1,pa.addressline2,pa.city, pa.state,pa.zipcode, s.status
We are currently using a Merged replication and Push subscriber to replicate the databse from DB1 to DB2 every hours. The replication process successed for first 20 hours but we found that the replication process cannot be completed after 20 hours with the following error codes:
Action Code Last Action Msg
4 The process could not query row metadata at the 'Subscriber'
363 The process could not deliver insert(s) at the 'Subscriber'.
Creating a burn down chart using a running total of cumulative hours with the following formula:
CumulativeHoursLeft:=CALCULATE ( SUM('Projects'[Budget hours]) - SUM ( 'hours'[Hours] ), FILTER ( ALL ( 'hours'[Date] ), 'hours'[Date] <= MAX ('hours'[Date]) ) )
Works great except that in a Line Chart using [Date] as the Axis and CumulativeHoursLeft as the value, I get these spikes on days for which the employee reported no hours. I do know what exactly the measure is doing in this instance and I do not get this in a table, those dates simply do not appear. I have tried both Categories and Continuous for the Line Chart. I have also tried filtering where [Date] is not blank.how to get rid of the spikes?
I have a dts job set up to transfer 550,000 records from a dbf file into a sql server. If I just let it run, there is a 9-10 minute delay, then it starts. If I try to schedule a job, it fails completely. I looked up ways to get it to execute quicker, mainly going to the advanced tab of the transform arrow and making the inserts 1000 at a time, the table locked, turning constraints off. Any advice on how to speed it up or why the job is failing?
Last week, we upgraded to sql2000 from 6.5. Everything went on fine. we re-compiled all the procedures. When i try to run a procedure, its running for long time - more than 10 hours (in sql 6.5 it runs for 50 mins). do i need to set any procedure cache?.
Also, the server CPU usuage is constantly high - more than 80%. It was fine till last week.
Our server is running. There are no locks, and server has been rebooted but the problem is still there. This has been going on for some time now. I intend to restart the server. Does anybody have a quick solution, please help. Thanks for your assistance!!
When first time I start my sql server is running faster. After 10 to 15 days later, sql sever performance is very slow. After I restart SQL service, to become normal.
Hi list, I'm a long time lurker on this list and really enjoy the discussions, although I rarely get a chance to participate.
Here is my situation: We are importing chunks of data (500 records at a time) from a C++ interface. The records have to be transformed before inserting into the target table which I am doing using a stored proc which is working fine. The records are in memory in C++ and the programmer is looping through the records building inserts into a temp table through ADO (which my proc picks up). The server business object is using the connection.execute method which is inserting one record at a time. That part of the process is taking over 15 seconds for 500 records which is the bulk of the total time.
My question is: Using ADO is there a better way to insert these records into the temp table? I see mention of a recordset interface but my programmers are new to ADO and since I am the DBA and have never used ADO, I am not sure what to tell them.
I have linked three SQL Servers together, and have written a stored proc that has a cursor that joined three tables from one of the linked servers. When I pull the SQL out of the cursor definition and run it in a query window it runs fine, but when I run the stored proc that simply steps through the same select result set it is too slow for words. It also throws a warning about serial isolation levels. Is there any way I can fix this.
Our main production server has started running slow, it is a dual zeon thingy with plenty of ram so hardware is not an issue.
Basically a service connects to the database and executes a few stored procs, the only way I can get the system up to speed again is to recompile one of the SPs but that is only a temporary fix.
Anyone had a similar thing?
Can anyone give me help on performance tuning in SQL server 2000.
I have a DTS package ON SQL 2000 which transfer data from AS400 to SQL 2000 using an ODBC Client Access 5.1 (The DSN was configured by a sysdmin on the AS400 so it is configured properly). When i execute the package manualy (by right click and "execute package") the package runns fine and ruterns data in no time (Eproximatly 30000 rows in 15 sec).
The problem starts when a job executes the same packagee!!! When i start the job, the DTS package is running Very Very Slow!!!! sometime it takes Hours to return a few rows! and it seems that it is stuck.
The SQLAgent is running as a NT Account with Administrator rights, and has full access to the AS400!! so the problem is not the Agent.
by monitoring the AS400, i have noticed that the job/DTS is retreaving the first fetch quickly , and then it is in a waiting status
i have tried everything and cant seem to get this problem fixed
Does anyone know what could be the problem? I Need Help Quick!!! Thank You
Hello All, I have two procedures being run one after the other. when I run proc1 it runs for about 15 min. Now the proc2 is dependent on proc1, when I run proc2 it runs for 45 min. If I run both the proc's simultaneously through .net code it takes more than 1 hour. Can anyone of you tell me where would be the problem.
I have SQLServer 6.5 SP5a update running on Windows NT 4.0 SP6 with 4 gig RAM and 4 processor.
Suddenly the SQL 6.5 jobs running on the production server started running very very slow. A job that suppose to run in 30 minutes are running like 2 hours and completing successfully.
(I suspect the after the Norton Anti virus automatic live update may be the reason but not the Second Vulnerability as mentioned by Microsoft Bulletin last week)
I check the SQLServer, ran the performance monitor, checked pagefiles, disk space, databases,memory, tempdb. Everything seems to be normal.
I rebooted the server, checked any other process making that slow. But no use.
Please help me out with this issue as this is a production and the CRM applications from the clients uses the database server.
I have been tapped to help with fixing a reporting tool. We have a Sql Server database/crystal reports(10) setup. I havent had the chance to look at the tables in the DB yet, but I was told that aggregate tables were used. In my past experience with crystal reports, we used database views to feed crystal reports (in Oracle). I was thinking that I could somehow use views instead of tables and then try to re-index the base tables and compile satitics (if theres such a thing with Sql Server). I was also going to look into bottlenecking and locking (table locking as opposed to row or page locking for the lookup tables...to reduce overhead on the main tables) but, I'm not sure if it'll make a difference since this is just a demo server with no major traffic hitting it yet.
The question is, does anyone have any experience with crystal reports running slow with Sql Server, what should I look out for??
I am having a problem ,SQL server is running very slow.This is happening some days only.For example my stored procedure ususally runs less than 2 minutes, some days will take 13minutes.I dont understand the problem.All the stored procedure having the same problem.Sorry, I am not a DBA,basically a devloper.Daily morning we are taking the DB backup and indexes already applied.DB size 10461.06 MB,RAM 4GB,CPU usage is less than 50%,This is a Cinema Database,so lot of users are accessing at same time(Web,IVR,cinema ticket counters etc).We are using SQL reports.Because of the stored procedure running slow,can not view the reports.pls advice..
please help me..If you need some more information please ask ..
Hi friends after working visual studio (on my report model project) few minutes it runs too slow. i mean clicking on entities ,attributes takes ages to finish. I opened task manager i see "devenv.exe" is taking more than 800,000 k !!
am using sql server 2005 standard edition. have you seen similar problem. Thanks for your help.
We have a stored procedure which is running fine on a SQL server 2000 from Query Analyzer. However, when we try to execute the same stored procedure from ADO.NET in an executable, the execution is hung or takes extremely long. Does anyone have any ideas or suggestions about how it could happen and how to fix. thanks
I support a website (ASP.NET 2.0) where recently users have been unable to insert data due to timeout issues. The functionality executes a query that inserts a single row into a SQL Server 2005 db table. I tried running this query from the backend, and it took 4:48 to insert a single row! Interestingly, after that initial agony any similar inserts I tried took no time whatsoever. I have checked the execution plan, but unfortunately don't really know what I'm looking for with inserts, as most of my experience with execution plans is with select statements. Any resources anyone could point me to for troubleshooting this would be much appreciated.
