Temporary Table Insert Performance Degraded After Migration To 2005 By 50%.
Apr 11, 2007
Hi,
I have a series of queries which have doubled in the amount of time they take to execute since moving to SQL Server 2005. The queries being performance tested are utilising hardware that is very similar to that of the comparison SQL Server 2000 server. They have 6 CPUs exactly the same and we have swapped RAM around to eliminate that difference.
There are 4 parts to the query suffering performance degredation.
1. Create temporary results table
2. Create (using SELECT INTO) and populate temporary working table 1 - 212,263 rows
3. Create (using SELECT INTO) and populate temporary working table 2 - 5,102 rows
4. Insert into temp results table matches found in temp work table 1 and temp work table 2 - 382 rows
On 2000, the queries take a total of 15 secs. On 2005, they take 30 seconds. Part four of the query takes approx 17 secs on its initial run. However, if i truncate the temp results table and re-run just the last query it only takes 1 sec. Query Plan caching?
I have reviewed the forum for a solution to the problem but with no luck. Many of the solutions presented appear to relate to permanant user tables. Solutions such as UPDATE STATISTICS and recompiling stored procedures have no positive effect. When reviewing the query plans, there are very little differences. Some expected between versions right?
The following code snippet is the query from part 4.
Code Snippet
INSERT #MatchingResults
(Table1IDNo, Table2IDNo, MatchRunNo)
SELECT DISTINCT #Table2.IDNo AS Table2IDNo,
#Table1.IDNo AS Table1IDNo,
1 AS MatchRunNo
FROM #Table1
INNER JOIN #Table2
ON ( #Table2.LastName = #Table1.LastName )
AND ( #Table2.AddressDetails = #Table1.AddressDetails )
AND ( #Table2.Country = #Table1.Country )
AND ( ( #Table2.FirstName = #Table1.FirstName) OR ( #Table1.FirstName = '' ) )
AND ( ( #Table2.Title = #Table1.Title ) OR ( #Table1.Title = '' ) )
The query plan shows a hash join on both servers. I have tried removing the distinct statement and forcing a Loop Join (query hint).
I have also run SQL Profiler. The only differences there appear to be with the "SELECT StatMan" statements.
On 2000, part of the query duration is 1719 and is as follows:
Code SnippetSELECT statman([AddressDetails],[LastName],[FirstName],[Title],[Country],@PSTATMAN)
FROM (SELECT TOP 100 PERCENT [AddressDetails],[LastName],[FirstName],[Title],[Country] FROM [dbo].[#TMCT04042007101009_________________________________________________________________________________________________000000000096] WITH(READUNCOMMITTED,SAMPLE 3.675520e+001 PERCENT) ORDER BY [AddressDetails],[LastName],[FirstName],[Title],[Country]) AS _MS_UPDSTATS_TBL OPTION (BYPASS OPTIMIZER_QUEUE, MAXDOP 1)
On 2005, part of the query duration is 5188 and is as follows:
Code Snippet
SELECT StatMan([SC0], [SB0000]) FROM (SELECT TOP 100 PERCENT [SC0], step_direction([SC0]) over (order by NULL) AS [SB0000] FROM (SELECT [AddressDetails] AS [SC0] FROM [dbo].[#TMCT04042007101009_________________________________________________________________________________________________00000000000E] WITH (READUNCOMMITTED,SAMPLE 7.946877e+001 PERCENT) ) AS _MS_UPDSTATS_TBL_HELPER ORDER BY [SC0], [SB0000] ) AS _MS_UPDSTATS_TBL OPTION (MAXDOP 1)
Its clear that the sampling rate is higher. I assume this could have something to do with it. Can this be modified?
Recently we have migrated our application from MSDE 2000 to SQL Server Express 2005(SP 1). This has significantly reduced the performance of our Windows.NET application which is developed using C#.
For example : While logging in to the application two databases are being attached.
Time taken in MSDE: 16 secs
Time taken in SQL Server Express 2005 : 58 secs Also note performance is degraded for normal screens where data is retrived from database using inline queries.
Questions:
1) Is there any special(optimum) configuration(installation parameters) while installing the SQL Server Express 2005 setup?
2) Is there any query optimization to be done w.r.t SQL Server Express 2005 ?
We highly appreciate any help towards resolving the above problem. Regards, Sasi
I did the migration by detaching the database from SQL 2000 running on W2K Pro and attaching it to SQL 2005 running on XP Pro. Some queries with simple aggregate functions such as AVG() have been slowed by at least an order of magnitude. I understand that SQL server can intelligently adapt to improve the performance. I am not sure how much it can improve at this point of time.
Is it possible to insert data into a table from a temporary table that is inner join? Can anyone share an example of a stored procedure that can do this? Thanks, xyz789
anyone can help me? i using sp to select a select statement from a join table. due to the requirement, i need to group the data into monthly/weekly basic.
so i already collect the data for the month and use the case to make a new compute column in the selete statement call weekGroup. this is just a string showing "week 1", "week 2" .... "week 5".
so now i want to group the weekgroup and disply the average mark. so i need to insert all the record from the select statement into the temporary table and then use 2nd select statement to collect the new data in 5 record only. may i know how to make this posible?
I'm trying to insert the results of a stored procedure call into a temporary table, which is not working. It does work if I use a non-temporary table. Can anyone tell me if this is supported or what I am doing wrong.
Here is an example:
-- DROP PROCEDURE testProc CREATE PROCEDURE testProc AS BEGIN SELECT '1111' as col1, '2222' as col2 END
-- this call will fail with message Invalid object name '#tmpTable'. INSERT INTO #tmpTable EXEC testProc
Will you recommend the usage of temporary tables in a SQL server database ? AFAIK, it boosts the performance. But recently I read one article in SQL Server performance.com[^] which confused me. Any insights on this would be helpful ?
After upgrading to SQL 7 (SP1), we have several SP's that have gone from taking 2-3 min to take 15-20. Each of these SP's creates at least one temp table, inserts into that table, then updates the records in that table. From our research, we can tell that the creation and inserts into the temp tables are fine. It is the updating of these tables that causes the problem. We can observe that the problem is happening by watching the processors go to and stay above 90%. If it were just a few SP's, we could easily fix it and go on, but because of 6.5's limit of 16 tables referenced in a SP, we had to use this method many times. Is there a fix out there for this or a configuration change I can make?
I have a table that includes the html-output of different parts of my pages. This table grows very big very fast, and rows older than 24 hours are useless.
My question is if it is possible to have temporary rows, whose are automatically deleted after these 24 hours? And then how to accomplish that?
How do I use table names stored in variables in stored procedures?
Code Snippetif (select count(*) from @tablename) = 0 or (select count(*) from @tablename) = 1000000
I receive the error 'must declare table variable '@tablename''
I've looked into table variables and they are not what I would require to accomplish what is needed. After browsing through the forums I believe I need to use dynamic sql particuarly involving sp_executesql. However, I am pretty new at sql and do not really understand how to use this and receive an output parameter from it(msdn kind of confuses me too). I am tryin got receive an integer count of the records from a certain table which can change to anything depending on what the user requires.
Code Snippet
if exists(Select * from sysobjects where name = @temptablename) drop table @temptablename
It does not like the 'drop table @temptablename' part here. This probably wouldn't be an issue if I could get temporary tables to work, however when I use temporary tables i get invalid object '#temptable'.
Heres what the stored procedure does. I duplicate a table that is going to be modified by using 'select into temptable' I add the records required using 'Insert into temptable(Columns) Select(Columns)f rom TableA' then I truncate the original table that is being modified and insert the temporary table into the original.
Heres the actual SQL query that produces the temporary table error.
Code Snippet Select * into #temptableabcd from TableA
Insert into #temptableabcd(ColumnA, ColumnB,Field_01, Field_02) SELECT ColumnA, ColumnB, Sum(ABC_01) as 'Field_01', Sum(ABC_02) as 'Field_02', FROM TableB where ColumnB = 003860 Group By ColumnA, ColumnB
TRUNCATE TABLE TableA
Insert into TableA(ColumnA, ColumnB,Field_01, Field_02) Select ColumnA, ColumnB, Sum(Field_01) as 'Field_01', Sum('Field_02) as 'Field_02', From #temptableabcd Group by ColumnA, ColumnB
The above coding produces
Msg 208, Level 16, State 0, Line 1
Invalid object name '#temptableabcd'.
Why does this seem to work when I use an actual table? With an actual table the SQL runs smoothly, however that creates the table names as a variable problem from above. Is there certain limitation with temporary tables in stored procedures? How would I get the temporary table to work in this case if possible?
Using SQL against a DB2 table the 'with' key word is used todynamically create a temporary table with an SQL statement that isretained for the duration of that SQL statement.What is the equivalent to the SQL 'with' using TSQL? If there is notone, what is the TSQL solution to creating a temporary table that isassociated with an SQL statement? Examples would be appreciated.Thank you!!
I decided to change over from Microsoft Access Database file to the New SQLServerCe Compact edition. Although the reading of data from the database is greatly improved, the inserting of the new rows is extremely slow.
I was getting between 60 to 70 rows per sec using OLEDB and an Access Database but now only getting 14 to 27 rows per sec using SQLServerCe.
I have tried the below code changes and nothing seams to increase the speed, any help as I would prefer to use SQLServerCe as the database is much smaller and Im use to SQL Commands.
Details: VB2008 Pro .NET Frameworks 2.0 SQL Compact Edition V3.5 Encryption = Engine Default Database Size = 128Mb (But needs to be changes to 999Mbs)
Where Backup_On_Next_Run, OverWriteQuick, CompressAns are Booleans, all other column sizes are nvarchar and size 10 to 30 expect for Full Folder Address size 260
14 to 20 rows per second (Was 60 to 70 when using OLEDB Access)
TRY 2
Using Record Sets
Private Sub InsertRecordsIntoSQLServerce(ByVal Group_Name1 As String, ByVal Full_Folder_Address1 As String, ByVal File1 As String, ByVal File_Size_KB1 As String, ByVal Schedule_To_Run1 As String, ByVal Backup_Time1 As String, ByVal Last_Run1 As String, ByVal Result1 As String, ByVal Last_Modfied1 As String, ByVal Latest_Modfied1 As String, ByVal Backup_On_Next_Run1 As Boolean, ByVal Total_Backup_Times1 As String, ByVal Server_File_Number1 As String, ByVal Server_Number1 As String, ByVal File_Break_Down1 As String, ByVal No_Of_Servers1 As String, ByVal Full_File_Address1 As String, ByVal OverWriteQuick As Boolean, ByVal CompressAns As Boolean)
cmd.CommandText = "SELECT * FROM BackupDatabase" cmd.ExecuteNonQuery() Dim rs As SqlCeResultSet = cmd.ExecuteResultSet(ResultSetOptions.Updatable Or ResultSetOptions.Scrollable)
Dim rec As SqlCeUpdatableRecord = rs.CreateRecord()
rec.SetString(1, Group_Name1) rec.SetString(2, Full_Folder_Address1) rec.SetString(3, File1) rec.SetSqlString(4, File_Size_KB1) rec.SetSqlString(5, Schedule_To_Run1) rec.SetSqlString(6, Backup_Time1) rec.SetSqlString(7, Last_Run1) rec.SetSqlString(8, Result1) rec.SetSqlString(9, Last_Modfied1) rec.SetSqlString(10, Latest_Modfied1) rec.SetSqlBoolean(11, Backup_On_Next_Run1) rec.SetSqlString(12, Total_Backup_Times1) rec.SetSqlString(13, Server_File_Number1) rec.SetSqlString(14, Server_Number1) rec.SetSqlString(15, File_Break_Down1) rec.SetSqlString(16, No_Of_Servers1) rec.SetSqlString(17, Full_File_Address1) rec.SetSqlBoolean(18, OverWriteQuick) rec.SetSqlBoolean(19, CompressAns) rs.Insert(rec) Catch e As Exception MessageBox.Show(e.Message) Finally conn.Close() End Try End Sub
20 to 24 rows per sec
TRY 3
Using SQL Commands Direct
Private Sub InsertRecordsIntoSQLServerce(ByVal Group_Name1 As String, ByVal Full_Folder_Address1 As String, ByVal File1 As String, ByVal File_Size_KB1 As String, ByVal Schedule_To_Run1 As String, ByVal Backup_Time1 As String, ByVal Last_Run1 As String, ByVal Result1 As String, ByVal Last_Modfied1 As String, ByVal Latest_Modfied1 As String, ByVal Backup_On_Next_Run1 As Boolean, ByVal Total_Backup_Times1 As String, ByVal Server_File_Number1 As String, ByVal Server_Number1 As String, ByVal File_Break_Down1 As String, ByVal No_Of_Servers1 As String, ByVal Full_File_Address1 As String, ByVal OverWriteQuick As Boolean, ByVal CompressAns As Boolean)
Banti writes "IF i create temporary table by using #table and ##table then what is the difference. i found no difference. pls reply. first: create table ##temp ( name varchar(25), roll int ) insert into ##temp values('banti',1) select * from ##temp second: create table #temp ( name varchar(25), roll int ) insert into #temp values('banti',1) select * from #temp
both works fine , then what is the difference waiting for ur reply Banti"
I need to return a table of values calculated from other tables. I have about 10 reports which will use approx. 6 different table structures.
Would it be better performance wise to create a physical table in the database to update while calculating using an identity field to id the stored procedure call, return the data and delete the records. For Example:
I am using SQL 2000 I have the following code. When saving it, I am getting an error: There is already an object named '##tbl' in the database
This is although #tbl is dropped. Is threrea way to avoid this error? the only work around I found was to create a string with the SQL command and call EXEC, but I don't like this solution as it prevents early compilation of the procedure.
declare @x int set @x=1 IF @x=0 begin create table #tbl ( abc int ) drop table #tbl end
IF @x=1 begin create table #tbl ( abc int ) drop table #tbl end
I have to create a temporary table for generating a report in VB. Pls help how to check the temporary table name in database. I want if exits than drop and create a new one.
The above database is running on Microsoft SQL Server 2000 and i would like to query for a report that looks something as below:
Description | Period 01 | Period 02 | Year to Date ========================================================= Sales | 20,000.00 | 10,000.00 | 30,000.00 Total Sales | 20,000.00 | 10,000.00 | 30,000.00
The above report would list 4 columns, with the last column being a calculated field as a sum of Period01 + Period02 Amount, sorted by GL_ID and group under a summation row called Total Sales & Total Cost.There would be a net amount appearing as Profit (Total Sales-Total Cost).
Guys, hope someone out there can help me with the sql command for the above report?
Since the second table is somehow only temporary table (having at the front ajax script), i need the third table to store user's ID and all the info from second table.
What would be the easiest way to do it, since the first table contains only one row per user and the second one 40 rows per user.
Declare @Temp Table(id_customer int, id_product varchar(10)) Insert Into @Temp(id_customer, id_product) Values(12, 'ABC104') Insert Into @Temp(id_customer, id_product) Values(12, 'ABC143') Insert Into @Temp(id_customer, id_product) Values(12, 'ABC103') Insert Into @Temp(id_customer, id_product) Values(13, 'ABC102') Insert Into @Temp(id_customer, id_product) Values(14, 'ABC101') Insert Into @Temp(id_customer, id_product) Values(15, 'AABC10') Insert Into @Temp(id_customer, id_product) Values(15, 'AABC11')
select * from @Temp Declare @results VarChar(100) Select @results = IsNull(@results + '; ', '') + IsNull(id_product, '') From @Temp where id_customer = 12 Select @results as all_products
and when i run "Select @results as all_products" what shall i do to get id_customer along? if i do "select id_customer, @results as all_products" I get error in return :(
Hi All, DECLARE @MyTableVar table( EmpID nvarchar(10) ) select login_id from cpm into @MyTableVar Above syntax is not working.... Actually i want to table variable and store the result returned by stored procedure. How to do that... Thanks and reagards A
I'm trying to drop a temporary table. I keep getting this error: Cannot drop the table '#temp_table', because it does not exist in the system catalog. I tried the following but it did not work, so help would be appreciated. if object_id('tempdb..#temp_table') is not nulldrop table #temp_table
Hello once again, My last question concerns (temporary) tables. In a web form (VWD Express, Asp,Net 2.0, VB, .mdf db, Ajax) I have two gridviews. One gets its data from textboxes and dropdownlists as the user clicks an ok button. This data is then sent to a "junk" table in the db. The other gridview retrieves this data "automatically" and displays the data, filtered and grouped by some SQL statements. When the user finally clicks an Insert record button, the data from this second gridview gets inserted in another table in the db and the data from the first "junk" table is disposed. I guess it shouldn't be necessary to insert, retrieve, and delete the data in and from the "junk" table. Even though it does work like a charm, I guess it would be better if I could do it wit hsome kind of "temporary table", array or the like so that the db wouldn't need to be updated to this extent. Still, I really like the simplicity of using gridviews and the rest of the page's code is complex enough, so I don't really want to substitute this behaviour for tons of hand-written, hard-to-follow, and hard-to-integrate code. What are my options? Any help would be greatly appreciated. Pettrer
Im trying to find a table and Drop in T-SQL using this script.
/* Start */ Use Students IF exists (Select * from information_schema.tables where table_name ='##Exams_result) drop table ##Exams_result go Create table ##Exams_result..............etc
/* end */
But I cant find my temporary table on this way... Any sugestion?
Luiz Lucasi lc@culting.com Rio de Janeiro - Brazil
When we are writing a big query that against to 25+ tables (it has lots of lookup and lots of business logic in it so it will have lots of left outer joins, right outer joins, inner join, UNION, group by, and lots of SQL server functions) should we put everything in one select statement? Or we should create lots of temporary tables and then join them together?
My friend told me that I should use one statement instead of lots of temp tables in order to reduce the physical read. He does not like the temp table idea since there will be lots of user running the query in the same time and it will use lots of space. But I dont believe that is true. (I believe when you put everything in one statement, SQL server still needs to create the result set in the temp db.) There will be lots of join going on and it could hurt the performance.
I believe that 1.We should create the temp tables and then join the temp tables together later to get the final results. we can create the index on the temp table at runtime. 2.If the order of columns in the index does not match to our where clause, then the index is useless. 3.If there is a SQL function in the where clause, the index is useless. (for example, where Datepart(yy, orderdate)=1998) 4.If there is a NOT in the where clause, the index is useless.
What do you think? Are you agree with me?
Following is the query. Is there any better way to do this?
Select d.Description as Division, c.Description as Region, b.StateCode as State, b.FacilityNumber, b.Name as FacilityName, (Case When a.ExitDate Is Not Null Then Convert(Varchar(10),a.ExitDate,101) Else 'Pending' End) as PriorStdSurveyExit, (Case When a.ExitDate Is Not Null And a.IJValid > 0 Then Convert(Varchar(10),a.ExitDate+89,101) Else '' End) as ThreeMonthWindow, (Case When a.ExitDate Is Not Null Then Convert(Varchar(10),a.ExitDate+179,101) When a.IJValid > 0 Then Convert(Varchar(10),a.ExitDate+179,101) Else 'Pending' End) as SixMonthWindow, (Case When a.ExitDate Is Not Null Then Convert(Varchar(10),a.ExitDate+450,101) When a.IJValid > 0 Then Convert(Varchar(10),a.ExitDate+450,101) Else 'Pending' End) as FifteenMonthWindow From ( Select a.FacilityNumber, a.ExitDate, IsNull(b.FacilityNumber,0) as IJValid From ( Select b.FacilityNumber, Max(a.ExitDate) as ExitDate From Survey a Inner Join Facility b on (a.FacilityID = b.FacilityID) Inner Join SurveyFormatCode c on (a.SurveyFormatCodeID = c.SurveyFormatCodeID) Inner Join SurveyTypeCode d on (a.SurveyTypeCodeID = d.SurveyTypeCodeID) Where a.VerbalResultDate is Not Null And c.Description Like 'Standard%' And d.Description Like 'LTC%' And a.ExitDate <= @InDate Group By b.FacilityNumber ) a Left Join ( Select Distinct b.FacilityNumber From Survey a Inner Join Facility b on (a.FacilityID = b.FacilityID) Inner Join SurveyFormatCode c on (a.SurveyFormatCodeID = c.SurveyFormatCodeID) Inner Join SurveyTypeCode d on (a.SurveyTypeCodeID = d.SurveyTypeCodeID) Inner Join ComplianceRatingCode e on (a.ComplianceRatingCodeID = e.ComplianceRatingCodeID) Where a.VerbalResultDate is Not Null And (c.Description Like 'Standard%' And d.Description Like 'LTC%') And e.Description Like 'Category 4%' And a.ExitDate <= @InDate Union Select Distinct b.FacilityNumber From ( Select b.FacilityNumber, Max(a.ExitDate) as ExitDate From SurveyCitation e Inner Join Survey a on (a.SurveyID = e.SurveyID) Inner Join Facility b on (a.FacilityID = b.FacilityID) Inner Join SurveyFormatCode c on (a.SurveyFormatCodeID = c.SurveyFormatCodeID) Inner Join SurveyTypeCode d on (a.SurveyTypeCodeID = d.SurveyTypeCodeID) Inner Join Citation f on (e.CitationID = f.CitationID) Inner Join SSLetterCode g on (f.SSLetterCodeID = g.SSLetterCodeID) Where a.VerbalResultDate is Not Null And d.Description Like 'LTC%' And a.ExitDate <= @InDate And g.SSLetterCode in ('J','K','L') Group By b.FacilityNumber ) a Right Join ( Select b.FacilityNumber, Max(a.ExitDate) as ExitDate From SurveyCitation e Inner Join Survey a on (a.SurveyID = e.SurveyID) Inner Join Facility b on (a.FacilityID = b.FacilityID) Inner Join SurveyFormatCode c on (a.SurveyFormatCodeID = c.SurveyFormatCodeID) Inner Join SurveyTypeCode d on (a.SurveyTypeCodeID = d.SurveyTypeCodeID) Where a.VerbalResultDate is Not Null And c.Description Like 'Standard%' And d.Description Like 'LTC%' And a.ExitDate <= @InDate Group By b.FacilityNumber
) b on (a.FacilityNumber = b.FacilityNumber) Where a.ExitDate >= b.ExitDate
) b on (a.FacilityNumber = b.FacilityNumber) ) a Right Join Facility b on (a.FacilityNumber = b.FacilityNumber) Left Join Region c on (b.RegionID = c.RegionID) Left Join Division d on (c.DivisionID = d.DivisionID) Order By d.Description, c.Description, b.FacilityNumber
when i run in query analyzer , using "exec SP_TEST" , it work and display result
but when run Exec Master..xp_CmdShell 'bcp "exec mydbtest..SP_TEST " queryout C:TEST.TXT -c -Slocalhost -Usa -Ppassword'
it give error "SQLState = S0002, NativeError = 208" and show Error = [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name '#TMP_CUSTOMER'.
I created the #Temporary table in MS SQL. Now i want to drop this #Temporary table, but i want to check first before i drop the this table. How to check the Temporary table exist or not in MS SQL?
We are having some issues with temporary tables (with # prefixes) within Stored Procedures.
When running a profile trace on them, the stored proc quite happily creates the temp # table (in fact several of them) but whenever it hits the first statement inserting data into one of them (and it doesn't matter which one), there is a 5-6 second delay.
By commenting out one and moving to the next piece of code, the same thing happens.
Following which, the rest of the Stored Prco runs fine and subsequent inserts into the # temp tables also run efficiently.
Is the stored proc getting recompiled perhaps ?
Any advice woul;d be appreciated.
we are running SQL Server 7.0, dont know whether that helps ?