Hi,
Here is the typical processing I do in Data Flow
1) read data file (data source)
2) do something with the data (e.g. derived column, script component, etc)
3) pass data as parameters into OLE DB command (stored proc) to take care of the rest.
Would it be faster if
I split the data into say 10 "chunks" (using split transform). Each chunk will go to its own OLE DB command (10 OLE DB command). Now I will have 10 concurrent processes running the stored proc ?
I recently installed SQL2005 Express on a Dell Precision workstation to be accessed by 6 users. The PC has XP Pro, 2GB memory, and 2.0GHz core2due CPU. This PC is dedicated for the database only. I also set XP Performance options in System Properties to Background services and System cache. Now, the users are experiencing slow response from the database. How can I make the database response faster? Thanks for your input.
Hi.We have stored procedure update specific table Each time it run it delete 5000- 6000 rows from table then insert 5000- 6000 rows with different information. It take up to 1 1/2 min execute.
1.Can force Sql server do not make entry for each insert and if yes would it increase speed of procedure ? 2. Is any other way increase speed of insert?
I need help makeing the following query run more efficently.
Code:
SELECT t1.ID,t1.firstName,t1.lastName,t1.address,t1.city,t1.state, t1.zip,t1.locationAddress,t1.locationCity,t1.locationState,t1.locationZip FROM Landlord_tbl t1 left outer join Mail_tbl t2 ON t2.potentialSitesID = t1.potentialSitesID WHERE t2.mailed_out_date is null and NOT(t1.firstName+t1.lastName) is Null GROUP BY t1.ID,t1.firstName,t1.lastName,t1.address,t1.city,t1.state, t1.zip,t1.locationAddress,t1.locationCity,t1.locationState,t1.locationZip ORDER BY t1.firstName, t1.lastName, t1.city, t1.state
BEGIN SELECT @AppointmentsCount = COUNT(tbl_SurgerySlot.SurgerySlotKey) FROM tbl_SurgerySlot INNER JOIN tbl_SurgerySlotDescription ON (tbl_SurgerySlot.Label = tbl_SurgerySlotDescription.Label AND tbl_SurgerySlot.PracticeCode = tbl_SurgerySlotDescription.PracticeCode) AND tbl_SurgerySlot.ExtractDate = @ExtractDate AND tbl_SurgerySlot.StartTime BETWEEN @DateFrom AND @DateTo AND tbl_SurgerySlotDescription.NormalBookable = 1 AND tbl_SurgerySlot.SurgerySlotKey NOT IN( SELECT tbl_Appointment.SurgerySlotKey FROM tbl_Appointment WHERE tbl_Appointment.ExtractDate = @ExtractDate AND tbl_Appointment.Deleted = 0 AND tbl_Appointment.Cancelled = 0 ) END ELSE
BEGIN IF @96hrPlusFlag = 1
SELECT @AppointmentsCount = COUNT(tbl_SurgerySlot.SurgerySlotKey) FROM tbl_SurgerySlot INNER JOIN tbl_SurgerySlotDescription ON (tbl_SurgerySlot.Label = tbl_SurgerySlotDescription.Label AND tbl_SurgerySlot.PracticeCode = tbl_SurgerySlotDescription.PracticeCode) AND tbl_SurgerySlot.ExtractDate = @ExtractDate AND tbl_SurgerySlot.StartTime >@DateTo AND tbl_SurgerySlotDescription.NormalBookable = 1 AND tbl_SurgerySlot.SurgerySlotKey NOT IN( SELECT tbl_Appointment.SurgerySlotKey FROM tbl_Appointment WHERE tbl_Appointment.ExtractDate = @ExtractDate AND tbl_Appointment.Deleted = 0 AND tbl_Appointment.Cancelled = 0 )
Any help would be really appreciated.... My stored procedure...
CREATE PROCEDURE business3rd7 @Fromdate DATETIME, @ToDate DATETIME AS
select distinct CONVERT(VARCHAR(10),Receipts.Companynumber1)+CONVE RT(VARCHAR(10),Receipts.Companynumber2) as co , Receipts.Premium1+Receipts.Premium2 as Premium, "CAN"=case when Receipts.transactiontype='CAN' then (receipts.premium1+receipts.premium2) else 0 end, "NET"=Receipts.Premium1+Receipts.Premium2-case when Receipts.transactiontype='CAN' then (receipts.premium1+receipts.premium2) else 0 end,
"#NEW"=case when Receipts.transactiontype='NEW' then count(Receipts.policynumber) else 0 end,
------- "$NEW"= case when Receipts.transactiontype='NEW' then (Receipts.Premium1+Receipts.premium2) else 0 end, "#REN"=case when Receipts.transactiontype='REN' then count(Receipts.policynumber) else 0 end, "$RENEW"= case when Receipts.transactiontype='REN' then (Receipts.Premium1+Receipts.premium2) else 0 end, "#AP"=case when Receipts.transactiontype='AP' then count(Receipts.policynumber) else 0 end, "$AP"= case when Receipts.transactiontype='AP' then (Receipts.Premium1+Receipts.premium2) else 0 end, "#SENT"=case when policy.Renewalofferdate between @Fromdate AND @ToDate then count(policy.policynumber) end,
"%"=case when case when Receipts.transactiontype='REN' then count(Receipts.policynumber)else 0 end =0 then 0 when case when policy.Renewalofferdate between @Fromdate AND @ToDate then count(policy.policynumber) else 0 end=0 then 0 else case when Receipts.transactiontype='REN' then count(Receipts.policynumber)else 0 end /case when policy.Renewalofferdate between @Fromdate AND @ToDate then count(policy.policynumber) else 0 end end,
"Current Year"= case when policy.Renewalofferdate between @Fromdate AND @ToDate then count(clubamount) end, "Previous Year"=case when policy.Renewalofferdate between DateAdd(year, -1, @Fromdate) AND DateAdd(year, -1, @ToDate) then count(clubamount) end, "#AA"=count(receipts.clubamount), "$AA"=sum(receipts.clubamount)
FROM Receipts,Policy where Receipts.Agencyid=Policy.Agentid group by Receipts.CompanyNumber1,Receipts.CompanyNumber2, Receipts.Premium1,Receipts.Premium2, Receipts.TransactionType,policy.Renewalofferdate, Receipts.Agencyid
Any ideas how to make reports faster when returning lots fo rows? I know you would need to work on your sql query etc.. Or maybe cache it. But i'm thinking of having a kind of middle tier thing that sits between your sql database and the reports itself. Any ideas would be appreciated
It seems when I run the query with the set staticts IO on then statistic reports back with the 'work table', and the query takes 30+ sec. if the worktable is ommited(whatever the reason?) the query take less 1 sec.
Here is my take, I believe work table is created in tempdb...and if not then whole query is using the cached page, am I right?
if I am right then the theory is, if I increase the (via sp_configure) server min memory setting and min query memory, the query ought use the cached page and return in less 1 sec. (specially there is absolutely no one but me on the server), so far I can't make it go faster...what setting am I missing to make it run faster?
Another question is if the query can not avoid but use the tempdb, is it going to always be 30 sec+ time? why is tempdb involvement make it go so much slower?
hi all, if i have a comma delimited string and want to insert each delimited substring into a table which of the following way is faster?pass the whole string into the a stored procedure and loop through the delimited string and pick out the substring and insert into the table orloop and pass the substring into a stored procedure and insert N times?or any other better ways someone could suggest me to do thanks!
I was just wondering if this can be done any faster? code-wise that is...
Don't mind the converts, can't do without them, as the data discipline for the source table isn't always reliable, while I have to be absolutely sure the destination data ends in the required format.
I have SQL file but it run slowly when comes to huge record. How do I make it faster. I do create an index but how to make use the index? Pls help me on this...
I'm sonewhat new to MS SQL Server and I'm wondering about which of thefollowing two queries would be faster:DECLARE @ResidencesBuilt intDECLARE @BarracksBuilt intDECLARE @AirBaysBuilt intDECLARE @NuclearPlantsBuilt intDECLARE @FusionPlantsBuilt intDECLARE @StarMinesBuilt intDECLARE @TrainingCampsBuilt intDECLARE @FactoriesBuilt intSELECT@ResidencesBuilt = SUM(CASE WHEN BuildingType = 0 THEN Built END),@BarracksBuilt = SUM(CASE WHEN BuildingType = 1 THEN Built END),@AirBaysBuilt = SUM(CASE WHEN BuildingType = 2 THEN Built END),@NuclearPlantsBuilt = SUM(CASE WHEN BuildingType = 3 THEN Built END),@FusionPlantsBuilt = SUM(CASE WHEN BuildingType = 4 THEN Built END),@StarMinesBuilt = SUM(CASE WHEN BuildingType = 5 THEN Built END),@TrainingCampsBuilt = SUM(CASE WHEN BuildingType = 6 THEN Built END),@FactoriesBuilt = SUM(CASE WHEN BuildingType = 7 THEN Built END)FROM BuildingsGROUP BY kdIDHAVING kdID = 2902Or:DECLARE @ResidencesBuilt intDECLARE @BarracksBuilt intDECLARE @AirBaysBuilt intDECLARE @NuclearPlantsBuilt intDECLARE @FusionPlantsBuilt intDECLARE @StarMinesBuilt intDECLARE @TrainingCampsBuilt intDECLARE @FactoriesBuilt intSET @ResidencesBuilt = (SELECT Built FROM Buildings WHERE BuildingType = 0AND kdID = 2902)SET @BarracksBuilt = (SELECT Built FROM Buildings WHERE BuildingType = 1 ANDkdID = 2902)SET @AirBaysBuilt = (SELECT Built FROM Buildings WHERE BuildingType = 2 ANDkdID = 2902)SET @NuclearPlantsBuilt = (SELECT Built FROM Buildings WHERE BuildingType =3 AND kdID = 2902)SET @FusionPlantsBuilt = (SELECT Built FROM Buildings WHERE BuildingType = 4AND kdID = 2902)SET @StarMinesBuilt = (SELECT Built FROM Buildings WHERE BuildingType = 5AND kdID = 2902)SET @TrainingCampsBuilt = (SELECT Built FROM Buildings WHERE BuildingType =6 AND kdID = 2902)SET @FactoriesBuilt = (SELECT Built FROM Buildings WHERE BuildingType = 7AND kdID = 2902)The data source is:kdID BuildingType Built2902 6 02902 7 02902 4 02902 0 802902 2 02902 1 52902 3 402902 5 10Or:CREATE TABLE [dbo].[Buildings] ([kdID] [int],[BuildingType] [tinyint],[Built] [int])INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 0, 80)INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 1, 5)INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 2, 0)INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 3, 40)INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 4, 0)INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 5, 10)INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 6, 0)INSERT INTO Buildings (kdID, BuildingType, Built) VALUES (2902, 7, 0)Analyzer says the first would be faster, but it has a lot of SUM()'s andwhatnot so I'm not too sure about this. There are also about 1000 rows inthe actual Buildings table. This will be a part of a stored procedure.
I want to know the # of users on our web site for each month in a given year. I'm looking for a faster way to do this--perhaps one that can leverage an index instead of reading the entire table! (My avg disk queue right now is above 7 and the query takes about 90 seconds).
Here's my current SP. Basically I'm calculating each month/year and using UNION to join them together, then pivot to rotate.
Hi all, I m new to this forum and this is my first question. I m having 2 pages in my web site ... page 1 query directly to db using sqldatasource, the second page query through a BLL then DAL by following the step in this tutorial (http://www.asp.net/learn/dataaccess/tutorial02vb.aspx?tabid=63).... Page 1 is using a "Like" query search and the Page 2 is the normal displaying some product detail.... Under normal circumstances, one will expect Page 1 will be way fastest than the Page 2... however the problem is Page 1 is in thunder speed while Page 2 takes 10 secs to load... 10 seconds is really not acceptable... I really couldnt figure out what happens... both Page 1 and Page 2 are using the same connection string which connection through a DSN.... How is the connection different by using sqldatasource and DAL?? Could someone please help.... ThanksP.S. I m using a Pervasive database
Hi y'all, I've recently run a profiler on my code and following query took 7 seconds: SELECT TOP 10 UI, COUNT(UI) AS Expr1 FROM table WHERE (UI <> 'custom_welcome') GROUP BY UI ORDER BY COUNT(UI) DESC Is it possible to rewrite this so my code gets faster? It's also possible that it's due to the size of the table? Thanks in advance! I'll let you know how long your query takes :)
I have a cursor prcedure that is pretty slow because as the cursor moves through the data I have three select statement on the same table to find other rows information. Is there a better way to do this? Simple Example of Code is: DECLARE MyVARABLES 1 to X DECLARE c1 CURSOR FOR SELECT MyData1, MyData2 to X FROM MyTable FOR UPDATE OF MyUpdateData --Start Cursor OPEN c1 FETCH NEXT FROM c1 INTO MyVariables --LOOP WHILE @@FETCH_STATUS = 0 BEGIN ----------------------- -- Get other rows data to add to this rows data ......GUESSING THIS IS THE SLOW PART as the table is LARGESELECT MyVar1 = MyData1 FROM MyTable WHERE MyTableColumns = MyVariables AND MyTableColumns2 <> MyVariables2 --FINDS OTHER ROW (I have three of these)
--Calculate & Update If MyVarable = 'this or that' BEGIN UPDATE MyTable SET MyUpdateData = MyVar1 * x *y WHERE CURRENT OF c1 END ------------------- -- NextFETCH NEXT FROM c1 INTO MyVarables 1 to xEND CLOSE c1 DEALLOCATE c1
Hi! I M basically an application developer & use simple sql queries in my programmings. I do not have much idea abt tuning/auditing part & thatswhy i m unable to answer them properly in my interviews. Can anybody give me some tips?????
Question 1: In a stored procedure, One SELECT stmt is there & depending upon the @rowcount, it updates around 14000 records which is also written inside this stored procedure. Instead of writing this way, there is some other way which is faster than this. Can anybody tell me the correct way???
Question 2:Can anybody give me few examples like this?????? I need them desparetly.
I´ve created a class to make some standard transaction development a little bit faster. The destructor seem to run, but something makes this object slow down the database, if SqlTransaction and/or SqlConnection isnt manualy handled with the method Commit(). Any ideas on how to handle the SqlTransaction and SqlConnection better?
public class DataTransaction { private bool blnError = false; private ArrayList arrErrorList = new ArrayList(); private SqlConnectionobjConnection = new SqlConnection(System.Configuration.ConfigurationSettings.AppSettings["ConnectionString"].ToString()); private SqlTransactionobjTransaction;
Previosuly I was executing 2 DTS packages one afte the other manually and together they took a CONSIDERABLE time. The 1st one was pulling data from the OLPT, doing transformations and populating the tables in my Datamart and the 2nd one was doing a FULL process of all the dimensions and cubes.
However I tried scheduling the DTSs as jobs and havethen merged the 2 resulting jobs as a SINGLE job having 2 sequential steps. To my surprise the resulting job takes less than half the time (actually even lesser) as compared with my original approach i.e. running the DTSs. And I am talking about major improvement in terms of completion of the tasks here :)
Am i getting over excited here or is this natural? I assume that if this is correct then jobs much be some sort of "compiled" version as compared to DTS and maybe that's why I have this terrific improvement in terms of execution times.
I have rewritten a stored procedure that consists of a single select that selects from a view. Essentially I combined the select in the view and the select in the sp into one select. I am now trying to determine if the new version is faster.
The estimated execution plan gives a ratio of 96% : 4% in favour of the new version when I run them together from a query window but when I try to time them I can't get a satisfactory result.
If I run each query once and display the difference between start and end time, they display 0. If I run each one 100, 200, etc times I get different results each time.
If I had a WHERE clause that had to compare a string to another string would it be faster one way or another if I broke it down to three different, smaller searches?
Previosuly I was executing 2 DTS packages one afte the other manually and together they took a CONSIDERABLE time. The 1st one was pulling data from the OLPT, doing transformations and populating the tables in my Datamart and the 2nd one was doing a FULL process of all the dimensions and cubes.
However I tried scheduling the DTSs as jobs and havethen merged the 2 resulting jobs as a SINGLe job having 2 sequential steps. To my surprise the resulting job takes less than half the time (actually even lesser) as compared with my original approach i.e. running the DTSs.
Am i getting over excited here or is this natural? I assume that if this is correct then jobs much be some sort of "compiled" version as compared to DTS and maybe that's why I have this terrific improvement in terms of execution times.
I have a live database and an archive database. I update the archive tables once a day from the live tables using:
INSERT INTO arc_table SELECT * FROM cur_table AS cur WHERE NOT EXISTS (SELECT * FROM arc_table AS arc WHERE arc.key = cur.key)
GO
This inserts newer records into the archive tables from the live tables.
I have two different methods to clean the live tables once a week but keep data from the previous week. Both methods have been verified to delete the same rows.
DELETE cur_table WHERE EXISTS (SELECT key FROM arc_table AS arc WHERE arc.key = cur_table.key) AND date_time < GetDate() - 7
GO
Second method modified from BOL - deletes identical rows
DELETE cur_table FROM (SELECT key FROM arc_table) AS arc WHERE arc.key = cur_table.key AND date_time < GetDate() - 7
GO
I read that "WHERE [NOT] EXISTS" is faster than "WHERE [NOT] IN" but this is the first time I have seen DELETE xx FROM (SELECT ----)
I'd like to know which procedure will be faster and/or better.
HelloI need this really faster in mS SQL 2000Usernumber (int)reportid (FK)reportreportid (PK)Category (int)SELECT A, B, C, D INTO UserCopy FROM UserWHERE User.reportid IN (SELECT MAX(report.reportID) AS maxReport FROM Report GROUP BY report.Category) AND user.number NOT IN (120,144,206,345,221,789,548,666,1204,4875,22,135, 777,444)can return a more than 1000 rows (an the table = 10.000 rows): SELECT MAX(report.reportID) AS maxReport FROM Report GROUP BY report.Categoryand the table user has a few millions rowsReport.ReportId is a Primary key for User.reportid (FK) for the moment it takes up to 3 minutes, i need to do that in 30 seconds maximumthank you for helping
Can anyone give me the basics of speeding up reports that use queries or views or nested views? Current reports are now taking over 2 minutes to show.We have thousands and sometimes even millions of records to report against.Queries have 4 and 5 table joins etc. We are using ASP.NET 2.0 in Visual Studio 2005 and Crystal Reports. Thanks
* We have a DAL that generates all SQL dynamically out of a nobject model. Standard very powerfull O/R mapper. * In the DAL, for CRUD operations, we generate the statements dynamically. As an example, let's take INSERT. * The insert is generated ONCE, with parameters, and cached. For every reuse, the parameters are replaced (in value), and the whole thing commited.
I see hte following negative: I can not easily batch multiple inserts. Parameters have to be unique per batch. So, if I want to batch two inserts, I need two sets of parameters.
Alternative:
Instead of generating the SQL with parameters, we generate the SQL as a string ready to be inserted for / with a String.Format, and then I encode the parameters and make one SQL String out of this. Now, please - don't say "sql injection", we are not that stupid, the layer handles this already, properly encoding all dangerous values.
With this approach, the SQL statement would be a string and not use any parameter. As a result, I could batch them up as much as I want (ok, up to a certain string size). I need to keep parameters around anyway (for blobs etc.), but most objects do not have blobs, and the SQL is prettty small. This small SQL could be batched significantly (100 statements per batch, propably mode) and be submitted to the database. As a result, the round trips to the databae would go down.
Now, my question is - which of the two approaches is more advisable, from a performance point of view? Again, stuff like SQL injection and ease of handling are totally irrelevant - the SQL never leaves the DAL and is generated in there, and we will go through a lot of complexitiy for higher performance.
Normally I would say batching should be better. SQL Server can auto-parameterize the statements (reusing the query plan), and / but the network round trips are the larger issue here.
I have a situation where currently, I check to see if a record exists and I do an update if it does, I do an INSERT if it does not. This happens about 200k times each night.
I'm trying to speed the application up that is doing this and I was wondering if it would be a better idea to just delete the records I'm about to recreate, before I start, then do an unconditional insert.
It would eliminate my IF EXISTS check but if an update is faster than an insert, it might even out.
Also, will SQL deal with the big delete and the data being recreated each night without too much bloating or fragmentation?
File I/O or SQL Database calls?Note that my SQL Server is on a separate server, so that cuts down on the web server resources needed per query, but it increases the time necessary for the query to happen.
