Performance - Slow Load Times For SSIS Packages
Feb 13, 2006Is there any information around what the SSIS packages are doing in the first 5-10 seconds of execution, and ways to speed this process up?
View 3 Replies
ADVERTISEMENT
Resources About Load/Run SSIS Packages Using .NET Remoting.
May 22, 2006Hello,
I don't have a specific problem, I'm looking about resources because this is the tipical case when something is working but I don't know why ..
On particularly I have to load and run a dtsx package from a window application and the package is located on the server.
I decided to use .NET Remoting.
I developed my "Remotable Object" which contains:
package = Microsoft.SqlServer.Dts.Runtime.Application.LoadFromDtsServer()
package.Execute()
etc....etc...
The window application calls the Remotable object, call the specific method and the package is loaded and run successfully.
When I studied the .NET Remoting there is also Server that expose the Remotable Object.
In this case the Server is SSIS Service but I would like to know witch port is used, which method( singlecall or singleton ) is used and so on!
where I can find this info?Someone knows some resource to advice?? or I'm working with SSIS not on a remote way?
Thankx
Marina B.
SSIS Packages That Load Different Text Files Into Tables
Jan 25, 2013So i have about 80 different SSIS PAckages that load different text files into tables. each package has a config file that contains password, server name and user id. I want to forward these packages to a use who will execute them. So instead of giving the user 80 packages to execute manually, how do i put these packages into one parent package so that the user has to only execute this parent package instead of executing 80 different packages NB. package exec is manually. (we will automate later) so this is a temp measure.
View 2 Replies View RelatedDTS To SSIS - Very Slow Performance
Feb 7, 2008I have a bunch of packages that take views and create tables from them. Some of the views are rather complex, but the packages themselves are very simple... drop and re-create a table using the data from a view on the same server. We create a new DB for each year, and this year we've upgraded to a new server with SQL 2005, so our DTS packages on the 2000 SQL server had to be recreated in SSIS on the new server. No problem, as I said the packages are really simple. But when I create the packages in SSIS they now take an extremely long time to execute, and I cannot figure out why.
For instance, one DTS package would take approximately 5 minutes to run when the view contained hundreds of thousands of rows and the underlying tables contained millions. But now, even with MUCH smaller tables (since it's the beginning of the year, new DB) the SSIS package I created on the new server takes over an hour, literally. The view that the SSIS package is using to create the table only takes about 15 seconds to execute in management studio (only about 16,000 rows). How can this possibly take so long??
the new server is virtually the same hardware-wise... 4 x 2400mhz, 4gb ram, win2k3 server
Load/Run SSIS Packages On The Server From A .net Exe Exception:Microsoft.SqlServer.Dts.Runtime.DtsPipelineException: Retrieving.
Jul 28, 2006Hello, I got an exception when I try to load/run a SSIS Package programmatically on the server using remoting.
The client machine has no SQL Server 2005 component installed..
This is the error I get from my exe application:
Microsoft.SqlServer.Dts.Runtime.DtsPipelineException: Retrieving the COM class factory for component with CLSID {E44847F1-FD8C-4251-B5DA-B04BB22E236E} failed due to the following error: 80040154.
---> System.Runtime.InteropServices.COMException (0x80040154): Retrieving the COM class factory for component with CLSID {E44847F1-FD8C-4251-B5DA-B04BB22E236E} failed due to the following error: 80040154.
at Microsoft.SqlServer.Dts.Runtime.Application..ctor()
And this the Remote component:
<Serializable()> Public Class SSISComponent
Inherits MarshalByRefObject
Public Function LaunchPackage(ByVal sourceLocation As String, ByVal serverName As String, ByVal packageName As String, ByVal packageVariables As SortedList) As Microsoft.SqlServer.Dts.Runtime.DTSExecResult
Dim packagePath As String
Dim myPackage As Package
Dim integrationServices As New Application
' Combine path and filename.
packagePath = Path.Combine(sourceLocation, packageName)
Try
If integrationServices.ExistsOnDtsServer(packagePath, serverName) Then
myPackage = integrationServices.LoadFromDtsServer(packagePath, serverName, Nothing)
Else
Throw New ApplicationException( "Invalid package name or location: " & packagePath)
End If
If Not packageVariables Is Nothing Then
For Each de As DictionaryEntry In packageVariables
myPackage.Variables(de.Key).Value = de.Value
Next
End If
LaunchPackage = myPackage.Execute()
Catch ex As Exception
Throw
End Try
End Function
End Class
This is the code that call the remote component to run the package on the Server:
Public Class Test
Private Function LoadAndRunPackageRemotly(ByVal PackageName As String, ByVal PackageVariables As SortedList) As Boolean
Dim launchPackageService As New SSISComponent
Dim packageResult As Microsoft.SqlServer.Dts.Runtime.DTSExecResult
Try
packageResult = launchPackageService.LaunchPackage(Me.SSISPackageLocation, Me.SSISServerName, PackageName, PackageVariables)
If packageResult = DTSExecResult.Success Then
LoadAndRunPackageRemotly = True
Else
LoadAndRunPackageRemotly = False
End If
Catch ex As Exception
Throw
End Try
End Function
End Class
I would like to inform that everything run inside a thread because the Package do a lot of things...
When I build up the setup for the exe inside the Detected Dependencies there are several DLL related to SQL Server (Microsoft.SQLServer.DTSRuntimeWrap.dll etc..etc..) that I don't register ..
Tis could be the problem?
If yes, somebody can tell me which one I have to register ? as COM or COMRelativePath?
Some other hints?
This error could be also caused by wrong permissions on the ssis?
Thank you very much for any help...
p.s. To use sql server agent will be my last option!!
Marina b.
Performance Issues, Size Of SSIS-packages
Feb 7, 2007Hello!
I have a general question concerning the impact on the performance of massive parallel data imports in one SSIS-package.
We have a Database on a SQL2005 SP1-Server (2 Xeons 3,8 Ghz, 4GB of RAM) for a report web-app which is updated every day with data of the last year/3 years. The data is extracted from several different DBs on multiple machines at different locations. Right now, there are imports/transformations from 7 companies at 3 locations. The table has ~80 columns and about 2 Mio. Rows. I built a SSIS-Package with one companies import and added the others by c&p-ing all the tasks in the package and changing connection parameters and values. Soon there will be 6 more companies to do imports with, and there will possibly be about 20 some day.
Now, when these 7 imports run parallel, there are 3 simultaneous imports from the same Source Server. Sometimes one of these imports seem to hang up. I cannot reproduce it, when I run the package 2 or 3 times, itīs gone. So I put some of the imports in line to reduce the parallel working tasks to 4. Then the problem dissappears. The "MaxConcurrentExecutables" Value is set to 6. "Retain same connection" is set "TRUE".
My questions, regarding stability and performance, are:
1.) Is it better to do those imports in seperate packages, if yes can I schedule multiple packages to execute parallel at the "SQL Server Agent"?
2.) Or should they be combined in one package, running (partly) parallel?
3.) What is the appropriate value for the "MaxConcurrentExecutables" Value and what options do I have to speed up those imports?
SLow Access Times
Aug 3, 2007Hii,
A bit new to SQL 2k5 but here goes, I recently installed SQL 2k5 on a preety decent box with about 4GB Ram and created a database called PointOfSale. As probably obvious by now, the application that accesses the Database is a PointOfSale application.
The store sells ladies clothing items, of which each is barcoded.
When an item is scanned via the barcode reader into the application, it takes like about 10-15 seconds for the item information to appear. Of course the item information resides on the database.
There are three registers and the same thing happens on all. I am running sp 2 for sql2k5 already.
Any thing I can do that would reduce the 15 sec delay significantly?
Any help at all would be appreciated immensley.
Regards,
Burt.
SQL Query Load Times
Feb 10, 2006I have an VB.NET web app which performs a fairly complicated SQL query. It seems in the morning, the 1st time the page is loaded (and query executed) it takes up to 10-15 seconds to complete loading. Sometimes it even times out. However anytime after that, the page loads up (even from another computer) in about 4-5 seconds. Can someone explain the reason for this and how I might fix the load times in the morning?
Thanks
Difficult Times Deploying A Few Packages To SQL Server And Running As A Job
Jan 4, 2007Hi Guys!
I have created a big list of packages, some calling others. They all work fine from my computer using Visual Studio.
When I try to deploy them (building them with deployment turned on and running them either directly from Management Studio or as a job) I get the errors with the password of connection strings. From what I read so far its the encryption process that kills it.
I have tried to add a password to some packages, but it still didnt work (only when run directly on my computer in management studio after deploying to SQL Server, but not as a job).
I have tried to change ProtectionLevel to SecurityStorage, wouldnt let me save in Visual Studio (I understand it is ot allowed in VS because you are saving to File System, how the hell am I supposed to save it to anything else? why is it showing there if its not even valid?).
If anyone can please give me the steps to doing it properly, that would be awesome. I simply need to run the packages from SQL Server! thats all! I have no idea why it has to be soooo difficult :/
Report Performance Slows Down As SP Is Run Multiple Times.
Oct 26, 2007Hi all,
In my project, I have a website and through that, I run my reports. But the reports take a lot of time to render. When I checked the profiler, it showed that the SP for the report is run around 4-5 times. Due to this, the report rendering takes a lot of time.
When, I ran the SP with the same set of Parameters in Query Analyser, it ran in around 18 seconds. But when I ran the report from web interface, it took around 3 minutes to completely show the data. And the SP has been run 5 times.
I am having serious problems with Report's performance because of this. Many a times, report just times out. I have set the timeout as 10 minutes. And because the Sp is run 5 times, the report times out, if there is huge amount of data.
Any help would be appreciated.
Thanks in advance.
Swati
Calling SSIS Packages From ASP.NET - Packages With File System Tasks End Abruptly
Jan 9, 2007I've run into a problem with SSIS packages wherein tasks that write or copy files, or create or delete directories, quit execution without any hint of an error nor a failure message, when called from an ASP.NET 2.0 application running on any other machine than the one where the package was created from. By all indications it appeared to be an identity/permissions problem.
Our application involves a separate web server and database server. Both have SQL Server 2005 installed, but the application server originally only had Integration services. The packages are file system-deployed on the application server, and are called using Microsoft.SqlServer.Dts.Runtime methods. For all packages that involve file system tasks, the above problem occurs.
When the above packages are run using the command prompt (either DTEXEC or DTEXECUI) the packages execute just fine. This is expected since we are using an administrative account. However when a ShellExecute of the same command is called from ASP.NET, the same problem occurs.
I've tried giving administrative permissions to the ASPNET worker process user to no avail.
I have likewise attempted to use the SQL Server Agent job approach but that approach might not be acceptable for our clients since it means installing SQL Server 2005 Database services on the application server.
I have read the relevant threads in this forum, namely http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1044739&SiteID=1 and http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=927084&SiteID=1 but failed to find any solution appropriate for our set up.
Anybody got any idea on how to go about this?
Slow Access On Page Load
Nov 15, 2006I have one aspx page that is very slow loading it is connected to SQL 2005 database. I am looking for any preformance tips out there
View 3 Replies View RelatedValidation Of For Loop Very Slow On Load
Aug 31, 2007Hi there,
I have recently added a nested for loop into my SSIS package (The 'parent' for loop loops through years with a variable @DAT_YEAR, the 'child' for loop loops through months of those years using @DAT_MONTH). The task contained within the 'child' loop is a data flow transformation which connects to a source ODBC database and writes out data to a text file a month at a time, supplying the @DAT_YEAR and @DAT_MONTH variables as expressions in the SqlCommand statement.
The logic of the data flow seems to work fine, however, I notice that when I load the package now, it seems to take forever (> 5 minutes, certainly) to regain control in Visual Studio, and the status message at the bottom left of the VS window reads "Validating For Loop Container".
I know that you can work in 'offline' mode to temporarily turn off the validation, but I cannot manage any of the connections in this mode - why is it taking so long? Is it actually doing the loop and validating the ODBC connection each iteration?
yours puzzled,
t.
Initial Report Load Is Slow
Apr 14, 2008Each day, the first user who launches our RS reports always gets a long wait time. Subsequent report launches are normal.
Does anyone know what is going on? If yes, what is the remedy?
Thanks
Integration Services :: Remotely Execute Packages On SSIS Server - Packages Are Deployed In File System
Apr 22, 2015We manage some SSIS servers, which has only SSIS and SSIS tools installed on them and not the sql server DB.
SSIS packages and configuration files are deployed on a NAS. We run the SSIS packages through DTEXEC by logging in to the server.
We want to allow developers to run their packages on their own on the server, but at the same time we dont want to give them physical access on the server i.e we do not want to add them into RDP users list on server properties. We want them to allow running their packages remotely on the server.
One way We could think of is by using powershell remoting and we are working on that. But is there any other way or any tool already present for the same.
Sp_fetch Executed A Lot Of Times, Decreasing Performance - ASP + SQL 2005
May 23, 2008Hey,
In a ASP application, which used Access and is now being migrated to SQL 2005, I have a medium-sized query with many inner joins. I optimized the tables, with indexes, etc, and if I run the query on the SSMS it returns me like 55 lines in less then a second. On the other hand, when the ASP page passes the query to the SQL 2005, it takes 10 sec. to get the data. Using the Profiler I found that the sp_fetch is being executed a lot of times. It looks like the query is decomposed in smaller pieces and the rows as selected one by one. I'm using OleDb.
How to make the ADO (the culprit in my opinion) to execute that query at once?
Thank you.
Stored Procedure - Page Load Slow
Jul 25, 2006Hi all,
I have a webpage with a Datagrid that populates using a table adapter from a Stored procedure that exists in my SQL Database...If I run the Stored procedure in SQL Directly then it takes 20 Secs to return all records...If I run the webpage then it takes just over 20 Secs..
Great you say..But If I have the sorting option set in ASP.net and I click on a column to sort then off the page goes for another 20 secs to sort the data..
Is there a better way to do what I am doing here that will speed up the page load..
Ie..the data is returned once and then sorted...
Is it Better / Quicker for me to create a table using the stored procedure and link to this from the website..Updating the table every couple of minutes ?
Any advice please ?
Ray..
Load Fact Table, Very Slow With Lookup - Data Flow Transformation
Apr 9, 2008
Hello,
I have developed some packages to load data into "Fact" tables in the data warehouse.
Some packages are OK, other ones not. What is the problem?: some packages load fact tables with lots of "Lookup - Data Flow Transformation" into the "data flow task" (lookup against dimension tables) but they are very very slow, too much slow to be choosen as a solution.
Do you have any other solutions to avoid using "Lookup - Data Flow Transformation"? Any other solution (SSIS, TSQL and so on....) is welcome to speed up the Fact table loading process.
Thank in advance
Slow Performance
Mar 24, 2007Hi all...
I need urgent help, about someting:
i've developed and deploy an aspnet web site (data works with sqlserver), but after a few minutes working with some users, the permormance slows and stop the site.
please help me what should i do......
Slow Performance Using DTS Sql 7
Mar 30, 2000I need to transfer a database from one server to another, I'm using the DTS utility because the servers have different sort orders. our database size is about 5GB which include about 2500 tables. Using DTS is taking many hours to transfer all objects and data. is there a better/faster way to do this?
Any help would be appreciated, Thank you
Slow Performance
Mar 31, 1999Help! I am new to SQL (6.5) and am designing a small database on a small LAN.
Everything seems to work fine until more than one user gets in the database.
It was originally an Access database and we upsized some of the information to SQL.
I have not upsized all the tables yet, and am not sure if that is causing a problem.
Since we are new to SQL we are trying to make all the changes in Access then upsize
everything. If this is the problem though, I will not be hesitant in upsizing.
Any ideas?
Slow Performance With Sp2
Dec 27, 2001Since sp2 a number of applications, specifically web ado access has slowed very badly and I seem to be seeing a high cpu usage too. Internal jobs such as backups and table defrags are not affected. I'm on a 4 way box with ample hardware .. stats indexes etc. are fine .. client access is just slow. I suspect mdac but I can't pin anything down and it's driving me crazy !!!!!
I've even built a new box with a fresh install but to no avail.
help ???
Not Again - Slow T-SQL Performance
Aug 24, 2004I'm running the following statements. They have been running for 48 hours and counting... In performance monitor, I can see the system is steadily 100% disc bound. Any idea how to get status or ETA on these? Any idea how I can speed up such actions? This seems unusually slow. There are no errors in the Error Log. Does other DBMS systems (such as Oracle) handle such scenarios more quickly or with better status information?
INSERT INTO Domains (Domain)
SELECT DISTINCT Domain FROM Stages WHERE Domain NOT IN (SELECT Domain FROM Domains)
UPDATE Stages SET DomainID = (SELECT Domains.[ID] FROM Domains WHERE Domains.Domain = Stages.Domain)
Stages is 173 million records
Domains is 2.4 million records
This is running on pretty decent hardware:
Windows 2000 SP5
SQL Server 2000 Enterprise Edition SP3a
Dual 3.06 GHz Xeons with HyperThreading enabled (4 virtual CPUs)
4GB RAM (OS 3GB switch is enabled to give SQL Server 3GB of RAM)
70GB SCSI boot drive
Data/Transaction Log is on a RAID 5, 30 spindle, fibrechannel SAN
Slow Performance
Aug 30, 2006Hi - I have MS SBA2007beta and Office Outlook 2007 BCM Beta. Both use SQL2005 and they are both as slow to point of being frustrating. Outlook2007 without BCM is fine and every other application runs fine if no SQL server apps are present.
I have this running on a 2.8ghz desktop with 1G memory and a Laptop of 1.8ghz and 1.5g memory. The laptop is far faster.
When SQL apps are running everything is slowed.
Slow Performance Using A View.
Jan 31, 2000Hello,
I have the same database on two different servers. One for production and one for testing. A view that I use runs in less than 2 seconds on the test system, but takes almost 2 minutes on the production server.
What I have noticed is on the test server the view will use an index. The production server ends up scanning a whole table. All indices are the same on both machines for the tables involved and I have updated the statistics. I even went through the process of creating a new table with its indices for the table that is being scanned. Both machines have had service pack 1 installed on them.
Any ideas?
Locks And Slow Performance
Jan 28, 2003An Application Times out with tons of SQL Server Locks but there is no log of any errors in SQL Servers. I checked all the Indexes and I reindex it but still same issues. CPU is 100% full at that time and users unable to do anything. Any ideas on how to fix it
View 1 Replies View RelatedSlow Performance In Sql Server
Jun 5, 2004Hi
We are facing performance related problem using Sql server 2000.
We have one stand alone P4 Pc (128 ram) and around 30 users access the sql server through network.
We have written our aplication in VB 6 and backend as Sql Server 2000. We have used Stored Procedure where ever necessary. We have used cursor location as Server side.
When we start with 5 users it is not slow, when all the users say 30 comes in it is slow down.
Can some one help to find out what is the problem.
Thanks
Help With SLOW SQL Server Performance
Nov 29, 2005I'm still new to SQL Server so some of my lingo/verbage may be incorrect, please bare with me.
The company I work for relies strictly on ASP and SQL Server for 85% of it's daily operations. We have some Access projects and some VB projects as well, but for the majority it's ASP and SQL Server.
Previously we had 2 T1 lines with something like 3MB a piece and a handfull of Dell Servers. Our main server is also a Dell running Windows Server 2003 and is hosted through a reputable company here in town. They have a host of fiber lines running all over so I know we're getting good throughput. We've actually just upgradded to a DS3 but we're still working out the kinks with that. Anyway, I just want to eliminate that up front - we have great connection speeds.
The problems lies, I believe in our database design. The company supposedly had a DBA come in and help setup the design some 3 or 4 years ago, however even with my limited knowledge I feel like something is just not working right.
Our main table is "Invoices" which is obviously all of our Invoices, ever. This table has an Identity field "JobID" which is also the Clustered Index. We have other Indexes as well, but it appears they're just scattered about. The table probably 30-40 fields per row and ONLY 740,000 rows. Tiny in comparison to what I'm told SQL Server can handle.
However, our performance is embarassing. We've just landed a new client who's going to be brining us big business and they're already complaining about the speed of their website. I am just trying to figure out ways to speed things up. SQL is on a dedicated machine I believe with dual Xeon processors and a couple gigs of ram. So that should be ok. THe invoices table I spoke of is constantly accessed by all kinds of operations as it's heart of what we do. We also have other tables such which are joined on this table to make up the reporting we do for clients.
So I guess my question is this. Should the Clustered Index be the identify field and is that causing us problems? We use this field alot for access a single Invoice at a time and from what I understand this makes it a good Clustered Index, because the index IS the jobID we're looking for. But when it comes time to do reporting for a client, we're not looking at this field. We just pull the records for that Clients Number. And we only have 1400 clients at this point. So if we were to make the "ClientID" field the Clustered Index, it would much faster to Zero in on the group of Invoices we wanted because the ClientID is ALWAYS included in our queries.
But because a "DBA" came in to design this setup, everyone is afraid to change it. I guess it's hard to explain without people sitting here going through the code and look at the structures of all our tables - but I guess what I need is like a guide of what to do to easily increase performance on SQL Server and the proper use of Clustered and Non-Clustered Indexs and how to mix and match those.
Sorry I wrote a book.
Ideas? This place has always helped me before, so thanks in advance!
MSSQL2000 Slow Performance Over VPN
Feb 11, 2007Hi,
I'm executing a stored procedure in my local LAN which executes another one in a loop and I update a Table. The number of loops is about 6300.
This operation takes about 25 seconds in my local LAN.
Then I try to execute though in a VPN which has an upload speed of 256 kbps. I open query analyser connect to the remote server which is must faster than mine and I just write exec mystoredprocname in order to execute the procedure. The performance is very very slow.
In 7 minutes 180 loops are completed out or 6300.
I really cannot understand this. What is the reason of such slow perfomance?? My ADSL model displays no activity when the procedure is executed. I just use the PRINT method in MSSQL in order to display the progress of the operation. I tried to comment it out but with no difference.
I also use SET NOCOUNT ON in order not to display the update results.
Can someone explain me the cause for this? Are there some tricks in order to improve the performance when a slow connection is used like a ADSL with a static IP? It seems that something wrong is happening here.
Best Regards,
Manolis Perrakis
Database Performance Is Very Slow
Jul 31, 2007dear friends,
i have dropped many objects and recreated in a database.suddenly my database became very slow. so please any one of friends give solution.
Slow Performance Of Like Operator
Jul 23, 2005I have a stored procedure that has some problems with slow performance.The procedure has 2 parameters@firstnamevarchar(100)@lastnamevarchar(100)These parameters can have values like a%, that is wildcard searches.The strange thing is that if one of the parameters has the value %, andthe other one a%, the performance is very bad.If i subsistute the variables with exactly the same values hardcoded inthe where-clause, the query is very fast.If both variables has some characters prepending the percent sign, theperformance is better.SELECT distinct u.user_id, u.username, u.status_idFROM statusnames sn, statuses s, users u, users_persons up, persons p,users_roles urWHERE p.lastname like @lastnameAND p.firstname like @firstnameAND s.status_id = u.status_idAND sn.statusname_id = s.statusname_idAND u.user_id = up.user_idAND up.person_id = p.person_idWhat makes SQL server behave so differently with variables andhardcoded values?
View 6 Replies View RelatedSlow Performance In SQL2005
Feb 19, 2008Hi,We have shifted one of our data processing to a new instance of SQL2005. For this a new DB was created in SQL2005 and it is not anupgrade of SQL200 DB.The data processing application is a VB6 application that runs a batchprocess to insert about4.5 million records everyday in this SQL 2005 database. I am usingADO(ver 2.6) Connection object to execute T-SQL insert statements toinsert the 4.5 mill. records.This same application used to insert the data in the SQL 2000databaseand used to take about 2.5 hours to do so. After moving it to SQL2005 the performance has really detriorated and it now takes morethan6 hours.Is there any configuration change that I need to do in SQL 2005?Please help.Note: There is also an instance of SQL 2000 running in the sameserver.TIA,Tawfiq
View 3 Replies View RelatedPerformance Of Dataflow Too Slow ___
Apr 12, 2007I was transfering more that 100,000 records from flat file to sql table
It took about 1 hour.Is this the way it is?????i used oledb command.
As the data passes by i got to insert to several table.Like i insert some of incoming data to one table then get the key from that table and insert rest of the data with the key field from previous table to another table.
In this case i felt OLedb would be best as we can use query.
I cannot use oledb destination as it has only error output(to insert some of incoming data and i want to have a look up to get the key but oledb des has only error output)
i cannot use sql destination as the database is sql server 2000.It dosent let me.
How can i increase the performance????Please let me know