I have a serious problem with my SSIS Package while executing using 32-bit DTExec and 64-bit DTExec.
Here are the details:
Environment:
Windows Server 2003 64-bit (Build 3790: Service Pack 2) SSIS 32-bit & 64-bit installed SQL Server 2005 (Microsoft SQL Server 2005 - 9.00.1399.06 (X64) - RTM)
SSIS Package details (compiled in 64 bit)
Script tasks only Microsoft Visual Basic .NET (using TRY...CATCH block) PreCompileScriptIntoBinaryCode = TRUE Run64BitRunTime = TRUE
Execution
Batch file that uses DTExec to execute the Package.
SCENARIO I am trying to exeucte the above SSIS package using both 32-bit and 64-bit DTExec to make it failure by providing invalid connection string. Here are the details,
Wrong connection String using 32-bit Execution
While establishing the connection the error message has been nicely captured in to my Exception block and writes into the log file.
Wrong connection String using 64-bit Execution
While establishing the connection the error has not been catpured anywhere (although I have TRY CATCH block) and it haults there itself with the message "Process is terminated due to StackOverflowException". Later I found that the error is due to the connection string along with the unhandled exception.
Please suggest any one of the following my findings, also if you have any other advice would be very much appreciated.
1. Shall I go ahead and fix the issue by handling those unhandled errors? (e.g Appdomain, application). I tried several but still not working using 64-bit DTExec.
2. Shall I go ahead and use 32-bit DTExec to execute the package? If so, is there any other major issue...like performance or anyother bug?
P.S: We cannot apply any service pack for SQL Server 2005 at the moment. Sorry abt it. If you have any specific hotfix for DTExec (without affecting SQL Server) then we can decide.
Sorry for the lengthy one and Thanks very much for you help in advance .
I have a legacy extraction ("E") .NET 1.1 application that is still going to be the driving force for our ETL process. We are going to be utilizing SSIS for "T" and "L". Now, the "E" phase is running on an application server where we have .NET 1.1 framework installed and working. The SSIS packages are running on a separate SQL Server. The problem here is - how do we call the SSIS package and be able to pass in the right parameters from this .NET app that runs on a separate box? We would like to use DTEXEC to call the remote SSIS packages through Integrated Security. The SSIS packages are stored as File System packages.
I am executing a package via vb.net with package.execute using the IDTSEvents interface inheriting DefaultEvents. I am trying to emulate DTExec functionality. The package runs fine and I am catching events, but I don't get nearly as many info and progress events as are put out by DTExec. From my understanding DTExec also used the managedDTS application object, is that not the case? If it uses this does anyone have an ideas on how it is grabbing more events? For example I don't get these events that DTExec shows ..
Info: 2006-09-26 14:22:27.97 Code: 0x40016041 Source: DWB02130 Description: The package is attempting to configure from the XML file "D:SSISConfigurationsDatabase.dtsConfig". End Info Info: 2006-09-26 14:22:28.05 Code: 0x40016040 Source: DWB02130 Description: The package is attempting to configure from SQL Server using the configuration string ""Configurations Database";"[dbo].[SSISConfigurations]";"DWB02130";". End Info
I also don't get as many progress updates during validation, I basically get 0 and 100% while DTExec shows several 1%, 3%, 10%, etc.
I'm a bit confused. On the command line of the job step property I entered dtexec /SQL... and got an error saying file not found, I assumed dtexec itself couldnt be found. So I tried /SQL .... by itself and got something that looked more like a security error. If I make the step property type "ssis" job appears to run fine, I receive my pkg's on success (rather than on failure) email but I know everything isnt fine because even if no data is ETL'd, first executable is supposed to (and always has in client) insert a row into an audit table and it doesnt. If I set job step "type" to t-sql and simply db email myself with t-sql command, everything is fine.
The first question is "Wouldnt dtexec need to be specified, how else could sqlagent know what I'm trying to run?" If answer is yes, what's wrong with my syntax or environment?
I have a manager SSIS package that calls a generic loader SSIS package, via the 'Execute Process Task'. The manger package (amongst other things) creates a config file and and then executes DTExec passing the config specification in the Arguments property, i.e. '
It works as expected, but, I always get the warning message 'File/Process "DTExec" is not in path.', I know that it is because the exe runs and loads the correct package which then performs the correct actions.
Have I dropped the ball here? I cannot see any reason for the warning as DelayValidation==True, I could set RequireFullFileName==True, but I do not know where SQL is installed on the various nodes.
I developed my package in BI studio and tested runnning it through BI. Now when i have to deploy the package to main server it will be executed via batch job. This package requires an input file name too. Now when i am trying to run the package on my loacal box using following command on command prormpt i get error given below.
COMMAND USED BY ME: DTExec /FILE 'C:ProjectIntegrationServicesSFRIntegrationServicesinpkgSFRLocation.dtsx' /SET Package.Variables[InputParamFileName].Value;'C:DataDumpInputLOC00022'
ERROR MESSAGE: Microsoft (R) SQL Server Execute Package Utility Version 9.00.1399.06 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved.
Started: 1:29:54 PM Error: 2008-02-22 13:29:54.96 Code: 0xC0011007 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Unable to load the package as XML because of package does not ha ve a valid XML format. A specific XML parser error will be posted. End Error Error: 2008-02-22 13:29:54.96 Code: 0xC0011002 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Failed to open package file "'C:ProjectIntegrationServicesSFR IntegrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The speci fied path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of either providing an incorrect file name was specified when calling LoadPackage o r the XML file was specified and has an incorrect format. End Error Could not load package "'C:ProjectIntegrationServicesSFRIntegrationServices inpkgSFRLocation.dtsx'" because of error 0xC0011002. Description: Failed to open package file "'C:ProjectIntegrationServicesSFRInt egrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The specifie d path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of eit her providing an incorrect file name was specified when calling LoadPackage or t he XML file was specified and has an incorrect format. Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Started: 1:29:54 PM Finished: 1:29:54 PM Elapsed: 0.047 seconds
PLease let me know solution to creating batch job for executing this pkg with input param.
It looks like you can run an SSIS package via the DTEXEC command. You can also specifiy a configuraiton file flag on the command. Is there anyway to specify a configuraiton filter if you stored it into a table vs and xml file?
I've worked for awhile to get an SSIS package that connects to Oracle to succesfully run via DTExecUI from a 64 bit SQL Server. I completely understand that DTExecUI runs in 32 bit mode, but I need to initiate the package from a stored proc so I need to be able to utilize the 32 bit DTExec from either a xp_cmdshell call or from an sp_start_job (thanks to MatthewRouche for the suggestion). Both Ora92 and 10g were installed on the SQL Server.
Here's what I've tried unssuccessfully
1. SQL Job - Just calling the package from an SSIS step. I wouldn't expect this to run because it is 64 bit DTExec. 2. SQL Job - A job that calls the 32 bit version of dtexec from cmdexec. Here is 2 examples I have tried, the first is because I came across several postings saying that there is a problem with the "(x86)" for the Oracle driver.
\server_nameE$ProgramFilesLegacyMicrosoftSQLServerX8690DTSBinnDTExec.exe /DTS "PackageName" /SERVER "SERVER_NAME" /SET Package.Variables[PLT].Value;"8H" /CONFIGFILE "\SERVERConfigFile.dtsConfig" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EW
\server_nameE$Program FilesMicrosoft SQL Server (x86)90DTSBinnDTExec.exe /DTS "PackageName" /SERVER "SERVER_NAME" /SET Package.Variables[PLT].Value;"8H" /CONFIGFILE "\SERVERConfigFile.dtsConfig" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EW
Both of these give me the error below:
System.Exception: System.Data.OracleClient requires Oracle client software version 8.1.7 or greater.
I'm assuming that it is running in 32 bit mode because this was at the beginning of the job history log:
Executed as user: "XXX". ....3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved
3. I then tried to call the package from a batch file on a file server that had integration services installed on. Got the same error as above.
I'm obviously missing something if I can get it to work in DTExecUI but not DTExec 32-bit.
Any ideas? Below are some postings I have already visited several times:
Windows 64 bit and Oracle http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=358365&SiteID=1
Connecting to Oracle on 64-bit (x64) machine http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=159581&SiteID=1
One paragraph that stands out on this posting is:
Find the location of the application that is generating the error. Check the path to this location and see if it contains any parenthesis. If so, you must relocate the application to a directory without any parenthesis in the path. This didn€™t work for me, unless I was doing it wrong.
Error retrieving numeric data from Oracle 10g on 64 bit Itanium SSIS server http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1183296&SiteID=1
Connection Problems to Oracle Database http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1307460&SiteID=1
If this posting was complete, it looks like what I was looking for.
I developed my package in BI studio and tested runnning it through BI. Now when i have to deploy the package to main server it will be executed via batch job. This package requires an input file name too. Now when i am trying to run the package on my loacal box using following command on command prormpt i get error given below.
COMMAND USED BY ME: DTExec /FILE 'C:ProjectIntegrationServicesSFRIntegrationServicesinpkgSFRLocation.dtsx' /SET Package.Variables[InputParamFileName].Value;'C:DataDumpInputLOC00022'
ERROR MESSAGE: Microsoft (R) SQL Server Execute Package Utility Version 9.00.1399.06 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved.
Started: 1:29:54 PM Error: 2008-02-22 13:29:54.96 Code: 0xC0011007 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Unable to load the package as XML because of package does not ha ve a valid XML format. A specific XML parser error will be posted. End Error Error: 2008-02-22 13:29:54.96 Code: 0xC0011002 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Failed to open package file "'C:ProjectIntegrationServicesSFR IntegrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The speci fied path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of either providing an incorrect file name was specified when calling LoadPackage o r the XML file was specified and has an incorrect format. End Error Could not load package "'C:ProjectIntegrationServicesSFRIntegrationServices inpkgSFRLocation.dtsx'" because of error 0xC0011002. Description: Failed to open package file "'C:ProjectIntegrationServicesSFRInt egrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The specifie d path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of eit her providing an incorrect file name was specified when calling LoadPackage or t he XML file was specified and has an incorrect format. Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Started: 1:29:54 PM Finished: 1:29:54 PM Elapsed: 0.047 seconds
PLease let me know solution to creating batch job for executing this pkg with input param.
I'm running several SSIS-packages in stored procedure using dtexec. Actually, there is a software that is running the procedure, but that's not important here.
The problem: if some of the packages failes, the whole procedure does not fail and I don't know if all the packages are successfully completed.
Here is a sample list of ssis-packages from procedure:
1) We are migrating data from SQL Server 2000 to 2005 DB.
2) We are developing SSIS packages in client, build, deploy and install and copying to SQL Server 2005 server to execute for migration.
3) For this we encrypt all packages with password, then decrypt all while executing in a batch file using dtexec. This works fine.
4) What does not work is executing a master execute task in which are individual execute tasks. This is so since the other packages are still encrypted.
5) We decrypted them separately first using dtutil using Decrypt and Copy/Move option but while executing error is thrown that these other packages are still protected by password.
6) Do we need to decrypt, then encrypt using default package protection level and then decrypt again.
7) How exactly to just execute a master package when packages are developed on client.
We made SSIS package in dev environment in windows 2008R2 and SQL server 2012. Same packages were placed on SAN disks in cluster environment and are invoked with security contexxt of admin user and wih dtexec utility ( we call this using an sp). rarely but procedure completed i see the log text file by SSIS package and find the return code 5 means package is unable to load. i canst found what are the exact reasons for return code 5.
In one of my SSIS package (which connect to DB2) i save the DB2 username and password in SSIS packge itself by using option "EncryptAllWithPassword". When I try to run the packge using dtexecui.exe it works perfectly fine. It prompts me for package password and then run fine.
Now I am trying to run it using command line dtxece.exe and i get fooolwoing error. Please suggest and help!!! DTExec.exe /FILE C:pkgDB2Load.dtsx /CONFIGFILE C:ConfigpkgConfigMaster.dtsConfig /Decrypt mypassword /CHECKPOINTING OFF /REPORTING EWCDI
Error Message: ---------------------- Description: An OLE DB error has occurred. Error code: 0x80040E21. An OLE DB record is available. Source: "IBM OLE DB Provider for DB2 Servers" Hresult: 0x80040E21 Description: " SQL30082N Attempt to establish connection fa iled with security reason "17" ("UNSUPPORTED FUNCTION"). SQLSTATE=08001 "
Our SSIS packages use the Web Service Task to call services to send email and write package failure data to a department wide database. These Web Service Calls are failing with HTTP 401 errors. It was caused by the passwords for the HTTP connections not being saved when the SSIS pkgs were saved to .dtsx files. I have tried saving the package with a password and the EncryptSensitiveWithPassword protection option. This password can be supplied when the package is called from another package or a package is executed in Visual Studio but cannot be supplied to DTexec to execute the package in a job. DTexec does have a /Password parameter but it is rejected if the package is loaded from a .dtsx file. This appears to be a bug in DTexec. It only accepts the /Password parameter when the package is loaded with the /SQL option. Specifying /Password and /File is not supported.
Is this a known bug? Are there any workarounds? Has anyone successfully called a Web Service from SSIS executed via DTexec?
I had got the below error when I execute a DELETE SQL query in SSIS Execute SQL Task :
Error: 0xC002F210 at DelAFKO, Execute SQL Task: Executing the query "DELETE FROM [CQMS_SAP].[dbo].[AFKO]" failed with the following error: "The transaction log for database 'CQMS_SAP' is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
But my disk has large as more than 6 GB space, and I query the log_reuse_wait_desc column in sys.databases which return value as "NOTHING".
So this confused me, any one has any experience on this?
I have a master package, which executes child packages that are located on a SQL Server. The Child packages execute other child packages which are also located on the SQL server.
Everything works fine when I execute in process. But when I set the parameter in the mater package ExecutePackageTask to ExecuteOutOfProcess = True, I get the following error
Error: 0xC00470FE at DFT Load Data, DTS.Pipeline: SSIS Error Code DTS_E_PRODUCTLEVELTOLOW. The product level is insufficient for component "Row Count" (5349).
Error: 0xC00470FE at DFT Load Data, DTS.Pipeline: SSIS Error Code DTS_E_PRODUCTLEVELTOLOW. The product level is insufficient for component "SCR Custom Split" (6399).
Error: 0xC00470FE at DFT Load Data, DTS.Pipeline: SSIS Error Code DTS_E_PRODUCTLEVELTOLOW. The product level is insufficient for component "SCR Data Source" (5100).
Error: 0xC00470FE at DFT Load Data, DTS.Pipeline: SSIS Error Code DTS_E_PRODUCTLEVELTOLOW. The product level is insufficient for component "DST_SCR Load Data" (6149).
The child packages all run fine when executed directly, and the master package runs fine if Execute Out of Process is False.
I have a simple package in SSIS that reads approx 2.5million rows from SQL2k5 source and exports it into 4 Foxpro files. The way I do it is as below 1) use and oledb source to read data from SQL2k5 2) use a conditional split component to divide the source into 4 streams ( I do a recId % 4 = 0, 1, 2, 3 to give 4 streams) 3) use an OLEDB destination to export to the 4 prexisting foxpro tables.
This thing runs amazingly fast when I run(F5) it from Business Inteligence studio, but it is atleast 10 times slower when rightclick the package and execute using dtexec.exe
I can't figure out why! any clues??
I would elimiate the 64 bit /32 bit problems because export to foxpro only works in 32 bit mode and my package has been configured to run in 32 bit mode.
I have an application that needs to execute dtsrun from a remotemachine. Basically I have a batch file on the remote box, whichhandles the execute. This works fine on a machine with client toolsinstalled. My problem is that I can't install the client tools on theremote box.Does anyone know what I need to install on the remote machine to getthis to execute, without installing the client tools? I've triedcopying dtsrun.exe and dtsrun.rll, but that didn't work
I have stumbled on a problem with running a large number of SSIS packages in parallel, using the €œdtexec€? command from inside an SQL Server job.
I€™ve described the environment, the goal and the problem below. Sorry if it€™s a bit too long, but I tried to be as clear as possible.
The environment: Windows Server 2003 Enterprise x64 Edition, SQL Server 2005 32bit Enterprise Edition SP2.
The goal: We have a large number of text files that we€™re loading into a staging area of a data warehouse (based on SQL Server 2k5, as said above).
We have one €œmain€? SSIS package that takes a list of files to load from an XML file, loops through that list and for each file in the list starts an SSIS package by using €œdtexec€? command. The command is started asynchronously by using system.diagnostics.process.start() method. This means that a large number of SSIS packages are started in parallel. These packages perform the actual loading (with BULK insert).
I have successfully run the loading process from the command prompt (using the dtexec command to start the main package) a number of times.
In order to move the loading to a production environment and schedule it, we have set up an SQL Server Agent job. We€™ve created a proxy user with the necessary rights (the same user that runs the job from command prompt), created an the SQL Agent job (there is one step of type €œcmdexec€? that runs the €œmain€? SSIS package with the €œdtexec€? command).
If the input XML file for the main package contains a small number of files (for example 10), the SQL Server Agent job works fine €“ the SSIS packages are started in parallel and they finish work successfully.
The problem: When the number of the concurrently started SSIS packages gets too big, the packages start to fail. When a large number of SSIS package executions are already taking place, the new dtexec commands fail after 0 seconds of work with an empty error message.
Please bear in mind that the same loading still works perfectly from command prompt on the same server with the same user. It only fails when run from the SQL Agent Job.
I€™ve tried to understand the limit, when do the packages start to fail, and I believe that the threshold is 80 parallel executions (I understand that it might not be desirable to start so many SSIS packages at once, but I€™d like to do it despite this).
Additional information:
The dtexec utility provides an error message where the package variables are shown and the fact that the package ran 0 seconds, but the €œMessage€? is empty (€œMessage: €œ). Turning the logging on in all the packages does not provide an error message either, just a lot of run-time information. The try-catch block around the process.start() script in the main package€™s script task also does not reveal any errors. I€™ve increased the €œmax worker threads€? number for the cmdexec subsystem in the msdb.dbo.syssubsystems table to a safely high number and restarted the SQL Server, but this had no effect either.
The request:
Can anyone give ideas what could be the cause of the problem? If you have any ideas about how to further debug the problem, they are also very welcome. Thanks in advance!
I want to learn about developing SSIS packages. I have installed trial version of SQL Server 2005 Express Edition. what additional software do I need to install to develop SSIS packages???
I feel stupid for asking this but I can't seem to find where to install the SSIS tutorials referenced in msdn.. specifically the "Deploying Packages Tutorial". In the SQL Server 2005 Developer Edition DVD that I have, I've chosen to install all of the samples etc. I have the AdventureWorks and AdventureWorksDW and a whole bunch of samples in "C:Program FilesMicrosoft SQL Server90SamplesIntegration ServicesPackage Samples" but under the directory "C:Program FilesMicrosoft SQL Server90SamplesIntegration ServicesTutorial" I only have sample data for "Creating a Simple ETL Package".
I know... feels like a dumb question but I have run the install wizard on both the dev edition dvd and the msdn library dvd several times and looked at all the options and can't find anything about tutorials under the SQL server installation or the workstation component installation. Please help. Thanks.
I have just installed SP2 on our server and this has caused a number of SSIS packages containing encripted data (SFTP tasks purchased from /nsoftware) to stop working. When executing them from business development studio they work fine, but when I install them into the server I get this error "The Validate method on the task failed, and returned error code 0x80131500 (You must specify an SSH host.). The Validate method must succeed and indicate the result using an "out" parameter." The SSH host is specified correctly in all tasks, because as I said I can run them from the development studio without any error. Does anyone have a clue on how to resolve this? Thank you in advance.
I have installed SSIS on server running SQL 2000 but I don't see the Business Intelligence or SSIS. I could see the SSIS service is running but do not see VS or BI version of VS.
On my laptop I've got SQL2005 developper edition SP1 with 3 SSIS-jobs scheduled through SQL-agent.
These 3 SSIS-jobs are in fact maintenance plans which are also used on our production SQLServers.
These take backups (full, transactional), Cleanup old backups, reorganize indexes, update stats ...
One of these jobs runs every hour to take transaction logbackups. After installing SP2 this started failing.
When I opened the sources of that job in visual studio It seemed like the unit of thime was changed for cleanup jobs fe cleanup bak-files after 4 days changed into after 4 hours. The backup task for transactional backups which had the property to do that for all databases whas changed into specific databases which no database selected. Even when I changed that into All databases and saved it, when I opened that backup task again, this was changed back into specific databases with no database selected.
I uninstalled SQL2005 and reinstalled it, including SP1 and everything works well again.
I restored a backup from SQL 2000 onto a new SQL 2005 server. The database restored and the matching user who was the owner of that database in SQL 2000 was created as a user in SQL 2005 as well as being designated as the "schema" for that database. I was having trouble logging in with the user that was created after the restore, so I created a new user and assinged the new user as DBO of the new database, set up the new users default database and schema, and everything was working fine, my database driven application worked perfectly under that user.
Then I installed SSIS, as it was recomended as a solution to another problem. After installing SSIS I can not run queries without specifying a schema name. For example this query will not work, it returns table not found:
SELECT * FROM <table_name>
However if I specify the schema name like this, the results return:
SELECT * FROM <schema>.<table_name>
This screws up my database application as it does not include the schema name in its transactions.
I do not understand why installing SSIS would have changed this.
I have installed integration service on a SQL Server 2005 development server and could not start the service. Windows Server 2003 Standard Edtion (SP2) + SQL Server 2005 Developer Edition (SP2) are installed on the server. When trying to start the service from SQL Server configuration tool or directly from service, got message saying that the service did not start in a timely fashion. Nothing is logged to event log.
Found this this kb. http://support.microsoft.com/kb/918644 The problem description is similar but not sure whether it is the same problem.
When installing service pack 2 the installation would not allow me to select ssis or tools. ssis works as long as you are access the machine using a 32 bit remote client however we get the following error when trying to use ssis on the 64 bit machine. How can I fix this problem without reinstalling SQL server?
Error loading type library/DLL. Connect to SSIS Service on machine "<computer_name>" failed:
Think this problem was cuased by installing the service pack 1 hotfixes in the wrong order?
None of the information I have found about this issue has help.