Dtexec - Can Run In A Job (CmdExec) But Not From Xp_cmshell
Nov 15, 2007
Hi all,
I've working a while, not at full time, but seeking the solution...Here what I want to do and what I've done till now:
I want to build a sql job, so I can run a package (witch loads 2 excel sheets into 2 tables) passing "dynamic" parameters, like convert(varchar,getdate(),112) in the format YYYYMMDD. From what I've found, I can do this with an Stored Procedure, which first set this variable, building a statement so it can be run by xp_cmdshell. For example:
And this statement runs with no problem in the SP until it reaches the step of loading the Excel Sheets into tables. Here it gives an error. It's about the JET driver. From what I've read in the forum and from the output error the problem seems to be that this statement executes the dtexec.exe 64bits, even the suggestion to change the property Run64BitRuntime set to False, it stills running from the dtexec.exe 64bits.
So, I changed the statement to point to dtexec.exe 32bits:
But executing this statement it does not even run. The errors are:
'C:Program' is not recognized as an internal or external command
operable program or batch file
This was very wird to me, because this was very mentioned by the moderators.
So, I copy this same statement and created a job with a step type of "Operating System (CmdExec)" and it runs great...With no problem with the extraction from excel source.
Now my questions are:
1. Why the step job CmdExec recognizes the path of dtexec.exe 32bits ('C:Program Files (x86)Microsoft SQL Server90DTSBinnDTExec.exe ), but trying to run with xp_cmdshell it gives the error mencioned above. And if there is another way to set dtexec.exe 32bits besides this way?
2. If I cannot run it throught xp_cmdshell, how could I pass a parameter like convert(varchar,getdate(),112) in the format YYYYMMDD instead of the static parameter 20070101.
I'm a bit confused. On the command line of the job step property I entered dtexec /SQL... and got an error saying file not found, I assumed dtexec itself couldnt be found. So I tried /SQL .... by itself and got something that looked more like a security error. If I make the step property type "ssis" job appears to run fine, I receive my pkg's on success (rather than on failure) email but I know everything isnt fine because even if no data is ETL'd, first executable is supposed to (and always has in client) insert a row into an audit table and it doesnt. If I set job step "type" to t-sql and simply db email myself with t-sql command, everything is fine.
The first question is "Wouldnt dtexec need to be specified, how else could sqlagent know what I'm trying to run?" If answer is yes, what's wrong with my syntax or environment?
I use this in a script window in SSMS: master.dbo.xp_cmdshell 'bcp myDB.dbo.DBlist out C:myCSV1.txt -c -t -Uusrtest -Pmypassword,' I get this error: User name not provided, either use -U to provide the user name or use -T for Trusted Connection any help pls
I have two servers A and B, both running SQL 7.0 SP3 on NT4 (SP6a). I need to run an hourly job on Server A that copies a file to Server B using xp_cmdshell. (I'm trying to set up log shipping where Server B will be a warm standby). As a test I tried running this cmd from my workstation (I have Sql Server installed on my workstation) and it worked: xp_cmdshell 'dir ServerBDmssql7ackup'. However, when I log in to Server A using sa account and attempt to run the same query, I get the error 'Access Denied'. The MSSQL service on Server A is running under the system account.
I have a serious problem with my SSIS Package while executing using 32-bit DTExec and 64-bit DTExec.
Here are the details:
Environment:
Windows Server 2003 64-bit (Build 3790: Service Pack 2) SSIS 32-bit & 64-bit installed SQL Server 2005 (Microsoft SQL Server 2005 - 9.00.1399.06 (X64) - RTM)
SSIS Package details (compiled in 64 bit)
Script tasks only Microsoft Visual Basic .NET (using TRY...CATCH block) PreCompileScriptIntoBinaryCode = TRUE Run64BitRunTime = TRUE
Execution
Batch file that uses DTExec to execute the Package.
SCENARIO I am trying to exeucte the above SSIS package using both 32-bit and 64-bit DTExec to make it failure by providing invalid connection string. Here are the details,
Wrong connection String using 32-bit Execution
While establishing the connection the error message has been nicely captured in to my Exception block and writes into the log file.
Wrong connection String using 64-bit Execution
While establishing the connection the error has not been catpured anywhere (although I have TRY CATCH block) and it haults there itself with the message "Process is terminated due to StackOverflowException". Later I found that the error is due to the connection string along with the unhandled exception.
Please suggest any one of the following my findings, also if you have any other advice would be very much appreciated.
1. Shall I go ahead and fix the issue by handling those unhandled errors? (e.g Appdomain, application). I tried several but still not working using 64-bit DTExec.
2. Shall I go ahead and use 32-bit DTExec to execute the package? If so, is there any other major issue...like performance or anyother bug?
P.S: We cannot apply any service pack for SQL Server 2005 at the moment. Sorry abt it. If you have any specific hotfix for DTExec (without affecting SQL Server) then we can decide.
Sorry for the lengthy one and Thanks very much for you help in advance .
I have tried to make a DTS task running a batc-file on the same server as the SQL program is running using the CmdExec command.
My Command is as follows:
CmdExec serverpathatchname.bat
The problem is that everytime the step runs it report the following error:
"The system cannot find the file specified."
In the bat file I am also trying the de-map/map a network drive with another loginname/password than the one used on the SQL-server -> is this i problem?
Hope to hear from you soon, this is a bit urgent!!
Please provide tips to this adresse: qtip@bigfoot.com
I have to verify a .CSV file exists before I run a BULK INSERT. I am using XP_FileExist in SQL 2000 to accomplish this. After the Bulk Insert is completed and validated, I need to Rename the file and Move the file to archive the folder. For testing I figure If I can rename the file I can move it. I suspect I have permission issues and need to provide the SQL Server Agent permissions to this folder and file. I have my PC setup as a SQL 2000 Server and am attempting to get this step only working on my local machine. I created a nightly Job that remanes a file that I created in a Job and that is all it does. I am running the Job as SA but am still having issues.
The step being executed by the Job is "Ren C:MyTestFile.csv C:MyTestFile1.csv" (with the quotes). If I run this statment (without the Quotes from a command prompt, the file is renamed.
I have set the Type as "Operating System Command (CmdEXEC)". The Job history shows "The process could not be created for step 2 of job 0x71D51027F920A140A2913234DB7FF509 (reason: The system cannot find the file specified). The step failed."
As I said, I suspect that it is a permissions issue as the command works from the command prompt. What is the windows account that the SQL Server Agent uses to commit these commands? I added "Everyone" with Full access to the folder and I still get the same failure.
I would appreciate any assistance anyone could provide. Thanks in advance!
Hi All, In creating 'steps' in JOBS, is it possible to execute many DOS CmdExec in one step, instead of creating several steps with a single DOS-cmd in each. For example:
If I created a job executing those 3 functions in 3 separate steps then it works fine. But if I put all those 3 DOS command in one step, it won't work. Somehow, SQL doesn't 'understand' it should execute after the end of each command OR I missed something here (apparently so!). I know if I put all those 3 DOS commands into a DOIT.BAT and execute it, it will work. But I want to use SQL Job to schedule it to run on a regular basis.
Anyone has run into this same problem? Thanks in advance. David Nguyen.
WE have a number of SQL 6.5 servers all of which run many scheduled tasks. However on one all tasks which use cmdexec to run batch files fail after 2-3 seconds. The history returns : 'No message' and the error logs just indicate that the tasks have faILED.
The identical tasks run on servers with what appears to be identical configurations and setup.
I would like to trap a return value from a cmdexec that is scheduled. The cmdexec returns 0 if it is a success and something other than 0 if it doesn't.
Can I raise an error from a command file. The command file calls a console application ( i.e. no interface ).
I have a Job that runs a cmdexec job step which executes a batch file in sql server 7.0 that runs fine In sql 2000 when i try to run that job it gives the following error and fails
---- the process could not be created for step 1 of job 0x677EF599B13FA743AA2D501D4C211AC4 (reason: The system cannot find the file specified). The step failed. ------ In fact i am not able to execute any cmdexec job in sql 2000 The owner of the above job is sa Does it have to do with SqlAgentCmdExec account which is set to corporate/administrator and has required permission.
Thanks for the invitation to post a question, so I will post one.
I need to create a job step that uses cmdExec.
This is the command line I entering: D:odbcTimeClockUpdatesinReleaseTimeKeepingNo nLogouts.exe
When I run this job the job fails. When I look at the job history, the only information I get is the date and time, user that ran the job and the fact that it failed. I haven't been able to get any CmdExec job to run at all. Can anyone tell me what I'm doing wrong?
Facts. 1. This exact same command is used by my network administrator using windows scheduler on the server. The only reason he wants me to create an sql server job is because it's mostly sql functions. 2. I know the Sql Server agent is running, because I have other jobs that are run. 3. I have verified that I have permission to run the file because I can go to the actual directory and run the exe. 4. Do I need to enclose my command in quotes i.e. “D:odbcTimeClockUpdatesinReleaseTimeKeepingN onLogouts.exe” 5. the path of the file I need to run is the path on the server and not the path on my local machine.
If you need any other information, please let me know Thanks for you help GEM
I want to copy a backup from the production server to another server periodically. The following is the DOS command and it works in DOS propmt d: is the local drive and g: is the mapping server --------------------------------------------------- "copy d:ackupackup_CustomerDB.dat g:mssqlackupackup_on_3Wednesdayackup_CustomerDB .dat"
I try to use the above command in the executive task with CmdExec as type, if failed. Then try on "cmd c/ copy d:ackupackup_CustomerDB.dat g:mssqlackupackup_on_3Wednesdayackup_CustomerDB .dat"
this time, it started to run with no error. But it run 40 miniutes and I cancelled it, since it should only run 10 minuts.
Can someone tell me how i can generate an exit code (Cmdexec). Ive tried running an .exe file in a job but when i do that the job hangs. When i check the history it says the exit code is somewhere in 27000. Now i'm wondering if it is possible for me to generate an exit code if the executable closes. PS. When i run the file from a batch file it works great
I have sql 2000 job that has been setup as a "Operating System Command (CmdExec) job. I am logged into the SQL Server as DomainNameSQLAdmin this domain account is part of the Administrator group on the SQL Server that the job in running on. This account has SysAdmin rights and is also starting the MSSQLSERVER Service and SQLSERVERAGENT Service on the same machine.
The job just copies files from one directory to another, here's the code.
D: cd MSSQLBACKUPAPAP_Primary xcopy *.* D:MSSQLBACKUPAP /s/y/d
When run as a job, with the owner of the job being DomainNameSQLAdmin the job fails with the following error message: Executed as user: DomainNameSQLAdmin. The process could not be created for step 1 of job 0x822E9AD29DCAAF4196369A46C7FE212A (reason: Access is denied). The step failed.
Here's the weird part if I open a command window on the sql server and run the batch it works fine.
I even tried executing the commands via xp_cmdshell but that didn't work either, I recieved the message: (1 row(s) affected) with the output being NULL and the file was never copied.
"Error executing extended stored procedure: Specified user can not login"
I have tried this through Enterprise Manager and get identical results, of course.
I have also tried all of the following: - different OS user accounts, including local system accounts with local admin rights; - assigning the OS account to a SQL login with System Admin role/rights; - specifically assigning the above SQL login with EXEC rights on the master.dbo.xp_CmdShell procedure; - verifying local security policy settings, as per the following link: http://support.microsoft.com/?id=283811; - pulling out my hair and banging my head against the wall.
I have created an Integration Services package on my development machine. The package contains a configuration file witch let's say is stored in c:projectsMyIntegrationServicesProjectmyConfigfile.dtsConfig (on my dev machine).
Then I have another "Production" machine where I import the SSIS package into an SQL database. I then create an sql-job with only one step, to run my SSIS package. This works fine if I configure the step to be an "SQL Integration services package" and configure it to use my configurationfile.
However I would like to configure this package as a CmdExec step. In the commandline, I specify /CONFIGFILE "d:....myConfigfile.dtsConfig" (the correct path on the prod machine). But it seems to be ignored, because when I execute the package I get an error telling me that the configuretion file c:projectsMyIntegrationServicesProjectmyConfigfile.dtsConfig cannot be found.
What I try to say is, it seems like it ignors the config-file I specify on the command-line and tries to reach the config-file on a location that's probably stored somewhere in the SSIS package from the time it was created on my development machine.
I have an SSIS package that runs fine through command pormpt although when I try to run it from a SQL Servr Agent Job CmdExec step it bombs out. Please help this has me stumped...the SSIS package uses an XML connection string so certain key settings such as connection strings and email info can be changed easily. Currently this is all on the same machine. I have not moved it beyond where I am developing.
On the command line I am using the following command...
Microsoft (R) SQL Server Execute Package Utility Version 9.00.1399.06 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved.
Started: 6:59:40 PM Progress: 2006-08-16 18:59:41.29 Source: Data Flow Task Validating: 0% complete End Progress Progress: 2006-08-16 18:59:41.29 Source: Data Flow Task Validating: 33% complete End Progress Progress: 2006-08-16 18:59:41.71 Source: Data Flow Task Validating: 66% complete End Progress Progress: 2006-08-16 18:59:41.73 Source: Data Flow Task Validating: 100% complete End Progress Progress: 2006-08-16 18:59:41.77 Source: Data Flow Task Validating: 0% complete End Progress Progress: 2006-08-16 18:59:41.77 Source: Data Flow Task Validating: 33% complete End Progress Progress: 2006-08-16 18:59:41.77 Source: Data Flow Task Validating: 66% complete End Progress Progress: 2006-08-16 18:59:41.77 Source: Data Flow Task Validating: 100% complete End Progress Progress: 2006-08-16 18:59:41.79 Source: Data Flow Task Prepare for Execute: 0% complete End Progress Progress: 2006-08-16 18:59:41.79 Source: Data Flow Task Prepare for Execute: 33% complete End Progress Progress: 2006-08-16 18:59:41.79 Source: Data Flow Task Prepare for Execute: 66% complete End Progress Progress: 2006-08-16 18:59:41.79 Source: Data Flow Task Prepare for Execute: 100% complete End Progress Progress: 2006-08-16 18:59:41.81 Source: Data Flow Task Pre-Execute: 0% complete End Progress Progress: 2006-08-16 18:59:41.84 Source: Data Flow Task Pre-Execute: 33% complete End Progress Progress: 2006-08-16 18:59:41.90 Source: Data Flow Task Pre-Execute: 66% complete End Progress Progress: 2006-08-16 18:59:41.90 Source: Data Flow Task Pre-Execute: 100% complete End Progress Progress: 2006-08-16 18:59:41.92 Source: Data Flow Task Post Execute: 0% complete End Progress Progress: 2006-08-16 18:59:41.92 Source: Data Flow Task Post Execute: 33% complete End Progress Progress: 2006-08-16 18:59:41.92 Source: Data Flow Task Post Execute: 66% complete End Progress Progress: 2006-08-16 18:59:41.92 Source: Data Flow Task Post Execute: 100% complete End Progress Progress: 2006-08-16 18:59:41.92 Source: Data Flow Task Cleanup: 0% complete End Progress Progress: 2006-08-16 18:59:41.93 Source: Data Flow Task Cleanup: 33% complete End Progress Progress: 2006-08-16 18:59:41.93 Source: Data Flow Task Cleanup: 66% complete End Progress Progress: 2006-08-16 18:59:41.93 Source: Data Flow Task Cleanup: 100% complete End Progress Progress: 2006-08-16 18:59:41.95 Source: Send Mail Task The SendMail task is initiated.: 0% complete End Progress Progress: 2006-08-16 18:59:42.09 Source: Send Mail Task The SendMail task is completed.: 100% complete End Progress DTExec: The package execution returned DTSER_SUCCESS (0). Started: 6:59:40 PM Finished: 6:59:42 PM Elapsed: 1.984 seconds
When I try to use the same command within SQL Server Agent Job using a CmdExec step I get the following error...
Description: The package is attempting to configure from the XML file "S:connectionscontacts.dtsConfig". End Info Warning: 2006-08-16 18:40:03.15 Code: 0x80012012 Source: contactsPackage Description: The configuration file name "S:connectionscontacts.dtsConfig" is not valid. Check the configuration file name. End Warning Warning: 2006-08-16 18:40:03.15 Code: 0x80012059 Source: contactsPackage Description: Failed to load at least one of the configuration entries for the package. Check configurations entries and previous warnings to see descriptions of which configuration failed. End Warning Info: 2006-08-16 18:40:03.20 ... Process Exit Code 1. The step failed.
I have a command line as following, with DTEXEC to launch the execution of a package and to set a value (13335) of an user variable called CIB (string type):
But I have got an error message saying that the object is not known in the package. My variable does exist in the variable window of the dataflow part. Thank you for telling me what to set so that the variable can be set by the command line.
I understand to schedule an SSIS package to run, I need to use the dtexec utility. I want to schedule the job right within SQL. I walked through how to set up the job, but I really could not decipher the proper syntax to use as a Step.
I have the following:
dtexec /File "C:...file.dtsx"
Is the the proper way to schedule and execute an SSIS package or is there some other way I should be doing this. The pacakge will run unattended nightly. I am using SQL Server 2005.
I have lots of Stored procedures in which we use 'dtsrun' to run DTS Packages. Now, after I upgrade the server to SQL 2005 will I have to change all the Stored procedure's to reflect 'dtexec' instead of 'dtsrun'?
Hi I have a SSIS package which pulls files from a network share and loads data into SQLServer Database. When I execute the application using DTExecUI , It runs fine without any issues , but where as when I run it using the command line arguments, It seems to go in sleep mode and nothing happens. I need to kill the package from Task manager. Following is the command I use to run my application
(Optional). Specifies a configuration file to extract values from. Using this option, you can set a run-time configuration that differs from the configuration that was specified at design time for the package. You can store different configuration settings in an XML configuration file and then load the settings before package execution by using the /ConfigFile option.
Does this mean that I can specify which configuration file to use during runtime? Or is just because I'm too desperate for that, I understood that way
The DTEXEC Utility has the capability of returning ReturnCodes which have specific meanings e.g.
ReturnCode=3 means: The package was canceled by the user.
Is it possible to set my own ReturnCode values?
For example, my Package contains a Script Task which contains code to search a folder for a file. If the file cannot be found then I make the Script Task fail and hence the Package fails. But, when the Package is invoked by the DTEXEC utility, then the ReturnCode is always set to 1. Is it posssible to set the ReturnCode from within the Package (in, say, a Script Task in the Event Handler) to a different value?
It is necessary to be able to set custom application-specific return codes because these return codes are passed to the Batch Scheduler which alerts Operations (Support) staff in the event of failure. Meaningful (custom) ReturnCodes expedite problem solving and are (usually) mandatory in large production environments.
If it is not possible to make a Package return my own returncodes via DTEXEC then can you suggest alternative solutions, please?
I'm trying to execute DTExec from a workstation and I got some help from a different group without luck maybe someone here can help me.
This is what I try so far.
1. My package run in command line from my sql box using dtexec with parameters. 2. Set the package with DontSaveSensitive and import into IS under MSDB with the same option setup. 3. Set package role to public. 4. Share DTS folder with everyone permission just for testing. 5. Execute the package from a workstation using //sqlServerbox/DTS/BINN/dtexec /dts "msdb/mypackage" /SER "MySQLServer" /set package.variables[myvariable].Value;"myvalue1,myvalue2" (myvariable is string and I can pass multiple values separate by commas) 6. Still getting error: Error: 2007-02-09 10:31:34.31 Code: 0xC0010018 Source: Execute DTS 2000 Package Task Description: Error loading a task. The contact information for the task is "E xecute DTS 2000 Package Task;Microsoft Corporation; Microsoft SQL Server v9; ? 2 004 Microsoft Corporation; All Rights Reserved;http://www.microsoft.com/sql/supp ort/default.asp;1". This happens when loading a task fails. End Error
I can't crack this problem and my Google-Fu is failing me. I am using DTEXEC from a stored procedure to set a global variable in an SSIS package and execute the SSIS package, but I am getting an error. I will include the SQL code and the error that I am getting:
Microsoft (R) SQL Server Execute Package Utility Version 9.00.1399.06 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. NULL Started: 3:17:02 PM Warning: 2008-03-13 15:17:02.76 Code: 0x80012017 Source: ProductionToDevelopment Description: The package path referenced an object that cannot be found: "Package.Variables[User:electMail].Properties[Value]". This occurs when an attempt is made to resolve a package path to an object that cannot be found. End Warning DTExec: Could not set Package.Variables[User:electMail].Properties[Value] value to 1. Started: 3:17:02 PM Finished: 3:17:02 PM Elapsed: 0.375 seconds NULL
All the examples I find over DTEXEC seem to follow what I am doing in the code and making variables in an SSIS package isn't rocket science, so I don't understand where I am going wrong.
I have seen people suggest indirect configuration as a solution, but when I set it up (using the environmental variable TEMP from the dropdown) it doesn't seem to help anything. I have used the exact path from the configuration wizard, though, so I'm pretty sure I have the path right.
HI, with a dataflow that has delay validation property = true, DTExec will not try to validate it when I call it with /validate option. Is there a way to see if the dataflow validate even though the delay validation property is set to true?
It was quite strange with dtexec.... The problem started with my package being scheduled under the SQL Agent. I have two packages... one with simple SQL tasks and another with complex loop,sql tasks and activex scripts. Both using the XML configuration files for those DB Connection strings, user names and passwords. First I tried to schedule the two packages under Sql agent using SSIS steps. Both failed and I found out that I should set the ProtectionLevel to non-default. So I set the protection level to 'DontSaveSensitive' and it works fine with the first package but the second package failed. After I tried many way (changing configuration files, setting and reinstalling the packages inside SQL DB etc..) I found a blog that I should try with CmdExe step in Sql agent scheduler rather than SSIS step. So I tried but still failing. the message said dtexec could not find the configuration xml file. Even I tried in the command prompt with the same command, it couldn't find the configuration. If I went into the directory where the configuration files are stored and run the command, it run fine. Seem like dtexec could not read the configuration from other directory. but still the sqlagent is failing... Somebody have a clue?
In my previous post here: https://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1044739&SiteID=1 Michael Entin provides a number of responses to my questions regarding programatic execution of remote SSIS packages.
Having experienced some significant reliability problems with the Microsoft.SqlServer.Dts.Runtime components from an ASP.NET process (the page either times out, or inevitably just stops responding), I have been prototyping the DTExec command option which Michael suggests as being a better approach to remote programability.
So, off I've been prototyping this all day today...
I have a stored procedure that wraps a call to xp_cmdshell which takes the DTS (DTEXEC) params as a big long argument. This scenario would hopefully allow me to call the sproc from an ASP.NET application.
The proc is deployed to a SQL 2005 machine running SSIS (which I now understand is a REQUIREMENT for targetting SSIS "remotely"). The package targets a seperate SQL 2000 machine and includes two connection managers which are set to use SSPI. I use configuration option files to allow for configurable connection manager target/sources.
In this scenario, it does not seem that the DTEXEC command runs in the same context as the caller. and as a result, a peculiar account called MACHINENAME$ is used (where MachineName is literally the name of the SQL 2005 machine). The account authentication fails (obviously) when the package tries to establish a connection to any of the connection managers because MACHINENAME$ does not exist on the connection manager servers.
Based on the following excerpt from the MSDN doc on xp_cmdshell, it would seem that MACHINENAME$ is probably the LOCAL SYSTEM, which is the process tha the SQL Server Service is running under:
When xp_cmdshell is invoked by a user who is a member of the sysadmin fixed server role, xp_cmdshell will be executed under the security context in which the SQL Server service is running. When the user is not a member of the sysadmin group, xp_cmdshell will impersonate the SQL Server Agent proxy account, which is specified using xp_sqlagent_proxy_account. If the proxy account is not available, xp_cmdshell will fail. This is true only for Microsoft® Windows NT® 4.0 and Windows 2000. On Windows 9.x, there is no impersonation and xp_cmdshell is always executed under the security context of the Windows 9.x user who started SQL Server.
Obviously, settting the ENTIRE SQL Server service to run as a fixed account or even a domain account is probably not appropriate for client sites. Any opinion to the contrary is welcome.
In reading Kirk Haselden's walkthrough for setting up a SQL Agent Proxy, this seems incredibly involved. Before I go through this exercise, can anyone validate that this is the way to go for doing SSPI?
A work-around is to use SQL Server Auth for the connection managers and use configuration files to try to obfuscate these details, but my preference would be SSPI/Windows Integrated.
Hi, I have a package which has 2 file system tasks and 2 data flow tasks all in a for each loop container. if i execute the package from the development studio its executes fine but when i try to run it from the command prompt using the dtexec utility..it just runs upto the first data flow task and then it hangs unexpectedly. Any help on this would be really appreciated.
