Regarding Running Batch Files And Different Packages
Mar 19, 2007
I have created a master controller package which runs as follows
deletes all the log files -> deletes few flat files on different drives -> preprocess task(execute package task) -> c# executable (execute process tasks) -> postprocess tasks (execute package tasks)
i need a create a task just before the preprocess task with an user input asking whether he wants to run a particular batch file before proceeding to preprocess. if the user says yes it should run a batch file followed by preprocess tasks, c# and post process or else it should directly goto preprocess, c# and post process (neglecting batch file task)
I'm trying to run a simple batch file from a SQL job (SQL 7.0 sp4). No errors are received but the job does not complete. When I try to run the job manually, I get a message stating "Error 22022: SQLServerAgent Error: Request to run job my_job (from User DomainAdminUser) refused because the job is already running from a request by Schedule 127 (Schedule 1).
The services are running as a domain admin account.
We are in the process of upgrading to a new SQL 2012 server but we have many packages that load data from dbf files created with FoxPro into one of our databases. We have not converted the packages and run them with DTS but we get the following error:
Error: 2013-08-26 11:05:27.36 Code: 0xC0209303 Source: BenchmarkLoad Connection manager "OLEDB NPIONE.Investment.middleTierSQL" Description: The requested OLE DB provider SQLNCLI.1 is not registered. If the 64-bit driver is not installed, run the package in 32-bit mode. Error code: 0x00000000.
[Code] .....
I searched for OLEDB and ODBC drivers for SQL 2012 64 bit but cannot find any that is newer than the 2005 that we have. What can I do short of changing the source files to overcome this issue?
I just created batch file to restart analysis services and the job for executing this bat file. <<Batch File >> NET STOP "SQL Server Analysis Services (MSSQLSERVER)" NET START "SQL Server Analysis Services (MSSQLSERVER)"
And I did the following step in sql agent: Create the job > Select CmdExec Job > In commend section, I wrote CALL C:/RestartSSAS.bat
When I run this job, it seems everything went right but at the end, the job was filed. However, I don't see any error on this step since the service was stopped and started successfully. The below is the error message:
Executed as user: D20WKZC1SYSTEM. C:WINDOWSsystem32>NET STOP "SQL Server Analysis Services (MSSQLSERVER)" The SQL Server Analysis Services (MSSQLSERVER) service was stopped successfully. C:WINDOWSsystem32>NET START "SQL Server Analysis Services (MSSQLSERVER)"The SQL Server Analysis Services (MSSQLSERVER) service is starting. The SQL Server Analysis Services (MSSQLSERVER) service was started successfully. Process Exit Code 0. The step failed.
I've also tried to create the package using 'Execute Process Task ' and run it. And it's successful but when I schedule it in sql agent then it failes.
Any thoughts? Please advise me and thanks in advance.
Is it possible to import packages in batch into MSDB (in lieu of right clicking and importing each one by one)? We have a lot of packages that are going through a lot of changes so a batch import would save a lot of time.
I need to create a program that will run Client Access to download data from the AS400 to a flat file, then run SQL DTS to import the data into a table for use by another software package.
I've created a bat file that does that using the CA RTOPCB command and the SQL DTSRUN utility. Problem is that it appears that I need to first check to see if there is a table in SQL and if there is to delete it. Otherwise, rather than overlay the existing data in the table it adds to it.
Is there a way to issue a SQL drop table in DOS? Or am I missing something that could be done in SQL?
The DTSRUN is using a local package and CA is using a transaction request.
I've have been sent a large number scritps to update one of the databases i look after. Problem is there are hundreds of individal script files, all of which need to be run.
Is there a script / tool i could use, which would run all of the .sql files in a certain directory and update the database?
I can't seem to find anything built into SQL Server. Job schedular would seem to be the closest thing, but it would still require you select each script needed to be run.
Is it possible using T-SQL to run a batch file located on different server, ie. PC1 has SQL Server on it, PC2 has the batchfile, I need to run the batchfile stored on PC2 on PC1.
Hi, this code run fine from within sql server query window. I want to put this code in a batch file and run the batch file.. it did not work. someone told me the revised code is :
bcp sdnetpro..nbtorder11 out d:databtorder11.txt /c /t,/r -Sservername -Usa -Ppassword"
This is not working. Please if anyone knows how to fix this, I would appreciate. One more thing, how can I confirm that the bcp has successfully done. do I have to create a log file and if so what is the code to create this log file. thanks Ali
I want to keep applications off of my database server so I have set up an application server (APPServer1). On APPServer1 I have a batch file that bcp’s data from DBServer1 into DBServer2 and is being passed the server name of DBServer2. On DBServer1 I have mapped a drive to the directory of APPServer1 and have created a task to run the batch job and pass the server name. So here’s my problem: when the scheduler runs the job, the bcp to DBServer2 fails, because it can not find DBServer2. When I execute the exact same command line in a DOS Box on DBServer1, the bcp works fine. I have verified that the server name is being passed correctly to the batch job in both methods.
Hello, I am looking for some way to run a sql query from a batch file on the SQL server. The query needs a user entered parameter, if that can be accomodated it would be great. I read about some osql command & tried to run it but it kept saying "Server does not exist or access denied." Please give me the steps to go about this. Thanks, -R
I had created 2 packages... one is the parent package and contains a 50 iterations loop running a secondary package for each iteration... i had reached the following conclusion:
My package takes an average of 5 seconds from the time it ends executing one iteration and starts another... after about 30 iterations... my average time between end and start increases significantly to about 12 secs or even more...
All packages have delay validation set to false, and receive several variables from the parent package... Has for the logging, it is done to files based on a path coming from a variable in the parent package.
To execute the parent package i am using dtexecui.exe and i consider this behavior rather strange... had anyone experienced the same? Can anyone test this?
My environment is a 4 x64 processors with 8gb memory, so i guess its good enough to get 0 secs from end to start
I need to be able to query a db on server A, in a batch file, return the result (DateTime value) into a variable, and then use that date as a parameter in a query that I will query on server b.
how to catch batch id from a running stored procedure. My intention is that when we run store procedures in batch we are running a lot of procedures and I would like to log each run and if the same procedure is running several times per day I need to separate the runs by a "batch id" for the specific run. I have created a logtable and a logprocedure that logs the start and end of a procedure run and also some values for the run. So I'm trying to find a way of fetching the "batch id" that the sp is running so I can separate the runs when analyzing the logtable. I have looked at metadata tables and also in the table sys.sysprocesses but I cannot find BATCH ID.
I am new to Sql Server [Express version] and am not even sure I'm making the right choice. So here I am, seeking advice.
My database needs are nothing sophisticated. They just involve: (a) create tens of thousands of separate data files each under a unique file name of up to 8 characters, with file names read in from a pre-determined file name list. (b) store/insert VOLUMINOUS numerical data into each of the data files, with the data indexed by date&time, plus maybe one or two additional character or string fields. (c) for each data file, retrieve a subset of its data, perform extensive numerical calculations, and then store the results in one or more separate corresponding files, e.g. if a file name in (b) is F12345, (c) creates F12345R1, F12345R2, F12345R3, etc. which stores different sets of calculated results.
Thus, this is purely a console application, doing a batch job, and requiring no graphical user interface. Both automation and speed are important here, due to the large number of data files that must be created and/or updated, and the very extensive numerical calculations on the data.
The goal is to automate the daily or weekly creation of each of the tens of thousands of Sql Server database files, insert fresh data (read in from a fresh ASCII file) into each file, numerically process the data and then store the results in one or more separate, corresponding result data files, with all the steps automated and without need for GUI. Once coding is done, the entire data processing session is expected to run for many hours, or even days, in an automated manner, and without human intervention.
What would be the most efficient way of doing this under Visual Basic Express (which is what I'm learning to use) by directly calling Sql Server Express without having to go through GUI to create database files? What is the proper interface utility or library to use to enable direct database function calls without the need to learn SQL language? Is Visual Basic and/or Sql Server even good choices for what I want to do? I want to be able to call the basic, simple database functions directly and simply from program code in a non-GUI, non-interactive manner for the tens of thousands of separate data files that will be used.
I really miss the good old days when one can do a straightforward batch job via a console application, with simple, direct calls to create new data files, insert and index fresh data, retrieve any subset of data to do extensive calculations, create new files to store the results, etc. all under automated program control and iterating through unlimited number of data files, until the job is finished, all without human intervention during processing.
Or am I missing something because all this can still be done simply and easily under VB and Sql Server? I've several books here about Visual Basic 2005 and Visual Basic 2005 Express, all showing how to create a database via a GUI utility. That's fine if one needs to create just one or two databases, but not hundreds, or even tens of thousands (as in my case) of them on the fly.
So, I am looking for the simplest and most direct database interface that will allow me to do the above under VB program code alone, and easily. For something as simple as I have described above, I don't think I should have to learn the SQL language or manually create each database file.
As you can see, I just want to get some heavy duty numerical processing job done over tens of thousands of data files as simply and efficiently as possible, and with as little fanciful detour as possible. So, ironically, I am trying to use Visual Basic without being cluttered by having to learn its "Visual" aspects, yet GUI is what most VB books devote to or emphasize heavily. Similarly, I would much rather use simple, "lean and mean", direct database function calls than having to learn a new vocabulary of "English-like" SQL language.
Yes, I'm not used to this tedious detour of learning the GUI aspect of VB, or learning the Structured Query Language of Sql Server, just to try to do something simple that I need to do in batch mode via a console application.
Are there any good books or other helpful URLs that will help a guy like me? Am I even using the wrong language and the wrong database to do what I want to do? What are the better alternatives, if any? Any advice, experience and pointers on any of the above issues raised would be very much appreciated. Thank you!
I have create a batch file to execute a stored proc to import data.
When I run it from the server (Remote Desktop) it works fine, but if I share the folder and try to run it from my pc, it doesn't do anything. I don't get an error, it just doesn't do anything. My windows user has admin rights in SQL. Why is it not executing from my PC?
I have about 1200 sql files in one of my folders. Almost all of these files do data inserts and updates, so they should be run only once. As and when required I have manually ran around 150 of them already. Whenever I ran any of these scripts, I log that file name into a log table in my sql server including the execution time. Since running 1000+ more files takes a lot of time, I want to automate running of these files through a batch file. But I also want to filter the files that are already run.My file list looks like follows.
select * from sqlfileexecutionlog FileNameRunTimeResult --------------------- DeleteDuplicateOrders.sql03/12/2014 14:23:45:091Success UpdateInventory.sql04/06/2014 08:44:17:176Success
Now I want to create a batch file to run the remaining files from my directory to my sql server. I also want to wrap each of these sql file executions in a transaction and log success/failure along with the runtime and filename into sqlfileexecutionlog table. As I add new sql files into this directory, I should be able to run the same batch file and execute only the sql files that have not bee run.
I want to write a batch file that will do just that. The problem is bcp or bulkcopy never works with my code, it doesn't recognize it. Maybe I'm going the wrong way to do this, but I could use some help. Ideally I'll make the batch to do the importing via bulkcopy or something like it of a text file into a table and use windows scheduler to automate it. Be as specific as you can please, I'm very new to sql server.
Hi, I'm building a portal for supervisors can view their agents errors. We have a team that have been using Access to import data and QC it. I want to get these records into SQL but at the same time, the agents need to import their own work. I've create a DTS package that will import the data into the SQL table from a folder the agents have access to. I tried to create the strored proc to be run from the ASP page however it's not working and I'm having a hard time figuring out why. Here is the procCREATE PROCEDURE DTS AS exec master..xp_cmdshell 'dtsrun /KEMTSQL02 /ADJUSTMENTS /E'GODoes anyone know what I'm doing wrong? Any help would be great. Thanks!
on the server i am planning to run my packages at runtime using sql server agent...i remove the text files from exisiting location(for eg C to different location(D.when ever i receive a file at c:
I want my package to run.
How can i set this up in the sql server agent?????how will the sql server agent know that this package had received a file.
or is there anyway i can run my packages automatically other than sql server agent
Hi, when call a sub-package from the parent package with the executeoutofprocess=true I have about 5-10 sec delay before sub-package starts running. ( this is a big delay for me as the sub_package is in a foreach loop)
the sub_package itself it's very simple and contains just one script task with a small script in in it. even if this delay is for validation I still can't understand why it is taking 5-10 ses. cheers
I have to create a package that executes other packages I've already created... Every one of these packages run with the same Configuration File, but if I try to execute the main one, including the path of the file, I get errors from the other packages because they can't find it... How can I manage to pass this file and its content to the other packages??
Here a little explanation of te process:
Main Package needs configuration file X.dtsConfig, and calls: package1, which needs configuration file X.dtsConfig; package2, .......
i have two packages, one that adds data to a Fact Table (package1) and one that updates the dimension tables (package2). i want package1 to run all day, as it waits for incoming csv files and i want it to stop every night at 11pm and run package2, which updates dimension tables. then package2 should stop at say, 1am and package1 would run again. i tried setting the Daily Frequency schedule of package1 to occur every 24hours starting at 3pm and ending at 3:15pm, just to test if it will stop at 3:15pm, but it didnt. how can i create a job schedule that will perform the behavior? thanks a lot!
Is there any way to check from SQL code if a specific SSIS stored package is running like a SQL system view or stored procedure? Management studio option works fine but not if you need to determine if a specific package already is running or not?
I need to execute a long running package (it takes about 16 hours to finish) to load a data warehouse for the first time with all historical data. This package it's a master package and execute other packages; I log the start time and the finish time of the package in a table to manage future incremental loads.
I executed the package on sql server where it is saved, but after 8 hours it was running, a new package was started automatically. Then two more packages started .. each every two hours.
I set the MaxConcorrentExecutable = 4, this could affect this strange behavoir ?
Before I decided to post this thread, I searched the forum and saw that I'm not the only one going crazy with running SSIS packages in a job agent.
My package has some warnings ('truncation may occur due to inserting data from data flow column 'xxx' with a length of 50 to a database column with a length of 20') but it runs correctly in VS (development).
If I run it from dtexecui, it runs correctly...
If I configure my job as a SQL Server Integration Services Package I get this error in History Started: 3:24:12 PM Progress: 2007-11-07 15:24:13.19 Source: Package1 Validating: 0% complete End Progress Progress: 2007-11-07 15:24:13.19 Source: Load T_TITLE Validating: 2% complete End Progress Progress: 2007-11-07 15:24:13.20 Source: Load T_TITLE Validating: 4% complete End Progress Progress: 2007-11-07 15:24:13.20 Source: Load T_TITLE Validating: 6% complete End Progress Progress: 2007-11-07 15:24:13.22 Source: Load T_TITLE Validating: 8% complete End Progress Progress: 2007-11-07 15:24:13.23 Source: Load T_TITLE Validating: 10% complete End Progress Progress: 2007-11-07 15:24:13.23 Source: Load T_TITLE Vali... The package execution fa... The step failed.
If I change to Operating System (CmdExec) type this is my error output: Started: 3:31:20 PM Progress: 2007-11-07 15:31:22.04 Source: Load T_TITLE Validating: 0% complete End Progress Progress: 2007-11-07 15:31:22.05 Source: Load T_TITLE Validating: 2% complete End Progress Progress: 2007-11-07 15:31:22.07 Source: Load T_TITLE Validating: 4% complete End Progress Progress: 2007-11-07 15:31:22.07 Source: Load T_TITLE Validating: 6% complete End Progress Progress: 2007-11-07 15:31:22.08 Source: Load T_TITLE Validating: 8% complete End Progress Progress: 2007-11-07 15:31:22.68 Source: Load T_TITLE Validating: 10% complete End Progress Progress: 2007-11-07 15:31:22.68 Source: Load T_TITLE Validating:... Process Exit Code 1. The step failed.
Is there a way to see a more meaningful output?
SSIS really frustrates me in the last couple of days...
I have a sql 2005 cluster. I recently moved a new server in place of an old server we had
Both the servers now have w2k3 sp2 sql 2005 sp1 both have 2000 dts designer package.
My problem is that i have a dts package that runs on sql01inst01 without any issues but the same package when i try to run it from sql02inst02 just hangs ... digging deeper i found that - when i try to parse a query / build query / try giving parameters in a Execute SQL task it says
"The selected connection does not provide sql support"
The connection it reference is a postgres connection looking up info from a udl file.
I have a meaty job that currently runs as a set of sql scripts under sql agent and I'm transferring this job to run as a SSIS package. I have managed to set it up so that it runs as a sql job (cheers guys!), and I'm watching the job run now, maxing out the CPU usage as it tries to run 10 different tasks at the same time . great stuff!
However, I've been thinking about how to view the progress of this job/package (the job only has one step; "run SSIS package"). With the current job as single scripts, in SQL agent you can view which step of the job you are at.... is it possible to somehow view which package task you're at (ideally with nice shiny yellow green and red updates like in debugging), or do you have to output each step result into a log file?
Oh, and another question! If i have a checkpoint in a package, and the package falls over, when the job is rerun through SQL agent, will it start at the checkpoint, or run from the start?
