SSIS - How Many Per Batch?
Jul 18, 2007
So, I€™m setting up an SSIS package and have been playing around with the batch size but would like to know what is or how to figure the best size for the migration.
Here are some details:
1 - I'm migrating about 72 million records from oracle to SQL server 2k5.
2 - The read (from oracle) is across a somewhat slow VPN connection
3 - The process is running on a separate machine than either DB but is in the same net as the sql server db (no vpn for the insert)
So, if there's a way I can calculate optimal records per batch that would be great unless there's just some set figure I should use given my configuration
Any suggestions are welcomed~
View 1 Replies
ADVERTISEMENT
Nov 18, 2014
I am using Variables to configure a lot in an SSIS Packages.
Over the years you add new variables that are usefull to your default-package.
However to use those in "older" packages you have to open add them manually.
Any way (e.g. skript) to add a set of variables to every SSIS-package in a folder ?
View 3 Replies
View Related
Apr 22, 2008
Is it possible to import packages in batch into MSDB (in lieu of right clicking and importing each one by one)? We have a lot of packages that are going through a lot of changes so a batch import would save a lot of time.
Thanks.
View 5 Replies
View Related
May 2, 2007
I am loading a lot of Excel and CSV files to SQL Server. Some loading may fail for various reasons. I want a file either be load as a whole or nothing. Currently I keep a list of failed filename and remove it at the end (I add a column for source file name).
Any better way to make sure a file is loaded as a whole or nothing?
Thanks,
View 3 Replies
View Related
Jan 29, 2008
Trying to run a SSIS package from a SQL job, and the package itself has a step that calls a process task that runs a batch file. The syntax in the process task I have is the following:
executable: c:windowssystem32cmd.exe
Arguments: /C e:SungardPTAencryptfile.bat
Working Directory: e:sungardpta
I keep getting the following in my log:
PackageStart,MIMKEIMC11N,MI rustserviceadmin,PTADailyTransactionExtract,{46F7381F-B345-47DC-BFC0-17CCF02A935A},{F82C7944-D28C-4F70-8CB7-F0BD7ED748D2},1/29/2008 1:59:11 PM,1/29/2008 1:59:11 PM,0,0x,Beginning of package execution.
OnError,MIMKEIMC11N,MI rustserviceadmin,EncryptFiles,{FCF5B653-CC05-4183-981B-F5EF4906DD09},{F82C7944-D28C-4F70-8CB7-F0BD7ED748D2},1/29/2008 1:59:12 PM,1/29/2008 1:59:12 PM,-1073573551,0x,In Executing "c:windowssystem32cmd.exe" "/C e:SungardPTAencryptfile.bat" at "e:sungardpta", The process exit code was "1" while the expected was "0".
OnError,MIMKEIMC11N,MI rustserviceadmin,PTADailyTransactionExtract,{46F7381F-B345-47DC-BFC0-17CCF02A935A},{F82C7944-D28C-4F70-8CB7-F0BD7ED748D2},1/29/2008 1:59:12 PM,1/29/2008 1:59:12 PM,-1073573551,0x,In Executing "c:windowssystem32cmd.exe" "/C e:SungardPTAencryptfile.bat" at "e:sungardpta", The process exit code was "1" while the expected was "0".
OnTaskFailed,MIMKEIMC11N,MI rustserviceadmin,EncryptFiles,{FCF5B653-CC05-4183-981B-F5EF4906DD09},{F82C7944-D28C-4F70-8CB7-F0BD7ED748D2},1/29/2008 1:59:12 PM,1/29/2008 1:59:12 PM,0,0x,(null)
PackageEnd,MIMKEIMC11N,MI rustserviceadmin,PTADailyTransactionExtract,{46F7381F-B345-47DC-BFC0-17CCF02A935A},{F82C7944-D28C-4F70-8CB7-F0BD7ED748D2},1/29/2008 1:59:12 PM,1/29/2008 1:59:12 PM,1,0x,End of package execution.
Any help would be greatly appreciated
View 4 Replies
View Related
Aug 20, 2015
We have a Job  that calls a SSIS package 2005 that does some processing and execute a BAT file.  This Job is being called by a web application.The BAT file creates a folder and named it based on the current date ( YYYY_MM) e.g 2015_07
echo %date:~10,4%_%date:~4,2%
md %date:~10,4%_%date:~4,2%
pause
exit 0
It was working okay in the SQL Agent 2005 server until we moved  to the new server SQL Agent 2012 using the same package SSIS package 2005. Now the issue is, instead of creating the folder based on YYYY_MM, it's now being created as YYYY_DD.I've checked the Regional settings of both server and they have the same "ENGLISH (United States) format. I even ran the code below and they're returning the same output echo %date:~ 10,4%_% date :~4,2%
I know the BAT file can be improved by not depending current locale in WINDOWS, but I just want to understand how this issue occurs and how does the regional setting being overridden?
View 2 Replies
View Related
Aug 7, 2007
HELP,
I need to take a variable from a tabel in SQL Server pass to a Batch file and execute the batch file. Right now I can exec the batch file with XP_CMDSHELL but how can I pass the variable to the batch file and loop through all the variables.
Please help
Phil
View 4 Replies
View Related
Dec 5, 2006
I am using the following batch file to execute a script that creates a db and all its objects in the local sql express:
sqlcmd -S (local)SQLExpress -i C:CreateDB.sql
This works fine, but I'm wondering if there's an easy way to put the script in the batch file, so users don't have to worry about putting the script in the C drive. I tried getting rid of the i parameter and pasting the script from the sql file into the batch file, but it didn't work.
Thanks,
Dave
View 1 Replies
View Related
Dec 26, 2006
Hi,
I want to schedule a daily job using sql server to update the info. in a sql server table. This is very new to me. Could you please forward me some helpful resources.
Thanks,
View 3 Replies
View Related
Jul 25, 2006
QuestionsI need to batch a set of update commands. Can that be done and if so what are the possible ramfications? Can one mix / match Deletes/Inserts and Updates into a SQL command via the semi-colon in a batch set? (Is this the appropriate forum?)
WHYI am unable to use the
dynamic sql building found in the dataset due to a multiple table
contstraints. I have built classes that will extract the differences
from two datasets and report on the updates/deletes and inserts
required. Since I have multiple SQL statements, it would be nice to combine them into one statement.
Platform.Net 2C#IIS6SQL Server 2000 is the eventual target database for the generated SQL
advTHANKSance
View 1 Replies
View Related
Mar 28, 2008
Hey guys, I periodically have to update maybe 300 records of the same column.. However, I've been udpating each record one by one.
Is there a way I can do something the following.ID Name1 john2 chris3 adam4 ben5 steveupdate table1 set name = name where id = ? but have it run through an entire set of these? Is there any way of automating that rather than having to update it manually? Do i need another table with just the id numbers and the column I want to change? or do i need a txt file?
View 6 Replies
View Related
Jan 22, 2002
hi,
I need some help in accomplishing this task.
I want to design a DTS task which will:
a)copy a certain given files from one directory to another
b)import the files into the tables
c)upon successful import delete the files from the original directory.
I done know much about scripting and need help in figuring out steps a) and c).
thanks
Zoey
View 2 Replies
View Related
Jul 31, 2001
Is there any way using to run already developed DTS pacalkes one after another.
Thanks
View 1 Replies
View Related
Sep 13, 2001
Do u know how to write a batch file
for example i will need to type the sql server name it has to connect to the server and run a script that I have
let me know if u have any ideas of doing it
View 2 Replies
View Related
Sep 13, 2001
Do u know how to write a batch file
for example i will need to type the server name it has to connect to the server and run a script that I have
let me know if u have any ideas of doing it
View 3 Replies
View Related
Mar 29, 2000
Can someone show me an example of the syntax required to execute multiple BCP commands within the same batch (*.bat) file?
Sorry if this is a bit of a basic question, but not being a programmer by profession, I need to plead ignorance. I've tried a few things, but I just can't seem to figure it out.
Thanks!
Rich
View 4 Replies
View Related
Oct 22, 2003
I have a problem running a batch file, now the problem is that when i run the batch file the command prompts the user for an input, but I have all the output of the bat file going into a log file. So when i run the bat file the process just sits there until i hit the 'y' key or unless i nput sumthin manually. This is a problem becuase this batch file is running on the UAT server as a job and there is no one there to input once the job is running. The commnd in the batch only requires an input once a month.
for eg if run: launch_scrt.bat and i want to put sum parameters such as 'y' or 'n' to avoid the manual input once the job is running. Any ideas?
Thanks in advance
View 11 Replies
View Related
Sep 26, 2005
Hi There,
I want to able to run a batch of sql statments (execute procedures) in SQl Server at a specific time. Just wondering if there is a method for that.
View 2 Replies
View Related
Jun 27, 2006
I like to schedule set of jobs in sql server 2005 to run in a particular sequence. How do I schedule batch jobs? Each job is a set of packages. Idea is to create a job with multiple steps and each step should be a one job but there is no option for the "sql server agent jobs" when mentioning each step.
Any idea how to create a set of jobs in sequence?
Thanks in advance,
Maria
View 2 Replies
View Related
May 6, 1999
What is the best way to run a scheduled task that fires off three scripts that need to be run sequentially? I could set up three different tasks but I don't know exactly how long each will take and they are interdependent.
View 2 Replies
View Related
Aug 10, 2004
Hi all
I have a batchjob which runs a DTS package now it calls it ok
However it is running against 2 Databases on 2 servers and I get a login error even tho I am using the sa login
does anyone know of any issues relating to this Hopefully with a soloution that does not involve a lot of work
thanks
Michael Grieve
View 7 Replies
View Related
Sep 27, 2004
In Horacle (thanks to someone else for the apt pseudonym), I am able to run a PL/SQL script saved as a file by simply typing in @ plus the path and file name. So, if I have a script called E:CreateTable.sql, I can simply enter @CreateTable.sql from the SqlPlus command line and Oracle will attempt to run all the commands in the E:CreateTable.sql file (there may be one or more commands).
Can I do the same thing in Query Analyzer (or even osql)? If so, what is the correct syntax?
Thanks,
hmscott
View 3 Replies
View Related
Jan 23, 2004
I'm using ASP.net to do a Select and I want to insert all the results into a table that is stored locally
I can put an SP local but cant put on the other DB
How would i achieve this batch insert? is it possible?
thanks
Mark
View 4 Replies
View Related
Apr 21, 2006
Dear friends
I want to be able to make a batch that adds a column to every table in one go
Find a table object
Attach table name to columnname called "Concurrency"
Add column called Concurrency timestamp
If present already, don’t do it
Anyway to do it or do I have to do it manually?
cm
View 3 Replies
View Related
Apr 3, 2007
Hi pals,
I need a small help from u all.
I need a Windows batch file script which does the ftp to remote machine and "puts" a file
say "data.xls" from a predefined dir say "c:uploads" and puts on the database server "d:dumps".
Regards,
Franky
View 2 Replies
View Related
May 17, 2007
I need to execute around 100 SQL server jobs in a batch. If I execute them in a batch will they get executed linearly or asynchronously?
------------------------
I think, therefore I am - Rene Descartes
View 1 Replies
View Related
Aug 12, 2007
Hai, i would like to do Bulk update to avoid the round trip to the database.
Means, In my UI im dsiplaying all the employee details who are related to one particular dept. Now i want to update the bonus to all the employees based on their category. UI changes are refelected in the Datatable(.NET). Finally the datatable changes i would like to update in the Database.
how can i do that..
sample code pls.
im very very new to sql
View 3 Replies
View Related
Jul 23, 2005
I am inserting records into a table - around 1 million records. I wantto do the insert 10000 records at a time using TRAN. My insertstatement is very simple:BEGIN TRAN T1Insert INTO TABLE A(ColA, ColB, ColC)Select B.TBLBCOLA, B.TBLBCOLB, B.TBLBCOLCFROM tbltable BWHERE blah balh...etc.COMMIT TRAN T1Any hep or link on the syntax will be helpful!ThanksSri
View 6 Replies
View Related
Jul 20, 2005
Hi AllThis is a belter that my little brain can't handle.Basically I have 1 SQL table that contains the following fields:Stock CodeStock DescReferenceTransaction DateQtyCost PriceBasically this table stores all the transaction lines of when a userbooks stock items into stock so that they can look at a journal ofthis goods in as and when they please.My task is that the user wants a list of all the stock items with thelast cost price that they were booked in at.So I think I have to find the last transaction date used for eachstock code and then bring this back as 1 row per stock code with theabove fields of data.How the whats-its can I do this? Is it acutally possible?Any help you can give is much appreciated.RgdsLaphan
View 3 Replies
View Related
Jul 20, 2005
I have a windows batch file that executes a SQL Server bcp command. Iwould like to obtain a return code if the bcp command fails. However,I cannot seem to find the return code (if any) for bcp. For example,if the bcp command is improperly formatted, or has a bad password, Iwant the batch file to return an error. Right now, my batch filesimply executes and returns success, even when the bcp command fails.Has anyone run into this before?Thanks!
View 3 Replies
View Related
Nov 24, 2006
Hi Folks,
I am running batch jobs(Nearly 50 queries) on Production server.It is taking nearly 5 or 6 hours(some times 10 hours also) to get the results.The problem is If I loose VPN connection I am not able to save data.I need to save the data upto the query executed before I loose VPN connection Please help me
View 6 Replies
View Related
Apr 30, 2008
Hi
I'm running into an odd problem. I have set a backup process on a user installation, and invoke the backup using a CMD file.
When the CMD file is executed from a DOS box, or windows GUI the SQLCMD executes, but, when the same batch file gets spawned by the AT command, the SQLCMD does not exeute.
The odd part is that I have the same exact setup on another SQL Express installation on another server and it works OK.
Ideas?
David L.
View 8 Replies
View Related
Oct 30, 2006
Hello everyone,
I have about 20 dtsx packages to run in a particular order.
Some create tables, other fill/convert data, and others just clean/remove temporary tables.
Everything works ok manually...
But, i need to run them automatically.
A batch script seems the quickest and more simple way to do...
So, I made a batch with something like this:
dtexec.exe file1.dtsx > output1.txt
dtexec.exe file2.dtsx > output2.txt
dtexec.exe file3.dtsx > output3.txt
(the ideia is to save the logs...)
My first question is...
I get a very usual "product level is insufficient" (0xC00470FE) error on some components...
It's wierd because manually works ok. But through the batch it returns lot's of these...
I searched for information and found this...
It's a nice topic, not enought to fix my problem.
My second question is...
A noob batch script question:
Can i make something like this:
dtexec.exe file1.dtsx > output1.txt
IF FIND /C /I "End Error" output1.txt != 0 GOTO EXITSCRIPT
The ideia is to search the log for errors, and if found any run other batch file...
Thank you so much for any help!!!
David
View 1 Replies
View Related
