I have around 10 packages for dim and fact Table load. Using deployment Utility i create setup and put the packages in to test environment.
Now my requirement is call all the packages in certain order from single package. Using Execute Package Task i could call only one package. how to achive this in deployment?
Thanks
Sam K
Hi Guru's,
I am facing a problem in deploying the SSIS packages.
My Scenario is like this
1. Have one Master SSIS Package.
2. Master SSIS Package inturn calls 3 Different packages using execute package task.
3. I have a common Configuration file for all these packages.
4. I am using File System deployment.
5. I have imported all the packages i.e master and the 3 child packages using SQL Server management studio.
6. Now i have created a job, and added the master package as a step.
7. I have a variable declared in the configuration file and the child packages will be getting their path from this variable.
The problem since i know the imported packages lies in "C:Program FilesMicrosoft SQL Server90DTSPackages<<Folder Name>>" i have poited the child package variable to point to this directory.
I still want to import all the packages and want to get this directory info dynamically thorugh the script so that i can assign this at run time.
I am using "Directory.GetCurrentDirectory" But it's giving me a different location when we run the master package.
Can any one has any idea or comments how to retrive the path of the master package?
Even a work around or any other option is also fine....
Regads,
Dev
Hi,
I need to insert a record in a master table and 20 records in a child table. I want to do this using stored procedure. Is it better to do it in stored procedure? Have somebody already tried this? Or is there any sample that I can use?
Thanks a lot!
I have a situation here. Please
advice me on this.
I have a master table and a child
table. They have a PK and FK relationship. The master table has an identity
column with auto increment set to true. This map as a FK in the child table. My
questions are:
Can I have a single form to insert a new record into
master and child table at the same time?This has to be accomplished without stored
procedures. Can it be done?Is it possible to do this with a single insert query?
If yes, can it be done with sql data source or
dataset or tableadapters’?Please point me towards appropriate link for doing
so.
Thank you.
I am currently using SQL Server
2005 and VS 2005.
I have a master package which calls two package one after an other.
1) I will be passing parameters to master package using dtexec with /set property to set the variables in the master package. I would like to know how can child package access these variables.
Hi All,
In my DataFlow i have OleDBDataSource and OleDB Command. Using these i am inserting data to master and child tables.
In OleDBDataSource , i am inserting into master table and returning the ID of newly inserted rows. Next in the OleDB Command, i am inserting to child table using the ID returned from OleDBDataSource.
It is working fine. Now i want to put this in the Transaction so that if it fails to insert into child table, the changes made to the master table should be rolled back. I tried by giving Transaction Supported for dataflow. But does not looks like it works for me. Please suggest me the best approach for this.
Thanks in advance
DV
I searched for an answer for this, and found a few good threads, but none of them seem to be doing quite the same thing I'm trying to do.
I have two packages, parent and child. Parent does basic file manipulation -- encrypting/decrypting, moving from server to server, backing up to archive, pulling/pushing to external server via FTP, etc. It is completely dumb to what child does. Child is the guts of the data work -- the ETL package.
The goal is to have this one parent package be used by several ETL child packages. But not only can parent use different children (depending on which config is used when executing parent), but child can also handle different tasks, depending on a configuration it uses.
Let me break it down further.
Three packages:
ParentPackage - Used by all jobs
ChildPackage1 - Used for processing orders
ChildPackage2 - Used for processing inventory
Four configurations (thus four jobs):
Job1Configuration - Used for processing orders for ClientA
Job2Configuration - Used for processing orders for ClientB
Job3Configuration - Used for processing inventory for ClientA
Job4Configuration - Used for processing inventory for ClientB
Job1Configuration is set up to tell ParentPackage to use ChildPackage1, and to provide ChildPackage1 w/ various data for ClientA -- and so forth.
To my understanding, there is no passthrough of the config data from parent to child. A child package doesn't "inherit" or otherwise receive the information from the parent package, unless explicit variables are set up. This won't work for us, because ParentPackage is dumb of any data that the child packages may need. In other words, we don't want to set up ParentPacakge with every possible variable that every child package may need.
Also, I'm not aware of a way of setting up a job to provide a child package directly w/ a package configuration.
Is there any way to do what I'm trying to accomplish?
If I didn't explain something clearly, let me know and I will try to clarify.
Thanks in advance for your help.
Jerad
Hello,
I was hoping someone might be able to clarify how configuartions work with child packages. My process has 3 levels of packages. The main packages called two child packages each of which calls more child packages.
I'm using Configurations to pass variables from parent packages to child packages. But each level of packages contains Data Flow Source/Destination connections. I was planning on having only 1 XML Configuration file in the main package that would allow the data connections for the main package to be configured for the target enviornment. When the job is scheduled I will be able to point a configuration file for use with the main packages. But will that configuration file be applied to all child packages that will run? All of the Connections Managers have the same name so I'd like to make the change in one file.
If not, am I forced to define a parent/child variable to pass the Connection String from the parent to each of the child packages? Use an expression to set the Connection String dynamically? This seems like a lot of extra work, so I'd really like to see if there is a way to do it with an XML configuration file. Even if I have to definte the same config file for each of my 20+ packages.
Thanks
-Dan
I have a fairly simple SSIS project that has nested parent-child packages.
I am trying to find the best way to manage the connections strings so as to make the package portable across machines and environments. Currently there is one "master" package which calls 6 child packages. 1 of these child package calls 3 child packages of its own.
For the database connections, I've settled on creating a standardized .dtsConfig file for each server/login. This is a relatively small number (intially 8) that I don't expect to grow much.
I've taken a different approach for the file-system connections used by Execute Package components that call the child packages. For each package that has child packages, I store all the connection strings (paths) to the child packages in a single .dtsConfig file. This works well for the top-level "master" package where I can pass in the .dtsConfig file (that has the paths to the child packages) as a run-time option in the Execute Package Utility.
However, this approach seems to fall apart for the 2nd generation package that in turn call 3rd generation packages because I don't know how to get the .dtsConfig file (with the 3rd generation .dtsx package paths) path to this downstream dtsx package.
Though I'm sure there are others, the only two solutionsI can think of now are
(a)don't nest packages beyond 1 parent/child relationship -- not really an option or
(b)Store the path of .dtsConfig files for each .dtsx package as an environment variable on each machine. This option is unappealing because it would require adding an environment variable for every .dtsx package that has child packages. I don't think it would take long for this to grow into a large number, that would make managing environment variables cumbersome.
So far my experience with SSIS has been that there was a simple solution for each scenario I had. So this hoop jumping I'm going through seems to indicate I am just missing something.
Is there a better way I am just not getting?
hello again!, this time I'm trying to run a Master Package from the SQL Server Agent but I can't set relative paths to the connections for all the child packages that the master package contains.
It only finishes execution when I set absolute paths for all connections in the connection manager within the SSIS Project.
Is there any property in the SQL Server Agent or mayby a workaround to solve this?
It would appear that if a Child package is called more than once from a Parent using the 'Execute Package' task, then after the first execute the Parent Package Variables are not applied to child package. I.E we build dimensions in a master database and these are then loaded to a number of topic specific datamarts. We simply pass Parent variables to the child that hold source & target connection strings, the first time the package is called the correct database is accessed, subsequent Executes ignore the variables and use the original values. Manipulating the (our) event queue to run the package once results in the correct behaviour
Are packages cached when they are called from a Parent? if so is there a flag that I have missed to force a reload each time a child is executed?
This has just become a big problen for us so any guidance would greatly appreciated.
Paul
HI, I have some parent parent packages that calls child packages. When I added a bunch of packages, I faced the buffer out of memory error. I then decided to set the child packages property ExecuteOutOfProcess to TRUE. I noticed that the execution time is longer now. Is this a good practice to set the ExecuteOutOfProcess to true? If so, is it normal that the execution time is longer?
Thank you,
Ccote
Hi there,
I have a Business Intelligence project containing 4 packages. One of the packages is a parent package that just runs the other 3 packages sequentially (passing in variables via Package Configurations).
When I set the properties of the Execute Package tasks in the parent package, it seems that I can only select SQL Server or File System locations for the child packages.
What I want to do is for the parent to run the packages in the Visual Studio Project so that I can debug the whole process. I don't want to be bothered deploying the child packages so that I can run them from the parent, at least not while I am debugging the whole process.
How do I get the children to run within Visual Studio as well as the parent? I can't for the life of me find anything on the MSDN about this.
Many thanks in advance.
Richard F
I have a parent package and three child packages.
In all packages they make connections to a SQL 2005 db which sits on the same instance as which the packages are being deployed.
In the Child packages these connections have the connection string set as a package property variable in Package configs and in the Parent package this variable exists correctly.
This variable in the Parent is itself defined in a PackageConfig which uses SQL Server as the holder for the configuration.
When I deploy the packages the connections to in the child packages fail, one connection by timing out, another with an acquire connection failure.
I have set the DelayValidation to false on all connections but it made no difference.
I have also increased the timeout to 180s but again still a failure.
When I deploy the parent package on its own it is successful, however when the child packages are deployed on their own these connections again fail to validate due to a timeout and connection failure, yet they are using the same connections defined in the parent.
The child packages are no larger than other single packags that I have deployed successfully with the same connection to the same server.
Is there anything special that I should be aware of regarding deployment of parent and child packages.
I have 6 tables which are very huge in row count and need to be partitioned for better manageability.
Little info: Every day, 300 Million records are inserted and 300 million records are deleted in below 7 tables. we maintain only 8 days worth of data in below tables which is the reason records which are older than 8 days are continuously deleted.
Master table which has [ID],[Timestamp]
Table Name: Sample - 2,578,106
Child tables: Foreign key [ID] is common for all the tables. There is no timestamp column in child table.
dbo.ConnectionDB - 1,147,578,048
dbo.ConnectionSS - 876,458,321
dbo.ConnectionRT - 118,133,857
dbo.ConnectionSample - 100,038,535
dbo.Command - 100,032,235
I would like to partition the above child tables based on the IDs that are inserted every 4 hours. Meaning, All IDs that are inserted in 4 hours window should be in a partition.
I have 6 tables which are very huge in row count and records needs to deleted which are older than 8 days.
Little info: Every day, 300 Million records are inserted in below 7 tables. we should maintain only 8 days worth of data in below tables. How to implement Purge script which can delete records in all tables in the same time and with optimized parallelism.
Master table which has [ID],[Timestamp]
Table Name: Sample - 2,578,106
Child tables: Foreign key [ID] is common for all the tables. There is no timestamp column in child table. So the records needs to deleted based on Min(ID) from Sample
dbo.ConnectionDB - 1,147,578,048
dbo.ConnectionSS - 876,458,321
dbo.ConnectionRT - 118,133,857
dbo.ConnectionSample - 100,038,535
dbo.Command - 100,032,235
I have one package that executes 4 child packages (5 total). All 5 packages are set to log information using a connection string with a variable set at runtime for the location. The child packages use a parent variable to get the log connection string from the parent package.
The parent package logs fine, but a strange behavior occurs with the child packages.
The child packages all log data to the location used in the configuration file variable during runtime (like it's supposed to). However, I would get an error right before the first child package finishes execution saying path not found. Just for giggles, I created the folder I use in my development environment on the production environment. The error goes away, but the log file is created in that folder with no data in it. Subsequently, the log file with the data is created in the location set in the configuration file. Now I have two log files!!!!
Anybody know how to fix this?!
My packages are now in the filesystem. I have given the parent package a password in the properties window, saved the package and moved it to the run location.
I am using the SQL Job Agent to schedule the execution. My step command line reads:
dtexec /FILE "C:Pathparentpackage.dtsx" /De mypass /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EW
Do the the 20 or so child and grandchild packages require passwords to run this way?
Thanks,
IanO
I have a package that executes 3 other packages in it. From the command line I pass in which config file to use (just contains db/seerver) does this setting get carried over into the child packages as well?
View 3 Replies View RelatedI currently have multiple (parent and child) packages using the same config file. The config file has entries for connections to a number of systems. All of them are not used from the child packages. Hence, my child package throws an error when it tries to configure using the same config file because it can't find the extra connections in my connection collection.
Does anyone have any ideas on the best way to go about resolving this? Is multiple config files (one for each connection) the only way?
Sachin
Hi,
I have been developing SSIS packages and recently I found a strange thing happening. When executing chilld packages from the parent packages the child packages dont open automatically in the Package Designer window. Hence I cant monitor the places where the errors/ row movement have happened in child pkgs without manually opening the them first and then executing the parent.
Sam
I am running a package with a parent child relationship.
in the parent package I am executing a loop that calls the child package each time it loops.
The child package is executing a SQL command that retuns data and attempts to load the data into a flat file.
The problem is that everytime the SQL changes the Metadata needs to change also.
How can I refresh the child packages Metadata dynamically.
I KNOW THAT I CAN'T REFRESH THE METADATA WHILE THE PACKAGE IS RUNNING.
But I can refresh the Metadata before the package executes.
Does anyone know how to script this?
Thank you
I am developing an application in vb.net 2005 using SQL Server 2000.
In this I have two tables SessionMaster and SessionChild.
Fields of session master - SessionMastId, Start_Date, End_Date, Session_Type,
Fields of session child - SessionChildId, SessionMastId, UserName, Comment.
SessionMastId and SessionChildId are primary keys of respective tables and also they are auto increment fields.
Please how to write trigger to insert record into both tables at a time.
Hi,
I've found this problem that when I change settings in my configuration file it does not automatically apply to all child packages which uses the same configuration file if run from a job in SQL Server Agent. I need to open the package and save it again from BIDS. I use one "load group" package to execute all other packages.
Is there a way from the job configuration to set a setting so the package allways will have the newest configuration?
Regards
Simon
I have a parent package that uses Execute Package Task against three child packages. They are all setup to use the same configuration file, and the packages are all located in the same folder. Purpose of the configuation is to use the same connection strings for each child package.
I removed the path from the config. file specification to facilitate deployment. During debugging, the child packages cannot find the config file when they are run via the parent unless the full path is specified. If run standalone, a child package can find the config. file without the path. As I said, all dtsx filaes and the config file is in the same folder.
Also, the password-removal issue is killing me in development. Everytime I touch the config. file, it removes the password from the connection string. I have "protection level" set to "don't save sensitive" - I'd appreciate help with this too.
thanks
I've been executing a package and passing a parent variable to the child using package configurations but I'd now like to do this using a script task. The script task would then programmatically load the package and execute it.
How do I do this and still use the parent variable?
I've found examples of how to load a package but I haven't been able to find out how to I load it specifying the parent variables.
I think it's possible. MSDN shows the available methods but the example is for the base method.
http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.dts.runtime.package.execute.aspx
Cheers,
Ben
I've not really used SSIS for a while, and I'm now building some in 2012 and trying to utilise some of the features in the 2012 SSIS catalogue; however I've hit a bit of a stumbling block.
What I'm trying to do is have a master/child package relationship, with several child packages and where the child packages themselves are dynamically called (i.e. the master package may call a different child package based upon some value or state of data already processed.)
When I try and create an expression for the PackageNameFromProject property... well, that property doesn't appear to set dynamically. I know how to do this for old style packages by creating expressions for the package name etc; but that way I can't use the package parameters I have from my master package.
Has anyone had experience of using Parent/Child packages while enlisting them in Transactions.
I tested this on a small sample and thought that I had got it to work, but in my real-world package it does not.
The parent package essentially calls three child packages.
In each child package there are multiple DFT's that import and transform data into SQL Server.
All data must be imported or not at all.
Therefore I created a FELC container into which three Exec child package tasks were placed.
The FELC is set to Trans Option 'Required' and the Exec child package tasks to supported.
Unfortunately upon failure of one of the DFT's in the child the data was not rolled back.
So initially we had in terms of container hierarchy for the Trans Option property:
Parent package Supported
FELC for calling child packages Required
Task execute child package Supported
Child package Suppored
Tasks Suppored
Looking at this more closely we thought that we would need
Parent package Supported
FELC for calling child packages Required
Task execute child package Required
Child package Required
Tasks Suppored
for it to work.
However, the latter now gives us failures with error messages on the tasks on the child packages.
[Execute SQL Task] Error: Failed to acquire connection "Conn ECARS1CEDImport". Connection may not be configured correctly or you may not have the right permissions on this connection.
Even more strange the first couple of tasks in the child pkg complete successfully even though they use the same connection listed in the error.
These tasks also have Event handlers.
I want to set multiple child packages running without waiting for them to complete in a parent SSIS package. The catch is that I will be running the same child package in multiple threads with different configuration variables set. I want to drive the config variables for the child packages via a SQL Server table, and start the execution of each child package from within a for-loop container.
Here's what I've tried, and why it didn't work:
1) Execute package task. Didn't work: Waits for the child package to complete before moving to the next step.
2) Kicking off the package via the "sp_start_job" stored proc in the msdn db. Didn't work: Can't specify config variable values.
3) Using the DTExec command line prompt. Didn't work: Same issue as 1). Waits for the package to complete.
Anything I'm missing? Any ideas on how to accomplish this?
Hi I have completed my first SSIS master package which runs a whole lot of child packages depending on value of expressions on workflow. (refer http://www.sqlis.com/306-3.aspx)
Each of my child packages is .dtsx file location and each Excute Package task uses the file connection.
The master parent package is also a dtsx file location which will be run by a SQL Server 2005 Agent
All good--problem is testing from BIDs--each time a Excecute package task is run--turns yellow a new tab appears apears in the design window --showing you that particular .dtsx file control flow detail. DTS never had this behaviour --can I turn this off in the BIDS ie as I have dozens of new tabs at run time which makes it very hard to keep track of the master package. All I want is the master package running from BIDs, and no new tabs appearing at run time???
Thanks kindly
Dave
How to pass variable from Parent to child and child to Parent Packages is this possible in SQL SSIS 2012. I need this only in SSIS 2012 ...
View 6 Replies View RelatedI have installed Office 2007 Beta 2 and SQL Server 2005 at my system.
I made 2 packages (C:Package.dtsx, C:Package1.dtsx). I am executing Package1 inside Package.dtsx using Executing Package Task.
In package1.dtsx there is some configuration that is coming at runtime(For each loop container having some file connections) , so I have put DelayValidation property of this package to True, but that did not work when I called this package from Package1.dtsx, then I found one workaround, I changed the ExecuteOutOfProcess of ExecutePacakge task to True,
Now the problem is that before installing Office 2007 the both were executing fine (it was taking some time in starting the second package because of OutOfProcess execution).
After installing Office 2007 I get the following error while executing the second package from first package.
Error: Error 0x800703E6 while loading package file "C:Package1.dtsx". Invalid access to memory location. .
This problem comes whenever the I execute the package having OutOfProcess property to true.
Is there any workaround for this, or I am wrong somewhere.
Thanks in advance.
Atul
I have 12 packages to execute in order. I made a table in my DB where i mentioned the name of each package and his order in execution.
I want to create a master package that get the name and order from my DB table to execute all packages.