Single Configuration File - Mulitple Packages - Elements Not Used In Each Package
Dec 18, 2007
Hi,
I'm fairly new to the SSIS world, and I've recently ported a bunch of dts packages over to SSIS. I'm an ASP.NET developer so I'm very familiar with the capabilities that configuration files give you, and I attempted to set up my solution as follows:
All of my "Data Sources" are at the project level, and added (with the same name) to each package. I wanted to have a single config file that had all of the project-level settings (i.e. connection strings, data file paths, etc). I then have a config for each package with the package level settings - i.e. variables, etc.
The problem becomes that all packages do not use all data sources. This results in an error when I try to open up a package for editing, it complains that it doesn't have a reference to data source XYZ that it is seeing in the configuration file.
Is there any way that I can get around this? If I have a password to a database change, I don't want to have to look through every config file and change it in multiple places.
We are trying to create a deployment utility for a solution. The issue we are facing is, we are using a single package configuration file and when we try to build the solution to create the deployment utility, the build process fails saying that the package configuration file already exists. THe reason for this is while trying to build, the utility copies the configation fiel for the packages, it copies for one, but for the second onward, when it tries to copies, it fails saying the file already exists.
Any idea how to overcome this, or else any suggestions how to perform the similar steps to create a deployment utility for a solution in which the packages share a single package configuration file.
I currently have multiple (parent and child) packages using the same config file. The config file has entries for connections to a number of systems. All of them are not used from the child packages. Hence, my child package throws an error when it tries to configure using the same config file because it can't find the extra connections in my connection collection.
Does anyone have any ideas on the best way to go about resolving this? Is multiple config files (one for each connection) the only way?
I've found this problem that when I change settings in my configuration file it does not automatically apply to all child packages which uses the same configuration file if run from a job in SQL Server Agent. I need to open the package and save it again from BIDS. I use one "load group" package to execute all other packages.
Is there a way from the job configuration to set a setting so the package allways will have the newest configuration?
I have two packages share same configuration file. There are 4 connections in the first package and 2 connections in the second. Configuration file contains properties for 4 connections. First package opens in VS designer with no error. The problem with second package. I get error that connection exist in the configuration but not in the package and "Could not copy file " Master_Configuration.dtsConfig" to the deployment utility output directory. Master_Configuration.dtsConfig' already exists." Any help?
Basically it seems that whilst you can indeed share a config file, it doesn't aggregate sets of say different connection managers, into a superset - you choose to reuse, but it actually overwrites. Any ideas, anyone??
I'm using an XML file to configure my dataBase connection string. I try to deploy my package on a new server and it works perfectly.
Then I made a second package which also need a configuration for the dataBase connection string. (I made the connection with the connection manager inside packages). The configuration is the same that for my first package, so I thought to use the same configuration file.
I can use the same configuration file but the problem is when I try to generate a deployment for my solution. I got an error which tell me that the xxx.dtsConfig file already exist and can not be copy again.
When I made the configuration in the second package I said that I want to reuse the file ... and I thought that for the deployment SSIS would know that it has to include that file only once ...
I am currently migrating from Oracle to MS SQL Server 2005 using SSIS. Since the new schema being used on the SQL Server is very different, we have created separate packages for each target table.
We have several different sets of data stored as separate Oracle schemas, and I would now like to change to a different schema for the OLE DB Source objects in my packages to use (Table/column names to remain the same). While I can go through all packages and search through the drop-down list to select the table of the same name from other Schema, doing so is an extremely slow process.
Is it possible to use configuration files to set a schema to be used for a number of OLE DB Sources over a number of packages?
At the moment I am doing a find and replace over the .dtsx files, but can't believe I have to resort to this for something that I imagine many people would want to do with SSIS.
I am planning to develop a single package that will download files from ftp server, move the files to internal file server and upload it in the database. But I want to run this package for multiple ftp file providers. For each provider the ftp server might be different and the transformation to upload the files into a database table might be different.
So can I create a single package and then multiple configuration files (xml), which will contain the details fo the ftp file providers and then pass the xml file as a parameter while executing the package. The reason being that the timings of fetching the files is different for each ftp file provider and hence cannot be combined into one.
I've read that a single config file can be used across multiple packages, but am unsure if it applies to what I'm trying to achieve. As an example, I have 2 packages, PackageA and PackageB. Each package has a connection manager defined - ManagerA and ManagerB and I want a single package configuration file to contain the connection string for each manager.
I can define the package configuration of PackageA with no problems. I choose XML file, define the filename and add the connection string to the config.
I then open PackageB Package Configuration and point it to the existing config file, IS tells me the file already exists and allows me to choose the option to "Reuse Existing". I then supply a configuration name and the wizard finishes, without letting me expose any properties. At this point, the configuration file still contains config details from PackageA.
If I then choose to Edit the package configuration in PackageB, I get a warning saying the config values in the config file (relating to PackageA) cannot be found, and the wizard continues. The subsequent PackageB properties which I add overwrite the existing PackageA properties.
So it would appear that what I'm trying to do cannot be done, and I need a separate configuration file for each package?
Here's the deal. I have a child package, (say, pack01.dtsx), which uses a dtsconfig file for its connection string, which can be called from other packages, but which also can be called by itself.
However I also have another package (say, pack02.dtsx) which uses the same dtsconfig file for its connection string. It calls on pack01.dtsx.
When I use DTEXECUI and run pack01.dtsx, specifying the proper .dtsconfig file, it goes well. But when I try and run pack02.dtsx, an error occurs saying pack01.dtsx connection cannot be established.
How do I pass the connectionstring being used by pack02 to pack01, without having to remove the configuration file setting of pack01? Can a Parent Package configuration and a configuration file try and map to the same property?
I have a DLL that acts as an interface between the application and the database. So you create a new instance to the database via:
MyDatabaseClass db = new MyDatabaseClass()
Each instance of the class creates a new connection to the database.
So if I were to have the following, then 4 connections would be made.
MyDatabaseClass db1 = new MyDatabaseClass() MyDatabaseClass db2 = new MyDatabaseClass() MyDatabaseClass db3 = new MyDatabaseClass() MyDatabaseClass db4 = new MyDatabaseClass()
My question is...What if instead of creating new connections upon each instantiation, I re-use the same connection through a "ConnectionManager" class. In this way, the above code will only create 1 connection instead of 4.
What do you guys think about each approach? Obviously the 2nd approach makes sense for non-web applications, but what about websites? Would that just cause that single connection to be overloaded??
Hi, I am using custom dll in script component in SSIS package. This dll is looking for some configuration settings and dsplays the message as "Configuration section could not be found in the configuration source" . Please tell me the configuration source it looks for.
I have about 40 packages in my job. One of the problems that I have encoutered is that I don't know how I could use same XML dtsconfig file for all of my files. Each SSIS calls different file.
The XML dtsconfig file is a replacment for my *.ini file in sql 2000. The dtsconfig file will hold paths to all different files. Each SSIS will take different file name at a runtime.
How can I have all the files paths in one dtsconfig file - and is it even possible to do that in SSIS?
I'm having difficulty configuring a package to look for an xml configuration file that contains values for source/destination connection strings.
The goal is to be able to have a package and its configuration file on one file system and to be able to copy/move the file dtsx and dtsx.config between build/staging/production servers while only having to set the xml value for the source/destination configurations.
In the package, I've created an xml configuration file called zzz.config. I took the absolute path out of the location of the xml file so that I have xml location = zzz.config instead of xml location = e:ssiszzz.config I'm assuming that when it gets to the point of validating the location of that file, it starts by looking in the current directory where the package is found.
All is well as long as I execute the package through the execute package utility by double clicking on the package file.
The problem lies when configuring a job in the sql server agent, it doesn't seem to pick up the xml configuration values because its pointing to the old server and not the current server stored in the xml file. I'm assuming its because the current directory is not the package directory at this point.
How can I get around this? Do I have to move the xml config file (zzz.config) to another directory where the agent will look for?
I know I could configure this as an environment variable but this contradicts the point of having movable packages without any extra tweaks.
The system creates a XML file but when I run the package I get the following error in the output pane. Information: 0x40016041 at FMC_People: The package is attempting to configure from the XML file "L:ProjectsVinciSSISDVLFMC loader ImportFMC Loader ImportFMC Loader ImportJACBE_IF_CONFIG.xml". SSIS package "FMC_People.dtsx" starting. Information: 0x4004300A at Dataprocessing_PEOPLE, DTS.Pipeline: Validation phase is beginning. Error: 0xC0202009 at FMC_People, Connection manager "JACBE_IF": An OLE DB error has occurred. Error code: 0x80040E21. An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040E21 Description: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.". Error: 0xC020801C at Dataprocessing_PEOPLE, FMC_ARE_PRESENT_destination 1 [22338]: The AcquireConnection method call to the connection manager "JACBE_IF" failed with error code 0xC0202009. Error: 0xC0047017 at Dataprocessing_PEOPLE, DTS.Pipeline: component "FMC_ARE_PRESENT_destination 1" (22338) failed validation and returned error code 0xC020801C. Error: 0xC004700C at Dataprocessing_PEOPLE, DTS.Pipeline: One or more component failed validation. Error: 0xC0024107 at Dataprocessing_PEOPLE: There were errors during task validation. SSIS package "FMC_People.dtsx" finished: Failure.
I don't get it. Where do I go wrong?
I tried the same with a DtsConfig file instead of an XML but to no avail. the way of working as described in BOL and in the book professional SQL SERVER 2005 Integration service seems to me perfectly similar.
does anybody know where is stored information about belonging a certain configuration file to a package? How can I connect a configuration file with an installed package without using SSISDeployment wizard?
I created a simple file system task to copy a file from one location to another on my local system.
I used variables to specify the source and destination paths and now in my configuration file if I configure the values to something else also the package is taking the initial default values and not the values specified in the configuration file.
So I've seen articles outlining how to execute a package in VB and how to pass variables along to the package, which is great.
But I'm wanting write a VB.NET app that will read the xml in a .dtsx package, looking for the available variables (and datatypes) that need to be passed to the package to configure it properly. Anyone done this? Is it best to use parent package variables? Or another type of configuration?
When looking at the xml of a .dtsx file, I can see a variables section that could be parsed... is there anything letting me know what is required versus optional? How do I know the datatype that I need to pass? Any help would be appreciated...
Started: 10:48:56 AM Info: 2008-01-24 10:49:02.92 Code: 0x40016041 Source: SsisEndBatch Description: The package is attempting to configure from the XML file "C:Program Files (x86)Microsoft SQL Server90DTSPackagesBridgeNet.BI.SSIS.EndBatc ENDBATCH TEST ENV.dtsConfig". End Info Warning: 2008-01-24 10:49:03.15 Code: 0x80012011 Source: SsisEndBatch Description: Cannot load the XML configuration file. The XML configuration file may be malformed or not valid. End Warning
We are facing some issues with SSIS and thought if you could be of some help.
Actually, instead of Deploying our SSIS projects we copy the SSIS files to a different server from where we execute the SSIS Packages. The reason being, we have a Core Version of the application and this is deployed to different markets and subsequent customisations are performed in the market specific version of the Packages. The problem is although we have a Package Configuration Xml File for our Connection object, the first time when we copy the Packages, we have to manually open each of the Package and change the URL of the Xml File for Configuration. I read through some of the articles on the internet and came to know that the Package Configuration does not support relative path and only absolute path is possible.
Can you please share some of your suggestions on the above problem.
I've added an XML package configuration file to my SSIS package and it appears to be working fine.
The XML file is located in the same folder as the solution file.
I've recently added the entire solution to Visual Sourcesafe 2005, and I achieved this by doing the following:
1. Right-clicked on the Project name in the solution explorer and selected "Add to source Control". 2. When prompted, provided my Sourcesafe login credentials and clicked "OK" 3. Accepted the default VSS project name and clicked "OK" 4. Clicked "Yes" to allow VSS to create a new project
Problem: The XML configuration file hasn't be added to sourcesafe along with rest of the solution.
I'm able to add the file manually myself, but this doesn't seem right to me. Am I missing something? Should the config file show as part of the Solution in Visual Studio 2005?
I've used XML package configuration in my packages in order to populate key variables. The configuration String is pointing to a local folder in my machine. After that, I've checked my whole solution into TFS.  I did check the checked in file but could not find the .dtsConfig XML file. The problem occurs when the other teammate checked out this solution from TFS into his own box. When he tried to open the solution, it gives warning (not error though) saying it could not find the package configuration file.machine does not have the same URL I had in my  box.  In situation like this, how can we fix in the multi-developers SSIS environment?
I am working on a project currently where we have many SSIS packages and we want to minimize the number of config files to be used. What I was thinking was using one master config file which will have Server/Database info which will be used by all the packages. Now the thing is there are multiple packages which are being executed from within a master package and I was wondering if this will be an issue. Is there anything else that I will have to keep in mind? Any help is appreciated.
I have been reading through different posts about how to manage SSIS configuration files across multiple environments. I have seen a few that mention a solution that turns out not to really work (or maybe I€™m doing something wrong); which is what I hope to find out here.
Solutions I am working with: 1: Put the XML Configuration file in the same directory as the SSIS package and only specify the config file name when setting it up in the package. I do not specify a path to the XML Configuration file, just the config file name.
ISSUE: This works when I first create the package and test it without closing down visual studio. Even after deploying the SSIS package and the config file and moving it between environments it works as long as the working directory of the DTExec command is the same directory as the SSIS package and configuration files. The problem comes in after I have closed Visual Studio and opened it again and loaded up the SSIS package. It then gives a bunch of errors stating that it can€™t locate the configuration files€¦. If I could fix this issue, then this would be a solution that I could live with. (I would much rather use a relative path to a different directory but I€™ll live with what I can get). Can anyone solve the issue of Visual Studio 2005 not recognizing the location of the configuration files when only the configuration file name, and not the path is specified?
-- The other odd thing is that I have been able to get this to work consistently with some of my packages but not others. I'm not sure what I did to get those packages to recognize the location of the xml config file.
Hi --I was wondering if this is a bug when I add new data in my table SSIS Confiurations and give wizard a new Configuration filter the package configuration wizard can not see the new values --the old values from the previous configuration are still showing---is there any known workaround or forced refresh I can do
thanks in advance Dave
Background:
SQL Package Configurations are most important because they provide the possibility of a central configuration store for your entire enterprise!!!!!!!! and is in my mind the only way to go
USE [ETLConfiguration] GO /****** Object: Table [dbo].[SSIS Configurations] Script Date: 05/23/2006 13:34:35 ******/ SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO CREATE TABLE [dbo].[SSIS Configurations]( [ConfigurationFilter] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL, [ConfiguredValue] [nvarchar](255) COLLATE Latin1_General_CI_AS NULL, [PackagePath] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL, [ConfiguredValueType] [nvarchar](20) COLLATE Latin1_General_CI_AS NOT NULL ) ON [PRIMARY]
I've run into a problem with SSIS packages wherein tasks that write or copy files, or create or delete directories, quit execution without any hint of an error nor a failure message, when called from an ASP.NET 2.0 application running on any other machine than the one where the package was created from. By all indications it appeared to be an identity/permissions problem.
Our application involves a separate web server and database server. Both have SQL Server 2005 installed, but the application server originally only had Integration services. The packages are file system-deployed on the application server, and are called using Microsoft.SqlServer.Dts.Runtime methods. For all packages that involve file system tasks, the above problem occurs.
When the above packages are run using the command prompt (either DTEXEC or DTEXECUI) the packages execute just fine. This is expected since we are using an administrative account. However when a ShellExecute of the same command is called from ASP.NET, the same problem occurs.
I've tried giving administrative permissions to the ASPNET worker process user to no avail.
I have likewise attempted to use the SQL Server Agent job approach but that approach might not be acceptable for our clients since it means installing SQL Server 2005 Database services on the application server.
I have read the relevant threads in this forum, namely http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1044739&SiteID=1 and http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=927084&SiteID=1 but failed to find any solution appropriate for our set up.
