What's The Best Practice...1 Package Many Config Or Many Package W/1 Config
Apr 4, 2007
Hello all,
I'd like to know what's considered a best practice in the following scenario...
We're in the process of converting DTS to SSIS, our current SQL2K setup has one DTS package per database, each database and package are mirror images of each other, with the exception of connection properties and the files path's they pull from. (Occasionally there may be a one off Execute SQL Task that differs from package to package...but this will be handled in a different manner outside of the package from this point on)
From a administration perspective this has been somewhat cumbersome, as every time there is a schema change we need to update the transformations in each individual package...after updating 120 packages my index finger has a bad case of carpel tunnel!
My thought with SSIS is that I can create one package, with many configuration files (one for each database).
Now my question is, 1) is this a good idea? 2) Anytime we create a new database (New Customer = New Database) and need to create a new xml config file, do I have to create the config in BIS/SSIS package then redeploy the package?...or Can I simply add a new config file to the appropriate directory and reference this config in the scheduled job?
I have a situation where I have Package A, which is called from Package B. Both packages have been designed to use config files for their connections, etc.
The issues arises when executing Package B. I can specify a config for Package B to use, but how do I tell the embedded package to use its correct config file?
By default, Package A has its config file pointed to development, but I would rather not have to save a special version of Package A that points to production.
I dont see any option within the execute package task.
Here's is the situation : I have a script component that uses the aspnet membership provider to create users from a csv file in an aspnetdb database. In my Business Intelligence Developpement Studio (BIDS) everything works well. I added the aspnet membership provider in DtsDebugHost.exe.config and when I run the package from the BIDS it does what it have to do.
The problems begin when I deploy the package in file system as well as in sql server. When I run it doesn't use the DTExec.exe.config to find where are added the membership provider. If I refer to that post : http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=541086&SiteID=1 When I run the package from SQL Server it should use the DTExec.exe.config as an app.config for my script component. But it doesn't use it at all and stop with an error that have nothing to do with that. I'm sure that it doesn't use because even when I have a wrong DTExec.exe.config it shows the same error.
If some one knows something about that it will be really great. I need to force my package when it's in stored packages to use one of the *.config as an app.config file.
I have manually imported my packages into SSIS (via StoredPackages/FileSystem/Import Package command). How can I utilize a ConfigFile with this package? I see that the ConfigFile can be selected when it is run (in dtexecUI ), but it doesn't "stick", and is only used for that particular run.
How can I point this manually imported package to the ConfigFile so everybody who runs it doesn't have to browse for the ConfigFile every time they run it?
I've created a solution with 5 packages in. I've scripted a config file for each package where I would be able to change the source file connection and the database servers (source and destination). On the first package, I am able to change the config attributes and the affects are picked up, but the rest of the packages do no pick up the changes.
For example, if the source file is "C:Filessource.txt" and this is changed to "C:Filessource123.txt" (Just to make sure it fails) it wont pick up the new changes and still uses "C:Filessource.txt".
Also, I tried changing the name of config file itself and the package still ran as if nothing had changed. It's as if it's not recognising the config file.
Dear all, I have built an SSIS package by using the BI Dev Studio and enabled its configuration xml file. The package have a variable called TranDate and I want to put it dynamically from a Calendar on my website (just like assigning a variable). I have successfully change the value of that variable in the configuration files (affected to the xml file). Then I loaded the package and executed it (through my web). It's still get the old value (which I have assigned while creating the package). I didn't understand that where else can the package get the value of that variable so it still get the old value (that value have never appeared in the xml configuration file anymore when I changed it).
Thanks for reading this, and I am looking forward to seeing any helps from you guys.
I have a package that executes 3 other packages in it. From the command line I pass in which config file to use (just contains db/seerver) does this setting get carried over into the child packages as well?
I'm currently trying to convert over packages from SQL 2000 to SQL 2005. The biggest obstacle at the start has to do with converting my "Dynamic Properties" control. I use it to read an .ini file and load the user name and password to my connections. With 2000 it's nice and easy 1 file 3 lines. While trying to convert it I€™ve had nothing but problems. I've tried the Registry Entry but it forces everything into the Current User and I can't use that I need local machine. There is no documentation and I can't have a xml file with 1000 different username and password settings because 1 person has to update that file with the new passwords. Any help would be GREATLY appreciated.
I am working on a project currently where we have many SSIS packages and we want to minimize the number of config files to be used. What I was thinking was using one master config file which will have Server/Database info which will be used by all the packages. Now the thing is there are multiple packages which are being executed from within a master package and I was wondering if this will be an issue. Is there anything else that I will have to keep in mind? Any help is appreciated.
After developing SSIS Package (.dtsx file) if I need to deploy to all environments dynamically, then how can I create .dts config file and mention properties in it ?Â
While trying to build SSIS Package Project, the following error is thrown. I have added configuration file recently to fetch the dynamic values such as connection string, output path etc.
I tried manually cleaned up the debug folder and tried build but still the error occurs.
Can any one help in suppressing this error?
Error 1 System.ApplicationException: Could not copy file "D:ExportExport.dtsConfig" to the deployment utility output directory "D:ExportinDebug". ---> System.IO.IOException: The file 'D:ExportinDebugExport.dtsConfig' already exists. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.File.InternalCopy(String sourceFileName, String destFileName, Boolean overwrite) at Microsoft.DataTransformationServices.Project.DataTransformationsProjectBuilder.CopyFiles(ICollection fileNames, String outputPath) --- End of inner exception stack trace --- at Microsoft.DataTransformationServices.Project.DataTransformationsProjectBuilder.CopyFiles(ICollection fileNames, String outputPath) at Microsoft.DataTransformationServices.Project.DataTransformationsProjectBuilder.CreateDeploymentUtility(IOutputWindow outputWindow) 0 0
Currently have a single hard coded file path to the SSRS config file which parses the file and provides the reporting services web service url. Â My question is how would i run this same query against 100s of servers that may or may not share the same file path as the one hard coded ?
Is there a way to query the registry to find the location of the config file of any server ? which could be on D, E, F, H, etc.Â
I know I can string together the address followed by "reports" and named instance if needed, but some instances may not have used the default virtual directory name (Reports).
Am I going about this the hard way ? Is there a location where the web service url exists in a table ? I could not locate anything in the Reporting service database. Basically need to inventory all of my reporting services url's.
I am new to SSIS. I need to transfer data from SQL Server 2005 Operational Database to SQL Server 2005 Report Database. The upload needs to work every night. There are few master tables and remaining are transactions tables.
I am planning to create 2 packages one for master tables and other for transactions tables. Is it the good approach?
Also few of transaction tables are heavy in terms of number of records. Will it better if i further break them in many packages?
I am using book "Microsoft SQL Server 2005 Integration Services Step by Step". Can you suggest any other good book?
Hi,My application is acting weird and the research I've done leads me to an answer why it's breaking, but not how to fix it.My web app is hosted by a remote company. I have the main website sitting in the root web directory and in a subfolder I have another application that will handle login and updating the content in the SQL 2K5 backend (not an Express DB).The website itself is able to pull in informaiton from the DB without any trouble, so I know it's able to connect to the DB and everything is peachy. The login app though is acting as though it can't get to the DB. I've applied the ASPNET schema to the DB using the aspnet_regsql.exe from the framework and the login application is using the same connection string as the main website.One thing to note here is that the website and the login app have seperate web.config files since they were developed as different projects.With my customized provider defined in the web.config for the login app I get this error on page load:"Parser Error Message: It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level. This error can be caused by a virtual directory not being configured as an application in IIS." If I comment out the custom provider, the page loads but when I try to create a user it tells me that "...the SQL Server couldn't be contacted. This may be because it doesn't accept remote connections." Thing of it is that it's not a remote connection and it's the same connection string and db as the other site.Now, the research I've done points at the two different web.config files, which makes sense. But how do I make the one web.config work for both the website and the subdirectory with the login app? Keep in mind that I can't create any new virtual directories or modify IIS since this is a remote hosting company.Can someone help me understand the fix with a little more clarity?
I'm kind of the first person at my new company to start a project using SSIS. My question(s) revolves around setup and configuration, maybe even best practices.
1. Connection Manager Config - What are some of the methods you use to config your connection managers? I was planning on creating a folder (C:SSISConfig or something) and keeping a config file for each connection there. That way it would be fairly trivial for each developer to also put said folder on there box for deving.
2. Solution set up - Are people creating solutions for each package or just using one Master solution and adding all the packages to that? Or maybe some combination, like having solutions for different areas of functionality or something..? I'm trying to figure out what the pain points are with different approaches.
If you have any other information or horror stories about SSIS that you think might help, feel free to pass them along.
I want to make an application using a distributed database (single logical database, parts of which are located at more than one location or more than one server). It acts as a single collection of data but the database itself is geographically in separate locations. How do I make it in SQL Server 2005?
I am trying to use a custom C# dll from within a script task. The C# dll relies on the Enterprise Library 2.0. Where do I put the enterprise library configuration when I am debugging a package and then when I have the package in production. I cannot find any documentation anywhere on this.
Does anyone have an example of a web.config file that works with Dynamic Data controls? I asked because I can't seem to arrange my "config sections" in any working order. I use 1 connection string in the web.config to establish a conversation with the database. Thanks in Advance.
I am using vs2005 and have a solution file with 4 projects, 2 of which are presentation layers (admin sections), 1 is a business logic layer, the other a data access layer. The data access layer project is simply a class library that will provide data access functionality. Im using a dataset (xsd) to create my data access objects. My question is how do I select a connection string from the web config file (which is in each of the presentation layer projects) as the data access layer is a class library there is no web.config. Using the connection string selector in the dataset wizard doesnt show any available connection strings.
Hi the data connectionstring will be generated automatically in web.config file after dragging a data table from database into webpage.for example: <configuration> <add name="SQLProfileConnString" connectionString="Data Source=.SQLEXPRESS;AttachDbFilename=|DataDirectory|MSPetShop4Profile.mdf;Integrated Security=True;User Instance=True" providerName="System.Data.SqlClient"/> </> But what if I have to write the connectionString part myself, I mean where I can get the syntax of it? Thanks a lot
When I connect locally, my app runs fine. But when I try to connect via FTP to a dedicated server, I can't connect at all. My connection string is in the web.config file. I noticed that the sessionState in web.config is automatically generated with the standard sa blank pwd. I still can't connect even when I change this to the correct uid and pwd. Does anyone know what I need to do? My conn string's data source is "." Please help.
I've defined my connection string in the web.config file: <add name="RealtorXConnectionString" connectionString="Data Source=localhostSQLExpress;Initial Catalog=RealtorX;User ID=sa;password=welcome" providerName="System.Data.SqlClient" /> Now I want to get that string and use it in some code where I'm going to run a query against the database. This query happens in PageLoad, and won't be bound to any UI component. How can I replace XXXXX below with the value stored in web.config? Dim strConnection As String = XXXXX Dim cn As Data.SqlClient.SqlConnection = New Data.SqlClient.SqlConnection(strConnection) cn.Open()
(Optional). Specifies a configuration file to extract values from. Using this option, you can set a run-time configuration that differs from the configuration that was specified at design time for the package. You can store different configuration settings in an XML configuration file and then load the settings before package execution by using the /ConfigFile option.
Does this mean that I can specify which configuration file to use during runtime? Or is just because I'm too desperate for that, I understood that way
In toder to make my SSIS packages portable, I need to be able to set connection string to sql package configuration. I thought I can do that via XML package, problem is, at execution time SQL server package configuration is queried first, then variables from XML. This way, I have no chance to set connection via XML. Any ideas on how to aproach this ? (I want to have as much configuration on SQL server so I can modify it easily from future GUI application.
I have a simple SSIS Package where the Connections are specified in Configuration files, which was built locally and I wanted to deploy it to a server. Looking at the different options I decided to try the "Import Package..." from Management Studio (SSMS), since I have limited access to the server (and no access to the file system). It does appear to have imported the package from the local file system to the servers "SSIS Package Store".
My question is, what happened to the Configurations files when a package is imported?
When I "Run Package" from SSMS, the Execute Package Utility shows Connection Managers with only part of the connection info, but it executes just fine. Also, it shows no Configurations at all.
My impression is that during "import" the connection managers are created from a snapshot of the current Configurations, so what ends up in SSIS are static connection managers, not Configurations. However I can't seem to find any documentation one way or the other. Also, I suspect that the only reason the partial connection info actually works is due to dtexecUI (?) running locally under my own account.
I have a simple SSIS package which includes one Analysis Services Processing Task. I have created a connection manager object (cmo) and assigned it to the AS task. Within the cmo I selected a specific user name and password to log on to the server with. Tested this [domain][user account] and it works. Now here is the problem. When I run the package either in debug mode or from a command line, the packages ignores the connection manager and attemps to use the account I am logged into the server with.
Using a config file to hold the credentials does not work either. When DTEXEC is used, the package attempts to use the credientials used to log in to the server.
I've been struggling with a problem I have. When I use DTEXECUI to run a package and manually add the config file, it gets ignored and there is no warning as such.
Here's the set up:
1) create SSIS package that uses a DB connection 2) enable config, add XML config, export the connection string, disable config 3) edit config file, change connection string 4) open package from Windows Explorer (double click) 5) add config file created in step 3
All of that works fine, but there are times when it doesn't. I think it depends on the editor used in step 3. If I use textpad and "pretty format" the XML (from the huge one-line built from within Visual Studio), then sometimes the changes aren't picked up. I can tell this because the "default" server in development is not accessable in production. When we "reformatted" the XML back to one line, the correct server was connected to.
I have no idea what the problem with the formatted XML was. I thought it was CRLF vs LF, but I have at least one config that formatted that way and it works fine. My current thought is character encoding when using TextPad.
My biggest problem is that DTEXECUI doesn't tell you that the config file isn't going to be used. It just ignores the "bad" config format and goes on its merry way using the information in the package itself. That really scares me.
Has anyone else seen this? What is the problem being introduced into the config file?
I am new to SSIS. I created a XML configuration file for my package. I found lots of data on how to do this but I can't find anything on how to get the values back out. I am trying to get them out in a Script file. Any help and maybe examples would be appreciated.
I get this error. Does any body came across this and found some solution?
Compiler Error Message: CS0433: The type 'Microsoft.Web.UI.ScriptManager' exists in both 'c:WINDOWS.000Microsoft.NETFrameworkv2.0.50727Temporary ASP.NET Fileswebsites_hrdbms3acda2bba63d28c5assemblydl332deaeca 05606e1_3759c601Microsoft.Web.Atlas.DLL' and 'c:WINDOWS.000assemblyGAC_MSILMicrosoft.Web.Extensions1.0.61025.0__31bf3856ad364e35Microsoft.Web.Extensions.dll'
I have created the application with ATLAS July CTP . currently upgraded to ASP.NET AJAX BETA 1. I face the above problem
Hello all I need to know the best way to work with databases (Sql Server) in my app.config. I have an VB application that binds data from a SQL server (All Customers install these application and retrieve data via VPN). I use two databases for each customer. The first DB has some commun tables/data and the second one is its ID (Until now I setup manually the connections because we don't have more that 10 Customers but we need improve that for next year). I know that a connection string is created in the app.config file . I don't know if it is possible from this way or if there is another way to do it.... but the goal is: 1. Customer is logged in the application -User Name, ID, Password 2. Customer retrieve a database with commun data and his own database (that is his ID). (From the app.config???) How can I setup my app.config in order that every customer retrieve his database automatically when it is logged? Thaks, I don't know if it's clear but I didn't find another way to explain my trouble
