Hello,
I want to store each SQL Server's (prod, test, etc) connection string in XML to make deployment & configuration easy. However, the connection string must be encrypted. Does anyone have any ideas or suggestions on how to accomplish both. I know how to encrypt it but I dont think SSIS can work with it.
Any help would be appreciated.
Suppose 2 environments on a single machine.
Each environment has different configuration settings....different
databases etc.
All the packages in the first environment have a hardcoded config files
referencing the local drive.
In order to create the second environment do I have to
go into each package and manually change the location of the
hardcoded config files. If I don't it will it not use the config files
from the first environment.
Thanks
Hi,
SSIS is behaving differently in different environments but the code is same.
One thing is nor working correctly that is I am converting a string data type column to float data type in data conversion. In our local environments the package is working fine but in production environments it is not working correclty. It is unable to convert the data it is throwing an error.
"The data value cannot be converted for reasons other than sign mismatch or data overflow"
Can anybody help me please?
Hi,
I have created an XML Configuration file within the Designer and have added
my exported my database connection details.
How do I get my TSQL Task to use this connection.
My ConnectionType is OLE DB.
For my Connection how do I pick up what is in my .dtsConfig file?
Thanks in Advance.
Hi,
My requirement is to have the connection strings for the source and destination to be a parameter for the packages that way they can be changed from the sql job scheduler.
Please let me know how to go about it.
Any help will be highly appreciated.
Thanks
Vijaykishan
We are deploying our SSIS packages into different folders. For example:
Test1
est2
production
Test1 points to the Test1 database, etc.
So, I configured my SSIS package to use a database connection called dbMAIN.
I then setup the SSIS File Configuration (XML) so that "dbMain" points to Test1. This xml file, called Global.dtsConfig, sits in C: est1, the same place as my ssis.dtsx file.
I want to copy my dtsx file and my dtsconfig file into est2 and production. The problem is that the the location of dtsconfig is HARDCODED in the ssis package!
How do I dynamically change it?
I tried to hardcode the location to ".Global.dtsconfig", but that did not work! Please advise!
Hi, I'm facing a problem that I'm trying to solve. I report it in the following hoping somone can help me!
the context:
- I have a solution containing almost 20 packages
- I have a main package containing the sequences (Sequence Container) calling all others packages (Execute Package Task)
- every package has the connections to its own DBsource ; so the package X has the its own connections to DBsourceA and DBsourceB, the package Y has the its own connections to DBsourceB, DBsourceC, DBsourceD and the destination DB is unique for all the packages.
My problem is:
I have the necessity to define in the best way a config (one for all the dtsx, possibly) that allow me to manage easily the switch from the developing env to the deployment env, basically for the DB connections.
Which is the best way to do this and can anyone tell me the steps to follow?
Hi all,
I have finished the development of an SSIS package but I would like to make it a bit more flexible by adding a configuration file. The script migrates a database from a source to a destination and syncs with another source.
So there is the source db and the destination db. However, I have noticed that standard SSIS configurations can get a little tricky. For instance, I have two connection managers for the destination db (one that points to the schema and one that doesnt). So my question is: Is there any way to create a config file thta you can input a few attributes like sourceDB, destinationDB, schemaName and get the package itself to parse and process it?
Thanks,
Yannis
I'm facing a little problem concerning package configuration that is using a SQL table.
I have 4 exactly the same Db's on the same SQL server, each with another name of course (Dbipa, DbIpB, DbIpC & DbIpD).
My developers created 4 exactly the same SSIS packages with 4 exactly the same jobs, each for his own DB only the connection settings are different.
Now I was trying to manage this with only one SSIS package by using an SSIS Package configuration table in each DB.
I thought it would be possible to start the package with the following parameter
/SET Package.Connections[SQL].InitialCatalog;DBIPA OR
/SET Package.Connections[SQL].InitialCatalog;DBIPB OR
/SET Package.Connections[SQL].InitialCatalog;DBIPC OR
/SET Package.Connections[SQL].InitialCatalog;DBIPD
But this doesn't work because before he changes the initial catalog in the connection named SQL, he already did the loading of the SSIS_Configuration table. So he point always to the same config table.
Is there a way to change the connection before the package configuration is being executed.
Ludo Bernaerts
1) We are doing data migration from SQL Server 2000 OLTP design to SQL Server 2005 OLAP design.
2) We have used SSIS packages and data flow tasks in which we mentioned connection strings for source and target containers.
3) We have a master execute package which contains series of execute packages in relational order.
4) When we execute this master package, we should be able to dynamically specify different source and connection strings for all packages.
5) In master execute package, we generate an XML configuration file using SSIS -> Package Configurations... using Connection Managers -> Particular connection
6) Now, we change connection string to point to another database after adding this new OLE DB connection in Connection Manager of each and every package.
7) When we save, build and execute master package it is still executing migration packages against the old database. Please let me know what I am missing.
Hi,
I need to configure my package using SSIS Configurations. When I opened SSIS Configuration wizard, press ADD and selected Xml Configuration, I found two options.
One being : Specify configurations directly.
next : config location is stored in env variable.
How to use the second option?[ env variable]. Say I want to configure my connection string for my destination database [ OLEDB Destination Component]. How to do that using second option.
Thanks
Kishan
Hello,
I am having a bit of trouble with SSIS Configurations.
- In BIDS, I have added a configuration file and specified the properties I want to expose.
- When I build the project, I get the standard bindeployment folder which contains the package file (.dtsx), the configuration file (.dtsconfig) and the deploymentmanifest.
- Before deploying the package, I edit the config file to have the settings I want it to for the environment I am deploying to.
- The package deploys OK
- When I work directly on the SSIS server (64 bit), I can go into SQl Mgt Studio, choose 'run package' and when I look in the connection manager window all the settings are as I desire ( I havent had to add a configuration file manually)
- When I work on a client machine, I connect to the SSIS server and choose 'run package' - the properties/connections are the same as on the server but the values themselves are completely different.
Why is this? Why when I run the package from the client (32 bit) do the configuration values appear to be completely different? How can I run the package remotely and pick up the configuration values that I deployed with?
Or have I misunderstood this whole configuration function?
Please, please help!
Mgale1
Hi,
I am new to SSIS. I have an SSIS package which we are storing it in the application server ( not in SQL Server ). We are storing the database name etc.. in package configuration. Where should I store the configuration file? In app server or in the database server. Also please tell how to execute the package stored as file system.
Thanks
ramesh
Hi,
I have an SSIS Package which reads the properties values from Registry. As per the constraint of SSIS I have created the keys under HKCU in the registry and things work fine from the designer.
But, when I schedule the SSIS Package to run under SQL Agent (which is running under Local System) the values of configuration do not seem to be picked up from Registry.
How I tested was:
I am picking up the server name for the Connection Manager from Registry and I set that value to an invalid servername. But, the job seems to be executing fine.
Using DTEXEC it fails as expected since, this is running under my credentials.
Also, SYSTEM has full control on the new regsitry keys.
Any information on this?
Thanks,
Gaurav
Hi,
I would like to know what is the difference between XML file and Database in SSIS configuration option.
Thanks,
K
Hello All,
In my project, developers enable configuration on the packages they develop. This means that if a project has 100 packages, each one has its own configuration file.
If our primary database server changes, I have to go an update 100 configuration files.
This defeats the purpose of having a configuration file in the first place.
I want to have a configuration file which contains common settings for all the packages in the project.
At the same time, I should have the flexibility of having individual configuration files for the project.
Is this possible and how can I do this?
http://blogs.conchango.com/jamiethomson/default.aspx has a lot of great tid-bits for SQL 2005. I am currently on a tight deadine for 25 SSIS packages that need to be able to move from Dev to QA to Staging to Prod. For the life of me I cannot get any of the packages to *READ* the config files created with the package config wizard. All I want to do is move the connection string out of the package so we can change the config file and not have to touch (hand edit) each package. Any help is appreciated!
View 13 Replies View RelatedThanks to some earlier help from this forum, I have my package running using configuration files and Windows Authentication. However, when I try to execute the package using Sql authentication, login fails for the user i am specifying. When I check out the configuration file, the connection string does not contain the password. I added the password to the connection string, but it does not cone back. I followed the instructions found on the following site:
http://www.nigelrivett.net/DTS/SSISConfigurationFile.html
After trying what was suggested there, I was still unsuccessful with my login attempt.
Anyone know how to set up a package to run using Sql authentication?
Thanks!
Craig
OK, here's the scenario:
I have some libraries (DLL) that I call from a regular console application. The data access methods get the connection string from the app.config, nothing special there.
The thing is: now I need to use the libraries inside a SSIS package. I call them from a script task, and everything is fine, except for the connection string.
Is there a way that I can get the connection string from the SSIS package configuration like I would from the app.config? Maybe some alternative to ConfigurationManager.AppSettings["ConnString"] in the DTS Runtime?
Thanks in advance...
Hi
I want know how to config SSIS Package With different connection Manager.
Sample .
When Design time i gave The Connection Manager(Devlopement Server) Name : STG1.XXXSchema
Server Name STG.ssss
Password : xxxxx
I make the The XML configuration file .
after deploy the Package i want from Development Server to Production Server . so i will change the Package Config File Open wiht Notepad .
Server Name STG.qqq
Password :xxxxxx
after Edit Connection String I run the Package . It Always Taking Whatever igave the Connection String When Design time only .is not taking what connection string edit hte in the Package Config File. Please Any one give the Solution .
How we can Change the connection string in XML Config File .Whne I tested . It always taking what ever we given in the Desing Time .
Thanks & Regards,
Jeyakumar .M
I have an SSIS package (which is already delivered to the customer, so I can't change it), but I want to switch off (disable) one of the elements. I use a dtsConfig file (XML), so I want to just send an update to that config file to switch off the element.
So, I used the Package Configurations Organizer to generate a new config file with the element switched off, and it generated the following:-
<Configuration ConfiguredType="Property" Path="PackageRun Data LoadForeach Data FileLoad Invalid Format Recs into rej_dat_format_errors.Properties[Disable]" ValueType="Boolean">
<ConfiguredValue>0</ConfiguredValue>
</Configuration>
So I copied these lines into my "production" config file and ran the package, but got the following warning message:-
Warning: 0x80012017 at ImportData: The package path referenced an object that cannot be found: "PackageRun Data LoadForeach Data FileLoad Invalid Format Recs into rej_dat_format_errors.Properties[Disable]". This occurs when an attempt is made to resolve a package path to an object that cannot be found.
Can anyone explain why? Is there some issue with having spaces in the Path name perhaps? Is there some syntax to get around it? Why did it generate a Path that is wrong?
Thanks
- AA
We have more than 100 packages which referenced many of the same data sources. In
ideal situation, we would like to create three master filters based on the dev, test, and prod servers and all packages can just reference the same filters. However since one package normally only use a subset of master connections defined in the configuration files, if only those subset of connections are set up in the package, when the
package is first open in BIDS, lots of errors will pop up saying that it could not find the connections defined in the configuration files. Even though this will not cost runtime error, it is still very annoying. Does anyone have better ideas on how to handle this?
I have an SSIS package, (actually many packages at this point because I can duplicate this problem each and every time) with an FTP Task and three connection managers (one for FTP, one for a file share, one OLEDB).
The package in its simplest form, grabs a file from an FTP site and transfers it to the file share. When there are no files found on the FTP site, the FTP task fails as expected.
Here is where it gets strange...I turned on a package configuration, with all variables / properties saved to a SQL table; all except for the OLEDB connection which I am leaving hardcoded in the package for the time being.
The problems I am seeing as soon as I activate the package configuration are twofold: The first occurs with the property "FileUsageType" defined to the file share connection. I have it set to "existing folder" in the SSIS tool, the SQL config table also has a row for this item defined with a value of 2. I have even tried manually changing this value in the SQL config table and the result is always the same error when I run the package:
Error: 0xC002F314 at FTP Task, FTP Task: File usage type of connection "Z:" should be "FolderExists" for operation "Receive".
If I remove this entry totally from the SQL config table, and leave that property hardcoded in the package then I am able to continue execution in the package and that particular error goes away. But then I come to weird problem #2: I stated above that when there are no files in the FTP directory to retrieve, then the FTP task fails, as I would expect. However, as soon as I activate the package configuration, even when there are no files at the FTP site, the FTP task doesnt fail anymore, it marks itself as successful instead. This is wreaking havoc on my precedence constraints that follow the FTP task naturally. As soon as I deactivate the package configuration, the FTP task behaves normally again.
The problems only seem to happen with config packages stored in SQL. I tested this also with XML file config instead and everything seemed to behave as it should. Unfortunately, I need to store these configs in SQL, not XML.
Does anyone have any ideas on what I need to look at to fix these weird issues?
Gentlemen
I've been working with SSIS (SQL 2005 SP1) and I have a problem adding a package configuration of the SQL Server type. Upon completion of all steps in the wizard, the "Target Object" and "Target Property" attributes of this configuration remain empty. Hence, the package cannot set this configuration when being executed.
Does anyone know if this is a bug or have I missed something?
Thank you
Karch
i am trying to configure my ssis package thru xml file option.
the location of the xml file should be pointing to a common folder name in different SQL Server (Dev, QA and Prod)
say for example
xml file need to be present in dev server (DEV01) at \DEV01outputssistest.xml
in QA it should be present in \QA01outputssistest.xml
i am able to do thru BIDS by changing individually but how do i do in package designer only once and change in xml later.
please advice.
How can I update a variable in the XML configuration file. Can I use the configuration object for it or do I need to use XML objects and do it just like any other XML file?
Another problem I have is, I know I can reference the same XML config file from 2 different packages. But every time I do that through the wizard, only the last package configurations are placed in the XML file.
Any help is appreciated!
Hello,
I need some clarification. I am trying to utilize the XML Direct Configuration in my SSIS packages to utilize database connections in the package. I am wanting to utilize this the same way you could use UDLs in the SQL Server 2000 DTS packages.
Currently, I am creating the dtsConfig file and saving it to my desktop. I then modify it with notepad and add the following configuration where it looks like this:
<DTSConfiguration>
<Configuration ConfiguredType="Property" Path="Package.Connections[ConnectionName].Properties[ConnectionString]" ValueType="String">
<ConfiguredValue>Data Source=[ServerName];Initial Catalog=[DBName];Provider=SQLNCLI.1;Integrated Security=SSPI;Auto Translate=False;</ConfiguredValue>
</Configuration>
</DTSConfiguration>
Once this is created, I am trying to "re-use" this in SSIS packages created moving forward, where they all point to this configuration for the same database connection.
What I don't understand is when "Enabling Package Configuration" and then pointing to this dtsConfig file doesn't create a connection in "Connection Manager" NOR does it provide a way to create a blank connection and point to this configuration.
I feel like I am missing something thats so "great" about XML configuration files. Any help would be appreciated.
Thanks,
Daniel Lackey
MCSD
It looks like you can run an SSIS package via the DTEXEC command. You can also specifiy a configuraiton file flag on the command. Is there anyway to specify a configuraiton filter if you stored it into a table vs and xml file?
View 1 Replies View RelatedHello,
I have several packages that share one configuration file located at a network location. Up until i installed sp2 everything worked fine, but now the packages "loose" the path to the configurationfile. It is replaced by some c:programsqlserverpackage.... path where no config file is located.
I can run packages if i specify the config file explicetly, but I really don't wanna do that, since it will not work in the context i'm working.
I am using the wizard to install my packages.
Anyone with similar problems ? Any help appriciated.
Note: This is not the issue that sp1 solved (for me at least). =)
Hello Everybody,
I have developed a SSIS package to pull data from a remote SQL Server.
I have specified Database related settings in a dtsconfig file.
I was just wondering if we have any way of encrypting the config file so that only my package can read information out of it.
Or is there any other better way where i can store my database configuration (uid, pwd) so that it is not viewed by anyone.
please help me with this issude, thanks in advance...
Regards,
Sudhir Kesharwani
Hello all,
I have an issue with my SSIS package not using the package configuration specified in my SQL server table. This is a simple scenario where I need to move data from a csv file to a SQL server table.
I have 2 connection Managers - one for the location of CSV file and another for the SQL database connection string. I chose to Enable Package configuration. Two configurations -
a. Environment variable to store the database connection string.
b. SQL server table configuration for flat file connection manager.
The location of the CSV file is dynamic..so the idea is to change the path as and when needed in the SQL table.
No errors..everything works fine except that the properties of the flat file connection manager is not being pulled from the SQL server configuration table. It uses the location I had specified during design time.
How do I force the SSIS package to use the configuration specified in the SQL table always.
I've been spending a lot of time on this with no success. Please help..
Thanks
Karthik
Can someone identify what is wrong with the XML config file that I i'm using with a SSIS package? The XML file was created by Visual Studio tool
Code Snippet
<DTSConfiguration>
<DTSConfigurationHeading>
<DTSConfigurationFileInfo GeneratedBy="BRIDGENETjpacella" GeneratedFromPackageName="SsisEndBatch" GeneratedFromPackageID="{91A1941E-E066-47F7-8DD4-9E260F38741D}" GeneratedDate="1/21/2008 4:35:22 PM" />
</DTSConfigurationHeading>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-DEV02.BNETDB2].Properties[InitialCatalog]" ValueType="String">
<ConfiguredValue>BNETDB2IMPORTTEST</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-DEV02.BNETDB2].Properties[ServerName]" ValueType="String">
<ConfiguredValue>CHI-DEV02</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-PROD01.BNETDB2].Properties[InitialCatalog]" ValueType="String">
<ConfiguredValue>BNETDB2</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-PROD01.BNETDB2].Properties[ServerName]" ValueType="String">
<ConfiguredValue>CHI-FILE01</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="PackageRun Report Executor.Properties[Executable]" ValueType="String">
<ConfiguredValue>C:ReportsReportProgramCHI_FILEBridgeNet.Reporting.ExecuteConsole.exe</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="PackageRun Report Executor.Properties[WorkingDirectory]" ValueType="String">
<ConfiguredValue>C:ReportsReportProgramCHI_FILE</ConfiguredValue>
</Configuration>
</DTSConfiguration>
When I run the pacakge, this is what happens:
Started: 10:48:56 AM
Info: 2008-01-24 10:49:02.92
Code: 0x40016041
Source: SsisEndBatch
Description: The package is attempting to configure from the XML file "C:Program Files (x86)Microsoft SQL Server90DTSPackagesBridgeNet.BI.SSIS.EndBatc
ENDBATCH TEST ENV.dtsConfig".
End Info
Warning: 2008-01-24 10:49:03.15
Code: 0x80012011
Source: SsisEndBatch
Description: Cannot load the XML configuration file. The XML configuration file may be malformed or not valid.
End Warning
I have been asked by the powers that be to make sure that my configuration database gathers some auditing information. I have looked over these and have no idea where to find the information in SQL Server. If you could tell me the table and database that the informations are located in, I could write the T-SQL to find them. IMPORTANT you do not have to answer all of them, I am grateful to those that pick and choose even one to help me with.
SQL Server Auditing
General/Access Auditing Items
Software Install (DB): Verify that the sample and demonstration databases are not installed and remove the temporary setup files created by SQL Server setup process
Authentication and Access Control
Default user accounts & passwords (DB): Default user account passwords will be changed and will follow the corporate password standard for frequency of change, length and complexity
User authentication (DB): Windows authentication is preferred. Mixed mode authentication can be used.
User passwords (DB): All user passwords (mixed mode, SQL authentication, Windows authentication) will follow the corporate password standard for frequency of change, length and complexity.
Sharing database user accounts (DB): Sharing of database user accounts is not permitted without an explicit exception documented and granted by IS Information Security.
Normal users submitting jobs (DB): Normal user jobs should not be run as SA. Create a role-based policy for normal users that must be allowed to run jobs. They should be included in the role created for job submission.
Monitoring and Reporting
Auditing (DB): For all database installs after XX/XX/XXXX, Auditing should be turned on and at minimum set to log failed connections. Logs should be saved to a different hard drive than the one which data files are stored.
SQL error log (DB): Processes should be in place to protect error log data so it can be reviewed for seven days prior to being overwritten or deleted from the system.
Networking
Cross-database ownership chaining (DB): Cross-database ownership chaining is disabled.
Code of stored procedures, triggers and views (DB): Who has access
Public access to SYSXLOGINS and SYSDATABASES tables: The public role will not have access to the SYSXLOGINS or the SYSDATABASES tables.
Public access to SYSOBJECTS and SYSCOLUMNS tables: The public role will not have access to the SYSOBJECTS and SYSCOLUMNS tables.
Public access to stored and extended stored procedures: The public role will not have access to stored or extended stored procedures.
Public access to xp_regread and xp_instance_regread: Restrict public role access to xp_regread and xp_instance_regread.
Public group access to mswebtasks table: The public group will not have insert, update, delete or select permissions to the msdb.dbo.mswebtasks table.
Temporary directories: For all database installs after xx/xx/xxxx, all temporary directories will be cleaned on a periodic basis.
Auditing II: Auditing should be turned on and at minimum set to log failed connections. Logs should be saved to a different hard drive than the one which data files are stored.
SQL Server instances visibility: All SQL Server instances should not be visible across the network. They should be set up with the hidden option activated.
Default ports: SQL Server should be configured to not use the default ports.