Hi,
I am new to SSIS. I have an SSIS package which we are storing it in the application server ( not in SQL Server ). We are storing the database name etc.. in package configuration. Where should I store the configuration file? In app server or in the database server. Also please tell how to execute the package stored as file system.
Thanks
ramesh
Deployed Report having SSIS package as source do not work when Indirect Package configuration is used in ETL package. It seems ETL package when called/executed from Report manager does not recognize environment variable to pick up the dtsconfig file.
The Report works when Direct package configuration is used to same dtsconfig file.
What could be the reason? Any solution for this? This will cause our build/deployment to QA and Prod very difficult.
Hi, I'm facing a problem that I'm trying to solve. I report it in the following hoping somone can help me!
the context:
- I have a solution containing almost 20 packages
- I have a main package containing the sequences (Sequence Container) calling all others packages (Execute Package Task)
- every package has the connections to its own DBsource ; so the package X has the its own connections to DBsourceA and DBsourceB, the package Y has the its own connections to DBsourceB, DBsourceC, DBsourceD and the destination DB is unique for all the packages.
My problem is:
I have the necessity to define in the best way a config (one for all the dtsx, possibly) that allow me to manage easily the switch from the developing env to the deployment env, basically for the DB connections.
Which is the best way to do this and can anyone tell me the steps to follow?
I'm facing a little problem concerning package configuration that is using a SQL table.
I have 4 exactly the same Db's on the same SQL server, each with another name of course (Dbipa, DbIpB, DbIpC & DbIpD).
My developers created 4 exactly the same SSIS packages with 4 exactly the same jobs, each for his own DB only the connection settings are different.
Now I was trying to manage this with only one SSIS package by using an SSIS Package configuration table in each DB.
I thought it would be possible to start the package with the following parameter
/SET Package.Connections[SQL].InitialCatalog;DBIPA OR
/SET Package.Connections[SQL].InitialCatalog;DBIPB OR
/SET Package.Connections[SQL].InitialCatalog;DBIPC OR
/SET Package.Connections[SQL].InitialCatalog;DBIPD
But this doesn't work because before he changes the initial catalog in the connection named SQL, he already did the loading of the SSIS_Configuration table. So he point always to the same config table.
Is there a way to change the connection before the package configuration is being executed.
Ludo Bernaerts
1) We are doing data migration from SQL Server 2000 OLTP design to SQL Server 2005 OLAP design.
2) We have used SSIS packages and data flow tasks in which we mentioned connection strings for source and target containers.
3) We have a master execute package which contains series of execute packages in relational order.
4) When we execute this master package, we should be able to dynamically specify different source and connection strings for all packages.
5) In master execute package, we generate an XML configuration file using SSIS -> Package Configurations... using Connection Managers -> Particular connection
6) Now, we change connection string to point to another database after adding this new OLE DB connection in Connection Manager of each and every package.
7) When we save, build and execute master package it is still executing migration packages against the old database. Please let me know what I am missing.
We have more than 100 packages which referenced many of the same data sources. In
ideal situation, we would like to create three master filters based on the dev, test, and prod servers and all packages can just reference the same filters. However since one package normally only use a subset of master connections defined in the configuration files, if only those subset of connections are set up in the package, when the
package is first open in BIDS, lots of errors will pop up saying that it could not find the connections defined in the configuration files. Even though this will not cost runtime error, it is still very annoying. Does anyone have better ideas on how to handle this?
Gentlemen
I've been working with SSIS (SQL 2005 SP1) and I have a problem adding a package configuration of the SQL Server type. Upon completion of all steps in the wizard, the "Target Object" and "Target Property" attributes of this configuration remain empty. Hence, the package cannot set this configuration when being executed.
Does anyone know if this is a bug or have I missed something?
Thank you
Karch
i am trying to configure my ssis package thru xml file option.
the location of the xml file should be pointing to a common folder name in different SQL Server (Dev, QA and Prod)
say for example
xml file need to be present in dev server (DEV01) at \DEV01outputssistest.xml
in QA it should be present in \QA01outputssistest.xml
i am able to do thru BIDS by changing individually but how do i do in package designer only once and change in xml later.
please advice.
It looks like you can run an SSIS package via the DTEXEC command. You can also specifiy a configuraiton file flag on the command. Is there anyway to specify a configuraiton filter if you stored it into a table vs and xml file?
View 1 Replies View RelatedHello Everybody,
I have developed a SSIS package to pull data from a remote SQL Server.
I have specified Database related settings in a dtsconfig file.
I was just wondering if we have any way of encrypting the config file so that only my package can read information out of it.
Or is there any other better way where i can store my database configuration (uid, pwd) so that it is not viewed by anyone.
please help me with this issude, thanks in advance...
Regards,
Sudhir Kesharwani
Hello all,
I have an issue with my SSIS package not using the package configuration specified in my SQL server table. This is a simple scenario where I need to move data from a csv file to a SQL server table.
I have 2 connection Managers - one for the location of CSV file and another for the SQL database connection string. I chose to Enable Package configuration. Two configurations -
a. Environment variable to store the database connection string.
b. SQL server table configuration for flat file connection manager.
The location of the CSV file is dynamic..so the idea is to change the path as and when needed in the SQL table.
No errors..everything works fine except that the properties of the flat file connection manager is not being pulled from the SQL server configuration table. It uses the location I had specified during design time.
How do I force the SSIS package to use the configuration specified in the SQL table always.
I've been spending a lot of time on this with no success. Please help..
Thanks
Karthik
Can someone identify what is wrong with the XML config file that I i'm using with a SSIS package? The XML file was created by Visual Studio tool
Code Snippet
<DTSConfiguration>
<DTSConfigurationHeading>
<DTSConfigurationFileInfo GeneratedBy="BRIDGENETjpacella" GeneratedFromPackageName="SsisEndBatch" GeneratedFromPackageID="{91A1941E-E066-47F7-8DD4-9E260F38741D}" GeneratedDate="1/21/2008 4:35:22 PM" />
</DTSConfigurationHeading>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-DEV02.BNETDB2].Properties[InitialCatalog]" ValueType="String">
<ConfiguredValue>BNETDB2IMPORTTEST</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-DEV02.BNETDB2].Properties[ServerName]" ValueType="String">
<ConfiguredValue>CHI-DEV02</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-PROD01.BNETDB2].Properties[InitialCatalog]" ValueType="String">
<ConfiguredValue>BNETDB2</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="Package.Connections[CHI-PROD01.BNETDB2].Properties[ServerName]" ValueType="String">
<ConfiguredValue>CHI-FILE01</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="PackageRun Report Executor.Properties[Executable]" ValueType="String">
<ConfiguredValue>C:ReportsReportProgramCHI_FILEBridgeNet.Reporting.ExecuteConsole.exe</ConfiguredValue>
</Configuration>
<Configuration ConfiguredType="Property" Path="PackageRun Report Executor.Properties[WorkingDirectory]" ValueType="String">
<ConfiguredValue>C:ReportsReportProgramCHI_FILE</ConfiguredValue>
</Configuration>
</DTSConfiguration>
When I run the pacakge, this is what happens:
Started: 10:48:56 AM
Info: 2008-01-24 10:49:02.92
Code: 0x40016041
Source: SsisEndBatch
Description: The package is attempting to configure from the XML file "C:Program Files (x86)Microsoft SQL Server90DTSPackagesBridgeNet.BI.SSIS.EndBatc
ENDBATCH TEST ENV.dtsConfig".
End Info
Warning: 2008-01-24 10:49:03.15
Code: 0x80012011
Source: SsisEndBatch
Description: Cannot load the XML configuration file. The XML configuration file may be malformed or not valid.
End Warning
I have been asked by the powers that be to make sure that my configuration database gathers some auditing information. I have looked over these and have no idea where to find the information in SQL Server. If you could tell me the table and database that the informations are located in, I could write the T-SQL to find them. IMPORTANT you do not have to answer all of them, I am grateful to those that pick and choose even one to help me with.
SQL Server Auditing
General/Access Auditing Items
Software Install (DB): Verify that the sample and demonstration databases are not installed and remove the temporary setup files created by SQL Server setup process
Authentication and Access Control
Default user accounts & passwords (DB): Default user account passwords will be changed and will follow the corporate password standard for frequency of change, length and complexity
User authentication (DB): Windows authentication is preferred. Mixed mode authentication can be used.
User passwords (DB): All user passwords (mixed mode, SQL authentication, Windows authentication) will follow the corporate password standard for frequency of change, length and complexity.
Sharing database user accounts (DB): Sharing of database user accounts is not permitted without an explicit exception documented and granted by IS Information Security.
Normal users submitting jobs (DB): Normal user jobs should not be run as SA. Create a role-based policy for normal users that must be allowed to run jobs. They should be included in the role created for job submission.
Monitoring and Reporting
Auditing (DB): For all database installs after XX/XX/XXXX, Auditing should be turned on and at minimum set to log failed connections. Logs should be saved to a different hard drive than the one which data files are stored.
SQL error log (DB): Processes should be in place to protect error log data so it can be reviewed for seven days prior to being overwritten or deleted from the system.
Networking
Cross-database ownership chaining (DB): Cross-database ownership chaining is disabled.
Code of stored procedures, triggers and views (DB): Who has access
Public access to SYSXLOGINS and SYSDATABASES tables: The public role will not have access to the SYSXLOGINS or the SYSDATABASES tables.
Public access to SYSOBJECTS and SYSCOLUMNS tables: The public role will not have access to the SYSOBJECTS and SYSCOLUMNS tables.
Public access to stored and extended stored procedures: The public role will not have access to stored or extended stored procedures.
Public access to xp_regread and xp_instance_regread: Restrict public role access to xp_regread and xp_instance_regread.
Public group access to mswebtasks table: The public group will not have insert, update, delete or select permissions to the msdb.dbo.mswebtasks table.
Temporary directories: For all database installs after xx/xx/xxxx, all temporary directories will be cleaned on a periodic basis.
Auditing II: Auditing should be turned on and at minimum set to log failed connections. Logs should be saved to a different hard drive than the one which data files are stored.
SQL Server instances visibility: All SQL Server instances should not be visible across the network. They should be set up with the hidden option activated.
Default ports: SQL Server should be configured to not use the default ports.
Hi,
I have a SSIS package called "MyExport" stored on the SQL server 2005 standard SP1. I have created an SSIS package configuration stored in a SQL table msdb.dbo.SSIS Configurations, with the configurationFilter = "Export2" with some configuration values which I can change programmatically as needed.
How do I execute this package with the configuration in the SQL table? I don't seem to have any problem when the package configuration is stored in an .xml file. The documentation is very poor or non-existant on trying to do execute the package with configurations stored in SQL server. I just can't seem to get the proper syntax.
Can someone give me an example of a dtexec command for the above or maybe some c# code?
In Execute Package Utility, when you select the Configurations option, it pops up a dialog box for a FILE based configuration file (*.dtsconfig). There is NO WAY to access a configuration stored in the [dbo].[SSIS
Configurations] table - which is where the SQL based configurations are stored. If you could pick a SQL based configuration, you would then need to pick the ConfigurationFilter for the configuration you want to use.
It's the same when you try to create a new job in SQL Server Agent - you can't select the package configuration stored in SQL server. When you get to the configurations tab, you can only add a file based configuration.
I've checked BOL as well, and there are no examples or discussion of this that I can find.
If anyone can point me in the proper direction, I'd appreciate it.
Thanks.
How do I create, say an "update statistic" maintenace package that works on multiple servers and databases?
For example, how do I get the package to operate on the following servers and tables:
ServerA, tableX, tableY, tableZ
ServerB, tableM, tableN
Also, I would like to save the package configuration and store it on SQL Server. So if I want to add more servers or tables to the list, the SSIS package will pick it up at run time.
Is this possible?
We have 6 Â SSIS packages which populates different sets of table by reading different set of excel file.We need to have a master SSIS package which will have the configuration (say xml) which consists of database connection details and file path details of child packages.what will the best way to achieve the desire results.
Package 1 use File x
package 2 Â use File y
package 3 use File z
....
package 6 use File a
The parent ssis package will have xml file as configuration which will store the all the six different file details for child packages along with database connection string.Is the above option feasible . or what approach will be the best possible way to achieve the results.Since the Triggering of SSIS package (Master SSIS package) will be from SQL Job
I am new to SQL server 2005 and have a config question:
I am controlling database connection info using XML indirect config - no problems there.
Essentially I am going to have a number of packages that need to use a common file path, that might change from one server to the next, e.g. Server 1: C:sourceFiles versus Server 2: D:sourceFiles. Within this directory the filenames will remain static. So in the flat file connection manager I'd like to use a variable to reflect the folder - but I don't want to have to create this for each package.
So, I thought I would create a system environment variable and create expressions for the connection managers - something like %SOURCE_DIR% + "file.csv" - but this does not evaluate correctly.
So then i though I could use the SQL server configurations table with a configurationFilter SOURCE_DIR and appropriate configuration value - but then how do I access this in the flat file connection manager to create a dynamic file name?
So essentially I want a variable/property available globally to all my packages and potential flat file connection managers that help me to centrally control file path locations.
Any help would be most appreciated.
On Visual Studio 2013 with SSDT for SQL Server 2014
I noticed something very strange.
I am viewing the package configuration selected option for a package:
Now, i just click on the - sign and then on the plus sign on the "Properties" branch. And the option is cleared:
The ONLY way i found not to mess everything is just to have ALL the options FULL expanded and THEN to select what i want.
Hi --I was wondering if this is a bug when I add new data in my table SSIS Confiurations and give wizard a new Configuration filter the package configuration wizard can not see the new values --the old values from the previous configuration are still showing---is there any known workaround or forced refresh I can do
thanks in advance Dave
Background:
SQL Package Configurations are most important because they provide the possibility of a central configuration store for your entire enterprise!!!!!!!! and is in my mind the only way to go
http://sqljunkies.com/WebLog/knight_reign/archive/2005/01/24/6843.aspx
Wizard results:
Name:
ETL
Type:
SQL Server
Connection name:
ETLConfiguration
Any existing configuration information for selected configuration filter will be overwritten with new configuration settings.
Configuration table name:
[dbo].[SSIS Configurations]
Configuration filter:
PT_CUST_ABR
Target Property:
Package.Variables[User::gsPreLoad].Properties[Value]
Package.Variables[User::gsPostLoad].Properties[Value]
Package.Variables[User::gsLoad].Properties[Value]
Package.Variables[User::gsFlatFilename].Properties[Value]
Package.Variables[User::gsFileName].Properties[Value]
Package.Variables[User::gsCDOMailTo].Properties[Value]
Package.Variables[User::gsCDOMailSubject].Properties[Value]
Package.Variables[User::giRecordCount].Properties[Value]
Package.Variables[User::giFileSize].Properties[Value]
Package.Variables[User::giBatchID].Properties[Value]
Package.Variables[User::gdFileDateCreated].Properties[Value]
Package.Connections[MyDatabase].Properties[ServerName]
Package.Connections[MyDatabase].Properties[InitialCatalog]
USE [ETLConfiguration]
GO
/****** Object: Table [dbo].[SSIS Configurations] Script Date: 05/23/2006 13:34:35 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[SSIS Configurations](
[ConfigurationFilter] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValue] [nvarchar](255) COLLATE Latin1_General_CI_AS NULL,
[PackagePath] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValueType] [nvarchar](20) COLLATE Latin1_General_CI_AS NOT NULL
) ON [PRIMARY]
Hello
I am trying to use Foreach loop container - Foreach File Enumerator
Is there easy way to retrieve enumerator configuration folder from variable - so I can easily move package from server to server?
I would like to have name of the folder we getting files from retrieved from variable by using this control
Thank you in advance
Armine Bell
Hi,
I am planning to develop a single package that will download files from ftp server, move the files to internal file server and upload it in the database. But I want to run this package for multiple ftp file providers. For each provider the ftp server might be different and the transformation to upload the files into a database table might be different.
So can I create a single package and then multiple configuration files (xml), which will contain the details fo the ftp file providers and then pass the xml file as a parameter while executing the package. The reason being that the timings of fetching the files is different for each ftp file provider and hence cannot be combined into one.
Is this possible?
Thanks for your help.
$wapnil
Hi guys,
Here's the deal. I have a child package, (say, pack01.dtsx), which uses a dtsconfig file for its connection string, which can be called from other packages, but which also can be called by itself.
However I also have another package (say, pack02.dtsx) which uses the same dtsconfig file for its connection string. It calls on pack01.dtsx.
When I use DTEXECUI and run pack01.dtsx, specifying the proper .dtsconfig file, it goes well. But when I try and run pack02.dtsx, an error occurs saying pack01.dtsx connection cannot be established.
How do I pass the connectionstring being used by pack02 to pack01, without having to remove the configuration file setting of pack01? Can a Parent Package configuration and a configuration file try and map to the same property?
Hope someone could help. Thanks.
Hey, I've a few jobs which call SSIS packages. If I run the SSIS package, it runs fine but if I try to run the job which calls this package, it fails. Can someone help me troubleshoot this issue? None of my jobs that call an SSIS package work. All of them fail.
Thank you
Tej
Hi Guys,
Here's the current scenario:
I am creating a windows application to manage my ssis packages.
My SSIS packages use package configurations, and by default connect to the "SSIS Configurations" table in sql server.
However, I want it to connect to a view instead of this table, so i create a view "vwSSIS_Configurations" and point the package configuration to use the view in sql server.
The application SHOULD be capable of altering the package configuration's configuration and switch its connection between the default "ssis configurations" table, and the view.
Is this possible? I cannot think of a way which I can have my application switch the package configuration's connection.
Thanks,
kervy
I have created a package configuration to read the connectionstring for the oledb connection manager.
Saved the file as Environment.dtsConfig in the same directory as the other packages.
Do the packages refer to the configuration file automatically?
Does it make a difference if I use the dtexec to start the ssis packages since there is now a configuration file?
Thanks
I would like to set package configuration.But dont know how to.
What i want to achieve is that i want to change connections based on test ,development and production environment of the same package...
How can i do that without me changing the connections manually to each task based on environment..
using package configuration i dont know how to...Please let me know.
Hello,
I'm developing some packages in SSIS and I need to implement a configuration file to load the server's connections and other parameters, so I can use this same connection for all packages.
Have anyone a step by step to perform this configuration ?
I exported a Connection Manager to a .dtsConfig file, but I don't know how to use this config file.
Any help is welcome !
Thanks,
Guber
Here is the scenario we are
trying to use; (it a bit long winded please bare with me)
There are 2 Packages Parent
and Child. Both have 2 Connection Managers (€śConfiguration€? & €śData Connection€?).
The €śData Connection€? has an invalid connection set to start with this is so
that we can deploy this to multiple places and use a SQL table to hold the
configuration.
At runtime €śConfiguration€? is
set to a valid SQL2005 database connection via the command line or in BI but €śData
Connection€? is left invalid.
The Package Configuration point
to an SQL server using €śConfiguration€? this has the connection string for the €śData
Connection€? Connection Manager (a valid connection string). The package errors
with can not acquire connection.
The first task is to put the
Connection string of the Connection manager €śConfiguration€? in to a variable.
Then the child package is called.
The Child package has the same
2 connections €śConfiguration€? & €śData Connection€? both are Invalid at this
time.
1) We use the Package
Configuration to populate a variable from the parent package which holds the €śConfiguration€?
connection string.
2) We use a Package
Configuration to populate then Configuration€? connection from the parent
package which holds the €śConfiguration€? connection.
3) We use a Package
Configuration to populate the €śData Connection€? Connection Manager using the €śConfiguration€?
connection. Which should be Set via step 2 here.
The First task on the child
package has a bit of script to check all the connection manager return a valid
connection to make sure database are up and running before connection. This
fails at the moment return in an error.
We want to use the same code
in multiple environments and as such don€™t want to make code changes that are really
only environment changes.
IE connection string to
servers. I can€™t seem to see anything saying we are doping anything wrong but if
the connections are all set to a valid connection before running this seems to
work. When invalid connection are used this seems to fall over.
I have created a package configuration to read the connectionstring for the oledb connection manager.
Saved the file as Environment.dtsConfig in the same directory as the other packages.
Do the packages refer to the configuration file automatically?
Does it make a difference if I use the dtexec to start the ssis packages since there is now a configuration file?
Thanks
Does it matter if the configuration is by xml,sql server,parent package variable,environment variable etc.
I use xml configuration which i kind of feel easy as i can change the connection directly by typing.Not used with others much..
Is there any security issue here .
Please let me know
Hi,
i am haveing package which is devloped in BIDS and i am executing the package from DotNet (Windows Application C#).during development my package is pointed to Server1,Database1.when we move the package to another test environment for a different server and database (Server2,Database2). we are giving the connection string details in dtsConfig file .but Package is not takeing the new connection string instead its takeing the old connection string (design time connection) and throwing error.what could be the problem
Invoking Code (C#)
--------------------
btn_SaExecute.Enabled = false;
string pkgLocation;
Package pkg;
Microsoft.SqlServer.Dts.Runtime.Application app;
DTSExecResult pkgResults;
pkgLocation = "D:JeganSSIS PrjsInvokeTestInvokeTestData1.dtsx";
app = new Microsoft.SqlServer.Dts.Runtime.Application();
pkg = app.LoadPackage(pkgLocation, null);
pkg.EnableConfigurations = true;
pkgResults = pkg.Execute();
Data.dtsConfig
----------------------
<?xml version="1.0"?><DTSConfiguration><DTSConfigurationHeading>
<DTSConfigurationFileInfo GeneratedBy="PSLSDCJegan.thangadurai" GeneratedFromPackageName="Data1" GeneratedFromPackageID="{30ACE86F-D0D3-4575-AD46-2E9C2F34D46B}" GeneratedDate="9/14/2006 12:54:01 PM"/>
</DTSConfigurationHeading>
<Configuration ConfiguredType="Property" Path="Package.Connections[XXXX.YYYY].Properties[ConnectionString]" ValueType="String">
<ConfiguredValue>Data Source=XXX;User ID=XXX;Password=XXX;Provider=MSDAORA.1;Persist Security Info=True;
</ConfiguredValue>
</Configuration>
</DTSConfiguration>
Thanks
Jegan
Hi all,
Hopefully a simple question. In my c# application i can call ImportConfigurationFile(<str>) to import an xml configuration file. How do i go about importing configuration settings from a SQL configuration package? I have created this but see no way to load it. I'd prefer to use the config file created in the SQL database.
Any help would be most appreciated.
Many thanks,
Grant
Hi all of us,
I'm looking any .dtsconfig file as sample for my first package configuration. I've got ssis packages which might be updated on development and then moved to production. First of all, I'd like dynamically change the connections but I don't know exactly how to do that. I'm taking a look on the wizard but when I choose a Sql Server request me a table.
Any sample or link for that would be welcomed!
TIA