DTS .ini File Versus SSIS XML Configuration - Guidance Needed.
Apr 25, 2008
Hello
I've been given the task of migrating a DTS package to SSIS (neither of which I am particularly familar with). The first job in the DTS package is to read a .ini file and set a bunch of variables. These variables are then used throughout the DTS package. After running the DTS package through the SSIS migration wizard this job turns into an execute script task and I can't see if it is still reading the .ini file. However, the only real purpose of this step is to allow different parameters to be passed in development, test, production etc. So I am thinking this whole step can be removed and effectively replaced with a package configuration (I'll probably use an XML file). My understanding is that by selecting the name/value pairs as appropriate in the XML package configuration file means this values will be passed in at runtime and achieve the same functionality. Is this the correct way to do this in SSIS, or do I still need the .ini file and variables?
Hello Dear Forum! I am new to this web. I am now days working with .NET Tecs especailly ASP.NET. I have developed a small school system in which i used SQL MDF file. I want to know about its approach if they want to make it online. and one more question, is MDF file is best for more than 10,000 Students? Can i convert the MDF into normal table format like SQL Server 2005 , i am using SQL Server Express. Take Care.Wating for reply. Developer
I have been working SQL server for many years as a web developer, however in my current position we are lacking an expert in maintenance and admninistration as the company has recently switched over to SQL Server 2005.
I therefore need to develop a greater understanding of the best practices with regards to administration and am a bit lost as there are so many options which seem like they might match my requirements. As I have never looked beyond the basic administration/maintenance side of things before I have no idea which options to investigate.
Basically I want to clear any records older than 3 months old on a scheduled/automatic basis. There seem to be various options that could do this, however I want to know what the best option is, rather than waste a lot of time and risk problems by resorting to trial and error.
We are already backing up this data, however I am not sure if I should combine these tasks. It is highly unlikely that we will ever have to restore these older records, however we do need to store the data for legal reasons.
I dont expect anyone to tell me everything, as I am happy to investigate, I just need a nudge in the right direction.
im trying to convert a sql stored procedure to a ssis project as part of a learning experience, however, im not sure where to even begin. im looking to see if someone can help my find my footstep.
basically, the sproc looks through a table and pulls two fields into a cursor, there is a third field, the where clause is if say that field is greater than 1.
basically the first field it pulls is the name of another sproc to run, and the this field says if i want to execute that sproc or not. so i want to rewrite this sproc in ssis to go through the table, and execute each sproc name based off of the first table.
my initial ssis thought was to add a for each loop, but i couldnt figure out how to go through the table. anyone offer some help on how this could be done? thanks
We are deploying our SSIS packages into different folders. For example: Test1 est2 production
Test1 points to the Test1 database, etc.
So, I configured my SSIS package to use a database connection called dbMAIN.
I then setup the SSIS File Configuration (XML) so that "dbMain" points to Test1. This xml file, called Global.dtsConfig, sits in C: est1, the same place as my ssis.dtsx file.
I want to copy my dtsx file and my dtsconfig file into est2 and production. The problem is that the the location of dtsconfig is HARDCODED in the ssis package!
How do I dynamically change it?
I tried to hardcode the location to ".Global.dtsconfig", but that did not work! Please advise!
In my project, developers enable configuration on the packages they develop. This means that if a project has 100 packages, each one has its own configuration file.
If our primary database server changes, I have to go an update 100 configuration files.
This defeats the purpose of having a configuration file in the first place.
I want to have a configuration file which contains common settings for all the packages in the project.
At the same time, I should have the flexibility of having individual configuration files for the project.
Thanks to some earlier help from this forum, I have my package running using configuration files and Windows Authentication. However, when I try to execute the package using Sql authentication, login fails for the user i am specifying. When I check out the configuration file, the connection string does not contain the password. I added the password to the connection string, but it does not cone back. I followed the instructions found on the following site:
I have an SSIS package (which is already delivered to the customer, so I can't change it), but I want to switch off (disable) one of the elements. I use a dtsConfig file (XML), so I want to just send an update to that config file to switch off the element.
So, I used the Package Configurations Organizer to generate a new config file with the element switched off, and it generated the following:-
<Configuration ConfiguredType="Property" Path="PackageRun Data LoadForeach Data FileLoad Invalid Format Recs into rej_dat_format_errors.Properties[Disable]" ValueType="Boolean"> <ConfiguredValue>0</ConfiguredValue> </Configuration>
So I copied these lines into my "production" config file and ran the package, but got the following warning message:-
Warning: 0x80012017 at ImportData: The package path referenced an object that cannot be found: "PackageRun Data LoadForeach Data FileLoad Invalid Format Recs into rej_dat_format_errors.Properties[Disable]". This occurs when an attempt is made to resolve a package path to an object that cannot be found.
Can anyone explain why? Is there some issue with having spaces in the Path name perhaps? Is there some syntax to get around it? Why did it generate a Path that is wrong? Thanks - AA
I'm new to SSIS (and quite new to SQL Server). I have a process which I'd like to automize via SSIS - just don't know how and couldn't figure it out yet by playing around with the program. Shouldn't be too difficult though.
First of all, that's the process as I do it now:
1) Load several flatfile sources (dumps of SQL tables) into an SQL database. 2) Add identifier rows (to some tables), set the primary and foreign keys so the database is "recreated" and I can work on it. 3) Do several simple transformations, aggregations and selects across tables and finally write a new table containing information for reporting stuff.
I succeded in loading flatfiles within the data flow view, doing some transformations and saving the output to a flatfile. What I didn't find out: how can I "recreate" the database enabling me to perform "SELECT/FROM/WHERE" statements across tables? Will I have to write the imported files to tables within a db (how?) or can I avoid this step?
A little guide (newbie friendly) would be great help!
Started: 10:48:56 AM Info: 2008-01-24 10:49:02.92 Code: 0x40016041 Source: SsisEndBatch Description: The package is attempting to configure from the XML file "C:Program Files (x86)Microsoft SQL Server90DTSPackagesBridgeNet.BI.SSIS.EndBatc ENDBATCH TEST ENV.dtsConfig". End Info Warning: 2008-01-24 10:49:03.15 Code: 0x80012011 Source: SsisEndBatch Description: Cannot load the XML configuration file. The XML configuration file may be malformed or not valid. End Warning
Hi, the message below is generated when I try to generate an SSIS configuraiton file. The link does not provide any help. Do you have any suggestion what could cause this error?
Thanks, Piet
==========
TITLE: Microsoft Visual Studio ------------------------------
Could not generate the configuration file. (Microsoft.DataTransformationServices.Wizards)
For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%u00ae+Visual+Studio%u00ae+2005&ProdVer=8.0.50727.42&EvtSrc=Microsoft.DataTransformationServices.Wizards.ConfigurationWizardPages.ConfigurationWizardSR&EvtID=CouldNotGenerateConfigurationFile&LinkId=20476
Hi, I am using custom dll in script component in SSIS package. This dll is looking for some configuration settings and dsplays the message as "Configuration section could not be found in the configuration source" . Please tell me the configuration source it looks for.
I wonder if anyone else out there has the same impression that I have: I find that DTS works much better than SSIS.
I find that DTS is so easy to use and reliable: it gets the job done and fast! On the other hand, SSIS seems to be so needlessly complex that it takes hours of troubleshooting just to get it to work, and sometimes it doesn't work at all. For example I have just spent hours trying to get SSIS to import a flat file with 300,000 rows. It just crashes and doesn't even give an error message so that one can fix it. On the other hand I have just now successfully accomplished the same task with DTS and it took me 5 minutes!
I honestly don't see a valid reason for using SQL Server 2005 instead of 2000. So far it's much more productive to use 2000.
This question probably overlaps a few different topic areas.
As I will be required to work with both Oracle and SQL Server I will be in a difficult position with SSIS(due to it's change in distribution).
Therefore I am having to look at alternatives.
With coding a can open a text file and parse it reasonably to my satisfaction. However getting the data into the database is incredibly slow.
I am using an Insert into for each line, which I am sure everone will shake their head over. This seems to be pretty slow even using transactions.
Is there any scope in using data tables or have the read on one thread and write on another.
Other than that is there an Oracle equivalent of SSIS which comes (probably get shot for asking that on a microsoft web site, but would probably get shot if I asked on Oracle forums as well).
In the past we had reasonable results in outputting to csv and then doing some sort of bulk insert, messy and irritating though this may be.
Any ideas on this area will be gratefully accepted.
I installed SQL Server 2005 Enterprise Edition (64-bit) and then applied SQL SP2. When I launch the "SQL Server 2005 Surface Area Configuration" and choose "Surface Area Configuration for Features" I'm trying to enable xp_cmdshell.
After ticking the checkbox, and clicking Apply, I receive the error below. Anyone have any ideas on how to fix this?
===================================
Alter failed. (Microsoft.SqlServer.Smo)
------------------------------ For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=9.00.3042.00&EvtSrc=Microsoft.SqlServer.Management.Smo.ExceptionTemplates.FailedOperationExceptionText&EvtID=Alter+Configuration&LinkId=20476
------------------------------ Program Location:
at Microsoft.SqlServer.Management.Smo.ConfigurationBase.Alter(Boolean overrideValueChecking) at Microsoft.SqlSac.Public.Smo.SetSetting(Credentials credentials, DatabaseFeature feature, Int32 value) at Microsoft.SqlSac.MainPanel.UserControlSSxp_cmdshell.ProcessOK(HashKey givenKey, String machineName) at Microsoft.SqlSac.MainPanel.FormFeatures.commitDBChanges() at Microsoft.SqlSac.MainPanel.FormFeatures.commitChanges()
===================================
Could not load file or assembly 'Microsoft.SqlServer.BatchParser, Version=9.0.242.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified. (Microsoft.SqlServer.ConnectionInfo)
------------------------------ Program Location:
at Microsoft.SqlServer.Management.Common.ServerConnection.GetStatements(String query, ExecutionTypes executionType, Int32& statementsToReverse) at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteNonQuery(String sqlCommand, ExecutionTypes executionType) at Microsoft.SqlServer.Management.Smo.ExecutionManager.ExecuteNonQuery(String cmd) at Microsoft.SqlServer.Management.Smo.ConfigurationBase.DoAlter(Boolean overrideValueChecking) at Microsoft.SqlServer.Management.Smo.ConfigurationBase.Alter(Boolean overrideValueChecking)
The SSIS package will need to accomplish the following:
1. The package will somehow need to know what the file name should be based on the last file processed. So for example if we just loaded the Jan 2015 file, the next monthly file to drop should be the Feb 2015 file. For the quarterly files, it should be Q1 2015, then Q2 2015, etc...
2. Based on the file (Monthly or Quarterly), the package needs to somehow split and process one way for Monthly and another way for Quarterly.
I have a stored proc that is returning the results I need for output to .txt file.
Is there a way in SSIS to commit 50K (or whatever number) row batches at a time or should I just handle this in the stored proc?
select * into #TempTable from SomeTable [WHILE LOOP] --throttle commit batches of 50K rowcount select * from #TempTable [END LOOP] drop table #TempTable
But If I'm doing this in SSIS, I can't drop the #temp table otherwise I have nothing to output right?
I cannot get the log file path read from the configuration.
If the path in the Connection Manager is invalid, package throws an error "SSIS logging provider has failed to open the log" instead of reading it from the config.file. What am I doing wrong?
Here is the portion of the config file. Everthing else is read from the config file correctly.
My requirement is to have the connection strings for the source and destination to be a parameter for the packages that way they can be changed from the sql job scheduler.
I'm trying to develop an application and I have been told right here in sql server getting started forum that SSIS or DTS are the best bits.
My old post has these questions:
Code Snippet
I'm making an application. I have 2 different sites running Sql Server 2005 workgroup and sql server express.
I have a 3rd main server and what I would need to do is to get some of the data from both of these sites, merge it and save it on Main server and then generate another file which should be placed in a directory on a network.
I don't have any idea of how I would do that. Its entirely a sql server app.
One person told me its an SSIS and the other one said to investigate replication.
I have checked a bit about both. I want to give it a try.
Where can i find tutorials on both of these. Complete tutorials to develop and design it.
Hi, I'm facing a problem that I'm trying to solve. I report it in the following hoping somone can help me!
the context:
- I have a solution containing almost 20 packages - I have a main package containing the sequences (Sequence Container) calling all others packages (Execute Package Task) - every package has the connections to its own DBsource ; so the package X has the its own connections to DBsourceA and DBsourceB, the package Y has the its own connections to DBsourceB, DBsourceC, DBsourceD and the destination DB is unique for all the packages.
My problem is: I have the necessity to define in the best way a config (one for all the dtsx, possibly) that allow me to manage easily the switch from the developing env to the deployment env, basically for the DB connections.
Which is the best way to do this and can anyone tell me the steps to follow?
I have finished the development of an SSIS package but I would like to make it a bit more flexible by adding a configuration file. The script migrates a database from a source to a destination and syncs with another source.
So there is the source db and the destination db. However, I have noticed that standard SSIS configurations can get a little tricky. For instance, I have two connection managers for the destination db (one that points to the schema and one that doesnt). So my question is: Is there any way to create a config file thta you can input a few attributes like sourceDB, destinationDB, schemaName and get the package itself to parse and process it?
I'm facing a little problem concerning package configuration that is using a SQL table.
I have 4 exactly the same Db's on the same SQL server, each with another name of course (Dbipa, DbIpB, DbIpC & DbIpD).
My developers created 4 exactly the same SSIS packages with 4 exactly the same jobs, each for his own DB only the connection settings are different.
Now I was trying to manage this with only one SSIS package by using an SSIS Package configuration table in each DB.
I thought it would be possible to start the package with the following parameter
/SET Package.Connections[SQL].InitialCatalog;DBIPA OR
/SET Package.Connections[SQL].InitialCatalog;DBIPB OR
/SET Package.Connections[SQL].InitialCatalog;DBIPC OR
/SET Package.Connections[SQL].InitialCatalog;DBIPD But this doesn't work because before he changes the initial catalog in the connection named SQL, he already did the loading of the SSIS_Configuration table. So he point always to the same config table. Is there a way to change the connection before the package configuration is being executed. Ludo Bernaerts
1) We are doing data migration from SQL Server 2000 OLTP design to SQL Server 2005 OLAP design.
2) We have used SSIS packages and data flow tasks in which we mentioned connection strings for source and target containers.
3) We have a master execute package which contains series of execute packages in relational order.
4) When we execute this master package, we should be able to dynamically specify different source and connection strings for all packages.
5) In master execute package, we generate an XML configuration file using SSIS -> Package Configurations... using Connection Managers -> Particular connection
6) Now, we change connection string to point to another database after adding this new OLE DB connection in Connection Manager of each and every package.
7) When we save, build and execute master package it is still executing migration packages against the old database. Please let me know what I am missing.
I need to configure my package using SSIS Configurations. When I opened SSIS Configuration wizard, press ADD and selected Xml Configuration, I found two options.
One being : Specify configurations directly.
next : config location is stored in env variable.
How to use the second option?[ env variable]. Say I want to configure my connection string for my destination database [ OLEDB Destination Component]. How to do that using second option.
I am having a bit of trouble with SSIS Configurations.
- In BIDS, I have added a configuration file and specified the properties I want to expose. - When I build the project, I get the standard bindeployment folder which contains the package file (.dtsx), the configuration file (.dtsconfig) and the deploymentmanifest. - Before deploying the package, I edit the config file to have the settings I want it to for the environment I am deploying to. - The package deploys OK - When I work directly on the SSIS server (64 bit), I can go into SQl Mgt Studio, choose 'run package' and when I look in the connection manager window all the settings are as I desire ( I havent had to add a configuration file manually) - When I work on a client machine, I connect to the SSIS server and choose 'run package' - the properties/connections are the same as on the server but the values themselves are completely different.
Why is this? Why when I run the package from the client (32 bit) do the configuration values appear to be completely different? How can I run the package remotely and pick up the configuration values that I deployed with? Or have I misunderstood this whole configuration function?
I am new to SSIS. I have an SSIS package which we are storing it in the application server ( not in SQL Server ). We are storing the database name etc.. in package configuration. Where should I store the configuration file? In app server or in the database server. Also please tell how to execute the package stored as file system.
Hi, I have an SSIS Package which reads the properties values from Registry. As per the constraint of SSIS I have created the keys under HKCU in the registry and things work fine from the designer.
But, when I schedule the SSIS Package to run under SQL Agent (which is running under Local System) the values of configuration do not seem to be picked up from Registry.
How I tested was: I am picking up the server name for the Connection Manager from Registry and I set that value to an invalid servername. But, the job seems to be executing fine. Using DTEXEC it fails as expected since, this is running under my credentials.
Also, SYSTEM has full control on the new regsitry keys. Any information on this?
I want to store each SQL Server's (prod, test, etc) connection string in XML to make deployment & configuration easy. However, the connection string must be encrypted. Does anyone have any ideas or suggestions on how to accomplish both. I know how to encrypt it but I dont think SSIS can work with it.
