Setting The OLEDB Connection Properties At Runtime
Jan 30, 2007
I used to do this in DTS but wondered how to do it in SSIS. I have a For Each ADO recordset loop that reads these four columns into variables:
DataSource Name, Name(for a WHERE clause), user and password.
For each "Building" there is a new server to connect to, with identical tables. I have 10 tables in the loop to pump from one database to a SQL 2005 database (i.e. 10 dataflows). I want to set the source connection information with the variables. My questions:
1. How do I set the connection information for the source connections? Do I just set the properties for the connection manager and then it sets for all the sources? What property do I set--do I need a script task?
2. How do I use the values from the "Name" column in a WHERE clause in each data flow?
I am trying to set the connection string in a connection manager at runtime. Here is what I have done:
1. Created a gv_DataSource, gv_Username and gv_Password
2. Created a ForEach Loop that reads DataSource, Username and password values from a variable (it is an For Each ADO loop Enumerator). The ADO recordset is read into by an Execute SQL task before the loop.
3. Mapped values from the recordset to variable in the ForEach loop's "Variable Mappings" page.
4. Used the variables in my Sybase OLEDB Connection Manager's "Expression" property, setting the "ConnectionString" property to:
5. I set the values in my database table for the connection-I set 2 connections for which I have Sybase OLEDB datasources setup.
When I run the package, I just get the first server's data twice, it doesn't set the second server's data during the second loop. I made sure the first one was working (i.e. the ConnectionString's property was being set by the data from the current variables) by setting the variables incorrectly in the variable properties page, and then running the package. So the first row of connection information is working, but the second loop around it doesn't seem to be working. I used a msgbox in a script task to show that the variables are mapping correctly in the loop, so it seems the second time around the connection information isn't taking from the variables.
I'm programmatically loading a package that was created with VS 2005. The last task in the package is a Send Mail Task. It has two properties, FileAttachments and ToLine, which are set to expressions whose values come directly from package variables. The package runs in debug and non-debug (under VS) correctly.
Here is the code I am trying to use to set a update parameter at runtime. (Depending on what linkbutton a user clicks on the STATUS_ID value will change.) SqlDataSource1.UpdateParameters("STATUS_ID").DefaultValue = 33332Here are my parameters: <UpdateParameters> <asp:Parameter Name="CUSTOMER_ID" Type="Decimal" /> <asp:Parameter Name="RECEIVED_BY" Type="String" /> <asp:Parameter Name="CALL_DATETIME" Type="DateTime" /> <asp:Parameter Name="AREA_ID" Type="Decimal" /> <asp:Parameter Name="CLASS_ID" Type="Decimal" /> <asp:Parameter Name="STATUS_ID" Type="Decimal" /></UpdateParameters>Sqldatasource1 is the name of my datasource control. Any thoughts?
I don't know if this topic should go in this forum category or in the SQLDataSource forum. I need to be able to provide SQL which varies based on what the user enters on a page. However, this isn't as simple a providing parameters in the SQLDataSource control because the SQL changes depending on what the user selects. For example, in a customer search page, they choose to search by either first name or last name from a drop down list and then enter the name to search for in a text box. The dataview should display the matching customers. How can I set this SQL based on the entered values while also keeping the paging and sorting capabilities of the dataview? You don't seem to be able to set the SQL for an SQLDataSource at run time.
This may be a very easy question to answer, but I cannot figure out the answer for the life of me. I have set up a simple SSIS project based on a template that gathers information from several servers. I would like for this package to be almost completely automated and run nightly. Is there a setting I can add to make it run everyday?
Is it possible to establish a set of default report and report body properties, perhaps in an ini file of some sort so that upon creating a new project and adding a new report item the developer does not have to go through the process of setting up all the interactive sizes, margins, paper sizes, etc each and every time? We have an established header that we incorporate on all reports and while in transition to the new Reporting Services it was my hope of finding something will create the first set of default conditions to help us automate the process just a bit.
I want to input a parameter to my package. I have a variable, and I tried changing its value from DTExecUI.
I did everything as mentioned in http://blogs.conchango.com/jamiethomson/archive/2007/03/13/SSIS_3A00_-Property-Paths-syntax.aspx but I still get an error -
Could not set "Package.Variables[User::var_input_step_Name].Properties[Value]" value to "RAW".
The variable was originally read from a XML config file. I stored the package to MSDB. I then ran dtexecui, selected my msdb package, and in Set Values, I entered property path and value. I also tried removing the "", and I even deleted the XML configuration. Still the same error.
What am I doing wrong? Any Package Settings to make?
I have a package that uses a for loop to iterate through an unknown amount of excel files and pull their data into a table. However, there will be cases when the file is corrupted or has some sort of problem so that either the transformation will fail or the excel data source will fail with an oledb connection error. Could anyone suggest a clean way to trap these errors? Specifically, the "Cannot Aquire Connection from Connection Manager", which is the excel connection.
I use SQL Server 2012 and visual studio 2010.I created SSIS Project with task "Execute Package". Control flow view as: Package1 (execute package) -> Package2 (data flow).Data flow in Package2 view as: ADO.NET source -> ADO.NET destination.
When I started Package2, it's work. I havn't errors.But when I started Package 1 I have error "Unable to get managed connection from the Connection Manager runtime". In execution log I see that ADO.NET source produced this error on verification stage. Package failed on verification stage, not on execution stage.Why when I started Package 2 it work, but when I started Package1 (and Package1 started Package2) it failed?
I have a package which contains a foreach container. Can anyone help me in setting the properties for the enumerators programmatically??? I am trying to set the properties for the enumerator "ForEach File Enumerator"
I have a need to access random entries in an IRowset object. Looking through the OLEDB documentation there seems to be several options:
1. Just use GetNextRows with positive or negative offset values.. 2. Use the IRowsetLocate :: GetRowsAt method 3. Use IRowsetIndex :: Seek to set the cursor 4. Use IRowsetBookmark :: PositionOnBookmark
Which of these are supported and/or recommended for SSCE?
Just to give you an idea of what I'm going for, the application will look something like the following on my own recordset classes:
This is the first time I have used SSIS, so please bear with the ignorance.
I have a super simple package that inserts x000's of rows into a temporary table. The data source is a file that the user will upload. I need to be able to tell the package what file to upload. I'm thinking the simplest thing would be to edit the connectionString property of the SourceConnectionFlatFile at runtime. Is this possible? What form should the file path be in (UNC, other)? And, are there any other considerations I should be aware of?
1. Deletes a SQL Server database. 2. Recreates it with the tables ready to take data. 3. Executes a Flow Task that puts the data into the tables.
So far so good. The problem I have is that everything is loaded correctly as long as the tables are in place before I try to execute the package. When the first step deletes the database I get an error because the connection manager cannot find the tables that are defined in the database that has been removed and the package seems to disregard the fact that before the data gets loaded the whole structure will already be in place. What should I do to convince the package that the connection manager will have the target database set up when the data needs loading?
I have SSIS packages that send success/failure email upon completion, and I'd like to add a note that identifies the server and database used. I can certainly add variables to the packages and use them when constructing the email, but I'd prefer to get the information directly from the OLE DB connection itself. Is there a way to access the connection string from within a control-flow VB script task? Furthermore, can I get the data source and initial catalog from that connection string, or do I need to parse it myself?
I'm trying to get a better idea of how I should set some of the SSCE connection properties. I will be deploying the application on a WinCE5 board with a 4G flash storage card and 512MB RAM. The application must run 24/7 with a medium amount of traffic. At any given point I expect up to 3 connections to the same database. I'd really like to keep performance high without risking any database corruption issues. As such, I need to make sure my connection properties are optimized.
I read from Joao's article that setting DBPROP_SSCE_MAXBUFFERSIZE = 1024 gave a considerable performance increase, while anything higher gave diminishing returns. Is this for a certain amount of RAM, or is it uncorrelated to RAM size?
What's the best practice for specifying DBPROP_SSCE_TEMPFILE_DIRECTORY and DBPROP_SSCE_TEMPFILE_MAX_SIZE? Is it possible in WinCE5 to directly put it in RAM? How big do these temp files get?
I've read of some bad experiences with autoshrink. Are there any reasons not to set DBPROP_SSCE_AUTO_SHRINK_THRESHOLD to 100 if the database will be compacted regularly?
Any other advice for the more obscure settings such as DBPROP_SSCE_FLUSH_INTERVAL or all the lock settings?
How can i change the connection string at runtime?
Scenario is, I want my customer to enter server name, database name, and some parameters at the installation time then i will use these entries to build the connection string and persist it into the app.config.
also not only at installation time but also at runtime while the program is running. i wanna make these entries as options which can be modified at any time
I want to transfer data from one server to another by using SSIS. i want the connection string to be dynamic and also according to the some other variable, the transforming data is changing.
Could you provide me the solution thet how i am able to change my connetction string dynamically and the other variable too.
i am using VS 2003 as front end and SQL server 2005 as a backhand.
Due to VS.NET 2003 i am able to create DTS packages but i have to migrate it and then anly i am able to use it in JOB in SQL server agent of SQL server 2005.
is that any code or any stored procedure from which i am able to migrate DTS packages to SSIS packages.
Hi all. Sorry about bothering you again but I have no choice, I'm afraid. This time I have to add a connection manager at runtime. I have a script that examines rows in a huuuuuuuuuuuuge file and it turns out that I will have to redirect different rows into 95 different text files. I would like to avoid the pain of creating the files and connection managers manually. I have a source script component that runs asynchronously (the only way basically it can run as a source) and I wonder whether this time, when I change the connction string on the manager at runtime inside the script, it would run properly. Can anybode tell me, please?
I designed a DTS package with eleven different connections. When I try to see the proberties of a connection the enterprise manager immediatly closes without giving me a warning. This happens only with two connections (in my example M1 and M2). This mistake has no influence to the execution of the package. In my opinion it is a fault of the DTS package desinger but may be that I made something wrong. Is there anybody who knows this mistake? Thanks for your help.
I am working on a SSIS package which is using a Script task, now I have all the connection properties set up in the .NET script using connection strings, what do I need to do if I have to set this up using a config file or something else which is more secure (I dont want to leave the connection information in the script). Please Advice.
My package needs to be a High Performance (target: 150,000 rows in 30 minutes) ETL solution. We are using all MS technologies - SSIS, SQL 2005, BIDS, etc. I need to loop the recordset executed by a Stored Proc in a Execute SQL Task in a Source Script Component.
If I use an ADO.NET Connection Manager, here is the code in the Source Script Component Public Overrides Sub CreateNewOutputRows()
Code 1
Dim sqlAdapter As New SqlDataAdapter
Dim dataRow As Data.DataRow
Dim ds As DataSet = CType(Me.Variables.rsSomeResultset, DataSet)
sqlAdapter.Fill(ds)
Iget: Error: System.InvalidCastException: Unable to cast object of type 'System.Object' to type 'System.Data.DataSet'.
Code 2
Dim oledbAdapter As New OleDb.OleDbDataAdapter Dim dataTable As DataTable
Error: System.ArgumentException: Object is not an ADODB.RecordSet or an ADODB.Record. Parameter name: adodb
It works all right when I use an OLEDB Connection Manager with the second code sample.
Question: In order to extract the maximum performance, wouldn't it be preferred to use ADO.NET with SqlClient Provider in an all SQL Server 2005 environment? Or will an OLEDB Connection provide comparable or equal performance?
If so, what code can I use? Since the recordset returned by the Stored Proc (in the Execute SQL Task) can only be captured in a System.Object variable and you can only use the overload of the Fill() method of the OleDbDataAdapter to accept an ADO Recordset or Record object.
Are there compelling reasons to use one versus the other? I am completely redeveloping all of our site from .ASP, VBScript and Access, to ASP.NET, VB.NET and SQL Server 2005 Express. I have a lot of code in ADO and OLEDB already, and would like to make the change to the System.Data.SQLClient classes if I should.
I have tried to connect ce database by using c++ and oledb, and failed in every turn, i dont know what i did wrong...
I created a new database by using create a new database sample in ssce bol, but i couldnt connect that database, it says invalid class string or authentication failed...
I know this has been a topic of discussion before and I have read and tried the suggestions to no avail. I have a few .dbf files to import into my sql database and I know that one needs to make a oledb connection to these files.
I am using the Jet provider and have tried setting the extended properties to all of the DBASE versions and still cannot get it to connect. These files' names are shorter than 8 chars so that issue is no problem in this case. I can actually get these files to show in a Microsoft Access 2008 database when I import it from there so I know that I must be missing something in my SSIS project.
After doing some reading on this topic, it would seem that this shouldn't be such a schlepp but, here I am
Hi - Im migrating packages in dts 2000 to SSIS 2005 using package migrator wizard. It loads a CSV to a table in DB. I need to change the server and file location paths after migration. While changing this, im encountering error - "the acquireconnection method call to connectionmanager failed
At one point in my package I call a simple web service that kicks off a process on a remote server using a Web Service Task . I need to be able to point the task at a test instance while I'm testing and a different one when it goes to production. The simple answer (you'd hope anyway) is to just store the ConnectionString for the HTTP Connection Manger in Package Configurations for TEST and PROD. I did this and when the package runs, the connection manager's connectionstring does get changed, but the web service task doesn't use this value. It uses the value defined in the WSDL file which in my case contains the URL for the TEST instance.
Am I missing something obvious? Do I really have to get the WSDLs from PROD and TEST and deploy both of them to the appropriate environment? What does the ConnectionString in the HTTP Connection Manager actually do then?
I have been working with flat file connection managers for a while now, and I have found it very useful to set properties on all types of SSIS objects using expressions that are set to SSIS package variables.
We have recently moved towards using a centralized application configuration infrastructure, so we are now loading up all application configuration information from our own system -- it's actually based off of the 2.0 ConfigurationManager.
I have developed a custom control flow task that will retrieve all of the SSIS variable information from our own application configuration system and set all of the SSIS variables as the first step in a package.
When I have been using this to set the SSIS Variables which are used in expressions for the HeaderRowsToSkip and DataRowsToSkip (there may be others, but I have not tested them yet) properties of flat file connection managers, these values are not being used -- instead it is using the values for the variables that are defined in the package itself (i.e. the default values that I set up in BIDS when developing the package). The ConnectionString property on the flat file connection manager does not exhibit this behavior.
Can anyone provide any insight on this? Is this a bug in SSIS, or is there something that I am missing here?
