My package needs to be a High Performance (target: 150,000 rows in 30 minutes) ETL solution. We are using all MS technologies - SSIS, SQL 2005, BIDS, etc. I need to loop the recordset executed by a Stored Proc in a Execute SQL Task in a Source Script Component.
If I use an ADO.NET Connection Manager, here is the code in the Source Script Component Public Overrides Sub CreateNewOutputRows()
Code 1
Dim sqlAdapter As New SqlDataAdapter
Dim dataRow As Data.DataRow
Dim ds As DataSet = CType(Me.Variables.rsSomeResultset, DataSet)
sqlAdapter.Fill(ds)
Iget: Error: System.InvalidCastException: Unable to cast object of type 'System.Object' to type 'System.Data.DataSet'.
Code 2
Dim oledbAdapter As New OleDb.OleDbDataAdapter
Dim dataTable As DataTable
Error: System.ArgumentException: Object is not an ADODB.RecordSet or an ADODB.Record. Parameter name: adodb
It works all right when I use an OLEDB Connection Manager with the second code sample.
Question: In order to extract the maximum performance, wouldn't it be preferred to use ADO.NET with SqlClient Provider in an all SQL Server 2005 environment? Or will an OLEDB Connection provide comparable or equal performance?
If so, what code can I use? Since the recordset returned by the Stored Proc (in the Execute SQL Task) can only be captured in a System.Object variable and you can only use the overload of the Fill() method of the OleDbDataAdapter to accept an ADO Recordset or Record object.
set rsSearch = Server.CreateObject("ADODB.Recordset") rsSearch.CursorLocation = adUseClient rsSearch.LockType = adLockBatchOptimistic
sSql = "some sql stmt" 'this works fine on its own
rsSearch.Open sSql, cnTraining
set rsSearch.ActiveConnection = nothing
cnTraining.Close set cnTraining = nothing
%>
My goal is to get a disconnected recordset. The problem here occurs when i try to use the adUseClient value (3) for the CursorLocation. If I don't use a 3 in the CursorLocation it works fine. Hoewever i'm almost certain i have to use the 3 in order to disconnect the recordset. Any ideas? Is my connection string not set up properly to get a disconnected recordset?
I have a package that uses a for loop to iterate through an unknown amount of excel files and pull their data into a table. However, there will be cases when the file is corrupted or has some sort of problem so that either the transformation will fail or the excel data source will fail with an oledb connection error. Could anyone suggest a clean way to trap these errors? Specifically, the "Cannot Aquire Connection from Connection Manager", which is the excel connection.
pCadors->Open (_T"select foo from bar", pCadoCon->m_lpDispatch, eCursor); //SQL: a select statement here
The CADOConnection and CADORs are ADO connection, RecordSet wrapper classes created with ClassWizard.
I successfully open an ADO connection using the OEM driver by this DSN less way, and then passed it to ADO Recordset. From ODBC trace, I found that under the cover of the ADO, this connection is used to execute the select statement, but after that it automatically opens an other connection. I do not know why and is it possible to disable this?
I have recently moved over from SQL Server 2000 to 2005 and am now having an issue with my application with what appears to be, the number of recordsets that I can open/close on a single connection.
Here is a snippet of what I'm doing, in VB 2005 using ADO... (assuming the connection is already open and working)
Dim RS1 As RecordSet.
DIM RS2 As RecordSet
RS1.Open("SELECT...")
Do While Not RS1.EOF
count = count + 1
Console.Writeline(count)
RS2.Open("SELECT...")
..some processing...
RS2.Close
Loop
RS2 = Nothing
RS1.Close
RS1 = Nothing
Now as I said this all works fine when I connect to an SQL 2000 server but on SQL 2005 it bombs out when 'count' is approximately 1940 with an exception saying that the login failed. If I have Server Management Studio open, that connection will then freeze and throw up an error about how only one usage of each socket address is normally permitted - I think thats more a red herring though.
Any ideas? I've been through all the server settings and can not seem to find anything about recordset limits or timeouts. The only way I've been able to get around this problem at present is to open a new connection object for each iteration to be used by RS2.
Thanks everyone. Any pointers would be much appreciated.
I used adodb connection and recordset in script task. but i have an error saying adodb is not defined. how do i add it to reference? or, is adodb can run in script task or only ado.net?
Are there compelling reasons to use one versus the other? I am completely redeveloping all of our site from .ASP, VBScript and Access, to ASP.NET, VB.NET and SQL Server 2005 Express. I have a lot of code in ADO and OLEDB already, and would like to make the change to the System.Data.SQLClient classes if I should.
I have tried to connect ce database by using c++ and oledb, and failed in every turn, i dont know what i did wrong...
I created a new database by using create a new database sample in ssce bol, but i couldnt connect that database, it says invalid class string or authentication failed...
I know this has been a topic of discussion before and I have read and tried the suggestions to no avail. I have a few .dbf files to import into my sql database and I know that one needs to make a oledb connection to these files.
I am using the Jet provider and have tried setting the extended properties to all of the DBASE versions and still cannot get it to connect. These files' names are shorter than 8 chars so that issue is no problem in this case. I can actually get these files to show in a Microsoft Access 2008 database when I import it from there so I know that I must be missing something in my SSIS project.
After doing some reading on this topic, it would seem that this shouldn't be such a schlepp but, here I am
I need to conect my software from a remote client to a server, mystandard connection:Provider=SQLOLEDB.1;Persist Security Info=False;UserID=sa;Password=;Initial Catalog=MyDb;DataSource=192.168.0.100;Network=DBMSSOCN;andProvider=SQLOLEDB.1;Persist Security Info=False;UserID=sa;Password=;Initial Catalog=MyDb;DataSource=192.168.0.100;Network=DBNMPNTW;make a error connection,i have tryed with ODBC connection and its OK only in Named Piped.Can any one help me write the right connection string?thanksrm
Hi I am using OLEDB Connection in script component. I get a login failure when I run the package. I use SQL Server authentication. Do you have to use windows authentication in script component? Thanks
An application vendor is telling me that their product will not run with SQL Server 2000 64-bit. Here is the layout: * Dedicated 64-bit machine running Windows Server 2003 Enterprise, 64-bit. * Separate application server running Windows Server 2003 - SBS. Application server uses DCOM and OLEDB to obtain data from SQL Server.
My question: Is there any way for the application server to know if the data supplier is running 32-bit or 64-bit? If so, then perhaps the application vendor is right. If not, then they are just trying to put me off and avoid fixing a problem. In this case, I need an authoritative statement as ammunition to take back to their support department. .... Warren
Im trying to connecting from SQL 2005 (win 64bit server) to a Oracle database via OLEDB but get the following message. The OraOLEDB.Oracle.1 provider is not registered on the local machine.
If we try ODBC we get the following message "error in initializing provider. Attemt to load Oracle client libraries threw BadImageFormatException.This problem will occur when running in 64bit mode whit the 32 bit Oracle Client components installed."
I have a big problem. Im writing an application in Sql 2005 Etl. This application use 2 kind of ole db connection. One towards a Sql Server, and the other towards As400 to use ole db towords As400 is necessary to save user and password to connect. There is a flag to save password.........and obviously a checked it. !!!!!
The problem is: If I stay inside Visual studio and i use this application.......all is correct. If I use my application in sql like a job.........loose password information and crash immediatly
I used to do this in DTS but wondered how to do it in SSIS. I have a For Each ADO recordset loop that reads these four columns into variables:
DataSource Name, Name(for a WHERE clause), user and password.
For each "Building" there is a new server to connect to, with identical tables. I have 10 tables in the loop to pump from one database to a SQL 2005 database (i.e. 10 dataflows). I want to set the source connection information with the variables. My questions:
1. How do I set the connection information for the source connections? Do I just set the properties for the connection manager and then it sets for all the sources? What property do I set--do I need a script task?
2. How do I use the values from the "Name" column in a WHERE clause in each data flow?
Hi, I am trying to Create an OLEDB Connection from SSIS to Inforrmix without any success. Would someone be able to help.We are running Informix version 9.4. We were able to create an OLEDB connection with SQL server 2000 and DTS. But, when trying to create an OLEDB connection using SQL server 2005 and SSIS, I am getting an error message. The connection is created successfully but cannot access databases and tables on informix. The error message when trying to load the Informix tables: 'Could not retrieve table information for the connection manager .... 'Ifxoledbc.e' failed with no error message available '
1. If I have millions of rows to be compared, then which cache type is prefereed for lookup, Partial or no caching?
2. If I have lookup connected to Oracle Oledb, cache type as partial and SqlCommandParam as following
select * from (SELECT ORDER_ID, OPER_KEY, STEP_KEY, SUM(OCCUR_COUNT) AS OCCUR_COUNT FROM SFWID_OPER_DESC_EXPLD GROUP BY ORDER_ID, OPER_KEY, STEP_KEY) refTable where refTable.ORDER_ID = ? and refTable.OPER_KEY = ? and refTable.STEP_KEY = ?
then it doesn't allow me to add the parameters from Advance tab of Lookup transformation edition, and raises following error
"Provider cannot derive parameter information and SetParametInfo has not been called"
When I configure and test and OLEDB connection SQL Server 2005 keeps stopping with an error message as shown below. The problem seems to happen when I open the connection for ediing in Data Flow Window. There is a consistent pattern of errors (in there Event Log) relating to the Evnet Log prior to this error occurring.
Any idea what the problem is?
I have to say that my experience with SSIS has not been very encouraging so far.
Event Type: Error Event Source: MSSQL$SQL2005 Event Category: (2) Event ID: 17311 Date: 23/01/2007 Time: 2:06:41 PM Description: SQL Server is terminating because of fatal exception c0000005. This error may be caused by an unhandled Win32 or C++ exception, or by an access violation encountered during exception handling. Check the SQL error log for any related stack dumps or messages. This exception forces SQL Server to shutdown. To recover from this error, restart the server (unless SQLAgent is configured to auto restart).
I have an application wrote in Power Builder 11 and use OLEDB connection for SQL SERVER 2005 DB. it works excellent till multiple users access the application. 2 seconds passes and locks appears and blocks all the DB! I realy don't know what is the reason. In past I used SQL Native driver and it works well with the same application!!! (other reasons caused me to change the connection. not this). Does someone can help me????????
I have OLE DB Connections set up in my connection manager (Native OLE DBMicrosoft OLE DB Provider for SQL Server). I would like to reference and query these connections from a custom task, written in C#. I currently reference it as follows:
What may be obvious to some (though wasn't to me, as I am new at this), when I run the task, I get an error saying that I cannot make this cast. After perusing the boards, I understand that this is because I am not making a cast to the right connection type. Well, that is where I am lost. What connection type (and corresponding library) do I need to reference? I want to continue to use the "Native OLE DB..." connection.
I am trying to set the connection string in a connection manager at runtime. Here is what I have done:
1. Created a gv_DataSource, gv_Username and gv_Password
2. Created a ForEach Loop that reads DataSource, Username and password values from a variable (it is an For Each ADO loop Enumerator). The ADO recordset is read into by an Execute SQL task before the loop.
3. Mapped values from the recordset to variable in the ForEach loop's "Variable Mappings" page.
4. Used the variables in my Sybase OLEDB Connection Manager's "Expression" property, setting the "ConnectionString" property to:
5. I set the values in my database table for the connection-I set 2 connections for which I have Sybase OLEDB datasources setup.
When I run the package, I just get the first server's data twice, it doesn't set the second server's data during the second loop. I made sure the first one was working (i.e. the ConnectionString's property was being set by the data from the current variables) by setting the variables incorrectly in the variable properties page, and then running the package. So the first row of connection information is working, but the second loop around it doesn't seem to be working. I used a msgbox in a script task to show that the variables are mapping correctly in the loop, so it seems the second time around the connection information isn't taking from the variables.
I have an SSIS package that I need to modify. It was developed in Visual Studio 2005 and runs faithfully in production on SQL Server 2005. Suddenly, I can't open the package on my own workstation. I see a big red "X" image and a message stating "Microsoft Visual Studio is unable to load this document." The real reason the document can't be loaded appears to be:
Error loading DataWarehouseLoader.dtsx: The connection type "OLEDB" specified for connection manager "Warehouse Logging OLE DB" is not recognized as a valid connection manager type. This error is returned when an attempt is made to create a connection manager for an unknown connection type. Check the spelling in the connection type name.
Does anyone know what causes this and what I can do? The package has been in production for 9 months and I've never seen this problem before.
howdy, i have a sqlserver2005 ssis package which connects to a mysql webserver to upload data. i am connecting via mysql oledb connection and am able to test connect ok through the oledb connection manager. However in my ole db destination editor i get an 0x80040E21 error when trying to preview a table . The full error is :
Error at Data Flow Task [Destination - Query [70]]: An OLE DB error has occurred. Error code: 0x80040E21.
Error at Data Flow Task [Destination - Query [70]]: Opening a rowset for "INFORMATION_SCHEMA.address_book" failed. Check that the object exists in the database. ------------------------------ ADDITIONAL INFORMATION:
Exception from HRESULT: 0xC02020E8 (Microsoft.SqlServer.DTSPipelineWrap)
I can see all the tables in the drop down box in connection manager. This has been driving me nuts for the last 3 days! I have tried changing some of the security connection settings to no avail, (although i am not sure on what these do).I have also tried setting up as an odbc connection using script components but had memory errors after copying several hundred records and the package just hangs. I can go into this in more detail however i dont think they are related issues and i would prefer oledb as i dont have to script my data transfer.
I have a SSIS which have a main connection to a SQL Server 2005 DB, and also another connection to a Oracle database, and fetch data from the Oracle db.
I have the 2 connection managers, and everything runs fine when I test it in dev studio. However, once I imported it to the DB, and run it on command line, I get the "Class not registered" error.
Log file as follows:
C:>dtexec /DTS "MSDB ST1INST101CRDA_D_GIM_CONN_TEST" /SERVER crpnycmsq34q /CONNECTION "CRD_CM";""Data Source=crpnycmsq34qST1INST101;Initial Catalog=CRD;Provider=SQLNCLI.1;Integrated Security=SSPI;"" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING V Microsoft (R) SQL Server Execute Package Utility Version 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 1:23:53 PM Info: 2008-02-28 13:23:53.91 Code: 0x40016040 Source: CRDA_D_GIM_CONN_TEST Description: The package is attempting to configure from SQL Server using the configuration string ""CRD_CM";"[dbo].[CMG_SSIS_CONFIG]";"GIM_SOURCE";". End Info Info: 2008-02-28 13:23:54.03 Code: 0x40016040 Source: CRDA_D_GIM_CONN_TEST Description: The package is attempting to configure from SQL Server using the configuration string ""CRD_CM";"[dbo].[CMG_SSIS_CONFIG]";"GIM_USER";". End Info Info: 2008-02-28 13:23:54.05 Code: 0x40016040 Source: CRDA_D_GIM_CONN_TEST Description: The package is attempting to configure from SQL Server using the configuration string ""CRD_CM";"[dbo].[CMG_SSIS_CONFIG]";"GIM_PWD";". End Info Error: 2008-02-28 13:23:54.55 Code: 0xC0202009 Source: CRDA_D_GIM_CONN_TEST Connection manager "GIM" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040154. An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040154 Description: "Class not registered". End Error Error: 2008-02-28 13:23:54.56 Code: 0xC00291EC Source: Execute SQL Task Execute SQL Task Description: Failed to acquire connection "GIM". Connection may not be configured correctly or you may not have the right permissions on this connection. End Error Warning: 2008-02-28 13:23:54.58 Code: 0x80019002 Source: CRDA_D_GIM_CONN_TEST Description: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors. End Warning DTExec: The package execution returned DTSER_FAILURE (1). Started: 1:23:53 PM Finished: 1:23:54 PM Elapsed: 1.547 seconds
I DID TRIED MY BEST to search before posting, and most of them are having problem with 32 bit and 64 bit executions. However, in my case, both are 32 bit (as shown in the log file).
In the above test, I'm setting DoNotSaveSensitiveData, and then fetch it with package configuration.
To make sure the package configuration is not creating pblm for me, I also tried to hardcode everything in the package, and then encrypt the whole ssis with password, and run the package with the /DECRYPT option, so that everything is already intact.
But I still get the same error. Log file as follows.
C:>dtexec /DTS "MSDB ST1INST101CRDA_D_GIM_CONN_TEST" /SERVER crpnycmsq34q /De "painful" /CONNECTION "CRD_CM";""Data Source=crpnycmsq34qST1INST101;Initial Catalog=CRD;Provider=SQLNCLI.1;Integrated Security=SSPI;"" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING V Microsoft (R) SQL Server Execute Package Utility Version 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 2:24:25 PM Error: 2008-02-28 14:24:27.11 Code: 0xC0202009 Source: CRDA_D_GIM_CONN_TEST Connection manager "GIM" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040154. An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040154 Description: "Class not registered". End Error Error: 2008-02-28 14:24:27.14 Code: 0xC00291EC Source: Execute SQL Task Execute SQL Task Description: Failed to acquire connection "GIM". Connection may not be configured correctly or you may not have the right permissions on this connection. End Error Warning: 2008-02-28 14:24:27.14 Code: 0x80019002 Source: CRDA_D_GIM_CONN_TEST Description: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors. End Warning DTExec: The package execution returned DTSER_FAILURE (1). Started: 2:24:25 PM Finished: 2:24:27 PM Elapsed: 1.422 seconds
Can anyone suggest what is missing ?? or what can be done ??
I have problem to extract data from DB2 table using SSIS dataflow task with OLEDB source selecting the data access mode with command. Here is the scenario.(Note:If I select the data access mode as a table then it works fine.)
If I use the data access mode as SQL Command then it is not working. i.e. the flow stays yellow in controlflow nothing happens in dataflow and I noticed the SQL dumper and proc completes. I couldn't find any log to verify the issue including the package execution results.Any of your thoughts will help me to resolve this issue.
I'm creating an xml configuration file to hold the connection string including the password (sql server authentication). My package protection level is set to 'EncryptSensitiveWithPassword'. I set up my connection manager and I check the 'Save my password' box. The 'Test connection' button reports that the connection is OK. I enable the package configurations and I go through the Package Configuration Wizard and I specify 'xml configuration file' and I choose 'Specify configurations settings directly' and I specify a configuration file name with the same directory as the package. I check the 'ConnectionString' property and the 'Password' property and then click 'Close'. Then I save my package changes. Now I look at the xml configuration file in a text editor and I see the Password property has an empty element:
<ConfiguredValue></ConfiguredValue>
Is it supposed to be empty? When I right-click the package in solution explorer and pick 'Reload with upgrade' then I have to enter the password, but the validation fails with 'Acquire connection' error. Should I just be saving the 'Connection string' property, or should I save all the connection elements (i.e. ServerName, UserName, Password etc.)? If I edit the xml configuration file and I type the password into the <ConfiguredValue> element above, and then I do 'Reload with upgrade', then the 'Acquire connection' validation error goes away. Could this mean that I am not able to encrypt the password? Thanks.
[One day I will come back here and answer questions, rather than only ask.]
I have a custom connection manager that returns an OleDbConnection. The only thing "custom" about the connection manager is how it determines its connection string. The object returned in AcquireConnection is an OleDbConnection.
OleDbConnection oleDbConnection = new OleDbConnection(this.connectionString); oleDbConnection.Open();
return oleDbConnection; }
Some of the objects within SSIS only work with an OleDbConnection, but mine is not recognized as such. Is there any way to have my connection recognized as a valid connection?
I am getting the following error when trying to EDIT a OLEDB Connection Manager. I am using VS2010. Im not coming up with any answers to this problem. I just reinstalled SQL SERVER CLIENT TOOLS 2012 That includes the SQL DATA TOOLS/VS 2010 But I still receive this error.
Error:Unable to find the requested .Net Framework Data Provider. It may not be installed. (System.Data)
