Importing From An OLEDB Datasource Into A SQL Database
Mar 18, 2008
Hi,
I have a proprietary OLEDB data source that works alongside a SQL database. Both are pretty big. I am trying to find a fast way of copying the data from the OLE source into a table in the SQL database. I am working in ASP.NET, and the easiest way is to have 2 connections, open the OLEDB and fill a datatable, then go through the table line by line and do a parametized insert. That works but takes far too long.
I am looking for something along the lines of "SELECT INTO tblB FROM tblA", or even "INSERT INTO tblB SELECT * FROM tblA", anything that is faster than the above. Can you do this in SQL but accessing an OLEDB datasource?
I have written a package which accepts variables for the server, initial catalog & table name.
I execute sql to drop the following stored procedure, then following sql statement to create it.
================================================================ SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO
CREATE procedure [dbo].[SP_CreateMatchProc] @sTable varchar(300) as BEGIN SET NOCOUNT ON declare @cmd nvarchar(2000) set @cmd = '''' Set @cmd = 'SELECT REPLACE(field1 + field2 + field3 + field4 + field5, '' '', '''') AS dBString ' + 'FROM ' + @sTable + ' ORDER BY <table>_ID COLLATE Latin1_General_CI_AS' exec (@cmd) END GO
Then in the Oledb source (validateexternalmetadata = false) I use "sqlcommand from variable" with a variable value of "SP_CreateMatchProc '<tableName>'"
The package runs fine in the IDE regardless of variable values, but when I created a batch file which calls dtexec I get a failure:
Error: 2007-10-04 08:46:42.82 Code: 0xC0202005 Source: Data Flow Task OLE DB Source [310] Description: Column "dBString" cannot be found at the datasource. End Error Log: Name: OnError Start Time: 2007-10-04 08:46:42 End Time: 2007-10-04 08:46:42 End Log Log: Name: OnError Start Time: 2007-10-04 08:46:42 End Time: 2007-10-04 08:46:42 End Log Error: 2007-10-04 08:46:42.82 Code: 0xC004701A Source: Data Flow Task DTS.Pipeline Description: component "OLE DB Source" (310) failed the pre-execute phase and returned error code 0xC0202005. End Error
with the ValidateExternalMetadata set to TRUE I get
Error: 2007-10-04 09:21:35.20 Code: 0xC004706B Source: Data Flow Task DTS.Pipeline Description: "component "OLE DB Source" (10621)" failed validation and returned validation status "VS_NEEDSNEWMETADATA". End Error
the most notable thing I see there is that it looks like a different ID (310) with out the validation and (10621) with it.
All: I am trying to code the following SQL into an OLEDB data source but it is not allowing me to do so because I think the variables are nested in multiple SQL statements. I have seen other posts that suggest using a variable to store the SQL but I am not sure how it will work.
I would also like to mention that the OLEDB source executes from within a For Each loop that is actually passing the values for the variables, which was one of the reasons I got stumped on how I could have a variable store the SQL.
Here is the SQL:
select b.ProgramID, b.ProductCode, b.BuyerID, b.Vendor,sum(a.Ordered) As Qty_Pruchased From SXE..POLine a INNER JOIN (SELECT VIR_Program.ProgramID, VIR_ActiveSKU.ProductCode, VIR_ActiveSKU.BuyerID, Vendor FROM VIR_Program INNER JOIN VIR_ActiveSKU ON VIR_Program.ProgramID = VIR_ActiveSKU.ProgramID INNER JOIN Vendor ON VIR_Program.VendorID = Vendor.VendorID WHERE ProgramFreq=?) b ON a.ProductCode = b.ProductCode WHERE a.TransDate >=? AND a.TransDate ? Group By b.ProgramID, b.ProductCode, b.BuyerID, b.Vendor
I've been using the new features with ASP.Net 2 for some months now. I use the gridview control a lot with the sqldatasource. I recently started a new job and am having problems connecting a sqldatasource control to a sql server 7 database. At my old job we used sql server 2000.
I get the error: "SQL server version needs be 2000 or higher" when I try to set up a new connection.. I read some documentation that said you can use asp.net 2 datasource controls with sql server 7. Does anyone know how to get round this issue. I don't want to have to go back to 1.x methods of connecting data controls to databases using code. Thanks
I've tried everything. The CreateUserWizard works fine, but then, I didn't have to connect anything for that. I've tried both SQLServer and MySQL. Connection Strings are correct, everything is correct as far as I can see. I've followed every step many many times. I'm about to go back to the old style and use VB6, because this is just stupid. The error I get is "An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)" If anyone could help me out with this I'd really appreciate it.
Hi I am trying to open a database connection to a Northwind database. I can open it with a datasource control and return data to a gridview, but can't programically. I get a sqlexception error. this is ust for learning. Protected Sub Button1_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button1.Click Dim S As New SqlConnection Dim builder As New SqlConnectionStringBuilder builder("Data Source") = ".SQLEXPRESS;AttachDbFilename=C:SQLNORTHWND.MDF" builder("Integrated Security") = True builder("Connect Timeout") = 30 builder("User Instance") = True S.ConnectionString = builder.ConnectionString Me.Label1.Text = S.ConnectionString S.Open() Me.Label2.Text = S.State.ToString S.Close() Me.Label3.Text = Me.SqlDataSource1.ConnectionString.ToString End Sub The text in label2 and Label three are identical except there are "" around the datasource. How come I can connect through the datasource control but not through code?
I got an error when i do an OLE db Source pointing to an sql 2000 database and executing a sql query inside the OLE Source. The ole source will point to an OLE DB destination which is an sql 2005 database.
But i got the below error:
Error at Data Flow Task [OLE DB Destination [245]]: the column firstname cannot be processed because more than one code page (936 and 1252) are specified for it.
Error at Data Flow Task [DTS.Pipeline]: "component "OLE DB destination" (245)" failed validation and returned validation status "VS_ISBROKEN".
Error at Data Flow Task [DTS.Pipeline]: One or more component failed validation.
Error at Data Flow TaSK: There were errors during task validation.
I have tried to connect ce database by using c++ and oledb, and failed in every turn, i dont know what i did wrong...
I created a new database by using create a new database sample in ssce bol, but i couldnt connect that database, it says invalid class string or authentication failed...
So I tried the typical IDBDataSourceAdmin:estroyDataSource() command to destroy an existing sql server 2005 express database and guess what it returned:
E_NOTIMPL (not implemented)
This leaves me in the dark on how I'm supposed to drop an existing database in SQL Native Client OLEDB. Any ideas?
// Sets properties in the Data Source and initialization property groups hr = m_pIDBProperties->SetProperties(uiNumPropsets, dbpropset); CHKHR(hr, L"Failed to set data source properties", _ExitCreate);
I have a VS 2005 Web Appplication solution that uses SQL Server Reporting Services 2005. One of the projects is a Report Server Project. In this project I have an RDL file that has: 1.)a non-shared XML DataSource (no credentials), 2.)a DataSet with a Command Type of Text and a Query String that is as follows: <Query> <XmlData> <Root> <LOAD LOAD_ID_PREFIX="" LOAD_ID="" TRUCK="" DIAGRAM=""/> </Root> </XmlData> </Query>
3.)a field that comes from the Database that is of VarBinary(Max) that is a JPEG Image
What happens is, is that by doing #2 above, at design time I can create tables and hook into my datasets and get the report all designed out the way I want. Then at runtime, I have a Stored Procedure that uses the "FOR XML AUTO" option so that I can represent my results from the database in an XML fashion which will allow me to replace the XML Schema in between the Root tags above (in #2) with my actual data (which still has the same schema). I then save the modified RDL to a database table field and move on to happily ever after......in this case, not. In this case, since an VarBinary is involved, unless I want to get a pointer back for the Diagram Column, I have to use the "FOR XML AUTO, BINARY BASE64" option. This returns an Encoded string to represent the Binary value that was in the Database....and this is the partial source of my angst....read on.
My approach that I described above worked flawlessly in all cases where an image wasn't involved. This report is the only report in the whole system I am working on that will have an image and it is the last one being done, so that's why I didn't run into this earlier.
I have tried all kinds of things at different levels, but the thing that I have tried that seemed to me to have the most potential was to use Embedded Code to take the Encoded ASCII Value and convert into something binary that the SSRS Reporting Engine likes (so my image will show), but I have been unsuccessful thus far. The error message I keep getting now from my Embedded Function is: "Invalid length for a Base-64 char array", which I have not been able to resolve thus far....and I do know that the Binary value in the database is good becuase I can see it in pages in my application otherwise.
This aside, is there anyone out there that knows how to take the value of a VarBinary(Max) field and insert it into the Text (XML) DataSet in the markup of an RDL file....and have it appear correctly when the report is rendered in something like the standard Report Viewer Control (or even when being previewed at design time inside of Visual Studio for that matter)?
Just to be clear, the root of my problem, as I see and understand it, is that I have a VarBinary value in a database that is being converted to a string (ASCII...I am presuming) due to how I am writing my stored procedure and then needs to go back to something else binary that the SSRS Engine will accept so that I can see my Image in the report when I render it.
I am wishful that someone out there understands my problem and has run into this and has a solution......but I am wishful of winning the lottery, too, and that hasn't happened yet...............
I am using ADOX to create linked tables in a jet database from an ODBC datasource. The tables in the ODBC data source does not have a primary key. so I am only able to create read only linked tables.But I want to update the records also. I tried adding a primary key column to the linked table while creating the link. but I am getting an error while adding the table to the catalog.
The error message is "Invalid Argument".
I use the following code for creating the linked table
Sub CreateLinkedTable(ByVal strTargetDB As String, ByVal strProviderString As String, ByVal strSourceTbl As String, ByVal strLinkTblName As String)
Dim catDB As ADOX.Catalog Dim tblLink As ADOX._Table
' Name the new Table and set its ParentCatalog property ' to the open Catalog to allow access to the Properties ' collection. .Name = strLinkTblName .ParentCatalog = catDB
' Set the properties to create the link. Dim adoxPro As ADOX.Property
Getting following error while trying to connect to ORACEL database using OLEDB manager.
Error at RADAR [Connection manager "RADAR_Updated"]: SSIS Error Code DTS_E_OLEDB_NOPROVIDER_ERROR. The requested OLE DB provider OraOLEDB.Oracle.1 is not registered. Error code: 0x00000000. An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040154 Description: "Class not registered".
I am trying to dynamically change my initial catalog in a SSIS project. Each day I get a snapshot of data from the production server. I am performing ETL on that database and loading it into my warehouse. I am trying to put something together so that each day I get the latest snapshot and if its newer then my most recent one I pass that new catalog into the connection manager. I can already run the sql script to compare the two catalogs but I can't populate the connection manager.
I experienced this problems on both Windows Mobile 2003 SE and Windows Mobile 5.0.
Its native development (c++, oledb, atl and mfc).
It's quite simple to reproduce...
1. open a database
2. open a rowset on table A (whatever, valid of course and with both IOpenRowset and ICommandText), read datas and close rowset
3. power off
4. power on
5. try step 2 with another table (failed on openrowset with error 0x80004005) or try table A (sometimes working because of cached memory, sometims failed on Read Datas).
6. being stuck ;-)
Our work-around was, in case we loose our connection (identified by error 0x80004005 on openrowset), we close it and re-open database... ugly for sure, but working.
What I'm looking now is to use some kind of "detection method" like what people in .Net develoment are using "if ConnectionState.Open <> ...) for reopening my database only on demand...
Hi All,Here is my code Dim strCon As String = System.Configuration.ConfigurationSettings.AppSettings("OwnerTrader") con = New SqlConnection(strCon) con.Open() Dim strselect As String = "" Try strselect = "INSERT INTO tbl_CSV(CSV_TIME,CSV_SIZE,CSV_LOCATION,CSV_COUNTRY,CSV_LAT,CSV_LON,CSV_COMMENTS)" strselect = strselect & " SELECT * FROM OPENROWSET('MSDASQL','Driver={Microsoft Text Driver (*.txt; *.csv)};DEFAULTDIR=C:UploadFiles;Extensions=CSV;','SELECT * FROM TestNew.csv')" cmd = New SqlCommand(strselect, con) cmd.ExecuteNonQuery() con.Close()i have defined connection string in web.config file and my csv file is inside C:UploadFiles with the name TestNew.csv file.Can u please check it out the code?is it correct or not.I am getting this error"SQL Server blocked access to STATEMENT 'OpenRowset/OpenDatasource' of component 'Ad Hoc Distributed Queries' because this component is turned off as part of the security configuration for this server. A system administrator can enable the use of 'Ad Hoc Distributed Queries' by using sp_configure. For more information about enabling 'Ad Hoc Distributed Queries', see "Surface Area Configuration" in SQL Server Books Online."Regards
I use ms sql 2000 and i wanted to import a new backup database into a new ms sql 2000 on a different machine but it does not work so i wanted to know if it can be done or i have to do something else thank u.
As a newbie, I'm having problems importing a copy of my SQLServer2k development database from my laptop (via a CD), to my production SQL Server2k database.
I've tried the DTS Import/Export wizard & also the Copy wizard, but can't seem to see the dialog that prompts & identifies the source path & filename.
I've tried setting up an ODBC DSN for the database, but get the same problem.
Coming from an Access background, I've been used to the copying of *.mdb files, and half expected that level of simplicity with SQLS2K - but clearly, I'm wrong and am missing something, which the on-line help isn't able to clarify for me...
Can anyone please give me any pointers for me to get a copy of my development database along with its stored procedures, from my laptop (not networked), to my desktop server. Both use SQLS2k.
SQL 2000, Windows 2000. I need to find a way to export our data fromour production database and import it into a test database. I can doit to a freshly created test database I just have trouble doing itonce the test database is already populated. Is there a script I canwrite that will automate it by1) Deleting the test database2) Creating a new one3) Exporting the production tables and views4) Importing into the new test databaseWhat are my options here?Thank you in advance.
sql server 2005 express edition, sql server management studio express, oracle tables (original source data currently linked through odbc) adp database file (new db where i'm trying to import the above oracle tables)
i need to import (or make copies) on a weekly/daily basis, of several oracle tables into a new adp database.
what is the fastest option? some of these tables hold over 1m records.
i have also heard of "linked servers" and "dts", but would like some experts advice before starting looking into this.
Hi when i am using OLDB Command for Update Existing Records the Follwing ERror Code I am getting . Any one pls help me on this one.
1)[DTS.Pipeline] Error: The ProcessInput method on component "OLE DB Command 1" (9282) failed with error code 0xC0202009. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.
2)
[OLE DB Command 1 [9282]] Error: An OLE DB error has occurred. Error code: 0x80040E10.
3)
[OLE DB Command 1 [9282]] Error: An OLE DB error has occurred. Error code: 0x80040E10.
Hello Everyone, I would like to import a text file which contains one string (a large integer) per line not separated by commas or anything else except a carriage return. Does anyone know of an easy way to store this in a database file? I'm open to suggestions if there is more than one way to save this kind of information within a database. I have SQL server 2005 developer edition if that helps in any way. I'm also starting to learn about Linq so if there is some other way you would store this information for that purpose I would love to hear about that as well. C# code is preferable, but I can use the automatic translators if that's all you have. By the way, I'm a newbie to this subject (if you couldn't tell). Thanks in advance. Robert
Hi, i have a new site built in .net using the login/membership system. i have a legacy database (access) of users from the previous site with email, name, password, etc.Is it possible to import this data straight into my new membership tables through some insert statement, software or through some other way?
Could some1 pls help me, how do I import a word doc into a sql server database, and it mus t be displayed in a panel I need to know how to do it, step by step
Does anyone out there have a good example of a VB Application for importing data from an Access DB into a SQL Server DB using DTS. Any help with this would be greatly appreciated.
I am trying to import tables from my access database.
1st trying simple table, In ent mgr i right clicked the db folder all task import data data source : Microsoft access file name : c:Data Management SystemDMS.mdb User name : mike (same used for lonin in access) password : (same used for lonin in access)
advanced button: jet Oledb: System database c:Data Management SystemSecured1.mdw
windows authentication server (local) database New -> dms
copy tables
now i see the items in my access database so i am guessing every step so far was correct
hey it worked !
ha,, hope u dont mind if i leave this here for anyone else, i ansd my own ?
Has anyone done this before, I have been given some flat files for a Sybase database to import into SQL server. Yet SQL server doesn't want to know. I have a .db and .log, but SQLserv er doesn't like either of them.
However I'm not sure what to do, I have been given a .db and .log . but SQL server doesn't like either of these, is their something I should do to make it compatable? First tie I've ever look at Sysserver so please use short words, Thanks Ed
I need to import data from excel to sql database. I have an excel that has a column called dept_name that is a string. In the database I have a table called tb_depts with the following columns: dept_id int, identity dept_name nvarchar (200)
In addition I have another table called tb_extUsers with the following columns: user_id int, identity user_name nvarchar (200) dept_id int
I need to import the excel into the table tb_extUsers however my problem is that the value in excel is the name of the department and not the department id and the column in tb_extUsers requires the id. How can i insert the correct dept_id into tb_extUsers ?
Can someone please assist me?
Thanks
Whisky-my beloved dog who died suddenly on the 29/06/06-I miss u so much.
Hey guys,I am trying to import data from an Excel spreadsheet into my SQLdatabase. I am running SQL 2005.I following Microsoft's instructions for creating a linked server, andit appeared to work. However when I run this query:SELECT * INTO test FROM OPENQUERY(EFORMS,'SELECT * FROM [Form Tracker Baseline$]')I get the following error:The OLE DB provider "Microsoft.Jet.OLEDB.4.0" for linked server"EFORMS" reported an error. The provider did not give any informationabout the error.Msg 7303, Level 16, State 1, Line 1Cannot initialize the data source object of OLE DB provider"Microsoft.Jet.OLEDB.4.0" for linked server "EFORMS".Test is the table that will receive the imported data. Eforms is thename of my linked server, and Form Tracker Baseline is the worksheetname.The error sounds like it can not locate my spreadsheet. Any ideas whyI am getting this message? Is there an easier way to do this import?Thanks,
How to check if a record is already in the database I want to import to before importing?
My table have 2 primary key, and the DTS is suppose to be schedule to run every night, though it can be run whenever a user want to get the most updated data.
