Sql Import/push To Oracle Faster
Jan 10, 2006
Looking for a faster method of moving data from SQL to Oracle.
I'm attempting to push a sql table into an oracle table (sql server 2000 oracle 7, 8, and 9). I have no problem doing this with either 'Oracle Provider for OLE DB' or the 'Microsoft OLE DB Provider for Oracle'. None of my data is being transformed so its a straight import. With the hardware I'm using it takes nearly 3 seconds to import 1000 rows. While this isn't too bad, I need to import upwards of 4 million rows and this results in unacceptable time results.
I do have an oracle script that imports the csv files of the tables, but I'm looking for an all inclusive sql solution.
Does anyone know of another method in SQL that I can use to push the data faster?
View 7 Replies
ADVERTISEMENT
Dec 13, 1999
Hello,
I was wondering if there was a different approach I should take in appending data to a table...
My destination table has about 94+ million records in it, and I have been taking two approaches to getting new files into this table:
1) I do a data pump task in a DTS to import the file to a trans (temp) table, which is truncated every time, and then do an INSERT INTO statement from the temp table to my destination table.
The import to the trans table only takes a few minutes (about 1 - 2 million records per file, but have short record lenghts,) but when I do the INSERT INTO statement, it takes upwards of 6 hrs to append.
2) I have tried doing a bulk insert task, going directly to the destination table (which defeats the purpose of my trans table to check out the data prior, but I feel the data is clean at this point.)
I am running the bulk insert right now, and it's been running for over 3 hours...so I'm going to assume this will take just as long as the INSERT INTO statement does like I did before.
My destination table does not have any indexes in it at all, and I don't need to do any transformations to the data when bringing it into SQL since the data is clean. Also, I have a default value constraint on one of my fields on the destination table.
Plus there are other ppl and applications hitting the server which could impact the overall processing, but nothing out of the ordinary is going on the server today. I know there are only so many ways to get a file into a table...but maybe someone knows a different way I should try this.
Thanks for anyone's suggestions!
Kael.
View 1 Replies
View Related
Mar 15, 2006
We need to configure a way to publish from development to testing and from testing to production. An easy, automated process would be best.
View 1 Replies
View Related
Mar 29, 2006
i am using sql server express edition.
i need to import (or make copies) on a weekly/daily basis, of several oracle tables into a new adp database.
what is the fastest option? some of these tables hold over 1m records.
so ideally i would like to schedule a refresh or update job outside busy hours.
I have heard of "linked servers" and "dts", but would like some experts advice before starting looking into this.
many thanks
luzippu
View 3 Replies
View Related
Dec 7, 2007
Hi All,
I'm trying to import from Oracle Rdb database into SQL server 2005 via ODBC without much success.
I've installed the correct Oracle Rdb driver and verified it all works ok by importing in to Access.
Using the import wizard within SQL mgt studio and the following connection strings I get a password error
Dsn=XRS;Driver={Oracle RDB Driver};sid=system
TITLE: SQL Server Import and Export Wizard
------------------------------
The operation could not be completed.
------------------------------
ADDITIONAL INFORMATION:
ERROR [HY000] [Oracle][ODBC][Rdb]%SQLSRV-F-GETACCINF, Oracle SQL/Services authorization failed
ERROR [HY000] [Oracle][ODBC][Rdb]%SQLSRV-F-GETACCINF, Oracle SQL/Services authorization failed
------------------------------
BUTTONS:
OK
------------------------------
I know the username/password combination to be correct.
Is SSIS the way to go. I've had a quick look at this and setup the ODBC connection but the column
mappings box reports the following...
error at data flow task [datareader source[1]; cannot aquire a managed connection
from the run-time connection manager.
Anyone point me towards any examples of reading in data via ODBC and storing in a SQL server database.
Should point out that the source is not Oracle RDBMS (7,8 or 9) but Oracle Rdb which is a
completely seperate product.
Thanks in advance
Dave
View 10 Replies
View Related
May 2, 2007
I am using Import and Export wizard and seeing a very weird behaviour.
I am able to choose 'Microsoft OLE DB Provider for Oralce' and provide server name, username/ password. The test connection results in a success. However when I try to move to next window/page by clicking next I get invalid oracle error ora-01017 - invalid username/ password.
what is going on here?
View 1 Replies
View Related
Apr 4, 2008
Hi all,
I recently encountered what will appear to me as a very strange problem with my packages involving data import from an Oracle 10g database. The packages are simple data imports which select and insert data from Oracle to SQLServer 2005 tables.
Till recently, the packages ran without any problems but a day ago, these packages starts to fail when nothing was changed in them. They will run for about 8 minutes, at which approximately 400,000 records will be selected and processed. After that these packages will fail. The failure rate is inconsistent in the sense that some of these packages will fail while some will pass, depending on the 'mood' of SSIS.
These packages are using Oracle's supplied OLE DB drivers and have been running well since day 1. Should I be concerned with Oracle terminating connection after 'x' minutes something like that?
I'm absolutely puzzled with these behavior and will appreciate it if someone can point me to the right direction. This server is a 64 bit server and SQL Server is installed as 64 bit as well.
Thanks in advance!
Nestor
View 6 Replies
View Related
Mar 6, 2006
I have a column in an Oracle source system with data type NUMBER(38,2). The value "-0.01" is causing problems when trying to import into a SSIS data-flow.
The only way I can import this into my data-flow is by using a Datareader connection manager using the ODBC Data Provider. My DSN is using the Oracle ODBC driver.
If I try and use the "Native OLE DBMicrosoft OLE DB Provider for Oracle" I get an error: "The data value cannot be converted for reasons other than sign mismatch or data overflow"
Judging by this post: http://microsoftdw.blogspot.com/2005/11/final-storyhow-to-get-data-out-of.html there aren't really any other combinations to try that will bring my data in as I want it.
I don't want to use ODBC though as:
Its an old technology
Its slow
I have to deploy an additional DSN.
Can anyone tell me why the other options don't work? Why does Microsoft OLE DB Provider for Oracle have a problem with "-0.01"?
-Jamie
View 2 Replies
View Related
Mar 3, 2000
Does anyone know how to import data from an Oracle view (on Unix machine) to the tables in NT/SQL server 7.0? At least point me to the right docoment resource, if available. Thanks a lot.
wendy
View 2 Replies
View Related
Mar 10, 2005
We will need to routinely import only changed data from an Oracle data base into a SQL database.
So we need an agent that will
1) Compare data in both databases (From disparate tables)
2) Import only that which is changes or new.
I am new to SQL server administration and am looking for a best practice method that we can be run on a weekly basis.
I am open to using third party software solutions, but would prefer a native MS SQL 2000 solution.
Can someone point me in the right direction?
Thanks.
View 1 Replies
View Related
Nov 29, 2006
Now this is the first time i have tried this with SQL 05 I am trying to Import data from an Oracle DB. I have the connection info:
Xtx20xxx = (DESCRIPTION = (ADDRESS_LIST =(ADDRESS = (PROTOCOL = TCP)(HOST =urlpart1.part2.part3.com)(PORT = 1521)))(CONNECT_DATA =(SID = Xtx20xxx ))
)
I used the Microsoft OLEDB provider for oracle from the dropdown,
The username and password I have also. I tried to enter in the Xtx20xxx as the server name, clicked "Properties"
put in the username and password I can given and clicked "Test Connection". No matter what I get the infamous ORA-12154: TNS:could not resolve service name error message. I am sure I am missing something obvious here but I have absolutely no clue and all the googling in the world has not given me any help. I do have the Oracle 10g client tools installed on this workstation (running Windows XP Prof SP2 and SQL 2005 STD).
View 5 Replies
View Related
Feb 11, 2007
Hello friends,
I am working for a project. At my college I used to work with oracle. Now as requirement changes I have to change my database to SQL Server.
Is there any simple way one can suggest me?
View 2 Replies
View Related
May 25, 2015
I'm using - Destination - Oracle driver - oraOLEDB.Oracle.1 (native ole dboracle provider for ole db)
Source - SQL driver - microsoft ole db prover for sql server. I want to import data from sql server to oracle. Challenge is, I have 1 million records on oracle. I have 100 records on sql server (these 100 records count will change daily). So, I thought of using 'lookup' task looking taking record from ms sql and fetch corresponding record from oracle. But when I use lookup, all records from oracle are loading into cache, which is taking approx 3 hrs.
View 4 Replies
View Related
Dec 26, 2006
Hi,
I am running SQL Server 2005 and have built a simple SSIS package to import data from an oracle database to my SQL Server 2005 database. When I run it in SSIS, it works and it imports just fine. When I schedule it, it gives me problems. Help!
It might be a problem with Oracle Provider client I installed. Is there a client version I can download and install? the one I downloaded from oracle doesn't work. I bet i did something wrong though.
Here is my version:
Microsoft SQL Server Management Studio 9.00.2047.00
Microsoft Analysis Services Client Tools 2005.090.2047.00
Microsoft Data Access Components (MDAC) 2000.086.1830.00 (srv03_sp1_rtm.050324-1447)
Microsoft MSXML 2.6 3.0 4.0 6.0
Microsoft Internet Explorer 6.0.3790.1830
Microsoft .NET Framework 2.0.50727.42
Operating System 5.2.3790
Microsoft Visual Studio 2005
Version 8.0.50727.42 (RTM.050727-4200)
Microsoft .NET Framework
Version 2.0.50727
Installed Edition: IDE Standard
SQL Server Analysis Services
Microsoft SQL Server Analysis Services Designer
Version 9.00.2047.00
SQL Server Integration Services
Microsoft SQL Server Integration Services Designer
Version 9.00.2047.00
SQL Server Reporting Services
Microsoft SQL Server Reporting Services Designers
Version 9.00.2047.00
Thanks,
Ryan
View 5 Replies
View Related
Apr 19, 2007
I am getting this error when connecting to Oracle db. I tried using Microsoft OLEDB provider for Oracle it give me error and tells me the error could not be retrieved from Oracle. When I try the Native OLDDB provider for Oracle I get
Warning at {0F67F2FA-E3F8-4F44-93EC-47D513A34FD4} [Orcale Database WPHP2 [1]]: Cannot retrieve the column code page info from the OLE DB provider. If the component supports the "DefaultCodePage" property, the code page from that property will be used. Change the value of the property if the current string code page values are incorrect. If the component does not support the property, the code page from the component's locale ID will be used.
Error at Copy DSS_SJV_Volume_History [DTS.Pipeline]: The "output column "COMP_UNIQUE_ID" (2007)" has a precision that is not valid. The precision must be between 1 and 38.
What do I do !!!!!!!!!!
Thanks
Shabnam
View 1 Replies
View Related
Sep 6, 2006
Im getting the following error when I try to export from Oracle 9i and import into SQL Server 2005 using Oracle and SQL OLE DB Provider's respectively.
Cannot retrieve the column code page infor from the OLE DB Provider. If the component supports the "DefaultCodePage" property, the code page from that property value will be used. Change the value of the property if the current string code page values are incorrect. If the component does not support the property, the code page from the component's locale ID will be used.
I know there is a way to change the property settings for the OLE DB Provider in the Data flow section of the development studio BUT ...
1) Is there a way to change this outside of the development studio?
2) I can't create the package to setup in the studio because even when I uncheck execute immediately and check save package, it still fails and never creates anything.
Any help is much appreciated
Thanks
Scott
View 2 Replies
View Related
Mar 2, 2005
Hi,
I do not know if it can be done:
how to import an Oracle dump file into SQL server ?
That dump file contains the content of a table, excported from Oracle.
Is there a way or another to import it to SQLServer 2000 ?
Thanks
View 2 Replies
View Related
Jan 23, 2007
Hi, I've a question about importing Oracle data and some fields are null. I get an error 'Conversion failed because the data value overflowed the specified type'. When i look in preview query result, via OLE db Source editor > Preview, this field contains '<value too big to display>'.
What do i do wrong? Can somebody help me?
Thanks in advance
Olaf
View 6 Replies
View Related
Jan 29, 2008
Hi,
Please do anyone know a VB 2005 code to export and import SQL server table to Oracle. Thanks.
View 3 Replies
View Related
Mar 17, 2008
Access Connection
create a new Connection Manager by right-clicking in the Connection Managers section of the design area of the screen. Select New OLE DB Connection to bring up the Configure OLE DB Connection Manager dialog box. Click New to open the Connection Manager. In the Provider drop-down list, choose the Microsoft Jet 4.0 OLE DB Provider and click OK.
Browse to the Access database file and connection set up---all good!!!
Dataflow task
Add an OLE DB Source component
Double-click the icon to open the OLE DB Source Editor. Set the OLE DB Connection Manager property to the Connection Manager that I created . Select Table from the Data Access Mode drop-down list.
I cannot see the tables set up as set up as pass-through table types to a Oracle 9i db
Any ideas please help
thanks in advance
Dave
View 2 Replies
View Related
Oct 20, 2006
Hi All,
I have become frustrated and I am not finding the answers I expect.
Here's the gist, we support both Oracle and SQL for our product and we would like to migrate our Clients who are willing/requesting to go from Oracle to SQL. Seems easy enough.
So, I create a Database in SQL 2005, right click and select "Import Data", Source is Microsoft OLE DB Provider for Oracle and I setup my connection. so far so good.
I create my Destination for SQL Native Client to the Database that I plan on importing into. Still good
Next, I select "Copy data from one or more tables or views". I move on to the next screen and select all of the Objects from a Schema. These are Tables that only relate to our application or in other words, nothing Oracle System wise.
When I get to the end it progresses to about 20% and then throws this error about 300 or so times:
Could not connect source component.
Warning 0x80202066: Source - AM_ALERTS [1]: Cannot retrieve the column code page info from the OLE DB provider. If the component supports the "DefaultCodePage" property, the code page from that property will be used. Change the value of the property if the current string code page values are incorrect. If the component does not support the property, the code page from the component's locale ID will be used.
So, I'm thinking "Alright, we can search on this error and I'm sure there's an easy fix." I do some checking and indeed find out that there is a property setting called "AlwaysUseDefaultCodePage" in the OLEDB Data Source Properties. Great! I go back and look at the connection in the Import and .... there's nothing with that property!
Back to the drawing board. I Create a new SSIS package and figure out quickly that the AlwaysUseDefaultCodePage is in there. I can transfter information from the Oracle Source Table to the SQL Server 2005 Destination Table, but it appears to be a one to one thing. Programming this, if I get it to work at all, will take me about 150 hours or so.
This make perfect sense if all you are doing is copying a few columns or maybe one or two objects, but I am talking about 600 + objects with upwards of 2 million rows of data in each!!
This generates 2 questions:
1. If the Import Data Wizard cannot handle this operation on the fly, then why can't the AlwaysUseDefaultCodePage property be shown as part of the connection
2. How do I create and SSIS Package that will copy all of the data from Oracle to SQL Server? The source tables have been created and have the same Schema and Object Names as the Source. I don't want to create a Data Flow Task 600 times.
Help!!!
View 8 Replies
View Related
May 7, 2008
Dear all,,
I used a code that import data from an excel file into a dataset,
now I want to insert the dataset into a table in my database(SQLserver database) using a VB.NET code
Could you help me?
Thanks in advance,,
Here is my code:
Imports System.Data.OleDbPartial Class _DefaultInherits System.Web.UI.Page
Protected Sub Button1_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button1.ClickDim connString As String = ConfigurationManager.ConnectionStrings("xls").ConnectionString
' Create the connection object Dim oledbConn As OleDbConnection = New OleDbConnection(connString)
Try
' Open connection
oledbConn.Open()
' Create OleDbCommand object and select data from worksheet Sheet1Dim cmd As OleDbCommand = New OleDbCommand("SELECT * FROM [Sheet1$]", oledbConn)
' Create new OleDbDataAdapter Dim oleda As OleDbDataAdapter = New OleDbDataAdapter()
oleda.SelectCommand = cmd
' Create a DataSet which will hold the data extracted from the worksheet.Dim ds As Data.DataSet = New Data.DataSet()
' Fill the DataSet from the data extracted from the worksheet.oleda.Fill(ds, "Sheet1")
' Bind the data to the GridView
GridView1.DataSource = ds.Tables(0).DefaultView
GridView1.DataBind()
Catch
Finally
' Close connection
oledbConn.Close()
End TryEnd Sub
End Class
View 5 Replies
View Related
Jul 22, 2001
I'm trying to replicate two very big databases with about 10 million
of 4000 characters each. The publisher is SQL 2000, subscriber
is SQL 7.0
The subscriber will also perform full text searches.
I'm trying to decide wheter I should use PULL or PUSH.
The publisher is operating on a very low quality/speed internet
connection, where the subscriber is enjoying a T1.
Help, anyone?
Thanks,
-Michael.
View 1 Replies
View Related
Jun 21, 2008
Hi to all.
How can we identify that a transactional replication is configured in pull or push subscription..
where we have to check it out.
View 2 Replies
View Related
Nov 2, 2006
While trying to push a tracked table using RDA.push, I get the following error:
Error Code: 80004005
The message cannot be built. The make message failed.
Minor Err: 28581
Source: Microsoft SQL server 2005 Mobile Edition.
All other tables in the database are getting pulled and pushed correctly. This table is different only in the larger number of columns, around 150. It has a primary key, no other constraints.
Any help to find the reason for this error will be greatly appreciated.
- Paul
View 7 Replies
View Related
Mar 14, 2008
Hi All,
I am developing an application in which i have to send data from local Sql Server compact edition database[Which is in a Windows Mobile Device,] to central server[SQL Server 2005]. I am using RDA method for communication
Can i use push method to send data from local DB to Central DB?
Is it must to use PULL method before using PUSH method?
Thanks in Advance..
View 8 Replies
View Related
Feb 20, 2006
Im using the rda.push to push the data back to the SQL SERVER.
My SQL table is OrderDetail with a blank data inside?
My push was unseccessful due to the follow error, can any one help?
[code]
rda.Push("OrderDetail", rdaOleDbConnectString, RdaTrackOption.TrackingOffWithIndexes)
[/code]
Error message:The table is not a tracked table
View 3 Replies
View Related
May 9, 2001
Hello all. Please excuse my ignorance, as this is not my territory. I administer a website which is hosted remotely. This site has SQL7 running the data to dynamically build the site. Every Sunday our hosting service runs a DTS package to push the data they have down to us, so we can run reports and analyze it. We recently upgraded to SQL2000, while our host has stayed with SQL7. Now our DTS is failing. They say it is because 7 cannot push to 2000. But they think that we could pull from them. How do I go about setting that up? Will the DTS wizzard walk me through most of it?
Thanks in advace
Adrian Miller
View 1 Replies
View Related
Aug 29, 2003
what would be the best way to push a registry setting to about 3000 PC's. A Batch file is my thought. Any ideas?
Thanks.
View 3 Replies
View Related
Jun 1, 2004
Hi All,
I have a problem,
I need to copy a large amount of data from one table and insert it into another table.
The design of the destination table is exactly the same as the source table except for the fact that it has one extra field.
Can I copy; in a single SQL statement; all rows in one table (that match given criteria) into another table allowing for the extra field?
View 4 Replies
View Related
May 20, 2008
I have a production database that I would like to have copied over to a backup database on a separate server every evening. I don't want to mirror, I just want the databases synced up every evening.
The servers are physically attached through a gigabit switch and the database is relatively small, so I don't think that speed will be an issue.
Could someone point me to an article about the best way to accomplish this?
Thanks.
http://www.dynamicajax.com
View 2 Replies
View Related
Sep 18, 2007
Hi, all,
I did not see this one coming, and I am not sure if I did something wrong.
How do you push data from sql05 to sql2k?
I set up a data flow task, with one sql05 connection magager and another sql2k connection manager. Then when I tried to map them, I cann't!
The message on the box said: The connection manager uses an earlier version of sql server provider. Bulk insert operations require a connection that uses a sql server 2005 provider.
I have been trying different source, destination and transformation, but seems like missing something.
Thanks!
View 3 Replies
View Related
May 29, 2007
What is better to use, Push or Pull Replication? What are the advantages and disadvantages?
View 1 Replies
View Related
