SSIS Import Of DBase Tables
Feb 4, 2007
I followed the steps outlined in
http://msdn2.microsoft.com/en-us/library/aa337084.aspx
and was successful at importing ".dbf" files from a dBase 7.0 folder.
However, this import was quite slow: approx. 2000 rows per second. Some of the tables I must import have over 8,000,000 rows, which would then take 4,000 seconds, or over one hour. This import rate seems unaffected by whether or not I put a "Copy Column" Data Flow Transformation task between the "OLE DB Source" and "OLE DB Destination" objects.
In SQL Server 2000 it was much faster to import such large tables. The 8 million row table takes only 14 minutes (800 seconds) with SQL Server 2000 -- giving a transfer rate approx. 5 times as fast.
Is there any way to speed up this process in SQL Server 2005? (In SQL Server 2000 there was a "Use fast load" option on the "Options" tab of the "Transform Data Task Properties." I have not yet found any similar option for SQL Server 2005.)
My receiving database is using the "Simple" model, so there is no need to create transaction entries for loading these large tables. There are also no indexes on these tables: the data are being loaded into new tables created in SSIS.
Thanks for your help with this issue.
View 3 Replies
ADVERTISEMENT
May 5, 2006
I saw this post by dterrie in the Wishlist thread and I just wanted to second it:
"How about bringing back a simple dBase import. The SSIS guys are clearly FAR out of touch with reality if they think people who handle data no longer need to work with dbf files. I've seen alot of dumb stuff in my day, bit this is just sheer brilliance. I just love the advice of first importing into Access and then importing the Access table. Gee, why didn't I think of such a convenient solution. I could have had a V-8."
I've been struggling with this the last couple days and finally decided to import the dBase III file into Access and then import that into SQL Server 2005. Imagine my surprise when I discovered this was the current recommended method.
That's just ridiculous. Can someone tell me why they would reduce some of the functionality of SQL Server from 2000 to 2005? This was a very easy process in SQL Server 2000...
View 3 Replies
View Related
Aug 24, 2007
Due to some legacy requirements, I'm using an OLEDB connection with the Jet driver and creating dBASE III tables (.DBFs) as the destination for a data export. The source is SQL Server 2000. The SQL table has a bunch of varchar columns, and I can't send them directly to the .DBF because SSIS complains that it can't (implicitly) convert from non-Unicode to Unicode. UNICODE?! dBASE III is ancient. Why would Jet/SSIS assume that these .DBFs are Unicode? As it stands, I have to put a data conversion task between the source and destination and convert all the columns. It's a real pain.
And no, I don't want to make all my SQL Server columns nvarchar. Is there any setting I can put in the connection manager or the connection string to prevent this error? I've already set AlwaysUseDefaultCodePage to True on the OLEDB destination component and the default code page is 1252. That didn't work; I still got the can't convert error.
View 3 Replies
View Related
Jul 20, 2005
Hi,SQL Server 2000 SP3Windos 2000 Server SP4I have a DTS package that imports data from a dBase IV databse withfiles located in two folders (dBF1 and dBF2). I use a transform datatask to transform the data.They were running properly, but last week we installed W2K SP4, andnow the transform task for files from dBF2 are not working properly.I have two tranform tasks to extract data from files in dBF2 folder.If I double click to open the transform data task of either of them,Enterprise Manager crashes with the errrormmc.exe applicatio ErrorThe instruction "xxxx" referenced memory at "xxx". The memory couldnot be read.Althoug the transform task for one of the files will run, the otherwill not run giving the messageError Source: DTS packageCatastrophic failureAlso, I have an Access database that has links to the same dBasefiles. For files from dBF2 folder, I'm able to see the data from oneof the files, but if I double click to see the data from the other,access crashes with no specific error message. Nothing has changed ondBase related files (permission wise).The transform tasks to extract data from the other dBase folder (dBF1)files are working fine, and data is accesible from Access.Any advice how to tackle this one?
View 2 Replies
View Related
Apr 28, 2004
I am trying to create a linked server to some dbase 5 tables. I have read several other posts about using the OPENROWSET option and that does work however, I would like to be able to create a linked server for easier access.
End Goal: I am developing an ASP based app that needs to join a DB5 table to a SQL table.
Any help on the linked server properties in Enterprise Manager for dbase5 or on the proper sp_addlinkedserver function for dbase5 would be highly appreciated.
Joe
View 2 Replies
View Related
Aug 22, 2006
I am having trouble connecting dbf of version dbase II. I can connect dbase III and dbase IV in ssis package,but can't connect dbase II.hope your help!
View 3 Replies
View Related
May 4, 2006
I am trying to get a handle on the new SQL Server Integration Services in SQL Server 2005. There is a legacy DTS package that I need to get working on our new server (using SQL Server 2005) but I can't seem to get it to work.First, I migrated it and attempted to run the package. It gave me many errors. So, I thought I would just rebuild it in integrated services. Everything was going smooth until I got to the real data transformation part.I added a data flow task and it took me to the Data Flow tab. Then I added a DataReader Source. I am trying to read from a dBase III file and I assumed I would be using an ODBC driver for that. Well... it has been hell trying to get it to see the dBase III file and access it.I have tried two solutions from the internet.This one and this one (by Wenyang) with no positive results.No matter what I do I still get an error in the bottom of the Advanced Editor for DataReader Source that reads: "Error at Sales Transformation [DataReader Source [81]]: Cannot acquire a managed connection from the run-time connection manager."Has anyone attempted anything like this before? Are there any SSIS experts around here that can drop me a few pearls of wisdom?Thanks in advance... I'm dying here.
View 3 Replies
View Related
Jun 16, 2015
I have a requirement where in i have around 15 different flat files , filenames are fixed but folder path can be changed(i think i should use a variable for folder path). These 15 files data should go to their respective tables in the database.
Whether I need to create separate data flow task for each file or separate package? In addition to these, example : while importing product data into product table, if product ID already exists, we need to ignore it and upload only the new records.
View 4 Replies
View Related
Aug 25, 2015
I have an excel file that has multiple sheets and I need to import data from each separate sheet to a separate table using SSIS.
E.g. Sheet A data should go to Table A and Sheet B data should go to Table B and so on. Is it possible to do this with out using script task.
View 6 Replies
View Related
Apr 26, 2008
Hi. I need to import excel file in database. i first need to do an unpivot task. the column names are dates and SSIS seems to be unable to pick up the column name as it is replaced by F2 F3 F4etc Can you advise of a solution. thanks ken
View 1 Replies
View Related
Nov 23, 2007
Hi,
Can I give a sequence to each table that I will try to import from one database to another by using import /export wizard
Regards
Vaibhav
View 1 Replies
View Related
Apr 13, 2007
Hi there,
I am having difficulties in importing table from one sql server database file to another sql server database file.
A few months ago, I converted access file to ms sql express file. I had made many changes on the ms sql express file, however, the data in this file isn't the latest as the old system is still being used. Now, I want to deploy my new system, I need to import in necessary tables from the old system database, as well as, I want to retain the tables and data I created on the ms sql express file that I have been using so far for development and testing.
May I know how to import tables from other database? Just as in ms access where we can import tables from other access file. I'm using sql express 2005 and sql server management tool. Any advice/help is very much appreciated.
Thanks....
View 2 Replies
View Related
Nov 6, 2007
Still really new at all this, but learning lots thanks to this forum.
I was wondering - is there a way to import Access tables into my SQL Server 2005 ?
(The Data and the Table Design?)
View 3 Replies
View Related
Apr 24, 2001
I would like to know:
How can I import tables from the Enterprise Manager to Excel (copy and paste)?
Is it possible?
thanks
View 1 Replies
View Related
Sep 7, 2006
I am really new to SQL Server so be patient i guess. I have installed sql server 2005 on my home computer but at first i wasnt able to connect to the server at school where i had started creating tables already. Instead of going up to the school i opted to do it from home and start over. Well I am wondering if there is a way to copy all the tables that i have created on my home computer and just import them into the database on the server at school. Hope this isnt confusing. Anyways, this forum always has helped me in the past, so i am confident someone will be able to point me in the right direction. Thanks in advance.
View 3 Replies
View Related
Mar 29, 2006
i am using sql server express edition.
i need to import (or make copies) on a weekly/daily basis, of several oracle tables into a new adp database.
what is the fastest option? some of these tables hold over 1m records.
so ideally i would like to schedule a refresh or update job outside busy hours.
I have heard of "linked servers" and "dts", but would like some experts advice before starting looking into this.
many thanks
luzippu
View 3 Replies
View Related
Nov 16, 2006
I have some Access tables that I want to import into SQL. I can do this using the AllTasks/ImportData. However if my Access table has a zero length string in a field this is imported into the SQL table as <NULL>. How can I make it import it as a zero length string ?
View 2 Replies
View Related
Feb 21, 2008
I am an SQL newbie and despite 3 hours on the Internet this morning I am still confused!
I have some XML tables that I want to import into my SQLExpress database.
I have followed the example at http://support.microsoft.com/kb/316005 How to import XML into SQL Server with the XML Bulk Load component.
I understand the example but when I run the vb code:
Set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkLoad")
objBL.ConnectionString = "provider=SQLOLEDB.1;data source=MySQLServer;database=MyDatabase;uid=MyAccount;pwd=MyPassword"
objBL.ErrorLogFile = "c:error.log"
objBL.Execute "c:customermapping.xml", "c:customers.xml"
Set objBL = Nothing
I get an error message saying that ActiveX cannot create object 'SQLXMLBulkLoad.SQLXMLBulkLoad'.
I have SSMSE and the databse on my PC harddrive.
What am I missing here?
Thanks in antcipation.
Chris
View 5 Replies
View Related
Apr 2, 2008
Hi, I am trying to import an XML file to SQL tables using SSIS, Data Flow Task --> XML Source --> OLEDB destination.
I have both the XML file and its xsd file.
The extraction of the XML file output 5 tables (mc, r, i, bc and c, have relationship among them). My problem is that one of the tables is empty (table mc). I tried to add a "root" node to the XML file (the xsd file was modified too), and all 5 tables were populated correctly. Is there any other way to fix this problem without manually changing the XML file?
The XML file looks like this:
<mc MajCmd="Atlantic">
<r Region="East">
<i Installation_UIC="M00146" Installation_Name="Cherry Point">
<bc BC="111" BC_Title="Airways">
<c Restoration_Cost_C3="0.0000" Comment_Type="" Override_C_Rating="" Comment="" />
</bc>
</i>
</r>
</mc>
The xsd file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<xschema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urnchemas-microsoft-com:xml-msdata">
<xs:element name="mc">
<xs:complexType>
<xsequence>
<xs:element name="r" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xsequence>
<xs:element name="i" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xsequence>
<xs:element name="bc" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xsequence>
<xs:element name="c" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
</xs:complexType>
</xs:element>
..........
</xsequence>
<xs:attribute name="BC" type="xstring" />
<xs:attribute name="BC_Title" type="xstring" />
</xs:complexType>
</xs:element>
</xsequence>
<xs:attribute name="Installation_UIC" type="xstring" />
<xs:attribute name="Installation_Name" type="xstring" />
</xs:complexType>
</xs:element>
</xsequence>
<xs:attribute name="Region" type="xstring" />
</xs:complexType>
</xs:element>
</xsequence>
<xs:attribute name="MajCmd" type="xstring" />
</xs:complexType>
</xs:element>
<xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element ref="mc" />
</xs:choice>
</xs:complexType>
</xs:element>
</xschema>
Thanks!
ChaInn
View 2 Replies
View Related
Mar 19, 2006
Is there a way in SQL 2005 Express Edition to import/export data from another MSSQL Server? Something similar to the right click/ import table functionality of MSSQL2000.
I've read similar threads which all recommend using MS Access but there has to be a better way, right?
regards,
dj
View 9 Replies
View Related
Jan 10, 2006
I have problems importing tables from db2 using the import wizard using IBM OLE DB Provider for DB2
The tables and data ('Preview...') can be viewed but when trying to map the columns ('Edit...' next to the tables) or import the data an error is recieved
[DB2/NT] SQL0443N Routine "SYSIBM.SQLCOLUMNS" (specific name "COLUMNS") has returned an error SQLSTATE with diagnostic text "SYSIBM:CLI:-727". SQLSTATE=38553
Schemas is used in both DB2 and Sql 2005
There is no problem importing to Sql 2000
View 1 Replies
View Related
Mar 9, 2007
I'm trying to migrate some tables from 2000 to 2005. I used the wizard in SSIS to create a package bringing them in.
The first database I did this on it worked great, in no time.
However, I tried this on another database and I get a LOT of these:
Information: 0x4004800C at {0417E15E-7306-4DAD-BEDD-1FDB17390EB0}, DTS.Pipeline: The buffer manager detected that the system was low on virtual memory, but was unable to swap out any buffers. 2 buffers were considered and 2 were locked. Either not enough memory is available to the pipeline because not enough is installed, other processes are using it, or too many buffers are locked.
Now, some things to note:
1. I'm trying to do about 170 tables at once - although none of them are bigger than a MB
2. Copying files between these 2 servers is also taking a long time
Is it too many tables at once? Is it the actual pipeline between the servers? The message above seems to indicate that it is a problem local to the machine, and maybe our file-copying problem is just another issue entirely.
View 3 Replies
View Related
Dec 17, 2006
Hi all,
I've just installed the Sql Server 2005 Express Ed + Management Studio Express and I can't get how to import data from a MS Access 2003 file. From what I've read is not possible to do through the Management Studio Express so is there any other tool to do that?
Thanks in advance,
Marc Soleda
View 7 Replies
View Related
Mar 5, 2007
how do i import or what is the best way of import from xml file to a sql server tables?how do i export or what is the best way of export from sql server tables to xml file?thanks
View 4 Replies
View Related
May 19, 2008
hi
guys i need to import all tables from sqlserver to informix ...............
any idea really appreciated
View 5 Replies
View Related
Nov 7, 2005
Hi all,
I am a newbie to .NET and would appreciate all your valuble
suggestions. I have and issue were I am trying to import data from a
few selected columns MS Access and a couple of columns in SQL
Server Table Y and trying to populate another table X . Both
tables X and Y are in the same Database . I am wondering if I could
design a custom package for this task.
Ananth
View 2 Replies
View Related
Jul 16, 2002
Quick Question!
If you have both invoice header lines and invoice detail lines in a comma delimited file, how can I get the data in the file to be imported into two different tables. I can produce a text file eg:
1,20,10/03/2002,39 High Street Any Town,,
2,20,Fluffy Slippers,2,Red,10.99
2,20,Pyjamas,3,Black,15.99
2,20,Trousers,1,Lilac,24.99
1,21,10/03/2002,11 Gibson Close,
2,21,Sandles,1,Black,12.99
2,21,Shoes,4,Blue,23.99
1,22,13/03/2002,45 Mill Street,
2,22,Womble Feet,4,White,16.99
2,22,Glass Slipper,1,Transparent,23.99
Lines with 1 in the first column should go in the InvoiceHeader table
Lines with 2 in the first column should go in the InvoiceDetails table.
I have tried with DTS but to no avial - ActiveX scrips in the Transform Data Task can only seem to access one data destination - it one table not two.
Any Ideas?
Julia
View 3 Replies
View Related
Nov 14, 2003
Can someone tell me if there's a way to export data from one database to another without losing the primary key information that has been set in the source database?
Please respond to URL. Thanks!
CC
View 1 Replies
View Related
Jan 29, 2014
I have 1 Excel spreadsheet. It has about 50 fields. I need to import this into my Database.The problem is; there are 2 tables that I need to import this into. These 2 tables are related with [ID].So, I need to split the data from 1 spreadsheet between 2 SQL server table. and keep the records related.
View 4 Replies
View Related
Feb 24, 2004
Hi,
I am using DTS in SQL server2k
What is the best way to insert CSV records, where 1 record maps onto multiple tables with parent/child or PK/FK relationship.
eg.
CSV record
(ID, param1, param2, param3)
ParentTable(ID as PK)
Param1Table(ID as FK in ParentTable, param1)
Param2Table(ID as FK in ParentTable, param2)
Param3Table(ID as FK in ParentTable, param3)
Currently i am using multiple Transform Data tasks with ActiveX script. But it seems to work slow.
Thanks and regards,
Henry
View 1 Replies
View Related
Mar 9, 2004
Hi There,
I am trying to create a DTS package that will import dbf tables. My problem is that the data was created back in the pre-dos 5 days, so to save room they took the dates and convert them to 2 characters.
Now I would like to import these tables and conver the date back. I do have a Function that I can run in FoxPro to "Unpack" these dates.
CODE
Function UnPackDate( cDate )
*!*PARAMETERS: f_datestr - Character (manditory). This should be a two
**!* byte string created with PACKDATE().
*!*
*!* RETURNS: A date from 1/1/1970 and 11/29/2126
If EMPTY(cDate)
Return {//}
Endif
Local nDate
nDate = (ASC( LEFT(cDate,1) ) * 256) + ASC( RIGHT(cDate,1))
Return ( CTOD("01/01/1970") + nDate - 8225 )
*================================================= ======================
Does anyone know how to take this function and make it work while doing an import?
Thanks
Sue
View 14 Replies
View Related
Apr 30, 2007
Hi
I have to import some Access 2003 tables into a SQL Server 2005. Since I have to do this every 6 months (different Access db into same SQL-Server) and since I have to check every single row I can't do it with the import data wizzard. I need to import the access tables into "temporary" tables in sql server and from there I'll put the rows into the correct tables...
Does anyone know how to import Access tables into SQL-Server 2005 with C#?
Thank you for your help!
Michael
View 9 Replies
View Related
Nov 28, 2006
Hi,
I want to import into an MDB table a csv file.
I'm trying to use the bulk copy table.
my function is:
SQL = "SELECT * INTO [my_table] FROM [ODBC;Driver=Micrsoft text driver (*.txt; *csv) ;Dbq=c:\;Extensions=asc,csv,tab,txt;].table.csv"
db.OpenEx( "Driver=Microsoft Access Driver .mdb);DBQ=c:\access.mdb;", CDatabase::noOdbcDialog );
db.ExecuteSQL( SQL );
when i run this function i get an error : "You cannot use ODBC to import from, export to, or link an external Microsoft Jet or ISAM database table to your database"
when i try to import in the same way a dbf file (insted the csv file) with VFP it's working well.
what seems to be the problem? how can i fix it? or if some one know how can i import a large csv file into access DB in an efficient diffrent way?
thanks ishay
View 1 Replies
View Related
