Copy Dbase To SQL Performance
Mar 6, 2002
I am searching for a better way to move data from DBase3 files into SQL Server 2k. The Dbase file has approximately 2 million records and after applying a SELECT statement to retrieve the data I need to move, there are still over 600,000 records. To move this data I have used DTS (data pump) to transform the data- THIS IS UNBEARABLY SLOW. Next, I created an exe to transform the data to a flat file and then used BULK INSERT to copy the data. The exe is slow, but the BULK INSERT is fast. I have moved the source and destination to the local Server to increase performance, but with little imporvement.
My question:
Is there a better way to move large amounts of data from DBase3 files to SQL Server without waiting 38 hours to move 600,000 rows? Thanks for your help.
View 1 Replies
ADVERTISEMENT
Mar 31, 2008
Monthly, I copy a table from one database to another database. I delete the original table and copy the table back speed the performance of the query on the order of 10 to 1. Why does this work?
Detail:
I have a legacy table that a small application queries about once a month. The table was poorly designed and the query runs a date range comparison on one field and has a sub query that runs string comparison against six fields. I cannot change the calling app or table design. When the app calls the query, the call times out due to the inordinate length of time. To fix this until next months query, I copy the table out, delete the original and copy back. What changes when the table is copied to another database and then copied back? The performance of the query changes from 10sec to 1.
View 3 Replies
View Related
Sep 23, 2015
My requirement is to sling a rowset from one place in SQL server into a table in another place in the most performant way. I want this to be parameterizable - I want to provide just a connection string and some SQL for the source and a connection string and a table name for the destination. The package should do the rest.
The solution I chose was an 2014 SSIS package with source and destination as ADO.NET connections configured from project variables. The package has a script task to bulk copy the data. For performance I disable the non-clustered indexes first.
But this performance precaution causes the bulk copy to timeout after delivering the correct rowcount to the destination table. What I can do to avoid this error?
Here's my script code:
//get hold of the source and a data reader from it
SqlConnection sqlconnSource = new SqlConnection();
sqlconnSource = (SqlConnection)(Dts.Connections["source"].AcquireConnection(Dts.Transaction) as SqlConnection);
SqlCommand sourcesqlCommand = new SqlCommand(SourceSQL, sqlconnSource);
sourcesqlCommand.CommandTimeout = 1500;
[Code] ....
This takes 128 seconds to put 13 million thin rows into my empty destination table and then throws an exception with this message:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
View 5 Replies
View Related
Jun 12, 2004
In my project, I need to transfer the database from dBase to MS SQL 2000. I only have the .dbf files for the dBase database. Can someone give me a help? Thanks in advance.
View 1 Replies
View Related
Aug 30, 2006
Hi,
I want to access dbase database, can anyone help?
I try :
SELECT * FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Data Source=D:Account;Extended Properties=DBASE III;',
TRANS.DBF)
and I got the following message :
OLE DB provider "Microsoft.Jet.OLEDB.4.0" for linked server "(null)" returned message "Could not find installable ISAM.".
Msg 7303, Level 16, State 1, Line 1
Cannot initialize the data source object of OLE DB provider "Microsoft.Jet.OLEDB.4.0" for linked server "(null)".
Thanks and I really appreciate any help
joe
View 3 Replies
View Related
Feb 4, 2008
HI, I have a Dbase Database stored in several subfolders. I allready made my connection with a sqldatasource and a gridview, and all works fine when i do a query with the tables in to root folder. The problem is when I try to include tables from the subfolders, the compiler send me a syntax error in the FROM Clausule
I Allready try this.
SELECT products.id, products.descri, condition.discount from products, clientscondition where products.line=condition.line
SELECT producst.id, products.descri, clientscondition.discount from products, clientscondition where products.line=condition.line
SELECT products.id, products.descri, condition.discount from products, condition in 'clients' where products.line=condition.line
SELECT producst.id, products.descri, clientscondition.discount from products, 'clients'.condition where products.line=condition.line
And many other different combinations. I would like to be helped if you know the correct syntax or a link where i can consult the syntax for VB.net and dbase with several folders
Another option a tried is having a query from 2 different datasources, but I don´t know how to make a reference to the 2 sqldatassources in a new one to be linked to the gridview
View 7 Replies
View Related
May 26, 2000
Has anyone had any experience using the openrowset function to access a dbase IV file? I can access the file using ADO from within VB but not getting anywhere using openrowset. If anyone knows the syntax or has an example it would be greatly appreciated.
Thanks in advance...
Jim
View 4 Replies
View Related
Feb 7, 2006
Hello a,,
I have a MS SQL database that I use MS ACESS to add data to and also to query.
My question is how can I setup the MS Access database so that it will not allow duplicate entries to be added to the MS SQL database? I know How I could do it using php via a webpage but I need to be able to do this using the MS Access program directly since thats what we use to add and search the dbase with.
thanks
View 5 Replies
View Related
Sep 13, 2004
I have some database files with .dbf from dBase and I want to transfer data from the .dbf files to Microsoft SQL database (2000). Can someone help me with that? Thanks.
View 4 Replies
View Related
Mar 23, 2006
Please anybody help me.
Can't get oledb for odbc work.Need an explanation what to put in each field of linked server properties General tab to get connected. Have ODBC server DSN working fine with Crystal reports.
Did this:
exec sp_addlinkedserver @server = 'ODBC_TEST' ,
@srvproduct = 'dBase',
@provider = 'MSDASQL',
@datasrc = 'DB4_GBS'
GO
-- Adding linked server login:
EXEC sp_addlinkedsrvlogin 'ODBC_TEST', 'false', 'sqlLogin', 'sqlpassword', NULL
Getting error : datasource name not found and no default driver specified.
I also attempt to create OLEDB ODBCless link and can't do it because after that I have to change path in TEMP and TMP variables and restart db, but I cann ot restart the server.
:eek:
View 10 Replies
View Related
Aug 28, 2007
I have been asked to look into converting a DBase II database on an AS400 to MS SQL Server 2000 or newer running on Windows 2003 or newer.
I know that it can be done but how hard will it be and how long could it take?
Are there any conversion packages out there that will do the majority of the leg work for us?
We also need to convert all of the screens written in RPG to a web based application. We want to get rid of the AS400.
Thanks
enak
View 5 Replies
View Related
Jul 20, 2005
I would like to see a Clipper/dbase DBF file as a table in SQL Server7.0. How can I SELECT rows from DBF file? Should I use OLE DB Provideror ODBC, and how?
View 10 Replies
View Related
Nov 7, 2007
Dear All,
Can some one help me to get or how to get an SQL SERVER dbase software cos have been trying to download bt i dnt get it right.
Urgent!!!!!!!!!!
OBALALU, Babatunde S.
View 5 Replies
View Related
Apr 16, 2007
I'm not sure if this is the right place for this but I have an issue.
I have a SQL2005 server with SAN drives for the MDF/LDF files. I have updated recently to SP2.
When I detach a Dbase using the appropriate command it strips the Security on the file level (which SP2 is supposed to fix) and I am unable to attach the database.
I am using a domain account to do this with from the query analyser window.
Anyone have any thoughts on this?
View 6 Replies
View Related
Aug 22, 2006
I am having trouble connecting dbf of version dbase II. I can connect dbase III and dbase IV in ssis package,but can't connect dbase II.hope your help!
View 3 Replies
View Related
Nov 21, 2006
Hello all...
I created several SSIS packages on a 32-bit system, using odbc to to access the dbase source file. Finally, after setting up a proxy, etc., we got the package to run and import data. We then needed to move the packages to a 64-bit system. So, we built our manifests, and installed all of our packages. We are using several dtsConfig files for our packages, and in there we specify the DSN that we will be accessing. I understand that MS does not support odbc drivers in a 64-bit world (http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=85703&SiteID=1). Has anyone been faced with this issue? How did you get past it? I accessed the 32-bit odbc drivers on the 64-bit system, and i setup a dsn that points to the dbase file, but when i run the job, the log file has an error that the dsn was not found. However, when I run the package itself, it runs as expected...
Please help!! :-)
Craig
View 1 Replies
View Related
Mar 4, 2004
You guys are probably going to tell me to go to school (training, and I'd like to)
My question is, I have a stored procedure in our MS_SQL 7 dbase that pukes out the info in a table with MSAccess. What I would like to do is, have a nice customized report be generated via http link. I designed and made such a report that queries the SQL sp_ with Crystal Report 8 I installed the browser plug in on a test workstation launched the report (via web) and I get prompted for access credentials (username & Password). Nothing works except the sa account in on the SQL server (and I am a sysadmin on that server),. I am giving that up in hopes to achieve this goal with a ASP page, but I really do not even know where to start. remember I have a storeed procedure that does the querying for me, so if there is a way to make "form that queries the sp_ maybe? ...I don't know really new at this.
View 1 Replies
View Related
Jun 5, 2000
Can i update a field in a table in 1 dbase based upon data in another table in another dbase.
could somebody let me know if this is possible.
Thanks
View 1 Replies
View Related
May 5, 2000
I Completely new to SQL, can anyone give me a URL to find this information.?
Thanks
View 1 Replies
View Related
Dec 25, 1999
Hi,
I have used the backup tool in MS Access 2000.ADP many times
To make a nice backup of my adp Projects server objects(tables & data) that
I have in my MS SQL Sever 7.0 dBase. and then have successfully restored them back to the same computer that I have backed them up to. .
But now I need to transfer server SQL 7.0 DAT backup files over to two other types of computer setups..
I need to transfer these files and import or restore them onto another
computer that has MS Access 2000 and SQL 7.0 on a NT Server 4.0 platform and
also to a computer that has MS Access 2000 and MSDE on a win98 platform..
Have tried many different things but cannot seem to find a way to transfer or restore the SQL 7.0 DAT files to a different named dbase NT server with SQL 7.0 or to a win98 client with a named dbase on a MSDE file. the ADP client portion is just a simple file transfer but these SQL Server 7.0 DAT backup files... argh!
I am completely stumped even after trying many things and consulting the SQL
server manual.
Christopher James
View 1 Replies
View Related
Mar 6, 2001
I've connect some .DBF file to SQL server 7 with linked server option. My table have about 200.000 records. I've tried with ODBC and Jet 4.0 option (found in www.mssqlserver.com) but I've very slow response if I use a "where" in the select instance.
In fact I've the response after 10 seconds and during this seconds the processor go to 100%.
Does anyone found a most speed way to do this?
Thanks,
Andrea
View 1 Replies
View Related
Apr 8, 2002
Hi,
I am stephen and am working with SQL server 2000 and .NET, i want to develop web pages(am just experimenting) using VB.NET and am using SQL server 2000 to retrieve data and publishing using IIS web server.
"the problem is when the webpage displays, it asks for authentication win NT but when i hit OK button. it displays the page."
i want to eliminate this windows NT pop-up box. is it the problem of IIS server security settings or SQL server settings?
any views or suggestions will be greatly appreciated.
stephen
View 1 Replies
View Related
Sep 18, 2001
I have a database in a restore mode. The SQL Enterprise Manager says the database is in a 'Loading' phase. How can this operation be cancelled?
Thanks,
Brian
View 1 Replies
View Related
Dec 29, 2006
besides using the costly EMS Data export application?
I'm using the SQL 2005 Express version......I heard the enterprise manager can do the job but I can't afford a standard edition yet....
View 11 Replies
View Related
Apr 28, 2004
I am trying to create a linked server to some dbase 5 tables. I have read several other posts about using the OPENROWSET option and that does work however, I would like to be able to create a linked server for easier access.
End Goal: I am developing an ASP based app that needs to join a DB5 table to a SQL table.
Any help on the linked server properties in Enterprise Manager for dbase5 or on the proper sp_addlinkedserver function for dbase5 would be highly appreciated.
Joe
View 2 Replies
View Related
Apr 17, 2006
I've followed the steps on how to use the ForEach loop container using a Flat Text file and it's working fine. I tried reading the text file and put it in my SQL table. However, I have some DBF files as well and the ForEach loop does not work at all with my DBF files.
Here is my scenario:
A connection manager pointed to C:DBF_FILES
On the Control Flow: a Foreach loop container with Foreach file enumerator. (checks for IFUL*.DBF files and retrieves the Fully qualified name.
Within the foreach loop container is my DataFlow (import)
Within the dataflow:
OLEDB datasource
data access mode: Table name or View Name variable
Variable nam: username::Filefound
Now from this point on, I'm already having problems, here is the error:
An OLEDB error has occured.
An OLEDB record is available. Source: "Microsoft OLE DB Provider for ODBC Drivers.
Data source name not found and no default driver specified.
Error at data flow task...
There is also some error that say i don't have a destination table specified. Plus the error is on the OLEDB datasource not having a destination table? I don't understand this part.
Please help!
View 22 Replies
View Related
Feb 4, 2007
I followed the steps outlined in
http://msdn2.microsoft.com/en-us/library/aa337084.aspx
and was successful at importing ".dbf" files from a dBase 7.0 folder.
However, this import was quite slow: approx. 2000 rows per second. Some of the tables I must import have over 8,000,000 rows, which would then take 4,000 seconds, or over one hour. This import rate seems unaffected by whether or not I put a "Copy Column" Data Flow Transformation task between the "OLE DB Source" and "OLE DB Destination" objects.
In SQL Server 2000 it was much faster to import such large tables. The 8 million row table takes only 14 minutes (800 seconds) with SQL Server 2000 -- giving a transfer rate approx. 5 times as fast.
Is there any way to speed up this process in SQL Server 2005? (In SQL Server 2000 there was a "Use fast load" option on the "Options" tab of the "Transform Data Task Properties." I have not yet found any similar option for SQL Server 2005.)
My receiving database is using the "Simple" model, so there is no need to create transaction entries for loading these large tables. There are also no indexes on these tables: the data are being loaded into new tables created in SSIS.
Thanks for your help with this issue.
View 3 Replies
View Related
Aug 24, 2007
Due to some legacy requirements, I'm using an OLEDB connection with the Jet driver and creating dBASE III tables (.DBFs) as the destination for a data export. The source is SQL Server 2000. The SQL table has a bunch of varchar columns, and I can't send them directly to the .DBF because SSIS complains that it can't (implicitly) convert from non-Unicode to Unicode. UNICODE?! dBASE III is ancient. Why would Jet/SSIS assume that these .DBFs are Unicode? As it stands, I have to put a data conversion task between the source and destination and convert all the columns. It's a real pain.
And no, I don't want to make all my SQL Server columns nvarchar. Is there any setting I can put in the connection manager or the connection string to prevent this error? I've already set AlwaysUseDefaultCodePage to True on the OLEDB destination component and the default code page is 1252. That didn't work; I still got the can't convert error.
View 3 Replies
View Related
May 4, 2006
I am trying to get a handle on the new SQL Server Integration Services in SQL Server 2005. There is a legacy DTS package that I need to get working on our new server (using SQL Server 2005) but I can't seem to get it to work.First, I migrated it and attempted to run the package. It gave me many errors. So, I thought I would just rebuild it in integrated services. Everything was going smooth until I got to the real data transformation part.I added a data flow task and it took me to the Data Flow tab. Then I added a DataReader Source. I am trying to read from a dBase III file and I assumed I would be using an ODBC driver for that. Well... it has been hell trying to get it to see the dBase III file and access it.I have tried two solutions from the internet.This one and this one (by Wenyang) with no positive results.No matter what I do I still get an error in the bottom of the Advanced Editor for DataReader Source that reads: "Error at Sales Transformation [DataReader Source [81]]: Cannot acquire a managed connection from the run-time connection manager."Has anyone attempted anything like this before? Are there any SSIS experts around here that can drop me a few pearls of wisdom?Thanks in advance... I'm dying here.
View 3 Replies
View Related
Oct 12, 2000
Hello,
I need a sql server 7 script that will migrate data from
my DEV dbase to my WEB dbase. The data desired is for the field
rig_id for rig number 0018. The shell of both dbases are similar.
The table is called rig. An snapshot is below:
DBASE name - DEV
TABLE name - rig
customer rig_id amount
Williams 0011 328
Borsha 0013 256
Kaila 0018 411
Yoida 0021 347
The rig table also exists on the WEB dbase. However, the Kaila data
for rig_id number 0018 needs to be added to the WEB dbase's rig table.
The Williams, Borsha, and Yoida data already exists on both tables.
I need to perform this migration w/ a TRANSACT-SQL stored procedure.
Thanks Very Much IN Advance,
Dave
gunnardl@yahoo.om
View 1 Replies
View Related
Sep 27, 2000
Hello,
I need help in generating a stored proc script (SQL Server 7.0)that will:
document the changes of each field in a dbase (SQL Server 7.0)
appending the deltas (changes) to the dbase on a different, say production server for each affected field
prompt the user (dba, developer, super user) for the fields to involve in the comparing and documenting of changes (deltas).
thanks in advance for your help!
ALSO: Does SQL Server 2000 handle this situation better than SQL Server 7.0
Gunnar
View 1 Replies
View Related
Feb 2, 2005
Hi,
I am trying to use linked tables to connect SQL Server 2000 to a legacy system using dbase III files. (I need real time read only access to these files)
I have created a linked table from SQL server to a folder on the C drive which contains the dbase III files, using an ODBC DSN which uses "Microsoft dbase driver (*.dbf)". DSN tested successfully using Excel. Linked server connection is then created using "Microsoft OLE DB Provider for ODBC drivers".
The dbase tables appear OK in Enterprise manager, but I cannot get a query to work in SQL Anayzer, using the 4-part name syntax.
My query is just :
SELECT * FROM LinkedTable...customers
Error message is "Invalid schema or catalog specified for provider 'MSDASQL'". Now I am pretty sure dbase files do not support any sort of schema / catalog set up, so I suspect SQL Server is looking for something it is not going to get.
One clue might be that in Enterprise manager, under the catalog column, I get the pathname to the dbase file, ie c:customers.dbf, which I cannot enter in the 4-part syntax.
Any suggestion welcome !!!
View 2 Replies
View Related
Apr 4, 2008
how to insert data from 1 database table to another database tables
pls help me out
i m very much thank ful to u
Yaman
View 5 Replies
View Related
