Best Practice To Copy A Lot Of Data Between Databases
Oct 30, 2007
Hi,
I have a database on sqlserver 2000 which contains some tables which receive Create/Read/Update statements from applications, and +-5 tables which contain a sort of read-only data: the applications are only reading from these tables.
Sometimes these 5 tables need to be updated with new data. Currently I am doing this as follows:
Execute a long-running operation (e.g. 1 week), which will add new data, against a Test-database (which is a restore from a backup of the original database)
Some people do some checks to see if the new data in the Test-database is correct
If (2) is OK, the only thing which needs to be done is copying the data of the 5 tables in testdatabase to the 5 tables of the production-database.For (3), I currently use a DTS-package which consists of a "Copy SQL Server Objects"-task. This task is configured to copy the 5 tables (objects) from testdatabase to productiondatabase. The data in these 5 tables is around 20GB, and this task takes a lot of time compared to a backup/restore of the same size of data. I already tried to speed it up by creating different filegroups and wanted to restore only 1 filegroup but you can read here that making a filegroup backup on a testdatabase and trying to restore it on a production database won't work. Is there some other way to speed this up? Is my current way of working good practice?
Thx!
View 1 Replies
ADVERTISEMENT
Jan 21, 2005
I am trying to copy information from one Database to another. Each of the databases reside on a different server.
My question is
1. How can I use the DSN name in a stored procedure?
2. Do I need to use dynamic SQL?
3. Any other ideas of how to implement this?
Thanks
View 1 Replies
View Related
Jun 3, 2001
Copy objects and data between SQL Server databases
"
Display the Select Objects to Transfer dialog box, where you can specify both objects and data to copy, if both the data source and destination are Microsoft® SQL Server™ databases. The objects you can transfer include tables, views, stored procedures, defaults, rules, constraints, user-defined data types, logins, users, roles, and indexes. You can transfer objects only between multiple instances of SQL Server version 7.0, from an instance of SQL Server 7.0 to an instance of SQL Server 2000, and between multiple instances of SQL Server 2000.
"
can I apply "Copy objects and data between SQL Server databases"
to run in two different sqlserver 2000 ( not an instance ) . what I mean is I have two different sql servers located in two different locations( I am not using an instance installation) can I still run the copy and get an identical database in both servers.
Q2. if I have two sql server 2000 with different collations (one is binary and the other is the default) will I be able to run the copy wizard and still have an identical copy of sql server in both servers.
I personally tried to run the copy wizard and IT NEVER WORKED FOR ME and I really do not know the reason.
Thanks for your input.
ali
View 2 Replies
View Related
Dec 5, 2006
Hello,
In our company we haven't tamed Visual Studio yet for the part of configuring
the application database. Therefore I was wondering if there is someone that
can give me hints or clues on what the best practice is for configuring Visual
Solution to handle an SDF database (SQLCE 3.0). I have search a bit on the
internet and this forum, but couldn't find a decent how-to or best practices
guide that copes with this specific problem.
As
a matter of facts, i am also curious for guidelines and tips on the best way to
configure and manage SQL Server databases under Visual Studio. I guess (read as
hope) that the configuration of a mobile and full blown
database will have some overlaps somehow.
I've found the Database Project which with to manage (Create/Alter) databases,
but is this also the best way to store versions of the database in an
source repository? Can such a project allow me to 'automagically' have a
correct database (so with tables and data) when i want to deploy or debug my
device application?
I am asking this because now our database designers and software engineers have
to do allot of manual actions to update the application with the most recent
database version. SDF databases are flying around all over the place, and in
order to test a specific version of the application, the related database has
to be copied manually on the device. We are not searching for replication
related solutions, nor adding the SDF itself to the source repository, but because
most of our applications are server-client based, it would be really super cool
if we could somehow couple both database definitions and generation together. My
feeling says me there ought to be some feature embedded somewhere deep into
Visual Studio that we missed that would (partially) simplify and automate this
whole process for us.
Another
related question is if it is possible to couple XSD generated datasets to an
SDF database. Is it possible to update the generated code (or the describing
XSD documents) from an SDF database? i again guess that this somehow should be
possible in oderder to keep both code and data in sync, and i do not like the
alternative to always update the XSD when the SDF architecture changes. Somehow
i cannot find out how to do this. i was initially searching for the other way
around: Use an XSD to generate both the DataSet codewrappers and the database
itself.
Any
information is welcome and many thanks in advance!
Peter
Vrenken
View 7 Replies
View Related
Jan 23, 2012
I am attempting to use the copy wizard to copy databases from SQL Server 2005 to SQL Server 2008 R2 w/ FP1.
The copy fails with a login failure to SQL Server 2005. I have a user id & password under Windows for both servers. I have a user id and password under SQL security with the called for admin security rights.
The 2005 server has two instances, 20 databases, two dozen maintenance plans, and over a hundred users. I really would like to use the utility so I don't have to recreate everything manually.
View 1 Replies
View Related
May 13, 2008
Hi, I am going to copy databases from my office's computer to my home's machine.
What is the best way?
I need the detailed steps.
Thanks
View 1 Replies
View Related
May 30, 2006
Hi,
Maybe a stupid question, but I'm trying to copy some databases from one
server to another. The copy databases wizard says the job is successful
and I can see that the job has been done on the remote server. But the
copied databases are not there. What do I do?
View 10 Replies
View Related
Jun 17, 2002
Hi,
I am preparing to move from sql7 to sql2000 and and considering the copy database wizard to move the databases. The thing that I am concerned with is this passage from BOL
"A database with the identical name on both source and destination servers cannot be moved or copied"
This to me says that the database name cannot be the same on source and destination servers. Yet, you would want the database name to be the same from an application point of view (developers get real nasty when you change database information on them).
Does this mean I cannot use the copy database wizard? Must I use sp_attatch_db?
Thanks in advance
View 2 Replies
View Related
May 16, 2007
Hi All,
I am new to sql server and the database concepts and just started learning.
I want to copy a database and local package from a old sql server to a new server.
can anybody guide me with the steps? I am very new to this field so if you could give me detailed steps
i would very much appreciate it.
thanks in advance,
View 9 Replies
View Related
Aug 14, 2007
Could anyone possibly help me out with an issue I am having...
I need to copy all my SQL2005 databases from Server1 to Server2. How can I, using SSIS, copy all the databases and not just 1 specific one (database transfer task)???
My actual task is much deeper than this, but this is the main problem I seem to be encountering and I cannot find this anywhere on the web...
Over and above the what I have mentioned, is there a way to also specify a name of the target DB the copy will go through to?
e.g. Server 1 has 3 instances, each one with a different copy of the same DB.
I would like to copy all 3 copies to Server 2 running 1 SQL instance, but to target DB's like DB_dev, DB_prod and DB_test.
The latter section is not so crucial at the moment, but the first part is really an issues I am struggling with at the moment...
Any help would be greatly appreciated.
Thanking You in advance!!!!!
View 1 Replies
View Related
Oct 17, 2006
Hello and thanks in advance for help,
I have configured several jobs in the SQL Server Agent which do a copy of databases from a cluster of SQL 2005 to another SQL 2005 server. I did these from the "Copy Dabatase Wizard". When I run these it works fine with "small" databases but with the two bigger ones I get the following error after 10 minutes since execution started:
Event Name: OnError
Message: An exception occurred while executing a Transact-SQL statement or batch.
StackTrace: at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(String sqlCommand)
at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(StringCollection sqlCommands)
at Microsoft.SqlServer.Dts.Tasks.TransferObjectsTask.TransferObjectsTask.TransferDatabasesUsingSMOTransfer()
InnerException-->Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
StackTrace: at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject stateObj, UInt32 error)
at System.Data.SqlClient.TdsParserStateObject.ReadSni(DbAsyncResult asyncResult, TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParserStateObject.ReadPacket(Int32 bytesExpected)
at System.Data.SqlClient.TdsParserStateObject.ReadBuffer()
at System.Data.SqlClient.TdsParserStateObject.ReadByte()
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet)
at Microsoft.SqlServer.Management.Common.ServerConnection.ExecuteWithResults(String sqlCommand)
Operator: DOMAINuser
Source Name: CLUSTERSERVER_BACKUPSERVER_Transfer Objects Task
Source ID: {3CF47485-1035-40E7-86C4-D679E253D38D}
Execution ID: {A10C4E99-0B35-416C-A560-81DB2C2D92C4}
Start Time: 11/10/2006 17:45:11
End Time: 11/10/2006 17:45:11
Data Code: 0
Any idea about the solution?
Regards,
Jorge
View 5 Replies
View Related
Feb 26, 2008
Hi, i'm wondering which is the best way to search data in a SQL Server.
I reach data using Data Sources and Data Views and also with OLE DB Source with a Data access mode: Named query.
I have to write the data into a Flat File. So, does any one knows which is the best practice for this? Or any one of the two are good choices?
Thanks for your help.
Beli
View 6 Replies
View Related
Jul 8, 2005
I need to learn how to move or copy a couple of tables from one database to another. The tables are defined but contain no data.
View 2 Replies
View Related
Jul 20, 2005
I am trying to find the best way to copy specific tables from one databaseto another when the source and target database names are not always thesame. Can you use variables to specify (or prompt the user) to providesource and target databases? The target database will exist with the thesame tables as the source. The tables to copy will always be the same.Example:UserA wants to copy 10 tables from Data1 to Data2UserB wants to copy 10 tables from Data4 to Data5I'm sure a script can do this in Query Analyzer but is there a more userfriendly method when the user has ony standard SQL tools?Thanks in advance.
View 1 Replies
View Related
Apr 26, 2006
I am developing an application that uses SQL Server Express. Everything is running great, except that I can't figure out how to copy a DataBase from my server at work onto my laptop so that I may test it at home...
How can I copy the DataBase from my Server at work to my laptop? I'm on the network at work, I just can't figure out how to copy it...
Please help! :)
Thanks,
Jacob
View 4 Replies
View Related
Nov 6, 2006
Two Windows 2003 server,one with SQL 2005 server,another with SQL Express.Is it possible to copy databases from SQL 2005 to SQL Express?Thanks.
View 6 Replies
View Related
Feb 27, 2006
I'm just wondering if it is bad programming practice to have many data sources on one page. I need a "Matrix" type of page that grabs data from many tables and so far I have 5 data sources. Is this common or is there something I should be doing to make it more efficient?
View 1 Replies
View Related
Oct 17, 2005
Here is the situation:Tables:AccountAccountDocumentAccountTestThen we have the following "Activity" table refencing "Account" tableabove:Table: ActivityColumns: ActivityID, AccountNo,...when "Activity" with accountno is created, we'd like to able to takesnapshop or copy all Account related data records from tables: account,accountdocument, accounttestand store it somewhere.This way even if account, accountdocument and accounttest tables changelater, then "Activity" accountno integrity in maintained.Do we need to create or replicate similar tables to store data foraudit?or Which other ways can this be achieved?
View 3 Replies
View Related
May 1, 2015
I was contacted by the SAN team to test backup/restore of larger databases using a split-mirror backup (BCV) or clone that is taken from production db server and copied to another sql box. They want to use this process once a week. I see the mounted drives with the data/log files. All looks good. Initially I attempted to attach the databases and received (Unable to open the physical file db.mdf Operating System Error 5 Access is denied). I manually granting SQLServerMSSQLUser$<computer_name>$<instance_name> on all of the physical files 20 total. That worked.
Since this will be weekly, the SAN team performed the copy again and now none of the databases can communicate with the newly copied files. NTFS permissions need to be set again. I'm getting (Operating System error 21: the device is not ready). Is there something that I'm missing in this process how the vendor BCV clones the data and SQL communicates with the copied files as I was thinking it would be more automated process?
View 0 Replies
View Related
Sep 1, 2006
Hi ,all here,
Would please anyone here give me any guidance and advices for best practice of data dictionary in SQL Server 2005?
I have restored a large insurance claims database with up to more than 300 tables, massively, most of them are empty tables, many of them dont have any keys, contrains, indexes, and more difficult, there is no any data dictionary for the database which gets me stuck in the understanding of the data at the moment. Thus I think data dictionary is very important for a database.
Will anyone gives me advices for that? Really need help.
Thank you very much in advance for any help.
With best regards,
Yours sincerely,
View 11 Replies
View Related
Feb 9, 2007
We are working on converting to SQL 2005 database. During the conversion we are having to rewrite a lot of code and doing a lot of intital testing and development on development data. This is causing our transaction logs to really big. I have created a maint plan that runs nightly that does a back up of database and tran log but throughtout the day the tran logs are getting really big and eating up a ton of disk space. Does anyone have suggestions on what sort of maint plan I can setup to run on my developement data where as at this point I am not concened about being able to roll back the database just keep is small as possible and "healthly"
All ideas are appreciated
Thanks
Chris
View 5 Replies
View Related
May 7, 2007
Hallo I need some hints because I would like to set up my service broker in the proper way.
First: I'm going to setup service Broker between 2 databases on the same sql server instance.
My Goal: I insert the data on table1 in the DB1.On table1 there is a trigger that begin conversation and send the message to the service on the DB2.
On the receiving queue there is an Activation that take the xml message, shred it and save the content on th table 2 on the DB2.
Actually this SP is my main concern because the shred and insert involves also several checks so it could take "a while".
Volume of data: it seems that there are 100-200 daily insert in the table1 on the DB1 and it is possible that some of them arrives at the "same time".
Very quickly:
On the DB1 the trigger:
Notice that I reuse always the same conversation and the send queue as RETENTION = OFF
begin transaction;
begin
set @dialog_handle = (select conversation_handle from sys.conversation_endpoints where far_service='ReceiveService');
if @dialog_handle is null
BEGIN
BEGIN DIALOG CONVERSATION @dialog_handle
FROM SERVICE [SendService]
TO SERVICE 'ReceiveService'
ON CONTRACT [MainContract]
WITH ENCRYPTION = OFF;
END
SEND ON CONVERSATION @dialog_handle
MESSAGE TYPE Message ('uyiuy')
commit;
END
On the DB2
ReceiveQueue is defined as:
CREATE QUEUE [dbo].[ReceiveQueue] WITH STATUS = ON , RETENTION = OFF , ACTIVATION ( STATUS = ON , PROCEDURE_NAME = [dbo].[OnReceivedTrade] , MAX_QUEUE_READERS = 100 , EXECUTE AS N'dbo' ) ON [PRIMARY]
But most important is the Activation's SP and this is the main code:
while (1 = 1)
begin
begin transaction
-- Receive the next available message from the queue
WAITFOR (
RECEIVE top(1)
@message_type=message_type_id,
@message_body=message_body,
@dialog = conversation_handle
FROM ReceivedQueue
)
if (@@ROWCOUNT = 0)
BEGIN
Rollback Transaction
BREAK
END
SET @ErrorSave = @@ERROR ;
IF (@ErrorSave <> 0)
BEGIN
ROLLBACK TRANSACTION ;
SET @ErrorDesc = N'An error has occurred.' ;
END CONVERSATION @dialog
WITH ERROR = @ErrorSave DESCRIPTION = @ErrorDesc ;
INSERT INTO [dbo].[tblLog] VALUES(@ErrorDesc,NULL)
END
ELSE
BEGIN
'this is the SP that does most of the work
exec [dbo].[sp_ShredXMLMessageToRelationalData] @message_body
END
commit transaction
I have to say that everything works fine but I don't think that it is completly performant.
I read several blogs/forums and now I'm confused on the best way to implement service broker messaging:
These are my questions:
1) I read that is the DB's are in the same instance It can improve performances the send the message directly on the Queue2.
Can be useful in my case? How to implement it?
2)In my case it is necessarly that I send a message back to the initiator?
3) In my case the conversation never end. I don't think that this is correct but in case of the data push who has to END the conversation? The target? the initiator? never end to avoid overhead (I read that big overhead is caused by the BEGIN and END DIALOG)?
3) where (initiator or Target) and how to handle the service broker error? In my case my applications are SPs and I need to inform the developer or DBA that something went wrong during the processing ( conversation or shredding stored procedure).
4) In my case is should be a good idea to think about how to solve the possibility of the "poison messages"
for sure I will find out other questions...
Any hint, link is appreciated!
Thank you very much!
Marina B.
View 17 Replies
View Related
Dec 11, 2007
Hi All,
I am using SQL Server 2000 Enterprise Edition fully patched. Database is in Simple Recovery mode.
I need to change a column's data type from "int(4)" to "smallint(2)". I know for sure that there will be no data (precision) lost, because I know the possible values that this column could have.
My problem is that the table I am dealing with has 600,000,000 records in it. I dropped all indexes before I tried to alter the table. But still it is taking forever and filling up my 280GB disk with transaction log file.
I know that in Oracle, if I want I can turn off logging and do these kind of modifications relatively faster.
I was wondering if there is a way of disabling logging before running this alter command.
What is the best practice to handle a situation of this sort?
I appreciate your help.
Thanks in advance,
Sinan Topuz
View 3 Replies
View Related
Mar 1, 2007
Here is my idea but I am looking for the best practice.
Each record can have 3 possibilites.
I would read and write the data 3 times to different tables and add an identity key on all 3 files then I would reassembe the data back together on the identity key and map the data to a fourth and final table.
Any other ideas?
Thanks,
Larry
View 2 Replies
View Related
Sep 19, 2007
I dont alot about sql server 2005(Express edition). For debugging purposes i want to copy the whole app_data folder(.mdf & .log files) on the production server to another folder on the same machine(or sometimes to a network folder). So when i copy and try to paste this App_data folder to a new location, i get this error message
"cannot copy ASPNETDB: it is being used by another person or program. close any programs that might be using the file and try again."
After reading the above message, i close visual web developer, stop the website in IIS and stop the SQLExpress service on the server and try again but still get the same message.
So how can i make sure that all the programs accessing these database files are closed such that i'm able able to copy them to a different location.
View 2 Replies
View Related
May 3, 2007
Hi There,
I need to pull data using input from one table in sql server 2005. I have to query against the sql server 2000 database and pull data into sql server 2005. I have a list of ids that I have to pass to a query to get the desired data. What is the best practice for this. Can I use SSIS or do I need to build an app in C#? Can somebody please reply back?
Thanks a lot!!
View 4 Replies
View Related
Dec 10, 2005
Hi,I have(had) an old Win2k Server server with about 30 web site databases(SQL 2000) that just went under due to hardware problems. Thankfully, Ihave backups of all the databases plus the MDF and LDF files from thehard drive.I want to move all of these sites and their data to a newer server(Win2003) running SQL2000.What's the best way to copy the database from the old server hard drive(now mounted as an extrnal drive to a local machine; I'm currentlyFTPing all of the web site directories from it to the new server)?Just upload the original data to the new server and then mount the MDFand LDF files within the new SQL server? Or do I restore the backupfiles in the new SQL2000?All of my previous data migrations have been DTS operations from onelive server to another, so no experience with either of the abovescenarios. I'll certainly have a lot more experience at one of them bythe time this weekend is through.Thanks for any help you can offer.
View 1 Replies
View Related
Apr 1, 2008
I'm in the middle of developing a Database for a hospital that measures its audits, inhouse operations, and finance. What we currently have and do everyday is collect data from a large database that is real time with patient data, progess, infomation, etc and dump it into a data warehouse that runs on TSI/Eclipsys. We run reports using a number of programs and dump it into Excel sheets that have charts, reports, etc. This Database for which I'm developing won't come soley from the TSI/Eclypsys source, but this is the only source thats updated regularly. I don't want to have in sync with TSI/Eclipysys in fear that every day when it updates data may be lost, not read, or worse won't be up date if there is a problem. My question is is it possible to run a query from Sequel Server 2005 that will take that data upon request using the reporing features on Sequel Server 2005. i.e. What if I need to run a report on measure B in department 12 from Jan 1-Feb 1, instead of being in sync, can I just write queries to take that information rather than double the data and take up twice the space and trouble. FYI, these datatypes rarely change in the TSI/Eclipsys data warehouse. This sure was long question and didn't intended it to be . Thanks for listening and hope to hear back.
Stephen
View 1 Replies
View Related
Jul 23, 2005
Just wanted to know what is a general rule of thumb when determining log file space against a database's data file.We allow our data file for our database to grow 10%, unlimited. We do not allow our log file to autogrow due to a specific and poorly written process (which we are in a three month process of remove) that can balloon the log file size.Should it be 10% of the Data file, i.e. if the Date file size is 800MB the log file should be 8MB?I realize there are a myraid of factors that go against file size but a general starting point would be nice.ThanksJeff--Message posted via http://www.sqlmonster.com
View 4 Replies
View Related
Aug 12, 2006
Can anyone provide an expample of bulk copying XML data to a SQL table. I am also looking at using column mapping so that I can map fields and also insert a new GUID into the key of the SQL table.
Many thanks
View 1 Replies
View Related
Dec 5, 2005
i have a claims_tbl that i am inserting into with claim information, the tbl contains a claim_seq number that i want to copy over to a user_info_tbl at the same time insert user information below is what i have tried with no luck
insert into user_info(claim_seq) select Max(claim_seq) from claim_tbl
update user_info set lname =upper(@lname), fname =upper(@fname), mname =upper(@mname), personnel = @personnel, p_position =upper(@position)where claim_seq = (Select Max(claim_seq) from user_info)
i am getting duplicate rows of information ..........can someone help
thanks
View 2 Replies
View Related
Jul 25, 2001
Hi!
What is the best way to copy data from SQL server to Access database located on differet machines?
Thank you,
Elena.
View 1 Replies
View Related
Sep 11, 2006
Hello,
there is a database with an table x and I want a copy of that in my testdatabase.
I am pretty sure, that I have done that the whole last week, but it's not working anymore.
I thinkt, that last time I used a statement like that to copy the table with structure and all.
insert into dbtest.dbo.x select * from db.dbo.x
Today it's only telling me, that the destinationtabel x is not present.
The table has about 100 fields and I swear, I didnt create it by hand in the testdatabase, I'm definitely too lazy for that.
I can't use the enterprise manager, I only can use sql.
There are still tables in my testdatabase, I copied last week. So what is wrong with my insert into?
View 1 Replies
View Related
