Copy Large Database
May 17, 2004
Does anyone know of a good way to copy a large database or backup of a database (~10 GB) across long distances (transatlantic) in a minimal amount of time?
I have tried regular file copies, BCP, DTC, my own handwritten extraction utility, PCAnywhere, and several file splitters, all to no avail. To complicate things, the source is SQL 7 with a different code page from the destination, which is SQL 2K. The file copies always seem to get interrupted after several hours.
View 6 Replies
ADVERTISEMENT
Sep 20, 2015
We have a situation that requires converting all INT columns from source database to BIGINT in destination.
By ‘copying’ term, I consider any operation like insert-select, merge, asynchronous replication or any other that will get all data synchronized until some moment in time, i.e. 2015-12-31 10:00 PST when downtime is planned. Before that moment it isn’t important if data in source and destination DB are in sync.
Copying process starts early enough to allow appliance of any technique/technology – there’s enough time to find best solution.
Requirements:
1. Destination database is superset of source, having all tables and all columns plus few more columns in few tables. Additional columns have default NULL or they are computed. INT columns in source are BIGINT in destination.
2. If source and destination are not in sync at the moment downtime begins, it is allowed to have few minutes to wait for synchronization. For instance, if transaction logs from last hour or so need to be shipped, that would be ok. If large table need to be synchronized by comparing row-by-row, it’s not ok.
3. Source database performance must not be compromised by any copying if it’s done before scheduled downtime. There are couple of options I have in mind involving MERGE command but its appliance might be limited by current DB design. Additionally, based on business logic, I cannot be absolutely sure if some row, created while ago, wasn’t changed meanwhile.
For some large tables, there is no indicator at all when any row has changed – I cannot rely on IDs nor there is column like ‘[date last changed] DATETIME’
I have been reading about various kinds of replication, i.e. log shipping but I’m not clear if any of these is applicable. Key points, if there is replication involved, are copying process must start from empty destination database plus synchronization must not compromise source DB performance So, how to do this using any kind of replication?
View 17 Replies
View Related
Jul 16, 2015
I have run the select query which returned one row. There is one column in it which has got large amount of data. I want to copy the complete content of that column(varchar(max)), but I am unable to do it. It's not the xml data. I don't want to do any conversions.
View 1 Replies
View Related
May 11, 2006
Hi,
I set up DB mirror between a primary (SQL1) and a mirror (SQL2); no witness. I have a problem when I issue command:
alter database DBmirrorTest
Set Partner = N'TCP://SQL2.mycom.com:5022';
go
The error message is:
The remote copy of database "DBmirrorTest" has not been rolled forward to a point in time that is encompassed in the local copy of the database log.
I have the steps below prior to the command. (Note that both servers' service accounts use the same domain account. The domain account I login to do db mirror setup is a member of the local admin group.)
1. backup database DBmirrorTest on SQL1
2. backup database log
3. copy db and log backup files to SQL2
4. restore db with norecovery
5. restore log with norecovery
6. create endpoints on both SQL1 and SQL2
CREATE ENDPOINT [Mirroring]
STATE=STARTED
AS TCP (LISTENER_PORT = 5022, LISTENER_IP = ALL)
FOR DATA_MIRRORING (ROLE = PARTNER)
7. enable mirror on mirror server SQL2
:connect SQL2
alter database DBmirrorTest
Set Partner = N'TCP://SQL1.mycom.com:5022';
go
8. Enable mirror on primary server SQL1
:connect SQL1
alter database DBmirrorTest
Set Partner = N'TCP://SQL2.mycom.com:5022';
go
This is where I got the error.
The remote copy of database "DBmirrorTest" has not been rolled forward to a point in time that is encompassed in the local copy
Thanks for any help,
KT
View 8 Replies
View Related
Nov 4, 2006
Hello,
if i have a given database (a model) and i want to copy this database in the same database instance. Is it ok to copy the mdf and ldf file and attach the files with a new database name in the same instance.
Or is the datebase name part of the .mdf file?
Regards
Markus
View 6 Replies
View Related
Oct 28, 2006
Hello I am a software developer with minimal SQL server administration skills. Currently I am using SQL Server 2000.I need to know if there is a way to copy a particular table from a database, and to copy the table into a different database.Basically on a project I am working on we are using a table named "Customers" from a database named QTR. We need to copy this database table into a different database named "Research". How can this be done? Is if very complicated?
View 1 Replies
View Related
Apr 23, 2007
guys,I have a project need to move more than 100,000 records from onedatabase table to another database table every week. Currently, usersinput date range from web UI, my store procedure will take those dateranges to INSERT records to a table in another database, then deletethe records, but it will take really long time to finish this action(up to 1 or 2 hours).My question is if there is some other way I should do to speed up theaction, I am thinking about use bcp to copy those records to datafileand then use bcp to insert it into SQL Server table. Is this the rightway to do it or should I consider other solution (then, what is thesolution.)Thanks a lot!
View 8 Replies
View Related
Jul 23, 2007
What happens if you have a website and the hard drive on your server is say 250GB. Then the database exceeds that and the database is 300GB in size.How would you spread your database into two different hard drives?Thanks
View 2 Replies
View Related
Apr 17, 2003
Hello,
I would like to know whether MS SQL Server 2000 Can handle Large Database size 28 gig, currently we are using Sybase but we want migrate to SQL Server 2000. Are there any performance issues?
Any help is appreciated.
Thanks
Sejal
View 1 Replies
View Related
Aug 17, 2006
We have many installations of our shopping cart database. One specific database is huge, now about 25 GIG, compared to the others that range from 20 to 75 MEG. The server this one resides on has three other instances of the same database that are normal size.
In a particular table in the large database there are 9700 rows taking 380MB, the same table in a normal db has 162000 rows and takes 6.MB. The tables are identical and the indexes are the same.
Any ideas out there?
View 6 Replies
View Related
Sep 12, 2006
I upgraded a database from SQL2000 to SQL2005 and it went pretty smooth. After the transition was made, I backed up the DB. The size of the database was as expected, about 5 GB. About 5 hours later, a maintenance plan executed a few optimization jobs in the following order: Reorganize Index, Rebuild Index, Shrink Database, Update Statistics. Soon after that, another job backed up the database and it was then 32 GB. During the time after the first backup, no one was really using the database.
I've been trying to track this down for several days. Does anyone have any ideas for me?
Thanks,
Bobby Crawford
View 1 Replies
View Related
Dec 4, 2006
Hello!
I have a database with one table inside. The table has six columns with the following datatypes:
col1 -->smallint (2byte)
col2 -->int (4byte)
col3 -->smallint (2byte)
col4 -->smallint (2byte)
col5 -->smallint (2byte)
col6 -->int (4byte)
I have insert 1.359.320 rows with data and the size of the sdf file is 40.116.224 byte.
According to my calculation:
1.359.320 * 2byte + 1.359.320 * 4byte + 1.359.320 * 2byte + 1.359.320 * 2byte +1.359.320 * 2byte + 1.359.320 * 4byte = 21.749.120 byte
I hope somebody can explain me, why the database is so large.
Thank you
Sascha
View 4 Replies
View Related
Sep 24, 2007
Hi guys,
I've been assigned the task of setting up access to our SQL Server 2005 box. A consultant developing for us has accessing to 2 databases and I've set this up fine. It appears however that one of these databases is re-copied over to the server every night to keep data reasonably current.
I'm not interesting in changing this method as I'm not the maintainer (as yet).
Basically I would like to know if I've setup access to this database (it works fine), when the database is updated (with an SSIS package) the account seems to get deleted. Do the original permissions from the source database overwrite those of its destination?
Cheers
View 1 Replies
View Related
Oct 8, 2004
Importing data from an Access database, I cannot overcome the limit of 1,000 records.
In DTS, I "copy one or more tables", select tables, run, and cannot see my 1,052 entries.
Where can I set a max size of ~1,500 in my sql target base?
View 1 Replies
View Related
Jun 15, 2000
Hi Everybody:
We now host 200 G data in SQL Server 7.0 in MSCS. The data will be 1 terabyte at end of the year. The possible solution to host large data with good performance is to migrate data to Oracle or UDB in Unix platform. But we would like to try any possibility in Microsoft world before we decide to migrate data to the other databases.
I need your experience and recommendation about hosting large database in SQL Server with good web performance. Somebody mentions Microsoft tera server. Who has experience using tera server which is still in the beta release? I would like to hear some successful story to handle the large data and understand precisely hardware, software, performance moitoring requirement.
Thank you very much.
Rich
View 2 Replies
View Related
Dec 14, 2006
Hi to All
I have large database with tables and views on the server around 80 to 100 mb amount of data.when i retrieve the data from the server it is taking long time Ex: "select * from vwcases" its taking 10 to 12 minuts time.this is my database problem.If you know the solution please reply back to me as soon as possible.
thanking You.
View 4 Replies
View Related
Mar 2, 2005
I need to query a database with about 3 million records using a 'like' statement on various varchar columns. I've have a non-clustered index on the columns included in my query, but the performance is still pretty slow. Can anyone suggest ways to increase the speed of my queries? Does full-text indexing improve performance or does it just facilitate more advanced searches (which I don't need)? Would char instead of varchar datatypes be faster?
thanks!
View 3 Replies
View Related
Jan 7, 2005
Hi Guys,
I am looking for a large datawarehouse database implemented in SQL Server to use it to run some examples in my thesis. if someone knows where I can find a large datawarehouse database (e.g Sales database) please tell me.
I appreciate any help.
Thanks
View 1 Replies
View Related
Jan 26, 2007
Hello all,
We have total 7 databases in the warehouse and these db are big. All the data files are approximately 300G now and they all growing more. Some are small db some are large db. (ranging from 20G to 50G of data files). Can someone please recommend me the best backup plan for this scenario. The regular backup through SQL 2005 takes too long.
Thanks for your help.
tocroi
View 1 Replies
View Related
Feb 24, 2006
I developed a vb app that imports csv data into an sql server db. The original text file is 36.5mb. The db after import is 230mb and the log file is 555mb. Is this normal?
View 8 Replies
View Related
Aug 11, 2007
I am developing an office automation software for a government department. To start with, I have decided to first automate the salary section of the department. Because of some issues like, TCO, easy support and maintenance, scalability etc., I have decided to use SQL Server 2005 database with VB 2005 front-end.
I have planned the layout for this first part and found that in only employee salary table, 40,000 records per month will be stored. That comes out to be around 480,000 records annually. This is for one table alone, excluding data in other tables.
Since each section of the department will be integrated into this later, this application will become the backbone the department. The employee service, leave, salary, allowance, deduction and other records shall be maintained.
For this type of mission-critical application, I want to get some queries cleared:
(1) What backup strategy should be followed? I want to have schedule and maual backups both. Is there any way to have a mirror image on another server?
(2) What considerations to keep in mind in the inital stage of database design?
(3) How to keep the design scalable and configurable to meet future needs?
View 6 Replies
View Related
Oct 23, 2006
Hi Experts
We are debating what is best:
1. To combine all the company's data in one large database, and use schemas and file groups to create logical and physical distribution on drives and namespaces
or
2. Distribute the data into smaller databases with related data - eg. products and product description in one db, Customers in another and orders and orderlines in a third db.
Just what are the pros and cons?
regards
Jens Chr
View 3 Replies
View Related
Nov 1, 2006
I'm using SQL Server Management Studio Express and I'm trying to figure out how to copy a table(s) from my local database to my web hosting database. I know how to do it in 2000, but it's completely different now. Is this feature not allowed on SSMSE? If so, then how do I deploy database tables to a web host?Also, how do you add local database(s) to SSMSE? I tried to use 'attach database' in SSMSE and it wouldn't allow me to navigate to My Documents folder where the database resides. Thanks...
View 8 Replies
View Related
Jun 21, 2007
Hello, All:
I have many, many Access databases that are roughly 1.5GB-3GBs each and they have millions of records. Each MS Access Database file corresponds to one Database in SQL server. I'm trying to simply transform the data as it is in Access to MS SQL 2005.
I'm using the 64 bit version of Windows Server 2003 and the 64 bit version of SQL 2005. The server is running four dual core AMD Operton processors and has 8GB of RAM with a 1TB RAID 5 configuration. I think the hardware should be sufficient but the SQL Server Import and Export Wizard can't seem to handle the large number of tables/records. If I do one table at a time, it works well; however, it produces the following error message whenever I try to import the entire database:
Pre-execute (Error)
Messages
Error 0xc0202009: {5A5BF7AD-E86B-4316-AD43-1912358C56F4}: An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft JET Database Engine" Hresult: 0x80004005 Description: "Unspecified error".
(SQL Server Import and Export Wizard)
Error 0xc020801c: Data Flow Task: The AcquireConnection method call to the connection manager "SourceConnectionOLEDB" failed with error code 0xC0202009.
(SQL Server Import and Export Wizard)
Error 0xc004701a: Data Flow Task: component "Source 64 - District Corporal Punishment Class" (5743) failed the pre-execute phase and returned error code 0xC020801C.
(SQL Server Import and Export Wizard)
Any ideas would be much appreciated!
Thank you,
Cody
View 1 Replies
View Related
Feb 19, 2012
I have a distribution database that has grown over 50g. Distribution cleanup takes over 15 hours to run and there are always very few rows in msrepl_Commands. Usually only a few hundred. Everything is being replicated fine, but I fear a storm coming. why distribution would be so large with so few rows? Also, Ghost Cleanup has been running all week, so I'm thinking maybe the records marked for deletion aren't being deleted?
View 1 Replies
View Related
Oct 9, 2014
I've just inherited the job of looking after our SQL server (2012). The server works fine but I am a bit concerned about the backup strategy we have in place. The current backup strategy is a SQL script that runs every 2 nights and backs up to an ISCSI box elsewhere on the network. One of the databases on the server is currently at 303GB, which will take about a full working day to restore to another server if this one fails. I am just looking at other ways which could reduce the amount of time to restore this if this server was to fail. i heard so people talking about sql clusters and configuring replication on posts by others, which i will look into if its the way to go, but i guess I am just looking what others would do if they were in the same position with such a large database.
View 1 Replies
View Related
Mar 21, 2008
Hi i wanna delete all the records from an large database 200 -300 tables, because i want make some changes an start from scratch,but keep the structures of the database key , index etc, i tried to generate script but when i run to many errors , plz help 10x
View 11 Replies
View Related
Feb 5, 2007
hi
is there a datatype thatcan handel a very large string ( i dont want to save it in a file and save the file in the bd ) i want to save it as it is
thanx a lot
View 7 Replies
View Related
Oct 23, 2007
Hopefully I'm posting in the right area. There is a database that has grown to about
41-42 GB in size in about a 2 month period. The previous database had grown to about
22 GB before it was purged out. I'm running this on SQL 2000, and I've tried running all
the DBCC SHINKFILE and SHRINKDATABASE commands to no avail. In this case,
the MDF file is the one that has grown out of control as opposed to the log file (LDF file).
Does anyone have any suggestions on what could be done to control the size?
View 17 Replies
View Related
Dec 29, 2004
If I want to copy the data from Table1 in Database A to Table2 in
Database B but Table1 column name is code , Table 2 column name is
vesselcode.
(Code = vesselcode)
How to copy all data from Table1 in Database A to Table2 in Database B ?
Do I need to write the SQL statment ? and Can I use Server Enterprise
Manager Tool?Thx a lot.
View 1 Replies
View Related
Jul 20, 2005
Hello,I need to copy a table from an 8i oracle database to a sqlserver 2000 database.Is it possible to use the command "COPY FROM ... TO ..." ?So, what is the correct syntax ?Thanks for your helpCyril
View 1 Replies
View Related
Jan 6, 2008
Hi,
I have a complicated sql server mobile database (.sdf) and need to create a SQL SERVER database with the same tables. How can I do it without scripting the whole thing? I thought of using the views.information_schema databases, but it is still a lot of coding.
thanks
View 3 Replies
View Related
Apr 26, 2001
hello everyone,
I have a question about the under what circumstances will a large database with mostly free space not shrink?
thanks
View 2 Replies
View Related
