I upgraded a database from SQL2000 to SQL2005 and it went pretty smooth. After the transition was made, I backed up the DB. The size of the database was as expected, about 5 GB. About 5 hours later, a maintenance plan executed a few optimization jobs in the following order: Reorganize Index, Rebuild Index, Shrink Database, Update Statistics. Soon after that, another job backed up the database and it was then 32 GB. During the time after the first backup, no one was really using the database.
I've been trying to track this down for several days. Does anyone have any ideas for me?
guys,I have a project need to move more than 100,000 records from onedatabase table to another database table every week. Currently, usersinput date range from web UI, my store procedure will take those dateranges to INSERT records to a table in another database, then deletethe records, but it will take really long time to finish this action(up to 1 or 2 hours).My question is if there is some other way I should do to speed up theaction, I am thinking about use bcp to copy those records to datafileand then use bcp to insert it into SQL Server table. Is this the rightway to do it or should I consider other solution (then, what is thesolution.)Thanks a lot!
What happens if you have a website and the hard drive on your server is say 250GB. Then the database exceeds that and the database is 300GB in size.How would you spread your database into two different hard drives?Thanks
I would like to know whether MS SQL Server 2000 Can handle Large Database size 28 gig, currently we are using Sybase but we want migrate to SQL Server 2000. Are there any performance issues?
We have many installations of our shopping cart database. One specific database is huge, now about 25 GIG, compared to the others that range from 20 to 75 MEG. The server this one resides on has three other instances of the same database that are normal size.
In a particular table in the large database there are 9700 rows taking 380MB, the same table in a normal db has 162000 rows and takes 6.MB. The tables are identical and the indexes are the same.
Hello! I have a database with one table inside. The table has six columns with the following datatypes: col1 -->smallint (2byte) col2 -->int (4byte) col3 -->smallint (2byte) col4 -->smallint (2byte) col5 -->smallint (2byte) col6 -->int (4byte) I have insert 1.359.320 rows with data and the size of the sdf file is 40.116.224 byte. According to my calculation: 1.359.320 * 2byte + 1.359.320 * 4byte + 1.359.320 * 2byte + 1.359.320 * 2byte +1.359.320 * 2byte + 1.359.320 * 4byte = 21.749.120 byte I hope somebody can explain me, why the database is so large.
Importing data from an Access database, I cannot overcome the limit of 1,000 records. In DTS, I "copy one or more tables", select tables, run, and cannot see my 1,052 entries. Where can I set a max size of ~1,500 in my sql target base?
We now host 200 G data in SQL Server 7.0 in MSCS. The data will be 1 terabyte at end of the year. The possible solution to host large data with good performance is to migrate data to Oracle or UDB in Unix platform. But we would like to try any possibility in Microsoft world before we decide to migrate data to the other databases.
I need your experience and recommendation about hosting large database in SQL Server with good web performance. Somebody mentions Microsoft tera server. Who has experience using tera server which is still in the beta release? I would like to hear some successful story to handle the large data and understand precisely hardware, software, performance moitoring requirement.
Hi to All I have large database with tables and views on the server around 80 to 100 mb amount of data.when i retrieve the data from the server it is taking long time Ex: "select * from vwcases" its taking 10 to 12 minuts time.this is my database problem.If you know the solution please reply back to me as soon as possible. thanking You.
I need to query a database with about 3 million records using a 'like' statement on various varchar columns. I've have a non-clustered index on the columns included in my query, but the performance is still pretty slow. Can anyone suggest ways to increase the speed of my queries? Does full-text indexing improve performance or does it just facilitate more advanced searches (which I don't need)? Would char instead of varchar datatypes be faster?
Hi Guys, I am looking for a large datawarehouse database implemented in SQL Server to use it to run some examples in my thesis. if someone knows where I can find a large datawarehouse database (e.g Sales database) please tell me.
Does anyone know of a good way to copy a large database or backup of a database (~10 GB) across long distances (transatlantic) in a minimal amount of time?
I have tried regular file copies, BCP, DTC, my own handwritten extraction utility, PCAnywhere, and several file splitters, all to no avail. To complicate things, the source is SQL 7 with a different code page from the destination, which is SQL 2K. The file copies always seem to get interrupted after several hours.
We have total 7 databases in the warehouse and these db are big. All the data files are approximately 300G now and they all growing more. Some are small db some are large db. (ranging from 20G to 50G of data files). Can someone please recommend me the best backup plan for this scenario. The regular backup through SQL 2005 takes too long.
I developed a vb app that imports csv data into an sql server db. The original text file is 36.5mb. The db after import is 230mb and the log file is 555mb. Is this normal?
I am developing an office automation software for a government department. To start with, I have decided to first automate the salary section of the department. Because of some issues like, TCO, easy support and maintenance, scalability etc., I have decided to use SQL Server 2005 database with VB 2005 front-end.
I have planned the layout for this first part and found that in only employee salary table, 40,000 records per month will be stored. That comes out to be around 480,000 records annually. This is for one table alone, excluding data in other tables.
Since each section of the department will be integrated into this later, this application will become the backbone the department. The employee service, leave, salary, allowance, deduction and other records shall be maintained.
For this type of mission-critical application, I want to get some queries cleared:
(1) What backup strategy should be followed? I want to have schedule and maual backups both. Is there any way to have a mirror image on another server?
(2) What considerations to keep in mind in the inital stage of database design?
(3) How to keep the design scalable and configurable to meet future needs?
1. To combine all the company's data in one large database, and use schemas and file groups to create logical and physical distribution on drives and namespaces
or
2. Distribute the data into smaller databases with related data - eg. products and product description in one db, Customers in another and orders and orderlines in a third db.
I have many, many Access databases that are roughly 1.5GB-3GBs each and they have millions of records. Each MS Access Database file corresponds to one Database in SQL server. I'm trying to simply transform the data as it is in Access to MS SQL 2005.
I'm using the 64 bit version of Windows Server 2003 and the 64 bit version of SQL 2005. The server is running four dual core AMD Operton processors and has 8GB of RAM with a 1TB RAID 5 configuration. I think the hardware should be sufficient but the SQL Server Import and Export Wizard can't seem to handle the large number of tables/records. If I do one table at a time, it works well; however, it produces the following error message whenever I try to import the entire database:
Pre-execute (Error) Messages Error 0xc0202009: {5A5BF7AD-E86B-4316-AD43-1912358C56F4}: An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft JET Database Engine" Hresult: 0x80004005 Description: "Unspecified error". (SQL Server Import and Export Wizard)
Error 0xc020801c: Data Flow Task: The AcquireConnection method call to the connection manager "SourceConnectionOLEDB" failed with error code 0xC0202009. (SQL Server Import and Export Wizard)
Error 0xc004701a: Data Flow Task: component "Source 64 - District Corporal Punishment Class" (5743) failed the pre-execute phase and returned error code 0xC020801C. (SQL Server Import and Export Wizard)
Any ideas would be much appreciated! Thank you, Cody
I have a distribution database that has grown over 50g. Distribution cleanup takes over 15 hours to run and there are always very few rows in msrepl_Commands. Usually only a few hundred. Everything is being replicated fine, but I fear a storm coming. why distribution would be so large with so few rows? Also, Ghost Cleanup has been running all week, so I'm thinking maybe the records marked for deletion aren't being deleted?
I've just inherited the job of looking after our SQL server (2012). The server works fine but I am a bit concerned about the backup strategy we have in place. The current backup strategy is a SQL script that runs every 2 nights and backs up to an ISCSI box elsewhere on the network. One of the databases on the server is currently at 303GB, which will take about a full working day to restore to another server if this one fails. I am just looking at other ways which could reduce the amount of time to restore this if this server was to fail. i heard so people talking about sql clusters and configuring replication on posts by others, which i will look into if its the way to go, but i guess I am just looking what others would do if they were in the same position with such a large database.
Hi i wanna delete all the records from an large database 200 -300 tables, because i want make some changes an start from scratch,but keep the structures of the database key , index etc, i tried to generate script but when i run to many errors , plz help 10x
Hopefully I'm posting in the right area. There is a database that has grown to about 41-42 GB in size in about a 2 month period. The previous database had grown to about 22 GB before it was purged out. I'm running this on SQL 2000, and I've tried running all the DBCC SHINKFILE and SHRINKDATABASE commands to no avail. In this case, the MDF file is the one that has grown out of control as opposed to the log file (LDF file).
Does anyone have any suggestions on what could be done to control the size?
I need to transfer a database (contining one table) containing over 35 million records from one server to another. I have two options at present : (a) Use DTS to do the transfer (b) Copy the mdf file across and sp_attach_db it
Does any1 have a better idea, or does any1 know which of the two methods will be the quickest?
Here's my delema, I have a file that's 308 bytes wide by 5.7 million records. The record length is fixed and the position and width of the known within the record. When I run DTS I recieve this error Msg MS DTS flat file provide and Err Diesdription: error creating file mapping view: not enough storage is available to process this command. Then when I try to continue with the wizard, it will not allow me to separate the data into the format that I need. Is there any other way to import this file using DTS?
I am asking this question on behalf of a friend. I have little knowledge of SQL 2005 but my friend is quite knowledgeable, although this is the first time he is dealing with large database for a client. So here's the story.
His client has a database containing 1.5 million books. Now he is setting up a website which will enable users to search books. Searching by ISBN is no problem as it only takes 1 seconds. The problem is, searching by Title takes more than 20seconds, which is unacceptable. My friend has only done smaller database and he just recently thought of implementing indexing and now looking for other ideas.
Each row contains book details such as Title, Author1, Author2, Author3, Publisher, Publication Date, ISBN, etc.
Can anyone who are more experienced in doing large database share with me some design ideas? His client is aiming for 8seconds or less.
I have come across a database system which isn't designed to work optimally. It is fairly large (~400GB) and performance of loading and querying is degrading (improper data types, fragmented indexes, non unique clustering key and other problems). So, I have quite a task in front of me, but I am up for the challenge. I figure this is not a unique situation, many of us would have come across this before. I have done this before too, but only for smaller databases, some of the operations here I expect to take a couple of hours or more to complete (depending on load/infrastructure speed etc, I know).
My plan is thus:
+ Take a full backup of the database + Set the recovery model of the DB to simple + Drop non clustered indexes + Drop clustered indexes + Remove PKs (wrong data types, too large!) + Narrow data types (add new column, update column in batches to old value, rename new column to old column) + Add PKs, which will create clustered indexes automatically based on PK ID + Create non clustered indexes + Run a SHRINKDB (normal operations I would never do this, but this is a special case, ensure log file is truncated to a logical size especially after all those table modifications...) + Set the recovery model of the DB to Full + Ensure everything works OK or better
We have a situation that requires converting all INT columns from source database to BIGINT in destination.
By ‘copying’ term, I consider any operation like insert-select, merge, asynchronous replication or any other that will get all data synchronized until some moment in time, i.e. 2015-12-31 10:00 PST when downtime is planned. Before that moment it isn’t important if data in source and destination DB are in sync.
Copying process starts early enough to allow appliance of any technique/technology – there’s enough time to find best solution.
Requirements:
1. Destination database is superset of source, having all tables and all columns plus few more columns in few tables. Additional columns have default NULL or they are computed. INT columns in source are BIGINT in destination.
2. If source and destination are not in sync at the moment downtime begins, it is allowed to have few minutes to wait for synchronization. For instance, if transaction logs from last hour or so need to be shipped, that would be ok. If large table need to be synchronized by comparing row-by-row, it’s not ok.
3. Source database performance must not be compromised by any copying if it’s done before scheduled downtime. There are couple of options I have in mind involving MERGE command but its appliance might be limited by current DB design. Additionally, based on business logic, I cannot be absolutely sure if some row, created while ago, wasn’t changed meanwhile.
For some large tables, there is no indicator at all when any row has changed – I cannot rely on IDs nor there is column like ‘[date last changed] DATETIME’
I have been reading about various kinds of replication, i.e. log shipping but I’m not clear if any of these is applicable. Key points, if there is replication involved, are copying process must start from empty destination database plus synchronization must not compromise source DB performance So, how to do this using any kind of replication?
Hi everybody, On executing the RESTORE command of SQL Server to restore from a backup of 78.3 MB, the "Server Application Unavailable" error message comes up.The error message in the Application log is as follows:aspnet_wp.exe (PID: 2184) was recycled because memory consumption exceeded the 152 MB (60 percent of available RAM). However using Query Analyser of SQL Server I am able to restore the database. What is the solution to this problem?
Hi, I work with a large team developing ASP.NET application that has a large database with over 50 complex stored procedures. It is proving more and more difficult and time consuming to centralise the development and update of the database changes and I was wondering if there were any best practises/tools that could be recommended. I have looked on the web for good articles and haven't found anything difinitive (except that Team Foundation Server is the way forward).. A brief background to the current process is that everyone develops on the same database, and then updates the stored procedure scripts in source safe (manually). Then when we do a new release someone builds a script of all the database updates and runs it. There are issues related to developers updating there stored procedures over other peoples and other concurrency. I am looking to move all the developers to start using local databases so that there work only effects them, but then this brings up problems of keeping all the local databases up to date whenever they get the latest source code. The only way I currently see is to build a database update program, that will run and update to the latest version. Surely this must be a common issue? Anyone have any good ideas/concepts? Also our setup is Visual Studio 2005, SQL Server 2005 and Source Safe 2005. Cheers, Andrew Thomas
We have a large OLAP database, about 2.5 TB spread out over 3 data files on three different drives, and recently someone ran a query that created a table that continued to grow until the data files filled the available disk space (about 3 TB total - 1 TB per drive).
Tonight I plan on running a full backup (it's in Simple mode) and running a ShrinkFile on all three files sequentially with TRUNCATEONLY just so it will remove the space after the last extent. Any way to tell ahead of time how much space this will recover?
Granted running a DB Shrink is one of those things you just don't do, but this is a one-time shot and unavoidable to get the file size back under control.
