Hello
Does anyone have experience/advice with large databases (5-10 Gig)? If so, I was wondering about
performance/other benefits of spanning a large database across multiple devices (different disks). Would anyone
vote for or against doing this?
Suggestions...
Thanks
Tim
Hello All,
I want to know if it's possible for a database on multiple devices to put specific tables on specific devices?? and if this is possible how to do it??
Thanks for help
Mohamed hani
Just wondering - on a straight RAID 5 system is there any advantage, performance or otherwise, to splitting a SQL 6.5 database over multiple database devices? Or for simplicity's sake am I just further off creating a single device and manually extending it as the database grows?
Thanks
RM
Hello--
I am trying to backup a VLDB (800+gig) to multiple devices. I am using 4 disk drives. Assuming all devices have the same performance, should the 4 devices fill equally? (IE 200gig per drive) If so, what would might happen if one of the 4 drives only has 100 gig free with the other 3 having 300 gig free each?
Thanks for the help--
Jeff
I have a database approximately 30 GB in sixe which need to be moved from one SQL server to another. Does anyone know the most efficient way of doing this, other then backing up to tape?
View 1 Replies View RelatedHi experts,
We have SQL Server 2005 installed in MS Windows server 2003 with 8 GB RAM. This server has 4 processors.
Ours is a VLDB and a single table has 400 million records occupying nearly 40 GB of space.
We find it vert difficult to meet the response time set by the clients in many occasions.
Should the RAM be atleast as big as the biggest table in the database ? Is this mandatory ?
Even any other suggestions for improving the performance are welcome.
Thanks & Regards,
Hariarul
Hello All,
I have been experimenting with SQL Server 2005 partitions. I loaded a terabyte of information into 2 tables. The first holds the document information and the second holds the actual binary document (in this case pdf). Most of the documents are about 1 megabyte in size, but the largest is 212 megabytes.
SQL Server has no problem storing the blobs. The problem occurs when I attempt to get the data.
I did some quick tests to test how fast I could pull the documents out. The largest took about 24 seconds. The 1 meg documents are sub-second.
Here is how the 212 meg doc breaks down:
Time to load datatable: 18.79 seconds
Time to load byte array: 3.84 seconds
Time to Write and open document: 0.01 seconds
If I access the file from a file server, the time is 0.04 seconds to begin showing the document.
As you can see, the longest time period is related to retrieving the data from SQL, and it is much slower that launching it from disk across the network. (note: the sql server and file server used to test are next to each other).
My question is, how can I speed up the access from SQL Server? I believe the keys are "partition aligned". Any suggestions would be appreciated.
I will add the table definitions and partition information as a reply since only 5000 chars are allowed in the post.
Hi There
I have a 50Gig OLTP production database that currently takes +- 50 minutes to backup, (normal sql flat file backup to disk).
This database will grow to +- a terrabyte by next year.
My major concern is how will i be able to backup this DB when it is that big in 2 hours or less.
I have been checking out my options, in terms of SAN snapshots/clones. Also multiple backup devices and using differential/filegroup/full backup strategy.
What i want to know is if anyone out there is backing up VLDB's what strategy/methos/tools are you using, even 3rd party tools for faster,smaller backups?
Any pointers/best practices for VLDB backups would be greatly appreciated.
Thanx
Hi experts,
We have SQL Server 2005 installed in MS Windows server 2003 with 8 GB RAM. This server has 4 processors.
Ours is a VLDB and a single table has 400 million records occupying nearly 40 GB of space.
We find it vert difficult to meet the response time set by the clients in many occasions.
Should the RAM be atleast as big as the biggest table in the database ? Is this mandatory ?
Even any other suggestions for improving the performance are welcome.
Thanks & Regards,
Hariarul
I hard that SQL Server 7.0 has problems when the database reaches
50 - 100GB, in areas such as backup, transaction logging, and database
admin and that by 100GB parallel queries are also affected.
Is this true ? Where I can get information on this ?
Thanks in Advance.
Regards,
Vidyadhar
I have a production database that is in the low gigabyte size andgrowing steadily. No issue there.I wish to completely refresh the development database daily on asecond server. What is going to be the fastest easiest way to do thiswith hindering performance on the production system ?Thanks,Craig
View 2 Replies View RelatedI need to manage the problem of negative performance implications when I fragment a 1TB+ DB. I want to perform Index Reorganization if fragmentation is no higher than 30%, and Index Rebuild if the fragmentation exceeds 30%.
Firstly can anyone recommend a script which uses sys.dm_db_index_physical_stats system to ascertain the
fragmentation level. Secondly, is there a technique I can employ to prevent the ONLINE operation completely killing performance on 27/4 production system?
ALTER INDEX REORGANIZE/REBUILD WITH (ONLINE=ON)
I've got a few VLDB's that we want to make smaller. Since the tables are running on legacy stuff, all of it's basically made with int's and char's and it's horriably inefficant.
The problem that I came across is when I made a new table with the best data types and copied the data from the old table, the table size was the exact size (excluding the index size). It was estimated that a total of ~20 GB would be saved with this change. As it turned out, 0 bytes of data were saved with the data types chagnes.
Why are the two tables the same, even though one has much more efficant data types?
If you want more information about the table I'm using:
391 columns.
50,147,035 rows.
65,295.625 MB in size.
I have been using AlwaysON AG for a long time now and currently have about 10TB of data across 120 databases and 3 AG groups for any application that is on SQL 2012 with great success. Each AG group is running on patch level 11.0.5058.0 with 2 synchronous replica(on different SANS) in Primary Data center and 1 ASYNC replica in DR. Migration has been a non-issue because none of the databases weren't substantial enough that I could not fit into my maintenance window which is 12-4AM on SAT morning.
My issue is that my last application to migrate to 2012 includes a 4TB TDE encrypted databases database which is about 10x larger than any of the previous ones I have migrated. The database takes 4 hours to backup after tuning extensively(I hate TDE!!)
The restore to the primary replica is instant because of seeding incremental but the issue comes from having to backup the database before adding to the availability group. 4 hours is my exact outage window and I can't get any more. My plan to migrate application is to -
First Outage Window
1) Restore Database from 2008 to 2012 Primary Replica
2) Change application ARECORD(or cname not sure which) to Primary replica
3) Run database on single node until next outage window
Week Later
1) Add database to availability group
2) Change ARECORD/CNAME to listener
What I don't like about this is I am going an entire week with 1 node instead of 3 which is worrisome. How to accomplish this I would love to hear from you or any type of comment from people who have worked with VLDB in availability groups and what you like/hate/loved about doing it. I am trying to go all in on this software and have loved it so far but getting worried when it comes to the VLDB migration.
The column I'm adding needs to be part of the clustered PK (it will be the last of three columns) so I need to recreate all the indexes.
My DB is set for FULL recovery mode ALLOW_SNAPSHOT_ISOLATION ON. I've tried two methods so far.
Method 1:
BEGIN TRANSACTION
CREATE TABLE dbo.Tmp_copyoftablewithnewfield
(
) ON PRIMARY
IF EXISTS(SELECT * FROM dbo.originaltable)
EXEC('INSERT INTO dbo.Tmp_copyoftablewithnewfield (<original fields>)
SELECT <original fields> FROM dbo.originaltable WITH (HOLDLOCK TABLOCKX)')
GO
DROP TABLE dbo.originaltable
GO
EXECUTE sp_rename N'dbo.Tmp_copyoftablewithnewfield', N'originaltable',
'OBJECT'
GO
<recreate PK constraint>
<rebuild indexes>
COMMIT
Pro's: Lets me add the new field in the spot I'd like it (not a big deal)
Con's: Tons of wasted space and time. It took about 15 hours.
Method 2:
SET XACT_ABORT ON
GO
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
GO
BEGIN TRANSACTION
<drop PK constraint>
<drop indexes>
ALTER TABLE [dbo].[originaltable] ADD
[newfield] [tinyint] NOT NULL CONSTRAINT [DF_originaltable_newfield] DEFAULT
((1))
<recreate PK constraint>
<rebuild indexes>
COMMIT TRANSACTION
Pro's: No making a copy of the entire table taking up 200GB more space in the db data file
Con's: My tempdb grew to accomodate the row versioning info for every row in the 200GB table. It took over 30 hours.
A lot of time and disk space is wasted with both.
Since the db is going to be unavailable to users I have some flexibility here. I was considering turning ALLOW_SNAPSHOT_ISOLATION OFF and then trying method 2 again which should stop the versioning in tempdb and then turning it back on.
I was also curious if setting the database recovery mode to SIMPLE would cut down on db log usage and then I could set it back to FULL when done.
Do these really need to be in a transaction? If there's some hardware failure or something unexpected I can just restore from backup and do the conversion again. If the presence of the transaction itself is causing more disk usage for logging or any other slowdown, I think I'd rather do without.
Given the amount of time this conversion takes, I wanted to get some
feedback other than "just try it" before doing any new tests.
Thanks.
Hi,
Im a Jr DBA and have been given an assignment by my lead to find information on the following.
We are to migrate existing db of size 4TB to a
DELL PowerEdge 2950[Mem:Up to 32GB]
OS : Windows Server 2003 Std Edition X64 SP2
DB : SQL Server Enterprise Edition x64
I am to find on how to design the db to provide optimum performance,fail over and consider the growing factor of the db.
1)What would be the recommended RAID settings?
2)Placement of the tempdb ?
3)Should we do clustering and why ?
4)What Data partioning would do to help?
5)Any Other aspects to be considered for sizing db ?
6)Placement of data files and log file on separate physical disk ?
7)Indexing?
I have read many sites.I would appreaciate if someone could write suggestions and opinions based on their current db design spec or previous experience,by selecting best db design points.Thank You.
Hi,
I would like to delete a data from a 750million row table in chunks of 10000,without blocking the users.As ours is a 24/7 shop I donot want to block the users for a long time.
Answer for this is highly appreciated.
Thanks
Samna
Does anyone out there use a NAS device to place there database and corresponding images on?
View 2 Replies View Relatedhi, I usually back up the database to a file within the sql server box. then use a tape backup to back up the whole directory of the sql server in the d: drive.
Is this right way to do or there is a better way ?
Ahmed
I know that you need to create a device in 6.5 to do backups, but in 7.0, it seems to take care of this for you. However, there is still a device folder in the Management folder of version 7.0's console tree. Why would it be better to back up to devices, rather than anywhere else on the hard drive. Is there any way to easily schedule the deletion of backups on these devices? I know that with 7.0's Maintenance plans, you can schedule old backups to be deleted. But how do you do that for devices?
joe
How do I create more than 10 devices in sql server 4.2?
View 2 Replies View RelatedHello
I just tried to make a typed resultset from a SSCE v3.5 database, and recieved this message from Visual Studio:
"Typed ResultSets cannot be generated for tables on Microsoft SQL Server Compact 3.5 databases present on devices."
On the device, speed and low memory consumption is important. So it seems to me that this should be present ONLY on devices, because on the desktop we can just use the DataSet generator and not care about the overhead.
Anyway, ripped from what would have been a nice feature, the alternatives that comes to mind are:
1. Making my own resultset generator
2. Use typed DataSets
3. Derive from SqlCeResultSet and store the Field names in it, to build my own typed resultset class.
Any suggestions on what to do?
I asked about this a few mos ago over in the CF forum, but I'm hoping there's a better solution someone (Erik?) knows about here.
Is there a "standard" deploy script for SQL Server CE on devices? There doesn't even seem to be one in the d/l from Microsoft. During development, Visual Studio handles the installation of SQL Server CE on the device but deploying it on a device that isn't partnered to a dev box seems to be a matter of "just copy the cab file over".
While I'm willing to do that if I have to, it raises some other questions. How do I know if it's already there? How do I know if what's there is a newer version or not (and if it is, how do I abort this step)? How do I make sure some other application hasn't installed it already? Do I really even need to worry this?
I can just have my installer tell ActiveSync to put it on the device, but without knowing the above I'm kind of nervous about it.
Hi All,,
I want to get a list of backup devices for a selected database
ex :
Northwind Database i need to get the list of back devices for this db
any one know how the query could be written
best regards
Wafi Mohtaseb
I have taken backups and the backup devices are stored in C: drive and now i want them to be moved onto the other drive.So how can i do it without creating new devices and scheduling them again.
Any help would be appreciated.
Thanks,
Kris
i need to initialise the backup devices as the the backup files have grown so big thats its occupy hard drive space , each device is for @ day for all databse we use , what do i do
thanks
safac
Hi all
I have a raid 5 NT server 4.0 with SQL 6.5 running.
I perform backups on tranlogs every hour and db`s every night.
I also have another 16G Nt server not raid 5.
Can someone tell me if it is possible to mirror a device or a database
into the regular 16G NT server from raid 5 server?
HOW???
HOw about database mirroring??
And also whats the best solution when it comes to mirrorring in NT server?
Hi guys
I am relatively new to SQL Server admin and I have been handed a task of creating backup devices for a particular SQL Server which has 204 databases.
Rather than go through and create Full, Incramental and Transaction Log backup devices for each database is ther anyway of doing the following
Create a script to go through and for each database name in 'sysdatabases'
create a folder e.g sqldataackup\%databasename%
and within each folder create a backup device called
%databasename%_full
%databasename%_inc
%databasename%_log
Also all these databases are running in 'Simple' recovery mode so obviously I need to change this to 'Full' to enable incramental and log backups - is this possible using the same script.
Hope someone can help as the thought of doing all of this individually for each database scares me silly!!! :)
Thanks in advance for any help
Hanley
Hi,
I am using EMS SQL Manager for SQL Server (www.sqlmanager.net). I am trying to learn how to do backups of my database.
I am up to this step -> http://www.sqlmanager.net/products/mssql/manager/documentation/hs21640.html
I don't think I actually understand what a device is, how it's created, where it resides, etc. I would like to backup my remote datasource to my local computer's harddrive.
Can someone please explain how this 'device' works and how I should set up?
Thank you so much for any help :-)
jo
Hi all,
Can somebody tell me how to display reports created in SSRS 2005 in PDA/mobile devices? How to connect to the PDA device?
I am new to SS7. We are planning to rearrange our system databases so that the logs are on one drive
and the data files are on a different drive. I found a really good white paper on this but just want to
verify that there is not a problem with doing this. In sybase there were some restrictions on doing this.
Any info you have would be appreciated.
Is there a way to shrink database devices in SQL Server 6.5 Other than dropping and restoring?
View 1 Replies View RelatedHello,
I am using EM to administer SQL 6.5 SP5a and have a few dBs that do not show backup history by expanding the backup device. The server is regestered as SA and MSDB does contain the the correct data in the sysbackuphistory table. MSDB DBCCs OK and other dBs show history. I manage multiple SQL 6.5 servers but this is the first time I've seen this and can find no info in the various KBs or NGs..
Anyone have any clues?
Thanks,
Jack