Currently I have 1 server running MSSQL 2005 Standard. There is no redundancy in my current solution. I'm working on a project that involves a separate installation. I have 3 goals:
1) Provide redundancy for the new installation
2) Provide high availability for the new installation
3) Provide reduncancy for the current installation
Here's what I'm thinking about doing:
1) Purchase 2 servers and a Dell MD3000i ISCSI storage box. Cluster the servers and install SQL Server on the cluster.
2) Install SQL Server on an existing box (single CPU license) for backup purposes. Enable log shipping from the new cluster to the backup server.
3) Enable log shipping from the existing installation to the backup box.
I've also though about offloading some of the static pricing operations to the backup installation. This would free up my existing installation to deal with the changing data.
A couple of questions:
-How does this strategy sound? Are there any configuration problems with it?
-Can I legally use a development edition for the backup installation (if all I do is log shipping to it)?
Pardon me for asking a question that I know has been asked before. I need to develop a backup strategy for our SQL Server and I am looking for any help that anyone can offer including recommending good books for reading.
In SQL Server 6.5, Is it generally better to dump the transaction log first, then the database or to dump the database and then run a dump 'tranlog with truncate only' option?
I am currently developing a system thats stores exchange stats in a db. Since our customers are companies with 20 employees up to 5 000 there a a big difference in the volume of data needed to be stored.
We currently thinking of supplying a SQL Server Express DB to the small customers and suggest a SQL Server to the bigger.
But since I would like to use the same structure for both types of customers I wonder how should i design the storeage.
Since the could be from 500 records a day up to 20 000. There are quite simple recordes with only simple datatypes. about 15 fields with no more than 10 chars each, mostly 2.
Should i separate the data in diffrent tables for a week or a day etc. Since I am only going to filter data on 1 or 2 fields the data will be easly indexed.
The reports generated will almost always only use 1-3 months of data, but historical reports have to be possible.
My question are ofcourse: Whats the best solution for me?
I've recently inherited a position where I am responsible for the well-beingof some DBs.2 (much) more important than others.The current recovery model, from what I can tell, is to do a full db/logbackup overnight.This .bak file is then written to tape as well as saved on the disk for 2days.Both these dbs are used fairly extensively 8-5pm and losing data would notbe good.The db sizes are approx 5gb and 3gb.This doesn't seem like the ideal situation to me. Everything I read tellsme... full backup periodically, differential nightly and transaction hourly.Agreed?If so then I have 2 questions:1. Is the best way to do this via a maintenance plan or by scripting andscheduling?2. What, if any, overhead can be expected with regular transaction backupsduring work hours?A bit of a pointer to #1 would be appreciated also.Thanks.
I have a bit of a problem with regards an indexing strategy. Well,basically there is no indexing strategy on a set of data I have atwork. Now, I didn't create the design as I would have allowed for this.OK, so there is primary key (clustered) indexes (mainly compositekeys), but no other indexes on the tables. As you would expect, theperformance leaves a lot to be desired. A hell of a lot. We haveseveral million rows in a lot of the tables. None of the queries seemto be overly complex so we can work through the applications at a laterstage.We have multiple copies (one per client per country) of the samestructure (I may have considered combining these to allow betterperformance). One specific SP that I need to run takes 40+ hourswithout indexes and 5 hours with some (130) basic indexes to get usstarted on a better design. These 130 indexes are the minimum I suspectwe need and from there, we can start to work out which ones we need.Now the test database (our worst performer) doubles in size to 20Gb,but the performance is much much better (as expected). The originalthinking behind the design was for storage concerns (server spacerecently upgraded) and for performance with bulk inserts.We have a lot of bulk inserts, but I suspect that these are not toobad, and the time taken for indexing these is negligable due to theperformance gains. I strongly suspect that we have a considerableamount of table scans going on, but the problem is that people heredon't understand indexing (yet) or have the time (probably because it'sall taken up using the poorly designed system). That's a whole seperateissue for me to address.So, finally getting round to my questions...Is there any good reference explaining in Layman's terms why you needbasic (or advanced) indexing ? Any links would be appreciated. I needthis to help explain to colleagues why a disk space increase andindexing will be far better than spending thousands on a new box anddoing the same (a common problem I suspect).How can I accurately estimate the amount of time taken to update anindex once data is bulk inserted. I don't want to re-index from scratchas this may take too long. Indexing my database first time round takesabout 1 hour 30 minutes.It's all part of an ongoing bit of digging into the system and re-doingit to make it work properly. I'm sure most of you will have been thereat some point or another.ThanksRyan
We are currently doing daily full backup of system & custom databases since database size is small. Is that good idea ? or better option would be weekly full & daily incrementatl ?
Do we need to do any special backup on system databases or transactional logs ?
I have concern about an sql server. The server has the operating system and sql server installed locally. The databases and transaction log files is stored on SAN. We used to have the database backup and transaction log backups stored locally on the server. We tape the database backup and transaction logs every 24h. If we lose the san and the server then we are stucked with no backup easily accessible. And on tape we loose of 24h data. We decided to put up stand alone server with no connection to the san and dump the backups file on this server. We also put a secondary server sql I case of emergency, to test backups and We are looking at the getting a mirroring or log shipping solution but we are not there yet, next year€™s budget. We still using some old server left from migrating to virtualization.
Then I read the €œPractical Troubleshooting The Database Engine book€? best practice not to avoid net work drive backups. Stuck aging. Back with the backup to local drive and robocopy them? Keep them on the network drive, start using? MIRROR TO in the BACKUP DATABASE? Today we are using the Backup Database Task in the SSIS.
I am running SQL Server 2005 x64 Enterprise under Window Server 2003 x64 Enterprise. After reviewing many posts and suggestions in this forum, I am developing a backup strategy that should include keeping my transaction log file in a manageble size.
Please examine the following proposed backup schedule and let me know if this is considered a sound plan. The scripts below will write to disk and each night and then be backed up to tape.
*** TASK 1 ***
Backup transaction log
/* This script backs up the DSS database transaction log to disk, overwriting any previous backup */
BACKUP LOG [DSS]
TO DISK = N'g:mssqlackuplogdss_log.bak'
WITH
INIT
, NAME = N'DSS-Transaction Log Backup'
GO
*** TASK 2 ***
/* This script shrinks the DSS database transaction log file
*/
BACKUP LOG [DSS] with truncate_only
dbcc shrinkfile(DSS_log)
**** TASK 3 ****
/* This script backs up the DSS database to disk, overwriting any previous backup */ BACKUP DATABASE [DSS] TO DISK = N'g:mssqlackupdatabaseDSS.bak' WITH DESCRIPTION = N'DSS Full Database Backup' , INIT , NAME = N'DSS - Full Database Backup' GO /* Backup validation to ensure the file is valid before storing it */ RESTORE VERIFYONLY FROM DISK = N'g:mssqlackupdatabaseDSS.bak' WITH FILE = 1 GO
This may seem like a silly question, but has anyone ever heard of a DBA or an Engineer deciding to not back up databases inside EM, and only relying on the RAID or third party software for redundancy?
SBS 2003 R2
SQL 2000
Veritas 8.6 open file agent, SQL agent, Exchange agent
In my current organisation they are using SQL Server.
They are using TSM (Tivoli Storage Manager) to back up the server on a nightly base.
However I feel that this is not the correct way. Suppose I need some data back, I call technical support and they would restore the server. If another user made some changes to another database that day, he would lose his changes.
Of course they could restore a file but I am not sure if this is correctly. What will happen to the transaction log for example ?
My idea is that they should backup the database using the normal sql backup commando, dump the data to a folder and backup that folder.
I have been developing a genealogy application using a SQL Server 2000 database and ASP .NET 2.0. In this application a process, Ged.Parse, converts data from the GEDCOM standard format (a heirachical file format that looks as if it was designed for 80-column cards) into my SQL Server database. As we started to load reasonable quantities of data into the system we found that the on-line response became abysmal. This problem was fixed by defining a number of secondary indexes (response times dropped to under a second, from previously exceeding 2 minutes and often timing out). Unfortunately however the processing time of Ged.Parse then tripled, and it may now take up to an hour to process a GEDCOM. I believe that this is a byproduct of defining several indexes that are not needed by Ged.Parse itself, but which are of course maintained as Ged.Parse inserts new records into the database. I am wondering what my best strategy is, apart from putting Ged.Parse into a background task and just letting it trickle away. (I will probably do this anyway). What I'd like to be able to do is to have Ged.Parse load records without creating the secondary indexes, and then create the indexes for the newly-added records as a penultimate step just before it makes them available for general use. Of course there is no way that you can do this: records in a table are either indexed or they are not. Proposed change: recode Ged.Parse to load data into temporary tables, say NewPeople, NewFacts, etc., with these tables having only the indexes required by Ged.Parse. Then, as the last process in Ged.Parse run a SQL procedure with code like: - Insert into People Select * From NewPeople Delete from NewPeople etc This is a reasonable amount of programming, so before I make this change could somebody tell me: will this be significantly faster overall, or is this likely to make little or no improvement compared to the present process in which Ged.Parse loads data directly into People, Facts, etc? Two facts that may influence the answer. First, all record relationships are through GUIDs, so records in NewPeople, NewFacts, etc would already have their final key values. Second: although Ged.Parse needs to form relationships between records, these relationships are only within the new records (created from the same GEDCOM), and Ged.Parse does not need to relate any of these new records to earlier records. Thank you, Robert Barnes.
Hi! It would be very nice if some people out there, using merge replication can tell me their strategy to get started with a new subscriber. Is it a good way to copy a publisher backup to the new subscriber and restore it there and say that schema and data is already here when creating the subscription? I experienced some troubles when I tried to add a new subscriber and used the initial snapshot transfer to get the db to the new sub. (no defaults are transferred...???) Or is it even better to use DTS?
I'd also be very grateful forinformations about RESCUE STRATEGIES in case of a major database problem of replicated databases!
Hi All, Can any one help me with this.. I've a critical application that can't be stopped for a second. I'd like to have an implementation that uses 2 sql 6.5 servers one as standby and which is ready and up to date to take place and run instead of the master server when it's down and when the master is back to work it's updated with the data entered to the standby. This process must be automatically to maximum extent. Thanks Mohamed
I am using SQL Server 2000 and trying to create a disaster recovery strategy that would run nightly and backup the database or at least the changes and would ftp these to a secure ftp site. For smaller database it is easy, I just take a full backup, zip up the file and ftp it to the secure backup site. This strategy does not work so well when the zipped up database is still close to 3GB. I have a pretty big window for doing everything but 3GB is just too much to ftp overnight. The recovery model is simple so the only other option seemed to be do a full backup once a month and take differentials nightly. The problem is I am offsight and the client may need to take a full backup during the day and my nightly differential would get screwed up.
There is a fairly low volume of transactions so the idea of just doing nightly backups on the data that has changed is the obvious choice but differentials don't seem to fit. Any ideas?
My maintenance plans are starting to acting weird. I'm building a custom script to manage the database backups on my server, but curious if anybody has some sample work that will allow me to avoid re-inventing the wheel.
A couple of primary constraints: I want to do a full backup daily (and only retain 1 day of full backups) Transaction Log backups every 20 minutes I'd like to loop through the databases on the server automatically to make this a little more flexible.
You have anything you'd like to share? Or, bits of knowledge worth sharing?
I have just created a logging table that I anticipate to have 10's of millions of rows (maybe 100's of millions eventually).
Basically its a very basic, narrow table, we are using it to log hits on images for a webserver.
My question is that we want to run queries that show how many rows are shown per day etc, however we want to make sure these queries which we are anticipating to be very heavy, do not slow down the system.
I have been recommended to have a seperate database (mirror/replica) for reporting so that the performance of regular activity will not be affected.
I assume this means I would need another server for this other database?
I am thinking there are probably some alternative solutions to this as well. Getting a dedicated server just for these queries really isnt an option.
In order to improvement it is not a problem to make some sacrifices. For example, having the data update every 15 minutes is more than acceptable.
I see certain websites I use employ this strategy of making data update every 15 minutes, but I am unsure what is likely going on behind the scenes. Also the queries are lightening fast when run. I am thinking that they have some sort of table that is populated with some computed data, so its quick to query.
Any thoughts or suggestions to give me some direction, are very much appreciated !
My SQL acumen stems from just a couple courses, and everything since fromthe trenches. Fun + angst over time.I'm needing some advice on joins. Though I understand the basics, I'mhaving problems abstracting from instances where it's easy to think aboutdiscrete key values (. . . and studentid = 1234) to entire sets of users,with the joins doing their work.For example, currently I'm going nuts trying to return dates for whichattendance has not been taken for students, but should have been. Studentshave active and inactive periods of enrollment in our schools, so we have ahistory table of when they were active and inactive -- as well as two moretables that layer other bounds on eligible dates (what range of dates fallwithin a given school's term? What of holidays and staff institute days?).I also have a populated calendar table, and a table where students areidentified. Finally, there's a site history table which is a REAL pain inthe butt for me to think about.CREATE TABLE Student (StudentID int IDENTITY(1,1) NOT NULL,CurrentStatus varchar(2) NOT NULL)CREATE TABLE Calendar (Dateid int NOT NULL,Date datetime NULL,Workdaybit NULL )CREATE TABLE DailyAttendance (StudentID int NOT NULL,AttendanceDate datetime NOT NULL,SiteID varchar(6) NOT NULL,Attend_Status varchar(2) NOT NULL)(the last field is, e.g., present or absent)CREATE TABLE StudentActivityHistory (StudentID int NOT NULL,StatusStartDate datetime NOT NULL,StatusEndDate datetime NULL,Activity_Statusvarchar(2) NULL,StudentStatusHistoryIDint IDENTITY(1,1) NOT NULL)(the activity_status is either A or I; the important records in this tableare the 'A' records. A student's most recent status record always has anend date of '12/31/9999 12:00:00 AM', whether that's an A or I record. Nodates not between start/end dates of students' A records would needattendance taken. students may have many periods of activity -- A records-- as well as many inactive periods.)CREATE TABLE SiteTerms (SiteID varchar(6) NOT NULL,Term varchar(3) NOT NULL,StartOfTerm datetime NOT NULL,Quarter varchar(2) NOT NULL,SchoolYear varchar(9) NOT NULL,EndOfTerm datetime NOT NULL)(different schools vary their term start and end dates. No dates notbetween term start and end dates would need attendance taken by studentsassigned to and active in that school during that period.)CREATE TABLE SiteExceptionDays (SiteID varchar(6) NOT NULL,SchoolDayStartTimedatetime NOT NULL,SchoolDayEndTime datetime NOT NULL,SchoolDayType varchar(2) NOT NULL)(there are two kinds of days -- partial attendance, and no attendance. Inshort, if the type of day is "N" no attendance needs to be taken forstudents assigned to that school and active on that day)CREATE TABLE StudentSiteHistory (StudentID int NOT NULL,SiteStartDate datetime NOT NULL,SiteID varchar(6) NOT NULL,SiteEndDate datetime NULL,StudentSiteHistoryIDint IDENTITY(1,1) NOT NULL)(Pain. The attendance table tells which site a student was assigned whenattendance was taken. To find which school a student was a assigned to ondays attendance was NOT taken, this table's implicated 'cause it's the onlyway of connecting everything else together)Dangitall, I know this can be done but I've beat my head against the wall.Due diligence has gotten me a headache and a hankerin' for whiskey, and I'mnot much of a drinker. Is there anyone in the group for whom this kind ofthing is a no-brainer? I'd just as soon get some tips on how to approachthis kind of thing, and figure it out myself with some guidance.Any takers? Gotta run, dang I'm late for something.TIA--Scott
hello all , I need help to implement this package that will update/add/delete a row from one table to another.
I€™m trying to create this package:
Insert an entire row using SSIS from one table to another based on condition:
A for add, D for delete, C for change
MASStable
Column0
FirstName
Lastname
,MiddleName
A
John
daniels
D
sarah
jones
G
C
yann
coleman
J
daniel
lope
If column 0 = €˜A€™ in masstable add entire row to Deathtable If column 0 = €˜D€™ in masstable Delete row from Deathtable where Masstable.LastName = Deathtable.LastName If column 0 = €˜C€™ in masstable Update row (some columns) where Masstable.LastName = Deathtable.LastName If column 0 = €˜ €˜ no tasks..
Hi all My company is going to open new working centers on different locations of my region. One of the problems we are suffering is that at some locations, the network communications infrastructure is very very very poor. So, in that locations we work with low bandwidths, and connections usually break down.
Because of this, we are thinking in distributing our database. I have been doing some tests on replication, reading the docs, etc... But I am still not sure which replication strategy should we use, and how to organize our database tables to allow replication work properly.
Our offices are going to share some data (a product catalogue, for example) which could be updated an queried from any of the offices. But there is also data which is not going to be shared, as product stocks for each location.
We were thinking on using transactional peer-to-peer replication. But now we are having serious doubts about this since in a previous post (http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1525643&SiteID=1) we were told we can't store non-shared data (stock) in the same table as the shared data (products) due to how database behaves when a publication is restarted. We know it would not be usual to republish, but we were thinking on crash recovery.
It would be fantastic if somebody could help us to decide what should we do to organize our database to allow the use of a proper replication scheme.
I have only ever been required to take a full back up of my main prod database every night.
Now the times they are-a changing , and it is now required to be able to restore the database up to the last hour.
I've never really done much with tran log / differential backups so I'm asking for some advice as to what should be the best strategy. We are not a 24/7 shop we work from 6:30 am to 6:30 pm every day, so I thought:
Full backup @ 7pm Backup tran log every hour after that starting @ 7am (as there are no changes overnight)
How does that sound? also when the tran log is backed up, is it truncated? Or do I need to shrink it? Basically I need to know what to do so it doesn't get too big!
I have an exception occur when the "open" method is called on the connection object. Although I have this within a try/catch block, it doesn't get caught by the SqlException block and instead only gets caught by the "catch all" exception block at the bottom. I know that method is causing the problem because I set the text of an asp:label element at several places in the script to find out where the exception is being thrown. Shouldn't this be caught by the SqlException block. Does anyone have any ideas why this might not be getting caught? Thanks for any thoughts.
I tried searching, but it appears the search is still broken.Some of you know my other posts.. pleading for help with a problem (that's really sort of fixed itself/not remanifested itself).Upon this whole ordeal I decided to re-evaluate my nonchelant backup strategy. Which currently is:Backup our DB daily w/ transaction logs. (overwrite the backup daily on the tape).Backup our DB weekly w/ transaction logs.I rotate the tape weekly, on Monday morning. So there is a weekly backup and the last backup from monday morning on each tape.Along with that I have some optimizations and shrinks I run. Reading through the SQL BOL. It seems my strategy is ok, however if we lost the db at the end of the day, we'd lose a full days worth of work.So. Here in lay the questions.The BOL states to possibly back up twice a week, with differential daily backups and 4 hour transaction logs (example).So, I sorta get that..I make a device, "COMPANY.BAK", which writes to my external 'tape' unit.I make a job to back up the complete DB on sunday. to "COMPANY.BAK". (Not sure if I tell it to truncate the transaction log?).I make another job to make nightly differential backups to "COMPANY.BAK" with the 'append to media' switch on.I make yet ANOTHER job to make a transaction log backup. Here's what I don't get. If I set this thing to run every 4 hours, since it's not a differential backup.. rather incremental.. doesn't it just overwrite itself each time? It doesn't append itself to the backup right? So how much data could be missing if the log is overwriting itself rather than appending to itself? Also, if I turn on truncate log on backup, aren't I missing possibly critical transactions from the log for a sucessful restore?I guess I'm a bit befuddled here.
I would like to have guide for backup estore strategy for production DB..if i have to apply for high availability of production DB..
consider scenario ..Full backup every week, differential backups every 6 hours, and logs every fifteen minutes.
suppose my DB is crashed at 8:14 AM and my last logs were backed up at 8 AM then by looking above startegy then i lost my DB from 8 am to 8:14 am???? if not then where it shud find???
also looking above scenario by taking backup for such high frequency is it full my disk?? guide me pls...
I need to manage the problem of negative performance implications when I fragment a 1TB+ DB. I want to perform Index Reorganization if fragmentation is no higher than 30%, and Index Rebuild if the fragmentation exceeds 30%.
Firstly can anyone recommend a script which uses sys.dm_db_index_physical_stats system to ascertain the fragmentation level. Secondly, is there a technique I can employ to prevent the ONLINE operation completely killing performance on 27/4 production system?
Hi,Could anyone tell me the backup strategy for a 1000GB database?Thank you!Peter Wang*** Sent via Developersdex http://www.developersdex.com ***Don't just participate in USENET...get rewarded for it!
I found that sometimes when the cache is filled up, the performance drops significantly. Anyone knows the caching strategy behind the Lookup Transformation? If it's LRU or FIFO for example, I may have to sort the input based on FK to get better performance. If it's frequency based then sort might not help. Cheers,
i just completed local testing of an ssis package that I'd like to schedule for execution thru sql agent nightly. I dont mind trying this for a while without deploying either to a database, but am not sure if this is possible. Should my strategy be to get both to a server first or is there a way to plumb them together and see the pkg kicking off nightly before deployment of either?
I'm currently developing an Integration Service Solution using Visual Studio 2005. In this solution I have 4 packages. The first one call the second one that call the third one and so on.
To be able to debug those packages, the child packages are executed by Execute Package tasks, using a file system connection that point to the .dtsx file located in my project folder. It works great.
My problem is that I now need to deploy my packages on SQL Server. Since the deployment server is not the same as the development one, my Execute Package tasks fail - this is logical, they are looking for a file that doesn't exist on the deployment server. I'd like my parent package to execute the child packages located on SQL Server.
Of course, I could modify all my Execute Package tasks to use SQL Server instead of a file system connection before deployment but this is very painful (mainly because I have to do the opposite when I need to debug my solution).
Is there a way to streamline that process so that my Execute Package tasks automatically use SQL Server when I set Deployment configuration and file system connection when I set Development configuration ?
Don't know if this is the right forum to be asking this, but I'll give it a try...
I'm relativelly a beginner in SQL Server and T-SQL in general. The problem I'm trying to solve is the following:
The big picture is that I have data coming from different data sources which I need to store on a database for later reference. Each data source might have a different set of measurements. For example, data source 1 might log Pressure and Humidity while data source 2 logs Pressure and Temperature. Once the data is present on the DB, the users can go ahead and retrieve data for a given [datasource/measurement/time interval] to generate reports or charts.
My implementation so far consists of two tables: series_info and series_data. series_info holds general information for a given series of measurements for a given data source (Pressure for data source 1, Pressure for data source 2, Humidity for data source 1 and Temperature for data source 2, in our example). Each series has a bigint index as primary key.
The table series_data contains all data relative to the series from series_info. Each piece of data has a bigint as a primary key, an associate time (which is always crescent) and a foreign key to the series it represents (in series_info).
Alright, everything is cool so far. However, whenever a user wants to retrieve data for given [data source/measurement/time interval], this takes very long, since all data is interposed in series_data and for every search it's necessary to find where the desired data actually lies.
One obvious solution for this would be to dynamically create a new table to hold the data for each series, but that would just make my database disorganized, since there would be thousands and thousands of tables.
Another thing that comes to my mind is to create a table with information of where lies the data for a given [data source / measurement] for given dates. So when the user requested data for a given [data source/measurement] between, say, january and february, we would first look at this intermediate table and find out that the data lies between indexes 1000 and 2000 on the series_data table, so the next SELECT command to series_data would already contain a restriction like WHERE index>=1000 and index<=2000. This should probably improve the speed of retrieval.
What do you guys (or girls) think? Maybe there's simply a classical solution for such a case.
Regarding SQL Server data, I am looking to implement the beset Data-Archive and Purge policy. Normal, we do SQL Backups and keep the history for some period , for example, 8 weeks, so we can go back and restore any data point in time upto 8 month in past. and we also do Tape backups.
Question is Where can I get nice article or documentation on this to best design such policy where I make sure that I am covered for point in time recovery of database (which is sql backups) and point in time recovery in far past, say, 3 years ago using tape backups, and I need to make sure that I don't repeat the same efforsts.
