Data Mart Maintenance
Aug 6, 2007
The database maintenance on our data mart SQL server is causing problems. The re-index process causes too much file growth and the processes are unable to complete due to disk space constraints. I want to write a process to re-index the database, but handle this a single file group at a time and then shrink the file group before going to the next one. I'm looking to write a maintenance routine (stored procedure) that will get all of the tables from a file group and re-index them. Any general ideas would be greatly appreciated.
View 1 Replies
ADVERTISEMENT
Nov 1, 2006
Hi, an Operational Data Store is for short-term data storing (a few days at most) for providing real-time response to specific queries, but when you need to query about historical data you have to go down in the chain and build a data mart, is that true?
I have just one corporative database (which I need to real-time query and report) and I think that you need a data mart when you need to unify several departments that have different names for the same product for example, or when those departments are from different regions in the country but when you have a "common language" in the company do you need a data mart or you just have a ODS? If you have just an ODS does it work for historical data?
Now I think that the ODS could feed the data mart on a regular time basis (for example every week) that's the way you deal with long term historical queries? I also think that I'm in the first case of the 3 layouts proposed here http://www.informit.com/guides/content.asp?g=sqlserver&seqNum=166&rl=1 because I don't have several ODS to integrate in one data mart and I'm working with just one company data but I don't know if this layout handles (in real time response) the long term historical queries that the management is asking me for.
View 3 Replies
View Related
Oct 9, 2015
our group is debating whether we should use views or data marts. WE have some huge queries that can have many joins up to 20 in some cases. One part of our group wants to use views to pull this data and another thinks we should create a data mart off an SSIS package and run it early every morning and then have the users access the data directly from there.
I am not really sure how a view would speed things up. If I have 20 users and they all call a view the view is created 20 times is that not correct? Since a view is basically just a stored sql statement (not a stored proc) I am not seeing how this is any more efficient.
View 8 Replies
View Related
Aug 31, 2006
i want to create a data mart from an existing OLTP database. for example northwind or i will create an OLTP database. i dont know how i can create data mart from OLTP database. i want to learn that step by step. help me??? please!!
View 5 Replies
View Related
May 30, 2006
Hi all,
I'm new to the forum, and fairly new to SQL, so please forgive me if I say something that sounds stupid at any time. I have been looking around this forum (and others) while trying to diagnose my current issues, and there seems to be a really good amount of knowledge about here so I thought I'd try my luck with a post.
We've been running SQL 2000 for around 2 years now, and had very few problems until we suffered a torn page error around 6 months ago (just before the time I started looking after the server).
Since then we've been getting intermittent database corruption (I implemented a nightly DBCC job as part of DB maintenance) and it seems to occur after our weekly index maintenance has run (also a scheduled DBMaint job). I have just confirmed this by restoring nightly backups to a test database - the current round of corruption has occured sometime between the Saturday and Sunday night backups (indexing runs at 1 am Sunday morning, and is the only activity between these two backups)
We have 9 small (all less than 200MB) databases which are used faily lightly by our salesforce. The corruption is occurring on different databases each time.
We are running SQL 2000 (SP4) on a Windows 2003 clustered server (2 x DL380s and an MSA500 G1) running in failover mode (active-passive). The raid 5 array does utilise disk caching, and I have wondered if this is involved somehow.
Our hardware support comany have checked the server thoroughly for HW issues, but have come up blank.
Here is the DBCC output from one of our corrupt databases from this morning:
dbcc checkdb (casestatus) WITH ALL_ERRORMSGS, NO_INFOMSGS
-----------------------
Server: Msg 2533, Level 16, State 1, Line 1
Table error: Page (1:20040) allocated to object ID 117575457, index ID 0 was not seen. Page may be invalid or have incorrect object ID information in its header.
Server: Msg 8976, Level 16, State 1, Line 1
Table error: Object ID 117575457, index ID 1. Page (1:20040) was not seen in the scan although its parent (1:19808) and previous (1:20039) refer to it. Check any previous errors.
Server: Msg 8978, Level 16, State 1, Line 1
Table error: Object ID 117575457, index ID 1. Page (1:20041) is missing a reference from previous page (1:20040). Possible chain linkage problem.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 1, text ID 1190656606208 is not referenced.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 3, text ID 1190656671744 is not referenced.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 5, text ID 1190656737280 is not referenced.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 7, text ID 1190656802816 is not referenced.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 9, text ID 1190656868352 is not referenced.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 11, text ID 1190656933888 is not referenced.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 13, text ID 1190656999424 is not referenced.
Server: Msg 8964, Level 16, State 1, Line 1
Table error: Object ID 117575457. The text, ntext, or image node at page (1:12449), slot 15, text ID 1190657064960 is not referenced.
CHECKDB found 0 allocation errors and 11 consistency errors in table 'Raw:LifeStatusFP' (object ID 117575457).
CHECKDB found 0 allocation errors and 11 consistency errors in database 'CaseStatus'.
repair_allow_data_loss is the minimum repair level for the errors found by DBCC CHECKDB (CaseStatus ).
-----------------------
The funny thing is that this particular database isn't even in use any more - I've had it in single user mode for about 2 months (and the users don't have any access to the back end stuff).
If anyone can shed any light on what's happening, I'd really appreciate this.
Thanks
Dave
View 5 Replies
View Related
Dec 10, 2007
Hi,
I really don't know how frowned upon my approach is here, but it was the only way I have been able I've been able to do it.
On my application, when users delete their account, it sometimes brings the db server to a COMPLETE crawl. The reason is some users who delete have many years of related data, and when the data deletes with them, its very slow.
To avoid this I've taken off many contstraints, and I do have some sprocs that deleted orphaned data at night.
thoughts on this approach ?
View 5 Replies
View Related
Aug 17, 2007
Hello,
I have a question that I hope someone can clear up for me. I have come across a number of different suggestions on DB maintenance, for example reindexing with the following script:
USE DatabaseName --Enter the name of the database you want to reindex
DECLARE @TableName varchar(255)
DECLARE TableCursor CURSOR FOR
SELECT table_name FROM information_schema.tables
WHERE table_type = 'base table'
OPEN TableCursor
FETCH NEXT FROM TableCursor INTO @TableName
WHILE @@FETCH_STATUS = 0
BEGIN
DBCC DBREINDEX(@TableName,' ',90)
FETCH NEXT FROM TableCursor INTO @TableName
END
CLOSE TableCursor
DEALLOCATE TableCursor
My question is, doesn't the maintenance plan have this functionality inherent in it when you create the maintenance jobs to reindex? Is there a benefit to scripting things out vs just using the maintenance plan wizard for this sort of thing and any of the items it covers? I came from an Oracle background where this was a no-brainer but I am a bit confused on the choices with SQL Server.
Thanks.
View 1 Replies
View Related
Jun 18, 2015
I am testing some maintenance tasks sql commands such as index rebuild, index reorg, update statistics and db integrity check on a SQL Server 2014 Database. This is a new non-production vendor database (DB Size 500 GBs, Log Size 25 GBs) which eventually will be created in production. Currently, it is in full recovery model and without log backups. The database has a whole lot of indexes. I am just trying to rebuild and reorganize all the indexes (that need it), in addition to trying to get an idea of how long these maintenance task will take and the space needed in the log file to complete these tasks/commands. I would like to execute these tasks manually (the first time) to gather the duration and space required information. Eventually, I would probably schedule a weekly job to perform this maintenance.
I ran the index rebuild task on the database and noticed that the log file grew by over 50 GBs. I killed the process and truncated and shrunk the log file back down.
1. Does the index rebuild, index reorg, update statistics and db integrity check commands all use the log file?
2. Does Indexs Reorg have less impact on log file then Index Rebuild?
3. Should a truncate log and shrink log file be performed after these maintenance commands?
4. Should a full database backup be performed after these maintenance commands? Or before the maintenance commands?
I have read and understand that shrinking is not good for the database (could lead to more fragmentation and more data file growth when data is added) and I know about rebuilding indexes when fragmentation is GT 30% and reorganizing indexes when fragmentation is GT 5% and LE 30%.
Since this is a non-production database maybe I should set the recovery model to simple, run the maintenance commands and leave the database in simple recovery model unless the vendor needs it in full recovery model for some unknown reason.
5. With the simple recovery model the log file should be reused in a circular manner and not grow during these maintenance tasks. Is this correct?
View 3 Replies
View Related
Mar 5, 2001
I have deleted a database from SQL Enterprise Manager. Anyone know a way to clear that database from my maintenance plan? I do not wish to just uncheck the deleted database or create a new database plan.
Thanks!
View 1 Replies
View Related
Jun 11, 2001
My index maintenance job that was setup through Enterprise manager database maintenance fails with the following notice. It ran great for several weeks then it started failing. Any suggestions!!
sqlmaint.exe failed. [SQLSTATE 42000] (Error 22029). The step failed.
View 1 Replies
View Related
Aug 31, 2001
Hi, anyone who administering the pretty big database not less than 30 Gb with the average number of rows in a table about 2M and more, please share you experience with maintenace of such a db. Esspecially i'm interesting in:
1) Indexes maintenance (When and how - just regular dbcc, maint. plan or some script to split the job twice and so on.)
2) Remove unused space from db. (not major)
The serever works 24*7, and it's transactional environment. SQL 7 sp3 on claster.
I run the sp. to rebuild all the indexes it takes about 2-3 hrs to determin the objects withfragmentation less than 80% and actually rebuilding, during this process the users experience the performance (specially for update/insert) problem. It looks like I need to change the plan or strategy to do this. Any thoughts appreciated!
Thanks in advance.
Dmitri
View 2 Replies
View Related
Oct 23, 2001
Hello All
I have been given a SQL Server 2000 database to look after which has been set up with a Database maintenance plan. The plan is set to backup the complete database and the transaction log. The backups are written to the local disk correctly but the plan is also set to remove any backup files (both database .BAK and transaction log .TRN) that are over one week old. Complete database .BAK files are written daily and the .TRN are written every hour daily. The .BAK files are removed ok automatically but the .TRN files are not - they are just slowly filling the disk. There does not seen to be anything different between the way the main database and the transaction log is set up in the maintenance plan.
I would be very grateful for any ideas
View 1 Replies
View Related
Feb 14, 2000
I am looking for opinions of setting up a database maintenance plan. I want to know if it is safe to trust the wizard and let it set up all of the jobs, or if it is better to write your own procedures to handle backups and maintenance as in 6.5. All sugestions and opinions are welcome. Thanks.
View 2 Replies
View Related
Feb 2, 2000
For SQL server 7.0, is it necessary to schedule database maintenance plan on a regulare basis? I know it is necessary for SQL server 6.5.
Thanks.
Su Ge
View 1 Replies
View Related
Aug 31, 2000
I have a strange thing in one of our Maintenance plans.
On the first tab where you check which databases you're including in the plan I have (say my database name is CAT) a 'CAT' and 'cat' database listed and the one chosen is 'cat'. However my database in all other views shows up in all caps. (even when I do an sp_helpdb)
The backups look like they're working, etc. but it just seems weird. If I go to create a new plan it only gives me the one option 'CAT' which is really what's there. I'm new and I'm thinking the database at one time was 'cat' and this is when the maintenance plan was created. Then it was renamed to 'CAT' and there's the two db's showing in the old mainenance plan.
What would you do? Create a new plan with "CAT" and just get rid of the old one with the weird 'cat' and 'CAT'?
Any other suggestions or ideas on what happened..
ann
View 1 Replies
View Related
Oct 4, 1999
I've created a database maintenance plan to backup a database, but it just
isn't happening, am i missing something. The maintenance plan appears to be
created successfully.
responses appreciated.
thanks
Todd Minifie
View 6 Replies
View Related
Feb 20, 2003
SQL7: I have added a Maintenance Plan to backup to 4mm dat tape the master and msdb SQL databases as well as another database relative to our application called WISE. This works fine; however, it appears to always append to the media as opposed to overwriting (preferred). Any help would be appreciated.....
View 1 Replies
View Related
Aug 22, 2005
Hi,
I am going to set up maintenance plans on all our SQL servers (7.0 and 2000). I have found several 'tutorials' on how to do this, but no one is describing the options in detail. Can you guys/gals please help me out? We have alot of small databases and some medium (1-2GB).
Thanks//Stefan
View 3 Replies
View Related
Feb 11, 1999
Help !!
I am running a database of 500-600mb 20-30% of which is new data daily (5 day old data being deleted as part of the nightly maintenance) And my nightly maintenance is regularly taking an hour plus.
CheckDB, New Alloc, Catalog, re-indexing and dumps are performed nightly (2am ish) and as the system is in constant use I cannot afford such a long task. I can't use weekly dumps/checkDB as we use transaction log replication and these are dumped every minute. I really need some suggestions on how I can improve matters. The deletion of old data in particular is taking a long time due to the use local variables but is there a faster way to do this :
OPEN tnames_cursor
FETCH NEXT FROM tnames_cursor INTO @connectionid
WHILE (@@fetch_status <> -1)
BEGIN
IF (@@fetch_status <> -2)
BEGIN
Select @dRent = DeliveredRetention from ControlDB..connectiontable
where ControlDB..connectiontable.Cid = @connectionid
Delete from MyDB..Table where Cid = @cid
and DateDelivered != NULL
and Datediff(hh,MyDB..Table.DateDelivered,getdate()) >= (@dRent*24)
END
FETCH NEXT FROM tnames_cursor INTO @connectionid
END
DEALLOCATE tnames_cursor
GO
These jobs have also started running out of locks and deadlocking on occaision which seems odd as the system has 10000 available (escalating at 2000)
Any Suggestions would be very much appreciated
Damon
View 1 Replies
View Related
Feb 1, 2002
hi,
I have SQL 2000 ProductionBox.It is in 24x7 environment.
We need to maintain data only for 30 days.
Even if I schedule deletetion of data on daily basis - SQL will take the lock as data we receive is too huge.
Secondly,Since the indexes are heavily used - defrag don't work for them.the only option I think I am left with is to rebuild the indexes.Though in SQL2K - index creation is on fly - but here we are talking of table sizes of 8-10GB.
I suggested my Boss to bring down the box for few hours for maintenance.but he insists that since this is 24x7 - Box can never be brought down.
I am finding it hard to convice him and do my job.
Any idea on how to rebuild the indexes & how to delete this many records(avg.50k per day-data of xml type)without creating a block is highly appreciated...or how to convince my boss to give me a window for maintenance...:)
TIA
View 3 Replies
View Related
Apr 12, 1999
SQLMaint is run once a week for the following a database on SQL 6.5. The following is the information for the database when I see it through the Enterprise Manager:
Data Size 650 MB
Data Space Available 0.00
Log Size 360
Log Space Available 359.99
The following is the syntax built by the DATABASE Maintenance Wizard:
SQLMAINT.EXE -D CATS -CkDB -CkAl -CkTxtAl -CkCat -UpdSts -RebldIdx 100 -Rpt E:MSSQLLOGCATS_maint.rpt
It runs once a week and takes about 40 mins and runs successfully. Last it run was 4/11/99 at 2:00 AM
The result set I get from sp_spaceused is as follows:
database_name database_size unallocated space
CATS 1010.00 MB 273.96 MB
reserved data index_size unused
------------------ ------------------ ------------------ ------------------
753710 KB 280360 KB 426494 KB 46856 KB
What I don’t understand is how come the data space available shows 0 in Enterprise Manager? Shouldn't SQLMAINT, which is run once a week, allow for correct information to be reported?
Could someone please explain.
Thanks
Shashu
View 2 Replies
View Related
May 6, 2001
Hello,
I am just getting started with SQL 2000 Server, and we have our database online, and starting transfer all the data from ACCESS to SQL 2000.
Need to know what type of maintenance I need to do to keep the data clean on SQL.
Any help would be appreciated.
View 2 Replies
View Related
Oct 1, 2004
Can you generate script for a maintenance plan?
I know how to script a job, I was wondering about a plan.
If not, whats the best way to record the configuration?
Thanks
Lystra
View 3 Replies
View Related
Dec 14, 2004
I have been having an extremely annoying problem with SQL server. About 3 to 4 time a day, it starts running some job that takes 30+ minutes to finish. The problem is that it bogs the system down, and consumes so many resources, that it it is almost impossible to run anything while the job is running. Most of the time, this job runs when the server is idle. And, much of the time, it has been idle for at least 30 minutes, and often longer. Also, there is excessive hard drive activity while this task runs.
I am unable to find out what is going on because Enterprise manager times out trying to connect to it, and other tasks remotely connecting either time out or get a network error trying to connect. I have task manager running all the time and it shows task 'sqlservr.exe' hogging the system when this is happening.
Can anyone shed any light on what is happening, why, and how I can stop this?? If it is performing maintenance, is there a way to get it to schedule this for specific times rather than during normal idle system activity?
View 11 Replies
View Related
Feb 15, 2005
We have Veritas' Backupexec running in our Enterprise and the Veritas Install actually installs MS SQL Server MSDN on each Server in the Enterprise.
It looks like it also sets up a default Maintenance plan within each of the MSDN Instances.
I guess my question is.. Can I manage the Maintenance Plans on these MSDN Instances via the SQL Server EM GUI from my desktop?? Seems like when I look at the Maintenance plans alot of the options are greyed out or not available. What I am trying to do is modify one of the maintenance plans to have the backups deleted after one week (One of the Instances has been running a complete backup on the Backupexec Databases for a year and there are a years worth of backups on the Server) but the option to "remove files older than" is 'greyed out' ??????
View 6 Replies
View Related
Nov 3, 2005
Hi. I'm totally new to this whole sql server world so bear with me on this. I tried creating a maintenance plan which consist of a backup of all of my db on the server and whenever I try to run the job it creates, it just hang there like I have done nothing. I have created the maintenance plan with the administrator account on the server and I have tried to run it like that but no dice. If any of you can give me any hint on what could be happening, I would be very appreciated.
Thanks!
View 2 Replies
View Related
Jul 20, 2007
Hello everyone,
I was interested in finding out if anyone knows any helpful websites, articles, and/or postings for recommendations on server maintenance.
Maintenance such as defrag, logging...and so on. Our company has a good backup system but we would like to improve on everything else (basic/advance maintenance).
If you have any best practices secrets or know of any good resources, it would be much appreciated to hear from you. If you need more info, just let me know. We have SQL Server 2005.
-Cindy.
View 1 Replies
View Related
Feb 20, 2008
Eh uhhhh where did the old '-DelBkUps 7DAYS' (delete files older than 7 Days) option go?
Is this a future product enhancement?
I fee like such a dirty noob.
View 3 Replies
View Related
Apr 7, 2008
hi everyone..
this is a little bit weird ..
i am trying to make a backup strategy. i am using sql2005.
when i go to maintenance plan. right click >> new maintenance plan...
nothing happens..
if i go with the maintenance plan wizard everything goes normally.
after doing the backup, if i right click on it and press
modify , nothing happens too.
what i mean by nothing happens is that it doesn't open the "design view".
the back up is doing normally.. but i need to set a range of 5 days before overwriting the oldest backup.
any idea what is going on or what am i missing?!
thank you
View 2 Replies
View Related
Apr 9, 2008
Windows Server 2003R2 w/ Sp2
SQL 2005 w/ Sp2
Created weekly (full backup) and a daily (differential backup) Maintenance Plans using the wizard. I formatted the server, installed the OS and SQL. Restored the full backup (No Recovery Mode), then restored the differential backup (Recovery Mode), tested and all worked well.
Then I noticed the original Maintenace Plans I created (Full and Differential) were gone; makes sense as I had formatted the server.
Is there a way to create a Maintenance Plan file or script that I can save and just add back to the server??
Hope that makes sense.
Any help appreciated.
Kerry
View 3 Replies
View Related
May 16, 2008
We have a training server and I've had a request that after each training session, we have the ability to quickly roll the training server back to its previous state so that the next group of people can be trained with the same examples.
Doing a restore requires that the maintenance (defrag, warm cache, etc..) also runs after that before the server performs fast enough.
So I was thinking of snapshots as an alternative. When you roll back to a snapshot, does that invalidate what's in the cache or have any adverse effect on query plans, stats, or indexes?
View 1 Replies
View Related
Feb 26, 2006
Hello 2 all,
Could someone advise and/or correct me with my thoughts on how I would do my db maintenance plans?
(db's on SQL2000 as 'full' model)
Backups:
1) Daily Transaction log backups scheduled frequently enough.
2) Full Backup scheduled daily. Good way to start I presume ;)
Maintenance: Would be scheduled daily if possible, on non-production hours and if not colliding with daily full BU schedule.
3) Full DB reorg data&indexes.
4) Update Query Optimizer Stats (although 'Auto Update Stats' is on)
5) Shrink the logfile (ldf) as I presume this will have grown due to previous maintenance jobs.
6) If 5 ok, alter ldf filesize back to new allocated size.
Rgds,
T.
View 6 Replies
View Related
Mar 9, 2006
Hi
As per my backup policy I need to do a full backup dialy & transactional backup every 30 minutes , can i use the Maintenance plan to do this
or should I write T-sql statements & create jobs.....
I mean whats the difference between the two methods.
Thanks
View 5 Replies
View Related
