I'm writing a small script to clean up our database. We have a couple of databases which contain many gigabytes of data.
The script fills a few temp tables, with price-id's which can be deleted (based on a few rules). Then it deletes all related data first, before actually deleting the price records themselves.
This works fine, except for performance issues. After 12 hours, I had to cancel the running script as it was taking too many resources.
My question is, how can I increase performance? Should I add 'commit' after deleting data in each table? Would it help to make it a stored procedure?
Script looks kinda like this:
SELECT priceId INTO #tobedeleted FROM prices WHERE ... (few rules)
DELETE FROM price_product WHERE priceId IN ... #tobedeleted
....
....
We have SQL SERVER 2000 Runnin on our server. We are trying to do a db cleanup, ie all tbls/views that were created earlier and are not needed any more, need to be deleted. However, is there a way, to do a cleanup in a better method, other than going thru the whole bunch of tbls/views manually,and determining which ones are needed or not and delete the ones that are not needed.
Which sp code part is efficient? 1st code from sp_MSdistribution_cleanup. Used for Replication Cleanup job. For 1500 rows, runs a day. Is something wrong with this code? 2nd code part is an alternative idea!. /************************************************** *****************/ delete MSsubscriber_status from MSsubscriber_status ss1 where publisher_id = @publisher_id and publisher_db = @publisher_db and subscriber_id = @subscriber_id and subscriber_db = @subscriber_db and job_id < @max_cleanup_job and job_id <> (select max(job_id) from MSsubscriber_status ss2 (index = ucMSsubscriber_status) where ss2.publisher_id = @publisher_id and ss2.publisher_db = @publisher_db and ss2.subscriber_id = @subscriber_id and ss2.subscriber_db = @subscriber_db) /************************************************** *******************/ select @maxCleanup_job = max(job_id) from MSsubscriber_status (index = ucMSsubscriber_status) where publisher_id = @publisher_id and publisher_db = @publisher_db and subscriber_id = @subscriber_id and subscriber_db = @subscriber_db delete MSsubscriber_status from MSsubscriber_status ss1 where publisher_id = @publisher_id and publisher_db = @publisher_db and subscriber_id = @subscriber_id and subscriber_db = @subscriber_db and job_id < @max_cleanup_job and job_id <> @maxCleanup_job
I need to restore a DB but it was prevented by a background process of "Ghost Cleanup". server is SQL2000 ENT. sp4.
It could not be killed, neither it was stoped after restart the server. Is there a way to change its running schedule and/or to kill it when I need to restore the db?
I would love to be able to run the distribution cleanup job with a switch that says cleanup all distributed transactions.
Because when I use peer to peer replication the @allow_initialize_from_backup publication property is set to true which is good. But it has the down side that transactions are stored the max retention period in the distribution database. I want to use the deafault 72 hours for my retention period so that the subscritions don't get deactivated but in a system with a high transaction rate there wil be a lot of transactions in 72 hours. This means that the cleanup job will have a tough time to figure out which transactions to delete so the cleanup job will run for a long time not a very big problem but the problem is that the cleanup job will keep the log reader agent from delivering trtansactions to the distribution database and the subscribers won't get their data in time.
Could Microsoft please give me a switch so I can choose when I want to save my transaction and when I want to delete them as soon as they have been delivered to all subscribers?
Is this a feature in SQL Server 2008? Could it be released in SP3 for SQL Server 2005. (The SP 2 cleanup job has a bug so I have to use the SP 1 verison of the cleanup job)
Ques; can "both" files the db backup (.bak) file "and" the (.txt)report file in a maint plan object be cleaned up at same time?
The object is working ok but trying to setup and match backup text reports to the db, I have way more .bak files than text files.
Created from/after db maint plan (which is working ok) from/in the object Maintenance Cleanup Task and the object selections are
Delete files of the following type: _backup files _maintenance plan text reports
File location: _delte specific file File name: __ _search folder and delete files based on an extension Folder:__ File extension:__ File age: _Delete files based on the age of the file at task run time... delete files oder than the following. . . .
I've noticed that wheter if you put the condition on files' age or not it generates exactly the same T-SQL statement, it says that the actual one could be different if you set conditions on the task, but if it is so why they work the saem way?
I have problem with 'Ghost cleanup' system process. It is locking up my tables and user transactions are keep getting wait status. So is there any way to disable or change the schedule of ghost cleaner? Thanks in advance..
sysmail_mailitems table in msdb stores data for emails sent using database mail. The DB was growing and I found sysmail_mailitems table is taking almost 85% space. What is the procedure to clean up data in this table. I did dirty way of removing FK's for the table, truncating data and re-establishing FK relationships.
Is there some way or settings somewhere to schedule cleanup on this table?
Using SQL Server 2005. Replication working fine except the distribution table is continuely growing. Started to configure maintenance job (catagory: REPL-Distribution Cleanup) Any suggested steps that will not break the replication. Is there a SP available that will address my problem.
I noticed that several connections were blocked by something called Ghost Cleanup (or something like that). I know what the cleanup does, but it often causes blocking for quite a while . . .
I am using Ola Hallegren's scripts to do backups. He uses @Cleanup Time to delete backups older than a certain number of hours. My situation is I do a full backup of a database on Sunday and then I have a few Differentials and then log backups for the rest of the week. When Sunday rolls around again and my full backup is finished, I would like to delete all the differential backups and log backups. Any way that I could accomplish this using Ola's scripts?
I need to setup a Server Audit in sql 2012 standard edition. The goal is to identify all procs, views, tables, functions that are not being used anymore. What is the best way to setup a server audit to accomplish this? What events need to be captured at a minimum so I can delete all the unused crap in this database. This is a smaller, growing company.
At a client site we just had a customer away on holidays, came back and went to replicate and received the error. I understand why this would happen after the default 14 days, and the obvious fix to prevent it from happening in the future is to extend the period, however I am hoping someone can offer me assistance on the best way to correct the issue now
Error is:
The Merge Agent failed after detecting that retention-based metadata cleanup has deleted metadata at the Publisher for changes not yet sent to the Subscriber. You must reinitialize the subscription (without upload).
The problem is we cannot lose the data that has been entered on this machine since the last replication. Writing scripts to manually save and pump this data will take days if not weeks (complex database). Is there any way to issue a command to cause a manual upload of the new information that is at the subscriber. Then I could just delete and recreate replication?
I have a 3 tier replication with one master server on tier1, 3 servers on tier 2 and up to 20 remote servers (tier 3) on each of the tier 2 servers. I recently had to delete the subscriptions to each of the tier 3 servers from the tier 2 servers and then recreate the subscription. All went well for most of the subscriptions, but for 3 subscriptions the subscription was dropped at the tier 2 level but failed to drop at the subscriber due to a time out. I went to the tier 3 servers in question and issued the sp_removedbreplication command that supposedly executed properly. On one of these 3 servers everything worked fine and I was able to re-subscribe successfuly and have everything continue to work properly. On the other 2 it appeared the re-subscription worked properly until users began to try to update the database through adds and updates (all controlled through stored procedures). Users receive an error message claiming an invalid object name ctsv_ followed by guid looking string of characters. I suspect it has to do with all of the stored procedures that were created by merge replication when the subscription was set up and initialized (no sync). These include numerous sp_ins_, sp_sel_, and sp_upd_ ... named procedures and a bunch of dt_ ... stored procedures as well.
How do I clean this up so that I can have users using their database again and then re-subscribe again short of rebuilding the database (a very time consuming process)?
After cloning our production SQL 2012 server to make DEV environment, there is a bunch of databases in "Suspect" mode. This is because the DB's and Logs have got out of sync.I can clean up each DB with this script:
ALTER DATABASE DB_NAME SET EMERGENCY GO DBCC CHECKDB (DB_NAME) GO ALTER DATABASE DB_NAME SET SINGLE_USER WITH ROLLBACK IMMEDIATE GO DBCC CHECKDB (DB_NAME, REPAIR_ALLOW_DATA_LOSS) GO ALTER DATABASE DB_NAME SET MULTI_USER GO
but this takes a long time replacing DB_NAME and executing for each DB.I tried putting it all into a script with variables, but I have done something wrong and it does not work. Also is there a way to just apply it to the suspect DB's?
USE master Go DECLARE @dbname sysname, @cmd varchar(1000), @cmd1 varchar(1000), @cmd2 varchar(1000), @cmd3 varchar(1000), @cmd4 varchar(1000) DECLARE db_recovery_cursor CURSOR FOR SELECT name from sysdatabases OPEN db_recovery_cursor
I added a maintenance CleanUp task pointing to the right folder I checked the include first-level subfolder box, and delete files based on the age of the file at task run time
I specified "delete files older than " 1 day
But the cleanup task is not working, I have to manually go a delete the files myself.
Am I missing something? should I run the maintenance task first? and then the backup task?
I am running a couple of sql 2000 SP3a servers with merge and snapshot replication. One server acting as publisher and distributor and the rest subscribers. On one of the server I have got the error below and have tried most of the suggestions by msdn. This server has not crashed ever before or any hardware problems. It has been running for a couple of months and no problems. This has not happened no any of the other servers. Any suggestions would be greatly appreciated as the only resolution I have left is to bring up a new instance, setup replication and see if this would resolve the issue. Stopping and starting of agents don't work.
Server: EASTSRV3 DBMS: Microsoft SQL Server Version: 08.00.0760 user name: dbo API conformance: 2 SQL conformance: 1 transaction capable: 2 read only: N identifier quote char: " non_nullable_columns: 1 owner usage: 31 max table name len: 128 max column name len: 128 need long data len: Y max columns in table: 1024 max columns in index: 16 max char literal len: 524288 max statement len: 524288 max row size: 524288
[4/18/2005 11:59:27 AM]EASTSRV3.ICASData: {call sp_MSgetversion } Percent Complete: 2 Connecting to Subscriber 'EASTSRV3' Percent Complete: 3 Retrieving publication information Percent Complete: 4 Retrieving subscription information Percent Complete: 4 The merge process is cleaning up meta data in database 'HO_Master'. Percent Complete: 4 The merge process cleaned up 0 row(s) in MSmerge_genhistory, 0 row(s) in MSmerge_contents, and 0 row(s) in MSmerge_tombstone. Percent Complete: 4 The merge process is cleaning up meta data in database 'ICASData'. The merge process could not perform retention-based meta data cleanup in database 'ICASData'. Percent Complete: 0 The merge process could not perform retention-based meta data cleanup in database 'ICASData'. Percent Complete: 0 Category:NULL Source: Merge Replication Provider Number: -2147199467 Message: The merge process could not perform retention-based meta data cleanup in database 'ICASData'. Percent Complete: 0 Category:COMMAND Source: Failed Command Number: 0 Message: {call sp_mergemetadataretentioncleanup(?, ?, ?)} Percent Complete: 0 Category:SQLSERVER Source: EASTSRV3 Number: 11 Message: General network error. Check your network documentation.
I have a problem with deleting old bak files from a Cleanup Maintenance Task. I have backups in subdirectories and I read in SQL Server 2005 Books Online that:
In contrast to earlier versions of the Maintenance Cleanup task, the SQL Server 2005 version of the task does not automatically delete files in the subdirectories of the specified directory. This constraint reduces the surface area of any attack that could exploit the functionality of the Maintenance Cleanup task to delete files maliciously. To delete the first-level subfolders, you must explicitly elect to do this by checking the Include first-level subfolders option in the Maintenance Cleanup Task dialog box.
I cannot find such an option in my Maintenance Cleanup Task dialog box. I do not think I installed any service packs yet, could this be the problem?
Hi. I'm very new to SQL Server and would greatly appreciate any advice with regard to my two questions below.
I've recently been given responsibility for a pair of clustered Windows 2003 servers running SQL Server 2005. Looking through SQL Server Management Studio, I found the following three maintenance plans:
- System Maintenance Plan (all system databases) - Plan 1 (1 database) - Plan 2 (4 databases).
All three maintenace plans had the following elements configured, in the following order: - Check Database Integrity - Backup Database (Full) - Shrink Database - Update statistics - Clean Up History
with the exception that Plan 1 doesn't have a 'Shrink Database' task (because, I'm told, its data is pretty static), but none had a Maintenance Cleanup Task, so I added one to each plan specifying 14 days of old backups to be kept in all cases except Plan 1, where I limited it to 2 days, the database being over 7 GB in size. All links between tasks in all plans are AND constraints, value 'Completion'. All .BAKs are written to tape by Symantec Backup Exec as part of the daily backup.
Two questions from this:
1. Since .BAKs are written to separate folders for each database, do I need, for example, four Maintenance Cleanup Tasks for the Plans covering four databases - one for each folder/database? I can't see a way of specifying that a Maintenance Cleanup Task should apply to multiple databases or search subfolders based on database name, so I assume I need four consecutive Maintenance Cleanup Tasks within a single plan? Is this the way to do it?
2.The data displayed when I right-click a maintenance plan and choose View History shows:
date and time run plan name task name duration log type log source
Since I added a Maintenance Cleanup Task to each plan, the only task shown in the Task Name column is Cleanup History (set to 'Older than 7 days'), and the duration is 00:00:00, where previously all tasks were shown and the overall duration was over a minute.
However, the .BAK files are being created (size looks OK), the application log contains entries suggesting the backups and other tasks completed successfully, and if I expand the SQL Server Logs node and view the current file I can see entries which give the same 'success' messages (presumably this log is the source of the 'success' messages in the application log?).
Why does there appear to be an anomaly between the 'View History' display and the SQL Server Log and application log? Should I be worried, or is what I can see in the SQL Server Logs and application log sufficient grounds for a good night's sleep?
I have read from a variety of sources that using the "WITH CLEANUP" option on a "END CONVERSATION" statement is bad and unnecessary. (Question #1) Is this true???
My code does not work properly if I don't use the "WITH CLEANUP" option. My code leaves closed conversation records in the queues if I leave out the "WITH CLEANUP" option. The "END CONVERSATION" statement is executing properly and flagging the conversation record as closed but the records don't get deleted. All the messages are going back and forth properly too.
My code is based on the HelloWorld ServiceBroker sample which does not use "WITH CLEANUP". When I run the sample scripts everything works great and the conversation records are deleted. However, this sample does not uses an activation stored procedure to receive messages and respond with the results. When I copy and paste the receive messages sample code into an activation stored procedure is when the problem comes up. It's the same code! (Question #2) Why am I getting different results depending how the code is executed/activated???
This is could be a tough one. I just hope somebody else has seen it too and figured it out. Thanks for the help!
We are using SQL Server 2005 SP2 to do transactional replication.
We and have a separate service account for the SQL Agents (sqladmin) vs. SQL Replication Agents (sqlrepadmin). It is my understanding this is a replication security best practice. The sqlrepadmin has full permissions on the snapshot share folder and it's subdirectories. The sqladmin account does not have permissions at all.
I have been getting an error message when we run the distribution clean up job. Executed as user: PRODsqladmin. Could not remove directory '\Tes01boxRepldatauncqabox01_DB01_TO_ORACLE20070905104896'. Check the security context of xp_cmdshell
I have dropped the publication and recreated which is what appears to have caused the error.
From http://technet.microsoft.com/en-us/library/ms151151.aspx
Note:
If a publication is dropped, replication attempts to remove the snapshot folder under the security context of the SQL Server service account. If this account does not have sufficient privileges, log in with an account that does have sufficient privileges and remove the folder manually. Removing a folder requires the Modify privilege if the folder is a local path or the Full Control privilege if the folder is a network path.
The note above implies that the SQL Server service account (sqladmin) needs permissions on the snapshot folder as well.
Finally my questions: Is there a workaround that will allow the distribution cleanup job to run as sqlrepadmin and perform the delete?
If both sqlrepadmin and sqladmin need permissions to the snapshot what is the reasoning from a security perspective of separating them out?
I've had problems trying to get a new maintenance plan on a new server to work.
I'm running a plan that does an integrity check, reindex, shrink, backup, the a maint cleanup to get rid of backups older than 2 weeks.
I'm getting failures that say:
Error Number -1073548784
Error Message: Executing the query "EXECUTE master.dbo.xp_delete_file 0,N'',N'BAK',N'2007-10-18T18:43:41',1 " failed with the following error: "Error executing extended stored procedure: Invalid Parameter". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Yes, this is a post SP2 server. Current version, with the patches I've applied looking to fix this is: 9.0.3054
Additionally, if you go in to edit a maintenance cleanup task, you cannot click on OK, or the T-SQL. Changing values within it does not help.
Hi. I'm very new to SQL Server and would greatly appreciate any advice with regard to my two questions below.
I've recently been given responsibility for a pair of clustered Windows 2003 servers running SQL Server 2005. Looking through SQL Server Management Studio, I found the following three maintenance plans:
- System Maintenance Plan (all system databases) - Plan 1 (1 database) - Plan 2 (4 databases).
All three maintenace plans had the following elements configured, in the following order: - Check Database Integrity - Backup Database (Full) - Shrink Database - Update statistics - Clean Up History
with the exception that Plan 1 doesn't have a 'Shrink Database' task (because, I'm told, its data is pretty static), but none had a Maintenance Cleanup Task, so I added one to each plan specifying 14 days of old backups to be kept in all cases except Plan 1, where I limited it to 2 days, the database being over 7 GB in size. All links between tasks in all plans are AND constraints, value 'Completion'. All .BAKs are written to tape by Symantec Backup Exec as part of the daily backup.
Two questions from this:
1. Since .BAKs are written to separate folders for each database, do I need, for example, four Maintenance Cleanup Tasks for the Plans covering four databases - one for each folder/database? I can't see a way of specifying that a Maintenance Cleanup Task should apply to multiple databases or search subfolders based on database name, so I assume I need four consecutive Maintenance Cleanup Tasks within a single plan? Is this the way to do it?
2.The data displayed when I right-click a maintenance plan and choose View History shows:
date and time run plan name task name duration log type log source
Since I added a Maintenance Cleanup Task to each plan, the only task shown in the Task Name column is Cleanup History (set to 'Older than 7 days'), and the duration is 00:00:00, where previously all tasks were shown and the overall duration was over a minute.
However, the .BAK files are being created (size looks OK), the application log contains entries suggesting the backups and other tasks completed successfully, and if I expand the SQL Server Logs node and view the current file I can see entries which give the same 'success' messages (presumably this log is the source of the 'success' messages in the application log?).
Why does there appear to be an anomaly between the 'View History' display and the SQL Server Log and application log? Should I be worried, or is what I can see in the SQL Server Logs and application log sufficient grounds for a good night's sleep?
We have a merge subscription which is reporting the following error at the subscriber:
The Merge Agent failed after detecting the retention-based metadata cleanup has deleted metadata at the Publisher for changes not yet set to the Subscriber. You must reinitalize the subscription (without upload)
I'm a bit confused about this - there have been no changes made at the publisher - why would we be getting this message?
I cannot go backward or forward on installing SQL Server 2005. Do not have programs installed, but setup disk claims it is already installed. Tried running Windows installer CleanUp and got the following error message when trying to install Run Time error -2147024882 System Error &H80004005
Oldest Day till I want to Purge[30 days of Old Data] : 03-17-2015
Running Cleanuphistory Procedure On Server:
NO of records are qualifying for deletion:0
Deleted Sysmail MAil Items, Old history: 1 Deleted Sysmail Log Old history: 1 0 history entries purged. <<<<<<< ----------------------- [HOW IT IS COMING ?? ??] Deleted SQL JOB Old HISTORY: 1
