Ongoing Purging Of Active Records Causes Deadlocks

Jul 19, 2007

Hi,

We have a Java application that runs against a variety of backends
including Oracle and MSSql 2000 and 2005. Our application has a
handful of active tables that are constantly being inserted into, and
some of which are being selected from on a regular basis.

We have a purge job to remove unneeded records that every hour "DELETE
FROM <tableWHERE <datafield< <sometimestamp>". This is how we are
purging because we need 100% up time, so we do so every hour. For
some tables the timestamp is 2 weeks ago, others its 2 hours ago. The
date field is indexed in some cases, in others it is not... the
DELETE is always done off of a transaction (autoCommit on), but
experimentation has shown doing it on one doesn't help much.

This task normally functions fine, but every once in a while the
inserts or counts on this table fail with deadlocks during the purge
job. I'm looking for thoughts as to what we could do differently or
other experience doing this type of thing, some possibilities include:
- doing a select first, then deleting one by one. This is a
possibility, but its SLOW and may take over an hour to do this so we'd
be constantly churning deleting single records from the db.
- freezing access to these tables during the purge job... our app
cannot really afford to do this, but perhaps this is the only option.
- doing an update of an "OBSOLETE" flag on the record, then deleting
by that flag... i'm not sure we'd avoid issues doing this, but its'
an option.

The failures happen VERY infrequently on sql2005 and much more
frequently on sql2000. Any help or guidance would be most
appreciated, thanks!

Bob

View 3 Replies


ADVERTISEMENT

Removing Non-useful Records And Duplicate DB Objects. ( Purging)

Oct 9, 2007



hi All,
I have to remove non-useful and duplicate records containing NULL , Blanks and extra spaces (either on left or right side of the column values) etc from all the tables in my server's DB XYZ weekly thru a a scheduled job with the help of a Stored Proc, that s i guess called Purging og DB. Plz help how i can do it with T-SQL.

Also i have to find out and remove all the duplicate DB objects(tables) from the DB .e.g. a table existing with name TABLE_TEST or TABLE_DEBUG etc for an original table TABLE , making sure no any of the base table is dropped.

Plz help me reagrding these two problems.

Thanks in advance for the quick replies.
Mohit

View 1 Replies View Related

Capturing A Warning - Ongoing Basis

May 17, 2004

Hi all,

I currently have a series of stored procedures that capture stock prices on a daily basis, then summarize the results into a daily, and further, a weekly summary of the "index" of a group of stocks. The data is accumulated from a (to use a highly technical unit of measurement...) bunch-O-individual rows of data using aggregate functions such as AVG and SUM.

The problem is that I occasionally get a warning on such aggregate statements which is the common one complaining thusly: "Warning: Null value is eliminated by an aggregate or other SET operation"

I know where it comes from, and I know how to code to protect the aggregate from complaining (i.e., AVG(ISNULL(yadayada,0)) ) but I am interested in figuring out a way to REPORT the statement that contains null values. I can, of course, capture ERRORS in selects, but is the same mechanism used to capture these NULL warnings on my aggregate statements? I don't necessarily want to know which individual row is causing it, just want to "tag" somehow the statement that results in the warning so I can go back after the run and check into it (after capturing local "pointer" info at the time the offending aggregate is invoked).

The code I use to capture errors and trace information follows:
UPDATE PortfolioPerformance
SET PrevDate = @PrevDate,
DailyPerChg = GPP.DailyPerChg,
DailySumPriceChg = GPP.DailySumPriceChg,
SumCurrPrice = GPP.SumCurrPrice,
SumPrevPrice = GPP.SumPrevPrice,
StockCount = GPP.StockCnt,
AvgHighPriceRatio = GPP.AvgHighPriceRatio,
AvgLowPriceRatio = GPP.AvgLowPriceRatio,
Volume = GPP.Volume
FROMPortfolioPerformance PP (nolock), VIEW_Get_PortfolioPerformance GPP
WHERE PP.PortfolioID = GPP.PortfolioID AND
(PP.CreateDate = GPP.CreateDate AND
PP.CreateDate = @CreateDate) AND
PP.PrevDate IS NULL

SELECT @RowCount = LTRIM(STR(@@ROWCOUNT)) /* capture rowcount so @m_error select doesn't clobber it*/

SELECT @m_error = @@Error IF @m_error <> 0 GOTO ErrorHandler
SET @TraceMsg = 'Completed Daily Portfolio Performance calculations (updated ' + @RowCount + ' rows)'
EXECUTE [dbo].[tracelog] 1, 'Index', 'sp_Set_PortfolioPerformance', @TraceMsg

NOTE: the aggregation in the above code is performed in the view referenced as "GPP", but that's outside the realm of the question, I think, so I won't bore you with the details of that just yet.

So I think if I can capture the warning like I do the errors, I can accomplish what I want to accomplish. I haven't yet been able to find any guidance in the Books Online, so do any of you have any pointers?

Thanks!
Paul

View 2 Replies View Related

Query To Display Most Active Records

Feb 11, 2005

Hello,

I have two tables, Promotion and Promolocation. The Promotion table is used to set up promotions or sales, and consists of a PromoID, StartDate, and EndDate. Each PromoID is referenced in the Promolocation table, which is used to assign items to a promotion for various locations or stores. The Promolocation table consists of PromoID, LocID, SkuID, PromoPrice, and DiscLevel.

There are times where an item or SkuID will exist in more than one promotion, however, our application is currently not intelligent enough to determine which promotion to use, so it sets the active promotion based on the StartDate being before other promotions' StartDate and the EndDate being after other promotions' EndDate.

I want to find all promoid's where a sku exists in more than one promotion. I want to signify which promotion is active, using 1 as the first active promotion, 2 as the next active, 3 as the next, etc. To determine which promotion is the first active promotion, the StartDate must be before any of the other promotions' StartDate, and the EndDate must be after other promotions' EndDate. If the promotions' StartDate is after the other promotions' StartDate but not before the other promotions' EndDate, and the EndDate is before or on other promotions' EndDate, then that's the second active promotion. If the StartDate is the same as other promotions' StartDate, but the EndDate is before other promotions' EndDate, then that's the third active promotion.

For example:

PromoID StartDate EndDate
------- --------- -------
PROMO1 1/1/2004 1/1/2006 (1st Active Promotion)
PROMO2 2/1/2004 1/1/2006 (2nd Active Promotion)
PROMO3 1/1/2004 12/1/2005 (3rd Active Promotion)

Here's a query I am using to display all active promotions:

select
pl.promoid,
pr.startdate,
pr.enddate,
pl.locid,
pl.skuid,
pl.promoprice,
pl.disclevel
from
promolocation pl
inner join
promotion pr
on
pl.promoid = pr.promoid
where
pr.enddate >= getdate()

Thanks for your help.

D

View 4 Replies View Related

SQL Server 2008 :: Add Subscriber To Ongoing Replication?

Feb 27, 2015

i have replication setup and working, there are 3 subscribers and now i want to add a 4th one. is it ok if i just add the subscriber and reinitialize replication?

View 1 Replies View Related

How To Retrieve 100,000 Records From Active Directory Using SQL Server

Nov 24, 2004

Hi,

I am new to Microsoft Technologies, can any one say how to connect/ retrieve data's from Active Directory through SQL SERVER from scratch. I need with some sample codes.....

Thanks in advance

raj d.s

View 1 Replies View Related

SQL Server 2012 :: Query To Count How Many Sessions Are Active And Remain Active Per Hour

Jan 22, 2015

I have a table with the following columns employeeSessionID, OpDate, OpHour, sessionStartTime, sessionCloseTime. I need to see how many users remain active per hour. I can calculate how many logged in per hour, but I am stumped on how to count how many are active per hour. I have a single table that stores login data. I have created a query that pulls out the only the data needed from the table into a temp table using this query. Also note it is possible that the sessionCloseTime is null if the device has not been logged out this would need to be counted a active.

TABLE NAME #empSessionLog Contains the time stamp data OpDate, sessionStartTime and sessionCloseTime.
OpDatesessionStartTimesessionCloseTime
2015-01-202015-01-20 14:32:59.1302015-01-20 14:33:14.6299166
2015-01-202015-01-20 06:58:33.7302015-01-20 15:27:16.9133442
2015-01-202015-01-20 09:56:22.8402015-01-20 17:56:29.7555853
2015-01-202015-01-20 05:59:18.6132015-01-20 14:05:19.0426707

[code]....

can see how many sessions logged in per hour with the following statement:

SELECT
opDate,
FORMAT(DATEPART(HOUR, sessionStartTime), '00') AS opHour,
Count(*) AS Total
FROM #empSessionLog
Group BY opDate, FORMAT(DATEPART(HOUR, sessionStartTime), '00')
Order BY opDate, FORMAT(DATEPART(HOUR, sessionStartTime), '00') ASCResults:
opDateopHourTotal
2015-01-20041

[code]....

Where I am stuck is how do I count the sessions that remain active per hour until the session is closed with the sessionCloseTime.

View 5 Replies View Related

There Is Not Enough Disk Space Error While Installing SQL In A Active/Active/Passive Cluster

Mar 5, 2008



Hi

I am having some teething problems while installing SQL on a 3 node cluster. Within the Cluster configuration I have 3 Cluster Groups with each of them having their associated disk resources. All these disk resources physically exist on a SAN.

The actual cluster is running absolutely fine and I can access all the disks from their respective owner node. The problem only starts when I start installing SQL Server 2005 on this cluster. I specify the Cluster group from the Cluster Group Selection and choose the desired partition and then the error message pops up

"There is not enough diskspace on the destination disk for the current SQL Server data files. To proceed, free up disk space to make room for data files, or install the data files to a different drive"

But the disk I am trying to install it on is 264Gb and none of it is used. I have also tried to change it to a different disk within the same Cluster group but to no avail. I have even tried to install it in a different cluster group all together but I get the same error message.

I have googled around havent found anything so far. The disks have got full permissions for the account I am installing SQL with.

Any help will be much appreciated.


Regards

Adnan

View 5 Replies View Related

Archival And Purging

Jul 17, 2004

How to implement a optimal archival and purging in MSSQL SERVER
databases

View 3 Replies View Related

Purging Old Tranlogs

Sep 23, 2005

hi !
I have a job that creates tranlog backups every 15 mins and makes a full database backup at midnight.
I want to purge the old logs after the full backup , how can I do that ?

thanks
Sami

View 2 Replies View Related

SQL Help With Purging Old Data

Feb 28, 2007

Does anyone know how to purge old data in a MS SQL server 2000database?

View 1 Replies View Related

Purging TempDB

Jun 20, 2007

Hi DB Experts



I have a question on shrinking tempDB. Currently, I am using MS SQL 2005 Server, I have a software that used SQL DB but I had created a separate database to store my data. The tempDB is totally not used at all.

Problem - Whenever I export data into my created database on MS SQL server, tempDB also grew, I noticed that the files grew so large that it crashed the server. I am running 50GB free space on my drive where by the MS SQL server was installed.

Question - May I know are there any solution to shrink or freeze the growth of tempDB size?





Best regards

TEWCT

View 7 Replies View Related

Active/Active SQL Server Clustering With Multiple Instances

May 12, 2008

Hi

I am newbie in SQL Clustering. I have set up a Windows Server Cluster with 2 nodes and am having the following problem with Physical Disk resource for cluster groups:

My Default Cluster Group (named Cluster Group) has IP Address, Network Name, Physical Disk and MSDTC resources. In addition to that my Default SQL Server instance resources are also in this group. I had this initial set up for Active/Passive mode.

Now I am trying to set up a SQL Cluster in Active/Active mode. For this I have to install another instance of SQL Server in the existing cluster and make a separate cluster group for its resources. I made a new cluster group (SQL Instance Group) with an IP Address and a Network Name resource for this new instance but I dont have any Physical Disk resource to allocate to it. As such while installing the SQL Server Instance I get stuck when I'm asked to select the quorum disk to be used.

Is it possible to configure two quorum disks, one for each group?
What's the concept of dedicated disks resource for each sql instance in a group? Is this same as the quorum disk? If this is not a shared disk how do I configure a dedicated disk resource for my second cluster group (SQL Instance Group)?

Anyone could please help me out with this?

View 12 Replies View Related

Are There Any Issues With An Active/Active Cluster With A Remote Mirror For DRP

Nov 30, 2006

Hi folks,

We are going thru the process of scoping an active/active cluster at one site.
I was wondering whether there will be any issues with mirroring (DB by DB) off the cluster into non clustered server at an alternate DRP site.

Regards, Brian
Sutherland

View 1 Replies View Related

Install Active Active Sql 2005 2 Node Cluster

Mar 12, 2008




Hi all, My aim to install active/active 2 node SQL 2005 cluster. I have installed sql cluster on one mode, which automatically installed on node 2.

it works great. However When i tried to install second virtual server, it is not allowing me to install.

it says already install, can anyone tell me how to install active active cluster

View 1 Replies View Related

Purging The Old Transaction Log Files

Aug 7, 2007

Hi,

In one of our DR Servers we have configured Custom Logshipping. In the folder where the .TRN files are getting copied there is a script to purge the files which are older than one day. Following is the code for the same.

@Echo Off
if exist filelist del filelist
date /t > rundate
for /f "tokens=2* delims= " %%i in (rundate) do set rundate=%%i
for %%i IN (*.trn) do echo >> filelist %%i %%~ti
for /f "tokens=1,2* delims= " %%i in (filelist) do if not %rundate% equ %%j del %%i
:pause
:Exit

Instead of removing the files older than 1 day, I need to keep 3 days transaction logs.

Being a novice I don't have much idea how to accomlish it. Can anybody help me with this?

Many thanks in advance,

Sandhya

View 1 Replies View Related

Purging The Transmission Queue

Mar 29, 2006

My transmission queue has lots of messages that will never, ever be delivered because the transmission_status = "The session keys for this conversation could not be created or accessed. The database master key is required for this operation."

How can I purge the transmission queue to get rid of this junk?

View 1 Replies View Related

Purging Deleted Database Names

Jul 1, 2002

When connecting to an SQL Server (v7 in this case) the log file shows that old and deleted databases are still being opened as part of the process. Further, these db names are now reserved and cannot be reused, even thought I've deleted them.

Is there any way I can purge all traces of these deleted db names?

You might have guessed, I'm a newbie, which is why there are a few dozen deleted DBs.

Thanks for any help
Max
atomax@gmx.net

View 2 Replies View Related

DB Maintenance - Purging Old File Feature

May 31, 2006

Hi,

I'm using SQL2K with SP4 2187. I have created a DB Maintenance wizard where the purging older than 1 day is set.

However, this feature seems not to be working, even if I tried two ways. Delete the scheduled job and recreate it - not successful, 2nd) delete the Maintenance Plan, still not successful.

Is this a bug or do I miss something here.

View 4 Replies View Related

Purging Data Off Of An MSDE Database.

Jan 17, 2007

How do I purge data off of an MSDE database. I only want to keep 6 months of data in the database. Right now I have data going back to 2004. I get errors about every 10 seconds. "Primary File Group is Full" is the error I am getting.

View 7 Replies View Related

Data-archiving And Purging Strategy

Mar 31, 2008



Regarding SQL Server data, I am looking to implement the beset Data-Archive and Purge policy. Normal, we do SQL Backups and keep the history for some period , for example, 8 weeks, so we can go back and restore any data point in time upto 8 month in past. and we also do Tape backups.

Question is Where can I get nice article or documentation on this to best design such policy where I make sure that I am covered for point in time recovery of database (which is sql backups) and point in time recovery in far past, say, 3 years ago using tape backups, and I need to make sure that I don't repeat the same efforsts.

Any advices or suggestion on this topic.

Thanks,

View 5 Replies View Related

Active/active Sql Server Config; Shared Dbs

Nov 2, 1999

Hi,

We are trying to set up an active/active configuration of a SQL Server
cluster, and we had a few questions.

Initially, we want to have 2 Database Servers that would share the same
Database (both reading/writing to the same tables).
However, from reading the MS docs, we find out that we can have what
they call an "active/active" configuration using a Cluster but they
need to have 2 different disk sets, i.e. having 2 separate databases.
If this assumption is correct, how does the data get synchronised
between the 2 databases (that are on the 2 different disks sets)?

Can anyone help? thanks
Axel

View 1 Replies View Related

ACTIVE/ACTIVE SQL Server 2000 Cluster

Oct 12, 2002

ACTIVE/ACTIVE SQL Server 2000 Cluster

This was my plan.

Physical box - box1,box2
virtual box - virtual1

i have a active/passive cluster virtual1 (primary node box1, secondary box2)

I am creating another virtual box on box2 as primary node by name virtual2)

whenever my installation setup runs my default server name is greyed out. it always requires a instance name.

Is that the way it should work?

i can only use instance for active/active failover?

please advice

View 1 Replies View Related

MSDTC In Active/Active MSCS Cluster

Jun 29, 2004

How do I install MSDTC in an active/active MSCS cluster environment?
If I run comclust.exe, I can only activate MSDTC on one of the nodes at a time.

I need DTC on both nodes.

(Opsys is W2K with SQL 2000)


//Mange :)

View 2 Replies View Related

Active/Active SQL Server Cluster And OLTP

Sep 27, 2005

Hello folks,

have you ever heard of an Active/Active SQL Sever 2000 Cluster deployed in a pure OLTP environment?

Some 8 years ago I have learned about a bespoke solution for the SAP ERP system (not the BW!) with DB2 Parallel Server for a huge German company. Then again, I would expect that Oracle RAC might fit into an OLTP environment, although I never heard about a real world implementation.

All this led me to believe clustering is good for failover purposes, and for decision support services, not quite for OLTP applications.

So if you see a point in Active/Active Clustering and OLTP please come forward and explain.

Cheers,

Johann

P.S: For those of you who want to know: Consultants from www.hiltes.com want us to deploy an Active/Active Cluster for their Fashion 3000 Net stock software.

View 6 Replies View Related

SQL 2000 Active/active Cluster And Different IP Adresses

Mar 22, 2007

We run several SQL 2000 SP4 instances on IA64 active/active cluster. The OS we run with is Windows Server 2003 SP1. We have different network cards : 2 network cards teamed for production purposes in domain X and 1 network card dedicated for problem solving in domain Y.
First we configured the cluster with only the 2 teamed network cards for production purposes in domain X. Later we introduced the second network card for problem solving in domain Y.
Everything looks fine. The (virtual) SQL instance listens on two different IP adresses on TCP port 1034. If we try to make connection via isql, EM or Query analyzer than we can directly contact the SQL instance from a workstation/server within domain X but this doesn't work form a workstation/server within domain Y. However if we use the specific TCP port in the connection in domain Y the connection is setup. We wish however not to use explicit TCP ports in setting up connections.
Has anyone experienced the same problem before or has anyone an idea how to solve the problem?

View 1 Replies View Related

Applying SP To 2005 Active/active Cluster

Jun 8, 2007

I have a 2005 active/active cluster and want to apply SQL server 2005 SP2 to both node.
I know that for active/passive , the sp can simple be installed on the active node(instance) and everything will be replicated to the other node.
bear in mind what I have SSIS and SSAS and SSRS running on the active/active cluster.

what is the rigth method for applying the SP to activeactive cluster?

Thanks

View 1 Replies View Related

SSIS Package That Would Perform Routine Purging

Jun 29, 2006

HiWe have need for an SSIS package that would perform routine purging of the growing data in some of the tables used to support notification services. While running Sql in a regular job would seem to suffice, an SSIS package would be more in line with the other processes for the alerts in terms of manageability. The SSIS package should follow the following guidelines:Deletes records from a given number of tables, based on a specified date column for each table and a specified number of days for each table, or other conditions. SystemAlertQueue: 30 days old based on the SubmitTimestamp column. SystemAlertChron: 30 days old based on the EventTimestamp column. SystemAlertNotifChron: 30 days old based on the NotifTimestamp column. WMICheck, related tables (based on WMICheckID, see WMIAlerts database diagram in SqlServer) 15 days old based on BeginCheckDate column. Each table€™s deletion routine should be distinguishable in the package. Does any body know how to do this.. Please help meRegardsDeepu M.I

View 1 Replies View Related

Purging A Database Data While Keeping The Structure Intact

Oct 15, 2006

Is it possible to purge all records in the database while retaining the the table structures. Even better yet, could I do it on a table by table basis?   If I simply delete all the records the identities for the tables do not revert back to 1.

View 2 Replies View Related

SQL Server 2012 :: Purging Data From Master And 5 Child Tables

Jul 5, 2014

I have 6 tables which are very huge in row count and records needs to deleted which are older than 8 days.

Little info: Every day, 300 Million records are inserted in below 7 tables. we should maintain only 8 days worth of data in below tables. How to implement Purge script which can delete records in all tables in the same time and with optimized parallelism.

Master table which has [ID],[Timestamp]
Table Name: Sample - 2,578,106

Child tables: Foreign key [ID] is common for all the tables. There is no timestamp column in child table. So the records needs to deleted based on Min(ID) from Sample

dbo.ConnectionDB - 1,147,578,048
dbo.ConnectionSS - 876,458,321
dbo.ConnectionRT - 118,133,857
dbo.ConnectionSample - 100,038,535
dbo.Command - 100,032,235

View 9 Replies View Related

Active / Active Cluster Win2K

Mar 5, 2001

Hi
While configuring an active / active cluster, do I need to run the SQL setup on both nodes?
The SQL2K setup installs binaries on localdisks of both nodes - hence the question.

thanks
Liju

View 3 Replies View Related

Active/Active Clustering Config

Feb 22, 2000

I have setup an active/active clustering environment for SQL Server, however it is 2 seperate virtual servers. How can I set them up to exist as one virtual server containing both active installations? Can this be done? I have two compaq 8500 w/8 processors each. I need to be able to cluster these configurations to take advantage of all 16 processors in one virtual SQL Server. Can it be done?

If you can answer these questions, you will have my undying gratitude.

Thanks in advance.

View 1 Replies View Related

SQL7 In Active/active Cluster

Sep 21, 2000

Hi
I am running some tests on SQL7 in an active/active cluster and have a couple of queries
1. When I create an ODBC DSN, why is the "Use the Failover SQL Server if the primary SQL Server" checkbox disabled
2. To test, I was running SQL queries from Access over the DSN created. When the Primary Server went down, I had to reconnect to re run the query - Is this normal?
3. Can someone point me to any documentation on the above scenario that would shed some more light?

Thanks
Liju

View 5 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved