Facing deadlock issues in my ETL job .
The driver table , which keeps track of what datamarts ran and for what date range gets updated frequently during the etl run . There can be as many as 250 updates issued on this table in a single second.
Now this table is a heap , and there are no indexes on it .
During these updates , we encounter deadlocks causing the ETL job to fail .
Will adding an index faciltate?
We have a highly transactional database. It was owned by a third party before but now both the database and the application is on our site and we are trying to improve this project. So, we have a big (902919 rows), heap table, which is getting bigger and bigger everyday and sometimes deadlocks occur. The table has only 4 columns, "token", "type", "value" and "cacheTime", unique index cannot be created. It has one index on "token"(char(36)) and "type"(varchar(50)) ("value" should also be included but it is nvarchar(max)).
<deadlock-list>
<deadlock victim="process670ba10c8">
<process-list>
<process id="process670ba10c8" taskpriority="0" logused="0" waitresource="RID: 6:1:14949918:1" waittime="417" ownerId="347523711" transactionname="SELECT" lasttranstarted="2014-07-01T17:43:35.233" XDES="0x117cf7da40"
[Code] ....
I am basically trying to update a table which reflects account transactions. Accounts get paid in full but occasionally balance payments can be reversed and I want to update the table to show this - I need to show which period the account was previously paid in full.I've created a simplified version of the scenario and below are a couple of examples of things I've tried that do not work. I understand why they do not work but I'm struggling to figure out how to update the 'PeriodPrevPaidInFull' field.
create table Trans
(
AccNo int,
Transaction_Period_Index int,
PeriodOpeningBalance money,
DebtBalance money,
PeriodPaidInFull int NULL,
PeriodPrevPaidInFull int NULL,
[code]...
So I've stumbled across an audit table on one of our systems that has reached a hearty 180M rows in size.
The table is a heap (no indexes whatsoever).
Each record has a datetime value indicating when it was created.
I need to delete everything that was created prior to the last 6 months; what is my best plan of attack?
IF (SELECT OBJECT_ID('t1')) IS NOT NULLDROP TABLE t1GOCREATE TABLE t1 (c1 INT, c2 INT)DECLARE @n INTSET @n = 1WHILE @n <= 454BEGININSERT INTO t1 VALUES (@n, @n)SET @n = @n + 1ENDSELECT name, indid, CASE indidWHEN 0 THEN 'Table'WHEN 1 THEN 'Clustered Index'ELSE 'Nonclustered Index'END AS Type,dpages, rowcntFROM sysindexesWHERE id = OBJECT_ID('T1')name indid Type dpages rowcnt---- ----- ---- ------ ------NULL 0 Table 2 454I have a table containing 454 rows of two columnsof type INT with each being 4 bytesc1 int = 4 bytes+c2 int = 4 bytes=8 bytes per rowIf I entered 454 rows : 454 * 8 = 3,632 byteseach SQL Page is 8KB = 8 * 1024 bytes= 8,192 bytesa data page header takes the first 96 bytesleaving 8096 bytes for data and row offsets.Each record uses a row offset at the end of the pageconsisting of 2 bytes. 454 * 2 = 908 bytes.8096 - 3632 - 908 = 3,556 bytes. Should this befree data bytes?For a heap table, does SQL add an internal uniqueidentifiercolumn also? or my question is when does SQL adda uniqueidentifier? I am reading Inside SQL 2000 andtrying to understand a few things.A uniqueidentifier of 4 bytes gets added when a clustered indexexists but it is NOT a UNIQUE clustered index. AND onlyif duplicate record is added those two records only geta uniqueidentifier value.But in my example it's a heap table with no indexes. Evenon a heap table with no indexes a ROWID or Uniqueidentifierget added? Based on the INSERT statement above allvalues are unique.So what am I missing to understand why 453 rowsmake one data page to be used whereas 454 rowsmake two data pages to be used?Thank you
View 1 Replies View RelatedI have bunch of heap tables and the fragmentation seems to be high, i am not sure whether i shall add index for them, as these tables are inserted and updated every day.
View 4 Replies View RelatedWe have some Deadlock alerts set up in SQL Agent that email us when the performance counter for deadlocks goes above zero. I've used the following script to identify the event file which has deadlock information in there.
select CAST(target_data as xml) as TargetData
from sys.dm_xe_session_targets st
join sys.dm_xe_sessions s on s.address = st.event_session_address
where name = 'system_health'
Now that is fine, and we're looking into that (number of deadlocks appear to be 0.5) but out of interest ran a SQL Profiler session to capture the details as well and nothing is showing, I've received a few alerts and the trace file has information in there - but profiler shows absolutely nothing (all deadlock events are captured)
We are using partitions and all the table are properly aligned as per the partition keys. When this particular sp, which is inserting data to a table from a different table based on the partitionkeys is called by Web UI where threading has been applied, dead lock appears.
Let me make it more clear.
ThreadOne:
Insert into table A(partitionKey,BatchId,...)
select * from table B where partitionkey = 1
ThreadTwo:
Insert into table A(partitionKey,BatchId,...)
select * from table B where partitionkey = 2
I can see sometimes it gives deadlock for this procedure, not sure about the reason, as far as I guess since the tables are partitioned and escalation is set to Auto the deadlock should not occur.
We recently upgraded to sql server 2012. We have xxx-D-011 as OLTP server and yyy-D-011 as distributor server.
The log is showing deadlocks every day between application queries/updates and replication jobs.
A fragment of the log about the deadlock is included below.
2014-07-10 15:31:05.94 spid13s deadlock-list
2014-07-10 15:31:05.94 spid13s deadlock victim=process37ced3498
2014-07-10 15:31:05.94 spid13s process-list
2014-07-10 15:31:05.94 spid13s process id=process37ced3498 taskpriority=0 logused=0 waitresource=OBJECT: 8:532249001:0 waittime=357 ownerId=860304057 transactionname=SELECT lasttranstarted=2014-07-10T15:31:05.090
[code]....
In order to troubleshoot some deadlocking that I am seeing on SQL Server, I am trying to capture the Deadlock XML by enabling the Events Extraction Settings option 'Save Deadlock XML events separately' and specifying a Deadlock XML results file.
Meanwhile, I am also tracing the Deadlock graph, Lock:Deadlock, and Lock:Deadlock Chain events. Yet the xdl file remains empty even though I am getting hits on the events themselves in the SQL Profiler trace.
Also, I have the following trace flag settings enabled.
TraceFlagStatusGlobalSession
1204110
1222110
Why the xdl file remains empty even though (I think) it should contain some XML for deadlocks that are actually happening?
Hi...
I have data that i am getting through a dbf file. and i am dumping that data to a sql server... and then taking the data from the sql server after scrubing it i put it into the production database.. right my stored procedure handles a single plan only... but now there may be two or more plans together in the same sql server database which i need to scrub and then update that particular plan already exists or inserts if they dont...
this is my sproc...
ALTER PROCEDURE [dbo].[usp_Import_Plan]
@ClientId int,
@UserId int = NULL,
@HistoryId int,
@ShowStatus bit = 0-- Indicates whether status messages should be returned during the import.
AS
SET NOCOUNT ON
DECLARE
@Count int,
@Sproc varchar(50),
@Status varchar(200),
@TotalCount int
SET @Sproc = OBJECT_NAME(@@ProcId)
SET @Status = 'Updating plan information in Plan table.'
UPDATE
Statements..Plan
SET
PlanName = PlanName1,
Description = PlanName2
FROM
Statements..Plan cp
JOIN (
SELECT DISTINCT
PlanId,
PlanName1,
PlanName2
FROM
Census
) c
ON cp.CPlanId = c.PlanId
WHERE
cp.ClientId = @ClientId
AND
(
IsNull(cp.PlanName,'') <> IsNull(c.PlanName1,'')
OR
IsNull(cp.Description,'') <> IsNull(c.PlanName2,'')
)
SET @Count = @@ROWCOUNT
IF @Count > 0
BEGIN
SET @Status = 'Updated ' + Cast(@Count AS varchar(10)) + ' record(s) in ClientPlan.'
END
ELSE
BEGIN
SET @Status = 'No records were updated in Plan.'
END
SET @Status = 'Adding plan information to Plan table.'
INSERT INTO Statements..Plan (
ClientId,
ClientPlanId,
UserId,
PlanName,
Description
)
SELECT DISTINCT
@ClientId,
CPlanId,
@UserId,
PlanName1,
PlanName2
FROM
Census
WHERE
PlanId NOT IN (
SELECT DISTINCT
CPlanId
FROM
Statements..Plan
WHERE
ClientId = @ClientId
AND
ClientPlanId IS NOT NULL
)
SET @Count = @@ROWCOUNT
IF @Count > 0
BEGIN
SET @Status = 'Added ' + Cast(@Count AS varchar(10)) + ' record(s) to Plan.'
END
ELSE
BEGIN
SET @Status = 'No information was added Plan.'
END
SET NOCOUNT OFF
So how do i do multiple inserts and updates using this stored procedure...
Regards
Karen
Hi,
We are running SAP 3.1H on MS SQL server 6.5.
We are having serius problems with printing.
Here are some error messages from the system log.
Deadlock occurred
DB error 1205 at UPD access to table TSP02
MSGNO 1205, MSGSTATE 2,
SEVERITY 13 Your server
command (process id 92) was
deadlocked with another
process and has been chosen
as deadlock victim. Re-run
your commandsrvname SAPPROD :
procname
YR200000015BKH4037TSP02 :
line 2
Spool: RT_UPDATE error 1 for table TSP02
DB error 0 at UPD access to table TSP02
Spool: RT_UPDATE error 1 for table TSP02
DB error 0 at DEL access to table TST01
Database error: TemSe->XRTAB(5)->1 for table TST01
key [005]SPOOL25171 ,1
Database error: TemSe->XRTAB(5)->1 for table TST01
key [005]SPOOL25171 ,1
Some error in TemSe management
DB error 0 at DEL access to table TSP01
The system log is full of these errors, we have a lot of inconsistency's in the temse objects.
Can you tell me wat the solution is or cult be for this problem??
Kind regards,
Kris.
I am looking to update a record from a previous row. So if there is a value of total goods in week 1, i want that value to carry forward to the value of goods in week 2. Is there any SQL as an example of the best way to accomplish this? I can query it using lag() which works great but i need the source data itself to update as the end-users are accessing the data via lightswitch, so when they save a change, i want the trigger (or whatever you recommend) to update the source table.
View 9 Replies View RelatedI have a SQL Server at version Microsoft SQL Server 2012 - 11.0.5343.0 (X64)
May 4 2015 19:11:32
Copyright (c) Microsoft Corporation
Enterprise Edition (64-bit) on Windows NT 6.3 <X64> (Build 9600: ) (Hypervisor).
I need to apply a cumulative update, either CU2 or CU3 for a third party tool used for monitoring to work. When I try to apply CU2 or any subsequent CU there is nothing that need the update so I cannot advance the version to 11.00.5548.
I need to discover the actual order in which locks are acquired on a table during a query.
This with a goal of analyzing the lock order of queries against the same table to prevent deadlocks.
I'm using SQL Server 2008 R2.
From Management Studio I execute:
begin transaction
<my query>
exec sp_lock
rollback transaction
In the output I see interesting information about which locks are acquired, but:
- are this locks ordered by the time they're acquired? That is, can I be sure that lock at row n is acquired before lock at row n+1?
- if not, how can I get this information?
I have a stored procedure that does the following
BEGIN TRANSACTION OUTERTXN
BEGIN TRANSACTION
Copy records from live to archive
END TRANSACTION with commit or rollback
execute sproc to write audit log with success or fail
IF transaction was committed
BEGIN TRANSACTION
Delete records from live the archive
END TRANSACTION with commit or rollback
execute sproc to write audit log with success or fail
End IF
END TRANSACTION OUTERTXN with commit if both inner transactions were successful or rollback if either failed
If either inner transaction rolled back execute sproc to write audit log saying whole process is rolling back End IfMy problem is that if the outer transaction rolls back then I am losing the two audit records because they are part of the transaction scope. I want these executes to commit even if the master transaction fails.
SQL Server 2005
We have a table that simply stores all changes to a specific record in another table and can get very large. The relationship is such there are many records in the transaction table for each record in the parent table. How many depends on how many times the record has been updated and can contain multiple entries for each column.
The transaction table contains a clustered index over a column that is defined as a UNIQUEIDENTIFIER. There are other indexes as well over business fields (basically foreign key columns). This obviously has some performance implications and becomes fragmented very quickly during heavy loads. Then as expected, the performance issues cascade to queries, etc.
Anyway, we are looking at two options - (a) removing the clustered index altogether and treat it as a heap or (b) add another column defined as identity (1,1) and make that the clustered index.
My initial research tells me the heap is not the way to go as there still may be performance issues with it. Using the second option guarantees me that all new data is always added to the end and will minimize the fragmentation. Keep in mind we do have regular maintenance jobs to rebuild / reorganize indexes and lob data.
Can anyone shed their thoughts on these two options for this situation?
Well I am the MS-SQL user I always face the problem in my application.After some time my application stop working goes in to the notresponding stage. Then I have to KILL some session which are SPID > 51.After this my problem is getting resolved after this I can able tocreate new sessions. Please help me ot with this issues.regards,Shailesh
View 2 Replies View RelatedI have read in a couple of books about overheads when a page split occurs on a heap (a table with no clustered index)
My question is - how (or why) would a page split occur on a heap (as the table is in no particular order)?
Hi guru,I've been new company for only a month and started analysing IndexFragmentation.After I ran DBCC DBREINDEX and capture data into permanent table, I 'veseen lots of tables with no indexes. These tables showed:Very low scan density,High extent fragmentationHigh Avg. Bytes Free per PageWhat are the best strategies to defragment tables with no indexes?I'm planning to make a rule that each table must have a clustered indexand this index must be created on the best column (highestselectivity).Please help.Thanks,Silaphet,
View 1 Replies View RelatedI have a big table (heap)... well, not so big, I have a small serverand I want to spread access to it across several new disks dedicatedonly to that table.I known its possible to do that creating a clustered index with "ONfilegroup" option but I want to maintain it as a heap, is there anyway to do this without dropping indexes/references - bulk unload -create table - bulk load - create indexes?.
View 4 Replies View RelatedI am facing Timeout Error as i am using SQL Server 2005 as a database and in database set pooling property as 0 and RemoteTimeout as 0 also as a front end i am using .net 2005 and in web.config file i set pooling=False but then also i am getting the following error.Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
What can i do then...?
Hi,
I have started experiencing the following errors
"Unable to allocate memory for a hash node heap"
"The system reports 99 percent memory load. There are 8584744960 bytes of physical memory
with 5799936 bytes free. There are 8796092891136 bytes of virtual memory with 8794956038144 bytes free.
The paging file has 7447801856 bytes with 5201920 bytes free."
The packages running have been running for the last year with no issues. Admintidley they were the only jobs running against the Instance.
I have now introduced additional databases and packages.
The package do not over lap when they run so there is currently no contention for resource.
I have no idea where to start looking to identify the culprit.
I have observed the available pagefile being completely consumed and physical memory. The only way I can get it released si by bouncing the instance.
the only thing that has changed is pagefile. The system has 8GB RAM and was configured with a 4GB pagefile. I have recently created a second pagefile on
an alternative disk of 12GB and reduced the 4GB pagefile to 100MB and left it on the root drive. When I view the pagefile size in properties, it says that there
is only 100MB even though it creates a 12GB pagefile.
Can anyone shed some light on my problem?
I appreciate your help
Regards
Marcus
table = PEOPLE
Name Money Type
----- ----- ----
Steve 400 R
Steve 100 R
Paul 500 R
Paul 100 R
Matt 500 R
Matt 200 R
Matt 0 T
Steve 0 T
Paul 0 T
I'm trying to add-up all of the Money values for each Name and store them into their names, but under Type 'T'.
after the update command it should look like this
Name Money Type
----- ----- ----
Steve 400 R
Steve 100 R
Paul 500 R
Paul 100 R
Matt 500 R
Matt 200 R
Matt 700 T
Steve 500 T
Paul 600 T
I have a backup application that uses SQL as the backend (both 2000 and 2005) The backup application performs a specific function that enters data into the Database but does not have any reporting. I am looking for a way to query the DB directly to see if there is any info I can grab. But the problem is I don't know where its stored. So my questions) are:
Is there anyway to tell what tables get updated by a certain process. For example if I run this one action how could I then tell what tables were effected or even what data was changed. I tried looking for a logging function that would list this but did not find it. I also tried looking for some type of real time monitor. I even tried looking for a way to search for records / tables that had been recently updated.
I am new to SQL so not sure I am using the correct terms but any help would be appreciated. Also this is SQL2000 and a test server
Thanks in advance for your time
I'm working on a project that is being built piece by piece. The first partis in place. I will occasionally need to either change thing within a table(only adding fields) or add stored procedures etc.What is the best way of making these changes to the production databasewithout interfering with the existing data? The users are remote and I won'tnecessarily have direct access to the server. I'll have to walk the primaryuser through the process.TIA--Jake
View 1 Replies View RelatedHi All
I am facing problem while processing my SSIS package
The error is:
"Invalid delimited data. Text qualifier must be followed by a column delimiter (except for the last column)."
The text qualifier is the double-quotes character “. Text fields are supposed to get a double quote at the beginning of the string and one at the end. The column delimiter is the upright bar or pipe character |.
Some of the descriptions in table column have double-quotes embedded within the text string. When SSIS encounters one of these embedded quotes, it thinks that is the end of the text string and expects to find the column delimiter character next. It doesn’t, because the " is embedded between other alphabetic characters, so it raises an error.
For example if my column value looks like Test"String"One
What old DTS doing:
DTS make the embedded " into "". The resulting text string then would look like this:
"Test ""strings"" one".
When running the same data into a database using DTS, DTS recognized automatically the "" in the string and changed it to " so in the target the string looked like this: test "string" one. No problem.
SSIS, however, does not allow and does not support doing this "doubling" of embedded " when you are using " as the text qualifier. So it exports the sample string above like this (I added the column delimiters as well):
|"test "string" one"|.
That then causes the error
Please help me out
T.I.A
This is the Trigger which is not working properly during Update, no anyrecord is going to be updated so pls help.I am updating the requisition table when any update in quantity inpodetails tableCREATE TRIGGER trig_updateRequistion ON ERP.DBO.TranPurchaseOrderDetailINSTEAD OF UPDATEASIF UPDATE(Quantity)BEGINUpdate RequisitionSlipDetailset RequisitionSlipDetail.PoQuantity =(Select PoQuantity from RequisitionSlipDetail whereItemCode=(Select CAST(i.ItemCode as nvarchar(20)) from inserted as i)andRSlip_No=(Select CAST(i.RSlip_No as int) from inserted as i))-((Select Quantity from TranPurchaseOrderDetail where Purchase_OrderNo=(Select CAST(i.Purchase_OrderNo as nvarchar(20)) from inserted as i))- (Select CAST(i.Quantity as int) from inserted as i))where RequisitionSlipDetail.ItemCode = (Select CAST(i.ItemCode asnvarchar(20)) from inserted as i) and RequisitionSlipDetail.RSlip_No =(Select CAST(i.RSlip_No as int) from inserted as i)Update TranPurchaseOrderDetail setTranPurchaseOrderDetail.Quantity =(Select CAST(i.Quantity as int) from inserted as i)where TranPurchaseOrderDetail.Purchase_OrderNo = (SelectCAST(i.Purchase_OrderNo as nvarchar(20)) from inserted as i)andTranPurchaseOrderDetail.ItemCode = (Select CAST(i.ItemCode asnvarchar(20)) from inserted as i)andTranPurchaseOrderDetail.PurchaseDetailId =(Select PurchaseDetailId from TranPurchaseOrderDetail wherePurchase_OrderNo = (Select CAST(i.Purchase_OrderNo as nvarchar(20))from inserted as i))END
View 1 Replies View Relatedhi everybody..
I have placed a textbox in a page footer of SSRS which is linked with a
databound text box in the body of the report.. which i intended would
give the report databound footer...
but now the problem is tat they dont show up on every page..
earlier they use to mostly not show up at all
or show up on the first or the last page...
could some1 help.??
once i render it to pdf..
all the page footers cant be seen.
regards.
savio
I need to update information for a user and if the user is classified
as a primary (@blnPrimary) then I need to update information for all
users within his agency (AgencyUniqueId). The issue is that the second
UPDATE to "cdds_User_Profile" always returns a rowcount of 0 (should be
1) even though the values for "@Original_AgencyUniqueId" and
"@Original_UserId" are correct. This is just a snippet of the whole
procedure. I'm trying to implement similar logic in other parts of the
procedure and I'm observing the same behavior there as well. Any help
anyone can provide is greatly appreciated. </p><pre>/*** Update User Profile ***/UPDATE [cdds_User_Profile] SET [FirstName] = @FirstName, [LastName] = @LastName, [Title] = @Title, [Phone] = @Phone, [AcctType] = @AcctType, [AcctStatus] = @AcctStatus, [LastUpdatedDate] = GETDATE() WHERE ([FirstName] = @Original_FirstName AND [LastName] = @Original_LastName AND [Title]=@Original_Title AND [Phone]=@Original_Phone AND [AcctType]=@Original_AcctType AND [AcctStatus]= @Original_AcctStatus AND [AgencyUniqueId] = @Original_AgencyUniqueIdAND [UserId] = @Original_UserId);IF @@ROWCOUNT = 0BEGINSET @err_message = 'Data has been edited by another user since you began viewing this information.'RAISERROR (@err_message,11, 1)ROLLBACK TRANSACTIONRETURNEND IF @@ERROR <> 0 BEGINROLLBACK TRANSACTIONRETURN ENDIF @blnPrimary = 1 BEGIN IF LOWER(@AcctStatus) <> LOWER(@AgencyAcctStatus)/*** Update Users Acct. Status ***//* update all users in same agency profile */UPDATE [cdds_User_Profile] SET [AcctStatus] = @AcctStatus,[LastUpdatedDate] = GETDATE() WHERE ([AgencyUniqueId] = @Original_AgencyUniqueIdAND [UserId] = @Original_UserId); IF @@ROWCOUNT = 0BEGINSET @err_message = 'Data for this agency has been edited by another user since you began viewing this information.'RAISERROR (@err_message,11, 1)ROLLBACK TRANSACTIONRETURNENDIF @@ERROR <> 0 BEGINROLLBACK TRANSACTIONRETURN ENDEND</pre><pre>
Hi,
I am trying to do selective updates for rows where a column matches a column in another table. I want to do something like this, only 'this' does not work, and nothing else I could think of (I tried joins also) worked. What am I missing? I hope this explanation makes sense.
UPDATE queryresultsmodel SET queryresultsmodel.tableforcedoutdate = getdate()
Where Exists (Select tablename from queryresultsmodel q inner join orphanul o on q.tablename = o.name)
Thanks for any help,
Judith
Hi,
Has anyone had any problems on one row updates on a table where you have defined horizontal and vertical partitioning of the data to be replicated?
When I execute an update clause that modifies just one row the log reader misses the modification and it does not get replicated to the other databases.
If I do the same update clause but on several rows then all the modifications are read by the log reader and the replication task goes ok.
What might be wrong?
-janne
Hello!
I have 2 database (DB1 and DB2) in 1 server. I want to reflect new changes made in DB1.tbl_pm_project to DB2.tbl_pm_project. Any idea on how to do it.
Thank you
JJ-hon