DB Design :: Optimizing Merge Statement By The Use Of Indexes On Target
Nov 13, 2015
This website describes how Merge statements should be optimized by the use of indexes on the target?source tables: [URL]..... It says that a clustered index should be created on the join column in the target and a unique covering index on the source table.
I have read in other articles that insert/delete/update statements perform worse on tables with clustered indexes as the leaf level pages will have to be reorganized.
Why in the case of Merge statement having a indexes actually improve the performance of insert/delete/update statements?
View 3 Replies
ADVERTISEMENT
Nov 8, 2015
I am new to use MERGE statement. The MERGE cannot find any match Cardnumber in the target table. It inserts row into an existing row on the target table causing SQL rejected with duplicate key not allowed. The CardNumber is defined as a primary key on the target table with no duplicate allowed. Below snippet stop when MERGE insert a row exists on the target. The source table contains multiple rows with the same Cardnumber because it is a transactional table with multiple redemptions.
If MERGE cannot handle many (source) to one (target) relationship, what other method that I can change to in order to update the target GiftCard table which keeps track of gift card balance?
Below is the error message:
Msg 2627, Level 14, State 1, Line 5
Violation of PRIMARY KEY constraint 'PK_GiftCard'. Cannot insert duplicate key in object 'dbo.GiftCard'. The duplicate key value is (63027768).
The statement has been terminated.
DDL of the target table:
CREATE
TABLE [dbo].[GiftCard](
[CardNumber] [varchar](50) NOT NULL,
[Year] [int] NULL,
[Month] [int] NULL,
[Period] [int] NULL,
[TransAmount] [decimal](10, 2) NULL,
[Code] ....
View 8 Replies
View Related
Feb 10, 2015
I have a table:
declare tableName table
(
uniqueid int identity(1,1),
id int,
starttime datetime2(0),
endtime datetime2(0),
parameter int
)
A stored procedure has new set of values for a given id. Sometimes the startime and endtime are the same, in which case I update the value of parameter. Sometimes I add a new time range (insert statement), and sometimes I delete a time range (delete statement).
I had a question on merge, with insert, delete and update and I got that resolved. However I have a different question regarding performance of the merge statement.
If my target table has hundreds of millions of records and I want to delete/update/insert a handful of records, will SQL server scan the entire target table? I can't have:
merge ( select * from tableName where id = 10 ) as target
using ...
and I can't have:
merge tableName as target
using [my query] as source on
source.id = target.id and
source.starttime = target.startime and
source.endtime = target.endtime
where target.id = 10
...
This means I cannot filter the set of rows in the target table to a handful of records where id = 10.
View 1 Replies
View Related
Nov 5, 2015
I've have a need with SQL Server 2005 (so I've no MERGE statement), I have to merge 2 tables, the target table has 10 fields, the first 4 are the clustered index and primary key, the source table has the same fields and index.Since I can't use the MERGE statement (I'm in SQL 2005) I have to make a double step operation, and INSERT and an UPDATE, I can't figure how to design the WHERE condition for the insert statement.
View 2 Replies
View Related
Aug 12, 2007
I am developing Payroll Solution for a large government department. I have created following tables:
Table: Allowance Master (AllowanceID smallint P.K, AllowanceName varchar(30) unique)
Table: DeductionMaster (DeductionID smallint P.K, DeductionName varchar(30) unique)
Table: EmpMaster (EmpCode LongInt P.K, EmpName varchar(60))
Table: Emp_Pay_Details (Salary_Month_Year Date, Emp_Code, Tran_Detail varchar(30), Tran_Type char(1), Amount number(20,2))
In Emp_Pay_Details table, the TranDetail field stores the name of Allowance/Deduction and TranType field stores A or D, for Allowance or Deduction. The sample records will look something like this:
SALARY_MONTH_YEAR EMPCODE TRAN_DETAIL TRAN_TYPE AMOUNT
-----------------------------------------------------------------------------------------------------------------------------------
Jan-2007 101 DEARNESS A 100
Jan-2007 101 VEHICLE A 200
Jan-2007 101 RECOV. D 5000
As you can see in the above table, the pay record for the month Jan-2007 is entered. Though I have shown only three rows. In real there are compulsorily 15 rows for each employee. These 15 rows shall be entered per month for each employee. Though we have taken Allowance Master table and Deduction Master table where we will store one time values for the total number of Allowances/Deductions that need to be applied on the month salary. The Emp_Pay_Details table stores the exact record of the salary.
In the Emp_Pay_Table, I have used Tran_Detail field to store either the name of Allowance (from Allowance Master table) or the name of Deduction (from Deduction Master table) and the Tran_Type column stores A or D for Allowance or Deduction.
Since this table will store 15 records for each employee per month and there are 5000 employees, so 15*5000 = 75,000 rows shall be inserted each month. Depending upon the growing size of the table, I want to know:
(1) How to optimize the table structure for better performance?
(2) Whether I should take a composite key of (Salary_Month_Year, EmpCode, Tran_Detail) as Primary Key or any other way?
(3) Any other SQL Server 2005 feature to use while fetching records from this table.
View 2 Replies
View Related
Mar 12, 2015
I'm not even sure this is possible but I'm using MERGE in a process that has 3 source tables (the process steps through each source table sequentially) and I need to delete from the Target database occasionally.
My current code is
sqlMerge = "MERGE " + TableName + " AS target USING @CData AS source" +
" ON target.TotRsp = source.TotRsp AND target.ClientRef = source.ClientRef AND target.dbPatID = source.dbPatID" +
" WHEN MATCHED THEN" +
" UPDATE SET dbPatFirstName = source.dbPatFirstName, dbPatLastName = source.dbPatLastName,
[Code] ....
The Target db data is made up from several different clients and when the MERGE runs it uses TotRsp, ClientRef and dbPatID to uniquely match a source row to the target row and if no match it inserts the source row.
My problem is that when this runs with Source A first, it will delete all merged data from Source B & C. Then when Source B runs it will insert all Source B data but delete all from A & C and so on.
Is there way that that I can include additional clauses into NOT MATCHED BY SOURCE THEN so it knows only to delete when data has come from say Source A. Once the data is in the target table there is no reference to which source table it came from tho.
If there isn't a solution I suppose I could always add an extra column to the target db to indicate which source it came from and then have something like
NOT MATCHED BY SOURCE AND t.Source = 'SourceA'.
That's quite a bit of work my end to do that tho so I'd like to be sure it works.
View 1 Replies
View Related
Mar 16, 2014
I am trying to insert new records into the target table, if no records exist in the source table. I am passing user specific values for insert, but it does not insert any values, nor does it throw any errors. The insert needs to occur in the LOAN_GROUP_INFO table, i.e. the target table.
MERGE INTO LOAN_GROUP_INFO AS TARGET
USING (SELECT LGI_GROUPID FROM LOAN_GROUPING
WHERE LG_LOANID = 22720
AND LG_ISACTIVE = 1)
AS SOURCE
[Code] .....
View 8 Replies
View Related
Jul 20, 2015
I have data that I want at multiple granularities, 5,15,30 and 60 minutes. To reduce repetition, I have put them all in the same table, so that there is a column for 5,15,30 and 60 minutes, with a filtered index on each of the columns that removes the nulls. This means that each day will have 288 slots, but only 24 of the slots are filled in for 60 min data, and all of them are filled for 5 minute data.
I have another column that specifies the interval granularity, and my first thought was to access my data through a join, where I can use a CASE statement, and depending on the data granularity necessary, it will look at a different column:
INNER JOIN Data d ON
AND d.settlement_key =
CASE st.interval_granularity
WHEN 5 THEN [5_min_settlement_key]
WHEN 15 THEN [15_min_settlement_key]
WHEN 60 THEN [60_min_settlement_key]
ELSE NULL END
Despite the presence of the indexes on the columns, then the process seems to be quite slow, I think probably due to the fact that any query plan isn't going to know beforehand which of the columns it is going to use for any given dataset, until it actually starts to run, so it may not be optimised.
How I could optimise this based on the given structure? Maybe there are hints to be added to the join, or maybe I can clear the query plan each time the SQL is run? My other option for dealing with the data of different granularity was to use one column and repeat the data multiple times, each at the different granularity, but this makes my data, row and table sizes much higher, as we are adding just a column for each additional granularity. Would this work any better in future versions of SQL server, maybe with column store indexes?
View 5 Replies
View Related
Feb 17, 2005
If I have a table with Col1 + Col2 as PK. But most of the queries use Col1, Col2, Col3 and Col4 in the queries, should I create another unique index on this combination? If not, my queries do an index scan. Just wondering is there a general rule for this kind of situation as I have a lot of such cases in my database.
View 13 Replies
View Related
Oct 5, 2006
Group,
I am getting the following error during replication of Database to a client:
The schema script 'Statutes_6.dri' could not be propagated to the subscriber. (Source: MSSQL_REPL, Error number: MSSQL_REPL-2147201001)
Get help: http://help/MSSQL_REPL-2147201001
Invalid locale ID was specified. Please verify that the locale ID is correct and corresponding language resource has been installed. (Source: MSSQLServer, Error number: 7696)
Get help: http://help/7696
Incorrect syntax near the keyword 'with'. If this statement is a common table expression or an xmlnamespaces clause, the previous statement must be terminated with a semicolon. (Source: MSSQLServer, Error number: 319)
The database is relatively small, only about 5 tables but there is a clustered Full-text Index.
Any ideas?
Thanks,
Frank
View 1 Replies
View Related
Jul 23, 2015
Currently we are facing some performance issue while accessing the archive data from the archive tables. the archive table is hugh and it contains around 100,000,000 records and this archive table is being used in few reports and in our commission cycles too. since we are facing performance issues we are rebuilding index once in a week on all the indexes on this archive table.
We have 1 clustered index and 5 non clustered indexes, every time when we rebuild all these indexes on this table it is taking more time, more often rebuilding the clustered index itself is taking approx. 1hr which is consuming more time. wanted to know is there any useful to rebuild clustered indexes or not, if yes then what would be the better way. if not then do we need to rebuild only non clustered indexes.
View 7 Replies
View Related
Jul 15, 2015
I have a series of indexes that are showing as statistics. Is it because they are filtered or are defined as constraints?
View 5 Replies
View Related
Jan 17, 2007
Hi,
We have a client that has a large (5Gb) database replicated to 13 subscribers, the publisher is Sql 2005, the subscribers are Sql Express. The publication has as few filtered articles too. I have found that after several months of continuous running Replication Monitor is taking a long time to report history on each subscriber.
Do people tend to rebuild the indexes on the system merge replication tables on production servers, or should the standard replication jobs take care of this?
Thanks for your help
Graham
View 6 Replies
View Related
Jul 14, 2015
I have a database around 500 GB. right now the database have only one data file and one log, it has only one filegroup also.all the indexes and table are placed in Primary Filegroup . we are going to separate them. the planing is to move all the indexes to Secondary filegroup and all the table will be in Primary filegroup.But there will be a problem while implementing it because there are around 600 tables and each table have at least 2 non-clustered index , so is there any way to move all the index to Secondary Filegroup.
View 3 Replies
View Related
Jul 14, 2015
In my enviornment i have one database with 6 ndf files and 5 ldf files and one mdf file.Actuvally what i am looking is to merge 6 ndf files into one ndf file and 5 ldf files into one ldf file.is it possible to do like this? , i tried using MOVE and TO option while backup is restorating but getting below error messages.
ERROR:
Msg 3176, Level 16, State 1, Line 4
File 'J:NDFabc.ndf' is claimed by 'Finance_data2'(4) and 'Finance_data1'(3). The WITH MOVE clause can be used to relocate one or more files.
View 4 Replies
View Related
Oct 31, 2006
Can someone point me to a resource for Table Design Considerations for Merge Replication? I have an ASP.Net/SQL2K5 app that I need to run on disconnected machines, then allow dfor data sync through merge replication. I assume that the first step is getting my tables indexed in a replication friendlt manner?
Many Thanks to anyone who can point me in the right direction!
View 3 Replies
View Related
May 14, 2008
When I'm designing a merge join in BIDs, in particular a large one with say a couple hundred columns on the left part of the join (and only 1-2 columns on the right side), it takes 2-3 minutes to close and I notice it pegs the CPU on the computer during that time. I'm developing on a quad dual core 64 bit machine with 32 Gb of RAM.
What makes this problematic is if you have a several merge joins in the pipe, and you do something like add a column. Then you have to re-open every merge join to include that column, waiting 2-3 minutes for it to close. So adding a column to a pipe with 10 merger joins will take 20-30 minutes, most of which is just waiting for edit boxes to close. I've tried the options of "work offline" and setting the ValidateExternalData to false in the merge join property but those don't help. Anyone encounter this problem?
Thanks,
Lee
View 6 Replies
View Related
Dec 28, 2003
Hello,
I have two tables: Orders and Appoitment. Each order can have up to 2 appointments. Now, I need a SELECT statement that gives me this:
ORDER APPT
1 appt1 appt2
2 appt1 appt2
and not this:
ORDER APPT
1 appt1
1 appt2
2 appt1
2 appt2
In other words, I want to merge the two appointments for each order. I tried using the merge statement but it does not work. Tried to google but saw nothing. My database is SQL server. Please help. Thanks
View 3 Replies
View Related
Aug 31, 2015
We have a current database table (PAF) that had a new column added to it named 'Email'. This table also has some other columns including one named [Employee Number].We also have an Excel spreadsheet that has 2 columns 'Employee Number' and 'E-mail Address'. I need to take the E-mail Address field from the spreadsheet, match it up with the employee number between the spreadsheet and PAF table, and then insert the email address into the database column.I'm guessing I would do this using a MERGE statement, correct?
View 4 Replies
View Related
Aug 17, 2006
Is there a merge statement in SQL Server 2000?
I want to combine update and insert statement into single statement as follows.
MERGE INTO MyTable
USING MyTempTable
ON MyTempTable.MatchingField1 = MyTable.MatchingField1
WHEN MATCHED THEN
UPDATE UpdateField1 = MyTempTable.UpdateField1
WHEN NOT MATCHED THEN
INSERT VALUES(MyTempTable.MatchingField1, MyTempTable.UpdateField1)
Currently if I try to run this stmt, it gives error "Incorrect syntax near the keyword 'INTO'."
Thanks
View 3 Replies
View Related
Aug 21, 2012
I am trying to update values in two tables (approx 1-2 million records).
I have decided to use a MERGE statement, however have been unsuccessful to do so.
Below is an example of the statement.
The error is receive is "Incorrect syntax near '('." right after the MERGE line.
Is it not possible to use this syntax ?
Example:
[CODE]
MERGE
( SELECT a.id, b.id2
FROM TableA A
JOIN TableB B
[Code]....
View 5 Replies
View Related
Mar 14, 2014
I am using Merge Statement. Here is my requirement, I don't want to Insert data if Client State is NY, but I want to update all data
When Not Matched
and State not in ('NY')
THEN INSERT
the problem is sometime data NY data is inserted and sometime don't.
View 4 Replies
View Related
Feb 11, 2015
Even rowchecksum is different in target from source then also update is not happening
alter Procedure SP_Archive_using_merge
AS
BEGIN
SET NOCOUNT ON
Declare @Source_RowCount int
Declare @New_RowCount int
[Code] ....
View 1 Replies
View Related
Jun 9, 2007
is there anyway i can merge select statement and insert statement together?
what i want to do is select few attributes from a table and then directly insert the values to another table without another trigger.
for example, select * from product and with the values from product, insert into new_product (name, type, date) values (the values from select statment)
View 3 Replies
View Related
Nov 9, 2007
Hello,
i have a little question.
Is it possible you can't perform an insert statement on a table wich is replicated with merge replication?
I set the replication up and everything works fine, but if i want to perfom an insert statement on the table, i get an error that the values i want to add aren't the same as the one in the table.
I know that merge replication creates a new column and I think that's the problem.
Can someone help me solve this or confirm that you can't perform an insert statement on a replicated table?
Masje
View 3 Replies
View Related
Sep 17, 2015
I have a stored procedure that runs (SQL Server 2012 (SP1) Standard Ed) daily and I never had any problem with this stored procedure. However, there is MERGE statement on the stored procedure and I see an error saying that the MERGE statement failed..Here are the stored procedure and error message:
-- FlushQueue
CREATE PROCEDURE [dbo].[FlushQueue] (@RowCount as int = 10000)
AS
BEGIN
SET NOCOUNT ON;
SET XACT_ABORT ON;
[code]....
The MERGE statement attempted to UPDATE or DELETE the same row more than once. This happens when a target row matches more than one source row. A MERGE statement cannot UPDATE/DELETE the same row of the target table multiple times. Refine the ON clause to ensure a target row matches at most one source row, or use the GROUP BY clause to group the source rows. [SQLSTATE 42000] (Error 8672). The step failed.
Table definition:---CREATE TABLE [dbo].[ImportDefinitions]
(
[NodeName] [varchar](20) NOT NULL,
[ProcedureName] [varchar](100) NOT NULL,
[FilePrefix] [varchar](20) NOT NULL,
[ImportDelay] [int] NOT NULL CONSTRAINT [DF_ImportDefinitions_ImportDelay] DEFAULT ((0)),
[code]...
View 13 Replies
View Related
Apr 10, 2008
Have a View where I need to enter a conditional IF_THEN_ELSE statement in a new field (field has an alias). If this were an Access query I would do following:
IIf([dbo.AR1_CustomerMaster.EmailAddress] Is Null, "A", "B")
How can I accomplish same in View design??
View 2 Replies
View Related
Aug 27, 2012
skip locked records in a MERGE statement and output the list of skipped records.
Through the documentation, internet posts and testing, I believe it is NOT possible. MERGE acts like a single atomic DML statement, and therefore cannot avoid locked records.
I can use the READPAST hint, which will skip the row-locked records. However, it could actually insert duplicate keys in certain cases (as it is ignoring records, i would guess), which would not be acceptable.
View 1 Replies
View Related
Jul 29, 2015
In the following t-sql 2012 merge statement, the insert statement works but the update statement does not work. I know that is true since I looked at the results of the update statement:
Merge TST.dbo.LockCombination AS LKC1
USING
(select LKC.comboID,LKC.lockID,LKC.seq,A.lockCombo2,A.schoolnumber,LKR.lockerId
from
[LockerPopulation] A
JOIN TST.dbo.School SCH ON A.schoolnumber = SCH.type
[Code] ...
Thus can you show me some t-sql 2012 that I can use to make update statement work in the merge function?
View 3 Replies
View Related
Nov 12, 2014
I was using Type 2 for one of our Fact table.... and need to put a flag to know which one is the Current record... I couldn't able to figure how to implement logic in the merge statement... This is an example Query ....I was using like this for my fact table...
Basically I need to track CustomerName and City... So I need a Currentflag (Y) for latest record....
MERGE INTO [dbo].[TargetCustomer] AS TRG
USING [dbo].[MyCustomers] AS SRC
ON TRG.[CustomerID] = SRC.[CustomerID]
AND TRG.[CustomerName]=SRC.[CustomerName]
AND TRG.[City]=SRC.[City]
[Code] .....
View 7 Replies
View Related
Oct 7, 2015
In a t-sql 2012 merge statement that is listed below, the insert statement on the merge statement listed below is not working. The update statement works though.
Merge test.dbo.LockCombination AS LKC1
USING
(select LKC.lockID,LKC.seq,A.lockCombo1,A.schoolnumber
from
[Inputtb] A
JOIN test.dbo.School SCH ON A.schoolnumber = SCH.type
JOIN test.dbo.Locker LKR ON SCH.schoolID = LKR.schoolID AND A.lockerNumber = LKR.number
[code]...
Thus would you tell me what I need to do to make the insert statement work on the merge statement listed above?
View 10 Replies
View Related
Aug 19, 2007
I am wondering if there is a direct query in this case:
I am developing a program to a company which simply sells services
One service may have different prices for different types of clients
The price of any service for any client can change at any time, and I should be able to trace these changes at any time
I made the following tables (simplified): (asterisk for primary key)
(Table) (Fields)
CLIENT_TYPES : ID*, ClientTypeName
SERVICES : ID*, ServiceName
PRICES : ServiceID*, ClientTypeID*, Price, Date*
ORDERS : ID*, Date, ClientTypeID
ORDER_SERVICES : OrderID*, ServiceID*
The field in bold is the area of the question
This is a sample data in the PRICES table:
ServiceID ClientTypeID Price Date1 1 100 1/1/20072 1 150 1/1/20071 2 90 1/1/20072 2 135 1/1/2007
Now if I want to update a price of service 1 for clienttype 1, I add the following row:
1 1 100 1/1/2008
So one product for one client can have any number of prices with different dates
The following query:
SELECT * FROM PRICES WHERE ClientTypeID = 1
will retrieve all prices with repeats for a specific client (#1 here)
What I want is a query to retrieve the most recent prices for specific client for all products, even if a query on query
If there is commemts on table design please tell me
thanks for any one who provides help
View 6 Replies
View Related
Aug 22, 2012
I use the merge statement in a sproc to insert, update and delete records from a staging table to a production table.
In the long sql, here is a part of it,
When Matched and
((Student.SchoolID <> esis.SchoolID
OR
Student.GradeLevel <> esis.GradeLevel
OR
Student.LegalName <> esis.LegalName
OR
Student.WithdrawDate <> esis.WithdrawDate
Student.SPEDFlag <> esis.SPEDFlag
OR
Student.MailingAddress <> esis.MailingAddress)
Then update
Set Student.Schoolid=esis.schoolid,
.....
My question is how about if the column has null values in it.for example
if schoolID is null in production table is null, but in staging table is not null, will the <> return true.or if either side of <> has a null value, will it return true.
I don't want it to omit some records and causing the students records not get updated.If not return true, how to fix this?
View 9 Replies
View Related
