Optimizing A Query To Delete Duplicates

Jul 20, 2005

I have a DELETE statement that deletes duplicate data from a table. It
takes a long time to execute, so I thought I'd seek advice here. The
structure of the table is little funny. The following is NOT the table,
but the representation of the data in the table:

+-----------+
| a | b |
+-----+-----+
| 123 | 234 |
| 345 | 456 |
| 123 | 123 |
+-----+-----+

As you can see, the data is tabular. This is how it is stored in the table:

+-----+-----------+------------+
| Row | FieldName | FieldValue |
+-----+-----------+------------+
| 1 | a | 123 |
| 1 | b | 234 |
| 2 | a | 345 |
| 2 | b | 456 |
| 3 | a | 123 |
| 3 | b | 234 |
+-----+-----------+------------+

What I need is to delete all records having the same "Row" when there exists
the same set of records with a different (smaller, to be precise) "Row".
Using the example above, what I need to get is:

+-----+-----------+------------+
| Row | FieldName | FieldValue |
+-----+-----------+------------+
| 1 | a | 123 |
| 1 | b | 234 |
| 2 | a | 345 |
| 2 | b | 456 |
+-----+-----------+------------+

A slow way of doing this seem to be:

DELETE FROM X
WHERE Row IN
(SELECT DISTINCT Row FROM X x1
WHERE EXISTS
(SELECT * FROM X x2
WHERE x2.Row < x1.Row
AND NOT EXISTS
(SELECT * FROM X x3
WHERE x3.Row = x2.Row
AND x3.FieldName = x2.FieldName
AND x3.FieldValue <> x1.FieldValue)))

Can this be done faster, better, and cheaper?

View 3 Replies


ADVERTISEMENT

Delete Duplicates

Mar 20, 2007

Some guy posted that the syntax: delete top 1 from some_table works for deleting duplicates. I am pretty sure that doesnt work but I wanted to check just in case it did because it would be a really easy to delete duplicates.

The Yak Village Idiot

View 3 Replies View Related

Delete Duplicates

Mar 11, 2008

Hi i need a query to check a table and if any duplicates of the column called "MessageID" and if there are any duplicates then delete them leaving just the one unique MessageID


so i have

MessageID, Number, Text.

12,33333333333,hello
12,33333333333,hello - Delete this one
12,33333333333,hello - Delete this one
14,55555555555,new

View 3 Replies View Related

Delete Duplicates

Jul 23, 2005

I'm having trouble figuring out how to delete some _almost_ duplicaterecords in a look-up table. Here's the table:CREATE TABLE [user_fields] ([fKEY] [char] (16) NOT NULL ,[SEQUENCE] [char] (2) NOT NULL ,[FIELD_LABEL] [varchar] (20) NULL ,[FIELD_VALUE] [varchar] (50) NULL ,[EXPORT_DATE] [datetime] NULL ,CONSTRAINT [PK_user_fields] PRIMARY KEY CLUSTERED([fKEY],[SEQUENCE])CONSTRAINT [FK_USRFLD_INV_DOCID] FOREIGN KEY([fKEY]) REFERENCES [OTHER_TABLE] ([PKEY]))Some values:fKEY SEQUENCE FIELD_LABEL FIELD_VALUE----------------------------------------------------------8525645200692B8919Co. ID #8525645200692B8920Co. ID #8525645200692B8921Co. ID #8525645200692B8913Co/Div/Dept8525645200692B8914Co/Div/Dept8525645200692B8915Co/Div/Dept8525645200692B8916Division8525645200692B8917Division8525645200692B8918Division8525645200692B8910Group8525645200692B8911Group8525645200692B8912Group8525645200692B891 HR ContactJOHN NOVAK8525645200692B892 HR ContactJOHN NOVAK8525645200692B893 HR ContactJOHN NOVAK8525645200692B8924Job Location8525645200692B8922Job Location8525645200692B8923Job Location8525645200692B894 Manager8525645200692B895 Manager8525645200692B896 Manager8525645200692B897 Recruiter8525645200692B898 Recruiter8525645200692B899 Recruiter85256D740081C3A413Co. ID #85256D740081C3A414Co. ID #85256D740081C3A410Co/Div/Dept85256D740081C3A49 Co/Div/Dept85256D740081C3A411Division85256D740081C3A412Division85256D740081C3A48 Group85256D740081C3A47 Group85256D740081C3A42 HR ContactDiana Tarry85256D740081C3A41 HR ContactDiana Tarry85256D740081C3A415Job Location85256D740081C3A416Job Location85256D740081C3A43 Manager85256D740081C3A44 Manager85256D740081C3A45 Recruiter85256D740081C3A46 RecruiterNote that fKEY 8525645200692B89 has three of every FIELD_LABEL, andfKEY 85256D740081C3A4 has two. Both, however, should have only one.Unfortunately, when I do a slect ... having count(*) > 1, I have nearly900 different fKEYs with some variation of this problem.It's just not coming to me how to delete the duplicates (except forsequence). I don't care which of the sequence values I keep but as amatter of preference I tried to do something using max(sequence) but,so far, everything I've tried deletes all records for any given fKEY.Help?Thanks.Randy

View 3 Replies View Related

Help On Find Duplicates And Delete Procedure

Aug 8, 2007

Hi everybody I need help on finding duplicates and deleting the duplicate record depending on name and fname , deleting the duplicates and leaving only the first one.

my PERSON table is this below:

ID name fname ownerid id2

1 a b
2 c c
3 e f
4 a b 1 10
5 c c 2 11

I have this query below that returns records 1 and 4 and 2 and 5 since they have the same name and fname

select * from ( Select name ,fname, count(1) as cnt from PERSON group by
name,Fname ) where cnt > 1


ID name fname ownerid id2

1 a b
4 a b 1 10

2 c c
5 c c 2 11


With this result I need to delete the second record of each group but update the first records with the ownerid and id2 of the second record that would be deleted... I don't know how to proceed with this..

thanks
alex

View 5 Replies View Related

How To Delete Duplicates Values From Table

Aug 13, 2013

Table Design --> Componenet (table name)

ID.
Country ID
BT-ID
Component
Activation

I need to delete the duplicate values from Componenet table

ID CountryID BT-ID Componenet Activation
A3CD GD58 TR77 RX 1
BER2 GD58 TR77 RX 1
XEW7 GD58 TR77 MX 1
O4T4 GD58 TR77 MX 1
PE78 GD58 TR77 GX 1

Expected Output

ID CountryID BT-ID Componenet Activation
A3CD GD58 TR77 RX 1
XEW7 GD58 TR77 MX 1
PE78 GD58 TR77 GX 1

View 14 Replies View Related

Howto: Delete Every Second Record If Duplicates

Jul 19, 2007

Hi.I have a "union" table which results of a union of two tables.Occasionally I could have duplicates, when the same PIN has been addedto both tables, albeit at different Datees/Times, such as:PINNameAdded Date100411A7/11/2007 10:12:58 AM100411A7/17/2007 10:54:23 AM100413B7/11/2007 10:13:28 AM100413B7/17/2007 10:54:39 AM104229C7/6/2007 2:34:13 PM104231D7/6/2007 2:34:25 PM104869E6/10/2007 11:59:12 AM104869E6/22/2007 2:40:18 PMThe question is - how can I delete by queries the first occurence(time-wise) of these duplicates - i.e. I would want to delete thefirst occurence of 100411 (A), the first occurence of 100413 (B), andthe first occurence of 104869 (E) in the example above - records C andD show only once, so they are fine.Is there a MsAccess solution ? Is there a SQL-server solution ?Thank you very much !Alex

View 2 Replies View Related

How To Delete Duplicate Rows Retaining Only One Of Them From Every Set Of Duplicates.

Apr 10, 2008

 
I have a table employee_test having the sample data. The rows with EmployeeID=6 are duplicate rows. I want to delete the duplicates retaining one row for the employeeid=6.
Note :- I don't want to use a temporary table. I want to do this using a single query or at the most in a SP query batch. Please advise. 




EMPLOYEEID

ENAME

SALARY

MANAGERID


1

Anee

1000

11


2

Rick

1200

12


3

JOHN

1100

13


4

ABC

1300

14


5

DEF

1400

15


6

DEF

1400

15


6

DEF

1400

15

View 22 Replies View Related

T-SQL (SS2K8) :: Delete Duplicates From Table Based On Two Columns?

May 20, 2015

Assuming I have a table similar to the following:

Auto_ID Account_ID Account_Name Account_Contact Priority
1 3453463 Tire Co Doug 1
2 4363763 Computers Inc Sam 1
3 7857433 Safety First Heather 1
4 2326743 Car Dept Clark 1
5 2342567 Sales Force Amy 1
6 4363763 Computers Inc Jamie 2
7 2326743 Car Dept Jenn 2

I'm trying to delete all duplicate Account_IDs, but only for the highest priority (in this case it would be the lowest number).

I know the following would delete duplicate Account_IDs:

DELETE FROM staging_account
WHERE auto_id NOT IN
(SELECT MAX(auto_id)
FROM staging_account
GROUP BY account_id)

The problem is this doesn't take into account the priority; in the above example I would want to keep auto_ids 2 and 4 because they have a higher priority (1) than auto_ids 6 and 7 (priority 2).

How can I take priority into account and still remove duplicates in this scenario?

View 3 Replies View Related

Optimizing A Query

Jun 3, 2004

Hello,
I am hoping someone here can help me optimize the following query:
SELECT
INCOMING.DATE_TIME, INCOMING.URL, INCOMING.HITS,
USER_NAMES.USER_LOGIN_NAME,
CATEGORY.NAME
FROM
(wsHQMay2004.dbo.INCOMING INCOMING INNER JOIN wsHQMay2004.dbo.CATEGORY CATEGORY ON INCOMING.CATEGORY = CATEGORY.CATEGORY)
INNER JOIN wsHQMay2004.dbo.USER_NAMES USER_NAMES ON INCOMING.USER_ID = USER_NAMES.USER_ID
WHERE
INCOMING.DATE_TIME >= '2004-05-01 00:00:00.00' AND
INCOMING.DATE_TIME < '2004-06-01 00:00:00.00'
ORDER BY
INCOMING.URL ASC


I am just hoping to get some tips on perhaps a better way to write this query as right now, due to the size of the incoming table, this query just takes forever.

Any advise will be apreciated.

Thanks.

View 5 Replies View Related

Need Help Optimizing A Sql Query...

May 28, 2004

we have an insurance program up and running in our regions and we get random reports of slowness. in an effort to track down all facets of slowness i am looking at all my sql code to make sure it is as efficient as possible. I know a little about SQL and writing SQL statements, enough to help me do my job well. but i do not write optimized code.

if request.form("selPolicyNum") <> "" then
sqlPolicyInfo = "SELECT PIEffectiveDate, PIExpirationdate from PIMaster where PIPolicyNum='" & request.form("selPolicyNum") & "'"
Set rsPolicyInfo = Server.CreateObject("ADODB.Recordset")
Set rsPolicyInfo.ActiveConnection = webLookupConn
'rsPolicyInfo.CursorType = adOpenDynamic
'rsPolicyInfo.LockType = adLockOptimistic
rsPolicyInfo.Source = sqlPolicyInfo
'rsPolicyInfo.CursorLocation = adUseClient
rsPolicyInfo.Open
'response.write sqlPolicyInfo
end if


if request.form("selPolicyNum") <> "" then
sqlRemarkInsert="INSERT INTO clRemarks (PolicyNum, AccountNum, UserID, RemarkBody, CreationDate, PolEffectiveDate, PolExpDate, RemarkCategory1, RemarkCategory2, RemarkCategory3, RemarkCategory4, RemarkCategory5, RemarkCategory6, RemarkCategory7, RemarkCategory8, RemarkCategory9, RemarkCategory10, RemarkCategory11, RemarkCategory12, RemarkCategory13) VALUES ('"
sqlRemarkInsert=sqlRemarkInsert & request.form("selPolicyNum") & "', '" & request.form("txtAcctNum") & "', '" & request("aysmenu")("userid") & "', '" & Replace(request.form("RemarkBody"),"'","''") & "', '" & CurrDate & "', '" & rsPolicyInfo("PIEffectiveDate") & "', '" & rsPolicyInfo("PIExpirationDate") & "', '" & request.form("chkIssuingInstructions") & "', '" & request.form("chkLossControl") & "', '" & request.form("chkLossHistoryClaims") & "', '" & request.form("chkReinsurance") & "', '" & request.form("chkMVRDriverIssues") & "', '" & request.form("chkBilling") & "', '" & request.form("chkGeneral") & "', '" & request.form("chkCancellationDNR") & "', '" & request.form("chkPolicyAmendments") & "', '" & request.form("chkExperienceRatingInfo") & "', '" & request.form("chkDiscretionaryPricing") & "', '" & request.form("chkPremiumAudit") & "', '" & request.form("chkQuotingInstructions") & "')"
else
sqlRemarkInsert="INSERT INTO clRemarks (AccountNum, UserID, RemarkBody, CreationDate, RemarkCategory1, RemarkCategory2, RemarkCategory3, RemarkCategory4, RemarkCategory5, RemarkCategory6, RemarkCategory7, RemarkCategory8, RemarkCategory9, RemarkCategory10, RemarkCategory11, RemarkCategory12, RemarkCategory13) VALUES ('"
sqlRemarkInsert=sqlRemarkInsert & request.form("txtAcctNum") & "', '" & request("aysmenu")("userid") & "', '" & Replace(request.form("RemarkBody"),"'","''") & "', '" & CurrDate & "', '" & request.form("chkIssuingInstructions") & "', '" & request.form("chkLossControl") & "', '" & request.form("chkLossHistoryClaims") & "', '" & request.form("chkReinsurance") & "', '" & request.form("chkMVRDriverIssues") & "', '" & request.form("chkBilling") & "', '" & request.form("chkGeneral") & "', '" & request.form("chkCancellationDNR") & "', '" & request.form("chkPolicyAmendments") & "', '" & request.form("chkExperienceRatingInfo") & "', '" & request.form("chkDiscretionaryPricing") & "', '" & request.form("chkPremiumAudit") & "', '" & request.form("chkQuotingInstructions") & "')"
end if
rsRemarkInsert=ComProConn.Execute (sqlRemarkInsert)

rsPolicyInfo.close
Set rsPolicyInfo=nothing


that is the code used to store a remark into the system. is this code optimized already or should some of the db parameters be changed to make things faster? this is just an example of many of the SQL statements that i may or may not have to fix. thank you for any and all help.

View 1 Replies View Related

Help Optimizing Query

Jun 23, 2008

I have the following query that works fine but i'm wondering if there is a way to optimize it better as when I analyze through sql profiler it is at the top of the list of using the cpu

SELECT DISTINCT site, d, (SELECT COUNT(id) FROM anP aPV2 WHERE aPV2.confirmed=1 and aPV2.stage=2 and aPV2.inserted=0 and aPV2.site=aPV1.site and aPV2.d>=aPV1.d and aPV2.d<=aPV1.d) AS mycount FROM anP aPV1 WHERE confirmed=1 AND stage=2 AND inserted=0 ORDER BY site,d

View 8 Replies View Related

Optimizing Query

Sep 18, 2007

Hi,

I have the below query written so that i do not insert entries that is already existing in the table. I am trying to put in 70000 entries at a single shot and it breaks down. Can anybody help me optimize the below query so that it doesnt break? Is there any other way I can write this query?

Please do help me with this. Thanks in advance. The table in which i am inserting the entries has a composite key composed of ACCT_NUM_MIN and ACCT_NUM_MAX. I am getting this from a table which doesnt have a primary key(CORE)


INSERT INTO CRF (CORE_UID,ACCT_NUM_MIN, ACCT_NUM_MAX,BIN, BUS_ID,BUS_NM,ISO_CTRY_CD, REGN_CD, PROD_TYPE_CD, CARD_TYPE)
SELECT UID , LEFT(ACCT_NUM_MIN,16), LEFT(ACCT_NUM_MAX,16), BIN, BUS_ID, BUS_NM, ISO_CTRY_CD, REGN_CD, PROD_TYPE_CD, CARD_TYPE FROM CORE o
WHERE NOT EXISTS (SELECT * FROM CRF i
WHERE o.ACCT_NUM_MIN = i.ACCT_NUM_MIN
AND o.ACCT_NUM_MAX = i.ACCT_NUM_MAX)

View 20 Replies View Related

Optimizing A Query

Nov 27, 2007

hi all

View 8 Replies View Related

Optimizing A Big Query

Feb 9, 2007

To start with, I'll give a simplified overview of my data.BaseRecord (4mil rows, 25k in each Region)ID | Name | Region | etcOtherData (7.5mil rows, 1 or 2 per ID)ID | Type(1/2) | DataProblemTable (4mil rows)ID | ConcatenatedHistoryThe concatenated history field is a nvarchar with up to 20 differentpipe delimited date/code combinations, eg. '01/01/2007X|11/28/2006Q|11/12/2004Q|'Using left outer joins (all from base, the rest optional) I've got aview something like:View (4mil rows)ID | Name | Region | etc | Data | Data2 | ConcatenatedHistoryQuerying it, it takes about 15-20 seconds to do this:Select ID, Name, Region, etc, Data, Data2, ConcatenatedHistory


Quote:

View 5 Replies View Related

Help With Optimizing Query

Feb 2, 2007

if I have the following tables:

Table Customer_tbl

cust_id as int (PK), ...

Table Item_tbl

item_id as int (PK), ...

Table Selected_Items_tbl

selected_item_id as int (PK), cust_id as int (FK), item_id as int (FK), ...

-------

With the following query:

select cust_ID from selected_items_tbl WHERE item_id in (1, 2, n) GROUP BY cust_id, item_id HAVING cust_id in (select cust_id from selected_items_tbl where item_id = 1) AND cust_id in (select cust_id from selected_items_tbl where item_id = 2) AND cust_id in (select cust_id from selected_items_tbl where item_id = n)

-------

Each of these tables has other items included. Selected_Items_tbl holds zero to many of the items from the item_tbl for each customer. If I am searching for a customer who has item 1 AND item 2 AND item n, what would be the most efficient query for this? Currently, the above query is what I am working with. However, it seems that we should be able to do this type of search in a single query (without subqueries).

View 3 Replies View Related

Optimizing Query To Run

Sep 27, 2006

I'm trying to get a query to run which looks at completed orders that have not had another paid order in 180 days. The database I'm running it against is very large so I can't get it to complete. Where's what I've got:

select Date =cast(cl1.cl_rundate as datetime(102)),count(cl1.cl_recno) as 'Completed Initials', cl1.cl_status as Status from dbo.vw_Completedorders cl1 where cl1.lob_lineofbusiness = 'aaa'
and cl1.cl_rundate > '20060801' and not exists (
select cl2.cl_company from dbo.vw_Paidorders cl2 where
cl2.lob_lineofbusiness = 'aa'and cl2.cl_company = cl1.cl_order and cl2.cl_rundate > '20060101' and datediff(day,cl2.cl_rundate,cl1.cl_rundate) < 180)
group by cl1.cl_status, cl1.cl_rundate

View 11 Replies View Related

Optimizing SELECT Query

Nov 7, 2007

Hi
I have one table (tableDemo) with following structure:
TSID                          bigint (Primary Key)TskID                         bigint Sequence                   bigint Version                      bigint Frequency                  varchar(500) WOID                         bigint DateSchedule             datetime TimeStandard             real MeterEstimated          real MeterLast                   real Description                 ntext CreatedBy                  bigint CreatedDate               datetime ModifiedBy                 bigint ModifiedDate              datetime Id                               uniqueidentifier 
I have 8000 records in this table. When I fire simple select command (i.e. select * from tableDemo) it takes more than 120 seconds.
Is there any techniques where I can access all these records within 2-3 seconds ?
Regards,
ND

View 2 Replies View Related

Optimizing The Query And Index

Nov 4, 2000

hi, what are the tools that I can use to Optimize the query/ index.

I know that if I am running a query on a table I create index on the fields where I use in the where clause, is this a right thinking.

Someone told me how do I determine what columns should be indexed, I told him the fields that I use in the where clause should be indexed to speed up the process of retrieving the data. ..... is this answer correct. if not please advice the correct one.
Thanks

Ahmed

View 1 Replies View Related

Optimizing Query Execution...

Dec 17, 2005

We have SQL Server 2000 and int is an Oracle linked server. I'm trying to run the following query...


SELECT DISTINCT a.auf_nr AS OrderNo,
e.ku_name AS Customer,
d.bestell_dat AS OrdDate,
d.liefer_dat AS DelvDate,
CAST(SUM(b.anz) AS FLOAT) Qty,
CAST(SUM((CAST(c.breite AS FLOAT) / 1000 * CAST(c.hoehe AS FLOAT) / 1000) * b.anz) AS FLOAT) SQM,
CAST(SUM(a.liefer_offen) - (SUM(a.anz) - SUM(b.anz)) AS FLOAT) AvailDelv,
CAST(SUM(a.liefer_anz) AS FLOAT) Delvd,
CAST(SUM(c.sum_brutto*a.anz) AS FLOAT) Value

FROM liorder..LIORDER.AUF_STAT a,
liorder..LIORDER.AUF_LIP_STATUS b,
liorder..LIORDER.AUF_POS c,
liorder..LIORDER.AUF_KOPF d,
liorder..LIORDER.KUST_ADR e

WHERE a.auf_nr = b.auf_nr and
b.auf_nr = c.auf_nr and
c.auf_nr = d.auf_nr and
d.kunr = e.ku_nr and
a.auf_pos = b.auf_pos and
b.auf_pos = c.auf_pos and
b.lip_status = 7 and
c.ver_art !='V' and
a.history = 0 and
a.rg_stat != 2 and
e.ku_name IS not null and
e.ku_vk_ek = 0 and
d.bestell_dat BETWEEN '01/01/2005' and '12/17/2005'

GROUP BY a.auf_nr,
d.liefer_dat,
b.lip_status,
d.bestell_dat,
e.ku_name,
d.kopf_tour,
d.kopf_firma

HAVING CAST(SUM(a.liefer_offen)-(SUM(a.anz)-SUM(b.anz)) AS FLOAT) > 0


..and it takes around 2 minutes to show the results even if the date range is of the same date. I even tried to use an indexed column but I still get the same slow execution time. I even tried to create a UDF so that the WHERE clause would be resolved remotely on the Oracle DB but still the same. Is there anyway I can do it in much more efficient and faster way?

View 1 Replies View Related

Poor Query Optimizing

Nov 4, 2007

Hi there,

I've got a SQL 2005 table(DataTable) with 5.5 million rows and a clustered index on an integer column. The column is a foreign key pointing to another table (LookupTable) that associates the integer with an nvarchar(50) column. Both integer and nvarchar(50) columns are in a clustered index.

I'm running equivalent select statements with slightly different where clauses, and am not seeing the performance similarities I'd expect.

select * from DataTable join LookupTable on DataTable.Integer = LookupTable.Integer

1. Setting the where clause to the nvarchar(50) value on the LookupTable takes 64 seconds.
2. Setting the where clause to Integer = (select integer from LookupTable where nvarchar(50) = 'mysearchstring') takes 61 seconds.
3. Setting the where clause to Integer = 526 takes 1 second.

Is this poor design in SQL, or is it a misconfiguration? I would have expected SQL to modify my query to essentially be #3 in all cases before touching the DataTable, which would cause all three versions to take 1 second.

Thanks,
Jason

View 2 Replies View Related

Optimizing This Duplicate Finding Query

Sep 11, 2005

Hi everyone!

I am parsing a database structed like the following for duplicates.


Code:


keywordnegativeexactbroadphrase
Alpha0.120.36
Alpha0.120.37
Alpha0.120.37
Alpha0.220.37
Alpha0.120.37
Alpha0.120.37
Alpha0.120.37
Beta0.43212
Charlie0.240.31
Delta0.72212
Epsilon410.31
Foxtrot250.22
Grape420.215
Hotel 230.13
Indigo0.3721
Juliet21220.14
Kilroy0.3110.15
Lemon0.2201
Mario0.2502
Nugget0.130.72
Oprah2141
Polo01225
Polo01224
Polo01225
Q-Bert31442
Romeo12100.1
Salty2200.1
Tommy10.30.132.4
Uri10.30.132
Uri20.20.132
Uri20.30.122
Uri20.30.131
Vamprie0.120.1315
Wilco0.210.21
X-Ray00.220
Yeti020.20
Zebra1310



The duplicates that this thread relates to are the kind with duplicate "keyword" entries AND dissimilar field entries; i.e. :


Code:


keyword negative exact broad Phrase
Polo 0 122 4
Polo 0 122 5



I've come up with an SQL query that seems to return all of these duplicates (save one of each type- the 'real', unique entry). However I think I made the query very inefficient. My SQL is very bad; this query will be running over tens of thousands of rows, so if it can be at all optimized I would greatly appreciate your help!

What I have so far is:



Code:


string query1 = "SELECT * FROM TableName" +
" WHERE EXISTS (SELECT NULL FROM TableName" + " b" +
" WHERE b.[keyword]= " + "TableName"+ ".[keyword]" +
" AND b.[negative]<> " + "TableName"+ ".[negative]" +
" ORb.[keyword]= " + "TableName"+ ".[keyword]" +
" ANDb.[exact]<> " + "TableName"+ ".[exact]" +
" ORb.[keyword] = " + "TableName"+ ".[keyword]" +
" ANDb.[broad]<> " + "TableName"+ ".[broad]" +
" ORb.[keyword]= " +"TableName"+ ".[keyword]" +
" ANDb.[phrase]<> "+"TableName"+ ".[phrase]" +
" GROUP BY b.[keyword], b.[broad], b.[exact]" +
" HAVING Count(b.[keyword]) BETWEEN 2 AND 50000)" ;



the algoritm seems to check every column of every row in order to determine a duplicate. Seems straightforward to me, but alas slow...

Is there a better/faster way I can do this? Thanks for you help!

View 1 Replies View Related

Optimizing Query - Product Tips

Jan 6, 2004

Hello everyone!
I've got a problem with a real slow query, I would be very happy if somebody has any idea to improve the speed of it...
The idea is to get the top 2 products, a customer hasn't bought wich are in his interest...

query (simplificated)
-------------------------------------------------
SELECT TOP 2 prodID, Title, Price FROM bestSold7Days WHERE
prodID NOT IN (SELECT prodID FROM orders INNER JOIN orderProducts ON orders.orderID = orderProducts.orderID WHERE (orders.custID=394))
AND
(prodType = COALESCE((SELECT TOP 1 products.prodID FROM orders INNER JOIN orderProducts ON order.orderID = orderProducts.orderID INNER JOIN products ON orderProducts.prodID = products.prodID WHERE (orders.custID=394) GROUP BY products.prodType ORDER BY SUM(orderProducts.PCS) DESC), 2))
-------------------------------------------------
end query

(COALESCE is for replacing if the customer hasnt ordered anything, or hasnt ordered anything of this type)...

Thanks for any time spent!

View 14 Replies View Related

Query For Duplicates

Mar 8, 2007

Hi All.


I have a Table with records and some of them are actually multiple records. I need to query SQL 2005 (compare) field values on more than one column to return these records.

I am an absolute beginner and need some guidance on how to start on this. Can anyone advise.

Thanks.

-Indio

View 4 Replies View Related

T-SQL (SS2K8) :: Rank Duplicates But Only Rows Involved In Duplicates?

Oct 22, 2014

I have a table with 22 million Business records. I can see that there are duplicates when I group by BusinessName and Address and Phone. I'd like to place only the duplicates into a table, with a ranking, oldest business key gets a ranking of 1.

As a bonus I'd like each group to have a distinct group name (although not necessary, just want to know how to do this)

Later after I run more verifications to make sure these are not referenced elsewhere I'll delete everything with a matchRank > 1 out of the main Business table.

DROP TABLE [dbo].[TestBusiness];
GO
CREATE TABLE [dbo].[TestBusiness](
[Business_pk] INT IDENTITY(1,1) NOT NULL,
[BusinessName] VARCHAR (200) NOT NULL,
[Address] VARCHAR(MAX) NOT NULL,

[code]....

View 9 Replies View Related

Query To Find Duplicates

Jul 19, 2004

Hi

I am trying to find when a name has been entered more than once into 1 database table.

I'm currently doing something like this (can't remember exactly, not at work)


SELECT COUNT(*) AS Cnt, Name
FROM tblTable
GROUP BY Name
ORDER BY Cnt Desc

This brings back all the Names in the database and tells me which are duplicates but I want to just have the results of the duplicate values and not the single values.

Hope you can help.

Thanks

View 2 Replies View Related

Query To Show Duplicates

Aug 16, 2005

mytable      fld1 int     primkey                  fld2 varchar(20),                  fld3 varchar(20)From the definition of the above table, how do i do i modify my below query to only select the rows with duplicates in fld2.  I know a groupby with a having count will display the duplicates for a given field, but i want my query to see all the fields and rows that are duplicates.     select fld1, fld2, fld3 from mytable                

View 1 Replies View Related

T-SQL (SS2K8) :: Duplicates In Sub Query

Sep 13, 2014

I have the piece of sql code here below that keeps giving out duplicates. How to resolve this.

isnull((select distinct (SUM(a1.ActualDebit) - SUM(a1.ActualCredit) ) from #MainAccount a1
LEFT OUTER JOIN
#BudgetAccount bb ON aa.AccountID = bb.AccountID AND a1.PeriodStartdate = bb.PeriodStartDate and
a1.DateMonth=bb.DateMonth and a1.Budget = bb.Budget WHERE a1.AccountID = aa.AccountID and
a1.Refdate >= @FROMDATE and a1.Refdate <= @TODATE GROUP BY a1.group1, a1.Group2),0)
As Actual_CurrentMonth,

View 1 Replies View Related

Search Query Help With Duplicates?

Sep 7, 2006

Hello everyone,

Obviously, I'm new here so I'd like to start by thanking anyone for their help with this very newbie SQL search query question.

I have been tasked with trying to figure out how to get search results from several different columns of two different tables within a database. I have successfully accomplished this, but one column has many many duplicates. I have read on these forums about using the DISTINCT command to eliminate this, but so far have been unsuccessful. Here is the search I'm using...(columns 1, 2, and 3 are from one table and columns 4 and 5 are from another. The results need to be in ascending order (alphabetical) of column1.

select column1, column2, column3, column4, column5 from table1, table2 order by column1;

The result of this is great except that column1 has many duplicates.

I tried entering the search like this...

select distinct column1, column2, column3, column4, column5 from table1, table2 order by column1;

...but that didn't seem to help.

Can someone steer me in the right direction?

Thanks for any and all assistance.

View 8 Replies View Related

I Use SQL 2000, Can You Use One Delete Query To Delete 2 Tables?

Nov 26, 2007

this is my Delete Query NO 1
alter table ZT_Master disable trigger All
Delete ZT_Master WHERE TDateTime> = DATEADD(month,DATEDIFF(month,0,getdate())-(select Keepmonths from ZT_KeepMonths where id =1),0) AND TDateTime< DATEADD(month,DATEDIFF(month,0,getdate()),0)
alter table ZT_Master enable trigger All
 
I have troble in Delete Query No 2
here is a select statemnt , I need to delete them
select d.* from ZT_Master m, ZT_Detail d where (m.Prikey=d.MasterKey)  And m.TDateTime> = DATEADD(month,DATEDIFF(month,0,getdate())-(select Keepmonths from ZT_KeepMonths where id =1),0) AND m.TDateTime< DATEADD(month,DATEDIFF(month,0,getdate()),0)
I tried modified it as below
delete d.* from ZT_Master m, ZT_Detail d where (m.Prikey=d.MasterKey)  And m.TDateTime> = DATEADD(month,DATEDIFF(month,0,getdate())-(select Keepmonths from ZT_KeepMonths where id =1),0) AND m.TDateTime< DATEADD(month,DATEDIFF(month,0,getdate()),0)
but this doesn't works..
 
can you please help?
and can I combine these 2 SQL Query into one Sql Query? thank you

View 1 Replies View Related

How To Run Delete Query / Delete Several Rows Just By One Click ?

Feb 16, 2008

I'm using SqlDataSource and an Access database. Let's say I got two tables:user: userID, usernamemessage: userID, messagetextLet's say a user can register on my website, and leave several messages there. I have an admin page where I can select a user and delete all of his messages just by clicking one button.What would be the best (and easiest) way to make this?Here's my suggestion:I have made a "delete query" (with userID as parameter) in MS Access. It deletes all messages of a user when I type in the userID and click ok.Would it be possible to do this on my ASP.net page? If yes, what would the script look like?(yes, it is a newbie question) 

View 2 Replies View Related

Hi, Pls See The Query Inside, Reg. Removing Duplicates..

Mar 17, 2008

-- declared variables
declare @database_name varchar(100), @table_name varchar(100), @primary_key_field varchar(100)
declare @list varchar(8000)
-- set values to variables
set @list = ''
set @database_name = 'data200802_dan'
set @table_name = 'other02'
set @primary_key_field = 'callid'

use database

select @list = @list + column_name + ', '
from information_schema.columns
where table_name = @table_name --table name
and column_name != @primary_key_field --unique identifier
select @list = substring(@list, 1, len(rtrim(@list)) - 1)

--above 5 lines btw came from a helper in the msdn forum. thanks

SELECT DISTINCT @list
INTO '#' + @table_name
FROM @table_name
@table_name + ':'
IF (SELECT COUNT(*) FROM @database_name + '.dbo.' + @table_name) = 0
BEGIN
INSERT INTO
@database_name + '.dbo.' + @table_name + '(' + @list + ')'
SELECT
@list
FROM
'#' + @table_name
END
ELSE
BEGIN
DELETE @database_name + '.dbo.' + @table_name +' ( ' + @list + ')'
GOTO @table_name
END
DROP TABLE '#' + @table_name



the query above is basically.. selecting all the fields from a table in database W/OUT their primary key. then putting them in a temp table.. delete all the records in the original table. then paste the records from the temp table into the original table.

is there a way for this to work? i don't know how to use the variables w/ this script. please help me correcting this query..

this is for removing duplicates btw. thanks!

View 3 Replies View Related

T-SQL (SS2K8) :: Join Query Produce Duplicates?

Aug 12, 2014

join three tables and wont be duplicate records.

I have tried and attached the computed results and also expecting results.

IF OBJECT_ID('tempdb..#tmpExam1')IS NOT NULL DROP TABLE #tmpExam1
IF OBJECT_ID('tempdb..#tmpExam2')IS NOT NULL DROP TABLE #tmpExam2
IF OBJECT_ID('tempdb..#tmpExam3')IS NOT NULL DROP TABLE #tmpExam3

[Code]....

View 4 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved