Finding Duplicate Records
Oct 29, 2007
Hello,
I searched for all the posts which covered my question - but none were close enough to answer what i'm trying to do. Basically, the scenario is thus;
Table1 contains values for UserID, Account code, and Date.
My query (below) is trying to find all the accounts assigned to a particular user ID, but also those duplicate account codes which belong to a second user ID. The date column would be appended to the result set.
The query I'm using is as follows;
select acccountcode, userid, date from dbo.table1
where exists (select accountcode from dbo.table1 where accountcode = table1.accountcode
group by accountcode
having count(*) > 1)
and userid = 'x-x-x'
order by accountcode
What I think this produces is a list of all files where a duplicate exists, but of course it leaves out the 2nd UserID...which is crucial.
Hopefully this makes sense. Any insight my fellow DBA's can share would be greatly appreciated!
Thanks,
D.
View 1 Replies
ADVERTISEMENT
Feb 18, 2014
I am trying to build a select statement that will
1). find records where a duplicate field exists.
2.) show the ItemNo of records where the Historical data is ' '.
I have the query which shows me the duplicates but I'm stuck when trying to only show records where the Historical field = ' '.
Originally I thought I could accomplish this goal with a sub query based off of the original duplicate result set but SQL server returns an error.
Below you will find a temp table to create and try in your environment.
create table #Items
(
ItemNovarchar (50),
SearchNo varchar (50),
Historical int
[Code] ....
Ultimately I need a result set that returns 'ATMR10 MFZ N', and 'QBGFEP2050 CH DIS'.
View 3 Replies
View Related
Feb 23, 2001
Hi,
I am putting my problem in an example as I Feel it would be clear.
Assume my table PEOPLE is having 4 columns with 6 rows, the SlNo being primary key.
SlNo Name LastName birthdate
1 A B x --
2 C B x |-- 1 pair (A, B, x)
3 D E y --|------------
4 A E y | |
5 A B x __| |-- 2'nd pair (D, E, y)
6 D E y ---------------
In this scenario, I need to find SlNo values having similar values in other columns. The o/p for above must be:
1
5
0
3
6
0 (0 needs to include in output for distinction in the sets)
(a)IS THIS POSSIBLE TO DO IN ONE SELECT STATEMET? and HOW?
(b)If I create another temp table tempPEOPLE and select distinct row information of the 2'nd, 3'rd and 4'th columns from the PEOPLE table and
then selecting SlNo's where the information match, I am able to get o/p
1
5
3
6
without 0...and I cannot makeout the distinct sets in this.
HOW DO I FIND THE DISTINCTION IN SETS?
Reshma.
View 1 Replies
View Related
Jan 16, 1999
I have a problem with a 3rd party piece of software. Doesn't matter which, really. The problem lies
in a table called payments, with a column called txnumber...the newest version of this software fails
a check during installation with the message "duplicate txnumber in payment table." Not sure how
this could have happened, since there is no way to manually assign the txnumber, but the point is
not important. What I'd like to do is figure out a sql script that will return only the duplicate number(s)
so that I can either remove or change them manually. Unfortunately, I'm not terribly familiar with sql.
Any suggestions?
View 2 Replies
View Related
Sep 11, 2005
Hi everyone!
I am parsing a database structed like the following for duplicates.
Code:
keywordnegativeexactbroadphrase
Alpha0.120.36
Alpha0.120.37
Alpha0.120.37
Alpha0.220.37
Alpha0.120.37
Alpha0.120.37
Alpha0.120.37
Beta0.43212
Charlie0.240.31
Delta0.72212
Epsilon410.31
Foxtrot250.22
Grape420.215
Hotel 230.13
Indigo0.3721
Juliet21220.14
Kilroy0.3110.15
Lemon0.2201
Mario0.2502
Nugget0.130.72
Oprah2141
Polo01225
Polo01224
Polo01225
Q-Bert31442
Romeo12100.1
Salty2200.1
Tommy10.30.132.4
Uri10.30.132
Uri20.20.132
Uri20.30.122
Uri20.30.131
Vamprie0.120.1315
Wilco0.210.21
X-Ray00.220
Yeti020.20
Zebra1310
The duplicates that this thread relates to are the kind with duplicate "keyword" entries AND dissimilar field entries; i.e. :
Code:
keyword negative exact broad Phrase
Polo 0 122 4
Polo 0 122 5
I've come up with an SQL query that seems to return all of these duplicates (save one of each type- the 'real', unique entry). However I think I made the query very inefficient. My SQL is very bad; this query will be running over tens of thousands of rows, so if it can be at all optimized I would greatly appreciate your help!
What I have so far is:
Code:
string query1 = "SELECT * FROM TableName" +
" WHERE EXISTS (SELECT NULL FROM TableName" + " b" +
" WHERE b.[keyword]= " + "TableName"+ ".[keyword]" +
" AND b.[negative]<> " + "TableName"+ ".[negative]" +
" ORb.[keyword]= " + "TableName"+ ".[keyword]" +
" ANDb.[exact]<> " + "TableName"+ ".[exact]" +
" ORb.[keyword] = " + "TableName"+ ".[keyword]" +
" ANDb.[broad]<> " + "TableName"+ ".[broad]" +
" ORb.[keyword]= " +"TableName"+ ".[keyword]" +
" ANDb.[phrase]<> "+"TableName"+ ".[phrase]" +
" GROUP BY b.[keyword], b.[broad], b.[exact]" +
" HAVING Count(b.[keyword]) BETWEEN 2 AND 50000)" ;
the algoritm seems to check every column of every row in order to determine a duplicate. Seems straightforward to me, but alas slow...
Is there a better/faster way I can do this? Thanks for you help!
View 1 Replies
View Related
May 29, 2007
Hi
i tried the following query and able to get the list of foreign keys with column names as well as referred tables and referenced column
select parent_column_id as 'child Column',object_name(constraint_object_id)as 'FK Name',object_name(parent_object_id) as 'parent table',name,object_name(referenced_object_id)as 'referenced table',referenced_column_id
from sys.foreign_key_columns inner join sys.columns on (parent_column_id = column_id and parent_object_id=object_id)
Order by object_name(parent_object_id) asc
but i am not able to get the fks created more than once on same column refering to same pk
Thanks in Advance
View 4 Replies
View Related
Feb 8, 2007
Yet another simple query that is eluding me. I need to find records in a table that have the same first name and last name. Because the table has a primaty key, these people were entered twice or they share the same first and last name.
How could you query this:
ID fname lname
10001 Bill Jones
10002 Joe Smith
10003 Sue Jenkins
10004 John Sanders
10005 Joe Smith
10006 Harrold Simpson
10007 Sue Jenkins
10008 Sam Worden
and get a result set of this:
ID fname lname
10002 Joe Smith
10005 Joe Smith
10003 Sue Jenkins
10007 Sue Jenkins
View 2 Replies
View Related
Jul 20, 2005
What is the best way to compare two entries in a single table wherethe two fields are "almost" the same?For example, I would like to write a query that would compare thefirst two words in a "company" field. If they are the same, I wouldlike to output them.For example, "20th Century" and "20th Century Fox" in the companyfield would be the same.How do I do this? Do I need to use a cursor? Is it as simple as using"Like?"
View 2 Replies
View Related
Nov 3, 2014
There are many duplicate records on my data table because users constantly register under two accounts. I have a query that identify the records that have a duplicate, but it only shows one of the two records, and I need to show the two records so that I can reconcile the differences.The query is taken from a post on stack overflow. It gives me 196, but I need to see the 392 records.
How to identify the duplicates and show the tow records without having to hard code any values, so I can use the query in a report, and anytime there are new duplicates, the report shows them.
SELECT
[groom_first_name]
,[groom_last_name]
,[bride_first_name]
,[bride_last_name]
[code]....
View 5 Replies
View Related
Mar 7, 2001
Hi,
OK, here's a question for all you SQL Gurus.
I want to find all the records that are in t2 but are not in t1.
(NOTE: table print outs at bottom of post)
I could write this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~``
select t2.t from t2 where t not in (select t from t1)
t
-----------
6
(1 row(s) affected)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~``
BUT it takes to long.
So I thought of this:
select t2.t , t1.t from t2 left join t1 on t2.t = t1.t where t1.t = null
BUT it doesn't work.
WHY NOT?
Is there anything similar that i can write???
It should work, heres the output for the simple left join:
select t2.t , t1.t from t2 left join t1 on t2.t = t1.t
t t
----------- -----------
1 1
2 2
3 3
4 4
5 5
6 NULL
7 7
(7 row(s) affected)
Thanks,
Benny Hill
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~``
select * from t1
t
-----------
1
2
3
4
5
7
(6 row(s) affected)
select * from t2
t
-----------
1
2
3
4
5
6
7
(7 row(s) affected)
View 4 Replies
View Related
Mar 10, 2004
I am trying to find records that are orphaned in a data base after thier parent record (in another table) has been deleted.
Something happened to the constraints and I had no idea until now this had happened.
I have the faint rustle of a query in query sort of thing but my knowledge needs expansion.
PS: sorry for the cross post - just noticed that I was in the wrong forum before.
Thanks,
Matt
View 11 Replies
View Related
Jul 20, 2005
I need to create a table that would be the result set of a comparisonbetween table a and table b? Table a and b first 2 fields will always bethe same (CustomerName and CustomerNumber). But if the Address1 fieldchanges in table a, I would like to throw that whole row into mycomparison table. Almost like a Select Into with a sub query that wouldinclude a WHERE TableA.field <> TableB.field. I would need to do thiscomparison for about 8 fields. Help appreciated for my syntax is prettybad. Thanks.Steve*** Sent via Developersdex http://www.developersdex.com ***Don't just participate in USENET...get rewarded for it!
View 1 Replies
View Related
Feb 18, 2008
I have the following generic table structure.
CREATE TABLE tableName (
...
, datetime_created datetime DEFAULT GetDate()
, created_by_user char(12)
)
I want to find out which records were created within a given period of time per user. For example I want to know if the user "georgev" (or any users) has created more than one record in this table within a 2 minute period.
Hopefully thsi will warm your brains this cold Monday morning :p
Any more information needed, let me know!
View 8 Replies
View Related
Jun 19, 2012
I've got a subquery that keeps throwing up an error but I can’t think of another way of completing the query.
Firstly I need to find records where TestQty =3
(this would find 2 records with TestQty =3, TestNumber, 7171003 and 7088650)
The TestNumber is not unique so I would like the final set of records to include all the records with TestNumber, 7171003 and 7088650
TestNumberTestQty
7088650____________3
7088650____________1
7088650____________2
7088650____________1
7088650____________2
7171003____________1
7171003____________3
7171003____________2
Code below:
[Code SQL]
USE TestWarehouse
IF ('dbo.TestItems') IS NOT NULL
DROP TABLE [dbo].[TestItems];
GO
CREATE TABLE [dbo].[TestItems]
( TestItem int not null IDENTITY (1,1)
[Code] .....
View 5 Replies
View Related
Jun 16, 2007
In the "tblEmailGroupLink" table...
I need to find all records with the "UnSubscribed" field having a "True" value. All these records will have a corresponding "Emailid" field.
In the "tblEmailAddress" table...
The same "Emailid" field has a corresponding "EmailAddress" field.
What needed is all the email addresses found in the "EmailAdddress" field of these records.
I'm very new at this so I hope I explained this right. I'd really appreciate any help I can get.
Thanks,
Bill
View 2 Replies
View Related
Mar 13, 2008
Here's what I'd like to be able to do: I have a queue that holds any number tasks, so something like this here:
queue_1 task_a
task_b
task_c
task_d
Workers are assigned to teams, Red team, Blue team, Green team. What I need to do is identify instances where all tasks for a given queue have been handled by one team. Once a task has been assigned to a queue any team can work on it, but when only one team has completed every task in a queue a bonus should be awarded.
I'm looking for this to award a bonus:
queue_num task_num team
queue_1 task_a red
task_b red
task_c red
task_d red
No bonus for any team here
queue_num task_num team
queue_1 task_a red
task_b blue
task_c red
task_d green
So the red team earns a bonus. Now, I have thousands of queues each containing any number of tasks. Using T-SQL how can I find all queues where only one team was responsible for completeing every task assigned to the queue? Do I have to use a cursor and eval each task coming through or is there a faster, more efficient way to handle this in SQL?
View 4 Replies
View Related
Jul 9, 2007
I need to find the name of the user that signed in to the web page. For this, I have a web page with a text box, the program should access a SQL database use, use the userID and return the name. My code (working) does the following:string mySqlQuery = "";
string strCnn = "";SqlDataReader myDataReader = null;
mySqlQuery = "SELECT c_name FROM database WHERE c_user='demo'";
//connect to the databasestrCnn = "Data Source=" + connectionServer + "; Initial Catalog=" + connectionDatabase + "; user id=" + connectionUserID + "; password=" + connectionPassword + "; Integrated Security=false";cnn = new SqlConnection(strCnn);
try
{
cnn.Open();MessageBox.Show("Success Opening the Database");
}
catch
{MessageBox.Show("Problems with the database");
}SqlCommand sqlSelectCommand = new SqlCommand(mySqlQuery, cnn);
tbName.Text = "me";
cnn.Close();
This works, I just need to retrieve the value of c_name. Any help, thanks
View 2 Replies
View Related
Jul 30, 2007
Does anyone have a good query that would return records from two tables that are found in one, but not it the other table? In my situation I have 2 tables that are duplicate tables and I need to find an additional 3000 records that were added to one of the tables. I also have a composite key so the query would have col1, col2 and col3 as the composite key. So far I have tried concatenating the 3 columns and giving the result an alias and then trying to show the ones that were not in both tables, but have been struggling. Thanks..
View 4 Replies
View Related
Jul 27, 2015
I have a CRM database that has a lot of tables and would like to be able to extract the last 'x' records in descending order from each table based on a common a field 'modifiedon' that is in every table and is auto populated by the system.
View 4 Replies
View Related
Nov 20, 2015
I have this 40,000,000 rows table... I am trying to clean this 'Contacts' table since I know there are a lot of duplicates.
At first, I wanted to get a count of how many there are.
I need to compare records where these fields are matched:
MATCHED: (email, firstname) but not MATCH: (lastname, phone, mobile).
MATCHED: (email, firstname, mobile)
But not MATCH: (lastname, phone)
MATCHED: (email, firstname, lastname)
But not MATCH: (phone, mobile)
View 9 Replies
View Related
Nov 6, 2015
I have 2 tables A, B with 1 to many relationship
Table A(ProductID), TableB(ProductID, FileID)
I need to find only the records in Table A that may have more than one FileIDs in Table B, since some ProductIDS have multiple FileIDs in Table B...
View 8 Replies
View Related
Jan 18, 2000
Is there a way to find duplicate records in a table using code. We have about 500,000 records in this table.
Thanks.
View 1 Replies
View Related
Jan 27, 2005
Hi All,
How to check for the duplicate records in the table? Thanks.
View 1 Replies
View Related
Jun 12, 2002
Yes, I know this subject has been exhausted, but I need help in locating the discussion which took place a few months ago.
Sharon relayed to the group a piece of software (expensive) which would help in my particular situation. I grabbed a demo and have gotten the approval for purchase. Unfortunately, I don't have the thread with me at work.
The problem:
Number Fname Lname Age ID
123 John Franklin 43 1
123 Jane Franklin 40 2
123 Jeff Franklin 12 3
124 Jean Simmons 39 4
125 Gary Bender 37 5
126 Fred Johnson 29 6
126 Fred Johnson 39 7
127 Gene Simmons 47 8
The idea would be to get only unique records from the Number column. I don't care about which information I grab from the other columns, but I must have those fields included.
If my resultant result set looked as follows, that would be fine. Or any other way, as long as all of the fields had information and there were only unique values in the Number field.
Number Fname Lname Age ID
123 Jeff Franklin 12 3
124 Jean Simmons 39 4
125 Gary Bender 37 5
126 Fred Johnson 39 7
127 Gene Simmons 47 8
If anyone remembers this discussion, mainly the date, I would really appreciate it.
Thanks
Gregory Taylor
MIS Director
Timeshares By Owner
View 1 Replies
View Related
Jul 13, 2007
Hi,
I have a field called user_no
i want to find out which ones are duplicates in the user_no field
the data in user_no is like this
111-222-345-666
so there are 10,000 records in the table and i want to find out the duplicate records in them
can someone tell me how my query will be
todd
View 1 Replies
View Related
Feb 17, 2008
I have two tables, one contains all work orders, the second contains records on work orders that are linked to customoer orders. I'm trying to create a query that will return specific fields from the table that contains orders in the linked order table, and only the work orders in the all order table that (work_order) do not exist in the linked order table (demand_supply_link). I have tried several queries and cannot get the results I desire. Here is the query I am currently trying.
SELECT DISTINCT WORK_ORDER.DESIRED_WANT_DATE as 'Want Date', DEMAND_SUPPLY_LINK.SUPPLY_BASE_ID as 'WO Id',
WORK_ORDER.DESIRED_QTY as 'End Qty', DEMAND_SUPPLY_LINK.SUPPLY_PART_ID as 'Part Id', CUST_ORDER_LINE.CUSTOMER_PART_ID as 'Cust Part',
OPERATION.RESOURCE_ID as Resource, PART.DESCRIPTION as Description, CUSTOMER.NAME as Name
FROM ((((DEMAND_SUPPLY_LINK INNER JOIN CUST_ORDER_LINE ON DEMAND_SUPPLY_LINK.DEMAND_BASE_ID = CUST_ORDER_LINE.CUST_ORDER_ID)
INNER JOIN WORK_ORDER ON DEMAND_SUPPLY_LINK.SUPPLY_BASE_ID = WORK_ORDER.BASE_ID)
INNER JOIN OPERATION ON WORK_ORDER.BASE_ID = OPERATION.WORKORDER_BASE_ID) INNER JOIN PART ON WORK_ORDER.PART_ID = PART.ID)
INNER JOIN (CUSTOMER INNER JOIN CUSTOMER_ORDER ON CUSTOMER.ID = CUSTOMER_ORDER.CUSTOMER_ID) ON CUST_ORDER_LINE.CUST_ORDER_ID = CUSTOMER_ORDER.ID
WHERE WORK_ORDER.DESIRED_WANT_DATE Is Not Null AND OPERATION.RESOURCE_ID in ('ASSY','FAB 1','PLAY TRK')
AND WORK_ORDER.STATUS='R'
UNION
SELECT distinct work_order.desired_want_date as 'Want Date', work_order.BASE_id as 'WO Id',
work_order.desired_qty as 'End Qty', work_order.part_id as 'Part Id', operation.resource_id as Resource,
part.description as Description
FROM WORK_ORDER INNER JOIN PART ON PART_ID=WORK_ORDER.PART_ID INNER JOIN OPERATION ON WORK_ORDER.BASE_ID=OPERATION.WORKORDER_BASE_ID
WHERE WORK_ORDER.DESIRED_WANT_DATE IS NOT NULL AND OPERATION.RESOURCE_ID IN ('ASSY','FAB 1', 'PLAY TRK')
AND WORK_ORDER.STATUS='R'
This is the error I receive:
Server: Msg 205, Level 16, State 1, Line 1
All queries in an SQL statement containing a UNION operator must have an equal number of expressions in their target lists.
The all orders table (work_order) will not have the other fields to link to as there is no customer order linked to them.
Can anyone help. Thanks!
View 14 Replies
View Related
Feb 9, 2004
Hi
Can anyone tell me how to stop a SQL query displaying duplicate records within a table
Thanks Alot
View 2 Replies
View Related
Apr 7, 2004
Can someone tell me the best procedure when trying to find duplicate records within a table(s)?
I'm new using SQL server and I have been informed that there maybe some DUPS within unknown tables. I need to find these DUPS.
If someone can tell me how to perform this procedure I would apprciate it. And if you reply can also include examples that i could follow for my records.
Thanks for the help?
-SQL Rookie
View 1 Replies
View Related
Apr 11, 2008
anybody know what sql statement can be used to pull duplicate records from an sql table.
View 5 Replies
View Related
Apr 6, 2006
Table1 has shop# and shop_id. Every shop# should have only one shop_ID. There has been a few data entry errors where a shop# has duplicate a shop_id. How to write a query for shop#s that have more than one shop_id?
View 12 Replies
View Related
Jan 6, 2008
I have a table with phone numbers.
I want to find if any phone number are repeated more then once. How can I accomplish this?
View 6 Replies
View Related
Feb 11, 2007
Hi,
Not so sure how simple this question is but here is what happened. I installed SQL Server 2005 on a new Win Server 2003. I exported the tables and their data from the old machine to the newly established database on the new machine.
It looks like all my records were duplicated. When I try to delete one of the duplicates it won't work because both rows are effected. I can't set my primary key now and if I try to create a new database with the primary key already set than the import fails.
Any one run into this before or know what's going on?
Any help ASAP would really be appreciated.
Thanks,
Alice
View 4 Replies
View Related
Apr 27, 2007
how to we check in for duplicate records without using sort (remove duplicateS)
i need to remove duplicates based on four columns.
please let me know
View 16 Replies
View Related