Removing Duplicates In The Table
Mar 21, 2014
The database has Name,Email, and skill. Though the name is distinct it is repeated as it has different skills. I would like to remove duplicate names and add the corresponding skill to the only one row.
From the stored procedure, combining 3 tables I got the output as:
NameemaildepartmentSkill
ArunemailidTech teamTechnical
ArunemailidTech teamLeadership
ArunemailidTech teamDecision Making
BinayemailidMarketingTechnical
BinayemailidMarketingDecision Making
I would like to remove the duplicate Name fields and combine the Skill in a single row as other fields are same.
So the output should be
NameemaildepartmentSkill
ArunemailidTech teamTechnical, Leadership, Decision Making
BinayemailidMarketingTechnical,Decision Making
View 2 Replies
ADVERTISEMENT
May 18, 2012
I have a very large table that can contain up 3 to 5 duplicate records. Every month around 100,000 new records come in. Sometimes it's an ammended record, other times is just duplicated by error.
Is it possible to keep the latest record dumped into the table and delete the others? Does SQL track the order of the data being dropped into the table?
The layout would look like this. There are 10-15 other columns in the table where adjustments can also be made.
Lease# Year Month Production
12345 2008 10 1,231
12345 2008 10 1,250
12345 2008 10 1,250
View 2 Replies
View Related
Aug 2, 2000
I need to remove the duplicates from a table. If a record exits 4 times in a
table I need to delete 3 records and retain only one occurance of that.
I need a query to do this.
Can anybody help.
-Rajesh
View 3 Replies
View Related
Jan 27, 2005
I have a members table and have added an extra few thoushand members to it. Now I need to remove the duplicates.
It doesnt matter which duplicate i remove as long as there are unique email addresses.
so here is the format of the table:
id
email
firstname
lastname
datebirth
if i do a:
SELECT COUNT(DISTINCT Email) AS Expr1
FROM Customer
it returns 21345
and
SELECT Count(Email)
FROM Customer
returns 28987
I can get the unique email addresses into another table by going:
SELECT DISTINCT emailaddress INTO DistinctCustomer
FROM Customer
but this will only return unique email addresses. How do i select distinct email address and all other fields into a new table? or just remove duplicates where email address appears more then once?
Thanks in advance
Tom the SQL beginner
View 5 Replies
View Related
Jul 20, 2005
HiI have inherited a web app with the following table structure, and need toproduce a table without any duplicates. Email seems like the best uniqueidentifier - so only one of each e-mail address should be in the table.Following http://www.sqlteam.com/item.asp?ItemID=3331 I have been able toget a duplicate count working:select Email, count(*) as UserCountfrom dbo.Membersgroup by Emailhaving count(*) > 1order by UserCount descBut the methods for create a new table without duplicates fail. My code forthe 2nd method is:sp_rename 'Members', 'temp_Members'select distinct *into Membersfrom temp_MembersTable....CREATE TABLE [dbo].[Members] ([MemberID] [int] IDENTITY (1, 1) NOT NULL ,[Username] [varchar] (10) COLLATE Latin1_General_CI_AS NOT NULL ,[Password] [varchar] (10) COLLATE Latin1_General_CI_AS NOT NULL ,[Email] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Title] [varchar] (10) COLLATE Latin1_General_CI_AS NOT NULL ,[FirstName] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Surname] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Address1] [varchar] (35) COLLATE Latin1_General_CI_AS NOT NULL ,[Address2] [varchar] (35) COLLATE Latin1_General_CI_AS NOT NULL ,[City] [varchar] (25) COLLATE Latin1_General_CI_AS NOT NULL ,[Country] [varchar] (25) COLLATE Latin1_General_CI_AS NOT NULL ,[Profession] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Publication] [varchar] (40) COLLATE Latin1_General_CI_AS NOT NULL ,[DateAdded] [smalldatetime] NOT NULL ,[SendMail] [smallint] NOT NULL) ON [PRIMARY]GOThanks B.
View 2 Replies
View Related
May 19, 2008
Using a sort transform to remove duplicates takes a very long time to execute.Is there a better way to remove duplicate information.
I have alot of duplicates within the data.
I would like to know a better way than using sort transform
Please let me know
View 2 Replies
View Related
Jan 12, 2006
I found a query on here to find all my duplicates (altered to fit mine)
----
select * from yourtable
where WCD_no in
( select WCD_no from yourtable
group by WCD_no having count(*) > 1 )
----
and it works great. However, when I try and delete one of these, I get the following error message.
"key column is insufficient or incorrect. Too may rows were affected by update"
Any idea on how I can delete these without having to drop the table or anything?
View 2 Replies
View Related
Sep 27, 2004
Hi There,
I was wondering if someone could help me with the results on this query, at the moment I am getting values repeated and I was wondering if it was possible to have some of the columns grouped, I have tried to have grouping at the end of the query but this still did not group the rows.
Thanks in advance for your answer - Sean
The structure that i'm trying to acheive is like the following:
with each colour having multiple quantitys for each size:
colourdesc| sizedesc | xs | s | m | l
-----------
black |qoh| | 0 | 2 | 0 | 7
-----------
white |qoh| | 0 | 0 | 0 | 0
-----------
!-- query results
+------------+------+------+-----------+--------+---------------+----------+
| qoh | size | shop | style | colour | colourdesc | sizedesc |
+------------+------+------+-----------+--------+---------------+----------+
| 0000000007 | 6 | 1 | 10475RIB1 | 2 | BLACK | L |
| 0000000000 | 6 | 1 | 10475RIB1 | 36 | CORRECTOR6771 | L |
| 0000000000 | 6 | 1 | 10475RIB1 | 15 | SPEARMINT6581 | L |
| 0000000000 | 6 | 1 | 10475RIB1 | 6 | WHITE | L |
| 0000000007 | 5 | 1 | 10475RIB1 | 2 | BLACK | M |
| 0000000000 | 5 | 1 | 10475RIB1 | 36 | CORRECTOR6771 | M |
| 0000000000 | 5 | 1 | 10475RIB1 | 18 | MERINGUE4016 | M |
| 0000000000 | 5 | 1 | 10475RIB1 | 6 | WHITE | M |
| 0000000002 | 4 | 1 | 10475RIB1 | 2 | BLACK | S |
+------------+------+------+-----------+--------+---------------+----------+
!--- query
select qoh
, shopsales.size, shopsales.shop, shopsales.style, shopsales.colour, co.colourdesc, sz.sizedesc
from shopsales
inner
join shops
on shopsales.shop = shops.shop
inner
join colours co
on shopsales.style = co.style
and shopsales.colour = co.colour
inner
join sizes sz
on shopsales.style = sz.style
and shopsales.size = sz.size
where shopsales.shop = 1
and shopsales.style = '10475RIB1'
View 1 Replies
View Related
Jul 5, 2007
I have Data like below
TableA
Col1 Col2 Col3
1 'Test One' 4/7/2007
1 'Test One-1' 4/7/2007
2 'Test' 4/7/2007
3 'Test Three' 4/7/2007
3 'Test Three-1' 4/7/2007
3 'Test Three-2' 4/7/2007
I need to delete the duplicate records[Latest record has to be remaining like that] from above table. i need the out put like below
Col1 Col2 Col3
1 'Test One-1' 4/7/2007
2 'Test' 4/7/2007
3 'Test Three-2' 4/7/2007
If any body know the solution please help me.
Regards
Hanu
View 12 Replies
View Related
Jan 3, 2008
Hi, I'm in the midst of an Access 2003 to SQL server 2000 upsizing project and have come across a table on Sql Server that has a field that looks like it's supposed to be the PK but it contains duplicates. What I'd like to do is to have a cursor start at the first value and increment the next value by 1. Could someone explain how I'd go about this?
Many thanks,
Peter
View 6 Replies
View Related
Jun 4, 2014
I have table like this and I need to remove the duplicates selecting the Min Date
IDDate Alloc ProdWater Tub Cas
3752011-10-10 00:00:00.00000014.70
3752011-10-11 00:00:00.00000014.70
3232014-04-24 00:00:00.000000352555.7
3232014-04-26 00:00:00.000000352555.7
3562014-04-01 00:00:00.000000318.8471.1
3562014-04-02 00:00:00.000000318.8471.1
3562014-04-03 00:00:00.000000318.8471.1
3562014-04-04 00:00:00.000000318.8471.1
3232014-05-20 00:00:00.000000649.1976.9
3232014-05-21 00:00:00.000000649.1976.9
3232014-05-22 00:00:00.000000649.1976.9
I would like my result set to be like this:
3752011-10-10 00:00:00.00000014.70
3232014-04-24 00:00:00.000000352555.7
3562014-04-01 00:00:00.000000318.8471.1
3232014-05-20 00:00:00.000000649.1976.9
View 2 Replies
View Related
May 6, 2015
I'm trying to pull records from a source/staging table where there is a duplicate row in it.I don't need that as the requirement is to garbage in /garbage out.when I do that from mart and use joins btw fact and dimensions, Im not getting this duplicate record as Im using distinct/group by. If I removed it, then it returns more than 3000 rows which is not correct. Is there a way I can keep these duplicates without removing group by...Im using correct joins and filters.
View 5 Replies
View Related
Mar 17, 2008
-- declared variables
declare @database_name varchar(100), @table_name varchar(100), @primary_key_field varchar(100)
declare @list varchar(8000)
-- set values to variables
set @list = ''
set @database_name = 'data200802_dan'
set @table_name = 'other02'
set @primary_key_field = 'callid'
use database
select @list = @list + column_name + ', '
from information_schema.columns
where table_name = @table_name --table name
and column_name != @primary_key_field --unique identifier
select @list = substring(@list, 1, len(rtrim(@list)) - 1)
--above 5 lines btw came from a helper in the msdn forum. thanks
SELECT DISTINCT @list
INTO '#' + @table_name
FROM @table_name
@table_name + ':'
IF (SELECT COUNT(*) FROM @database_name + '.dbo.' + @table_name) = 0
BEGIN
INSERT INTO
@database_name + '.dbo.' + @table_name + '(' + @list + ')'
SELECT
@list
FROM
'#' + @table_name
END
ELSE
BEGIN
DELETE @database_name + '.dbo.' + @table_name +' ( ' + @list + ')'
GOTO @table_name
END
DROP TABLE '#' + @table_name
the query above is basically.. selecting all the fields from a table in database W/OUT their primary key. then putting them in a temp table.. delete all the records in the original table. then paste the records from the temp table into the original table.
is there a way for this to work? i don't know how to use the variables w/ this script. please help me correcting this query..
this is for removing duplicates btw. thanks!
View 3 Replies
View Related
Dec 20, 2013
I have a query which finds duplicate spec_items linked to a work order. What I want to do it remove the duplicates (and in some cases there will be more than one) leaving only the record with the highest [sr.id]
select sr.id, sr.linked_to_worknumber, sr.spec_checklist_id from spec_checklist_remind sr inner join spec_checklist_remind sc on sc.linked_to_worknumber = sr.linked_to_worknumber
group by sr.id,sr.linked_to_worknumber, sr.spec_checklist_id
Having sr.spec_checklist_id = 30 and count(*)>1
order by sr.linked_to_worknumber
Sample of query results:
idlinked_to_worknumberspec_checklist_id
18323IP-DN-03377430
24229IP-DN-03377430
18294IP-DN-03380830
28027IP-DN-03380830
18295IP-DN-03381030
19924IP-DN-03381030
In the 3 cases, I would want to retain only records with ids 24229,28027 and 19924.
View 9 Replies
View Related
Jun 2, 2015
I have an existing stored table with duplicate rows that I want to delete.Using a cte gives me
WITH CTE AS
(
SELECT rn = ROW_NUMBER()
OVER(
PARTITION BY employeeid, dateofincident, typeid, description
ORDER BY Id ASC), *
FROM dbo.TableName
)
DELETE FROM cte
WHERE rn > 1
This is what I want to do basically. But this is only deleting in my CTE, is there anyway I can update my existing table "TableName" with this, without using temp tables?
View 4 Replies
View Related
Jan 26, 2015
I ran this query to populate a field with random numbers but it keeps populating with some duplicate records. how I can remove the duplicates?
UPDATE APRFIL
SET ALTATH = CONVERT(int, RAND(CHECKSUM(NEWID())) * 10000);
Below are sample output that I need the dupes not show. The table already exist and its sql 2008
155957
155957
155968
155974
155976
15599
155990
155997
155997
156005
156008
View 2 Replies
View Related
Nov 2, 2015
INSERT
INTO [Table2Distinct]Â Â Â Â Â Â Â Â
([CLAIM_NUMBER]Â Â Â Â Â Â Â Â
,[ACCIDENT_DATE]
[code]....
I used the above query and it still inserts all the duplicate records. What is wrong with my statement?
View 5 Replies
View Related
Oct 22, 2014
I have a table with 22 million Business records. I can see that there are duplicates when I group by BusinessName and Address and Phone. I'd like to place only the duplicates into a table, with a ranking, oldest business key gets a ranking of 1.
As a bonus I'd like each group to have a distinct group name (although not necessary, just want to know how to do this)
Later after I run more verifications to make sure these are not referenced elsewhere I'll delete everything with a matchRank > 1 out of the main Business table.
DROP TABLE [dbo].[TestBusiness];
GO
CREATE TABLE [dbo].[TestBusiness](
[Business_pk] INT IDENTITY(1,1) NOT NULL,
[BusinessName] VARCHAR (200) NOT NULL,
[Address] VARCHAR(MAX) NOT NULL,
[code]....
View 9 Replies
View Related
Apr 15, 2008
Hi,
I have table which stores the fund name and its data. We get quarterly information from the fund co. Suppose if the user wants to add a fund thats not in our database we let then add a ClientFundId and a FundName. But may be after sometime the fund company may add that fund in the next quarter.. So how do i get rid of Duplicated Data..
In the ClientFundId column we can a 9 letter Aplhanumeric or a 5 letter character but if the fund co.. provides those values the 5 letter characters are stored in Ticker column and the 9 letter words are stored in Cusip column.. So i just wrote this query hoping i could retrieve the duplicate values but it didnt list any..but i found one this is my query..
Select
FundId,
Cusip,
Ticker,
ClientFundId,
FundName,
ShortName
From Fund
Where
ClientFundId = Ticker
or
ClientFundId = Cusip
Any help will appreciated
Thanks
Karen
View 18 Replies
View Related
Mar 24, 2004
hi, what is a good way to kill the duplicates from a table. when i say killing duplicated, i mean killing all the rows for the repeated row.
WorkTempID ItemNo Seq
100196 RTP-22 1
100197 RTP-22 2
100198 RTP-22 3
100199 RTP-22 3
100200 RTP-22 4
100201 RTP-22 4
100202 RTP-22 5
100203 RTP-22 5
********************************************************
see how Seq 3, 4 and 5 are repeated? so for the output i want.
WorkTempID ItemNoSeq
100196RTP-221
100197RTP-222
********************************************************
i DO NOT want this as the output. i already know how to achive this using DISTINCT keyword
WorkTempID ItemNoSeq
100196RTP-221
100197RTP-222
100198RTP-223
100200RTP-224
100203RTP-225
View 12 Replies
View Related
Dec 11, 1999
does someone have a querry to display the duplicate records in a table.
Table:
zipcode dma
My data upload is failing because there is a primary key on zipcode and the source data (42k records) has about 50 duplicate zipcode records in it. It is possible that there is a unique combo of zipcode / dma but I need to identify the duplicate records to determine that.
View 1 Replies
View Related
May 28, 2008
I have this script bellow which does what it is supposed to. However it only outputs the cust_id. I want it to show all the columns in the table. How would I do this?
SELECT cust_id
FROM cust_table
WHERE cust_name in ('Billy','John') and rownum < 100
GROUP BY cust_id
HAVING COUNT(*) > 1;
View 6 Replies
View Related
Aug 6, 2014
how i can check for duplicate entries for example if a serial number has already been inputted and a user tries to input the same serial number.. how can i get a trigger or some sort to check for duplicates and then prompt that the number has already been entered.
View 7 Replies
View Related
Aug 7, 2014
Im trying to look for duplicates in a table field.. field name is alphanumericCol and table is a user defined table...This is my trigger:
ALTER TRIGGER [dbo].[DUPLICATES]
ON [dbo].[AMGR_User_Fields_Tbl]
FOR INSERT, UPDATE
AS
DECLARE @Alphanumericcol VARCHAR (750)
-- This trigger has been created to check that duplicate rows are not inserted into table.
-- Check if row exists
SELECT @Alphanumericcol
FROM Inserted i, AMGR_User_Fields_Tbl t
WHERE t.AlphaNumericCol = i.AlphaNumericCol
AND t.Client_Id = i.Client_Id
-- (@Alphanumericcol = 1)
-- Display Error and then Rollback transaction
BEGIN
RAISERROR ('This row already exists in the table', 16, 1)
ROLLBACK TRANSACTION
END
The result i get is, if i input a duplicate number it fills in a null in the field so my question is how do i get it to tell me its duplicate and let me insert a new one
View 3 Replies
View Related
Jan 29, 2008
Hi All
I have the dbo.OperatingHour It has many duplicates and I want to remove duplicates permanently
The statement below works but when I open the table there are no changes
Insert into OperatingHour(Weekdays, Wednesdays, Fridays,Saturdays, [Sundays/Public Holidays])
(SELECT DISTINCT Weekdays, Wednesdays, Fridays,Saturdays, [Sundays/Public Holidays] FROM OperatingHour)
View 2 Replies
View Related
Mar 11, 2006
I need help flagging duplicate records in ome tables I have.For example if I have Table1 which conatins Field1, Field2 and Field3like belowField1 Field2 Field3 Field4Paul 18 Null NullPaul 18 Null NullJohn 19 Null NullHow would I;1. put a 'Y' in Field3 to mark the two records which are duplicates.2. put a 'Y' in Field4 to mark ONLY ONE of the duplicate records.Regards,Ciarán
View 2 Replies
View Related
Jul 20, 2005
There is a table with a single column with 75 rows - 50 unique / 25duplicates. How would pull back a list of the rows that have/areduplicates?This is a question that I got in an interview. I didn't get it,obviously....Thanks,Tim
View 1 Replies
View Related
Aug 13, 2013
Table Design --> Componenet (table name)
ID.
Country ID
BT-ID
Component
Activation
I need to delete the duplicate values from Componenet table
ID CountryID BT-ID Componenet Activation
A3CD GD58 TR77 RX 1
BER2 GD58 TR77 RX 1
XEW7 GD58 TR77 MX 1
O4T4 GD58 TR77 MX 1
PE78 GD58 TR77 GX 1
Expected Output
ID CountryID BT-ID Componenet Activation
A3CD GD58 TR77 RX 1
XEW7 GD58 TR77 MX 1
PE78 GD58 TR77 GX 1
View 14 Replies
View Related
Dec 3, 2006
If we want to remove the duplicate row and leave only one row instead of 2 or 3 rows for example with the same column values.
2/ The same question but when all the columns of the row are duplicate except the id field.
Thanks a lot.
View 3 Replies
View Related
Jun 15, 2007
How do I only select rows with duplicate dates for each person (id)? (The actual table has approximately 13000 rows with approximately 3000 unique ids)
p_key id date pulse
--------------------------------------
1 32 5/25/2006 80
2 32 5/25/2006 85
3 32 4/26/2006 81
4 32 6/15/2006 82
5 15 1/20/2006 75
6 15 3/25/2006 80
7 15 3/25/2006 83
Result table I am looking for:
p_key id date pulse
------------------------------------
1 32 5/25/2006 80
2 32 5/25/2006 85
6 15 3/25/2006 80
7 15 3/25/2006 83
Thanks.
View 8 Replies
View Related
Jul 23, 2005
Hiya everyone,I have two tables in SQL 2000. I would like to append the contents ofTableA to TableB.Table A has around 1.1 Million Records.Table B has around 1 Million Reocords.Basically TableA has all of the data held in TableB plus 100,000additional records. I would only like to import or append these newadditional records. I have a unique index already setup on Table B.Any ideas pretty pretty please?Paul.Ps. (Have been messing around with DTS but get a unique violation error- Which is kinda what I want I guess, but would like SQL to ignore theerror and only copy the new data - if only)
View 9 Replies
View Related
May 15, 2015
I have a table with call data (ContactID, Queues Entered, Call Status, Date & Time Stamps etc). Each entry relating to a contact ID goes onto a new row. The first row for a contact is the date and time it is created. It then captures the queue (or queues) it enters before it is answered. Finally, it captures when the call is released (Completed).
I'm trying to link all this data into one single row per contact ID to make it easier to report on.
I started off by using DISTINCT to pull back all of the Contact ID's. I then used a Left Join to pull back the date and time of creation. I created a further Left Join to pull back the first queue that it entered and so on.
When I did this, I started getting duplicates. This is because some calls enter more than one queue.
How can I do this so that it only has one ContactID per row. Also, for the Queue, is there anything I can do to ensure it pulls back the first Queue it enters? (These are time stamped). Subsequently, I would then need to add the second and third queue it enters in other columns. (A call can enter a maximum of 3 queues).
View 8 Replies
View Related
Aug 4, 2006
Hi,
What sql would I use to remove all the nulls in a table?
thanks!
View 11 Replies
View Related
