I ran this query to populate a field with random numbers but it keeps populating with some duplicate records. how I can remove the duplicates?
UPDATE APRFIL
SET ALTATH = CONVERT(int, RAND(CHECKSUM(NEWID())) * 10000);
Below are sample output that I need the dupes not show. The table already exist and its sql 2008
155957
155957
155968
155974
155976
15599
155990
155997
155997
156005
156008
Hello All,
We all were new at one point.... any help is appreciated.
Objective:
Combining two 49,000 row tables and remove records where there is only 1 column difference. (keeping the specified column value removing the one with a blank.)
Reason:
I have 2 people going through a list, coding a specific column with a single letter value. They both have different progress on each sheet. Hence I am trying to UNION them and have a result of their combined efforts without duplicates.
My progress/where I'm stuck:
Here is my first query/union:
SELECT * FROM [Eds table]
UNION SELECT * FROM [Vickis table];
As shown above, I have unioned these 2 tables and my results removed th obvious whole record duplicates, but since 1 column is different on these, a union without criteria considers them unique.....
an example of duplicates that I must remove are as follows:
142301 - Product 5000 - 150# - S (Keep)
142031 - Product 5000 - 150# - "" <--- Blank (Remove)
I am trying to run another query on my first query results so I don't mess my first query up. Here it is:
SELECT DISTINCT [Prod #], [Prod Name], [Prod Description], [Product Type]
FROM [Combined Tables]
WHERE [Product Type]<>" ";
Please Help! Thank you in advance.
--------------------
5 minutes away from pulling my last one!
BaldNAskewed
I need to remove the duplicates from a table. If a record exits 4 times in a
table I need to delete 3 records and retain only one occurance of that.
I need a query to do this.
Can anybody help.
-Rajesh
I have a members table and have added an extra few thoushand members to it. Now I need to remove the duplicates.
It doesnt matter which duplicate i remove as long as there are unique email addresses.
so here is the format of the table:
id
email
firstname
lastname
datebirth
if i do a:
SELECT COUNT(DISTINCT Email) AS Expr1
FROM Customer
it returns 21345
and
SELECT Count(Email)
FROM Customer
returns 28987
I can get the unique email addresses into another table by going:
SELECT DISTINCT emailaddress INTO DistinctCustomer
FROM Customer
but this will only return unique email addresses. How do i select distinct email address and all other fields into a new table? or just remove duplicates where email address appears more then once?
Thanks in advance
Tom the SQL beginner
HiI have inherited a web app with the following table structure, and need toproduce a table without any duplicates. Email seems like the best uniqueidentifier - so only one of each e-mail address should be in the table.Following http://www.sqlteam.com/item.asp?ItemID=3331 I have been able toget a duplicate count working:select Email, count(*) as UserCountfrom dbo.Membersgroup by Emailhaving count(*) > 1order by UserCount descBut the methods for create a new table without duplicates fail. My code forthe 2nd method is:sp_rename 'Members', 'temp_Members'select distinct *into Membersfrom temp_MembersTable....CREATE TABLE [dbo].[Members] ([MemberID] [int] IDENTITY (1, 1) NOT NULL ,[Username] [varchar] (10) COLLATE Latin1_General_CI_AS NOT NULL ,[Password] [varchar] (10) COLLATE Latin1_General_CI_AS NOT NULL ,[Email] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Title] [varchar] (10) COLLATE Latin1_General_CI_AS NOT NULL ,[FirstName] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Surname] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Address1] [varchar] (35) COLLATE Latin1_General_CI_AS NOT NULL ,[Address2] [varchar] (35) COLLATE Latin1_General_CI_AS NOT NULL ,[City] [varchar] (25) COLLATE Latin1_General_CI_AS NOT NULL ,[Country] [varchar] (25) COLLATE Latin1_General_CI_AS NOT NULL ,[Profession] [varchar] (50) COLLATE Latin1_General_CI_AS NOT NULL ,[Publication] [varchar] (40) COLLATE Latin1_General_CI_AS NOT NULL ,[DateAdded] [smalldatetime] NOT NULL ,[SendMail] [smallint] NOT NULL) ON [PRIMARY]GOThanks B.
View 2 Replies View Related
Using a sort transform to remove duplicates takes a very long time to execute.Is there a better way to remove duplicate information.
I have alot of duplicates within the data.
I would like to know a better way than using sort transform
Please let me know
I found a query on here to find all my duplicates (altered to fit mine)
----
select * from yourtable
where WCD_no in
( select WCD_no from yourtable
group by WCD_no having count(*) > 1 )
----
and it works great. However, when I try and delete one of these, I get the following error message.
"key column is insufficient or incorrect. Too may rows were affected by update"
Any idea on how I can delete these without having to drop the table or anything?
Hi There,
I was wondering if someone could help me with the results on this query, at the moment I am getting values repeated and I was wondering if it was possible to have some of the columns grouped, I have tried to have grouping at the end of the query but this still did not group the rows.
Thanks in advance for your answer - Sean
The structure that i'm trying to acheive is like the following:
with each colour having multiple quantitys for each size:
colourdesc| sizedesc | xs | s | m | l
-----------
black |qoh| | 0 | 2 | 0 | 7
-----------
white |qoh| | 0 | 0 | 0 | 0
-----------
!-- query results
+------------+------+------+-----------+--------+---------------+----------+
| qoh | size | shop | style | colour | colourdesc | sizedesc |
+------------+------+------+-----------+--------+---------------+----------+
| 0000000007 | 6 | 1 | 10475RIB1 | 2 | BLACK | L |
| 0000000000 | 6 | 1 | 10475RIB1 | 36 | CORRECTOR6771 | L |
| 0000000000 | 6 | 1 | 10475RIB1 | 15 | SPEARMINT6581 | L |
| 0000000000 | 6 | 1 | 10475RIB1 | 6 | WHITE | L |
| 0000000007 | 5 | 1 | 10475RIB1 | 2 | BLACK | M |
| 0000000000 | 5 | 1 | 10475RIB1 | 36 | CORRECTOR6771 | M |
| 0000000000 | 5 | 1 | 10475RIB1 | 18 | MERINGUE4016 | M |
| 0000000000 | 5 | 1 | 10475RIB1 | 6 | WHITE | M |
| 0000000002 | 4 | 1 | 10475RIB1 | 2 | BLACK | S |
+------------+------+------+-----------+--------+---------------+----------+
!--- query
select qoh
, shopsales.size, shopsales.shop, shopsales.style, shopsales.colour, co.colourdesc, sz.sizedesc
from shopsales
inner
join shops
on shopsales.shop = shops.shop
inner
join colours co
on shopsales.style = co.style
and shopsales.colour = co.colour
inner
join sizes sz
on shopsales.style = sz.style
and shopsales.size = sz.size
where shopsales.shop = 1
and shopsales.style = '10475RIB1'
The database has Name,Email, and skill. Though the name is distinct it is repeated as it has different skills. I would like to remove duplicate names and add the corresponding skill to the only one row.
From the stored procedure, combining 3 tables I got the output as:
NameemaildepartmentSkill
ArunemailidTech teamTechnical
ArunemailidTech teamLeadership
ArunemailidTech teamDecision Making
BinayemailidMarketingTechnical
BinayemailidMarketingDecision Making
I would like to remove the duplicate Name fields and combine the Skill in a single row as other fields are same.
So the output should be
NameemaildepartmentSkill
ArunemailidTech teamTechnical, Leadership, Decision Making
BinayemailidMarketingTechnical,Decision Making
I have Data like below
TableA
Col1 Col2 Col3
1 'Test One' 4/7/2007
1 'Test One-1' 4/7/2007
2 'Test' 4/7/2007
3 'Test Three' 4/7/2007
3 'Test Three-1' 4/7/2007
3 'Test Three-2' 4/7/2007
I need to delete the duplicate records[Latest record has to be remaining like that] from above table. i need the out put like below
Col1 Col2 Col3
1 'Test One-1' 4/7/2007
2 'Test' 4/7/2007
3 'Test Three-2' 4/7/2007
If any body know the solution please help me.
Regards
Hanu
Hi, I'm in the midst of an Access 2003 to SQL server 2000 upsizing project and have come across a table on Sql Server that has a field that looks like it's supposed to be the PK but it contains duplicates. What I'd like to do is to have a cursor start at the first value and increment the next value by 1. Could someone explain how I'd go about this?
Many thanks,
Peter
I have a very large table that can contain up 3 to 5 duplicate records. Every month around 100,000 new records come in. Sometimes it's an ammended record, other times is just duplicated by error.
Is it possible to keep the latest record dumped into the table and delete the others? Does SQL track the order of the data being dropped into the table?
The layout would look like this. There are 10-15 other columns in the table where adjustments can also be made.
Lease# Year Month Production
12345 2008 10 1,231
12345 2008 10 1,250
12345 2008 10 1,250
I have table like this and I need to remove the duplicates selecting the Min Date
IDDate Alloc ProdWater Tub Cas
3752011-10-10 00:00:00.00000014.70
3752011-10-11 00:00:00.00000014.70
3232014-04-24 00:00:00.000000352555.7
3232014-04-26 00:00:00.000000352555.7
3562014-04-01 00:00:00.000000318.8471.1
3562014-04-02 00:00:00.000000318.8471.1
3562014-04-03 00:00:00.000000318.8471.1
3562014-04-04 00:00:00.000000318.8471.1
3232014-05-20 00:00:00.000000649.1976.9
3232014-05-21 00:00:00.000000649.1976.9
3232014-05-22 00:00:00.000000649.1976.9
I would like my result set to be like this:
3752011-10-10 00:00:00.00000014.70
3232014-04-24 00:00:00.000000352555.7
3562014-04-01 00:00:00.000000318.8471.1
3232014-05-20 00:00:00.000000649.1976.9
I'm trying to pull records from a source/staging table where there is a duplicate row in it.I don't need that as the requirement is to garbage in /garbage out.when I do that from mart and use joins btw fact and dimensions, Im not getting this duplicate record as Im using distinct/group by. If I removed it, then it returns more than 3000 rows which is not correct. Is there a way I can keep these duplicates without removing group by...Im using correct joins and filters.
View 5 Replies View Related-- declared variables
declare @database_name varchar(100), @table_name varchar(100), @primary_key_field varchar(100)
declare @list varchar(8000)
-- set values to variables
set @list = ''
set @database_name = 'data200802_dan'
set @table_name = 'other02'
set @primary_key_field = 'callid'
use database
select @list = @list + column_name + ', '
from information_schema.columns
where table_name = @table_name --table name
and column_name != @primary_key_field --unique identifier
select @list = substring(@list, 1, len(rtrim(@list)) - 1)
--above 5 lines btw came from a helper in the msdn forum. thanks
SELECT DISTINCT @list
INTO '#' + @table_name
FROM @table_name
@table_name + ':'
IF (SELECT COUNT(*) FROM @database_name + '.dbo.' + @table_name) = 0
BEGIN
INSERT INTO
@database_name + '.dbo.' + @table_name + '(' + @list + ')'
SELECT
@list
FROM
'#' + @table_name
END
ELSE
BEGIN
DELETE @database_name + '.dbo.' + @table_name +' ( ' + @list + ')'
GOTO @table_name
END
DROP TABLE '#' + @table_name
the query above is basically.. selecting all the fields from a table in database W/OUT their primary key. then putting them in a temp table.. delete all the records in the original table. then paste the records from the temp table into the original table.
is there a way for this to work? i don't know how to use the variables w/ this script. please help me correcting this query..
this is for removing duplicates btw. thanks!
I'm looking for a bit of SQL code that will select some entries randomly from an SQL table.
For instance I'd like to feed a parameter in that contains the number
20, and the returned record contains 20 randomly and distinct selected
records.
Anyone know how this can be done? (never came across randomly select records) Appreciate any help
I need to randomly pick one or more records from a query e.g
select c_id, c_name
from c_table
where cat_id = 52
There may be more than one records for cat_id = 52. and I need to pick 3 of them randomly.
Thanks in advance!
I have a query which finds duplicate spec_items linked to a work order. What I want to do it remove the duplicates (and in some cases there will be more than one) leaving only the record with the highest [sr.id]
select sr.id, sr.linked_to_worknumber, sr.spec_checklist_id from spec_checklist_remind sr inner join spec_checklist_remind sc on sc.linked_to_worknumber = sr.linked_to_worknumber
group by sr.id,sr.linked_to_worknumber, sr.spec_checklist_id
Having sr.spec_checklist_id = 30 and count(*)>1
order by sr.linked_to_worknumber
Sample of query results:
idlinked_to_worknumberspec_checklist_id
18323IP-DN-03377430
24229IP-DN-03377430
18294IP-DN-03380830
28027IP-DN-03380830
18295IP-DN-03381030
19924IP-DN-03381030
In the 3 cases, I would want to retain only records with ids 24229,28027 and 19924.
I have an existing stored table with duplicate rows that I want to delete.Using a cte gives me
WITH CTE AS
(
SELECT rn = ROW_NUMBER()
OVER(
PARTITION BY employeeid, dateofincident, typeid, description
ORDER BY Id ASC), *
FROM dbo.TableName
)
DELETE FROM cte
WHERE rn > 1
This is what I want to do basically. But this is only deleting in my CTE, is there anyway I can update my existing table "TableName" with this, without using temp tables?
Is there a query or a way to convert duplicates value in a column to non duplicates.
View 14 Replies View RelatedHi
i have created a quiz software in VB. i have used ini files to fetch questions.. now i have thought of changing it to SQL.. i have created a table with questionno, question, option1,option2,option3,option4 and the correctanswer.. now i want to select the questions randomly each time the question session is started.. please help me out as i need to complete this project for my school...
Hai friends,,
I have a table name "Student"
it contain 2 fields no ,name
no name
1 Raja
2 Larsen
3 Ravi
4 Ankit
5 Eban
my questions is I have a webform random.aspx
whenever any user open a webform random.aspx
it should display anyone of name in a random order..... CODING:-
Dim cn As New System.Data.sqlclient.SqlConnectionDim rd As sqlDataReader
cn.ConnectionString = "Persist Security Info=False;User ID=sa;Initial Catalog=master;password=david;"
cn.Open()Dim cmd1 As New SqlCommand("select no,name from Student", cn) ' how i can chage to random order....
rd = cmd1.ExecuteReader
rd.Read()
Response.Write(rd(0) & "." & rd(1))
rd.Close()
thank u..
Ambrose...
I have a web app that calculates tax filing status and then stores data about the person.
Facts
The insert is done through a stored procedure.
All the sites that this program is being used are connecting through a VPN so this is not an external site.
The duplicate records are coming from multiple sites (I am capturing there IP address).
I am getting a duplicate about 3 or 4 times a day out of maybe 300 record inserts.
Any help would be greatly appreciated.
There are many sqlcmdInsert.Parameters("@item").Value =
cnTaxInTake.Open()
sqlcmdInsert.ExecuteNonQuery()
cnTaxInTake.Close()
And that is it.
create table #sample
(
Name varchar(100),
value int
[code]....
From that I wanted to delete some records based on following condition. randomly select any number of records but sum(value) = 125 and name = xxx
I need to group up the records randomly into ‘n’ number of batches. That can be done by NTILE, but I want group up similar records in single group.
Say for example, following is the list of records I have in my table which I want to group into 5 batches
A123
A124
A124
A123
A127
After Ntile I will get the below,
Desired output is, Need output like Ntile but all same id should reside in single batch
Even if I n=5, maximum possibility of batches are 3 only.
I'm trying to Update a column table based on values from another table but I need these values are random.My query looks like this and doesn´t work
DECLARE @Rowini int,
DECLARE @lastrow int
SET @Rowini = 1
SET @Lastrow = 80000
[code]...
This one has set me back many, many hours on this project; it's about got me ready to dump SSIS & just roll a custom .NET solution in C#.
I need to create import packages for quite a few very wide flat files (130 - 180+ columns, not my design). Many of these columns have data > 50 characters long.
I change column widths on the data flow source using the Advanced Editor, via Input and Output Properties ==> Flat File Source Output ==> External Columns.
About 50% of the time, the changes vanish after clicking OK to dismiss the Advanced Editor. There is no warning message or output announcing that the editor failed to persist its changes, or that it set some columns' properties back to the defaults, or why. The column's widths just silently revert back to 50.
If the cause and resolution aren't known, does anyone know of a way to accomplish any of the following workarounds?
Create a data source connection by importing an external text schema defining the flat file's column names, data types and sizes, or
Change a property on multiple source columns en-masse, or...
Get at a text version of the file containing the Data Flow Component's definition, so we can edit wide import schemas without racking up thousands of mouse-clicks?Thanks! A virtual cheeseburger to anyone with answers.
I'm having a problem using the Flat File Source while using the underlying .Net classes to execute SSIS Packages. The issue is that for some reason when I load a flat file it Empty's out columns randomly. Its happening in the Flat File Source Task. By random I mean that most of the times all the data gets loaded but sometimes it doesnt and it empty's out column data. Interestingly enough this is random and even the emptying out of columns isnt a complete empty, its more like a 90% emtpying. Now you'll ask that is the file different everytime and the answer is NO. Its the same file everytime. If I run the same file everytime for 10 times it would empty out various columns maybe 1 of those times. This doesnt seem to be a problem while working with dtexec or the Package Executor utility. Need Help!!
View 9 Replies View RelatedUsing the SqlDataReader, I am retrieveing records from a SQL Server 2005 DB table. Since I am using the SqlDataReader to retrieve the records from the DB, I have to use the Read method of the SqlDataReader like this:Dim sqlReader As SqlDataReaderWhile(sqlReader.Read) Response.Write(sqlReader.GetValue(0) & "<br>") Response.Write(sqlReader.GetValue(1) & "<br>") Response.Write(sqlReader.GetValue(2) & "<br>")End WhileThe records retrieved can only be accessed inside the While loop. I want to access the records outside the While loop as well. Is there anyway by which I can do this, maybe by populating the recordset in an array variable & then using it outside the While loop?
View 3 Replies View RelatedHI all I have this table below and i need to get records from col1 >1 and col2 >1
col1 col2 language
1 a s
1 b p
3 c e
3 c f
SELECT col1, col2, language COUNT(*) AS Expr1
FROM table1
GROUP BY col1, col2
HAVING (COUNT(*) > 1)
with the above query it gives me this
col1 col2 expr1
3 c 2
the language column is not displayed what i want is not the exact count but all the records which is duplicate and must resemble the result below
col1 col2 expr1 language
3 c 2 e
3 c 2 f
I need the result above inorder to normalize the table and bring out the language part into another table
thanks
Hi,
Need to remove the duplicated rows from a table which has text/ntext/image type columns. The table does not have any PK/Unique column. (I accept its a bad data model). But currently changing the data model is not possible. Hence doing changes in application.
I couldn't do 'SELECT DISTINCT * from table', since the table has text columns. Though there is no PK constraint, If I know that col1 and col2 are join PKs in the table, Is that possible to select the distinct rows from such a table.
Please advise,
Thanks,
MiraJ
Dear All
This is query i have written is giving following output.
FormCodeRefCodeSerialnoDateTime
R1-196H1-68A12232138/6/2007 19:38:11
R1-196H1-68A12232138/6/2007 19:38:14
R1-205H1-67XS23124148/6/2007 19:36:08
R1-205H1-67XS23124148/6/2007 19:36:10
R1-220H1-66F433365348/6/2007 19:30:27
R1-220H1-66F433365348/6/2007 19:30:29
R1-400H1-64ER53436648/6/2007 19:24:23
R1-400H1-64ER53436648/6/2007 19:24:26
R1-408H1-65TE4626268/6/2007 19:24:23
R1-408H1-65TE4626268/6/2007 19:25:00
I want the output like this,it should take only Min Datecreated
FormCodeRefCodeSerialnoDateTime
R1-196H1-68A12232138/6/2007 19:38:14
R1-205H1-67XS23124148/6/2007 19:36:08
R1-220H1-66F433365348/6/2007 19:30:27
R1-400H1-64ER53436648/6/2007 19:24:23
R1-408H1-65TE4626268/6/2007 19:24:23
I have a table that holds the following
1 7530568 87143 OESCHD 1/5/2006 6:31:58 AM
1 7530568 87143 OESCHD 1/5/2006 7:02:36 AM
for each 7530568 ordernumber there should only be one OESCHD status.
This is the query I'm using to insert the data sent to me.
INSERT INTO ORDER_EVENTS
SELECT d.division as division,
dt.orderNum as orderNum,
dt.poNum as poNum,
dt.statusCode as statusCode,
dt.statusChangeDate as statusChangeDate
FROM dt_Order_Events dt INNER JOIN
division d ON dt.division = d.divisionShort INNER JOIN
status s ON s.division = d.division AND s.statCode = dt.statusCode
WHERE directive <> 'C' AND
dt.orderNum IN (SELECT orderNum FROM ORDER_HEADER)
This works fine when used with in the hourly transactional update. But When I ran it for the Bulk UpDate (so we'd have historical data) it allowed orders to have statuses to many times.
I am not a SQL guru, I have no idea how to write a sql statement or stored proc that will remove the duplicate records. or how to change what I have to prevent further ones.
Any help would be apreciated.