Queries :: SELECT DISTINCT Not Removing Duplicate Records In Staging Table
Jan 29, 2015
I have a database with an import process which normalises incoming data and appends to various tables. No issues with that. I also have a function within that process which counts the number of new entries for a summary popup when the process has completed.This works by querying the staging table, prior to the append, into a recordset and using the .RecordCount to increment the count (multiple files can be imported at once so this effectively provides a running count, per file, to give a total for the whole import)
I thought it was working fine but this morning I noticed that the count which appeared on the popup was 1 greater than the number of actual new records. I checked the source files and noticed that, for whatever reason, there was a duplicate entry in there. So I presume that's why the count was out by 1.
There's no integrity issue in the main tables as the composite primary keys ensure that duplication shouldn't be a problem. Indeed, the record in question, duplicated in the source, appears only once in the main table post-import. So not too worried about that.
However, I need the count in the popup to be accurate (it tells the users how many new entries require further investugation). And what's puzzling me is that I use DISTINCT in the query, which I would have thought should eliminate any potential dupes in the recordset and thus provide the correct count. It seems it doesn't?
Code:
Public lngNewBalancesTBI As Long ' Defined in a separate module...
-------
Dim dbs As Database
Dim rst As Recordset
Dim strSQL As String
[code]....
Why the dupe, which is still present in the staging table, also makes it over to the recordset, even though I'm using DISTINCT?
I have a table with duplicate records (which is ok) and I want to return distinct data for each client.
It works fine when there is only two fields returned however, when I add a third field to the query it no longer returns only the Distinct records - I am getting Duplicates returned.
I.E
SELECT DISTINCT tblClient.ClientNo, tblClient.Name FROM tblClient
Works fine with only the Distinct records for each client returned
However
SELECT DISTINCT tblClient.ClientNo, tblClient.Name, tblClient.Address, tblClient.OrderValue FROM tblClient
Now returns Duplicates!
Is there a limit to the number of fields to be returned using DISTINCT or what else could be the problem? Should I be doing this some other way?
I'm having a problem removing duplicate records from a table. It should be easy but I can't suss it.This is an example of the data in my table:
Code:
RefDateStatus 113007111/06/2015Do Not Pay 114454306/07/2015Do Not Pay 115760714/05/2015Do Not Pay 116520705/05/2015Do Not Pay 117670108/05/2015Do Not Pay 118036218/05/2015Do Not Pay 118517015/05/2015Do Not Pay 178734020/07/2015Do Not Pay 182809915/07/2015Do Not Pay 184226010/07/2015Payment Due 184226022/07/2015Payment Query
As you can see, there are 11 records here but the last two records have the same reference number. I need my query to show the first 9 records + the record from the bottom two with the latest date (22/07/2015).
Im trying to write a query that shows all the container movements. Yet when I run the query qryFullHistory I get a duplicate value for container Off Island. Ive tried adding some criteria that says that the DateRequested has to be between the ImportDate and ExportDate but that doesnt seem to work. There are duplicate entries for container Off Island in tblContainerDetails as the same container has arrived and left and then returned on another voyage. Yet there is no entry for the second voyage in the tblMEMRContainer.
A brief description of the tables is: tblMEMR Movement requests details tblMEMRContainers the containers that were moved on the movement request. There can be more than 1 container for each request. tblContainerDetails details and dates for the container when it arrived and when it left
There are other tables but these are the 3 that are used in the query.
I have a table with at least 13.000.000 records. There are many duplicates records... For example
ID Name Family mobile car number chassis Register_Year 1 Roy Jalbout 9999 123456/G ASF4546 2005 2 Roy Jalbout 9999 854658/G GRK554JFJD 2009 3 Tony Elishah 1234 854658/G GRK554JFJD 2012 4 Sam Markos 5478 854658/G GRK554JFJD 2014 5 Roy Jalbout 9999 123456/G ASF4546 2005
As you can see ID Number 2,3,4 have the same car but every one bought from another so it's not duplicates
The duplicates here is the ID's number 1,5. So how to remove the duplicates. I remember you i have at least 13.000.000 records. I try to make a query to find duplicates then i make a copy of the original table and than i should make a primary key then append the data from the original table to the copy table, but here i have more than one criteria
When I made a query to find duplicates the result was 680.000 records and every one have a minimum 2 duplicates an maximum 4 duplicates so it's about 2.500.000 duplicates records at least....
Ok well I thought I was doing this right but it appears I have not. I am drawing data from two tables. Both of them related by the person's name. One table I am pulling the unique ID number, the other table contains the name. I have a PO (Purchase Order) field being pulled from the table that holds unique ID. Or lack there of, so I am pulling all the records that contain no PO number. However duplicate unique ID's are being pulled. The way my query is set up now is:
Unique ID Number Name Count (Number of Procedures) Charges PO Number - Criteria: "Is Null"
I have also right click in the table view area, in the properties I selected Unique values. I have to assume that this would have held back any Dupes from the output. However this is not the case. When I looked in the help files the only thing I could find was:
http://office.microsoft.com/en-us/a...0840151033.aspx Quote: "If you want to see only unique values in a result set, you can specify that you want to exclude duplicates from the result set.
In the Database window, click Queries under Objects, click the query you want to open, and then click Design on the database window toolbar. Right-click the background of the Diagram pane, then choose Properties from the shortcut menu. Select DISTINCT values. The Query Designer inserts the keyword DISTINCT in front of the list of display columns in the SQL statement.
Note If you use the DISTINCT keyword in Microsoft SQL Server, you cannot modify the data in datasheet view."
The problem with this is my properties box does not offer a Distinct Values option. So my question is. Is there an expression I can place in the Unique ID field that will exclude any duplicate records.
Hi, Wish if some one could help me ASAP. I have a table which contains name, tel, email i need to import only records which have distinct email. I do need need to import data of all three fields but only which has distinct email. As there are number of record which are duplicate. They have different names but same email. So i need to condition only for distinct email but dump the data in a new table with all three records. so same names can have different email. but same email can't have duplicate email. So need only records which have distinct email. Please help .......
I currentlyt have a search form that searches criteria based on a textbox. After entering data a macro runs, and opens up a query with the results. After that query opens i have macro that runs a duplicate query off of the query initally opened based on the search results. What i would like to have is a macro that removes the duplicate records from the duplicate query and display the remaining records.
tbl_A has column "ID" with duplicates. tbl_B has column "ID" with distinct values.i want to find all the distinct IDs in tbl_A that are present in tbl_B.I've tried the following with no luck:
SELECT DISTINCT tbl_A.ID FROM tbl_A LEFT JOIN tbl_B ON tbl_A.ID = tbl_B.ID WHERE tbl_A.ID IN tbl_B.ID;
Is it possible to change this query to search for either a Name or by an employee ID number? The query runs a report on after update. Currently I can only enter the name not the employee ID
Code: SELECT DISTINCTROW First(Inventory.Employee) AS FirstOfEmployee, Inventory.[Employee ID] FROM Inventory GROUP BY Inventory.[Employee ID] ORDER BY First(Inventory.Employee);
I have the following SQL which returns rows of distinct numbers that are calculated from a field.
SELECT Distinct (Left([ProjectID],4)) AS NumberOfCalls FROM tblProject;
ProjectID looks something like this : 1307-IND-001 and NumberOfCalls looks like : 1307
I need to count the number of unique "NumberOfCalls" entries that there are in a list of about 50 rows. Currently the SQL returns a list of numbers like this:
1307 1311 1401 etc...
I just want NumberOfCalls to show "3". Is there an easy way to do this?
I am trying to write a query that will select distinct values from three columns. The table name is Hoods. The columns are C1, C2, and C3. All three columns contain colors and can have the same values, but I only need to have a particular color selected once. For instance, all three columns contain "Black", but "Black" should only be listed once. The result of the query will be used in a drop down list.
The following works for one column:
SELECT DISTINCT [Hoods].[C1] FROM Hoods WHERE C1<>"";
I am currently trying to make a form which will generate a report for the users. Currently the form has 3 combo boxes where users select which fields they want displayed from the table in the report. They also have sorting options next to them (Asc or Desc). I am now trying to put in a multi select list box under each combo box to allow users to filter the report to their liking. Currently I am trying to pull distinct values from the specific field of the table and display them in the listbox for users to select.
I have tried using vba to set me.filter1.rowsource = SELECT DISTINCT me.combo1 FROM EVAP_Database, but this doesnt seem to work. I have also tried a few other codes and still no luck.
I currently use the following to import a csv file into a staging table and then append and save the data to a table - however it doesnt use the headings from the csv file and creates an error table and a blank record where the headings should be apart from one column where it copies the name.
Would it be possible to use the headings as field names or just omit them completely and use the standard f1 f2 f3 etc access generates in the tmp table?
Code: With Application.FileDialog(msoFileDialogFilePicker) .Title = "Select the CSV file to import" .AllowMultiSelect = False .Filters.Clear .Filters.Add "CSV Files", "*.csv", 1 .Filters.Add "All Files", "*.*", 2
While I am not new to Access, I am not well versed in its abilities as far as combo boxes go. What I have is a Form where a combo box allows you to pick from a table records 'record ID #' in order to fill in the data of that record to the rest of the form.What I want to do is use a query to select specific records from this table and allow the combo box to show only these 'selected Record IDs' for user selection.
Most employees have 10 to 15 rows. I only want to pull ALL rows for employees ONLY if there is a distinct count from DATE_PAID greater than 1. There are cases where an employee might have two pay checks generarted for the same PAY_DATE.
Using a database 2010. I have a monthly import from a spreadsheet to one of my tables. Sometimes it has the same previous data from prior month or months that has not changed. I don't want to keep appending this data to my table if none of the fields have changed.
Is there a way to append data to table only if fields have changed in an append query? Ex. (Name, Address, PhoneNum, Rate) If all fields are the same; don't append. If one or more fields are changed; append.
After some research I thought I had found a neat way to SELECT records from a table based on an 'IN' clause and sort them in the same order as the values for the 'IN' clause... i.e.
Code: SELECT Unique_No, Table_Name, List_Order FROM My_Table WHERE Table_Name = 'Titles' AND List_Order IN (3,1,15,4,5,12,7,2) ORDER BY INSTR('3,1,15,4,5,12,7,2', List_Order)
Unfortunately, this returns list_order 5 just after 15 and list_order 2 just after 12, thus
I'm creating a job/timesheet database and have a problem...
I have a main timesheets table that includes every job/piece of work thats been done (sometimes multiples for each employee each day). Each record contains a ref to the employee, ref to the contract, ref to the type of work, time taken and date.
There are then various tables that support the main table, which provide actual employee names etc.
I want to produce a query that provides the number of hours undertaken on each day of a particular week for each employee. So I'd have one row for each employee from the employees table, then a field for the sum hours for Monday through Friday.
The way I was going about this was
- to create five query's for the main timesheets table that would limit the entries to the five days in question - Monday-Friday. That worked fine.
- to then create a query that takes the employee name and using each of the five day based queries perform a Sum Total on the time field to give me the total hours worked for each day for each employee. That also worked fine, but the trouble is it removes any null values. So, if I only include the first day there might be 80 employees listed, but when I include the second day as well that goes down to 72 - presumably as 8 employees who entered timesheets for day 1 didn't for day 2. By the time I add all five days, I have almost no employees. I assumed that there would be a query level property to set, but i can't see one.
Also, is there a simpler way to do what I want? If I can get this working I'd like to replicate to breakdown by contract, job type etc in the same way as I have with employees.
I am using Access 2010 on a Windows 7 laptop. I need a query to provide a list of ID's that have more than one occurrence of IDandDate combined but haven't been successful getting past syntax errors.
Using this table structure as an example: ID - defined as text field Date - defined as date/time field TestValue
This legacy table contains a record for each test. The table should be unique based on the ID and Date combination but was never restricted to that rule. I am converting to a new table but need to identify the duplicate entries so they can be addressed by business folks.
I have a table that has mutliple records for the same person. I know how to create a query to remove duplicates but i don't know how to create a table to remove duplicates but keep the most recent record. Is this possible?
ie client status date 123 A 1/1/07 123 C 2/5/06 123 A 9/3/07
