How To Handle A Large Data Cleaning Task?
Jan 11, 2006
I need to create a program that will regularly import a text file of over one million records into an Access table. I've been give a list of about fifty different updates to perform on the data to clean it.
I can't imagine performing all these updates in one query. However, creating fifty individual queries seems horribly inefficient from a processing perspective.
I'm accustomed to stepping sequentially through a table in FoxPro, which seems ideal to me for this type of situation. What is the best way to handle this in Access?
View Replies
ADVERTISEMENT
Nov 16, 2007
Hi
I have a table that contains telephone numbers.
The data is in a variety of formats.
Here are some examples:
087 123456
087-123456
(W)087-123456
(H)123456
I would like to strip out hyphens, brackets, spaces, etc.
I hope to do this in a straighforward make table query.
Is there a function available that will strip out non-numerics characters?
Thanks
Noel
View 1 Replies
View Related
Jul 21, 2007
I'm attempting to normalize an enormous table with order data, but I'm running into some problems. The table currently contains many duplicates, of which also included the actual order information (yikes!), but I managed to normalize it almost all the way down. It appears that different accounts can be used on orders, and these order numbers are being recycled for some reason months down the line (don't ask my why they're reusing them for future orders because I have no idea either, they should be creating new order numbers). Of course, the Order number is the primary key in my table as it should be. I guess the same thing can occur with the sales rep. Anyway, I'm struggling to find the "best practice way" to deal with this situation. I'm almost tempted to create an intermediary "transaction table" or something like that between the main general order information (which at this point will basically be the Order Number and Customer ID only), then include a table with the account information and sales rep info, then have that link to the Order Detail with the products, quantity, order number and various dates for those order numbers. Order maybe it should be a separate, related table, but not between the general order information and the order details? Can anyone tell me if I'm on the right track for this situation? It was a total curveball that the rep and account information could be different on these orders.
Option 1:
Order (Order #, Customer) -> Transaction Information (Order #, Account Type, Sales Person) -> Order Details (dates, products, quantities, etc)
Option 2:
Order (Order #, Customer)---> Transaction Information (Order #, Account
| Type, Sales Person)
|
|-> Order Details (dates, products, quantities, etc)
View 2 Replies
View Related
Oct 26, 2005
I have a question and I'll apologize in advance because it may or may not really be an Access question.
I have a large ASCII spreadsheet that I have imported into Access. It has 4 columns, and has these headings: CodeID, Group, Division, Account. Many Division and Account combinations can exist within a Group, and there can be many Groups under a specific CodeID. Unfortunately the way I received the data was in a grouped fashion, like so:
CodeID Group Division Account
1234 1111 0001 0001
0002 0002
2222 0001 0001
0003 0003
1235 1111 0001 0001
0003 0003
etc.
Is there a way - easy or otherwise - to populate the blanks with the appropriate information? Essentially to "ungroup" I guess. I need to do this so I can sort and modify the data. Obviously as it is I can't do much with it.
Note, when I imported the data I had Access add unique counter as a primary key. That is allowing me to keep the data in order, but doesn't help beyond that.
Any suggestions???
View 7 Replies
View Related
Mar 31, 2008
Hi guys,
I have a query that returns a large data set (~100k rows/month).
I am currently inefficiently exporting week-by-week to Excel to carry out some manipulation (mainly a set of "if" statements to calculate differences between records that have the same site id).
Is there a way to write a function in Access to carry out the calculations? I'm not too familiar with Access VBA apart from coding functions for forms etc.
View 12 Replies
View Related
Nov 27, 2006
Hey all
I have a table that contains a list of news items. For each item there is a link to an appropriate image and a load of text.
I display a list menu of news items showing the picture and the first few lines of text. At the moment for the first few lines of text I have a separate field and just copy and paste the first few lines of the main article into it.
My question is is there a way where instead of having to have a separate field with just the first few lines in I can somehow just retrieve the first few lines of data from the full article field when displaying the menu listing?
Hope this makes sense, and any ideas greatly appreciated.
Many thanks
View 2 Replies
View Related
Aug 18, 2003
I ran into errors recently when I tried running a large UNION query (about 6,900 news articles) in a desktop Access database, that will eventually be migrated to SQL Server. I've got two tables, each having the same typed fields. I joined then through the following query:
SELECT ID,Name,Body FROM Table1 UNION SELECT ID,Name,Body FROM Table2;
The problem was that in the resultant table containing the conjoined records, one of the fields (Body, a MEMO field) copies only the first 250 characters or so, truncating the rest of the data.
I was thinking this might have something to do with telling Access how to type the data when copying it over. Is there a way to explicitly tell a query the data type of each field to be used so that the data can be copied over properly?
Thanks!
View 3 Replies
View Related
Dec 7, 2007
Morning all,
I'm having a problem with mdb file size. I'm importing a large amount of data from a number of tab delimited text files via a simple transfertext function. The process goes: empty the tables in the database, then import the data into the tables.
All this works fine, but the file size rockets to over 1.5Gb. When I then compact and repair, it goes down to 420Mb. I'm not deleting and recreating the tables, and at no point is there 1.5Gb worth of real data, so what's causing this?
N.B. I realise I can call compact and repair following the import, but this is going to take too long as they are user-initiated imports.
View 4 Replies
View Related
Mar 27, 2013
Best way to display "notes: field that gets updated/added to multiple times, in a form. (Access 2007)
I've got a DB to track information about product changes. Each month we have a meeting and discuss problem products. I'm concerned with a notes/comments section. Currently it is a memo field in the source table that can be added to, so the notes are cumulative. But we want to auto-add a date to each new entry (in case the person inputting the data forgets, and so that notes can be sorted by order on a report).
There may be many comments on a certain product or there may be none. Sometimes the note may be very long, other notes on the same product may be very short. I created a new table with a one-to-many relationship to case ID (to link it to the main form), auto-populate a date field, and field for notes in order to link multiple notes to each case (and of course an autonumber PK).
I can put the subform in the original form, but then the note field will be the same size for every note, though notes are not all uniform in size/length, and it ends up taking up a LOT of room on the main form. So I created a button to click to open the subform, but it brings up all records for all cases, not just the currently selected case. Again, have the problem of one uniform field size for all notes, regardless of size/length.
Then I considered a report to view past history (field size can shrink or grow as needed) then a button on the report that opens a form for data entry/entry of new note. But again the report displays all records for all cases, not just the currently selected case.
View 2 Replies
View Related
Jun 16, 2015
I have 250 separate worksheets with a lot of data to put into Access. Problem is the data is 120,000 rows in each worksheet and a lot of duplicate date eg..DATE, NAME, TIME,are some of the column headings and there are multiple rows with same DATE or NAME. That is just how I received the data. I would like to transfer all records into Access as quickly and efficiently as possible.
View 4 Replies
View Related
Sep 10, 2013
I have created a rather large data entry form for one of our departments which will be used to run a mail merge document. The merge is run off of a query of the form, rather than the tables themselves...I think that is what I've read is the right thing to do?
Anyways, most of it is working great so far but I've run into an issue where a few of the form fields are combo boxes. In access and in the query the data looks correct, but when you look at it in word (in edit recipient list) and after the merge, it will have a file path name instead.
So a combo box that has "Medical Plans and Visual Plans" in that field in the query, comes over saying "c:Users
ameAppData
Ex: New Hires are provided information about Medical Plans and Visual Plans.
Ex: New Hires are provided information about MeC:UsersbrooksAppData
It always shows the first two letters before putting the filepath name in there.
I have tried doing it multiple ways...a lookup to another table, lookup right in the field itself....get the same results.
View 3 Replies
View Related
Jun 9, 2013
My DVD library database I have a memo field which has the description of the movies.
In some cases, like my tv show records, I have a lot of data because of the episodes and episode descriptions.
I'm currently displaying this data using a split form with the datasheet on the left
I must use a small font in the memo field in order to display all data.
I could use a larger font and simply scroll but I'm looking for an alternative.
I've even thought of creating a hyperlink filed and point to the movie on IMDB's web site. But when I click the link nothing happens even though I copied and pasted the link into the field.
View 8 Replies
View Related
May 5, 2006
Ok, I am working on an exsisting database at work that has alot of code/forms/reports and so on. Whoever worked on this before me apparently didn't believe indenting code or anything like that. Now I don't want to to go all this code to clean up the structure of the code. I remember back a few yeras ago I found an app for VB that did this for you. No clue where I came across it at or even what it was called but I was wondering if there was anything available like that for Access?
View 3 Replies
View Related
Sep 5, 2014
In given table name is formatted as:
HERNANDEZ JR, FELIX / BRADY, ANDEL
and I need to make it to "Felix Hernandez" How would I be able to clean up "/" , "JR," in my query.
View 8 Replies
View Related
Feb 21, 2014
In a table, there's a column, "Siebel or Sales Ref". Some of these will have a long sales reference number or an ID from a Siebel sales system (pants), in the format 1-XXXXXXX. The problem is that some have two IDs, separated by a forward slash (/), so like 1-ABC1234/1-DEF5678. He wants to go through the data, and where there's a double ID, split it out, and create a new row with the second ID.
I'm not sure this can be done in Access, I would have done a loop in Excel, but it's his database.
Code:
INSERT INTO ActualBaseData
SELECT *
FROM (SELECT 'ITQ ID', Deadline, Lot, 'Bid Progress', 'Framework Type', 'Tender Type', 'Siebel or Sales Ref'
FROM TempBaseData
WHERE InStr('Siebel or Sales Ref', '/') = 0
[Code] .....
View 6 Replies
View Related
Oct 27, 2014
I want to release an Access file which has eveything locked down and cleaned up so that my Form is showing full screen (and nothing else) for an optimal user experience.
View 2 Replies
View Related
Dec 28, 2007
PROBLEM: String Manipulation
"Cleaning up Mom's Christmas address labels"
I need guidance on the best string manipulation functions (Instr, Left, Right) to cleanup my mother's Christmas address list of 300+ names.
I have successfully imported the text file into Excel and exported to Access; fieldnames: FULLNAME, ADDRESS, CITYSTATEZIP
I have found instructions on how to breakout FULLNAME field into FIRSTNAME and LASTNAME.
But within the FULLNAME field are many combinations of titles (Mr., Mr. & Mrs., Dr., HON.) with inconsistent periods applied.
Which one of string manipulation functions:
Instr
Left
Right
would be best for extracting these various titles from this name field?
I understand the concepts behind the above functions, but not enough experience using them to understand the tedious syntax or which string manipulation function would be best for extracting the varying title entries to a separate created field called TITLES.
So far, I have deduced this will be a multi-step process. But asking for guidance:
1.) Which string function is best suited for this?
2.) Example of the function syntax for an update query?
2.) Suggested order to administer update queries?...
to extract misc titles from the FULLNAME field.
I am a novice-casual Access user.
Thanks, Greg
(If someone would copyright these steps into a book called "Cleaning Up Mom's Christmas Address List"... I am sure they could retire from sales on Amazon. :-)
View 14 Replies
View Related
Jun 12, 2012
I want to create a report for the Monthly cleaning plan of a hotel. For each day, how many rooms need new sheets, how many need new towels etc.
At this point I can generate a report for any given day.
This could be an example of what I want to achieve
Code:
------------ Date | Date+1 | Date+2 | Date+3
New Sheets 2 1 0 2
New Towels 1 3 0 1
Full Clean 0 1 2 0
"Date" is a date tat you can set, after which you'll get the following 30 days("Date+1","Date+2" etc)
I thought that a CrossTab query would give me what I want,but using the wizzard I can't get the result that I want.
Haven't worked with crosstab queries before so maybe I'm doing something wrong, or maybe this isn't even possible with a crosstab query.
View 4 Replies
View Related
Jan 1, 2014
So I have two tables,
table1: (company_name, company_code, year_month, rating)
table2: (company_name, company_code, year_month, asset, debt, equity,...)
What I would like to do is to call up all the data in table1, and then call up (asset, debt, equity,...) from table2 where the company_code and year-month are equal. What do you think is the best way to do this? Keep in mind a few things:
1. I'm dealing with hundreds of thousands of data lines
2. I may need to get more data similar to table2 in the future and call it up in the same way
I thought I should create a primary key "company_code-year_month"for both tables, but that method doesn't seem like a good one, and if I were to have another table, will have to create the same primary key for that table as well?
View 3 Replies
View Related
Jul 11, 2005
Hi, here's the scenario...
I'm building an ASP based email application which is to send emails to different user groups. The email addresses in the database need to be able to be assigned to more than one user-group, and an administrator needs to be able to add user-groups to the database.
So effectively, there may be hundreds of user groups and hundreds of users.
What is the best way to store this info in an Access database? Do I let the administrator create a new table column in the userEmails table whenever there is a new user-group added, or do I store a series of userGroupIds in a text field along with each email address and delimit them so that I can split them into an array?
I'm really not sure how to go about this, so any help would be much appreciated.
Thanks...
View 2 Replies
View Related
Jul 10, 2006
Hello,
What is the best way to handle notes on a form?
The DB is for recording Quotes and Orders taken over the phone.
Is it best to have a seperate table w/ a NotesType field so that all notes are stored in a central location? Or is it better to store notes for each quote/order in the Orders / Quotes table itself?
Also, if data is entered into these fields using subforms, is it possible to have the notes displayed outside of the subform in a box format, to allow plenty of typing room?
If anyone has a good example of a smart notes system implementaion, it would be very appreciated.
Thanks!
View 3 Replies
View Related
May 25, 2005
I have Four tables, CustomerTBL(holds all the customers for both companies); NoteTbl; AgeingTBL1 (this is for one company) ;AgeingTBL2 (this is for a child company)
I need to keep the 2 ageings seperate.
I want to creat a form from the customer table (so you can make edits to contact info and such)
Then I would like a subform with the ageingtbl1 and the note table to be linked together, but I only want the records from the customer table and note table to come back that match the ageing for that perticular customer.
I think I made this more confussing then it needs to be
View 2 Replies
View Related
Aug 13, 2007
I have a parent form (called PR_CR_Form) that has a subform called SubFrm_DefectClassification. I am trying to log all changes to the Access database. I am using the VBA code from the Microsoft website for article 197592. It works great. However, it only allows me to capture the changes to the main form. The VBA code has the following statement:
Set MyForm = Screen.ActiveForm
This makes MyForm point to the parent form (PR_CR_Form). What I would like to do is to make MyForm point to the subform (SubFrm_DefectClassification). However, I can't seem to do this. I tried the following:
Set MyForm = Forms!PR_CR_Form!SubFrm_DefectClassification
but this didn't work.
I would very much appreciate it if someone could please tell me how to do this.
Thank you!
-Al Oberneder
View 2 Replies
View Related
Dec 3, 2006
Hi guys,
I've noticed a few posts on the subject of images but there's lots of different scenarios and I'm hoping someone with far more knowledge can give me some basic pointers for my specific case so I can research it more fully.
Basically part of my database is storing properties, then there's a separate table for property photos as each property can have between 0 and infinity photos (theoretically, the max. is likely to be between 5 or 10).
From my understanding so far it will be best to store a path to each picture as a text field in the database as we're looking at 100+ photos easily from the start which I understand would cause a huge database using OLE linking.
I've got a couple of articles on how to take the path and display it as a picture on the form, not tried it yet but had a read through and it all seems fine.
What I need to ask is the best way to do a couple of things...
When viewing a property via a form is it easy-ish/possible to display thumbnails and captions for all the associated images? Could it then be set up so you could click each image to view it full size?
I need a slightly more user friendly way of adding the photos than just typing in file paths. What I imagine the cleanest way would be blank boxes which you'd click, get a Browse... window to pick the image then once you'd picked one it would place a thumbnail in the box, or something similar. However I'm not sure how well that would scale to a non-specific number of images (ie 0 - 10).
I have no fixed specifications for displaying/adding these images so if you have other ideas/ways of doing it then please suggest - the above is just what I imagine would be 'nice' - if it involves way too much coding then it's not worth it at this stage.
Any help / pointers would be most appreciated :) Let me know if you need more info.
Chris :)
View 1 Replies
View Related
Nov 5, 2007
I'm responsible for an Access 2000 database which three people have open more or less constantly and another four open from time to time.
The database is on a P2P network and is split, with the back end in a single location. The network is wireless.
At the moment there are no problems with this setup but changes are being contemplated which will mean that eight people will have the database open constantly with another four using it occasionally.
For other reasons ( There were only three other wireless networks within range when we started, now there are eight), we are switching the network from wireless to cable. The back end will be transferred to a new PC with a powerful processor and loads of memory.
Does anyone have experience of allowing that many users to log on at the same time in a P2P network? And was the experience good or bad?
View 6 Replies
View Related
Dec 10, 2007
Any suggestions on how to handle delimiters when creating an SQL query string?There's a query form with a number of text boxes. User can enter text in one or more boxes. The VBA code checks the text boxes and generates the query string from what's been entered.eg. strSQLString = "SELECT * FROM [myTable] WHERE [Surname] = ' " & txtSurname & " ' "If the user enters a name such as O'Neil this will result in an invalid SQLString because of mis-matched single quotes."SELECT * FROM [myTable] WHERE [Surname] = ' O ' Neil ' "How can I handle input text which may contain delimiters?
View 3 Replies
View Related
