Hello there. I am Completely new to SQL and this forum, and this problem that I have may appear to be very basic to you guys but still... I was wondering if I could get some help with a database I am trying to make in MS Access.
I have used the Access TransferText function to import data from a text file into a table with an ID attached to each line, eg.
ID Text
1 Hello world
2 This is an example
3 Of my database
I want to merge the data, or copy it into a field in a new table to get:
ID Text
1 Hello World
This is an example
Of my database
2 [more imported text from a different table]
and i have been advised that SQL is the best way to do this. Is it possible to have line breaks in a field within microsoft access, or would it have to be structured as
ID Text
1 Hello World This is an Example Of My Database
2 ...
I am trying to create a dimension table and I am pulling in data from two tables to create it. I need all records from table A, any records from table B that are not in table A, and I need to use the fields from B for those records that do match. What would be the best way to approach this, merge join + derived columns, union all + aggrigation? Any suggestions?
It seems like it's harder to do this in ssis rather then just doing it in the database.
We have a data warehouse staging database in which we capture change history for hundreds of tables from a source system. In the source system, records are updated in place, but in our data warehouse we capture these changes by "terminating" the existing record and adding a new record reflecting the changes. In the data warehouse we add two columns to every table -- effective_date and expiration_date -- which indicate the dates the record was in effect in the source system. By convention, an expiration_date of 6/6/2079 means the record is currently still active in the source system. Each day we simply compare yesterday's version of the record (in the data warehouse) against today's version (in the source system). If differences are found in any of the columns, we terminate the record and add a new one, setting those dates appropriately.
In this example, the employee_id column is the natural key in the source system. We add the effective_date and expiration_date in the data warehouse, so those three columns together make up the key in the data warehouse. The employee_name, employee_dept, and last_login_date columns all come from the source system as well.
In the select output, you can follow the trail of changes for each of these three employees. Bob moved from dept 7 to 8 at some point; Frank didn't change departments at all; Cheryl moved from dept 6 to 9 and later back to 6. However, the last_login_date was updated frequently for all these employees.
We've tracked hundreds of tables this way for years, some with hundreds of columns. For optimization purposes, I'm now interested in trimming the fat a bit. That is, we track changes in many columns that we don't really need in our data warehouse. Some of these columns are rapidly-changing, causing all sorts of unnecessary terminate/inserts in the data warehouse. My goal is to remove these columns, reclaim the disk space and increase the ETL speed. So in this example, let's get rid of the last_login_date column.
alter table mytbl drop column last_login_date select * from mytbl order by employee_id, effective_date
Now in the select output, you can see we have many "effective duplicate" records. For example, nothing changed for Bob between 1/1/2014 and 1/31/2014 -- those really should be one record, not three. Here's the challenge: I'm looking for an efficient way to merge these "effective duplicates" together, through set-based sql updates/deletes/inserts (hoping to avoid any RBAR operations). Here's what the table ultimately should look like (cheating to get there):
Note that Bob only has two records (he changed department), Frank only has one record (no changes), and Cheryl has three records (two department changes).
My inclination would be to drop the unwanted columns, then GROUP BY all the remaining columns from the source system, and taking the MIN effective_date and MAX expiration_date. However, this doesn't work for cases like Cheryl's -- she moved to another department, then back again, so that change history needs to be retained.
As I mentioned, we have hundreds of tables, and I'd like to strip out dozens (maybe hundreds) of unused columns, so ultimately there will be millions of these pseudo-duplicates that need to be merged together. These are huge tables, so I really need to find an efficient set-based approach to this.
I'm trying to avoid a large amount of manual data manipulation.
Here's the background: Legacy system that has (well let's call apples apples) pretty much no method of enforcing data integrity, which has caused a fairly decent amount of garbage data to be inserted in some tables. Pulling one of the [Individuals] table from within this Legacy system and inserting it into a production system, into the Table schema currently in place to track [Individuals] in this Production system.
Problem: Inserting the information is easy, how to deduplicate the records that exist within the staging table that the legacy [Individuals] table has been dumped into in production, prior to insertion. (Wanting to do this programmatically with SQL or SSIS preferably, so that I can alter it later to allow for updating existing/inserting new)
Staging Table Schema:
; CREATE TABLE [dbo].[stage_Individuals]( [SysID] [int] NULL, --Unique, though it's not an index intended to identify the [Individuals] [JJISID] [nvarchar](10) NULL, [NameLast] [nvarchar](30) NULL, [NameFirst] [nvarchar](30) NULL, [NameMiddle] [nvarchar](30) NULL,
[code]....
Scenario: There are records that duplicate the JJISID, though this value is supposed to be unique for every individual. The SYSID is just a Clustered Index (I'm assuming) within the Legacy system and will be most likely dropped when inserted into the Production [Inviduals] table. There are records that are missing their JJISID, though this isn't supposed to happen either, but have valid information within SSN/DOB/Name/etc that can be merged into the correct record that has a JJISID assigned. There is really no data conformity, some records have NULLS for everything except JJISID, or some records will have all the [Individuals] information excluding the JJISID.
Currently I am running the following SQL just to get a list of the records that have a duplicate JJISID (I have other's that partition by Name/DOB/etc and will adapt whatever I come up with to be used for those as well):
; select j.* from (select ROW_NUMBER() OVER (PARTITION BY JJISID ORDER BY JJISID) as RowNum, stage_Individuals.*, COUNT(*) OVER (partition by jjisid) as cnt from stage_Individuals) as j where cnt > 1 and j.JJISID is not nullNow, with SQL Server 2012 or later I could use LAG and LEAD w/ the RowNum value to do my data manipulation...but that won't work because we are on SQL Server 2008 in this environment.
[URL]
With, the following as a potential solution:
GSquared (3/16/2010)Here's a query that seems to do what you need. Try it, let me know if it works.
Performance on it will be a problem, but I can't fine tune that. You'll need to look at various method for getting this kind of data from the table and work out which variation will be best for your data. Without access to the actual table, I can't do that.
; WITH CTE AS (SELECT master_id, MIN(ID) AS first_id, MAX(Account_Expiry) AS latest_expiry FROM #People GROUP BY master_id) SELECT P1.master_id,
[code].....
Unfortunately, I don't think that will accomplish what I'm looking for - I have some records that are duplicated 6 times, and I'm wanting to keep the values within these that aren't NULL.
Basically what I'm looking for, is to update any column with a NULL value to the corresponding Duplicate [Individuals] record value for that column.
**EDIT - Example, Record 1 has a JJISID with NULL NameFirst & NameLast BUT Record 2 has the same JJISID and values for NameFirst & NameLast. I'm wanting to propogate the NameFirst & NameLast from Record2 into Record1
I have a database full of different types of leads some for company A some for company B and so on, each doing a different service. However the leads from B can be used for A and leads from A can be used for B, so I want to merge the data.
Example:
Phone Number Name Home Owner Credit Insurance 727-555-1234 Dave Thomas Yes B 727-555-1234 Dave Thomas Gieco
I would like the end result to be one record:
Phone Number Name Home Owner Credit Insurance 727-555-1234 Dave Thomas Yes B Gieco
Since these were imported into SQL they all have a unique ID, here are the current labels
writing the query for the following, I need to collapse the continuity. If the termdate for an ID is one day less than the effdate of the next id (for the same ID) i need to collapse the records. See below example .....how should i write the query which will give me the desired output. i.e., get min(effdate) and max(termdate) if termdate is one day less than the effdate of next record.
hello im working on a project with a friend and store source files in a subversion server. since we are both working on the same DB everytime one of us makes a change we discover a problem in merging our changes.
is there any tool that can help in merging and making diff?? is there any alternativa?
I have two SQL tables with the following structureInstructor tableINIDINNameINEmailInstructor PhotoIPIDINIDIPPhotoNow my new Instructor table got a new fieldINIDINNameINEmailINPhotoQuestion, how can I merge the Instructor Photo table IPPHoto field into Instructor table INPhoto field? Thank you.
I wonder about the possibility of merging two identical databases on two different servers upon recovering from connection failure between them, using triggers. In order to create a simple synchronization
I have a website wih about 50000 pageviews permonth. I am using multiple access database for each section. for example for photogallery there is a separate database, for jokes there is another one. Now I am thinking to convert my all access databases into MSSQL Server2005 databases.There are about 5 access databases I want to merge them and convert them into 1 MDF file.How to do this?I am very new to SQL Server. Please Help I update access databases in MS ACCESS and upload them in server.When I will use SQL Server Databases How will I update them? From VS2008? Any other method? Or Should I think about creating WebBased control panel for my website like CP present in every CMS(joomla, dotnetnuke etc)? ....................................................................................................................... I have VS2008 and SQL Server 2005 Express.
I have two tables each with a date field. I have to combine these two tables and sort on the date. I want the date from each table to be on the same column though, not in seperate columns.
I have two sql server databases...one holds nothing but personalisation information. The other holds various other types of data. I would like to merge these two DBs into one (including all stored procedures,etc.), but being a relative noob to SQL Server, I can't seem to figure out exactly how to accomplish this. Neither database holds anything that the other holds; data-wise, stored procedure-wise or any other wise. ;-) Can anyone point me in the right direction?
We've got an incident database on our server and we want our consultants could add, modify, delete incidents on our customers sites with their laptop. When they come back we can also synchronise our incidents database with their modifications.
I defined the database on our server ( sql7 sp3) as distibutor and publisher, the other computers coulb be only suscribers.
It would be very easy except for a case :
when a line is modified on our server database and the same line in the same table is modified on the suscriber computer, the changes on publisher (distiributor) seems to have priority on the suscribers. But we would like to control this : we want the suscribers, when they come back, if this case appears, could choose good merging or not if it's possible, and if it is not, we would like at less they could have a message saying they will lost that they change (incident by incident if it's possible)
So what is possible to resolve it by the best way ?
Help!!, Does anyone know of a way that I can merge three seperate fields into one field in my sql statement. EG. SELECT catnumber, catnumber2,catnumber3 FROM Categories WHERE Customer = 'xxxx' and itemnumber = 1 or 2 or 4
I want the catnumber, catnumber2, catnumber3 all in one field so I can sort by it. If anybody could help me it would be greatly appreciated. Thanks Mike
I need to figure out what is the easiest way to do this is. I have two tables that have the following columns:
Code:
TABLE 1 AcctNbr, GiftRef, Gdate, PayCode, Amt
Table 2 GiftRef, Amt, MotvCode, FundId
I need to merge the two tables with GiftRef being the common table between them. I have exported the tables from MSSQL to Access and am wondering what the easiest way to do this is. Is there an SQL Query that can merge the two tables on export by GiftRef? Is it easier to do this in Access now that I've exported the two Tables? Any help on this would be awesome. Thanks.
My client drops for me many files like this on a shared drive M: daily 1_Test.csv 2_Test.csv 3_Test.csv
I would like you to advice on how to write a SQL code (that can include DTS if possible) that will take this files, merge them into one (since they have same columns) and send them to another client as one file. But it must only take the files of the same date and must not resend files i have send already. I need this to be an automated process since the files are dumped into M:drive midnite and I need this code so that I can schedule it as a job and run around 4h00am.
I'm pretty new to SQL altogether. I kinda gave myself a crash-course when the software I was using, Papertrack, was having problems. It all started when I tried to move the database. Now there are two separate databases that I can see on SQL Enterprise manager. I picked up the book and figured some things out, but still don't know how to merge the databases. I called the support for the software and got to know them on a middle-name basis. Is there a way I can merge the two databases so that the software can pull queries from both and access the files? To anyone that likes a challenge,
I have 2 tables that have the exact same structure and have some common values. T1 = a,b,c,d,e T2 = e,f,g,h
I want to create a single distinct table which captures all unique in both tables. I have a common fields ID in both, I can match on. I don't want duplicates which is what's happening now..
I have a function that returns a range of numbers and a query that returns a varchar field and an int field. I would like to replace the int field in the query with unique ranged numbers from the function(one per row).
Any ideas?
Thanks, Mark
-- use ifms
select * from dbo.fnRange(5101,5152)
select --distinct s.OrgKeyId, 10420 TableNumber, 5100 DetailCodeNumber, s.NewStratalabel CodeDescription, s.NewStratalabel CodeAbbreviation, 1 CodeActiveFlag, Getdate(), 0 distinct s.NewStratalabel, 5100 from _StrataLabel_xref s Left Outer Join(select * from org_Detail_Code where tablenumber = 10420 and orgkeyid in (Select distinct OrgkeyId from _StrataLabel_xref)) o on o.OrgKeyId = s.OrgKeyId and o.CodeDescription = s.NewStratalabel where o.Detailcodenumber is null order by s.NewStratalabel
Hi, We have we had a database in 2006 with around 90 tables and later we created a new database in 2007 with more tables for same application. Now we want to have a database which will contain all records from the two above database. how should we go about this??
I have a query that I'm working on, but instead of giving the query, I wanted to ask a basic syntax question. If more info is needed, let me know. If you have 2 rows that have a common relationship, but differing information in some fields, can you merge them all onto one row? I've done this with Sum(case) expressions, but I don't want to 'add' anything. In the following example, the ActivityID refers to a break. ActivityID can be:
0=Pick up 1=Drop Off 2=Lunch 3=Break
So if I wanted to see 2 breaks on 1 row in the following example, would this be possible:
I have a question. I want to perform the following task:
1) UPDATE wdoctyp SET ldoclock=0 WHERE docType='LEAVE' 2) SELECT idoclastsn FROM wdoctyp WHERE docType='LEAVE' 3) UPDATE wdoctyp SET ldoclock=1, idoclastsn=idoclastsn+1 WHERE docType='LEAVE'
can I perform it in one SQL statement? Actually the reason I want to do this is because I want to prevent ldoclock from locking forever due to computer failure.
**what i wanna do is to get the following result :
<transport> <train> 10 </train> <plane>10 </plane> <metro>10</metro> <bm>10</bm> <voiture><avion> 10 </avion><train>10</train><mercedesC230> 10 </mercedesC230></voiture> <nissan> <Z>10 </Z> </nissan> </transport> --------------------------------------------------------------------------------------------------------------------------------------------------------------- this means that i want to merge all the child nodes that belong to the same parent node. thanks for any help
I Have 3 columns in source (ID1,ID2,ID3) ..i want make a lookup of 'Dnumber' column in the target table with each of the 3 columns.Has to map the unmatched ones(ID's) in the 'Dnumber'
Iam fine till getting the unmatched ID's from each of the column from source ..but getting problem in mapping these 3 columns in to 'Dnumber'
Ex: Lets say got the unmatched are like this ,
ID1 ID2 ID3 1 4 6 2 5 7 3 8
Output should be :
Dnumber 1 2 3 4 5 6 7 8
I Can't use UnionAll..bcoz 3 columns should map to the same target column.Not sure about Merge Join
Yesterday I unintentionally removed some records from a database. I need to rebuild the database to the point just before I ran the DELETE query. My restore is 12 hours before that point. I'm pretty sure that's what the transaction log is for, right? Is there a tutorial somewhere on how to do this? Or maybe software I can buy to make it easy?
I need to merge database's from 2 different locations into a master database. I am concerned with the identities being duplicated, and the need to map/translate the ids in a sequential order. Also this needs to happen on a nightly basis , would i need to write a custom script ?? any starters , pointers would be appreciated
