Large Export Of Data From One DB Of One Structure To Another
Aug 31, 2006
Hi guys,
Hopefully this is the right place to ask.
Basically we have have two larges databases, one of which is updated from the other monthly.
For exaplination purposes:
DB1 = Source DB
DB2 = Destination DB
The problem that I require a soltion to is, how do I insert rows from a table in DB1 to DB2 and recover and store the identity of the new row against the ID of the existing row. This is so that I can then matain constraints when it comes to inserting rows into the next table and the next and so on.
This process of storing the ID's as lookups will need to be done for almost every table of which there are 20.
The best Idea we have at the minute is to create a table with two colums for each table (drop it and recreate it after each table has exported) that contains the two ID's, new and old.
This will require using a cursor for each row in the existing table, inserting it in the new table and the using @@Scope_Identity to get the new ID and then insert the two values into the temp table.
This too me feels like it will be very slow, particuarly when I bear in mind how much data we have.
Does anyone have any better ideas? (Sorry if the explaination isn't great, its difficult to get accross)
Thanks
Ed
View 1 Replies
ADVERTISEMENT
Feb 15, 2007
Hi,is there a way of exporting database structure not the data in it?Thanks.
View 1 Replies
View Related
Mar 13, 2008
Hi,
I've tried to use the export command to export a mining model from management studio, but it returns that
export statement is not supported for OLAP mining structures, Ive checked the EXPORT(DMX) reference I can't see any note that it is not applicable on OLAP structures.
Can anyone confirm this?
-Rgds,
Sheryaar
View 3 Replies
View Related
Mar 30, 2008
I'm trying to export data from an SQL table using classic ASP. In the past, I have created tab delimited text files using code similar to the following code:
Response.AddHeader "Content-Disposition", "attachment;filename=results.txt"
Response.ContentType = "application/octet-stream"
Response.Write "Field1"&vbTAB
Response.Write "Field2"&vbTAB
Response.Write "Field3"&vbTAB
Response.Write vbCRLF
RSb.Open "SELECT * FROM tblData1 WHERE ID=1 ORDER BY txtField1", con, 3, 3
Do Until RSb.EOF
Response.Write RSb("txtField1")&vbTAB
Response.Write RSb("txtField2")&vbTAB
Response.Write RSb("txtField3")&vbTAB
Response.Write vbCRLF
RSb.MoveNext
Loop
RSb.Close
This always worked fine in the past because the tables were small. Now, the tables are exporting 100,000 records and getting errors. I'm setting the page timeout and the sql connection timeout, but I'm still getting errors. I'm not exactly sure what's happening, but the export stops after exporting about 5Mb. It varies where it stops, so I'm thinking it's timing out somewhere.
Is there a better way to do this? Possibly use an export function in MS SQL that would export faster?
TIA
- PR
View 4 Replies
View Related
Oct 27, 2006
I can't find the answer to these problems. I'm using SQL Server 2005 Developer Edition. I have a report with 15,036 rows and about 15 columns. It utilizes a table and a report header and footer. One column has data containing hyperlinks to another report.
1) When I export this report to Excel, the result is 14.2MB file. Why is this file so huge, and how can I make it smaller?
2) Is there anything I can do to prevent the inclusion of the hyperlinks in the exported Excel file?
Thanks,
Rosie
View 4 Replies
View Related
Jun 15, 2001
When using DTS (in SQL 7) to export via OLE DB a large varchar to a text file, it clips it at 255 chars.
No other data access drivers seem to work, either. This is lame! I cannot use bcp as a work
around, because i want to use quoted comma-delimited, which it doesn't support, and I
am using query-based export, where the query calls a stored proc, which bcp also doesn't
support.
Are there any new versions of MDAC that fix this? Anyone know a workaround? My current hack fix
is to split my field into 2, but this is a grubby fix that hassles my reciptients.
This is a pretty fundamental limitation to a major product!
dn
View 1 Replies
View Related
Oct 13, 2006
Hi,
I am currently importing (and exporting) binary flat files to and from Db fields using the TEXTPTR and UPDATETEXT (or READTEXT for export) functions. This allows me to fetch/send the data in manageable packet sizes without the need to load complete files into RAM first.
Given that some files can be up to 1Gb in size I am keen to find out a new way of doing this since the announcement that TEXTPTR, READTEXT and UPDATETEXT are going to be removed from T-SQL.
I had a quick foray into SSIS but couldn't find anything suitable which brings me back to T-SQL. If anyone knows a nice elegant way of doing this and is prepared to share, that would be grand.
Thanks for your time,
Paul
View 9 Replies
View Related
Jan 4, 2008
How can I Export Database with foreing Key and primary key.
Operation is that
SQL2005 Management Studio/Database/Tasks/Export Data
Before Version is SQL2000 we can Selected Copy Object and data between server and then Use Default Options click checked and Select Copy Index, Copy Foreing Primary key vs vs
But this options is not found in the SQL2005 Management Studio/Database/Tasks/Export Data wizard or I can't found it.
How can I export foreing Key and primary key with SQL2005 Management Studio/Database/Tasks/Export Data wizard.
Best Regards,
Athena.
View 1 Replies
View Related
Jul 20, 2005
Ok, I haven't been doing too much database work lately and my brain has gonesoft.I need some help with proper structure.My database is being used to track television shows.Any given show will be associated with at least one production company (callit ProdCo) but possibly up to 3 production companies (never more than that).I have:tblNetworkNetID (numeric, Identity)NetName (char, 50)tblShowDataBookingNum (numeric, Identity)BookingNumExt (numeric, Identity)Now, if I was just dealing with one ProdCo I would add it tblShowData as aforeign key from tblNetwork; no problem.How do I structure it so that I can combine 1 to 3 ProdCo's as one referencein tblShowData.I tried:tblNetworkComboID (numeric, Identity)Net1Net2Net3The problem is, obviously I can only join the FK from tblNetworks to one ofthe NetX fields in tblNetworkCombo.I don't quite know where to go from here. Any help would be appreciated.
View 3 Replies
View Related
Oct 28, 2004
i have a database on one server that I want to duplicate on a new server. I do not want to copy the data only the structure.
I created some SQL scripts from the current db as I have read that is what I should do.
My question is where do I use this script on my new server to create the new tables etc..
Do i use the import feature. Where do I run my sql script?
thanks
View 2 Replies
View Related
Dec 27, 1999
Are there any known problems with replicating structure changes (New table, etc...), as well as data in MSSQL Server 7.0?
View 2 Replies
View Related
Mar 4, 2004
Hello E'body,
I have to export a SQL Server Table (Structure+Data) to an access mdb file at run time. How can i do it? please help.
Lax.
View 1 Replies
View Related
Nov 14, 2014
I have a database (MSSQL). To demonstrate the problem let me show a fictive Tablestructure. I don't want to discuss about how to save the data differntly, because the structure is fix and I can't change it.To get this result I would do a sql query with a lot of joins like that:
SELECT firstname, lastname, email.value, phone.value
FROM Customer
INNER JOIN
(
SELECT Customer_Properties.id, Customer_Properties.value
FROM Customer_Properties
[code]...
I don't think that this is really performant and the SQL-Queries get very complicated. Give it a other methode for that? I can't change the data structure.
View 3 Replies
View Related
Apr 23, 2008
Hi
Can Any one help me out of this in my project I have to migrate only datastructure or schema and not the data inside the table using SSIS. How can I do that ?
View 9 Replies
View Related
Dec 1, 2006
I have a SQL 2005 database that I am using with a website. This basic website will be sold to other companies and ran on their servers with different URLs. Since, All of these databases will store different data, I'm not sure how I can make updates to original database and replicate those structure changes to the other DBs without changing the data also. Is there a way to automate the replication of structural DB changes without replicating the data along with it?
Thanks,
Kirk
View 5 Replies
View Related
Oct 4, 2015
I am studying indexes and keys. I have a table that has a fixed width of data to be loaded in the first column which is parsed in a view based on data types within the fixed width specifications.
Example column A:
(name phone house cost of house,zipcodecountystatecountry)
-a view will later split this large varchar string basedÂ
column b: is the source filename of the data load (varchar 256)
....
a. would there be a benefit of adding a clustered or nonclustered index (if so which/point in direction on why)
b. is there benefit of making one of these two columns a primary key (millions of records) or for adding a 3rd new column as a pk?
c. view: this parses the data in column a so it ends up looking more like "name phone house cost of house zipcode county state country" each having their own column.
-any pros/cons of adding indexes (if so which) to the view instead of the tables or both for once the data is parsed?
View 4 Replies
View Related
Oct 18, 2007
We have SQL Enterprise Manager (8.0). Is there any way to Copy a Table from one database to another with only the Structure (design) - not all the Data?
I can't find any option in the Import Data Wizard that only copies the Table structure.
Any help is appreciated. Thanks
View 1 Replies
View Related
Mar 19, 2008
I have written 5 content-management systems and am getting a little bored writing a lot of the same functionality and tweaking it for different datamodels. Is there a way within .NET to have visibility of a database's structure (ie, data type, column names, foreign keys etc)? I'd like to write up a dynamically generated form for any given table, displaying appropriate form controls based on data type and foreign keys.
View 7 Replies
View Related
Sep 20, 2006
new to SQL Server 2000. We have an obsolete database that we need to
save off for x number of years. DB2 has utilities (DB2Look/Export)
that allows for the export of the data along with a schema and script
that enables the future recreation of the structure of the databases
and tables to include RI etc. You can save off the architecture and
relationships of the tables as well as the data.
Does SQL Server have anything similar?
Failing that, our plan is to backup the data and logs then image the entire disk.
Thanks in advance.
Gerry
View 8 Replies
View Related
Feb 20, 2004
I have a table that looks like this:
ID Type
123 Phone
123 Meeting
123 Phone
and I would like the data to look like this
ID phone Meeting
123 2 1
How do I do this?
View 3 Replies
View Related
Jul 23, 2005
Hi all!I have an application that needs to copy the database structure fromone database to another without using the "Generate SQL Script"function in Enterprise Manager. I'd like to do this from within astored procedure. Can someone recommend the best approach for this?I've seen references to using SQL-DMO from a stored procedure using thesp_OA* procs in other postings to this group but was wondering if therewas an easier way? Can I use bcp and then use xp_cmdshell from withinmy stored procedure? It's not clear to me from the documentationwhether bcp copies both structure and data or just data? Is there abetter way?Thanks in advance for any help!Karen
View 1 Replies
View Related
Feb 2, 2006
Hi SQL gurus,I have a table structure question. I will have a table 'Models' thathas one to many 'incomes' and one to many 'costs'. These 2 entitieshave exactly the same structure, which is 7 smallmoney and a name. Isit better to create a table 'Incomes' and a table 'Costs', with boththe same number of fields like this :Incomes-------------in_idmodelin_1in_2in_3in_4in_5in_6in_7in_nameCosts-------------c_idmodelc_1c_2c_3c_4c_5c_6c_7c_nameor is it better to create one single table that will contain bothentities like that :Incomes_Costs-------------ic_idmodelic_1ic_2ic_3ic_4ic_5ic_6ic_7ic_nameic_isIncomewhich only differs from the 2 above by the isIncome field to know whichrow is an income and which row is a cost.I'd like to know which method is the best in terms of performance andgeneral structure and would greatly appreciate if you explain a littlethe reasons that drove you to suggest me a method over the other.Thanks all for your time!ibiza
View 4 Replies
View Related
Feb 19, 2007
I have been working on my windows app, using SQL Server 2005 (& C#) but I now have loads of junk data, so I want to copy the structure without any data to a new set of files that can become the production version.
I assume I can use backup and restore in future when I want a copy of my production data to use for future testing.
How would I take just one table from my test database and add it to my production database without retyping the design?
Apart from the size limitation is there any advantage in buying a version of SQL Server over using the Express edition?
View 1 Replies
View Related
Feb 13, 2008
I'm looking to create a new version of a database but with completely reloaded data - so I want to retain the database structure but none of the data.
Is there an easy way (read: without a DBA on staff) to copy an existing database structure - preferably with views and stored procedures - and not take the data along for the ride?
Thanks in advance for suggestions...!
View 7 Replies
View Related
Sep 25, 2003
In order to export data to a 3rd party provider, I build five separate tables to store the data. Every table has a different layout, except for the first four columns. They are record type, SSN, employee id and another id number.
Basically, I have to sort that data by SSN then by record type. Each employee will have multiple records.
However, that data will need to be "merged" into one table to be exported.
I have created a table that defines the first four columns, but then has one large "filler" field that will contain the rest of the data. How can I copy data from five different tables with five different layouts into one table?
Any suggestions?
Thanks,
Steve Hanzelman
View 6 Replies
View Related
Feb 15, 2013
We have the following scenario: We receive CSV files every month for which SSIS packages were built to process the data. The following problems occur from time to time:
1. The structure of the CSV file changed (e.g. column added or removed)
2. There were no footers in the data, but now footers started to appear
3. Date format changed (e.g. used to be mm/dd/yyyy, but became mm.dd.yyyy)
4. Number format changed (e.g. from 2000 to 2,000)
Currently we have person who manually opens each file, and using our "validation document" validates to ensure none of these or similar problems occur. We would like to move away from this manual process if possible. I understand that items 3. and 4. could be caught by loading data into a staging table with VARCHAR data types, and performing validation before moving it any further.
Item 2 is a bit questionable (meaning depending on the footer size SSIS load could fail or not).
Item 1, however, is a sure fail of the SSIS package that directly loads the data into a table.
Thus I feel the two possible options are:
1. Create a custom script that will run through the file, row by row, apply all the necessary validations and report an error or continue if all checks out
2. Use some 3rd party tool to validate the files (semi-manually) before kicking off the SSIS processing.
View 3 Replies
View Related
Mar 11, 2004
Hello E'body
I have an application with MSAccess as front end and SQLServer as backend. have quite a bit of tables. i wanted to write a stored procedure which exports a SQL Server table (both Structure & Data) to a new Access MDB file. i know with the use of DTS its possible but i need to code it down. i need to perform this at runtime. so can anybody help.
Its urgent.
Bye.
Lax
View 1 Replies
View Related
Jul 23, 2005
Is there a way to copy the structure from one database to anotherwithout affecting the actual data?For example, I added new fields and stored procedures to a db in mydevelopment environment. There are a lot of them.I now want to make the same changes to my production environment db butdon't want to affect the data. Only the fields, stored procs &constraints and stuff pertaining to the structure of the db should bechanged.Does anyone know of a program that can do this or can this be done inMS SQL Enterprise Manager.Any help would be appreciated.7078895
View 2 Replies
View Related
Aug 24, 2015
I have a large fact table spread across tens of partitions (appx. 1TB each). I found that the business does not need much of the columns in the table. So, as an optimization action, I decided to get rid of these un-needed columns.What is the efficient way to achieve this? Can I simply drop these columns from the table, or use a new table with the reduced structure?
View 2 Replies
View Related
Oct 15, 2006
Is it possible to purge all records in the database while retaining the the table structures. Even better yet, could I do it on a table by table basis? If I simply delete all the records the identities for the tables do not revert back to 1.
View 2 Replies
View Related
Jun 30, 2006
I have created two table with same data structure. I need realtime effects (i.e. data) on both tables - Table1 & Table2.
Following Points to Consider.
1. Both tables are in the same database.
2. Table1 is using for data entry & I wants the same data in the Table2.
3. If any row insert, update & delete occers on Table1, the same effect should be done on Table2.
4. I need real time data insert, update & delete on Table2.
I knew that using triggers it could be possible, I have successfully created a trigger for inserting new rows (using logical table "Inserted") in Table2 but not succeed for update & delete yet.
I want to understand how can I impletement this successfully without any ambiguity.
I have attached data structure for tables. Thanx...
View 10 Replies
View Related
Apr 7, 2007
Hi all
I have a Table with Following structure ( a Tree Structure )
PK Parent Level Code
--- -------- ------- ------
1 0 0 100
2 1 1 101
3 1 1 102
4 2 2 103
5 3 2 104
6 4 3 105
The same as following Tree as you can see
1__
| 2__
| 4__
| 6
| __
3__
5
I need a query to return the following Result. I think it is possible only through Nested sub-Queries But i don't know how to do that
Could any one help me.?
PK Parent Level Value First-Parent' Code 2nd-Parent's Code 3rd-Parent's Code
---- -------- ------ ------- --------------------- ---------------------- -----------------------
1 0 0 100 NULL NULL NULL
2 1 1 101 100 NULL NULL
3 1 1 102 100 NULL NULL
4 2 2 103 101 100 NULL
5 3 2 104 102 100 NULL
6 4 3 105 103 101 100
Any help greatly would be appreciated.
Kind Regards.
View 2 Replies
View Related
Aug 8, 2006
Hello. I want to ask about the possibility of copying both a tablestructure and it's contents from aSQL server table to a table within MS access. The problem cannot besolve with a permanent table structure at the target location.The names of the columns are essentially data with the application andso are subject to change. I am targeting a solution using SQL QueryManager.The approach I have tried (with failure) isSELECT *INTO <linkedserver table>FROM <local table>This should create and copy. However, I am not sure if this isachievable with this approach.Refer to the dialogue;-------------------------------------------------------USE MASTERGOEXEC sp_addlinkedserver@SERVER = 'Freddie',@PROVIDER = 'Microsoft.Jet.OLEDB.4.0',@SRVPRODUCT = 'OLE DB Provider for Jet',@DATASRC = 'C: empHMIS_Recipe.mdb'-- I am not sure if this is requiredEXEC sp_addlinkedsrvlogin 'Freddie', false, 'sa', 'Admin', NULLSELECT * FROM Freddie...FRED -- This is OKSELECT * INTO #Temp FROM Freddie...FRED -- This is OK-- This fails - Refer errorSELECT * INTO Freddie.FRED65from #tempServer: Msg 2760, Level 16, State 1, Line 1Specified owner name 'Freddie' either does not exist or you do not havepermission to use it.-- This also fails and I thought reflected the above select with naming- Refer errorSELECT * INTO Freddie...FRED65from #tempServer: Msg 117, Level 15, State 1, Line 2The object name 'Freddie...' contains more than the maximum number ofprefixes. The maximum is 2.EXEC sp_dropserver 'Freddie',@droplogins = 'droplogins'------------------------------------------------------------Thank you.Regards JC...
View 3 Replies
View Related