Import A XML File In SQL2005 DB Using BulkInsert?
Apr 15, 2008
Hi All,
I am dealing with a lot of XML file data that I load in a SQL 2005 database using Integration Services. Let me give you a high level idea of I how I do it.
The way I do it is that a) I create a blank Integration Services Project, then b) In the "Control Flow" tab I put the "Data Flow Task" by simply dragging "Data Flow Task" on to the "Control Flow" page. c) Then in the "Data Flow" tab I drag the "XML Source" as my data flow source, and drag "OLEB Destination" as my data flow destination; then I create a connection between the "XML Source" and "OLEDB Destination" and point the "XML Source" to my XML file, and I point the "OLEDB Destination" to the Database where I want to load the XML file. I configure the mappings etc if needed and run the package. This works fine for me. But if the XML file is large it takes hours to load the data. I am dealing with huge size XML files and I want an alternative in Inegration Services probably "Bulk Insert Task" but dont know how to use it.
Can someone please explain how I can use bulk insert as a fast alternative to load the data. Please list the steps.
Thans a milliom in advance.
Zee
View 3 Replies
ADVERTISEMENT
Sep 28, 2007
Hi,
I am bulk inserting the data from a file into a table using the following format.
9.0
1
1 SQLCHAR 0 10 "
" 1 MSISDN SQL_Latin1_General_CP1_CI_AS
This works perfectly for me.
But my problem is that the firmat may not be generated in Windows and i can't user "
" as row delimeter.
It could be generated in Unix and want to use LF as the row delimeter.
Questions :
1) How do i specify this kind of new line (LF ) in formatfile?
2) Is there anyway that i can specify both the delimeters and let it use which ever is appropriate ?
Please provide some inputs on this??
Thank u so much
~Mohan
View 3 Replies
View Related
Dec 15, 2005
hi
i try to restore a bak file from another sql2005 server to my sql2005 server, but it show the error message as below :
TITLE: Microsoft SQL Server Management Studio Express
------------------------------
An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.Express.ConnectionInfo)
------------------------------
ADDITIONAL INFORMATION:
Cannot open backup device 'C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLBackupackup.bak'. Operating system error 5(error not found).
RESTORE HEADERONLY is terminating abnormally. (Microsoft SQL Server, Error: 3201)
For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=09.00.1399&EvtSrc=MSSQLServer&EvtID=3201&LinkId=20476
------------------------------
BUTTONS:
OK
------------------------------
pls some one can help me ???
thanks
chaus
View 62 Replies
View Related
Feb 8, 2007
Hello, I hope I am posting this question in the right forum. I have been working on some ASP applications and using them to connect to a MySQL database. I would like to move all of the data from MySQL over to SQL2005 Express. The problem is that the express version doesn't offer an import/export tool for CSV files. I am able to export my table data from MySQL to a CSV file, how can I import this data into my SQL2005 Ex database? Thanks.
Jim Miller
View 2 Replies
View Related
May 19, 2008
How do i import As400 data in to SQL2005(standard version) using SSIS?? Please help!!!
View 1 Replies
View Related
Oct 26, 2006
hi,
i have six files ( 3 MDF and 3 LDF) from a backup of a MS SQL 2000 server.
I installed MS SQL 2005 and wanted to Restore these Dataset. but they cant be accessed. doesnt SQL 2005 support these files ?
How can i use these files to access them with SQL 2005 ?
Thx,
View 1 Replies
View Related
Mar 16, 2007
Hi, there;
I want to importing data from ODBC, I created DataReader Source which use a .NET Provide Odbc Data Provider and connected successfully. My destination is a OLE DB Destination that points to SQL2005. I set the SQL command as "SELECT * from ....".
I also have problem to create new table in SQL2005 using SSIS Import and Export Wizard, it doesn't know the source table table schema (two date type column). So I create the new table manully and run the package, I got error:
SSIS package "Package1.dtsx" starting.
Information: 0x4004300A at Data Flow Task, DTS.Pipeline: Validation phase is beginning.
Information: 0x40043006 at Data Flow Task, DTS.Pipeline: Prepare for Execute phase is beginning.
Information: 0x40043007 at Data Flow Task, DTS.Pipeline: Pre-Execute phase is beginning.
Information: 0x4004300C at Data Flow Task, DTS.Pipeline: Execute phase is beginning.
Error: 0xC02090F5 at Data Flow Task, Source - Query [1]: The component "Source - Query" (1) was unable to process the data.
Error: 0xC0047038 at Data Flow Task, DTS.Pipeline: The PrimeOutput method on component "Source - Query" (1) returned error code 0xC02090F5. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.
Error: 0xC0047021 at Data Flow Task, DTS.Pipeline: Thread "SourceThread0" has exited with error code 0xC0047038.
Error: 0xC0047039 at Data Flow Task, DTS.Pipeline: Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
Error: 0xC0047021 at Data Flow Task, DTS.Pipeline: Thread "WorkThread0" has exited with error code 0xC0047039.
Information: 0x40043008 at Data Flow Task, DTS.Pipeline: Post Execute phase is beginning.
Information: 0x402090DF at Data Flow Task, Destination - PITest CMF [169]: The final commit for the data insertion has started.
Information: 0x402090E0 at Data Flow Task, Destination - PITest CMF [169]: The final commit for the data insertion has ended.
Information: 0x40043009 at Data Flow Task, DTS.Pipeline: Cleanup phase is beginning.
Information: 0x4004300B at Data Flow Task, DTS.Pipeline: "component "Destination - PITest CMF" (169)" wrote 0 rows.
Task failed: Data Flow Task
SSIS package "Package1.dtsx" finished: Success.
Can any one know what's wrong here?
And it is quite pain that you have to specify the table name every time you want to create a new table in SQL2005. It is so easy in SQL2000!!!
Thanks.
View 1 Replies
View Related
Sep 3, 2007
Hi,
i need to import data from sql 2005 into sql mobile, is there anyway to do it?
Need help urgently....
thanks
Joel
View 1 Replies
View Related
Jul 13, 2007
Is there a possibility to import a table from sql 2000 into sql 2005 express?
I have a table exportet from sql2000 to csv, adjusted "FALSE" and "TRUE" into 0 and 1 and saved it to csv. Now I want to replace a table in the sql2005 DB by the csv. (The first row contains the header, each row contains a record.
I'm a n00b, as you can see ;)
Thanks for your help ;)
Peter
View 3 Replies
View Related
Mar 22, 2007
Hello everybody,
I'm a new SQL2005 user.
I'm
trying to import data from MySQL version 5x into my SQL2005 by wizard.
I created a DNS file and tested successfully using MySQL Connector/ODBC
v5.
Everthing seems fine but at the last step selecting data
source. The SQL2005 wizard forced me to choose using SQL command option
instead of selecting tables/views from a list. Anyone can tell me why?
My collegues faced the same case as they help me to find the reason.
I'm sure that there are a few source objetcts in MySQL source.
Any help will be appreciated!!!
khanhmy
View 1 Replies
View Related
Jun 18, 2006
I had to upgrade to SQL2005 when my customer did so. Everything went OK until I had to load additional data to the datamart. The SQL2000 task to load data from flat text files was straight forward and literally took me seconds tyo set up and execute and has never given me problems. I have now spent 3 days straight trying to do the same in SQL2005 going in ever wider circles. I have got SSIS up and going, I saved a dtsx package that won't run without a multitude of errors ranging from truncation errors (which I tried to avoid by meticulously ensuring all field definitions on the import and export side are defined and correct) to errors relating to not being able to connect to the database I am working on or of not having the right security access. I am at my wits end. I need help!
View 8 Replies
View Related
Feb 26, 2008
hi all,
can any one give me syntax example of how to use BULKINSERT
Thanx for any help
sql9
View 3 Replies
View Related
Jan 24, 2005
Dear All,
I am importing all the files from a particular folder to a table on my database KB. It is working perfectly if i use it on the same system where the DB exists and not working from
the network. Also this scritp is working perfectly on sqlserver7 and not working on sqlserver2000.
SQL2000 returns error near comma
set @query ='BULK INSERT '+ @Tablename + ' FROM "'+ @Filepath+@Filename+'"
WITH ( FIELDTERMINATOR = ","(returns error here) ,ROWTERMINATOR = "")'
USE TESTDB
--Table Creation Starts here
Create table Account([ID] int IDENTITY PRIMARY KEY, Name Varchar(100),
AccountNo varchar(100), Balance money)
Create table logtable (id int identity(1,1),
Query varchar(1000),
Importeddate datetime default getdate())
--Table Creation ends here
---Stored Procedure Starts here
Create procedure usp_ImportMultipleFiles @filepath varchar(500),
@pattern varchar(100), @TableName varchar(128)
as
set quoted_identifier off
declare @query varchar(1000)
declare @max1 int
declare @count1 int
Declare @filename varchar(100)
set @count1 =0
create table #x (name varchar(200))
set @query ='master.dbo.xp_cmdshell "dir '+@filepath+@pattern +' /b"'
insert #x exec (@query)
delete from #x where name is NULL
select identity(int,1,1) as ID, name into #y from #x
drop table #x
set @max1 = (select max(ID) from #y)
--print @max1
--print @count1
While @count1 <= @max1
begin
set @count1=@count1+1
set @filename = (select name from #y where [id] = @count1)
set @query ='BULK INSERT '+ @Tablename + ' FROM "'+ @Filepath+@Filename+'"
WITH ( FIELDTERMINATOR = ",",ROWTERMINATOR = "")'
--print @query
exec (@query)
insert into logtable (query) select @query
end
drop table #y
--sp ends here
Exec usp_ImportMultipleFiles 'c:myimport', '*.csv', 'Account'
If i use the above Exec like
Exec usp_ImportMultipleFiles '\kb-02C$MyImport', '*.csv', 'Account'
I am getting the following error:
Could not bulk insert because file '\kb-02C$MyImportAccess is denied.' could not be
opened.
Operating system error code 5(Access is denied.).
C Drive and MyImport folder is shared on system kb-02
Also If if execute this script it should insert the following line to logtable for all the available files on myimport folder.
How do i modify the following line in the above script to get the syntax given below:
set @query ='BULK INSERT '+ @Tablename + ' FROM "'+ @Filepath+@Filename+'"
WITH ( FIELDTERMINATOR = ",",ROWTERMINATOR = "")'
Required Syntax
BULK INSERT kbAccount FROM 'c:myimporta.csv' WITH ( FIELDTERMINATOR = ',' ,ROWTERMINATOR = '')
Would appreciate your valuable HELP.
thanking your valuable help in advance.
K006B
View 1 Replies
View Related
Jan 2, 2008
Hi,
I am trying to import comma delimitted text files which contain data for any particular country.
I have created SSIS package to import data into sql server table. table structure is exactly same as text file except one additional column which is to identify the CountryID. Text files we recieved does not contain CountryID column.
I am using BULK INSERT Task in SSIS to import the data as text files are quite big (up to 1 GB). during processing I do determine the country id for that text file as file name contain countrycode which can be use for lookup and get the countryid.
cyn_bills_20071208_032242.txt
CYN is the country code and country table has got country id for it.
I am using Format file with BULK INSERT task as table has additional column which does not present in source file.
currently after inserting all rows into table i have to run an UPDATE statement to update the value of CountryID column which is very expensive some time taking more then 30 min for (30 million rows)
Can anyone tell me how to insert an expression when using BULK INSERT command / Task in SSIS. there must be a way to insert litterals using Bulk Insert.
Thanks in Advance.
Furrukh Baig
View 6 Replies
View Related
Dec 13, 2007
I can skip the first line with Transform Data Task, it look like can not skip the first line in BulkInsert.
But bulkinsert is faster, anybody can help?
Thanks
View 1 Replies
View Related
Jan 12, 2006
Hi all,
when trying to ímport files to our database server from a client, I keep getting an error:
- Validating (Error)
Messages
Error 0xc00470fe: Data Flow Task: The product level is insufficient for component "Source_txt" (1).
(SQL Server Import and Export Wizard)
Error 0xc00470fe: Data Flow Task: The product level is insufficient for component "Data Conversion 1" (175).
(SQL Server Import and Export Wizard)
... doing the same import when logged on the server, hasn't been giving me any errors, how come. I can from my client without trouble import tables from other DB servers but when ever it is files it won't do it.
I tried as mentioned in other threads rerun setup to re-install SSIS, but as it was already installed it wouldn't re-install. My next move would be to make a clean install, but not sure it would help, as I think this is a buck.
best regards
Musa Rusid
View 1 Replies
View Related
Jul 23, 2007
Hi all,
I have scheduled a package to export data from the table to the execl file in the local directory, it work fine. However, when the next time you execute the package it give error on 'file already exists'. Prephap this is due to previously already created a file in the directory.
I try to use drop option in the ssis during creating the package for export, but the result still the same. I do not know the drop option is working, can someone tell me is that working fine? if not what is the solution or possible cause.
Thanks for your help.:)
View 1 Replies
View Related
Jun 23, 2008
hi
i have shrink log file of database use dbcc command
Message
Executed as user: sa. Cannot shrink log file 2 (abc.ldf) because all logical log files are in use. [SQLSTATE 01000] (Message 9008) DBCC execution completed. If DBCC printed error messages, contact your system administrator. [SQLSTATE 01000] (Message 2528). The step succeeded.
View 2 Replies
View Related
Oct 23, 2006
Is it possible to restore SQL2000 backup to SQL2005? Or I have to restore the DB to SQL2000 and then upgrade it to SQL2005?
Can someone provide me ways to upgrade SQL2000 DB to SQL2005?
Thanks,
Hiten
View 5 Replies
View Related
Nov 6, 2007
I am using the import tool to import a small excell file into SQL.
I am getting the following error
Error 0xc00470fe: Data Flow TAsk: The product level is insufficient for componene "source - Current_customer$" (1)
The file name I am importing is Current_customer, which contains 4 fields
Id
last
first
zip
View 3 Replies
View Related
Mar 28, 2007
Currently, we have the following setup...
SQL 2005 Std, 8gb ram, boot ini switches for /PAE and /3GB, AWE enabled and the min/max memory settings for a single instance respectively set to 1024 and 7168. Performance-wise the system is a bit slower than before (with SQL2000) and there seems to be alot of paging. The SQL service is also running under the "LocalSystem". In this case, do I need to specify the local "SYSTEM" to have rights to "Lock pages in memory"? I have a feeling there is a misconfiguration somewhere. Can someone point me in the right direction? Thanks.
View 4 Replies
View Related
Oct 10, 2006
Dear all I create this html file on the fly in my asp.net application abd what i would like to do is to store it inside my sql2005 database. What would be the best way?The html file itself is not really big. Probably not more then 600 - 800 characters most. I was thinking the text type fields of the database and then when retreiving it dump it inside in a file and save the file with html extention. Are there any better sugestions? Thank you for your time
View 3 Replies
View Related
Jul 29, 2007
Hi ,
I am trying to link with Linked server to MDB file, when i try to create the linked server i i cant find the Jet Ole db provider, how i can link it ?
View 1 Replies
View Related
Feb 11, 2008
First, a couple of important bits of information. Until last week, I had never touched SISS, and therefore, I know very little about it. I just never had the need to use it...until now. I was able to convert my first 3 flat files to SQL2005 tables by right clicking on "SISS Package" and choosing "SISS Import and Export Wizard". That is the extent of my knowledge! So please, please, please be patient with me and be as descriptive as possible.
I thought I could attach some sample files to this post, but it doesn't look like I can. I'll just paste the information below in two separate code boxes. The first code box is the flat file specifications and the second one is a sample single line flat file similar to what I'm dealing with (the real flat file is over 2 gigs).
My questions are below the sample files.
Code Snippet
Record Length 400
Positions Length FieldName
Record Type 01
1,2 L=2 Record Type (Always "01")
3,12 L=10 Site Name
13,19 L=7 Account Number
20,29 L=10 Sub Account
30,35 L=6 Balance
36,37 L=1 Active
37,41 L=5 Filler
Record Type 02
1,2 L=2 Record Type (Always "02")
3,4 L=2 State
5,30 L=26 Address
31,41 L=11 Filler
Record Type 03
1,2 L=2 Record Type (Always "03")
3,6 L=4 Coder
7,20 L=14 Locator ID
21,22 L=2 Age
23,41 L=19 Filler
Record Type 04
1,2 L=2 Record Type (Always "04")
3,9 L=7 Process
10,19 L=10 Client
20,26 L=6 DOB
26,41 L=16 Filler
Record Type 05
1,2 L=2 Record Type (Always "05")
3,16 L=14 Guarantor
17,22 L=6 Guar Account
23,23 L=1 Active Guar
**There can be multiple 05 records, one for each Guarantor on the account**
and the single line flat file...
Code Snippet
01Site1 12345 0000098765 Y 02NY1155 12th Street 03ELL 0522071678 29 04TestingSmith,Paul071678 05Smith, Jane 445978N 05Smith, Julie 445989N 05Smith, Jenny 445915N 01Site2 12346 0000098766 N 02MN615 Woodland Ct 04InfoJones,Chris 012001 01Site3 12347 0000098767 Y 02IN89 Jade Street 03OWB 6429051282 25 04Screen New,Katie 879500
As you can see, each entry could have any number of records and multiples of some of the record types, with one exception, every entry must have a "01" record and can only have one "01" record. Oh, and each record has a length of 400.
I need to get this information into a SQL 2005 database so I can create a front end for accessing the data. Originally, I wanted one line for each account and have null values listed for entries that don't have a specific record. Now that I've looked at the data again, that doesn't look like a good idea. I think a better way to do it would be to create 5 different tables, one for each record type. However, records 2 through 5 don't have anything I can make a primary key. So here are my questions...
Is it possible to make 5 tables from this one file, one table for each of the record types?
If so, can I copy the Account number in record 01, position 13-19 in each of the subsequent record types (that way I could link the tables as needed)?
Can this be done using the SISS Import and Export Wizard to create the package? If not, I'm going to need some very basic step by step instructions on how to create the package.
Is SISS the best way to do this conversion or is there another program that would be better to use?
I know this is a huge question and I appreciate the help of anyone who boldly decides to help me! Thank you in advance and I welcome anyone's suggestions!
View 13 Replies
View Related
Feb 4, 2006
Hi
I have new bought the SQL server 2005 enterprise, but it have 2 CDs, so what are the differences between CD1 and CD2? and so which one should i install first? or is it necessary to install both two or just need to install one of them?
And about my original sql 2000 database, can i just attach it's MDF file into the sql 2005 engine, or which import wizard can load the sql 2000 MDF into sql 2005? or do i need to keep the sql 2000 engine before do this?
thx
View 1 Replies
View Related
Jan 29, 2003
Guys.
This has been an issue for me. It happens once in a while.
I am importing a File from a shared folder thru DTS package. Once in a while the DTS package fails due to the following error.
"Error Opening Data File: Process cannot access the file because it is being used by another process"
Is there anyway I can specify that the file is going to be opened in read only, so that the DTS will not fail?
Any other solution/suggestion?
-MAK
View 1 Replies
View Related
Oct 12, 2007
Can you open/use a database created in SQL2005 in SQL2005 Express?
Thanks for the help!
Max
View 4 Replies
View Related
Dec 27, 2005
I just upgraded my SQL 2000 server to SQL2005. I forked out all that money, and now it takes 4~5 seconds for a webpage to load. You can see for yourself. It's pathetic. When I ran SQL2000, i was getting instant results on any webpage. I can't find any tool to optimize the tables or databases. And when I used caused SQL Server to use 100% cpu and 500+MB of ram. I can't have this.Can anyone give me some tips as to why SQL 2005 is so slow?
View 3 Replies
View Related
Jan 4, 2007
Hi
We have Sql2005 x64 bit standard edition server installed in windows 2003 64 bit editio server,
currently due to buisness requirements we need to have sql2005 x64 bit enterprise edition, please let me know how do i do the upgrade or change.
is it possible to retain all our custom settings in the standard edition after changing to enterprise edition.
This has to be done for our production and very critical, please help
Thanks
Samuel I
View 4 Replies
View Related
Dec 28, 2005
I want to import XML file as the '@doc' value when I execute 'sp_xml_preparedocument', many thanks!
View 1 Replies
View Related
Sep 28, 2001
Hello every one
i unstalled my sql server and installed it again, and i want the old database. so how i can i import the old database file .mdf, ldf here. i tried to copy this 2files in to data folder, but its not showing database files in the server.
please help me out,i havent taken any backup before uninstallation
with regards
kittu
View 1 Replies
View Related
Jan 2, 2002
Im looking to write a DTS script to import a CSV file into an existing table. I have made sure that all columns correspond. I ideally want to create a stored procedure and a variable to be entered as the file name.
How can I do this?
View 1 Replies
View Related
Apr 15, 2004
How do you import a dbf file to sql and dump it into a table please help . iam totaly new at this.
View 2 Replies
View Related