Import A XML File In SQL2005 DB Using BulkInsert?

Apr 15, 2008

Hi All,

I am dealing with a lot of XML file data that I load in a SQL 2005 database using Integration Services. Let me give you a high level idea of I how I do it.

The way I do it is that a) I create a blank Integration Services Project, then b) In the "Control Flow" tab I put the "Data Flow Task" by simply dragging "Data Flow Task" on to the "Control Flow" page. c) Then in the "Data Flow" tab I drag the "XML Source" as my data flow source, and drag "OLEB Destination" as my data flow destination; then I create a connection between the "XML Source" and "OLEDB Destination" and point the "XML Source" to my XML file, and I point the "OLEDB Destination" to the Database where I want to load the XML file. I configure the mappings etc if needed and run the package. This works fine for me. But if the XML file is large it takes hours to load the data. I am dealing with huge size XML files and I want an alternative in Inegration Services probably "Bulk Insert Task" but dont know how to use it.
Can someone please explain how I can use bulk insert as a fast alternative to load the data. Please list the steps.

Thans a milliom in advance.

Zee

View 3 Replies


ADVERTISEMENT

Specifyin The Row Delimeter As Newline (Unix Style) In Bulkinsert Format File

Sep 28, 2007

Hi,

I am bulk inserting the data from a file into a table using the following format.



9.0
1
1 SQLCHAR 0 10 "
" 1 MSISDN SQL_Latin1_General_CP1_CI_AS

This works perfectly for me.

But my problem is that the firmat may not be generated in Windows and i can't user "
" as row delimeter.
It could be generated in Unix and want to use LF as the row delimeter.

Questions :

1) How do i specify this kind of new line (LF ) in formatfile?
2) Is there anyway that i can specify both the delimeters and let it use which ever is appropriate ?

Please provide some inputs on this??

Thank u so much
~Mohan

View 3 Replies View Related

Sql2005 Database Restore From Another Sql2005 Backup File Error.

Dec 15, 2005

hi

i try to restore a bak file from another sql2005 server to my sql2005 server, but it show the error message as below :

 

 

TITLE: Microsoft SQL Server Management Studio Express
------------------------------

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.Express.ConnectionInfo)

------------------------------
ADDITIONAL INFORMATION:

Cannot open backup device 'C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLBackupackup.bak'. Operating system error 5(error not found).
RESTORE HEADERONLY is terminating abnormally. (Microsoft SQL Server, Error: 3201)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=09.00.1399&EvtSrc=MSSQLServer&EvtID=3201&LinkId=20476

------------------------------
BUTTONS:

OK
------------------------------


 

 

 

pls some one can help me ???

 

thanks

chaus

View 62 Replies View Related

SQL2005 Express CSV Import

Feb 8, 2007

Hello, I hope I am posting this question in the right forum. I have been working on some ASP applications and using them to connect to a MySQL database. I would like to move all of the data from MySQL over to SQL2005 Express. The problem is that the express version doesn't offer an import/export tool for CSV files. I am able to export my table data from MySQL to a CSV file, how can I import this data into my SQL2005 Ex database? Thanks.



Jim Miller

View 2 Replies View Related

Import As400-DB2 Data Into SQL2005

May 19, 2008

How do i import As400 data in to SQL2005(standard version) using SSIS?? Please help!!!

View 1 Replies View Related

Import MDF And LDF Backup From SQL2000 To SQL2005

Oct 26, 2006

hi,

i have six files ( 3 MDF and 3 LDF) from a backup of a MS SQL 2000 server.

I installed MS SQL 2005 and wanted to Restore these Dataset. but they cant be accessed. doesnt SQL 2005 support these files ?

How can i use these files to access them with SQL 2005 ?





Thx,

View 1 Replies View Related

Import Data From ODBC To SQL2005

Mar 16, 2007

Hi, there;
I want to importing data from ODBC, I created DataReader Source which use a .NET Provide Odbc Data Provider and connected successfully. My destination is a OLE DB Destination that points to SQL2005. I set the SQL command as "SELECT * from ....".
I also have problem to create new table in SQL2005 using SSIS Import and Export Wizard, it doesn't know the source table table schema (two date type column). So I create the new table manully and run the package, I got error:


SSIS package "Package1.dtsx" starting.
Information: 0x4004300A at Data Flow Task, DTS.Pipeline: Validation phase is beginning.
Information: 0x40043006 at Data Flow Task, DTS.Pipeline: Prepare for Execute phase is beginning.
Information: 0x40043007 at Data Flow Task, DTS.Pipeline: Pre-Execute phase is beginning.
Information: 0x4004300C at Data Flow Task, DTS.Pipeline: Execute phase is beginning.
Error: 0xC02090F5 at Data Flow Task, Source - Query [1]: The component "Source - Query" (1) was unable to process the data.
Error: 0xC0047038 at Data Flow Task, DTS.Pipeline: The PrimeOutput method on component "Source - Query" (1) returned error code 0xC02090F5. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.
Error: 0xC0047021 at Data Flow Task, DTS.Pipeline: Thread "SourceThread0" has exited with error code 0xC0047038.
Error: 0xC0047039 at Data Flow Task, DTS.Pipeline: Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
Error: 0xC0047021 at Data Flow Task, DTS.Pipeline: Thread "WorkThread0" has exited with error code 0xC0047039.
Information: 0x40043008 at Data Flow Task, DTS.Pipeline: Post Execute phase is beginning.
Information: 0x402090DF at Data Flow Task, Destination - PITest CMF [169]: The final commit for the data insertion has started.
Information: 0x402090E0 at Data Flow Task, Destination - PITest CMF [169]: The final commit for the data insertion has ended.
Information: 0x40043009 at Data Flow Task, DTS.Pipeline: Cleanup phase is beginning.
Information: 0x4004300B at Data Flow Task, DTS.Pipeline: "component "Destination - PITest CMF" (169)" wrote 0 rows.
Task failed: Data Flow Task
SSIS package "Package1.dtsx" finished: Success.

Can any one know what's wrong here?

And it is quite pain that you have to specify the table name  every time you want to create a new table in SQL2005. It is so easy in SQL2000!!!

Thanks.

View 1 Replies View Related

Import Data From Sql2005 Into Sql Mobile

Sep 3, 2007

Hi,
i need to import data from sql 2005 into sql mobile, is there anyway to do it?
Need help urgently....

thanks

Joel

View 1 Replies View Related

[sql2005 Express] Import A Table From Sql2000

Jul 13, 2007

Is there a possibility to import a table from sql 2000 into sql 2005 express?
I have a table exportet from sql2000 to csv, adjusted "FALSE" and "TRUE" into 0 and 1 and saved it to csv. Now I want to replace a table in the sql2005 DB by the csv. (The first row contains the header, each row contains a record.

I'm a n00b, as you can see ;)

Thanks for your help ;)

Peter

View 3 Replies View Related

How To Import MySQL Data Into SQL2005 Using Wizard?????

Mar 22, 2007

Hello everybody,

I'm a new SQL2005 user.
I'm
trying to import data from MySQL version 5x into my SQL2005 by wizard.
I created a DNS file and tested successfully using MySQL Connector/ODBC
v5.
Everthing seems fine but at the last step selecting data
source. The SQL2005 wizard forced me to choose using SQL command option
instead of selecting tables/views from a list. Anyone can tell me why?
My collegues faced the same case as they help me to find the reason.
I'm sure that there are a few source objetcts in MySQL source.

Any help will be appreciated!!!

khanhmy

View 1 Replies View Related

SQL2005 Equivalent Of The Data Import Task In SQL2000

Jun 18, 2006

I had to upgrade to SQL2005 when my customer did so. Everything went OK until I had to load additional data to the datamart. The SQL2000 task to load data from flat text files was straight forward and literally took me seconds tyo set up and execute and has never given me problems. I have now spent 3 days straight trying to do the same in SQL2005 going in ever wider circles. I have got SSIS up and going, I saved a dtsx package that won't run without a multitude of errors ranging from truncation errors (which I tried to avoid by meticulously ensuring all field definitions on the import and export side are defined and correct) to errors relating to not being able to connect to the database I am working on or of not having the right security access. I am at my wits end. I need help!

View 8 Replies View Related

BULKINSERT

Feb 26, 2008

hi all,

can any one give me syntax example of how to use BULKINSERT

Thanx for any help

sql9

View 3 Replies View Related

BULKINSERT Problem

Jan 24, 2005

Dear All,

I am importing all the files from a particular folder to a table on my database KB. It is working perfectly if i use it on the same system where the DB exists and not working from
the network. Also this scritp is working perfectly on sqlserver7 and not working on sqlserver2000.

SQL2000 returns error near comma

set @query ='BULK INSERT '+ @Tablename + ' FROM "'+ @Filepath+@Filename+'"
WITH ( FIELDTERMINATOR = ","(returns error here) ,ROWTERMINATOR = "")'



USE TESTDB

--Table Creation Starts here

Create table Account([ID] int IDENTITY PRIMARY KEY, Name Varchar(100),
AccountNo varchar(100), Balance money)

Create table logtable (id int identity(1,1),
Query varchar(1000),
Importeddate datetime default getdate())

--Table Creation ends here

---Stored Procedure Starts here

Create procedure usp_ImportMultipleFiles @filepath varchar(500),
@pattern varchar(100), @TableName varchar(128)
as
set quoted_identifier off
declare @query varchar(1000)
declare @max1 int
declare @count1 int
Declare @filename varchar(100)
set @count1 =0
create table #x (name varchar(200))
set @query ='master.dbo.xp_cmdshell "dir '+@filepath+@pattern +' /b"'
insert #x exec (@query)
delete from #x where name is NULL
select identity(int,1,1) as ID, name into #y from #x
drop table #x
set @max1 = (select max(ID) from #y)
--print @max1
--print @count1
While @count1 <= @max1
begin
set @count1=@count1+1
set @filename = (select name from #y where [id] = @count1)
set @query ='BULK INSERT '+ @Tablename + ' FROM "'+ @Filepath+@Filename+'"
WITH ( FIELDTERMINATOR = ",",ROWTERMINATOR = "")'
--print @query
exec (@query)
insert into logtable (query) select @query
end

drop table #y

--sp ends here

Exec usp_ImportMultipleFiles 'c:myimport', '*.csv', 'Account'

If i use the above Exec like

Exec usp_ImportMultipleFiles '\kb-02C$MyImport', '*.csv', 'Account'

I am getting the following error:

Could not bulk insert because file '\kb-02C$MyImportAccess is denied.' could not be

opened.
Operating system error code 5(Access is denied.).

C Drive and MyImport folder is shared on system kb-02


Also If if execute this script it should insert the following line to logtable for all the available files on myimport folder.

How do i modify the following line in the above script to get the syntax given below:


set @query ='BULK INSERT '+ @Tablename + ' FROM "'+ @Filepath+@Filename+'"
WITH ( FIELDTERMINATOR = ",",ROWTERMINATOR = "")'

Required Syntax

BULK INSERT kbAccount FROM 'c:myimporta.csv' WITH ( FIELDTERMINATOR = ',' ,ROWTERMINATOR = '')

Would appreciate your valuable HELP.

thanking your valuable help in advance.
K006B

View 1 Replies View Related

BULKINSERT - Any GURU ?

Jan 2, 2008



Hi,

I am trying to import comma delimitted text files which contain data for any particular country.

I have created SSIS package to import data into sql server table. table structure is exactly same as text file except one additional column which is to identify the CountryID. Text files we recieved does not contain CountryID column.

I am using BULK INSERT Task in SSIS to import the data as text files are quite big (up to 1 GB). during processing I do determine the country id for that text file as file name contain countrycode which can be use for lookup and get the countryid.

cyn_bills_20071208_032242.txt

CYN is the country code and country table has got country id for it.

I am using Format file with BULK INSERT task as table has additional column which does not present in source file.
currently after inserting all rows into table i have to run an UPDATE statement to update the value of CountryID column which is very expensive some time taking more then 30 min for (30 million rows)

Can anyone tell me how to insert an expression when using BULK INSERT command / Task in SSIS. there must be a way to insert litterals using Bulk Insert.

Thanks in Advance.
Furrukh Baig

View 6 Replies View Related

Load Data With BulkInsert In DTS, How To Skip The First Line?

Dec 13, 2007

I can skip the first line with Transform Data Task, it look like can not skip the first line in BulkInsert.
But bulkinsert is faster, anybody can help?

Thanks

View 1 Replies View Related

Error Regarding File Import Through Import Wizard

Jan 12, 2006

Hi all,

when trying to ímport files to our database server from a client, I keep getting an error:

- Validating (Error)
Messages
Error 0xc00470fe: Data Flow Task: The product level is insufficient for component "Source_txt" (1).
 (SQL Server Import and Export Wizard)
 
Error 0xc00470fe: Data Flow Task: The product level is insufficient for component "Data Conversion 1" (175).
 (SQL Server Import and Export Wizard)

... doing the same import when logged on the server, hasn't been giving me any errors, how come. I can from my client without trouble import tables from other DB servers but when ever it is files it won't do it.

 

I tried as mentioned in other threads rerun setup to re-install SSIS, but as it was already installed it wouldn't re-install. My next move would be to make a clean install, but not sure it would help, as I think this is a buck.

best regards

 

Musa Rusid

View 1 Replies View Related

Sql2005 Export File To Directory

Jul 23, 2007

Hi all,
I have scheduled a package to export data from the table to the execl file in the local directory, it work fine. However, when the next time you execute the package it give error on 'file already exists'. Prephap this is due to previously already created a file in the directory.

I try to use drop option in the ssis during creating the package for export, but the result still the same. I do not know the drop option is working, can someone tell me is that working fine? if not what is the solution or possible cause.

Thanks for your help.:)

View 1 Replies View Related

Shrink Log File Database In Sql2005

Jun 23, 2008

hi
i have shrink log file of database use dbcc command

Message
Executed as user: sa. Cannot shrink log file 2 (abc.ldf) because all logical log files are in use. [SQLSTATE 01000] (Message 9008) DBCC execution completed. If DBCC printed error messages, contact your system administrator. [SQLSTATE 01000] (Message 2528). The step succeeded.

View 2 Replies View Related

Restoring SQL2000 Bkp File On SQL2005

Oct 23, 2006

Is it possible to restore SQL2000 backup to SQL2005? Or I have to restore the DB to SQL2000 and then upgrade it to SQL2005?

Can someone provide me ways to upgrade SQL2000 DB to SQL2005?



Thanks,

Hiten

View 5 Replies View Related

Error While While Importing Excel File Into SQL2005

Nov 6, 2007



I am using the import tool to import a small excell file into SQL.
I am getting the following error

Error 0xc00470fe: Data Flow TAsk: The product level is insufficient for componene "source - Current_customer$" (1)

The file name I am importing is Current_customer, which contains 4 fields
Id
last
first
zip

View 3 Replies View Related

SQL2005 Memory And Paging File Issue

Mar 28, 2007

Currently, we have the following setup...

SQL 2005 Std, 8gb ram, boot ini switches for /PAE and /3GB, AWE enabled and the min/max memory settings for a single instance respectively set to 1024 and 7168. Performance-wise the system is a bit slower than before (with SQL2000) and there seems to be alot of paging. The SQL service is also running under the "LocalSystem". In this case, do I need to specify the local "SYSTEM" to have rights to "Lock pages in memory"? I have a feeling there is a misconfiguration somewhere. Can someone point me in the right direction? Thanks.

View 4 Replies View Related

Storing Html File Inside SQL2005 Database

Oct 10, 2006

Dear all I create this html file on the fly in my asp.net application abd what i would like to do is to store it inside my sql2005 database. What would be the best way?The html file itself is not really big. Probably not more then 600 - 800 characters most. I was thinking the text type fields of the database and then when retreiving it dump it inside in a file and save the file with html extention. Are there any better sugestions? Thank you for your time

View 3 Replies View Related

SQL2005 X64bit Linked To Msaccsses Mdb File Problem

Jul 29, 2007

Hi ,
I am trying to link with Linked server to MDB file, when i try to create the linked server i i cant find the Jet Ole db provider, how i can link it ?

View 1 Replies View Related

Converting Flat File To SQL2005 Table (Flat File From H***)

Feb 11, 2008

First, a couple of important bits of information. Until last week, I had never touched SISS, and therefore, I know very little about it. I just never had the need to use it...until now. I was able to convert my first 3 flat files to SQL2005 tables by right clicking on "SISS Package" and choosing "SISS Import and Export Wizard". That is the extent of my knowledge! So please, please, please be patient with me and be as descriptive as possible.

I thought I could attach some sample files to this post, but it doesn't look like I can. I'll just paste the information below in two separate code boxes. The first code box is the flat file specifications and the second one is a sample single line flat file similar to what I'm dealing with (the real flat file is over 2 gigs).

My questions are below the sample files.


Code Snippet
Record Length 400

Positions Length FieldName

Record Type 01
1,2 L=2 Record Type (Always "01")
3,12 L=10 Site Name
13,19 L=7 Account Number
20,29 L=10 Sub Account
30,35 L=6 Balance
36,37 L=1 Active
37,41 L=5 Filler
Record Type 02
1,2 L=2 Record Type (Always "02")
3,4 L=2 State
5,30 L=26 Address
31,41 L=11 Filler
Record Type 03
1,2 L=2 Record Type (Always "03")
3,6 L=4 Coder
7,20 L=14 Locator ID
21,22 L=2 Age
23,41 L=19 Filler
Record Type 04
1,2 L=2 Record Type (Always "04")
3,9 L=7 Process
10,19 L=10 Client
20,26 L=6 DOB
26,41 L=16 Filler
Record Type 05
1,2 L=2 Record Type (Always "05")
3,16 L=14 Guarantor
17,22 L=6 Guar Account
23,23 L=1 Active Guar
**There can be multiple 05 records, one for each Guarantor on the account**


and the single line flat file...



Code Snippet
01Site1 12345 0000098765 Y 02NY1155 12th Street 03ELL 0522071678 29 04TestingSmith,Paul071678 05Smith, Jane 445978N 05Smith, Julie 445989N 05Smith, Jenny 445915N 01Site2 12346 0000098766 N 02MN615 Woodland Ct 04InfoJones,Chris 012001 01Site3 12347 0000098767 Y 02IN89 Jade Street 03OWB 6429051282 25 04Screen New,Katie 879500





As you can see, each entry could have any number of records and multiples of some of the record types, with one exception, every entry must have a "01" record and can only have one "01" record. Oh, and each record has a length of 400.

I need to get this information into a SQL 2005 database so I can create a front end for accessing the data. Originally, I wanted one line for each account and have null values listed for entries that don't have a specific record. Now that I've looked at the data again, that doesn't look like a good idea. I think a better way to do it would be to create 5 different tables, one for each record type. However, records 2 through 5 don't have anything I can make a primary key. So here are my questions...


Is it possible to make 5 tables from this one file, one table for each of the record types?

If so, can I copy the Account number in record 01, position 13-19 in each of the subsequent record types (that way I could link the tables as needed)?

Can this be done using the SISS Import and Export Wizard to create the package? If not, I'm going to need some very basic step by step instructions on how to create the package.

Is SISS the best way to do this conversion or is there another program that would be better to use?
I know this is a huge question and I appreciate the help of anyone who boldly decides to help me! Thank you in advance and I welcome anyone's suggestions!

View 13 Replies View Related

What Are The Differences Between CD1 And CD2 Of The SQL Server 2005 Enterprise? And Can I Just Attach The Sql2000 MDF File Into Sql2005?

Feb 4, 2006

Hi
I have new bought the SQL server 2005 enterprise, but it have 2 CDs, so what are the differences between CD1 and CD2? and so which one should i install first? or is it necessary to install both two or just need to install one of them?
And about my original sql 2000 database, can i just attach it's MDF file into the sql 2005 engine, or which import wizard can load the sql 2000 MDF into sql 2005? or do i need to keep the sql 2000 engine before do this?
thx

View 1 Replies View Related

DTS File Import When File Is Locked

Jan 29, 2003

Guys.

This has been an issue for me. It happens once in a while.

I am importing a File from a shared folder thru DTS package. Once in a while the DTS package fails due to the following error.

"Error Opening Data File: Process cannot access the file because it is being used by another process"

Is there anyway I can specify that the file is going to be opened in read only, so that the DTS will not fail?

Any other solution/suggestion?

-MAK

View 1 Replies View Related

Can You Open A Database Created In SQL2005 In SQL2005 Express?

Oct 12, 2007



Can you open/use a database created in SQL2005 in SQL2005 Express?

Thanks for the help!

Max

View 4 Replies View Related

Upgraded SQL2000 To SQL2005, SQL2005 VERY Slow

Dec 27, 2005

I just upgraded my SQL 2000 server to SQL2005. I forked out all that money, and now it takes 4~5 seconds for a webpage to load. You can see for yourself. It's pathetic. When I ran SQL2000, i was getting instant results on any webpage. I can't find any tool to optimize the tables or databases. And when I used caused SQL Server to use 100% cpu and 500+MB of ram. I can't have this.Can anyone give me some tips as to why SQL 2005 is so slow?

View 3 Replies View Related

Sql2005 Standard To Sql2005 Enterprise Edition

Jan 4, 2007

Hi

We have Sql2005 x64 bit standard edition server installed in windows 2003 64 bit editio server,

currently due to buisness requirements we need to have sql2005 x64 bit enterprise edition, please let me know how do i do the upgrade or change.

is it possible to retain all our custom settings in the standard edition after changing to enterprise edition.

This has to be done for our production and very critical, please help



Thanks

Samuel I

View 4 Replies View Related

How To Import XML File Into SQL.

Dec 28, 2005

I want to import XML file as the '@doc' value when I execute 'sp_xml_preparedocument', many thanks!

View 1 Replies View Related

How To Import .mdf File

Sep 28, 2001

Hello every one

i unstalled my sql server and installed it again, and i want the old database. so how i can i import the old database file .mdf, ldf here. i tried to copy this 2files in to data folder, but its not showing database files in the server.

please help me out,i havent taken any backup before uninstallation

with regards
kittu

View 1 Replies View Related

Import A CSV File

Jan 2, 2002

Im looking to write a DTS script to import a CSV file into an existing table. I have made sure that all columns correspond. I ideally want to create a stored procedure and a variable to be entered as the file name.

How can I do this?

View 1 Replies View Related

How Do You Import A DBF File

Apr 15, 2004

How do you import a dbf file to sql and dump it into a table please help . iam totaly new at this.

View 2 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved