Dynamically Importing Csv File Into MS SQL

Feb 14, 2007

I have an app written in delphi v5 with a MS SQL Database and I'm struggling with an import feature i need

The user needs a function to import Contact data from a txt/csv file into the Contact Table

Details of feature:
the user enters the different parameters into the Delphi app e.g.
File Location
Files Includes Column Headings or not
Whether the file Comma or Tab Seperated
Mapping the fields

Then i need to use those parameters and run some sort of import routine putting the data into the specific table.

The tables consists of 3 fields - First Name, Surname, Mob Number - but these fields can be in any order in the file. for example Col 1 (in file) needs to go into Field 3 in DB. this is sorted in the Mapping Fields above

How do I do this using MS SQL??

I've been looking at using the BULK INSERT command e.g
BULK INSERT Contact
FROM 'C:Import_data.csv'
WITH
{
FIELDTERMINATOR = ',',
ROWTERMINATOR = '
'
}
but at the minute i just get error -
[Microsoft][ODBC SQL Server Driver]Syntax error or access violation.

Is this the correct command to use??

Do you know any websites that can point me in the right direction??

I have an app written in delphi v5 with a MS SQL Database and I'm struggling with an import feature i need

The user needs a function to import Contact data from a txt/csv file into the Contact Table

Details of feature:
the user enters the different parameters into the Delphi app e.g.
File Location
Files Includes Column Headings or not
Whether the file Comma or Tab Seperated
Mapping the fields

Then i need to use those parameters and run some sort of import routine putting the data into the specific table.

The tables consists of 3 fields - First Name, Surname, Mob Number - but these fields can be in any order in the file. for example Col 1 (in file) needs to go into Field 3 in DB. this is sorted in the Mapping Fields above

How do I do this using MS SQL??

I've been looking at using the BULK INSERT command e.g
BULK INSERT Contact
FROM 'C:Import_data.csv'
WITH
{
FIELDTERMINATOR = ',',
ROWTERMINATOR = '
'
}
but at the minute i just get error -
[Microsoft][ODBC SQL Server Driver]Syntax error or access violation.

Is this the correct command to use??

Do you know any websites that can point me in the right direction??

Hope all this makes sense

Thanks

View 1 Replies


ADVERTISEMENT

Importing Text File: How To Dynamically Change The Row Delimiter

Jul 22, 2004

Hi,

I have a dts package that imports a number of text files into a SQL Server 2000 database table. The package has been set up to accept a text file with a row delimiter of carriage return and line feed ({CR}{LF}). Some of the text files I receive only have a line feed ({LF}) as the row delimiter and the dts package fails the file. Is there an activex script i can use that will scan the file and change the row delimiter as required?

i was going to use the filesystemobject which allows me to read a line at a time, however the Readline method doesn't read the new line character. The text files are too big to read into one variable and then do a replace.

Any help would be appreciated

I am using SQL Server 2000 SP3, Windows Server 2000 and Windows XP Professional. All systems are fully patched

Regards Justin

View 3 Replies View Related

Reporting Services :: Parsing SSRS Config File And Dynamically Changing File Path Of Config File In Code

Sep 2, 2015

Currently have a single hard coded file path to the SSRS config file which parses the file and provides the reporting services web service url.  My question is how would i run this same query against 100s of servers that may or may not share the same file path as the one hard coded ?

Is there a way to query the registry to find the location of the config file of any server ? which could be on D, E, F, H, etc. 

I know I can string together the address followed by "reports" and named instance if needed, but some instances may not have used the default virtual directory name (Reports).

Am I going about this the hard way ? Is there a location where the web service url exists in a table ? I could not locate anything in the Reporting service database. Basically need to inventory all of my reporting services url's.

View 2 Replies View Related

Importing Data To An Existing Database Column From An .xsl File Or .cvs File

May 15, 2006

good morning,

 I want to load data that i receive everydays from my customers in .xls file format (excel) or cvs file format,  to the database that i have created on this purpose. but  when trying to do that whith SSIS; i got an error message .... that i can't import redudant data in my database column.

 

Best regards.

View 1 Replies View Related

File Name Dynamically

Apr 11, 2008



I have CSV file as source for SSIS package every time the filename will be changing like trd_1990M1_1990M12.csv

trd_1991M1_1991M12.csv , trd_1992M1_1992M12.csv so it will vary as per user selection . i need to run the same SSIS package to execute the package whichever the file name.

Please let me know the solution for that how to pass the file name dynamically to SSIS package.

View 3 Replies View Related

Reading From A File Which Changes Its Name Dynamically

Oct 7, 2007

Hi,I have a requirement to load data from a flat file.The flat file has a naming convention and thus a timestamp is appended to it. eg : RMRS_07102007_001.delThis makes the file name different each day, Is there any method where by I can provide a dynamic name in the file connection manager.P.S. I tried a variable in For each loop with file enumatror and stored that in a variable , but I was not able to use that variable in File connection manager. pls guide if i was missing anything.Thanks in Advance, Ashish Prasad

View 1 Replies View Related

Dynamically Creating A Excel File In DTS

Feb 13, 2006

Hi
Some one Please guide me... How to create a Excel File Dynamically in DTS.

Once i run my DTS Package the result should be moved to a new Excel sheet( This should be created dynamically).

I tries in this way but it says....Syntax error at the DTS GlobalVariables.....

Function Main()

Dim appExcel
Dim newBook
Dim oSheet

dim oPackage
dim oConn

Set appExcel = CreateObject("Excel.Application")
Set newBook = appExcel.Workbooks.Add
Set oSheet = newBook.Worksheets(1)

oSheet.Range("A1").Value = "au_lname"
oSheet.Range("B1").Value = "au_fname"
oSheet.Range("C1").Value = "phone"
oSheet.Range("D1").Value = "address"
oSheet.Range("E1").Value = "city"


DTSGlobalVariables("fileName").Value = "C:\" & Month(Now) & "-" &
Day(Now) & "-" & Year(Now) & "-" & Hour(Time) & "-" &Minute(Time) & "-" &
Second(Time) & ".xls"

With newBook
.SaveAs DTSGlobalVariables("fileName").Value
.save
End With

appExcel.quit

set oPackage = DTSGlobalVariables.parent

set oConn = oPackage.connections(2)
oConn.datasource = DTSGlobalVariables("fileName").Value

set oPackage = nothing
set oConn = nothing

Main = DTSTaskExecResult_Success

End Function

View 2 Replies View Related

Create File Location Dynamically

Apr 10, 2008

hi,

Can you guys please give me the steps of how to create a destination file dynamically. What i mean is for example i want to get everything from a table and ssis should create a file distination according to today's date and save it in a specific folder.

thanks

View 7 Replies View Related

Dynamically Naming An Excel File

Dec 17, 2007

Hi,

I am trying to import a SQL dataset run through SQL Execute task to an excel file.

I would like the excel file to be generated each night.

My question is

1. How would I dynamically name the excel file to be report_daterun?

2. Also I would like to automatically zip the file up can ssis do this as well?


thanks

View 1 Replies View Related

FLAT FILE SOURCE DYNAMICALLY

Dec 1, 2007

hi all

I have a table which has the paths where the source files need to be collected.

So I have a set of files which I need to collect. So I need to iterate through the result set which retrieves the paths

where the files can be found and set this dynamically to connection string of flat file connection manager.

Can anyone please let me know how to do this.

Can you be a little elaborate as to how we can do this because I am really new to SSIS.

Please help me as I am new to SSIS.

Thanks

Sai

View 1 Replies View Related

Dynamically Creating File Name With Todays Date

Jun 15, 2008

Hi

My SSIS package automatically creates Excel files everyday.
i want the excel files to get todays date when they are created .
Eg:06152008 for today and 06162008 for tomorrow
please help

View 3 Replies View Related

Import More Than One Excel File Using SSIS (Dynamically)

Dec 6, 2007



I have one share folder ,every month end-user will copy & paste excel file into particular share folder. Ok .
Now i have to create new SSIS package as schedule should run every month to find the file and then load automatically into Sql server tables and then move those excel file to another share folder if file successfully loaded only.
The excel file name will be changing every month. but the format wont change. If any body knows this process or steps.
Please share with me .

Thanks in Advance.

View 5 Replies View Related

Change Flat File Source Dynamically

May 21, 2007

I am a relative newbie to SSIS. I have been tasked with writing packages to import data from our clients. We have about 100 clients. Each client has a few different file formats. None of the clients have the same format as each other. We load files from each client each day. Each day the file name changes. I have done all of my current development work with a constant file name in a text file connection manager.



Ultimately we will write a VB application for the computer operator to select the flat file to load and the SSIS package to load it with. I had been planning on accomplishing this thru the SSIS command line interface. Can I specify the flat file to load via a variable that is passed through the command line? Do I need to use a Script Component to take the variable and assign it to the connection manager?



Is there a better way to do this? I have seen glimpses of a VB interface to SSIS. Maybe that is a better way to kick off the packages from a VB app?



Thanks,

Chris

View 9 Replies View Related

Dynamically Create Text File As Destination

Feb 16, 2007

I am trying to create a text file from an SQL query on a SQL table. I would like the SSIS package to prompt for the file name and path. The text file is tab delimited and the text qualifier is a double quote.

Thanks,

Fred

View 9 Replies View Related

Dynamically Exporting Data To Access File

Jun 19, 2007

I am looking a way to export SQL Server 2005 DB tables to Access file dynamically. Like if i have added or removed any tables from SQL DB then when i run SSIS package it should export that table with data to Access file. is there any easy way to do this. if it is not them please someone please tell me which controls should i look at and which technique i should use to do this.

View 4 Replies View Related

Dynamically Changing Configuration File Path

Apr 30, 2008

Before I launch into a long description of the issue, I was wondering whether its possible to dynamically amend the file path held in a configuration string? (I'm looking at the Package Configurations Organizer)

Basically recreating the /CONF switch in the dtexec utility

So a parent package could somehow tell a child package to pull its configuration from a selection of different xml config files at run time... I couldn't see a variable in the child package to set from the former

My only idea at the moment is to replace the Execute Package Task with an Execute Process Task, and directly invoke the child using dtexec & an expression to dynamically set the /CONF switch..

thanks

James

View 3 Replies View Related

DB Engine :: How To Pass CSV File Dynamically From A Path

Jul 14, 2015

I have a query to pass .csv file data into sql table using bulkinsert . I used :

CREATE TABLE #CSVTest
(CouponCode VARCHAR(200))
/* Insertion From csv Source To Table */
BULK
INSERT #CSVTest
FROM 'C:Apple.csv'

[Code] ....

This worked very fine to me. Now, the thing is i should pass the filename as parameter and so that what ever file i specified in execute command , query should take that file and push that file data from .csv file to sql table.

View 2 Replies View Related

How To Dynamically Map The Coloumn To A Flat File Destination?

Jun 5, 2007

Hi All,



I am struck at one point. I am trying to this operation and not able to go further.



1. I have got the dataset to a variable in the control flow.

2. I am looping through the dataset based on a coloumn.

3. Now inside my For each loop i have a dataflow task.

4. In the data flow task i am trying to build a dynamic query using the OLEDB Source and i have selected SQL Command from variable. And the variable build the Query as select * from @othervariable.

Now my question is

Can i send the data of each of the Query resultset to an out put text file using Flat File Connection? If yes pls guide me how? I have tried to create a flat file connection but i am failing how to map the data comming from step 4 dynamically for every query, since every query gives you a different resultset with different coloumns.



Thanks in advance..



Regards,

Dev.

View 15 Replies View Related

Importing A Csv File Using Dts

Jun 22, 2007

 Hi i'v e installed the file: SQLServer2005_DTS.msiBecause i've heard that i need that to import a csv file into a ms sql database. now i have no idea how to work it, like to even make it open.Where would i open it from? I reckon i'v eread enough literature about using it, to be able to have a bash... thanks in advance...  

View 3 Replies View Related

Importing .DAT File

Oct 5, 1999

Hi, this is my first time and was wondering what's the best way to do the following:
i've sql server 6.5. also have a sql database(.DATfile) in a folder.
i need to import this database to my server. should i just LOAD/import it? or do i need to first create a database device on my server(how much size should i allocate etc) and then load it??

thanks a lot

rohit

View 3 Replies View Related

Importing Txt File

Jun 24, 2002

Hi guys,
I have to write a Store procedure which will pickup a txt file from a destination, read it and update some table.
How to pick and read a txt file in a Stored procedure.
Thanks in advance
Vineet

View 1 Replies View Related

Importing .csv File

Jun 11, 2007

Hi,
i am having problems while importing a .csv file.
when i import a .csv file in sql server I get the folowing error message.
"Cannot create an OLE DB accessor. Verify that the column metadata is valid".

Any help would be appreciated

View 3 Replies View Related

Importing A .txt File

Jul 23, 2005

I have a .txt file that I need to add to an existing table, which ismade up of Varchar, Char, numeric and int fields.What is the best way to do it.The first thing I tried was importing the .txt file, and then goinginto the design and changing the field type, but I hit problems on theNumeric fields.Then I tried changing the field types from varchar to different typesat the transfom stage of importing, but that failed too.Regards,Ciarán

View 2 Replies View Related

Importing A CSV File

Nov 22, 2007

I have a CSV file that i need to import into a SQL table. The problemis the values in the first column are not brackited in "". There areover 700K rows. Is there an easy way to fix the data or have SQLcorrectly import the the data?The data looks like this1, "xxx","zzz"2, "aaa","bbb"an so on...Thanks

View 1 Replies View Related

Need Help Importing .CSV File

Sep 19, 2006

Hi all-

I am in need of some help importing a .CSV file into a SQL Server 2005 Express database. I can't use SSIS because it's SQL Server Express. The operation needs to run as a parameterized Stored Proc which I will call from ASP.NET.

The problem is that I need to get the only 2 columns from the text file, then "tack on" 3 more columns that will have data that changes, which will come in as parameters to the T-SQL stored proc. This prevents me from using a straight BULK INSERT.

The data in each of these columns will be the same for each record in that column - that is, every record in every field in column 3, for example, will be the same.

Some of the files will have 4 million records and up, so speed is of the essence here. I tried using BULK INSERT to dump the data into a #temporary table - which took 38 seconds and was acceptably fast - but then, my next step was to dump the additional data into the other columns using UPDATE... SET. I gave up on this after the query ran for THIRTY MINUTES! My next step was going to be to move the data from the temporary table into the target permanent table somehow, but I never got that far when I saw how long the previous step took...

It's a little odd, because I can do the same thing in MS Access in under 5 minutes by using a SQL statement like this in my ASP.NET code:

"INSERT INTO [Target_Table] (field1, field2, field3, field4)" & _
"SELECT F1, F2, '" & strSource & "', Now() AS DateTimeStamp " & _
"FROM [Text;HDR=NO;DATABASE=" & strPath & ";].[" & strFilename & "]"

strSource is a string that the User enters (properly vetted for security); strPath and strFilename are strings holding the path and filename to the .CSV file. I create a unique filename from the file the user uploads. This works in under 5 minutes for several millions of records in MS Access, as I said above. I've had no luck getting anything similar to work in SQL Server, though.





Anyway, does anyone have any ideas? This is somewhat urgent, as the project was about to go "live" when it was discovered that the actual, live data had grown to the point where Access couldn't hold it, and a move to SQL Server Express was necessary.

Thanks in advance,



-Andrew

View 9 Replies View Related

Importing CSV File With URL Using DTS???

Sep 19, 2005

I need to be able to create a DTS package that imports a CSV file which is loated at URL.  I.E.  HTTP://www.url.com/csv/thefile.xls I tried copying the URL an pasting it in the file location when in SQL wazird but I got an error message.

View 1 Replies View Related

Importing CSV File

Jun 4, 2007

I'm a newbie to SSIS, so there is probably an easy solution:



I'm importing from a csv file into a db table, but would like to remove the quotation marks from around the data before inserting into the table.



Are there any suggestions of how to remove the "data" marks? Would a Foreach Loop container work for this?

View 1 Replies View Related

SQL Server 2012 :: Dynamically Grow A Data File?

Dec 5, 2014

Is it possible to dynamically grow a data file for TempDB?

I have some TSQL that I want to execute (this is raw tsql, not something I'd typically share) when I get an alert that TempDB is growing.

SET NOCOUNT ON;
DECLARE @TempDB TABLE (FileID tinyint, name varchar(50), size bigint)
DECLARE @TotalSize decimal(19,4), @TotalFiles int, @CurrentTempDBSize bigint, @SQL varchar(1000), @DBName varchar(50)
SET @SQL = ''
INSERT INTO @TempDB

[code]....

ENDThe code runs fine, but doesn't actually increase the size of the file.

View 7 Replies View Related

Can't I Dynamically Read From Any File? - Connection Manager Does Not Exist

Feb 23, 2007

I have a Parent package and a child package.

I have hard coded the value for the User::FilePath variable in the parent package.

I am mapping this variable to the value of the same variable in the Child Package.

I created a directory and sql file: "C: empsqlb.sql". I have verified that the path variable value is passed to the child package by using a Script task with a messgbox call.

How do I define an execute sql task to execute sql file: @[User::FilePath] & "sqlb.sql". I'm using this expression for the SqlStatementSource property. I have entered the OLEDB server information and specified the SQLSourceType = fileconnection.

However I get the error:

[Execute SQL Task] Error: Connection manager "D:
lewisdevZsqlb.sql" does not exist.

What am I doing wrong?

View 3 Replies View Related

Dynamically Populate Flat File Connection String

Mar 12, 2008

Hi,
I tried to follow the widely talked about method to dynamically populate the connection string property of my flatfileconnection manager from a variable. I keep getting the following non-fatal error.

TITLE: Microsoft Visual Studio
------------------------------

Nonfatal errors occurred while saving the package:
Error at Package [Connection manager "FFCM"]: The file name ""C:ProjectsSSISHLoadTOutputOut.csv"" specified in the connection was not valid.

Error at Package: The result of the expression "@[User::CsvFullFileName]" on property "ConnectionString" cannot be written to the property. The expression was evaluated, but cannot be set on the property.


Here is what I am trying to do.
I have a foreach loop that iterates through a list of xml config files and reads the config information including the destination csv file name and does a data transformation.
So I created a flatfile connection to a csv file did my data mappings.
Created a package level variable to hold the destination file path
In the Flat file conn. manager's properties -> expression -> set the @[User::CsvFullFileName] (which even evaluates fine)

When I try to run the package I keep getting the above mentioned non-fatal error..I checked the path and it is valid. I even tried

the c:\projects\...notation and the UNC path notation...all seem to give the same error


Anyone experience this before ? any thoughts would be appreciated.

Thanks

View 5 Replies View Related

Dynamically Creating SSIS Package For Each Flat File

Apr 18, 2007

Trying to figure out the best method of reading in a number of flat files, all with different number of columns and data types and outputting them to a database.



Here's the problem: They are EBCDIC encoded and some of the columns are packed decimal. I've set up one package that takes the flat file, unpacks the decimal (Using UnpackDecimal component) and then sending the rest through a second component to go from EBCDIC -> ASCII.



What I need is a way to do this for every flat file based on the schema for that flat file. One current solution is to write a script/app to create the .dtsx XML file and then execute that for each flat file. It appears like this may be possible, but I haven't gotten far enough to know for sure. So my questions are this:



1) Is there an easier way to do this (ie somehow feed the schema to the package and use it to dynamically set up the column makers and determine which columns get fed to the unpack decimal component.



2) If there isn't a better way, will dynamically creating the .dtsx XML file based on the necessary input/output columns for each flat file work? If so, what is a good source of information on this (information about how the .dtsx XML file is set up, what needs to be changed/what doesn't, etc).



Thanks,

Travis

View 1 Replies View Related

How To Dynamically Change Source Text File In Sql Server Dts

Sep 25, 2006

Hi ,

I have a table (source file) where all sourse file(its all text file) name and location stored. i have a dts packeage, through this package all the sourse file will upload. So I need to dynamically change the source file name in dts packages.

How i can do it??????

Can any body plz help me out!

Thnaks

SAndipan

View 1 Replies View Related

Dynamically Configure Text File Destination In SSIS

Feb 7, 2008

Good Afternoon,

So this one has been bugging me for a while and I am ready to punt...

Is it possible to dynamically create a text file destination in SSIS and then pump the results of a query stored in a variable to this text file?

So my package looks like this
1) SQL task that pulls back a list of tables to be exported
2) For Each Loop ADO enum that passes the table name to a SQL Task that builds the select...ie select * from <DTS.Variables()>
3) Data flow task that sets the command from variable from step 2
4) Text File destinaiton that is built using a varable as the connectionstring

I am delaying validation in steps3 and 4 above without any luck...basically I am curious if i can even do what I am thinking I should be able to do here...I get as far as getting metadata errors because SSIS can't seem to handle dynamically filling the pipeline with the columns from the variable/SQL statement.

Am I missing something? Is this possible?!

Thanks in advance.
Dave

View 6 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved