BCP Task And Dynamic Import And Export Of A File Via Package Variables
Jul 3, 2006
I have a requirement to create many SSIS packages and no datatransform is required so the BCP task looks a good contender providing it can do both import & export
is it possible to parse the values in bold as package variables into the BCP task. If so how?
BULK INSERT ipcs_wvg.dbo.extract
FROM 'D:IPCSextract.csv'
WITH (FORMATFILE = 'D:ipcsqueryextract.xml');
Thanks in advance
Dave
I have a global database called ETL Configuration for all my SSIS packages that uses a single table. So I can create three global variables
USE [ETLConfiguration]
CREATE TABLE [dbo].[SSIS Configurations](
[ConfigurationFilter] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValue] [nvarchar](255) COLLATE Latin1_General_CI_AS NULL,
[PackagePath] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValueType] [nvarchar](20) COLLATE Latin1_General_CI_AS NOT NULL
) ON [PRIMARY]
Here is the data I would put in here
ConfigurationFilter = 'MySSISPackageName'
ConfiguredValue = 'D:IPCSextract.csv'
PackagePath = 'Package.Variables[User::gsFileName].Properties[Value]'
ConfiguredValueType = 'String'
ConfigurationFilter = 'MySSISPackageName'
ConfiguredValue = 'D:ipcsqueryextract.xml'
PackagePath = 'Package.Variables[User::gsFormatFile].Properties[Value]'
ConfiguredValueType = 'String'
ConfigurationFilter = 'MySSISPackageName'
ConfiguredValue = 'ipcs_wvg.dbo.extract'
PackagePath = 'Package.Variables[User::gsTableName].Properties[Value]'
ConfiguredValueType = 'String'
--Database connection info
ConfigurationFilter = 'MySSISPackageName'
ConfiguredValue = '.mssql2005'
PackagePath = 'Package.Connections[MyDatabaseName].Properties[ServerName]'
ConfiguredValueType = 'String'
ConfigurationFilter = 'MySSISPackageName'
ConfiguredValue = 'MyDatabaseName'
PackagePath = 'Package.Connections[MyDatabaseName].Properties[InitialCatalog]'
ConfiguredValueType = 'String'
I have looked at lots of options to automaticly create SSIS packages and have a hunch that that simple can be better: All these solutions look way to complex to what I want to achieve--
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=17484&SiteID=1
http://lakshmik.blogspot.com/2005/05/how-to-programmatically-create-ssis.html
Create DataFlow Package Sample
http://msdn2.microsoft.com/en-us/library/ms161541.aspx
SMOTableList Sample
http://msdn2.microsoft.com/en-us/library/ms161564.aspx
View 4 Replies
ADVERTISEMENT
Sep 4, 2006
Hi,
I am not comfortable with DTS 2000 but I need to execute a encapsulated DTS 2000 package from a SSIS package. The real problem is when I need to pass SSIS variables to DTS 2000 package. The DTS 2000 package have 3 global variables that I can identify on " Execute DTS 2000 Package Task Editor - Inner Variables ". I believe the SSIS variables must be mapped on " Execute DTS 2000 Package Task Editor - OuterVariables ". How can I associate the SSIS variables(OuterVariables ) to "Inner Variables"? How can I do it? Much Thanks.
João
View 8 Replies
View Related
Jul 3, 2006
Hi Everyone,
I created a script task that uploads a file to a HTTPs site. The script task works well and now I want to turn this script task into a custom component...
I'm having problems on how to call the variables I have in the package, I already locked them for reading, but how am I suppose to access them and how can I put their values into strings so I can easily call them in the WebClient.UploadFile method?
Also, I am very new to custom tasks in SSIS.. and would appreciate if anyone of you can send me something (code, link, etc.) of a very simple custom task without any form or properties, that I can try to study.
Basically, what I want my custom task to do is,
1. Read the varialbes in the package.
2. use the variables values as input in my WebClient.UploadFile("varURLstring", "PUT", "varFilePath")
as you can see, my custom task is really simple... but I'm a noob here and I don't know where to go.
Thanks in advance,
Kervy
View 1 Replies
View Related
Feb 23, 2006
Hi,
I need to run an SQL command, in which one of the fields is based on the current time, represented as a string in a specific format.
I used a script task to create the string from the current time, and store it in a package variable named "newDate". Later, in the execute sql task, I have the following update statement: "update table_1 set field_1 = ?" (OLE DB connection is used)
and in the parameters mapping I set parameter 0 to the string package variable "newDate".
The problem is that on runtime, the sql executes with the default value of the variable - i.e. it doesn't take the new value.
I checked it and placed another script task after the first one, that simply tries to read the variable "newDate" and print it to screen, but even here the default value is used.
how can I use the new value of the variable?
thanks.
View 7 Replies
View Related
Aug 14, 2007
Hi,
I am having trouble getting the Execute SQL Task to recognize the package-level variables in an SSIS package. The tasks fail execution. The package contains three Execute SQL tasks and none of them receive the variables. However, when I replace the '?' in the code with the variable values they execute fine. These are the steps I have taken thus far:
Made sure the three variables are package level.
Verified I'm using an OLE DB valid connection.
Verified the variables are properly mapped in the parameters mapping section of the task properties.
Tried all 4 types of variable to parameter mapping (even though '?' in the query and '0,1,2..' in the name should work).
Changed the Bypassprepare property to 'True'.
Breakpoints and watch windows to verify the variable values.
I actually tried the simple tutorials on a one-liner SQL statement that uses variables, but had the same problem. I know there must be something simple that I'm missing here, but any info would be greatly appreciated!
Thanks!
View 5 Replies
View Related
May 8, 2014
I created a package from the SQL Server Import and Export Wizard.I can run this .dtsx file " Import_Table.dtsx " on my local machine no problem, which then executes the package on the remote server of my hosting company.Do they need to install SQL Server on their machine for it to work?Is there some files they can download to make it work?
View 3 Replies
View Related
Apr 17, 2008
Hi,
First post, new to SQL Server, the usual apologies apply ;-)
I used the Import/Export Wizard in SSIS to define a source and target for data migration. I accidentally omitted ticking off a box to force the drop and re-creation of the target table. I can't find any way to edit a package, once I've finished with the wizard! If someone would point me in the right direction, I'd appreciate it.
Thanks,
Chap
View 4 Replies
View Related
Mar 28, 2008
Hi, I have one package with a complex Control Flow. I need to import a dataflow from another package into it. Cut and Paste is working but it looks like my package corrupts after that. (for example, login to database has problem with authentication now)
Is there any import or export tool for this?
Regards
View 5 Replies
View Related
Apr 28, 2004
I have a package (Package1) that is run from another package (Package2) via a Execute Package Task. I set a Global Variable called sErrorMessage in the in Package1 and would like to access that Global Variable in an ActiveX Script Task in Package2. How can I do this?
View 6 Replies
View Related
Feb 13, 2007
Hi,
Let's say I have a package taking as parameter "InvoiceID". I want to execute this package as a child in another package. The parent package gets the list of invoices to produce and calls the child package for each entry of the list.
How do I pass the InvoiceID to the child? I know I can use the parent's variables from the child but I don't want the child package to be dependant on the parent package. For example, I might want to execute the "child" package as a stand-alone in development (providing it with a predefined InvoiceID). I might also want to call the same child package from another parent package with other variable names.
What I would like to do is "push" the value instead of "pulling" it. I know it's possible using the command line and the /SET option (ex.: /SET Package.Variables[InvoiceID].Value;' 184084)... Is it possible using the Execute Package Task?
Thanks
View 4 Replies
View Related
Feb 1, 2007
Hi,
I'm trying to simplify the deployment process of my project. I already had some troubles with the config files but lets say I solved that issue. I'm going to read a flat file and set the variables of my packages from this file. I was thinking to use a Script Task to do that but I will need to copy this task in every package (I have at least 30). So if I want to make some change this will be painful.
Then, I came up with the idea of creating a Custom Task called Config File Task. I'm working on this but I got stuck trying to get the variables from the package that is running my Config File Task.
This is the code I had in the Script Task:
Dim streamReader As New StreamReader(Dts.Variables("ConfigFilePath").Value.ToString)
Dim line As String
Dim lineArray As String()
Dim variableName As String
Dim variableValue As String
Dim readConfigurations As Boolean = False
While (streamReader.Peek() <> -1)
line = streamReader.ReadLine()
If line = "[CONFIGURATIONS]" Then
readConfigurations = True
ElseIf line = "[/CONFIGURATIONS]" Then
readConfigurations = False
Else
If readConfigurations And line <> "" Then
lineArray = line.Split("|".ToCharArray())
variableName = lineArray(0).Trim()
variableValue = lineArray(1).Trim()
If Dts.Variables.Contains(variableName) Then
Dts.Variables(variableName).Value = variableValue
End If
End If
End If
End While
Dts.TaskResult = Dts.Results.Success
All I want to do is set the variables that exists in each package from the config file. In my UI Class (ConfigFileTaskUI.cs) I can have access to the variables via the TaskHost which is passed as an argument of Initialize() method.
Any thoughts? I'd really appreciate some help!
P.S. I've been working on this for 2 days!
View 11 Replies
View Related
May 29, 2007
Hi,
My Warm Wishes to All !!
I ve created a Import Export Package by right clicking the specified Database and Save it as SQL Server Type thru the wizard and execute immediately , and it works . But I dont know where it get saved and cant able to see so that I could be able run once more. More over I want to schedule the package for that what I have to do ??? Plz help me
Thanks
Aravind
View 2 Replies
View Related
Dec 17, 2007
Hello All,
I am a newbe at MS SQL 2005, so if this has already been answered elsewhere, please just point me in the right direction.
I have successfully used the Import/Export wizard in "execute immediate" mode to import a table from Oracle 10g to MS SQL 2005. Works like a charm, but when I take the saved pacakge (saved as a *.dtsx server side file) - with no changes to the package and execute it from the either the "dos" prompt or from Windows explorer it fails at the login to Oracle step. Yet when I check the package's "Connection Managers" source connection, all the settings, user-ids and passwords look fine. The package does indeed execute, but I receive an ORA-01017:invalid username/password within the trace file of the package, but nothing has been changed from the Import/Export wizard which ran beatifully, thus verifying the appropriate acccess rights to Oracle.
Any advice would be greatly appreciated.
Doug
View 7 Replies
View Related
Oct 13, 2006
Hello all,
I have been struggling trying to read and/or write package level variables from within my custom task. I'd like to be able to get and set values from within the Execute method of my custom task. I have searched this forum and the books online and can't seem to find the answer. I thought maybe I could use an expression on my task (mapping the package variable to a custom task public property) but that doesn't seem to be working for me. I also would have thought I could use the VariableDispenser object from within my task but the collection is empty. I have 3 package level variables configured and can't seem to find a way to access them (with intentions of getting/setting). Could someone point me to a good doc or provide an example that may accomplish this? Thanks!
(I'm using package level variables as a means of passing simple information between tasks that are not using a DB, if there is a better way I'm open to suggestions.)
Jay_G
View 3 Replies
View Related
Nov 7, 2007
Hi,
Is there anyway where i can pass the package name to an "execute package task" and run it through loop. My loop task will pass the package name which is assigned on a variable and the execute package task should pickup the name and run, is this possible, if yes how?
View 3 Replies
View Related
Dec 15, 2005
hello ,
I want to download data from an AS/400 to a SQL 2005 Database.
Within Sql server 2000 it was possible to do more than 1 transformation with just 1 source connenction and 1 destination.
In SSIS I have created:
- new solution
- data source
- Data source viewer ( I made named queries to format the data as i want it to be in Sql)
Although it is possible to view the named query data it does not seem to be possible to import the data in the data viewer into my sql 2005 database. (For each dataflow I have to define each query again)
I have added the data source in the connection manager.
Also I am searching to import more than 1 file in 1 package !
I can use however a Dts-2000 package but why then use SQL server 2005
Hope somebody can help me
Regard,
Ronny
View 7 Replies
View Related
Apr 25, 2007
I have a 'file system task' moving files from one server to another for processing. I have defined the path and filename as separate variables. When I attempt to pass them together into the task I receive an error.
Source Path & Filename:
@[User:: DataSourceFolder]+"\"+ @[User::CD_PaidClaimSource]
Outputs:
\umrdwh2FTPCLAIMDAILY.DATA
Destination Path Now: (Filename not specified per another thread.)
@[User::WorkingFolder]
Original Destination Path: (Which did not work.)
@[User::WorkingFolder]+"\"+ @[User::CD_PaidClaimSource]
I receive validation errors:
Error 1 Validation error. CLAIMDAILY Move : Failed to lock variable "\umrdwh2FTPCLAIMDAILY.DATA" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.". ClaimLoading_MASTER.dtsx 0 0
How do I overcome this without hard coding the path in the task?
View 11 Replies
View Related
Oct 9, 2006
I am having an issue with the File System Task.
I was wondering if there is a way to 'Move File' with the File System Task inside of a For Each Loop container but to dynamically set the Destination path variable.
Currently, this is what I have:
FileDestinationPath variable - set to C:TestFiles
FileSourcePath variable - set to C:TestFiles
FileNameAndLocation variable - set to blank
For Each Loop Container €“ Iterates through a folder C:TestFiles that has .txt files in it with dates in the file name. Ex: Test_09142006.txt. Sets the file path (fully qualified) to the Variable Mapping FileNameAndLocation.
Script Task (within For Each Loop, first step) €“ Sets the FileDestinationPath to the correct dated folder within C:TestFiles. For example, if the text files I want to move are for the 14th of September, it takes FileDestinationPath and appends the date folder to the end of it. The text files have a date in the file name (test_09142006.txt) and I am picking this apart (from FileNameAndLocation in the For Each Loop) to get the folder date. (dts.Variables(€œUser::FileDestinationPath€?).Value = dts.Variables(€œUser::FileDestinationPath€?).Value & €œ€? Month & €œ_€? & Day & €œ_€? & Year & €œ€?) which gives me €œC:TestFiles 9_14_2006€?.
File System Task (within For Each Loop, second step) €“ This is where the action is supposed to occur. I want it to take the FileDestinationPath and move the FileNameAndLocation file (from the For Loop) into this folder for each run.
Now as for my problem. I want this package to run everyday but it has to set the FileDestinationPath variable dynamically according to that day€™s date. Basically, how do I get this to work since I can€™t hard code the destination path variable from the start? I have the DestinationVariable on the File System Task set to the FileDestinationPath variable, after the script task builds it. However, using FileNameAndLocation as the SourceVariable on my File System Task tells me that the €œVariable €œFileNameAndLocation€? is used as a source or destination and is empty.€?
Let me know if I need to clarify further€¦...I may be missing something very simple. Any help would be greatly appreciated!
View 10 Replies
View Related
Aug 25, 2007
Hi frnz ,
i want to know how to import a CSV File and export CSV File From database SQLSERVER-2005 Using ASP.NET With C#
IF any one knows Help me
Vinnu
View 1 Replies
View Related
Mar 4, 2014
I've got a package in SSIS 2012 that has an Execute SQL task in the control flow level.
The SQL in question does an Upsert via the SQL merge statement. What I want to do, is return the count of records inserted and records updated (No deletes going on here to worry about). I'm using the output option to output the changed recs to a table variable.
I've tried returning the values as:
Select Count(*) as UpdateCount from @mergeOutput where Action = 'Update'
and
Select Count(*) as InsertCount from @mergeOutput where Action = 'Insert'
I've tried setting the resultset to both Single rowset and Full rowset, but i'm not seeing anything returned to the package variables I've set for them (intInsertcount and intUpdatecount).
View 2 Replies
View Related
May 25, 2006
I'm using a DTS package to copy data from a comma delimited text file into a datatable. The text file has close to 160,000 records. The transfer fails at record 120,000. The error message says "Column Delimiter not found". The particular row in the .csv file seems to have nothing wrong with it as far closed quotes and missing commas are concerned. Even if I delete the row, the error is thrown at the same row.
Microsoft has acknowledged this as a bug in SQL Server 2000 and has a service pack to fix it. But installing SP4 is not an option for me at this time. The workaround on Microsoft websire about adding a character or a line didn't work for me.
Is it possible that there's some character or a marker in the text file that stays even if I delete the row? I don't know how to fix this problem. Does anyone here knows a workaround?
View 7 Replies
View Related
Feb 27, 2008
Hi All,
I have a source files folder where the files generated everyday.
My goal is pick the latest file and copy this single file to another folder.
I used the Foreach loop container and got the latest file and stored the file name to a varible i.e. LatestFile
Then i want to use the File System Task to copy this to the destination.
On the beginning, I could not setup the Latestfile since I don't its name then, so when I setup the Source Connection property of the File system task, it is not allowed to leave the SourceVarible as blank!
Any suggestion?
Thanks
Micor
View 3 Replies
View Related
Apr 23, 2015
Where is a package visible when running the Data Import/Export wizard, choosing to save a package, and choosing "SQL Server" as the location? When I make an SSIS connection in Management Studio I do not see the package under the "MSDB" node.
View 4 Replies
View Related
Sep 26, 2006
How would I import a dynamic file name into a known table?I have the file name in a variable. So what object in SSIS do I use to do this. Thanks
View 2 Replies
View Related
Jun 12, 2008
Reading through the forums, I found some great imformation for importing/exporting an excel spreadsheet via a stored procedure, however, the amount of data I have won't fit in excel. Can someone help me Import a CSV file via a stored procedure? and Export a CSV file via a stored procedure? I don't know if XML is the answer or if there is another way. For many reasons, I don't want to use DTS or Bulk. We are currently using SQL Server 2000.
Thank you,
Stacy
Thanks,
Stacy
View 1 Replies
View Related
Oct 11, 2006
Firstly, I hope this question isn't asked too frequently but I found no existing reference to this situation....
I had a bunch of stored procedures in SQL 2k which imported and exported data to and from flat files using TEXTPTR, READTEXT, UPDATETEXT etc... The flat files were continuously changing so the filepath was a parameter for the sp.
The reason I used the pointer to flat files is because I didn't want to
load the files in memory before commiting them ie. with TEXTPTR and
UPDATETEXT I can import a 1Gb binary file 80000 bytes at a time and
keep (precious) memory usage down.
I was accessing this procs from a C# application.
Since these methods are going to be phased out by the guys at MS what is the best way of importing/exporting very large binary files in SQL 2005?
As far as I can tell SSIS requires a Flat File Source Manager object which needs a static filepath - not good.
Hope you can help,
Paul
View 10 Replies
View Related
Jan 9, 2008
Hi,
I need to export some data from SQL 2005 to a flat file, The data and flat file names will be dynamic and will be be fired programaticaly so I can't use DTS or SSIS.
In SQL2000 I did it using bcp, but that's quite a security hole so I don't want to use external utilities. I'll need to do something similar on another machine to import the data as well.
I find it strange there's no easy way to do this!
Thanks.
View 4 Replies
View Related
Dec 24, 1998
I needs export data on table to text file so I can process this data
with another database engine ie. Informix.
Can anybody help me to solve this problem ?
View 2 Replies
View Related
Jun 13, 2006
i am handling a project where user can choose the excel file and the field in the excel file to export into sql server 2005. which mean there will be dropdownlist where the user can choose the field and so on. anyone know how to do this?
View 2 Replies
View Related
Feb 5, 2007
Hi,
I'm trying to import text file (generated by UNIX - collation ISO LATIN 2) into the database using SQL SERVER 2005 Import and Export Wizard. I have got a problem with importing a decimal number, because in that column are not only decimal numbers (that's OK), but there are also spaces (not null, the column is filled by spaces and it looks like | |). When I'm trying import that file, then will occur the problem of truncation and import stops.
I can import that data by BULK INSERT, but I would like to import it by Import and Export Wizard at once without using subsequent conversions.
View 2 Replies
View Related
Aug 29, 2006
Just attempting to import a simple tab delimited text file into my SQL Server 2005 database using the SQL Server Import and Export wizard. Column names are specified within the first line of the file. The Header Rows to Skip field value is listed as 0, but the wizard indicates that "The field, Header rows to skip, does not contain a valid numeric value".
Why isn't zero (0) a valid numeric value? I don't want to skip any rows. PLUS, I get the same error when trying to export to a text file although the header rows to skip field does not exist. I can increase the number to 1 or more, but the wizard will skip part of my data .. unacceptable.
What am I missing here? I installed SP1 of SQL server 2005, but that did not help.
Thanks in advance.
View 1 Replies
View Related
Feb 20, 2008
Hi,
I have a DTS package scheduled to run every day creates a text file on Server A and then moves the text file from Server A to Server B. But recently, the package is moving incomplete text file from Server A to Server B. Not every day. There is no problem in generating the text file.
So far, I verified the memory on Server B - No Probelms.
Changed the package so it copies the text file from Server A to Sever B instead of moving, then delete the text file on Server B. - Didn't work out.
Any idea?
View 3 Replies
View Related