I am building a generic SSIS where it takes a text source file and converts to a destination source file . However in the process I want to set the row delimiter and the column delimiter dynamically through the package variable. So that I can use it for any file transfer.
I saw the option for row delimiter in the file connection string property but did not see any column delimiter option.
I am getting an error importing a csv file both using SSIS and SSMS. The csv is comma delimited with quotes for text qualifiers. The file gets partially loaded and then gives me an error stating The column delimiter for column "MyColumn" was not found. In SSIS it gives me the data row which is apparently causing the problem but when I look at the file in a text editor at the specific row identified the file has the comma delimiter and it looks fine. I am using SQL Server 2008.
View 9 Replies View Relateddeclare @var varchar(8000)
set @var='Name1~50~20~50@Name2~25.5~50~63@Name3~30~80~43@Name4~60~80~23'
---------------------
Create table #tmp(id int identity(1,1),Name varchar(20),Value1 float,Value2 float,Value3 float)
Insert into #tmp (Name,Value1,Value2,Value3)
Values ('Name1',50,20,50 ), ('Name2',25.5,50,63 ), ('Name3',30,80,43 ), ('Name4',60,80,23)
select * from #tmp
I want to convert to @var to same like #tmp table ..
"@" - delimiter goes to rows
"~" - delimiter goes to columns
I have the following in my table. As you can see the Geo field is null (which is of type Geography), what I'm trying to do is populate that with an update statement by comparing the latitude and longitude for each row so I can then work out the distance, I've been following this tutorial to workout the distance which is giving me the desired results
[URL] ....
The issue I have is passing in the points i.e latitude and longitude and then updating the Geo field for the row, I currently have 11938 rows in my DB so I need to run this against all of them. I thought I had it I was thinking of update then select from the source but then I realized I need to pass the lat and long in for the points and that's where I got confused
I have a SQL Task that updates running totals on a record inserted using a Data Flow Task. The package runs without error, but the actual row does not calculate the running totals. I suspect that the inserted record is not committed until the package completes and the SQL Task is seeing the previous record as the current. Here is the code in the SQL Task:
DECLARE @DV INT;
SET @DV = (SELECT MAX(DateValue) FROM tblTG);
DECLARE @PV INT;
SET @PV = @DV - 1;
I've not been successful in passing a SSIS global variable to a declared parameter, but is it possible to do this:
DECLARE @DV INT;
SET @DV = ?;
DECLARE @PV INT;
SET @PV = @DV - 1;
I have almost 50 references to these parameters in the query so a substitution would be helpful.
Dan
Is it possible to use a table's fieldname as an SQL variable?
i.e. can the below be made to somehow work:
Code:
SELECT Firstname, Surname, myVariable
FROM ContactDetails
WHERE myVariable = [user input];
- or simply -
Code:
SELECT Firstname, Surname, [user input]
FROM ContactDetails;
---
The "user input" being any other chosen column/fieldname from the ContactDetails table (e.g. Street, City, Postcode, etc.).
i'm using Access and ASP - in case that makes a difference.
any help would be greatly appreciated.
Is there anyway to pass data from a sql database to a variable in a ssis package. I'm trying to get a few fields from a sql database into some variables in my package and send an email using these variables with the send mail task?
Thanks,
I am trying to pass a variable set by a Row count transformation to a derived column transformation. There are actually three separate Row Count transformations storing to different variables. In the derived column trans. the expression i am using is @[User::ClientCount] and the variable type is Int32 and I am passing it to an I4 database column. After the derived column trans. all rows just come out with 0s in the row count columns. Does anyone know why this would be? Please let me know if you need more information.
View 3 Replies View RelatedHi,
I am creating SSIS package which using Execute SQL Task to which I am supplying one .sql file code is
delete from Test where ID = @ID
I defined @ID variable to SSIS and also path where .sql file placed but i am not able to execute this package i am getting error like can not find C:@Path file......
As i got information that passing such variable to .sql you need ADO.net connection so I changed SQLSERVER Database connection string to ADO.net .... after that when i set hard core value for these variable i.e for @ID and @Path then it runs sccessfully but by setting parameter i am getting above error
Any suggestion that will be gr8 help for me
T.I.A
After several hours of trying, I trow the towel in the ring and come here to ask a question.
Source system uses a timestamp column in the transaction tables. which is equal to a non-nullable binary(8) datatype (sql 2000 bol).
What I want to do is get the timestamp at the start of the transfer and at the end of the transfer of data. and store these in a controltable
I try to do this in 2 sql execute tasks:
sqltask 1: "select @@DBTS AS SourceTimestamp" and map the resultset to a variable. Here come's the first problem what variable type to take ?
DBNULL works (meaning it doesn't give errors) (BTW: is there a way to put a variable as a watch when debugging sql tasks ?)
INT64 and UINT64 don't work error message that types for column and parameter are different
STRING works
Then I want to store this variable back in a table of a different data source
sqltask2: "insert into controltable values(getdate(), ?)" and make an input parameter that takes the previous timestamp ...
if I took DBNULL as a type for the variable there doesn't seem to be a single parameter type that works ???
if i take STRING as a type for the variable I have to modify the sql to do the explicit conversion from string to binary so I change CAST(? as binary). It doesn't return any error but the value stored in the table is 0x00000000000 and not the actual timestamp.
Any help on this one ? Why are the INT64/Bigint not working here, you can perfectly do a convert(bigint, timestampfield) in sql ?
How came the SQL datatypes, and the variable datatypes, parameter datatypes are so badly alligned to each other (and all seem to use different names) ?
tx for any help
Dirk
Hi
I have having trouble getting my hands around how to retrieve variables from a parent package. I read about the Environment variables and Configuration File at the parent package level and the Parent Package variable at the child level.
Here are my questions:
1. Can you only store/retrieve 1 variable in a config file at a time?
2. Does the child package have to define the variables and if so, do they have to be the same names as the parent package?
This seems so more more complex then the DTS2000 way of passing variables to and from packages.
Any help would be appreciated.
Thanks in advance,
Gordon Radley
public Sub Main()
Dim url, destination As String
destination = Dts.Variables("report_destination").Value.ToString + "" + "Report_" + Format(Now, "yyyyMMdd") + ".xls"
url = "http://localhost/ReportServer?/ssis_resport_execution/ssis_ssrs_report&rs:Command=Render&ProductID=" + Dts.Variables("ProductID").Value.ToString + "&user_id" + Dts.Variables("user_id").Value.ToString
+ "&rs:Format=EXCEL"
SaveFile(url, destination)
Dts.TaskResult = ScriptResults.Success
End Sub
How to pass more than one variable values in ssis as parameter values to ssrs. With the above code its showing as empty.If i am taking single variable i am able to render the data into excel sheet.
I have defined a variable Var_Query_SQL and passed the below query using expression but it is showing error. where am i going wrong.
"SELECT
sample_id ,
sample_time ,
trans_date ,
product = mh.[identity] ,
comments = s.m_smp_comment
[URL] ...
I've created a varible timeStamp that I want to feed into a stored procedure but I'm not having any luck. I'm sure its a simple SSIS 101 problem that I can't see or I may be using the wrong syntax
in Execute SQL Task Editor I have
conn type -- ole db
connection -- some server
sql source type -- direct input
sql statement -- exec testStoredProc @timeStamp = ?
if I put a value direclty into the statement it works just fine: exec testStoredProc '02-25-2008'
This is the syntax I found to execute the procedure, I don't udnerstand few things about it.
1. why when I try to run it it changes it to exec testStoredProc @timeStamp = ? with error: EXEC construct or statement is not supported , followed by erro: no value given for one or more requreid parameters.
2. I tired using SQL commands exec testStoredProc @timeStamp and exec testStoredProc timeStamp but nothing happens. Just an error saying unable to convert varchar to datetime
3. Also from SRS I usually have to point the timeStamp to @timeStamp and I dont know how to do that here I thought it was part of the parameter mapping but I can't figure out what the parameter name and parameter size should be; size defaults to -1.
Thank you, please help.
I am trying to import data from a text file to SQL database through a DTS package. i am always getting "Column Delmimeter not found" error at record number 8000. i copied the 8000th record into the beginning of the file to test if there is any problem with data but still i got the error at 8000 th record. i know it was a problem before and was fixed with sp1. i installed SP4 and still gettting the same error.
View 4 Replies View RelatedI have flat file source from which data is imported to a Sql table.The target column is int and input column is string .The column has some numeric values and some blank values.when I tried to convert into int values it fails.
View 7 Replies View RelatedAnyone got any Ideas on how to set tax...TXZip as a variable in the following script segment?
Select
RecordID = Identity(int,1,1),
Zip_ZipCode,
Zip_SignatureCode
Into #Temp
from tempstatezip
where State = @State
Delete From tax...TXZIP ------------------------------
While (Select Count(*) from #temp)>0
Begin
Set @rowVal = (Select top 1 RecordID from #Temp)
Id like to set this up in a store procedure so I can pass the state variable to it.....
Having problems setting a linked server as a variable thou.....
Example of what I want.....
Delete from @StateVar
But as a linked server I always get...
declare @StateVar Varchar(11)
Set @StateVar = 'Tax...TXZip'
Select * from @StateVar
Msg 137, Level 15, State 2, Line 3
Must declare the variable '@StateVar'.
Any Ideas?
i have a url that opens my report with no entries in the parameters i then add &Search=96200 at the end of the URL and still the parameter is blank
View 2 Replies View RelatedI have an inline table valued function (function1) where I need to call another inline function which requires 4 TVPs (function2).
Unfortunately, given the limitations of my php calling program I cannot pass a TVP to function1. How can I make that call to function2 (using empty TVPs) from function1 without putting any TVP parameters on function1?
I could do this by creating a multistatement wrapper function that declares the empty tables and passes them to function1, but I was trying to avoid doing this. Is there any other way?
function1:
ALTER FUNCTION [dbo].[function1] (
@ListId INT,
@StartSize INT = NULL,
@EndSize INT = NULL,
@StartDate DATE = NULL,
@EndDate DATE = NULL
[Code] .....
I have created a variable in a SQL Task and assigned it to a string variable. When I debug the container with a breakpoint, I can see the correct date value being assigned to the variable.
I have an ADO Net source setup to an Oracle connection. I need to pull the Oracle data down that has an updated date greater than the updated date in my ODS.
My issue is that the variable is not being passed through to my expression that I use for an ADO Net source.
"SELECT * FROM BI_EDW.GL_JE_HEADERS WHERE LAST_UPDATE_DATE > To_Date('" + (DT_WSTR, 19) @[User::varLastUpdateDate] + "','yyyy-mm-dd hh24:mi:ss')"
I'm in a flat file connection manager editor and I have a flat file where the columns are separated by a space. Does anyone know how to specify a space in the column delimiter option? I've tried {' '} and {s} but these don't work. Not sure what the syntax is for indicating a space. Thanks in advance
View 1 Replies View RelatedI'm posting this because I found this solution after much digging.
The goal here is to incorporate a variable parameter within a OPENQUERY and, ultimately build a dynamic Where clause for use within a OPENQUERY linked server routine. I'm posting because I spent a lot of time trying to get this to work and also, have seen other posts here that hinted it wasn't doable.
First of all - there a good quick article that gets close for FoxPro and possibly works as is for ACCESS:
http://support.microsoft.com/default.aspx?scid=kb;en-us;314520
Here's code for a solution:
DECLARE @OPENQUERY nvarchar(4000),
@TSQL nvarchar(4000),
@FAMILY CHAR(10)
SET @FAMILY='Touring'
SET @OPENQUERY = 'SELECT * FROM OPENQUERY(VFP,'''
SET @TSQL = 'select cov,family,model from vinmast where family='+'['+@FAMILY+']'')'
EXEC (@OPENQUERY+@TSQL)
All shown are single quotes.
In Visual Foxpro, ' ' or " " or [ ] can be used a delimeters
In addition, if wanting to build a dynamic where clause, you could do something like:
SET @TSQL = 'select cov,family,model from vinmast '
IF <some condition met to include FAMILY filter>
Begin
SET @TSQL=@TSQL+'where family=['+@DUTFAMILY+']'''
SET @TSQL=@TSQL+ ')'
End
-----------------
Here's the entire Stored Procedure:
CREATE PROCEDURE dbo.ewo_sp_DUTLookup
(
@DUTPROJECT char(25)=NULL,--Project
@DUTFAMILY char(10)=NULL,--Family
@DUTMODEL char(20)=NULL,--Model
@DUTYEAR char(4)=NULL,--Model Year
@DUTBEGIN char(25)=NULL,--Beginning of COV/DUT number
@DEBUG int=0
)
AS
DECLARE @OPENQUERY varchar(4000),
@TSQL varchar(4000),
@TWHERE varchar(4000),
@intErrorCode int
select @intErrorCode = @@ERROR,
@TSQL='',
@TWHERE=''
IF @intErrorCode=0
Begin
SET @OPENQUERY = 'SELECT * FROM OPENQUERY(VFP,'''
SET @TSQL = ' select dut_pk,cov,family,model,project,modelyr from vinmast '
End
set @intErrorCode = @@ERROR
IF @intErrorCode = 0 and
@DUTFAMILY is not NULL or
@DUTMODEL is not NULL or
@DUTPROJECT is not NULL or
@DUTYEAR is not NULL or
@DUTBEGIN is not NULL
set @TWHERE=' where '
-- Check for Family criteria
If @intErrorCode = 0 and @DUTFAMILY is not NULL and Len(@TWHERE)>0
SET @TWHERE=@TWHERE+' family=['+@DUTFAMILY+'] AND '
set @intErrorCode = @@ERROR
-- Check for Model criteria
If @intErrorCode = 0 and @DUTMODEL is not NULL and Len(@TWHERE)>0
SET @TWHERE=@TWHERE+' model=['+@DUTMODEL+'] AND '
set @intErrorCode = @@ERROR
--Check for Project criteria
If @intErrorCode = 0 and @DUTPROJECT is not NULL and Len(@TWHERE)>0
SET @TWHERE=@TWHERE+' project=['+@DUTPROJECT+'] AND '
set @intErrorCode = @@ERROR
--Check for Model Year
If @intErrorCode = 0 and @DUTYEAR is not NULL and Len(@TWHERE)>0
SET @TWHERE=@TWHERE+' modelyr=['+@DUTYEAR+'] AND '
set @intErrorCode = @@ERROR
--Check for beginning of DUT
If @intErrorCode = 0 and @DUTBEGIN is not NULL and Len(@TWHERE)>0
Begin
SET @DUTBEGIN=RTRIM(@DUTBEGIN)
SET @TWHERE=@TWHERE+' substr(cov,1,'+cast(len(@DUTBEGIN) as char(20))+')=['+@DUTBEGIN+'] AND '
End
set @intErrorCode = @@ERROR
IF @intErrorCode=0 AND substring(@TWHERE,Len(@TWHERE)-3,4)=' AND '
Begin
set @TWHERE=Substring(@TWHERE,1,Len(@TWHERE)-3)
select @intErrorCode=@@ERROR
End
SET @TWHERE=@TWHERE+''')'
IF @debug<>0 and @intErrorCode=0
Begin
print @intErrorCode
print @OPENQUERY
print @TSQL
print @TWHERE
print @OPENQUERY+@TSQL+@TWHERE
End
IF @intErrorCode=0
EXEC (@OPENQUERY+@TSQL+@TWHERE)
GO
Peter
I need to use a value retrieved in one data flow in the second data flow. What's the best way to do this?
How do I copy the column retrieved to a variable so I can use that variable in the second data flow?
Is it possible to assign multiple columns from a SQL query to one variable. In the below query I have different variable (email, fname, month_last_taken) from same query being assigned to different columns, can i pass all columns to one variable only and then extract that column out of that variable later? This way I just need to write the query once in the complete block.
DECLARE @email varchar(500)
,@intFlag INT
,@INTFLAGMAX int
,@TABLE_NAME VARCHAR(100)
[code].....
Hello Everyone,
Hope someone will be able to help me out here.
I have a text file exported from my DTS package and it requires an '!' as a custom column delimiter.
Does anyone have any idea how I can use the '!' mark instead of the Tab or Vertical Bar as my Column Delimiter?
Would appreciate any suggestions.
Thanks,
Kay
Hi,
is there any way to change the delimiter in a FlatFile Source Adapter dynamically? You can do that with the row delimiter but there is no expression for the column delimiter... Is there any other way besides changing the XML code?
Thanks,
hi guys,
i am working on a project witch involves creating packages on-the-fly to import data from txt/csv/xls files according to some definitions on the database.
so far, i have been doing fine.
now we are planning the ASP.net page that enables the customer to define the input file format that will be imported to the system. we want it to have the same listBox as the FlatFileConnectionManager Editor has to define some properties, such as - column delimeiter.
the code to set the column delimiter looks like this:
SSISRunTime.IDTSConnectionManagerFlatFile90 myFilecn = null;
myFilecn = (SSISRunTime.IDTSConnectionManagerFlatFile90)package.Connections["InputFileConnection"].InnerObject;
DtsConvert.ToConnectionManager90(package.Connections["InputFileConnection"]);
SSISRunTime.IDTSConnectionManagerFlatFileColumn90 col
col = myFilecn.Columns.Add(); //.....
string colDelimiter ="|" ; // it actually gets the data from the database... but it is the same thing
col.ColumnDelimiter = colDelimiter;
when we deal with the simple characters- "," , ";" , "|" ... we have no problems with setting the delimiter. but how can i set the delimiter to Tab? or {CR} ? {LF}?
i tried to look at the dtsx- XML , and i see that the column delimiter that is defined when i choose Tab is: _x007C_, but when i try to do something like this:
col.ColumnDelimiter = "_x007C_" ;
it doesn't work. the same happens when I try "{t}" or "Tab".
how do i solve this problem, and enable the user to select Tab as a column delimiter?
Thanks!
Good day everyone,
I have a package that reads data from a CSV file, transforms it and finally loads it in a destination DB table.
My current problem lies in the parsing of the input flat file. I shall illustrate it using a small example.
Source File:
P;Product-1;Short Description for product 1
P;Product-2;Short Description for product 2
Problem:
I configured the flat file connection manager to use semicolon as the column separator. But then I have received some sample flat files where I found that the semicolon might be sometimes used as content of a column data.
Possible Solutions:
I have thought about 3 different solution and I would like to get your feedback and recommendations about them.
Alternative 1:
Use a complex column delimiter, which wouldn't be used in the data.
Example:
P#~#Product-1#~#Short Description for product 1
P#~#Product-2#~#Short Description for product 2
Question 1:
- Is it possible to define such a customized column delimiter for the Flat File Connection Manager?
- If yes, how can I do this?
Alternative 2:
Use double quotes around the data, which the Flat File Source Adapter must somehow recognize and trim before pushing the data down the Data Flow.
Example:
"P";"Product-1";"Short Description for product 1"
"P";"Product-2";"Short Description for product 2"
Question 2:
- Is it possible to configure the Flat File Source Adapter to work as described?
- If yes, how can I do this?
Alternative 3:
Use a Script Component and write the needed code for parsing the Flat File.
Question 3:
- Do you have further suggestions/ideas for solving this parsing problem?
Thanks in advance and my regards,
Samar
My Requirement IS : 1<sup>st</sup>run: if the record does not exist in the table insert the record (file_name, last_modified_file_date) and create a copy in the archive folder with file_name_currentdate.csv
Daily run: retrieve the last_modified_file_date from the input file and check if the retrieved date is greater than the last_modified_file_date in the table:
If true: create a copy of the input file in the archive folder and update the last_modified_file_date in the table with the retrieved date
If false don’t do nothing because the file has been archived in one of the previous runs.I have already retrieving the modified date and File Nae iserting into Filename Table: (That table has 2 columns which are FileName and FileDate) so In script task everytime the variable getting Modified date(retrieve the last_modified_file_date from the input file). How I can Compre the existing table record and variable. I have already imported the all Filenames and Modified into table like below.
I have a flat file that is row delimited by x00 x0D x0A. Any ideas on how to specify the row delimiter in the Columns section of the Flat File Connection Manager?
View 2 Replies View RelatedI have a file with a header row which contains the date of the file and under that all the columns without a header.
In SSIS package I skip row and manually name the different columns.
However, I want to use the column in the header row to store the date value in an SSIS variable and use that variable to write it to our staging table. How can I do this when I skip the header row?
Public Class ScriptMain
Inherits UserComponent
Dim smpid As String
Dim Prdt As String
Dim rcnt As Int64
[code]...
Using the Vb script above I am expecting to read the first row from a flat file source and transferring the data into two variable using script component.
SQL server 2008.
Script task Custom Properties:
Script Task Input Columns:
I get the following errors one after the other:"The collection of variables locked for read and write access is not available outside of PostExecute." "Object reference not set to an instance of an object."
Is there a way to display a column alias as part of the result set column labels?
View 9 Replies View Related