Is there way to rename parameters Param_0, Param_1 in OLEDBCommand transformation? I am trying to create table driven packages using BIML. I am using OLEDBCommand Transformation to update rows. But since, I will not be sure of how many parameters and order of the parameters, I was planning to rename the parameter programmatically, so that accordingly I can build the update statement and add filter condition.
View 1 RepliesHi,
I have application connected to MS Access DB using OleDB. When creating commands (Insert/Update/Select) I use OleDbParamater class to insert data into command. Examples :
Select ::
OleDbCommand select_cmd = new OleDbCommand("SELECT * FROM " + ObjectTable.TableName + " WHERE " +
ObjectTable.idObject + "=@" + ObjectTable.idObject + " AND " +
ObjectTable.idObjectUnder + "=@" + ObjectTable.idObjectUnder);
Update ::
OleDbCommand update_cmd = new OleDbCommand("Update " + ObjectTable.TableName + " SET " +
ObjectTable.idParent + "=@" + ObjectTable.idParent + " , " +
ObjectTable.idParentUnder + "=@" + ObjectTable.idParentUnder + " , " +
ObjectTable.License + "=@" + ObjectTable.License + " , " +
ObjectTable.Type + "=@" + ObjectTable.Type + " ," +
ObjectTable.Language + "=@" + ObjectTable.Language + " , " +
ObjectTable.Name + "=@" + ObjectTable.Name + " , " +
ObjectTable.Checksum + "=@" + ObjectTable.Checksum + " , " +
ObjectTable.VText + "=@" + ObjectTable.VText + " , " +
ObjectTable.VInt + "=@" + ObjectTable.VInt + " WHERE " +
ObjectTable.idObject + "=@" + ObjectTable.idObject + " AND " +
ObjectTable.idObjectUnder + "=@" + ObjectTable.idObjectUnder);
Parametes:: (Adding in separate method -> AddParameters(OleDbCommand command); )
command.Parameters.Add("@" + ObjectTable.idObject, OleDbType.BigInt).Value = this.IDUpper;
command.Parameters.Add("@" + ObjectTable.idObjectUnder, OleDbType.BigInt).Value = this.IDUnder;
command.Parameters.Add("@" + ObjectTable.Name, OleDbType.VarChar).Value = this.Name;
command.Parameters.Add("@" + ObjectTable.idParent, OleDbType.BigInt).Value = GetUpper(this.IDParent);
command.Parameters.Add("@" + ObjectTable.idParentUnder, OleDbType.BigInt).Value = GetUnder(this.IDParent);
command.Parameters.Add("@" + ObjectTable.License, OleDbType.BigInt).Value = this.License;
command.Parameters.Add("@" + ObjectTable.Language, OleDbType.BigInt).Value = this.Language;
command.Parameters.Add("@" + ObjectTable.Type, OleDbType.BigInt).Value = (int)this.Type;
command.Parameters.Add("@" + ObjectTable.VText, OleDbType.VarChar).Value = String.IsNullOrEmpty(this.VText) ? null : this.VText;
command.Parameters.Add("@" + ObjectTable.VInt, OleDbType.BigInt).Value = this.VInt;
command.Parameters.Add("@" + ObjectTable.Checksum, OleDbType.BigInt).Value = this.Checksum;
Question: Does the order of adding parameters to command matter? Because allways when the order of parameters added is diffrent from order in command text, I get weird Exceptions . I thought that the name matters, not the order, but it seems that system doesn't care about the parameter's name, it just picks next parameter in command.Parameters when putting values. How is it?
We are using the cache transformation in our project , while doing the cache transformation our disk space goes to 0 MB free and SSIS package execution not completes even after 3 hr..Initially we have around 34 GB free space on C: drive .Our server configuration is 64 RAM. We are caching the data from table which contains around 21 million records.We changed the path in properties (“BLOPTempStoragePath”,”BufferTempStoragePath”) of Data Flow task of SSIS in which we are using Cache Transformation.
View 6 Replies View RelatedUsing the following:
SQL Server: SQL Server 2012
Visual Studio 2012
I have created an SSIS package where I have added an Execute SQL Task to run an existing stored procedure in my SQL database.
General:
Result Set: None
Connection Type: OLE DB
SourceType: Direct Input
IsQueryStoredProcedure: False (this is greyed out and cannot be changed)
Bypass Prepare: True
When I use the following execute statement where I am "Hard Coding" in the parameters, the stored procedure runs successfully and it places the data into the table per the stored procedure.
SQLStatement: dbo.sp_ml_location_load @system_cd='03', @location_type_cd=Store;
However, the @system_cd parameter can change, so I wanted to set these parameters up as variables and use the parameter mapping in the Execute SQL Task.
I have set this up as follows and it runs the package successfully but it does not put the data into the table. The only thing I can figure is either I have the variables set up incorrectly or the parameter mapping set up incorrectly.
Stored procedure variables:
ALTER PROCEDURE [dbo].[sp_ml_location_load]
(@system_cd nvarchar(10), @location_type_cd nvarchar(10))
AS
BEGIN .....................
Here is my set up, what is wrong here:
I Created these Variables:
Name Scope Data Type Value
system_cd Locations String '03'
location_type_cd Locations String Store
I added these parameter mappings in the Execute SQL Task
Variable Name Direction Data TypeParameter NameParameter Size
User::system_cd Input NVARCHAR@system_cd -1
User::location_type_cd Input NVARCHAR@location_type_cd -1
I used this SQLStatement: EXEC dbo.sp_ml_location_load ?,
It runs the package successfully but it does not put the data into the table.
I'm after running into something in SSIS 2012 that I fail to grasp.
I have a package that provides a service to other packages. In order to provide that service it needs 4 parameters provided by the caller. So naturally I'm thinking I make those 4 parameters 'required'.
The caller uses Execute package task and provides the 4 parameters on the parameter mapping tab.
Yet the packages fails with the error message that one or more required parameters weren't provided.
Using the following:
SQL Server: SQL Server 2012
Visual Studio 2012
I have created an SSIS package where I have added an Execute SQL Task to run an existing stored procedure in my SQL database.
General Tab:
Result Set: None
Connection Type: OLE DB
SourceType: Direct Input
IsQueryStoredProcedure: False (this is greyed out and cannot be changed)
Bypass Prepare: True
SQL Statement: EXEC FL_CUSTOM_sp_ml_location_load ?, ?;
Parameter Mapping:
Variable Name Direction Data Type Prmtr Name Prmtr Size
User: system_cd Input NVARCHAR 0 10
User: location_type_cd Input NVARCHAR 1 10
Variables:
location_type_cd - Data type - string; Value - Store (this is static)
system_cd - Data type - string - ??????
The system code changes based on the system field for each record in the load table
Sample Data:
SysStr # Str_Nm
3 7421Store1
3 7454Store2
1815061Store3
1815063Store4
1615064Store5
1615065Store6
1615066Store7
7725155Store8
STORED PROCEDURE: The stored procedure takes data from a load table and inserts it into another table:
Stored procedure variables:
ALTER PROCEDURE [dbo].[sp_ml_location_load]
(@system_cd nvarchar(10), @location_type_cd nvarchar(10))
AS
BEGIN .....................
This is an example of what I want to accomplish: I need to be able to group all system 3 records, then pass 3 as the parameter for system_cd, run the stored procedure for those records, then group all system 18 records, then pass 18 as the parameter for system_cd, run the stored procedure for those records and keep doing this for each different system in the table until all records are processed.
I am not sure how or if it can be done to pass the system parameter to the stored procedure based on the system # in the sys field of the data.
Hi Gurus,
I have a Dataflow Task which has an OLE DB Source calling a SP with parameters (?, ?). Then this OLE DB Source is conencted to a Lookup Transform which also calls a SP but on a different database. I am unable to figure out how to pass parameters in a Look up Transform.
In the 'Use Results of an SQL Query' pane of Lookup Transform:
Code SnippetEXEC GetMonthlyDataExtract 4, 2007
( I am passing month and year values) this works ok.
But when I chage to
Code SnippetEXEC GetMonthlyDataExtract ?, ?
It says EXEC not supported. Also I can not figure out how to configure parameters since 'Reference Table' Tab of the Lookup Transform does not have any option where we can attach variables to parameters.
Also I am interested to map parameters to variables not to input columns.
If mention if that is not possible or any other alternative.
Your help will be appreciated.
Thanks,
Paraclete
Hi there,
In order to prevent lookup errors in a lookup transformation, I've decided to go for an OleDb Command Transformation.
This transformation should check the lookup and, if it turns out to be null, ir returns a dummy value. Otherwise, it would return the lookup value.
This should be done by doing something like this:
select coalesce( (select ID_Table2 from ID_Table2 where FK_Table1 = ?), 0)
suposing Table2 has an atribute called "FK_Table1" that should match a column in the data flow.
Now, such command result in this message:
"An OLE DB record is available. Source "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Syntax eror, permission violation, or other nonspecific error".
But, it I remove the coalesce and type the following command:
select ID_Table2 from ID_Table2 where FK_Table1 = ?
It presents me no errors and allows me to continue.
Did i did anything wrong or is this something that is not possible to be done?
I know i have the option to use a script task to do this operation, but that would turn the maintenance process a little more difficult.
Otherwise, i know i could also re-direct the error from the lookup transformation and handle it. Though, my package has about 10 lookups and that would turn my package a lot more complex than
Thanks in advance
Best Regards
André Santana
Hi,
When I execute a stored procedure from an OLE DB Command transformation, where the sp takes a parameter and RetainSameConnection=TRUE and DelayValidation=TRUE are set, I get the error
"Syntax error, permission violation, or other nonspecific error"
If I take out the param or set RetainSameConnection=FALSE on the connection, all is fine again?
Has anyone has come across this?
Cheers
Hello,
I'm using an OLEDB Command in a DataFlow which performs a parametric query to update thousands of rowsets but it is very slow.
Is there an alternative ?
What does Invalid object name "MyTable" mean?
I checked the table and column names. They are both correct.
Will the error mean something else, e.g. wrong data type, or no data?
TIA,Jeffrey
Dim strConn As String = ConfigurationManager.ConnectionStrings("MyConnectString").ConnectionStringDim oConn As New OleDbConnection(strConn)Dim oDBCommand As New OleDbCommand("MyStoredProceduret", oConn)oDBCommand.CommandType = CommandType.StoredProcedureoDBCommand.Connection.Open()Dim rtnValue As String = oDBCommand.ExecuteScalar() This is the error source file
This is the SPCREATE PROCEDURE MyStoredProcedure ASSELECT SettingsReqSchdTimeout FROM SettingsGO
I have begun using SSIS and I am a little taken aback by the complexity of it especially since I just want to do a simple data transformation such as in DTS.
Are there any tutorials for data transformation for SSIS on the web/this forum and what if I want to do a simple transformation from Access to SQL Server?
Hey all - got a problem that seems like it would be simple (and probably is : )
I'm importing a csv file into a SQL 2005 table and would like to add 2 columns that exist in the table but not in the csv file. I need these 2 columns to contain the current month and year (columns are named CM and CY respectively). How do I go about adding this data to each row during the transformation? A derived column task? Script task? None of these seem to be able to do this for me.
Here's a portion of the transformation script I was using to accomplish this when we were using SQL 2000 DTS jobs:
'**********************************************************************
' Visual Basic Transformation Script
'************************************************************************
' Copy each source column to the destination column
Function Main()
DTSDestination("CM") = Month(Now)
DTSDestination("CY") = Year(Now)
DTSDestination("Comments") = DTSSource("Col031")
DTSDestination("Manufacturer") = DTSSource("Col030")
DTSDestination("Model") = DTSSource("Col029")
DTSDestination("Last Check-in Date") = DTSSource("Col028")
Main = DTSTransformStat_OK
End Function
***********************************************************
Hopefully this question isnt answered somewhere else, but I did a quick search and came up with nothing. I've actually tried to utilize the script component and the "Row" object, but the only properties I'm given with that are the ones from the source data.
thanks in advance!
"Lord, beer me strength."
hi all,
My issue is the inner join transformation in SSIS.
See i ll explain my problem clearly now.....
Actually i m just checkin if the inner join performed in business intelligence studio usin the inner join transformation and the inner join performed in the management studio using queries are same. Logically both the resultset should match isn't but in my case it is not so. It is very important for me to figure out where the problem is because i m goin to use lotsa inner join transformations in my current project.
I ll appreciate if someone can help me to figure out this problem. May be you can also tell me the detailed steps in adding the inner join transformation and also how it works.
Thank you,
I have the following query:
SELECT EMPID,EMPNAME from EMPLOYEE
where EMPID = (SELECT MAX(EMPID) from EMPLOYEE group by EMPNAME,insert_date)
Here one can use above query in Dataflow of SSIS and specify SQL to create temporary table and later can use as lookup to join to other table.
Is there any way in SSIS to directly do the MAX of EMPID in lookup and join to the main source table.
Any help is really appreciated.
Thanks.
I have a flatfile source to which different flatfiles will be passed as input,this is connected to an OLEDB destination which changes along with the sourcefile.
But when the new file is given as input, the OLEDB mappings are not getting refreshed.It is showing an error.
Actually this was implemented in DTS, and they have used an activex script for the transformation.
what shd I use in SSIS?
Please help me..
Sharmada
Hi all,
I've got to change values in my source database as follows:
Source: Target:
X 1
Y 1
Z 2
Can I create a lookup table and us a look up task in SSIS to do this or do I need to script it?
Thanks
F
I'm not exactly sure how to ask this question ... but here goes!!
I want to get an idea of how SSIS actually executes transformation tasks.
Do transformation tasks (eg a lookup) complie down to managed code or are the executed as SQL commands in a SQL server database?
Thanks.
Hi
I have migrated a DTS that had some activeX transformation tasks within data pump flow tasks.
Those parts were migrated as "DTS 2000 tasks" .. so activeX transformation tasks aren't possible in SSIS ?
I know ActiveX script tasks are but for transformations ?
1. IF i leave these Encapsulated DTS 2000 tasks in the migrated SSIS package, will it run independently of the original
DTS or does it need the old DTS running to "call" that part from ? (I hope im making sense here)
is it possible to load this functionality internally into the new SSIS ?
2.
How could I (if i can't do ActiveX transformation tasks) achieve this is SSIS ?
can I achive this using the script tasks in SSIS ?
Hello All,
Can someone help me out in providing the STEPS to solve this problem. My scneario is, I've a table which has got 2 fields and 5 default row values have been filled in. Now, using the above, duirng package runtime, it need to dynamically create additional field and has to store values like for.e.g (0001 America). I'm getting the following error while executing the ssis package.
1. [DTS.Pipeline] Warning: Component "Derived Column" (1170) has been removed from the Data Flow task because its output is not used and its inputs have no side effects. If the component is required, then the HasSideEffects property on at least one of its inputs should be set to true, or its output should be connected to something.
2. [DTS.Pipeline] Warning: Source "OLE DB Source Output" (87) will not be read because none of its data ever becomes visible outside the Data Flow Task.
Please suggest with your valuable solution at the earliest.
Thanks
Vaiydeyanathan.V.S
HI,
How do I loop through all columns in a row using a script
transformation? For example if I want trim all columns.
If I want to trim one column this is a simple script:
Public Class ScriptMain
Inherits UserComponent
Public Overrides Sub MyAddressInput_ProcessInputRow(ByVal Row As
MyAddressInputBuffer)
Row.City = Trim(Row.City)
End Sub
End Class
But what if I want to do that for all columns? I don't want to name
them all like this:
Public Class ScriptMain
Inherits UserComponent
Public Overrides Sub MyAddressInput_ProcessInputRow(ByVal Row As
MyAddressInputBuffer)
Row.Column1 = Trim(Row.Column1)
Row.Column2 = Trim(Row.Column2)
Row.Column3 = Trim(Row.Column3)
...
...
Row.Column997 = Trim(Row.Column997)
Row.Column998 = Trim(Row.Column998)
Row.Column999 = Trim(Row.Column999)
End Sub
End Class
Is there a simple foreach column in Row.columns option?
-- Joost (Atos Origin)
I just ran across an interesting problem, that makes no sense. I
built
an SSIS package that updates a column, using an transformation
script.
Testing in Debug mode everything runs perfectly, but when I have SQL
sever agent run the package it insert null into the new column.
Any thoughts or suggestions would be greatly appreciated.
r/ Anthony
First let me say, I really can't believe this chain of events myself--and they are happening to me.
I am upgrading several DTS packages to SSIS on what will be my new production server. These packages create tables, export them to a flat file, and ftp them off to other locations.
What is happening (on the SSIS side) is that the OLE DB Source is reordering some of the columns on its own (moving them to the end of the table/file. Then when my pickup/load routines run, the data is out of place and they fail.
Can anyone please explain what is happening here with the mapping. I have evaluated the table and the columns are in the order that I expect. When I preview the source table in the OLE DB Source Editor the columns are in the correct order/alignment, but when them in the OLE DB Source Editor --Columns section within BIDS the order is changed arbitrarily.
I have been somewhat successful (2 out of 3) in being able to re-map the data, but this last table just doesn't want to change.
Thanks in advance for any help and/or information you can provide
Dear all,
Ive made a SSIS package which might take source columns from a plain text file and copy them to the Sql table.
A long time ago, when you did the process I did by dts and that stuff included a pump task which had ActiveX Script transform column with VbScript stuff inside so that, how do I for to do the same with SSIS??
Ive got a couple of tasks: Flat File Source and OleDb Source Destination but its useless at all for that goal.
Does anyone have ever used or faced this hended?
Thanks for any input,
How would you do the following in SSIS?
SELECT a.TestID,
a.TestCode
FROM TableA a
WHERE UPPER(RTRIM(a.TestCode)) IN SELECT (SELECT UPPER(RTRIM(b.TestCode)) FROM TableB b)
Of course the above query is missing a few things but with ETL the where clause UPPER(RTRIM does not appear to be something that has an object or property that I can use in the Lookup.
Please correct and educate me.
I am trying to put the following as an expression in the SSIS Derived Column Transformation Editor.
DATEADD(DAY, DATEDIFF(DAY, 0, GETDATE()), 0)
It is not allowing it. This works fine in a regular SQL statement.
Does anyone know how I can get this to work?
Hi,
I have an example situation that seems like it should have a super easy solution, but my jobs keep failing.
Here we go. . .
I have a SQL Server 2005 table as my source in a data flow task.
This table contains raw data.
We'll call it FACT_Product_Raw - which contains a field called ProductType varchar(1)
Let's say that ProductType contains values of "A" or "B" or "C" - or for that matter, some null and garbage values
I have a lookup table, LOV_Product_Types
This table contains 3 fields that will transform my raw data table
We'll call these fields ProdTypeID smallint, ProdTypeRaw varchar(1) and ProdType smallint
It contains pairs such that A = 1, B = 2, and so on.
Here's what I want to do.
I want to ADD a field to FACT_Product_Raw that contains the "looked up" value from LOV_Product_Types.
Let's say that I want to add the ProdTypeID field to my _Raw table.
I have used the _Raw table as both my source and destination
It blows up every time.
Help.
Thanks,
David
Dear All,
I have a table A with a KEY column and SSN column.
KEY = 12 digits ( first 3 digits are Department Id , and last 9 digits are SSN)
I have a table B with SSN column only.
both KEY and SSN columns are Primary keys so duplicate entries must be Avoided.
Table A is intended to be popluated weekly from TXT file (SSIS package RUN). I want to achieve somethign like this..!
P-Code sample:
for each Row in TXT file
if TXTfile.KEY = TableA.KEY then
skip and Read/Go to next Row in TXT file
else
INSERT TXTfile.KEY into TABLEA.KEY
SSN_Var = EXTRACT the SSN part (SSNpart.READ)
if SSN_VAR.Exists In TableB.AnyRow then
skip
else
Insert into TableB
End If.
End If
End For Loop.
-----------------------------------------------------------------
Using SSIS controls, what will be best flow and logic to achieve this.....?
any sample scripting code ????
Many Thanks.
Hello My package flow is like This
OLEDB SOURCE --> LOOK UP TRANSFORMATION
|
lookup output / error output(for new Inserts)
(updated records) /
/
DERIVED COLUMN DERIVED COLUMN
TRANSFORMATION1 TRANSFORMATION1
| |
v v
OLEDB COMMAND TRANSFORMATION OLEDB DESTINATION (inserting new records to
(Updating records in destinationTable)
destination)
in this senario new records r insrted properly
but though package runs without error records not get updated in Destination.
In OLEDB COMMAND my query is like below,
UPDATE TARGET_SCD_1 SET
CURRENTSTATUS = ?,
CURRENTSTATUSEFFECTIVEDATE=?,
PROPOSALEFFECTIVEDATE=?,
UNDERWRITINGEFFECTIVEDATE=?,
TECHLAPSEEFFECTIVEDATE=?,
WITHDRAWNEFFECTIVEDATE=?,
DCSEFFECTIVEDATE=?,
PREPOSIONEFFECTIVEDATE=?,
INFOURCEEFFECTIVEDATE=?,
LAPSEEFFECTIVEDATE=?,
SURRENDEREFFECTIVEDATE=?,
FLCEFFECTIVEDATE=?,
CANCELLEDEFFECTIVEDATE=?,
DCIEFFECTIVEDATE=?,
REC_UPT_DT=?
WHERE O__NUM = ?
In advanced editor of OLE DB I hv created additional 16 paramater columns though i assign datatype as numeric to tht columns when i press refresh automatically it changes to DT_STR.
My destination table columns r numeric .
I though due to this datatype mismatch the error came So i change the datatype of dest to varchar to make compatible with OLEDB Comand Transformation. THN also no Use NO UPDATES
package is running without error but records not get updated.
if change the flow like below
OLEDB SOURCE --> LOOK UP TRANSFORMATION
|
lookup output / error output(for new Inserts)
(updated records) /
/
DERIVED COLUMN DERIVED COLUMN
TRANSFORMATION1 TRANSFORMATION1
| |
v v
OLEDB COMMANDTRANSFORMATION /
(Updating records in /
destinationTable) /
/
/
UNION ALL TRANSFORMATION
|
v
OLEDB DESTINATION
In This Case The updated record get inserted in the target as wel as the old remains as it is means m getting one additional record.
kindly help me to figure out the bug
M frusted with this issue please.............
Hi,
I am pretty new to SSIS. I am transferring some rows from 2 source tables to 1 destination table.
The 2 source tables have 1000 rows.They act as the 2 inputs to a merge join transformation where i perform the join between the 2 tables based on a couple of fields. But for some reason the output of the merge join gives me about 1018 rows .Shouldnt the destination also have only 1000 rows?
How do i solve tis problem?
Thanks in advance
Sat
Im from Russia, sorry if my english is not very good.
Here's the case:
1)-------------------------------
I made a DTS-package in sql2000 that transfers the [sql table] into [dbf file] via jet4.
First i create (in delphi) the empty dbf with the same name and columns same as in sql table.
Second, I run my DTS with variables - source and destination table names
In DTS there is source, destination and transformation . After I send the Variables(table names)
, the transformation "arrow" needs to be "refreshed" to make column names in both tables correspond each other. For that in transformation I chose ActiveXScript Mode and wright VB Script:
'**********************************************************************
' Visual Basic Transformation Script
'************************************************************************
' Copy each source column to the destination column
Function Main()
dim i
For i = 1 To DTSSource.Count
DTSDestination(i) = DTSSource(i)
next
Main = DTSTransformStat_OK
End Function
And it works
2)------------------------------
I want to do same thing in sql2005 SSIS but don't figure out how...
I managed to make a package that recieves (in variables) table names and runs correctly.
But after I change those variable names into any other it crashes -
Description: "component "OLE DB Source" (1)" failed validation and returned validation status "VS_NEEDSNEWMETADATA".
Of cource this happens 'cause I didn't "refresh" the transformatoin (and maybe also source and dest), but I don't know how.
Anyone can help ?!
Hi,
I worked with informatica before and bit new to SSIS,
i have one simple question..
How can i achieve the same functionality as External Procedure Transformation (Informatica) in SSIS?
like..i want to use a DLL in my package which contains some set of business rules.
Please let me know.
Charles
i have too many DTS packages to migrate to SSIS, and while examining a DTS package in BIDS (converted with the migration utility) i tried to edit the resulting migrated package, which opened the DTS interface with the two connection icons joined by the big fat arrow with a gear on it...not exactly what i had in mind, iow, it looks like SSIS on the outside, but its still DTS on the inside.
So I stripped out a series of components from a more complex package hoping that simplifying it would reveal the contents of old DTS Transformations tab at least partially set up in a Derived Column transformation.
Can i get there from here, or must i recreate every stinking definition in a derived column manually from the ground up?
thanks very much for your help