SSIS Custom Data Flow Component - Variable Type Converter

Jun 27, 2007

Hi all,



I am creating a customer data flow component for SSIS for use in a package. I've got some custom properties that I am exposing using the supplied advanced editor (no custom property editor here).



Some of my properties are enumerated types, and I have deciphered how to get those properties to show as dropdown lists of their respective enumerations. (For those of you who may be looking as hard as I did as to how to accomplish this, see the end of this post.)



I also have a few properties which request SSIS package variable names - such as an file name variable. However, I can't figure out how to tell the advanced editor that the property is looking for an SSIS variable, so that it can show a dropdown list of package variables, much like virtually any other Microsoft supplied Data Flow component can.



Is there a Type Converter I could specify for those custom properties? Is there another way to instruct SSIS that my custom property is expecting a variable? Or do I need to code a custom UI for editing my Data Flow Task?



To create a dropdown list of values for a custom property that represents an enum, do the following:

1. Create your enum definition, such as "public enum ThisIsMyEnum { one, two }"

2. Create a new class that inherits from TypeConverter, such as "public class MyEnumConverter : TypeConverter"

3. Override "CanConvertFrom", and return true if "sourceType == typeof(string)"

4. Override "CanConvertTo", and return true if "destinationType == typeof(string)"

5. Override "ConvertFrom", and return the enum value (such as "one" or "two" in my example) that corresponds to the string passed in the parameter "value"

6. Override "ConvertTo", and return a string that corresponds to the enum value passed in the parameter "value"

7. Override "GetStandardValuesSupported" and return true

8. Override "GetStandarValuesExclusive" and return true to indicate that ONLY the enum values should be accepted

9. Override "GetStandardValues", and return a new StandardValuesCollection constructed with Enum.GetValues() of your enum, such as "return new StandardValuesCollection(Enum.GetValues(typeof(ThisIsMyEnum)));"

10. Just above your "public enum" declaration, add a "TypeConverter" attribute to link your type converter to your enum, such as "[TypeConverter(typeof(MyEnumConverter))]"

11. In "ProvideComponentProperties", after you've created your custom property like this: "IDTSCustomProperty90 propEnum = ComponentMetaData.CustomPropertyCollection.New()", add another line to specify the TypeConverter property of the property to the full assembly name of your type converter, like so: "propEnum.TypeConverter = typeof(MyEnumConverter).AssemblyQualifiedName;"

View 11 Replies


ADVERTISEMENT

Expression Editor On Custom Properties On Custom Data Flow Component

Aug 14, 2007

Hi,

I've created a Custom Data Flow Component and added some Custom Properties.

I want the user to set the contents using an expression. I did some research and come up with the folowing:





Code Snippet
IDTSCustomProperty90 SourceTableProperty = ComponentMetaData.CustomPropertyCollection.New();
SourceTableProperty.ExpressionType = DTSCustomPropertyExpressionType.CPET_NOTIFY;
SourceTableProperty.Name = "SourceTable";






But it doesn't work, if I enter @[System:ackageName] in the field. It comes out "@[System:ackageName]" instead of the actual package name.

I'm also unable to find how I can tell the designer to show the Expression editor. I would like to see the elipses (...) next to my field.

Any help would be greatly appreciated!

Thank you

View 6 Replies View Related

Expression Issue With Custom Data Flow Component And Custom Property

Apr 2, 2007

Hi,



I'm trying to enable Expression for a custom property in my custom data flow component.

Here is the code I wrote to declare the custom property:



public override void ProvideComponentProperties()

{


ComponentMetaData.RuntimeConnectionCollection.RemoveAll();

RemoveAllInputsOutputsAndCustomProperties();



IDTSCustomProperty90 prop = ComponentMetaData.CustomPropertyCollection.New();

prop.Name = "MyProperty";

prop.Description = "My property description";

prop.Value = string.Empty;

prop.ExpressionType = DTSCustomPropertyExpressionType.CPET_NOTIFY;



...

}



In design mode, I can assign an expression to my custom property, but it get evaluated in design mode and not in runtime

Here is my expression (a file name based on a date contained in a user variable):



"DB" + (DT_WSTR, 4)YEAR( @[User::varCurrentDate] ) + RIGHT( "0" + (DT_WSTR, 2)MONTH( @[User::varCurrentDate] ), 2 ) + "\" + (DT_WSTR, 4)YEAR( @[User::varCurrentDate] ) + RIGHT( "0" + (DT_WSTR, 2)MONTH( @[User::varCurrentDate] ), 2 ) + ".VER"



@[User::varCurrentDate] is a DateTime variable and is assign to 0 at design time

So the expression is evaluated as: "DB189912189912.VER".



My package contains 2 data flow.

At runtime,

The first one is responsible to set a valid date in @[User::varCurrentDate] variable. (the date is 2007-01-15)

The second one contains my custom data flow component with my custom property that was set to an expression at design time



When my component get executed, my custom property value is still "DB189912189912.VER" and I expected "DB200701200701.VER"



Any idea ?



View 5 Replies View Related

Reuse Existing Data Flow Components In A Custom Data Flow Component

Aug 29, 2007

Hello,

Is it possible to use existing data flow components (Merge Join, aggregation,...) in a custom data flow component?

Thanks,

Yoann

View 15 Replies View Related

Use Of A SSIS Variable Of Type “Object� Inside Script Component And Task Component

Mar 16, 2007

In a Data Flow, I have the necessity to use a SSIS variable of type €œObject€? inside Script Component and assign to it the content of 'n' variables of string type.
On exiting from the script the variable of type object should contain something like in the following lines:
AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDD
€¦€¦€¦€¦€¦€¦€¦.
€¦€¦€¦€¦€¦€¦€¦.
On exiting from the data flow I will use the variable of type Object in a Script Task, by reading each element in a cyclic fashion.
Is there anyone who have experienced something like this? Could anyone provide any example of that?
Thanks in advance!

View 3 Replies View Related

Custom Data Flow Component Column Mapping Question

Feb 7, 2007

Hi,

I'm having my first go at developing a destination adapter which will send data to an update Web Service.

I've got some rather big gaps in my understanding. I've been following the various samples I've found on the net and have validated my mapping and picked up all the available column names and datatypes which are appearing in the Input and Output Properties tab of the Advanced Editor but I only have a tab for "Input Columns" and not "Column Mappings".

Which method defines the availble columns for the user to map?

Let me know if I haven't given enough information.

cheers

View 1 Replies View Related

Problem Assigning Value To Package Variable From Data Flow Script Component

Sep 28, 2005

In my Script Component properties I have included "ClientReportGroupId" as a ReadWrite variable. This variable is declared as a Package Variable.

View 23 Replies View Related

The Return Of Problem Assigning Value To Package Variable From Data Flow Script Component

Jul 10, 2006

I have a Data Flow Script Component(Destination Type) and in the properties I have a read/write variable called User::giRowCount

User::giRowCount is populated by a Row Count Component previously in the Data Flow.


After reading http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=97494&SiteID=1 it is very clear that you can actually only use  variables in the PostExecute of a Data Flow Script Component or you will get an error
"Microsoft.SqlServer.Dts.Pipeline.ReadWriteVariablesNotAvailableException: The collection of variables locked for read and write access is not available outside of PostExecute."




What I need to do is actually create a file in the PreExecute and write the number of records = User::giRowCount as second line as part of the header, I also need to parse a read/write variable such as gsFilename to save me hardcoding the path

(Me.Variables.gsFilename.ToString),(Me.Variables.giRowCount.ToString)

 -they must go in the PreExecute sub --workarounds please-here is the complete script component  that creates a file with header, data and trailer --Is there any workaround

Thanks in advance Dave
 
Imports System
Imports System.Data
Imports System.Math
Imports System.IO
Imports System.Text
Imports System.Configuration
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
 
Public Class ScriptMain
    Inherits UserComponent
    'Dim fs As FileStream
    Dim fileName As String = "F:FilePickUpMyfilename.csv"
    'Dim fileName = (Me.Variables.gsFilename.ToString)
 
    Dim myFile As FileInfo = New FileInfo(fileName)
    Dim sw As StreamWriter = myFile.CreateText
    Dim sbRecord As StringBuilder = New StringBuilder
 
 
    Public Overrides Sub PreExecute()
 
        sbRecord.Append("RECORD_START").Append(vbNewLine)
 
    End Sub
 
 
 
    Public Overrides Sub ParsedInput_ProcessInputRow(ByVal Row As ParsedInputBuffer)
 
        sbRecord.Append(Row.ProjectID.ToString)
        sbRecord.Append(Row.TransactionRefNum.ToString)
        sbRecord.Append(Row.BillToCustomerNum.ToString)
        sbRecord.Append(Row.BillToAccountNum.ToString)
        sbRecord.Append(Row.BillToLineNum.ToString)
        sbRecord.Append(Row.BillToReassignmentNum.ToString)
        sbRecord.Append(Row.ChargeCode.ToString)
        sbRecord.Append(Row.NotificationMethod.ToString)
        sbRecord.Append(Row.AdjustmentAmount.ToString)
        sbRecord.Append(Row.AdjustmentDate.ToString)
        sbRecord.Append(Row.ReparationGivenFlag)
        sbRecord.Append(Row.BillingSystemProcessingErrorCode.ToString).Append(vbNewLine)
       
    End Sub
 
    Public Overrides Sub PostExecute()
        sbRecord.Append("RECORD_COUNT").Append((vbTab))
        sbRecord.Append(Me.Variables.giRowCount.ToString).Append(vbNewLine)
      sbRecord.Append("RECORD_END").Append(vbNewLine)
       'Now write to file before next record extract
        sw.Write(sbRecord.ToString)
        'Clear contents of String Builder
        sbRecord.Remove(0, sbRecord.Length)
 
 
       'Close file
        sw.Close()
 
    End Sub
 
 
End Class

Has anyone got a workaround

thanks in advance

Dave

View 6 Replies View Related

Xml Ssis Data Flow Component?

Jul 2, 2007

There is a table with a column that contains Xml documents. For each record from my Data Flow Source, I want to pass in the Xml document and the node to interrogate, and return the value contained in the node. Like the Crm component, this is probably one I will have to write from scratch in C#, but I would like to avoid having to create the custom component if it already exists in the public arena.



Does anyone know of any Xml Ssis Data Flow Components that are downloadable for free?

View 3 Replies View Related

Logging Data Flow Component Events In SSIS

Nov 14, 2007

Does anyone know how to hook up to data flow pipeline events via custom solution (C#)? I am trying to write code to log start and end times of components(lookup,merge joins etc) in a data flow task. I tried with a class inheriting from the EventsProvider class but it didn't work as this is only for container tasks. Any ideas will be greatly appreciated.

View 3 Replies View Related

SSIS Programming Setting Name To Data Flow Component

Aug 28, 2007

Hi,

I am creating a SSIS Programmatically. I facing problem in setting name to data flow compenent.

here is the sample code

IDTSComponentMetaData90 oledbItemMasterSource;

oledbItemMasterSource = dataFlow.ComponentMetaDataCollection.New();



oledbItemMasterSource.ComponentClassID = "DTSAdapter.OLEDBSource";

oledbItemMasterSource.Name = "InventTable";

oledbItemMasterSource.Description = "Get Item Master";



After creating this package i found that 'InventTable" is not set to OLE Db Source component. where as i can set name to OLE DB Flat File destination component.

Kindly suggest me a way fix this issue.

Thanks-
aravind

View 1 Replies View Related

SSIS Data Flow Source Component To 'read' A PDF File

Feb 13, 2008

At our business we are getting a lot of PDF documents that are being hand keyed into a database. Has anyone heard ior know of a SSIS Data Flow Source component that I coud use to read thos documents into a data stream (?) and process?

View 5 Replies View Related

Why Dataflow Component Doesn't Appear In The List Of SSIS Data Flow Items?

Sep 5, 2007

Hi,
I developed SSIS Data Flow Component and placed dll file into the DTSPipelinecomponents. Then I registered the component in the GAC.

But when I try to add the required component into toolbox that there is not this one in the list of SSIS Data Flow Items. What does it mean?

Thanks in advance.

View 3 Replies View Related

A Custom Component For Use As A VIEW In SSIS- Is It Possible To Create One MERGE Like Component With More Than 2 Inputs

Aug 13, 2007

Hi all
I'm into a project which uses a lot of views for joining 2 or more tables. Using the MERGE component in SSIS will be a huge effort coz it only has 2 inputs and I gotta SORT the input too.
Isnt it possible to have a VIEW like component that joins more than 2 tables and DOESNT need sorting??
(I've thought about creating views in database engine but it breaks my data floe in SSIS and is'nt a practical solution)

View 4 Replies View Related

Add A Variable From Custom Component

May 19, 2006

Hi

I am writing a custom transformation component that utilises a user variable.

Before using the variable at run time I am checking that the variable exists, but it would be nice to be able to add it if it does not. I cannot find any documentation on the subject, though I can see that the Variables class is derived from a ReadOnlyCollectionBase.

Is there a way to add a user variable with package scope from a custom component, either at run time or design time?

Thanks . . . Ed

View 1 Replies View Related

Variable Data Type In SSIS

Aug 31, 2007

how to pass the numeric(12,0) data type to a variable in SSIS? what kind of variable data type should I choose?
I am trying to assign object_key column ( numeric(12,0)) to a variable in SSIS

If i select int32 , it keep giving me an error:
Error: 0xC001F009 at Row by Row process: The type of the value being assigned to variable "User::Object_Key" differs from the current variable type. Variables may not change type during execution. Variable types are strict, except for variables of type Object.

View 3 Replies View Related

SQL 2012 :: How To Capture Data Flow Component Name Dynamically While Package SSIS Package Is Executing

Jun 3, 2014

I would like to fetch the data flow component name while package is executing. Since system variable named [System::SourceName] only fetches name of the control flow tasks? Is there a way to capture them?

View 5 Replies View Related

Developing Custom Components By Extending The SSIS Stock Data Flow Components

Sep 7, 2006

Everything I've read says that custom data flow components are built by inheriting from the Microsoft.SqlServer.Dts.Pipeline.PipelineComponent class.

But the stock components such as the Derived Column data flow transformation must each be implemented by their own class. So how do I base my custom components on those classes? The documentation for the PipelineComponent class doesn't list any such subclasses.

View 1 Replies View Related

Custom Transform Component, Change Type Or Add Output Column

Jun 26, 2006

Would anyone happen to have any pointers or know of any good code examples to either programmatically change the type of an input column when it is passed through the component, or add a new column to the output? I am extracting data from an Oracle database which is in Julian date format (represented within SSIS as a DT_NUMERIC column) and I need to to either transform the input column holding it into a date column, or to dynamically add a new output column holding the transformed data.

Many thanks

View 1 Replies View Related

Is There A Way To Set A Variable In A Data Flow From A SQL Statement (like In Control Flow)

Jan 12, 2006

I'm currently setting variables at the package level with an ExecuteSQL task.  This works fine.  However, I'm now starting to think about restartability midway through a package.  It would be nice to have the variable(s) needed in a data flow set within the data flow so that I only have to restart that task. 

Is there a way to do that using an SQL statement as the source of the value in a data flow? 

OR, when using checkpoints will it save variable settings so that they are available when the package is restarted?  This would make my issue a moot point.

View 2 Replies View Related

Can Custom Properties In A SSIS Component Be Disabled

Feb 26, 2008

Is there a way of disabling a custom property in a component so that during design-time the property is grayed out? I looked around the properties of IDTSCustomProperty90 and nothing sticks out.

Thanks
Mike

View 5 Replies View Related

SSIS Custom Component/task Examples

Feb 8, 2006

Hi,

Near the end of 2005 Microsoft made available some sample C# apps that implemented custom SSIS tasks and components. I think Doug Ladenshlager may have had a heavy hand in building them.

Does anyone know where they are? I can't darned well find them!

-Jamie

View 1 Replies View Related

Resources For SSIS Custom Component Development

Dec 13, 2007

Does anyone here have a favorite site or set of sites with resources on SSIS custom component (tasks, transforms, log providers, etc.) development? I've been searching around to try to find this on my own, but so far I've had little luck.

Darren - I've noticed that you always seem to have the most insightful responses to questions related to SSIS .NET development (such as your recent response to evaluating expressions in a custom component) so I am particularly interested in anything that you have to share.

Thanks in advance, everyone!

View 6 Replies View Related

Removing A Custom Component From SSIS Designer

Mar 12, 2007

How do I remove a custom component from SSIS Designer pernamently? I can easily remove it from the tool box, but I want to get rid of it completely.

I tried searching in MSDN for an answer to this question but I could not find it.

Thanks

View 6 Replies View Related

Custom SSIS Control Flow Task Implemented In C++

Nov 12, 2007

Hi Guys,

This is a question to the SSIS development team. I would like to know what are the requirements to implement custom SSIS Control Flow task in C++ . There is a documentation describing the process when implementing a managed task, but no such documentation exists for implementing a task in C++.

Thank you,
Ivan

View 10 Replies View Related

SSIS Custom Component DerivedColumn Programmatically Problems

Feb 13, 2006

dear experts,
i'm trying to build a package programmatically from client c# application. I'm working to create three dataflow components:
- OleDB Source
- Derived Column Transformations
- OleDb Destination.

My package works good, but i have several problems when insert an expression as Value of an IDTSCustomPropriety90 object, like this one:
[LEN](#firsname.lineageID) &gt 5 ? <<if true>> : <<if false>>

Simple expressions like concatenate strings, for example "#firstname.lineageID" + "#lastname.lineageID" seem to work good.

Thanks a lot in advanced.

Paganelli Francesco

View 19 Replies View Related

SSIS Custom Component, Output Buffer Problem

Mar 27, 2007

Hi Guys,

I created a SSIS custom component, transformation (Asynchronous) with one Input collection and 2 output collections.

The SSIS Package which includes the Component I created works well in the Business Intelligence Studio, but when the same Package is run in the 'Execute Package utility' It fails to run. ( when you Double click on the dtsx file)

The cause of the failiure is

public override void PrimeOutput(int outputs, int[] outputIDs, PipelineBuffer[] buffers)

method receives only one output buffer when executed using the 'Execute Package Utility' { outputs = 1 , buffer.Length = 1 } ( when executed in the BI studio, the method receives parameters of both the output buffers that I expect { outputs = 2 , buffer.Length = 2 } )

The property ComponentMetaData.OutputCollection.Count = 2 as well. Yet the PrimeOutput method provides only 1 buffer.

The Validation Succeeds on both instances, which I assume means that Meta Data is Provided Properly.


What would be the reason for the same pakage to run in 2 different ways like this,

What might I have missed out to do, to make the package run in different ways on 'Business Intelligence Studio' and 'Execute Package Utility'

Thanks a lot



Below are some of the lines from the ProvideComonentProperties Method which deals with the output Collection, Isn't this sufficient for the PrimeOutput to provide 2 output buffers?





ProvideComponentProperties()









public override void ProvideComponentProperties()
{

RemoveAllInputsOutputsAndCustomProperties();
base.RemoveAllInputsOutputsAndCustomProperties();
base.ProvideComponentProperties();

//other function calls

IDTSOutput90 output1 = ComponentMetaData.OutputCollection[0];
output1.Name = "Output1";
output1.Description = ".......................";
extracted.SynchronousInputID =0;


IDTSOutput90 output2 = ComponentMetaData.OutputCollection.New();
output2.Name = "Output2";
output2.Description = "..........................";
output2.SynchronousInputID = 0;

//other function calls
}

View 3 Replies View Related

Need Help With Script Component In Data Flow

Jul 26, 2007

I have a int value that I pulled from a table in a database and it is stored in a variable. I would like to increment this value in a script component and then insert it into a field in my ole db destination. Is there any example out there of using a script component to do something similar to this.

Do I want to select source, destination or transformation?

Can I/how do I access my variable from within the script component.

Any asistance or examples you can point me towards would be greatly appreciated.

View 3 Replies View Related

Custom Data Flow

Mar 26, 2007



To all the SSIS gurus,



I have a situation where I have multiple source tables, I need to populate the destination tables that have the same schema as the source ones. I dont want to do the repetitive task of creating a Dataflow for each source-destination load.



I want to create one custom dataflow component and loop through all my source tables and provide destination tables dynamically.



Is there a way to do that? Any custom data flow component out there??



Your ideas are greatly appreciated.



Regards,

View 5 Replies View Related

Do We Have To Alawys Use Slowly Changing Dimensions (SCD) Component In The Data Flow For The Loading Of Table Data?

Feb 28, 2008

Hi, all experts here,
Do we always have to use SCD component for the loading of data into data warehouse to handle changes of rows?
I am looking forward to hearing from you and thank you very much in advance for your help.
With best regards,

View 4 Replies View Related

Enabling Expression Builder For Custom SSIS DataFlow Source Component

Mar 13, 2007

Hi,

I have implemented a custom source component that can be used as the data source in the Data Flow task.

I have also created a custom UI for this component by using the IDtsComponentUI .

But my component does not have the capability of setting the custom properties via the DTS Variables using the Expression Builder.

I have looked around for samples on how to do this, but I can only find samples of how to do this for custom Control Tasks, i.e. IDtsTaskUI.

My question is, How can implement the Expression Builder in my custom Source component + custom Source UI. Or do you know of any samples which I can look at.

Thank you,

Jameel.

View 1 Replies View Related

SSIS Custom Component Will Only Partially Update In VS Causing Debugging To Fail

May 29, 2006

When I try to debug the break points will always say the source code is different from the current version, but the custom component in the GAC has the new version number. The other strange thing is the toolbox will not reset to the original version meaning it will not remove the custom components. The funny thing is after I compile the custom components and restart VS the custom component runs with the new code changes. I can see the new features I added, but the debugger and toolbox still seem to be broken.

I have tried the following
1) Reset the tool box.
2) uninstall all my custom dll from the GAC €œC:WINDOWSassembly€?
3) remove all my custom dll from €œC:Program FilesMicrosoft SQL Server90DTSPipelineComponents€?
4) restart VS 2005
5) reselect the custom components.
6) reboot my computer.


It seem like VS has another cache. For the tool box or something.

Does anybody have any suggestion?

View 10 Replies View Related

System Configuration File For Custom Dll Used Script Component In SSIS Packages

Jan 14, 2008



Hi,
I am using custom dll in script component in SSIS package. This dll is looking for some configuration settings and dsplays the message as "Configuration section could not be found in the configuration source" . Please tell me the configuration source it looks for.

View 3 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved