Is It Possible To Version Custom SSIS Components?
Jul 17, 2006
Is it possible to use versioning at all with SSIS components?
My experience so far is that I have not figured out how version them at all, because if I change the version number, then apparently the <property name="UserComponentTypeName" ...> has to be changed in every single reference to the component, in every dataflow, in every single dtsx file that uses the component at all -- this will be a tremendous headache as some of this stuff is in source control. It seemsl clearly infeasible to check out and substitute and change all these references, and check them all back in, except very infrequently -- surely not feasible to do it for each small change.
I am in the habit of versioning dlls and assemblies, and actually changing at least a minor build number for each actual change, so I am a bit unhappy to find that apparently I cannot feasibly do this for a custom SSIS component :(
(I just find a dislike at apparently being forced to leave the version number invariant when I make code changes.)
What I would be very happy with, is if there were a way to adjust all the references in all the dtsx files, to refer to only the major and minor build numbers say, specifying that I am not picky about the revision numbers below those, so that it will find the latest component in the GAC matching the major and minor numbers.
That way I could still update the component version with each code change, which would make me happy, and not have to edit a zillion dtsx occurrences (not having to do that would also make me happy).
Anyone know if that is possible?
I'm hoping it is, and that it is only my ignorance at component references preventing me from doing so.
View 4 Replies
ADVERTISEMENT
Sep 7, 2006
Everything I've read says that custom data flow components are built by inheriting from the Microsoft.SqlServer.Dts.Pipeline.PipelineComponent class.
But the stock components such as the Derived Column data flow transformation must each be implemented by their own class. So how do I base my custom components on those classes? The documentation for the PipelineComponent class doesn't list any such subclasses.
View 1 Replies
View Related
May 28, 2008
Whenever I make a breaking change to a custom SSIS component/tasks and update the Assembly Version, it seems to break my packages beyond repair, telling me it can't load the task:
Error loading Package1.dtsx: Error loading a task. The contact information for the task is "". This happens when loading a task fails.
All of the properties of said task now show:
Could not get value for property 'c-155-designer-name'. Specified cast is not valid.
Typically, a "breaking" change when it comes to code just means that you need to update your components to adhere to the new contract of the updated signatures. But with SSIS, it seems the only solution to this is to completely remove the component, and re-add the new version, and re-enter all of the property values/expressions. If I have a package containing 10 instances of a task that only had one property removed, for example, this results in a very time-consuming process of fixing my package.
So my questions:
1) Am I doing something wrong in my versioning/deployment that is causing my packages to unnecessarily break?
2) If this is just "by design" and the way it's meant to behave, what is the best practice for making breaking changes to custom tasks/components used by many packages? Should I just never change the assembly version, even when it is a breaking change (this seems to be less disastrous)?
3) As a last resort, if I'm stuck with having to fix the broken tasks, is there a better way to fix them rather than having to completely remove them, re-add them, and re-set all of their properties/expressions?
Thanks in advance.
Jerad
View 3 Replies
View Related
Oct 10, 2007
Hi,
As a hobby, I am creating custom SQL Server Integration Services components, which are installed as an add-on to out of the box components. They are mainly used to reduce the development time and minimize the maintance overhead. I would like to know, how I should market them? What are the mediums to use? Since this is the first time, I am creating a software for mass markets, I would really appreciate your help while marketing this baby.
Just FYI, the first commercially available package that I develop is for creating text based files. My package connects to any OLE DB datasource. Executes the query and stores the output in a text file. It is very useful while developing database driven custom feeds. Instead of working with multiple dataflow tasks, data flow sources, you just need to configure one single component and the rest is taken care for you. Since, the package implements the standard Microsoft Task Editor, it is very easy to learn.
Thanks for your help.
Cem
View 3 Replies
View Related
Nov 15, 2007
I have a SQL Server 2005 Package (32 bit, 4 processors/16 GB of RAM, 3 GB switch for SSIS) with 1 DataFlow and 5 Custom Components:
> An OLEDB Source that reads a stoed proc from Sybase
> 3 Transform Components that perform: 6-required fields check, 4-lookups (3 cached, 1 non-cached), and a calculation that is performed(called) from a .NET Library.
> A SQL Destination that uses the .NET BulkCopy Class to insert the data.
It takes about 3 hours to load 1 million records that contain 46 columns (the largest being a varchar(100)). I have tried a good number of tuning options as well as indexing/SQL techniques to improve perfromance.
Some pieces of info:
1) We dynamically create the buffer during runtime based on an SSIS variable.
2) The actions of the components are determined at runtime through config files.
We do these two things so we can have multiple data flows run through the same package for better scalability and flexibility. Since we are dynamically creating the buffer I can NOT use any of the out of the box components because of lineage and metadata inconsistency issues.
3) I noticed on PerfMon that I am spooling buffers to disk after 100K rows.
I realize this is bad, but what are my alternatives?
Should I go to 64 bit? Is there anything else I can do?
Thanks in advance,
Anthony
View 10 Replies
View Related
Jun 30, 2006
Hi,
Can anyone tell me where I might find the Class and Sequence Diagrams for the SSIS framework (for Custom Components)?
I've just started trying to create some Custom Transform Components and
I'm really struggling to get my head around the component lifecycle
(i.e what methods are called when, with what arguments, and why) with
just the BOL documentation to guide me.
Thanks in advance,
Lawrie
View 1 Replies
View Related
Jun 25, 2006
Hi All,
I am not sure if this is a correct forum to discuss on the document posted @ http://www.microsoft.com/downloads/details.aspx?familyid=1c2a7dd2-3ec3-4641-9407-a5a337bea7d3&displaylang=en on SQL Server Integration Services (SSIS) Hands on Training - Creating Custom Components.
I am assuming Microsoft Developers are constantly monitoring this forum.
In the document - SSIS Creating a Custom Transformation Component .doc on Page 2 -
Exercise 1 - Writing the no-op data flow transformation component -
Task 1 - Create a new C# Class Library Project
The textual description talks about creating a new Visual C# Class Library project in VS 2005 but the screenshot accompanying it shows the creation of new "Integration Service Project" in VS 2005.
Please change the screenshot appropriately to avoid confusions.
Thanks,
Loonysan
View 1 Replies
View Related
Feb 12, 2007
Hi,
In my case I have to log the errors raised by any task in a package to either windows event log, text file or SQL server. Also I need to send an email notifications to a group of people telling them about the error.
Now can I use SSIS package logging for logging the errors into the required destinations. I mean right clicking on the package and selecting Logging, then adding the required log providers and enabling the events for logging into those. I think I have to upfront select the log providers to log the error, I will not have the liberty to log the error to the destination, the name of which is passed as a variable to the package. This is okay with me though.
Now what will a custom log provider help me to do in this case. Also can I somehow configure my package to call the send mail task everytime an error is raised.
Also, one more option can be developing a package that only does the error handling. It will take in the paramters or the error codes and descriptions, the destination to write to and a flag to send mail or not for that particular type of error.
What do you think? Kindly advise.
Thanks in advance for your help and time.
Regards,
$wapnil
View 3 Replies
View Related
Jan 22, 2008
Hi again, All!!
I'm having a problem with problems with some custom components I built. I have a custom Connection Manager and Data Flow Source that I have built.
My local PC is 64-bit, but I followed the MS instructions on building and deploying custom components. They run fine inside of VS2005, as long as I have 'Run64BitRuntime" set to 'False'. When I try to run packages using the custom components in 64-bit mode, I get:
Error: 0xC0014005 at : The connection type "<MyConnectionManagerType>" specified for connection manager "PRI" is not recognized as a valid connection manager type. This error is returned when an attempt is made to create a connection manager for an unknown connection type. Check the spelling in the connection type name.
I suspect I need to either change the way I build or deploy the custom components. Can someone shed some light on this?
Thanks,
Frank
View 1 Replies
View Related
Jun 15, 2007
Is it possible to create a custom SSRS .NET component that can view the report parameters and values contained within?
Thanks!
View 3 Replies
View Related
Mar 13, 2006
Hello,
I had an idea that it would be nice to be able to extend the functionality of an existing SSIS component or task by inheriting from it. Perhaps in a similar way to how it is possible to extend user controls in .Net.
e.g. The rowcount is a useful component but it would be good to create a new component that inherits from it and then override the PostExecute() method to fire an Information event containing the number of rows. That's a very simple example but I think you get the gist.
Does anyone think that would be useful? Or even made possible?
I was also wondering whether it would be possible for the SSI team to make all the icons used within SSIS available as .ico files so that we could modify them for tasks/components that might do similar things.
Thanks
-Jamie
[Microsoft follow-up]
View 1 Replies
View Related
Jun 30, 2006
Hi,
Is it possible to make use of VB.Net for Applications and the VBA editor from your own Custom Transform Components (like the script component does)?
I would like to create a component that generates boiler-plate VBA code and that then allows developers to add their own specialised code in certain methods using the editor (just like the script component)
If so, is there any documentation anywhere on how to go about this?
Thanks in advance,
Lawrie
View 4 Replies
View Related
Dec 4, 2007
I've been trying to find a solution to adding javascript and/or HTML to a report (not reportviewer). After a search I found that you can add javascript to a textbox under action properties and it works fine. Is there a way to add custom code anywhere on the page.
The problem is the charts that come with SRS is limited so I'd like to add my own. I'd also like to add some ajax functionality as well. I know it's possible because a company called Dundas is doing it but not exactly sure how they implemented it. We considered Dundas but they are just too expensive for the solution we need.
Any help would be much appreciated.
View 8 Replies
View Related
Sep 7, 2007
Is there a way to use a custom component programmatically (I'm using C# to create and run my packages) without installing a compiled .dll of the class into the SQL folders and the GAC? For instance, can I instantiate my custom component and somehow pass it to be used in a task in my code.
View 1 Replies
View Related
May 21, 2006
I need VB.NET source code showing how to create a Custom Component. in SSIS (All the ones I've seen are written in C#.)
TIA,
barkingdog
View 1 Replies
View Related
Apr 11, 2008
Hi all,
I'm wondering if anyone's accomplished this before - I've been unable to find a whiff of info on how to do this so far.
I'm creating a custom component that I'd like to give a "Derived Column" type of ability to. By that, I mean I'd like to populate a property of my component with an expression (including references to input columns, package variables and functions) and be able to evaluate it at runtime - per row processed by the component.
I would also appreciate any information as to how to provide the interface to allow the user to build such an expression as well - is there a UI function in SSIS I can call to pop up the "expression builder"?
Thanks!
View 6 Replies
View Related
Feb 14, 2006
Hi,
I have a 2 custom components - source and destination.
I want to create an error output for each, to allow the users of my component to handle errors the way they choose.
I only found a property in IDTSOuptut90 named isErrorOut - a boolean property indicating whether this output is an error output or not.
Does anyone have additional documentation / articles / code samples regarding how to really populate the rows in the error output?
Thanks
View 1 Replies
View Related
Mar 29, 2007
I've been having an issue with the integration of a third-party DLL into a custom data flow component.
The company sent me a C# project that generates a simple console application. The project includes a class that calls their DLL with DllImport. The console application runs fine.
I created a stand-alone class using the C# class they sent to expose the methods of their DLL. In my custom component, I'm referencing this class to pass data to and from their DLL.
The first method from that stand-alone class that my component encounters simply gets their installation path from the registry and does not use DllImport. That path retrieval works fine. The next method calls a function that is declared with DllImport. Each time the call fails with "System.DllNotFoundException = {"Unable to load DLL AMZip.dll': Exception from HRESULT: 0xE06D7363"}".
I've copied this DLL to countless locations (e.g., the PipelineComponents directory, the project/solution bin directory) and included these paths in all manner of path variables.
What am I missing here? Their DLL is not strong named (does this matter since I'm using DllImport?), my stand-alone class is, and of course, the custom component itself is. I appreciate the help.
View 9 Replies
View Related
Dec 14, 2007
Hi all,
I have downloaded Microsoft SQL Server 2005 Enterprise Evaluation Edition ISO from the following site and burned it onto a DVD.
http://www.microsoft.com/downloads/details.aspx?FamilyId=6931FA7F-C094-49A2-A050-2D07993566EC&displaylang=en
Prior to this I have installed Visual Studio 2005 Professional Edtion. During setup I have selected all the native client components i.e. Management Studio, Business Intelligense Studio. But after the install I am not able to see them under All Programs-> Microsoft SQL Server 2005. On further searching I have come across this link
http://channel9.msdn.com/ShowPost.aspx?PostID=142941
First of all, there is no sqlrun_tools.msi under setup folder of the dvd. And also when I tried to change component from Add/Remove programs and following the instructions as specified on clicking Change Installed components all that I see are: Connectivity Components and Software Development kit.
Could someone tell me how can I install Management Studio/ Business Intelligence Studio seperately? I don't want to go through the process of re-installation.
View 4 Replies
View Related
Oct 21, 2005
Hi all,
Yesterday, I just downloaded the SQL Server 2005 Express (SQLEXPR.EXE) and tried to install it in my Windows XP Pro PC that is on Microsoft NT 4 LAN System. I got the following error: Microsoft SQL Server 2005 CTP setup The Microfost .NET Framework 2.0 is not installed. Please install before running setup.
This morning, I downloaded the NET Framework Version 2.0 Redistributable Package Beta 2 (x86) from the Microsoft Download Center and installed it in my Windows XP Pro PC. Then I tried to install the SQL Server 2005 Express (SQLEXPR.EXE) again. This time I got a new error: SQL Server 2005 Setup has detected incompatible components from beta versions of Visual Studio, .NET Framework, or SQL Server 2005. Use ADD or Remove Programs to remove these components, and then run SQL Server 2005 Setup again.
I am confused and lost in this installation problem. Was the .NET Framework Version 2.0 Redistributable Package Beta 2 (x86) a wrong Microsoft .NET Framework 2.0 I downloaded and installed in my PC? Where is the right Microsoft .NET Framework 2.0 for the SQL Server 2005 Express? Please help and advise.
Thanks in advance,
Scott Chang
View 23 Replies
View Related
Jan 7, 2008
I have an ssis package that downloads a SQL Server 2000 database and restores it to a SQL 2005 server. Ninety-five percent of the time it runs just fine, but every so often the job fails and I get the following set of error messages in the log file. (I have had to delete some of the proprietary information for this message to satisfy my boss like putting in [Database name] in place of the actual name of the database in the error message).
OnError, Restore backup file to DB,,,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,Converting database [Database name] from version 539 to the current version 611.
OnError,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,Converting database [Database name] from version 539 to the current version 611.
OnError,Restore backup file to DB,,,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,Database [Database name] running the upgrade step from version 539 to version 551.
OnError,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,Database [Database name] running the upgrade step from version 539 to version 551.
OnError,Restore backup file to DB,,,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,The transaction log for database [Database name] is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases
OnError,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,The transaction log for database [Database name] is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases
OnError,SQL5,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,RESTORE could not start database [Database name].
OnError,SQL5,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,RESTORE could not start database [Database name].
OnError,SQL5,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,-1073548784,0x,Executing the query "RESTORE DATABASE [Database name]
FROM DISK = '[Path to back up file]' WITH REPLACE,
MOVE 'ASP_Live_Data' TO [Path to MDF file],
MOVE 'ASP_Live_Log' TO [Path to LDF file]" failed with the following error: "RESTORE DATABASE is terminating abnormally.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
OnError,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,-1073548784,0x,Executing the query "RESTORE DATABASE [Database name]
FROM DISK = '[Path to backup file]' WITH REPLACE,
MOVE 'ASP_Live_Data' TO [Path to MDF file]',
MOVE 'ASP_Live_Log' TO [Path to LDF file]" failed with the following error: "RESTORE DATABASE is terminating abnormally.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
OnTaskFailed,Restore backup file to DB,,,1/6/2008 2:59:31 AM,1/6/2008 2:59:31 AM,0,0x,(null)
Does anyone have any suggestions?
View 1 Replies
View Related
Jun 2, 2006
Hi,
I'm finding that the standard components often just don't quite meet my
needs, but would only need some fairly minor changes to save me and my team a lot of
work (and produce more elegant solutions). So I was just wondering whether the source code was available for the standard components that come with SSIS, or if there is anyway to extend their functionality?
Or do you just have to start form scratch?
Thanks,
Lawrie
View 4 Replies
View Related
Jan 2, 2008
Hi
I am looking for a function or so to format a number to two digits plus one decimal point.
Some examples:
- if input=4 then output= 04.0
- if input=40 then output= 40.0
- if input=4.1 then output= 04.1
- if input=4.32 then output= 04.3
Any idea?
View 1 Replies
View Related
Oct 10, 2007
hello all -- I'm an SSIS newbie who is exploring (custom) programming functionality through third party components, to be plugged into SSIS, for SMTP, FTP, PGP, TAR, WinZIP, basically for a scheduling & automation utility.
Is there such a concept of third party components (and marketplace for this?) or is VB a much better choice for this vis-a-vis components variety, $$ and robustness ?
thanks much,
Cos
View 1 Replies
View Related
Feb 10, 2006
Especially interested in the CodePageConvert.
Installing the .msi just creates a project folder in my Visual Studio directory. I'm unclear how to get from this point to being able to choose this component from my Toolbox items in SSIS. There was a readme file that talked about gacutil.exe and .snk files that was a above my head.
Can anyone dumb it down for me?
View 6 Replies
View Related
Mar 3, 2006
How can we install the SSIS runtime components alone.
We are calling SSIS packages from .NET code. We would like to know what are the minimum components we need to install in the machine in which the .NET code is running.
Thanks,
Loonysan
View 4 Replies
View Related
Aug 4, 2006
What SSIS components must be loaded onto the server?
To run an SSIS ETL program, exactly what executable, components, programs, etc. must reside on the server that will be running the program. Our developers write an test their ETL programs locally, on their desktops or laptops, but must upload the scripts to the production server for execution. Obviously, they are not going load Visual Studio and SQL 2005, etc. up to the server just to run the scripts (tho SQL 2005 may be on the same box as one of the DB engines used for ETL).
Any info would be appreciated.
View 7 Replies
View Related
Jan 15, 2015
MSDN states the following on: Readable Secondary Replicas (AlwaysOn Availability Groups) for SQL Server 2014:
Limitations and Restrictions:
Change tracking and change data capture are not supported on secondary databases that belong to a readable secondary replica:
Change tracking is explicitly disabled on secondary databases.
Change data capture can be enabled on a secondary database, but this is not supported.
This confuses me: You can not track the changes. However you can enable CDC?
The scenario I am trying to achieve is to use SSIS CDC components on an asynchronous secondary replica. Is this possible? If not what would be other viable approaches?
View 0 Replies
View Related
Jun 13, 2006
Greetings SSIS friends,
I have been attempting to implement one of our numerous ETL processes in SSIS but hit a brick wall when I tried replacing a complex stored procedure with a series of Merge Join components.
In the end, I had to settle with using a SQL task which merely calls the stored procedure and this proved to be the better option as the other version where I used SSIS components only took forever to run.
How do people cope with complex transformations?! Do you guys opt for pure TSQL to perform complex transformations and use SSIS components for control flow+simple(ish) data flow tasks?
I am confused.
View 23 Replies
View Related
Sep 29, 2007
Hi,
I have a SSIS Package which I would like to modify using SSIS API. I need to put new component between some two existing data flow's components. During this process I need to disconnect two data flow's components using SSIS API. How can I do that?
Thanks,
Rafal
View 1 Replies
View Related
Feb 6, 2008
How do i access global variables in SSIS scripts ?
the Dts.variables("VarName").Value
should work but doesn't ?
do i need to reference the variable another way ?
View 3 Replies
View Related
Jun 30, 2006
Hi,
In another thread Jamie Thomson very informatively said "The components in SSIS
are deliberately atomic (i.e. they do something very specific) so that
its easy to put them together to build something greater than the sum
of the parts". Which does make a lot of sense. However, I've been finding that I end up having to create exactly the same "pattern" of combined transform components again and again in order to solve the same problem but in different dataflows (or even within the same dataflow). Cut-and-paste-tastic! In order to obtain real re-use, it seems to me like SSIS is crying out for an easy way to create new components by using composition - i.e. the ability to take commonly-used combinations of existing components and create new "super" components (without having to write Custom Transform Components in C#/VB.Net and handle everything in code).
Does anyone know if this sort of functionality is likely to make it into SSIS in the forseeable future?
Regards,
Lawrie
View 6 Replies
View Related
May 10, 2007
The following is a list of questions that I have not been able to obtain concrete answers. I am probably missing something:
1) ReadWriteVariables -- can the updated value for a ReadWriteVariable be accessed within the same data flow? It appears not as I think the PostExecute() fires at the completion of the data flow not the end of the Script Component. Secondarily, the Script Component is a non-blocking transformation so the component does not "see" the end of the pipeline prior to sending data down stream.
2) Record Count -- Because of #1 above, How could you calculate a record count for a data stream? It does not appear that one can calculate the number of records for a data stream within a data flow and then access the count from within the same data flow.
3) FinishOutputs() -- Is the concept of FinishOutputs() applicable to Script Component Destinations? Asked another way, is FinishOutputs() executed at the end of the data stream regardless of whether there are "real" outputs for the component? I can create a "Dummy" output to create FinishOutputs() but is this ok?
4) Script Component -- It appears that the Script Component Source, Transformation or Destination are really defined based on the columns defined in "Inputs and Outputs". Can you convert an Source script component to a transformation script component by simply adding an Output?
Sorry for these basic questions but I am not getting it completely. As you can tell...
View 12 Replies
View Related
