The Component Metadata For Component DataReader Source Could Not Be Upgraded To The Newer Version Of The Component.
Jan 23, 2007
Hi,
I have a package which reads an Access file from a folder. My connection manager to this file is .NET providers for OledbMicrosoft Jet 4.0 OLE DB Provider.
Package works from my computer. But when I execute it on the server as a SQL Agent job, I get
The component metadata for "component "DataReader Source" (1) could not be upgraded to the newer version of the component. The PerformUpgrade method failed.
I copied the mdb file to a folder on the server which my packages have no problem reading data from.
My packages run under the same domain account as defined in proxies.
I have a package that has a data lfow task. this task imports data from a db2 database (using the IBM Ole DB provider fro db2) and adds it to sql server database table. This package was created on the server. then though version control (using TFS source control) I check out the package on my local machine. and when I open the package I get the foll 3 errors.
Error 1 Validation error. Import Account Num from BMGP_BDR: DTS.Pipeline: The component metadata for "component "DataReader Source" (1113)" could not be upgraded to the newer version of the component. The PerformUpgrade method failed.
Error 2 Error loading BMAG Download Xref Tables - bmag.dtsx: Microsoft.SqlServer.Dts.Pipeline.ComponentVersionMismatchException: The version of component "DataReader Source" (1113) is not compatible with this version of the DataFlow. [[The version or pipeline version or both for the specified component is higher than the current version. This package was probably created on a new version of DTS or the component than is installed on the current PC.]] at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostCheckAndPerformUpgrade(IDTSManagedComponentWrapper90 wrapper, Int32 lPipelineVersion)
Error 3 Error loading BMAG Download Xref Tables - bmag.dtsx: The component metadata for "component "DataReader Source" (1113)" could not be upgraded to the newer version of the component. The PerformUpgrade method failed.
Attempting to create a data flow task to copy data from AS/400 (DB2) to SQL2005, using an existing System DSN ODBC connection defined on the SQL2005 host.
Problem:
When adding the DataReader Source component to the package, I cannot assign the Connection Manager. Designer issues the error message:
"The runtime connection manager with the ID "" cannot be found. Verify that the connection manager collection has a connection manager with that ID."
Editing the DataReaderSrc component shows only one row under the Connection Managers tab:
The datareadersrc component editor displays the warning message: "Not all connection managers have been set. Set all connection managers.". Clicking the Refresh button causes the error message to be displayed "The runtime connection manager with the ID "" cannot be found. Verify that the connection manager collection has a connection manager with that ID."
I am prevented from assigning my Connection Manager object the DataReaderSrc.
The package already contains one Connect Manager object:
Provider: .Net Providers/Odbc Data Provider System DSN
I have created an SSIS On my PC, That extracts data out of Lotusnotes. When I run it manaully from Visual studion-execute package, it works just fine. However, when I go tO Management studio, chose the servername, then go to Stored Package-MSDB- And chose Packagename, and hit Run PKG from there, it gives me the error below. Server has 2005 AND IS WINDOWS 2003 OS.
The product level is insufficient for component "datareader source"
Can you tell me why the same pkg will work fine when executed from Visual studio-But when I go the run it from Storedpackages-MSDB-it fails.
I am loading/executing packages from c# and I need to populate a temp table from user input and pass this table as a variable to the datareader source components sql command. I am using expression to build this query, but I am getting design time error when I have this command..
"select id, (SysDate + 28) as ExpiresDate from Table1 where id in (Select Id from" +@[User::Table2]+")"..
I have declared Table2 as a variable of type Object and I am creating Table2 in C# and I am assigning that Table to the user Table. But in the design mode, I am getting an error...expression cannot be evaluated.
In a Data Flow, I have the necessity to use a SSIS variable of type €œObject€? inside Script Component and assign to it the content of 'n' variables of string type. On exiting from the script the variable of type object should contain something like in the following lines: AAAAAAAAAAAAAAAAAAAAAAAAAAAAA BBBBBBBBBBBBBBBBBBBBBBBBBBBBB CCCCCCCCCCCCCCCCCCCCCCCCCCCCC DDDDDDDDDDDDDDDDDDDDDDDDDDDDD €¦€¦€¦€¦€¦€¦€¦. €¦€¦€¦€¦€¦€¦€¦. On exiting from the data flow I will use the variable of type Object in a Script Task, by reading each element in a cyclic fashion. Is there anyone who have experienced something like this? Could anyone provide any example of that? Thanks in advance!
Hi all I'm into a project which uses a lot of views for joining 2 or more tables. Using the MERGE component in SSIS will be a huge effort coz it only has 2 inputs and I gotta SORT the input too. Isnt it possible to have a VIEW like component that joins more than 2 tables and DOESNT need sorting?? (I've thought about creating views in database engine but it breaks my data floe in SSIS and is'nt a practical solution)
I am writing a custom dataflow transformation component and I need to get the name of the preceeding component.
I have been trying to find a way to get a reference to the Package object, MainPipe object or IDTSPath90 object (connecting to the IDTSInput90 of my component) from my component because I think from there I can get to the information I want.
No idea where this bug crept in from. Have been using SSIS for 1.5 years now without hitting this problem.
I had a script component opening an XML document and parsing it using XPATH. I added some code that uses StreamReader / Streamwriter (closing one stream before starting the other). The code works without issue in my C# app.
And it ran without issue 2-3 times in SSIS. Then suddenly after running my package again, the script component says it completes successfully, yet nothing happens. I set a breakpoint on the first line of code - it never hits it. I add a msgbox as the first line of code - and it never displays.
I then close my package / exit out of ssis ... and then re-open it. When i open my script component, all of my code is GONE. All references that I added are gone.
I tried adding the streamreader/writer process to a dll I created from my c# app ... and added the DLL to the package -- same result.
I can reproduce this on 2 different computers.
Anyone experience this problem ? Any idea how to stop it ? Or debug it ?
Here is a slimmed down code sample of what causes the error :
Public Class ScriptMain Public Sub Main() Try Dim xmlDoc As New XmlDocument xmlDoc.Load("c:ulkasync_86281519_20070628045850225_4.xml") MsgBox("xmlLoaded") --this doesn't display once the package starts "acting up" Catch ex As Exception MsgBox(ex.Message) UpdateXML("c:ulkasync_86281519_20070628045850225_4.xml", ex.Message) End Try Dts.TaskResult = Dts.Results.Success End Sub Private Sub UpdateXML(ByVal fileName As String, ByVal message As String) Try Dim invalidChar As String = message.Trim().Substring(message.Trim().IndexOf("0x"), 4) Dim rd As StreamReader = New StreamReader(fileName) Dim xml As String = rd.ReadToEnd() Xml = Xml.Replace(invalidChar, String.Empty) xml = xml.Replace("", String.Empty) xml = xml.Replace("<![CDATA[<![CDATA[", "<![CDATA[") xml = xml.Replace("]]>]]>", "]]>") MsgBox("replaced") rd.Close() Dim wr As StreamWriter = New StreamWriter(fileName) wr.Write(xml) wr.Close() Dim xdoc As XmlDocument = New XmlDocument() xdoc.Load(fileName) Catch ex As Exception UpdateXML(fileName, ex.Message) End Try End Sub End Class
I am getting the following exception when attempting to read from a DataReaderDestination:
System.Exception was unhandled Message="Could not obtain a DataReader object from the specified data flow component." Source="Microsoft.SqlServer.Dts.DtsClient" StackTrace: at Microsoft.SqlServer.Dts.DtsClient.DtsCommand.internalPrepare(Boolean fReaderRequired) at Microsoft.SqlServer.Dts.DtsClient.DtsCommand.ExecuteReaderInThread() at Microsoft.SqlServer.Dts.DtsClient.DtsCommand.ExecuteReader(CommandBehavior behavior) at CA3DataImportTool.ViewSSISOutput.btnRun_Click(Object sender, EventArgs e) in C:Documents and Settings heinMy DocumentsVisual Studio 2005ProjectsCA3DataImportToolCA3DataImportToolViewSSISOutput.cs:line 35 at System.Windows.Forms.Control.OnClick(EventArgs e) at System.Windows.Forms.Button.OnClick(EventArgs e) at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.ButtonBase.WndProc(Message& m) at System.Windows.Forms.Button.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam) at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg) at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32 dwComponentID, Int32 reason, Int32 pvLoopData) at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context) at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context) at System.Windows.Forms.Application.Run(Form mainForm) at CA3DataImportTool.Program.Main() in C:Documents and Settings heinMy DocumentsVisual Studio 2005ProjectsCA3DataImportToolCA3DataImportToolProgram.cs:line 18 at System.AppDomain.nExecuteAssembly(Assembly assembly, String[] args) at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args) at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly() at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart()
If I use the example from SQL Server BOL (http://msdn2.microsoft.com/en-us/library/ms135917.aspx), and make a new package for the sample in the same project, the sample works. The only thing that I can see that is significantly different between my code and the sample is that my DataReaderDestination has a lot more data in it, but here's the relevant code:
string dtexecArgs;
string dataReaderName;
DtsConnection dtsConnection;
DtsCommand dtsCommand; //IDbCommand
IDataReader dtsDataReader;
DataTable dtsTable;
dtexecArgs = @"/FILE ""C:Documents and Settings heinMy DocumentsVisual Studio 2005ProjectsCA3DataImportToolML3000_IntegrationProjectPackage.dtsx"" ";
dataReaderName = "DataReaderDest";
dtsConnection = new DtsConnection();
dtsConnection.ConnectionString = dtexecArgs;
dtsConnection.Open();
dtsCommand = new DtsCommand(dtsConnection);
dtsCommand.CommandText = dataReaderName;
dtsDataReader = dtsCommand.ExecuteReader(CommandBehavior.Default); // EXCEPTION HERE
I'm having a tad bit of trouble getting output from an asynchronous component that I've written and am looking for some insight.
This component takes in a name string passed from upstream and parses the name components into standardized output fields. I'm using an asynchronous component because if the name string contains two names ("Fred & Wilma Flintstone") I'm outputting one row for Fred and one for Wilma. I've gotten it to run and with debugging have observed what appeared to me to be proper execution, but zero rows are flowing out of it.
In my ProvideComponentProperties method, I add the three fields and there associated metadata to the OutputColumnCollection. Is this method where this should occur? It's before the PrimeOutput method, so I didn't know if I should be creating the output columns in ProcessInput (i.e., after the output buffer is provided by PrimeOutput.)
In ProcessInput, I'm using AddRow for each input row and another if it contains a second name, setting the value for each index using the buffer's SetString method, to no avail. I can observe it to this point, but then don't know what's in that output buffer (if I'm using the wrong buffer index value, etc)
Hi, I have a datareader component of which i am dynamically setting its sqlcommand statement with expression (click the background of dataflow > properties > expressions). Now my sql select statement has about 600 fields so that makes my expression statment "select field1, field2, .....from table1 where field2 >=" + @[User::dateforfield2] but when i evalute the expresssion (which is right), i get the error: A string literal in the expression exceeds the maximum allowed length of 4000 character and i think its because of the fields in my select statment causing my string literal to grow more than 4000 characters. Is there any way to increase the max string literal for expressions. Please help.
I started this thread as the last attempt to sort the issue out.
I have an SSIS package that loads data from a .csv file into my database. It works fine on my developer machine. I start it programmatically or from Management Studio or from Visual Studio, and it works. Then I deploy it to the MSDB database on the computer on which it will have its final place. There again I can start it from Management Studio or from Visual Studio (using the source file), and it works. But when I start it programmatically, it just fails telling me:
Package Warnings: Package Errors: The version of component "****" (11773) is not compatible with this version of the DataFlow. Component "component "Derived Column" (13627)" could not be created and returned error code 0x80070005. Make sure that the component is registered correctly. The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "Flat File Source;Microsoft Corporation;Microsoft SqlServer v9; (C) 2005 Microsoft Corporation; All Rights Reserved; http://www.microsoft.com/sql/support;1". component "Agreement File" (11773) failed validation and returned error code 0xC0048021. One or more component failed validation. There were errors during task validation.
Maybe the source of the problem is that this machine is a 64-bit one, while the developer machine is 32-bit one. But why does the package run fine when I start it from Management Studio?
I have an XML data file and an associated XSD file with properly defined datatypes. However, the datatype of all the data elements are always "string" datatype. For example, in my current xml file, all the data elements are of Decimal datatype which is properly defined in XSD file. However, datatype of all the output columns are of string datatype.
I have a problem of performance with my XML source component. In my XSD file, I have over 300 outputs, for each output of the XML component I generate a Row file (so I generate 300 Row File in the same Data Flow) that I integrate into a table in another Data Flow. When I debug just the data flow of the XML source, it takes much time over 10 hours and the XML source component keeps a yellow color, and it generate empty Row Files, but when I interrupt the debug process it generates the right row files. For information I use a machine with 8 microprocessor and 4GO of Ram, and my XML file has 850KO. So my question is, how can I increase the performance of my package without splitting the XML file. Thank you in advance
Any knowledge of an SSIS source pipeline component which reads the JSON, a data interchage format.
JSON looks pretty tempting for heavy data interchange (somewhat human read-able, name/value pairs + arrays, nesting, lighter weight than most xml serializiers), and if its gaining momentum, I should think a source component would follow on (most likely third party)
I want to use Script Component as Source, but I don't know how to code the output rows. Someone can give me some clue or some sample codes? Thanks in advance.
I need to iterate through a bunch of packages and replace the source component in all data flows with a different source component type. That much I've got figured out, but I'm wondering the best way to fix all the LineageID references downstream.
In figured there would be some helper classes to do that for you or something, but the best I could come up with so far is build a dictionary... the key being the LineageIDs from the source component I just deleted... the dictionary value being the corresponding new column's LineageID. Then I run the following code. Is there a better way?
Also, derived column transforms end up broken because the expression is still referencing the old LineageIDs, so I had to write a RefreshExpression function and run it on each input and output column downstream. That function replaces the Expression (which uses the LineageID) with the FriendlyExpression (which uses the column name). Also, I had to name the new source component the same name as the old so that two-part names (i.e. [My Source].[My Col]) in the derived column expressions would still be valid. There's got to be a better way! And I'm worried that derived column transform isn't going to be the only special case.
While working on a recent project, I needed to import a number of old mainframe generated text reports. Since they didn't work on the one-record/one-row construct, the Flat File Source won't work.
So, I created a custom component that allows you to specify a Regular Expression pattern, and it will parse a text file and return columns. Each capture in the RegEx pattern is returned as a column, and you can also specify column names in the regex with the standard (?<colname>) syntax. The code is fairly basic really, but if you are comfortable with regular expression syntax, it can handle a huge variety of unusual text file formats. It also includes a UI editor for the RegEx pattern property that will allow you to test the pattern against a sample text file; it will highlight each row in blue and each column in green (see screenshot)
Since this might have applications for other folks, I added it to SourceForge under GPL. So if you might find that functionality useful, please check out http://sourceforge.net/projects/textregexsource/ and send me some feedback. There isn't much there yet, so check out the screenshots/news/release notes for the basics.
If there is demand, I'll create a more robust install package and documentation. Also looking for feature suggestions.
I am using XML source component to integrate an xml file into an sql server data base. So for this I have tried to test only this XML component with a small XML file (43 KO) and its XSD (434KO) (not generated using XML component), so my package contain a dataflow with only the XML source component. When I execute the package, in the progress window I get:
1- Validation step 100%, (generate warning, because the outputs are not used)
2-Preparation of execution step 100%
3-Excution step
But when the "Execution step" starts it does not stop and it does not fail, so the XML component keeps the yellow color indifinely and it generate temporary file in "Temp" repository.
I run this package in a box with SQL server 2005 SP2 with, 8 Processor and 4GO of RAM.
So is there any solution or explication for this problem. Is it a limit of SSIS? , and how can I increase the SSIS RAM.
I also generated an XSD file using the XML component, but I get the same problem.
A colleague of mine has discovered some behaviour in the XML Source component that I am having trouble understanding or explaining. Here is the (obfuscated) XML document that we are looking to parse:
I'd like to draw your attention to the bit I've highlighted in red. SSIS has incorrectly defined this attribute as an unsignedLong. if you look at the data above you'll see that it has leading zeros therefore we need it to be interpreted as a string. Unfortunately (and here's the problem) there doesn't seem to be a way for us to change the generated xsd and thus refresh the external column metadata. The only way to change it is to go into the Advanced Editor and manually change the external column and output column metadata.
Is this by design? It seems very limiting if you ask me to not let us have control of what the metadata should be.
I am debugging a Data Flow task in my SSIS package. When I run the package in debug mode, one of the OLEDB Data Sources turns red. I have rerouted all Error Output to a flat file, and put a Data Viewer on that path: no rows get sent. When I click the Preview button on this component in Design mode, I see the expected data and get no error messages. The connection does a simple table access...no SQL command. I don't see anything different between this component and other OLEDB sources in the same package that don't trigger any errors. I've tried dropping and re-creating the component with the same results.
in the acquireconnection method Using the below statment I can get a connection Object
oledbConnection = cmado.AcquireConnection(transaction) as OleDbConnection;
from the connection object I can get the connectionstring from the object by calling
oledbConnection.connectionstring() property which will have all the details like DataBase, UserName & other Inofrmation but there is no password Info.
How to get the password Information, I need that information since I will use that info to make OCI calls to fetch the data from the Oracle database in m,y custome source component.
Hi,I have developed a custom source component to get data from a generic odbc connection with some special caracteristics.
The component works fine by getting and mapping the output fields etc...
The only two problems existing are that when i run the task it says that the data flow has no components inside... how is this possible since i have my source mapped to a flat file inside that data flow?
This is the error:
SSIS package "BVEIT000D.dtsx" starting.
Warning: 0x80047034 at BVEIT000D_<EMPRESA> TXT, DTS.Pipeline: The DataFlow task has no components. Add components or remove the task.
Information: 0x4004300C at BVEIT000D_<EMPRESA> TXT, DTS.Pipeline: Execute phase is beginning.
SSIS package "BVEIT000D.dtsx" finished: Success.
The other problem is that if i want to <ignore> a certain source column the component already shows me an error saying that the no column with ID 0 was found...
Any one with experience in creating custom components?
I need to create an ODBC source script component that outputs into SQL Server. When I debug I get the following error message:
Error at Data Flow Task [Script Component [1]]: System.InvalidCastException: Unable to cast object of type 'System.Data.Odbc.OdbcConnection' to type 'System.Data.SqlClient.SqlConnection'. at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.HandleUserException(Exception e) at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.AcquireConnections(Object transaction) at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostAcquireConnections(IDTSManagedComponentWrapper90 wrapper, Object transaction)Error at Data Flow Task [DTS.Pipeline]: component "Script Component" (1) failed validation and returned error code 0x80004002.
Public Class ScriptMain Inherits UserComponent Dim connMgr As IDTSConnectionManager90 Dim sqlConn As SqlConnection Dim sqlReader As SqlDataReader
Public Overrides Sub AcquireConnections(ByVal Transaction As Object) connMgr = Me.Connections.PP sqlConn = CType(connMgr.AcquireConnection(Nothing), SqlConnection) End Sub
Public Overrides Sub PreExecute() Dim cmd As New SqlCommand("SELECT Solution_Code_From, Solution_Code_To FROM Solconv", sqlConn) sqlReader = cmd.ExecuteReader End Sub
Public Overrides Sub CreateNewOutputRows() Do While sqlReader.Read With SolutionOutputBuffer .AddRow() .solcodefr = sqlReader.GetString(1) .solcodeto = sqlReader.GetString(0) End With Loop End Sub
Public Overrides Sub PostExecute() sqlReader.Close() End Sub
Public Overrides Sub ReleaseConnections() connMgr.ReleaseConnection(sqlConn) End Sub
I have an OLEDB Source component with a Oracle OLEDB connection manager. In my SQL statement I must do something like this ...
SELECT * FROM OracleTable WHERE convert(datetime, OracleDate) = parameter
I've dynamicaly build the statement like this : http://blogs.conchango.com/jamiethomson/archive/2005/12/09/2480.aspx
Unfortunately this doesn't work because the variable is a datetime value and not a string.
Maybe another way is to store the date in a table in my SQL database and include it in my Oracle SQL statement ? This would mean that there are multiple connection managers being used in one SQL statement ... I have no idea if this would be possible ...
I'm wondering if it is possible to create a flat file source on the fly while bypassing the following step:
On the Connection Managers page, add or create the Flat File connection manager, using a descriptive name such as MyFlatFileSrcConnectionManager. Then close the Script Transformation Editor.
I want to create the connection totally in script, yet i'm having a hard time proving this out...does anybody have any experience with this?
I am trying to figure out how the Excel source component determines that a column in an Excel sheet is a datatype of float when there is no data in the sheet, only column headers.
I created a package and dragged an 'Execute Sql task' from the toolbox onto the control flow. I made the ConnectionType = 'Excel' and then Connection = 'New Connection' and used the dialog to specify a path and also checked the 'First row has column names' box. In the SqlStatement I put the following:
create table `Tab1` (Id integer ,Description nvarchar(255) )
Then I ran the package and it sucessfully created the workbook and the 'Tab1' worksheet.
Now I create a new package and I drag a 'Data Flow Task' from the toolbox onto the control flow. I edit the Data flow task and drag an 'Excel Source' onto the 'Data Flow'. I edit the 'Excel Source' component and click the 'New Connection' button and I browse to the workbook file I created above. In the 'Name of excel sheet' drop down, I select 'Tab1' and click OK. Now when I open the Advanced Editor for the 'Excel Source' and then click on the 'Input and Output Properties' tab, I expand the 'Excel Source Output' node. Then I select the 'External columns' and I select the 'Id' column and I see that the datatype is 'double-precision float'.
I am wondering how SSIS knows that it is a float even though there is as yet no data in the table only the column headers. There does not appear to be any numeric formatting either. Interestingly, if I close the Advanced Editor and then open the workbook and I merely click the 'Save' button without having made any change, then if I again look in the Advanced Editor, it now prompts me that the datatype of the 'Id' column has changed. It seems that the datatype has changed to Unicode string. Can anyone explain why this happens? Why did simply clicking the 'Save' button in the excel sheet cause the datatype to change from float to string. What actually changed in the worksheet? I am using Excel 2003, SQL Server 2005 SP2 and Visual Studio 2003 Pro SP1. Thanks for any insight into this.
