SSIS XML Source Adapter Can't Make XSD Location An Expression
Nov 21, 2007
I am using the "XML Source Adapter" in an SSIS package.
I notice that although you can specify the XML filename as an expression, the XSD appears to have to be a fixed file path.
This is a problem for me since the path for the XSD is different in my development than it will be in production (in production it's on drive E:, which I don't have).
I'd like to have the file location specified in the config file, but since I can't make it an expression how can I do that?
Also, since they don't have Connection Managers I can't switch DelayValidation on.
Hi, Quick question on how SSIS handles queries from Data Source in a Data Flow. I noticed that when I run a particular query from Query Analyzer it takes forever. But, when I run the same query in SSIS data source in a data flow. The query results are immediate.
The query plan is already cached in SQL.
Is this just something which I am seeing incorrect or is there some bit of optimization in there in SSIS. As per my understanding SSIS does not optimize the source query.
I created a SSIS package to extract data from a flat file source and load them into a table in a data base. After I created the package i checked it in to source control(perforce).
But the problem is once a month new flat source file comes and data should be updated. Once the new flat file comes, is there anyway that SSIS package can identify the path of the flat file and execute the package automatically? In Flat file source only the data will be changed. Not location or data type or anything. Can i use parameters to do that?
I have implemented a custom source component that can be used as the data source in the Data Flow task.
I have also created a custom UI for this component by using the IDtsComponentUI .
But my component does not have the capability of setting the custom properties via the DTS Variables using the Expression Builder.
I have looked around for samples on how to do this, but I can only find samples of how to do this for custom Control Tasks, i.e. IDtsTaskUI.
My question is, How can implement the Expression Builder in my custom Source component + custom Source UI. Or do you know of any samples which I can look at.
I am currently writing a custom source adapter that extracts data from a JD Edwards OneWorld system. In the custom user interface of the source component I need to allow the user to set a query (a custom property), and then refresh a list of output columns that will be extracted based upon that query (similar to the list shown in the advanced editor).
My question is, can I apply the custom property change to the component and build my output column list without closing and restarting my custom UI form? I understand that the IDTSComponentUI interface being implemented allows for transactional editing of the component, in that the changes to the component are not applied until I have returned a result in the implemented Edit() method. However is there a way to apply changes without returning this result (and closing my UI)?
Essentially I am looking to have similar behaviour to that of the 'Refresh' button in the advanced component editor form.
I am using an "XML Source Adapter" in an SSIS Package to try and load an XML file into a database table. But having trouble with a date field. The XSD defines the field as:-
So in the Data Flow I have a "Data Conversion Transformation" converting [DT_WSTR] to [DT_DBTIMESTAMP]. However, when I run the package I get:-
Error: 0xC02020C5 at Load XXX File, Data Conversion 1 [13550]: Data conversion failed while converting column "DateTime" (15100) to column "DateTimeDT" (15422). The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
So what how do I fix it? I don't mind losing some miliseconds of precision, so I have tried configuring the error output of the Data Conversion for this field to "ignore failure" for "Truncation". But that seems to have no effect.
I'm working on a new project using SSIS in SQL Server 2005 and have an issue that I need resolved.
I'm loading an XML file into SSIS using the XML Source Adapter. This maps to an OLE DB destination which reads the data into a SQL table (please see below for the table structure). In between this stage is a data conversion to convert the Unicode characters from XML into non-Unicode characters.
I am a business user trying to build an incremental ETL package with SSIS. I have a working prototype on SQL Server 2005 where I select the max(ID) from the last successful run and pass that value into a variable. Then, in my Data Flow step, I select an OLE DB source adapter and use this variable in a custom select statement.
Here's my challenge....the live data is actually in a Postgres DB so I have to use a Data Reader Source adapter. When I try to pass my variable to this adapter the job bombs out. Does anyone know how to dynamically update the query text inside a Data Reader source adapter using variables or otherwise?
Hi I am using Oledb Provider for Db2 from IBM. (There is another one from microsoft) Through this provider I am constructing a oledb connection manager.
This connection manager I am Using in Oledb Source adapter. Now when I set Data Access mode as Table or view I Am able to preview the data.
But when I use Data Access Mode as SQL Command try to preview I get this Error
TITLE: Microsoft Visual Studio ------------------------------ Error at s_NEWMAPPING10 [EMP [1]]: An OLE DB error has occurred. Error code: 0x80040E21. ------------------------------ ADDITIONAL INFORMATION: Exception from HRESULT: 0xC0202009 (Microsoft.SqlServer.DTSPipelineWrap) ------------------------------ BUTTONS: OK ------------------------------
I want to use the excel sheets located on a share point site as source, for this i downloaded the adapter from the following link URL....IÂ got the connection manager to work, but when coming to load the excel files to the destination location its not doing what is intended. For example i want the destination location my local drive for the excel files, how can i do that which destination should i use?
Is it possible to move lines based on the values from dataset(trying to draw graph kinda thing by joining points using lines) ?Let me know if I am not clear here.
Is there any way in Reporting Services (2005) to set the Top and Left properties of the TextBox control with an expression? What I would like to do is use two integer report parameters, and alter these properties based on the parameters.
I am trying to make all reports on SSRS location specific. e.g I only people who work in certain locations will be able to view certain parameters.
I figured that I can do this using User!User.ID and an access control list. I have tested this by manually creating a sample ACL but I want to be able to automatically populate the list with data from AD.
I can find lots of stuff online that allowed me to extract information from AD but I cannot see a way of creating a full list which includes memberships.
e.g:- USERNAME MEMBER OF User1 location1 User1 location2 User1 location3 User2 location1 User2 location4
I am able to get users by specifying a group and getting memberships by specifying a user but I am not able to get a full list.
I want to be able to programmatically set the schema location for an XML source. I first thought it would be a simple task using expressions and variables but it doesn't appear to allow anything in the way of setting it at runtime. Is this possible?
Is there a way to set the XSD programatically? I don't want to hard code an XSD filename/location.
Alternatively, is there a way to set the location in a relative path name (i.e. relative to the path of the package)? I did see a similar question here but that wasn't answered. I tried but I wasn't getting anywhere.
As far as I can see there is no way to set this with an Expression either.
Another alternative I thought of would be to set this with using the SSIS object model but I have no experience with this and I don't know if it can be done at run time. If it can be done I would appreciate a pointer to some further information on doing this.
I have developed a SSIS package on my desktop and the package involves loading of XML data into a database. The XML does not have inline schema and I generated the XSD file from SSIS.
I used Derived Column and Data Conversion to load the data into the database.
Now, i want to migrate the package to a server. Now, when i change the path of the XML and XSD files, all the tasks show error such as
"Input column 'Last_Updated' (4433) has lineage id 3586 that was not previously used in data flow...."
Why is this so? I am using the same XML/XSD files after i moved to the server.
I have a flat source file(.csv) which I am importing to SQL server. Now, if the source file is not available at the specified location, then the SSIS package should retry to execute n times( say 3 times) after certain time interval. The number of retries and the time interval should be configurable.
I have used Flat File connection manager for the source and OLEDB connection manager for the destination.
I am quite new to SSIS. Any help would be highly appreciated.
Hi, I want to create pckg with two data source, the pckg will know to take data from the right DS after IF will check if getdate() returned endweekday it will take from DS1 else DS2.
I have a knowledge of SISS and CRM 3.0 but don't get any proper steps to create such Integration Service in SQL Server 2005 using CRM 3.0 as external data source.
I don't get any option of external data source for CRM.
So if anybody know about this then let me know steps to do that.
I have created a report parameter called ServerName. However, when I deploy the report and report viewer opens, I get the following error:
An error has occurred during report processing. (rsProcessingAborted) Error during processing of the ConnectString expression of datasource €˜AdventureWorks€™. (rsDataSourceConnectStringProcessingError)
Is there some really obvious step I am missing here? The relevant parts of my rdl are below.
Thanks in advance
<DataSources>
<DataSource Name="AdventureWorks">
<ConnectionProperties>
<ConnectString>="data source=" & Parameters!ServerName.Value & ";initial catalog=AdventureWorks"</ConnectString>
My report works with XMLDP data provider and obtains source data as a result of a web method call. This memthod returns all data necessary for the report. Thus, all of about 20 report datasets query the same web method with the same parameters and then select appropriate data with different ElementPath expressions. It's so inefficient and time-consuming! I cant have respective web method for every dataset. How can I configure SSRS to call web method only once while the report is executing? Or how can I leverage the performance at all?
I'm trying to generate the data for a 2-column table, where both columns are defined as NOT NULL and the second column is a uniqueidentifier.
In SQL Server Management Studio, this works fine:insert into table_3(column_a, column_b) select table_1.column_a, (case when table_2.column_b is NULL then newid() else table_2.column_b end) as column_b from table_1 left outer join table_2 on table_1.column_c = table_2.column_c
That is, column_b of the SELECT result has no NULL values, and all 35,986 rows are successfully inserted into a previously empty table_3. (If I comment out the INSERT INTO clause and project table_2.column_b instead of "(case ... end) as column_b", the SELECT result includes 380 rows with a NULL in column_b, so I know the case expression plus the outer join are working as expected.)
But when I use the SELECT query as the SQL command in an OLE DB Source component that is connected directly to the OLE DB Destination for the result table, I get this error:There was an error with input column "column_b" (445) on input "OLE DB Destination Input" (420 The column status returned was: "The value violated the integrity constraints for the column.".
And sure enough, when I modify the result table to allow NULL in column_b, truncate it, and re-run the data flow, it inserts the exact same 380 rows with a NULL in column_b among the 35,986 rows.
So what is SSIS doing to screw up the results of the SELECT command???
Set up reports to use expression-based data sources and everything works fine as far as executing and viewing the reports. The ConnectionString expression includes the credentials, and the "No credentials required" option is selected.
Unfortunately, am getting this when trying to create a subscription for same report. WTF?
eportingServicesService!processing!3!12/20/2006-12:20:07:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Cannot create a connection to data source 'XXXData'., ; Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Cannot create a connection to data source 'XXXData'. ---> System.InvalidOperationException: The ConnectionString property has not been initialized. at System.Data.SqlClient.SqlConnection.PermissionDemand() at System.Data.SqlClient.SqlConnectionFactory.PermissionDemand(DbConnection outerConnection) at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.ReportingServices.DataExtensions.SqlConnectionWrapper.Open() at Microsoft.ReportingServices.ReportProcessing.ReportProcessing.ReportRuntimeDataSourceNode.OpenConnection(DataSource dataSourceObj, ReportProcessingContext pc) --- End of inner exception stack trace ---
I am having a huge xml file with nested section. i also have a xsd file for that xml. i have a destination table where the data from the xml should be loaded into. i am using the xml source transformation. But o get all the data i need to use multiple merje joins to get the data in a single row which i can insert into the destination.i was not quiet convinced with using so many joins.
so i tried using the script source transformation where i am using xml objects to get the node and dynamically construction the data row. and the output is then inserted into the destination. on comparing the two approach the one using the script source is working much faster than the xml source transformation.
i wanted to know is there any limitaion using the script source to parse through xml files. also i would like to know any other better way of getting the data from xml source without using the joins.
I have Transfered my dts packages from sql 2000 to sql2005 by directly migrating the rows from Sysdtspackages table on sqlserver 2000 to Sysdtspackages table on Sqlserver 2005.Now im able to see all my DTS of sql2000 in sqlserver 2005 Management studio under MANAGMENT--->LEGACY--->Data transforamtion services and i have all the corresponding records in sysdtspackages table of MSDB database on SQLSERVER 2005.
Now i have to schedule a job for executing these dts packages. In the job schedule window, when i try to select my dts packages on the SSIS package store for the package source and go to SSIS --->MSDB, IM NOT ABLE TO FIND MY DTS PACKAGES? WHERE ARE MY DTS PACKAGES GONE. HOW CAN I SCHEDULE.
i can find another table by name 'sysdtspackages90' on msdb database.do i have to migrate the data from sysdtspackages to sysdtspacakges90 ?
How to load files with similar format , from two different locations into same database with same ssis.
Lets say Location 1: C:LoadFilesCust1APP_123445.txt Location 2: D:LoadFilescust2VDD_543121.txt
Currently we have one ssis which loads and process files from C:LoadFilesCust1 only. we have to modify the existing package it to load files from Location 2 (D:LoadFilescust2) as well. Also while loading, the ssis should assign a value to existing column CustID depending upon the file name. File names always start with APP_ in first location. VDD_ in second location
Assign CUSTID as 100 if file name starts with APP_ Assign CUSTID as 200if file name starts with VDD_
how do I copy a folder from an FTP location using the FTP task in SSIS. Currently, I can only move the files in the folder one after the other but I want to copy the folder at once.
