SSIS XML Source Adapter Can't Make XSD Location An Expression
Nov 21, 2007
I am using the "XML Source Adapter" in an SSIS package.
I notice that although you can specify the XML filename as an expression, the XSD appears to have to be a fixed file path.
This is a problem for me since the path for the XSD is different in my development than it will be in production (in production it's on drive E:, which I don't have).
I'd like to have the file location specified in the config file, but since I can't make it an expression how can I do that?
Also, since they don't have Connection Managers I can't switch DelayValidation on.
Thanks.
View 24 Replies
ADVERTISEMENT
Jun 1, 2006
Hi,
Quick question on how SSIS handles queries from Data Source in a Data Flow. I noticed that when I run a particular query from Query Analyzer it takes forever. But, when I run the same query in SSIS data source in a data flow. The query results are immediate.
The query plan is already cached in SQL.
Is this just something which I am seeing incorrect or is there some bit of optimization in there in SSIS. As per my understanding SSIS does not optimize the source query.
Thanks,
Gaurav
View 3 Replies
View Related
Oct 24, 2007
Hi All,
I created a SSIS package to extract data from a flat file source and load them into a table in a data base. After I created the package i checked it in to source control(perforce).
But the problem is once a month new flat source file comes and data should be updated.
Once the new flat file comes, is there anyway that SSIS package can identify the path of the flat file and execute the package automatically? In Flat file source only the data will be changed. Not location or data type or anything.
Can i use parameters to do that?
Thanks
View 6 Replies
View Related
Mar 13, 2007
Hi,
I have implemented a custom source component that can be used as the data source in the Data Flow task.
I have also created a custom UI for this component by using the IDtsComponentUI .
But my component does not have the capability of setting the custom properties via the DTS Variables using the Expression Builder.
I have looked around for samples on how to do this, but I can only find samples of how to do this for custom Control Tasks, i.e. IDtsTaskUI.
My question is, How can implement the Expression Builder in my custom Source component + custom Source UI. Or do you know of any samples which I can look at.
Thank you,
Jameel.
View 1 Replies
View Related
Jun 22, 2006
I am currently writing a custom source adapter that extracts data from a JD Edwards OneWorld system. In the custom user interface of the source component I need to allow the user to set a query (a custom property), and then refresh a list of output columns that will be extracted based upon that query (similar to the list shown in the advanced editor).
My question is, can I apply the custom property change to the component and build my output column list without closing and restarting my custom UI form? I understand that the IDTSComponentUI interface being implemented allows for transactional editing of the component, in that the changes to the component are not applied until I have returned a result in the implemented Edit() method. However is there a way to apply changes without returning this result (and closing my UI)?
Essentially I am looking to have similar behaviour to that of the 'Refresh' button in the advanced component editor form.
Thanks
View 3 Replies
View Related
Jan 4, 2007
Hi All,
Using a data access mode of SQL Command, I have a stmt that returns one date ( e.g select max(LastChangedDate) from TblA ).
I want to store the result in a variable. How do I do this?
Cheers,
Tamim.
View 13 Replies
View Related
Oct 30, 2007
I am using an "XML Source Adapter" in an SSIS Package to try and load an XML file into a database table. But having trouble with a date field.
The XSD defines the field as:-
<xs:element minOccurs="1" name="DateTime" type="xs: string" />
The XML looks like this:-
<DateTime>12/31/2007 10:12:14.123</DateTime>
And the database table column is defined:-
[DateTime] [datetime] NOT NULL,
So in the Data Flow I have a "Data Conversion Transformation" converting [DT_WSTR] to [DT_DBTIMESTAMP].
However, when I run the package I get:-
Error: 0xC02020C5 at Load XXX File, Data Conversion 1 [13550]: Data conversion failed while converting column "DateTime" (15100) to column "DateTimeDT" (15422). The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
So what how do I fix it?
I don't mind losing some miliseconds of precision, so I have tried configuring the error output of the Data Conversion for this field to "ignore failure" for "Truncation". But that seems to have no effect.
I'd be happy to change the XSD to:
<xs:element minOccurs="1" name="DateTime" type="xs:dateTime" />
But the "XML Data Flow Source" doesn't seem to support the dateTime data type.
Any suggestions appreciated. Thanks.
View 1 Replies
View Related
Apr 3, 2006
I'm having alot of trouble figuring out the proper way to log from inside my custom source adapter.
I couldn't find my useful information in MSDN, it mostly applies to logging from inside a script task.
I'd like to log my messages along with all the other SSIS package log entries, which I have going to the dts log table.
I'm assuming I should be using some functionality from Microsoft.SqlServer.Dts.RunTime, probably the LogProvider.
Can anyone advise?
View 3 Replies
View Related
Jul 3, 2007
I'm working on a new project using SSIS in SQL Server 2005 and have an issue that I need resolved.
I'm loading an XML file into SSIS using the XML Source Adapter. This maps to an OLE DB destination which reads the data into a SQL table (please see below for the table structure). In between this stage is a data conversion to convert the Unicode characters from XML into non-Unicode characters.
CREATE TABLE [dbo].[XMLTest](
[Forename] [varchar](50) COLLATE Latin1_General_CI_AS NULL,
[Surname] [varchar](50) COLLATE Latin1_General_CI_AS NULL,
[PostCode] [varchar](50) COLLATE Latin1_General_CI_AS NOT NULL
)
The problem I have is with blank strings in xml are being inserted into the SQL table as NULL values and not blanks (see the xml below).
<Root>
<People>
<Forename>Duncan3</Forename>
<Surname></Surname>
<PostCode>ME16 0WH</PostCode>
</People>
</Root>
Correct me if I'm wrong but a null value in xml means the node does not exist (See the xml below)?
<Root>
<People>
<Forename>Duncan3</Forename>
<PostCode>ME16 0WH</PostCode>
</People>
</Root>
Please also find the schema attached for reference.
<?xml version="1.0"?>
<xschema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Root">
<xs:complexType>
<xsequence>
<xs:element minOccurs="0" name="People">
<xs:complexType>
<xsequence>
<xs:element minOccurs="1" name="Forename" type="xstring" />
<xs:element minOccurs="0" name="Surname" type="xstring" />
<xs:element minOccurs="1" name="PostCode" type="xstring" />
</xsequence>
</xs:complexType>
</xs:element>
</xsequence>
</xs:complexType>
</xs:element>
</xschema>
View 5 Replies
View Related
Apr 16, 2008
I am a business user trying to build an incremental ETL package with SSIS. I have a working prototype on SQL Server 2005 where I select the max(ID) from the last successful run and pass that value into a variable. Then, in my Data Flow step, I select an OLE DB source adapter and use this variable in a custom select statement.
Here's my challenge....the live data is actually in a Postgres DB so I have to use a Data Reader Source adapter. When I try to pass my variable to this adapter the job bombs out. Does anyone know how to dynamically update the query text inside a Data Reader source adapter using variables or otherwise?
View 3 Replies
View Related
Feb 9, 2006
Hi
I am using Oledb Provider for Db2 from IBM. (There is another one from microsoft)
Through this provider I am constructing a oledb connection manager.
This connection manager I am Using in Oledb Source adapter.
Now when I set Data Access mode as Table or view I Am able to preview the data.
But when I use Data Access Mode as SQL Command try to preview I get this Error
TITLE: Microsoft Visual Studio
------------------------------
Error at s_NEWMAPPING10 [EMP [1]]: An OLE DB error has occurred. Error code: 0x80040E21.
------------------------------
ADDITIONAL INFORMATION:
Exception from HRESULT: 0xC0202009 (Microsoft.SqlServer.DTSPipelineWrap)
------------------------------
BUTTONS:
OK
------------------------------
I am not able to figure out how to resolve this.
View 3 Replies
View Related
Nov 16, 2015
I want to use the excel sheets located on a share point site as source, for this i downloaded the adapter from the following link URL....IÂ got the connection manager to work, but when coming to load the excel files to the destination location its not doing what is intended. For example i want the destination location my local drive for the excel files, how can i do that which destination should i use?
View 2 Replies
View Related
Sep 13, 2007
Is it possible to move lines based on the values from dataset(trying to draw graph kinda thing by joining points using lines) ?Let me know if I am not clear here.
Thanks
View 1 Replies
View Related
Sep 5, 2007
Hi all,
Is there any way in Reporting Services (2005) to set the Top and Left properties of the TextBox control with an expression? What I would like to do is use two integer report parameters, and alter these properties based on the parameters.
Any help very much appreciated.
Ed Allison
View 1 Replies
View Related
Mar 6, 2014
I am trying to make all reports on SSRS location specific. e.g I only people who work in certain locations will be able to view certain parameters.
I figured that I can do this using User!User.ID and an access control list. I have tested this by manually creating a sample ACL but I want to be able to automatically populate the list with data from AD.
I can find lots of stuff online that allowed me to extract information from AD but I cannot see a way of creating a full list which includes memberships.
e.g:-
USERNAME MEMBER OF
User1 location1
User1 location2
User1 location3
User2 location1
User2 location4
I am able to get users by specifying a group and getting memberships by specifying a user but I am not able to get a full list.
i have created a ADSI link server.
View 11 Replies
View Related
Jun 6, 2006
I would Like to know whether anyone tried to SSIS from SQL to AS400.
I am getting Exception when I set destination adapter ( for As400).
Whats the acceptable value .
View 1 Replies
View Related
Jul 14, 2006
I want to be able to programmatically set the schema location for an XML source. I first thought it would be a simple task using expressions and variables but it doesn't appear to allow anything in the way of setting it at runtime. Is this possible?
View 4 Replies
View Related
Nov 8, 2007
Hi,
Is there a way to set the XSD programatically? I don't want to hard code an XSD filename/location.
Alternatively, is there a way to set the location in a relative path name (i.e. relative to the path of the package)? I did see a similar question here but that wasn't answered. I tried but I wasn't getting anywhere.
As far as I can see there is no way to set this with an Expression either.
Another alternative I thought of would be to set this with using the SSIS object model but I have no experience with this and I don't know if it can be done at run time. If it can be done I would appreciate a pointer to some further information on doing this.
Thanks,
Mark
View 4 Replies
View Related
Jun 26, 2007
Hi,
I have developed a SSIS package on my desktop and the package involves loading of XML data into a database. The XML does not have inline schema and I generated the XSD file from SSIS.
I used Derived Column and Data Conversion to load the data into the database.
Now, i want to migrate the package to a server. Now, when i change the path of the XML and XSD files, all the tasks show error such as
"Input column 'Last_Updated' (4433) has lineage id 3586 that was not previously used in data flow...."
Why is this so? I am using the same XML/XSD files after i moved to the server.
Please advice a fix for this?
Regards,
Vikram
View 6 Replies
View Related
Sep 26, 2007
I have a flat source file(.csv) which I am importing to SQL server. Now, if the source file is not available at the specified location, then the SSIS package should retry to execute n times( say 3 times) after certain time interval. The number of retries and the time interval should be configurable.
I have used Flat File connection manager for the source and OLEDB connection manager for the destination.
I am quite new to SSIS. Any help would be highly appreciated.
View 1 Replies
View Related
Apr 21, 2015
So if my Parameter value is...
MCCL, MCMA, MCPL, MCPR, MCRE, MCSE, MCSN, MCVA, MCCL, MCEP, MCMA, MCREÂ out of 43 possible choices,
What is the expression syntax to hide my tablix column based on my parameter value?
=iif(instr(Join(Parameters!ColumnVisibility.Value,","),"MCCL","MCMA","MCPL"...)>0,false,true)
Just not sure of the syntax...or do I have to qualify each one???
= IIf(Parameter!PF1.Value = "MCCL" OR Parameter!PF1.Value = "MCMA"..., TRUE, FALSE)
View 6 Replies
View Related
Oct 14, 2007
Hi,
I want to create pckg with two data source,
the pckg will know to take data from the right DS after IF will check if getdate() returned endweekday it will take from DS1 else DS2.
any suggestion how to do it??
10X.
View 1 Replies
View Related
Jan 8, 2008
Hi All,
I have a knowledge of SISS and CRM 3.0 but don't get any proper steps to create such Integration Service in SQL Server 2005 using CRM 3.0 as external data source.
I don't get any option of external data source for CRM.
So if anybody know about this then let me know steps to do that.
Thanx in advance.
View 5 Replies
View Related
Jan 10, 2012
I am putting the below query in a OLEDB SOURCE through a variable (it is a select statement with a where clause from one date to another).
"select TestRecordtype, request_id from department
where LOAD_TMSTP between
(select max(END_TMSTP) LOG) and
(TO_DATE("+RIGHT("0" + (DT_STR,4,1252)DATEPART( "dd" , @[System:tartTime] ), 2) + "-"
+RIGHT("0" + (DT_STR,4,1252)DATEPART( "mm" , @[System:tartTime] ), 2) + "-"
+RIGHT("0" + (DT_STR,4,1252)DATEPART( "yy" , @[System:tartTime] ), 2) + " "
+RIGHT("0" + (DT_STR,4,1252)DATEPART( "hh" , @[System:tartTime] ), 2) + "."
+RIGHT("0" + (DT_STR,4,1252)DATEPART( "mi" , @[System:tartTime] ), 2) + "."
+RIGHT("0" + (DT_STR,4,1252)DATEPART( "ss" , @[System:tartTime] ), 2) +",'DD-MM-YY HH24.MI.SS')) "
View 1 Replies
View Related
Jun 6, 2007
Hello, I am trying to use a data source expression but I'm getting an error. Here is my data source expression:
="data source=" & Parameters!ServerName.Value & ";initial catalog=AdventureWorks"
I have created a report parameter called ServerName. However, when I deploy the report and report viewer opens, I get the following error:
An error has occurred during report processing. (rsProcessingAborted) Error during processing of the ConnectString expression of datasource €˜AdventureWorks€™. (rsDataSourceConnectStringProcessingError)
Is there some really obvious step I am missing here? The relevant parts of my rdl are below.
Thanks in advance
<DataSources>
<DataSource Name="AdventureWorks">
<ConnectionProperties>
<ConnectString>="data source=" & Parameters!ServerName.Value & ";initial catalog=AdventureWorks"</ConnectString>
<DataProvider>SQL</DataProvider>
</ConnectionProperties>
<rdataSourceID>(some stuff here)</rdataSourceID>
</DataSource>
</DataSources>
...
<ReportParameters>
...
<ReportParameter Name="ServerName">
<DataType>String</DataType>
<Prompt>ServerName</Prompt>
</ReportParameter>
</ReportParameters>
View 3 Replies
View Related
Jan 11, 2007
My report works with XMLDP data provider and obtains source data as a
result of a web method call. This memthod returns all data necessary
for the report. Thus, all of about 20 report datasets query the same
web method with the same parameters and then select appropriate data
with different ElementPath expressions. It's so inefficient and
time-consuming! I cant have respective web method for every dataset.
How can I configure SSRS to call web method only once while the report
is executing? Or how can I leverage the performance at all?
View 1 Replies
View Related
Oct 26, 2006
I'm trying to generate the data for a 2-column table, where both columns are defined as NOT NULL and the second column is a uniqueidentifier.
In SQL Server Management Studio, this works fine:insert into table_3(column_a, column_b)
select table_1.column_a, (case when table_2.column_b is NULL then newid() else table_2.column_b end) as column_b
from table_1 left outer join table_2 on table_1.column_c = table_2.column_c
That is, column_b of the SELECT result has no NULL values, and all 35,986 rows are successfully inserted into a previously empty table_3. (If I comment out the INSERT INTO clause and project table_2.column_b instead of "(case ... end) as column_b", the SELECT result includes 380 rows with a NULL in column_b, so I know the case expression plus the outer join are working as expected.)
But when I use the SELECT query as the SQL command in an OLE DB Source component that is connected directly to the OLE DB Destination for the result table, I get this error:There was an error with input column "column_b" (445) on input "OLE DB Destination Input" (420
The column status returned was: "The value violated the integrity constraints for the column.".
And sure enough, when I modify the result table to allow NULL in column_b, truncate it, and re-run the data flow, it inserts the exact same 380 rows with a NULL in column_b among the 35,986 rows.
So what is SSIS doing to screw up the results of the SELECT command???
View 6 Replies
View Related
Dec 20, 2006
Set up reports to use expression-based data sources and everything works fine as far as executing and viewing the reports. The ConnectionString expression includes the credentials, and the "No credentials required" option is selected.
Unfortunately, am getting this when trying to create a subscription for same report. WTF?
eportingServicesService!processing!3!12/20/2006-12:20:07:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Cannot create a connection to data source 'XXXData'., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Cannot create a connection to data source 'XXXData'. ---> System.InvalidOperationException: The ConnectionString property has not been initialized.
at System.Data.SqlClient.SqlConnection.PermissionDemand()
at System.Data.SqlClient.SqlConnectionFactory.PermissionDemand(DbConnection outerConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.ReportingServices.DataExtensions.SqlConnectionWrapper.Open()
at Microsoft.ReportingServices.ReportProcessing.ReportProcessing.ReportRuntimeDataSourceNode.OpenConnection(DataSource dataSourceObj, ReportProcessingContext pc)
--- End of inner exception stack trace ---
View 15 Replies
View Related
Sep 14, 2007
Hi
I am having a huge xml file with nested section.
i also have a xsd file for that xml.
i have a destination table where the data from the xml should be loaded into.
i am using the xml source transformation. But o get all the data i need to use multiple merje joins to get the data in a single row which i can insert into the destination.i was not quiet convinced with using so many joins.
so i tried using the script source transformation where i am using xml objects to get the node and dynamically construction the data row. and the output is then inserted into the destination.
on comparing the two approach the one using the script source is working much faster than the xml source transformation.
i wanted to know is there any limitaion using the script source to parse through xml files.
also i would like to know any other better way of getting the data from xml source without using the joins.
Hari
View 7 Replies
View Related
Dec 5, 2007
Hi friends,
I have Transfered my dts packages from sql 2000 to sql2005 by directly migrating the rows from Sysdtspackages table on sqlserver 2000 to Sysdtspackages table on Sqlserver 2005.Now im able to see all my DTS of sql2000 in sqlserver 2005 Management studio under MANAGMENT--->LEGACY--->Data transforamtion services and i have all the corresponding records in sysdtspackages table of MSDB database on SQLSERVER 2005.
Now i have to schedule a job for executing these dts packages. In the job schedule window, when i try to select my dts packages on the SSIS package store for the package source and go to SSIS --->MSDB, IM NOT ABLE TO FIND MY DTS PACKAGES? WHERE ARE MY DTS PACKAGES GONE. HOW CAN I SCHEDULE.
i can find another table by name 'sysdtspackages90' on msdb database.do i have to migrate the data from sysdtspackages to sysdtspacakges90 ?
Please help out.
Regards
Arvind
View 6 Replies
View Related
Oct 15, 2006
I have only SQL engine installed to my D drive. (SQL Server 2005 no SP1)
I want to install SSIS to my D drive but i can not.
It only gives me the option to install to the default drive C:...
saying that "the sql shared path has already been configured"
Is this a BUG ?
No Tools nor Notification Services are installed (which i understood to be related to the same path as SSIS)
thanks,
raviv
View 5 Replies
View Related
May 18, 2015
How to load files with similar format , from two different locations into same database with same ssis.
Lets say
Location 1: C:LoadFilesCust1APP_123445.txt
Location 2: D:LoadFilescust2VDD_543121.txt
Currently we have one ssis which loads and process files from C:LoadFilesCust1 only. we have to modify the existing package it to load files from Location 2 (D:LoadFilescust2) as well. Also while loading, the ssis should assign a value to existing column CustID depending upon the file name. File names always start with APP_ in first location. VDD_ in second location
Assign CUSTID as 100 if file name starts with APP_
Assign CUSTID as 200if file name starts with VDD_
View 1 Replies
View Related
Aug 19, 2015
how do I copy a folder from an FTP location using the FTP task in SSIS. Currently, I can only move the files in the folder one after the other but I want to copy the folder at once.
View 3 Replies
View Related
