Strip DTD Out; Help With Some XSLT Inside Of SSIS XML Task
Sep 4, 2007
I'm reading over this thread: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1884062&SiteID=1
and I'm kinda lost as to what to do to strip out the dtd from an XML file I am downloading. I do NOT know XSLT and for that reason, I can't follow his logic.
My SSIS package downloads my XML file just fine, now I need to do a strip of the DTD line in my XML Task.
The person who provided the solution in the above post said to do this...
Code Snippet
Operation Type: XSLT
Source Type: Variable
Source: Variable's name containing the xml text
Save Operation Result: True
DestinationType: Variable
OverwriteDestination: True
Destination: Variable's name which is to contain the original xml minus the DTD.
SecondOperandType: Variable
That stuff I understood. I'll replace variables with my files because they are stored that way, but from what I can tell, that's not my problem.
The stuff he says below this comment is going over my head like a ton of bricks. I can't figure out how to do it.
This is the kind of line of my XML that I want to strip out.
https://www.myaddy.com/pbdr.dtd"[]>
and then he said this...
Code Snippet
Since XSL doesn't know about DTDs, telling it to copy everything strips out DTDs. Then use the Variable specified in the Xml task's SecondOperand as the Source data for the xml source.
A note on how to paste a multi-line xml document into a Integration Services String variable:
Integration Services String variables textboxes are not multi-line, in the Windows sense of a line (CR+LF),
So, in order to paste multi-line text (which xml docs almost always are), save a temporary copy with a unix line ending.
That is, create an xml file in visual studio, and paste your sample original xml in there. Go to File/Advanced Save options, and save the xml with the the settings of Encoding: Unicode (utf-8 without signature) - CodePage 65001, and most importantly, set the Line endings dropdown to "Unix (LF)".
After selecting "OK", copy and paste the text from Visual Studio's xml file editor into the IS variable, and you'll note all the xml data appears.
So can anyone walk me through a dummies version of what he is suggesting to do?
Can I use the XML task in SSIS to create an excel document? If so which operation type is best to do it with, XSLT?
I'm trying create excel files dynamically with nothing more than a SQL statement that I'm passing as a variable which generates XML. I would then like to take that variable and combine it with a template and create an excel document. Any help would be appreciated. I do not want to use the data flow because it requires all transformations ahead of time.
I've a xml file and I want to substitute some of the values from database in this xml using XSLT transformation making use of XML task. How can this be achieved?
In the XML Task if you set the OperationType to XSLT, how do you pass arguments to the Transform like you would in .Net by using the XsltArgumentList class? Thanks.
Where does output from <xsl:message> stylesheet elements go? It's not in the Progress or Output window, and there doesn't seem to be a property that controls the destination for messages.
In a Data Flow, I have the necessity to use a SSIS variable of type €œObject€? inside Script Component and assign to it the content of 'n' variables of string type. On exiting from the script the variable of type object should contain something like in the following lines: AAAAAAAAAAAAAAAAAAAAAAAAAAAAA BBBBBBBBBBBBBBBBBBBBBBBBBBBBB CCCCCCCCCCCCCCCCCCCCCCCCCCCCC DDDDDDDDDDDDDDDDDDDDDDDDDDDDD €¦€¦€¦€¦€¦€¦€¦. €¦€¦€¦€¦€¦€¦€¦. On exiting from the data flow I will use the variable of type Object in a Script Task, by reading each element in a cyclic fashion. Is there anyone who have experienced something like this? Could anyone provide any example of that? Thanks in advance!
Running this code on my PC via VS 2005 .Net version 2.0.50727 on the server (shown in IIS) Code is in ASP.NET 2.0 and is a VB.NET Console application SSIS 2005
Problem & Info:
I am bringing in an Excel file. I need to first strip out any non-detail rows such as the breaks you see with totals and what not. I should in the end have only detail rows left before I start moving them into my SQL Table. I'm not sure how to first strip this information out in SSIS specfically how down to the right component and how to actually code the component to do this based on my Excel file here: http://www.webfound.net/excelfile.xls
Then, I assume I just use a Flat File Source coponent or something to actually take the columns in the Excel and split into an OLE DB Datasource to shove each column into a corresponding column in my SQL Server Table. I have used a Flat File Source in the past to do so with a comma delimited txt file but never tried with an Excel.
Desired Help:
How to perform
1) stripping out all undesired rows 2) importing each column into sql table
I have a variable in SSIS that I want to access inside the Script Task. I assigned the variable in the ReadOnlyVariables in the Script Task property. How do I access it?
Can we include a Send Mail Task inside a transaction? We intend to callback a sent email in case there is an error in the subsequent task, if it is possible.
We are designing an ETL solution for a BI project using SSIS.
We need to load a dimension table from a source DB into the DW; inside the DW there are two tables for each source dimension table: a current dimension table and a storical dimension table.
The source table includes technical columns that indicate if each record was processed correctly by ETL procedures.
Each row in the source table can be a new record, or an exisisting one. If it's a new record, a corresponding new record should be inserted into the current dimension table of the DW; if it's an exisisting one, a new record should be inserted in the storical dimension table and the existing record in the current dimension table should be updated.
Furthermore for each record we need to update the source table with an error code. If the record was processed with succes the code is zero, otherwise it contains an error code.
I try to use a single data flow task, using the 'ole db command trasformation' task for managing update and 'ole db destination' task for managing insert.
The problem is that some insert and update should be executed inside a single transaction, for each record; for example if the source contains a new record I should insert the record in the current dimension table and update the source record with error code zero if every think goes fine.
There is some way to group task in the data flow pane in a single transaction ?
I've been looking around but haven't yet found the syntax for usage of global variables in an SQL Task.
I've set the global variable Id (see code below):
if (select field from table where id = @[User::id]) is null select top 1 1 as response from table else select top 1 0 as response from table
My objective with it is to set another global variable (@isNull). Supposably, when the selection returns null, I should set the variable to null, I did it by using the selections and mapping the response to that variable (is ther a better way to do so?).
When I try to execute this, it says the variable has not been defined. Here is the error:
Error: Must declare the variable '@'.
I've also tryed it withou the brackets and the User:: thing in the beggining, (@id directly) and here is the response:
Error: Must declare the variable '@id'.
How should I access the global variables in the SQL code? (BTW, I've checked the field in execution time and it is set to 23, the correct Id, so the block that preceedes this one is working properly)
Hi, I'm trying to loop thru a table and insert records into another table in ssis. So far I have been able to get the data using a execute sql task set up to store the full result set into a variable called data. I then drug a foreach loop container out and selected the foreach ADO enumerator and used my variable data as the ADO object source variable. I then set up a new variable under variable mappings with index 0 to get the collection values. How do I take that variable and update another table using another sql task inside the foreach loop container? Is this possible?
I have a class that utilizes the ThreadPool together with a ManualResetEvent to queue up work items (in this case, FTP downloads) and then waits until all work items have completed before proceeding.
I then use this class inside a Script Component. However, I am getting a "remote procedure call failed" every single time the FTP component starts a download. This same class works perfectly in a WinForm app that I created as a test harness. Also, the synchronized version of the FTP download component works perfectly well from the Script Task -- it's only when I "wrap" the synchronous version in a threaded version that it fails.
Also, when the "remote procedure call failed" exception occurs, sqldumper.exe runs for a split second and creates a minidump. I, unfortunately, cannot inspect the dump because VS2k5 throws a native error when I try to "debug" the dump.
Is it possible to utilize multithreaded code inside SSIS? I noticed that SQL Server prohibits the use of threading as part of its hosting requirements (http://msdn2.microsoft.com/en-us/library/ms172338.aspx) but I'm not sure how that would impact (if at all) the SSIS host.
I have tables like the one below for my Stage and dimension tables:
Stage Table
accountid
name
address
Dimension Table
accountkey ---- surrogate key (DW key)
accountid ---- business key (transaction's primary key)
name
address
I used slowly changing dimension to detect the changes for the records inside my Dimension table. But I had a problem when a new record exists in the stage table. The accountkey is set as the primary key and it gets its value from a different table which stores the last account key that was created. I cannot load all the changes unless i have a business key. Is there a way that i can get the "last key" from a different table in the data flow area and then supply it together with the other fields in the new output branch of the slowly changing dimension?
I am using an XML task for validating the XML data against the Schema XSD. I have more XML Files with same the schema. I have to used a for loop container which has an XML task for validate XML. The loop container gets the XML File into variables name "XMLFileName" which the loop current file, in turn, used by XML Task for validation.
XML task is configured in the option "Validate". I have provided the XML Data in variable name "XMLFileName" also get the name from the loop container and XSD file content File Connection. XML Task stored the result in the another variable name "OKFormat". FailOnValidationFail property set to false.
It had the error when I run the package, the error msg as below:
Error: 0xC002F304 at XML Task, XML Task: An error occurred with the following error message: "Data at the root level is invalid. Line 1, position 1.".
Error: 0xC002928F at XML Task, XML Task: Property "New Source" has no source Xml text; Xml Text is either invalid, null or empty string.
Task failed: XML Task
After that, I had to change the source type from Variable to File Connection, and had to test fixed to some xml file it had ok for validate the XML file againt XSD.
I don't know this is the wrong setting or bug of SSIS, anyone can guide me through.
I am looping through xml files and validating them with the XML Task. On sucess I want to perform task 2, on failure I want to perform task 3. When one file fails to validate all subsequent tasks fail.
I have also tried putting the result in a variable instead of using the success constraint. I get the same result, the variable keeps the failed value after the first file fails. Any advise is greatly appreciated.
in a SSIS 2012 pkg, I'm trying to specify a SELECT TOP ? myColumn FROM myTable inside an Execute SQL task, but unsuccessfully.Is it possible to parameterize the TOP clause?
I am using the "Transfer SQL Server Objects Task" to copy some tables from database A to database B including data.
The tables, primary key constraints, Foreign key, data and all transfers nicely except for "DEFAULT CONSTRAINTS" on the tables.
I have failed to find any option in the "Transfer SQL Server Objects Task" task to explicitly say "copy default constraints". So I guess logically it should happen automatically but it doesn't. I hope it is not a bug :-)
In short, does the €œTransfer SQL Server Objects Task€? support distributed transactions?
In trying to use a €œTransfer SQL Server Objects Task€? in a container using a transaction on the container. The task is set to support the transaction. It is setup to copy table data from several tables from a non-domain server (sql server 2000) to a domain-based server (sql server 2005). I get an error stating, €œThis task can not participate in a transaction€?.
I am wondering if it means exactly what it says €“ this task in SSIS can€™t participate at all. Or does it mean that it won€™t in this scenario for some reason. I attempted a simple copy of data from mssql 2005 to mssql 2005 (same server) and the task still failed). MSDTC appears to be running properly on my machine and such (I can do a simple distributed transaction across linked server to the 2000 server in Query Analyzer (QA)). Also, MSDTC appears to be working on both servers with distributed transaction query tests in QA.
Here€™s the error info€¦
SSIS package "Development BusinessContacts and Products Migration.dtsx" starting. Information: 0x4001100A at Copy BusinessContacts Data: Starting distributed transaction for this container. Error: 0xC002F319 at Copy BusinessContacts database table data 1, Transfer SQL Server Objects Task: This task can not participate in a transaction. Task failed: Copy BusinessContacts database table data 1 Information: 0x4001100C at Copy BusinessContacts database table data 1: Aborting the current distributed transaction. Information: 0x4001100C at Copy BusinessContacts Data: Aborting the current distributed transaction. SSIS package "Development BusinessContacts and Products Migration.dtsx" finished: Failure. The program '[4700] Development BusinessContacts and Products Migration.dtsx: DTS' has exited with code 0 (0x0).
A common issue that I run across with clients is they want only want to process a file if it's finished transmitting to the server. This SQL Server 2005 task reads the properties of a file and writes the values to a series of variables. For example, you can use this task to determine if the file is in use (still be uploaded or written to) and then conditionally run the Data Flow task to load the file if it's not being used. You can also use it to determine when the file was created in order to determine if it must be archived.
I've got a serious problem with SSIS packages. We've built an application -done with Framework 2.0- which throw SSIS packages on demand -according a criteria on NTFS- Everything's fine but those packages which own ADO .Net connections. When our application meet some of them appears in our log:
I attached you the snippet of code:
pkg = app.LoadFromSqlServer(ObjSSIS.sRutaDts & "" & ObjSSIS.sSSISName, ObjSSIS.sServer, "usrSSIS", "ninot", EventsSSIS) Till here, fine no throws any event.
sResultDts = pkg.Execute(Nothing, Nothing, EventsSSIS, Nothing, Nothing) 'here throws OnError event when meet the task issued
It doesn't acquire the connection "<server.database.user>." It could be that the connection is not fine customized or maybe permissions are not suitable for this connection.
At the same time we've verified that all data sources are available. In a fact that same packages from the server is launched without incidences (manually)
Please let me know any comment or further information or any link related with this problem. With OLE Db connections we haven't problems at all.
I have a RPC which gives me multiple - single record rows like
Robert|K|Half|TX|1123823|1423904 -- This is one such record that i get in 1 Column
Now I need to split the above record into 6 Fields and populate in my local DB I know i could use any .NET Language and accomplish this. But Iam limited to using SSIS/T-SQL Proc's
I am working on a technical design of data integration ETL package which will be moving data from SQL Server Source to DB2 destination. I currently have two options, when moving data to DB2(IBM AS400). I can call a AS400 Stored Procedure, and pass in the data to the stored procedure, and perform the insert processing within the AS400 environment or I could do inserts from SSIS in a DFT and write individually to AS400 tables. My question is from a performance and good practice perspective, which method should I move forward with. I need a possible list of pros- and cons when using AS400 Sproc vs using SQL within SSIS? I would really appreciate response from individuals who have done something similar in the past. Thanks a lot and I am really looking forward to responses.
I have created a package which executes every 10 mins. Last week end for maintenance purpose, I shutdown my database. Now as an initial execution process, my package does the default validation steps on which the database connection validation step fails. As this is the default functionality of SSIS I am not able to capture this error. Is there anyway to capture this error inside SSIS Package?
I've created my own posting for this. The original post was here, I apologize: http://forums.microsoft.com/forums/ShowPost.aspx?PostID=2906512&SiteID=1
According to the poster it's not possible. But there has to be some way to do it? Reflection (don't know how)?
I need to get a reference to the task host in an SSIS Task component.
Basically the scenario is this:
I have a custom task I have created. However I would like to validate that the ExecValueVariable is infact a string variable during the validate event of the task. I know how to verify its a string variable. But I can't figure out how to read what the user selected (such as User::Myvariable). The only way I've been able to figure out how to do it, but it only works if you open my custom task UI.
What I did is this:
I've implemented IDtsTaskUI and during the initialize section I wrote:
Sub Initialize(ByVal taskHost As TaskHost, ByVal serviceProvider As IServiceProvider) Implements IDtsTaskUI.Initialize ' Store the TaskHost of the task. Me.taskHostValue = taskHost Dim myTask As CustomTask= CType(taskHost.InnerObject, CustomTask) myTask.myTaskHost = taskHost End Sub
My Task is named: CustomTask. I have a public variable in my task as follows:
Public NotInheritable Class CustomTask Inherits Task Implements IDTSComponentPersist Public myTaskHost As TaskHost = Nothing
Therefore I pass back the taskhost value to the CustomTask class, and voila I have it.
Problem is, this only works if the custom task calls the initialize method, and this only happens when you open the custom editor.
I then do the validation in my CustomTask class and it works fine, but myTaskHost is null/nothing until you actually open the custom task UI
I want to delete and recreate the master Database Device for purpose of shrinking the size of the device . I dont want to loose the Master db.
I thought of transfering the Master db to a different device, Delete and recreate the master device and then tranfer back the master db . and this is all done through the MS SQL enterprise Manager (SQL 6.5 ) Is this the safest eay to proceed with this task?
