Script Destination Component Code Sample Here That Creates A Csv Or Xml File
Jul 11, 2006
After much work and thanks to all of you who helped on this here is a code sample that can be adapted. From the dataflow task add an OLEDB source component, a row count component and finally a Script Destination Component.
On the Script Destination Component rename the Input node of the imports and outputs tree view to "ParsedInput"
The readonly User: variables that start with gs can be read in the PreExecute method
The readwrite User: variable giSuccessCount can only be used in the post execute task because it is populated by the Row Count Component which is the previous object in the Dataflow
The xml code is adapted from an idea in Donald Farmers book
enjoy
Dave
Now if someone can make a Script Source Component that can read a file with a header , data body and trailer that would b egreat!
' Microsoft SQL Server Integration Services user script component
' This is your new script component in Microsoft Visual Basic .NET
' ScriptMain is the entrypoint class for script components
Public Class ScriptMain
Inherits UserComponent
Dim sw As StreamWriter
'In addition to using the Imports System.Xml statement a reference must be added to the
'System.Xml assembly (Select Project-Add Reference from IDE)
Dim xWriter As XmlTextWriter
Dim OutputFileType As String '.csv or .xml
Public Overrides Sub PreExecute()
'Read Only variables
Dim gsPickUp As String = Me.Variables.gsPickUp 'D:ftprootOutAvid'
Dim gsPickUpFilename As String = Me.Variables.gsPickUpFilename '1_AVID_'
Dim gsPickUpFileExtn As String = Me.Variables.gsPickUpFileExtn '.csv'
Dim gsMemoText As String = Me.Variables.gsMemoText 'Memo Text : credit adjustment'
Dim gsStatementText As String = Me.Variables.gsStatementText 'Statment Text : credit adjustment'
Dim gsRunMode As String = Me.Variables.gsRunMode 'UPDATE'
Dim fileName As String = gsPickUp & "" & gsPickUpFilename
fileName = fileName & (Format(Now(), "yyMMdd").ToString)
'MsgBox(fileName)
OutputFileType = gsPickUpFileExtn
If OutputFileType = ".csv" Then
fileName = fileName & gsPickUpFileExtn
sw = New StreamWriter(fileName) 'connection to dest file
'Header records
sw.Write(gsRunMode)
sw.Write(Environment.NewLine) ' end of line
sw.Write(gsMemoText)
sw.Write(Environment.NewLine)
sw.Write(gsStatementText)
sw.Write(Environment.NewLine)
sw.Write(Environment.NewLine) 'Spacer
End If
If OutputFileType = ".xml" Then
fileName = fileName & gsPickUpFileExtn
'xWriter = New XmlTextWriter(Me.Connections.XMLConnection.ConnectionString, Nothing)
'xWriter.WriteStartDocument()
'xWriter.WriteComment("Customer file parsed using script")
'xWriter.WriteStartElement("x", "customer", "http://some.org/name")
'xWriter.WriteAttributeString("FileName", Me.Connections.XMLConnection.ConnectionString)
xWriter = New XmlTextWriter(fileName, Nothing)
xWriter.WriteStartDocument()
xWriter.WriteComment("Customer file parsed using script")
xWriter.WriteStartElement("x", "customer", "http://some.org/name")
xWriter.WriteAttributeString("FileName", fileName)
End If
End Sub
Public Overrides Sub ParsedInput_ProcessInputRow(ByVal Row As ParsedInputBuffer)
If OutputFileType = ".csv" Then
Dim delim As String = ","
If OutputFileType = ".csv" Then
'Create the trailer
sw.Write(Environment.NewLine) ' blank line
sw.Write("RECORD_COUNT: " & Me.Variables.giSuccessCount.ToString) 'ReadWrite Varible
sw.Write(Environment.NewLine)
sw.Flush() 'send the stream to file
'Close file
sw.Close()
End If
If OutputFileType = ".xml" Then
xWriter.WriteStartElement("RecordCount")
xWriter.WriteString(Me.Variables.giSuccessCount.ToString)
xWriter.WriteEndElement()
xWriter.WriteEndElement()
I have been developing VBA apps in Access and Excel for sometime and am fairly proficient in VBA. Now we are moving all of our data to SQL Server 2005. I am in need of learning how to write code for the Script Component of a data flow task. And so I have a couple of questions.
First, are there any books you recommend for learning ???? (I'm not even sure what I need to learn: .NET? ADO.NET?)
And as a follow-up, any good websites that provide good reference documentation?
And my second question is more specific to my current problem. If I had a bit of code to get me started, I'm sure I could scream all the way to the bottom of the hill.
Data source is coming from a sort task where the data is sorted by STATUS and then MOD_DATE and the AUDIT_ID. I need to read each row and compare it to the next row. If STATUS is the same, discard the second row. When STATUS is different, send the first row to the output (to be used by the next task in the data flow). Using the "different" row from step 3, go to step 2.
I know how to write if statements, case statements, for/next statements. I'm just not understanding how to read the rows in and then send them back out. I've been searching for some sample code but everything I find tends to be solving much bigger issues.
Any help you can provide would be much appreciated.
I'm trying to create a simple Data transfermation. I have a flat file that came of a unix server.. it's 177 bytes wide.. thought it was 175, but when I created the flat file connector, I could see some extra characters on the end.
My output is going to be an excel spreadsheet, I only want two columns from the input. I created an oledb jet 4.0 connection. and followed instructions from here :
On my first attempt to dataflow, I ran into unicode errors and had to do this:
ran into a problem with unicode errors. went to the source for the flat file. for the output column in question changed to Unicode string [DT_WSTR].
When I run , here are the errors I get:
[OLE DB Destination [513]] Error: An OLE DB error has occurred. Error code: 0x80040E09. [DTS.Pipeline]
Error: The ProcessInput method on component "OLE DB Destination" (513) failed with error code 0xC0202009. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.
[DTS.Pipeline] Error: Thread "WorkThread0" has exited with error code 0xC0202009.
[GanchoFileSource [1]] Information: The total number of data rows processed for file "\ammia01dev04D$JCPcpmgancho_venta_20070321.sal" is 19036.
[GanchoFileSource [1]] Error: Setting the end of rowset for the buffer failed with error code 0xC0047020.
[DTS.Pipeline] Error: The PrimeOutput method on component "GanchoFileSource" (1) returned error code 0xC0209017. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.
[DTS.Pipeline] Error: Thread "SourceThread0" has exited with error code 0xC0047038.
Books online mention the existence of sample code for several custom tasks, including the one mentioned in the title. But, when I try to find this code in the location mentioned it is nowhere to be found.
I have run a search on the rest of my drive and come up empty.
No idea where this bug crept in from. Have been using SSIS for 1.5 years now without hitting this problem.
I had a script component opening an XML document and parsing it using XPATH. I added some code that uses StreamReader / Streamwriter (closing one stream before starting the other). The code works without issue in my C# app.
And it ran without issue 2-3 times in SSIS. Then suddenly after running my package again, the script component says it completes successfully, yet nothing happens. I set a breakpoint on the first line of code - it never hits it. I add a msgbox as the first line of code - and it never displays.
I then close my package / exit out of ssis ... and then re-open it. When i open my script component, all of my code is GONE. All references that I added are gone.
I tried adding the streamreader/writer process to a dll I created from my c# app ... and added the DLL to the package -- same result.
I can reproduce this on 2 different computers.
Anyone experience this problem ? Any idea how to stop it ? Or debug it ?
Here is a slimmed down code sample of what causes the error :
Public Class ScriptMain Public Sub Main() Try Dim xmlDoc As New XmlDocument xmlDoc.Load("c:ulkasync_86281519_20070628045850225_4.xml") MsgBox("xmlLoaded") --this doesn't display once the package starts "acting up" Catch ex As Exception MsgBox(ex.Message) UpdateXML("c:ulkasync_86281519_20070628045850225_4.xml", ex.Message) End Try Dts.TaskResult = Dts.Results.Success End Sub Private Sub UpdateXML(ByVal fileName As String, ByVal message As String) Try Dim invalidChar As String = message.Trim().Substring(message.Trim().IndexOf("0x"), 4) Dim rd As StreamReader = New StreamReader(fileName) Dim xml As String = rd.ReadToEnd() Xml = Xml.Replace(invalidChar, String.Empty) xml = xml.Replace("", String.Empty) xml = xml.Replace("<![CDATA[<![CDATA[", "<![CDATA[") xml = xml.Replace("]]>]]>", "]]>") MsgBox("replaced") rd.Close() Dim wr As StreamWriter = New StreamWriter(fileName) wr.Write(xml) wr.Close() Dim xdoc As XmlDocument = New XmlDocument() xdoc.Load(fileName) Catch ex As Exception UpdateXML(fileName, ex.Message) End Try End Sub End Class
I have several SSIS packages which are run in sequence. But some strange reason sometimes a SQL Dump is created. If I restore database and restart the same ETL is works, or it creates another SQL Dump in a different place. Any idea on how to debug this problem?
SQL Server 2005 Developer Edition SP1 ( should I upgrade to SP2? ) Windows 2003/r2 64b ( latest patches )
Microsoft Visual Studio 2005 Version 8.0.50727.762 (SP.050727-7600) Microsoft .NET Framework Version 2.0.50727
Installed Edition: IDE Standard Microsoft Visual Studio 2005 Premier Partner Edition - ENU Service Pack 1 (KB926601) This service pack is for Microsoft Visual Studio 2005 Premier Partner Edition - ENU. If you later install a more recent service pack, this service pack will be uninstalled automatically. For more information, visit http://support.microsoft.com/kb/926601
SQL Server Analysis Services Microsoft SQL Server Analysis Services Designer Version 9.00.2047.00 SQL Server Integration Services Microsoft SQL Server Integration Services Designer Version 9.00.2047.00 SQL Server Reporting Services Microsoft SQL Server Reporting Services Designers Version 9.00.2047.00
Does anyone have any sample code for using ntext data type, when inserting or reading big block of texts? How can you read a big huge block of text back into c#?
This is a sample code from an MSDN help site. I copied it and pasted into an open new query. I tried to execute it and got two errors:
USE AdventureWorks; GO DECLARE @tablename sysname SET @tablename = N'Person.AddressType' table_loop: IF (@@FETCH_STATUS <> -2) BEGIN SELECT @tablename = RTRIM(UPPER(@tablename)) EXEC ('SELECT ''' + @tablename + ''' = COUNT(*) FROM ' + @tablename ) PRINT ' ' END FETCH NEXT FROM tnames_cursor INTO @tablename IF (@@FETCH_STATUS <> -1) GOTO table_loop GO
The errors are:
Msg 208, Level 16, State 1, Line 1 Invalid object name 'PERSON.ADDRESSTYPE'.
Msg 16916, Level 16, State 1, Line 9 A cursor with the name 'tnames_cursor' does not exist
The database is connected. Table Person.AddressType is a part of it.
I am creating a database for an application through script. After the tables, views, and sp's are created, the database is populated with data. After all of this (and before the application is even run), the log file is about 700MB. If I shrink the database, it takes the log down to 1MB. The mdf file is about 165 MB before and after it has been shrunk.
I have two questions: 1. Is there something I should look for in my database scripts or is there a setting that could prevent this from being created so large.
2. Is there a script I can run in my sql code after the database has been created and populated to shrink it.
hi, i learn by practice and i would like an asp code using c# that connects to sql database can retrives information from a table. can you please copy paste me some code or give me exact link a?
I was thinking of using a CTE to return history, have you ever done this?
I know this can be done in other ways but I wanted to wrap this CTE in a view so people could join to it.
Here is an example table: KEY ASOFDATE A 1/1/2007 A 3/1/2007 B 1/1/2007 B 2/5/2007 B 3/6/2007
I want the results to look like: KEY START END A 1/1/2007 3/1/2007 A 3/1/2007 NULL B 1/1/2007 2/5/2007 B 2/5/2007 3/6/2007 B 3/6/2007 NULL
This would be even better:
ID KEY ASOFDATE 1 A 1/1/2007 2 A 3/1/2007 3 B 1/1/2007 4 B 2/5/2007 5 B 3/6/2007
I want the results to look like: ID KEY START END 1 A 1/1/2007 3/1/2007 2 A 3/1/2007 NULL 3 B 1/1/2007 2/5/2007 4 B 2/5/2007 3/6/2007 5 B 3/6/2007 NULL
If somebody have worked on providing the code for the selection of delivery method and delivery schedule page in aspx . Then please share it with me. I am working on creating a Subscription page using aspx rather then reporting services page.
Please let me know if there are any controls provided by reporting services or some 3rd party for this.
when loading the transformed data into OLE DB destination, there is no options to truncate destination table first. Have to insert a middle step to run script to truncate the destination table first.
I'm very confused. We even has the options of keeping or deleting the data in destination table in SQL2000 DTS package. Why we don't have this option in SQL2005??
Does anyone know where to find a simple sample showing you how to use asp.net 2.0 with the login control to direct you to another aspx page using SQL 2000 as the database on a hosted server? I have been beating my head against the wall trying to figure this out and would be very appreciative if anyone knows a place out there that shows this? I can find many sites that show you how to do this with SQL 2005, but not SQL 2000. Thanks - Chris
I am trying to write a ssis surrogate key data transform, my problem is I can't find an example how to add a column to the incoming columns and add some data to it. If anyone has a sample, can you please post it. I found a script option that works but I would like an actual transform.
I am looking for some examples of code for an appliction which will let run RS reports from a custom application. The user will choose the reports from a menu structure
We have an exiting application which we use to run crystal reports which we are going to be migrating to RS. We use custom web controls to capture the required parameter values for the reports, and hope to reuse the same controls (with minimal rework) to capture the parameters values for the RS reports.
What I am after is some code which will
a) let me run the report and display it. I don't want to display the report parameters in the URL etc as we generate some report parameter values based upon the user's ID and do NOT want them displayed back to the users
b) export the report to PDF, CSV etc depending upon the option chosen by the user.
The application will be developed in VS 2005 - VB. Does any one know of some links to some sample code
Hi, there; I try to import data from ODBC using C# programmatically. Is there any sample code we can have a look. Like how to read schema from ODBC source table and then create source column...
I am creating and running a package programmatically. I have the source component set up fine, and the destination component seems good, but when the package is run, it gets the error message: "Excel destination failed validation and returned validation status "VS_NEEDSNEWMETADATA" ". This would lead me to believe that I need a ReinitializeMetaData() call, but I already have that (see below). How do I fix this? Thanks for your help.
' Create and configure an OLE DB destination.
Dim conDest As ConnectionManager = package.Connections.Add("Excel")
In the "Example: Detecting a Poison Message" section, it reads: This Transact-SQL example shows a simple, stateless service that includes logic for handling poison messages. Before the stored procedure receives a message, the procedure saves the transaction. When the procedure cannot process a message, the procedure rolls the transaction back to the save point. The partial rollback returns the message to the queue while continuing to hold a lock on the conversation group for the message.
Is there any sample code to demo the SSB send messages with same sql instance?
my case is very simple:
I want write a stored procedure to send a xml to another database. The stored procedure is called by tables triggers when some data is changed under the specific conditions.
Hello, I am trying to create a simple package programmatically. I am following the examples in the BOL, and from some advice here. I am getting stuck at creating an Excel Destination and setting its sheetname. Everything works fine, including setting the output Excel filename. I get a runtime exception when I try to set the sheetname via SetComponentProperty. Is there another way, or am I doing something wrong? Thanks for any info you may have.
' Create and configure an OLE DB destination.
Dim conDest As ConnectionManager = package.Connections.Add("Excel")
Create a table at the beginning of a package (using a ExecuteSQLTask component) and then use the created table as a OLE DB destination component, later on the package.
Is this possible in SSIS?
The problem I run into is that I have to point the OLE DB destination component to a table and set up mappings, however as the table does not exist until the package is running, it does not seem to be possible.
Which is slightly similar to what I want, but the table I create would not be a temp tables, and I need to set up mappings and I don't see how this is possible.
How to upload excel data to sql server 2000 thorugh .net application. I want like one upload button should be there,we have to browese corresponding excel file and then we need to upload to database. before uploading it has to ask appending or replace everything in table. both options appending and replacing shoulb be there. How to upload ? any sample code is there plz give me.
I have an SSIS package that produces an Excel output file say File1 The Excel output file is created from a previous script task by copying a standard excel template to File1 after the copy, File1 has the disclaimer, legend etc... and a header row at row 10. So data rows should only start at row 11.
I was googling and found that people who read Excel file using an Excel source component have been successful reading from a range by the use of the OPENROWSET property. It is said to set this property to Sheet1$A5-B999 to start reading from row 5.
I tried to set the OPENROWSET property to be Sheet1$A11-B999 but am getting an error "Check that object exists" (Sheet1 does exist) So I guess it is the range that has an improper syntax or something else is wrong.
I am writing a custom dataflow destination that writes data to a named pipe server running in an external process. At runtime, in PreExecute, I check to see if the pipe exists. If it does not exist I want to be able to fail the component and gracefully complete execution of my package.
Question is, how do I do this? If I make a call to ComponentMetaData.FireError() that will only create an OnError event and not actually stop the execution. So, how to I halt the execution of my component and return the error message?
Is there a default destination component used when a new data flow is created? The reason I ask is simply curiosity. I have an xml file with 2 pieces of data: item A and item B. A should simply get copied out of the file. B should undergo a quick transform. I set up an XML source such that two columns are mapped correctly to the XML source data of A and B. I set up my data transform task as well. So, if I leave those two components on the .dtsx page with no other components, then will there be a default data flow destination already created? ...OR, do you always have to have a destination component?
Here is the situation. I have created a package that takes 50 columns from a comma delimited flat file. I then validate and clean the data. Next I add two columns that were not in the original source file. These two columns need to be in the 5th and 9th column position when the file is then re-written to a text file. How do i get those two columns to write out in the desired order? Any ideas?
I am writing a Custom Destination component with a custom UI. The UI contains a combo box which contains the connection names of type €œFLATFILE€?. I also have provided a button which would create a new connection of type €œFLATFILE€? by making a call to CreateConnection method of IDtsConnectionService. The combo box gets properly updated showing the connections of type €œFLATFILE€? but on clicking on the new Connection button the application hangs up. Am I missing something or is there some other way to do it?
The function are the events handlers which are called by the UI.
if (connectionService != null) { ArrayList temp_Connections = connectionService.GetConnectionsOfType("FLATFILE");
args.AvailableColumns = new AvailableColumnElement[temp_Connections.Count]; for (int i = 0; i < temp_Connections.Count; i++) { ConnectionManager runtimeConnection = (ConnectionManager)temp_Connections; args.AvailableColumns.AvailableColumn = new DataFlowElement(runtimeConnection.Name, runtimeConnection); }
args.AvailableColumns = new AvailableColumnElement[temp_Connections.Count]; for (int i = 0; i < temp_Connections.Count; i++) { ConnectionManager runtimeConnection = (ConnectionManager)temp_Connections; args.AvailableColumns.AvailableColumn = new DataFlowElement(runtimeConnection.Name, runtimeConnection); }
I am running SQL 2005 9.0.1399 and VS 2005 8.0.50727.42 (RTM.50727.4200) on Windows Server 2003 Enterprise Edition SP1. Any suggestions would be welcome.
The code sample is an extension to the RemoveDuplicates sample (Dec 2005) which comes along with the SQL Server.
Does anybody have a working Java code sample that connects to an SQLServer 2005 database on a remote host, via the default named pipe, from a client using the SQLServer 2005 JDBC driver? Could you post it, or a pointer to it?
I've gotten java.sql DriverManager.getConnection() to work fine with TCP/IP connections before. But I'm a newbie with named pipes, and unclear on how the connection string/properties are different. I've tried to piece it together from multiple docs and threads, but haven't found sample code that quite fits my situation. I think a simple working example would best clarify the syntax.
The server is not using SQL Express. Most SQLServer configuration options are defaults; the named pipes protocol is enabled.