DTS - Datapump, Transform Data

Feb 24, 2001

Hi,

On my MS SQL Server 2000, I am trying to create a generic way to load tables into my datawarehouse.

I have as input to the process a large number of table definition(s) stored individually as files on my server. And, ascii delimited data files in various locations but mostly accessible via NFS mounts.

I created two DTS package in MSSQL2K that in theory represents what I want to do:

package1
... invoke package2 with global variables to load a system of related tables

package2
... check for a trigger file
... set the "Execute SQL Task" statement to my first file
... run the "Execute SQL Task" which drop/add's a table
... set a "Connection" to a data source file that I want to use
... run the transformation
and, with that my package starts to fall apart
... set the "Execute SQL Task" statement to the next file, and
...... goback and execute it

I can't figure out how to set the table in the transformation section to the table I want to use. And, I assume next to have the transformations links between the source and new table relinked.

The source files contain in the first row the column names as found in the tables I just created.

thanks,

Dave Rowsome

View 1 Replies


ADVERTISEMENT

DTS DataPump Blob Fields

Feb 16, 2001

Hello all,

I've spent quite a few hours on this one... I've successfully gotten tables to datapump between source and destination datbases, however... I get the following message when I hit a table with a blob field:

Step 2 failed, error: x80040E21, x80040000 + 3617
General error -2147217887 (80040E21). (Microsoft Data Transformation Services (DTS) Data Pump (80040e21): Cannot instantiate Storage Object for source column 6 ('description') Blob data. Your provider may require that all Blob columns be rightmost in the source result set.) (Microsoft OLE DB Provider for ODBC Drivers (80040e21): Errors occurred)

I've written a VB com dll that does a simple datapump... but am stuck on this "Storage Object" instantiation thingy...

Any recommendations as to what I can put in my datapump object to allow just a simple xfer of blob types?


Thanks...
- Ed
edh@americanid.com

View 1 Replies View Related

DTS Datapump Task Error

Aug 29, 2000

Hi,

I get this error when I execute a job calling a dts pkg. The package itself is fine and executes without any error.It's transferring around 400000 records. Shceduling the pkg gives the following error.
Can any body led a hand please?


DTSStep_DTSDataPumpTask_1 DTSRun OnProgress: DTSStep_DTSDataPumpTask_1; 1000 Rows have been transformed or copied.; PercentComplete = 0; ProgressCount = 1000 DTSRun OnProgress: DTSStep_DTSDataPumpTask_1; 2000 Rows have been transformed or copied.; PercentComplete = 0; ProgressCount = 2000 DTSRun OnProgress: DTSStep_DTSDataPumpTask_1; 3000 Rows have been transformed or copied.; PercentComplete = 0; ProgressCount = 3000 DTSRun OnProgre... Process Exit Code 1. The step failed.

Thanks in advance
Barath M

View 3 Replies View Related

Teradata Datapump Issue

Nov 8, 2007

Wether I'm trying to access a datapump task in an imported package in SSIS, or in 2000 enterprise manager

they crash once created. I can not alter or redo mappings or anything else.
After enterprise manager churns for 5-10 min it crashes.

SSIS partially crashes: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. After this I can no longer click on anything and have to end task on Visual studio.

I can do a disconnected edit on the package in enterprise manager and alter the statement, but as far as I know you can't really change much as far as mappings back in there.
This also means that my 2000 packages have been limited to 1 teradata pump per package.

This happened on my laptop, 2 coworker's desktops, and now that i've migrated to a new high end developer box it's happening on it also.

Any feedback/ideas?

View 1 Replies View Related

DataPump Performance Problem

Oct 17, 2006

I'm using an SSIS under SQL 2005; i transfer data with a DataFlow Component:

DataSource = OLEDB Source;

Destination = OLEDB Destination

The source is a View returning about 100000 records; if i execute it in query analyser it takes few seconds. But if i use it as DataPump Source it takes HOURS to execute it...

I've noticed that If i modify the SELECT query inserting a "TOP 10000000000", it takes few seconds..

Why? Am i missing something?

View 4 Replies View Related

Data Transform

Jun 22, 2008

Hi friends,
Can somebody tell me how to do this-
How can we Analyze existing code used to transform data into the Operations Data Warehouse, and make changes to correspond to upcoming changes in the SAP data sources.
Thanks

sk

View 1 Replies View Related

Transact SQL :: How To Transform Data

Jul 30, 2015

how I can transform this input (in SQL Server 2008 R2)

SiteID    EdateTime
L5500     2015-07-30 05:00:48.000
L5500     2015-07-29 05:00:49.000
To be…
SiteID    StartDate                                  EndDate
L5500     2015-07-29 05:00:49.000         2015-07-30 05:00:48.000

View 6 Replies View Related

Scripting DTS DataPump To A Text File Destination

Oct 31, 2000

I am scripting a DTS Package using VB. The problem I am having is that I get the following error when I execute the package from Visual Basic:

Step Error Source: Microsoft Data Transformation Services Flat File Rowset Provider
Step Error Description:Incomplete file format information - file cannot be opened.

In the DTS Designer when I open the DataPump Transformation and click on the Destination tab I am prompted With the Define Columns dialog. I define the columns and click execute. The package works properly after doing this when executed from Enterprise Manager. Does anyone know how to script the file definition creation??? I have searched high and low through BOL and other resources and am not finding anything. I doesn't seem like it should be this hard to figure out so, maybe I am missing something.

View 1 Replies View Related

Data Transform And Compare With In A Query

Jul 23, 2005

Okay all I have a problem. I have two list of adresses and phonenumbers. I do not have control over the contents of the lists The onlyunique field between the two is the phone number. I need to be able toinner join the two lists on phone number.This would normally be straigt forward but the problem is that they areformated different and one of them does't even have a control on theformating.*Phone numbers are US phone numbers only.The one list (table) that does have a control uses this formatAAA3334444where AAA is the area code333 is the 3 digit prefix4444 is the four digit suffixthe second list (table) does not have any standardized formating andcan be filled with extraneous spaces, parentheses and dashes not tomention the leading 1 in some instances.I thought that I could do some kind of regular expression to do acomparison but I havn't as yet found a good resource to tell me how todo it or if it is even possible. Or maybe break up the one I know hasa standardized format into something like this:'%AAA%333%4444%' and doing my comparison that way. however It is veryimportant that only those list items in both list that are truly thesame place be listed.suggestions and solutions are apreciated

View 3 Replies View Related

Transform Data To Excel Sheet Or Xml

Apr 21, 2006

Does SQL 2005 Express have standard functionality on board to export data as an excel sheet? or as XML in a specified format?

View 4 Replies View Related

How To Transform A Data Task From ODBC

Nov 8, 2007

Hi

I need to Transform Data from a Transoft Data Source (ODBC) into an OLE DB Connection.

The Solution that I Currently have is to Transform using the DTS 2000 Package Task.

Is there a way that I can perform this task by using SSIS

Regards
Que

View 1 Replies View Related

SQL DTS Packages Data Transform Task

May 25, 2006

Hello, I am working on a module to extract data from a Teradata server to SQL database. I am using a DTS package to extract the data and need to make the data source name (database name and object name) configurable at runtime and to be read from a Config table in the SQL database. What do you suggest is the simplest and most efficient method to do this?

I am trying to use a dynamic SQL query in the data tranform task with the data source as a query. But its giving me a strange syntax error. Can we use a dynamic SQL in the query option of DTS transform task source?

Thanks, Meriya

View 1 Replies View Related

DTS Error When Using Text File As Destination In DataPump Task

Nov 18, 2004

I want to export to a Text File (destination) from a SQL query using a DTS DataPump Task. My query has 28 columns, some how when I try to define the destination columns for my text file the Microsoft Managment Console Crashes completely.

I tough may be my query has two many columns only to find out that it has one to many. If I ramove a column form my query, any column. I get no error at all.

¿Is there a limit to a Text File destination connection?

View 3 Replies View Related

DTS 'Transform Data Task Property' GUI Window

Apr 15, 2005

I am importing data from xls file to a db table with a dts. In the time of the dts creation I am using 'Transform Data Task Properties' GUI window to map incoming xls fields (source) to the table columns (destination).
Question: Is there any way to invoke the 'Transform Data Task Property' GUI window in dts runtime and use it to change the mapping dynamically in the run time?
Thanks, Vadim.

View 3 Replies View Related

SQL Server 2014 :: Transform Data Into Column

Jan 8, 2014

I have data like this in the table :

IntRecpieID strName intMealtypeID Total
100 ‘A’ 1 20
101 'B' 2 30
100 'A' 3 40

Desired Output required:

IntRecpieID StrName 1 2 3

100 'A' 20 Null 40

101 'B' Null 30 Null

View 5 Replies View Related

Transform Data Task Vs Stored Procedure

Mar 9, 2007

Hi everybody

I'm still trying to learn the advantage of having stored procedures. I have a DTS that uses a Transform Data Task to append the result of a view into a table. All operations are done locally in the server.

Do I have any advantage if I write a stored procedure to insert the view into the table, and then call the stored procedure in the DTS, in stead of using the Transform Data Task ?


Thanks in advance for your thoughts
ds9

View 1 Replies View Related

Inconsistent SSIS Data Transform Behavior

Jan 15, 2008



High all,

I have a very simple SSIS package that is moving data from a DB2 database to a Teradata box. I've run it around 10 times, twice it pushed data over, the balance of the time, it executes with no error, but moves nothing over. In the "incomplete" runs, a command line box pops up for half a second, then the package ends.

Does anyone have ideas as to why this behavior is occurring?

Thanks,

Mark

View 1 Replies View Related

Transform Data Task (DTS2000) And SSIS

Jun 21, 2005

My first period using SSIS in a real-world application convinced me that there

View 4 Replies View Related

DTS Data Transform Task: How To Hide DB And Schema Names?

Mar 25, 2008

Hi All,

I'm stuck up with a strange problem.
When i try to setup a Transform Data task in DTS, the table drop down shows fully qualified table name.

i.e. <database name>.<schema name>.<table name>

as you can see in the attached screenshot. With this I cant see the full table name and am not able to make the correct selection.

Where can i change the properties so that it displays only the table name?

Thanks
Rohit

View 1 Replies View Related

SSIS Package, How To Transform Data Type Fields.

Oct 12, 2006

Hello,

I have an OLE DB Source and i want to transform the data type fields of the table before i export the table in an OLE DB Destination.
Is there a way to transform numeric value to float, and numeric to nvchar?

Thank you in advance.

View 5 Replies View Related

Lookup Transform - Incompatible Data Types Mistery

Jul 10, 2007

I have a package that works fine in development. I move the package over to test and it fails validation in the lookup transform.



Error 46 Validation error. Data Flow Task - PO Lines Interface: Lookup - LIST PRICE [29621]: output column "LIST_PRICE_PER_UNIT" (29667) and reference column named "LIST_PRICE_PER_UNIT" have incompatible data types. SPO_TO_ORACLE_PO.dtsx 0 0




What strikes me as odd is the fact that I don't have a way of specifying the data types. I just specify the column I wish to return as a new column with the same name. Anyway, why would this work in one instance but not another?



thanks

John

View 4 Replies View Related

Nested Tables With Data Mining Training Transform?

Nov 23, 2007

Hi ...I can't figure out how to put nested tables into the Data Mining Model Training Transform (SSIS). Can anybody help me? some example please...!!!??
Diego B.

View 3 Replies View Related

SSIS Equivalent To DTS Transform Data Task Properties

May 3, 2007

I am trying to read in a flat file, transform the fields and store into a destination database.



In DTS, this works using Transform Data Task Properties. I define the columns and then have a VB script on the Transformations tab that changes any bad data.



Is there a way to do this in SSIS that I can define the column transformations and re-use my VB scripts?



Linda

View 16 Replies View Related

Fuzzy Grouping Transform Corrupts Pass-through Data

Aug 2, 2005

We are working with a client and are using Fuzzy Group transform for de-duping, and hierarchy creation for a national account list.

View 4 Replies View Related

Parameter Error When Performing A Transform Data Task From Access To SQL Server 2K

Dec 21, 2005

I have an Access 2.0 database that holds call data on a mapped drive. I am running MS SQL Server 2000. I can open it and view the records inside. I can even run the query below and get results, if I removed the CallDate and CallTime parameters.

SELECT CallDate, CallTime, Mid(CallRecordData, 68, 3) AS Extension, 'I' AS Direction, Mid(CallRecordData, 34, 11) AS Called,
Val(Mid(CallRecordData, 18, 2)) + Val(Mid(CallRecordData, 21, 2))/ 60 AS Minutes, Val(Mid(CallRecordData, 21, 2)) AS Seconds
FROM CallRecords
WHERE (CallDate = ?) AND (CallTime >= ?) AND (CallTime < ?) AND (Mid(CallRecordData, 30, 1) <> '9')

When I preview in the Transform Data Task, I get:
Package Error
Error Source: Microsoft JET Database Engine
Error Description: No value given for one or more required parameters.

When I look at the parameters, they are listed. I check their values, and they have the appropriate values (DateCalled, String, 07/14/2005) (StartTime, String, 06:30) (EndTime, String, 07:00)

When I run it in the build query or in Access with a linked table to the source, I can enter the values when asked for them and it works.

Thanks for any help you can provide.

View 2 Replies View Related

Ho Do I Get The Details From A DTS Transform Data Task Into An SSIS Derived Column Transformation?

Mar 7, 2008

i have too many DTS packages to migrate to SSIS, and while examining a DTS package in BIDS (converted with the migration utility) i tried to edit the resulting migrated package, which opened the DTS interface with the two connection icons joined by the big fat arrow with a gear on it...not exactly what i had in mind, iow, it looks like SSIS on the outside, but its still DTS on the inside.
So I stripped out a series of components from a more complex package hoping that simplifying it would reveal the contents of old DTS Transformations tab at least partially set up in a Derived Column transformation.
Can i get there from here, or must i recreate every stinking definition in a derived column manually from the ground up?
thanks very much for your help

View 2 Replies View Related

SSIS Data Mining Model Training Transform (Nested Tables)

Oct 26, 2006

I can't figure out how to put nested tables into the Data Mining Model Training Transform (SSIS). I can do a simple case table, but how do you get those nested tables with DM Training Transformation? Any ideas? Samples?

Thanks in advance,

-Young K

View 3 Replies View Related

REPLACE In Derived Column Transform Causing Repeat Data In Rows That Should Be Blank

Jul 25, 2006

W2k3 server, SQL 2005.
@@version = Microsoft SQL Server 2005 - 9.00.1399.06 (Intel X86)
Standard Edition on Windows NT 5.2
(Build 3790: Service Pack 1)

I have my first SSIS package almost working, but I'm having an odd problem and can't find any information to help resolve it.

I'm importing from a flat file (csv) to an existing table (append). I've got a Derived Column transformation in the middle to do some data cleanup. It's all working except for one little problem...

One of the transformations is 'REPLACE([Column 3],"^","; ")', output to a new column. (The input file has a field that uses carets as delimiters between an unknown number of items; I'm changing that to semicolons for easier reading.) Not all rows have data in this column, some will have one item, some will have multiple items.

The REPLACE works except that it fills in repeated data for all the blank rows.

Example:

Incoming data is:

1 Smith,Jane^Jones,Jane

2 Brown,John

3

4 Adams,James^Adams,Jim

5

6 White,Debra

Data inserted into the table is:

1 Smith,Jane; Jones,Jane

2 Brown,John

3 Brown,John

4 Adams,James; Adams,Jim

5 Adams,James; Adams,Jim

6 White,Debra

I've tried to use a Conditional to skip the empty rows, but I can't get that working at all (get syntax errors no matter what I put in).

Any suggestions on how to fix this would be most appreciated!

Thank you.

View 5 Replies View Related

Moving Files (split From An Existing Thread-SSIS Equivalent To DTS Transform Data Task Properties)

May 7, 2007

Hi JayH (or anyone). Another week...a new set of problems. I obviously need to learn .net syntax, but because of project deadlines in converting from DTS to SSIS it is hard for me to stop and do that. So, if someone could help me some easy syntax, I would really appreciate it.



In DTS, there was a VBScript that copied a set of flat files from one directory to an archive directory after modifying the file name. In SSIS, the directory and archive directory will be specified in the config file. So, I need a .net script that retrieves a file, renames it and copies it to a different directory.

Linda



Here is the old VBScript Code:

Public Sub Main()

Option Explicit

Function Main()

Dim MovementDataDir

Dim MovementArchiveDataDir

Dim MovementDataFile

Dim MovementArchiveDataFile

Dim FileNameRoot

Dim FileNameExtension, DecimalLocation

Dim CurMonth, CurDay

Dim FileApplicationDate

Dim fso ' File System Object

Dim folder

Dim FileCollection

Dim MovementFile

'======================================================================

'Create text strings of today's date to be appended to the archived file.

FileApplicationDate = Now

CurMonth = Month(FileApplicationDate)

CurDay = Day(FileApplicationDate)

If Len(CurMonth) = 1 Then

CurMonth = "0" & CurMonth

End If

If Len(CurDay) = 1 Then

CurDay = "0" & CurDay

End If

FileApplicationDate = CurMonth & CurDay & Year(FileApplicationDate)

'=====================================================================

' Set the movement data directory from the global variable.

MovementDataDir = DTSGlobalVariables("gsMovementDataDir").Value

MovementArchiveDataDir = DTSGlobalVariables("gsMovementDataArchiveDir").Value

fso = CreateObject("Scripting.FileSystemObject")

folder = fso.GetFolder(MovementDataDir)

FileCollection = folder.Files

' Loop through all files in the data directory.

For Each MovementFile In FileCollection

' Get the full path name of the current data file.

MovementDataFile = MovementDataDir & "" & MovementFile.Name

' Get the full path name of the archive data file.

MovementArchiveDataFile = MovementArchiveDataDir & "" & MovementFile.Name

DecimalLocation = InStr(1, MovementArchiveDataFile, ".")

FileNameExtension = Mid(MovementArchiveDataFile, DecimalLocation, Len(MovementArchiveDataFile) - DecimalLocation + 1)

FileNameRoot = Mid(MovementArchiveDataFile, 1, DecimalLocation - 1)

MovementArchiveDataFile = FileNameRoot & "_" & FileApplicationDate & FileNameExtension

If (fso.FileExists(MovementDataFile)) Then

fso.CopyFile(MovementDataFile, MovementArchiveDataFile)

' If the archive file was coppied, then delete the old copy.

If (fso.FileExists(MovementArchiveDataFile)) Then

fso.DeleteFile(MovementDataFile)

End If

End If

Next

fso = Nothing

folder = Nothing

FileCollection = Nothing

Main = DTSTaskExecResult_Success

End Function

View 6 Replies View Related

Transform To Remove Rows From Data Set A That Match Rows In Data Set B On A Given Key?

Jun 28, 2006

Hi,

I have a common requirement in numerous SSIS processes to take my main input data set and to remove all rows from it that match a second input data set on a given key and output this as the main output. I also want to output (as a second output) all the rows from the main input data set that did match on the given key. However, I don't want to merge in data from the second input, nor am I interested in rows from the second input data set that have no match in the main input.

E.g. If I have the following data:

Main input:
Key Name
--- ----
1 Steve
2 Jamie
3 Donald

Second Input
Key DontCareAboutThisField1
--- -----------------------
1 ...
3 ...
4 ...

Then I would like the following output:

Main Output
Key Name
--- ----
2 Jamie

Second Output
Key Name
--- ----
1 Steve
3 Donald

Can I do this with a standard transform, or will I have to write my own? Any help on this would be greatly appreciated!

Thanks in advance,

Lawrie

View 1 Replies View Related

Transform Data From DB On Sql Server 2000 To Another On Sql Server 2005?

May 8, 2007

Hello everybody,



Can anyone give the best way to transform data from a DB on sql server 2000 to a DB on sql server 2005?



Note: - the source DB and the destination DB are different in their structure

- We need the conversion way to be scripted to can be done on different site.



thanks



View 2 Replies View Related

DTS Transform

Jul 13, 2004

I get this error:

column 6 ('CheckDate', dbType_dbtimestamp), status 6: dat over flow
invalid character value for cast specification

Here is the sql generated :

CREATE TABLE [DMS].[dbo].[Master] (
[Date_Entered] smalldatetime NULL,
[Initials] nvarchar (10) NULL,
[VendorName] nvarchar (50) NULL,
[CheckNum] nvarchar (20) NULL,
[ExpenseType] nvarchar (10) NULL,
[CheckDate] smalldatetime NULL,
[CheckAmount] money NULL
)


IM guessing i need to change date_enteed type or do a cast

any help pls

!~<M

View 3 Replies View Related

Look UP Transform

Apr 10, 2008



Hi ,

i am using to Lookup transform for the matching records , if my source records are matching with lookup records it work fine , if not it through the following error


[Lookup [34]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "component "Lookup" (34)" failed because error code 0xC020901E occurred, and the error row disposition on "output "Lookup Output" (36)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.



Please advice , how i can solve this .




Thanks in Advance
Raja Ragothaman

View 1 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved