Numeric Datatype To Ssis Variable Datatype Conversion Problem
Apr 24, 2008
Good afternoon,
I have an issue with an ssis variable datatype.
The scenario is as follows:
I have a stored procedure:
PROCEDURE [dbo].[sp_newTransaction]
@sourceSystem varchar(50),
@txOut NUMERIC(18,0) OUTPUT
AS
insert into scn_transaction (sourceSystemName) values(@sourceSystem);
SELECT @txOut = @@identity
Whose purpose is to perform an insert into a table and return me the identity value of the inserted record, which I'll then use throughout the rest of my package. The identity column in the inserted table is numeric(18,0).
I execute the stored proc with the following sql with an OLE DB connection manager:
exec sp_newTransaction ?, ?
The first parameter is a string variable from earlier in the package, and the second is the output parameter. I have the following parameter mappings to the execute sql task:
The proc is correctly called, and the row insesrted, however I get a type conversion error when SSIS attempts to map the return parameter to my package variable... I've tried all sorts of combonations, and can't seem to get it to execute.
At one point I wasn't returning a numeric, but rather an int from the stored proc, and all was well until I went to use the variable in a derived column later in the package, and the type was converted quite incorrectly (a 1 was 77799789080 or some such), indicating a type conversion error likely related to the encoding of the number.
I'd like to keep the datatypes as numeric and make ssis use those - any pointers are greatly appreciated as to what type my package variable should be to allow proper assignment of a sql server numeric type to it.
I have to develop a package to get some data from a 10g Oracle database and load it to a SQL Server 2005 database.
In my design I used Oracle Provider for OledDB as my source provider following Microsoft recommendations found here : http://msdn2.microsoft.com/en-us/library/bb332055.aspx
As I have to use a parametrized query against my Oracle database I create a variable with my query that I override at runtime with a script task to update the parameters values.
One of the return column is of numeric datatype.
To handle this known issue with Oracle sources I edited through the advanced editor my Oledb source component to manually correct the precision.
Everything works fine with one of my package.
So I repeat the whole process while designing an other one with the same purposes.
As for the previous one I first unit test every components and everything works. But when running the full package it raised an error code 0x80004005 Error while validating post executing phase of the component (sorry for the poor translation from french to english ...)
I open my dataflow where the component is and found no warnings or errors. I edited through the advanced editor my Oledb source component and found that there's an error on the precision scale of the numeric return column. The message warn me that the precision must be between 1 and 38. I checked this column and in the "OLE DB Source output / External Columns" the precision is 0 though I previously manually corrected it. I upadted again for a 18 precision as it is for this column in "OLE DB Source output / Output Columns". But it never keeps my changes ......
I delete the component and recreate it to force the metadata to be updated but always the same thing, the precision scale for this column in the "OLE DB Source output / External Columns" never keeps my 18 value.
Here is a description of the issue I'm facing about decimal datatype conversion from DB2 to SSIS throught Microsoft OLE DB Provider for DB2.
I first discovered it when I tried to use a LookUp trans. and selected a decimal typed field. I was unable to validate it and clicking the OK button threw the error copied below:
TITLE: Microsoft Visual Studio ------------------------------
Error at Data Flow Task [DTS.Pipeline]: The "output column "MFIXFRA" (317)" has a value set for length, precision, scale, or code page that is a value other than zero, but the data type requires the value to be zero.
Exception from HRESULT: 0xC0204019 (Microsoft.SqlServer.DTSPipelineWrap)
------------------------------ BUTTONS:
OK ------------------------------
Some workarounds on this issue led me to two other strange behaviours:
- If I create a dataflow task to perform a raw copy of a DB2 table to a brand new table in SQLServer destination (by clicking "new" instead of choosing an existing destination table), generated "create table" script for decimal fields is quite different from source table. For example, a source field declared Dec(15,2) is translated as Dec(29,5), and so on.
- Quite same behaviour if I try to derive a column using as derived column trans. If the source column is a decimal, it's scale and precision are interpreted ramdomly.
I have to read data from an MDB file into SQL Server. Simple, but I need to place a literal into every row. In my OLE DB Source I would use:
SELECT '001' AS col1, foo, bar FROM Table1
But the problem is that I pass my value for col1 into the package from C# using Managed DTS.
So I set up my OLE DB Source using "SQL command from variable." And I have two variables one to receive the value of for col1 (col1) and one that is an evaluated expression that has a dynamic sql statement (dynSql).
col1 is a String as is dynSql. dynSql looks like this:
"SELECT " + @[col1] + " as col1, foo, bar FROM Table1"
Now as long as I set the value of col1 to '001' that should work. Right? Well not exactly. The destination for col1 in SQL Server is a char( 3 ). But the package makes the col1 column from the source a DT_WSTR. Which causes the famed: "Text was truncated or one or more characters had no match in the target code page.".
I'm designing a database that will not be that large (I would be surprised if it ever surpassed 50,000 rows across all tables), but will be accessed quite often, so I am doing my best to optimize its structure, such as doing 3NF, selecting appropriate data types, etc.
I have a few columns that will contain numeric data (such as an invoice number (from an external source) or location ID). However, one of my classes in college was about database design, and we were taught that if you won't be doing mathmatic computation on a field (such as the invoice or location fields I mentioned earlier), then you should use a string literal type (char, varchar, etc.)
Unfortunatly, the professor did not explain much as to why this should be done. From the standpoint of semantic analysis, these types of fields are probably more labels than they are numbers. However, I don't find that very convincing (or even helpful).
In short, my question is that given a column holding numeric data that isn't worked on in a mathematical sense, is it really better to mark it as a string literal than a number? Any articles or studies I can read relating to this?
I would think that comparisons would be faster with int, as well as data storage (though, as mentioned, that's not as big of a concern). Sadly, Google doesn't have many helpful resources. (Lots of stuff on char vs varchar, though.)
I read the topic from JROdden and this case is similiar but...
I got several varchar fields with values like 1.2 1.3 ... these I can covert with select CONVERT(dec(5,2), fieldname) as fieldname
In fact I also solved undefined- and NULL-values with. CONVERT(decimal(12, 2), CASE WHEN GESCHKOSTMAX IS NULL OR GESCHKOSTMAX < '0' THEN '0' ELSE GESCHKOSTMAX END) as GESCHKOSTMAX,
But now there are values like 1,4 and these ones neither CONVERT nor CAST will handle.
I tried the SELECT DISTINCT KMPAUSCHALE FROM extr_INTFIRMA WHERE (isnumeric(KMPAUSCHALE) = 1)
and get 0,40 0.25 0.30 and so on...
The error is: [Microsoft][ODBC SQL Driver][SQL Server]Error converting datatype varchar to decimal. (or float or numeric (whatever I tried))
I think the easiest way would be to insist on higher data quality but I also would like to solve this interesting challenge.
Thanks for any hints
By the way, I followed rudys link to http://rudy.ca/afdb.html and now I know how I could protect myself !!!!
There must be a voice in my head saying: Try the db-forum, try it and stay happy... ;-)
I am getting a "Syntax error converting the varchar value '10,90' to a column of data type int." error when I run the following procedure:
@myList varchar(200)
SELECT column1 FROM table1 WHERE table1.ID IN (@myList)
When @myList is a single value, I get no errors. However, when @myList is a comma separated list like in the message above, I error out. I am using SQL Server 2000.
How else can I build this list of IDs? Thank you in advance for your comments.
Dear All: I am in the process of developing a code generation tool to generate automatically: 1- Business Layer objects 2- Object Layer objects 3- Data Layer objects
The code follows the same technique used in IssueTracker Starter Kit.
I faced somehting wierd today while trying to convert between SQL Data types to C# data types:
Check the image please, the problem is that, different value number is being given to each column type, by using both: syscolumsn.type and syscolumns.xtype, which one to use ? which is the best used to convert to C# ? Are there any place where data types of SQL Server are being converted to C# data types ?
I am using DTS to import a DB2 table from the mainframe and export the table in text format to a shared folder. I want to convert two fields to a date format yyyy-mm-dd. RELSE_Date is in this format and Updtts is a timestamp coming from the mainframe. The text file is all vchar. In dts here is my query pulling from the mainframe SELECT A.PROD_ORDER_NUMBER, A.DYE_SEQUENCE, A.STYLE, A.COLOR, A.SIZE, A.STATUS_IND, A.STATUS_CODE, A.QUANTITY_REC_DC, B.SHIP_OTFQ_QTY, A.MRP_PACK_CODE, A.LABELS_PRINTED, date(A.RELSE_DATE) as RELSE_DATE, date(A.UPDTTS) as UPDtts, A.LOCATION FROM DB2.WP1_PO_CUST_REQ A, DB2.WP1_DYE_LOT_REQ B WHERE A.PROD_ORDER_NUMBER = B.PROD_ORDER_NUMBER AND A.DYE_SEQUENCE = B.DYE_SEQUENCE AND A.STYLE = B.STYLE AND A.COLOR = B.COLOR AND A.SIZE = B.SIZE AND (A.PROD_ORDER_NUMBER LIKE '__K____') When I parse the query it accepts it. When I run the DTS I get an error stating (SQL STATE 220077 SQLCODE -180) The sting representation of a datetime value has invalid syntax. Does anyone have a suggestion on how to convert these fields before the text file is exported or another output format that is similar to .dat in comma delimited format? Thank you.
Will it be possible to convert a varchar data type column (having values that are numbers or null only) to a numeric data type column having the number values and the null values not being replaced by 0.
Col A Col A (varchar) (int or numeric) 123 123 124 124 NULL 0 <expected blank> 125 125
I tried using CAST(Col A as int) and it converts the NULL to 0. Will CAST(ISNULL(ColA,'') as int) work? The reason for this requirement is the value 0 will give different meaning to the data. Any insights will be appreciated.
I have a table named 'Table1' which contains a column 'Name'. The data type of column [Name] is varchar(50).
When i try to change its datatype to binary by trying following code
ALTER TABLE Table1 Alter Column [Name] Binary(5000)
It gives following error.
" Creation of table 'bp_MAIN' failed because the row size would be 10021, including internal overhead. This exceeds the maximum allowable table row size, 8060. "
So, how can i change the datatype of this column ?
I have a table named 'Table1' which contains a column 'Name'. The data type of column [Name] is varchar(50).
When i try to change its datatype to binary by trying following code
ALTER TABLE Table1 Alter Column [Name] Binary(5000)
It gives following error.
" Creation of table 'bp_MAIN' failed because the row size would be 10021, including internal overhead. This exceeds the maximum allowable table row size, 8060. "
So, how can i change the datatype of this column ?
here is my problem: i have a variable @sid_x as binary(16) = 0x4CF254AB0BA5D411AA3E00508BC5C413 and i want to use it as argument in sp_addlogin statement.
select @sqlcmd = 'sp_addlogin "test", @sid = ' + @sid_t /* this doesn't work, because @sid_t is binary... */
select @sqlcmd = 'sp_addlogin "test", @sid = ' + convert (char (20), @sid_x) /* this doesn't work either, because it doesn't convert to binary text */
my question, is there any way i can get @sid_x in follow text format 0x4CF254AB0BA5D411AA3E00508BC5C413 ?
Can anyone tell me how to calculate datatype conversion times in SQL Server 7? I have a varchar (15) field that I tried to convert to integer using the table design GUI in Enterprise Manager. The table holds about 72,000,000 records about 1k apiece in size.
It's been running for about an hour now with no seeable results. In Performance Monitor I don't see any page reads happening, so is this indicative that the process died? Enterprise Manager is no longer responsive - even if I open another session.
I'd like to know how long I can reasonably expect this conversion to take. Also, how can I abort this request safely if I want to?
I have a field in a table that stores date of birth. The field's datatype is char(6) and looks like this: 091703 (mmddyy). I want to convert this value to a datetime datatype.
What is the syntax to convert char(6) to datetime?
I am facing a problem while using SQL Server with VB application.Implicit conversion from datatype text to nvarchar is not allowed.Use the convert function to run this query.When i see the trace file, i see one stored procedure called but nolines of code get executed, and immediately after that the ROLLBACKTRANSACTION occurs and the applications fails.But to my surprise i am able to do the same thing on a differentmachine using the same application and the same database on the sameserver with the same user id.Can anyone explain the reason of occurance of this problem.I require this very urgently, so i will be oblized if anyone can comeup with a quick response.Kind Regards,Amit Kumar
The DATEDIFF(s, time1, time2) function output is of the 'datetime' datatype. How can I alter the 'datetime' to 'int' (or 'bigint') datatype to make it compatible with other variables in my calculations?
I have image type col.I'm trying to do the following,DECLARE @Data varbinary(16)SET @Data = (select imageCol from Table1 where id=3)As image datatype returns varbinary value, so I want to store image col value to a varbinary variable(or any other type variable, eg., varchar). But getting following error,========================================Server: Msg 279, Level 16, State 3, Line 2The text, ntext, and image data types are invalid in this subquery or aggregate expression.========================================Is there anyway to store image datatype value to a variable?Cheers.
I've created a stored procedure that converts an input string in richtext format (input as type TEXT) to plain text. I would like to be able to return this newly converted string, but I need to have some way of storing it in a local variable. My problem is that since I can't use the TEXT datatype as a local variable, I have no way of storing the large amounts of text I converted within the procedure. The VARCHAR(8000) just isn't large enough for my purposes. Anyone have any suggestions on how to go about doing this?
hi everyone, How do i declare a global variable in my package which takes a numeric value like User::VAR1 = 200402 and later on work on it
Later in the properties of the Dataflow i want to have this expression..
"select * from " + "TAB1" + " where Date=" + @[User::VAR1]
Here i want to subtract 190000 from @[User::VAR1] to get it in to myformat i.e the DATE format in the table
I can only see String datatype and othe datatypes wont allow me to to do any kind of manipulation in the expression: and to be more specific what are Int16 Int32 Int64 Double
I tried to use all of the above but the expression doesnt allow me as it says:
TITLE: Microsoft Visual Studio ------------------------------
Nonfatal errors occurred while saving the package: Error at Extract: The data types "DT_WSTR" and "DT_I8" are incompatible for binary operator "+". The operand types could not be implicitly cast into compatible types for the operation. To perform this operation, one or both operands need to be explicitly cast with a cast operator.
hi everyone, How do i declare a global variable in my package which takes a numeric value like User::VAR1 = 200402 and later on work on it
Later in the properties of the Dataflow i want to have this expression..
"select * from " + "TAB1" + " where Date=" + @[User::VAR1]
Here i want to subtract 190000 from @[User::VAR1] to get it in to myformat i.e the DATE format in the table
I can only see String datatype and othe datatypes wont allow me to to do any kind of manipulation in the expression: and to be more specific what are Int16 Int32 Int64 Double
I tried to use all of the above but the expression doesnt allow me as once i use any of the above it says :
TITLE: Microsoft Visual Studio ------------------------------
Nonfatal errors occurred while saving the package: Error at Extract: The data types "DT_WSTR" and "DT_I8" are incompatible for binary operator "+". The operand types could not be implicitly cast into compatible types for the operation. To perform this operation, one or both operands need to be explicitly cast with a cast operator.
HI,I have a table with IDENTITY column with the datatype as INTEGER. Nowthis table record count is almost reaching its limt. that is totalrecord count is almost near to 2^31-1. It will reach the limit with inanother one or two months.In order to avoid the arithmentic overflow error 8115, we would likechange the datatype from INT to BIGINT. we hope this will solve ourproblem.How do I approch this datatype conversion?. Since the data count ishuge, that leads to a long down time of database.we need better approach or solution for this problem?. kindly give mea better solution that will reduce the total downtime of the productiondatabase.?.Regards
i have so doubts in my mind and that i want to discuss with you guys... Can i use more then 5/6 fields in a table with datatype of Text as u know Text can store maximu data... ? acutally i am trying to store a very long strings values into the all fields. it's just popup into my mind that might be table structer would not able to store that my amount of data when u use more then 5/6 text datatypes...
and another thing... is which one is better to use as data type "Text" or "varchar(max)"... ? if any article to read more about these thing,, can you refere to me...
Why can varchar datatype variable only 4000 byte? For example: in a storedprocedure declare @aa varchar(8000) ...... while select @aa=@aa+@otherinfo end when the length is more than 4000 ,the data in the behind will be lost
After several hours of trying, I trow the towel in the ring and come here to ask a question.
Source system uses a timestamp column in the transaction tables. which is equal to a non-nullable binary(8) datatype (sql 2000 bol).
What I want to do is get the timestamp at the start of the transfer and at the end of the transfer of data. and store these in a controltable
I try to do this in 2 sql execute tasks:
sqltask 1: "select @@DBTS AS SourceTimestamp" and map the resultset to a variable. Here come's the first problem what variable type to take ?
DBNULL works (meaning it doesn't give errors) (BTW: is there a way to put a variable as a watch when debugging sql tasks ?)
INT64 and UINT64 don't work error message that types for column and parameter are different
STRING works
Then I want to store this variable back in a table of a different data source
sqltask2: "insert into controltable values(getdate(), ?)" and make an input parameter that takes the previous timestamp ...
if I took DBNULL as a type for the variable there doesn't seem to be a single parameter type that works ???
if i take STRING as a type for the variable I have to modify the sql to do the explicit conversion from string to binary so I change CAST(? as binary). It doesn't return any error but the value stored in the table is 0x00000000000 and not the actual timestamp.
Any help on this one ? Why are the INT64/Bigint not working here, you can perfectly do a convert(bigint, timestampfield) in sql ?
How came the SQL datatypes, and the variable datatypes, parameter datatypes are so badly alligned to each other (and all seem to use different names) ?
