I can't take full credit for this. I want to share this with Jeff Moden who did the important research for this calculation here.
All I did was just adapting some old code according to the mantissa finding Jeff made and optimized it a little
Some test codeDECLARE@SomeNumber FLOAT, @BinFloat BINARY(8)
SELECT@SomeNumber = -185.6125, @BinFloat = CAST(@SomeNumber AS BINARY(8))
SELECT@SomeNumber AS [Original], CAST(@SomeNumber AS BINARY(8)) AS [Binary], dbo.fnBinaryFloat2Float(CAST(@SomeNumber AS BINARY(8))) AS [Converted], @SomeNumber - dbo.fnBinaryFloat2Float(CAST(@SomeNumber AS BINARY(8))) AS [Error]
And here is the code for the function.CREATE FUNCTION dbo.fnBinaryFloat2Float ( @BinaryFloat BINARY(8) ) RETURNS FLOAT AS BEGIN DECLARE@Part TINYINT, @PartValue TINYINT, @Mask TINYINT, @Mantissa FLOAT, @Exponent SMALLINT, @Bit TINYINT, @Ln2 FLOAT, @BigValue BIGINT
WHILE @Part <= 8 BEGIN SELECT@Part = @Part + 1, @PartValue = CAST(SUBSTRING(@BinaryFloat, @Part, 1) AS TINYINT), @Mask =CASE WHEN @Part = 2 THEN 8 ELSE 128 END
WHILE @Mask > 0 BEGIN IF @PartValue & @Mask > 0 SET @Mantissa = @Mantissa + EXP(-@Bit * @Ln2)
SELECT@Bit = @Bit + 1, @Mask = @Mask / 2 END END
RETURNSIGN(@BigValue) * @Mantissa * POWER(CAST(2 AS FLOAT), @Exponent - 1023) END Thanks again Jeff!
I am keep getting an arithmetic overflow converting float to type numeric when running a script that looks something like this.
insert into table1 ( column1 ) select column2 from source server.
column1 is a numeric (28,8) and column2 is float. there are about 2000000 records in column2, and I know that when I tried just copying the top 1000000 wasn't a problem.
does anyone know what could be causing this problem???
*it's not because the data in column2 is out of range.
why does converting integer to float take so long? Its a column with about 5 Million rows. I want to avoid cast(inumber1 as float) / cast(inmuber2 as float), thats why converting them. Queries should be a bit faster after that.. hope so :) Thanks a lot
I have a data type float with a value of 10000487930 that I'm trying to insert into a data type nvarchar and am getting the result of '1.00005e+010'. I've tried cast(field as nvarchar) however this is not working. What might fix this? I cannot change the insert table data type.
Hello,I would like to convert a field from ntext field found in one databasetable to float field found in another database table. The reason why Iwant to do this is a long one.I have tried the following and playing around with the following:declare @valuePointer varbinary(16)<Row cursor logic to initialize @valuePointer to be a pointer to thesource ntext field>update TargetFloatTable set TargetFloatTable.TargetFloatValue =CAST(CAST(@valuePointer AS nvarchar) AS float)where TargetFloatTable.Id = @Idbut is not working for me.Hoping someone out there can help.Thanks,Cally
Select Cast('100.1234' as float)give me the result 100.1234Now when I convert it back to char I want exactly 100.1234Select Convert(char(100),Cast('100.1234' as float))Gives me 100.123 (Here I was expecting 100.1234)When I doSelect STR(Cast('100.1234' as float),25,4)I get back the result as 100.1234However here I am not sure how many digits do I have after the decimalpoint. If I put some value likeSelect STR(Cast('100.1234' as float),25,8)I get 0's appended to it, which is again not desired.Thanks in advance,Jai
Is this possible? I'm trying to user a lookup task and the data I want to compare is a varchar to float. How can I do this? I tried using the data conversion task and it didn't work and also tried cast and convert. Is this even possible or is there a way around it?
I have a field in my database that is stored as varchar. The values are usually contain a decimal, and should have really been a float or decimal. In order for me to do analytics in my BI environment, I need to convert this to a float or decimal.
eg of values.
10.00 20.00 0.00 15.00
or could be missing when I use cast(value as float) or cast(value as decimal(9,2)) or convert(float, value) I get an error
Can you guys help me out? I m trying to sum up some varchar-typed field. I need to convert it to float before doing the summing up so I m using "Cast".
I do get the answer but its not the correct figure. My SQL statement is as follow:
SELECT Sum((Cast(Qty1 as float)) + (Cast(Qty2 as float))) as intAnswer FROM TableName
Hello I have some problems with converting varbinary to float in T-SQLstored procedure. In my C# application i have table of structures withdouble type fields. Size of this table is variant. I have to send thistable to SP. Because of performance i want to send it only once andwhole. So I have to cut this varbinary blob in my Stored procedure toreceive structure field values. But I have problems with double fields- conversion from varbinary to float in T-SQL is not allowed. Pleasehalp me - thx for any advise.Maciej
I have a column type of float and How to convert it show like this
default value =39260.80 MY db use this ',' seperator instead of '.' wanna convert 39,260
This code is working if using '.' seperator
SELECTÂ parsename(convert(varchar,CAST(floor('39260.80') AS MONEY),1),2) not work using ',' seperator SELECTÂ parsename(convert(varchar,CAST(floor('39260,80') AS MONEY),1),2)
I'm trying to move records from a SQL table with a float column to a DB2 database that has the column defined as Decimal (8,2) It keeps crashing saying it has a type mismatch problem. I tried changing my source command to pass in the column already converted and it still crashes on this. I also tried doing a data conversion task to do the conversion and I still get the same error. Any ideas?
when I run below query I got Error of Arithmetic overflow error converting numeric to data type numeric declare @a numeric(16,4)
set @a=99362600999900.0000
The 99362600999900 value before numeric is 14 and variable that i declared is of 16 length. Then why this error is coming ? When I set Length 18 then error removed.
I'm getting the above when trying to populate a variable. The values in question are : @N = 21 @SumXY = -1303765191530058.2251000000 @SumXSumY = -5338556963168643.7875000000
When I run, SELECT (@N * @SumXY) - (@SumXSumY * @SumXSumY) in QA I get the result OK which is -28500190448996439680147097583285.072256 ie 32 places to left of decimal and 6 to the right When I try the following ie to populate a variable with that value I get the error - SELECT R2Top = (@N * @SumXY) - (@SumXSumY * @SumXSumY)@R2Top is NUMERIC (38, 10)
I seem to always get the "Fun Stuff" to try and figure out. I have an entire table that was pumped out of Oracle. I even hate saying that word!
There are a couple columns that are Float data type, and they are storing phone numbers as a Float data type. I am not able to CAST these into anything that is legible.
This is one of the values that I made up that look like some of the others.
how to convert float to timestamp in single select query..for exp. i have float as 1.251152515236 ,i want to convert this to datetime and from datetime to timestamp... i.e. 26:11:00
Hi there, I've a question regaarding datatype conversions... I can't convert the above mentioned datatype. I always get an error message that the conversion fails. Doesn't matter If convert or cast is used.
How would you convert the above mentioned variable into float???
I have been given some data from a Mainframe (AS400?) which has some fields coded in Packed Decimal. I have been able to load the data into a SQL2005 database table, but I now need to convert the Packed Decimal data in the binary(6) field to the appropriate integer (or float) value.
The field contains values such as the following:-
0x20202020200C
0x202020022025
0x20202020DFFA
I don't know how to interpret these. Has anyone got a function that can do this for me?
I've read several articles online that explain how packed decimal works, but none tell me how to interpret the last of my three examples. Can you help?
I have imported data from excel file. When data came to SQL table, the typeof AMOUNT column was varchar. I tried to convert and cast amount type ofamount column to number type but it does not allow me to convert.What is the best way of importing data into SQL and type stays the same asit was in excel file ?Or anyone has any better solution, please let me.Thanks.
Hi all, I defined an user string type varible in the package as AccountLen. I am trying to use this varible in the Expression of Derived Column transformation. I want to retrieve a part of column, i.e: Right(Column1, @AccountLen), this is always wrong because the AccountLen is string type. How I can convert it to the numeric so that can be used in the RIGHT function? Thanks