Hi,
When i try to insert a value in to a field of datatype float(8), it is storing the wrong values(approximate values).
For example, if i try to insert 2.62 , it takes it as 2.6200000000000001
But i can see the correct values displayed in the frontend.
Heard that there is a fix available for this issue.
Could any of you help me in getting the details about that fix?
Thanks in advance.
~ Chaitanya
In SQL Server for a field of datatype float(8),
if i insert a value 2.62 , it saves it as 2.6200000000000001
like so for other values also.
But in frontend i can see the right values.
Could any of you help me in fixing up this issue?
Thanks in advance.
~ chaitanya
I have this problem of inserting my query into database field. My code is as of below.
The @AVERAGESCORE parameter is derived from
Dim averagescore As Single = (122 * 1 + 159 * 2 + 18 * 3 + 3 * 4 + 0 * 5) / (122 + 159 + 18 + 3 + 0)
and the value returned is (averagescore.toString("0.00"))
However, I have error inserting the averagescore variable into a field of datatype float during the transaction. I have no problems when using non transactional sql insert methods. What could be the problem?
Try
Dim i As Integer
For i = 0 To arraySql.Count - 1
myCommand = New SqlCommand
Dim consolidatedobjitem As ConsolidatedObjItem = arraySql(i)
myCommand.CommandText = sqlStr
myCommand.Connection = myConnection
myCommand.Transaction = myTrans
With myCommand.Parameters
.Add(New SqlParameter("@AVERAGESCORE", consolidatedobjitem.getaveragescore))
End With
myCommand.ExecuteNonQuery()
Next
myTrans.Commit()
myConnection.Close()
Catch ex As Exception
Console.Write(ex.Message)
myTrans.Rollback()
myConnection.Close()
End Try
I can't take full credit for this. I want to share this with Jeff Moden who did the important research for this calculation here.
All I did was just adapting some old code according to the mantissa finding Jeff made and optimized it a little
Some test codeDECLARE@SomeNumber FLOAT,
@BinFloat BINARY(8)
SELECT@SomeNumber = -185.6125,
@BinFloat = CAST(@SomeNumber AS BINARY(8))
SELECT@SomeNumber AS [Original],
CAST(@SomeNumber AS BINARY(8)) AS [Binary],
dbo.fnBinaryFloat2Float(CAST(@SomeNumber AS BINARY(8))) AS [Converted],
@SomeNumber - dbo.fnBinaryFloat2Float(CAST(@SomeNumber AS BINARY(8))) AS [Error]
And here is the code for the function.CREATE FUNCTION dbo.fnBinaryFloat2Float
(
@BinaryFloat BINARY(8)
)
RETURNS FLOAT
AS
BEGIN
DECLARE@Part TINYINT,
@PartValue TINYINT,
@Mask TINYINT,
@Mantissa FLOAT,
@Exponent SMALLINT,
@Bit TINYINT,
@Ln2 FLOAT,
@BigValue BIGINT
SELECT@Part = 1,
@Mantissa = 1,
@Bit = 1,
@Ln2 = LOG(2),
@BigValue = CAST(@BinaryFloat AS BIGINT),
@Exponent = (@BigValue & 0x7ff0000000000000) / EXP(52 * @Ln2)
WHILE @Part <= 8
BEGIN
SELECT@Part = @Part + 1,
@PartValue = CAST(SUBSTRING(@BinaryFloat, @Part, 1) AS TINYINT),
@Mask =CASE WHEN @Part = 2 THEN 8 ELSE 128 END
WHILE @Mask > 0
BEGIN
IF @PartValue & @Mask > 0
SET @Mantissa = @Mantissa + EXP(-@Bit * @Ln2)
SELECT@Bit = @Bit + 1,
@Mask = @Mask / 2
END
END
RETURNSIGN(@BigValue) * @Mantissa * POWER(CAST(2 AS FLOAT), @Exponent - 1023)
END
Thanks again Jeff!
Peter Larsson
Helsingborg, Sweden
Ok I have upgraded my works database from a poorly designed Access database to a SQL database. The previous system allowed NULL values and duplicates to be inserted into a field that should NOT ALLOW NULL Values or duplicates. Therefore, this issue has now been moved across to my new system as I cannot set these constraints on the field that has multiple NULL values.
My solution would be to use a sequential operator, so whatever = NULL would be changed to a sequential number that us as administrators would know was a bogus number starting at something like = 999999900 counting up from that. There are only 250 records that would require updating.
To make things more interesting this field is not a integer type, its a Nvarchar type as its a Hardware ID. Both numerical and characters are require.
I have some engineering data in my table and the db designer is representing it with a float datatype. Here's what is happening. If I query on a record based on id num and get a row and put it in text boxes in my Windows App, min_riv_hd_dia (the float) is 0.026<14 zeroes>2. If I try to query and get that same record again but this time based on id num and min_riv_hd_dia equal to 0.026<14 zeroes>2, I get no row found. If I just do a select on this row based on id number, sql server displays it as 0.026. But if I query with 0.026 as my value, still the row is not found. If I query min_riv_hd_dia > 0.026, the record is found.
So my question is, how can I tell the exact value that must be input in my search criteria in order to find this row?
Thank you so much if you can help!
For example, I have a float datatype field with a value of .2501
I select using ROUND(Field,3) and get .25 as a result, but what I need to display is .250 (three decimal places.) How can this be done?
Thanks,
Paul
I'm using this query to sum the values. The cost column is a float datatype and what could I differently do here to sum the cost. I'm unable to sum the cost.
Also is there any way I change the datatype to int for Cost column without losing the data.
select ID, MAX(Date) Date, SUM(Cost) Cost, MAX(Funding) Funding from Application
group by ID
Hi,
how to convert nvarchar datatype to float?
Regards
Prashant
I am using sql express 2005 and sql server 2005 with C# 2.0.
I am a bit confused about which data type i should be using for several fields.
Right now I am declaring all of my fields in sql server as float for everything except for money fields which are using money.
When loaded into C# these fields are converted to double and decimal because C# does not have a float datatype.
Should I be using Decimal or Double for everything instead?
Here are a few examples
QtyInvoiced (float) - holds the number of items invoice
possible values look like this 1.0, 1.25 or 1.5
PercentDiscount (float) - holds a percentage
possible values look like this
10.25, 20.50, 50.00
I appreciate the help.
Hi,
I have a excel file and i am trying to import zip codes to the database... but the some of the zip codes start with 06902 but the excel file treats them as float but i want to treat them as varchar...
How can i do it.
Regards
Karen
Need to know if the varchar datatype field will ingore leading zeros when compared with numeric datatype ?
create table #temp
(
code varchar(4) null,
id int not null
)
insert into #temp
[Code] .....
What is the difference between Money and (Float or Decimal) Datatype. If we use Float or Decimal instead of Money, will we loose any functions..?
View 4 Replies View RelatedHi, friend,
l've 2 variables A & B from source database which i used to calculate C with the following formula, i.e.
C = 100 * A/B
In case of B = 0 (DIV BY ZERO), C should be equal to -9999.
l've set the datatype for A & B are INT, C is FLOAT at targetting database.
So l used to derived column to calculate C as,
B == 0 ? -9999 : 100 * A/B
After l run the package, l realize that all my C is INTEGER rather than FLOAT.....it seems that SQL server has evaluate the wrong datatype for me....anyway to overcome this?
Here's the deal:
I import a flat file from a legacy system, and then convert it into a single table. That works simply enough.
Then I have a SP that querys that table using a parameter for an accountID. My business tier calls that SP and returns the results to the calling tier (my web application). Easy enough...
Now for the question. The people who created the flat file (written in COBOL) decided to use "codes" to represent data. So, for instance, if I'm looking for the account plan, I can expect to see characters like ], or [, or +, etc... These characters have a special meaning, like:
] = Plan A
[ = Plan B
+ = Plan C, and so on.
Currently, the web application displays those characters, but I want it to display the actual plan name. Is there a way that when I execute the SP, the SP could pull the necessary records, and whenever it encounters a certain "plan" character, it could convert it into a "readable" name? Say that it sees that the plan_type field has a value of "]" for twenty records, so it converts those twenty records' plan_type value from "]" into "Plan A"? I'm not sure if I can do that, but I want to at least evaluate the option if I can.
I've evaluated other options, like using a CASE statement in my code, but I shot that down quickly...for obvious reasons. I don't wanna be changing my web application or business tier each time these guys update a plan name, or add a new one, delete an existing one, etc...
I've also thought about creating a dictionary table than contains the plan's code and its name, and then just INNER JOIN the first table with the dict table. This would keep my SP very simple (it's very straight-forward right now, and I like that). That way, if a plan name is ever changed, or a new one is added, I simply update the dict table using a simple query. However, if my SP is doing the conversion, I could just as easily update the SP.
Either of these methods would work for me, and I *do* know how to do the latter (dict table). However, there are quite a few other fields that I may have to do this for. I believe when I left for the day on Friday, my last count was 14 fields total that needed translation. That would mean 14 different dict tables! That could certainly affect my SP performance with all those INNER JOINS!
Therefore, I'm certainly interested in figuring out if it's possible to do the former method (SP), and then I shall decide which method is best for my situation.
Feel free to include your thoughts on which process you think is better as well. I'm really riding the fence with this one. However, if I can't find out how to change field values in my SP, then obviously I'll make a decision very quickly...
Thanks in advance.
I have an SSRS report that's being built from a stored procedure given 2 variables. The error I'm receiving is
An error has occurred during report processing. (rsProcessingAborted)
Cannot read the next data row for the dataset Sheet2. (rsErrorReadingNextDataRow)
Conversion failed when converting the varchar value '.5' to data type int.
This doesn't make any sense. The variable in question is being stored as a varchar. I don't do any typecasting in the stored procedure. Executing the stored proc in sql server studio works fine. In the query designer I replaced the stored procedure call with just the code for the stored procedure and it works fine as well. Switching it back to call the stored procedure throws the same error.
Our prod server is using 2008, our dev is running 2010, and I have 2013 Visual Studio. It has to do with the version of SSRS. I checked the datatype of the fieldtype and rd:TypeName was System.Int32. Changing it to System.String did nothing. Leaning even more towards the issue being SSRS itself. I even pulled the working version off of prod and it gave me the same error in Visual Studio.
Why is SSRS trying to convert a varchar to an int? It should be a double if anything or just leave it alone and not try typecasting it at all.
Hi all
I have a Float Field in my table with following values:
1.63
7.42
35.71
0.58
every thing is ok BUT when i Select Data from this table in
QUERY ANALIZER environment with (Select * from mytable) , Query Analizer shows me the following values:
1.6299999999999999
7.4199999999999999
35.710000000000001
0.57999999999999996
Why? while i don't have this problem in Enterprise manager !!!
Thanks in advance.
Kind Regards.
How can I remove -1.#IND values from a float column in SQL Server 2000 (8.00.2039 SP4).
So far I've tried
- Using Query Analyzer, setting the value to a proper float value. I receive error €œA floating point exception occurred€?
- Using Query Analyzer, deleting the record with the invalid float. I receive error €œA floating point exception occurred€?
- Using Enterprise Manager, setting the value to a proper float value. I receive error €œA floating point exception occurred€?
- Using Enterprise Manager, deleting the record with the invalid float. I receive error €œA floating point exception occurred€?
- Previously I was able to convert the column to nvarchar, but now this fails because the table is being replicated.
I don€™t really care what happens to the records, I just need them to go away.
Thanks
In the below scenario we are inserting some time related fields in Temp table.But its data type is varchar. Once data loading is finished in the temp table (Data is loading by source team SQOOP and they are unable to load if the source datatype is having Date or datetime) we have to alter the column datatypes. somehow, some character data in inserted in date columns (look at into 3rd insert statement). while altering the table it is failing. Can we do any alternative for this (Means if any varchar data that is non convertible to date can we make as null)
INSERT INTO ##TEMP_TEST
SELECT '2014-09-30','2017-10-06','Nov  6 2014  6:11AM','Nov  6 2014  6:11AM'
UNION SELECT '2014-09-29','2017-10-06','Nov  6 2014  6:11AM','Nov  6 2014  6:11AM'
UNION SELECT '2014-09-28','2017-10-06','Nov  6 2014  6:11AM','Nov  6 2014  6:11AM'
GO
INSERT INTO ##TEMP_TEST SELECT NULL,NULL,NULL,NULLÂ
[Code] ....
Hi
I was using a simple query to insert data in fields of type decimal
INSERT INTO Table (stringValue,intvalue) VALUES ('myString ',
" + numericvalue + ")
a query looked like this
INSERT INTO Table (stringValue,intvalue) VALUES ('myString',125)
I had to make a change to accept float point values ( 0,012 ) so I change the fields from decimal to float in the DB
Now I'm having problems in the query because of the comma. The new query looks like this
INSERT INTO Table (stringValue,intvalue) VALUES ('myString',0,012) so because of the comma the engine see three values instead two
How can write the query to insert this float values?
thanks
I have a table with three columns: UniqID, Latitude, and Longitude.
I need to write a query to identify when the latitude has more than 6 decimal places past the decimal. Same with Longitude. Values in these attributes can be a negative number. These fields are FLOAT.
I have a field called PPH and this field type float. so, this field could be null, 0.0, or values. I am using case statement:
 Â
[PPH] = case when h.HoursFloat = 0 then 0 else
round(s.Pledges / h.HoursFloat,2) end,
I am trying to add where if h.HourFloat is null then 0 but, it does not work. I have tried different ways such as
[PPH] = CASE isnull (h.HoursFloat,0)Â
when 0 then 0 else ....
How to add another condition to check on null then 0.
STDEV() gives incorrect values with reasonable input.
I have a table filled with GPS readings. I've got a column LATITUDE (FLOAT) with about 20,000 records between 35.6369018 and 35.639890. (Same value to the first 5 digits of precision---what can i say, it's a good gps.)
Here's what happens when I ask SQL Server ("9.00.1399.06 (IntelX86)") to compute the standard deviation of the latitude:
// Transact-SQL StdDev function:
SELECT STDEV(LATITUDE) FROM GPSHISTORY
WHERE STATTIME BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
0
// Zero. ZERO??!?!!
//Let's re-implement Std Dev from the definition using other aggregate functions:
DECLARE @AVERAGE FLOAT;
SELECT @AVERAGE = AVG(LATITUDE) FROM GPSHISTORY
WHERE GPSDATE BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
SELECT SQRT(SUM(SQUARE((LATITUDE - @AVERAGE)))/COUNT(LATITUDE)) FROM GPSHISTORY
WHERE GPSDATE BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
6.03401924005392E-06
// That's better. Maybe STDEV is using fixed point arithmetic?!?
SELECT STDEV(10 * LATITUDE)/10 FROM GPSHISTORY
WHERE GPSDATE BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
4.77267753808509E-06
SELECT STDEV(100 * LATITUDE)/100 FROM GPSHISTORY
WHERE GPSDATE BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
1.66904329068838E-05
SELECT STDEV(1000 * LATITUDE)/1000 FROM GPSHISTORY
WHERE GPSDATE BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
8.11904280806654E-06
// The standard deviation should, of course, be linear, e.g.
DECLARE @AVERAGE FLOAT;
SELECT @AVERAGE = AVG(LATITUDE) FROM GPSHISTORY
WHERE GPSDATE BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
SELECT SQRT(SUM(SQUARE(100*(LATITUDE - @AVERAGE)))/COUNT(LATITUDE))/100 FROM GPSHISTORY
WHERE GPSDATE BETWEEN '2007-10-23 11:21:00.859' AND '2007-10-23 17:00:00.062' AND GPSDEVICEID = 0x004A08BC04050000;
6.03401924005389E-06
// Std Dev is a numerically stable computation, although it does require traversing the dataset twice.
//
// This calculation is not being done correctly.
//
// Incidently, SQRT(VAR(Latitude....)) gives 4.80354E-4, which is also way off.
I will redefine STDEV to use a stored procedure similar to the above, but the algorithm used to compute VAR, STDEV etc should be reviewed and fixed.
-Rob Calhoun
Am converting varchar field to float and summing using group by and next inserting to varchar field(table).
while inserting float value it is converting to exponential ex:1.04177e+006 but if i execute only select statment actual float value will get display ex:1041765.726
My question is why it is converting while inserting ? and how to avoid it.
select query : SUM(CONVERT(float,(rtrim(REPLACE(REPLACE( column1, CHAR(13), ' '), CHAR(10), ' '))))) as AggregateValue
I am importing a couple SAS datasets to SQL Server 2008 for a project. The dates are in a float format, they show up as DT_R8 in SSIS. How can I convert these values to SQL server datetime? I have tried dozens of methods I found on-line with no success, I keep getting 'Arithmetic overflow error converting expression to data type datetime.' errors.
View 0 Replies View RelatedHere's a portion of the current statement.
UPDATE EngagementAuditAreas
SET numDeterminationLevelTypeId = parent.numDeterminationLevelTypeId,
numInherentRiskID = parent.numInherentRiskID,
numControlRiskID = parent.numControlRiskID,
numCombinedRiskID = parent.numCombinedRiskID,
numApproachTypeId = parent.numApproachTypeId,
bInherentRiskIsAffirmed = 0,
bControlRiskIsAffirmed = 0,
bCombinedRiskIsAffirmed = 0,
bApproachTypeIsAffirmed = 0,
bCommentsIsAffirmed = 0
FROM EngagementAuditAreas WITH(NOLOCK) ...
And what I need is to conditionalize the values of the "IsAffirmed" fields by looking at their corresponding "num" fields. Something like this (which doesn't work).
UPDATE EngagementAuditAreas
SET numDeterminationLevelTypeId = parent.numDeterminationLevelTypeId,
numInherentRiskID = parent.numInherentRiskID,
numControlRiskID = parent.numControlRiskID,
numCombinedRiskID = parent.numCombinedRiskID,
numApproachTypeId = parent.numApproachTypeId,
bInherentRiskIsAffirmed = (numInherentRiskID IS NULL),
bControlRiskIsAffirmed = (numControlRiskID IS NULL),
bCombinedRiskIsAffirmed = (numCombinedRiskID IS NULL),
bApproachTypeIsAffirmed = (numApproachTypeID IS NULL),
bCommentsIsAffirmed = (parent.txtComments IS NULL)
FROM EngagementAuditAreas WITH(NOLOCK)
Thanks.
I have a statement that works in compatibility mode 8 of SQL Server2000 while it fails in 7:declare @P3 intexec sp_prepexec @P3 output, N'@P1 float', N'select custname fromcustomer where custnr = @P1', 12600034custnr is a varchar(15). As long as the float p1 is less than1,000,000 the query returns correct results, but as soon as p1 isequal or greater than 1,000,000 the resultset is empty. I couldfigure, that above 1 million, sp_exec converts the float intoexponential format and compares this format to the string containingfixed format values. As said above, in mode 8 the statement returnscorrect results above 1,000,000.Please don't ask me, why we use float variables - I don't have anyinfluence on the statement - we isolated the statement with theprofiler.We are also currently dependent on the compatibility mode 7.I would be very grateful if you have any ideas, help or other issuesfor me.Thank you,Peter
View 1 Replies View RelatedHello everyone. How to make a query that returns the datatype of a field.
Thanks & Best Regard.
-Ron-
here is my business/data object for some reason I only get the first character back, say value is Charlie, I only get C public string GetUserName(long UserId)
{
try
{
string userName;
DiscussionDB data = new DiscussionDB();
List paramlist = new List();
paramlist.Add(data.CreateParameter("@UserId", UserId));
paramlist.Add(data.CreateParameter("@UserName", "", ParameterDirection.Output, DbType.String, 20));
data.ExecuteNonQuery("dbo.Discussion_User_Name", ref paramlist);
userName = paramlist[1].Value.ToString();
return userName;
}
catch
{
throw;
}
}
ALTER PROCEDURE [dbo].[Discussion_User_Name]
@UserId bigint,
@UserName varchar(20) output
AS
SET NOCOUNT ON
SET @UserName = (Select [Name] from Discussion_Member WHERE UserID = @UserID)
if (@UserName is null or @UserName = '')
BEGIN
SET @UserName = (Select UserName from Membership_User WHERE UserID = @UserID)
END
I have a field with a datatype of text. In SQL it looks like:
Body(text,null)
This field is used for notes. For certain records users will enter a date.
20150425
I'm looking to do a select statement that will grab only those records with a date and make them an int field. This doesn't seem to work.
select convert(int, body)
from b
where left(body,2)='20'
Hi,
I have to export data from a database in a special form (the customer wants it this way): every record in one line, and all fields in a large string. Thus, only by convendion it is defined the first seven characters being the central number, the next fivr characters the number of this spare part type, the following 30 the description, and so on...
Just at the end of every record has to be CF-LF.
all works well so far by with this code (in PHP) -
except one thing - see details below the code snippet):
$q1 = "SELECT dirc_matdata.* FROM dirc_vorgang, dirc_matdata ";
$q2 = "WHERE dirc_vorgang.rep_out> '".$anf_dat."'";
$q3 = "AND dirc_vorgang.rep_out < '".$end_dat."'";
$q4 = "AND dirc_matdaten.auftrags_nr = dirc_vorgang.auftrags_nr";
$q5 = "INTO outfile '".$drive.":/matdata.dat'";
$q6 = "FIELDS TERMINATED BY '' LINES TERMINATED BY '
';";
$query = $q1.' '.$q2.' '.$q3.' '.$q4.' '.$q5.' '.$q6;
The INT (integer) fields appear in the (plain text) output file with wrong field lengths. Although I had set the first field (i.e. the number of this type ofparts) to a length of 5, this field appears in the outfile being 10 characters long -- the maximum length of a normal INT type field (and twice as long as it should be).
Or (when set to filed type "MEDIUMINT"), it appears in the length of 8 characters only. But I need this data exactly 5 characters long in this application.
BIG QUESTION: How can I make mySQL (Ver. 4.1) write this fields into the outfile with their correct length as defined in the database structure?
Is this flaw already known in the community?
Is there any sort of "workaround"?
Thank you for every hint!
-ah-
I've got this sql statement that keeps returning the wrong data. (it's related to a previous post, but is different)
Code:
SELECT C.NAME, OL.PART_ID, SL.SHIPPED_QTY
FROM CUSTOMER C
INNER JOIN USERS U ON C.ID = U.ID
INNER JOIN ORDERS O ON C.ID = O.ID
INNER JOIN ORDER_LINE OL ON O.ID = OL.ORDER_ID
[Code] .....
The qry pulls like this: (wrong way)
HTML Code:
company partID Qty
Acme Inc ABC123 3
Acme Inc ABC123 4
Acme Inc ABC123 100
Acme Inc KLM444 3
Acme Inc KLM444 4
Acme Inc KLM444 100
Acme Inc QRP456 3
Acme Inc QRP456 4
Acme Inc QRP456 100
It should be:
HTML Code:
Co. part Qty
Acme Inc ABC123 3
Acme Inc KLM444 4
Acme Inc QRP456 100
The Qty field has duplicates. I've tried various ways, but I cannot make this work.
I have following select-statement:
select [date], [close], AVG([close]) over (order by [date] rows between 2 preceding and current row ) as [ma]
from dax2
My Problem is that the first 2 rows in column [ma] are not correct. They Show a value since it is not a 3 days average. In the first row in column [ma]is the same value as in [Close]. In the second row in column [ma] is the average value of the first and second value of column [Close].
How can i achieve that this "erroneous" values are not inserted or rather are shown as null.