Floating Point Truncation
Mar 24, 2004How can I truncate a floating point number to required number of decimal points
Eg:
100.642364074 to 100.64 and 67.643929847 to 67.645
Thanks.
How can I truncate a floating point number to required number of decimal points
Eg:
100.642364074 to 100.64 and 67.643929847 to 67.645
Thanks.
I need to write a floating point value to a column in a SQL server database. I have the following line ofcode, that formats the float value before passing it off to the stored procedure.
Please look at the line of code below. Does the 8 after SqlDbType.Foat mean that this will be a floatingpoint with 8 signigicant digits past the decimal point?
mySqlCommand.Parameters.Add("@Price", SqlDbType.Float, 8).Value = float.Parse(InitialPrice);
Hello all,
I can't see any reason for this error, not having a high level understanding of maths I thought I'd post it and hope someone could share some light on it.
I yesterday got called by a client who said that a payment for £15 + VAT was being passed to their payment gateway as 17.62 when it should be 17.63. The VAT calculation is performed in a SQL Server 2000 stored procedure. In the end I tracked it down and it wasn't a propblem with my calculation.
The price was coming out as 17.63 fine. The stored procedure then had to return this price in pence (17.63 * 100 = 17.63). When I put in a print statement with this calculation it was correct but when I output the variable that the result was assigned to it was coming out as 1762.
The variable that the result was being put into was of real datatype.
I then wrote a udf to test this. Here is the function:
CREATE FUNCTION dbo.POUNDS_TO_PENCE
(
@POUNDVALUE real
)
RETURNS INTEGER
AS
BEGIN
RETURN @POUNDVALUE * 100
END
As you can see nothing very special.
If you run this runction and pass in 17.63 it will return 1762!!!
The bit I don't get is if I change the @POUNDSVALUE intput variable to type float it returns the correct amount.
I've also found that the same problem occurs when passing in £30 + VAT (35.25) + 1pence. So, 35.26 comes out as 3525 instead of 3526. This is the case if you keep doubling the number (and adding a few pence here and there).
Does anyone know why this is or is it a bug in the processor?
The SQL books online say the following about the float and real data types:
--------------------------------------------------------
float and real (T-SQL)
Approximate number data types for use with floating point numeric data. Floating point data is approximate; not all values in the data type range can be precisely represented.
Syntax
float[(n)]
Is a floating point number data from - 1.79E + 308 through 1.79E + 308. n is the number of bits used to store the mantissa of the float number in scientific notation and thus dictates the precision and storage size. n must be a value from 1 through 53.
n is Precision Storage size
1-24 7 digits 4 bytes
25-53 15 digits 8 bytes
The Microsoft® SQL Server™ float[(n)] data type conforms to the SQL-92 standard for all values of n from 1 to 53. The synonym for double precision is float(53).
real
Floating point number data from –3.40E + 38 through 3.40E + 38. Storage size is 4 bytes. In SQL Server, the synonym for real is float(24).
--------------------------------------------------------
Apart from the fact that it says 'Approximate number data types' I can't see any difference between the data type apart from the ranges.
Anyone any ideas?
Thanks
Tom Holder
Hi.
Intermittently, the following error has been displayed when a SQL job on our box (SQL Server 2000 Service Pack 4 running on Windows 2000) runs.
'A floating point exception occurred in the user process. Current transaction is canceled.' The error number given is 3628 though I've also seen a 4xxx number (not at machine at the moment so cant be precise for the latter).
The intermittent problem over the last two days has become more regular and now does not appear to be related solely to the SQL job. For example, when connecting to the box using my local copy of SQL 2005 Management Studio and I attempt to browse the database objects I get this problem.
To the best of my knowledge, this server has not had any software installed on it and/or nothing has changed on it recently.
It has now got to the point that this particular job no longer runs. The job in question, in case this is significant, is a vanilla stored proc that returns data using OPEN ROWSET to communicate with an Access database (this connectivity has been verified to be okay, in that other similar jobs all run just fine)
If anybody can shed some light and/or point me in the right direction I would be extremely grateful.
Cheers
CG
Hi,I would like to know, if I need to do some floting point operations(mainly multiplication and division) on each roll of a table, should Iread the data out from the DB and do the calculation with a programminglanguage, say C#, or should I just use sql to do it on the sql server.An obvious advantage of doing it in the sql server is that you dontneed to transfer the data between the sql server and he applicationserver.But I am not sure if there are any other factors that will overridethis advantage: like the performance of doing lots of floting pointoperations in sql server.How is the performance of doing floting point operations in sql servercomparing to C# or other languages?Also are there any other factors that should be considered for thiscase andare there any other advantages to do this is sql server or in c#?ThanksBenny*** Sent via Developersdex http://www.developersdex.com ***Don't just participate in USENET...get rewarded for it!
View 1 Replies View RelatedI'm trying to perform a calculation on a field in SQL Server thatshould return a value with a decimal point. My problem is that thevalue returned is truncated without the decimal point. Is there asetting that needs to be turned on in SQL server to allow this?for exampleSelect 20/3should return 6.6666667but instead I get 6
View 1 Replies View RelatedHi all
I'm a newbie in SQL server and please excuseme for this silly question, Could anyone tell me when i should use which of the following types:
Decimal
Float
Real
I've mixed up !!! all of them can have floating point BUT what's the difference? some advise please!
Thanks in advance.
Kind Regards.
Hi SQL people,
I have rating system on pages in my website, each page being rated one to five by users. At the back end, an ASP.NET page displays the average rating and number of ratings for each page. As the rating is stored as an integer, the SQL statement returns an integer average. I would like to get a floating point average. Currently the statement used to return the average looks like this:
select AVG(Rating) as AverageRating from Ratings where [RatingPage] = @RatingPage
Is there a simple way to modify this to return a floating point average without iterating through the records, or converting the Rating field to a float?
Thanks for helping!
I have the following query:
SELECT AVG(respondent_question.answer)
Now, the 'answer' column is an integer. However, I want the query to return a floating point number with one decimal place (i.e., 5.4, 2.3)
Since the column is an integer, the query returns only an integer by rounding to the nearest integer. How can I do this? Thanks!
Hi,
I got below error in the SQL Server Production Server and i checked in the microsoft site it needs to install SQL Server service pack 4 to resolve the
problem.
"A floating point exception occurred in the user process. Current transaction is canceled"
I need help that i want to reproduce this below problem in the SQL Server environment and tried several ways but no luck.
Please advise me how to reproduce the problem.
Would be appreciate your help.
Regards
Sathish
I'm having a problem that I think is due to corrupt data. Depending on
the column I use in my order by clause two problems are occuring.
1. No results are returned and I get this error:
A floating point exception occured in the user process.
2. Results are returned but there are a different number of rows depending on which columns I use in my Order By clause.
Examples
SELECT * FROM SymbolStats
ORDER BY calc_date, symbol
Returns - 12207 rows but only includes one of the 25 dates in the table.
----------
SELECT * from SymbolStats
ORDER BY current_hv
Returns - 0 rows.
----------
SELECT * from SymbolStats
ORDER BY average_hv
Returns - floating point error
With more conditions in the WHERE clause the number of results returned varies greatly.
The
fact that different numbers of rows can be returned from the same query
only differing in how they are ordered seems like a bug.
Does this sound like corrupt data? If so, what are the best methods for fixing it?
Thanks,
patrick
I'm getting this error:
System.Data.SqlClient.SqlException: A floating point exception occurred in the
user process. Current transaction is canceled. at
System.Data.Common.DbDataAdapter.Update(DataRow[] dataRows, DataTableMapping
tableMapping) at ...etc etc
when updating one particular table, but not when updating another using nearly the same code.
I've searched and searched on the inet, but with no joy.
Microsoft have several different manifestations of it, under various
different circumstances, but the solutions all seem to involve
installing SP4, which by the looks of it is a massive procedure that
you have to go to university to be competent to do.
In any case, according to Enterprise Manager, I already have SP4
included. On the other hand, the one file of mine I checked the date of
against MS's hotfix file list was 2002 not 2003, so who knows?
My Service Manager is v 8.00.760, and it's the Development Edition (ie it's all running on the one machine).
According to the info on sqlDataAdaptor.update, the
sqlError that gets returned as part of the sqlException contains
fields Class, which gives the seriousness of the error, and State,
which identifies the exact error. I get Class=16 (user-fixable)
and State=1, which we have to go to SQL Books Online to discover the
meaning of. SQL BO probably sounded like a good idea to W.Gates,
but it took me half an hour to get anywhere near finding out what that
'1' meant. When I did, it was (guess): 'A floating point
exception occurred in the user process. Current transaction is
canceled.' !!!
Can anyone please tell me what exactly might be causing the error?
Graham Rounce
I am trying to understand why SQL Server gives me significantly lower precision than many of the other sources that have tried when using the POWER function. My environment is 2008 R2 SP2 (10.50.4000.0 X64, Standard edition) on Windows 2008 X64 SP1
DECLARE @x FLOAT(53) = 1.0004;
DECLARE @y FLOAT(53) = 1.0/12.0;
SELECT POWER(@x,@y)-1; -- Answer: 3.33270904724348E-05
GO
DECLARE @x FLOAT(24) = 1.0004;
DECLARE @y FLOAT(24) = 1.0/12.0;
[URL] ....
Answer: 3.33272237835747E-05
I also tried using Windows Calculator.Answer:3.3327223783495255846580902358195e-5
And an online "high-precision" calculator from [URL] ...
Answer: 3.332722378349525584658E-5
The best SQL results compare only to the fourth digit with other results. Everything else agrees to the 10th or 12th digit.
Is the precision of arithmetic calculations on SQL Server that bad, or am I doing something wrong here? Is there another more precise alternative?
I did the following. The results speak for themselves
DECLARE @x FLOAT(53) = 1.0004;
DECLARE @y FLOAT(53) = 1.0/12.0;
DECLARE @z FLOAT(53) = POWER(@x,@y)-1; -- Answer: 3.33270904724348E-05
SELECT POWER((1+@z),12); -- 1.00039999839968
-- using results from other sources (c#, windows calc, casio.com)
SELECT POWER((1+3.33272237835747E-05),12) -- 1.0004
SELECT POWER((1+3.3327223783495255846580902358195e-5),12)-- 1.0004
SELECT POWER((1+3.332722378349525584658E-5),12) -- 1.0004
It is not important what I am trying to do, but in case it will work, I am trying to calculate the monthly return for a 90-day T-bill given the compounded annual return. x is the annualized return.
sql server 2000 sp4 build 2184runs stats with maintenance plansp_msforeachtablestored procedure updating full stats for each table and the resultsare identicalA floating point exception occurred in the user process. Currenttransaction is canceled.all pointers are to move to latest service packAny pointers on how to avoid this errorThanks for your timeMassa
View 6 Replies View RelatedI am trying to run "select * from <table> where <columnname> = 0.0" inquery analyzer<columnname> is a floatI get the following error:Server: Msg 3628, Level 16, State 1, Line 1A floating point exception occurred in the user process. Currenttransaction is canceled.I get the same error when I use the float <columnname> through anapplicationI have tried upgrading to SP4, without any luck - still get the errormessage.Any help would indeed be appreciated.Thanks -AH
View 1 Replies View RelatedWe get the following error message."a floating point exception occured in the user process. currenttransaction is cancelled".this message comes when trying to excute a stored procedure. Thisexception is unpredictable.OS : Windows 2000 (SP3)Version: SQL server 2000 (SP3).
View 1 Replies View RelatedHi,I'm running SQL Server Version 8.00.194 on Windows 2000.I am am running this query:select TOP 2000TheoVolImpliedfrom OptionTradeswhere ReutersSymbol = 'IBM.N'and TheoVolImplied > 0.0TheoVolImplied is of type float, precision 15, length 8.When I run this query I get this error:Server: Msg 3628, Level 16, State 1, Line 1A floating point exception occurred in the user process. Currenttransaction is canceled.If I run this query:select TOP 2000TheoVolImpliedfrom OptionTradeswhere TheoVolImplied > 0.0It works fine with no problems.If I run this query:select TOP 2000TheoVolImpliedfrom OptionTradeswhere ReutersSymbol = 'IBM.N'It works fine with no problems.Anyone have any ideas about what might be wrong?
View 2 Replies View RelatedHi,
I have a data file that has numeric data that looks like:
1.123456
And this column is defined as a DT_NUMERIC(18.6) in the flat file conn mgr.
As an experiment, I changed the destination column to a NUMERIC(18,0) - hoping that this would throw a truncation error at the flat file task level (where I have Truncation on all columns set to "fail component").
Not a peep. It loaded the data into the table, chopping off the 6 digits after the decimal point.
You would THINK that this would cause an error, but no. Why is this? The flat file task complains about all kinds of things, but this is such a gross error, you would think it would catch it!
Thanks
Hello all,
First off, I appreciate the time that those of you reading and responding to this request are offering. My quesiton is a theoretical and hopefully simple one, and yet I have been unable to find an answer to it on other searches or sources.
Here's the situation. I am working with SQL Server 2005 on a Windows Server 2003 machine. I have a series of databases, all of which are in Full recovery mode, using a backup device for the full database backups and a separate device for the log backups. The full backups are run every four days during non-business hours. The log backups are run every half hour.
Last week, one of my coworkers found that some rarely-used data was unavailable, and wanted to restore a database to a point in time where the data was available. He told me that point in time was some time back in November.
To accomplish this, I restored the database (in a separate database, as to not overwrite my production database) using the Point in Time Recovery option. I selected November from the "To a point in time" window (I should note that this window is always grey, never white like most active windows, it seems), and the full database backup and the subsequent logs all became available in the "Select the backup sets to restore" window.
I then tried a bevy of different options from the "Options" screen. However, every restore succeeds (ie: it doesn't error out), but seems to be bringing the database back to a current point in time. It's never actually going back to the point in time I specify.
My questions are as follows:
a) Is it possible to do a point in time recovery to a point in time BEFORE the last full database backup?
b) If so, what options would you recommend I use? (ie: "Overwrite the existing database", restore with recovery, etc etc).
I again appreciate any and all advice I receive, and I look forward to hearing from anyone and everyone on this topic. Thank you.
Ryan
for example:
SELECT CAST(CAST(getdate() AS datetime) AS float)
how can i convert the return select value to nvarchar????
I'm running SQL Server 2005 on a Server 2003 machine serving both our home network as well as a remote site through a point-to-point T1. While file transfer speeds are up to par, the remote site's interaction with SQL Server (Point of sale system) is very slow. After testing I am certain that it has nothing to do with the actual physical machine in place neither is it an issue with the program itself since speeds are as they are supposed to be over the home network lan. It seems that there might be a packet size issue or something of the sort. Has anyone dealt with this before or have any thoughts?
Thanks,
Peter
If you are running in Full Recovery Mode and do a full backup every night but never do a backup of the log during the day does the log file ever truncate? From what I read this should be in Simple Recovery Mode but I'm wondering what happens in the case that I mention in the first sentence. Thanks.
View 1 Replies View RelatedHello Everyone and thanks for your help in advance. I am working on importing a flat text file into SQL Server 2005 and am having problems. The flat file is a CSV text file with " being used as a text qualifier. Each line is broken by a CrLf combination. When I try importing this file into a SQL Server 2000 table using the same datatypes and sizes for each column, it works perfectly fine with the data importing as expected. However, in SQL Server 2005, again using the identical column datatypes and sizes, the import fails giving me warnings such as:
* Warning 0x802092a7: Data Flow Task: Truncation may occur due to inserting data from data flow column "Column 0" with a length of 50 to database column "MLS_ID" with a length of 10. (SQL Server Import and Export Wizard)
Virtually every columns gives this type of warning, yet I don't understand why since the columns are all variable in length (every message says a column length of 50) and all are delimited rather than fixed size. Then later in the import, errors occur something like:
* Error 0xc02020a1: Data Flow Task: Data conversion failed. The data conversion for column "Column 15" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.". (SQL Server Import and Export Wizard) * Error 0xc020902a: Data Flow Task: The "output column "Column 15" (70)" failed because truncation occurred, and the truncation row disposition on "output column "Column 15" (70)" specifies failure on truncation. A truncation error occurred on the specified object of the specified component. (SQL Server Import and Export Wizard)
I haven't got a clue as to why this is happening. For the record, on the flat file source screen, I have ensured that delimited has been selected rather than fixed width. Any help on this issue owuld be greatly appreciated. Thanks.
Can anyone help with this error I am receiving when I perform the following
piece of code???:
INSERT INTO MEMBELIG
SELECT MEMBELIG_temp.*
FROM MEMBELIG_temp
WHERE (LOB = 'PPO')
I am receiving the following error:
Server: Msg 8152, Level 16, State 9, Line 1
String or binary data would be truncated.
The statement has been terminated.
Any help would be greatly appreciated, I am new to this stuff.
Thanks,
Rey
Hi,
Could someone please let me know what are the exact steps to follow to truncate the transaction log files? As these log files grow very fast and there seems to be no space in the drive.
Currently am using the below steps to truncate the log file:
Step1:Use the below syntax:
backup log <database name> with no_log
Step2:shrink the log file. Right click the correct database and choose shrink file ->chosse the log -> ok
I would be grateful if someone can give me a proper solution.
Thanks.
Hello All,I am attempting a bulk load of fixed position flat file data via bcpand I have noticed that I get a Right Truncation error when trying toload a row where the last column value is NULL.For example:Flat file row:0000016MFMT file:7.031 SQLCHAR 0 7 "" 1 RECORD_KEY2 SQLCHAR 0 1 "" 2 SEX3 SQLCHAR 0 1 "
" 3 HEIGHTIn this row, the height info is null and I get a right truncationerror. The row below, with height info goes in fine:Flat file row:0000016M510Let me know what I am doing wrong!Thanks in advance
How is it possible to avoid truncation errors in MS SQL? For example,if I run the followingdeclare @a as decimal(38,8)declare @b as decimal(38,8)declare @c as decimal(38,8)set @a = 30.0set @b = 350.0set @c = @a/@bselect @cset @c = @c*@bselect @cI get 29.99990000 instead of 30.0. Is there a way around this?ThanksBruno
View 6 Replies View RelatedHi,I'm trying to upload a large number of log entries currently stored astext files into a database table using bcp. For a few rows I get a"right truncation" error and the offending rows are not uploaded to thetable.I don't want to increase the size of the table varchar fields becauseit's only about a dozen out of almost million rows that have thisproblem ... I want to provide an override - i.e. if a row will resultin truncated data, truncate but still bulk copy the offending row. Isthat possible?I couldn't find such an option in the documentation.Any help is greatly appreciated.Thanks,Mudassir Latif
View 2 Replies View RelatedHello,I am attempting to write a stored procedure that builds and executes adynamic SQL statement which can be up to 8000 characters long.Therefore, I have declared a variable of type varchar(8000) which,according to the documentation, is the maximum acceptable length ofsuch a variable. Unfortunately, however, SQL Server seems allowvarchars to only be half this size: the resulting string keepingsgetting truncated to 4000 characters as reported by the len function.Is there setting somewhere that would fix this behavior or somework-around that I can employ that would allow me to execute a dynamicsql statement that is longer than 4000 characters?(note: I am not using the sp_executesql proc as it maxes out at 4000; Iam simply calling EXEC which, according to the docs, should be fine)Thank You.
View 1 Replies View RelatedWe use SQL Server 2005 x64 Enterprise and I have created a SSIS routine to replace a legacy DTS routine that reads from a Data Reader Source and writes to a SQL Server 2005 database. The field I am receiving the truncation error on is "Description" and it is set as nvarchar(50), which it always has been, and the old DTS routine works fine on it. I checked the contents of description and the maximum number of characters in any row is 28. I have tried changing it to nvarchar(max), nvarchar(4000) and ntext but it still fails with a truncation error. Any leads on how I may solve this issue?
All,
I am having a strange problem that I have been looking at for a day now, and my head is straing to hurt as I have banged my head on the desk so many times. I have written an extensible set of classes that allow me to build SSIS packages dynamically via a web front end. I am finding this code is working OK, but I have this silly bug.
The code is trying to generate an SSIS package that does somethign very simple, and transfer data from a 10 coumn table with a mix of data types, move through another component that adds a couple of extra columns on basic on some variables, then map it onto a OLD DB destination. This code works fine, until I start using strings of various lengths.
When the package runs, it fails validation with errors saying that truncation may occur as I am trying to put a 100 character string, into a 50 character string. The error is logical as you wouldn';t want to do that, but this is not what I am doing. I am actually transfering data from a 50 character string into a 100 character string. When I try it with a table where the strings are the same length at both ends, or no strings are involved, everything works fine, and the data goes from the source to the destination.
I must be setting something slightly wrong which only triggers this problem when the sizes don't match, but the data flow direction is fine, and the data types match. I have included the code from the piece of code that 'writes' the output part of the package. If anyone has any idea what might be going wrong, I would be forever in their debt!
Code Block
private void ConfigureInputColumns(Package package, MainPipe pipe, IDTSOutput90 SourceColumnCollection)
{
_InstanceOfDestination.AcquireConnections(null);
_InstanceOfDestination.ReinitializeMetaData();
IDTSPath90 path = pipe.PathCollection.New();
path.AttachPathAndPropagateNotifications(SourceColumnCollection,_Destination.InputCollection[0]);
IDTSInput90 input = _Destination.InputCollection[0];
IDTSVirtualInput90 vInput = input.GetVirtualInput();
int iIndex = 0;
foreach (IDTSVirtualInputColumn90 vColumn in vInput.VirtualInputColumnCollection)
{
// Call the SetUsageType method of the destination
// to add each available virtual input column as an input column.
_InstanceOfDestination.SetUsageType(input.ID, vInput, vColumn.LineageID, DTSUsageType.UT_READWRITE);
}
IDTSExternalMetadataColumn90 exInputColumn;
foreach (IDTSInputColumn90 inColumn in _Destination.InputCollection[0].InputColumnCollection)
{
// create the MAP
// What we need to do here, is say what source column, goes to what destination column.
// we read by index, and we need to map the specifics, we could control just 3 of 20 columns
// to whatever column we wanted in the destination......
// We know the name of the inColumn - It is called inColumn.Name, we need to find the column
// we want to map with by finding its name.
if (inColumn.Name == "BatchID" || inColumn.Name == "ValidationStatus")
{
exInputColumn = _Destination.InputCollection[0].ExternalMetadataColumnCollection[inColumn.Name];
// map it
_InstanceOfDestination.MapInputColumn(_Destination.InputCollection[0].ID, inColumn.ID, exInputColumn.ID);
}
else if (Mapping.Map.ContainsKey(inColumn.Name))
{
exInputColumn = _Destination.InputCollection[0].ExternalMetadataColumnCollection[Mapping.Map[inColumn.Name]]; // inColumn.Name
// map it
_InstanceOfDestination.MapInputColumn(_Destination.InputCollection[0].ID, inColumn.ID, exInputColumn.ID);
}
}
_InstanceOfDestination.ReleaseConnections();
}
I have always assumed that when you backup a SQL Server database the transaction log is automatically truncated so that there is no need to explictly truncate it. It makes sense to me, you would not normally need logs from before the most recent backup. BOL, with all its talk about check points etc, seems to hint at this but I can't find an explicit statement to this effect.
Anyone?
Dick Campbell
Hello Guys I am using XML files and dumping data to sql server 2005 , i have field called as rate which is having money as datatype and i am getting following error
LoadDataXML to XML Source -- LoadDataXML [907]: The value was too large to fit in the output column "RATE" (95245).
please help me out with the solution of this ...the data which is coming from xml file is unsigned itneger single bit and my database is having money .so should i use a conversion task in between if any body can give idea about this that would be great , if you want more information tell me ...
thanks
krish