Subject | Re: [firebird-support] Problem with rounding |
---|---|
Author | Mark Rotteveel |
Post date | 2009-08-11T16:18:39Z |
Svein Erling Tysvær wrote:
--
Mark Rotteveel
>> select * from zaok(67.865,2) - returns 67.86 - wrong resultYou are right, that could also be the cause of the (perceived) problem.
>
> Why is this wrong? In your procedure, you've defined the input parameter LICZBA as double precision and double precision is never precise. So when you specify 67.865, you're not passing the actual number 67.865 to the stored procedure, but an approximation that may be close to 67.8650000001 or 67.8649999999 (well, probably you have to add more digits to see the difference with double precision). Casting the first of these approximations to NUMERIC (18,2) would result in 67.87, the latter 67.86.
>
> The rule is simply never to assume a floating point number to be exact. Don't even think that 67.865 = 67.865 when using a floating point number.
--
Mark Rotteveel