Subject Re: [firebird-support] Problem with rounding
Author Jacek Borowski
Hi,

Thank you for yours reply.

> Svein Erling Tysvær wrote:
>>> select * from zaok(67.865,2) - returns 67.86 - wrong result
>>
>> Why is this wrong? In your procedure, you've defined the input parameter
>> LICZBA as double precision and double precision is never precise. So when
>> you specify 67.865, you're not passing the actual number 67.865 to the
>> stored procedure, but an approximation that may be close to 67.8650000001
>> or 67.8649999999 (well, probably you have to add more digits to see the
>> difference with double precision). Casting the first of these
>> approximations to NUMERIC (18,2) would result in 67.87, the latter 67.86.
>>

Ok, but why results on other platforms (Linux and Windows) are different ?
On Windows results are always "correct", even, when we are using double
precision declaration.
I've changed double precision to numeric(18,6) and now results on Linux are
ok.
Finaly, I wrote my own function and it works fine with double precision
declaration.

>> The rule is simply never to assume a floating point number to be exact.
>> Don't even think that 67.865 = 67.865 when using a floating point number.
>
> You are right, that could also be the cause of the (perceived) problem.
>
> --
> Mark Rotteveel

Many thanks,
Jacek