Subject Re: [firebird-support] double type in Stored procedure showing lots of decimals.
Author Michael Weissenbacher
>> I have a stored procedure writing a*b, then I get this a*b to be rounded to 8 decimals.
>>
>> a = 2.05
>> b = 36.5902439
>>
>> I stored the data in a dummy table, and this is where I find that sometimes the a*b gives me the value of 75.009999995, but sometimes it gives me the value of 75.0099999949999. And this has caused my value after rounded to 8 decimals not to be exact the same, where it gives 75.01 for the first time, but 75.00999999 for some other time.
>>
>> Anyone experiences this before ?
>>
> Standard Float / Double behaviour, see:
> http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems
>
> regards,
> Michael
>
Sorry to double-post, another interesting link:
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Bottom line: not a Firebird problem at all.

hth,
Michael