Subject | Problem with floating point precission |
---|---|
Author | Alejandro |
Post date | 2004-07-28T18:37:39Z |
Hi!
I'm trying to do some floating point operations in a select, but
there aren't table columns involved (at least not for certain
results).
Say, for example that I have:
select 7.0 / 800.0, x.Other_Data
from A_Table x;
My first column is returning 0, because only two decimals are being
used and result is below that. I've checked it with larger values and
in fact the maximum precision I get is 0.01
Is there some parameter or configuration option which allows for
increased precision?
Thanks in advance!
I'm trying to do some floating point operations in a select, but
there aren't table columns involved (at least not for certain
results).
Say, for example that I have:
select 7.0 / 800.0, x.Other_Data
from A_Table x;
My first column is returning 0, because only two decimals are being
used and result is below that. I've checked it with larger values and
in fact the maximum precision I get is 0.01
Is there some parameter or configuration option which allows for
increased precision?
Thanks in advance!