Subject Why does FB evaluate numbers this way? Paul R. Gardner 2006-06-23T14:03:22Z
select (3/2)
from rdb\$database

The result should be 3/2 = 1.5. Instead I get 1. Why?

select (3.0/2.0)
from rdb\$database

The result is finally 1.50. Wht not just 1.5?

I understand what it's doing with significant digits, but it don't
understand why. In the original example, I had 2. This is the same as
2.0, 2.00, or 2.00000000000000000000000000. It's not like the tenth
through the billionth decimal place holders are NULL and therefore could
be anything, they are assumed as 0 because they were not specified. 2
is 2 and not 2.01 or 2.0000000000000000000000001. To me, the answer
should have been 1.5.

OK, so let's say I get past my issue on the significant digits. Why was
the answer 1? It should have been rounded to 2 under any rounding
convention. Typically x.5 is rounded to x+1. In bankers rounding you
round towards the even number which is the same result: 2.

Any thoughts?

Paul

[Non-text portions of this message have been removed]