Subject Re: [firebird-support] Why does FB evaluate numbers this way?
Author Helen Borrie
At 12:24 AM 24/06/2006, you wrote:

> > select (3/2)
> > from rdb$database
> >
> > The result should be 3/2 = 1.5. Instead I get 1. Why?
>
>3 is an integer, 2 is an integer. You are performing an integer
>division.

The real reason that 3/2 = 1 is that integer/integer must give an
integer result. There are no "rounding rules": the answer is
categorically only the integer part. All in accordance with the SQL standard.


>select (3.0/2) will also bring up 1.5
>A float or decimal type is a "larger" type than integer and the larger
>type will determine the type of the operation.

It might be true by observation but for the DECIMAL and NUMERIC
types, the rule has nothing to do with which is "larger". These are
fixed (or more correctly, scaled) types. the result of division and
multiplication yields a subdecimal part that is the sum of the scales
of all participating numbers. (And INTEGER is a scaled type with a
scale of 0).

So - 4.5/2 gives 2.2, while 4.5/2.0 gives 2.25.

Floating point numbers are different again. If 4.5 is a float or
double precision type, then 4.5/2 gives 2.25. However, float 4.3/2
gives 2.15000009536743, albeit the last 8 digits must be discarded
because the precision of float is good only for the first 7
significant figures.

./heLen