Den 2019-07-29 kl. 05:49, skrev Geoff Worboys
geoff@... [firebird-support]:

>
> Hi,
>
> Firebird v2.5
>
> Can anyone explain why the following example gives inconsistent
> results. The first two I consider correct (rounded up to 8.42),
> the last four I consider incorrect (rounded down to 8.41).
>
> EXECUTE BLOCK
> RETURNS (
> "Result" DOUBLE PRECISION
> ) AS
> DECLARE A DOUBLE PRECISION;
> DECLARE B DOUBLE PRECISION;
> DECLARE C DOUBLE PRECISION;
> BEGIN
> A = 170.0;
> B = 4.95;
> C = 100.0;
>
> "Result" = round( a * (b / c), 2 );
> SUSPEND;
> "Result" = a * (b / c);
> "Result" = round("Result", 2);
> SUSPEND;
>
> "Result" = round( a * b / c, 2 );
> SUSPEND;
> "Result" = a * b / c;
> "Result" = round("Result", 2);
> SUSPEND;
>
> "Result" = round( (a * b) / c, 2 );
> SUSPEND;
> "Result" = (a * b) / c;
> "Result" = round("Result", 2);
> SUSPEND;
> END
>
> I was changing over from a UDF of my own that used an explicitly
> DOUBLE PRECISION input parameter. When I use that UDF in the
> above code, all results return the same 8.42 value.
>
> Geoff Worboys
>

Geoff,

The value 8.415 (and probably other part results) cannot be exactly
represented in binary form, just like 1/3 cannot in decimal form. You
get an infinite number of binary "decimals". That means that even the
double precision variables that you use will store rounded results.
These small rounding errors will result in some expressions to land just
under the exact result 8.415, and will thus be rounded down. Other will
land just above 8.415 and be rounded up.

If you need this to be handled in an exact manner, you will have to use
some data type that does not suffer from this kind of rounding errors.
Note: all floating point types have this problem.

Regards,
Kjell Rilbe



[Non-text portions of this message have been removed]