Subject | Round gives inconsistent results for DOUBLE PRECISION expressions |
---|---|
Author | Geoff Worboys |
Post date | 2019-07-29T03:49:59Z |
Hi,
Firebird v2.5
Can anyone explain why the following example gives inconsistent
results. The first two I consider correct (rounded up to 8.42),
the last four I consider incorrect (rounded down to 8.41).
EXECUTE BLOCK
RETURNS (
"Result" DOUBLE PRECISION
) AS
DECLARE A DOUBLE PRECISION;
DECLARE B DOUBLE PRECISION;
DECLARE C DOUBLE PRECISION;
BEGIN
A = 170.0;
B = 4.95;
C = 100.0;
"Result" = round( a * (b / c), 2 );
SUSPEND;
"Result" = a * (b / c);
"Result" = round("Result", 2);
SUSPEND;
"Result" = round( a * b / c, 2 );
SUSPEND;
"Result" = a * b / c;
"Result" = round("Result", 2);
SUSPEND;
"Result" = round( (a * b) / c, 2 );
SUSPEND;
"Result" = (a * b) / c;
"Result" = round("Result", 2);
SUSPEND;
END
I was changing over from a UDF of my own that used an explicitly
DOUBLE PRECISION input parameter. When I use that UDF in the
above code, all results return the same 8.42 value.
--
Geoff Worboys
Telesis Computing Pty Ltd
Firebird v2.5
Can anyone explain why the following example gives inconsistent
results. The first two I consider correct (rounded up to 8.42),
the last four I consider incorrect (rounded down to 8.41).
EXECUTE BLOCK
RETURNS (
"Result" DOUBLE PRECISION
) AS
DECLARE A DOUBLE PRECISION;
DECLARE B DOUBLE PRECISION;
DECLARE C DOUBLE PRECISION;
BEGIN
A = 170.0;
B = 4.95;
C = 100.0;
"Result" = round( a * (b / c), 2 );
SUSPEND;
"Result" = a * (b / c);
"Result" = round("Result", 2);
SUSPEND;
"Result" = round( a * b / c, 2 );
SUSPEND;
"Result" = a * b / c;
"Result" = round("Result", 2);
SUSPEND;
"Result" = round( (a * b) / c, 2 );
SUSPEND;
"Result" = (a * b) / c;
"Result" = round("Result", 2);
SUSPEND;
END
I was changing over from a UDF of my own that used an explicitly
DOUBLE PRECISION input parameter. When I use that UDF in the
above code, all results return the same 8.42 value.
--
Geoff Worboys
Telesis Computing Pty Ltd