Subject | rounding variant |
---|---|
Author | Radu Sky |
Post date | 2006-05-09T12:06:04Z |
Hello,
Could somebody please explain why these results? I tried googling and
searching in NG but I couldn't find a logical explanation
12.34/27=0.45703703703703703703703703703704
ISQL results:
SELECT 12.34/27 from rdb$database => 0.45
SELECT CAST(12.34/27 AS double precision) from rdb$database =>
0.4500000000000000
SELECT CAST(12.34/27 AS DECIMAL(18,2)) from rdb$database =>0.45
Now(note the 27.0):
SELECT 12.34/27.0 from rdb$database => 0.457
SELECT CAST(12.34/27.0 AS double precision) from rdb$database =>
0.4570000000000000
SELECT CAST(12.34/27.0 AS DECIMAL(18,2)) from rdb$database =>0.46
I used to use the
CAST(<some decimal>*(100+<some_smallint>)/100 AS DECIMAL(18,2))
and I thought that it should return the banker's rounding
Thanks
Radu
Could somebody please explain why these results? I tried googling and
searching in NG but I couldn't find a logical explanation
12.34/27=0.45703703703703703703703703703704
ISQL results:
SELECT 12.34/27 from rdb$database => 0.45
SELECT CAST(12.34/27 AS double precision) from rdb$database =>
0.4500000000000000
SELECT CAST(12.34/27 AS DECIMAL(18,2)) from rdb$database =>0.45
Now(note the 27.0):
SELECT 12.34/27.0 from rdb$database => 0.457
SELECT CAST(12.34/27.0 AS double precision) from rdb$database =>
0.4570000000000000
SELECT CAST(12.34/27.0 AS DECIMAL(18,2)) from rdb$database =>0.46
I used to use the
CAST(<some decimal>*(100+<some_smallint>)/100 AS DECIMAL(18,2))
and I thought that it should return the banker's rounding
Thanks
Radu