Subject Re: IB6 BUG #221589: numeric fields and mathematical operations
Author lobolo2000
Thanx Svein.
I had nearly lost hope in getting a reply to my post.

SELECT 10.5/100 FROM RDB$DATABASE gave 0.105 (and not 0.1) as I would
expect. But anyway, this query is a little bit different than the one in my
post as 0.1 is not an intermediate result. Having intermediate results with
a low precision would be awful. I would expect from any decent expression
evaluator to use doubles or better for intermediate results. And my version
of IB (6 open source) is consistent in this respect.

Am I missing something?

Regards

----- Original Message -----
Date: Thu, 23 Aug 2001 09:02:42 +0200
From: Svein Erling Tysvær <svein.erling.tysvaer@...>
Subject: Re: IB6 BUG #221589: numeric fields and mathematical operations

This is due to IB rounding intermediate values, and is actually visible even
if you reduce your
statement to

SELECT 10.5/100 FROM rdb$database, which gives 0.1 as the result.

To get the correct result, you could simply change this statement to

SELECT 10.5/100.00 FROM rdb$database, which will give you the correct number
of decimals (0.105).

HTH,
Set


_________________________________________________________
Do You Yahoo!?
Get your free @... address at http://mail.yahoo.com