Subject Re: [firebird-support] CAST/ Round disparity 1.5->2.1
Author Michael Möhle
Hm,

Linux Server Version: LI-V2.0.1.12855 Firebird 2.0, Dialect 3 = 65,56
Linux Server Version: LI-V2.0.1.12855 Firebird 2.0, Dialect 1 = 65,56

Windows XP Server Version: WI-V2.1.0.17798 Firebird 2.1 D3 = 65,57
Windows XP Server Version: WI-V2.1.0.17798 Firebird 2.1 D1 = 65,56

Windows XP Server Version: WI-V2.0.1.12855 Firebird 2.0 D3 = 65,57
Windows XP Server Version: WI-V2.0.1.12855 Firebird 2.0 D1 = 65,56

and now???




_____

Von: firebird-support@yahoogroups.com
[mailto:firebird-support@yahoogroups.com] Im Auftrag von Helen Borrie
Gesendet: Freitag, 1. August 2008 15:39
An: firebird-support@yahoogroups.com
Betreff: Re: [firebird-support] CAST/ Round disparity 1.5->2.1



At 08:09 PM 1/08/2008, you wrote:

>You can probably poke your finger at this with disdain but this simplified
>query has always returned 65.57 up thru FB1.5.5
>But now in FB2.1 it returns 65.56
>
>SELECT
> CAST(CAST(1.5 AS NUMERIC(18,2))
> * CAST(43.71 AS NUMERIC(18,2))
> * (CAST(0.00 AS DOUBLE PRECISION)/100+1.00)
> AS NUMERIC(18,2))
> AS ITEMTOTAL
> FROM RDB$DATABASE
>
>The zero value in the double cast is a constant in this context but it's
not
>in real life.
>It has something to do with this cast as double precision - it's changed in
>FB2.1.
>My focus here is on loss of consistency, not precision.
>
>Does anyone know why consistency has been altered? Is it because some grave
>fault was discovered i.e. precision was in error all this time?

I get the same result (65.57) on Fb 1.5.5, 2.04 and 2.1.1 (on separate Win2K
hosts, all 32-bit).

./helen





[Non-text portions of this message have been removed]