Subject RE: [firebird-support] CAST/ Round disparity 1.5->2.1
Author Alan McDonald
> At 08:09 PM 1/08/2008, you wrote:
>
> >You can probably poke your finger at this with disdain but this
> simplified
> >query has always returned 65.57 up thru FB1.5.5
> >But now in FB2.1 it returns 65.56
> >
> >SELECT
> > CAST(CAST(1.5 AS NUMERIC(18,2))
> > * CAST(43.71 AS NUMERIC(18,2))
> > * (CAST(0.00 AS DOUBLE PRECISION)/100+1.00)
> > AS NUMERIC(18,2))
> > AS ITEMTOTAL
> > FROM RDB$DATABASE
> >
> >The zero value in the double cast is a constant in this context but
> it's not
> >in real life.
> >It has something to do with this cast as double precision - it's
> changed in
> >FB2.1.
> >My focus here is on loss of consistency, not precision.
> >
> >Does anyone know why consistency has been altered? Is it because some
> grave
> >fault was discovered i.e. precision was in error all this time?
>
> I get the same result (65.57) on Fb 1.5.5, 2.04 and 2.1.1 (on separate
> Win2K hosts, all 32-bit).
>
> ./helen

WI-V2.1.1.17910 Firebird 2.1

C:\Documents and Settings\Administrator>"c:\program
files\firebird\firebird_1_5\
bin\fsql.exe" longblack:smm -user smcreator -password alalal
SQL> SELECT
CON> CAST(CAST(1.5 AS NUMERIC(18,2))
CON> * CAST(43.71 AS NUMERIC(18,2))
CON> * (CAST(0.00 AS DOUBLE PRECISION)/100+1.00)
CON> AS NUMERIC(18,2))
CON> AS ITEMTOTAL
CON> FROM RDB$DATABASE;

ITEMTOTAL
=====================
65.57

SQL>

So something changed briefly
Alan