| Subject | Re: [firebird-support] Re: casting to decimal(9,2) gives different answers if used with a group by clause |
|---|---|
| Author | Kjell Rilbe |
| Post date | 2009-04-17T04:33:48Z |
wibbleian2 wrote:
cast to take care of the old bad choice of double, to make sure you get
a calculation based on an exact non floating point type with enough
decimal places, then you cast the result to the type you want to display.
It's not strange in any way. It's the way to do it.
Kjell
--
--------------------------------------
Kjell Rilbe
DataDIA AB
E-post: kjell@...
Telefon: 08-761 06 55
Mobil: 0733-44 24 64
> Applying a dodgy hack the following works "fine":What's so dodgy and "hacky" about it? It,s perfectly logical. First you
>
> select parent_id, cast(cast(sum(qty) as decimal(18,3)) as decimal(18,2))
> from tmp group by parent_id;
>
> At least it gives the correct answer in this scenario...
cast to take care of the old bad choice of double, to make sure you get
a calculation based on an exact non floating point type with enough
decimal places, then you cast the result to the type you want to display.
It's not strange in any way. It's the way to do it.
Kjell
--
--------------------------------------
Kjell Rilbe
DataDIA AB
E-post: kjell@...
Telefon: 08-761 06 55
Mobil: 0733-44 24 64