| Subject | Re: casting to decimal(9,2) gives different answers if used with a group by clause |
|---|---|
| Author | wibbleian2 |
| Post date | 2009-04-16T15:23:02Z |
--- In firebird-support@yahoogroups.com, Kjell Rilbe <kjell.rilbe@...> wrote:
Ian
>This is the nub of the question!
> Why the rounding errors differ between group and no group, I cannot say,
> but it's not that strange.
>
> What you should do is to cast to decimal BEFORE adding. Like this:Unfortunately that would not give the correct result. Assume 12.125 + 12.125 - applying the rounding before summing would give 24.26 - rather than the required 24.25!
>
> select sum(cast(qty as decimal(9,2))) from tmp;
>
Ian