Subject Re: casting to decimal(9,2) gives different answers if used with a group by clause
Author wibbleian2
--- In firebird-support@yahoogroups.com, Helen Borrie <helebor@...> wrote:
>
> >SQL> select parent_id, cast(sum(qty) as decimal(9,2)) from tmp group by parent_id;
> >
> > PARENT_ID CAST
> >============ ============
> >
> > 1 130.27
>

Casting to 18,2 made no difference. Applying a dodgy hack the following works "fine":

select parent_id, cast(cast(sum(qty) as decimal(18,3)) as decimal(18,2)) from tmp group by parent_id;

At least it gives the correct answer in this scenario...

> From a design point of view, it will also be important to decide whether a double is what you actually want for your Quantity data. It will obviously matter if it's weighing gold dust...but not if you're selling cables or fabric by the metre...and definitely wrong if your quantities are unitary, e.g. a dozen red roses or 100 goldfish. ;-)
>

Double is probably not the correct type - both too precise and not accurate enough - but I'm a few years passed being able to change that now ;-).

Thanks

Ian

p.s. for the last few months I've being doing mobile java development - and posting lots of stupid questions to assorted support lists. Its nice to come back here after a long break - the questions seem to get answered so fantastically quickly compared to all the other I've used. Well done Firebird community!