Subject Re: [firebird-support] Re: casting to decimal(9,2) gives different answers if used with a group by clause
Author Mark Rotteveel
> I also have stashed in my head that "group by" appears to do some internal
> rounding that I wasn't expecting so I wont be surprised if this happens
> again. I wont like it if it does, but I wont be surprised. This is the
> general solution to my question. If you can call it a solution.

No it doesn't do some internal rounding, as some one posted earlier: you are using floating point calculation, which by definition is not precise. The result can also depend on the order of the addition, the order of addition might be influenced by the group by.

> Obviously the other solution is to persuade customers that the result of
> multiplying lots of precise numbers together and then rounding arbitrarily
> in different places will give you rounding errors that can be comfortably
> ignored. But I've not met a user that can be persuaded of anything, so I'll
> go back to waiting for the revolution.

No, you aren't multiplying or using precise numbers, floating point numbers are not precise.
The actual solution is: don't use floating point numbers when working with financial data, either use something like NUMERIC(18,2) / DECIMAL(18,2) (or 18,3 or 18,4 depending on your requirements or accounting rules) or INTEGER or BIGINT with decimal shifting in your own code.

Mark
--
Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01