Subject RE: [firebird-support] Problem with division
Author Helen Borrie
At 10:14 PM 4/02/2004 -0800, you wrote:
>Thanks Helen.
>Next ?:
>How would I then take that PCT field and multiply it by the total amount
>being prorated and assign it to a money field AMT numeric(18,2)?
>I've tried:
>AMT = Pct * 1700.00
>
>But I get the error:
>"unsuccessful execution caused by system error that does not preclude
>successful execution of subsequent statements.
>Integer overflow. The result of the integer operation caused the most
>significant bit of the result to carry."
>
>and I just tried (FC = 1700.00):
>AMT = Cast((FC * Pct) as Numeric(18,2));

You need to keep control of the scale of these numbers. When you multiply
or divide, the scales of the operands are added together and the powers of
10 stack up. You have 18 digits in total to work with - when you go over
that, you're busted.

So work out whether you need to keep pushing the scale higher and higher.
It doesn't make much sense for "things you count". If these are
high-precision calculations on measurements (rather than bean-counts) you
should be using double precision numbers.

/heLen