Subject Re: [firebird-support] Arithmatic Overflow in FB 1.0
Author Ivan Prenosil
Multiplying and dividing produce result with scale equal to
sum of the scales of the operands. In your case
> dBegPrice = ((dPrice * dPercent) / dConvert);
the scale will be 21! which is too much even for int64.
You have to put rounding somewhere ...

Ivan



----- Original Message -----
From: "sgharp" <steve@...>
To: <firebird-support@yahoogroups.com>
Sent: Wednesday, July 14, 2004 6:20 PM
Subject: [firebird-support] Arithmatic Overflow in FB 1.0


> Hi All,
>
> I'm getting an error
>
> "Arithmetic overflow or division by zero has occurred.
> arithmetic exception, numeric overflow, or string truncation."
>
> in a stored procedure. I've been able to duplicate it in the
> following.
>
> create procedure spTest
> AS
> declare variable dPercent Numeric(16,7);
> declare variable dConvert Numeric(16,7);
> declare variable dPrice Numeric(16,7);
> declare variable dBegPrice Numeric(16,7);
> Begin
> dPrice = 4;
> dPercent = .42;
> dConvert = 30;
> dBegPrice = ((dPrice * dPercent) / dConvert);
>
> suspend;
> End
>
> Why is this a problem? When I define my Numerics as (12,3) instead
> of (16,7) the problem goes away but I need a greater precision.
>
> I'm not sure whether this is an FB error or is coming from
> IBExpert. I caused an access violation in IBExpert that made me use
> task manager to end the process.
>
> Thanks,
> Steve