Subject | Re: [firebird-support] Store proc calc (rounding) fails |
---|---|
Author | Helen Borrie |
Post date | 2005-10-31T01:47:35Z |
At 01:17 AM 31/10/2005 +0000, you wrote:
unless it was IB 5.6. IB 6 and Firebird follow SQL standards re integer
division: integer/integer yields an integer result. So, in IB 5.6, 10/3
would give you a double precision (3.333333...) while IB 6 and Firebird
will give you 3.
That's probably enough to get you to a solution. If you want decimal
parts, you need to have them in one or other of the operands of a
multiplication or division operation. Note also that casting a mixed
number as integer will also truncate, not round.
./heLen
>Greetings,OK, first, I don't believe you got right results with this in InterBase,
>
> This was a stored procedure that worked in Interbase 6 (open source)
>and is failing in Firebird.. There may be several issues but I'll hit
>them one-at-a-time.. (from debugging in Database Workbench 2.6)
>
>The following line fails with the error
>
> "Array/BLOB/Date not allowed in arithmetic..."
>
>update TMP_FINANCE_CALC
> set FINANCE_CHG = (CAST( ((BALANCE * :IN_RATE )*100) AS INTEGER))/100;
>
>The intention was to round the field to a two place decimal. The
>field FINANCE_CHG is Double Precision, as is the BALANCE field as well
>as the input parameter IN_RATE.
>
>The statement works IF I remove the input parameter as in
>
>update TMP_FINANCE_CALC
> set FINANCE_CHG = (CAST( ((BALANCE * 0.018 )*100) AS INTEGER))/100;
>
>I need to have that input parameter in the calculation..
>
>I have a similar statement later in the procedure that is also
>failing.. (probably for the same reasons..)
>
>L_FINANCE_CHG = ( CAST((L_FINANCE_CHG * 100) AS INTEGER) )/100;
>
>where L_FINANCE_CHG is a declared variable as a Double Precision.
>
>Suggestions, Ideas, kick in the pants....???
unless it was IB 5.6. IB 6 and Firebird follow SQL standards re integer
division: integer/integer yields an integer result. So, in IB 5.6, 10/3
would give you a double precision (3.333333...) while IB 6 and Firebird
will give you 3.
That's probably enough to get you to a solution. If you want decimal
parts, you need to have them in one or other of the operands of a
multiplication or division operation. Note also that casting a mixed
number as integer will also truncate, not round.
./heLen