Subject | Re: [ib-support] rounding in IB 5.6 (Sco) with perl |
---|---|
Author | David K. Trudgett |
Post date | 2001-09-04T07:04:08Z |
On Tuesday 2001-09-04 at 16:18:10 +1000, Bill Katelis wrote:
(i.e., as floating point instead of a scaled Int64, for instance).
In a pre-V6 database (your case), or a V6 Dialect 1 database, NUMERIC(15,2)
will be stored as DOUBLE PRECISION. V6 Dialect 3 will store it as an INT64.
numerics (a scaled 64-bit integer), you shouldn't need to do rounding
in your application. As you are using InterBase prior to V6, you may
have more difficulty with that, because prior to V6 there was no
int64 native datatype in InterBase.
If you can't upgrade your database to V6 Dialect 3, then rounding the
values before you use them may solve your problem. You may need to
ensure you round your numbers before storing them in the datbase also.
command-line demo works for positive numbers, and always rounds .5
upwards.
perl -ne '$val = int($_ * 100 + 0.5)/100; print "$val\n"'
David Trudgett
> Hi,The values in the database are not being stored exactly
>
> I have a field in the db which is numeric(15,2)
>
> When I sum in isql I get:
>
> SUM
> ======================
>
> 1858.7500
>
> Field values are as follows:
>
> COIN
> ======================
>
> 222.25
> 333.33
> 444.55
> 666.66
> 111.11
> 0.00
> 0.00
> 80.85
>
> When I extract using perl and add the values I get: 1858.74
>
> Why?
>
(i.e., as floating point instead of a scaled Int64, for instance).
In a pre-V6 database (your case), or a V6 Dialect 1 database, NUMERIC(15,2)
will be stored as DOUBLE PRECISION. V6 Dialect 3 will store it as an INT64.
> Should I be rounding before adding as a previous post suggests?If you can ensure the database is storing the numbers using exact
numerics (a scaled 64-bit integer), you shouldn't need to do rounding
in your application. As you are using InterBase prior to V6, you may
have more difficulty with that, because prior to V6 there was no
int64 native datatype in InterBase.
If you can't upgrade your database to V6 Dialect 3, then rounding the
values before you use them may solve your problem. You may need to
ensure you round your numbers before storing them in the datbase also.
>There are many different ways to round numbers. The following
>
> If so, does anyone know how to round to 2 decimal places in perl (apart
> from sprintf)
command-line demo works for positive numbers, and always rounds .5
upwards.
perl -ne '$val = int($_ * 100 + 0.5)/100; print "$val\n"'
David Trudgett