Subject Re: [ib-support] rounding in IB 5.6 (Sco) with perl
Author Bill Katelis
David,
Thankyou for your reply ....

What is the storage difference between numeric(15,2) and numeric(15,4) ....

ie, if I change the field scale on this column could this make a difference ?

bill

"David K. Trudgett" wrote:

> On Tuesday 2001-09-04 at 16:18:10 +1000, Bill Katelis wrote:
>
> > Hi,
> >
> > I have a field in the db which is numeric(15,2)
> >
> > When I sum in isql I get:
> >
> > SUM
> > ======================
> >
> > 1858.7500
> >
> > Field values are as follows:
> >
> > COIN
> > ======================
> >
> > 222.25
> > 333.33
> > 444.55
> > 666.66
> > 111.11
> > 0.00
> > 0.00
> > 80.85
> >
> > When I extract using perl and add the values I get: 1858.74
> >
> > Why?
> >
>
> The values in the database are not being stored exactly
> (i.e., as floating point instead of a scaled Int64, for instance).
>
> In a pre-V6 database (your case), or a V6 Dialect 1 database, NUMERIC(15,2)
> will be stored as DOUBLE PRECISION. V6 Dialect 3 will store it as an INT64.
>
> > Should I be rounding before adding as a previous post suggests?
>
> If you can ensure the database is storing the numbers using exact
> numerics (a scaled 64-bit integer), you shouldn't need to do rounding
> in your application. As you are using InterBase prior to V6, you may
> have more difficulty with that, because prior to V6 there was no
> int64 native datatype in InterBase.
>
> If you can't upgrade your database to V6 Dialect 3, then rounding the
> values before you use them may solve your problem. You may need to
> ensure you round your numbers before storing them in the datbase also.
>
> >
> >
> > If so, does anyone know how to round to 2 decimal places in perl (apart
> > from sprintf)
>
> There are many different ways to round numbers. The following
> command-line demo works for positive numbers, and always rounds .5
> upwards.
>
> perl -ne '$val = int($_ * 100 + 0.5)/100; print "$val\n"'
>
> David Trudgett
>
>
> To unsubscribe from this group, send an email to:
> ib-support-unsubscribe@egroups.com
>
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/