Subject Re: [IBO] Rounding problem
Author Geoff Worboys
Riho,

> IBO 4.2.Hc, BCB5

> My query contains AVG(AMOUNT) field, where AMOUNT NUMERIC(15,2).
> I show the results in Grid and QReport.
> QReport uses TIBOQuery, which I initialize with original query:
IBOFStatement->>AssignSQLWithSearch(DSource->Dataset);

> In my TIB_Grid some AVG values are 1 cent smaller (1.63 instead
> of 1.64) than results seen in QReport. Also when I run my query
> in IBAdmin application, the AVG values are similar to QReport
> values.

> Looks like there is some rounding problem in TIB_Grid.
> Everything is with default settings, only TIB_Query has
> FieldsDisplayFormat.Strings = 'AVG=0.00', but changing that
> doesn't change anything.

Some additional information may be required here.

What version of Firebird (or interbase) are you using? What
SQL dialect is the database? What version and dialect of FB
(or IB) was the data created under?

These questions are directed at discovering whether the field
is really double or integer based in database storage. That
is; If the database was originally created as dialect 1 and
either still is dialect 1 or has since been upgraded to a newer
version or dialect, then the NUMERIC(15,2) field will be a
floating point (DOUBLE PRECISION) value in the database. If
created in dialect 3 then the field will be a true scaled
integer.

It seems likely to me that we are dealing with a field that is
represented as a floating point. I have never really studied
the AVG code to see what type it returns under various
conditions. I suspect that the problem may be that TIB_Query
is looking at the data type (NUMERIC(15,2)) and treating it as
a scaled integer - whereas TIBOQuery is reading it as a
floating point.

This is only guesswork at this point - hence the need for more
information.

--
Geoff Worboys
Telesis Computing