Subject Re: [IBO] Rounding problem
Author Geoff Worboys
>> It seems likely to me that we are dealing with a field that is
>> represented as a floating point. I have never really studied
>> the AVG code to see what type it returns under various
>> conditions. I suspect that the problem may be that TIB_Query
>> is looking at the data type (NUMERIC(15,2)) and treating it as
>> a scaled integer - whereas TIBOQuery is reading it as a
>> floating point.

> TIBOQuery reads it as a BCD, not as a floating point.


Where are you taking that from? (Experiment or assumption?)

In the GetDataTypeAndSize function in IBODataset.pas there is
this code:

if SQLScale <> 0 then
NewDataType := ftFloat;
case SQLType of
if IsCurrencyDataType then
NewDataType := ftBCD;
NewDataSize := -SQLScale;

Notice that an SQLType of float or double will result in the
NewDataType being returned as ftFloat.

If the database is dialect 1 (or was upgraded from dialect 1)
then the underlying SQLType is probably double. So it appears
to me that such instances will result in iboquery with ftFloat.

Disclaimer: I have not setup any experiments to test all this.
Just passing on my guesstimate that dialect 1 vs dialect 3
issues could be confusing this situation. Hence the request
for more information.

Geoff Worboys
Telesis Computing