Subject Re: [IBO] Rounding problem
Author Geoff Worboys
>> It seems likely to me that we are dealing with a field that is
>> represented as a floating point. I have never really studied
>> the AVG code to see what type it returns under various
>> conditions. I suspect that the problem may be that TIB_Query
>> is looking at the data type (NUMERIC(15,2)) and treating it as
>> a scaled integer - whereas TIBOQuery is reading it as a
>> floating point.

> TIBOQuery reads it as a BCD, not as a floating point.

Daniel,

Where are you taking that from? (Experiment or assumption?)

In the GetDataTypeAndSize function in IBODataset.pas there is
this code:

if SQLScale <> 0 then
begin
NewDataType := ftFloat;
case SQLType of
SQL_FLOAT,
SQL_FLOAT_,
SQL_DOUBLE,
SQL_DOUBLE_: ;
else
if IsCurrencyDataType then
begin
NewDataType := ftBCD;
NewDataSize := -SQLScale;
end;
end;

Notice that an SQLType of float or double will result in the
NewDataType being returned as ftFloat.

If the database is dialect 1 (or was upgraded from dialect 1)
then the underlying SQLType is probably double. So it appears
to me that such instances will result in iboquery with ftFloat.

Disclaimer: I have not setup any experiments to test all this.
Just passing on my guesstimate that dialect 1 vs dialect 3
issues could be confusing this situation. Hence the request
for more information.

--
Geoff Worboys
Telesis Computing