Subject | Re: [IBO] Rounding problem |
---|---|
Author | Geoff Worboys |
Post date | 2005-04-20T07:32:26Z |
>> It seems likely to me that we are dealing with a field that isDaniel,
>> represented as a floating point. I have never really studied
>> the AVG code to see what type it returns under various
>> conditions. I suspect that the problem may be that TIB_Query
>> is looking at the data type (NUMERIC(15,2)) and treating it as
>> a scaled integer - whereas TIBOQuery is reading it as a
>> floating point.
> TIBOQuery reads it as a BCD, not as a floating point.
Where are you taking that from? (Experiment or assumption?)
In the GetDataTypeAndSize function in IBODataset.pas there is
this code:
if SQLScale <> 0 then
begin
NewDataType := ftFloat;
case SQLType of
SQL_FLOAT,
SQL_FLOAT_,
SQL_DOUBLE,
SQL_DOUBLE_: ;
else
if IsCurrencyDataType then
begin
NewDataType := ftBCD;
NewDataSize := -SQLScale;
end;
end;
Notice that an SQLType of float or double will result in the
NewDataType being returned as ftFloat.
If the database is dialect 1 (or was upgraded from dialect 1)
then the underlying SQLType is probably double. So it appears
to me that such instances will result in iboquery with ftFloat.
Disclaimer: I have not setup any experiments to test all this.
Just passing on my guesstimate that dialect 1 vs dialect 3
issues could be confusing this situation. Hence the request
for more information.
--
Geoff Worboys
Telesis Computing