Subject Re: [firebird-support] SP float results rounded strangely (dbExpress)
Author Geoff Worboys
> Via my Delphi+dbExpress program (and also EMS IB/FB Manager)
> this procedure gives
> 0.666666686534882

> but I think it should rather give
> 0,666666666666667

> isql gives
> 0.66666669

> so it seems to be client issue. How big is float in Firebird?

> Any recommendations how to avoid this incorrect rounding in
> dbExpress?

"float" (as per the various documentation available) is a 32bit
float value - significant to only 7 digits - which is what you
can expect from C++ "float" data types and "single" in Pascal.

So 0.666666686534882 is accurate to 7 digits.

The fact that EMS or Delphi/dbExpress may be showing you more
digits than are significant is a problem for them.

If you want higher precision cast to DOUBLE PRECISION which is
a 64bit floating value - 15 digits.


For other datatype see the Interbase 6 documents available at:
http://www.firebirdsql.org/index.php?op=doc&id=userdoc

Even better get Helen's book:
http://www.ibphoenix.com/main.nfs?a=ibphoenix&page=ibp_firebird_book

--
Geoff Worboys
Telesis Computing