Subject Re: [firebird-support] SP float results rounded strangely (dbExpress)
Author Martijn Tonies
Hi,

> CREATE PROCEDURE FLOAT_PROBLA
> RETURNS (RESULT FLOAT)
> AS
> BEGIN
> select
> 2 / cast (3 as float)
> from dual
> into result;
>
> SUSPEND;
> END
>
>
> Via my Delphi+dbExpress program (and also EMS IB/FB Manager) this
> procedure gives
> 0.666666686534882
>
> but I think it should rather give
> 0,666666666666667

Should it? This is a floating point precision issue...

> isql gives
> 0.66666669
>
> so it seems to be client issue. How big is float in Firebird?

A client issue? Howso? "isq" returns an answer just as wrong as
Delphi.

> Any recommendations how to avoid this incorrect rounding in dbExpress?

Try "double precision" as the data type, that should give you better,
but still floating point, results.

With regards,

Martijn Tonies
Database Workbench - tool for InterBase, Firebird, MySQL, Oracle & MS SQL
Server
Upscene Productions
http://www.upscene.com
Database development questions? Check the forum!
http://www.databasedevelopmentforum.com