Subject | Re: [firebird-support] SP float results rounded strangely (dbExpress) |
---|---|
Author | Martijn Tonies |
Post date | 2005-09-06T08:46:59Z |
Hi,
Delphi.
but still floating point, results.
With regards,
Martijn Tonies
Database Workbench - tool for InterBase, Firebird, MySQL, Oracle & MS SQL
Server
Upscene Productions
http://www.upscene.com
Database development questions? Check the forum!
http://www.databasedevelopmentforum.com
> CREATE PROCEDURE FLOAT_PROBLAShould it? This is a floating point precision issue...
> RETURNS (RESULT FLOAT)
> AS
> BEGIN
> select
> 2 / cast (3 as float)
> from dual
> into result;
>
> SUSPEND;
> END
>
>
> Via my Delphi+dbExpress program (and also EMS IB/FB Manager) this
> procedure gives
> 0.666666686534882
>
> but I think it should rather give
> 0,666666666666667
> isql givesA client issue? Howso? "isq" returns an answer just as wrong as
> 0.66666669
>
> so it seems to be client issue. How big is float in Firebird?
Delphi.
> Any recommendations how to avoid this incorrect rounding in dbExpress?Try "double precision" as the data type, that should give you better,
but still floating point, results.
With regards,
Martijn Tonies
Database Workbench - tool for InterBase, Firebird, MySQL, Oracle & MS SQL
Server
Upscene Productions
http://www.upscene.com
Database development questions? Check the forum!
http://www.databasedevelopmentforum.com