Subject Re: [firebird-support] Re: Mathematics problem
Author Miki Avramovic
Thanks,

I solved the problem. I send message only like comment for other
programers.

See this:
Testing with Fb/1.5.2 SS [WI-V6.3.2.4731 Firebird 1.5] (dialect 3):

1.
select (109.87 - 5*109.87/100) from rdb$database
= 104.38
2a.
select (109.87 - 5*(109.87/100)) from rdb$database
= 104.42
2b.
select cast((109.87 - 5*(109.87/100.00)) as decimal(9,2))
from rdb$database
= 104,38
3.
select (109.87 - (5*109.87)/100) from rdb$database
= 104.38

Thanks for your time.

Best Regards!
Miki


> Hi Miki,

> select 
> cast( cast(109.87 as numeric(18,2))-cast(109.87 as numeric(18,2))
> *cast(5.0 as double precision)/100 as double precision) from
> rdb$database

> gives, 104.365  as expected exact result.

> There must be a problem with you, check yourself again.

> if you want to escape from rounding and cutting problems as
> possible, don't mix
> different type values in a calculation.
> for example 100 is an integer but doesnt effect here luckly.

> operations between 18,2 and double precisions may cause some
> precision losts.

> Convert all off them to double precision or
> anough to save numeric(n,p) type before calculation.

> + modify your data browser tool configuration about display of
> numeric values.
> (for example ibexpert uses (*,3) and rounds value before display)

> Ragards.
> Ali






> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

> Visit http://firebird.sourceforge.net and click the Resources item
> on the main (top) menu.  Try Knowledgebase and FAQ links !

> Also search the knowledgebases at http://www.ibphoenix.com

> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++






> YAHOO! GROUPS LINKS


>  Visit your group "firebird-support" on the web.
>    To unsubscribe from this group, send an email to:
>  firebird-support-unsubscribe@yahoogroups.com
>    Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.