Subject | Re: [firebird-support] Re: Mathematics problem |
---|---|
Author | Miki Avramovic |
Post date | 2005-10-08T11:53:35Z |
Thanks,
I solved the problem. I send message only like comment for other
programers.
See this:
Testing with Fb/1.5.2 SS [WI-V6.3.2.4731 Firebird 1.5] (dialect 3):
1.
select (109.87 - 5*109.87/100) from rdb$database
= 104.38
2a.
select (109.87 - 5*(109.87/100)) from rdb$database
= 104.42
2b.
select cast((109.87 - 5*(109.87/100.00)) as decimal(9,2))
from rdb$database
= 104,38
3.
select (109.87 - (5*109.87)/100) from rdb$database
= 104.38
Thanks for your time.
Best Regards!
Miki
I solved the problem. I send message only like comment for other
programers.
See this:
Testing with Fb/1.5.2 SS [WI-V6.3.2.4731 Firebird 1.5] (dialect 3):
1.
select (109.87 - 5*109.87/100) from rdb$database
= 104.38
2a.
select (109.87 - 5*(109.87/100)) from rdb$database
= 104.42
2b.
select cast((109.87 - 5*(109.87/100.00)) as decimal(9,2))
from rdb$database
= 104,38
3.
select (109.87 - (5*109.87)/100) from rdb$database
= 104.38
Thanks for your time.
Best Regards!
Miki
> Hi Miki,
> select
> cast( cast(109.87 as numeric(18,2))-cast(109.87 as numeric(18,2))
> *cast(5.0 as double precision)/100 as double precision) from
> rdb$database
> gives, 104.365 as expected exact result.
> There must be a problem with you, check yourself again.
> if you want to escape from rounding and cutting problems as
> possible, don't mix
> different type values in a calculation.
> for example 100 is an integer but doesnt effect here luckly.
> operations between 18,2 and double precisions may cause some
> precision losts.
> Convert all off them to double precision or
> anough to save numeric(n,p) type before calculation.
> + modify your data browser tool configuration about display of
> numeric values.
> (for example ibexpert uses (*,3) and rounds value before display)
> Ragards.
> Ali
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Visit http://firebird.sourceforge.net and click the Resources item
> on the main (top) menu. Try Knowledgebase and FAQ links !
> Also search the knowledgebases at http://www.ibphoenix.com
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> YAHOO! GROUPS LINKS
> Visit your group "firebird-support" on the web.
> To unsubscribe from this group, send an email to:
> firebird-support-unsubscribe@yahoogroups.com
> Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.