Subject | Problem with rounding .. any advice. |
---|---|
Author | Adrian Wreyford |
Post date | 2008-11-11T11:17:26Z |
I have the following situation:
I enter a Cash value in an IB_Edit, then copy it to string.
I then remove any spaces, R or $ signs
code below:
var
PostEntry, TempTotal: Double;
s: String;
....
s:= IB_EditPayment.text; Here I enter 495.10
delete(s,pos('R',s),1);
delete(s,pos('r',s),1);
delete(s,pos('$',s),1);
while pos(' ',s)>0 do
delete(s,pos(' ',s),1);
TempPaymentStr := s;
PostEntry := StrToFloat(TempPaymentStr); // PostEntry is 495.1
I then retrieve TempEntry
TempEntry := Prax.IB_QueryEntry.FieldByName('TransDebit').AsCurrency - Prax.IB_QueryEntry.FieldByName('TransCredit').AsCurrency;
TransDebit value contains 495.10
TransCredit = 0.00 or can be null
When comparing the two supposedly equal values:
if PostEntry > TempTotal then
... it branches here, but both PostEntry, and TempTotal Contains 495.10
else
... and should thus branch here.
This must be some floating point thingy with 495.10000000000000003456 or something like that, throwing me.
How do I circumvent this behaviour.
PS TransCredit and TransDebit defined as Numeric(15.2) ODS 11.1 FB2.1.1.17910-0
[Non-text portions of this message have been removed]
I enter a Cash value in an IB_Edit, then copy it to string.
I then remove any spaces, R or $ signs
code below:
var
PostEntry, TempTotal: Double;
s: String;
....
s:= IB_EditPayment.text; Here I enter 495.10
delete(s,pos('R',s),1);
delete(s,pos('r',s),1);
delete(s,pos('$',s),1);
while pos(' ',s)>0 do
delete(s,pos(' ',s),1);
TempPaymentStr := s;
PostEntry := StrToFloat(TempPaymentStr); // PostEntry is 495.1
I then retrieve TempEntry
TempEntry := Prax.IB_QueryEntry.FieldByName('TransDebit').AsCurrency - Prax.IB_QueryEntry.FieldByName('TransCredit').AsCurrency;
TransDebit value contains 495.10
TransCredit = 0.00 or can be null
When comparing the two supposedly equal values:
if PostEntry > TempTotal then
... it branches here, but both PostEntry, and TempTotal Contains 495.10
else
... and should thus branch here.
This must be some floating point thingy with 495.10000000000000003456 or something like that, throwing me.
How do I circumvent this behaviour.
PS TransCredit and TransDebit defined as Numeric(15.2) ODS 11.1 FB2.1.1.17910-0
[Non-text portions of this message have been removed]