Subject Re: [IBO] Problem with rounding .. any advice.
Author Helen Borrie
At 10:17 PM 11/11/2008, you wrote:
>I have the following situation:
>
>I enter a Cash value in an IB_Edit, then copy it to string.
>I then remove any spaces, R or $ signs
>
>code below:
>var
>PostEntry, TempTotal: Double;
>s: String;
>....
>s:= IB_EditPayment.text; Here I enter 495.10
> delete(s,pos('R',s),1);
> delete(s,pos('r',s),1);
> delete(s,pos('$',s),1);
> while pos(' ',s)>0 do
> delete(s,pos(' ',s),1);
> TempPaymentStr := s;
>
> PostEntry := StrToFloat(TempPaymentStr); // PostEntry is 495.1
>
>
>I then retrieve TempEntry
>TempEntry := Prax.IB_QueryEntry.FieldByName('TransDebit').AsCurrency - Prax.IB_QueryEntry.FieldByName('TransCredit').AsCurrency;
>
>TransDebit value contains 495.10
>TransCredit = 0.00 or can be null
>
>When comparing the two supposedly equal values:
>if PostEntry > TempTotal then
> ... it branches here, but both PostEntry, and TempTotal Contains 495.10
>else
> ... and should thus branch here.
>
>This must be some floating point thingy with 495.10000000000000003456 or something like that, throwing me.
>
>How do I circumvent this behaviour.

..see Martijn's comment...

But I see another problem here. (There seems confusion, though: the var you declare is TempEntry but then you go on to refer to TempTotal..)

Assuming you really mean TempEntry...the trap is that it can be null:

TempEntry := Prax.IB_QueryEntry.FieldByName('TransDebit').AsCurrency - Prax.IB_QueryEntry.FieldByName('TransCredit').AsCurrency;

TransDebit value contains 495.10
TransCredit = 0.00 or can be null <----

When TransCredit is null, TempEntry is null also, unless you explicitly coerce TransCredit to be zero if it is null.


>PS TransCredit and TransDebit defined as Numeric(15.2) ODS 11.1 FB2.1.1.17910-0

That's why you shouldn't cast them as floating point numbers or plug them into floating point variables. Declare the vars as Currency type and use AsCurrency when casting them. Don't forget to flag the CURRENCY attribute for the column in ColumnAttributes.

Helen