Subject Rounding error
Author Gustavo
Hello:

I use Delphi 5, FireBird 2.0 and IBX and I have the following problem.

I have a table named PMOVCC with a field MONTOPEND of type DOUBLE PRECISION and sometimes, when after an UPDATE statement it has to be null, it has a value like 7,105427357601E-15.

I will give an example.

1. In one record of the table, the field MONTOPEND has the value 34.27.

2. I execute the following statement:

UPDATE PMOVCC
SET MONTOPEND = NULLIF(COALESCE(MONTOPEND,0) - 34.27,0)
WHERE (CLAVEMOVIM = 'XXX')

3. Now, the field should be NULL, but it has the value 7,105427357601E-15.

It´s a strange thing because this doesn´t happen always. Sometimes it works good!


My questions are:

1. Is there a way to avoid this rounding error?

2. Is there a way to configure FireBird not to use more than N decimal digits in any calculation? If this coud be done, I could configure not to use more than 8 or 9 digits and then, even if there is a rounding error, the result will be right.

Thanks in advance

Gustavo

[Non-text portions of this message have been removed]