Subject | problem with round |
---|---|
Author | Helmut Steinberger |
Post date | 2004-03-08T10:05:48Z |
in the before update trigger of a table I set a field of the type
numeric (10, 2) depending on the value of another numeric field.
I need it rounded to 1 or 2 decimals depending on another field.
Here is a sample of the trigger:
if (new.num_decimals = 0) then
new.value1 := round (new.value2);
else
if (new.num_decimals = 1) then
new.value1 = round (new.value2 * 10) / 10;
else
new.value1 = round (new.value2 * 100) / 100;
If I do so, in some cases new.value1 is set to 0, allthough it must
be a value other than 0.
If I change the trigger to:
if (new.num_decimals = 0) then
new.value1 := round (new.value2);
else
if (new.num_decimals = 1) then
new.value1 = round (new.value2 * 10) / 10.0;
else
new.value1 = round (new.value2 * 100) / 100.0;
it works as expected.
Why is there a difference between dividing through 100 and 100.0?
There should be the same result in both cases, but there isn't.
For testing just use 0.58 for new.value2 and 1 for new.num_decimals.
Helmut
numeric (10, 2) depending on the value of another numeric field.
I need it rounded to 1 or 2 decimals depending on another field.
Here is a sample of the trigger:
if (new.num_decimals = 0) then
new.value1 := round (new.value2);
else
if (new.num_decimals = 1) then
new.value1 = round (new.value2 * 10) / 10;
else
new.value1 = round (new.value2 * 100) / 100;
If I do so, in some cases new.value1 is set to 0, allthough it must
be a value other than 0.
If I change the trigger to:
if (new.num_decimals = 0) then
new.value1 := round (new.value2);
else
if (new.num_decimals = 1) then
new.value1 = round (new.value2 * 10) / 10.0;
else
new.value1 = round (new.value2 * 100) / 100.0;
it works as expected.
Why is there a difference between dividing through 100 and 100.0?
There should be the same result in both cases, but there isn't.
For testing just use 0.58 for new.value2 and 1 for new.num_decimals.
Helmut