Subject | Wierd math??? |
---|---|
Author | Joe Martinez |
Post date | 2003-02-20T20:27:50Z |
Ok. Take the following query:
select txamount from laytransactions where layid = 5
The result is:
-7.34
7.34
-29.36
-8.09
37.49
-0.04
Now, I run the following query:
select sum(txamount) from laytransactions where layid = 5
The result is:
-8.53483950180589E-16
What the heck? Why isn't it zero???
I am using FB 1.0. The txamount column is a double precision. But,
shouldn't summing a bunch of numbers with exactly 2 decimal places yield a
result with two digits, and not some random very small number???
-Joe
select txamount from laytransactions where layid = 5
The result is:
-7.34
7.34
-29.36
-8.09
37.49
-0.04
Now, I run the following query:
select sum(txamount) from laytransactions where layid = 5
The result is:
-8.53483950180589E-16
What the heck? Why isn't it zero???
I am using FB 1.0. The txamount column is a double precision. But,
shouldn't summing a bunch of numbers with exactly 2 decimal places yield a
result with two digits, and not some random very small number???
-Joe