Subject | Wierd math??? |
---|---|

Author | Joe Martinez |

Post date | 2003-02-20T20:27:50Z |

Ok. Take the following query:

select txamount from laytransactions where layid = 5

The result is:

-7.34

7.34

-29.36

-8.09

37.49

-0.04

Now, I run the following query:

select sum(txamount) from laytransactions where layid = 5

The result is:

-8.53483950180589E-16

What the heck? Why isn't it zero???

I am using FB 1.0. The txamount column is a double precision. But,

shouldn't summing a bunch of numbers with exactly 2 decimal places yield a

result with two digits, and not some random very small number???

-Joe

select txamount from laytransactions where layid = 5

The result is:

-7.34

7.34

-29.36

-8.09

37.49

-0.04

Now, I run the following query:

select sum(txamount) from laytransactions where layid = 5

The result is:

-8.53483950180589E-16

What the heck? Why isn't it zero???

I am using FB 1.0. The txamount column is a double precision. But,

shouldn't summing a bunch of numbers with exactly 2 decimal places yield a

result with two digits, and not some random very small number???

-Joe