Subject Re: [ib-support] Wierd math???
Author Nando Dessena
Joe,

JM> -7.34
JM> 7.34
JM> -29.36
JM> -8.09
JM> 37.49
JM> -0.04

JM> select sum(txamount) from laytransactions where layid = 5

JM> -8.53483950180589E-16

JM> What the heck? Why isn't it zero???

In a sense, it is. :-)

JM> I am using FB 1.0. The txamount column is a double precision.

That's the source of your troubles. Double precision is not, err,
"precise".

JM> But, shouldn't summing a bunch of numbers with exactly 2 decimal places yield a
JM> result with two digits, and not some random very small number???

It should, but since you are using floating point numbers your numbers
do not have "exactly" 2 decimal places. Actually, their decimal point
floats. Search the archives for thorough discussions about this
problem.
Ciao
--
Nando mailto:nandod@...