Subject | Re: Standard Deviation calculation using select query ? |
---|---|
Author | Ali Gökçen |
Post date | 2005-07-14T11:41:46Z |
Hi Mario,
REAL is NOT DOUBLE PRECISION(8 bytes float),
it is single precision (4 bytes float).
you are doing 2 selects.
if your db on heavy load(too much users and inserts), then there may
be some commits between these 2 selects, and you may get some
incorrect stddev result.
(if your transaction is not in snapshot isolation level)
use snaphot transaction for this operation or
use single select operation as i write in previous mail or
eliminate the usage of new records via ID filtering or
etc etc..
Regards.
Ali
> I tried to declare the code below within the SP code but that didnot
> work.did you send DOUBLE PRECISION to sqrt?
> DECLARE EXTERNAL FUNCTION SQRT
> DOUBLE PRECISION
> RETURNS DOUBLE PRECISION BY VALUE
> ENTRY_POINT 'IB_UDF_sqrt' MODULE_NAME 'ib_udf';
> At the end, I had to apply Delphi SQRT built-in function to the SP
> output variable.
>
REAL is NOT DOUBLE PRECISION(8 bytes float),
it is single precision (4 bytes float).
> select avg(xindex), count(xindex) from xtablebe careful here please,
> where xDate > :FirstDate
> into :avgindex,:countindex;
> if (:countindex > 1) then
> select (sum((xindex - :avgindex)*(xindex - :avgindex)) /
> (:countindex - 1)) from xtable
> where xDate > :FirstDate
> into :stddev;
you are doing 2 selects.
if your db on heavy load(too much users and inserts), then there may
be some commits between these 2 selects, and you may get some
incorrect stddev result.
(if your transaction is not in snapshot isolation level)
use snaphot transaction for this operation or
use single select operation as i write in previous mail or
eliminate the usage of new records via ID filtering or
etc etc..
Regards.
Ali