Subject Re: [ib-support] Really strange calculating behaviour of IB
Author Geoff Worboys
> The values the database stores don't have to be scaled. You can do
> the scaling yourself using your external application or UDFs.

This idea actually demonstrates one work around for the present
problem. Again using the second demo that was posted, the following
gives the correct response...

CAST( (
( (CAST((money3 * 100000) as NUMERIC(18,0)) * 100000)
/ CAST((1.95583* 100000) as NUMERIC(18,0))
) ) AS NUMERIC(18,5) ) / 100000
as money3c

Of course the above work-around is of only limited value, since it
limits the integral value to 8 digits (99 million) - after which
overflow conditions will occur.


Note: If you change the divisor to 1.95582 as in the example below...

CAST( (
( (CAST((money3 * 100000) as NUMERIC(18,0)) * 100000)
/ CAST((1.95582* 100000) as NUMERIC(18,0))
) ) AS NUMERIC(18,5) ) / 100000
as money3c

and compare the result with...

(money1/1.95582) as money1a,

You will see that the first calculation results in truncation (and not
rounding) at the 5th decimal place. This is strictly correct, since
we are technically manipulating integers, but may not be what is
expected.


--
Geoff Worboys
Telesis Computing