Subject | Re: [IBO] Numeric Fields |
---|---|
Author | Geoff Worboys |
Post date | 2001-03-03T02:15:58Z |
Just to clarify...
My previous example of the IB6 dialect 1 bug was incorrect. The
original example was...
In IB6 dialect 1:
select cast(3500 as numeric(9,2)) * cast(1 as numeric(9,4))
from rdb$database
gives: -794,967296
AFAICT this is because...
(3500.00 * 1.0000) = 3500.000000
The result requires 10 digits which will not fit into a 32bit integer.
In IB5.6 the calculation would have resulted in an overflow if
assigned to numeric(9,5) or lower anyway - but could be successfully
cast to such a value. Whereas in IB6 the invalid value still results.
My apologies for the incorrect info before. It is unlikely that this
has anything to do with the original problem, but having stated the
bug incorrectly to begin with I thought I should post the correction.
Geoff Worboys
Telesis Computing
My previous example of the IB6 dialect 1 bug was incorrect. The
original example was...
In IB6 dialect 1:
select cast(3500 as numeric(9,2)) * cast(1 as numeric(9,4))
from rdb$database
gives: -794,967296
AFAICT this is because...
(3500.00 * 1.0000) = 3500.000000
The result requires 10 digits which will not fit into a 32bit integer.
In IB5.6 the calculation would have resulted in an overflow if
assigned to numeric(9,5) or lower anyway - but could be successfully
cast to such a value. Whereas in IB6 the invalid value still results.
My apologies for the incorrect info before. It is unlikely that this
has anything to do with the original problem, but having stated the
bug incorrectly to begin with I thought I should post the correction.
Geoff Worboys
Telesis Computing