Subject | Re: IB integer division issue |
---|---|
Author | csswa |
Post date | 2002-05-31T01:58:25Z |
It appears that dialect 3 rounds down or zeros any integer division
when the result is less than one:
select (300 / 10) * 3.333 from rdb$database;
dialect 1 result: 99.990
dialect 3 result: 99.990
select (3 / 10) * 3.333 from rdb$database;
dialect 1 result: 1.000
dialect 3 result: 0.000
Why the behaviour difference between dialects???
Regards
Andrew Ferguson
-- A is for Andrew. B is for Andrew. C is for Andrew . . . You get
the point.
when the result is less than one:
select (300 / 10) * 3.333 from rdb$database;
dialect 1 result: 99.990
dialect 3 result: 99.990
select (3 / 10) * 3.333 from rdb$database;
dialect 1 result: 1.000
dialect 3 result: 0.000
Why the behaviour difference between dialects???
Regards
Andrew Ferguson
-- A is for Andrew. B is for Andrew. C is for Andrew . . . You get
the point.
--- In ib-support@y..., "Wilson, Fred" <fred.wilson@b...> wrote:
> BTW, for what it's worth, under FB1.0 (W2000), dialect 1 database
>
> select (3 / 10) * 3.333 from rdb$database;
> ======================================
> 1.000
>
>
> Best,
> Fred Wilson
> SE, Bell & Howell
> fred.wilson@b...