Subject | IB *NOT* divisionally impaired - final |
---|---|
Author | csswa |
Post date | 2002-05-31T02:43:37Z |
To clarify the situation and put it to bed:
My original SP was dividing two integers which resulted in a zero
value because the result was less than one.
CASTing each integer involved seemed to work, which led me to believe
that I would have to cast all integers involved in the calc, which I
saw as longwinded and crazy. Hence subsequent rant.
After much discussion here, it turns out that there is a discrepancy
(bug???) in results returned by dialect 1 and dialect 3 dbs. The
same calc I was using in dialect 3 *would* have returned the correct
floating point answer in dialect 1, e.g.
select (2 / 5) from rdb$database;
dialect 1 result: 0.400
dialect 3 result: 0
I did not realize this earlier because my testing was inadvertantly
using integer division that was always giving a <1 result, hence I
was constantly getting zeros from that bit of the calc.
So let me state loudly and emphatically, IB is not arithmetically
challenged! This 'result<1' bug/discrepancy was causing my grief,
not any overall IB design problem... or at least I hope this is
correct.
What remain now is to hear if these results from different dialects
is expected behaviour or not...
I've chased my tail enough on this :-) so I'll give you all a rest.
Have a great weekend, all.
Regards,
Andrew Ferguson
-- All this, and with brains the size of a pea.
My original SP was dividing two integers which resulted in a zero
value because the result was less than one.
CASTing each integer involved seemed to work, which led me to believe
that I would have to cast all integers involved in the calc, which I
saw as longwinded and crazy. Hence subsequent rant.
After much discussion here, it turns out that there is a discrepancy
(bug???) in results returned by dialect 1 and dialect 3 dbs. The
same calc I was using in dialect 3 *would* have returned the correct
floating point answer in dialect 1, e.g.
select (2 / 5) from rdb$database;
dialect 1 result: 0.400
dialect 3 result: 0
I did not realize this earlier because my testing was inadvertantly
using integer division that was always giving a <1 result, hence I
was constantly getting zeros from that bit of the calc.
So let me state loudly and emphatically, IB is not arithmetically
challenged! This 'result<1' bug/discrepancy was causing my grief,
not any overall IB design problem... or at least I hope this is
correct.
What remain now is to hear if these results from different dialects
is expected behaviour or not...
I've chased my tail enough on this :-) so I'll give you all a rest.
Have a great weekend, all.
Regards,
Andrew Ferguson
-- All this, and with brains the size of a pea.