Subject | Re: 3 * 1/3 = 0 ??? |
---|---|
Author | rogervellacott |
Post date | 2002-08-30T14:24:47Z |
Wow. I was using IB_SQL, but I can't see how the tool can make any
difference. The following also gives 0
CREATE PROCEDURE TEST_PROC
RETURNS ( ANSWER DECIMAL ( 18, 4 ) )
AS
BEGIN
answer = (1/3)*3;
suspend;
END
SELECT * FROM TEST_PROC;
difference. The following also gives 0
CREATE PROCEDURE TEST_PROC
RETURNS ( ANSWER DECIMAL ( 18, 4 ) )
AS
BEGIN
answer = (1/3)*3;
suspend;
END
SELECT * FROM TEST_PROC;
--- In ib-support@y..., "Wilson, Fred" <fred.wilson@b...> wrote:
> Well, you'll be happy to know, that on FB1.0, using QuickDesk
>
> SELECT (1/3)*3 FROM RDB$DATABASE = 1.000
>
> Best regards,
> Fred Wilson
> SE, Bell & Howell
> fred.wilson@m...
>
>
> -----Original Message-----
> From: rogervellacott [mailto:rvellacott@p...]
> Sent: Friday, August 30, 2002 7:07 AM
> To: ib-support@y...
> Subject: [ib-support] 3 * 1/3 = 0 ???
>
>
> I always thought this was true
>
> (1/3)*3 = 1
>
> But Interbase and Firebird (and probably all the other RDBMSs) say
> this equals 0.
>
> Try
>
> SELECT (1/3)*3 FROM RDB$DATABASE
>
> I know there was some discussion of this on this group some time
ago,
> and no doubt some convincing explanations were given, but I am a
> simple soul, and the more I try to cope with it, the more obviously
> wrong it is.
>
> Did you know that
>
> 1/3.00 = 0.33
> and
> 1.00/3 = 0.33
>
> You might think this was a useful way of rounding. But no,
> 3.333333/1.00 = 3.33333300
> so unless you have already determined how many decimal places there
> are in every element of your formula, you cannot know how many
> decimal places there will be in the answer.
>
> In my view, this behaviour invites errors. Not only is it
> unintuitive, it gives the wrong answers to simple calculations, and
> unpredictable answers to complex calculations, and that is not a
good
> idea.
>
> Any chance of changing it next version? If integer operations are
> desired, how about a div? Please don't tell me that current
> behaviour is good. It isn't.
>
>
>
> To unsubscribe from this group, send an email to:
> ib-support-unsubscribe@e...
>
>
>
> Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/