Subject | Re: The integer part of a number |
---|---|
Author | Adam |
Post date | 2006-12-03T03:28:54Z |
--- In firebird-support@yahoogroups.com, "mauriciobornacin"
<mauriciobornacin@...> wrote:
Under the SQL standard, integer / integer the truncated division as an
integer. So:
24 / 60 = 0.4 (really) = 0 (according to standard).
You can see this by running:
select 24/60 from RDB$DATABASE;
According to SQL standard, adding an integer to a numeric returns a
numeric, so
0 + 0.495 = 0.495
You can see this by running:
select 0+0.495 from RDB$DATABASE;
Converting from a numeric to an integer rounds the value (pretty sure
that is part of the standard too).
You can see this by running:
select cast(0.499 as Integer) from RDB$DATABASE;
select cast(0.5 as Integer) from RDB$DATABASE;
Combining these three facts shows why you get the result you get.
case, either explicitly cast the numerator or denominator as some sort
of numeric, or implicitly cast it by adding .00000 etc.
Adam
<mauriciobornacin@...> wrote:
>Yes.
> Can anybody explain why
> select cast(((24/60) + 0.495)as integer ) from table;
> equals 0 and
> select cast(((24/60) + 0.5)as integer ) from table;
> equals 1
Under the SQL standard, integer / integer the truncated division as an
integer. So:
24 / 60 = 0.4 (really) = 0 (according to standard).
You can see this by running:
select 24/60 from RDB$DATABASE;
According to SQL standard, adding an integer to a numeric returns a
numeric, so
0 + 0.495 = 0.495
You can see this by running:
select 0+0.495 from RDB$DATABASE;
Converting from a numeric to an integer rounds the value (pretty sure
that is part of the standard too).
You can see this by running:
select cast(0.499 as Integer) from RDB$DATABASE;
select cast(0.5 as Integer) from RDB$DATABASE;
Combining these three facts shows why you get the result you get.
>My guess is that you are not expecting 24/60 to return 0. In this
> I'm trying to round on minutes and make a phone rate calculation
>
case, either explicitly cast the numerator or denominator as some sort
of numeric, or implicitly cast it by adding .00000 etc.
Adam