Subject | Re: [firebird-support] numeric overflow with constants? |
---|---|
Author | Richard Wesley |
Post date | 2006-06-02T20:22:55Z |
On Jun 1, 2006, at 15:28, Alan McDonald wrote:
select CAST(0.65000000000000002 AS double precision) / CAST
(0.34999999999999998
AS double precision) from rdb$database;
I guess my question is "Why?" Or more precisely, "What is the the
implied type of these constants without casting?"
________________________________________________________
Richard Wesley Software Engineer
Tableau Software
http://www.tableausoftware.com/ hawkfish
tableausoftware com
>> Why does the following generate a numeric overflow, and how do IThe solution we came up with is similar:
>> prevent it from doing so?
>>
>> select 0.65000000000000002 / 0.34999999999999998 from rdb$database;
>
> select CAST(0.65000000000000002 AS NUMERIC(18,2)) / CAST
> (0.34999999999999998
> AS NUMERIC(18,2)) from rdb$database;
select CAST(0.65000000000000002 AS double precision) / CAST
(0.34999999999999998
AS double precision) from rdb$database;
I guess my question is "Why?" Or more precisely, "What is the the
implied type of these constants without casting?"
________________________________________________________
Richard Wesley Software Engineer
Tableau Software
http://www.tableausoftware.com/ hawkfish
tableausoftware com