Subject | Re: [firebird-support] Rounding large numbers |
---|---|
Author | jft |
Post date | 2008-08-08T11:49:06Z |
Marrtins,
I'm not exactly sure what you want to do.
If you want to take 1482835105.67891 and round it to 4 digits giving 1482835105.6789,
or take 1482835105.67896 and round it to 4 digits giving 1482835105.6790 then this may help.
These iSQL results are from a 2.0.0 server:
First - to duplicate your existing column definitions of DOUBLE PRECISION:
SQL> select cast(1482835105.67891 as double precision) from rdb$database;
CAST
=======================
1482835105.678910
SQL> select cast(1482835105.67896 as double precision) from rdb$database;
CAST
=======================
1482835105.678960
Second - try casting them as decimal(18,4) (which incidently seems to have inbuilt rounding!):
SQL> select cast(cast(1482835105.67891 as double precision) as decimal(18,4)) from rdb$database;
CAST
=====================
1482835105.6789
SQL> select cast(cast(1482835105.67896 as double precision) as decimal(18,4)) from rdb$database;
CAST
=====================
1482835105.6790
If this is not what you are after, can you give us a specific example please?
HTH,
John
I'm not exactly sure what you want to do.
If you want to take 1482835105.67891 and round it to 4 digits giving 1482835105.6789,
or take 1482835105.67896 and round it to 4 digits giving 1482835105.6790 then this may help.
These iSQL results are from a 2.0.0 server:
First - to duplicate your existing column definitions of DOUBLE PRECISION:
SQL> select cast(1482835105.67891 as double precision) from rdb$database;
CAST
=======================
1482835105.678910
SQL> select cast(1482835105.67896 as double precision) from rdb$database;
CAST
=======================
1482835105.678960
Second - try casting them as decimal(18,4) (which incidently seems to have inbuilt rounding!):
SQL> select cast(cast(1482835105.67891 as double precision) as decimal(18,4)) from rdb$database;
CAST
=====================
1482835105.6789
SQL> select cast(cast(1482835105.67896 as double precision) as decimal(18,4)) from rdb$database;
CAST
=====================
1482835105.6790
If this is not what you are after, can you give us a specific example please?
HTH,
John
> -------Original Message-------
> From: marrtins_dqdp <marrtins@...>
> Subject: [firebird-support] Rounding large numbers
> Sent: 08 Aug '08 20:38
>
> Hello!
>
> I have few servers with Firebird versions. On some servers live
> version 2.0.4 on others 2.1.0
>
> I need to round one column with precision 4 digits. It is done via
> trigger. Column is defined as DOUBLE PRECISION
>
> On 2.1.0 I do
> new.TOTAL = ROUND(new.TOTAL, 4);
>
> On 2.0.4 I do
> new.TOTAL = ROUND(CAST(new.TOTAL * 10000 AS INTEGER)) / 10000;
>
> Both worked fine till today new.TOTAL hit 1482835105.6 in one row,
> which gave me 0.0000 on v2.1.0 and `arithmetic exception, numeric
> overflow, or string truncation` on v2.0.4
>
> Someone may test it
> v2.1.0
> SELECT ROUND(1482835105.6, 4)
> FROM RDB$DATABASE
>
> v.2.0.4
> SELECT ROUND(CAST(1482835105.6 * 10000 AS INTEGER)) / 10000
> FROM RDB$DATABASE
>
> My question is - how to round such large numbers?
>
> Thank you!