Subject | Re: op between different datatype |
---|---|
Author | Ali Gökçen |
Post date | 2005-05-03T09:27:26Z |
Hi Alberto,
it is not about Firebird or SQL.
It is about mathmatic science.
if the numbers are integer:
operations
'+', '-' and '*' are give exact result with overflow possibility.
'/' usually gives trunced result if X and Y both are not even numers
on X/Y.
if the numers are reel (dotted):
operations
'+', '-' are give exact result(W.O.Pos.). X-Y is same as X+(-Y) here.
'/' and '*' usally give trunced result. 100.0/10.0 same as 100.0*0.1
numeric(15,3) * numeric(15,4) gives numeric(15,7) here.
As you seen sharing(dviding) creates problems everywhere.
So, you must be careful about what you do. You should compute
your formules before computer with extrem values.
if you can accept some losts, then you can use double precision in
your problem.
after calculations you can store the result as (15,4) or whatever
you want with precision lost.
you can convert field types using CAST function at run time.
(look at the referance pdf )
Ali
--- In firebird-support@yahoogroups.com, "Alberto Pesce"
<palberto@l...> wrote:
it is not about Firebird or SQL.
It is about mathmatic science.
if the numbers are integer:
operations
'+', '-' and '*' are give exact result with overflow possibility.
'/' usually gives trunced result if X and Y both are not even numers
on X/Y.
if the numers are reel (dotted):
operations
'+', '-' are give exact result(W.O.Pos.). X-Y is same as X+(-Y) here.
'/' and '*' usally give trunced result. 100.0/10.0 same as 100.0*0.1
numeric(15,3) * numeric(15,4) gives numeric(15,7) here.
As you seen sharing(dviding) creates problems everywhere.
So, you must be careful about what you do. You should compute
your formules before computer with extrem values.
if you can accept some losts, then you can use double precision in
your problem.
after calculations you can store the result as (15,4) or whatever
you want with precision lost.
you can convert field types using CAST function at run time.
(look at the referance pdf )
Ali
--- In firebird-support@yahoogroups.com, "Alberto Pesce"
<palberto@l...> wrote:
> Hi all,0)/100))))
> In a store procedure I do :
>
> SELECT SUM(someTable.X * (someTable.Y * (1-(coalesce(someTable.Z,
> FROM someTablepreclude successful execution of subsequent statements.
> WHERE someTable.W = :CODICE
> INTO :A;
>
>
> note :
> A is a numeric(15,5)
> someTable.X is numeric(15,5)
> someTable.Y is a numeric (15,3)
> someTable.Z is a numeric(5,2)
>
> I get:
>
> Unsuccessful execution caused by system error that does not
> Integer overflow. The result of an integer operation caused themost significant bit of the result to carry.
>
> How can I solve this whitout change the datatype of X,Y, Z, A
> Thanks!
> Alberto
>
> [Non-text portions of this message have been removed]