--- In firebird-support@yahoogroups.com, Helen Borrie <helebor@...>
wrote:

>
> At 09:15 PM 24/03/2007, you wrote:

Helen,

You are genious, thank you very much. I tought that solution is
how to reduce number of decimals, but I couldn't remember CAST. you
alwasy know write asnwer. How you make it ?

Thanks and Best Regards

Zile

P.S. Thanks also to John for good will to try to help

>
> "The last straw that broke the camel's back". Your total
> 9209924.62677999958 is within the range of 18 digits of precision
> (the limit for a 64-bit integer); when you add one more number to
> it, it is overflowing to 19 digits.
>
> >I would apreciate help from someone how to resolve
> >this. I use FB 2.0, I tried all decimal datatypes (NUMERIC,

DECIMAL,

> >FLOAT, DOUBLEPRECISION) as datatype for result - column VP - no
> >solution.
>
> You have:
>
> I_CENA*I_KOL ----> 4 places of decimal (call this A)
> 1-(F_KASP/100.00) ---> 4 places of decimal (call this B)
> ((100-I_RAB)/100) --> 3 places of decimal (call this C)
>
> then you are doing
>
> A * B * C --> 11 places of decimal, which leaves 7 digits on the

left

> side of the decimal point. You are summing these and, once the

total

> flips up to 10 million, you have run out of precision. so you are
> getting the overflow.
>
> You could save 2 places in the middle part of the calculation by
> making (F_KASP/100.00) into (F_KASP/100). Depending on your

numbers,

> it might be enough to keep your totals in range, or you might need

to

> review the whole calculation and see whether you can free up some
> more precision by casting some of the intermediate numbers as

double

> precision, before finally casting the result as numeric(18.2).
>
> Forget floats, they have at best 7 places accuracy.
>
> ./heLen
>