> Greetings,
>
> I've got a CFO waiting for me....
>
> Not getting along well with the changes in FB 1.03 regarding simple
> arithmetic.
>
> Division is killing me. Whoever thought/thinks dividing one integer into
> another gives an integer result has got to be nuts (ok, maybe
> it's me that's
> nuts :)
>
> 2 things:
>
> 1) trying to divide a simple percetage by 100:
>
> pct = 33.33 / 100
>
> where 33.33 is a varibale defined as: numeric(5,2). 100 is constant.
> pct is a varibale defined as: numeric(6,5). (I've tried
> different things here).
>
> I cannot get it into a variable as .3333. Comes out as
> .333299994468689
>
>
> 2) then I want to take that pct (once it's correct) and multiply that by a
> currency
> 391.06. I get an integer overflow on this one!
>
> Complete proc follows:
>
> create procedure sp_krs_test_b
> as
> declare variable bpct numeric(5,2);
> declare variable spct numeric(5,2);
>
> declare variable pct numeric(6,5);
>
> /*declare variable bmargin numeric(18,2); tried this w/ no luck
> so now I'm
> just guessing below */
> declare variable bmargin double precision;
>
> begin
> /* Procedure Text */
> Execute Procedure sp_Get_Tdr_Comp_Pcts_N ('01', '99')
> Returning_Values :bpct, :spct;
>
> pct = bpct / cast(100 as numeric(6,2));
>
> bmargin = (391.06 * pct);
>
> suspend;
> end
>
>
>
> Any help will be GREATLY appreciated.
>
> Kevin

Some people might recommend a different way but in these types of calcs I
always use double precision for the raw numbers and then I use the UDF fixed
point/rounding/truncating or f_dollar value functions in conjunction with
CAST to decimal calls.
The FIXED point function will always give you exactly the decimal places you
want (as string), the cast will finally turn it back to a number.
e.g. here's a volume sum in cubic metres to three places
SUM(CAST(F_FIXEDPOINT(S."LENGTH"/1000*S.WIDTH/1000*S.HEIGHT/1000,3) AS
DECIMAL(9,3)))
the dimensions are stored as FLOAT but are millimeters.
As long as you are consistent you have no problems.
Alan