Subject | Re: [ib-support] Stored proc refuses to divide two numbers! |
---|---|
Author | Martijn Tonies |
Post date | 2002-05-28T17:10Z |
Hi,
integers, the formula returns 0 (0.3 rounded).
Read some more on numerics in dialect 3...
You can solve it by (not tested):
ret_result = cast(100 as double precision) / cast(300 as double precision);
Martijn Tonies
InterBase Workbench - the developer tool for InterBase and Firebird
http://www.interbaseworkbench.com
Upscene Productions
http://www.upscene.com
"This is an object-oriented system.
If we change anything, the users object."
> I have the following procedure. It takes two integers, divides them,100 / 300 probably returns 0, right? Because both 100 and 300 are
> returns a float:
>
> CREATE PROCEDURE SP_SAMPLES_PER_QUARTER (
> PAR_DAYS INTEGER,
> PAR_SAMPLE_COUNT INTEGER)
> RETURNS (
> RET_RESULT DOUBLE PRECISION)
> AS
> begin
> --1-- RET_RESULT = 100 / 300;
> --2-- RET_RESULT = CAST((100 / 300) AS DOUBLE PRECISION);
> --3-- RET_RESULT = 0.123;
> --4-- RET_RESULT = 1 + 1;
> EXIT;
> end
>
> The two test lines (3 and 4) correctly return their values. Lines 1
> and 2 return 0. What am I missing here? Do I need to cast
> something??
integers, the formula returns 0 (0.3 rounded).
Read some more on numerics in dialect 3...
You can solve it by (not tested):
ret_result = cast(100 as double precision) / cast(300 as double precision);
Martijn Tonies
InterBase Workbench - the developer tool for InterBase and Firebird
http://www.interbaseworkbench.com
Upscene Productions
http://www.upscene.com
"This is an object-oriented system.
If we change anything, the users object."