Subject | Re: [ib-support] Stored proc refuses to divide two numbers! |
---|---|

Author | Martijn Tonies |

Post date | 2002-05-28T17:10Z |

Hi,

integers, the formula returns 0 (0.3 rounded).

Read some more on numerics in dialect 3...

You can solve it by (not tested):

ret_result = cast(100 as double precision) / cast(300 as double precision);

Martijn Tonies

InterBase Workbench - the developer tool for InterBase and Firebird

http://www.interbaseworkbench.com

Upscene Productions

http://www.upscene.com

"This is an object-oriented system.

If we change anything, the users object."

> I have the following procedure. It takes two integers, divides them,100 / 300 probably returns 0, right? Because both 100 and 300 are

> returns a float:

>

> CREATE PROCEDURE SP_SAMPLES_PER_QUARTER (

> PAR_DAYS INTEGER,

> PAR_SAMPLE_COUNT INTEGER)

> RETURNS (

> RET_RESULT DOUBLE PRECISION)

> AS

> begin

> --1-- RET_RESULT = 100 / 300;

> --2-- RET_RESULT = CAST((100 / 300) AS DOUBLE PRECISION);

> --3-- RET_RESULT = 0.123;

> --4-- RET_RESULT = 1 + 1;

> EXIT;

> end

>

> The two test lines (3 and 4) correctly return their values. Lines 1

> and 2 return 0. What am I missing here? Do I need to cast

> something??

integers, the formula returns 0 (0.3 rounded).

Read some more on numerics in dialect 3...

You can solve it by (not tested):

ret_result = cast(100 as double precision) / cast(300 as double precision);

Martijn Tonies

InterBase Workbench - the developer tool for InterBase and Firebird

http://www.interbaseworkbench.com

Upscene Productions

http://www.upscene.com

"This is an object-oriented system.

If we change anything, the users object."