Subject Re: [firebird-support] 32 bit vs 64 bit arthmetic
Author Tim Ward
On 18/01/2016 17:34, 'Leyne, Sean' Sean@... [firebird-support] wrote:


> So I've got this stored procedure which takes DECIMAL(18,6) parameters and
> returns DECIMAL(18,12) results, and internally copies stuff into and out of
> DOUBLE PRECISION variables which it uses to call a UDF to do the actual
> calculations.
> And the results are different (in the 8th decimal place or so) between
> running on 32 bit and 64 bit systems.

Please post the logic for the SP and a reproducible example, this may be worthy of a bug tracker ticket.

I have confirmed that the versions of the underlying library (GDAL) are different on the two systems, and for that and other reasons that the best use of my time at the moment is doing other things, so I will be assuming that it's the change to the GDAL code that resulted in the slightly different output and will not be researching it any further.

Tim Ward