Subject Re: [IBO] Rounding problems
Author Geoff Worboys
> //to calculate the amount exclusive VAT :
> // amount-including-vat divided by 100+vat% times 100
> // (766.05 / (100 + 21.00)) * 100
> // (766.05 / 121) * 100 = 633.0991 the fields are
> // numeric(18,2) so it rounds on two decimals and should give
> // 633.10 but it gives 633.09 what am I doing wrong ?

Multiply first, divide last.

From the Borland online help about currency...

- - - -
Currency is a fixed-point data type that minimizes rounding errors in
monetary calculations. It is stored as a scaled 64-bit integer with
the four least-significant digits implicitly representing decimal
places. When mixed with other real types in assignments and
expressions, Currency values are automatically divided or multiplied
by 10000.
- - - -

Note that it is based on an integer. That is:
(5 div 2) = 2
NOT 3 (which would be the result if it was rounded).


So by doing the divide first you end up with:

(766.05 / (100 + 21.00)) * 100
=> (6.3309 * 100) // 6.3309917355 was truncated to 4 places
=> 633.0900
=> 633.09 // after rounding to 2 places


If you multiply out first you get...

(766.05 / (100 + 21.00)) * 100
=> (76605 / 121)
=> (633.0991) // 633.0991735537 was truncated to 4 places
=> 633.10 // after rounding to 2 places



You alternative to this is to perform the calculates with doubles and
then convert back to numerics (currency or whatever). This can often
reduce the confusion when dealing with division, percentages etc.


HTH

Geoff Worboys
Telesis Computing