Subject | Re: [IBO] Rounding problems |
---|---|
Author | Svein Erling Tysvær |
Post date | 2001-08-07T09:16:46Z |
Lee, are you doing this in Delphi or through SQL?
// amount-including-vat divided by 100+vat% times 100
// (766.05 / (100 + 21.00)) * 100
// (766.05 / 121) * 100 = 633.0991 the fields are
// numeric(18,2) so it rounds on two decimals and should give
// 633.10 but it gives 633.09 what am I doing wrong ?
If through SQL you need to be careful how you do things. I tried various
selects and this one seemed to work:
SELECT CAST(766.05 / (121.00/100) AS NUMERIC(18,2))
Note that you need two zeroes after 121 to avoid truncation of 1.21 to
simply 1. In Delphi you may have to multiply by 100, round and then divide
by 100 or check the last digits yourself and increase if necessary. Just be
careful to test how it works with large numbers where multiplying by 100
isn't possible without exceeding the 18,2.
Set
// amount-including-vat divided by 100+vat% times 100
// (766.05 / (100 + 21.00)) * 100
// (766.05 / 121) * 100 = 633.0991 the fields are
// numeric(18,2) so it rounds on two decimals and should give
// 633.10 but it gives 633.09 what am I doing wrong ?
If through SQL you need to be careful how you do things. I tried various
selects and this one seemed to work:
SELECT CAST(766.05 / (121.00/100) AS NUMERIC(18,2))
Note that you need two zeroes after 121 to avoid truncation of 1.21 to
simply 1. In Delphi you may have to multiply by 100, round and then divide
by 100 or check the last digits yourself and increase if necessary. Just be
careful to test how it works with large numbers where multiplying by 100
isn't possible without exceeding the 18,2.
Set