Subject | Re: [firebird-support] DSQL, API and Numeric DataTypes |
---|---|

Author | Ann W. Harrison |

Post date | 2006-05-15T14:40:02Z |

Alan McDonald wrote:

a separate scale factor as part of the definition of the field.

Obviously there's no way to represent 123.456 as an integer.

When performing comparisons, Firebird adjusts for the scale

factor so 123456 (9,4) evaluates as less than 123456 (9,3), even

though the stored value is the same. If your application gets

the value as an integer with a scale factor, it must make the

same adjustment - some languages have support for fixed point

scaled numbers, some don't. C, for example, doesn't, so in

C, your application must distinguish between 123456 (9,4) and

123456 (9,3).

However, when you send the value back to the database, using

an integer type and a declare scale, you need to send back the

value as an integer. If your application converts 123.456 in

whatever type it uses to an integer, the value you get is the

integral part - 123. When Firebird gets that value it applies

the scale factor of 3, and gets .123 - not at all what you wanted.

So when reading scaled integers from Firebird, multiply by ten

times the scale factor times minus 1 to get the intended value.

When passing scaled numbers back to Firebird, multiply by ten

time the scale factor.

Does this make sense?

Regards,

Ann

> In the API guide, under "Handling Numeric and Decimal Datatypes", itFirebird stores scaled numbers up to 18 digits as integers with

> discusses the requirement of adjusting retrieved values for the scaling

> factor.

a separate scale factor as part of the definition of the field.

Obviously there's no way to represent 123.456 as an integer.

When performing comparisons, Firebird adjusts for the scale

factor so 123456 (9,4) evaluates as less than 123456 (9,3), even

though the stored value is the same. If your application gets

the value as an integer with a scale factor, it must make the

same adjustment - some languages have support for fixed point

scaled numbers, some don't. C, for example, doesn't, so in

C, your application must distinguish between 123456 (9,4) and

123456 (9,3).

However, when you send the value back to the database, using

an integer type and a declare scale, you need to send back the

value as an integer. If your application converts 123.456 in

whatever type it uses to an integer, the value you get is the

integral part - 123. When Firebird gets that value it applies

the scale factor of 3, and gets .123 - not at all what you wanted.

So when reading scaled integers from Firebird, multiply by ten

times the scale factor times minus 1 to get the intended value.

When passing scaled numbers back to Firebird, multiply by ten

time the scale factor.

Does this make sense?

Regards,

Ann