Subject | Re: [firebird-support] Firebird 2.1 Dialect 3 Decimal Number Limits |
---|---|
Author | Ann W. Harrison |
Post date | 2009-04-01T21:47:02Z |
colincoleman2002 wrote:
decimal digits. So the legal range is not -999,999.99 to
+999,999.99 but -999,999 to +999,999 and the scale factor
is added later by multiplying by 10 to the scale factor
minus one.
Firebird stores fixed precision numbers as binary integers of
various sizes - 2 byte, 4 byte, and 8 byte. Firebird enforces
the size limit of the integer type, not the decimal limit.
A two byte integer can store values from -32768 to +32767
so numbers with a precision of 4 or less are stored in two
byte integers.
Numbers with a precision between 5 and 9 are stored in four
byte integers, which is what you got when you asked for
decimal (6,2). That has a range from -2,147,483,648 to
+2,147,483,647. The value 30,000,000.00 is stored as
3,000,000,000, so it overflows the type and rolls over into
a negative number.
Numbers up to 18 digits are stored in 8 byte integers, and
they get pretty big. Back when InterBase started, it would
hold the gross national debt (of the US) expressed in milli-Lira.
Since then the national debt has grown and Italy has switched
to the euro.
Cheers,
Ann
> Hi ,The 6 in decimal(6,2) is the precision. It includes the
>
> I thought I new this but im am stummped, I have a Domain
> that we have defined as Decimal(6,2) We have launched
> our project and started importing data into the fields...
> all goes well until we find that our imported figure of
> 30,000,000.00 is converted into -12,949,672.96
decimal digits. So the legal range is not -999,999.99 to
+999,999.99 but -999,999 to +999,999 and the scale factor
is added later by multiplying by 10 to the scale factor
minus one.
Firebird stores fixed precision numbers as binary integers of
various sizes - 2 byte, 4 byte, and 8 byte. Firebird enforces
the size limit of the integer type, not the decimal limit.
A two byte integer can store values from -32768 to +32767
so numbers with a precision of 4 or less are stored in two
byte integers.
Numbers with a precision between 5 and 9 are stored in four
byte integers, which is what you got when you asked for
decimal (6,2). That has a range from -2,147,483,648 to
+2,147,483,647. The value 30,000,000.00 is stored as
3,000,000,000, so it overflows the type and rolls over into
a negative number.
Numbers up to 18 digits are stored in 8 byte integers, and
they get pretty big. Back when InterBase started, it would
hold the gross national debt (of the US) expressed in milli-Lira.
Since then the national debt has grown and Italy has switched
to the euro.
Cheers,
Ann