Subject Re: [firebird-support] column precision
Author Helen Borrie
At 08:58 PM 12/04/2004 -0600, you wrote:
>I have several columns that I am trying to hold their precisions to
>xxx.x. I originally had the column's types set as floats but kept
>getting 0.xxxxxxxxx. I read that if you change the type to
>numeric(9,2) you will get xxxxxxxxx.xx.

No, you will get xxxxxxx.xx. 9 is the precision (exact number of
significant figures) and 2 is the scale (says effectively how many of the
rightmost numbers are decimal parts). If you want a fixed decimal number
with 11 digits, then you need to define a numeric(11,2).

The version and dialect of the database matter. It needs to be a dialect 3
database AND the column defined as numeric(m,n) *since* you converted to
dialect 3. Old dialect 1 fixed decimal definitions keep their old geometry.

> I use sql explorer to view
>the table setup and when I chage the type to numeric(9,2) and then
>apply the changes, it changes to the type integer(4,1).

This isn't saying much at all. You would need to inspect these values with
a tool that is capable of recognising fixed numeric types. That again
depends on database version and dialect; but there is no version of the
BDE that properly represents dialect 3 data types and SQLE was never any
good at dealing with fixed numerics.

> This
>effectivly rounds all of my results to the nearest integer x. I am
>not sure what else to try????

The conversion from FLOAT to NUMERIC(9,2) would not be very effective,
since FLOAT has insufficient precision to be represented as 9 digits fixed
precision. If you are really seeing only 5 digits of precision in the
converted data and you expected more, then therein lies the cause. Did you
really have numbers that were of precision 9, that you stored in FLOAT? Or
was precision 5 adequate for your actual data? If you haven't actually
lost precision by storing them in FLOAT, you may be OK.

First do a reality check, abandon BDE tools and use isql or another
tool--such as ib_sql, free from www.ibobjects.com, --to inspect your
data. Also, with one of these tools, query the relevant system tables
(RDB$RELATIONS and RDB$RELATION_FIELDS) to look at the definitions that are
actually stored. (It's easier with IB_SQL - click Browse on the main
Connection tab, and check System Inf on the Relations page of the data browser.

/heLen