Subject Re: [ib-support] character sets and arrays and blobs
Author Geoff Worboys
> >To transfer normal text fields without problems I have
> >to resort to BLR so that I can specify the character
> >set id for each field (blr_text2 etc). The standard
> >API mechanisms assume the connection character set
> >leading to possible translation problems in various
> >situations (particularly with character set NONE
> >fields).
> That seems vaguely unlikely since it would make a
> database with multiple character sets unusable from SQL...

That was what I thought, but a search of the API reference for
"character set" does not show any means of specifying alternative
character sets for non-blob text fields. So to read/write text fields
from a connection the only character set control you appear to have is
via the connection itself (and of course the field definition itself).

If you setup the connection using character set NONE, you can read
fields with any character set. But you cannot reliably write to a
text field with a character set other than NONE. ("Transliteration"
errors result if the data contains high ASCII chars.)

If you setup the connection using some other character set (presumably
the most commonly defined or database default character set), then you
are unable to reliably read text from character set NONE text fields.

I am presuming that applications using multiple character sets get by
because IB/FB will (hopefully reliably) translate between character
sets automatically. Except that character set NONE cannot be written
to other character sets.

If the database contains character set none fields the only way to
reliably read/write those fields is using a connection with character
set none, but you cannot use that connection for your other character
set fields.

I seem to be going around in circles but you probably get the gist :-)

There are always character set NONE fields in a database (in the
metadata), so there must be a way around this problem, and that was
what sent me looking at burp to see how gbak did it. There I found
out about blr_text2, blr_cstring2 and blr_varying2 that support
character set specification. I have yet to finish the code I am
writing to try these out, but they appear offer the only means of
truly using multiple character sets over a single

> >Is anyone able to tell me whether it always OK to
> >simply read/write blobs without application of
> >matching character set filters?
> Yes.

This is convenient for me, but it does seem inconsistent with the
normal text fields. That is; Normal text fields get automatically
translated according to the connection character set (if the above
statements are correct), whereas the connection must take special
action to obtain the translations for text blobs. Probably not a big
deal, just inconsistent.

> >Arrays...
> No, it reflects the general disregard for blobs. However,
> array data will be transferred without regard to content
> - byte value for byte value.

Thanks. Again this makes it easier for me, but again it seems
inconsistent with the treatment of normal text fields.

> Sorry not to have a better answer for text fields
> - perhaps someone else...

I hope to find out soon whether the blr code I am writing will avoid
the character set problems I experienced via the API (via IBO).

Thanks for you help.

Geoff Worboys
Telesis Computing