Geoff,

>> This sounds like something that would definitely help me reach
>> my current pending milestone for the 4.x branch.
>
> Hmmm... I was not really trying to do anything that you were
> already working on. From your previous posts, and from the way
> that v4.9 was altered, it did not seem that you were working on
> a unicode version of IBO v4... and I did not want to wait for a
> v5 if I could get it working in v4.

Let me try and clarify. Both IBO 4.x and IBO 5.x should work with all of the
character sets that Firebird/InterBase support.

The difference between IBO 4.x is you will have to deal with ANSISTRING
being the default AsString data type and have to use the AsWideString in
order to get the proper representation of the multi-byte character sets.

With IBO 5.x it will use the native STRING type in Delphi 2009+ that will
require no special handling to work with wide characterset data.

>>> One of the experimental bits of this process is that I have
>>> tried to bring across some ideas from my C++ unicode work...
>>> where we can actually convert text (and text blob) fields
>>> from whatever character-set they are provided in (Utf8,
>>> Win1251 etc) over to unicode. [...]
>
>> This sounds very useful, but it would also be a very
>> significant undertaking.
>
> Actually most of the work is done by Delphi and Firebird. Since
> FB v2.1.x the relevant character-set is given in XSQLVAR for
> text and text-blob fields. At this point they seem to have
> skipped array fields (but then so do I so it has not worried me
> much.) [Not saying I have got the code perfect yet... but any
> mistakes are probably mine rather than Delphi/FB.]
>
> The "trick" will be to get the functionality to work in with
> older versions of Firebird where the relevant character-set is
> not easy to get... and in these cases I have tried to make it
> so that the code would have much the same effect as it had
> before. ie. Use the connection character set. The logic being
> that older versions of Firebird did not really support this
> concept well so people should have been using an explicit
> character set on their anyway.
>
> And I still support the old UTF8 column attribute in the new
> code - override the default character-set processing.
>
> At this point I have made no attempt to implement support for
> custom character sets (I am happy for that to be an IBO v5
> thing :-).

Ok, perhaps I did not understand what you said property. I prefer to leave
the handling of character sets to the database and the OS as much as
possible. I would not eagerly delve into a project that required too much in
the way of custom transliteration routines on the client.

>> I do have test suites but they are heavily integrated into my
>> development environment. Likely they would be difficult to
>> uproot and transplant in another environment. What we ought
>> to do is get me up to speed with what you are doing and see
>> how much can be merged for an official 4.10 release. I will
>> take care of the bulk of the testing so that you can focus on
>> helping me in this area that I find a bit daunting. Your
>> assistance here will be a great catalyst to help me get some
>> solid traction and hit the milestone of proper characterset
>> support for the 4.x IBO code base so that I can turn my focus
>> to IBO 5.x more fully.
>
> Hmmm... I seemed to remember you had something like that, which
> is why I asked about it.
>
> I am not a fan of DUnit, but now that it is part of Delphi I
> have started migrating some of my test code. A new specialised
> base-class has allowed me to force it into something closer to
> what I wanted.
>
> Any chance of sharing relevant (core-only) bits of testing code
> with me? I could migrate that to the new test regime as part
> of the larger project. Being experimental I was hoping to find
> the worst of the bugs before getting too embarrassed by showing
> others my mistakes. ;-)

Gotcha! I'll see what I can come up with.

Of course one of them is I go through the sample applications and drill the
special aspect it is designed to demonstrate. Also, since some of them were
originally submitted as bug applications I revisit the issue to make sure it
is still solid. That would be a very good place to start.

> I do want to do at least some testing here before passing on
> the code - no point sharing until the biggest major holes are
> filled. If you cant share I will start writing at least a few
> tests of my own.

Start with the sample applications and I'll see what I can do from there.

Thanks,
Jason