Dimitry Sibiryakov wrote:

> Right, the buffer ban contain anything from N to 4N, but find out how
> many exactly is not a trivial task. It is more complicated that
> (SizeOfBuffer/BytesPerCharacter).

It is more complicated than that to find out the correct number of
bytes, but not to find out N. N is exactly equal to the formula above.

>>You seem to think that Milan's code does both in the same step by
>>converting to UTF16. This is not correct. If you have a buffer like this:
>>N
>>O
>><space>
>><space>
>><space>
>><space>
>><space>
>><space>
>>for a char(2) utf8 column, then a conversion of that buffer into utf16
>>would result in a 16 byte string containing 8 codepoints. You still
>>wouldn't know N or how to determine N. Utf16 does handle codepoints that
>>are 4-byte in utf8. Utf16 uses surrogate pairs, so even in utf16 not all
>>codepoints are two bytes. Some are four bytes.
>
> Ok, but look here: for CHAR(N) you can find out N as
> (SizeOfBuffer/BytesPerCharacter). Note, that fbclient has no idea about
> BytesPerCharacter. To get right string you must cut off (N-RealLength)
> trailing spaces (in your example you must cut off 6 spaces).
> But how one can determine "RealLength"? The only way is scan buffer
> and count codepoints which in UTF8 can have size from 1 to 4 bytes.
> Milan's code is aware of UTF8 and can calculate RealLength, fbclient
> isn't.

But the application's work would be a lot easier if N (or
"bytesactuallyused") were contained in the (X)SQLVAR struct. fbclient
can't derive either from what it currently receives from the server, but
the server should know N at least, and could pass it in XSQLVAR.sqlscale
unless I've missed something.

> And anyway: will it help much if instead of
> ('N','O',' ',' ',' ',' ',' ',' ') the buffer will contain
> ('N','O',\0,\0,\0,\0,\0,\0)?..

Yes, if the struct docs says that the buffer will be padded with spaces
up to N codepoints and \0 beyond that. But that would also require that
\0 can't occur in the actual data value. I suppose that is and should
still be allowed, so i that case the change in padding would not help.

But note that noone has asked for that change in padding. What we've
been asking for is N or "bytesactuallyused". And that *would* help.

While the application *can* get N by querying the charset system table,
it is a rather complicated task and also not very intuitive that it is
required. If, on the other hand, N is passed in XSQLVAR.sqlscale, and
this fact is documented where the rest of XSQLVAR is documented, the
application programmer will at least be made aware that something has to
be done with the returned buffer, and it will be rather intuitive to
learn *what* has to be done.

Kjell
--
--------------------------------------
Kjell Rilbe
DataDIA AB
E-post: kjell@...
Telefon: 08-761 06 55
Mobil: 0733-44 24 64