Subject Clumplets and formats.
Author Alex Peshkov
Clumplets and formats.

Currently (in FB 2.1) we support a lot of clumplets formats. I will mention
some of them:

1. Tagged DPB - traditional DPB
2. UnTagged DPB - used somewhere near to it, for example - client IP address,
itself stored in DPB.
This are best formats all over engine. The only limitation is size of each
clumplet - it can't exceed 255 bytes. For most cases it's enough, but there
are already at least one place where this limit becomes problematic - trusted
authentication. Size of authentication item can be more then this count of

3. SpbAttach - used in isc_service_attach()
4. SpbStart - used in isc_service_start()
5. SpbItems - used to request data in isc_service_query()
6. SpbInfo - returns data in isc_service_query()
Hard to imagine more nightmare - let me not comment services parameters blocks

7. Tpb - used in isc_start_transaction()
Almost fine, but some items require additional data to be put after it, while
most - not.

8. WideUnTagged - used in DatabaseSnapshot to store data for monitoring
Almost like DPB (2), but uses 32-bit for length of each clumplet.

It seems it too much for us, is not it?

Original DPB idea was to be able to easily skip unknown parameters. This is
fundamental DPB feature, making it possible for new client to attach to old
server - it easily skips what it can't process, and therefore can proceed
with the rest of DPB. But for formats 4,6,7 this is not true - and it already
caused problems when adding new services, that were missing in previous
versions of firebird.

I suggest to replace all this varieties with single common format (it may be
tagged or not tagged). Use of current DPB everywhere is not desired cause 255
bytes per clumplets may be not enough for trusted authentication, file names
in services and in the future may be something else. To forget once and
forever about size limit at the same time avoiding long zero sequences
travelling around network, I suggest to use format of coding clumplet length
similiar to one used to keep recno in ODS11 index. Lower 7 bits of each byte
contains 7 bits of clumplet length, upper bit set to 0 zero means this is the
last byte in the length of clumplet, 1 - next byte is treated as following 7
bits of clumplet length. This lets represent clumplets of virtually unlimited
size, at the same time remaining safe when passed across network between
computers with different architecture (no need to use isc_vax_integer's
family of functions). Slowness of it (bad for indices) doesn't matter when
used in clumplets length.

Code changes are minimum - in places where code is already rewritten to use
ClumpletReader and ClumpletWriter classes all we need is to change types of
clumplets and tags, and there are not much places where new classes were not

When attaching to old servers, client library can itself convert new format to
old one - provided clumplets in it fit in old format.

I suggest to schedule new parameter block formats for the release next after
implementing MT engine with shared data cache.