| Subject | Re: [Firebird-Architect] Data Stream Encoding |
|---|---|
| Author | Jim Starkey |
| Post date | 2005-05-03T15:39:51Z |
Steen Jansdal wrote:
prepend the encoding in memory with a vector of offsets to the specfiic
data item and a variable indicating how many offsets are known. If the
offset isn't know, the code needs to skip over items from the last known
to item required, keeping track of the intervening offsets on the way.
The skip operation take a pointer to an item and returns the pointer to
the next, and is a very cheap operation, but not one you'd want to do
n-squared times.
encoding and decoding could be done very, very cheaply and that the
scheme gave relief from the problem is fixed length physical lengths in
a world of variable length characters, it became apparent that it could
be used in other contexts. There isn't any reason that they need to be
the same -- they never meet -- but if the same scheme solves both
problems, why not? There is one significant difference in requirements
for the two applications, however. The encoding for the API must be
platform independent, but the ODS encoding has no such need. The API
encoding stores binary numbers must significant byte first. The ODS
ecoding could use native byte order, but frankly, I don't see any
performance advantage in doing so -- a move to a union and a load is
probably more expensive that a shift loop, especially for very short
integer lengths, which will always be the most frequent.
But let me turn around the question: Can you think of any efficiencies
that would justify making the API and ODS encodings different?
Jim Starkey
Netfrastructure, Inc.
978 526-1376
>Jim Starkey wrote:Sorry. I left out a few details. A way to handle the problem is to
>
>
>>It slightly complicates decoding in an important case. Assume we were
>>using the scheme to encode records on disk. To fetch value n, it is
>>necessary to parse value 0 to n, but no more.
>>
>>
>
>Please tell me how it's more complicated than your original design.
>You also have to parse value 0 to n.
>
>
prepend the encoding in memory with a vector of offsets to the specfiic
data item and a variable indicating how many offsets are known. If the
offset isn't know, the code needs to skip over items from the last known
to item required, keeping track of the intervening offsets on the way.
The skip operation take a pointer to an item and returns the pointer to
the next, and is a very cheap operation, but not one you'd want to do
n-squared times.
>BTW I thought we were talking about a communication protocol, notThe idea began is part of the API. When it became clear that both
>a disk record layout. Do they need to be the same?
>
>
encoding and decoding could be done very, very cheaply and that the
scheme gave relief from the problem is fixed length physical lengths in
a world of variable length characters, it became apparent that it could
be used in other contexts. There isn't any reason that they need to be
the same -- they never meet -- but if the same scheme solves both
problems, why not? There is one significant difference in requirements
for the two applications, however. The encoding for the API must be
platform independent, but the ODS encoding has no such need. The API
encoding stores binary numbers must significant byte first. The ODS
ecoding could use native byte order, but frankly, I don't see any
performance advantage in doing so -- a move to a union and a load is
probably more expensive that a shift loop, especially for very short
integer lengths, which will always be the most frequent.
But let me turn around the question: Can you think of any efficiencies
that would justify making the API and ODS encodings different?
>--
>
Jim Starkey
Netfrastructure, Inc.
978 526-1376