Subject Re: [firebird-support] How to store a string in Firebird which includes #0's in it?
Author River~~
Post date 2018-08-04T10:59:43Z
Hi again Chuck

Sorry I was a bit cryptic there.

There follows a quick primer in logical bitwise operations 

If it helps, use it. If not, code the "top bit setting" any way that makes sense to you. I would always recommend you code it the way you understand rather than the way some geeky C programmer says is more efficient!

When you do a bitwise AND or OR each bit is processed separately. The result of each bit is the AND of the corresponding bits in the input.

z and 1 is always a
z and 0 is always 0

z or 1 is always 1
z or 0 is always 0

Because each bit is processed in parallel on a 64bit machine you can do 64 of these in one cycle.

x80 (usually written with a leading zero 0x80 in most languages,  and read out loud as hex eight zero, means the base16 number 80) is binary 10000000. So in C the code

data = data | 0x80 

performs a logical OR bit by bit. The first bit will always be 1 regardless of the data, and the other seven bits preserved. 

If you are sure the data is  <128 that is the same as

data = data + 128

If you were not sure you would need to code an if statement as well 

If you are using 64bit integers you can set all the top bits at once with

data = data | 0x8080808080808080



Similarly in a single byte 0x7f is binary 01111111 so

data = data & 0x7f

clears the top bit, and in 64bits that would be coded as

data = data & 0x7f7f7f7f7f7f7f7f

Programmers who habitually need to code bitwise stuff prefer using hex because of the way adjacent bytes just concatenate when you write them in hex.

You may know that binary 10000000 is 128 and 01111111 is 127 but I bet you don't know the decimal integer value of that long hex value. Nor do I, but I do know what it would look like in binary. It's eight lots of 7f so in binary it's just eight lots of 01111111 

Most modern languages have ways to do bitwise logical ops (though Pascal for example disguises them as set operations)



River~~
who has written too much machine code and assembler in the past  ;)


On Sat, 4 Aug 2018, 10:44 Chuck Belanger phytotech@... [firebird-support], <firebird-support@yahoogroups.com> wrote:
 

Thank you guys for your thoughtful response!

I really like the idea of just using 7 bits and keeping the 8th a '1', thus ruling out the possibility of ever having a #0. It is an easy conversion in my routine and storage.

As far as I know NONE or OCTET is the closest thing to rawbyte, but as I mentioned (and another responded) the string is being truncated at the first #0. His comment is probably true: somewhere in IBO this is happening. Waiting on the IBO forum to respond.

Adding another table with a couple of columns seems like too much, too. I was thinking of abandoning my attachment to the BIN to Char conversion, which ultimately is arbitrary. A simple encryption of the BIN string would probably work well, too, albeit longer, but really, even if 8x as much storage we are not talking about much.

BLOBs just seem like over kill for storing a max of 64 chars!

Not sure I follow how to OR, AND the bytes.  What does x80 and x7F do?

Thank you, both!

Chuck


On 8/4/2018 12:46 AM, 'River~~' river14april@... [firebird-support] wrote:
Hi Chuck

I am a Firebird newbie so don't know if Fbd will let you do what you want. Perhaps there is a fixed length raw byte field you could use, or perhaps a blob?

If it does not, then a more efficient work around than the one yours would be to store only seven bits to a byte, forcing the top bit to 1.

This is an inefficiency in terms of storage space but the complexity on retrieval would be simplified greatly, as would the home grown code to repack the data. And you save the storage overhead of the subrecord sequence numbers.

The packing routine would move 7 bytes of data into each 64bit word, and vice versa on retrieval.


An even cruder hack to save packing and repacking the data is to process it in 7bit form throughout. Before storing you would OR each byte with x80 and on retrieval AND it with x7F. That would mean having a dummy group every eighth group that nobody ever belongs to, which moves the complication elsewhere in your code.

Either of these workarounds would be more efficient for data access, more efficient in running your custom code  and easier to write that code too.

Could that be worth an extra 1/7 cost in storage space?

Regards
River~~

On Sat, 4 Aug 2018, 01:30 Chuck Belanger phytotech@... [firebird-support], <firebird-support@yahoogroups.com> wrote:
 

Hello:

I have been using a string of '0000's and '11111's to indicate access to
the underlying position in a database. I have been storing this string
of '000111' combinations by converting to a set of characters
representing each 8 "bits" of the actual 0s and 1s.

The problem I am facing is how to store those occasional strings that
look like this:

'ΓΌ@'#0'0'#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0#0

It is not so much all the null characters after the '0', but that null
character between the '@' and the '0'. The above string is created and
stored in a string just as you see it, but how do I store it in Firebird
, preferably as-is in a table field?

I have tried rawbytestring but that truncates the string at the first #0
character, thus losing the information in the '0' (or some other
character) that follows.

I have not run across this issue before, but as my list of possible user
access groups grows, it is apparent that some users have 8 positions
without having access to any of the related groups, thus the #0 in the
string. And that would be OK, except that they do have access to some
group or groups after those 8 positions. Right now, the access code
truncates that part of the access code information because of the #0.

My only solution right now is to create a new table in a one:many
relation, such that I can have more than one record per Access code.
Each record would hold the characters up to the first #0 character
behind the character set. I would then store the number of #0 characters
that follow the string. That way I can iterate through the records and
reconstruct the original '0000's and '11111's string.

Any thoughts?

Thank you,

Chuck

I did post this on IBObject forum, too. I do use IBO for my database
access. I use Firebird 3.0 on WIndows, mainly desktop. I use NONE as my
default Character Set.

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus



Virus-free. www.avast.com