> I suggest you don't save it. It won't be the same with line feeds
> deleted.
>

OK. Thanks for the warning :-)

> Run length encoding is nothing more than compressing multiple
> occurrences of like bytes. It works well on trailing blanks and nulls.
> It doesn't do much at all for anything else.
>

OK. I'll understand. But so:

You asked about a crypto library - so I thought, that crypto fields
would make it into vulcan (maby I am wrong). And I thought, that you
can't search for crypto fields (would break crypto, if you can see the
value in the index file) so I thought it would be easy to implement a
compressed field which could be the same mechanism (with blobs you
could do this as I have learned - blobfilters :-) ).

> OK, I'll concede that 35 million records a day is a fair bit.
> Creativity is definitely going to be required. Have you considered
> improbability quanta?
>
>

No :-)

These are just proxy log files from a smaller company ;-). They are
actually laying in front of me: 879 MB of fine gzipped log files :-)
Extraced: 6 GB :-)

I anonymize, normalize and aggregate them and store the result in the
database - but my boss want's to have the whole normalized log for 1
month in the database :-( (he asked for 1/2 a year but ...) . And this
is for what I will use external files.

Chris