At 02:37 PM 5/2/00 -0700, Bill Karwin wrote:

>
>Since we're talking about plug-ins for other things, how about permitting
>custom external format I/O modules.
>

This seems like a good thing, but is a non-trivial undertaking
to do right. The problem is that the external file mechanism
has to play nicely with many parts of the engine, including
some very non-trivial pieces.

The existing external file mechanism was basically designed to support
RMS files on VMS (I had a wrong idea that the VMS world might like
a database interface to RMS; oh well). It uses the internal
format mechanism and system tables to map fields onto records.
It has been extended to map flat Unix files. The data stream
oriented calls are straightforward -- open, fetch, modify, close.
An external file manager, fortunately or not depending on your
aspirations, has to interact with the optimizer.

A more general architecture would need some more elaborate
mechanism to perform mapping from the simple relational model
onto a complex file structure, not to mention a mechanism for
populating the appropriate system tables through a DDL statement.
Yes, the active system table mechanism would work just fine, but
is culturally incompatible with a DDL oriented user base.

The hook to making it work could be RDB$EXTERNAL_DESCRIPTION,
which as far as I can tell is unused. Like the authentication
plug-in that we're obstensibly discussion, RDB$EXTERNAL_DESCRIPTION
could use an Apache-inspired meta-syntax that interacts with
a loaded plug-in to pass configuration information.

I think this would be a nice thing to have, but there are a lot
of other stuff on the plate. Of course, in an open source
project nobody (but Ann, and they only to her employees and
sometimes (rarely) her husband) can anybody what to do. I would
rather see a authentication plug-in so I can rid my machine of
isc4.gdb, but those are my druthers. Mostly my job is to see
if it gets done it gets done right.

I don't mind talking about this -- it's a pretty good pedagogical
subject. But I don't want to give the impression that his is a
good "baby's first open source project."

Jim Starkey