Subject Re: [Firebird-Architect] Re: External "procedural engine" modules
Author Jim Starkey
paulruizendaal wrote:

>Thank you for taking part and lifting the focus of the discussion to
>a very architectural level. However, your message is a bit too terse
>for my feeble mind. I would appreciate a bit of elaboration.
Sorry. Terse is not a new habbit. I'll elaborate. Lucid is Ann's

>>The right way to implement PL/SQL is an interpreter sitting
>parallel to
>>but separate from looper that implement PL/SQL semantics.
>Yes, that is my current conclusion too: I wasn't satisfied with what
>BLR/looper could do for my needs and hence I did a parallel
>interpreter. JVM would be another kind of parallel interpreter.
No. A JVM could be a perfectly fine sandbox in which to run stored
procedures and triggers, but it could never function as an alternative
to a database engine. Simple too much C/C++ code to interface to and
the garbage collection policies are radically different.

I'm suggesting that a purpose-built PL/SQL interpreter parallel to the
BLR interpreter sharing optimizer, RSB, EVL, and VIO interfaces.

>>and BLR semantics are handled by a painfully tortuous "looper"
>>to allow a statement to stall in a non-threaded environment.
>I don't understand from 'designed' onwards. Please explain a bit if
>more than casually relevant.
The architectural challenge of building a database engine in the
pre-thread world was being able to stall an executable request for
client interaction. I had implemented VAX Datatrieve is a coroutine as
one solution to the problem, and while it worked, I didn't want to go
back there. So I designed the BLR langauge and engine architecture to
allow execution state to be saved in a per-request impure area rather
than the stack. If threads had been an option, I never would have done
it that way. Ironically, I ended up implementing the first VMS
threading package that exploited the stuff the VMS guys did to allow
Datatrieve to run as a coroutine, but that's a different story.

Looper, the BLR intepreter engine, functions by calling language
statement nodes with a particular state. Each language node returns a
successor node, state, and stall flag. If the stall flag is set, loop
saves the state and returns to the interface layer. When the client
continues the request with a send or receive, looper continues where it
left oft.

The right place for looper is a software museum. But for the sake of
backwards compatibility, it must be left in place. But it really should
be supplanted by other interpreters that a) rely on threading, and b)
implement modern SQL semantics directly. But since all good software
engineers count, "zero, one, infinite", it makes more sense to morph the
engine architecture to support an arbitrary number of interpreters.
PL/SQL is definitely on my short list of utterly detestible languages,
but it is hard to deny its economic position in the world. And, in any
case, a hand held out to rescue an Oracle user striving for redemption
is a good deed.

>>There is no reason to extend BLR (which
>>akin to extending the dodo) when an relatively simple and more
>>alternative engine would do the trick.
>Hmmm. This sounds to me as you recommend a single interpreter. The
>first line suggests recommending two parallel interpreters. I'm
>confused now (which happens often & quickly :^) ).
No. The opposite. I'm suggesting at least three interpreters (existing
BLR, industry standard SQL (ugh), and PL/SQL (double, yea triple, ugh).

>Perhaps you mean that RSB, EVL and VIO should work with two "ping-
>pong-ing" interpreters, one for procedural code (eg. JVM, BTC,
>whatever) and one for DML/DDL code (perhaps a trimmed form of the
>current BLR, perhaps something a bit akin to the sqlite VDBE, but
>more sophisticated - see or
>perhaps something different still)
The only proper database interface for Java code is JDBC, meaning any
embedded JVM must have a JDBC interface. Whether the JDBC interface is
implemented against the BLR interpreter, an "industry standard SQL"
interpreters, or a purpose built JDBC interpreter is a fun architectural
question in it's own right. Netfrastructure, of course, is built around
a native (i.e. C++ JDBC interpreter).

But you never want to expose database internals (meaning, in specific,
the optimizers, the RSB mechanism, the EVL mechanism, VIO, or IDX/BTR).
These beasts are non-architectural and should remain that way.

>I suppose 1 procedural engine should be native to the engine (to at
>least support PSQL, but perhaps also PL/SQL). It would probably not
>be all that hard to add an API for external engines, like a JVM or
>external, compiled C/C++ procedures.
Again, no. External code should never, ever have access inside the
engine address space in anything other than a controlled sandbox for
reasons of security and reliability. UDF's, of course, violate this
rule. When I implemented them, there were no alternatives, and the
functionality was critical. But as Dale would say, that was then and
this is now.

>A potent statement from a potent source. But how should I read it?
>1. Stop thinking abour extending BLR and go for the separate
>interpreter , or

>2. Stop prototyping new directions until a rewrite of the internal
>interpreter is done
>Please elaborate.
No! Don't rewrite the internal interpreter. Add another. Or maybe
two. Or an smallish infinity.

[Non-text portions of this message have been removed]