Subject Re: [Firebird-Architect] Re: External procedures: implementation
Author Jim Starkey
Roman Rokytskyy wrote:

>>Sorry, but the old style API doesn't work inside the engine, so it
>>isn't an alternative.
>Would you like to explain this in details? It would be interesting not
>only for me, I think.
The Firebird architecture has its root in the DEC Standard Relation
Interface (DSRI) in the mid-1980s when DEC was the second largest
computer manufacturer. We designed DSRI to provide consistent database
management services across a group of operation systems to an open ended
set of database systems encompassing procedure and process based
software products, bundled database servers, and database machines.
Interbase's version of DSRI was called the Open System Relational
Interface, and differed from OSRI only in mechanical details. The scope
of OSRI, however, was greater than DSRI, supporting heterogeneous
connectivity (DSRI was VAX only) and gateways to DSRI and non-DSRI-systems.

OSRI is a structured OO interface (designed before there were OO
languages other than Smalltalk), where an object as represented by an
opaque handle. Objects were formally created (attach_database,
compile_request, start_transaction, open_blob) and formally destroyed
(detach_database, release_request, commit_transaction, close_blob).
Communication between the client and the OSRI world revolved are
object/handles and messages.

DSRI/OSRI was designed to make it easy to implement very low overhead
transmission layers (aka plumbing) to establish connectivity from a
client program to a database engine somewhere on the network or even
straddling networks (there used to be many competing network types).
Examples of transmission layers are the Y-valve that polls subsystems to
find one that will accept a connection, the remote interface (remote
procedure call client), the remote server (RPC server), and gateways to
other product sets. Database engines formed the far end of the
connection chain.

The thing that makes transmission layers easy and faster is handles.
Most transmission layers create a local object to hold state, including
the handle for the next transmission layer, and return the address of
the object as their handle (a minor variation is to return the index to
an internal vector of object pointers). Since information is structured
as messages, transmission layers are free to ignore message content,
passing the address and length of the message to the next (or previous)

In Firebird 1.0 to 2.0, the database attach sequence work like this:

1. The client calls isc_attach_database
2. The client library y-valve fields the isc_attach_database call
3. The client library y-value call attach_database in the remote
4. The remote interface establishes a connection to the database
server running on the local machine. The remote interface then
passes the database connection string and database parameter block
to the remote server.
5. The remote server receives the database connection string and
database parameter block from the remote interface
6. The remote server calls isc_attach_database
7. The server library y-valve fields the isc_attach_database call.
8. The server library y-valves polls subsystems for a connection to
the named database.
9. The database engine fields the attach call from y-valve and
attempts to open the named database. If successful, it returns
success and the address of a database attachment block to the y-valve.
10. The y-valve allocates a block to hold the identify of the
subsystem that accepted the attach call and the handle returned by
that subsystem. The y-valve returns success and the address of
its attachment block to the remote server.
11. The remote server allocates an attachment block to hold the
y-valve database handle, then sends the remote interface a message
indicating success and an index into the server's attachment vector.
12. The remote interface receives the success message from the remote
server, allocates a control block to hold the socket and
attachment index from the remote server and return success and the
address of this block to the client y-valve.
13. The client y-value receives success and the remote interface's
handle, allocates a control block to hold the identity of the
subsystem that accepted the calls (the remote interface) and the
remote interface's handle, and returns success and the address of
the control block to the client.

In Rdb/* 1.0 to Interbase 3.* and Vulcan, set #3 would poll additional
subsystems for an embedded engine or local gateway.

Note that the database handle is encapsulated in the server y-valve, the
server y-valve handle is encapsulated in the remote server, the remote
server's handle is encapsulated in the remote interface, and the remote
interface's handle is encapsulated in the local y-valve which is
returned to the user. No transmission layer knows anything about the
next layer other than that necessary to pass the calls through.

The problem will trying to call the client interface (in fact, the
y-valve) from the database engine is that y-valve is expecting a handle
it generated (anything else is an error), and there is no way for the
engine to determine what the y-valve return to its client as a handle.
And without a valid handle, the y-valve can't know how route calls or
establish the correct handles.

In Firebird 1.5 and 2.0, the "execute statement" breaks the layering by
calling from the engine back into the y-valve. This only works if the
engine can make internal calls into the y-valve, which is a violation of
layering. If Firebird 2.0 were refactored to make the y-valve and
engine separate libraries, "execute statement" would have been
unimplementable. Sometime in the next month or two I will re-implement
"execute statement" in Vulcan using the InternalConnection classes.


Jim Starkey
Netfrastructure, Inc.
978 526-1376