Subject Re: External procedures: implementation proposal.
Author paulruizendaal
> Paul, this means the Fyacle and Roman's external procedure
> mechanism will not run with Vulcan. I think it is a serious
> mistake to implement a major feature knowing that it cannot be
> supported in the future. If you need that feature in the immediate
> feature, I think it would be better if you maintain it in your
> private tree and not attempt to merge it into Firebird.

Jim, I think you have misunderstood me:
- Fyracle currently is a private, publicly visible tree
- I am not planning to keep the 1.5.x tree alive forever; I want to
move forward to FB2, and FB3/Vulcan -- rather sooner than later
- I see FB1.5.x based Fyracle as "build one to throw away"; I don't
mind refactoring/re-implementing/re-architecting stuff for the
FB3/Vulcan version of Fyracle.
Inefficient? Perhaps, perhaps not.

> Splitting "the server" into separate libraries for the client
> library and providers is critical for future development. Without
> the ability to access gateways to Interbase, legacy versions of
> Firebird, and multiple engine versions it will be very difficult to
> move Firebird forward in the future.

Agree 100%

> It would also mean writing off Vulcan, which I hope nobody is
> willing to consider.

Nope. My plans depend on Vulcan/FB3 being a run-away success.

> I haven't the slightest objection to Roman's external procedure
> mechanism appearing in a private version, but I have a great deal
> of trouble understanding how we can accept code that breaks our
> architecture and can't be integrated with versions now under
> development.

Agree. See above. In my view, this discussion is premature.

> >For instance, it would be possible for the engine to expose a jdbc-
> >style connection object and for the external SP interface code to
> >expose a isc_* api like call interface, generating its own handle
> >id's on the fly, just like the Y-valve currently does. The IscDbc
> >code builds a jdbc-like api on top of the isc_* api; the reverse is
> >equally feasible.

> That could be done, but would be horribly inefficient. The
> internal SQL is designed to bypass the SQLDA handling, and
> consequently is significantly faster than DSQL. Taking Java JDBC
> code, turning it into DSQL API (SQLDAs and all) then remapping the
> DSQL back to JDBC strikes me has a miserable exercise in creative
> squandering of processor cycles.

Agree once again. However, reworking all access libraries (jdbc,
odbc, .Net, delphi stuff, ...) just to make them work in external
stored procedures is a bit much right now. Using the isc_* api hack
in FB1.5.x and offering a (cheap CPU cycle squandering) isc_* compat
layer in Vulcan/FB3 is an acceptable road to go, IMHO. And, yes, I
would support gradual migration to a better (jdbc-like) access api
for both regular and external SP use.

> >In my mind, the big question for FB3 external SP's is not so much
> >the API, but how to isolate failures in the external code from
> >messing up the engine. PL/SQL, java, .net/mono could perhaps be
> >supported "in process" with sufficient stability, but how about
> >everything else?

> Failures, basically, can't be contained. There is nothing that
> Firebird can do to detect, prevent, or recover from external code
> writing random stuff in the virtual address space. I have never
> liked running external code in the engine, but until Java, there
> was no alternative other than to forgo UDFs and blob filters.

Hmmm... thinking long term road map (i.e. beyond this discussion),
why do we run all stuff in-process? Why not run untrusted UDF's and
untrusted external procedures in a separate process? Marshall data
back and forth just like webservers do under the fast-cgi protocol?

> Personally, I care less about multiple language support inside the
> engine than other people seem to.

"Horses for courses"

PSQL, PL/SQL caters to the folks that want to do code with a high
bandwith to the relational side. Java seems to be increasingly
accepted as the portable SP language. Mircosofties may prefer CLR
based stuff.

I do not think I am informed enough about all use cases to make the
call on what to support and what not. Well designed interfaces leave
it up to the user.