m_theologos wrote:

> --- In Firebird-Architect@yahoogroups.com, Jim Starkey <jas@...>
> wrote:
>
>> m_theologos wrote:
>>
>>> --- In Firebird-Architect@yahoogroups.com, "Ann W.
>>> Harrison" <aharrison@> wrote:
>>>
>>>
>>>> There have been designs for it for decades. There is a practical
>>>> problem however, which is optimization. We currently optimize to
>>>> minimize the number of records in the intermediate data sets.
>>>>
>>>>
>>> Agree. But I think that is much more important to have _now_
>>>
> simple
>
>>> (not simplistic) working features with a good field of
>>>
> improvement
>
>>> (in order to attract users), rather than to have a perfect
>>>
> feature
>
>>> after, let's say, 3 years. (See also the Paul Ruizendaal's
>>>
> message)
>
>>> IMHO, perhaps is better to support in the next version the most
>>> common case, which is, I think, the most simplest ie. all
>>>
> databases
>
>>> locally on a superserver architecture, eliminating in this way
>>>
> the
>
>>> communication problems and also having the advantages which
>>>
> Adriano
>
>>> dos Santos Fernandes pointed out (the same address space, easy to
>>> find structs aso.).
>>>
>>>
>>
>>
>> Are you suggesting that we implement a dumb-ass, non-extensible design that solves 15% of the problem and can't be extended to the remaining 85%? It's not like this hasn't been thought about and discussed. We'd be a happy to go over the issues if you'd like, but waving your hands and suggesting that they can be ignored doesn't strike me as a
>> particularly smart way to proceed.
>>
>>
>
> No, I didn't suggest this. But as far as I analyse the problem I see
> two (rather) distinct cases. If you pay attention at my first
> message, you'll see that, in fact, I proposed more or less the same
> ideea that you did. But now I think that is a rather different
> situation when we have the DB files stored locally, so, we can simply
> (I think) do our job (in a superserver architecture even easier, I
> think...), and the other case is the one which both of us tried to
> cover by using a server-to-server communication engine (which is
> _very_ important to be a very good one because, IMHO, this
> communication layer can be used in the future to do other things like
> clustering). In fact, the Firebird itself has a different approach
> (internally of course) when it deals with a local fdb (through XNet),
> and when deals with a remote one. It doesn't connect through TCP/IP
> at 127.0.0.1 (and this is the way it should be, IMHO).

There are more cases than this, particularly when an engine can also be
client process local in Vulcan. But in the end, there are two broad
cases: all engines are in the same process or engines are in different
processes. I think you will agree that the second case, engines in
different processes, there is no commonality between existing engine
code and that necessary to support cross process, so extending the
engine to do this is not significantly different from creating a
separate provider.

That leave two databases running in the same process. Again, there are
two cases. The databases are the same ODS version or they are different
(Interbase prior to V5 and Vulcan and later support this). You have to
understand that an engine cannot operate on a database with an
incompatible ODS. That leave the case where one engine is operating on
two (or more) databases. This is not what you might think. The two
database instances have independent data structures with independent
synchronization and memory managers. Even if the engine has pointers to
the roots of the two sets of data structures, it is constrained to enter
each at the top so the proper interlocking is performed and the
structures are protected from concurrent access. This dictates that
lower level functions in instance A will have to make high level calls
on instance B to avoid corrupting B. So it isn't as simple as a join
RSB referencing indexed RSBs in each of two databases. Database,
Connection, Statement, and Transaction objects are also necessary to
handle verb backout and transaction rollback. And since a transaction
now straddles two databases, a two phase commit is required to
synchronize them. Two phase commit handling, however, is handled in the
Y-valve / dispatch module, a completely different layer. So that code
must he functionally replicated in the engine.

There really aren't any shortcuts here. There are two alternatives:
Extend the engine to do something it was never designed to do that would
at least quadruple its complexity, or build a separate provider that
would decompose and analyze a cross database SQL command, generate
commands that operate against the target databases independently, and
synchronize / merge the results.

Writing a new provider to support integrated cross database access has
many advantages. One is that it isn't dependent on particular engine
versions. It would work on all versions, Interbase, Firebird V1,
Firebird V2, and Vulcan. It wouldn't break the engine during its
development. It would support cross database access across process and
server boundaries. It would support cross database access across major
version and even across a mixture of Firebird and Interbase databases.
It would have a release cycle independent of the current Firebird
engine. It also could reuse major encapsulated classes from Vulcan --
the SQL parser, memory managers, sort, the mover, etc.

--

Jim Starkey
Netfrastructure, Inc.
978 526-1376