Subject Re: [Firebird-Architect] External Engines Implementation Details
Author Adriano dos Santos Fernandes
Vlad Khorsun escreveu:
>> Vlad Khorsun escreveu:
>>
>>>> As you see in my external engine API proposal, it creates something we
>>>> current don't have internally. One attachment can have different
>>>> properties depending on the context.
>>>>
>>>>
>>> Is it really good and necessary ?
>>>
>>>
>> Yes. In External Engine, there may be two logical connection types.
>>
>> One is for the Engine itself manage metadata. Not all engines is dumb,
>> they need to inspect metadata to load things (example: .class file
>> stored in blob). And planning security from the start, such type of
>> information should not be available to users. This logical connection
>> generally can work with UTF-8 and should have SYSDBA rights.
>>
>
> Why SYSDBA ? It can be any account granted to read corresponding items.
>
No. Class files should not be readable by anyone that can execute it.

Also, external engine could manage security, metadata, i.e., all things
FB does internally for PSQL handling.

Let's not limit power of new features in favor of we-need-something
(java/.net SP).

>
>
>> Another connection type is the one to execute
>> functions/procedures/triggers. This one needs user rights.
>>
>
> This is current (or internal) connection. And we don't need to do something
> with its rights.
>
Rights for this should be identical of the client/user. But different
from the above case.

>
>> And we can't guess the charset they want.
>>
>
> We must not guess it. They just must work with attachment charset.
>
No Vlad... I may write my application in one programming language or
operating system and write the external function in another programming
language or operating system.

Isn't for such thing we have client charset?

They are definitively two different clients.

>
>> Also, it's another client, running in
>> different environment from the user, so user charset should not
>> interfere with it.
>>
>
> Hmm... EE may ask engine about charset of current connection and work
> with it as is (no transliteration) or we may introduce transliteration (INTL)
> services for EE to do it.
>
Why we need to introduce different things if existing mechanism works?

>
>>>> For example, it can use one charset to the client and another to the
>>>> external engine.
>>>>
>>>>
>>> Why do we need such complexity ? I think we must make life of external
>>> engine (EE) developers much easier than our ;) I mean - there is no sense
>>> to force EE to work with all possible charset's supported by FB.
>>>
>> I don't think your argument is correct. This "complexity" makes our
>> (engine developers) live a bit less-easier. EE developers can just pass
>> a charset name.
>>
>
> Pass charset name into DPB when obtaining handle of current connection ?
> Hmm... ok. But then engine must remember it on per-EE basis for every attachment
>
First, EE may cache connection handle. Engine needs to remember charset
of each pseudo attachment.
Very like it current works with normal attachments... Coincidence? ;-)

> and perform transliteration on every call of ESP\EUDF, right ?
>
Using normal transliteration rules.

>
>>> At my view
>>> two is enough - NONE (no transliteration between engine and EE) and UTF-8
>>> (as common denominator for all charsets).
>>>
>>> Not all applications require more than one charset to work with. Most european
>>> and american developers may work with one-byte encoding - plain ASCII, i think.
>>> They can use NONE (i called it fb_string_ascii in Fyracle) in most cases. All other
>>> will work with good known UTF-8.
>>>
>>>
>> This may be true only in managed (JVM / .NET) environments.
Jaybird, for example, do his jobs with charsets at client side.

>> I don't
>> think we should break all INTL concept of transliteration and client
>> charset for nothing.
>>
You said to have NONE and UTF-8 transliterations. But we can have
transliteration for any charset without any additional price.

>
> I don't understand both statements. Explain, please
>
>
>>>> It also can have user or sysdba rights.
>>>>
>>>>
>>> Can you explain more on this ?
>>>
>>>
>> Already explained above... For EE metadata handling. And also, this is
>> required for future internal SQL metadata handling.
>>
>
> I see two *different* connections - one is for internal needs (metadata handling, etc)
> and second is in user's context.
>
So do you see that our current GDML calls need to be accounted as
different connection from the user?

I've tried to create something compatible with future
SQL-replacing-GDML-in-the-engine. Pseudo attachments seems the way to go
for both things.

>>>> Different charsets are also needed for future implementation of internal
>>>> SQL, as engine SQL is expected to work with metadata (UTF-8) charset
>>>> independently of user charset. The same for user / sysdba rights.
>>>>
>>>> To make it work, I need to:
>>>> - create a REATTACH y-valve internal function
>>>> - create the concept of "pseudo attachments"
>>>>
>>>> A "pseudo attachment" is resolved in DSQL and engine entry points and
>>>> changes the real attachment properties and continues with it.* This
>>>> seems to not be a problem for multi-thread, as multiple threads can't
>>>> use the same attachment simultaneously. (is this simple correct?)
>>>>
>>>> * Dynamic properties can also live in the thread data, but this is
>>>> enough for current implementation IMO.
>>>>
>>>>
>>> Is it something like ALTER SESSION ?
>>>
>>>
>> Not exactly.
>>
>> For user, it's more like two attachments.
>>
>
> I can understand two attachments...
>
User attachment x Function execution attachment: definitively same
attachment:
- same rights
- different client charset

EE attachment: an extension of the engine:
- have sysdba rights
- may work with engine charset (utf-8), but why limit it?

>
>
>> For engine, only entry points functions understand the pseudo
>> attachment, transforms it to real attachment. For monitoring purposes,
>> there is only one attachment.
>>
>
> ...but i don't think "pseudo attachment" is a way to go.
>
>
>>>> I also need to communicate y-valve handles to engine, as Vlad does in
>>>> exec. stmt branch. But the solution Vlad is using doesn't work with
>>>> CONNECT and TRANSACTION START triggers, as he said.
>>>>
>>>>
>>> Yes. I think when DSQL will be moved inside engine this issue will disappear
>>>
>>>
>>>
>>>> I think these handles could be communicated to the engine via internals
>>>> DPB/TPB.
>>>>
>>>>
>>> Are you going to pass y-valve handle (or its address) via DPB/TPB ?
>>>
>>>
>> This solve the problem with database triggers.
>>
>>
>>> Or i, very probably, not understand something ?
>>>
>>>
>> Since it's from y-valve to internal engine, I see no problem except
>> establish a generic DPB/TPB mechanism for this.
>>
>
> I don't like this hack. I prefer to introduce private jrd8_attach_database2 call
> with additional argument to do it. I prefer much more to leave it as is until
> integration of DSQL into engine.
>
This is a thing we may discuss latter. :-)

>
>
>>>> - ugly API, where engine doesn't understand client (DSQL) structures
>>>>
>>>>
>>> Not sure i understand this
>>>
>>>
>> Vulcan Engine8 class can handle only BLR messages, and Dispatcher can
>> handle DSQL structures. Both are derived from Subsystem that implements
>> both ways throwing error.
>>
>> It's very bad designed.
>>
>
> I'd said this is because DSQL is not properly integrated into engine (still !) ;)
Anyway, why provider should deal with BLR messages and XSQDA/VAR structs?

But let's concentrate on more important things for now. :-)


Adriano