Subject Re: [Firebird-Architect] Metadata cache
Author Vlad Khorsun
> Alex Peshkov escreveu:
>>>> When transaction created\altered\dropped object, it creates new
>>>> version and attached it into local cache. All prepared within this
>>>> transaction statements which used altered\dropped object marked as
>>>> obsolete.
>>>>
>>> Why? In this case, I think the DDL command should fail if there are
>>> prepared statements in the same transaction using the object.
>>>
>>
>> Why should it fail? What's wrong with disabling access to prepared statements
>> in current transaction for the future?
>>
> In my proposal, manual or automatically invalidation of prepared
> statement was the worst thing of it. When statement is allocated and
> prepared, it appears it should run correctly if it doesn't have logic
> problems. When execution fails it is generally problem of that specific
> run. But invalidation cause each all subsequent run to fail and it's not
> very logical to have to re-prepare the statement.

If engine have enough information it can re-prepare statement silently.
This is the case of stored procedure\trigger\etc. But not the case of
user statements. Speaking more general - if engine knows SQL text or BLR
it can re-prepare statement transaparent to user application.

> Applications may be caching prepared statements (like we do in EPP
> files, but using DSQL) and this statements will fail forever.

This is up to application developer. He know what kind of statements
is cached, when to change metadata, and how to handle corresponding error.

>>>> When transaction commits it marked all changed objects instances in
>>>> global cache as obsolete, removed its from global cache and replaced its
>>>> by local cache objects.
>>>>
>>> So metadata will always works in read-committed mode?
>>>
>>
>> Yes. This is not good, but looks like quite possible compromise for FB3.
>>
> The problem I see is "unlucky" transactions getting partial changes from
> system tables.

We must prevent such condition of course. And metadata object's locks may
help us, isnt's is ?

>>>> When new object added into global metadata cache, its signaled (via
>>>> special AST) to all other processes and they loaded new object version
>>>> from database into its global metadata cache, marked old instance as
>>>> obsolete, etc.
>>>>
> Will them just be marked as "must-read" when necessary?

Explain, please

>>> What is not clear for me:
>>> 1) Prepare statement in one transaction, transaction is committed and
>>> statement is run on others transaction
>>>
>>
>> How is it related with DDL?
>>
> With relation of transaction prepared statements invalidation... AFAIU,
> it's proposed that only statements prepared in the same transaction as
> the DDL one will be invalidated if DDL touch the objects.

No. Statement prepared in DDL tx will be invalidated immediately after
each DDL statement, all other prepared statements *may be* invalidated after
DDL tx commits.

> So statements prepared in another transaction will run with old object
> definitions

They will continue its execution if its started before commit of DDL tx.

> but own transaction statements will not? Why they should be
> invalidated then?

Because its must not see old object's definitions. New object's definitions
visible only to DDL tx (until it commits) so statements, prepared in DDL tx must
be invalidated sooner than all other statements.

>>> 2) Metadata integrity checks (see my reply to Alex) when old versions
>>> are involved
>>>
>>
>> OK, I also do not see a good way to solve it without per-process (not global)
>> RW lock per cache (i.e. per database). Luckily, this lock will be taken
>> for 'write' only for a very small period of time - when transaction-level
>> cache is to be merged into global one.
> Does Att2 commit will fail in this situation? How it will be done in CS?
>
> Att1: create procedure P3 as begin execute procedure P1; end
> Att1: could execute P3

New instance of P3 will call old (current) instance of P1

> Att2: alter P1 parameters
> Att2: could execute P1

New instance of P1 executed

> Att1: commit
> Att2: commit

P3's internal request will be invalidated and engine would load and parse
P3's BLR on next run.

Regards,
Vlad