Subject RE: [Firebird-Architect] WAL and JOURNAL
Author Samofatov, Nickolay
Hi, Oliver!

> > > Will it be easy to send set of pages to the remotes such as only
> > > committed transactions are reflected in the remotes ?
> >
> > Yes, the net effect will be that the remote/slave databases will
> > "see" committed transactions, even though the disk pages may have
> > uncommitted changed.
> So we're clear about the fact that those slave would be guaranteed to
> be read-only beasts : only accepting read-only transactions or another
> mechanism to guarantee those slaves are read-only ?

Why do you think I was fixing readonly databases and transactions

> Are we sure those uncommitted changes eventually physically present on
> the slave pages could not interact badly with running queries ?

They cannot. Versioning still works. VIO_chase_record_version should
still work correctly for readonly databases.

> If I start a "long-running" query in a default read-only transaction
> (concurrency) on a slave, and new page updates come in from master
> with new committed transactions, how this will work ? I should need to
> see the old records that might well not exists anymore in the recent
> pages. I'm not sure I'm not lost in the thing, but I fear some issue
> here.

Very good point, you hit probably the most hairy part.
1. Updates are applied to slaves atomically (via delta file magic)
2. We use generational replication of physical pages. Each generation
has its number (SCN - event number incremented each time write
redirection mode changes for the master database)
3. We may treat each propagated generation as a transactional snapshot
and associate it with transaction ID. Since we can do that we may
calculate oldest snapshot watermark for each generation.
4. We may keep per-generation snapshot usage accounting in slave
databases. This may be done via keeping numbers of generations in mirror
database header page (the only page I'm going to allow limited writing
on a slave).
We may introduce new lock type used on slaves to track actual snapshots
usage (or use transaction locks for this purpose since effective IDs of
transactions should correspond to used generations).
5. It looks like we need to stall oldest_snapshot counter for master
database to keep record versions needed to reproduce snapshot for the
oldest active generation across the mirror set.

There is a couple interesting things needed to be done to make it really
1. Some measures are needed to be done to prevent very long-running
snapshots and outdated mirrors from inhibiting master performance.
2. Transactions running on slave databases should use effective IDs
based on current generation number when transaction was started. I think
tra.cpp will require some minor modifications to make it work.

> Olivier Mascia