Dmitry Yemanov wrote:

>"Jim Starkey" <jas@...> wrote:
>
>
>>However, a change to a generator is performed with a simple mark (no
>>force-write). This is safe because the generator page will always be
>>flushed before a transaction is committed.
>>
>>
>
>It's performed with a force-write mark since FB 1.5.1. A generator page
>could be actually flushed after the transaction data. There were a number of
>bugreports regarding this case recently - after a server crash the existing
>PK identifiers were generated, because the generator page was not flushed
>properly.
>
>

That's a misdiagnosed bug, then. Making generators forced write is a
terrible performance decision, and is unnecessary. As long as the
generator page (and all other transaction modified pages) are flushed
before the tip, there is no problem, assuming the operating system does
it's job and eventually writes pages to the disk.

If you are talking about "server crash" meaning that the operating
system has crashed, then we're talking about something else, and your
fix still doesn't work. There are two cases. If the operating system
is performing synchronous writes, the generator page will still be
written before the tip, so no committed transaction can have a
non-unique generator. If the operatingsystem is not performing
synchronous writes (the default case), forcing the engine to write the
generator page immediately has no effect on the OS write strategy and
does address the problem you think you saw.

I think you need to go back and re-think the diagnosis and solution of
the bug. If a transaction doesn't commit, it doesn't make any
difference whether or not any generators it may have acquired remain
unique or not.

But there is no doubt that making generator pages forced write (meaning
write on release) is going to have catestrophic performance impact on
applications making heavy use of generators. An application loading
data using generators to manufactor unique keys will have a (page size)
/ (record size) increase in page writes.

>Then your group commits differ from Charlie's ones ;-) And I'm afraid I miss
>something important in your proposal.
>
>If a group commit thread is waked up 5 times per second to flush the
>modified pages of the transactions being committed, it means that either
>every commit waits for t <= 0.2 seconds or commits are async/defered. If
>every commit notifies a group commit thread and waits for it to do its job,
>then I fail to see both "five times a second" and a "group" here. Perhaps,
>tomorrow my brain will work properly and I'll understand your proposal. If
>you'll have a few minutes to forward me in the necessary direction, it would
>be much appreciated.
>
>
>

I'll try it again. A call to isc_commit_transaction waits until the
next group commit cycle. On average, it will incur a cycle/2 latency.
This must be balanced against the system cost of avoidable write operations.

Every transaction in Firebird 2 and earlier requires two OS page writes,
one for the header, the other for the TIP. This bounds the transaction
throughput. If the OS or disk system caches the write, it's less bad,
but still bad.

The more transactions the system is pumping, but better group commits
gets. If, on average, there are a dozen transactions pending per cycle
and a 20 ms cycle time, this translates to 58 fewer writes per second.
I don't have the number available to compute the cross over point where
average transation time (including commit wait time) drops because of
reduced page writes, but I don't think it's a very large number. In my
example, let's assume a typical transaction requires 400 ms. An average
wait of 100 ms takes that to 500ms. In Firebird 2, with the same
transaction load of 60/second, there will be 24 transaction starting or
finishing during the 400 ms period generating 48 avoidable page writes.
It is certain that the write times for those 48 writes will induce a
delay of much more than 100 ms.


[Non-text portions of this message have been removed]