Nickolay Samofatov wrote:

>[... snip ...]
>
>
>Yes. There will be minimum contention when we will be adding multithreading
>to FB2.
>I have a plan how to make CS version multithreaded. It includes usage of
>thread-local
>memory pool. It will reduce thread contention to a very small amount.
>
>

I like Nickolay's plan, and it mirrors my experience with converting
single threaded apps to multithreaded apps. The approach usually
minimises locking requirements, and works out fairly clean.

Much as Ann pointed out you could merge classic into a multiprocessor
model by making each thread run as a seperate classic. Each thread
starts out having completely their own data, cache etc. Then
selectively move access to objects into shared (and synchronized)
resources. This process has worked successfully (for me) in several
projects in moving single threaded apps (often a service) to a multi
thread server.

But I admit, I've been thinking much as Jim has pointed out, that a db
server differs from a lot of applications in that although each thread
does have individual data, that at it's core, the whole application
shares one huge data source, against which the fine grain locking may be
a better alternative. (I don't have a lot of exposure to the fine
grained locking/threading models)

I've had a bit to do with linux clustering, and keeping distributed
cluster processing (as Paul suggested) is also worth keeping in mind,
although that has other problems in addition to communication between
threads, since it will require sharing/duplicating/splitting disk
database image over the nodes as well - but it's worth keeping in mind.

Im also mindful of the state of the current code, and the workload that
would be needed to do fine grained locking or other major changes correctly.

I think that Nickolay's idea has some promise, as he suggested, classic
can already do multiprocessing, it has a migration path that would work
at all stages. I suspect that eventually it could "talk to" or "be
integrated with" a module that used a fine grained threading/locking
model to access the large shared data source as well.

And importantly, he's willing to work on it, and an upgrade to classic
ain't going to hurt, and would likely also identify problems with super
server as well.

So the best answer I see is going to be to try a few prototypes', see
how they perform, and if a threaded classic does show real promises (as
I think it will) and doesn't impact much on super, then we can slip it
in as an upgrade.

At this stage however, (apart from just getting things stablised for
fb1.5) even just a few test scripts or programs, that exercise/load test
the engine in a way useful for determining multiprocessor
response/problems would be handy.

Now the copout clause:

Since I've only glanced at the threading code enough to see that I
though super server multithreading was a cludge that I didn't like too
much - I judge others, Mike, John, Jim (Im not sure if he's looked at
the current super code) Dmitry and Nickolay as having more weight in
this debate.

Just my 2c worth.

Cheers

Mark

PS: When ready I now have access to a linux cluster of 64 1.2GHz
processors (32 dual processors), associated with my real paying work,
and I would love to test a distributed fb server on it, but sadly it'll
take a long time till fb is in that class.