At 02:42 PM 10/15/02 -0700, John Bellardo wrote:

>
> As best I can determine all our major platforms support the ability to
>send sockets between processes, so this "load balancer" has a constant
>(close to single context switch) overhead.
>

Interesting. I hadn't run into that trick. It certainly
changes the analysis. Could you send some pointers on how
this is done in Linux and Win32?

Apache apparently forks with the listening socket bound,
then serializes the "accept" calls. The inline comments
state that non-serialized accepts are unsafe on all platforms.
What are the tradeoffs between this technique and sending
a connected socket?

Unless the code has been removed, classic servers can be
configured by a runtime switch to function as either a
single client (forked off by inetd or xinetd) or multi-client
(bind to 3050 and listen for connections) servers. Adding
code to maintain a pool of child servers to which incoming
connections are distributed sounds like a very interesting
and general idea.

>>
>> The engine currently uses the page locks issued by the cache
>> manager to synchronize access to pages within a process knowing
>> that a thread switch can't happen while a page is in use.
>
>I'm not 100% sure. But I am sure that no matter what direction we
>choose there will be some very tricky parts of the code that need to be
>modified. Here are two different approaches I can think of off the top
>of my head:
>
>1. two level locks - Each process is responsible for allocating its
>own locks internally to threads. But it must maintain an externally
>visible (to other processes) lock with sufficient rights to cover all
>of its inner-process locks.
>

This is probably the way to go. The semantic of page locks and
internal resource locks are quite different. External locks
require blocking ASTs and deadlock detection. Internal locks
are presumed deadlock free. The idea of siezing the lock table
semaphore for each synchronized thread resource.

Which brings to mind another problem. Lock grants and blocks
are communicated in classic by signals. Pthread semantics
require that signals be delivered to all threads. And, if
I remember correctly, none of the pthread synchronization
primitives are safe for use in signals. Any good ideas of
how to handle this problem?

A little historical context might be interesting here. Version
3 of Interbase was the first to support multi-threading. It
used by own thread manager for VMS and Apollo brand new (SR 8.2?)
threads. V3 was ready to release except for a very rare
hang under heavy load on Apollo. After a great deal of
exhaustive work we were able to point a definitive finger
at the Apollo thread manager. The problem was immediately
escalated to Apollo system architect. After studying the
problem for a few days he called back. Our problem was an
"unfixable" conflict in OS architecture between threads
and signal delivery. Our only choice was to rewrite the
lock manager and virtually everything associated with events
to eliminate all dependence on signals in multi-threaded
servers. The release was delayed by at least half a year.

There other examples of disasterous interactions between
threads and signals on Linux. Gdb, for examples, catches
signals before their delivery to process being debugged.
Pthreads, however, uses signals for thread wakeup. The
interaction consumes 100% of system resources when running
gdb with a multi-threaded program under load.

So be afraid of signals and threads. Be very afraid.

Jim Starkey