Paul,

> The way I understand it, in the 2005 refinement the buffer role is taken
> on by the snapshot. Both T1 and T2 will see their own update, but not the
> other before commit. When T1 and T2 want commit they broadcast the update
> message (also to themselves), but do not yet commit locally. Only upon
> receipt of the message back the local commit will happen.
>
> Node A and B receive T2 first. Node A will do nothing yet, just queue the
> message. Node B, seeing that T2 is a local transaction waiting to commit
> will now commit locally and it will succeed. It broadcasts an (unordered)
> "commit ok" message. Upon receipt of that message, Node A will also apply
> and commit the pending T2 message (which will always succeed).
>
> T1 is delivered next on both node A and B. Node B will do nothing yet,
> just as node A did for T2 above. Node A again sees that T1 is a pending
> local transaction and attempts to commit. It will fail, as committed T2 has
> updated the record after T1 began. It broadcasts an (unordered) "commit
> abort" message. Upon receipt of that message, Node B will discard message
> T1.

Ah, right... Now I see an error in my thinking - I was thinking that T2 will
get a lock conflict on update on Node A, but you're right - it will get conflict
only if T1 commits first. Right. Ok, then this approach is clever and works :)

> I think you are right on total order being required, but I'm not 100% sure
> yet that Lamport's (partial) causal order cannot be made to work. Whether
> this is worth considering depends on the additional communication needed to
> move from causal to total. If the extra network overhead is low, it is not
> worth doing -- assuming causal would work at all.

Can check JGroups about the total order overhead, since I have no experience
with Spread.

> It is my understanding that Spread can totally order 300..500 messages per
> second on a 100Mpbs lan, and perhaps 3,000 on a 1000Mbps lan. Assuming that
> the average business app user generates 1 update *transaction* per minute,
> the total order design can support nearly 200,000 concurrent users
> (assuming no other capacity constraints exist). This is well in excess of
> the load on a single Salesforce.com instance, so perhaps relaxing the
> ordering level is not worth thinking about.

I guess that highly depends on the size of the message, I would expect a higher
throughput. Will come back with numbers later.

Roman