>> Which event in this case you want to post when user changes its team
>> without sending user and/or team id so that cache does not poll the
>> complete user/team table but only the changes?
>>
> table team_change_log(posted timestamp, team_id int, player_id int,
> change_type int)

Ok, the application can then read the changes log. Though requires a
separate table for each event type we want to post. Quite an "overhead".

> We've had this discussion about a million times. It always terminates
> with "oh, I didn't think of that", but it's usually an informative,
> educational sort of discussion. Shall we begin?

Sure. As you note, they are informative and educational. You know about
system design more than me, so I'm all ears.

> Starting point: Events must be robost and reliable, meaning specifically
> that event notifications must be delivered, even if they occur at
> inconvenient points such as notification or processing of the prior
> event of the same name.

This is how messaging, at least JMS messaging works. Application posts a
message, messaging subsystem ensures that messages are delivered reliably to
one or all registered listeners (depends whether we talk about queues or
topics). Message listener is called from another thread. In EJB that means
that separate thread is also an independent transaction context. Delivery
can be transactional or not (in case of transactional delivery, message is
removed from queue only when message listener successfully finishes
processing).

So, till now events are like other messages in the J2EE environment where
messages are posted to a topic (multicast delivery).

> Futhermore, due to network latency and transaction atomicity, any
> database state other than "something happened" is necessarily stale by
> the time an event can be delivered and a new transaction started.

This means that application must be designed so, that it first starts the
transaction, then inspects the database state, the makes the decision what
to do, then commits its transaction. Again agree.

However, where's the difference between reading the changes log and taking
care of the last state (assuming that change history itself is not
important) and reading the final value from the original table? (If the
change history would be important, we would design the entity to have a
version timestamp anyway).

Back to our example.

What would be wrong with the design when the event contains ID of the
team/user that was modified? Instead of reading the complete team/user
change log application inspects exactly the team/user that was changed.
Instead of "something happened" we send a "something happened to user 123"
event.

> Many things could have happened in that time including, potentially,
> intervening transactions that may mask or invalidate data associated with
> an event.

True, it would be wrong approach to rely only on data contained in the event
to modify the table.

However, history table does not solve that issue too, does it? It is the
application that has to respect the last recorded entry in the log.

Can you give an example where containing data in the event would be harmful,
but storing the same data in log table would be the right solution?

> We call that middle tier the database, which exists to manage data and
> transaction consistency. I don't believe this can be done in a
> different tier, and that we're better off looking at the real problem
> rather than trying to make a bad workaround (a middle tier) less bad.
> We should be looking for solutions to obviate the need for a middle
> tier, not fix it.

I think you are right here again. However, if middle-tier is a data manager
(EJB layer) and transaction coordinator (JTA layer), then it is viable
solution as long as it is the only data and transaction manager (and does
not compete with database). And in many cases it is much more easier to
invalidate the the complete object graph in the middle-tier then to design
the same via the events ("easier" means the amount of code needed).

And I suspect that middle-tier being data and tx manager is quite common
setup in many applications, at least two enterprise-class applications which
I co-design/develop (insurance company automation), work this way - data are
modified through middle tier only - there is no current requirement to do
this other way and I suspect that all extensions will go that way too. And
messages that are floating in the system contain not only the "something
happened" events, but also "something happened with entity 1" events. So far
I do not have any problem with that design (there are other problems, but
mostly those are issues with the UI components). The third application of
the same class shares database with out world, but they have clean
separation of the responsibilities - no conflict there too.

So, seeing your reply, I wonder, what's wrong with the design where data
needed for information processing are send within the event itself instead
of writing them into the table in the database?

I will think about this issue more, since I'm not yet convinced that there
should be only "something happened" events in the system, while you seems
to know that for sure. But I would be grateful for any example that would
help my understanding.

Roman