"Put data doesn't particularly live anywhere specific. It can have
many locations synchronized with replication or it can be on a disk
attached to an archive node (and since the archive node doesn't do SQL,
it has to move somewhere else to be processed)."

Replicate too much and scalability collapses.

"What you really want to do is to direct multiple requests for the same
data to one of a smallish subset of nodes. After a short while, those
nodes will have most of what they need to handle the requests directed
at them."

This in effect is the node partioning I referred to. Also note that the
one issue high scalability engineers run into is the cost of joins in
web scale systems (many advocate denormalising data to deal with it).

"I'd go so far as to say counter productive."

Indeed and that sets the issue.

Paul