"If you take a look at the comments on the ServerSide article on
Coherence which URL I droped here you will find an interesting
discussion about de pros and cons (mostly cons) of fat data nodes."

Thank you, I've gone through the comments.

For clarity, I was not advocating "fat data nodes". My point was that
if data migrates to many nodes under the expected workload you will
end up with great read performance and poor write performance,
similar to a mysql master/slave farm. You might implement a clever
multicast fused cache type write replication, but even then write
scaling beyond 50 nodes will probably be poor.

Note that the whole discussion is based on assumed data locality. If
you have to go through the data in a near random way, it seems nobody
has found a solution other than scaling up. See for instance message
203411 in that ServerSide thread or the earlier link about the
LinkedIn architecture. Locality seems a safe bet in most real world
workloads though.

The dynamic partitioning of Coherence, combined with keeping data on
a limited number of nodes seems to have proven to scale well, at
least up to 100 nodes if I can believe the ServerSide thread.

Paul