Phillip,

> As to disks: large-capacity solid-state disks are coming onto the

market

> nowadays; they're still expensive, and they're certainly not as fast

as

> actual RAM, but they're still a marked improvement over traditional
> platters. Is there a proof that 'durable' media will always be the
> bottleneck?

Nope!

There are a growing number of solutions which pushing the performance of
'durable' storage.

In addition to the commonly known SSDs, there are enterprise SSD
solutions like the ZeusIOPS Solid State Drive which provide 20X
performance improvements
(http://www.stec-inc.com/technology/ssd_performance.php).

Then, there are a number of products like Fusion-IO ioDrive
(http://www.fusionio.com/products.aspx) and Violin Memory
(http://www.violin-memory.com/) (in the case of Fusion-IO, for a very
reasonable price point ($30 USD/GB -- 80GB for $2400)

In addition to these hardware solutions/advancements, there are also a
large number of solutions which aim at improving disk performance and
replication, so that read-only replication would not be something that
software (of any type) need be responsible for -- it would be handled at
the OS/device level.

These solutions include SANs (EqualLogic), WAN file systems (Panasas),
as well as Ramback ["...a new virtual device with the ability to back a
ramdisk
by a real disk, obtaining the performance level of a ramdisk but with
the durability of a hard disk..."] (http://lwn.net/Articles/272534/)

All of these solutions significantly raise the storage performance bar!


This combined with the fact that CPU speed increases have are now
limited due to power/HVAC issues in data centers; the speed of a single
thread (and therefore a database connection) has reached a plateau.

Storage performance is increasing at a much faster rate than CPU, such
that I believe that within 3 years, at the enterprise level, it will no
lower be a limitation for database operations. (Consider that within
the last 2 months Intel and Micron jointly announced that they had
developed a new NAND flash chip which has twice the performance of
current chips)


If we want to talk about where things are heading...

The current big push in computing is the integration of GPU (Graphics
Processor Unit) into standard computing languages/models. A GPU can
perform some operations 100X faster than a CPU.

This push is due to the fact that recent AMD/ATI and nVidia graphics
cards have GPUs which can be used for this computing. So, now these
commodity components can be leveraged -- no, special hardware is
required. (although there is some *very cool* GPU hardware available
from AMD and nVidia for HPC purposes, AMD FireStream 9250 and Telsa T10P
which operate at 200 gigaflops and 1 *teraflops* respectively)

I would suggest that finding GPU programming techniques/algorithms which
could be used for database operations (sorting, index 'folding') would
have a significant larger benefit for engine/database performance.

Unfortunately, GPU programming is nothing like traditional linear
programming because of the specialized functionality of GPUs.


Sean