Subject | Re: R: [Firebird-Architect] Well, here we go again |
---|---|
Author | Jim Starkey |
Post date | 2008-06-16T15:27:52Z |
Paolo Fainelli wrote:
cluster databases that grew up to power cell phone switches. It looks
like a close relative to the MySQL ndb storage engine (also from
Ericsson) and the Clustra product bought and disappeared by Sun. All of
these are designed to do ultra-relable, extremely fast single key
lookups. All are primarily in-memory databases with persistent disk
storage as an afterthought and/or option. Experience with both Clustra
and ndb has shown that they translate very poorly into general database
applications.
So, no, Nimbus isn't anything Mnesia at all. Different market,
different technology, different everything.
There is a strong trend in cloud computing away from relational
databases, incidentally. Google's BigTable, Amazon's SimpleDB, and
Microsoft's unannounced cloud database service all abandon the
relational model for different single table, complex record models. The
motivation is simple: Existing relational products are designed around
single node technology that has neither the reliability nor scalability
these guys require. Making a single row update ACID in a cloud is a
more tractable problem than making arbitrary ACID.
I think we all know what's wrong with the single table, complex row
model. The challenge today is to figure out how to make a relational
(or semantic) database run in a cloud, be fully ACID, and scale to very
large capacity.
The arithmetic is encouraging. Both Falcon and InnoDB can pump in the
neighborhood of 30,000 DBT2 transaction per minute on a four processor
system. Call that X. If you eliminate the logging and disk writes
required for ACID, these systems (at least Falcon) would run somewhere
between 2X and 4X, but wouldn't be ACID. If you substitute different
structure below the SQL engine that uses network replication rather than
a disk, things get more interesting. There will aways be a point of
diminishing returns where the delta increase in replication cost cancels
the benefit of an additional node. But if this point happens at a
hundred nodes, we will still see a 100X gain in performance over a
single node system. And as the number of cores and hardware threads
increase, the point of diminishing will continue to grow. My guestimate
is that the limit for clouds made of four code machines is probably
100X, but a system made with 16 core, 8 hardware threads per core plus a
higher speed hardware interconnect should be able to reach 1000X without
much trouble.
--
James A. Starkey
President, NimbusDB, Inc.
978 526-1376
> Something like Mnesia ??No, not at all. Mnesia comes from a strange bunch of Scandinavian
>
>
>
> http://www.erlang.org/documentation/doc-5.0.1/lib/mnesia-3.9.2/doc/index
> .html
>
>
>
cluster databases that grew up to power cell phone switches. It looks
like a close relative to the MySQL ndb storage engine (also from
Ericsson) and the Clustra product bought and disappeared by Sun. All of
these are designed to do ultra-relable, extremely fast single key
lookups. All are primarily in-memory databases with persistent disk
storage as an afterthought and/or option. Experience with both Clustra
and ndb has shown that they translate very poorly into general database
applications.
So, no, Nimbus isn't anything Mnesia at all. Different market,
different technology, different everything.
There is a strong trend in cloud computing away from relational
databases, incidentally. Google's BigTable, Amazon's SimpleDB, and
Microsoft's unannounced cloud database service all abandon the
relational model for different single table, complex record models. The
motivation is simple: Existing relational products are designed around
single node technology that has neither the reliability nor scalability
these guys require. Making a single row update ACID in a cloud is a
more tractable problem than making arbitrary ACID.
I think we all know what's wrong with the single table, complex row
model. The challenge today is to figure out how to make a relational
(or semantic) database run in a cloud, be fully ACID, and scale to very
large capacity.
The arithmetic is encouraging. Both Falcon and InnoDB can pump in the
neighborhood of 30,000 DBT2 transaction per minute on a four processor
system. Call that X. If you eliminate the logging and disk writes
required for ACID, these systems (at least Falcon) would run somewhere
between 2X and 4X, but wouldn't be ACID. If you substitute different
structure below the SQL engine that uses network replication rather than
a disk, things get more interesting. There will aways be a point of
diminishing returns where the delta increase in replication cost cancels
the benefit of an additional node. But if this point happens at a
hundred nodes, we will still see a 100X gain in performance over a
single node system. And as the number of cores and hardware threads
increase, the point of diminishing will continue to grow. My guestimate
is that the limit for clouds made of four code machines is probably
100X, but a system made with 16 core, 8 hardware threads per core plus a
higher speed hardware interconnect should be able to reach 1000X without
much trouble.
--
James A. Starkey
President, NimbusDB, Inc.
978 526-1376