| Subject | Re: [IB-Architect] Next ODS change (was: System table change) |
|---|---|
| Author | Jason Wharton |
| Post date | 2000-11-15T19:24:06Z |
Dalton,
What was the average CPU utilization during these long term data crunching
jobs?
Do you think you were I/O bound or CPU bound?
My *guess* is that in today's hardware scenario we can afford to expand in
the computational demands of maintain a database. Ideally this shouldn't
purchase less I/O but the ability to do other things previously discounted
as too "expensive" to return significant value.
One example would be to store the TID (transaction ID) of the record when it
was inserted and maintain it permanently in order to have persistent
bookmarks without having to hold up garbage collection. This does increase
the size of the record stored on disk, increasing the I/O to deal with the
record, but I think there are some ways to make it up.
With this foundation of a persistent DB_KEY in place, it would be possible
to add in MANY very useful capabilities to the engine. (I'd rather think of
them as sockets or modules that are pluggable as we don't want to bloat the
core engine.)
One example would be full text indexes could be maintained in a way that
would reduce the amount of "linkage" data to be stored. With a persistent
bookmark available there wouldn't have to be a surrogate key striped all
throughout the FTS index. Instead it would have a direct shot at the DB_KEY
and make it perform beautifully.
Other things like replication could be introduced more reliably at the
server/engine level with this type of foundation in place. In short, it
would be possible to have a clustered array of parallel databases if a group
of servers could be setup in a high-speed, row level, replication.
Call it IB Enterprise...
You need full text searching? Plug it in.
You need clustered replication among an array of servers for fail-over and
load balancing? Plug it in...
You need day-to-day incremental and differential backups? Plug it in.
Perhaps only dreaming...
FWIW,
Jason Wharton
CPS - Mesa AZ
http://www.ibobjects.com
What was the average CPU utilization during these long term data crunching
jobs?
Do you think you were I/O bound or CPU bound?
My *guess* is that in today's hardware scenario we can afford to expand in
the computational demands of maintain a database. Ideally this shouldn't
purchase less I/O but the ability to do other things previously discounted
as too "expensive" to return significant value.
One example would be to store the TID (transaction ID) of the record when it
was inserted and maintain it permanently in order to have persistent
bookmarks without having to hold up garbage collection. This does increase
the size of the record stored on disk, increasing the I/O to deal with the
record, but I think there are some ways to make it up.
With this foundation of a persistent DB_KEY in place, it would be possible
to add in MANY very useful capabilities to the engine. (I'd rather think of
them as sockets or modules that are pluggable as we don't want to bloat the
core engine.)
One example would be full text indexes could be maintained in a way that
would reduce the amount of "linkage" data to be stored. With a persistent
bookmark available there wouldn't have to be a surrogate key striped all
throughout the FTS index. Instead it would have a direct shot at the DB_KEY
and make it perform beautifully.
Other things like replication could be introduced more reliably at the
server/engine level with this type of foundation in place. In short, it
would be possible to have a clustered array of parallel databases if a group
of servers could be setup in a high-speed, row level, replication.
Call it IB Enterprise...
You need full text searching? Plug it in.
You need clustered replication among an array of servers for fail-over and
load balancing? Plug it in...
You need day-to-day incremental and differential backups? Plug it in.
Perhaps only dreaming...
FWIW,
Jason Wharton
CPS - Mesa AZ
http://www.ibobjects.com
----- Original Message -----
From: "Dalton Calford" <dcalford@...>
To: <IB-Architect@egroups.com>
Sent: Wednesday, November 15, 2000 11:29 AM
Subject: Re: [IB-Architect] Next ODS change (was: System table change)
> Hi Jason,
>
[lots of interesting stuff snipped]