Subject | Re: [IB-Architect] GBAK processing |
---|---|
Author | Markus Kemper |
Post date | 2000-04-26T18:04:33Z |
Sean,
Not looking at the code, I believe that it is single
threaded. From what I've seen with customers, the
most painful part of GBAK is time to recovery from
a failure/corruption. Time to recovery being either:
Restore from a Backup
(gbak -c ...)
Backup (corrupted db) and Restore
(gbak -b ...)
(gbak -c ...)
Any way you slice it this process gets more painful
as the database grows thus, I suspect that a different
strategy will need to be considered for really large
databases.
We've bounced around a couple ideas as to how we might
speed up gbak as it is today.
1) Add functionality to gbak to create a virtual
'backup' file. You can do this with UNIX/Linux
today outside of the gbak program. You might be
able to do it on Windows too if you have something
like MKSToolkit but, I have not tried it.
This syntax is from memory not testing today, so
please forgive me if I am incorrect but, the idea
is to write the backup file to a pipe and read
from it in the background thus, performing the
two step operation in one.
mknod gpipe
gbak -c gpipe new_database.gdb &
gbak -b old_database.gdb gpipe
2) Creating indexes is a large part of the expense
in restoring a database. Currently a full table
scan is required for each index even if they are
on the same table. Meaning that
create table foo (
f1 integer,
f2 integer )
create index idx_1 on foo ( f1 )
create index idx_2 on foo ( f2 )
would cause two full table scans to build the
indexes during a restore. The idea was to modify
gbak to build multiple indexes at a time, thus
only having to do one full scan per table when
building database indexes.
Markus
Not looking at the code, I believe that it is single
threaded. From what I've seen with customers, the
most painful part of GBAK is time to recovery from
a failure/corruption. Time to recovery being either:
Restore from a Backup
(gbak -c ...)
Backup (corrupted db) and Restore
(gbak -b ...)
(gbak -c ...)
Any way you slice it this process gets more painful
as the database grows thus, I suspect that a different
strategy will need to be considered for really large
databases.
We've bounced around a couple ideas as to how we might
speed up gbak as it is today.
1) Add functionality to gbak to create a virtual
'backup' file. You can do this with UNIX/Linux
today outside of the gbak program. You might be
able to do it on Windows too if you have something
like MKSToolkit but, I have not tried it.
This syntax is from memory not testing today, so
please forgive me if I am incorrect but, the idea
is to write the backup file to a pipe and read
from it in the background thus, performing the
two step operation in one.
mknod gpipe
gbak -c gpipe new_database.gdb &
gbak -b old_database.gdb gpipe
2) Creating indexes is a large part of the expense
in restoring a database. Currently a full table
scan is required for each index even if they are
on the same table. Meaning that
create table foo (
f1 integer,
f2 integer )
create index idx_1 on foo ( f1 )
create index idx_2 on foo ( f2 )
would cause two full table scans to build the
indexes during a restore. The idea was to modify
gbak to build multiple indexes at a time, thus
only having to do one full scan per table when
building database indexes.
Markus
> Is GBAK (the backup process) a single threaded function, performing 1
> task at a time, or is it multi-threaded?
>
> Since I think the answer is that it is single threaded, I'll ask a
> couple of other questions.
>
> Would a multi-threaded backup process perform faster then the current
> model?
>
> What issue would prevent (or made very difficult) GBAK being made
> multi-threaded?
>
> I been thinking about a different approach to GBAK where it creates
> treads (up to a certain limit) for each DB object to be backed up, with
> these threads reading the DB information and building "backup
> information" to be written out by a single "writer thread". I
> understand that the system structure would need to be processed by a
> single thread and written to disk before any application data.
>
> Sean
>
> ------------------------------------------------------------------------
> Join Garden.com's affiliate program and enjoy numerous benefits.
> To learn more click here:
> http://click.egroups.com/1/2753/3/_/830676/_/956769935/
> ------------------------------------------------------------------------
>
> To unsubscribe from this group, send an email to:
> IB-Architect-unsubscribe@onelist.com