Bob et al,

> I have a Firebird 1.5 database that's been in production for about
> four years (started with IB5, moved to IB6, moved to FB). The
> database size at this point is 16.5GB. The applications that use this
> database have morphed from the original inception as data reporting
> and analysis tools to interactive systems with direct financial impact
> to the business.

...

> I'm wondering how the 'big boys' handle their 100GB - Terabyte
> databases? Are they throwing more hardware at the problem as the size
> increases? Is there something I am missing?

The 'big boys' use DB engines which support image and incremental backup
approaches.

The new NBackup utility to be included in the Firebird v2.0 release adds
a database page level backup mechanism which implements support for both
of these requirements.

The image backup mode, as the name implies, allows for a complete
database image to be made while the database is operational. It runs
FAST! (I just tried it on a slow server -- P3 1.4GHZ with older/slow IDE
drive -- it took 80 sec to make a backup of a 1.4GB database).
Restoring the database couldn't be easier; simply copy the backup on top
of the 'live' (now bad) database or delete bad database and rename
backup to name of 'live' database.

The incremental backup mode allows for the user to implement a backup
strategy which supports multiple incremental backup levels (hourly,
daily, weekly, monthly, etc...). The backup files are not database
images but collections of the database pages which have been changed
since the last incremental backup of the same level.

Again, this runs really FAST! Remember: in most large databases only a
small portion of the data changes daily/weekly -- as most of the data is
historical in nature. So, the incremental backup would have to read all
the database pages but only write-off to the backup the changes, which
would mean that a backup would run easily 2-3x faster than a full
backup.

Naturally, with incremental backup the restore process is a little more
involved, and it requires using the NBackup utility to correctly restore
the database pages.

NBackup, due to the way the image backup works, will also allow you to
'lock' the database and allow you to take an online backup of the
database using your favorite backup utility or use the OS copy file
command, and then 'unlock' the database backup to return normal
operation.


Sean