Subject Re: [ib-support] When do FB/IB 'flush' to disk?
Author Phil Shrimpton
On Friday 30 November 2001 17:06, you wrote:


> > For each record imported there can be up to 75 'lookups'
> > and/or insertions,
> > and up to 50 calculations, all done in stored procedures.
> Have you profiled the performance of your import SP (indexes, etc) to
> see if some optimization can be made?

Yes, it has problably had about 4 man-years development time on it <g>

> As someone commented -- 25-30 hours to import 200Mb seems... excessive!

Thats 200Mb of _new_ data, there are lots of updats and 'summing' going on.

It does work better than expected, on a decent Linux/Solaris box, we get a
days worth of data in in arounmd 3-4 hours, on Windows it is closer to 8,
but still acceptable. If it wasn't fot the flush problem and having to turn
Forced writes ON, I would not have know it would have taken 25-30 hours!

> Personally, I would rather you really, really optimized the import
> routine and always ran with Forced Writes = On (on windows) then depend
> on closing the DB connection to prompt Windows to flush the pending disk
> changes.

The inport process is not perfect, but improving it is not going to gain that
much, aprt from a millisecond here and a millisecond there. Disconnecting
and reconning the client app every 12 hours is an acceptable swork around in
this case.

Linux 2.4.4-4GB
11:04pm up 2 days, 11:58, 1 user, load average: 0.70, 0.41, 0.25