> > For each record imported there can be up to 75 'lookups'
> > and/or insertions,
> > and up to 50 calculations, all done in stored procedures.
>
> Have you profiled the performance of your import SP (indexes, etc) to
> see if some optimization can be made?
Yes, it has problably had about 4 man-years development time on it <g>
> As someone commented -- 25-30 hours to import 200Mb seems... excessive!
Thats 200Mb of _new_ data, there are lots of updats and 'summing' going on.
It does work better than expected, on a decent Linux/Solaris box, we get a
days worth of data in in arounmd 3-4 hours, on Windows it is closer to 8,
but still acceptable. If it wasn't fot the flush problem and having to turn
Forced writes ON, I would not have know it would have taken 25-30 hours!
> Personally, I would rather you really, really optimized the import
> routine and always ran with Forced Writes = On (on windows) then depend
> on closing the DB connection to prompt Windows to flush the pending disk
> changes.
The inport process is not perfect, but improving it is not going to gain that
much, aprt from a millisecond here and a millisecond there. Disconnecting
and reconning the client app every 12 hours is an acceptable swork around in
this case.
Phil
--
Linux 2.4.4-4GB
11:04pm up 2 days, 11:58, 1 user, load average: 0.70, 0.41, 0.25