Subject Re: Import of > 1 mio. records
Author dwilson805
--- In firebird-support@yahoogroups.com, "thomas_besand"
<thomas.besand@w...> wrote:
>
>
> Hi everybody,
>
> I'm developing a database application, which will administer several
> hundred tables with up to 1.5 mio records of text data each.
> Right now I'm working on the import side of the beast.
> I noticed that in a long running import routine the firebird
> server(1.5.2.4731) is getting slower and slower until it virtually
> grinds to a standstill.
> Also Taskmanager (Win2k) shows it's eating up more and more memory.
> Take a look at this excerpt from a log file, I'm writing during import
> (the HEARTBEAT item is written every 10,000 records):
>

Thomas,

It sounds like you aren't committing the transaction periodically.
Try issuing a commit every 10,000 to 15,000 records. Your transaction
flow should look something like this:

start transaction
import 15,000 records
commit transaction
start transaction
import 15,000 records
commit transaction

Make sure you use a real hard-commit: don't use commit-retaining.

HTH,

Dan.