Subject Re: [IBO] Pumping data uses a lot of memory
Author Dmitry Beloshistov
Hello, williamvdw2005!
You wrote to <IBObjects@yahoogroups.com> on Mon, 05 Dec 2005 13:59:18 -0000:

w> I am pumping about a million records from one database to another
w> using IB_Datapump. This process is using a lot of memory causing my
w> server to run out of virtual memory.

I`m use external tools for data dump/pump - IBEScript from IBExpert suite.
One of solution - use own writed utility for dump and pump data by blocks
(1000-10000 records in one records portion (block) per one transaction
between start/commit.)
Simple algorithm:
1) Make on-disk based SQL file for pump.
2) Read portions of records for pumping from this file into IB_Script
3) Start transaction
4) Execute script (IB_Sctipt.Execute)
5) Commit transaction
6) Repeat step 2 (while not EOF in input file)
In this case
1) server commit only one portion of records in each transaction commit
time.
2) we have only one portion of records in memory

WBR,Dmitry Beloshistov AKA [-=BDS=-]