Subject RE: [IBO] How to improve bulk insert
Author Jason Wharton
Keep in mind if you do a Commit it could cause your source cursor to close.
My understanding of Firebird/InterBase is that these incremental commits
should not be necessary. The only thing that would be painful is if you did
lots of them and then had to do a rollback. Otherwise, it is optimistic and
a commit is simply an adjustment to the transaction and all the record
versions stay just as they are. In which case, why do incremental commits
along the way?

Jason

> -----Original Message-----
> From: IBObjects@yahoogroups.com [mailto:IBObjects@yahoogroups.com]On
> Behalf Of henry FRANQUET
> Sent: Monday, January 23, 2006 3:42 PM
> To: IBObjects@yahoogroups.com
> Subject: Re: [IBO] How to improve bulk insert
>
>
> --- In IBObjects@yahoogroups.com, Aage Johansen <aagjohan@o...> wrote:
> >
> > henry FRANQUET wrote:
> > > Hi,
> > > I use TIB_DataPump and IB_cursor to insert data into a
> database (a
> > > empty one created with backup and restore only metadata).
> > > As I have Tables with about one hundrer thousand
> records, I perform a
> > > commit retainning on after execute event.
> ---snip ---
> > > How can I improve inserting ? Is there a counter I could use, for
> > > doing a commit every 1000 inserts ?
> > >
> > > I have seen in forums, somebody's speaking of inserting
> thousand's
> > > insert a second, but even when doing insert without
> IB_DataPump and
> > > committing every 1000 insert, I can't go further 400
> inserts a second
> > > with 2 fields table. Should I use virtual disk for
> temporary files,
> > > disable forced write and enable it after inserts done ?
> > >
> >
> > It depends on several factors: hardware (cpu/disk).
>
> Yes, I have seen than the size of the cache of the disk is critical.
> And disabling Forced write, just increased the speed by a factor of 2
>
> > Aslo, are records
> > produced by a fast system?, do you prepare more than once?,
>
> No TIB_Datapump does the job
>
> > are there triggers?, any indexes to be maintained?
>
> None, I have disabled all triggers and indexes
>
> > I haven't used the Datapump, but TIB_DSQL is quite fast when you're
> doing
> > simple inserts (and otherwise, as well).
>
> I'll try to replace my TIB_Cursor with a TIB_DSQL
>
> > I wouldn't worry if commits were done every 10.000 records (or even
> less
> > frequently).
> It's done after inserting every table, and so about 50.000 to 100.000
> records. I have tried to commit on the event AfterExecuteItems, every
> 5000 records, but I get an error witch I can't explain
> > A few thousand inserts a second is not an ambitious goal. Even on
> > equipment that isn't "state of the art".
> Yes, but even with forcedWrite off, I hardly get one thousand.
> >
> > --
> > Aage J.
> Thanks for your answer
>