Subject | RE: [IBO] Conversion from dBase to Firebird using IBO |
---|---|
Author | Lindsay Gauton |
Post date | 2003-01-02T18:27:46Z |
One of my major systems is updating 3 million+ records everyday. The
only way to do it is via stored procedures - any other method would just
take too long
Thanks
Lindsay
-----Original Message-----
From: chris_michalec <Cmich22@...> [mailto:Cmich22@...]
Sent: 02 January 2003 15:50
To: IBObjects@yahoogroups.com
Subject: [IBO] Conversion from dBase to Firebird using IBO
I have a conversion utility that is taking data from dbase tables
into a firebird gdb. The table in question has about 300,000 records
and over 100 fields. I am using IB_Scripts and posting every couple
thousand records. Initially it starts off fast, but after about
100,000 records or so, it slows to a crawl. This puts the conversion
utility at finishing the table in about 3 days (which is just too
long). I have the scripts posting in their own transaction that I
create each time and commit. I have also tried committing the main
physical transaction (which in my understanding from reading the
documentation should actually be taken care of on it is own, but hey
I thought I'd try). I thought maybe it was the BDE trying to scroll
the something that huge, so I called FetchAll on the dataset to just
pull back everything. The machine this is running should have more
than enough resources to handle this. Any other thoughts out there
that maybe I haven't tried?
Thanks for the help
________________________________________________________________________
___
IB Objects - direct, complete, custom connectivity to Firebird or
InterBase
without the need for BDE, ODBC or any other layer.
________________________________________________________________________
___
http://www.ibobjects.com - your IBO community resource for Tech Info
papers,
keyword-searchable FAQ, community code contributions and more !
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
only way to do it is via stored procedures - any other method would just
take too long
Thanks
Lindsay
-----Original Message-----
From: chris_michalec <Cmich22@...> [mailto:Cmich22@...]
Sent: 02 January 2003 15:50
To: IBObjects@yahoogroups.com
Subject: [IBO] Conversion from dBase to Firebird using IBO
I have a conversion utility that is taking data from dbase tables
into a firebird gdb. The table in question has about 300,000 records
and over 100 fields. I am using IB_Scripts and posting every couple
thousand records. Initially it starts off fast, but after about
100,000 records or so, it slows to a crawl. This puts the conversion
utility at finishing the table in about 3 days (which is just too
long). I have the scripts posting in their own transaction that I
create each time and commit. I have also tried committing the main
physical transaction (which in my understanding from reading the
documentation should actually be taken care of on it is own, but hey
I thought I'd try). I thought maybe it was the BDE trying to scroll
the something that huge, so I called FetchAll on the dataset to just
pull back everything. The machine this is running should have more
than enough resources to handle this. Any other thoughts out there
that maybe I haven't tried?
Thanks for the help
________________________________________________________________________
___
IB Objects - direct, complete, custom connectivity to Firebird or
InterBase
without the need for BDE, ODBC or any other layer.
________________________________________________________________________
___
http://www.ibobjects.com - your IBO community resource for Tech Info
papers,
keyword-searchable FAQ, community code contributions and more !
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/