Subject | RE: [ib-support] %100 CPU usage with interbase, is it usual? |
---|---|
Author | Helen Borrie |
Post date | 2001-12-28T22:10:10Z |
At 05:53 PM 28-12-01 +0200, Kursat wrote:
You must provide more information!
If you are using IBOQuery and its Delete and Insert methods for bulk data loading then your approach is totally WRONG. Worse, I suspect you are using Autocommit which will be adding hugely to your overhead. I can't imagine what you are using IB_Script for !!!
You don't mention what the source of your input file is but you should either arrange to receive it in fixed-length format or write a script (e.g. using Perl, as David T. suggested) to convert it.
Then, all you have to do is declare an external table on the basis of your file's format and use a single DML statement to pump the data into another table (if that's what you need to do) or to call a stored procedure to process the external data directly.
With IBObjects, you simply need to use the TIB_DSQL component - NO DATASETS!! Use one to perform and commit the Deletes of old records first and another to fire off the statement you need to repopulate the table. You can use a TIB_SessionProps to control the batching of the input statements so that you can commit in batches of 10,000 records.
Alternatively, you can set up your own datapump using a TIB_Datapump...
regards,
Helen
>I have only one table, 3 field, and 1.000.000 record txt file, IKursat:
>read data from txt file with a while loop using IBODatabase, IBOQuery,
>IB_Script, first delete all the record, and then read the data one by one.
>insert into GDB and then post. thats all. PIII 650 my computer is, 192 RAM,
>WinNT
You must provide more information!
If you are using IBOQuery and its Delete and Insert methods for bulk data loading then your approach is totally WRONG. Worse, I suspect you are using Autocommit which will be adding hugely to your overhead. I can't imagine what you are using IB_Script for !!!
You don't mention what the source of your input file is but you should either arrange to receive it in fixed-length format or write a script (e.g. using Perl, as David T. suggested) to convert it.
Then, all you have to do is declare an external table on the basis of your file's format and use a single DML statement to pump the data into another table (if that's what you need to do) or to call a stored procedure to process the external data directly.
With IBObjects, you simply need to use the TIB_DSQL component - NO DATASETS!! Use one to perform and commit the Deletes of old records first and another to fire off the statement you need to repopulate the table. You can use a TIB_SessionProps to control the batching of the input statements so that you can commit in batches of 10,000 records.
Alternatively, you can set up your own datapump using a TIB_Datapump...
regards,
Helen