Maurice Ling wrote:

> Oldest active 1569536
> Next transaction 1584914
> Sweep interval: 20000

This should tell almost everything. AFAIK, you are reaching a gap of
20000 records and probably the sweep kicks in, and keeps running in
background even when Python does not query anything.

To confirm that this is the problem, you can turn sweep off. If it
really turns out that way, you'll need to take care about transactions:
when to start, when to commit, what type of transaction level to use in
different parts of application.

I never encountered problems you're having, but difference between OAT
and NT never gets higher than 100 in mine databases.

> 3 concurrent running scripts does the following:
>
> script A: loads new data into PMC_* tables, process raw text and
> inserts results into TEXT_REPLACE_ABSTRACT and
> TEXT_REPLACE_ABSTRACT_UPRO tables. Does that 300 times. Then do a
> select * from TEXT_REPLACE_ABSTRACT_UPRO, process each record, then
> insert the results into ML_SVO table and delete the processed record
> from TEXT_REPLACE_ABSTRACT_UPRO table. [this script is a pure CPU
> eater. FB process called from this runs at 99.7% without any drop in
> intensity.]

Ok, now copy/paste this text and add the places where you start and end
transactions.

> script B: select <primary key> from TEXT_REPLACE_ABSTRACT table and
> script C: select * from NAME_HUNT_WAITING table (1 field but 870k

For these too.

> What I find interesting is this...... When ran non-concurrently, FB
> process uses >96% CPU after after about 36-42 hrs (script B and C will
> take about 10 weeks to run), FB seems to behave itself and no longer
> using that much CPU.

P.S. I missed the beginning of thread, but I sure hope you're using
Classic Server for all this.

--
Milan Babuskov
http://www.flamerobin.org