Subject Re: [firebird-support] performance problem after "large" deletes (bug?)
Author Helen Borrie
At 01:29 PM 22/11/2004 +0100, you wrote:

>hm, ok i just tried that .. and got some interesting results:
>
>on the fbsd box:
>
>running gfix -sweep just after the delete magically finishes withing
>10 seconds and after that the select also finishes within 8 secs.
>the gfix -sweep here also looks alot more "healthier" as it really
>produces a lot of physical disk i/o .. and almost no cpu load.
>
>great :-)
>
>but on the linux box (unfortunately the production server)
>
>gfix -sweep runs and runs and runs (and it looks like as it will
>again be running for the mysterious 50000 secs) ... :-(
>
>again almost no disk activity, just 100% cpu .. and ofcourse a select
>in parallel (started after the gfix was started) also never returns ..

At least you can deduce the problem is environmental. Check that you're
running an appropriate version of the server for your Linux release. The
latest kernel versions need the NPTL kits (or to have the NPTL disabled).


>but, even if we just focus on the fbsd box ...
>
>it does not explain why the garbage collection, initiated
>by a client's select statement, takes >50000 seconds?!?
>(with 100% cpu; i'd think of garbage collection as a disk i/o
> bound process e.g. not constantly 100% cpu)

GC does do disk i/o but it also does a lot of stuff in memory and uses CPU
-- but not 100% of CPU. Something else is causing that and I'd put my
money on threading library incompatibilty. GC operates on the transaction
status bitmap (a memory structure) and the transaction inventory pages,
both in memory and on disk.


>so i still think we must be hitting a bug or somethin ...

somethin....


>did you look at the stats from
>my last select? don't they look a bit "strange" for such a small
>database?

Not after mega-deletes in big transactions. Batch operations are managed
better if you hard-commit about every 8-10,000 rows.


>hm, can i somehow disable that garbage-collection stuff on selects?

Not currently...in fact, if you are using Superserver, the GC runs as a
background thread and users don't clean up other users' garbage. There are
alternative options coming in Fb 2.

>setting sweep intervall to "0" does not seem to make a difference
>again results in a 50000 seconds "wait" on the first select ...

Sweep interval isn't a time interval, it's a threshold. Setting it to 0
disables auto-sweeping; but auto-sweeping is never going to happen in the
scenario you're running.

Sweeping and GC aren't the same. Sweep does what GC does and more things
besides.

This absolutely is my last support response today (or should I say
"yesterday").

bye
./hb