Subject Re[2]: [IBO] Stored procedure problem
Author Daniel Rail
Hi,

> Well, it's a long story. We've wrestled with it for 2+ years.

> We do massive imports on a regular basis. We process phone company data,
> millions of phone calls per month. We process for some 35 companies right
> now and growing. Each has there own database. Our largest database gets up
> to 3-4 GB of data per month. We keep the call data for 2 months and delete
> data older than 2 months. We do this spread out over 3 dual processor
> servers. FB can only use one processor in each though, of course.

Now that FB 1.5 classic server Windows version is out, you can use
dual processors(or any SMP configuration), since each connection is a
separate process. If you were running Linux, then the classic server
was already available. I know FB 1.5 is still in beta, but it's
getting stable. And, if I'm not mistaken, one or two of the
developers are already using it with production databases.

> This data has to be summarized and extracted for billing and reports each
> month, also. We have changed our apps to do this many times a month so that
> we don't have to do it all at once at the end.

> FB/IB (we've used both) can sometimes run away with the processor for no
> apparent reason. All of our sweep intervals are zero. We have tried
> letting it go to see if it will stop and sometimes it does. But there have
> been times when we have left it run all weekend and come in on Monday and it
> is still going. So we have to kill it.

I think that the garbage collection can still be performed even if
the sweep is set to zero. I don't remember quite the circumstances
when it happens.

> I've talked with support many times about the problem and have done all we
> know to do but it still happens. Although it has been less of a problem
> than it was. One thing we have found is that it occurs more regularly if we
> do deletes. For example, say we have not cleaned up one of the databases
> for awhile so we have to delete a 100,000 or more records at once. It
> appears to work just fine but we go to exit the app and it freezes up and FB
> is running away with the processor. We have done our commits, etc, at the
> end of processing so what is FB doing on close of the app. All we do there
> is database.close.

This is probably the garbage collection quicking in to clean up the
old record versions. In other words, it's doing the sweep when the
last connection disconnects from the database.

> We have changed to doing weekly backup and restores and
> only restoring the recent data. That works better but is a pain if you can
> imagine doing this for 30 companies each having a minimum of 3 GB of data.
> Takes quit a long time. We spread them out over the week but is still quite
> high maintenance for what is supposed to be a low maintenance system.

I see your point.

> We have also experienced database corruption now and then for no apparent
> reason. Even in our development server where we handle much less data.
> Repair refuses to fix it sometimes so we have to scrap it and start over.

> I'm hoping DBISAM will work better. If it doesn't we will have to go to
> something much more expensive. We are just trying to keep our costs down
> for our customers.

My suggestion to you is to load test it with the amount of data that
it is going to manage, before actually using it in production. That
way, you'll see if it will do the work.

Daniel Rail
Senior System Engineer
ACCRA Group Inc. (www.accra.ca)
ACCRA Med Software Inc. (www.accramed.ca)