Subject | Re: [ib-support] Re: Database grows rapidly (3 to 770MB) when using BLOB views/UDFs |
---|---|
Author | Claudio Valderrama C. |
Post date | 2002-03-22T02:59:32Z |
""Ivan Prenosil"" <prenosil@...> wrote in message
news:20020321173833.4387315014@......
Is the term "garbage collection" the problem or the process itself? Where do
you think that the engine stores a temporary blob when the UDF invokes the
callback function put_segment? In RAM as a raw chunk? No, in database pages.
It's using database pages. If you assign it to a row in a table, it becomes
permanent. Quick operation. If you don't like it, it will have to be cleaned
afterwards. Call it garbage collection or whatever you want, it doesn't
matter. It's using database pages. It's causing the db to grow. There's a
difference between BLB_close and BLB_cancel, just read
jrd/blb.c
isc_cancel_blob calls internally BLB_cancel, that tries to delete the blob
from the pages. For me, it means that those pages are marked as free. Since
the pages are free, the next temporary blob the UDF writes to might use the
same pages instead of allocating new ones.
/**************************************
*
* B L B _ c a n c e l
*
**************************************
*
* Functional description
* Abort a blob operation. If the blob is a partially created
* temporary blob, free up any allocated pages. In any case,
* get rid of the blob block.
*
**************************************/
SET_TDBB (tdbb);
/* Release filter control resources */
THE LINE ABOVE IS MISLEADING, the call
=> if (blob->blb_filter)
=> BLF_close_blob (tdbb, &blob->blb_filter);
IS MISSING.
if (blob->blb_flags & BLB_temporary)
delete_blob (tdbb, blob, 0); <===== FREE PAGES
release_blob (blob, TRUE);
}
That's said, if you look at BLB_close, it doesn't call delete_blob. It
defers the page cleaning to normal garbage collection.
static void delete_blob (
TDBB tdbb,
BLB blob,
ULONG prior_page)
{
/**************************************
*
* d e l e t e _ b l o b
*
**************************************
*
* Functional description
* Delete all disk storage associated with blob. This can be used
* to either abort a temporary blob or get rid of an unwanted and
* unloved permanent blob. The routine deletes only blob page --
* somebody else will have to worry about the blob root.
*
**************************************/
It's boring and unprocedent here to post the code in this function. I hope
the explanation shown above that appears at the top of the function
suffices.
I still hold that calling isc_cancel_blob() may be a partial solution to the
problem. It will prevent chaotic db pages allocation. The isc_close_blob()
call has not the same effect.
C.
--
Claudio Valderrama C. - http://www.cvalde.com - http://www.firebirdSql.org
Independent developer
Owner of the Interbase® WebRing
news:20020321173833.4387315014@......
> > From: Claudio Valderrama C.your
> ...
> > I think that isc_cancel_blob() is much better than isc_close_blob() in
> > case because the returned blob will be taken by BLB_cancel() internally.you
> > BLB_cancel() does an immediate job of trying to get rid of the db pages
> > allocated by the temporary blob. Instead, with the isc_close_blob() path
> > rely on garbage collection to proceed later...??
>
> Hmmm, how can the temporary blob (returned from UDF) be garbege collected
> if it was not assigned to any row (and thus converted to permanent blob)
Is the term "garbage collection" the problem or the process itself? Where do
you think that the engine stores a temporary blob when the UDF invokes the
callback function put_segment? In RAM as a raw chunk? No, in database pages.
It's using database pages. If you assign it to a row in a table, it becomes
permanent. Quick operation. If you don't like it, it will have to be cleaned
afterwards. Call it garbage collection or whatever you want, it doesn't
matter. It's using database pages. It's causing the db to grow. There's a
difference between BLB_close and BLB_cancel, just read
jrd/blb.c
isc_cancel_blob calls internally BLB_cancel, that tries to delete the blob
from the pages. For me, it means that those pages are marked as free. Since
the pages are free, the next temporary blob the UDF writes to might use the
same pages instead of allocating new ones.
/**************************************
*
* B L B _ c a n c e l
*
**************************************
*
* Functional description
* Abort a blob operation. If the blob is a partially created
* temporary blob, free up any allocated pages. In any case,
* get rid of the blob block.
*
**************************************/
SET_TDBB (tdbb);
/* Release filter control resources */
THE LINE ABOVE IS MISLEADING, the call
=> if (blob->blb_filter)
=> BLF_close_blob (tdbb, &blob->blb_filter);
IS MISSING.
if (blob->blb_flags & BLB_temporary)
delete_blob (tdbb, blob, 0); <===== FREE PAGES
release_blob (blob, TRUE);
}
That's said, if you look at BLB_close, it doesn't call delete_blob. It
defers the page cleaning to normal garbage collection.
static void delete_blob (
TDBB tdbb,
BLB blob,
ULONG prior_page)
{
/**************************************
*
* d e l e t e _ b l o b
*
**************************************
*
* Functional description
* Delete all disk storage associated with blob. This can be used
* to either abort a temporary blob or get rid of an unwanted and
* unloved permanent blob. The routine deletes only blob page --
* somebody else will have to worry about the blob root.
*
**************************************/
It's boring and unprocedent here to post the code in this function. I hope
the explanation shown above that appears at the top of the function
suffices.
I still hold that calling isc_cancel_blob() may be a partial solution to the
problem. It will prevent chaotic db pages allocation. The isc_close_blob()
call has not the same effect.
C.
--
Claudio Valderrama C. - http://www.cvalde.com - http://www.firebirdSql.org
Independent developer
Owner of the Interbase® WebRing