I was at my wits end trying to figure this out and you will never
guess what the problem was.
I had the database buffers set as follows:
Pages:10000
Memory 80mb

I lowered this to 2048 pages (16mb) and the issue went away
completely.
I am not sure I completely understand why this would affect blobs.

But with it set to 2048 the fbserver process holds steady at around
23mb even if I upload 10 10mb files in a row.

So it turned out not to be my code, but a db setting and I have no
Idea why I had it set that high. I must have read somewhere that
haveing the buffers set high would increase performance, and maybe
it did in FB 1.0

So maybe this will help someone else.

Thanks