Subject | Re: [firebird-support] Re: Binary blob performance in extremely large databases |
---|---|
Author | Steve Wiser |
Post date | 2008-04-09T18:49:40Z |
Hi David,
We really don't do anything special... The binary blobs are created as
sub_type 0 with a segment size of 4096. The database page size is
8192. The only thing special I guess is that we put the image db on 1
server and the data db on a different server.
We pull the images 1 at a time though based on the key so we don't
really grab a whole bunch of TIFF or PDF files and send them over the
network when we can help it. I would say that if you use a generator or
something for the PK and just pull blobs using it you should be fine. I
am not sure what other people do though.
Does that help at all?
-steve
dkeith2 wrote:
We really don't do anything special... The binary blobs are created as
sub_type 0 with a segment size of 4096. The database page size is
8192. The only thing special I guess is that we put the image db on 1
server and the data db on a different server.
We pull the images 1 at a time though based on the key so we don't
really grab a whole bunch of TIFF or PDF files and send them over the
network when we can help it. I would say that if you use a generator or
something for the PK and just pull blobs using it you should be fine. I
am not sure what other people do though.
Does that help at all?
-steve
dkeith2 wrote:
>
> Steve -
>
> Thanks for the reply. Your response is something that I half
> expected, but didn't have the blob experience to be sure. We have one
> SQL Server db that is 105GB; that is probably an intermediate size.
> One customer gave us a sample that just in TIFF blobs alone is > 130GB.
>
> The company that I currently work for stopped recommending FB for
> our larger customers some time ago. Apparently it had to do with the
> time involved in fetching blobs from large db's, say > 50GB.
>
> To be clear, I don't think that we've done comprehensive research
> and tuning for this issue. So any pointer, recommendations, references
> etc. would be appreciated. Nothing you recommend could be too simple
> or complex. We just need to acquire a more comprehensive knowledge of
> how to efficiently manage extremely large quantities of blobs in FB.
>
> Thanks for any reply.
>
> David Keith
>
> --- In firebird-support@yahoogroups.com
> <mailto:firebird-support%40yahoogroups.com>, Steve Wiser <steve@...>
> wrote:
> >
> > Hi Keith,
> >
> > How large is extremely large? We have a fdb of about 175 GB storing
> > TIFF and PDF blobs and it runs fine...
> >
> > -steve
> >
> > dkeith2 wrote:
> > >
> > > Are there any references available that would detail the tricks /
> > > traps relating to the storage and retrieval of large quantities of
> > > blobs in Firebird?
> > >
> > > We have a system that is implemented in both FB and SQL Server. FB
> > > performance is good or optimizable in all areas except for binary blob
> > > storage.
> > >
> > > I read that blob data is stored in pages and if the blob is bigger
> > > than the page it spans multiple pages, which causes the slowness etc.
> > >
> > > We have found that when databases get large in FB performance really
> > > suffers. In SQL Server you don't notice a difference.
> > >
> > > Are there ways to optimize the performance of blob lookup and
> > > retrieval in FB? Our blobs are binary TIFFs, so I don't think filters
> > > would help.
> > >
> > > Thanks for any pointers/suggestions.
> > >
> > > David Keith
> > >
> > >
> >
>
>