Subject | Re: [firebird-support] Large DB after batch import: index rebuild/set stats, backup/restore? |
---|---|
Author | Ann W. Harrison |
Post date | 2010-11-09T20:18:41Z |
Kjell Rilbe wrote:
import with indexes off, then create the indexes.
depending on what you do next. If the database continues
to grow significantly across all index ranges, then having
some space in the index reduces future splits. If you've
got most of the data you expect to have, maximize packing
density to reduce the number of index reads.
to be split is the last one in the index. Normally, a split puts
half the data in each page. If the split is the last page in the
index, the new page has only the record that didn't fit. That
means that if you load in key order, the index will be created
dense.
Ann
>If you had asked before you did the load, I would have said,
> We're preparing a database for an online system and have imported a
> large amount of data. The DB file is currently 42 gigabyte. There are
> two tables with over 110 million records and then some with about 1-2
> million records. Most other tables only have a few hundred or a few
> thousand records. No records are deleted.
import with indexes off, then create the indexes.
>Absolutely.
> Page size is 4096.
>
> Now, after the batch import finishes, I assume the indexes could use
> with some "conditioning". But what's my best option?
>
> 1. Just set statistics on all of them?
>Will produce dense index pages - which may be good or bad,
> 2. Rebuild all indexes with deactivate/reactivate?
depending on what you do next. If the database continues
to grow significantly across all index ranges, then having
some space in the index reduces future splits. If you've
got most of the data you expect to have, maximize packing
density to reduce the number of index reads.
>Overkill.
> 3. Do a backup/restore cycle with gbak?
>Don't activate the indexes until the data is stored.
> 4. Other options?
>The normal index loading does have an optimization when the page
> Would you handle indexes differently depending on selectivity? For
> example, the "largest" table, with over 110 million records, has a
> primary key where key values are incremental (not always +1, but always
> +something). Is it better to leave that index without a rebuild?
to be split is the last one in the index. Normally, a split puts
half the data in each page. If the split is the last page in the
index, the new page has only the record that didn't fit. That
means that if you load in key order, the index will be created
dense.
>How deep are your indexes?
> Also, what considerations should I make regarding page size? Maybe I
> should bump up the page size?
>Good luck
Ann