Subject Re: [Firebird-Architect] Bulk loader.
Author Jim Starkey
Adriano dos Santos Fernandes wrote:
> Jim Starkey wrote:
>
>> Vlad Khorsun wrote:
>>
>>
>>>> If the server knew it was a bulk load, it could lock the table, load the
>>>> data, and build the indexes using a sort and fast load / fast merge
>>>> algorithm (more or less equivalent to dropping the indexes, loading, and
>>>> redefining the indexes but easier). On the other hand, "locking the
>>>> table" isn't exactly a classic concept.
>>>>
>>>>
>>>>
>>> Hmm, why do we need to lock the table ?
>>>
>>>
>>>
>> If indexes are completely disabled, no. Otherwise the records will
>> appear no to be there. If the load is done under a single transaction,
>> however, this isn't an issue.
>>
> And what about duplicate keys handling?
>
>
Falcon handles those with in-memory, per-transaction structures called
deferred indexes. Deferred indexes are search as per transaction
visibility rules; for duplicate detection, all are searched. At commit
time, the transaction's deferred indexes are squirted into the serial
log in key order. The gopher thread does the index merge post-commit,
after which point the deferred index object can be flushed. Firebird,
of course, doesn't have a serial log, but the index merge idea could be
adapted for superserver or Vulcan, but not classic.