Subject Re: [Firebird-Architect] Bulk loader.
Author Jim Starkey
Alex Peshkov wrote:
> On Saturday 03 November 2007 22:50, Vlad Khorsun wrote:
>
>>> Vlad Khorsun wrote:
>>>
>>>>> If the server knew it was a bulk load, it could lock the table, load
>>>>> the data, and build the indexes using a sort and fast load / fast merge
>>>>> algorithm (more or less equivalent to dropping the indexes, loading,
>>>>> and redefining the indexes but easier). On the other hand, "locking
>>>>> the table" isn't exactly a classic concept.
>>>>>
>>>> Hmm, why do we need to lock the table ?
>>>>
>>> If indexes are completely disabled, no. Otherwise the records will
>>> appear no to be there. If the load is done under a single transaction,
>>> however, this isn't an issue.
>>>
>> I thought only about singe transaction case. More - about single
>> statement.
>>
>
> Why do we need to execute buld load not in single transaction? Certainly,
> that's all about single transaction. Better if it will happen also inside
> single request.
>
>
>
Of course single transaction.

Incidentally, there's a trick for constraints. Execute each constraint
at row level. If it passes, fine. Otherwise track the row id. At
commit time, re-evaluate the failures and act accordingly. This is
probably significantly faster than performing all constraint checks at
commit time.

For most loads, bad data is more of theoretical than practical problem.