Subject | Re: [ib-support] Importing Data Question? |
---|---|
Author | Svein Erling Tysvær |
Post date | 2002-07-09T16:32:51Z |
Hi Tony!
disk drive to replace the tape!
Seriously, it looks as if you're doing something very wrong - it shouldn't
take one hour to import 20 000 records unless each record contains several
blobs with high quality pictures (i.e. every record being a megabyte or
more). Since you do not mention anything about transactions, my best guess
is that they are your source of trouble. It takes time to start and commit
a transaction, and your performance may indicate that you do a commit for
every record (and you cannot import anything without using transactions).
I'd say that for imports you could take all within one transaction or
commit, say, every 10000th record.
I do not guarantee you that this is enough to import 2000 records a second,
but with some tuning that should be very feasible.
Set
>I am currently importing about 20,000 records and it takes about oneI'd recommend you to upgrade your Commodore 64, or at least buy yourself a
>hour or so using a tcp connection running on the same server.
disk drive to replace the tape!
>I tried the ibdatapump product and it does something with exclusiveWell, that should be possible without exclusive access to the database.
>access and can import 4000 records in a couple of seconds.
Seriously, it looks as if you're doing something very wrong - it shouldn't
take one hour to import 20 000 records unless each record contains several
blobs with high quality pictures (i.e. every record being a megabyte or
more). Since you do not mention anything about transactions, my best guess
is that they are your source of trouble. It takes time to start and commit
a transaction, and your performance may indicate that you do a commit for
every record (and you cannot import anything without using transactions).
I'd say that for imports you could take all within one transaction or
commit, say, every 10000th record.
I do not guarantee you that this is enough to import 2000 records a second,
but with some tuning that should be very feasible.
Set