Subject | RE: [ib-support] Importing Records |
---|---|
Author | Mike Grover |
Post date | 2001-08-16T18:42:23Z |
I made a quick app that split my big file into a bunch of 1.7 gig files
( 5 million records per file)
it take about 20 - 25 minutes to insert one file.
so if i call a sql like this:
INSERT INTO EXPERIAN SELECT * FROM EXP1;
INSERT INTO EXPERIAN SELECT * FROM EXP2;
INSERT INTO EXPERIAN SELECT * FROM EXP3;
INSERT INTO EXPERIAN SELECT * FROM EXP4;
etc...
each will be in it's own transaction?
mike
-----Original Message-----
From: Paul Reeves [mailto:paul@...]
Sent: Thursday, August 16, 2001 1:23 PM
To: ib-support@yahoogroups.com
Subject: Re: [ib-support] Importing Records
Mike Grover wrote:
or 1m lines and declare each file as an external table. Then each
'insert into ... select from' is executed within its' own transaction.
I'm not sure that this will necessarily be an improvement on the method
you are using. For bulk inserts external tables usually win hands down.
You, however, have rather a lot of data, so it will take a little trial
and error to find the best approach.
Paul
--
Paul Reeves
http://www.ibphoenix.com
taking InterBase further
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get VeriSign's FREE GUIDE: "Securing Your Web Site for Business." Learn about using SSL for serious online security. Click Here!
http://us.click.yahoo.com/KYe3qC/I56CAA/yigFAA/dpFolB/TM
---------------------------------------------------------------------~->
To unsubscribe from this group, send an email to:
ib-support-unsubscribe@egroups.com
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
( 5 million records per file)
it take about 20 - 25 minutes to insert one file.
so if i call a sql like this:
INSERT INTO EXPERIAN SELECT * FROM EXP1;
INSERT INTO EXPERIAN SELECT * FROM EXP2;
INSERT INTO EXPERIAN SELECT * FROM EXP3;
INSERT INTO EXPERIAN SELECT * FROM EXP4;
etc...
each will be in it's own transaction?
mike
-----Original Message-----
From: Paul Reeves [mailto:paul@...]
Sent: Thursday, August 16, 2001 1:23 PM
To: ib-support@yahoogroups.com
Subject: Re: [ib-support] Importing Records
Mike Grover wrote:
>You need to break the external file up into smaller files - perhaps 500k
> I used INSERT INTO ... VALUES (...)
> >From a Delphi6 program I wrote
>
> How Can I insert a certain number of records from a external file?
>
or 1m lines and declare each file as an external table. Then each
'insert into ... select from' is executed within its' own transaction.
I'm not sure that this will necessarily be an improvement on the method
you are using. For bulk inserts external tables usually win hands down.
You, however, have rather a lot of data, so it will take a little trial
and error to find the best approach.
Paul
--
Paul Reeves
http://www.ibphoenix.com
taking InterBase further
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get VeriSign's FREE GUIDE: "Securing Your Web Site for Business." Learn about using SSL for serious online security. Click Here!
http://us.click.yahoo.com/KYe3qC/I56CAA/yigFAA/dpFolB/TM
---------------------------------------------------------------------~->
To unsubscribe from this group, send an email to:
ib-support-unsubscribe@egroups.com
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/