Subject import data from external table (huge file)
Author zimrilin2000
Hi all!

I'm trying to import the data from a 2.6 Gb fixed length format
external file with 35 million records into a regular table inside the
database. Each line in the external table file is 76 characters long.
I'm using a stored procedure to do the import, because I need to
detect and store duplicates from the file in another table (violation
of table primary key) and continue with the processing

I execute the SP (EXECUTE PROCEDURE INSERT_EXTERNAL_TABLE) with EMS
QuickDesk and after 2 hour processing (the bdd file grows until it
gets 5 Gb size) I get the following error:


EXECUTE PROCEDURE INSERT_EXTERNAL_TABLE
fmSQLScript.Script:
Unsuccessful execution caused by a system error that precludes
successful execution of subsequent statements.
I/O error for file "D:\SRC\INSERTFILES\DATA.DAT".
Error while trying to open file.
El dispositivo no reconoce el comando.
.

I'm using Firebird Firebird-1.5.2.4731_win32

This is the stored procedure

CREATE PROCEDURE INSERT_EXTERNAL_TABLE
AS
DECLARE VARIABLE A VARCHAR(13);
DECLARE VARIABLE B VARCHAR(10);
DECLARE VARIABLE C VARCHAR(9);
DECLARE VARIABLE D VARCHAR(10);
DECLARE VARIABLE E VARCHAR(20);

BEGIN
/* Procedure body */

FOR SELECT A,B,C,D,E FROM EXTERNAL_TABLE
INTO :A, :B, :C, :D, :E
DO
BEGIN
INSERT INTO TABLE (A,B,C,D,E)
VALUES (:A, :B, :C, :D, :E);

WHEN SQLCODE -803 DO
BEGIN
INSERT INTO TABLE_TMP(A,B,C,D,E)
VALUES (:A, :B, :C, :D, :E);
END
END

EXIT;
END

Is the file too big to do the import?
Is there any other way of doing the import?


Thanks

Diego