Subject Re: [firebird-support] Help With External Table Issue
Author Germán Balbi
Post date 2014-10-29T13:06:57Z
look for CHR(13)+CHR(10) at the end of lines

open external table with editor (I use textpad) is good for big data.

good luck
Germán


El Jueves, 11 de septiembre, 2014 20:14:24, "'Edward Mendez' Emendez001@... [firebird-support]" <firebird-support@yahoogroups.com> escribió:


 
Hi All,
 
I think I found out why I am having the problem.
 
I found release notes on Firebird 2.5 and saw this following…
 
Large External Table Support Enabled
Vlad Khorsun
Previous Firebird versions used 32-bit I/O when working with external tables, limiting the size of the external
file to < 2 GB. The mechanism has been enhanced to use 64-bit I/O on filesystems that support it, effectively
eliminating the 2 GB limit.
Tracker reference CORE-2492.
 
So I am guessing that even though FB 2.1.4 DB is 64bit architecture, it is still using the 32 bit I/O for external Tables.
 
I was able to verify that this works well in FB 2.5.3.
 
Thanks,
 
Ed Mendez
 
From: firebird-support@yahoogroups.com [mailto:firebird-support@yahoogroups.com]
Sent: Thursday, September 11, 2014 5:51 PM
To: firebird-support@yahoogroups.com
Subject: RE: [firebird-support] Help With External Table Issue
 
 
Hello All,
 
We have a Firebird 2.1.4 running on CentOS 5.5
 
I am having a problem trying to insert data into an External Table and then read it back.
 
Here is out output from fbsvcmgr;
fbsvcmgr service_mgr -user sysdba -password masterke  -info_server_version -info_implementation
Server version: LI-V2.1.3.18185 Firebird 2.1
Server implementation: Firebird/linux AMD64
 
And from uname –a
Linux <hostname>  2.6.18-194.3.1.el5 #1 SMP Thu May 13 13:08:30 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
 
So it looks like I have 64 bit OS and 64 bit Firebird install
 
I have an external table definition of about 176 fields in this particular external table. 88 real fields and 88 additional fields used to track NULL status for each field.
 
I am populating the table as so…
 
INSERT INTO EXT_TABLE ( Field1, field2, field3…)
Select Field1, field2, field3… FROM SRC_TABLE;
 
 
There should be about 1.4 Million rows written out to the External Table.
The weird thing is that my insert statement completes successfully. I get no Errors.
But when I try to do an select count(*) on the external table it bombs with the following error message.
Unsuccessful execution caused by a system error the precludes successful execution of subsequent statements.
I/O error for file “@2”.
Error while trying to open file.
Invalid Argument.
 
On the Linux server it wrote out a file about 4.3 GBs.
 
If I do a select count(*) it bombs, but if loop via the FOR SELECT and add a WHEN ANY CLAUSE I get a count of about 667,000 row count.
 
I saw documentation for External Tables being 2GB for 32bit implementations, but the I can’t find one for firebird 2.1.4 - 64bit.
 
Maybe I might be hitting a Linux Limitation, but our FIREBIRD Database on this same server is 240GB.
 
Any help would be greatly appreciated.
Thank you,
 
Ed Mendez