Subject Re: [firebird-support] External Table Guidelines
Author Milan Babuskov
Hardy Sherwood wrote:
> 1. I understand that external tables is the best way of loading large
> quantities of data into internal tables. Is this correct? If not, what is
> a better method?

It is the quickest, but it has limitations. As you note in point 2.
CHARs are recommended, but that way you need to use a special markers
for NULLs, which is very error-prone IMO. The other limitation is that
it doesn't support BLOBs. I wrote FBExport to work around these
limitations. It is somewhat slower, but much more robust:

http://fbexport.sourceforge.net

If you need to import data from external sources, you can create a small
program to dump it in FBExport format before loading. The format is very
simple, and explained in detail here:

http://fbexport.sourceforge.net/FBExport_file_format.html

> 3. I've attempted to use external tables that contain binary data. I
> ran into problems because the slack bytes Firebird introduced to align 2
> byte, 4 byte and 8 byte binary values to "appropriate" boundaries did not
> always seem to conform to "published" rules. Has anyone else observed this
> inconsistency? Is there anyway to suppress the generation of slack bytes
> for alignment of binary data to appropriate boundaries?

You just replied to yourself here. 3. is the answer to 2. i.e. why CHAR
is recommended.

> 4. Helen Borrie suggested that as a starting guideline to insert about
> 8,000 rows at a time when inserting data from an external to an internal
> table. She also suggested that there are other factors. I assume that
> these other factors concern the number of columns and maybe the number of
> bytes of data per row. I'd appreciate any additional guidelines on how much
> data should be inserted at one time from an external to an internal table?

AFAIK, there is no formula to calculate the exact number of rows for
maximum performance. You have to test yourself on your real data to see
how it behaves.

Good luck.

--
Milan Babuskov
http://www.flamerobin.org