Subject | Re: [ib-support] Large External Files? |
---|---|
Author | Dalton Calford |
Post date | 2001-09-10T14:09:15Z |
We have used extreamly large external files on NT (long, long ago) when we
were porting data from one data structure to another.
We have extreamly large databases, and it seems to me that the files would
have been 4 GB or larger. I remember questioning if the external files were
accessed as a random access file or a stream. If they are a stream, then
they are naturally scanned on every read, and can be as large as the file
system allows. If they are randomly accessed, then they would be limited to
the size that the calling program can safely manage (IB in this case).
Since external files only allow inserting (append to end of file) or
selecting, I believe (and experience has shown) that you can build an
external file as big as NTFS will allow (18 GB last time I tried).
Just don't expect great performance.
I would not use IB 5.6 or less to access random access files that are greater
than 2GB in size, even though some people have gotten away with GDB sizes of
up to 4GB. (signed vs unsigned 32bit integers)
The nice thing would be to import from other data structures (Like MySQL or,
perhaps even, Interbase......)- maybe with firebird 3.XX or higher.
best regards
Dalton
were porting data from one data structure to another.
We have extreamly large databases, and it seems to me that the files would
have been 4 GB or larger. I remember questioning if the external files were
accessed as a random access file or a stream. If they are a stream, then
they are naturally scanned on every read, and can be as large as the file
system allows. If they are randomly accessed, then they would be limited to
the size that the calling program can safely manage (IB in this case).
Since external files only allow inserting (append to end of file) or
selecting, I believe (and experience has shown) that you can build an
external file as big as NTFS will allow (18 GB last time I tried).
Just don't expect great performance.
I would not use IB 5.6 or less to access random access files that are greater
than 2GB in size, even though some people have gotten away with GDB sizes of
up to 4GB. (signed vs unsigned 32bit integers)
The nice thing would be to import from other data structures (Like MySQL or,
perhaps even, Interbase......)- maybe with firebird 3.XX or higher.
best regards
Dalton
On Saturday 08 September 2001 23:01, you wrote:
> Mike,
>
> > Will Firebird Beta 2 support external text files over 2 gig?
>
> Does Beta 2?
>
> I don't know. Have you tried it or are you asking before you do? ;-)
>
> Also, since you run on Win NT/2000 the actual limit should be 4 gig.
>
>
> Will Firebird v1.0?
>
> Good question! ;-)
>
> At this point, I think that whatever support is present in the Beta will
> be the functionality of the final version.
>
> Supporting large external files is _not_ a typical problem. In fact,
> it's the first time that I can recall the question ever being asked, in
> 5 years. So, this is not a _high_ priority.
>
>
> Sean