Subject | Re: [firebird-support] Re: Fastest way of moving data in "AppendUpdate" mode to Firebird |
---|---|
Author | Tim |
Post date | 2006-02-24T07:23:13Z |
Look up "EXTERNAL TABLE" in the docs. :)
It's very easy to use. And real quick too. :D
Basically what you do is define a table as an external file
like this :
CREATE TABLE "EXT_FOO"
EXTERNAL FILE 'C:\NETPOS\NPSTITCH\FILES\FOO.TXT'
(
"FOO1" CHAR(19) CHARACTER SET ASCII,
"FOO2" CHAR(10) CHARACTER SET ASCII,
"NEWLINECHAR" CHAR(2) CHARACTER SET ASCII
)
Now Firebird treats this table as a table in the DB.
The only problems are that the path to the table is fixed - it is
part of the DB metadata. So you have to have a file there whenever
you create the DB, even if the file is empty.
The second problem I have had (in the past) is accessing the external
file when FB is finished with it : it locks the table, and releasing
the lock is a pain.
However, once you have your external table, everything else is trivial.
You create another table in the DB :
CREATE TABLE "IMPORT_FOO"
(
"FOO1" CHAR(19) CHARACTER SET ASCII,
"FOO2" CHAR(10) CHARACTER SET ASCII
)
And then (once you now have this "in" the database) you can do stuff like
FOR SELECT FOO1, FOO2 FROM EXT_FOO
INTO
:FOO1, :FOO2
DO
BEGIN
INSERT INTO IMPORT_FOO (FOO1, FOO2)
VALUES
:FOO1, :FOO2;
END
etcetera, etcetera, etcetera ...
HTH
Tim
At 09:03 24/02/2006, you wrote:
It's very easy to use. And real quick too. :D
Basically what you do is define a table as an external file
like this :
CREATE TABLE "EXT_FOO"
EXTERNAL FILE 'C:\NETPOS\NPSTITCH\FILES\FOO.TXT'
(
"FOO1" CHAR(19) CHARACTER SET ASCII,
"FOO2" CHAR(10) CHARACTER SET ASCII,
"NEWLINECHAR" CHAR(2) CHARACTER SET ASCII
)
Now Firebird treats this table as a table in the DB.
The only problems are that the path to the table is fixed - it is
part of the DB metadata. So you have to have a file there whenever
you create the DB, even if the file is empty.
The second problem I have had (in the past) is accessing the external
file when FB is finished with it : it locks the table, and releasing
the lock is a pain.
However, once you have your external table, everything else is trivial.
You create another table in the DB :
CREATE TABLE "IMPORT_FOO"
(
"FOO1" CHAR(19) CHARACTER SET ASCII,
"FOO2" CHAR(10) CHARACTER SET ASCII
)
And then (once you now have this "in" the database) you can do stuff like
FOR SELECT FOO1, FOO2 FROM EXT_FOO
INTO
:FOO1, :FOO2
DO
BEGIN
INSERT INTO IMPORT_FOO (FOO1, FOO2)
VALUES
:FOO1, :FOO2;
END
etcetera, etcetera, etcetera ...
HTH
Tim
At 09:03 24/02/2006, you wrote:
>Can you post some examples of this.[Non-text portions of this message have been removed]
>This sounds fantastic.
>
>Regards,
>
>Robert.
>
>--- In firebird-support@yahoogroups.com, Tim <tim@...> wrote:
> >
> > I have to import text files containing 50 000+ records into our DB
> > every so often.
> >
> > So what I do is rewrite the input file in the format I require
>(fixed
> > length fields)
> >
> > I have declared this rewritten table as an external file in
>Firebird.
> >
> > Then I run a SP to import the data in batches of 10 000 records a
> > time into the table.
> >
> > It cut the import time from (worst case) 2.5 hours to about 3 or 4
>seconds.
> >
> > HTH
> >
> > Tim
> >
> > At 07:54 24/02/2006, you wrote:
> >
> > >We currently use running SQL statements to inject
> > >data into our Firebird database.
> > >Is there any faster way?
> > >
> > >Like a rawdata pump or something?
> > >
> > >Regards,
> > >
> > >Robert.
> > >
> > >
> > >
> > >
> > >
> > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > >
> > >Visit
> > ><<http://firebird.sourceforge.net>http://firebird.sourceforge.net
> >http://firebird.sourceforge.net
>and
> > >click the Resources item
> > >on the main (top) menu. Try Knowledgebase and FAQ links !
> > >
> > >Also search the knowledgebases at
> > ><<http://www.ibphoenix.com>http://www.ibphoenix.com>http://www.ib
> phoenix.com
> > >
> > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > >
> > >
> > >
> > >
> > >SPONSORED LINKS
> > ><<http://groups.yahoo.com/gads?>http://groups.yahoo.com/gads?
>t=ms&k=Technical+support&w1=Technical+support&w2=Computer+technical+su
>pport&w3=Compaq+computer+technical+support&w4=Compaq+technical+support
>&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+support&c
>=6&s=196&.sig=-XIO8GxY6hqd3NaD5WSEyw>Technical
> > >support
> > ><<http://groups.yahoo.com/gads?>http://groups.yahoo.com/gads?
>t=ms&k=Computer+technical+support&w1=Technical+support&w2=Computer+tec
>hnical+support&w3=Compaq+computer+technical+support&w4=Compaq+technica
>l+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+
>support&c=6&s=196&.sig=B29J78SYXnNTjjMFBMznqA>Computer
> > >technical support
> > ><<http://groups.yahoo.com/gads?>http://groups.yahoo.com/gads?
>t=ms&k=Compaq+computer+technical+support&w1=Technical+support&w2=Compu
>ter+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+t
>echnical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+tec
>hnical+support&c=6&s=196&.sig=7_je1A94xs82CFXUjEqA6g>Compaq
> > >computer technical support
> > ><<http://groups.yahoo.com/gads?>http://groups.yahoo.com/gads?
>t=ms&k=Compaq+technical+support&w1=Technical+support&w2=Computer+techn
>ical+support&w3=Compaq+computer+technical+support&w4=Compaq+technical+
>support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+su
>pport&c=6&s=196&.sig=2zMAuRCo5cJrVBr1Bxa3_w>Compaq
> > >technical support
> > ><<http://groups.yahoo.com/gads?>http://groups.yahoo.com/gads?
>t=ms&k=Hewlett+packard+technical+support&w1=Technical+support&w2=Compu
>ter+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+t
>echnical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+tec
>hnical+support&c=6&s=196&.sig=_ytYU7aXb57AVaeUfmvLcA>Hewlett
> > >packard technical support
> > ><<http://groups.yahoo.com/gads?>http://groups.yahoo.com/gads?
>t=ms&k=Microsoft+technical+support&w1=Technical+support&w2=Computer+te
>chnical+support&w3=Compaq+computer+technical+support&w4=Compaq+technic
>al+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical
>+support&c=6&s=196&.sig=4hRo6NXYavRAbTkaYec5Lw>Microsoft
> > >technical support
> > >
> > >
> > >----------
> > >YAHOO! GROUPS LINKS
> > >
> > > * Visit your group
> > >
> "<<http://groups.yahoo.com/group/firebird-support>http://groups.yahoo.com/group/firebird-support>firebird-
>support" on the web.
> > > *
> > > * To unsubscribe from this group, send an email to:
> > > *
> > > <mailto:firebird-support-unsubscribe@yahoogroups.com?
>subject=Unsubscribe>firebird-support-unsubscribe@yahoogroups.com
> > >
> > > *
> > > * Your use of Yahoo! Groups is subject to the
> > >
> <<http://docs.yahoo.com/info/terms/>http://docs.yahoo.com/info/terms/>Yahoo!
> Terms of Service.
> > >
> > >
> > >----------
> >
> >
> > [Non-text portions of this message have been removed]
> >
>
>
>
>
>
>
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>Visit
><http://firebird.sourceforge.net>http://firebird.sourceforge.net and
>click the Resources item
>on the main (top) menu. Try Knowledgebase and FAQ links !
>
>Also search the knowledgebases at
><http://www.ibphoenix.com>http://www.ibphoenix.com
>
>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>SPONSORED LINKS
><http://groups.yahoo.com/gads?t=ms&k=Technical+support&w1=Technical+support&w2=Computer+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+technical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+support&c=6&s=196&.sig=-XIO8GxY6hqd3NaD5WSEyw>Technical
>support
><http://groups.yahoo.com/gads?t=ms&k=Computer+technical+support&w1=Technical+support&w2=Computer+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+technical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+support&c=6&s=196&.sig=B29J78SYXnNTjjMFBMznqA>Computer
>technical support
><http://groups.yahoo.com/gads?t=ms&k=Compaq+computer+technical+support&w1=Technical+support&w2=Computer+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+technical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+support&c=6&s=196&.sig=7_je1A94xs82CFXUjEqA6g>Compaq
>computer technical support
><http://groups.yahoo.com/gads?t=ms&k=Compaq+technical+support&w1=Technical+support&w2=Computer+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+technical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+support&c=6&s=196&.sig=2zMAuRCo5cJrVBr1Bxa3_w>Compaq
>technical support
><http://groups.yahoo.com/gads?t=ms&k=Hewlett+packard+technical+support&w1=Technical+support&w2=Computer+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+technical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+support&c=6&s=196&.sig=_ytYU7aXb57AVaeUfmvLcA>Hewlett
>packard technical support
><http://groups.yahoo.com/gads?t=ms&k=Microsoft+technical+support&w1=Technical+support&w2=Computer+technical+support&w3=Compaq+computer+technical+support&w4=Compaq+technical+support&w5=Hewlett+packard+technical+support&w6=Microsoft+technical+support&c=6&s=196&.sig=4hRo6NXYavRAbTkaYec5Lw>Microsoft
>technical support
>
>
>----------
>YAHOO! GROUPS LINKS
>
> * Visit your group
> "<http://groups.yahoo.com/group/firebird-support>firebird-support" on the web.
> *
> * To unsubscribe from this group, send an email to:
> *
> <mailto:firebird-support-unsubscribe@yahoogroups.com?subject=Unsubscribe>firebird-support-unsubscribe@yahoogroups.com
>
> *
> * Your use of Yahoo! Groups is subject to the
> <http://docs.yahoo.com/info/terms/>Yahoo! Terms of Service.
>
>
>----------