> So the requirements of my job dictate that
> 1) Its interbase/firebird compatible.
> 2) the frontend does absolutely everything.
> 3) its easy enough to use that somebody that does not even know what CSV
> stands for, could use it.
> 4) its fast enough to get the jobs done in the required amount of time. The
> same day turn around on processed files. Currently only possible since I am
> doing it manually.

I did not follow this thread from the start, but for what it's worth,
here's what i've found: isql.exe managed to feed a text file (containing
500.000 inserts statement, commit every 10.000 inserts, around 130MB in
size) into the database (Firebird 1.5, windows xp pro, single processor
athlon 2500) in approximately 2:45 (165 seconds).

On average, that's around 3000 inserts per second. YMMV, of course.
Also, varying the number of inserts between commits might make a
difference in speed, but i haven't tried it yet.

The table looks like this, by the way:
CREATE TABLE PRODUCT (
PRODUCTID integer NOT NULL,
NAME varchar(100) NOT NULL,
DESCRIPTION varchar(200),
DATACURRENCY1 numeric(18,4),
DATACURRENCY2 numeric(18,4),
DATACURRENCY3 numeric(18,4),
DATAINTEGER1 integer,
DATAINTEGER2 integer,
DATAINTEGER3 integer,
DATASTRING1 varchar(100),
DATASTRING2 varchar(100),
DATASTRING3 varchar(100),
PRODUCTCODE varchar(100)
);

So the 'raw length' of the record is around 800 - 850 bytes. A line from
the insert script looks like this :
...
insert into product(productid, name, productcode,
datacurrency1, datacurrency2, datacurrency3,
datastring1, datastring2, datastring3,
datainteger1, datainteger2, datainteger3)
values (gen_id(gen_product,1),
gen_id(gen_product,0),
gen_id(gen_product,0),
1, 2, 3,
'1'','2','3',
1, 2, 3);
...

It would not be too difficult to build a program (you're using vb, iirc)
to read and parse the source csv file, and emit a big sql insert script.
Then you can just use isql.exe to import it into the database. Is this
workable for you? If you need to go faster than this, you'd probably
need to go to the API level... <shudder>...:)

Salam,
sugi.