Subject Re: RE: At my wits end
Author Steve Staneff
Ed,


Mileage does indeed vary. As a quick and possibly irrelevant test, I just copied 1,665,884 records from one table to another, and it took less than 3 minutes (I left the room and it was done when I came back). That's better than 9250 reads & inserts per second. This was using WinXP, FB 1.5rc7, and IBConsole on a 2.8ghz pentium.


That said, you mentioned that you're bringing in records from CSV-formatted files on a routine basis. This is the kind of thing that scripting languages were made for. I'd suggest looking into either Perl or Python. Personally, I use Python for just this sort of thing, and the CSV and Kinterbasdb modules mask the complexity on both ends. I'm not a programmer, but I've employed them, and I much prefer writing a script myself whenever possible (no offense intended to the programmers out there...).


Steve


Epstein, Ed wrote:
> 1) Use the API directly - Don't have the skill
> 2) Use an external program written to use the API - SQL scripts, import
> engines like FBExport, etc. I really needed a way to do it internally and
> they did not even reach an accetable speed.
> 3) Use external tables. This does not work since I would have to come up
> with a method of delivering them to the server. This has proved to be quite
> impossible due to circumstances that I cannot control.
>
> I took the advice of this forum and paid to have a component written to
> execute the SQL statements for me and use only the API to do this. No ADO
> layers, etc. Just pure looped API statements.
>
> This does EVEN worse than ADO and OLEDB. I received multiple testing
> programs from potential coders and none of them could even break 100 tps,
> let alone the 1500 tps i need.



--
Steve
_______________________________________________
Dr. Stephen T. Staneff
President, Construction Data and Research, Inc.
Adjunct Professor of Civil Engineering, UNM
reply to: staneff@... <or>
staneff@...