Subject Re: [IBO] Pumping data
Author Robert Martin
Hi Carlos

> RM> I have used TCP/IP as my connection method (my std method) and wanted to
> RM> see if 'Local' would increase performance. I can't get it going, I keep
> RM> getting unavailable database (no other apps are using it). Do you think
>
> Any chances that you are using recent fbclient.dll/gds32.dll (>= FB
> 2.0) to access old (ie: FB 1.5) server? There was a change in local
> protocol in FB 2.0 and it is incompatible with old servers.

I have tested my dev machine and the brand new test bed and Local works
on the new machine but no my dev system. So it would seem you are
right, there must be some sort of dll problem. I am running an old 2.0
on my dev machine (but have never used 1.5). The new machine is FB 2.5.


> RM> it would improve performance?
>
> Probably, since the TCP layer would be skipped, unless you already hit
> your I/O limits.
>
>
>
My testing shows a significant improvement

FB 2.5.2 pumping 1 tables worth of data (6,999,773 recs)

Tcp/IP

1621s to process (~27minutes)
4318 rec/s

Local

1286s to process (~21.5 minutes)
5443 rec/s

So from the above it would appear that using local protocol results in a
26% speed improvement !

That's a great result, I am stunned at the impact TCP/IP has on the
process !

With respect to TCP/IP.
What is strange about the situation is the the HDD is not being heavily
used (the data transfer is quite low according to win 8 task mgr), the
CPU shows around CPU is around 25% (on a 4 core machine) and memory
usage is about 1.8GB of 16GB. As I am using FB SuperClasic I thought if
I ran multiple instances of the pump in separate threads I might be able
to get better performance (each thread copying a different part of the
table using 'SELECT FIRST x SKIP y' source SQLs) however while I could
get the CPU up (Around 35%
with 3 threads), performance was no better (slightly worse as it had to
do SQLs to determine the # of records).

Cheers
Rob