Hi,

After investigating this local ipc issue I have found that it ends up a bit
of a trade off. The problem being that the client library is not thread safe
when used in this mode.

I had ngds synchronized on the db_handle during each native method call.
This works OK(I have given this some heavy long term testing) when the
client library connects via tcp/ip to the server. When a local connection is
used although my single threaded benchmark works ok my multithreaded tests
lock up.

Synchronizing on the same object during each native method call so that only
one thread at a time is in native code seems to rectify this - and I have
not yet found a problem with this although I have dont really know if this
is safe. I intend to post to ib-support regarding this. If anyone has any
ideas or info regarding this problem I would be interested to hear.

So it seems the trade off is efficent client/server comms - lower(almost no)
concurrency or less efficent client/server comms - higher concurrency.

I have seen that the XNET protocol aims to solve this problem.

When I come too re-applying these changes I could try to produce a ngds
implementation that can work with either type of synchronization and switch
between them depending on if a local ipc or tcp/ip is used - then the client
can make the decision as to which mode is best.

I see now that I have a few phases of modifications I would like to
propose - these being:

1) The original set of changes that intergrate ngds with the driver. These
lead to very few modifications of code above of the gds package.

2) An extension to this set of changes which makes it possible the implement
the java XSQLDA object as a peer for the actual native XSQLDA structure
using java.nio.ByteBuffer to write/read directly to/from the underlying
XSQLDA structure that is supplied to the interbase c api.

3) Further modifications above gds which allow the direct ByteBuffer method
to work as efficently. eg. Having statement fetcher not fetch fetchsize of
rows but rather fetch size gets passed to the client library and each
resultset.next call causes a fetch in the underlying api - the out_xsqlda of
which is used (pretty much)directly by the ResultSet.getXXX methods to read
straight from the XSQLDA structure - eliminating much copying that occurs.

I will look at re-applying each of these seperatly and in turn once the
release has been made.


Thanks
Ryan


-----Original Message-----
From: Ryan Baldwin [mailto:ryan.baldwin@...]
Sent: 14 April 2003 14:38
To: Firebird-Java@yahoogroups.com
Subject: [Firebird-Java] Type 2 Driver Modifications - They are faster
honestly.


Hi,

I just thought I would report on a discovery I made regarding the type 2
driver modifications I made available. As the stand the binaries I made
available will not show any performance improvement on a local connection -
they will show a slight performance degredation.

I was looking at the possibility of using java 1.4s java.nio.ByteBuffer in
direct mode to reduce the amount of copying done mapping the XSQLDA
structure to native code(which has come along very well) when I found I
could not reproduce my original benchmarks using the type 2 driver.

A lttile head scratching later I remembered that while doing these tests the
database path name was hard coded into the dll as somthing like
"c:\database.fdb". The version I released builds this from the url and ends
up somthing like loopback/3050:c:\database.fdb. I realised that having the
loopback/3050 on the front seems too(for abvious reasons) make use of TCP/IP
while removing this it seems to use some kind of local ipc. My twice as fast
inserts came back when I hard coded "c:\database.fdb" back in.

So if anyone has tried the type 2 driver modifications and was disapointed
with the performance improvments this will be why. I will make avaiilable
binaries soon that fix this along with over-synchronization and a resource
leak on exception handling that was present in the last binaries I made
available.

I've got some modifications comming together that make use of
java.nio.DirectBuffer so that the java XSQLDA acts as a peer for a native
XSQLDA. As data is written to a prepared statement for instance, it gets
written dirctly to the underlying XSQLDA structure which eventually is
passed to the interbase api.

So far it's zero copy(other then the copying of the variable to the XSQLVAR
of course) for writing variables ( all but blob and non one byte per char
strings ) to prepared statements. I intend to complete this to also be zero
copy for reading from a result set and make propper use of the client
libraries fetch buffer - and use the java.nio.DirectBuffer for ferrying blob
segments into and out of interbases c api.

This has required encapsulating further the XSQLDA structure and kindof
moving responsibility for encoding and decoding to this( so for instance I
can use java.nio.ByteBuffer.setLong to a long to a XSQLVAR in a single
call). Hopefully though through this expermimenation I will be able to
re-apply these changes reasonably quickly and neatly once the release branch
is made and make them available for review. This should eventually give us a
higly efficent type 2 driver implementation.


Thanks
Ryan








To unsubscribe from this group, send an email to:
Firebird-Java-unsubscribe@yahoogroups.com



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/