At 02:09 AM 2/23/01 +1100, Mark O'Donohue wrote:

>

Mark, thanks for taking the time to put together the thoughtful
and enlightening response.

Much of my thinking is based around a basic tenant: The only
major platforms of the future are client browser and web server.
Client server computing is as much legacy today as MVS was last
decade. The only significant growth market for a DBMS is the
web environment.

The dominant performance metric of the web environment are
latency and scalability. Given contemporary hardware, unless
you're Amazon or Ebay, if you can deliver low latency, you're
probably sufficiently scalable for 98% of applications.

The trend in web applications is from static to dynamic
page generation. Most web sites are essentially 100%
static with the odd cgi script to catch form input. But
the good sites are generally 100% dynamic for user directed
content, adaptive display, etc. The crud in the middle
is using crappy ASP technology.

The strategic goal of Firebird should be to support completely
dynamic web sites. But 100% dynamic page generation requires
a great deal more database access (in number of queries) than
a simple but dumb ASP page. Any institutional latency (such
as an intermediate server) gets multiplied by the number of
database interactions until the delay is no longer acceptable
to the human at the browser, and the site fails.

Traditional database performance measurement is based on throughput,
not latency. Throughput is meaningless in a web environment.
Nobody is ever going to ask for a report on inventory; the user
is going to ask the number of a particular item on hand.

Firebird latency performance has been seriously neglected. Aside
from issue of Java connectivity (which is critical) there are
a couple of areas that need immediate attention:

1. The super server detaches a data when the last server
connection goes away. Restarting the engine is extremely
expensive. A keep alive timeout should be added.

2. Compiled queries are not retaining and reused. The compilation
of a SQL query is very expensive. Repetitive compilations of
the same SQL string is stupid, slow, and pointless.

Netfrastructure is designed specifically for dynamic page generation.
I sweat microseconds between Java application code and the engine
(the JVM runs inside the engine). The idea of passing a database
call through a DataOutputStream to a socket OutputStream across
JNI to a socket, down the 6 ISO layers, across the wrire, up the
6 ISO layers, into a socket, across JNI, into an InputStream,
throught a DataInputStream, to some intermediate code, allocating
many objects requiring thread synchronization, across JNI again,
to the Firebird library, and than trace the whole torturous path
back makes my skin crawl.

The latency problems in Firebird are easily fixed. The latency
problems induced by Interclient essential preclude Firebird from
playing in the dynamic web world, which is the future.

Everybody has probably noticed that when a page takes a long time
to load that either a) the line has dropped, or b) it's an ASP
page. ASP is wasting alot of time parsing the same page over
and over to find the scripting language, but mostly the delay
is dribbling across COM layers communicating with a database
somewhere. Bad now, not extensible to 100% dynamic page
generation.

>Jim Starkey wrote:
>
>> At 10:00 AM 2/22/01 +1100, Mark O'Donohue wrote:
>>
>>
>> The user written Java code is, as it should be, isolated from
>> the implementation. Casual database use probably doesn't care
>> about performance, and the security hit of passing all passwords
>> and data through an extra server probably isn't that bad.
>

If the casual user is running a local client, you're absolutely
right. If the casual user is running a browser, you're wrong.
Which do think it will be in two years?

>The optimal solution (for the java world) is to have a java
>implementation on the client side all the way down to the protocol used
>on the wire to the server. Basically a rewrite of all client software,
>and talk directly to the 3050 socket. This would be a type 4 driver,
>and wouldn't have the performance hit of going through another server
>layer. - but is from what I understand dificult to do here.
>

I want application code pure Java for software engineering
reasons. But I want high performance plumbing. If you track
the evolution of JavaSoft's base classes from 1.0 to 1.3 you
will see an appalling shift from pure Java to native (sic), primary
for performance reasons. Once across an architectural interface,
the best strategy is to get the hell out of Java into C++.

>I don't mind having a type 2 driver with better performance, but I would
>always prefer to have a type 4 driver hanging around because the
>marketting people always tend to put their head in asking if we can
>supports clients running on the amiga/sgi irix/commodore64/hp3000/palm
>pilot/netBSD platform.
>

Good point. So we'll give interclient/interserver a reprieve.

>> If
>> you have ever looked at the implementation of the lower layers
>> of Java communication code, take a look. It's not pretty and
>> it's not fast.
>
>Sorry do you mean java socket code, or borland interclient <->
>interserver socket communications?
>

No, the actual implementation of the Java classes and the native
methods behind them. Almost all of Java IO requires multiple
method calls per byte.

>
>Which is why the best java solution is to have the client java all the
>way down to the 3050 socket writes.
>

Doing an interface that talks directly to 3050 is certainly
feasible and probably not very difficult. The protocol is
currently defined in terms of XDR, but could/should have a
restatement as a straightforward line protocol. The more
difficult and expensive (in performance) is the mapping from
jdbc semantics to Firebird semantics. Maybe a careful study
would reveal that it's not too bad, maybe even preferred.

>
>It sounds interesting, I assume you provide C (or C++) api's that mirror
>the java classes. It sounds good, for the C++ people and it's possible
>that from that a pure java type 3 driver could be created fairly
>easily. That would be creating an "interserver" (although please! let's
>not call it that) replacement built on your current type 2 jdbc
>interface, which provides a java server socket listener handling
>requests from the 100% java clients.
>

Yes. The api is actually a set of C++ abstract class definitions
modelled very closely on the Jdbc interfaces. They could be
used either directly or in a birdserver (I'm developing a deep
aversion to the prefix "inter" in anything other "net".)

>Thinking about it, there is a simple way of doing this with the Proxy
>class that came out in 1.3. You use the Proxy class to forward requests
>to an interface over a socket to the server. Actually I already have
>the code since I have done exactly that for an upgrade of a recent program.
>

I supposed I'll have to learn something about Proxy classes. Happens.

>(Hmm ... I started writing about how I got to the Proxy class, but it
>turned into a history of how I've been working with generating c++/java
>object infrastructures from tables for years. In particularly
>infrastructures that use generic SQL and a runtime switchable interface
>to talk to many different backend databases including to remote internet
>servers over sockets (inetd things), It turned into another essay so I
>cut it)
>

Write it up and send it around.

>
>Which leads us into another issue, One advantage that interclient does
>have is that at least the source has been released into the public
>domain ;-). I don't begrudge you getting paid for what you do, I think
>its quite reasonable, particlarly given the circumstances that it all
>went belly up, but I suspect at current rates of uptake, that your odbc
>driver still has quite a way to go before it will be released. And I
>presume the jdbc component would also take some time.
>

The issue isn't money, per se, but the problem that the code contains
a big hunk of Netfrastructure. I'm just plain adverse to subsidizing
my competition. If the competition wants to pay for it, well, that's
business.

>
>Cheers
>
>Mark
>

Jim Starkey