i Jim

Jim Starkey wrote:

> At 02:09 AM 2/23/01 +1100, Mark O'Donohue wrote:
>
>
>
> The trend in web applications is from static to dynamic
> page generation.

[snip]

>
> The strategic goal of Firebird should be to support completely
> dynamic web sites.

I agree strongly in doing dynamic pages, but in my experience it hasn't
been much of an extra hit on the database, but I suppose it depends on
what you are doing. For transactional stuff payments processing/buying
theatre tickets etc it was certainly a much more cohesive solution to do
all the page generation in code or from a database.

I agree with trying to remove any intermediate server.

>
> Traditional database performance measurement is based on throughput,
> not latency. Throughput is meaningless in a web environment.
> Nobody is ever going to ask for a report on inventory; the user
> is going to ask the number of a particular item on hand.

Time taken navigating from one web page to another was the main issues
we found. In some cases we found if you run a java applet on the client
side, then you can actually do quite a lot of work (entirely on the
client side) before you need to post a message back to the server.
This reduces a lot of the network traffic, and also reduced the work
load on the server considerably.

This is why I argue a little later that what you call a web server I
think will actually be a more general application server.

>
> Firebird latency performance has been seriously neglected. Aside
> from issue of Java connectivity (which is critical) there are
> a couple of areas that need immediate attention:
>
> 1. The super server detaches a data when the last server
> connection goes away. Restarting the engine is extremely
> expensive. A keep alive timeout should be added.
>
> 2. Compiled queries are not retaining and reused. The compilation
> of a SQL query is very expensive. Repetitive compilations of
> the same SQL string is stupid, slow, and pointless.

These sound like good things to persue. They will take time to get
arround to, since we're all here in our spare time, but on a positive
note, we are here and still going forward slowly.

>
> Netfrastructure is designed specifically for dynamic page generation.
> I sweat microseconds between Java application code and the engine
> (the JVM runs inside the engine).

Ok and I can now really see where you are comming from, and am glad your
sweating microseconds over it since that's the business end of the
stick, and for a tightly coupled web server/db server and high
performance this is the best way to go.

But performance is not always the key factor in a system. I think some
sort of remote proxy solution ontop of your current jdbc would work
fairly well in providing a solution where ubiquitious platform coverage
is required (capture that graphic articst market that exclusively use
MACs :-).

If the server component then connected directly to the database, rather
than though another socket connect to 3050 then performance might not be
too bad at all. Hmm I wonder if interserver can use a local connection
to the db or whether it defers always to routing back through the 3050
port server. (Just showing my ignorance here ;-).

I'll put down a few notes about it a few weeks on how I used the proxy
class, after Im over my current work hump.

I agree with the ASP/COM notes and would add that by not being
compiled asp systems miss the chance of catching 99% of the errors that
plague IIS web sites.

> The idea of passing a database
> call through a DataOutputStream to a socket OutputStream across
> JNI to a socket, down the 6 ISO layers, across the wrire, up the
> 6 ISO layers, into a socket, across JNI, into an InputStream,
> throught a DataInputStream, to some intermediate code, allocating
> many objects requiring thread synchronization, across JNI again,
> to the Firebird library, and than trace the whole torturous path
> back makes my skin crawl.

Thats what's driving the market for 1GHz machines ;-).

>
> The latency problems in Firebird are easily fixed. The latency
> problems induced by Interclient essential preclude Firebird from
> playing in the dynamic web world, which is the future.
>
> Everybody has probably noticed that when a page takes a long time
> to load that either a) the line has dropped, or b) it's an ASP
> page.

c) the ASP/COM object has hung with msg box on server console.

> ASP is wasting alot of time parsing the same page over
> and over to find the scripting language,

At least jsp compiles it's pages, this helps.

>> Jim Starkey wrote:
>>
>>> At 10:00 AM 2/22/01 +1100, Mark O'Donohue wrote:
>>>
>>>
>>> The user written Java code is, as it should be, isolated from
>>> the implementation. Casual database use probably doesn't care
>>> about performance, and the security hit of passing all passwords
>>> and data through an extra server probably isn't that bad.
>>
>
> If the casual user is running a local client, you're absolutely
> right. If the casual user is running a browser, you're wrong.
> Which do think it will be in two years?

I agree, and think that it will be client <-> app server <-> db server.
Even with this basic structure there is still going to be plenty of room
for a lot of inter and intra server processes that communicate without
contact with the user.

We don't trust the clients anymore and I agree the communication client
<-> app server will generally not contain any SQL. Of course if MS
still have the browser, client <-> app server will be c#, and SOAP.

>
>>> If
>>> you have ever looked at the implementation of the lower layers
>>> of Java communication code, take a look. It's not pretty and
>>> it's not fast.
>>
>> Sorry do you mean java socket code, or borland interclient <->
>> interserver socket communications?
>>
>
> No, the actual implementation of the Java classes and the native
> methods behind them. Almost all of Java IO requires multiple
> method calls per byte.

That is pretty much true of all of java. In the early days, (and doing
web page generation) string handling was terribly slow, (objects need
more work that simple allocated memory) and it's been interesting
following the progress and what they have done under to hood to speed
that up, they went from (20 x c++, 4 x vb) to slightly better than vb in
the time I was watching this carefully, they should be better now.
Numeric calculations last I heard were approaching c speeds mainly
because the jit got them into native code quickly. However It still
doesn't help speed up where the coding in the base java classes is
itself is poorly optimised for speed.

In the latest wonderfully fast IBM jdk they achieved speedup times of up
to 60x for a lot of the IO by optimisation probably the code your
talking about. I expect over time that some more improvements will
continue to be made in the IO/socket area (and cpu's will thankfully
contine to get faster).

> Doing an interface that talks directly to 3050 is certainly
> feasible and probably not very difficult. The protocol is
> currently defined in terms of XDR, but could/should have a
> restatement as a straightforward line protocol. The more
> difficult and expensive (in performance) is the mapping from
> jdbc semantics to Firebird semantics. Maybe a careful study
> would reveal that it's not too bad, maybe even preferred.

That's encouraging to hear, and would be interesting to have a look at
what the mappings would involve. But I suspect it's a bit of a pipe
dream at the moment and think that interclient or in your case a proxy
providing a type 3 solution is going to be our solution for a quite a while.

>
> Yes. The api is actually a set of C++ abstract class definitions
> modelled very closely on the Jdbc interfaces. They could be
> used either directly or in a birdserver (I'm developing a deep
> aversion to the prefix "inter" in anything other "net".)

They sound interesting, and I would think pretty useful additions to
have ;-).

>
>
> The issue isn't money, per se, but the problem that the code contains
> a big hunk of Netfrastructure. I'm just plain adverse to subsidizing
> my competition. If the competition wants to pay for it, well, that's
> business.

As I understand the odbc driver is slowly on it's way to becoming
opensource, but it will be a while under the current plan. I understand
your reservations about the rest of your stuff, since it is your stuff,
and represents a considerable amount of
effort/time/commitment/investment/etc on your part. If we are really
lucky someone will come forth with the money and pay you for it.
But failing that, I think at some stage your going to have to decide
if/how you want this all to work in an opensource world, and/or
integrated into the FB code. then you are going to have to find a model
under which your happy to release at least some of your code, and a way
that it will complement rather than detract from your own business.
(You might even want to adopt Jason's licence model). I don't think
that there is any hurry, since where firebird is going it will go slowly
for the moment and it will take some time for the dust to settle.

In the meantime I dont want to scare you off from sharing at least your
design ideas and thoughts here ;-).

Cheers

Mark

(Oh look another essay, I should probaly have spent the time more
productively by say checking in the gbak changes made recently).

>
>

--
Your database needs YOU!
http://firebird.sourceforge.net


--
Your database needs YOU!
http://firebird.sourceforge.net