From: jas@... (Jim Starkey)

>>It is, and will be, tackled by the hardware people; it is
>>their problem.
>>
>Not at all. They don't know anything about software or operating
>system or how to attack complex problems.

How is this unlike a housewife claiming that an army of cooks
know nothing about cooking just because she has spent a lifetime
in the kitchen... While evolution produced a lot of dead-ends,
we did end up with creatures that fly and bacteria that feed on
sulphur deep on the ocean bed --all this during my short lifetime.

>This has the same problem that has Roman upset -- the inability
>to do direct access into a compressed file. I tried to convince
>him decompressing the blob was good enough, but this clearly
>doesn't work for a multi-gigabyte database file.

For a multi-gigabyte database file, anything seems to be not
good enough --except perhaps storing it on a cluster with RAIDs.

>I'll buy encryption at the disk level, but not compression.

Fair enough.

>>If it is the network throughput you're concerned, 10Gb/s is
>>around the corner. And, with 10Gb/s you have a problem that
>>has to be solved by specialized silicon: some sort of offloading
>>engine, or else the CPU will be monopolized by TCP alone.
>>
>You are confusing bandwidth and throughput. They aren't the same.
>Bandwidth reduces the per-bit transmission cost but doesn't nothing
>about the per-message cost to dribble up and down the operating
>system's protocol stacks.

I am not so sure I am confusing them. Offloading iSCSI (and generic
TCP) onto silicon seems to be doing wonders for Alacritech.

>Adaptive switches are good use of custom hardware. Unlike most
>specialized hardware, they make it easier for many clients to
>shared common resources. Ethernet boards have gotten smarter,
>and could be smarter still, but protocol and connection processing
>needs to be done in the OS for a variety of reasons.

Might I suggest you take a look at, say, Alacritech's products and
discuss them with some real users --link in my prev post.

>>There are various TCP offload engines (ToE) NICs that
>>does just that in HW.
>>
>A lot of people have made smart ethernet controllers. But if the
>operating system needs to support flexible IP packet handling (like
>Linux, for example), having the semantic in the board doesn't help.

Ok, Alacritech for one does not (yet) release drivers for
Linux --apparently not because they can not but because they
don't want their IP out in the open.

Anyway, you might even know the guy named Larry Boucher
personally from Shugart Associates days, and if you do, you would
do us a great favor to discuss whether his (Alacritech's) stuff
is really that good.

BTW, I have no personal and/or financial interest in Alacritech
or any other vendors mentioned here or earlier.

>This is an unavoidable problem. Suppose you have a compression or
>encryption or full text search board that is scathingly hot. To
>get at it from user mode, you need operating system support
>to control and schedule access to the hardware. So to get at
>it, you need to change protection mode, probe a bunch of addresses,
>try to acquire the hardware. If the hardware is busy doing someone
>else's work, the task must stall, be marked unrunnable, and queued
>for the hardware resource. So what you often get is an idle
>processor waiting for the "accelerator". The alternative is
>letting a super-scalar processor with a huge cache loose on the
>problem.

I am no guru on this sort of thing, but how is this different
from any other hardware interface --say a disk controller-- whereby
if the blighter is busy with something else you end up twiddling
your thumbs..

>Guess who usually wins?

Cell Processors :-)

>>Which brings back to the point where I was wondering why you
>>focus on problems that are about to be solved for all, instead
>>of adding new features, data types, ACL type security etc.
>>that Firebird/Vulcan needs.
>>
>I try to solve problems in order of importance. The number
>one problem was that Firebird central server didn't take advantage
>of SMP machines. That's fixed. Then next big problem is that while
>Interbase 3 was disk bound on a 68020, Firebird is compute bound
>on a 2.8 GHz superscalar processor with a half gig of on chip cache.
>
>There are only three or four ways to do something faster. One is
>to use a faster processor. Another is to do it on multiple
>processors in parallel. The third way is to do it smarter.
>And the best is to use memory so don't do the same damn thing
>over and over. We're towards the end of faster processors, so
>the future is with parallel processing (SMP).

From other posts I remember that you are a fan of Google in the
sense that they have stolen the whole database show and ended up
as the global database.

But, in this lineup, you seem to be overlooking the fact that Google
uses thousands (millions) of generic PCs... I for one would love
to use Firebird/Vulcan on a cluster of Via Epias with 200 GB HD,
1 GB RAM. Dead cheap.

Question is how. The answer: I don't know, I am just a passing face
here; you are the architect, you tell me :-)

>But the biggest potential is to be smarter. This means looking
>simultaneously at the big picture and the little details to see
>where the wheels are spinning, then fix them.

I am not even hinting that you don't do a thorough job --by George
you do! :-)

What I am baffled about is the apparent discrepancy (to me, at
least) between your own mantra and the things you're focusing on
in this thread... If I am not mistaken you wave the banner that
RAM and disks are practically free and you refuse to optimize
things regarding them. Fine. I agree with you whole heartedly
--and others will fall in line later, I hope :-)

What I am getting at is that the issue of bandwidth and throughput
is (in your words) like peace in the Middle East --much too big
and complicated to be solved by a single visionary. Leave it
for others to worry about --pretty please.

Frankly, not even 20-30 % increase in performance will be more
attractive than being able to do what I really want to do with
Firebird/Vulcan.

In my humblest opinion, the Big Bad Wolf must go back to doing what
only he can do: let Firebird/Vulcan embrace the masses --add more
features, full international support, make it even more user
(developer) friendly, etc.. and, if that means waving 2 fingers
to that mythological SQL committee, so be it.

>ACL type security? Sir, Interbase was born with ACLs. Borland
>hid them because they hadn't been blessed by the SQL Committee.

Everyone, hands up for 2 fingers, then.

>>Just my 0.02 of some currency.
>>
>My lecture is gratis. Keep you 0.02.

Very well then --except that mine was meant to be a gift; alas,
all I have is worthless currency.

Cheers,
Adem