paulruizendaal wrote:

> Hi Jim,
>
> Trying to understand your point of view:
>
> "Actually, I have something quite different in mind. I'm pondering a
> (non-stored) procedure language in which a client can define an
> arbitrarily complex set of operations to be executed on the server to
> minimize round trips between the client and the server. A procedure
> could be cached on a server, but wouldn't necessarily persist on the
> server.
>
> The architecture of Nimbus allows an open ended set of protocols
> between client and server, so I don't have to pick one and live with
> it. At the moment, at least, I have the freedom to pick or invent a
> candidate and experiment. And, since nothing that I'm aware of fits
> the bill, there's room to be creative."
>
> For the moment my mental model is of an HTTP-like protocol, or even
> HTTP itself. The client would do a POST, with the actual set of
> operations in its body and the reply would carry the result in some
> encoding (also open ended, like MIME types).
>

Nimbus listens on a know port for connections. The first piece of gook
on a connection must be a piece of XML describing the protocol and any
authentication information required for that protocol. If the incoming
XML is the least bit suspect, the connection is close.

That pretty much precludes HTTP.

The JavaScript protocol would be simple. The client would send the text
of JavaScript request and a set of attribute values pairs to assigned to
the JavaScript connection object. The server would look for a
"compiled" version of the request (hash lookup on JavaScript text) and,
if necessary, parse and otherwise prepare the script. It would then
execute the script, serializing the return value as the response. The
client side would deserialize the result, exposing it though a C/C++ API.

> I'm not sure how you envision the potential caching. I assume you
> match a post body with a response. For this to work the procedures
> must be deterministic or set a non-cacheable flag for the result. You
> may want to read up on this part of HTTP, although having written
> some web servers yourself you probably are already familiar with the
> issues.
>

Know them issues. Caching is simple. Two scripts are equivalent if
they're identical.

> If the procedure language relies on caching to be efficient, chances
> are that high scalability is shot, due to cache invalidation issues.
> Or will you accept replies with potentially stale data?
>

Data will not be cached, only the prepared scripts themselves.

JavaScript is so loosey-goosey that compilation beyond a parse is pointless.

> "The basic requirements for the language are that it computationally
> complete (call, branch, loop, throw/catch, etc.) and support some
> notion of object that can be used to return complex results to the
> client. Bootstrapping off a generally accepted language is a plus.
> Running in a defensible sandbox is an absolute requirement.
>
> I'm looking for a server side solution. The original idea of Java, for
> example, was thin client applets. They didn't pan out, but Java turned
> out to be a dandy server technology. Java is excellent for embedding
> application level semantics into a database server, but as a vehicle
> for expressing more or less ad hoc client requests, leaves a great
> deal to be desired."
>
> I think what you are trying to phrase is that you need a language
> that is good at expressing the manipulation of record sets. As so
> much these days is in the line of C++/Java the discussion tends to be
> object collections and manipulation thereof. Perhaps this is too
> narrow a view. Perhaps there is value in looking at concepts in other
> language families.
> - 'relational': PSQL, PL/SQL and alike offer primitives to operate on
> record sets, mostly one record at a time, but sometimes record sets
> at a time (eg. PL/SQL collections). You could consider to generalise
> these. Perhaps (an enhanced) GDML has another life in it.
> - 'array oriented': you could look to APL or J for inspiration, or
> look at Lua, which uses a table as its core datatype (objects are
> implemented as specialisations of tables).
>

None have the computational power of JavaScript. And being able to
return a cluster of objects to the client is very nice.

I'm not adverse to defining a new language (I've designed at least a
dozen), but the world doesn't really need another language if an
existing language is sufficient.

I've ruled out COBOL, ADA, and pdp-11 Assembler.

> "Take the DBT2/TPCC benchmark, for example. Yes, it could be coded in
> Java and installed on the client. A more convenient alternative that
> didn't require DBA privilege to install but still capable of
> expressing a full transaction as a single client/server exchange
> would be even nicer."
>
> Agree. Note that doing a full transaction for each request is needed
> if you wan to be RESTful, and that a solution needs to be found for
> long running user transactions ('starting to edit a record before
> lunch, finalising it after lunch'). One would need a first
> transaction to read the record, and then a second to do the update.
> It would be nice if the language offered some no sweat "check no
> concurrent update since the read" feature.
>

RESTful and transactional are mutually exclusive.

Long running transactions... Let's see. In Rdb/ELN, Interbase, and
Firebird, versions are stored on disk, requiring garbage collection. I
got tired of hearing about sweep, so Falcon versions are only in
memory. That leads to a major pain the butt when memory gets low.
Nimbus has an even better solution that requires neither versions in
memory or disk garbage collection.

Yeah, I think I know that problem.

> "What I'm considering is extended JavaScript for SQL access (probably
> a hundred ways to do this). The client would pass a JavaScript
> request (or the name of a predefined JavaScript object) to the
> server, the server would execute the given procedure, and return
> whatever the procedure returned to the list. The return might be an
> object, a string or number, or list of records and values, a number
> of lists of records and value, correlated or serial, or whatever."
>
> As I understand your question, you are looking for a language system
> that (i) easily queries a database (in SQL, from your earlier posts),
> (ii) has the ability to concisely manipulate component queries into
> results, and (iii) can communicate the final result in an easily
> objectifyable format, for example json or xml or whatever. Correct?
>

Yup.

> "JavaScript is a little loosey-goosey on types for my taste, all
> numerics as doubles doesn't please me, but otherwise looks
> interesting. I've never seen an ad hoc database stored procedure that
> I've like. Heavy duty server side stuff is better in Java, but Java
> is a tad heavy for the application."
>
> I guess that this requirement relates to highly lightweight
> instantiation of procedure execution, in order not to be dependent on
> caching and hence kill high scalability. From this perspective Lua
> may be your best shot, especially as the compilation into bytecode
> can happen on the client or on the web/app server (the compiler is
> only some 35kB). It has a proven JIT compiler into x86 native as well
> (also weighing in at some 35kB).
>
> "Any other interesting suggestions?"
>
> Perhaps not interesting, but I've been looking at doing a custom
> language for SQLite similar to what you seem to be looking at. The
> interesting thing is that there is quite a bit of similarity between
> the Lua VM and the SQLite VM (I guess you met Richard Hipp whilst you
> and Ann were at the OpenSQL camp last month).
>

At first glance, Lua and JavaScript have a great deal of similarity,
though it appears that JavaScript supports objects and exception
handling while Lua does not. On the other hand, Lua is a smaller
language (does a language truly need 12 assignment operators?).

> The thing that I cannot yet resolve is how to enable easy MVC style
> development without using elaborate frameworks. The other toughy is
> how to keep the app developer learning curve short. Something
> JavaScript-like has a big plus there, as any web app developer will
> be familiar with it.
>
> The problem with JavaScript (and Lua and ...) is that when used in an
> OO way, every property/method access becomes a (hash)lookup, rather
> than an indexation and that slows everything down. You can compile
> around that (eg. Google's V8), but then you lose the lightweight
> thing. Or we simply dump the OO mindset.
>

I tend to use a central string table, so there is only one instance of a
string. This allows a hash on the string address and an equality match
by address compare. It doesn't eliminate the hash lookup(s), but it
reduces the cost to a minimum.

In fact, however, most database transaction are contain little more than
SQL and control flow, so in most cases, a lot of lookups are not required.

It also helps to think about the round trip each line of code may be
eliminating.

> I guess the only correct solution for all these issues is 42 :^)
>
>

Sorry, JavaScript supports only floating, so make that .42e2

--
Jim Starkey
President, NimbusDB, Inc.
978 526-1376