Jonathan Neve wrote:

>Hi all,
>
>I just have a little question/suggestion. When new features are added to
>the engine, that require the use of a new keyword, is it really
>necessary to ban that keyword from use as a column/relation? Couldn't
>the parser know, in the context, whether it's a field or a keyword? Or
>at least in certain cases perhaps? I don't know the internals of how all
>this works, but wouldn't it be possible in, for example, the "alter
>table" statement, for the engine to understand that "alter table table1
>add type integer" means that we're adding a field called "type", whereas
>"alter table table1 alter col1 type integer" means that we want to
>change the type of a field called "col1"? Would this make SQL
>interpretation slower/more difficult? Is that why it's not been done up
>till now?
>
>
>
>

The problem is with yacc. Tokens are declared in yacc by a %token
declaration, such as

%token PRIMARY

Tokens are referenced directly in the grammar like the following:

primary_constraint : PRIMARY KEY column_parens constraint_index_opt
{ $$ = MAKE_NODE (nod_primary, (int) e_pri_count, $3, $4); }
;

Yacc/bison assigns an integer to each declared token that is used in the
the generated parser. It also generates a file containing all tokens
defined as macros of the keyword number:

#define PRIMARY 401

When lex sees the word "primary", it must recognize it as a token and
return the lexeme 401 to yacc/bison. Unless it knows more about
context, it can't know whether to return "primary" as a known token or a
potential identifier string. An identifier string won't match the token
"PRIMARY" and the token 401 won't match an indentifier string.

The obvious trick is to make lex sensitive to the context, which is
known in the industry as a hack. Too many hacks, and the structure
crumbles.

The solution is to ditch yacc/bison in favor of a more flexible parser
that doesn't suffer with this disease. Writing a parse is a pretty
trivial exercise once you get the hang of it. But it is boring,
particularly with a very large grammar.


--

Jim Starkey
Netfrastructure, Inc.
978 526-1376