''(Note: these notes apply to a new design for Ratcl, not the current v4 code)''

*** Overview ***

Main differences between v4 and v6:

* The v6 design adds support for missing values and greatly improves on the functionality of mutable views (this was carried over from a "v5" interim design).
* In addition, v6 adds the Lua [http://www.lua.org/] scripting engine, which has typed data (including ''nil'') and for which there exists a JIT on x86 platforms [http://luajit.org/].
* The lowest level of built-in view operators are coded in C, as before.  On top of this, all remaining built-in vops are coded as small Lua functions.  The Tcl ''vopdef'' extension mechanism is not affected, it remains the way to add additional vops coded in Tcl.
* The "Vector" typedef in C on which everything else relies has been changed slightly.  This was the main reason to re-implement v4 in the first place.
* The C core is more modular, there are now three layers: a core C-only layer, a Lua module on top of that, and lastly a Tcl extension which wraps it all into a single extension.
* Name changes: the C core is now in a "base" directory (used to be "core", but that name gives some problems on Unix).  There is a Lua wrapper called '''lvq''' which allows using and testing this from Lua (it does not include Lua, it's a dynamically loadable module for it).  And lastly, '''tvq''' is the name of the C-coded extension for Tcl which ''does'' include the complete Lua runtime.  Ratcl remains a Tcl-only package which loads tvq, similar to the v4 setup.

Apart from that, many many implementation choices made in v4 remain identical in
v6, major portions have been carried over with little change, including MK file
load/save. Only a few basic C-coded vops have been ported so far (mid Dec 2007),
the more advanced ones such as set operators, joins, grouping/ungrouping will be
ported later.

*** Organization ***

My focus in v6 is getting the main data structures ''exactly'' right, and
structuring the code in such a way that it can be grown and even portions redone
without too much impact on the rest. With the extra Lua codebase, and the fact
that the result can be used from C, from Lua, from Tcl, and standalone, this
structuring is more important than before. I'm adding test suites for all this
at various levels, and need to have decent debugging support all the way down to
mysterious C-level crashes of course. Oh, and let's not forget portability.

The new C/Lua/Tcl layering is helping to organize the code already. Much of the
C code in v4 turns out to be for implementing various vops, all this code is now
treated as optional and placed in separate .c + .h files. The core engine is
located in base/vlerq.c with the public interface defined in base/vlerq.h - the
rest of the code includes v_defs.h for internal definitions needed by vops, etc.

*** Calling conventions ***

Right now (mid Dec 2007), I'm still exploring which mechanism to use to pass
arguments around, probably the v4 mechanism will be kept at the C level. This
uses the following API for view operators:

    vq_Type MyViewOp (vq_Cell args[])

The input parameters are passed in args[[0]]..up, the result value is placed in
args[[0]] and its type passed back as return value of the function. The type of
the arguments is ''implicit'', i.e. there must be agreement between caller and
callee, this is not enforced through the API because vq_Cell does not include
type information. For that reason, the core uses an additional naming convention
for the vops, by adding a suffix. For example:

	vq_Type MyViewOpCmd_VVIF (vq_Cell args[])

This identifies the vop as taking two views, one integer, and a float as input
argument.

*** View operators ***

As v4 shows, it's easy to come up with dozens and dozens of different view ops.
The trick will be to come up with a ''small'' set of operators which are then
combined in various ways to provide additional operators. Many of these
''derived'' operators will be coded in Lua.

Getting to grips with Lua is going to be unavoidable if you want to dive in and
participate in the core development of the Vlerq project - all I can say is: the
language is clean and small (see [http://www.lua.org/manual/5.1/]), although of
course the more advanced features such as metatables and closures do require
some time to master and appreciate.

It will be quite a challenge to come up with a good collection of vops: clear,
complete, yet not overwhelming. And of course well-documented - the [v6vops]
page is an attempt to start this process off. Please feel free to edit/extend
that page (via svn: it's at svn://svn.equi4.com/vlerq/ratcl/doc/v6vops.txt).

-jcw