DN612149375

Subject: tcllib: experiences and suggestions for improvement - DN [1]

Mark Harrison <markh@usai.asiainfo.com> - 17 Apr 2000 - comp.lang.tcl

 As promised, here is my followup message regarding my
 experience with the preliminary release of tcllib:

 Good things
 -----------

 Even in its nascent state, it includes a lot of useful
 functions.  Having a standard library is good, etc...

 Bad things
 ----------

 1.  Varying interfaces

 As has been noted over the past 7-8 years, there is not
 a standard Tcl component model.  Therefore, everybody that
 writes a Tcl package has to create his own.  Different
 parts of tcllib follow different models of how components
 are put together.  Some do the FILE-like thing:

     set id [create-something]
     do-something $id

 Some do the widget-like thing:

     something foo
     foo do-something

 Some do subtle variations of the above:

     set foo [something]
     $foo do-something

 Some (like ::struct::tree) combine the two:

     tree t
     set foo [t insert root]
     t get $foo

 This applies to many other popular packages as well
 as to tcllib.  One of the first things a Tcl package
 author has to do is create his own packaging scheme.

 2.  It's ugly to create a package

 The ::struct components make an attempt to encapsulate
 their structure into abstract data types.  From the
 user level, it mostly succeeds (but see below for more
 comments):

     % stack foo
     foo
     % foo push 1 2 3
     % foo size
     3

 But from the implementation viewpoint, the code is just
 plain awful.  I spend a lot of time explaining Tcl
 programming to people, and is just plain difficult
 to explain how to implement an ADT using this method.
 It also confirms everyone's worst accusations regarding
 Tcl as being a hacky (bad sense) and one-off language.

 3.  Performance suffers

 Some ADTs, such as trees, need to have data associated
 with them.  The current method is to have "get" and
 "set" methods for each of the data types:

     tree t
     t set root -key name "Mark"
     puts [t get root -key name]

 Unfortunately, this kills a lot of the speed improvement
 introduced to Tcl over the past several revisions.  Some
 simple timing tests I've done, consisting of appending
 elements to a list:

     N    plain    incr    tcllib
     1000    15552    15645    835529
     2000    28475    30526    1936207
     3000    43242    45564    3013813
     4000    54734    60202    5957176
     5000    356977    85017    8348434
     6000    89041    100088    10148768

 plain:  a simple for loop, "lappend mylist $i" (control)
 incr:   itcl class, "lappend mylist $i", mylist is a class var.
 tcllib: tree, using "get, lappend, set", as above

 A component model is needed
 ---------------------------

 Having a standard tcl-level component model solves a lot
 of these problems.  This should be no news to anybody...
 We've rehashed the megawidget issue repeatedly, and this
 is just the language-level (non-gui) part of that discussion.

 Let's look at an example... I picked the smallest tcllib
 struct, ::struct::stack and converted it to an itcl class.
 This was a straightforward process and took about 20 minutes.
 I replaced the crufty initialization code with an itcl class,
 moved the data element to the class, and used the itcl::body
 command to incorporate the actual "real" code into the class.

 The code shrunk by about 25%, from 114 NCSL to either 81 or 74 NCSL,
 depending if the class methods were coded inline or after the class
 definition.  More important than the actual number of lines is the
 content of the lines that went away.  Let us look at the
 initialization code, and one of the small function bodies.

 I will explain the new version... I leave it to somebody with
 more patience and/or fortitude to explain the current version.
 Notice how you have to have (and explain) arrays of stacks,
 stack counters, and lines like:
 interp alias {} ::$name {} ::struct::stack::StackProc $name

 new:
 ---

 # create a class ::struct::stack which has the usual stack
 # operations.  Each instance of stack has a list "mystack",
 # which is a list of the data which has been pushed onto
 # the stack.  so after "push 1 2 3", mystack = {1 2 3}.

 namespace eval ::struct

     class stack {
         variable mystack

         method clear {} {}
         method peek {{count 1}} {}
         method pop {{count 1}} {}
         method push {arg1 args} {}
         method rotate {count steps} {}
         method size {} {}
         constructor {} {}
     }

     body stack::clear {} {
         set mystack [list ]
         return
     }

     body stack::peek {{count 1}} {
         #...
     }
     #etc...
 }

 old:

 namespace eval ::struct::stack {
     # The stacks array holds all of the stacks you've made
     variable stacks

     # counter is used to give a unique name for unnamed stacks
     variable counter 0

     # commands is the list of subcommands recognized by the stack
     variable commands [list \
             "clear"     \
             "destroy"   \
             "peek"      \
             "pop"       \
             "push"      \
             "rotate"    \
             "size"      \
             ]

     # Only export one command, the one used to instantiate a new stack
     namespace export stack
 }

 # ::struct::stack::stack --
 #
 #       Create a new stack with a given name; if no name is given, use
 #       stackX, where X is a number.
 #
 # Arguments:
 #       name    name of the stack; if null, generate one.
 #
 # Results:
 #       name    name of the stack created

 proc ::struct::stack::stack {{name ""}} {
     variable stacks
     variable counter

     if { [llength [info level 0]] == 1 } {
         incr counter
         set name "stack${counter}"
     }

     if { ![string equal [info commands ::$name] ""] } {
         error "command \"$name\" already exists, unable to create stack"
     }
     set stacks($name) [list ]

     # Create the command to manipulate the stack
     interp alias {} ::$name {} ::struct::stack::StackProc $name

     return $name
 }

 # ::struct::stack::StackProc --
 #
 #       Command that processes all stack object commands.
 #
 # Arguments:
 #       name    name of the stack object to manipulate.
 #       args    command name and args for the command
 #
 # Results:
 #       Varies based on command to perform

 proc ::struct::stack::StackProc {name {cmd ""} args} {
     # Do minimal args checks here
     if { [llength [info level 0]] == 2 } {
         error "wrong # args: should be \"$name option ?arg arg ...?\""
     }

     # Split the args into command and args components
     if { [llength [info commands ::struct::stack::_$cmd]] == 0 } {
         variable commands
         set optlist [join $commands ", "]
         set optlist [linsert $optlist "end-1" "or"]
         error "bad option \"$cmd\": must be $optlist"
     }
     eval [list ::struct::stack::_$cmd $name] $args
 }

 proc ::struct::stack::_clear {name} {
     set ::struct::stack::stacks($name) [list ]
     return
 }

 Why itcl?
 ---------

 Let's not beat this dead horse any farther than necessary,
 but here are the reasons I think itcl is a good candidate
 for a standard component system.
  - it's the most popular
  - it works well
  - the poll on dev.scriptics.com shows good itcl numbers
  - etc, etc, blah blah

 (and review the old conference proceedings... when people
 say "big-name company X uses Tcl on big-name project Y",
 deep in the fine print you usually see they are using itcl.)

 I'm not keen to rehash the hot discussion topic  of the
 1990's (objects), but let us at least agree that hot discussion
 topic of the 1980's (abstract data types) has been resolved.
 We can have discuss the usual fine points

  - the CLOS model is better than the C++ model
  - but objects aren't REALLY necessary
  - objects are so oversold
  - itcl is slow
  - blah blah

 but let us try to move forward in some meaninful way on
 this issue.

 Mark.

 PS, you can reference all the above code at
 http://www.markharrison.net/tcllib

 PPS, sorry this got to be so long... but I think these
 are some important issues for the longterm viability
 of Tcl.

 --
 Mark Harrison                     markh@usai.asiainfo.com
 AsiaInfo Computer Networks        http://www.markharrison.net
 Beijing / Santa Clara             http://usai.asiainfo.com:8080