[[title

          Tla 1.3.1 Librification Experiment Progress Report
                                 ---
                              2005-02-21
]]

  This document refers to the `tla' development branch:

  [[cartouche

	/Archive:/ `lord@emf.net--librify-tla-2005'

	/Version:/ `tla--factor-1--1.3.1'

        *(See <"links on the home page" -- $/index.html> for 
        help finding my archives.)*

  ]]



* Context

  Last week I reported having just spent a week modifying the `libawk'
  part of `tla' to permit string-sharing between multiple relational
  and associative table entries.  (<"The bug report describes that
  work." -- $/bugs--devo--2005/all-bugs/libawk-api-bogosities.html>)

  I was surprised at how quickly that work went.  I had only meant
  to spend some time estimating how long the task would take.
  Working on the estimate, I couldn't resist working on the
  modification itself, and when the deadline for the estimate arrived
  I already finished the work I was supposed to estimate!   (I
  provided an estimate of "-3 days" -- undertaking the
  already-complete task should count as adding three days to our
  schedule. :-)

  That got me thinking: the `libawk' cleanups were a small part of the
  long list of changes that would be necessarily to incrementally
  transform `libarch' from its state in 1.3 into a more librified and
  more portable piece of code: friendly to scripting languages, GUIs,
  alternative front-ends, extension languages, and non-unix platforms.

  I have long assumed (<"as described elsewhere" --
  ../attic/tla-2.0.html> that an incremental librification starting
  from the 1.3 code base had poor chances for success.   

  But if the fixes to `libawk' took "-3 days", perhaps the rest of
  incremental librification wouldn't be so impractical.


* A Two Week Experiment

  Today marks the middle of what will be a two week experiment in `1.3
  librification.'

  The form of the experiement is that I am spending these two weeks to
  librify as much of `libarch' as I can subject to some constraints:


* Results After Week 1

** Executive Summary

  The experiment tries to answer the yes or no question:

  [[cartouche

     /The Executive Question/

     Is persuing the librification effort for 1.3.1 a practical
     strategy for persuing the high-level objectives for GNU Arch
     (such as Windows support, Unicode support, scripting and
     extension language support, a demonstrably/visibly robust
     implementation, etc.)?

  ]]

  One week into the experiment I will leave my betting money at 90/10:
  there is a 90% chance that the answer to the executive question will
  be a clear "yes" at the end of the two week experiment (which will be
  28-Feb-2005).

  One modest but interesting demonstration of the benefits of
  librification might be <"the improvements to error reporting" --
  ./tla-error-sample.html> that it might lead to.


** Technical Summary

  See also <./tla-fn-anatomy.html>.

*** Last Week

  I spent most of the first week laying down a foundation for
  librification.   That included:

  /Factoring the source tree./  I set up a framework for splitting
  up files into multiple directories organized around modular
  and "modular cluster" boundaries.   The old contents of `libarch'
  now live in a directory called `libarch-compat'.   Those files will
  be incrementally deprecated: removed one by one and replaced by
  librified replacements in other `libarch-*' directories.

  /Setting up a new front end./ My plan is to librify "from the top
  down" as much as possible.  I'll work in a loop: pick a an arch
  subcommand; rewrite it to use only librified code (no code from
  `libarch-compat'); repeat until there are no unlibrified commands.
  Therefore, I set up a new `tla.c' (the home of `main').  The new
  front end first looks for a librified version of the subcommand.  If
  it doesn't find one, it runs the subcommand from `libarch-compat'.

  /Designing and implementing the error signalling mechanism./
  `libarch' needs to consistently and robustly signal errors rather
  than (in the manner of 1.3 and prior) often simply exitting on
  discovery of an error condition.  Part of what I did this week was
  to install run-time systemf support (`./src/tla/libach-errors') for
  error management.

  /Rebuilt `libawk'./  The `libawk' cleanup modified all callers into
  `libawk' to be robust in the face of a `libawk' implementation that
  shared strings between multiple table entries.   It also modified 
  the existing `libawk' code to actually share strings
  opportunistically, resulting in at least a significant run-time
  space savings.   Many librified functions will need `libawk'-style
  functionality but the existing `libawk' implementation does not
  provide for error signalling and recovery and, in other ways, does
  not conform to the requirements for a fully librified `libarch'.
  Last week, collecting ideas and code-scraps from both the existing
  `libawk' and the code base for `tla 2.0', I built a new
  implementation of the functionality in `libawk'.  The new `libawk'
  (now called `libarch-values'), in addition to be librified, adds
  support for table entries whose values are of types other than just
  string (e.g., integer-valued table entries).

  I also started on librifying the `my-id' command.  That invovled
  working on revised support for option parsing, on the API for
  functions implementing tla sub-commands, and work on writing
  librified versions of the low-level functions for manipulating a
  user id.

  This work went well in a few senses.  I was able to cut-past-edit a
  certain amount of code from both `tla 1.3' and `tla 2.0' to write
  what I needed in this context.  A great deal of the new code I
  simply rewrote, from scratch: this was code that is a minor
  variation on code that I've rewritten from scratch 3 or 4 times over
  the past few months.  The resulting code seems to work well,
  although testing has been scattershot.   I'm satisfied with
  the emerging calling conventions.

** Next Week and Possibly Beyond

  In week two I have a little more work to do on the foundation:
  string primitive operations; better option parsing; the beginnings
  of a more portable file system protocol stack.

  Beyond that I want to librify as much as I can in the remaining
  time.

  I'll consider the experiment to have produced a distinctly positive
  result (meaning that this approach to librification is worth
  persuing) if I can get through librifying the `file-id' command and
  some commands that pertain to per-user (`~/.arch-params')
  parameters.  Such an outcome implies an efficient framework for
  reimplementing CLI parsers, progress on librifying namespace
  management, project tree file system access, project tree arch
  control file access, and `~/.arch-params' access.

  A positive outcome will warrant a follow-on series of three "wind
  sprints": one each to librify `inventory', `mkpatch', and `dopatch'.
  Past experience has shown that, once those commands are in place,
  implementing (in this case, librifying) the rest of `tla' is a
  relative cake walk.
  



* Librification Experiment Constraints


  This experiment asks how long it will take to make a "clean up pass"
  over `libarch' such that, at the end of the process, the constraints
  described below are satisfied throughout the implementation of
  `tla'.

  [[cartouche

           /\\Librification Experiment Constraints/\\


    *Upward compatability* -- for several roughly one week intervals
    it is anticipated that only part of `libarch' will be librified.
    Nevertheless, `tla' must be fully operable at those intervals,
    passing both `make test' and changeset burn-in tests.  The intent
    is that it should be possible (and ideally useful) to merge
    partially-complete librification work into the mainline early and
    often.

    *Perfect Error Handling* -- Librified parts of `libarch' must have
    perfected error handling.  That means that they do not exit the
    process except under truly uncontinuable conditions -- most errors
    are signalled to callers.   Resource allocation and deallocation
    must be robustly handled across all paths, including
    error-signaling paths through the code.

    *Abstract String Handling* -- No part `libarch' code should make 
    presumptions about the internal representation of strings.
    Strings should be manipulated purely via procedural interfaces
    based on an ontology of code-point-index-addressable sequences
    of unicode codepoints.   Where specific codepoint values /must/ be
    presumed, only graphical and space ASCII characters should be
    referred to.

    *Reinforced On-disk Representation Abstractions* `libarch' has
     long internally had a rough layering of its filesystem access.
     The `vu' layer, from hackerlab, provides a low-level indirection
     above Posix system calls; for each of project-trees,
     `~/.arch-param' directories, and file-system archives arch
     includes a roughly procedural interface.  Within those three
     primary disk formats are ad-hoc formats for specific
     subcomponents (e.g, for files in `~/.arch-params' or for patch
     logs in `./{arch}').  Two of these subsystems (project tree and
     archive formats) have proven to need major restructuring for a
     clean port to Windows-based platforms.  Throughout the code,
     abstraction barriers are unevenly preserved with leaks across
     them exposing details of path names, descriptors, and so forth.
     A librified libarch needs to clarify the layering in these
     components and ensure that the API to them is sufficiently
     abstract that changes to them (such as for a Windows port) can be
     made easily.

   *Customizability, Extensibility, and Self-Documentation* Third
    party developers have made very clear the demand for robust
    scripting language bindings to `libarch'.  Work on arch GUIs, IDE
    bindings, and alternative front-ends suggests a similar demand.
    Some desirable capabilities in the core of arch, such as
    file-type-specific `diff' compuation and `patch' application,
    suggests a demand for an arch which is not merely *scriptable*
    (callable as primitive routines from a scripting language) but
    *extensible* (can be configured to call out to extension language
    routines during core operations).  The APIs, data types, error
    handling conventions, and available documentation used in a
    librified `libarch' must be scripting and extension language
    friendly.

  ]]



* Copyright

 /Copyright (C) 2004 Tom Lord/

 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
 the Free Software Foundation; either version 2, or (at your option)
 any later version.

 This program is distributed in the hope that it will be useful,
 but /WITHOUT ANY WARRANTY/; without even the implied warranty of
 /MERCHANTABILITY/ or /FITNESS FOR A PARTICULAR PURPOSE/.  See the
 GNU General Public License for more details.

 You should have received a copy of the GNU General Public License
 along with this program; if not, write to the Free Software Foundation,
 Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

 See the file `COPYING' for further information about
 the copyright and warranty status of this work.



[[null
   ; arch-tag: Tom Lord Mon Feb 21 08:56:34 2005 (communications/progress-2005021.txt)
]]

