My goodness, it's been a year since the previous entry. Much has happened. Much has evolved. Luckily, this isn't an account of the passage of time, just an ad-hoc collection of occasional musings.
In this installment, I'd like to talk about existing technology for a change. No dreaming about what could be, what my hopes are. Nope. Now.
I would like to start by reflecting a bit on how new software is created. It comes down to two aspects: state and flow. Software (any type of engineered product, probably) is normally built by defining a product, and then implementing it. So it's a bit like taking a rock and chiseling away at it until it is a work of art. It's also the opposite: using clay to build up the result from scratch. Whichever way you look at it, it's definitely state, and we're involved in making changes to that state - you start with nothing and you end up with something. We "build" software, we're called "authors", and we like to think of ourselves as "architects". The analogies with physical construction processes are clear, even though software is not about real matter.
But we're also deeply involved with process. We go through compile cycles and test cycles. We version our releases, we update code, and most importantly, we "ship" products and packages. We tend to spend inordinate amounts of effort on "fixing" things (well, I do, more than I care for) and on adding features, tweaking performance. We'd be nowhere if the state of our productions did not evolve in very clearly defined directions over time.
Software takes a lot of work, because state and flow don't go together well. The moment a certain state has been reached, the next requirement pops up, or some unforeseen consequence pops up, requring new action. It just never ends. I know of no software package which is simply good, and done. I see a box of matches, and can only marvel at how it does what it should do, and simply... is!
With software, we seem to never be able to leave good enough alone. Why is this? Is software never of sufficient value? I doubt it. It did what I wanted yesterday, why would it be different today? Ironically, of all things,software could in fact be timeless, something which few other engineered results can, since it has no wear and tear at all. Somehow, we can't make software be "done".
Of course software probably never is perfect. But what is? A matchbox with matches isn't "perfect" in any absolute sense either. But hey, they work for me, I'm definitely not looking for better matches. It would be very unusual to see matches advertised as being better! Improved! Faster! Who cares?
So why is software always in flux? Why do we keep moving on? I have no answer for this, but find it puzzling nevertheless. And worth thinking about.
Anyway, since change is a given, we ought to make sure we can effectively deal with it, right? But oddly enough, we're truly lousy at that kind of flow. Try thinking of adding one new feature or fixing one bug - and then how much of the effort will go into getting that change done - all the way to the destination system of clients and customers. Even when test suites have been highly automated - it's still tedious.
My conclusion so far is that we've complexified software development to the point where it has introduced a range of "self-inflicted problems" - and they CAN be avoided!
After this rather lengthy digression, I'd like to describe "Starkits", a technique which has been gaining acceptance as a development model for Tcl/Tk for some time. Starkits are deeply tied to that fundamental tension of state versus flow.
The normal lifecycle of a software product includes steps such as: design, coding, testing, packaging, deployment, and installation. It's been like that for decades, and with today's fast-paced product cycles it's been honed to perfection. But largely, this model has remained the same for many years.
There's a problem with this model. The trouble is that software tends to have an enormous number of dependencies - on the tools and libraries being used (and their configuration) during development, on the properties of the "target" system, and on the way deployment is performed. Software is horribly brittle - one tiny change, and it may end up not working. Which is why we set up huge scaffolds for revision control, release management, and installation to prevent problems at the target site.
It works. It's also a huge price to pay. All this effort is not spent on core design and core testing. Worse still, the extra steps all require additional design and testing as well!
Starkits aim to address this very issue. A quick intro: starkits are a way to "wrap" all files needed to run an application at the target site, without having to unwrap - i.e. a starkit stays wrapped up during actual use. A starpack is a refinement of this idea, whereby the main runtime executuble is also part of the packaged file. With starkits, deployment is about copying a file and running it. No install, no unpack, no dependencies. And no cleanup either: uninstallation is simply a matter of deleting the file.
Now by itself, the concept of package wrapping has been around for some time. But the real story of starkits is what I'd like to call a developer's lifestyle revolution. Let me explain.
The way I set up a new project, is to create a subdirectory for it, and then copy everything I need into it. Yes, copy, not link or symlink. This is done using some simple conventions. Then, I develop the code and test/debug it, 100% as usual. This is the heart of software development, this is where the real work gets done. This is where the state of the product evolves. This is what I get paid to do.
But when it comes to "shipping", starkits kick in. The whole area is wrapped into a starkit. No makefiles, no conditionals, no cleverness. Boom! It was a directory tree, and now I have this single file called a starkit (or starpack). Ship it!
The rest is fun. That's right, starkit deployment is a joyful experience - because It Just Works (TM) ! It has taken some years to get Tcl/Tk to support this model, but today it really is that simple. When collaborating with colleagues halfway across the globe, I just wrap and send a starkit. Same for showing progress to clients. Depending on the project and the portability of it all, it has happened several times that I found out later that they were not even using the same platform (portability is a story in itself - Tcl/Tk is a scripting language).
If you have doubts, or if you still struggle with the hassles that hunt you after you have created your software, then check out starkit technology. Starkits are not just a wrapping technique - they change the way software development is done.
© October 2005