Why compilers are doomed

by Coen Siegerink

Ok, I'll get straight to the point: software compilers as we know them today are on their way out. And with them also compiled languages such as C and C++...

Don't believe it? Let's look at some facts:

the purpose of a compiler is to generate fast machine code
experience says: 80% of the time is spent in 20% of the code
the size (and hence: complexity) of software is exploding
frameworks and special purpose libraries are widespread
re-use is a necessity, if only to keep s/w faults within bounds
open source software is spreading like wildfire

Let's add these things up: it is extremely likely that 3/4 of the time is spent in generic open source code. Another way to say this, is that compilation only matters for the generic parts of an application. Yet another way to say it: most developers need not care any more, and can focus on selecting tools, designing algorithms, and implementing more powerful concepts for the field they are working in.

But there is more, much more:

scripting languages are rapidly gaining acceptance (Python, etc)
binary software deployment is waiting around the corner
to mention the obvious: silicon is blazingly fast

To rephrase the above as arguments why compilation is about to go:

interpreted dynamic languages offer lots of advantages (RAD)
compilers/linkers: less and less people know or care about them
it's been a long time coming: productivity matters more and more

Imagine life without compilers: there would no longer be that awful schism between "developers" and "end users". The developer is still identified today as "the guy who has compilers and linkers installed on his machine". That in itself is ok, but considering how many people are introduced to computers every day, and the inevitable issue of making them do things for you (i.e. programming), it is only normal that more and more people will have no clue on what this thing called "compilation" is all about. IMO, this is good - compilation is an artifact of the way software development started out, it has been around for decades, it can now be jettisoned as no longer needed.

Thank you, everyone who made these beasts perform amazing tricks, we are all grateful for where it has taken us - now we can move on...

Wait a minute. Don't we desperately need the most advanced compiler systems for the "engines" of today? Don't we need them to produce the most advanced virtual machines, scripting environments, and operating systems? Should these all be written in assembler again?

Look at it this way: if the key performance issues are in generic code, if that code is open source, and if binary distribution becomes a standard way of deploying software... do you really think we need to distribute and maintain one or more compilers on every system? Of course not. What will happen is that the code base where performance matters will shrink, and that a few groups of people will pull every trick they know and invent to make that work fast. The rest will not ever want to get involved. With open source, they need not worry about losing control, or not being able to tweak things, they will simply not bother. I know I don't go tune my car - it's fast enough.

Furthermore, I predict that assembly language will come back more than before. Some person somewhere will invest and create some very fast special-purpose versions of algorithms, and we will be able to plug them in. This platform-specific code will be optional, with different people providing such versions for x86, PPC, Sparc, etc. When binary distribution is common, all it takes is three geniuses...

Related note: keep an eye on Forth, that language invented by Charles Moore in the 1960's. Its assembly-like qualities, and the fact that it smoothly bridges the gap from portable to CPU-tuned software, will return in force in the coming years. Not to make Forth programmers from all of us, but for some people - with 80% of all performance in generic open source code, such a temptation will make this inevitable.

Is this a revolution? Not really. We have seen exactly the same happen at the firmware level. Though proprietary, and with a level of investment which exceeds software development by far, the current chip industry (i.e. Intel, AMD, Motorola, etc) has done a good job of producing raw silicon power, and advancing that state of the art. As a programmer, would you want to get involved in VLSI chip design? Well, get ready for it: forget predominantly-compiled apps - fast!

So where does that leave today's tools and languages? My take on it:

scripting languages such as Python, Perl, and Tcl will boom
binary distribution (with compile/validation farms) will boom
performance will be an issue for generic s/w and special libs
there will be more compile-to-C / interpret hydrids like Dylan
some languages will generate C for internal use, like Squeak
C / C++ will become fringe languages, taking gcc down with 'em
Java and its hotspot technology will turn out to be irrelevant
compilers, linkers, and object code debuggers will fade away
"make" will be replaced by some sort of workflow-oriented tool
platform specialists will focus on generic software refinements
language/scripting tool specialists will focus on "pluggability"
look for Forth extensions coming soon to Python / Tcl / Perl
once Forth is there, on-the-fly machine code becomes feasible
modularity will become the key design issue, even more than OO
new extension architectures will span languages and platforms
re-factoring and re-factoring browsers will become standard
"the right TOOL for the job" becomes "the right LANGUAGE ..."
computer science classes will drop compilers as mandatory topic

Here ends my crystal ball gazing for now...

As someone who has invested over a decade in C and C++ each, I can't say I'd be sorry about such a trend, despite the fact that it will make my skills less relevant for the future. Today's deployment hassles are unbelievably stifling and unproductive - and anything which gets working software into the hands of people who want to use it is bound to be a major step forwards from the

build -> test -> package -> distribute -> build -> test -> use

nightmare we live in today.