I identify as a writer of both fiction and code. Most of what I post about here is on the fiction side of things, but I wanted to write about why I still work on software*.
Did you know that software is one of the most complex artifacts that people make, possibly the most complex?
I’m not referring here to number of components, although I could – the number of lines of code in a piece of software can easily be in the millions. I mean that the elements of a program are deliberately distinct and interact in surprising ways.
Consider a bridge. Bridges are interesting structures which have to be made to deal with the specifics of the locale, but how many distinct components are there? Lots of girders, cables, rivets and bolts; road surface panels and railings. Lots of parts, but how many types of rivets? Three? Plus two kinds of cable in different lengths, cable anchors, four or five railing components… there are hundreds or thousands of parts in a bridge but a tiny fraction of that number of distinct components.
A lot of physical engineering design is making standard components that can be used consistently**. Software is written to define the minimal subset of components that will capture all the behaviour of the system – the things that make up software are as different from each other as possible, and big systems can have hundreds or thousands of unique components.
Now consider the interactions between those components.
Sane software design minimises the communication between components – this is called loose coupling, in the jargon – but even the best-designed code may act strangely. Sometimes design decisions are made which have no impact at all in the original context of the decision, but can have enormous consequences later on. How about a C++ compiler that launches a web browser? Sounds bizarre, but what was a benign effort at reusing a common library in the normal case became a complete system lock-up if you were trying to compile multiple files at once.
These kinds of weird interactions happen occasionally in physical engineering (just look at the Tacoma Narrows bridge, for example) but they happen all the time in software. They are bugs of course and as such shouldn’t happen, but it’s the nature of this absurdly complex system that they occur.
I sometimes gasp in amazement that computers work at all.
There Is No Physics
Another interesting aspect of software is that there is no practical limit on what your program can do***, so the actual constraint is on what the program should do. A great deal of good software design is figuring this out, and it is akin to finding the plot and the characters for a novel. Doing this analysis comprehensively is enormously expensive and so ends up only being applied to safety-critical or inaccessible systems. NASA does this exceptionally well, for instance.
The current state of the art in making software good enough but cheaply enough to be affordable is to use agile methods: this is like driving a car where you constantly adjust course or react to road conditions, but the feedback you get is from user reactions. It’s not perfect, but it’s effective.
There’s a remark I’ve heard attributed to Alan Turing while he was building the first programmable computers**** that once they’d built the hardware then writing the programs would be easy. If programs had stayed as tight little balls of mathematical theorems then this might actually have been true, but as computers grew in size the programs they could run grew in complexity faster still.
Put simply, software is interesting because it is hard.
[*] apart from the risks of switching careers.
[**] none of this is to denigrate physical engineering – there are a lot of very interesting problems in engineering. Indeed, you’ll note that I don’t mention software engineering because I don’t feel that software is yet rigorous enough to deserve that label.
[***] ignoring for the moment such things as computability, problems with no algorithm for solution, and things that are generally hard.
[****] although I can’t find a citation, so it may be apocryphal