Cliff L. Biffle blogged a great write-up of a debugging odyssey at Oxide with the title Who killed the network switch? Here’s the bit that jumped out at me:
At the time that code was written, it was correct, but it embodied the assumption that any loaned memory would fit into one region.
That assumption became obsolete the moment that Matt implemented task packing, but we didn’t notice. This code, which was still simple and easy to read, was now also wrong.
This type of assumption is an example of an invariant, a property of the system that is supposed to be guaranteed to not change over time. Invariants play an important role in formal methods (for example, see the section Writing an invariant in Hillel Wayne’s Learn TLA+ site).
Now, consider the following:
- Our systems change over time. In particular, we will always make modifications to support new functionality that we could not have foreseen earlier in the lifecycle of the system.
- Our code often rests on a number of invariants, properties that are currently true of our system and that we assume will always be true.
- These invariants are implicit: the assumptions themselves are not explicitly represented in the source code. That means there’s no easy way to, say, mechanically extract them via static analysis.
- A change can happen that violates an assumed invariant can be arbitrary far away from code that depends on the invariant to function properly.
What this means is that these kinds of failure modes are inevitable. If you’ve been in this business long enough, you’ve almost certainly run into an incident where one of the contributors was an implicit invariant that was violated by a new change. If you’re system lives long enough, it’s going to change. And one of those changes is eventually going to invalidate an assumption that somebody made long ago, which was a reasonable assumption to make at the time.
Implicit invariants are, by definition, impossible to enforce explicitly. They are time bombs. And they are everywhere.
Surfing Complexity 
Any code that is maintained long enough will have this issue in spades. When I worked at Sun, we often found assumptions in kernel code about things that had been true ten (or twenty!) years earlier, but now seemed laughable. Software gardening is critical; we just have to revist our (or our predecessors) assumptions every so often. Every part of the software “garden” needs spading at some point.
I disagree.
The article hinges on this incorrect assumption. To provide a counterexample, I’ll mention that mere modularity defeats this assumption. Functionality can be fully specified, in some limited context, and never thereafter changed. An implementation of a data structure would qualify as an example; its internal representations, affecting such things as optimizations, can change, but the interface to this functionality doesn’t change. It follows that systems should comprise smaller modules, so that ideally only the top-level, representing the particular program itself, needs to change.
I use the Ada programming language, which makes these invariants explicit. SPARK goes even further with Ada. However, this quote is an argument in support of higher level programming systems, which have fewer and fewer assumptions. It’s entirely possible to specify something at so high a level that it need never change, and a system which processes mathematical equations would be an example. Writing every program at a level where worthless details pop up repeatedly and constantly is a recipe for disaster, yes.
This quote is an argument in support of small modules.
Very cute, but this is wrong for the reasons I’ve explained. Software can be perfect. If software be kept small, rewriting it becomes an alternation from changing it.
When I write code in Common Lisp, and especially APL, I don’t really suffer from implicit invariants, although some do remain. In addition to every method I’ve mentioned so far, these implicit invariants should be documented where they can’t be made explicit otherwise.
If invariants are known at the time of development, unit tests should be written to verify their continued value. Autoconf setups are full of these checks and are important for invariants not under the developers control, like return values in essential library functions that change between platforms. Behavioral testing suites are designed to validate that externally facing interfaces haven’t spontaneously started behaving differently. Linus Torvalds has noted that outsiders don’t depend just on your API (ABI), but also the behavior of that interface.
Behavioral dependencies also include timing, like when a call starts taking too long, they might start hitting timeouts. The end user treated the time it takes your code to do an operation as an invariant, although many developers are wise to this assumption nowadays.
Unfortunately, if useful, perfect software could be written, we’d have a lot more of it. Too many leaky abstractions.