The Two Greatest Software Systems Ever Built — And Why They Couldn’t Be More Different

Process-driven engineering vs obsessive craftsmanship

分享
The Two Greatest Software Systems Ever Built — And Why They Couldn’t Be More Different
Photo by CHUTTERSNAP on Unsplash

IT HISTORY

The Two Greatest Software Systems Ever Built — And Why They Couldn’t Be More Different

I want to tell you about two pieces of software.

One controlled a 120-ton machine carrying humans into space, made hundreds of decisions per second, and was not allowed to crash. Not ever. Not once. It ran for 30 years.

The other was a typesetting program. Built by one man. Because he looked at the proofs of his own book and decided the kerning was wrong.

Both are, by any serious measure, the best software humans have ever produced. And they were built on philosophies so opposite that they seem to argue with each other over time.

Space Shuttle Columbia on the launch pad at Kennedy Space Center, March 1981
Space Shuttle Columbia, five weeks before its first flight. Five computers would control every millisecond of what happened next. (NASA / Public Domain)

The Software Was Never Allowed to Fail

It’s April 12, 1981. Columbia sits on the launch pad at Kennedy Space Center.

120 tons empty. 2,000 tons of fuel. Two humans on board. In minutes, it will climb toward space at 17,500 miles per hour.

Everything that happens next — ignition sequence, attitude corrections, sensor fusion from thousands of inputs — is decided by software. Multiple times per second.

This software cannot crash. It cannot be rebooted mid-flight. A timing error of two-thirds of a second would throw the vehicle 5 kilometers off course.

The engineers who wrote it are sitting at desks in Houston, not watching the launch. They already know it will work.

NASA Mission Control Center during Apollo 16, 1972
Mission control during Apollo 16 (NASA)

17 Bugs in 11 Versions

The shuttle software team had 260 engineers. No famous names. No rock stars. The system was deliberately built not to depend on any particular person’s brilliance.

What they had was a process.

The numbers are almost impossible to accept.

The last three versions of the flight software — 420,000 lines each — contained exactly one bug per version.

Eleven consecutive versions. 17 total bugs. A commercial product of similar complexity typically ships with 5,000.

This is not a rounding difference. This is a different civilization.

What They Were Willing to Give Up

One-third of the entire development cycle was spent writing no code. Engineers sat with NASA and defined requirements — in writing, in obsessive detail, until both sides signed off on every sentence. Nothing changed without mutual agreement.

One example: adding GPS navigation to the shuttle touched 6,366 lines of code — 1.5% of the program. The documentation for that one change: 2,500 pages. Every branch, every failure mode, was committed to paper before a single line was edited.

Total documentation for the shuttle software system: 30 volumes. 40,000 pages.

IBM AP-101 Space Shuttle General Purpose Computer before and after 1991 upgrade
The IBM AP-101 flight computers. Four voted on every decision; the fifth ran completely different software, waiting to take over. (NASA / Public Domain)

The team also maintained two databases that served as a collective memory.

The first tracked every line of code — every version, every change, every approval, going back to the beginning. You could pull up any line and read its full history. Every line had a genealogy.

The second tracked every bug ever found, going back nearly 20 years. Not just what the bug was, but how it was introduced — and how it had survived every stage of review before finally being caught.

That second database wasn’t used to blame anyone. It was used to find the hole in the process and close it. Every escaped bug triggered a root-cause analysis. Every root cause triggered a process update.

Version by version, the net got tighter. Until the bugs essentially ran out.

When the team was asked whether this level of discipline killed creativity, the answer was blunt: yes.

Engineers follow the manual. There are people watching them follow the manual. You cannot improvise.

But the creativity didn’t disappear — it went somewhere else. You couldn’t be creative about the software. You could be creative about the process.

And they were. The Software Engineering Institute eventually modeled several of its own standards on what this team had built. The shuttle team didn’t adopt best practices. They became them.

The Man Who Stopped Over Kerning

Donald Knuth at the Open Content Alliance reception, 2005
Donald Knuth, 2005. He paused the most important work of his career to fix a typography problem. (Jacob Appelbaum / CC BY-SA 2.5)

In 1974, Donald Knuth won the Turing Award — computing’s highest prize. He was 36. He was halfway through The Art of Computer Programming, a seven-volume work his colleagues were already treating as scripture. He’d finished three volumes.

The ACM couldn’t wait. They gave him the award before he was done.

Then, a few years later, Knuth stopped writing.

His publisher had switched to digital typesetting. He received the galleys for the second edition of Volume 2. He looked at the pages. The mathematics looked wrong. Not catastrophically — most readers would never notice. But the kerning was off. The spacing between symbols was subtly broken. The beauty of the original hot-metal printing was gone.

To most people, this is a minor annoyance. You note it. You move on.

Donald Knuth is not most people.

He decided that before he could continue writing the most important work of his career, he needed to fix typesetting. Don't complain about it. Fix it. Himself. From scratch.

He took a sabbatical. He spent the next decade building TeX.

The Version Number That Converges Toward π

TeX changed academic publishing overnight. Physicists, mathematicians, computer scientists — anyone who needed to express precise notation in print — adopted it. Forty-five years later, it’s still the standard. If you’ve read a serious math or physics paper in the last four decades, you’ve read something typeset with TeX.

But what makes TeX remarkable isn’t its longevity. It’s the confidence embedded in its version number.

Most software: 1.0, 2.0, 3.0. Some use years. TeX uses neither.

TeX’s version number is an approximation of π.

3 → 3.1 → 3.14 → 3.141 → 3.1415 → 3.14159…

Tex version

Each release adds one more digit. TeX converges toward perfection the same way π converges toward a constant that never fully arrives: always closer, never done.

Knuth has announced that, upon his death, the version will be permanently set to π. Any remaining bugs will be reclassified as features.

He is not being ironic.

The Bug Bounty That Stopped Paying Out

To back up that confidence, Knuth created a bug bounty.

Find a real bug in TeX: $2.56. The bounty doubles every year — $5.12, $10.24, $20.48, $40.96.

Knuth has spent his career studying growth functions. He knows exactly what exponential doubling means. He made the offer anyway.

The bounty eventually reached $327.68 — and stopped. Not because he closed it. Because the bugs ran out.

A Knuth reward check. The people who received them didn’t cash them. They framed them. (Wikimedia Commons / CC BY-SA 3.0)
WIKI/Knuth_reward_check

The people who found bugs and received checks didn’t deposit them. They framed them. A signed check from Donald Knuth — proof that you found something he missed — was worth more as an artifact than as money.

Five Minutes a Day

Alan Kay — inventor of object-oriented programming, 2003 Turing Award winner — tells a story about a programming competition in the late 1960s.

Every Thanksgiving, Silicon Valley’s best programmers would gather for a contest. John McCarthy, the father of artificial intelligence, wrote the problems. The prize was a turkey.

The year Knuth competed, he beat everyone. Fastest execution time. Most efficient algorithm. On the slowest machine in the room — a batch-processing computer that returned results in cycles, not interactively.

Kay asked him how.

“When I was learning to program, I was happy if I got five minutes of machine time a day. I had to make the program run correctly on the first try. No debugging. Best algorithm from the start.”

Five minutes a day. First try. No iteration.

The scarcity had become a capability. Decades of constraint had carved a standard into him that nobody raised on fast, forgiving machines ever had to develop. When he finally sat down to write TeX, that standard went in with every line.

Two opposite philosophies. The same destination: software so close to perfect that the gap is almost philosophical.

Neither has been reached again.

One team chased bugs until there were none left. One man offered money until nobody could claim it.

Different roads. Same ending.


The shuttle software team’s story was first told by Charles Fishman in Fast Company, 1996 — “They Write the Right Stuff”. It remains one of the finest pieces of writing on software engineering. Donald Knuth’s account of building TeX is in his five-volume “Computers and Typesetting” series. Both are worth your time.


Support writers. All our non-profit’s offerings here.

Click here to subscribe to the IT Chronicles newsletter.
Click for Wordsmith, Mystery Writing, Write Like Stephen King, more

By the EIC Susan Brearley with Ideogram