In Tech, Reasoning by Analogy is Bad

One of the easiest ways to “settle” a technical discussion is to resort to an analogy. It took me three minutes of browsing HN (why do I keep doing that to myself?) to find the first one today. When operating systems and computers are discussed, a car analogy is sure to pop up within minutes.

I want to argue that reasoning by analogy is bad when technology is involved. And, more generally, that analogies and “common sense” are bad things to rely on in matters of science and engineering.

The operating-systems-as-cars cars analogy started as a running joke that no one took quite seriously. Unfortunately, the wonders of SEO means that all I can find on Google now is recent Pinterests posts instead of the original, but I can show you another example from the same era, only with airlines instead of cars.

Unfortunately, we’re actually, seriously using these things today as part of (supposedly) meaningful discussions. You’ll see exchanges like these, which then quickly devolve into even more obvious nothingness:

A: It’s preposterous that I can only install signed apps from a single app store, or that I can’t fix my phone myself, it’s my device so I should be able to do what I want with it. B: Well, we learned our lesson from the automotive industry, where thankfully you can’t install any buggy, unvetted software on your car, or any potentially incompatible component. This is a good thing!

(This, and all other examples in this post are real, but I re-worded things a little to keep things anonymous and protect folks from the ghost of USENET past).

Reasoning by analogy enjoyed a veritable Golden Age, as part of an era which we now broadly refer to as “the Middle Ages”. The Middle Ages were not a dark age of ignorance, despite what we’re sometimes led to believe, but we have thankfully grown out of it and its many dubious habits.

Here’s why I think we shouldn’t take this one up again.

Science and engineering have rigorous first principles that are sufficient for almost everything except bikeshedding. When discussing the relative merits of restricting install origin, for example, there is no reason (let alone merit) to figure out analogies with other industries.

First, everyone understands the security and reliability benefits, what’s being discussed is a) whether the trade-off is worth it, b) whether the trade-off should be forced on paying users, and sometimes c) if whether the company doing the forcing can actually deliver the security benefits and d) whether or not the security benefits are really what they’re interested in.

Second, the automotive industry doesn’t do all of the lockdown because they care for the good of their customers (which history has shown that they tend to do only to the extent of regulatory requirements). Most of it is done to protect from litigation and and bad publicity. Both are expensive when you manufacture horseless metal carriages that travel at 120 kph on public roads, with 80 years of regulatory history on what automotive manufacturers are responsible for. Phone manufacturers have none of those problems, especially not the regulatory one.

Besides…

…you never really know where to stop with the analogy, so they rarely give you a good model. Applications with extensive customization options, some would say, are like recording studios — you have to fiddle with controls a lot but you get unique quality out of it. Sure, the reply goes, but you need highly trained audio engineers for that, whereas applications with no controls enable everyone to get things done efficiently and productively, with good enough results.

Easy, right? It’s common sense: lots of knobs to fiddle, so you fiddle with them a lot, and you waste a lot of time, but you get really good sound. No knobs to fiddle, no time to waste — you get “good enough” sound, but faster.

Let me bring out my most annoying phrase: well, actually…

Ask anyone who’s been to a recording studio: the knob fiddling gives you the wrong impression. You actually do things very efficiently and productively there, because unless you’re The Rolling Stones and have your own recording studio, you’re on a clock. (Also, keeping five people in a 4×4 sound-proof room playing the same songs eight times until it all sounds right tends to take a violent turn after 14 hours).

In fact, removing some of the knobs from the recording studio equipment would make recording sessions longer, not shorter, because when you can’t get a good (let alone the right) sound through processing, you have to get it through real-life physics, like using different guitar strings, filling the bass drum with blankets and pillows, and re-arranging microphones.

So what should we get out of this analogy? That applications with fewer customization options make you slower and less productive, at least as long as your work potentially involves re-arranging microphones in a sound-proof room?

Analogies rely on “common sense”, and it’s hard to say when common sense observations are right. “Common sense” is responsible both for Olber’s paradox, which is a milestone of astrophysics, and for the belief that the Earth is flat, which is definitely not. It tells you that an ant can’t reach the end of a rubber band that’s stretched faster than the ant can crawl, even though it can.

Not only can common sense be horrendously wrong — we’re so convinced about the validity of common sense observations that oftentimes we don’t stop to think about it at all.

If you can’t reason about something quantitatively, or from first principles, it might be a shortcoming of that field, and you’re only making it worse by prolonging it. A good example here is the hamburger menu. There are endless debates about whether it’s good or bad. Some of them attempt to derive an explanation from first principles, or to derive a logical answer by quantifying things (how long it takes to find something, how much you have to scroll, how easy it is to tap).

But no answer is reached, and the issue is re-visited weeks afterwards.

What the field of UX design needs in order to settle this once and for all is not the elusive right analogy, a holy grail of “you can think of it as…”. What it needs is a way to quantify things like how intuitive a control is, how easy it is to perform an operation, and where you draw the line between “this should work without having to read the manual” and “you should be reading the manual for that”.

Not an analogy, but a historical observation: audio technology would be a lot worse if we tried to settle discussions about amplifier design based on “how warm it sounds” as opposed to harmonic distortion.

Analogies aren’t useless, but they’re an intermediary stage of understanding, and you shouldn’t stop there. There’s a famous “water circuit analogy” for electrical circuits, which likens voltage to pressure and current to debit. It’s good enough to explain some basic facts, but if you want to make informed decisions about any matter related to electricity, there’s really no alternative to learning Ohm’s law and Kirchhoff’s equations.

When elaborating a new model for a phenomenon, you do work with analogies. Finding a value through the gradient descent technique is “like” descending a hill, but you can’t write a program with that, or find anything with it — the technique is useless until you write down the math.

It’s the same with anything else: “it’s like a car” can be a good step to understanding things, but you’ll never make a real, useful, and valid assessment just with that.

Leave a Reply

Your email address will not be published. Required fields are marked *