Wednesday, May 13, 2015

What random feels like

In the pilot episode of Numb3rs, CalSci professor Charlie Eppes (David Krumholtz) describes how people confuse randomness with uniformity. "You've distributed yourselves at equal intervals, while true random patterns will include clusters... [Simulating randomness] is pretty difficult."

Whether it's nature or nurture, we're really bad at knowing 'random' when we see it. This is what lets Eppes solve the case-of-the-week. This is why the Fire Emblem games badly underexaggerate their probabilities of peons hitting other peons -- because it takes 80% real odds for something to feel like 70% certainty.

Battle animation from *Fire Emblem: Sword of Seals*.
"If a unit has a 15% displayed chance... their actual odds of hitting are a paltry 4.65%... if a unit has an 85% displayed hit chance [it's] actually a more reliable 95.65%."


I recently installed TagTime for Android, a barebones app that randomly polls you every forty-five minutes on average1 to ask you what you're doing at that exact moment. It's a completely random sample of your day.

(The results are distressing. Over the last four days, approximately 1 in every 8 pings has occurred while I was on Facebook, i.e. three hours a day, give or take a generous error term. [Image link])

The thing about the 'random sample' part: it's really random. (Specifically, it's a Poisson process). The key part here is that it's totally memoryless -- you can't say "it's been nearly forty-five minutes so I'm going to do something productive now so I can write that down when the app next pings me". Nor can you say "Well, it just pinged me a second ago, so it's safe to hop onto Twitter for a few minutes". There's no gaming it (and thus no need to worry about gaming it). At any given moment, minute to minute, your odds of getting pinged are exactly the same.

As I mentioned, this really doesn't feel like how we expect random distributions to behave.

Google play review: "Randomizer is terrible I left the frequency at 45 minutes, but the actual average is about half that. Not sure what kind randomizer you're using, but it doesn't work well. EDIT: After running this for about a day, I'm uninstalling this app. It didn't ping me for 6 straight hours, now it's pinging every 5-10 minutes. This is worthless."

The reviewer in the above image, in addition to not understanding probability, gives a fairly good description of what having the app switched on feels like. Sometimes you'll get a ping from it three times in two minutes. Sometimes you'll watch an entire movie without it going off. A lot of the time it feels like I'll do a solid hour of work and then open Facebook for a minute and then immediately get pinged asking what I'm up to2. There's no predictability to it.

At any given minute, there's about a 2% chance of the app pinging me. Thus I've started to learn what "a 2% chance" really feels like. It's strange.

A 2% chance feels small but not that small. It sounds like nothing but it still manages to happen anyway, once or maybe twice an hour (and every now and then, five times).

There's perhaps a larger note about probability (fine, about independent probabilities) here: the odds of getting pinged during a given second are less than one in a thousand. And so on, and so forth. One in a thousand, one in a million... those odds may be small, but sometimes, over once an hour, even, things that unlikely still do happen.

It may be obvious on paper, but it feels a lot stranger in the real world when you first really notice it.

That seems like a good lesson to internalise.


1 For a user-defined value of forty-five.

2 In these cases I'd still say "I was on Facebook". Even if it doesn't feel fair, on average, over the course of weeks, the pings will cover a representative sample of your time.

On a similar note, a lot of the employee surveys at work come with instructions to answer the question about how you're feeling that very day, even if you're having an unusually up or down week. Across the whole sample, they assure us, a genuine picture of "typical" will appear.