The mean misleads, part 2: more data for the doubters

Some background

A few weeks ago, I published The mean misleads: why the minimum is the true measure of a function’s run time, where I explained that taking the means of the run times of two functions (with the aim of selecting the faster one) will produce incorrect results more often than if you take the minimum.

This was met with quite a few people telling me that I was wrong, which meant one of two things:

• I was wrong
• I did a poor job of explaining why the mean is a bad metric

To assess my wrongness, it would help if I knew in what way I was wrong. But here’s a curious thing: of all the people who told me I was wrong, not one of them offered any evidence.

But upon reflection, I suppose I didn’t really provide any convincing evidence either. The first post contained a lot of logic and reasoning and rather complicated charts. I suspect that the open-minded folks who took their time to soak it all up saw the point. But when skimmed, the article was easy to dismiss. Particularly if a reader had been primed by Reddit comments, which were unanimous in their decision that I was wrong.

The other thing I failed to get across was that I was comparing the mean and the min, and not saying that these were superior options to analyzing your data and using your brain to make intelligent decisions. But many seemed to think that’s exactly what I was suggesting. So just to…