Selection Bias

I’ve been mentally wrestling with how to write about bias for a few months now. It is at once such a simple concept, but also incredibly complex because of just how pernicious they can be. In so many aspects of life, we have some sort of feedback mechanism that we can look at to help guide us when we stray off our desired path, but our biases are so ingrained in who we are as a person that we often have no idea we are being impacted by them.

In short, a bias is a prejudice in favor of, or against, something. The list of potential biases is long, but for this post, I wanted to start with an example of one that is present virtually everywhere – and that is selection bias. Selection bias occurs when the selection of individuals, groups or data for analysis is gathered in such a way that true randomness is not achieved.

For starters, let’s look at something that our headlines are filled with – polls. We often think that polls are reliable sources of information, mostly because we see a poll having something like 1000 (or more) participants, and that that sample size is enough to be random. But all you have to do is start asking questions about the polls to see that they are not necessarily random.

First of all, only those participants who responded are included. If it’s a phone poll, only those people who pick up and answer the call are being included. The reasons why someone may or may not answer a random call are many (very busy so they screen calls, only have a landline and are retired and spend a lot of their day outside, among others), but selection bias is already at work.

Another - how were questions in the poll phrased? Depending on who is conducting the poll, the pollsters may be looking for a certain result, so there could easily be subtle keywords in the questions that nudge respondents in a particular direction.

I’m leaving the list of examples at two because I’m trying to make a point (quickly) – but the potential places where selection bias creeps into polls are numerous. Conducting a good poll is a battle against biases, and reputable polling organizations put a lot of effort into reducing potential sources of bias. Yet in the end, they still exist.

We can't avoid it - but if we can at least be honest with ourselves and think about where selection bias is creeping into our thought process, and what pieces of information we use to make decisions, we can start to mitigate it.

Recently, my wife and I were questioning how gourds propagate – commenting that in school, we learned that fruits propagate by being appealing to animals, who eat them, and then excrete the seeds in their solid waste. But gourds don’t taste good, so what gives? I mentioned that we are just used to fruits that we think taste good because they contain sugars that are sweet to us as humans. Boom – selection bias.

Have you ever gotten into something (a hobby, an activity - hell, just exposure to what your kids are interested in) and felt 'Hey, this thing is going through a popularity surge - it's everywhere I look nowadays!' and then when things change, you think 'That fad is over'. Boom - selection bias.

It's everywhere.