The small science hack that disproves "data"
The Daily Drip
Show me the data.
It's so obvious, right? We all want to see the evidence that something works before we try it.
That goes for shampoo, a used car, or new legislation.
We want someone to "prove it."
Here's a quick story about data.
Sometime in the early 2000's, Blockbuster said:
"Our data show that people still like the experience of walking into a retail shop and picking a movie."
That was true. That's what people self-reported.
The problem is that people don't always know what they want.
So, when faced with the decision of trekking all the way out to a Blockbuster store or sitting at home on the couch and ordering this new thing called Netflix for $7 . . . well, we all know the end of the story.
The data weren't wrong, they just weren't what the executives thought.
So, the next time you see someone touting "data" to support their point of view, ask them where it came from.
Ask what they're trying to achieve. Ask how we know that these data are, in fact, the data we're looking for.
In science, the process for getting around this is called a "null hypothesis." It just means that instead of trying to prove what you think is true (your hypothesis), it's more rigorous to try and disprove it.
It's basically a little hack that we use to get around our human bias to only pay attention to the evidence that supports our existing world-view.
I think if we all started from the null-hypothesis a lot of pain and confusion would be avoided.
Subscribe to Strategic Altruism
Welcome to Strategic Altruism where I wrestle with entrenched ideals and philosophies to form mostly unpopular opinions about how to be a good person. It's a racket.