Looking for falsifying evidence

Here is a puzzle I saw in Gruber’s flash card for elementary school children.

More people who smoke will develop lung cancer than those who do not smoke.

What research will show smoking does not cause lung cancer?

This is not an argument about smoking, Big Tobacco, or morals. I like this question because it is simple, popular and familiar to most of us. The first statement makes us draw the most obvious conclusion – smoking causes lung cancer. Likely we wont look past this statement. And that is what makes the question very interesting.

The questions are, given all our knowledge and pre-conceived notion (so to speak), if you were asked to falsify the causation claim,

1. What research you will do?
2. What data you will seek?

This twist makes us stop, ignore our System 1 (Kahneman) and think. Finding one more example to support the claim is not difficult. Finding falsifying evidence is not only difficult but requires a different thought process.

You see numerous such causation claims in pulp-non-fiction business books (7-Habits, In Search of Excellence, Good to Great, Linchpin, Purple Cow) and blogs. Mom and apple-pie advice about startup, running a business, marketing etc. bombard us everyday in twitter. Our System 1 wants us to accept these. After all these are said by someone popular and/or in power and the advice is so appealing and familiar.

Penn Jillette of Penn and Teller wrote,

“Magic is unwilling suspension of disbelief”

For example the audience cannot ask to walk up the stage to look inside boxes. They have to accept the magician’s word for it. That is unwilling suspension of disbelief. When it comes to gross generalizations and theories supported only by the very data that is used to form them (e.g., What can we learn from xyz) we don’t have to suspend disbelief. We have the will to seek the evidence that will falsify the claim.

Do we stop and look for falsifying evidence or find solace in the comfort of such clichéd advice?

By the way, the answer to the Gruber puzzle is in looking for lurking variable. And there is none.