This is a brief conversation I heard on Talk of the Nation, the program segment was on drowning deaths:
CONAN: Of the cases that you’ve studied, would you think that what percentage would think were – these were preventable deaths?
Dr. MODELL: Well, it’s hard to put a percentage because it’s a skewed series. The only ones in the series are people who died and then ended up in court, not the ones that were saved because the lifeguards or someone else did the proper things. So the numbers don’t mean that much
I should not be surprised but after reading so many blogs and even WSJ’s opinion pieces that are filled with selection, survivorship and other cognitive biases, I did not expect someone participating in a on-the-air conversation to notice and deftly avoid selection bias.
Here is an article, ” A Selection of Selection Anomalies” (PDF)
One such case discussed in the article was the work of Abraham Wald:
During WW-II, the military observed some planes were returning with bullet holes in some parts and wanted to reinforce those parts. Wald was asked to help with this project. Intuitively it would make sense to add more armor to the parts that got hit so often. But intuition here has selection bias, the military is only looking at planes that survived and landed safely despite getting hit in these areas. Wald reasoned planes that are getting hit in other parts most likely did not survive and hence recommended adding armor to the parts that did not have bullet holes.
It is not enough to look for evidence that support our notions, we need to look for evidence that will contradict it.
Iterative Path 