Let us revisit the Madagascar movie scene I wrote about in my article on anecdotes and hypothesis testing. The main characters, Alex, Marty, Gloria and Melman find themselves on the lonely beach after they fall from the ship. Their conversation about goes like this
Yeah, here we are. Where exactly is here? Melman: San Diego. San Diego? Melman: White sandy beaches, cleverly simulated natural environment, wide open enclosures, I'm telling you this could be the San Diego zoo. Complete with fake rocks. Wow! That looks real.
Clearly, Melman was satisfied with anecdotes that fit his hypothesis that they were in San Diego, ignoring others that would falsify it. In the context of business decision making, be it a Startup’s product launch or a big corporation’s M&A, when we rely only on anecdotal evidence, we seek only what is available, convenient, spectacular, recent and support our hypothesis, ignoring those that contradict it.
Question arises, whether this methodology of decision making based on anecdotal evidence would suffice given the high cost and time requirement of gathering information the right way. Yes, collecting the right information and collecting them with rigor costs you time and money but the cost and time argument is independent of whether or not there are cognitive biases in seeking anecdotes. These biases push us to look for narratives that will make Madagascar into San Diego and ignore or cast aside those that say otherwise.
For the cost argument, we should ask,
- What is the value of information we seek? If the difference in the outcomes is trivial and is less than the cost to collect the information, then there is no need to spend precious resources.
- If we have already made up our mind and only want the proverbial data lipstick, then why bother with any type of data collection?
Yes, it takes time to collect information the right way and we are all told, in business books, business schools, by business Gurus and at work( and yes this is selective quoting) how we need to act with imperfect information, avoid analysis paralysis, fail fast, and how making a timely wrong decision is better than no decision at all. This is especially true in the Western culture, where our Heros are Gun totting cowboys who speak in Black and White, who shoot first and ask questions later and who are quick to take a stand on Monday and stick to it on Wednesday, no matter what happened on Tuesday.
But such arguments confuse bravado with confident leadership, running fast with getting to where we wanted, acting on wrong information with taking calculated risk based on imperfect information and cogitation with inaction. Look at the business history that is rife with failures,
- 250 of S&P 500 companies from 10 years ago are not in business anymore (source: iShares)
- Trillions of dollars have been written off because the investments were not what they were imagined to be (source: 10-K statements)
- According to SBA, 10% of Small businesses fail every year
- 90% of tech ventures fail within 3 years
- During the last 9 years: 24,000 companies funded by VCs, only 3% exited at $100 m+ (source)
Where is all those fast and “timely” actions taking us?
If the outcomes have high level of uncertainty associated with them and mean the difference between success and failure, shouldn’t you commit resources to reduce the uncertainty?
“In God we trust stories, everybody else seek data”
Tags: Customer Metric, Hypothesis