Be Sufficiently Skeptical – Get Second Opinion on Business Prescriptions

Do you trust your doctor?

Someone you found after several referrals from your friends and social network?

Someone you have been loyal to for several years?

Someone who is over-worked, seeing patient every 10 minutes, double booked, and has no time to catch up on recent advances or practice evidence based medicine?

Someone who gets her/his scoop on latest medical advances from pharma sales reps?

In an article in WSJ, Holly Finn writes about the necessity of getting second medical opinion. What she says about doctors applies to all the management, marketing and business gurus you meet on the social media and the incessant business advice they dole out everyday in their blogs, Forbes articles and books.

Finn writes,

A 2010 Gallup poll found that 70% of Americans are so respectful of their doctor’s advice that they never get a second opinion or do additional research. We apply more scrutiny to choosing bluejeans, buying flat-screen TVs, ordering lunch. We should all get our heads examined.

Looking at the market for pulp-non-fiction books and marketing wisdom we see out there  and their continuing popularity over years I am forced to conclude the same about how we treat our gurus.

We love them. Re-tweet them. Re-blog them. We spread their incorrect theories based on spurious correlations and subject to hundreds of cognitive biases as indisputable business wisdom.

There is marketing lesson from Grateful Dead.

Some urging us to be weird.

There is this one guru who insults you outright by making a diagnosis that you have lizard brain.

Of course don’t forget entrepreneurial wisdom from 10 year olds.

We do not question them, ask what-else, seek contrary evidence or do our own independent research to check the validity. We take comfort in the power, position, pedigree and popularity of the gurus. In essence we are so respectful of our gurus that we suspend our skepticism and genuflect unconditionally in front of their wisdom.

We should all get our heads examined.

Don’t defer to your gurus. Be sufficiently skeptical. Question them. Seek second opinion.

Next time you read yet another indisputable wisdom coming at you from popular gurus, I am happy to provide that second opinion. Write to me before you hit that Retweet button.

At the very least, get a second opinion on the lizard brain. It is most likely not true.

The Real Limitations of Evidence Based Marketing

This is not a reply to Seth Godin’s post. While his post triggered this, the usage of the term Marketing is different between the two articles. To me, Mr. Godin’s article’s use of marketing reads more about marketing communication and messaging. I cannot agree more on the need to present the message in more interesting and acceptable format than just gobs of facts and data. We cannot hope that the rational automatons will figure it out, years of behavioral economics research has shown that we are anything but rational.

Marketing, to me,  means strategic marketing – activities that come way before we get to messaging. Like choosing the segment to target, the product versions to deliver, the pricing strategy and how to reach these customers.  This article is about that definition of marketing. However, Mr. Godin’s use of the phrase “Evidence Based Marketing” for a messaging tactic will create confusion in the minds of many. Using data and facts in messaging is a tactic, it is not Evidence Based Management.
Read on.

Evidence based marketing is flawed, it is rife with multiple errors. The common error types are

Inherent Data Errors: Data is noisy and incomplete. Sometimes the readings can be just plain wrong due to probabilistic nature of data source.
Environmental Errors: Context and environment collude to further muddy the water by tricking our eyes, minds and tools. Due to no fault of our own we see mountains when there aren’t any (like John Ross’ Croker Mountains).
Observer Effect: Call it Heisenberg uncertainty principle or Hawthorne Effect. The very act of reading the data can taint it.
Methodological Errors: Call it sampling error or survey error, our methods of data collection are inadequate. They introduce their own errors that are sometimes unquantifiable.
Data Collection Errors: We do not know what or how to measure. We ask the wrong survey question or add the wrong stimulus  but treat the received data as answer to the question we had in mind.
Analysis Errors: Test for statistical significance is flawed and misused. Not just Type-I and II errors, the very method of hypothesis testing is flawed. We collect data conditioned on the hypothesis being true and declare, “since data fit the hypothesis we conditioned to be true, the hypothesis is true”. We take comfort, wrongly, in using lower p-values. When data fit the hypothesis we stop, ignoring the fact that data can fit any number of hypotheses.
More Analysis Errors: We take comfort in adopting more rigorous and computationally intensive analytical methods. We treat correlation as causation and even regression as causation, look to the past to guide the future. We ignore lurking variables that could explain common causation.
Yet, Evidence Based Marketing is preferable to the alternative – based on guts, fads, Guru’s revelation, opinion of the person with highest title, and just plain observations.

The alternative is rife with errors that are not easy to see or measure – Cognitive errors. We suffer from multiple cognitive biases like Recency Effect, Availability Bias, Selection Bias and Survivorship Bias.Worse, the alternative rely on Commitment Bias and Agreement Bias (peer pressure) to push through a version of truth. Here the truths are unfalsifiable and it is forbidden to look for falsifiable evidence. Anyone stepping outside the accepted norm is labeled and shamed into compliance.

Evidence Based Marketing is flawed but it is clear and explicit about the flaws and lets the decision maker be the judge. The good practitioners know its flaws and bad ones cannot overlook it for too long.

Evidence Based Marketing eliminates the need for a charismatic leader and survives across leadership changes. The Alternative relies on the words and presence of the leader and the methods change when leadership changes. One myth is replaced by another.Both Evidence Based Marketing and its Alternative will take us to Madagascar but will try to tell us it is actually San Diego. Evidence Based Marketing would do that with data selection error, Alternative will just tell with authority.

When  uncovering new data shows otherwise, Evidence Based Marketing will reduce its certainty and state the likelihood that its prediction is correct.

Evidence Based Marketing is about forming a repeatable and falsifiable theory. The practitioners know that their is theory is true only because there isn’t data to prove it otherwise.

Which pill is it going to be for you?