It is likely better to speak in absolutes

You read only interesting findings because only those get published, get written about and popularized in social media. Experiments that find no statistically significant difference don’t leave the filing cabinets of researchers because no one wants to read a story where nothing happens. This is such an experiment, where there was not enough evidence to reject the null hypothesis.

Let us start at the beginning. This experiment is about people’s perception of a person’s competence based on whether the person speaks in absolutes with no room for alternatives or whether the person speaks in terms of likelihood, accounting for alternative explanations.

There are several examples of those who speak in absolutes with no self-doubt. Read any CEO interview (enterprise or startup), management guru’s book or Seth Godin’s blog. Examples are,

“Revenue grew because of our marketing”
“Sales fell because of Europe”
“Groupon works, it really works”

An example of speaking in terms of likelihood comes from Nobel laureates in economics,

“Answers to questions like that require careful thinking and a lot of data analysis. The answers are not likely to be simple.”

Hypotheses: You do start with hypotheses before any data analysis don’t you?

Here are the hypotheses I had about speaking in absolutes/likelihoods and perception of competence.

H1: Business leaders are judged to be more competent when they speak in absolutes. Conversely, using terms like “likely” may be perceived as wishy-washy and hence signal incompetence.

H2: Scientists are judged to be more competent when they use likelihoods and avoid absolutes. (Because scientists are expected to think about all aspects and anyone who zones in on one factor must not know how to think about acenarios)

Of course the null hypothesis is there is no statistically significant difference in perception of competence based on whether the subject in question speaks in absolutes or in likelihoods.

Experiment Design: So I designed a simple 2X2 experiment, using SurveyGizmo. You can see the four groups, Company Executive and Scientist as one dimension, Absolutes and Likelihoods on the other. I designed a set of 4 statements with these combinations. When people clicked on the survey they were randomly shown one of the four options.

Here is one of the four statements

This was a very generic statement meant to speak about results and what could have caused it. I avoided specific statements because people’s domain knowledge and preconceived notions come into play. For example, if I had used a statement about lean startup or social media it would have resulted in significant bias in people’s answers.

Based on just one statement, without context, people were asked to rate the competence of the person. Some saw this about Scientists, some about a Company Executive.

Note that an alternate design is to show both Absolute and Likelihood statement and ask the respondents to pick the one they believe to be more competent. I believe that would lead to experimental bias as people may start to interpret the difference between two statements.

Results:  I collected  130 responses, almost evenly split between four groups and did t-test on the mean rating between the groups (Scientists: Absolute/Likelihood, Executive: Absolute/Likelihood, Absolute: Executive/Scientist, Likelihood: Executive/Scientist). And you likely guessed the results from my opening statements.

There is not enough evidence to reject the null hypothesis in all the different tests. That means and difference we see in competence perception of those speaking in absolutes and likelihoods is just random.

What does this mean to you?

Speaking in absolutes, a desired trait that leaders cultivate to be seen as competent and decisive leader, has no positive effect. Including uncertainties does not hurt either.

So go right ahead and present simplistic one size fits all solutions without self-doubt.  After all stopping to think about alternatives and uncertainties only takes time and hurts ones brain with no positive effect on the audience.

Caveats: While competence is not an issue I believe trust perception could be different. That requires another experiment.

Results from the Quiz on Probability of Retweet

Remember the question I posed some time back on finding probability of a tweet with a link being retweeted? The quiz was a fun way to make the audience realize for themselves the futility of any tips they see on improving retweets. 390 people took the quiz and answered it, (40 because I asked, 350 because Avinash Kaushik asked).

Here are the results. Big thanks to SurveyGizmo for its amazing survey platform. The reports pretty much write themselves. You should not even try anything else for running your surveys. For all percentages you see the base is 390 responses.

For the first question I presented the only data that I saw in the report I quote. The answer distribution looks like this

One could say, close to two third are likely to believe and accept whatever is implied by the commentary associated with a partial finding. While 36% asked for more data  only a third of them asked for the right data, the percentage of tweets retweeted, that will help them answer.

After they answered the first question I provided the additional data, percentage of RTs. I provided as optios the correct answer , the two wrong answers from previous question and two bogus answers. The answer distribution looks like this

About one in five found the answer (answer is 16%). It is likely that, even in the presence of additional data,  four out of five people can be convinced to accept a different answer. For example, when a spurious conclusion is presented in the form of a fancy infographic or presented by someone popular. When you see 5000 people tweeted an infographic that talks about scientific ways to improve retweets, it is hard to stop and do the math.

The takeaway is, it is hard for folks to stop and take a critical look at social media findings reported. It is even harder to seek the right additional data and do the math. So most yield to mental shortcuts and answer the easy question.

Note that this 16% number is calculated only to show you what is the average.  But average hides segments. Likely there are multiple segments here. For some, link or not, everyone of their tweet may be retweeted.  The only takeaway is this is a probability calculation and not a recipe and as we collect more data the probability will change.


Everyone is a Business Expert

If escape from the poverty of your own imagination is your reason for exploiting the stories history offers, or if you are taking refuge from another discipline in the belief that history is easy, without bothering to do the basic work, you will deserve to fail. –  FELIPE FERNÁNDEZ-ARMESTO writing on recent spate of poorly written history books

I considered writing a longer article on the many business experts,  the “lessons” they want to teach us from their everyday observations,  and their opinions on Netflix, HP, etc. But anything I add beyond this quote is redundant. Just substitute the word ‘history’ with ‘business strategy’ in the above quote.

Unfortunately, some of the new business writers, despite their lack of basic work won’t fail. Some will go on to become incredibly popular and will dictate social thought. The most popular ones will become role models for the next batch of business experts who go on to make their own astute observations about business from their trips to Justin Bieber concerts and farmer’s market or taking a taxi ride.

But that is not my concern, it should be your concern if you are adopting their ideas or hiring them.

Ideas that are convenient, popular, and acceptable have become sacrosanct

There is incessant and increasing attack on our intellect. We are fed simple ideas based on convenient samples and bombarded with regurgitation of weak ideas extended by other popular gurus.

As Galbraith wrote first in introducing The Conventional Wisdom by John Kenneth Galbraith, ideas that are convenient, popular and acceptable have become unquestioned truths. Times may change, new fads replace old fads – but our acceptance of what is convenient, familiar and popular as sacrosanct remains the same.

We now have instant communication, huge follower-ship, everywhere connectivity. Yet these remain the channels for spreading the same type of unquestioned ideas that suffer from cognitive biases and analytical errors.

Here are some nuggets from Galbraith’s essay, see how relevant these remain to what we see in social media:

  1. On attacking ideas: Anyone who attacks such [weak, incorrect] ideas must seem to be trifle self-confident and even aggressive. The man who makes his entry by leaning against an infirm door gets an unjustified reputation for violence. Something is to be attributed to be poor state of the door.
  2. Acceptable==Truth: Audiences of all kinds most applaud what is merely acceptable. In Social Comment, the test of audience approval, far more than the test of truth, comes to influence comment. (The more an idea gets blogged about, Retweeted …)
  3. Tribe Think:  Ideas come to be organized around what the community as a whole or particular audiences find acceptable. (Voting in Quora)
  4. Self-Esteem: We also find highly acceptable what contributes most to self esteem. The individual knows he is not alone in his thoughts – that he has not been left behind and alone.
  5. Power of Titles and Positions: Before assuming office, he ordinarily commands no attention. But on taking up his position, he is immediately assumed to be gifted with deep insights. Think of how we treat the ideas of those have the title “author, speaker” in their Bio. What would otherwise have been labeled as commonplace advice suddenly gets anointed as “inspiring advice” or “deep insight”.

It is time to call out that the Gurus have no robes!

Other articles:

  1. Evidence Based Management
  2. Informed Decision Making
  3. Fallacies of Cure-all Prescriptions