You’re stuck in an elevator with someone who read a tweet about a study on women in boards

Let me start by repeating what I wrote a while back on the faulty analysis by Credit-Suisse,

Here is an undeniable fact – considering ~50% of the population is women and 50% of the board of directors are not women, we can safely say women are underrepresented in corporate boards. Now let us return to the reported study in question that makes a faulty case for adding women board members. My arguments are only with errors in the research methods and its application of faulty logic. Nothing more.

I will add to this my case against those who keep quoting such studies as evidence for their side.

The title of this article is inspired by Adrienne LaFrance. She wrote a blog post titled , “You’re stuck on an elevator with someone who loved that Sarah Lacy article. Now what?”

In that article Ms. LaFrance makes point by point argument to what seem to be silly questions from a clueless elevator companion who fell in love with Ms.Lacy’s post.  For one such question Ms.LaFrance quotes as evidence the research by Thomson-Reuters,

You should check out this study from earlier this year that showed how diverse corporate boards outperform those with no women. You’d think that a company like Twitter would put its business interests first.

She isn’t alone in quoting this study, almost everyone taking   Mr. Vivek Wadhwa‘s side use this study. I am not sure how many read this report or looked at its methods and caveats. Let me do that in this article.

Here is the link to the said research report.

  1. Does the board matter?: The study starts with unverified assumption that a company’s board matters to its performance and then goes on to see differences in performance between boards. If your hidden hypothesis you took for granted is false it does not matter what your stated hypothesis is.
    What the study does is, If A=TRUE,  then A(With Women) > A(Without Women)
    You can see that if A =FALSE, the rest does not matter.You might want to stop here as nothing else matters after this error.
  2. Control Variable: When you want to study the influence of a single variable you want to make sure all other variables are held constant. But when you read this report it is clear that they have no way to do that. They started with composition of a company board in 2007 then compared the performance of a group of companies over a period. There are two many uncontrolled variables during this period  (tech trends, market trends, industry verticals, etc.) and these affected different companies differently.
  3. Error in Comparing Averages?: The comparisons are done on averages. There is a group of companies with mixed board and then there is another without women in board. The two groups are compared against another group, the benchmark which consist of companies of both kinds.

    The report says companies with women on board did marginally better or same as the benchmark while those with no women on board did 10-15% lower than the benchmark.

    First you notice that the difference in performance is not as significant s those who quote the study. Next you want to ask a simple clarifying question here.  If the benchmark has both types of companies, if one subset is  underperforming by 10-15%, shouldn’t the other subset outperform by 10-15% to bring it back to benchmark average?

    The only explanation I can think of is average hides details here. There must be a few companies in each side that are significantly different from the arithmetic mean for that group and they account for the difference. If you leave out these samples and compare again, the difference will likely vanish.

Now to the question of what to do when stuck in an elevator with someone who merely heard about the Thomson Reuter’s study?

Just smile and nod.

 

Does preschool lead to career success?

If you are reading this article it is highly likely your child has been in preschool or will attend preschool. But pick randomly any child from US population, you will find that only 50% chance the child goes to preschool.

The rest either stay home, where they play with parents or caregivers, or attend daycare, which may not have an educational component. Preschool isn’t mandatory, and in most places it’s not free. (Source : WSJ)

What is the observed difference in their later performance of those who attended preschool and those who didn’t?

According to Dr. Celia Ayala, research says preschool attendance points to stellar career.  She said,

“Those who go to preschool will go on to university, will have a graduate education, and their income level will radically improve,”

50% of children don’t get to attend preschool because of economic disparity. Seems only fair to democratize the opportunities for these children and provide them free preschool when their parents can’t afford them.

I do not have a stand on fairness but I have a position on the reported research and how they drew such a causation conclusion.

First I cannot make judgement on a research when someone simply says, “research says”, without producing the work, the data that went into it and the publication. Let us look at two possible ways the said research could have been conducted.

Cross-sectional Analysis – Grab a random sample of successful and unsuccessful adults and see if there is statistically significant difference in the number of those who attended preschool.  As a smart and analytically minded reader you can see the problem with cross-sectional studies. It cannot account for all different factors and confuses correlation with causation.

Longitudinal Analysis – This means studying over a period of time. Start with some preschoolers and some not in preschool and track their progress through education, college and career.  If there is statistically significant difference then you could say preschool helped. But you, the savvy reader, can see the same problems persist.  Most significantly it ignores the effect of parents – both their financial status and genes.
A parent who enrolls the child in preschool is more likely to be involved in every step of their growth. Even if you discount that, the child is simply lucky to start with smart parents.

So the research in essence is not actionable. Using it to divert resources to invest in providing preschool opportunity to those who cannot afford is not only misguided but also overlooks opportunity cost of the capital.

What if the resources could actually help shore up elementary, middle or high-school in low-income neighborhood? Or provide supplementary classes to those who are falling behind.

Failing to question the research, neglecting opportunity costs and blindly shoveling resources on moving a single metric will only result in moving the metric but with no tangible results.

Where do you stand?

When an idea stands alone, separated from its originator

I recently heard a NPR piece about the discovery of a copy of da vinci’s  Mona Lisa painting. Martin Bailey from The Art Newspaper had this to say in that piece

BAILEY: Well, I think maybe too much mystique has built up about this picture, the “Mona Lisa.” I mean, it is after all the world’s most famous painting. But people don’t look at it fresh. They look at it almost as an icon. And if you go to the Louvre, people aren’t actually really looking at the painting. They just want to sort of be in the same room with it. And for me, the beauty of the copy is that it actually makes us look at the painting as a painting, and I hope it will have that effect on other people too.

What Bailey says for world’s most famous painting and our reaction to it applies to famous ideas by famous gurus and our reactions to those.  We don’t seem to look at the ideas as just an idea when it is uttered by someone popular or with status.   As Galbraith wrote, anyone with position is assumed to be gifted with deep insights.

We may not grasp the idea nor we may analyze whether or not it is applicable to our case. But we just want to be part of the “conversation”. We tweet, retweet and write about it just to be in the same room as the Guru and his idea.

What if we are able to separate the idea from the one who said it?

What if I copied the idea of a Guru, word for word, intonation for intonation and stated it as mine? Will the mere act of copying make it stand alone?

Will we see the idea for what it is with all its biases?

 

Measure what is relevant, not what is available, convenient and sounds magical

Recently I saw a TV ad for a vitamin supplement. I do not know whether it is FDC approved or not. The Ad shows a woman drinking orange juice directly from the container (about 1 gallon). As she guzzles it with pain, the voice over says,

“You have to drink a whole gallon of orange juice to get the amount of vitamin-C in a single tablet of of our brand of Vitamin-C. It is the only tablet that gives you vitamin-C without the calories”

Very likely a true statement. But is it relevant? A single tablet of most such supplements has 1000mg of Vitamin-C and a single 8oz glass of orange juice  has 30-60mg. So a gallon bottle has 480-960mg of Vitamin-C. So a true claim.

What is our daily need for Vitamin-C? About 30mg. What will your body do with all those extra 970mg of Vitamin-C but to excrete it? Even if our body keeps the additional 970mg, what benefit comes of it? or worse, what are the side effects? What about other nutrients present in the glass of orange juice that are not accounted for?

This is  clever messaging, focusing customers attention on metrics that are readily available, prominent, convenient and advantageous to you even though the metrics may be completely irrelevant.
It is easy to dismiss such metrics when the claims are outrageous, when someone without authority/power/pedigree states them or after someone breaks it down. However, what if such a claim is stated by someone in authority, with pedigree/title or with some semblance of data and analysis?

  1. Group Buying sites that tout their ability to deliver an year worth of traffic to businesses – But at what cost?
  2. Marketing Managers who quote the number of times their product story was quoted in media and blogs – But at what cost and what is the incremental revenue?
  3. A Social Media guru urging businesses to invest in social media presence and measure the investment’s sucess with a new metric “Return on Engagement” – But at what cost and what is the incremental revenue?
  4. Shifting investor and market’s attention to growth in deferred revenue – but what if it came from pulling in future sales at a discount?
  5. Measure how big the customer smile is after they leave your store, it is a clear indication of profitability (okay I made this up, but the rest are not)

Proponents of such spurious metric and their followers are likely to answer the cost and incremental revenue question with, “in the long term it will pay off”. Your questions should be, How long a long term?, What other options would deliver the same “long term return”?, What is the long-term return?
Do you measure what is relevant or what is prominent, convenient, available, magical and irrelevant?

Sufficient but not Necessary!

The traditional media and the social media are peppered with stories on how one can achieve success like other successful entities.  Examples include, 7 habits, Good to Great, and numerous blog articles that follow the similar pattern.  Almost all of these articles look at a successful business or a person and look for observable positive traits . Then they attribute the success to the presence of such positive traits.

The general arguments against such studies include:

  1. Treating correlation as causation
  2. Different biases (survivorship, selection, availability, hindsight)
  3. Methodology errors like omitted variable bias
  4. We can’t stop because the data fit an hypothesis, data can fit any number of hypotheses.

Even if we set all these flaws aside and accept that indeed the success was the direct result of the positive traits there is another problem. These traits may be sufficient to the success but are they necessary?

Take an extreme example (for illustration). Let us say you observe a tall person in a fruit orchard. You observe her effortlessly pick much more fruits than others thanks to her height which gives her access to more opportunities. Her height was sufficient to get more fruits, but was it necessary?

Next time you see articles on “6/7/8/9 ways to do marketing/product-launches like Apple/Google/twitter/GratefulDead”, even if you look past the biases you should ask if the methods are relevant to your situation and are indeed necessary for your success.