You’re stuck in an elevator with someone who read a tweet about a study on women in boards

Let me start by repeating what I wrote a while back on the faulty analysis by Credit-Suisse,

Here is an undeniable fact – considering ~50% of the population is women and 50% of the board of directors are not women, we can safely say women are underrepresented in corporate boards. Now let us return to the reported study in question that makes a faulty case for adding women board members. My arguments are only with errors in the research methods and its application of faulty logic. Nothing more.

I will add to this my case against those who keep quoting such studies as evidence for their side.

The title of this article is inspired by Adrienne LaFrance. She wrote a blog post titled , “You’re stuck on an elevator with someone who loved that Sarah Lacy article. Now what?”

In that article Ms. LaFrance makes point by point argument to what seem to be silly questions from a clueless elevator companion who fell in love with Ms.Lacy’s post.  For one such question Ms.LaFrance quotes as evidence the research by Thomson-Reuters,

You should check out this study from earlier this year that showed how diverse corporate boards outperform those with no women. You’d think that a company like Twitter would put its business interests first.

She isn’t alone in quoting this study, almost everyone taking   Mr. Vivek Wadhwa‘s side use this study. I am not sure how many read this report or looked at its methods and caveats. Let me do that in this article.

Here is the link to the said research report.

  1. Does the board matter?: The study starts with unverified assumption that a company’s board matters to its performance and then goes on to see differences in performance between boards. If your hidden hypothesis you took for granted is false it does not matter what your stated hypothesis is.
    What the study does is, If A=TRUE,  then A(With Women) > A(Without Women)
    You can see that if A =FALSE, the rest does not matter.You might want to stop here as nothing else matters after this error.
  2. Control Variable: When you want to study the influence of a single variable you want to make sure all other variables are held constant. But when you read this report it is clear that they have no way to do that. They started with composition of a company board in 2007 then compared the performance of a group of companies over a period. There are two many uncontrolled variables during this period  (tech trends, market trends, industry verticals, etc.) and these affected different companies differently.
  3. Error in Comparing Averages?: The comparisons are done on averages. There is a group of companies with mixed board and then there is another without women in board. The two groups are compared against another group, the benchmark which consist of companies of both kinds.

    The report says companies with women on board did marginally better or same as the benchmark while those with no women on board did 10-15% lower than the benchmark.

    First you notice that the difference in performance is not as significant s those who quote the study. Next you want to ask a simple clarifying question here.  If the benchmark has both types of companies, if one subset is  underperforming by 10-15%, shouldn’t the other subset outperform by 10-15% to bring it back to benchmark average?

    The only explanation I can think of is average hides details here. There must be a few companies in each side that are significantly different from the arithmetic mean for that group and they account for the difference. If you leave out these samples and compare again, the difference will likely vanish.

Now to the question of what to do when stuck in an elevator with someone who merely heard about the Thomson Reuter’s study?

Just smile and nod.

 

Making a faulty case for women in corporate boards

Here is an undeniable fact – considering ~50% of the population is women and 50% of the board of directors are not women, we can safely say women are underrepresented in corporate boards.Now let us return to the reported study in question that makes a faulty case for adding women board members. My arguments are only with errors in the research methods and its application of faulty logic. Nothing more.

Here are the major highlights of the study

  1. The study analyses the performance of close to 2,400 companies with and without board members from 2005 onwards.
  2. Over the past six years, companies with at least some female board representation outperformed those with no women on the board in terms of share price performance
  3. We can now see a much clearer inverse correlation (–0.65 and –0.76 for Europe and the USA respectively) between the relative share price performance of companies with one or more women on the board compared with those with no women on the board and the overall market.
  4. The report identifies seven key reasons why greater gender diversity could be correlated with stronger corporate performance

I see several issues with this type of analysis and subsequent argument attributing causation.

  1. Does the board matter? We are not talking company executives who actively set strategy and make operational decisions everyday. We are talking about the board that meets for a day or two, four to six times a year and in most situations are not the experts in the vertical the company operates in. Do boards really have a role to play in a company’s stock performance? Some of the previous works on this topic found no contribution by the boards in a company’s strategy or stock growth. I am yet to see a report that looked at percentage of changes in stock price that can be attributed to the board.
  2. Leap from Correlation to Causation First the report found a correlation in stock performance then it goes on to provide seven reasons on why the gender diversity is helping stock performance. The implied statement here is, women board members were the reason for stock performance, let us tell you why they made the difference. That is an untenable leap from correlation to causation. It buries us with plausible narratives on how women board members would have made the difference without telling us how it can draw the causation argument.
  3. Flaw of averages  The study compared average stock performance of the two groups. Not pairwise comparison and not frequency comparison. Averages hide lots of detail. Take for instance Apple and S&P 500. While S&P 500 is down 13% from its 2007 highs, without Apple it would have been down addition 2%. What about such big movers in the two groups that disproportionately affect the averages? If we take them out, what would the performance difference look like?
  4. Not asking why some corporations seek diversity in boards  Isn’t the question here, what type of corporations would seek to strengthen its board and seek to add  diversity (not just demographics but ideas too) to help it perform better? Such a corporation is likely already doing many things right that is contributing to its performance. Attributing its performance to appointing women directors is confusing the cause.

For those interested, there is indeed one women member, Andrea Jung (ex-AVON chief) in Apple board. Can one attribute any part of Apple’s stock performance to Andrea Jung?

Since Sheryl Sandberg was appointed to Facebook board their stock has taken a turn for the worse, can one attribute causation to the appointment?