Quality of Decisions vs. Quality of Results

Do you judge a person by the results of their action or the decision making process that led to those results?

Judging from the most overused terms on LinkedIn we all seem to focus exclusively on results.   That is likely a result of our focus on being, “action oriented”.  Action leads to results while (decision) analysis leads to paralysis – so the saying goes.

When you judge only by the results can you identify sheer dumb luck?

Writing in the context of Poker betting, Poker theorist David Sklansky, writes

What matters is the quality of your decisions, not the results that come from them.

Seems irrelevant to quote Poker process to businesses? Not if you recognize the essence of business is decision making under uncertainty. We make “investment decisions” everyday with resources and people.

We don’t know the exact market size and which segment to target first. We do not know what prices to set. We don’t know what benefits customers value more and hence what products to build. We do not know what channels to develop and what promotions to run.

If we have a rigorous decision making process  we can estimate market sizes and pick the segment that offers best chance of success, tease out distribution of customer willingness to pay and set a price, we use statistical models to determine customer value distribution, we evaluate channel and promotion impact from past data and pick the most likely one to succeed.

There is lot of uncertainty in the results. Other factors may crush your market size. Your pricing could be all wrong. Customers may reject the main value proposition due to other factors. Your channel choice may fall flat because customer buying behavior changed. Besides all these your competition may take steps that nullifies all your actions.

Think about it, the uncertainty of outcome is not that different from Poker. Or is it Poker more like business strategy?

When you judge a person only based on the results of their actions you fail to understand if they are capable of delivering similar results in future especially when conditions are radically different. Someone with solid (and evolving) decision making process but with failed projects in the past however has the wherewithal to operate under uncertainty in the future.

It is like drafting a quarterback who threw a few game winning Hail-Mary’s vs one that plays the long game.

A person’s results do not tell the full story or tell the wrong story. Their decision making process however tells you the real story.  Let me rephrase the Poker quote,

It is the quality and repeatability of the decision making process that matters more than the results.


Today we use reason, mathematics and experimental test …

What Stephen Hawking writes about our model of Universe and its law seem to apply to business, management and marketing as well.


According to Viking mythology, eclipses occur when two wolves, Skoll and Hati, catch the sun or moon. At the onset of an eclipse people would make lots of noise, hoping to scare the wolves away. After some time, people must have noticed that the eclipses ended regardless of whether they ran around banging on pots.

Ignorance of nature’s ways led people in ancient times to postulate many myths in an effort to make sense of their world. But eventually, people turned to philosophy, that is, to the use of reason—with a good dose of intuition—to decipher their universe. Today we use reason, mathematics and experimental test—in other words, modern science.


Just look at the many different recipes, tools and best practices that are being produced everyday. Even when the methods use math, they lack rigor and most importantly lack a solid theory. In the absence of reliable way to evaluate these models, what is popular, acceptable and said by a Guru in high position has become indisputable business science. In the end we are all chasing away the wolves with our loud noises!

There is nothing new in marketing

This is a long quote from a 1967 article published in Journal of Industrial Economics*. This paper was written as a response to Galbraith’s theory of Consumer Sovereignty.

The sensible manufacturer works with the environment, not against it. He tries to satisfy desires, latent and patent, the consumer already has; it is much cheaper than creating new ones.

First, he tries to identify these desires. To do this he now has all the aids of marketing research. If he only researches into which detergent the consumer considers to wash cleanest, he may miss the fact that the consumer now also wants her detergent to be pleasantly perfumed.

That is why so many of the new products even of the biggest firms fail miserably in test market. It is rarely because they are poor products technically. It is because there is something in their mix of qualities that fails to appeal to the consumer.

Once the manufacturer has found out what he thinks the public wants, he has to embody it in a product.

When the manufacturer does find an answer at a reasonable price, he still has to sell it to the public. He may think the answer will work; he may feel the price to be reasonable. He does not know whether the public will see it as he does.

If you go further back you most likely will find yet another article saying the same thing in more arcane language.

Fast forward to present day and you have exactly the same concepts stated above packaged in so many different ways. Every Guru has a name for it, they want us to believe none of the existing methods work.

Unfortunately, when the audience suspends its skepticism or the Guru is popular enough, their re-packaged ideas take roots.

There really is nothing new in marketing. Only new catch-phrases that fit the language of the time.

*You will find a copy of the said paper from your local library EBSCO host.

Not the Product Manager you have in mind

When you think of best in class product managers, what companies come to mind?

All the valley companies? In fact don’t we have a label, “West Coast Product Management”? True or not sounds cool like NFL’s  West Coast Offense. If you were to do unaided recall survey among most in social media it is highly likely we will find these in the top of the list, (in no particular order)

  1. twitter
  2. facebook
  3. Dropbox
  4. Pinterest
  5. Square

When you think about what sets these top notch product managers apart, what traits come to your mind? There is a question in Quora that slices it even further,

What distinguishes the Top 1% of Product Managers from the Top 10%?

If we assume normal distribution of product manager quality levels,  this question asks what distinguishes those who are 2.34 sigma over mean from those who are merely 1.285 over mean.  That is some precision.

And the answer that received 2500 votes lists  a  long list key traits. I do not know how one can measure many of them objectively. And this most popular and long answer not once mentions the word, “customer”.  Other not so popular answers list customers and understanding customer needs.  But none of the answers take it to next step – from understanding and serving customer needs to getting fair share of value created by serving those needs.

Even my own survey (that used forced point allocation ) on product management skills, did not include aspects of customer value creation and value capture. If you think about it, all the traits listed in Quora (Design, Copywriting, …) or in my survey (Strategic thinking, Hustle, …) these are really secondary to the  True North function of a product manager.

  1. Understand customer needs – Analytic skills, Usability analysis, etc.
  2.  Decide on (prioritize) needs to serve based on value created and the share of value you  can get – strategic thinking, forecast and measure, …
  3. Build an offering, Maximally Valued Product,  that does it better than alternatives and in cost effective way – Simplify, Design, Hustle, Influence, make technical trade-offs
  4. Position it in the minds of customers  – Presentation skills, Copywriting and the rest
  5. Make it easy for customers to get it –  Sales enablement, Buying experience etc

After all, what is a product but a value delivery vehicle? And all those great design, frictionless UI and copywriting do not make a product until you define a set of customers whose needs you meet and who want to pay you for fulfilling that need.

Is that the product manager you have in mind?


Knowing and Avoiding Selection Bias

This is a brief conversation I heard on Talk of the Nation, the program segment was on drowning deaths:

CONAN: Of the cases that you’ve studied, would you think that what percentage would think were – these were preventable deaths?

Dr. MODELL: Well, it’s hard to put a percentage because it’s a skewed series. The only ones in the series are people who died and then ended up in court, not the ones that were saved because the lifeguards or someone else did the proper things. So the numbers don’t mean that much

I should not be surprised but after reading so many blogs and even WSJ’s opinion pieces that are filled with selection, survivorship and other cognitive biases, I did not expect someone participating in a on-the-air conversation to notice and deftly avoid selection bias.

Here is an article, ” A Selection of Selection Anomalies” (PDF)

One such case discussed in the article was the work of Abraham Wald:

During WW-II, the military observed some planes were returning with bullet holes in some parts and wanted to reinforce those parts. Wald was asked to help with this project. Intuitively it would make sense to add more armor to the parts that got hit so often. But intuition here has selection bias, the military is only looking at planes that survived and landed safely despite getting hit in these areas. Wald reasoned planes that are getting hit in other parts most likely did not survive and hence recommended adding armor to the parts that did not have bullet holes.

It is not enough to look for evidence that support our notions, we need to look for evidence that will contradict it.

Belief that Entrepreneurship is risky fosters risky ventures

You have seen my attempt to analyze data provided by a VC firm on how they decide to invest in startups. Contrary to what they thought they were doing there was just one factor that decided investment decision.

Do VCs make informed evidence based decisions by meticulously rating startups like the data we saw led us to believe? That requires a meta analysis across all VC firms and someone just did that.

These are some quotes from a  article by http://goo.gl/WivWG“>Stanford GSB Professor, Jeffrey Pfeffer, on the need for Evidence Based Management in Entrepreneurial environments:

  1.  … it has become conventional wisdom, accepted by all the parties ranging from entrepreneurs to those who provide them financing, that a high rate of failure is an inevitable consequence of doing new things, inventing new technologies, and opening up new markets—activities which are inherently risky and uncertain because they involve doing things that have not been successfully done before. Because this conventional wisdom suggests that a high failure rate is inevitable, there is often little effort expended trying to improve decision-making in new venture activity.
    (In other words, people start ventures without trying to validate customer demand and VCs invest based on all kinds of criteria but validity.)
  2. Many of the VC firms do what they do without much introspection or reflection, partly as a result of the egos and self-confidence of the VC partners. People who have survived and prospered in the venture industry have obviously done well, and those VC’s who don’t do well generally don’t last. Therefore, it is axiomatic that most fund managers (those who survived and prospered) believe they are much above average in their abilities and in their decision making.
    (Hey, smart people succeed. If not they wouldn’t have succeeded, would they?)
  3. Positive qualities get attributed to the people, groups, or companies that enjoy those good outcomes whether or not these qualities are true or causal. This means that high-performing VC’s will be perceived as having individual skill as a consequence of their performance, whether or not such skill actually exists.
    (No wonder we bow at the altar of success. This finding was first stated decades ago John Kenneth Galbraith in his Conventional Wisdom essay.)
  4. Entrepreneurs, too, mostly have strong egos, which is what is required to take on something new where the risks of failure are high. But this overconfidence among entrepreneurs and those that back them makes it difficult for people involved in creating new businesses to question things and to learn from setbacks and other experience.
    (Everyone is killing it! Disrupting status quo! I hope they would stop at that and not write seemingly erudite articles on brain science.)
  5. Most venture capitalists and entrepreneurs believe that outstanding individual people make the difference, leading them to focus on finding and recruiting stars and to eschew much attention to process, including decision making processes.
    (If you are already successful you are perceived to be outstanding and thought of as having success potential)
  6. Few of the participants in entrepreneurial activity suffer significant consequences from unsuccessful decisions, and therefore many players have less incentive than one might expect to improve their decision-making  – VCs get guaranteed principal and Entrepreneurs often, although not always, are working with other people’s money, so their financial downside, except in terms of the opportunity costs of their time, are also limited
    (Because failure is most often seen as an unavoidable risk of being an entrepreneur, there are few if any career risks for starting something that doesn’t work out!)