Decision Making Under Uncertainties – Statistical Modeling

Whether it is filling out March Madness bracket or making investment decisions on a venture it all comes down to scenario analysis taking into account the variabilities. Because it is the variance that kills you.

In past years The Journal used to run Blindfold March Madness Bracket to help us make data driven decisions to fill out the bracket and pick the winner. This year they simplified it for us and yet made it extremely sophisticated. They codified the priorities, added a way to introduce a level of randomness and made a sophisticated statistical modeling for scenario analysis.

In essence we all can be Nate Silver with this model.

Whether it is filling a bracket for entertainment or a key investment decision the process comes down to what WSJ recommends:

  1. Choose a starting point: What is important to you?
  2. Customize your priorities: Assign relative priorities – like is it the people? market? traction?
  3. Account for unknowns
  4. Understand your biases and quantify the level of bias you have – for example how you “feel” about the person
  5. Run statistical simulation

How do you make decisions under uncertainty?

Freemium has run its course

This was originally published in Gigaom.

  • “We are now seeing the end of the freemium model — signing up users for free and trying to upsell,” said Christian Vanek, CEO of the Boulder-based SurveyGizmo, in a recent phone conversation.
  • “6.5 million unique users is not all that it’s cracked up to be. I don’t want hits. I want revenue. I want a real business,” said Matt Wensing, founder and CEO of Stormpulsein an interview with Mixergy.
  • “Make a product people want to pay for,” said Marco Arment, founder of Instapaper, in a Planet Money interview.

Three easily available examples do not make indisputable evidence against freemium. Just like Dropbox, Evernote and RememberTheMilk do not make a case for freemium. But these three quotes reflect a return to the roots of marketing — starting with customer needs, choosing the needs you want to serve and getting your fair share of the value created.

In the oft-cited Hershey’s experiment that started the free-mania, behavioral economists from MIT tested customer preference for Hershey’s and Ferrero Rocher chocolates at two different price points. For one group, they offered Hershey’s at one cent and Ferrero Rocher for 26 cents. For another, they offered the chocolates at zero cents and 25 cents respectively. When the Hershey’s chocolate was free and the Ferrero Rocher chocolate was 25 cents, 90 percent of the participants chose Hershey’s. $0 price seems to have done the magic in driving customer adoption. The result became the foundation of the freemium school of thought — free is free marketing. First use the free version to drive adoption and build a large customer base, and then find ways to monetize that base by upselling the paid version and selling extras.

Ninety percent is an eye-catching statistic in books about the freemium model, but let’s stop and ask some basic questions about running a profitable venture.

  1. What do you know about your target customers?
  2. What urgent needs do the free and paid versions meet for these customers?
  3. Will the products remain relevant in the customers’ future?
  4. If fifty other sellers stand next to you and give away free Hershey’s chocolates, Skittles etc., what will happen to your share of the market?
  5. As a startup founder, which customers should you focus on first with your limited resources?

The five questions above are the key principles of marketing. Unfortunately, choosing a freemium model does not help answer these questions. Worse, it muddles the answers by misdirecting startup founders to focus on the product rather than customer needs. Stormpulse, a Web-based platform for managing weather risk, learned that free can attract all the wrong customers. The company’s CEO Matt Wensing told Mixergy, “Free brought us recreational users who tried us for superficial reasons, while those who found real value were the enterprise customers.”

Here is an alternative, which unlike freemium is neither new nor a fad:

Start with the customers, not your product. The product could be new but the customer needs are not. Whether it is a “bits” product with zero marginal cost or “atoms” product with non-zero marginal cost, customer needs come first.In fact, it is not a product until you have identified a set of customers whose needs you meet and who want to pay you for that value.

Make your choice. Stormpulse and the online survey platform, SurveyGizmo, both realized that a successful strategy involves making choices. They couldn’t go after every customer who is willing to try out their products. Instead, the leaders at both organizations chose to focus on enterprise customers, because these customers not only value the products but also have the budget to pay for them. Getting 90 percent of customers to take free Hershey’s chocolates with the hope that they will pay more for extras or will upgrade later is not a strategy. In fact, the presence of free products drags down the expected valueof a customer. Which is another reason why SurveyGizmo decided to downplay its free offering.

Get your fair share of the value created. As Instapaper’s Arment said, charging for the product is still the simplest of all business models. Product innovation does not mean business model innovation. If your product adds compelling value to customers, charging for it is simply getting your fair share of the value created. You do not have to be ashamed of making a profit.

A small percentage of a very large number is indeed a large number, but can your startup stay solvent while you wait for the conversion to kick in? Freemium only offers the hope that non-paying users will fall in love with your product and start paying for it. Shooting an unlimited number of free bullets and hoping some will hit the target is a shotgun approach to monetization. It’s time to take a deliberate and targeted approach. Or as Vanek told me in our conversation, “it is time to retire the shotgun.”

A Frequentist and a Bayesian Frequent a Bar

A Frequentist and a Bayesian have been going to this bar for the past five evenings. The bar is a special kind that offers only two kinds of beers – one is Amber and the other is Dark.

The first evening the Frequentist orders Amber and the Bayesian orders Dark.

The second evening the Frequentist orders Amber and the Bayesian orders Dark.

The third evening the Frequentist orders Amber and the Bayesian orders Dark.

The fourth evening the Frequentist orders Amber and the Bayesian orders Dark.

The fifth evening the Frequentist orders Amber and the Bayesian orders Dark.

This is sixth evening.

What are the chances the Frequentist will order Amber?

What are the chances the Bayesian will order Dark?

Your answer will tell you whether you are a Frequentist or Bayesian. A mindless dashboard driven manager or a hypothesis setting, data seeking, and dynamic decision maker.

Life is a series of probabilities – our job as decision makers is to seek relevant data to reduce the uncertainties.

For extra credit, what are the chances one of the two will be run over by bus or the bar shuts down due to numerous reasons? (Black Swan)

Recipe for Minimum Viable Versioning

jetblue-evenmoreProduct versioning is the simplest and fairest mean to price discriminate. While it may sound unpalatable to the idealistic minds, price discrimination is actually better for customers and businesses. That is when done right. Else it becomes operationally expensive, ineffective in market or worse turns off customers. That is where even established businesses fail. I provide a very simple and testable recipe to do product versioning.

First I want to expand on the benefits of price discrimination. Price discrimination is charging different customers different prices based on their willingness to pay. It is fair when customers willingly pay their prices over marketers enforcing it. It is clearly good for businesses because it helps maximize profits. It is good for customers for many reasons

  1. It allows them to experience a product they otherwise would not be able to afford- like a no frill discount airline that makes overseas travel possible for many.
  2. It allows customers to pay for only what they value and not pay for extras.
  3. It gives them the choice – sometimes they may want to splurge, while most times sticking to basics.
  4. it gives them the flexibility and convenience – like express line at amusement parks.

When done right, for businesses, the clear benefit is profit maximization. There are also other significant benefits

  1. Understand customer segmentation, purchase occasions and behavior.
  2. Optimize capacity – be it manufacturing capacity of breakfast cereal brand or web scale capacity of a cloud service.
  3. Refine (pivot) current product offerings to better tune to customer needs.
  4. Find the right price points for the products without tying down to a single wrong price.
  5. Clear separation of customers – like Nordstrom and Nordstrom Rack.
  6. Surface customers who have even higher willingness to pay.

So it is better to offer multiple versions of your product even if it in its MVP (Minimum Viable Product) stage. This may sound contradictory to the concept of MVP, after all an MVP is meant to test the demand and discover product-market fit with limited resources. How can a startup afford to invest in multiple product versions when it has only limited resources and is lean by definition?

Offering multiple versions of the same product does not mean investing in yet another product line and doubling your development cost. Especially it is not recommended for a startup that is trying to validate its MVP in the chosen customer segment. That leads us to a real simple and effective recipe for Minimum Viable Versioning.

The underlying economic principle is – customer value perceptions are different and if they perceive a value difference (real or not) they will gravitate towards the version that they believe offers them the most value.

Think about that for a minute. This means  you do not build three or four different products but make the customers think there are three or four products with different value offerings. That is you can build just one product and yet price discriminate if you can nudge your customers to believe there are multiple products.

Take for example the most quoted example of a printer  price discrimination where IBM added a chip to slow down print speed. While you could do such a product line change that does entail manufacturing costs and operational complexities.

What if IBM didn’t really add a delay chip but simply changed the specs to say slower speed?  A customer who values speed of printing is not going to be tempted by the lower priced “slower” version while the price sensitive customers are happy to buy the same.

Applying this recipe to the present day cloud services, you may offer a version that is limited in capacity but may not build any code to enforce that restriction. For example a 2GB capacity limit and 10GB capacity limit for two versions. Those who value more capacity will self select to higher priced larger capacity version and the price sensitive customers will stick to the lower capacity version.

Some of those who picked your 2GB limited version may end up using more than that. But so what? Most who believe they want more than 2GB self selected to your higher priced version and you gain better market adoption, better understanding of segment-version fit and better profit without changing the product one bit.

Remember the economic principle – it is not the real value difference as much as the perceived value difference. Value is in the minds of customers with their wallets out and ready to pay.

So go ahead and build one MVP. But offer three versions (Why three?) that CLEARLY and EXPLICITLY communicate to customers the value differences so they can self select. While measuring product market fit with MVP you now can measure price perception and segment-version fit. If you find exceptional demand for your “low-end” version, find a way to build it at lower cost so you get to keep more of the price as profit. Until then, they do not need to know it is the same product.

When you take risks based on data

Today is Super Bowl XLIX. One scoring play we are most likely to see today is the point after kick. Teams seldom miss it. It is a sure one point. They do have the option of going for two instead of taking the sure one point. Teams go for the two only when they are forced to. That is when they re in fourth quarter and a single point still puts then at a disadvantage or keeps the door open for the other team.

Successful two point conversion is not given. It is risky. But the time to take risk is when you have time and resources to recover if it does not pan out as expected. Not when when that is the only option. Armed with data it is easier to take risks than with just gut.

Why don’t they go for it every time? Should they go for it? Let us look at some data.

Today’s Super Bowl features defending champion Seattle Seahawks and three time champions New England Patriots. Here is how the team statistics look like (source)



In the past seven years Seahawks attempted five fewer than Patriots and have less than half the success rate of Patriots.
So it is likely Seahawks won’t go for two unless they are forced into it and are most likely to miss it.

Patriots have a 50% average success rate over the past seven years. That is an expected value of 1, same as assured point after kick. So they should go for two every time. There is only upside for them.

Price Discrimination At The Altar of God – Revisited

A while back I wrote about the single price model at a Hindu temple to bless cars, known among the community as “car puja”.  I then wrote about the value left on the table by not setting different prices,

The last time I walked past one such event at a East Bay Hindu Temple, I saw two cars parked side by side  waiting to be blessed. One was a BMW X series and the other a Toyota Corolla.  The price the temple charges for the ceremony? A standard fee of $25.

The value to customer is of emotional kind and not economical. The perceived value to the customer is of private kind and is different for each customer. For instance, the value of the ceremony to someone who bought Lexus is much more than the value to someone who bought a Corolla.

That said, the value of the blessing to a first time car buyer or someone who  bought the maximum they can afford (even if it were a Corolla) is high.

This screams out for practicing price discrimination.

Price discrimination is not about squeezing every bit of consumer surplus from every customer (in one model it is). It is about getting customers to willingly pay the price they can pay and still feel happy about it.

A regular reader of this blog sends an example of price discrimination at the altar of god done right. This comes to us all the way from Notre Dame cathedral in Paris. As one walks along the caverns of the cathedral one sees near every niche a bunch of votive candles lit. Next to the lit candles are stacks of new ones that you can pick up and light as offering.

The price? In reality it is €0.  But it is more nuanced than that. The set price is 0 but you are encouraged to leave donations. This is where the price discrimination done right comes in. A simple method would have been to offer just one type of candle and ask visitors to pay what what they want and expect them to pay their true value of lighting a votive candle. What the good folks at Notre Dame have done is a nicer and better form of pricing.

They offer the visitors not one candle type but three candles types – a very basic small sized version, a middle version and a highly decorative large version. The price? These are free remember? There are no prices but only suggestions, setting an anchor to donate and offering them three different options.


Price discrimination, despite its bad rap, is a good thing for all.