No Dearth of Pricing Experiments

Experiments are great. They help us test our hypotheses. If the experiments are not designed to test a specific hypothesis for a specific goal then what is the point of conducting them? If the experiments do not teach us something new or move us further in the direction of your goal, why bother?

Pricing experiments are no exceptions. There is no dearth of them. But most experiments we see and read in the social media fall in the category no net new value category. It is true that pricing is a very hard problem. There are many known unknowns and unknown unknowns. It takes effort, data and some very boring and laborious work to chip away at the problem. When faced with the difficult question of, “What pricing method will help maximize my business value by driving profit and revenue?”, most substitute it with a simpler question, “What pricing trick can I do to get a few sales and media attention?”.

The real question requires you to understand customer segments, their needs (what job do you want them to hire your product for?), the budget customers pay from for the product, the alternatives competing for the customer job to be done, the cost of alternatives,  the incremental value your product creates over all other alternatives.

The substitute question is easy to frame and answer. Let us assume something and call it an experiment. If it succeeds it was to our credit. If it did not, it was an experiment anyways. After all pricing is a very hard question, no wonder we did not succeed.

One such case is what we see from Everyone, an online fashion retailer. They say this about their pricing model,

Everlane About

Their assumption, which seem to be taken almost axiomatically is,

“Customers want to know the costs of products they buy”.

So they show their detailed costs breakdown, their overheads and margins. Have we seen this before? Yes, in the extreme form of cost based pricing I illustrated a few years ago,

The simplest counter argument that negates their premise is that customers do not buy product to offset our costs. They have a need to fill or a job to be done. They hire the right product that will get the job done under constraints. Our costs are just that, our costs. Not a concern for our customers.

Everylane states pricing in fashion retail is messed up with 8X markup. It is true. I have explained this detail before. But that does not mean we throw away first principles and opt to answer the simple but wrong question.

Pricing is about getting your fair share of value you create for your customers. As a online retailer Everlane seems unclear what unique value it creates for its customers. So it is hoping customers will gladly pay to offset its costs and cover its margin.

Wrong and futile!

Because I am all about the base rate

bayesPictureTis the season for predictions. If one has an audience one seem compelled to make predictions.  You are better off reading the book Superforecasting than this article. The book explains in depth the simplest elements you need in making predictions and forecast.

It starts with – Base Rate – which is how frequent does the said event happen in general relative to all other events. For example

  1. What percentage of tweets are retweeted?
  2. What percentage of people are named Bill?
  3. What percentage of startups achieve $1B valuation?
  4. What are the chances of you winning Survivor when you start the season with 19 others?

The next step is an iterative process that refined this prior knowledge by seeking new information and refining your estimate. That is the posterior probability.

Most likely you won’t read the book, so I present here these two concepts set to the tune of Megan Trainor’s song.


Because you know I’m all about that base-rate

‘Bout that base-rate, no tails

I’m all about that base-rate

‘Bout that base-rate, no tails

I’m all about that base-rate

‘Bout that base-rate, no tails

I’m all about that base-rate

‘Bout that base-rate… base-rate… base-rate… base-rate

Yeah, it’s pretty clear, I ain’t no sigma two

But I can predict it, predict it, like I’m supposed to do

‘Cause I got that Bayesian that all the gurus chase

And all the right tunables in all the right places

I see the magazine workin’ that Crystalball

We know that shit ain’t real, come on now, make it stop

If you got logic, logic, just raise ’em up

‘Cause every inch of you is curious from the bottom to the top

Yeah, my mama she told me “don’t worry about your data size”

(Shoo wop wop, sha-ooh wop wop)

She says, “Bayesians like a little more posterior to hold at night”

(That booty, uh, that booty booty)

You know I am wont to be stick figure xkcd comic doll

So if that what you’re into, then go ‘head and move along

An average Fitbit user takes 7000 steps a day

Screenshot 2014-12-18 at 10.09.15 AMI saw a full page Ad in WSJ from Morgan Stanley calling out their work on Fitbit IPO. One number that stood out was 23.2 trillion steps in total since the inception.

Sounds impressive especially when we hear comparatives like, “”enough to walk around the earth 350,000 times”. But what does this number mean and is it really that impressive?  Why does this number matter for Fitbit’s stock performance?

The same Morgan Stanley Ad says Fitbit sold 30 million devices till third quarter of 2015. That is not enough to do a simple math of average steps per day per person. We need number of devices sold each year and an estimate of average number of months (days) they are used till date.

The recent S1 filing from Fitbit gives us the per year numbers

2012  1.279   million
2013  4.476   million
2014  10.904  million
2015  13.097  million  (9 months)

Some or most of  the devices bought in 2012 and 2013 likely got upgraded to newer version. And assuming higher percentage of purchases near year ends let us  assume the number of months in use are 33, 24, 16 and 6 for the devices bought in 2012, 2013, 2014 an 2015 respectively.

With 30 days per month,  the 23.2 trillion total steps break down to 7049 steps per day.

That is the average and its 30% below the 10,000 steps Fitbit has been convincing its users to take. There is no science behind the 10,000 steps.

We do not have access to the distribution, Fitbit does. Obviously they those not to disclose this distribution. There is a set of people who beat 30,000 steps per day. Likely these are less than 5% (if higher then the average for the rest gets even lower). So we can safely state Average Fitbit user takes 7000 (or fewer) steps per day.

Is that really higher than what we otherwise take?

Regardless, does this matter? Do these extra steps really have an impact on fitness?

Definitely this means nothing to stock performance for Morgan Stanly to quote this with such prominence.

This New Year Make a Resolution to Measure What You Eat

new-years-resolutions2_dreamstime_m_17232559Most of us will make New Year resolutions for 2016 and most of those resolutions will fall by the way side within a few weeks. The most common one that shows up in Top 10 list is  – “Lose weight”. We make very specific and measurable ones like, “Lose 20lbs by June 30th”. Or somewhat amorphous ones like, “Eat right”, or “Diet”.

The fact that a goal is measurable doesn’t mean it is achievable. Setting a single milestone could lead to postponement, “we can always catchup in the last two months”. Not seeing progress or not meeting the goal can lead to increased disappointment and goal abandonment.

Amorphous goals will lead to scattershot approach. You try anything and everything without ways to know the end goal or measure progress, “If you don’t know where you are going, you might not get there”.

As someone who put this to work for the past 4 years I would like to recommend a different approach to goal setting for eating healthy or losing weight. Instead of making goals that can be measured make measurement as your goal. More precisely, logging.

fitnesspalUse an app like MyfitnessPal (something that makes logging easier and avoids logistical challenges).

Don’t yet worry about eating right, dieting or losing weight. Remember that is not your New Year resolution.

Log everything you eat. From the milk you add to your morning coffee to the bag of chips you snag in the afternoon. Everything.

Log the food item before you consume it so you stay on top of it. Set aside 2 minutes at the end of the day to close out the day’s log, adding anything you missed out.

Log every day. Make this a ritual. If you miss logging for a day, unlike lost exercise days you can always go back and fill out the previous log. The important thing is to log.

Another great thing about this goal is you can drop it after sometime. You need to keep this is up only until you can say which food item has how many calories, how many carbs, protein and fat.  When you can look at that slice of pizza from Pizza My Heart or the sandwich from Wolfgang Puck and see in your mind its nutritional value, you are done.

When you reach this state of knowing what you eat you are already on the path of knowing whether or not you are eating right. Before taking your next meal or snack you will know whether it is a healthier choice or not.

This is a simple and repeatable goal with no failures. Well the only failure is not logging. Other than that there are no disappointments and guilty feelings like you would feel from missing a day of exercise or after bingeing on the buffet. Missing a day or two of logging is not going to make you feel guilty and is easy to get caught up.

No scales to shame you. No fitness band to mock your inactivity.

The result is a complete knowledge of your food intake.

So this New Year make a resolution to just log.


Why IDC Estimate of Apple Watch Sales is Wrong

:Last week IDC came up with its estimate of Smart watch and wearable sales for calendar third quarter of 2015. They placed Apple watch second in sales volume at 3.9 million units in just 3 months. They did not share with us any assumptions and their methodology is unclear. It is fair and safe to say there are assumptions and methodology behind these numbers. Yet the biggest problem is not sharing with us the confidence level for the numbers, in other words we do not know the margin of error in these numbers.

Let us subject the 3.9 million numbers to test.

Last month after Apple announced its fiscal fourth quarter (calendar Q3) numbers I did an analysis of Apple Watch sales numbers. Apple does not report revenue from Watch but combined them under “Other Hardware” category.  This category previously did not include iPod but now now includes iPod, Apple TV and other accessories.


For the quarter in question this category revenue was $3,048 million, taking out estimated iPod sales (which is on the decline) the Accessories revenue was $2,728 million.

For the IDC estimate of 3.9 million units, even at the lowest average selling price of $400 for the Watch, revenue from Watch will come to $1560 million. That leaves $1,168 million for Apple TV and other accessories.

The Accessories revenue before Watch has not seen significant growth or decline but rather varies about a mean of $1,541 million with a standard deviation of $340 million. To see where IDC’s numbers fall in this, take a look at this chart below.


For IDC estimate of Apple watch unit sales to be true the revenue from other accessories has to fall below $1168 million. The chances this IDC Apple watch estimate is correct is a measly 12.5%. In the rest of the case the other accessories revenue are higher than $1168 million and hence  Apple watch revenue will be lower given we know the total Accessories revenue (listed in last column in table above).

The correct way to state is to give a range with a level of confidence. So here it goes (for ASP of $400)

  • We are 68.6% confident Apple sold between 2.07 to 3.77 million watches
  • We are 95.44% confident Apple sold between 1.21 to 4.6 million watches
  • We are 99.72% confident Apple between .366 to 5.47 million watches

When can we expect analyst firms to get numeracy right?

To Give or To Get – Calling into Question Walmart Ad #FillTheTruck


Walmart is running a heartwarming Ad about children and gifts.

Walmart calls this a Social Experiment and the TV version of this Ad concludes with the claim that 80% of the children chose to give than get. Sounds plausible. Right?

I do not make this about the children nor am I calling into question their true unselfish giving nature. It could very well be the case but the point is Walmart’s experiment has several errors that make us dismiss this completely.

The Ad starts by asking a set of children, “What do you want for Christmas?”. They all answer with their favorite things. Then the questioner asks them, “What does it feel like to open the present”. The respondents oblige by sharing their true feelings. So we can say they sufficiently primed the subjects, putting in the mindset to fell good about getting presents.  A video caption tells us, “Some people think kids only care about getting presents”

Then they pose the experiment question,

Okay, you have a choice. You can pick out any toy in Walmart and keep it. Or you can give it to a kid who does not get many presents during the holidays.

Everyone featured in this video choose to give and make a statement. So does this prove that children overwhelmingly choose to give than get? Before you decide look at these errors

  1. We do not know if the subjects are randomly chosen.
  2. This is not a controlled test with control group and treatment group. There is no baseline measurement either.
  3. The subjects know they are part of the experiment. They are in Walmart and told about their choice. So they adapt their behavior to suit the experiment (Demand Characteritics)
  4. They experimenter is right there with the subjects and when posed the question the children are more likely to give an answer that they believe will please the experimenter. Or they are more likely to give an answer that make them look good. This is the Social Desirability bias.
  5. Their choice to pick any toy from Walmart. May be the subjects do not like any toys from Walmart. What if the option is cash? Or a more popular option not available at Walmart?
  6. This is also the case where they have the option to get a new present above and beyond what they are going to get from their family. It is likely most thought this was not their present to begin with (opposite of endowment effect). So there is likely higher propensity to give away something not owned. Consider two variations:
    • Ask the child to pick any toy first. Tell them it is theirs to keep. That is, create endowment. Then give them the choice.
    • Ask them to agree to give away a randomly selected present from what they receive from their family.

      In either cases you will find markedly different results.

It is quite possible children are lot more generous and may be more than willing to give than get. But we do not have data and Walmart’s social experiment fails to provide that data.

Finally the Ad ends asking parents to give this choice to their children and post the video. That results will be more telling.