The Single Question Trap – Why the Mystical 0-10 Scale?

In the first part of this new series I raised a few questions about the popular Net Promoter Score metric. As a reminder we are not yet discussing whether or not the metric works but sticking to really basic questions. As a fan of NPS if you are tempted to say, “you are missing the big picture, forget the scale”, we are not done yet, we are starting with questions, say, a smart Martian coming to Earth would ask if she comes across the metric.  It behooves you to consider the foundation before talking about upper levels.

Let us take up the first three questions

  1. Why is the scale 0 to 10 – why not 1 to 10, or 1 to 5 or  0 to 100?
  2. What other scales have you tried, measured and compared before arriving at this 0-10, 11-point scale?
  3. If you are going to reduce the wider scale into just three groups, why not simply ask them whether or not they will recommend, doesn’t care or will speak against your brand?

I have not heard a justification from the Promoters of this metric but their likely explanation would be, “the 0-10 scale and not anything else is the most stable scale and has highest predictability among all other scales considered”.

Fortunately for us someone else with better data and analytical bent had asked this question before. Gina Pingitore of JD Power and Associates published the results from her research in Marketing Research Journal. She looked at thousands of customer data from five different industry verticals and compared four different Net metrics to determine if the 0-10 scale is key to predictability.

Note how comprehensive her data set is and how she limited her comparison to just Net metrics and not Interval scale metric. A Net metric is one, like NPS, subtracts one from another to report difference. She compared 0-10 NPS against,

  1. Net Delight metric  that uses a 10 point scale  and reports percentage difference between 10 and 1-5.
  2. Net Satisfied Metric that uses a 10 point scale and reports percentage difference between top two (10, 9) and 1-5.
  3. Net Committed a 4-20 point scale that compares percentage over 17 and percentage below 11
  4. NPS 4 a 4 point scale instead of 0-10 scale for NPS that reports percentage difference between 4 and 1-2.

And the results? Different scales did yield different results which is expected as it is based on the idiosyncrasies of each. Next the study compared the correlations (remember just correlation) between each metric and customer level behavior.  At the aggregate level the 0-10 scale used by NPS provided no significant advantage over other scales. And more surprisingly, contrary to what the Promoters believe, the 11-point scale does not yield the strongest relationship to any of the customer behavior that is known to affect firm’s revenue.

In the one customer behavior it performed strongest it was only marginally better than the 4-point NPS scale.

There you have it. The 0-10 is not rooted in any research nor its results better than any other metric one could have used. One could have simply asked the respondents to classify themselves as Promoter, Passive or Detractor.

What we likely have here is a 11-point scale that need be what it is and per the studies does not offer any advantage over simpler scales. A less forgiving reviewer may call into question the very motive, stating that creating a unique 0-10, 11 point scale allows for trademarking the scale whereas a  1-10 scale may resemble existing metrics, like ACSI, and not easy to brand and trademark.

There you have it, results from peer reviewed work comparing the efficacy of the 0-10 scale. What is your answer?

Let us take the other topics in coming posts.

3 thoughts on “The Single Question Trap – Why the Mystical 0-10 Scale?

  1. Perhaps JDPA has a reason to steer customers to their secret special sauce proprietary model instead of a solution they could implement themselves.

    Like

Comments are closed.