BAD Metrics

This is a guest post by Ron Shevlin.  Ron  is a senior analyst at Aite Group where he specializes in retail banking issues including sales and marketing technologies, customer and marketing analytics, social media, customer experience and consumer behavior. Find Ron on Twitter at @rshevlin or at his Snarketing 2.0 blog.


bs_metricOrganizational Effectiveness experts (sometimes called Change Management experts) have a saying: “You get the behavior you pay for.”

Simply put, it means that an organization’s rewards and incentive structure dictates employees’ behaviors. Pick the wrong metrics, get the wrong behavior.

If your primary measure of customer relationship success is the Net Promoter Score (NPS), you’ll get employees who will stop at nothing from getting customers to give them a top-box score on a survey. Unless, of course, you’re smart enough to understand the behaviors that drive the intention to refer.

And if you are smart enough to understand that, then you wouldn’t need to measure “intention to refer.”

NPS is an inferior metric because (among other reasons) it captures attitude, not behavior. Even worse, it captures attitude regarding future behavioral intention. So you intend to refer a company to your friends or family? What’s the actual likelihood that you actually do it? You intended to lose weight, stop smoking, and get your financial life in order last year (and every year before that). But that didn’t happen, did it?

NPS proponents tout the metric as a “simple” metric, but simple doesn’t make it right, nor does it make it free (or cheap). There is always a cost to collecting survey data from customers, and always trade-offs that must be made, because it’s impractical to survey every customer.

The old world of marketing relied on survey data to collect attitudes, however, because, even with its limitations, it was more effective and efficient than capturing the actual behaviors of customers. The old world of marketing also relied on demographic data about customers and prospects because they were relatively easy to capture, both through external sources as well as from internal data collection efforts.

The adoption of online and mobile technologies holds promise for a new set of metrics, however. Metrics based on actual customer behavior, and not just attitudes and demographics.

It’s not that marketers haven’t had behavioral data about customers. They have. But typically that behavioral data has been limited to actual purchase behavior.

The purpose of the attitudinal and demographic data, however, is to help predict and understand sales behavior.

The new technologies give marketers an opportunity to create, track, and measure a new set of metrics. Metrics based on actual consumer behavior that relate to non-purchase related behavior regarding product research and referral behavior.

The challenge marketers must meet is how to create a balanced portfolio of metrics that include behavioral, attitudinal, and demographic (i.e., BAD) factors.