Careful what you ask for in WTP studies

In a seminal work titled “How the questions shape the answers” published in American Psychologist (1999), Norbert Schwarz describes how responses are influenced by question wordings, format and context. Schwarz writes,

“Self reports a fallible source of data and minor changes in question wording, question format or question context can result in major changes in the obtained results”

This is especially a more pronounced problem when it comes to survey questions that ask customers for their willingness to pay (WTP) for a product. When you directly ask a customer questions like:

  1. will you buy this product at  10?
  2. how much will you pay for product?  a) $4   b) $8   c) $10   d) $10    e)  $12
  3. will you buy this product if this were not offered free any more?

The researcher run the risk of getting answers that are not in any way  a true representation of what the customers will actually do. These kinds of questions assume that customers know how much they value the service and  customers are willing to disclose it. Another flaw in WTP studies is treating customer WTP as a fixed number in the minds of customers while it has been shown to be malleable (Thomas and Menon, Journal of Martket Research, 2006).

I saw a report from Forrester research on US customer WTP for online newpapers.  I admit I have not read the report but only their promotional blog post about it. The report claims 80% are not willing to pay for content From what I read I am not satisfied with study or its methods. The survey question was:

If the Web sites for the newspapers and magazines you read were no longer free, how would to prefer to pay for that content?

  1. Wouldn’t access them if I have to pay
  2. Subscription access to access all online content
  3. Subscription that combined print, web, and mobile device access
  4. Individual payment for each article read

The biggest flaw I find is anchoring – the question clearly reminds that the content has been free. The question  was too generic, asking  about newspapers and magazines you read and not about a specific newspaper or magazine. The respondents could be thinking of all newspapers, even those they read occasionally while answering this question.  There were no questions reminding respondents of value they get or to rank the online news sources by importance.   If the question had been,

If your most favorite newspaper cannot financially support the free online access, would you be willing to pay in one of the following ways?

  1. Subscription access to access all online content
  2. Subscription that combined print, web, and mobile device access
  3. Individual payment for each article read
  4. Wouldn’t access them if I have to pay

… the results would have completely different.

Based on their survey, Forrester  recommends:

  1. Publishers should continue to offer free, ad-supported products to the 80% of consumers who won’t pay for content online; and
  2. Publishers should offer consumers a choice of multichannel subscriptions, single-channel subscriptions, and micropayments for premium product access.

I do not agree. Even if we assume the 80% number is correct, does providing free provide higher profit than charging?  Do newspapers rally want higher reach (because of the Ad revenue)?

If a newspaper publisher really wants to find customer willingness to pay for content they need to do more targeted study of their readers,  use methods like Conjoint analysis to tease out the segments, how much customers in each value the product and focus on methods that help improve customer reference price before charging for content.

The net is the results are unreliable and Forrester’s recommendations are plain wrong.