Did you utter these words or hear someone else do so recently in deciding the next step of product development or market development? It is not all bad because at least they considered taking some input and collecting some data before making decisions. But this is also totally wrong approach if you think 10-20 people talking a room under observation can help you decide.
The recent Journal Report from The Wall Street Journal had a very good piece titled, “Focus Groups Fall Out of Favor”. The article makes the case that in the present day Big Data and social media there is no role for focus groups. Reading the article, it was clear why – as suspected most people were using focus groups incorrectly for making big decisions.
I have written in detail in the past about how focus groups are misused. I recommend you read
- Anyone can convene a group, ask questions and write it up
- Watching a focus group going horribly wrong
- Who makes the hypothesis?
I will add to those key mistakes pointed out in the Journal article. In many of these cases the focus group is not only a poor fit but also the wrong tool.
- “The focus-group approach famously went awry for Coca-Cola Co. in the 1980s when extensive—and pricey—focus-group testing and market research led the company to replace Coca-Cola with a New Coke formula widely rejected by consumers.”
We do not know for sure Coke used focus group to decide New Coke. If they indeed did only use focus groups then we know why it failed. Stated preferences of extremely small samples is not reliable method to change taste of billion dollar business.
- “a moderator would ask a room full of moms which of the company’s products they liked most. The moms always responded that they fed their children the company’s healthiest offerings when, in fact, the best sellers are high-calorie snack food.”
If the goal was to find which product performed better, why would they ask that from a focus group? Shouldn’t they look at the sales data, by geography, by stored and combined with store card data to see cohorts and distribution? Not only were the moms stating what made them sound better moms but also the required data is readily available with better accuracy and precision elsewhere.
- “Kathy Fish, Procter & Gamble Co.’s chief technology officer, says that men, when asked how many swipes it takes to shave their face, will usually estimate 10 to 15. In reality, it’s more like 30 to 40. “
Even if the participants gave accurate response that would have been useless as the sample size is too small to draw any meaningful conclusions. This again is wrong use case for focus group. If P&G needed this data they should have used large panel of users who would log this data daily. Or in the world of IoT (Internet of Things) a connected razor that can collect this data easily.
Finally I will leave you with a big learning from the Journal article.
“It’s very expensive to find that progressive-leaning, millennial new mom who has the time to attend a focus group,”
This is indeed a very profound statement. This tells you why focus groups do not provide enough data to move on to decision phase. Focus groups only help you uncover hypotheses that you can test with larger group of people. The people in focus groups are not going to be the target segment. And that is perfectly okay if you are going to test the hypothesis you developed with the right target segment in a simpler manner.
How do you make decisions?