We already know what to do, we just need to validate it with a survey …

Through my blog and Berkeley network I get periodic requests from aspiring startup founders and small business owners  to help them do a “survey” for them. There is a general pattern in all these requests:

  1. They already have the product going or have the idea all figured out.
  2. They are skeptical of the phrase “Marketing Research” and services that charge high fees for it.
  3. They are highly technical, sharp and committed individuals
  4. They are reaching out only because someone they respect had asked them, “have you checked this with your target customers?”.
  5. They already know what they want to do, what the customers want, what the product offering should be, pricing should be – they only need to do a survey to validate these.
  6. They want me write survey questions like, “Would you prefer auto-turbo fluid motion v7.4 over nitro-fusion hydraulic v2.3? and will you pay $100 more for that?”
  7. They also want to ask a lot of essay questions, “What are all some of the problems you find with social media XYX?”
  8. They want to run the survey on their blog readers, twitter followers, facebook friends, …
  9. They are not willing to pay anything for me because all I have to do is write up a survey for them.

I turn away all such requests. The primary reason is, I currently advise two small businesses, one from Boulder and one here and I cannot take on more unpaid work.

But there is another reason as well: they are solving the wrong problem.

If these people are convinced about their path, looking only for validation and not willing to change their path based on the data, why bother with a customer survey?

You should not jump to do a survey if you have not formed  a few hypotheses that you want objectively tested.

You should not launch a survey that is not designed based on extensive exploratory process – one that involves multiple customer interviews and focus groups.

If you have not talked to single potential customer you are targeting, running the survey is the least of your concerns.

A survey is ineffective if you have not uncovered distribution of your customer profiles, likes and attitudes.

Asking respondents to write an essay of their problems and likes in a survey question is simply wrong!

A survey is ineffective  if you have not uncovered your “customer speak” and you continue to use your techno jargon speak.

A survey is ineffective if you slapped together a few questions that don’t help answer your decision.

A survey is ineffective  if you collect data from where it is convenient, like the drunkard searching for lost keys under the light because it was dark where he lost it.

A survey is ineffective if you have not identified the target population and found a way to reach them and are not willing to spend resources to reach the right target.

If you are convinced you are in San Diego, when you could be in Madagascar, looking only for white sandy beaches to validate your conviction and not willing to seek data that will show otherwise – you do not need a survey!

8 thoughts on “We already know what to do, we just need to validate it with a survey …

  1. Eric
    Thanks for the comment. As I talk to many of colleagues, I find that this is not just limited to small startups. Even PMs in big corps want to do a survey first because the email list is there.

    On Boulder, can’t wait to get back there.


  2. Great post and hello from another Boulder startup! You’ve highlighted a problem that rears its head continually in the startup circles here, and this quote:

    “If you have not talked to single potential customer you are targeting, running the survey is the least of your concerns.”

    is a big sensitivity for me. I’m astounded by how much energy tech entrepreneurs put into AVOIDING talking to a single potential customer. I don’t know if it can be chalked up to cockiness, naivety, or simple shyness but it’s a huge problem that contributes to a lot of wasted time and ultimately failed businesses.


  3. Anne Martin
    Thanks for reading and I appreciate you selecting this article for your class. I know I don’t do a good job of editing my articles, hope that does not deter your class.



  4. I plan to distribute this article to my undergraduate math classes when we discuss surveys and elementary statistics. It is so difficult to teach them that the math is not the important part. The crucial parts are identifying the problem, identifying the population, selecting a valid sampling technique, and writing neutral but meaningful questions. In other words, the time-consuming, analytical thinking part. Thanks for writing a clear and concise article.


  5. Srđan
    Thanks for the note. You point out valid scenarios on why they may want just a survey. Marketing research (survey is just one component of it) is useful only if we are willing to change our business direction based on the results. I do not believe in, “good to know” surveys. If we have made up our mind our biases nudge us to seek only data that supports our position.

    On a more geeky note, I don’t like to ask willngness to pay questions on surveys.

    I am aware of Customer Dev, but survey of convenient samples isn’t that. I should stop saying more on CustDev, in case @vlaskovits is reading 🙂

    On the URLs, I will recommend SurveyGizmo, not that I have anything against SurveyMonkey. SurveyGizmo makes it ridiculously simple to design surveys and provides so many powerful post processing functions. Of course, it is also a great startup from my home town Boulder 🙂

    Thanks for reading



  6. Handling those emails doesn’t seem much fun, but you may want to know that Customer Discovery, Pivots and Customer Validation are all parts of a very focused methodology which makes Product Market Fit a priority. It’s called Customer Development.

    I think it’s quite possible that you’re being approached by startups that have done some basic Market Research steps, and are just looking for a way to survey their user-base simply and effectively.

    I guess from now on you can reply with two URLs, this one and SurveyMonkey 🙂


Comments are closed.