Are Customer Surveys a Waste of Time?

July 26th, 2005 by Bob Bly

It sounds like a good idea: survey customers to find out what they want, and then let their answers shape your product development and marketing.

But in reality, it?s often a bust.

A case in point: according to an article in BusinessWeek (8/1/05, p. 38), in 2002 the Gap began an intensive program of focus groups, surveys, and other market research.

But in the fiscal quarter ending 4/30/05, sales fell 4% ? and analysts expect them to drop another 2% for the quarter ending on 7/30/05.

The reason: eight former employees and two analysts say the Gap ?has shifted too far toward research and away from the instinct and emotion favored by many successful clothing merchandisers.?

My question is: how much time and money do YOU spend on customer surveys and other market research? And how strongly do the answers influence your product development and marketing?

Share

This entry was posted on Tuesday, July 26th, 2005 at 1:22 pm and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

19 responses about “Are Customer Surveys a Waste of Time?”

  1. Julie said:

    Hi Bob,
    In my experience (10+ years in DM publishing)customer surveys can be very misleading. Recipients tend to give answers they link you want to hear. Case in point … a previous employer spent over 100k developing a product that customer survey respondents said they would purchase. At the end of the day … they didn’t purchase and the company lost a bundle.

    My advice … treat survey responses as minor guage of interest … tread lightly and test, test, test!

  2. Steve Slaunwhite said:

    Here’s a scary story. A few years ago an insurance company hired me to create a direct mail package.

    When the design and copy were completed to draft form, my client had a brilliant idea. “Let’s show this to a focus group of potential customers.”

    Nothing I said could dissuade him.

    Based on feedback from the group — which included eliminating all references to the word FREE (“Too trite”, they said) — my client made sweeping changes to the package.

    The package was mailed. And it flopped.

    What went wrong? Not the focus group, according to my client. “The focus group can’t be wrong,” he asserts. “After all, they’re the customers!”

  3. Peter said:

    Yeah, a lot of the time customers really don’t know what they want. Aside from prosumers (who may very well be engineers or designers in another field) the masses simply clamor about waiting for the next best thing. Not exactly the best group of folks to ask, eh? This kind of thing permeates game design discussion quite frequently. Everyone wants to predict what the next blockbuster title will be – based on current trends, (genre, setting, etc) when in reality you could spend millions of dollars on focus groups of current title enthusiasts and come up with an ultra-generic flop.

  4. Michael Vanderdonk said:

    At the moment I’m spending about 20% of my time on one particular survey, but this is a special case as I’m asking _everyone_ not just my customers. Normally I’m not asking specific questions like ‘would you buy this product’ because, as has been stated, most people will say yes, until you ask for the order…

    I’m asking questions like what causes you the most problems? What is the biggest problem in your business at the moment? How would you like to see this problem fixed? Yes they are generic questions, but they get me awareness of exactly where I can or can’t assist customers.

    When it comes to developing and testing the final product, that’s done ‘live’. I run the product on a test group to discover where the product can be improved or changed, then run DM, sales and pricing tests to find best fit.

    Enjoy,
    Michael.

  5. Bob McCarthy said:

    Don’t dismiss surveys out of hand. It depends on how you use them. Sure, if you ask people for their opinions, then all you get are opinions.

    I prefer to use surveys to collect detailed information about customers and prospects. I ask qualifying questions – the same type of questions a sales person would ask when qualifying over the phone or in person.

    I use surveys to generate leads with prospects, identify cross-selling opportunities with current customers and reactivate lapsed customers. If clients want to use the data as market research too, they can – but that’s not our primary objective.

    As a direct marketer, I see the survey as the ultimate involvement device. Unlike letters and other mailers that are scanned quickly by the reader, a survey has to be read slowly and carefully.

    I use surveys all the time for my clients – with terrific results that contribute directly to the sales process.

    Bob McCarthy

    The Direct Response Coach

  6. John said:

    Listening to your customers is important… period. What I’m hearing from some of these comments is that people believe their customers are liars.

    The first step is to LISTEN to your customers. I’d argue that surveys are a great way to do that. Focus groups are a billion times more likely to lead the customer to an answer they think you want to hear.

    But the second step, and this is important, is to actually IMPLEMENT the lessons you learn.

  7. Gordon said:

    The 2 main problems I’ve seen with surveys are:
    1. Upper management tries to get answers that they want to hear. Which leads to:
    2. Poorly written questions.

    Case in Point: When I worked for HP I created a survey to test the customer response to our service manuals. Upper management wanted to include the question “Are you satisfied with (this section) of the manual?” Reason they wanted to know this: so they could go to *their* managers and say “See! Our customers are satisified!”.

    I disagreed and insisted on asking two more questions: (1) “Is (this section) important to you?” and (2) “Are your needs being met?”. The results were astounding. A lot of customers were “satisfied”, but their needs weren’t being met in a lot of important areas. Based on that we were able to create some real changes.

    Lesson learned: Customers may say they’re “satisfied”, but what’s really important is finding out where their needs are not being met.

    GP

  8. Justin Hitt said:

    With my business-to-business clients I’ve had success with surveys asking customers to describe the results they want to achieve, or to ask the questions they want answered.

    Picking top results and questions from a list on a very simple survey works well too, but beyond that get very mixed results from surveys.

    I’ve had results with customer service surveys limited to a few questions on a single page, usually faxed back or presented on the web.

    What you ask and what you’ll do with the information collected is important too. It’s good to solicit feedback from prospects and customers whenever possible, but you need well designed questions. (Which is probably a whole post in itself.)

    Sincerely,

    Justin Hitt
    Strategic Relations Consultant
    +1 (757) 282-7779

  9. Marc Stockman said:

    Just like with any other marketing tool, surveys can be used correctly or incorrectly.

    I would never give a blanket statement saying surveys are always good or bad because it depends on a variety factors including a) skill of the survey designer, b) purpose of the survey, c) skill of the survey analyzer, d) how the survey analysis is used and more.

    In my experience, and as some have mentioned in this thread, surveys are a great tool to identify interest in a variety product ideas as to help actually create the content of the product. I’ve also had success asking deep emotional questions to get a 360 degree psychological view — awareness, emotions, imagery, jargon used, belief systems, common complaints, common desires, objections, etc. You’ll be shocked at what they’ll tell you if you ask. This is then used as additional fuel to HELP with writing copy…but not to be relied on completely.

    You can’t rely on the survey only — you have to use intuition, gut, competitive research, market research, dry testing, etc. to confirm what you find in your surveys.

    Price testing in surveys is difficult for the reasons other people have listed; likewise, although I’ve personally never done them, getting focus group feedback on direct mail packages seems to be ineffective because you’d have to have a focus group of 1,000 people just to get 10 people who might seriously buy. Sure, you could try to pull in only “potential buyers”, but that process itself has many problems.

    Bottom line, I suggest using surveys as another weapon to add to your intelligence arsenal. Solely relying on it is probably foolish, but completely ignoring what your customers tell you is probably equally as foolish.

    Life is full of shades of gray even though we always like to think in absolutes to make things simpler — that’s a hard lesson I’ve had to learn over the years…one I am still learning, in fact.

  10. Adam S. said:

    Yeah, I can’t help feeling that this question is ill-posed. Many surveys fail because many surveys are poorly written. Comment #1 is a case in point: asking people whether they will purchase is probably a waste of time. Tools like conjoint analysis are more appropriate for pricing studies.

    And comment #2 is another case in point: don’t ask focus groups to write copy for you. Instead, do some testing and analysis of the campaign.

    But surveys, particularly in this age of internet, can be quite useful and easy tools for getting quick feedback. Perhaps the fashion industry needs to rely more on guy, but in my business (tech), the customer’s opinion is all too often ignored.

  11. Jonathan said:

    Everyone is talking about the companies, but don’t forget about the survey takers. I personally have filled out 400hours worth of surveys, and have only recieved 2$. After that much time, and i’m sure there are many others’ you tend to give botched results, and yes i even participated in the gap survey’s. I tried to answer honestly, but most of the questions were just repetitive option boxes, placing them against their competitors. I believe greenfield online did them. You also tend to get the same survey multiple times so they can fill their quotas. Most survey companies don’t pay anything but give you about the same chance as getting struck by lightning than getting a prize/cash from them. You wan’t the best results, make sure you the survey takers get paid, even a small amout like 2$ will make 100% better results, and chose a few for online focus groups, chat rooms, direct response to another individual will get you better results than a piece of binary paper will. Last, leave you question’s open ended. Never force someone into answering the question the way you want them too, and give them plenty of whitespace to verify why they chose that answer if they wan’t too. I think up there I read someone talking about customers being satisfied, then giving you different results when asking why. oop, there goes my email, probably another survey that pay’s nothing. But just the chance I’ll fill it out anyway.

  12. dish network satellite said:

    dish network satellite

  13. Wahaj said:

    Its got every thing on it. Very interesting and informative. I just cant express in words what a great site this is!

  14. mexico said:

    silted philosophers title availabilities wheeled relabeling recreative – Tons of interesdting stuff!!!

  15. accept credit cards citibank credit card application status said:

    roof,civics canyon autodecremented response scanned brick …

  16. Auto Insurance Liberty Mutaul said:

    flowering overtone swings,posture denouncing!pitying Greenbelt rinser

  17. Tara said:

    I think that the listening to the voice of your customers will allow companies to grow and prosper. Hasn’t anyone heard of the blind coke taste test. People blind folded preferred pepsi to coke so coke changed their product and people hated it, if they had surveyed and used the information they would have found that people who loved coke, hated the new product. I don’t think you should use one bad review but integrate the majority of feedback back into your business, I think it is crucial. Survey companies like Vovici make making your survey, deploying it and analyzing very simple and professional and it made for smaller to enterprise size companies. The web based survey companies now make it easy to manage necessary feedback!

  18. http://www.youtube.com/watch?v=FZOuC8FraM0 said:

    Hey There. I found your blog using msn. This
    is an extremely well written article. I will make
    sure to bookmark it and return to learn more of your useful information.
    Thanks for the post. I will certainly comeback.

  19. Look At This said:

    Hello, i think that i found anyone stopped at our blog site therefore i got below to come back the actual pick out? . I’m in search of things to increase this site! Maybe its adequate to implement a number of your thoughts!

Leave a Reply