In my last post, I gave you five ways to increase your survey results. Today, we'll dive into a few survey question mistakes you should avoid.
Well written questions are critical for a survey's success in providing you with accurate information and for its completion rate. While you may know what you want to ask in your survey, the way you ask the questions can shape your respondents' perspective on something and even cause them to unintentionally provide inaccurate answers. Good survey questions should allow thoughtful answers without leading the respondent to one answer or the other, and should not leave them feeling angry or confused. Writing an effective survey can be challenging, but if you avoid these common survey writing mistakes, you will have a great survey in no time.
Survey Mistake #1: Not Setting Clear Objectives
The most important step in a survey is figuring out what you actually want to know. It’s important to make your objectives really clear up front to make survey creation as easy as possible. Stay away from vague goals like measuring “satisfaction.” What do you mean by satisfaction? Do you want to know if your public awareness campaign was engaging and fun? Or if it was informational? Or persuasive? Focus on your end goal. What decisions do you need to make or behavior do you need or change based on the feedback you get.
Survey Mistake #2: Not Segmenting Your Audience
If you want to find out what types of products millennials like/dislike, you wouldn't send a survey out to every one of your customers—just those who fit the demographic.
Survey Mistake #3: Asking the Difficult Questions First
If you start your survey with the wordy, difficult questions that require your participants to really think, you will likely lose many of your potential respondents within the first few questions. Putting these questions first makes your survey appear to be longer and more difficult than it may actually be, which can easily deter survey takers. It is best to begin with easy questions, such as the participant's age, how often they use your products, etc, as these questions won't scare your respondents away. Your respondents are also more likely to completely read and answer the easier questions in the beginning, so even if they become agitated and do not provide the most thoughtful of answers for your more difficult questions, you will still receive some data from the easy questions!
Survey Mistake #4: Asking Double-Barreled Questions
This is one of the most commonly made survey mistakes and can be really harmful to your survey data. Often in survey questions, one will try and ask too many questions at once. This may be to reduce the amount of questions asked in the overall survey, or may be completely unintentional, but it can be detrimental to the success of your survey. It is important to ensure that you ask only one question at a time and avoid double-barreled questions. Double-barreled questions can confuse your survey participants and cause you to receive inaccurate data. An example of a double-barreled question is:
Do you and your children find the new Saturday morning DIY workshops valuable?
To avoid this, you must turn this into two separate questions, like this:
Do you find the new Saturday morning DIY workshops valuable?
Do your children find the new Saturday morning DIY workshops valuable?
This may add some length to your survey, but the more accurate results will certainly be worth it.
Survey Mistake #5: Asking Leading Questions
A leading question is legal jargon for any question that "puts into the witness' mouth the words to be echoed back, or plainly suggests the answer which the party wishes to get from him." While it may be tempting to use leading and biased language to get a survey taker to provide positive comments about your company, it will not help you to receive the information you really need to help your institution make any necessary improvements. Your data may show that customers find your customer service to be great, but if you did not give them an option to say otherwise, or if you directed them into providing that response, you won't receive any comments telling you that you need to improve your service, which really only harms your institution's success.
Leading question: "How great do you think our customer service is?"
As you can see, this question tricks respondent into saying the service is great, even though the survey taker may think otherwise.
A better way to ask this question would be: "On a scale of 1–10, with 10 being great and 1 being poor, how would you rate our customer service?"
Survey Mistake #6: Asking Questions that are Totally Off-Topic
Before crafting your survey questions, first identify the objective of your survey. What is it that you want to conclude from this survey? Once you know what information you are trying to gather, you will have a better direction to guide your survey questions in a way that will help you get the information you need. If you only need information regarding the usage of a product, do not ask questions about a different product.
Off-topic questions may cause the survey taker to quit the survey and may cause more work for you as you interpret your survey. You want to get the most out of your survey, and ensuring you only ask the necessary questions is a good way to achieve this.
Survey Mistake #7: Making Every Single Respondent Answer Every Single Question—Whether It Pertains to Them or Not
Even if you have the right audience for a particular survey, if you ask “Have you called our service center within the past year,” and the respondent says “no,” then you wouldn't want the next five questions to be about the experience they had when calling the service center. Many survey builders have the option to include skip logic, so that recipients aren't bombarded with all the questions on your survey—just the ones that pertain to them.
Once you have completed crafting your questions, it is important that you conduct a test run with your survey to ensure there are no errors and that it is easy to understand and complete. It's a best practice to test the survey timing, question formatting, question order, and distribution channels. After the test run, you are ready to send out your survey!
This post was originally featured on SWBC's LenderHub blog. Revised October 2015.