Effective Questions for a Successful Survey
Robert Freesmeier, Market Research & Data Analyst
In working with the Federal Government, we’re all familiar with requirements, deliverables, resources, and scheduling. These terms each relate to an essential aspect of survey research: details. Survey research is dependent on the details―while still maintaining sight of the big picture of course. Once the details for survey-based research are determined, the next step is the survey design. Survey design is where the details are developed.
But let’s back up for a second to that big picture. In market research, there are a few ways to conduct primary research. A survey is just one method for obtaining primary data (focus groups and interviews are others). Prior to implementing any form of research, a decision must be made on which method is best and that decision is determined by understanding your research objectives. A survey will not yield useful data until the research objectives are defined and measured.
Even after deciding to execute a survey, there are elements to consider before choosing an appropriate survey method (i.e., online, telephone, or face to face). Each method has its own unique advantages. A one-on-one, in-depth interview with your entire population sounds ideal but other factors like budget and scope may limit your resources. Addressing every possibility now for making the right decision will save time, money, and perhaps a little sanity later.
After determining your research objectives, budget, and scope your next step is formulating the survey questions. To design a robust and results-oriented survey, your questions should be targeted and address different aspects of your research objectives. Here are several tips for keeping those survey design details in mind:
Make sure your questions are applicable to the respondent – This requires the appropriate use of “skip logic” to exclude irrelevant information. At the very least, provide response options such as “Not Applicable” or “I don’t know” to ensure exhaustive responses.
Ensure your questions are technically accurate – Using a Subject Matter Expert (SME) is beneficial here. For example: “To the nearest inch, how tall is your horse?” This question sounds innocent enough, right? You may think so until you realize horses are actually measured in hands. A respondent armed with this knowledge will instantly lose faith in the survey and potentially doubt the security of his or her information.
Ask one question at a time (avoid the dreaded double-barrel) – This seems logical, but it is surprising how often people combine multiple questions. For example, “On a scale of 1 to 10, how would you rate the audio quality of the music you hear on your computer or iPod?” If music skips on a PC but is heard clearly on an iPod, which answer do respondents choose? Questions are often answered inaccurately due to this type of confusion. When a respondent incorrectly answers the question, the result is measurement error. Here’s a quick tip: Do a word search for the conjunction “and” as well as “or.” If you’ve included those conjunctions, chances are you are asking two or even three questions in one.
Use simple language – Also known as “Plain English,” this writing style offers concise and easy-to-read questions. In survey design, simple language leads to increased comprehension and reduced burden on the respondent. Additionally, when a respondent understands the question clearly, your data validity increases.
Make sure “yes” means “yes” and “no” means “no” – Avoid questions with double negatives. In other words, a respondent should not say “yes” to mean “no.” Take the following question, for example: “Should your supervisor not be responsible for performing your annual review?” This approach is another way the respondent can be tricked into answering incorrectly.
Ensure the response options match what is being asked – This requires the researcher to take a step back and understand the question’s purpose. If a question asks a respondent how many days per week he or she works from home, response options should be some variation of 1 through 7, since there are only 7 days in a week.
Keep in mind that these tips for making a survey field ready are not exhaustive. There are other procedural measures to consider. One of these measures is an in-depth quality control or technical review. The technical review should be followed by a piloting session in which the survey undergoes a test run in a mock setting. The pilot will capture a sample perspective of the target audience, further addressing additional measurement error.
Administering a survey properly requires a copious amount of attention to―you guessed it―detail. With these tips in mind, hopefully your next survey will provide the accurate data you need to properly address your research objectives.
Market Research & Data Analyst
Robert has 5 years of experience in the market research field and 4 years experience implementing survey solutions to both Information Experts’ federal and commercial clients.