Hello! We are Sara McGarraugh and Kassira Absar, consultants at The Improve Group, and strong advocates for surveys as a tool for gathering feedback to help you understand if and how your organization is achieving its goals and where improvements or changes can be made. Many of the organizations we work with use surveys, but they aren’t sure whether their surveys are getting reliable or meaningful information for making decisions. So, we’ve come up with some simple tips to help you craft quality surveys that do just that:
1. Ask one question at a time.
This may seem like an obvious tip, but it’s a mistake we see in surveys all the time. What’s the quick check to help you avoid it? Look for “and” at any part in your question (or answer options)! Now split your question at the “and” to make it two questions. For example, “Do you think workers should have more vacation time and shorter days?” would change to, “Do you think workers should have more vacation time?” and “Do you think workers should have shorter days?”Done!
2. Use simple and familiar words.
Avoid using jargon or complex words that may cause confusion. By using simple language, you are ensuring that your survey is accessible to people at all reading levels. For example, when crafting a survey, I like to focus on using words that a ten year old would understand.
3. Formatting can make all the difference.
From the attractiveness of the survey to the attention to detail of increasing response consistency, formatting can make or break your results. Keep fonts and colors simple. Limit lines and increase the amount of white space. Are open-ended response boxes the appropriate size for the length of response you want? For example, If you want 1 sentence or word, don’t leave your box open to look like you want a paragraph. Similarly, if you want a number, make sure the space is small, and consider restricting the text to only receive numeric responses rather than written text (e.g. 7 vs. seven). These are just a few ways you can improve formatting to increase response consistency!
4. Use mutually exclusive response options.
Survey response options must be mutually exclusive, which means they do not overlap with one another. For example, “what is your age range?” should be sure to capture the entire range of potential responses without one age falling into two different responses: 5-19, 20-29, 30-39, 40-49, 50 and older. Some assumptions you are making here are, that you are not asking someone younger than 5 to take your survey, that you are mostly targeting or wanting to know about differences between those in their 20s, 30s and 40s, and you are not capping the potential age at which someone may be taking your survey (I know a 95 year old who is still at the top of his game).
5. If you are asking a question that could elicit negative or positive responses (e.g. strongly agree à strongly disagree), state both positive and negative sides in the question statement.
For example, “to what extent do you agree or disagree with this statement…” This helps to reduce bias from “leading” the respondent to respond in the way you may be suggesting.
6. Use specific and concrete words to specify the concepts clearly.
Look at each word or groupings of words in your question to see if there are any alternative ways it could be interpreted. For example, “how many times did you eat together as a family last week?” In this context, what constitutes “eating?” Meals? Snacks? Passing each other in the kitchen while grabbing a granola bar? What else strikes you as potentially unclear in that sentence?
Bonus tip: Pilot testing your survey (even with your co-worker or a family member) can help you discover how questions may be interpreted differently and make adjustments before you launch your survey to respondents!
Learn more at our Survey Camp
The tips above are a starting point for more effective survey design. On August 19, we will be facilitating a three-hour virtual Survey Camp that dives into these best practices and much more! You will learn about what makes an effective survey and have an opportunity to practice creating or updating your own survey.