Understanding your clients’ opinions and perspectives is one of the most important parts of the evaluation process. Today, many of us take for granted the convenience and ease of using technology or paper surveys for our data collection needs. The reality is that these methods may not always be the most appropriate as organizations today serve an increasingly diverse population who may not be familiar with or be comfortable with technology or pen-and-paper methods.
What are the options?
Your organization has many options for collecting data; however, there are also many trade-offs one must consider when choosing the data collection method for your evaluation.
Interviews
Interviews, face-to-face or telephone, are a great alternative for administering surveys to populations who do not have access to web surveys or are unable to complete a paper survey. Interviews can yield more detailed information from open-ended questions and have higher response rates than paper surveys. However, interviews tend to be longer and more expensive than a mail or web survey and are also subject to the possibility of interview bias.
Focus Groups
Focus groups are planned discussions with a small group of people led by a trained facilitator to learn about ideas and perceptions on a certain topic. The group dynamics in focus groups can provide insights that a personal interview cannot. Focus groups, however, are not representative of an entire population and can be dominated by certain individuals or get off topic without a skilled facilitator.
Observation
Observations of individuals or groups are often done to supplement interviews by documenting the environment and behaviors such as facial expressions, gestures, tone of voice, and other non-verbal indicators. Observations are particularly helpful in situations for individuals who have barriers to communicating their thoughts orally or in writing. On the downside, observations can be time consuming and may require observers to receive some training on discerning significant from trivial observations.
Other Participatory Methods
Language, literacy or cultural considerations can often be mitigated using non-verbal methods such as maps, diagrams, or other picture activities. This past spring, The Improve Group Senior Research Analyst Danielle Dryke used Image Grouping© while working on an evaluation project with Mercy Corps in Niamey, Niger in Africa. Image Grouping©, is a data gathering technique developed by The Improve Group that allows the full participation of illiterate populations. In one case, female beneficiaries were asked to use images representing indicators to respond to issues laid out in quadrants (e.g. things that have improved since working with a program, things that have gotten worse). The quadrants change for each project to reflect that issues at hand.
Choosing a Method
There are many factors you should consider when choosing a method for your evaluation study including the target population, the timeframe for collecting data, and your budget for collecting information using the method chosen. The following page outlines the pros and cons of different survey research methods and considerations for choosing a method: Different Methods of Collecting Information
Need some more advice on picking an appropriate data collection method? We would be happy to talk through your thought process with you and provide our advice! Reach out to The Improve Group at info@theimprovegroup.com.