Hi! I’m Amy Cyr, a senior consultant here at IG. For this month’s IG-ology, I want to tell you some of the considerations we take into account when choosing a data collection method. As evaluators, we often have conversations with clients about what they want to learn and how they can gather the right data to answer their questions. While some data collection methods seem omnipresent and like the easy choice—for example, surveys—there are pros, cons, and considerations to make when selecting any method.
For example, surveys can be a great way to get input from a lot of people; however, they don’t provide as much opportunity for in-depth responses as some other methods. Interviews, meanwhile, can help you dig deeply into a topic—for example, to ask customized follow-up questions based on answers given. But interviews have their own drawbacks: they are time-consuming in that you can only get one respondent’s data at a time; because of this, the interview method may mean either a smaller sample size or a higher cost compared to other methods. Participants may also be less open to talking about taboo topics with an interviewer. We also use workshops as participatory data collection; these offer the ability for participants to not only share information, but also generate ideas. On the flip side, workshops require participants be available to attend the workshop at a given time and place; they also may be less ideal for sensitive topics as they are semi-public. Other participatory data collection methods—like we used at the Minnesota Council of Nonprofits conference and wrote about in a past IG-ology—can also be useful in many circumstances.
We could go on and on about different methods, from observations to PhotoVoice. But the gist is that we have a lot to consider when choosing a data collection method, from the time and money available for evaluation to the sensitivity of the topic. Ideally, you can take a mixed-methods approach, drawing connections across what you find in surveys, focus groups, records review, and other methods. And of course, the most important factor when making data collection decisions is the community you are hoping will participate. Whether your stakeholders are over-surveyed, have had negative experiences with evaluation, and have the time in their day to participate are all considerations to take into account when selecting a data collection method. People who face the greatest barriers to participating may be those from whom you are most needing to hear. So, it is important to use an equity lens when designing data collection activities.
We enjoy discussing this with clients and coming to decisions on what will work best. Are you grappling with questions about what data collection method is right for you? We would be happy to set up a one-on-one conversation with you and one of our consultants. Contact us at firstname.lastname@example.org and we can tackle this together!