Danielle Dryke (center) with Program Beneficiaries
Our recent project in Niamey, Niger was a true joy for me. Prior to joining the Improve Group, I worked and lived abroad for five years and although I’ve traveled since then I have not had the opportunity to complete an international work assignment for almost three years. It was a great reminder of the many things to consider while completing evaluations in new cultural contexts. The following list is by no means comprehensive, but includes a few of the issues the Improve Group and staff from Mercy Corps considered while designing our evaluation. Local partners and staff are great resources to troubleshoot with and to plan strategically with to ensure that your evaluation is a success!
A few things to be aware of in a new cultural context:
- Gender relations. How will the relationships between men and women impact the data collection, interpretation and analysis during your evaluation? In Niger, like many places in the world, women will not speak up in a mixed-gender group. We designed the evaluation to ensure that male and female program participants were always segregated by gender during image grouping sessions and focus groups. In contrast, in professional office settings and among the urban educated population it is perfectly acceptable to interview men and women together and so interviews with project staff were completed in mixed-gender settings.
- Power structures. What are the power dynamics in the culture that you are in? How are you gaining access to the population and how are those persons perceived? Who defers to whom and similar to gender relations, what do you need to do to ensure that your data is not compromised as a result? In cases where we felt that age dynamics might play a role, we separated participants by general age category to ensure that young adults were not challenging power dynamics by speaking out with elders present.
- Basic social norms. What are the basic courtesies, rituals and behaviors expected of you? And if you do not obey the norms how will it impact the way that you are perceived, the rapport you build with your respondents, and the success of the evaluation? For example greetings are the most basic and important way of building rapport in West Africa. Prior to starting every session I learned a greeting in the local language and went around to every respondent personally greeting and shaking their hands. Additionally, all of the clothing that I wore was very conservative with long skirts and long sleeves, and during sessions I would cover my head with a scarf as a form of respect.
- Expectations of incentives. What is the norm in the area you will be working in for offering incentives? For example when I worked in Mali it was normal to offer rural participants a large bar of soap and a kilo of sugar as a thank you for their time. Whereas, for day-long meetings in the city, meals, coffee breaks with snacks and a per diem were the norm.
- Translators as facilitators. Generally speaking if you are working with a translator that individual should be a local and will act also as the facilitator for data collection activities. So it is important to consider both their skill as a translator and as an effective evaluation facilitator.
- Literacy levels. Can your participants read? Are they proficient in one language and not another? Do your methods match their abilities appropriately? Our entire data gathering was done using oral and visual methods to ensure full participation no matter the ability level of the participants. If you are planning a survey, be sure that you budget to allow for facilitators to administer it as an interview.
- Permissions and confidentiality. It is important to consider both the requirements of the country where you are performing the evaluation and those of the funding agency. These might be very different. Some funders want to know that you have been through an IRB process in their home country, whereas others have no guidelines at all. Depending on the nature of the study it may be enough to inform the participants orally prior to beginning any data collection session.
- Concepts of time. What time do you need to tell participants to be present in order to have them there when you want? In Niger, all of the participants were informed that meetings would commence 30 minutes prior to the actual start time to ensure that everyone would be present and ready to go. In some cultures the variation may be as much as two hours, so be sure to work that into your planning, particularly if you are on a tight schedule.
- Taboo or sensitive subjects. What subjects are off-limits? How can you present sensitive topics in a way that you will gain access to the information that you need without offending respondents? We had a discussion with the translator and facilitator to ensure that he was prepared to present any sensitive topics in a way that would avoid offending participants and would allow them to answer honestly without becoming uncomfortable.