Good communication is a key tenant of Improve Group work and never is it more important than when working with populations who speak a different language than the evaluators. While we have a number of staff who speak multiple languages (basic to advanced skills include: English, Spanish, French, Malinké, Hebrew, Khmer/Cambodian, Hmong, Bambara and Wolof), a number of our projects have required additional language sets. When seeking an appropriate translator, a question that always arises is how to confirm and affirm that the translations accurately reflect the responses of the participants.
The Mercy Corps MILK II evaluation of a drought-relief program in Niger that I assisted on is a great example of successfully navigating languages to come to accurate conclusions. Our study involved translation from English to French, and French to 5 different Nigerien ethnic languages. In one case, we even had our French translator ask the question in Hausa, and then one of the other participants translated it again for a different participant who came from a small ethnic group in the area.
Using this Nigerien evaluation as a case study, we took away valuable tips to help us prepare for future work with participants speaking different languages:
- If possible, find a translator that also has experience collecting data (matching the data collection methodology to their experience, e.g. group facilitator experience if the translator is going to help with a focus group). The ability to manage people can drastically affect the comfort of participants, and how open they will be to share their information and story. Our translator in Niger had previously provided facilitation and translation services for our client in the past. He also grew up in Niamey, adding to his ability to connect and put participants at ease.
- Budget for time and resources to meet and review the protocols with the translator before data collection. Translation is not an exact science, it is an art of interpretation, as meaning across words and cultures can change. Thus, making sure the translator understands the purpose of the evaluation, and what you hope to learn is essential to making sure the translation of questions is accurate both to the participants and in translation back to you as the evaluator.
- Budget for time and resources to discuss responses with the translator after data collection. No matter if you are having direct translation in the moment, or if the translator does all data collection and recording, take time to review what people said and how meaning could be different because of the culture. Translators who come from the participants’ culture are the most valuable in this situation; they can provide additional context to responses. After each data collection session, our Improve Group team sat with the translator to review our initial thoughts and reactions to participant responses. We found that we had missed some things as we translated from French to English on our notepads, and our translator also was able to talk about certain cultural references like food items consumed, that contributed to our knowledge of how people were living.
- Provide multiple methods for participants to respond. We mixed an image-based data collection tool using stickers and numbers with a follow up focus group. This allowed us to corroborate stories people told in the focus group about their livelihoods with their household survey. As people explained the “quantitative” portion of what their household looked like through their images, we could double check that the translation matched the images and colors on each person’s sheet of paper. This also revealed when participants were confused about the question and had improperly responded the first time.
- Keeping cultural norms in mind, consider collecting data as a group. Although not recommended if gathering sensitive data, collecting data from a group of participants that know each other may also increase accuracy of translated responses as more people can clarify meaning and help their friends, family and neighbors respond. In our study, we found that some individuals had a hard time recalling what their household looked like at two different time periods and got confused with the translated time stamp. Neighbors and friends assisted in their memories of that time and allowed us to gain representative information. Likewise, as we triangulated and confirmed responses through our mixed-methods approach, other participants in the group who had better grasped the activity instructions helped to verify the responses and images of whoever was speaking.
What are some other ideas, challenges or light bulbs that you have had when working across multiple languages using a translator? (See accompanying article by Danielle Dryke-Lessons Translate Through Many Languages).
Note: The American Evaluation Association recently posted an article by Jeff Williams on Pre-Pilot Language Testing. It contains an interesting table of the changing demographics of languages spoken at home, along with some tips and tricks for those engaging populations that have English as a second, third, or fourth tongue, or have no exposure to the language at all. To read more about it, click here.