The Improve Group is always excited to learn about creative data collection and analysis techniques. Because we work with people from all walks of life, from all over the globe, we want to be inclusive and respectful throughout our practice. We have discovered that photos can be a useful tool in learning about how programs affect their participants’, and have had great success incorporating pictures into our work.
For example, in our work with youth, we have found innovative ways to use images. The Arts Residency Interventions in Special Education (ARISE) program of Performing Arts Workshop of San Francisco asked 3-5 grade students to draw pictures on their pre and post surveys. The change from pre- to post-test in what children drew was analyzed to understand how the program impacted the participants’ lives. We found changes in things like representations of movement and collaboration. The pictures were also a tremendous help in reporting results, as we were able to show how students changed over time.
We were curious how others in our field are using pictures, so our staff attended some relevant sessions at AEA Evaluation 2012. In the “Visual Evaluation Methods: An Introduction to Using Art and Photographs in Program Evaluation,” Melissa Cater, discussed youth perceptions of gardens and the outdoors. Children were asked to take photos and draw pictures of things found in a garden to talk about each item’s purpose in a garden. The youth were of all ages, and the photographs and drawings allowed them to express themselves without being limited by language – overcoming difficulties in assessing child opinions and attitudes[1]. Furthermore, because children were given free range to express themselves, there wasn’t a risk of them trying to give a “correct” answer.
Another photo-related presentation at AEA titled, “Telling Our Story: School Transformation Documented Through an Ethnographic Photo Gallery,” shared findings of an initiative aimed to increase reading among elementary students, in part through beautification efforts and re-purposed space in the school building. Presenter Leanne Kallemeyn and her colleagues shared how, as part of a suite of data collection tools, photography was used to show changes that were happening through both before and after pictures of the intervention. The Performing Arts Workshop is currently incorporating photography as part of a 3-year PACT21 program where teaching artists develop 21st Century Skills in elementary students in the San Francisco Unified School District. Photos of students doing theatre arts, creative writing, spoken word and dance are uploaded to Tumblr pages where classroom teachers and artists write about the changes they are seeing as part of an on-going documentation and assessment process of the program’s impact. Participants play a key role in the evaluation by gathering data, analyzing what they see and identifying other questions to explore using the camera.
The appeal of photography is that it results in a tangible product and encourages us to interpret our surroundings. An evaluator can apply this method to many kinds of questions such as: What physical environmental impact is there as a result of a healing garden at a hospital? How do youth show leadership in their communities?
As with all methods evaluators use, photography can be used to respond to a specific set of questions, circumstances or values of the program and evaluation. While it has limits, photography is an important tool you can consider using in your evaluation practice.
[1] Borgers, N., de Leeuw, E., & Hox, J. (2000). Children as respondents in survey research: Cognitive development and response quality 1. Bulletin de Methodologie Sociologique, 66(1), 60-75.
[1] Borgers, N., de Leeuw, E., & Hox, J. (2000). Children as respondents in survey research: Cognitive development and response quality 1. Bulletin de Methodologie Sociologique, 66(1), 60-75.
