The most recent issue of the American Journal of Evaluation (v. 27, n3, Sept. 2006) presents an interesting ethical scenario: when is an evaluator no longer external? In the scenario, an evaluator has been working with an agency for a number of years, and a significant portion of their income comes from the single agency. A foundation, interested in funding a replication of a program that was found to have promising results by the evaluator, is concerned that the evaluator is not external.
September 14, 2006 - 3:47pm
Today we had the opportunity to join Tom DeCaigny, executive director of Performing Arts Workshop, as he visited with corporate giving staff from St. Paul Travelers, one of their funders. Although we've worked directly with funders who require evaluation, this is the first time we've had the opportunity to sit down and learn about the interests of those who primarily support programs.
September 8, 2006 - 4:16pm
All good ideas... are difficult to implement! The Improve Group has some tools that we are making better use of in our own planning, however. Thanks to a SharePoint site, our staff are able to host discussions, post ideas and share lists of resources that are open to any contributers. We can also create sub-sites for any of our projects which serves us well for managing some of our larger projects.
September 1, 2006 - 6:48am
We are in the process of piloting retrospective pre-test surveys for one of our clients. The retrospective pre-test has been shown in research to be the most valid way to capture change as a result of a program or intervention, particularly among young people. It asks participants at the end of the program to rate their own status at the beginning of the program and their current status; so a sample question might be:
For each question below, circle your answer.
August 14, 2006 - 2:13pm
One of the things we work very hard to do is collect data in the language that is most comfortable for respondents. We have translated print surveys into Spanish, Vietnamese, Hmong and Chinese; we've conducted focus groups in Spanish and Hmong, and conducted telephone interviews or surveys in a number of languages.
August 11, 2006 - 1:40pm
What do you do when your evaluation finds mixed results? How do you let your client know that what you've proven is only modest success? This is one of the biggest challenges for external evaluators.
August 3, 2006 - 1:14pm
Last spring, several Improve Group staff attended the "Every Girl, Everywhere" event of the Girl Scout Council of Greater Minneapolis. I've worked with a lot of programs that serve youth and always enjoy the chance to hear directly from kids.
