When I attended the recent 2014 American Evaluation Association national conference in Denver, I sought out several sessions focused on the evaluation of work by coalitions and collaboratives. Among the many fascinating topics to explore at the conference, why focus here?

Current work – A number of the Improve Group’s current and recent clients (Pine Technical College HOPES Initiative, Brooklyn Bridge Alliance, Suburban Ramsey Family Collaborative, Region IV South Adult Mental Health Initiative, Rape and Abuse Crisis Center of Fargo-Moorhead) are looking to accomplish their goals through coalitions or collaborations. We have used different tools and frameworks, client insights and our own ingenuity in these situations. But, we recognize this is a growing field and want to stay on top of what others are learning and practicing!

Importance - We recognize that complex social problems have multiple roots and thus demand response of a similar nature. Collaborations bring the valuable expertise, experience, perspectives and ideas of multiple actors to create opportunities for solving social problems. It seems like a recognition of the interconnected nature of our world and many of our life experiences.

Difficulty - An evaluation of a collaboratively led initiative is not just a super-charged program evaluation. Many key parts of an evaluation – actors, participants, time frame, attribution – are all reframed. Theories of change become more apt than traditional logic models, and anticipated time frames stretch to allow coalitions and partnerships to form. No single activity or actor can or should claim responsibility for impact, and outcome indicators are at a community or population level.

Some interesting things to chew on from these sessions:

  • Coalitions go through stages of development, often cycling through stages more than once because of internal and external changes. This should be expected and not seen as a failing of a collaborative because it is cycling through an earlier stage of development again.
  • Network analysis is often used to understand change over time in interconnectedness between actors. However, more is not always better. Focusing on the pattern of relationships and what relationships are strategically needed may yield more insights.
  • While there are a number of tools and frameworks to understand the effectiveness of a coalition or collaborative’s functioning, an evaluation must look at what they are doing, even if it is early indicators of change. Often, evaluations focus on the collaborative itself, and how well it is measuring up to what we know makes an effective collaborative. This is because the path of the kind of social change these groups are working towards can often be long and indirect. An evaluator can take this longer time frame into the context, but this does mean we cannot begin watching community measures for change in the first few years of work.
  • Community readiness for change is a critical lever to enable the change that collaboratives seek. We are currently using a framework called the Community Readiness Model developed by the Tri-Ethnic Center for Prevention Research at Colorado State University with several current clients to help them assess community readiness and adapt strategies accordingly.