Ten Big Ideas from Evaluation 2015

Last month, several members of the Harder+Company team attended and presented at Evaluation 2015, the annual conference of the American Evaluation Association in Chicago. We had a great time connecting with colleagues, making new friends and exploring the Windy City! We also learned a great deal from our fellow attendees. Here we share our top ten takeaways from our perspective as presenters and participants.

  • Clients are increasingly more data savvy
    We live in a data-rich world and everyone is becoming more comfortable accessing and using all kinds of different data. As evaluators, we need to align the sophistication of the data we present with the interests and abilities of our clients. (Aimee Fribourg)
  • Clients want visually appealing products
    Clients are increasingly interested in reports and deliverables that use both visuals and text. But most evaluators are not graphic designers. Evaluators need to think about building in-house design capacity or partnering with a graphic design firm to ensure they can meet these expectations. As Kylie Hutchinson so aptly put it, “Dammit Jim, I’m an evaluator,” not a graphic designer. (Kevin Rafter, Aimee Fribourg)
  • We’ve evolved beyond the “final report”
    It’s increasingly important to tailor evaluation products to meet the needs of different stakeholder groups and different communication platforms. Our own session on creative dissemination methods with First 5 LA was packed with evaluators looking for meaningful ways to share findings with their clients. (Aimee Fribourg, Joelle Greene)
  • Developmental evaluation is taking hold across the globe
    Developmental evaluation was everywhere at Evaluation 2015. This approach is picking up steam as evaluators and stakeholders alike see the power and utility of the approach. A key question in this work is, “Are we doing the right things?” versus “Are we doing things right?” In our work with First 5 LA, for example, we have used developmental evaluation to surface emerging themes, challenges and promising practices in real time. (Arianne Mar, Aimee Fribourg)
  • Evaluation is political
    Our colleagues reminded us of the inherent political nature of research and evaluation. Many sessions emphasized the political role of the evaluator, especially when making judgements about framing, prioritization, and recommendations for distribution of resources. We should make space for conversations about power and politics in our work. (Justine Marcus)
  • Stakeholder engagement is critical and challenging
    At our own interactive poster session on the topic of stakeholder engagement, our fellow evaluators expressed a genuine desire to engage stakeholders in the evaluative process and recognized the value of doing so. However, resources, time, local politics, and dissenting opinions can all pose obstacles to meaningful stakeholder engagement. (Paul Harder)
  • We need to ask truly evaluative questions for our work to make a difference
    Michael Scriven reminded us what truly evaluative questions look like and the power of asking them. “Was the program implemented as intended?” is not an evaluative question; “How well was the program implemented?” is an evaluative question. As evaluators we should be bold and support our clients in asking those truly evaluative questions about their work. (Joelle Greene)
  • Evaluative rubrics are gaining popularity and traction
    Rubrics are increasingly being used to evaluate program and initiative outcomes. The use of rubrics requires stakeholders and evaluators to explicitly define what constitutes quality and value for programmatic outcomes along with indicators and more traditional forms of evidence. Jane Davidson’s blog is a helpful resource to learn more about using rubrics. (Joelle Greene)
  • Video is hot
    In the age of Instagram, Snapchat, and Facebook perhaps it’s no surprise that video is king. We saw creative uses of video throughout the evaluation process – whether as a data collection method or to share findings. Video is interactive, engaging and feels current, especially to youth. This made us reflect on our own great experiences using video as an evaluation tool in our work with Corporation of Supportive Housing and City of Oakley, among others. (Laurent Reyes)
  • Grassroots organizations need evaluation too
    The Little Village Walking Tour was a powerful reminder of the important work that community-based and grassroots organizations do in vulnerable communities. It inspired our team to think about ways to reach out to smaller non-profits and support their evaluative needs and thinking. (Laurent Reyes)

This collection of observations shows how the evaluation field is experiencing an important shift towards paying more attention to communications as an important tool for making our work more effective!