As evaluators, we are always evolving our practice to support learning and inform strategy. Sometimes this means rethinking the assumptions we hold about the optimal balance between technical rigor and storytelling in order to best represent the impact of our clients’ work.
Two recent experiences developing case studies for the David and Lucile Packard Foundation and the DentaQuest Foundation challenged us to think deeply about both of these things. We looked beyond many of the usual models and sources that guide our work, and ultimately landed on an approach informed from an entirely different field: investigative journalism.
Highlighting Bright Spots: The Evaluator’s Dilemma
When funders see successful initiatives but can’t yet fully articulate what contributes to that success, they sometimes look to highlight a “bright spot” in order to spur conversation about what it takes to translate promising practices from one setting to another. That was the case for both the Packard and DentaQuest Foundations, who each encouraged us to use case studies as a way to tell the stories their grantees’ successes. Case studies are especially appropriate for answering the nuanced “how” and “why” questions that can be so elusive when programs are taking place in complex systems.
But as we shaped our case study approach for each organization, we saw a need to look beyond traditional methodology for two reasons. First, the typical case study focus on “bright spots” can create an imbalance in an evaluation if it only looks at known successes and leaves out other important insights. Second, by developing case studies, we shifted our focus to working with communities to tell their own story. Our team thought hard about how we might balance our commitment to rigor (a systematic, structured, and objective approach to studying an issue) with the knowledge that participants’ testimony would be both proud and subjective.
Taking Inspiration from Investigative Journalism
After some exploration, we saw an opportunity to infuse elements of investigative journalism into our approach to these two case study projects. Inspired by Mark Lee Hunter’s Story-Based Inquiry: A manual for investigative journalists, we developed an approach that was far less structured than more convential case study methods. But that open structure created the necessary space to engage our interview informants not as traditional research subjects but as partners with agency and expertise.
The table below shows how our journalism-inspired case study approach differs from the more traditional evaluation case study approach.
Throughout this process, we learned that there are many ways the principles of investigative journalism can apply to the case study method, and we tailored our approach according to the unique needs and context of each engagement.
We also used three other hallmarks of investigative journalism in our approach: (1) the central role of hypothesis-setting, (2) the use of diverse data sources, and (3) ongoing analysis and assessment. This approach helped us follow and adapt to new storylines as they evolved, and ultimately enabled us to produce stories that are true to the lived experience of constituents.
Hallmarks of investigative journalism
Community Voice as an essential element of an investigative journalism-inspired approach
As responsive, practical evaluators, we recognize the need to tailor our methods to the particular context of a project. In cases where we need to limit our lens to objective, measureable facts, the investigative journalism approach and the focus on bright spots may not be appropriate. But the purpose of our work was to illustrate and not evaluate, to spur conversation and not to judge or assess. Infusing investigative journalism into our case study methodology helped us to stay true to this purpose.
Through these two projects, we tried to surface authentic stories and share findings in a way that honors local stakeholders as the experts of their own work. The end result? We craft our case study reports in partnership with community leaders. In choosing to honor the value of their judgement, we hoped to motivate them and other key partners to continue to grapple with the implications of their stories, including the opportunities to translate key findings and lessons to other areas.