Originally posted on AEA365, which is sponsored by the American Evaluation Association (AEA):
Hi fellow evaluators, this is Domonique, Vanessa, Mili, Jared, and Cristina of Harder + Company Community Research who works with public and social-sector organizations to promote lasting change. This blog post focuses on evaluation technical assistance (TA) as a practice for advancing equity through evaluation.
Evaluators play a critical role in providing insights to inform decision-making among key interested parties. By centering equity in their approach, evaluators can facilitate a process that generates insights from equitable methodologies, thereby enhancing the validity of insights.
Historically, evaluation processes have been informed by values that exclude the voices of those who are a part of the evaluation, such as grantees. This exclusion often perpetuates harm by generating findings in unproductive ways. Instead, evaluators can embrace processes that value partnership with grantees. Embracing such values can disrupt cycles of harm and contribute to new norms for evaluation.
In this post, we highlight reflections from an evaluation of a County initiative aimed at addressing structural inequities and promoting healing to delve into how evaluation technical assistance (TA) can support capacity building among grantees, enabling them to influence and actively participate in evaluation design and implementation.
The TA we provided offered support to grantees to ensure they had the information and resources needed to engage in evaluation activities. Our approach began with an evaluation orientation to introduce the evaluation team, share values guiding our evaluation approach (e.g., flexibility), and highlight our intention to partner with grantees to identify the best ways to support their investment and participation in the evaluation. Following the orientation, we held individual conversations with each grantee to understand their interests, capacity, and concerns regarding the evaluation. Based on insights from these conversations, we administered an online survey to learn more about each grantees’ unique TA needs. We used insight from both touchpoints to further refine TA activities. We drafted descriptions of those offerings in a TA guide for grantees’ reference to proactively ensure all grantees, regardless of their previous evaluation or TA experience, had clarity and transparency about the support available to them.
Our TA activities included:
- One-on-one meetings: Thought partnership regarding the context of program data and building relationships between grantees, evaluators, and funders.
- Grantee group meetings: Dialogue related to programming and evaluation among all grantees.
- Office hours: Drop-in times for grantees to receive support for data collection, reporting, and quality improvement.
- Resources: Materials such as data dictionaries.
- Training: Pilot/roleplay the use of tools and strategies for addressing anticipated challenges.
Lessons Learned
Grantees appreciated the collaborative and flexible nature of the TA support they received. Another indicator of the effectiveness of this TA approach was the high engagement across all evaluation activities. This feedback suggests that a comprehensive approach to TA can advance equitable participation in evaluation.
Considering the initiative’s aim to promote healing across historically marginalized communities, it is important to highlight that all key individuals leading this effort, including the program officers and the evaluation team, were Black or Latino/a. They shared lived experiences with grantees and the communities they served. This diversity and expertise facilitated transparency and candid conversations within the evaluation team, funder, and grantees. This context was crucial in building trust with grantees and contributed to the successful implementation of the TA approach.
The launch of TA during that evaluation also set the tone for later evaluation activities by reinforcing a collaborative rather than compliance tone. From this foundation, we were able to share decision-making about data collection activities, identify and contextualize discrepancies in data, and uncover program learnings that might have been missed without the open lines of communication and relationship building that were essential outcomes of the TA approach.