As CIVIC’s Senior Advisor for Monitoring, Evaluation and Learning (MEL), I attended the American Evaluation Association annual conference with new questions about how CIVIC can expand MEL in its projects. I came to CIVIC in August after a stint working for one of Washington’s “Beltway Bandits,” and I was keen to work for a mission-driven organization. We live in an era plagued by numerous global armed conflicts, and the idea of working for an organization that works to protect civilians in conflict was particularly appealing. Since I began my work at CIVIC, I’ve faced the same question. How do you measure the absence of something, or of something that has been prevented? In CIVIC’s case, how do you measure the number of civilians not harmed in conflict as a result of our work?
There isn’t one clear answer to this question. At the conference I attended sessions that that I believed would provide me with the pieces to help solve this puzzle. In an in-depth session on “Most Significant Change,” I learned about a qualitative methodology focused on collecting stories from stakeholders. In CIVIC’s case, these stakeholders include governments, international and regional organizations, armed actors, communities. The seminar discussed how to engage these stakeholders about what has changed in their lives as a result of CIVIC’s work. Using a rigorous process of analyzing, sharing and most importantly learning from the stories provides insights and evidence into CIVIC’s work.
One of the sessions I found particularly interesting was “M&E for Countering Violent Extremism (CVE): Lessons from the Field.” While there are numerous differences between CVE and civilian harm mitigation work, the MEL challenges faced are similar. For example, both fields’ face the question of: How do you measure an absence of something or something prevented? In the case of CVE, how do you measure the number of people who don’t become extremists? In CIVIC’s case, how do you measure the number of civilians not harmed in conflict? The answer may lie in implementing more rigorous measurement of the steps in the causal chain needed to reach the desired impact.
The last session I attended was about the intersection of “Systems Thinking and Outcome Harvesting.” This session brought together two approaches that can be used to evaluate a project in a complex context. Organizations should use Systems Thinking to consider all relevant actors in a system and how these actors impact both each other and the project. Understanding these relationships is fundamental when analyzing project outcomes via Outcome Harvesting. Understanding how the actors’ function and affect project results is crucial to understanding how and what outcomes/impacts are achieved and to manage adaptively.
Although I did not leave the conference with all of the answers, I did learn new approaches for applying rigorous mixed MEL methodologies in the complex systems where CIVIC works. I’m excited to pilot these methodologies and approaches to better understand what works to protect civilians in conflict.