Monitoring, Evaluation, and Learning

CIVIC’s updated monitoring, evaluation, and learning (MEL) approach is tied to CIVIC’s 2021-2025 strategic plan. It is designed to track data on CIVIC’s organizational goal, results, and thematic priorities. CIVIC reports on organization-level standard quantitative and qualitative indicators. These indicators provide data to assess the impact of CIVIC’s programs.

CIVIC’s custom approach has five elements that facilitate rigorous monitoring, evaluation, and learning:

Learning and evidence-building;
Outcome-focused;
Complexity-aware;
Systems thinking; and
Adaptive management.

With learning and evidence-building at the center, these elements provide a framework to effectively and efficiently monitor, evaluate, and learn from CIVIC’s programs at both program and organizational levels.

CIVIC MEL Approach Elements

Element

Learning and evidence-building

Description

CIVIC endeavors to be a learning organization throughout all aspects of its work, and learning forms the hub of CIVIC’s practical MEL approach. CIVIC operationalizes its learning approach through: routine meetings where learning is shared; program and organization learning agendas; effective application of adaptive management; ad hoc learning discussions; thematic communities of practice; and ongoing contextual analysis.

Selected CIVIC Examples

Learning agendas: in response to a program learning question, CIVIC’s Ukraine program is currently drafting a lessons learned paper on the development of a civilian casualty mitigation team. The findings will be included in an organization-level analysis of the development of several CCMTs (Afghanistan, G5/Sahel, Somalia).

Thematic communities of practice: currently two thematic groups (community-based protection and training) meet routinely and share learning, strategies, approaches, etc. across programs. CIVIC hopes to expand this approach to additional thematic areas of interest.

Outcome-focused

CIVIC uses an outcome-focused MEL approach to learn from and advance the evidence base for its work, test its theory of change, and assess positive change in civilians’ lives. This outcome-focus reflects a progression from CIVIC’s implementation-/output-based approach and includes the following elements:

Aligned with CIVIC’s new strategic plan and updated theory of change.
Improves outcome capture and analysis for stakeholders across CIVIC.
Supports collection and analysis of higher-level results to inform CIVIC understanding of key questions on sustainability, cost-effectiveness, and civil society collective action.
Informs prioritization of MEL tasks and resources.
Supports a robust results-based culture.

Outcome indicators: to operationalize CIVIC’s outcome-focused MEL approach, CIVIC’s organization indicators are focused on the outcome level (rather than output level). For example, in addition to tracking the number of armed actors trained, CIVIC tracks the percent of armed actors trained demonstrating improved application of practices to protect civilians. This outcome-focus helps CIVIC better understand training effectiveness and the impact of training on improved protection of civilians.

Complexity-aware

CIVIC’s MEL approach is tailored to two challenges of working in conflict-affected settings. First, programs frequently operate in complex environments where cause-and-effect relationships are unclear/unknown. Second, programs frequently operate in ever-changing environments. To address these challenges, CIVIC’s MEL approach includes both traditional and complexity-aware monitoring/data collection methods/approaches. Complexity-aware monitoring requires the application of three principles:

Synchronize monitoring with the pace of change.
Attend to program monitoring’s blind spots.
Considers relationships, perspectives, and boundaries.

Complexity-aware methods/approaches/tools:

Nigeria program: uses a Results Journal to capture preliminary results to understand the program’s contribution.
Iraq program: conducts ongoing contextual monitoring to understand how contextual shifts may impact program implementation.
Ukraine program: tracks meaningful milestones on the road to the approval for a national protection of civilians policy.

Systems thinking

CIVIC programs frequently operate in complicated (known but challenging to access causal pathways) and potentially complex (unknown or unclear causal pathways) systems. Systems thinking provides an approach to consider the ways in which all the actors within a given system interact and impact one another. By using ongoing contextual monitoring, collaborating with stakeholders, and an adaptive approach, programs will have an understanding of the system in which it operates and provide a better basis for determining if and how the activity/approach/tool/ program is successful (or not).

Stakeholder mapping to support systems thinking: the Peacekeeping program conducts a detailed stakeholder mapping exercise as a part of the program design process. The stakeholder map informs the program’s understanding of the system in which the program operates: the actors, how they interact with CIVIC and each other, and how change may occur. Understanding these components of the system in key to achieving results and understanding successful activities and approaches.

Adaptive management

CIVIC will create an ongoing learning program that supports programs’ adjustment and overall decision-making through an adaptive management approach (e.g., continuous, systematic, iterative, and planned use of emerging knowledge and learning throughout program implementation). An adaptive approach facilitates decision making for any needed program adjustments. These adjustments include scaling/expanding what is working, stopping what is not working, and other necessary changes as a result of progress toward achieving program milestones and results.

Adaptive management for COVID-19: in Nangahar, Afghanistan, the program-supported Community Protection Committee advocated to the Taliban to not fight in the area as it will disrupt communities’ access to health services during a pandemic. Also, in Afghanistan, the program developed and shared resources on COVID-19 and security forces with the Ministry of the Interior.

Vulnerable Groups: CIVIC recognizes that vulnerable groups (e.g., women and girls, children, older persons, persons with disabilities, the LGBTQ community, indigenous people, ethnic and religious minorities, internally displaced persons) are affected differently by conflict. CIVIC-facilitated Community Protection Groups (CPGs) or comparable structures or individual civilians engaged (applying a CBP approach) must demonstrate that they are willing and able to impartially, jointly, and inclusively assess the different POC risks of their communities taking all POC-relevant vulnerabilities into account.¹

Conflict Sensitivity: Conflict sensitivity is “the practice of understanding how aid interacts with conflict in a particular context, to mitigate unintended negative effects, and to influence conflict positively wherever possible, through humanitarian, development and/or peacebuilding interventions.”² If it is learned that a program was in some way causing or increasing tensions between sides of the conflict or local communities or increasing risk to civilians, engagement would be paused, and local civilian leaders would be convened to help better understand the dynamics at the local level.

Guiding Tenets: Three tenets guide all CIVIC’s MEL activities. These principles are all closely linked and must all be in place to ensure the safety and security of CIVIC staff, stakeholders, partners, and beneficiaries.

Do No Harm: In accordance with global best practice and CIVIC’s Safety and Security protocols, CIVIC, in both program implementation and MEL work, endeavors to do no harm to all program staff, stakeholders, and beneficiaries.
Transparency and accountability: CIVIC views transparency and accountability from two perspectives – internal and external – and from multiples levels (e.g., program and organization). Internally, the MEL teams share data and analysis back to CIVIC program staff. Externally, CIVIC endeavors to be both transparent and accountable with/to partners, stakeholders, and beneficiaries.
Responsible collection and use of data: “The physical representation of information in a manner suitable for communication, interpretation, or processing by human beings or automatic means. Data may be numerical, descriptive, audio or visual.”³