Developing effective, inclusive and gender responsive MEAL systems

Dear Members,

Happy New Year!

Developing effective, inclusive and gender-responsive systems for monitoring, evaluation, accountability and learning (MEAL) is high on the agenda of development and humanitarian actors.

If a programme or project has a good monitoring, evaluation and learning (MEL) or MEAL system, an independent evaluation should take advantage of it. The evaluation can then focus more on the reflective and learning aspects and spend less time and resources on collecting data and information.

Programmes that take advantage of an effective MEL/MEAL function are designed to address the real needs of all people. Actors can measure performance, reflect and learn important lessons. However, often MEL/MEAL systems are limited to ensuring compliance and measuring outputs, outcomes and impact and rarely include cross-cutting issues, such as gender and leave-no-one-behind principles.

With this idea in mind, I proposed a discussion on the theme “developing effective, inclusive and gender-responsive MEAL systems”, in which 19 members participated over the course of two weeks or so. Opinions were invited on both emergency response and development, including agriculture and rural development, and focused on:

• How can we develop effective and inclusive MEAL systems?
• What are the constraints and how do we address them?

I would like to thank all participants. Your contributions made the discussion rich and successful.

Below is an outline of the main messages.

How can we develop effective and inclusive MEAL systems?

1. MEL/MEAL systems often focus too much on accountability to donors and on gathering data on results as set out in the results framework or theory of change. Instead, they need to be driven by questions defined by primary users.
2. Exploring assumptions matters as much as, if not more than, measuring indicators. Undermining deliberation on assumptions obstructs successful reflection and learning.
3. Although numbers generated by M&E are important, they are arguably not as important as learning how they came about.
4. For a MEL system to be gender-responsive, it must ascertain the following: inclusivity and intersectionality, the principle of representation, the principle of participation, and equal power relations between (or within) different groups.
5. In promoting effective, inclusive and gender-responsive MEL systems, citizen monitoring can be one way to go. It can take the form of a citizen scorecard or some other accountability monitoring tool (such as participatory tools).
6. Good MEL/MEAL systems build on good programme/project design. Unless a programme or project mainstreams disability, the elderly, children and different sexual orientations from the design phase, evaluation results commonly do not include such groups of people satisfactorily. Hence, we need to ensure the quality and transparency of the data and make sure that the views of all stakeholders, especially the disadvantaged, are taken into account.
7. A good logical model/theory of change is based on the integration of gender analysis and inclusivity. All actors in the chain ‒ the government and its branches (sectoral ministries, regions, municipalities and village committees, including village chiefs or their representatives) ‒ should be involved.
8. Indicators must be specified based on the different groups of food-security beneficiaries (pregnant women, people living with disability, children, the elderly, female-headed households, polygamous households where women have their own kitchens and are not dependent on their husbands).
9. Gender issues also apply to other vulnerable populations: people with disabilities, people undergoing forced migration, indigenous populations, children, etc.
10. Evidence-based programme or project design is a strong foundation for establishing effective, inclusive and gender-responsive MEAL systems. Preparations must be made very early on, upstream of the MEL process. The 4R method could be very effective here, as it identifies project stakeholders based on responsibilities, rights, relationships and revenues.

What are the constraints and how do we address them?

11. The level of available and deployable resources comes into question when developing MEAL systems. We need to understand the human resources available in terms of expertise (people who practice evaluation and who have expertise in gender issues), time and funds, especially at country or regional level. Where the required resources do not exist, efforts to develop such capacity should be made from the outset of MEAL system development.
12. In emergencies and crisis-affected settings, there are multiple constraints and limitations, including issues of secure access to project sites and intended informants, the availability and reliability of data, and the possible bias of key informants due to their location or affiliation. These constraints can be addressed by using local partners/experts, third-party monitoring and/or modern technology that allows remote monitoring and data collection.
13. Because they have been mostly limited to programmes/projects, MEL/MEAL systems have not been able to broaden their scope to learning and its importance to national growth and development. Ultimately, MEL/MEAL systems should be institutionalized in governmental and non-governmental agencies in all sectors.

 “Reflection and learning … comes in revealing the unknown through listening to and learning from those in need ‒ excluded and underserved communities ‒ not measuring those in charge.” (Daniel Ticehurst). Daniel’s assertion emphasizes why we need to advocate for inclusive and gender-responsive MEAL systems.

In conclusion, gender-responsiveness is complex. Even humanitarian and development agencies whose programmes have a large gender component can face serious challenges in ensuring gender-responsive MEAL systems, be it at project, programme or organizational level.

Dear EvalForward members

My contribution on the MEL&MEAL.

Monitoring and evaluation systems cut-across all sectors of the economy: this makes them an important means for learning, where researchers and policies makers should go for information, knowledge and update with relevant statistics and data for government policies, programmes and projects. However, to be fully integrated as a learning entity and process, past and present information and data for government policies, programmes/projects should be kept/stored and institutionalized for learning processes and for independent and other bodies to go for information and data collection for research purposes and policies.

Often, in monitoring and evaluation, the primary interests are outcomes and impacts of the programmes/projects leaving others unattended, including government policies that affect the economy growth and development and gender issues, where inclusiveness should be most important to all stakeholders. Making MEAL/MEL an avenue of learning would create and broaden the scope of monitoring, evaluation and learning systems: it could become a national learning institution which would serve and make inclusiveness important to all. Inclusiveness will ensure participatory mechanism in the development of the system itself.

Basically, a programme or project sustained would be based on a monitoring and evaluation system because it is what reviews the outcome and impact on the society. In this case it becomes a sustainable effect, and a means in which programmes/projects thrive continuously. It should be institutionalized to create a pathway and avenue for learning.

Meanwhile, because they have been mostly limited to programmes/projects, MEL/MEAL systems have not be able to broaden their scope to learning and their importance to national growth and development. Institutionalizing them would eliminate this limitation. Rural development has not be fully considered nor captured and integrated because of accessibility of communities for information and data collection. Emphasis should be placed on rural development to ensure information is well gathered for update and development. This would also help agricultural development and emergency needs at any time.

Ultimately, if MEL/MEAL is a system let it be institutionalized to serve as a national databased for all sectors.

Thanks.

Esosa

Hello experts!

Making everything is clear in terms of baseline, ToC, and CLA as put by Sylvain drives me to the idea of also basing efforts for building gender-responsive MEAL on findings of a Gender Analysis (of course guided by gender experts).

Greeting to you members!

In fact, you may agree that there are several agencies where Gender is a key component of their programming. However, these agencies still face challenges in institutionalizing gender-responsive MEAL. For instance, I am completing my dissertation titled “Factors influencing gender-responsive monitoring in non-governmental organizations…”. Based on the findings and our discussions here on Eval Forward, I can conclude that even humanitarian or development agencies whose programmes have a large component of Gender can face serious challenges in ensuring gender-responsive monitoring systems—be it at project or programme or organizational level). Therefore, I also agree with you, Kien that “The parameter of inclusion, effectiveness and efficiency are easier BUT GENDER RESPONSIVE IS SO COMPLICATED”. 

I am thinking that a more Evidence-Based Approach should be used by the agency in order to strengthen gender-responsiveness at project, programme, and organizational level.

Looking at the responses, I am not sure we are addressing the questions raised by ERIASAFU LUBOWA. First and foremost gender responsiveness is the like the highest level in the gender continuum.It has attributes or traits (if you like) that must be integrated in the MEAL cycle and the contraints and their solutions should be contextual like Daniel Ticehurst has deliberated it may not one size fits it all , depending on the approach that MEAL team chooses to use not methods. 

Here is my take  on how to make the MEAL system gender-responsive,

For any system to be gender-responsive it must ascertain the following:

• Inclusivity and intersectionality should be factored in from the predesign to design to implementation to closure
• The principle of representations- Focus should be in using the feminist MEAL approach so that processes are contributed into by both genders. The decision-making functions (parliaments, company boards, union leadership, etc.) are important indicators for women’s and men’s access to voice and power. However, it is not enough to simply count women and men. It is equally important to consider the actual outcomes of decisions taken by decision-making organs and their effects on gender equality
• The principle of Participation has to be intentional to the design team at the outset: Achieving higher participation of underrepresented or disadvantaged sex in a given activity (training course, discussion of a new policy, etc.) is always desirable. Nonetheless, a project is not necessarily gender-responsive or contributing to gender equality just because a high rate of women has taken part in its activities
• Finally focused should on be Equal power relations involves addressing power relations between (or within) different groups. This means evaluators need to acquire a full understanding of the context in which any changes have taken place. It also means evaluations need to be conducted in a way that is sensitive to the empowerment of disadvantaged groups.

With the above aspects addressed the MEAL team should integrate the above concepts in their MEAL systems  in the following ways:

Integrate all the gender-responsive concepts into the MEAL System for instance

Conduct a gender-responsive situation analysis

The MEAL plan has to be gender-responsive

Use Mixed Methods that are gender-responsive

Use tools that are gender-responsive

Finally, the MEAL Eco-system should be respectful, participatory, reflective, transparent, and accountable across the agricultural and or food systems value chain.

Dear EvalForward members,

I need ideas and support on MEAL best practice in the banking sector. 

Thanks 

Hello dear experts!

Thank you also for your contributions, and I bring my share of knowledge in the more specific sense of "gender in the food sector - agriculture". Without repeating the contributions of others, it is important that everything is clear, in terms of:

1. indicators, which must be well specified according to the different groups of food security beneficiaries (pregnant women, people living with disabilities, children, the elderly, female-headed households, polygamous households where women have their own kitchens and are not dependent on their husbands) to facilitate the disaggregation of the specific questionnaires related to them and, above all, the contribution of the activities to meet the indicators. It turns out that the specification of the indicators will also allow the questions to be determined according to the role of the different beneficiary groups to enable the reporting of information on these indicators according to the different groups.

2. Baseline or initial assessment: to control and identify preliminary issues in order to define the indicators it is necessary to specify the households that cultivate, the households that own arable land etc...., the categories of people that cultivate etc.

3. Project theory of change: This will define the activities, outputs, and outcomes in order to monitor progress and fidelity of the implementation of the activities during the implementation of the project; and to enable quarterly review of the activities. With the objective of adapting the activities, move on to the training of actors in order to plan well, for the achievement of effective results.

4. CLA (Collaboration, Learning and Adaptation): Adaptation, training and learning on lessons learned from the results, this to bring stakeholders to review activities, re-plan them according to an effective budget and its staff trained in relation to the evaluation result in order to improve the result and meet the indicators.

Sylvain LIMBISA NAKITUMBA

[Translated from French]

Dear Friend,

This is an exciting topic. In my opinion, any system/ procedures can operated in a step by step approach. In every step, we could develop a set of measurements and indicators for every stage of the MEAL system. The parameter of inclusion, effectiveness and efficiency are easier but gender responsive is so complicated. From my side, I am looking to forward hearing any feedback from experts and scientists in this field..

Have a nice weekend everyone.

Hello everyone, here is my contribution to the topic.

• How can we develop effective and inclusive MEAL systems?

Yes, we can, and in my opinion, to do so:

It is necessary to involve all the actors of the chain, namely the government and its branches (the sectoral ministries, the regions, the municipalities and the village committees including the village chiefs or their representatives).

It is also necessary to take into account other experiences already gained in the field.

You have to take into account the needs of people with disabilities in addition to gender.

Finally, I think you should take into account the market for the product targeted at the time of the emergency.

• What are the constraints encountered and how can they be solved?

Some difficulties encountered:

• The non-involvement of certain actors at the start of the project;
• The mastery of the methods used by the affected populations;
• The failure to take into account the study of the product before and during the disaster or crisis;

In order to mitigate, a few issues need to be explored:

• Effective involvement of all actors at all levels;
• Experimenting with other methods not mastered by the target populations;
• Always consider the market before the crisis and after the crisis before engaging.

[Contribution translated from French]

Thank you Eriasafu for this relevant topic. To develop an inclusive MEAL system, the chances are very good chance in starting with developing an all-inclusive theory of change and capturing the baseline of all-inclusivity issues. This then ensures inclusivity in creating the log frames and work plans and eventually trickles down to indicators that include both qualitative and quantitative; inclusive interventions conducted; collecting data that is disaggregated; Data collected by applying diverse methods of qualitative and quantitative methods and reporting. And also inclusivity in getting feedback from the beneficiaries.

The project/program team may fail to understand what is meant by inclusivity if the implementation starts at a later stage.

Dear fellow EvalForward members,

Following the insightful contributions shared by other partners in evaluation here, mine are a small addition to the discussion adopted from Save the Children's approach of including the Gender Quantum in MEAL with regards to overall programming.

In the journey from gender exploitative programming and evaluation to such which is gender-sensitive, or in better circumstances gender transformative;

• Institutions are urged to begin with proposals or plans that are inherently informed by a gender assessment or have an integrated needs assessment that considers gender issues.
• Having specific individuals (boys, girls, men, women) considerations for analysis and not generally; households, people, families or communities
• Clearly describing relevant and unique gender inequalities such as discriminatory social norms, gender barriers, etc.
• Expressing how the different gender inequalities experienced by girls, boys, women and men will affect the fulfilment of project objectives, including in relation to equitable benefits and participation for girls, boys, women and men.

When these factors and others are considered in program/project development; the following specifics can be considered at MEAL planning to ensure Gender sensitivity/transformity;

• Quantitative indicators (including targets) should be disaggregated by sex and age
• Indicators should be strategically set to close gender gaps (disproportionate differences between sexes).
• M&E methodology should include working in gender-safe spaces (e.g. conducting data collection separately with girls, boys, women and men, and using female enumerators with female stakeholders and male enumerators with male stakeholders [gender matching]).
• The MEAL framework should explicitly include ongoing gender analysis, enabling the identification of gender gaps.
• The system should track the changes in knowledge, attitudes, perceptions and behaviours related to gender equality at the outcome level.
• Results statements are included in the M&E plan that measures outcomes related to changes in gender equality (e.g. progress towards closing a gender gap or addressing gender-based barriers).

These considerations should also reflect in budgeting so that the MEAL system and programming operations should not face unintended bottlenecks in ensuring gender inclusivity overall;

• The budget proposal should allocate financial resources for the full implementation of the project's gender-related activities. This can either be through;
  • Budget lines dedicated to specific gender equality focussed activities; and/or,
  • Budget lines dedicated to activities that explicitly include gender equality components (e.g. construction of latrines which will respond to the specific needs of girls and boys).
• The project budget line(s) dedicated to gender equality technical support during project implementation (e.g. gender equality advisor etc.)
• The project includes budget line(s) dedicated to gender equality training and capacity strengthening for local staff, local partners, beneficiaries and/or key stakeholder

Lastly, on a perspective of sustainability of interventions;

• The program/project explicitly identifies and addresses discriminatory social norms and institutions
• The program/project advocates for legislation and policies that promote gender equality (e.g. replacing laws that discriminate against females or males, or advocating for the inclusion of gender equality components in existing laws.)

Adopted from Save the Children Zambia - Gladys Musaba

Dear all,

Greetings from Kigali, this is a very insightful discussion and really helpful to my current assignment. I am working on a gender responsive M&E system for the education sector in Rwanda and wanted to thank the Evalforward members who have made contributions so far. 

I look forward to reading all contributions and I hope to share lessons later when I complete my assignment.

Kind Regards,

Judith

As we continue to share ideas, think of how our ideas could be used to benefit the development of inclusive and gender-responsive MEAL systems for typical Food Security, Agriculture and Rural Development interventions.

Hello fellow discussants,

Thank you for your ideas. I strongly believe that any one developing or strengthening their M&E system would find all these ideas very helpful. Please, allow me to add.

It is interesting to learn that “purpose and utility” should be considered at the onset when developing a MEAL system.

I believe regardless of other purposes of the MEAL system, “Reflection and Learning” are important aspects to consider when setting the purpose and utility of the system.

On the accountability part of the MEAL system, accountability processes and activities too would consider the various backgrounds within the targeted groups (the communities, or beneficiaries, or the crisis affected people). For example, the feedback and complaints mechanism should be accessible by ALL; hence, making the accountability system inclusive in a way.

Based on my experience, unless the program or project or the evaluation itself mainstreamed disability, elderly people, children, and different sexual orientations right from the design phase, it is common to see evaluation results not talking about such groups of people satisfactorily. This is also where the aspect of HRBA should come in strongly when developing and operationalizing MEL systems (as Serdar mentioned).

Certainly, all the ideas posted put together give a good understanding of how we can develop effective and inclusive MEAL systems, the constraints involved, and how to address them.

Daniel, you mentioned that “I believe such reflection comes in revealing the unknown through listening to and learning from those in need, not measuring those in charge - excluded and underserved communities”. Really, this emphasizes why we need to advocate for inclusive and gender-responsive MEAL systems.

In order to develop effective, inclusive and gender-sensitive MEAL systems, it would probably be appropriate to question the level of available and deployable resources:

1. First, the question of human resources. What human resources are available locally? If we are looking for people who practice evaluation and who have expertise in gender issues, of course in our countries this expertise exists, but where is it at present?

They are in the United Nations system, they are in some international NGOs, they are in some ministries. Consequently, if we are looking for people who have expertise in evaluation and gender, and who are available to conduct evaluation missions, we are quickly in trouble.

This is reflected in the quality of the evaluation work carried out on gender issues, which has a lot of weaknesses, because the people who carry out these evaluations do not have a good grasp of gender analysis tools. These people do not have sufficient expertise to analyse the complexity that is generally found in these issues. As a result, we will see reports that address the issue of men's and women's participation, but do not address other issues such as social norms and practices that are harmful to girls and women, unequal power relations and participation in decision-making spheres, the sexual division of labour and the workload of men and women, gender-based violence, sexual harassment, sexual exploitation and abuse, ...

Sometimes gender mainstreaming is done around one axis or part of the deliverable, and thus sacrifices the transversality of gender mainstreaming. Thus, gender is only found in a dedicated section, whereas it should be found in all the new analysis sections, whatever the part of the work. Moreover, the commissioning of the evaluations is itself responsible. Gender is commissioned as an additional element to an evaluation, whereas it is the evaluation process that should take gender into account. The evaluator should analyse gender relevance, gender effectiveness, .... and not do a gender section.

Have the resources responsible for monitoring and evaluation in the ministries been trained in gender? I leave it to each of you to provide the answer that applies to your field.

2. The "time" resource is also to be taken into account. There is always a lack of time to carry out the required activities.

Let's take common tools as an example:

• The activity profile of stakeholders in a community or the profile of access to and control over resources are really basic tools in gender analysis: in a community, it takes about 3 hours to build this in a participatory way with the participants on a site and therefore it requires time.
• The daily agenda profile of men and women doing the same activity in the same context, a tool that allows to highlight the workload of men and women, divided between production issues, reproduction issues, political and social-community activities, also takes a lot of time to build.
• Monitoring and evaluation frameworks often do not integrate gender, and the evaluation team must then first construct what might have been the framework for monitoring gender change, before 'looking' for the effects and impacts that may have occurred! All this takes time.

3. There is also the issue of financial resources. The financial resources foreseen to carry out gender activities remain insufficient. What financial resources are the stakeholders prepared to mobilise? Up to now, as much as we may say, gender has tended to be perceived and practised as an appendix, as something that is added to something that already exists. You write a project and then you come and look at what you can add on gender issues.

As a result, the budgets allocated to gender issues are usually very low, which does not allow for the real activities that should be carried out. Exercises such as gender budgeting are not undertaken when projects are in formulation!

The mobilisation of these three resources forms a set of challenges that mean that gender issues are not really taken into account in projects, programmes and policies but also in evaluations. How much time do we have to make such systems work?

This observation on the consideration of gender issues also applies to other vulnerable populations: people with disabilities, people undergoing forced migration, indigenous populations, children, etc.

Thaddée Yossa

[translated from French]

Hello Eriasafu and Serdar,

Very useful thread to learn about gender-responsive and inclusive M&E. Thanks, Serdar for your important points. Adding to what you (Serdar) have written  (hope you have your permission!), I find the following important areas while setting up the gender-responsive and inclusive M&E: 

• Users: We find it is important to start with the user of the data while designing any M&E system. Usually, we have accountability to the donors/funders. However, the inclusive M&E is supposed to benefit the people who share their information, and the policymakers to take appropriate policy decisions. Hence, I'm inclined to citizen's monitoring, which can be in a form of a citizen scorecard or some other tools (such as other participatory tools). 
• Set of objectives - Do we have explicit mention of women, youths, and gender and sexual minorities (GSMs), for example (per context)? 
• Indicators -  Do we have both qualitative and quantitative indicators, measuring the changes/outcomes/impacts on various groups of people (women, youths, GSMs)? 
• Tools - Do the tools help getting disaggregated data, both qualitative and quantitative? 
• Data collectors - Who is going to collect the data? Do we maintain enough sensitivity to the confidentiality and privacy of the participants? Do we have rightly trained people? 
• Data analysis and reporting - Does the data analysis and reporting equally prioritize all groups? If we consider who are going to be engaged in data analysis and reporting when designing an M&E system, it helps shape it, accordingly. 

My two cents, 

Binod

Dear Eriasafu,

Thank you for raising these important questions and emphasizing the need for integrating the evaluation function with those of monitoring and learning. All these important functions are inter-connected and as such are integral parts of the effective project management cycle. It would, indeed, be the most effective approach if an evaluation could be integrated in the overall results-focused management system from the start, rather than be carried out as a “one-off” exercise occurring only at certain point of the project/programme cycle.

I would like to highlight a few points that may stimulate further discussion on the questions that you have posted.

Purpose and utility

Before any monitoring and evaluation system is established or developed, its main users/stakeholders and developers need to be clear on the purpose and utility of the system. For example, the system that is mainly generating data for higher-level (corporate) reporting is different from the system that focuses on measuring benefits to the local community or beneficiary-level impact monitoring.  Any system developed need also be pragmatic, and take into account available capacities and resources, without creating or adding levels of complexity or functions that may not be used subsequently.

Human rights and gender-related considerations.

The monitoring and evaluation systems should adhere to the rights-based approaches, potentially identifying the effect of programmes on people realizing their human rights and identifying potential best practices of ensuring respect for human rights while implementing operations and programmes. The monitoring and evaluation processes and activities should duly integrate gender-related considerations, measuring any effect of programmes on women, girls, men and boys, and assessing benefits and deprivations. These can be achieved by integrating gender-sensitive indicators of performance in projects’ results and resources frameworks, identifying potential sources of gender-disaggregated information, and ensuring that gender-disaggregated data is collected and used for monitoring and evaluation, to the extent possible. 

Addressing constraints

Monitoring and evaluation activities should take due consideration of the projects and programmes’ operating environment and in particular potential constraints and risks. In emergencies and crisis-affected settings, there are multiple constraints and limitations, including issues of secure access to project sites and intended informants, availability and reliability of data, possible biases of key informants in crisis-affected settings due to their location or affiliation. These constraints can be addressed by using local partners/experts, using third party monitoring, and/or use of modern technology that allows remote monitoring and data collection (e.g. remote sensing via satellites, geospatial data available, digital data collection, mobile phone-based data platforms, remote sensing with satellites, etc.)

Do No Harm

In all contexts, and particularly in crises-affected settings, “Do No Harm” principles should be applied in planning and undertaking the monitoring and evaluation activities, bearing in mind potential sensitivities and tensions. The proposed M&E approaches should be planned in a manner that do not affect providers of monitoring and evaluation data, and do not exacerbate existing tensions, and/or worsen relationships between the informants and the other local actors/communities. 

Kindest regards,

Serdar Bayryyev (FAO)

Dear all,

I am really interested in this debate, monitoring and evaluation are based on a logical model, the theory of change and will have to be based on the integration of gender analysis and inclusiveness.
 

Good morning to all,

The topic under discussion is of great interest, given the monitoring and evaluation aspect of development projects that has been raised: leaving no one behind and taking into consideration the concerns related to gender and the disadvantaged. It is very important to think about this. But to do this, the necessary preparations must be taken very early on, upstream of the monitoring-evaluation process. A methodological approach exists in this respect and is used in the field of natural resource management, but also in all other areas of socio-economic development. It is the 4Rs method. It is a method that allows all stakeholders to be identified in a participatory way, without the risk of forgetting any. The 4R method stands for Responsibilities, Rights, Relationships and Revenues. The 4Rs method is very effective because it identifies project stakeholders on the basis of duties, rights, relationships and revenues. The effectiveness of the method therefore comes from the fact that it is used from planning to monitoring and evaluation. When the method is used, monitoring and evaluation opens the way to focus sessions for rich and comprehensive information. I am willing to discuss this further, if necessary.

Dr Émile N. HOUNGBO

Agroéconomiste, Directeur de l'Ecole d'Agrobusiness et de Politiques Agricoles, Université Nationale d'Agriculture de Porto-Novo (Bénin)

If the programme or project document  to monitor and evaluate is logical and evidence based, it is easier to develop an MEAL system or framework

We should therefore start by having a logical and evidence based document on a programme or project for MEAL

Abubakar Muhammad Moki(PHD)

Uganda

We need to ensure the quality and transparency of the data and then make sure that the views of all stakeholders, especially the disadvantaged, are taken into account.

Constraint: The independence and politics of evaluation! 

We don't learn from our mistakes and keep doing things that don't work - we need to respect the standards of evaluation and accept our failures. 

Dear Eriasafu,

Many thanks for the post, and good to be in touch on the subject of monitoring, much neglected and given short thrift by the evaluation community.

I like your observation on how time complying to demands of collecting data all the way to the top of the results framework or theory of change, often missing out on the assumption along the way, crowds out time for reflection and learning. I believe such reflection comes in revealing the unknown through listening to and learning from those in need, not measuring those in charge - excluded and underserved communities.

So, how to resolve the issue you raise as to how "MEL/MEAL systems are limited to compliance, outcomes and impact, and rarely include cross cutting issues such as gender and leave-no-one behind principles."

It strikes me as ironic how, as monitoring is all about learning, it, itself, shows a limited capacity to learn about its past. The pursuits of measuring outcomes and impact are not so much limiting as they are mis-guided. Even if you had more time, outcome and impact indicators generate limited value for learning purposes. This is easier said than done in comparison to measuring indicators laid out in some needy theory of change or logic model. Indicators do what they are supposed to do, they measure things that happened, or not, in the past. They don’t tell you what to do. Monitoring does and should not entertain using rigorous – as a statistician would define the term - methods geared to academic concerns and obsessive pursuits of measuring and attributing intervention effects.

Monitoring has different requirements as highlighted above; that is, if it is to help managers resolve their decision uncertainties. Your claim ignores the hegemony of mainly transient, academically inclined western evaluators, and those in the monitoring and results measurement community, addicted to single narratives, and rigid methodological dogmas. Monitoring needs to free itself from these mechanistic approaches; and managers need to step up, afford primacy to the voices and needs of indigenous communities, and take ownership to ensure monitoring does generate insights for decision-making purposes that benefit those who legitimize, not just measure the predicted results defined by those who fund, development and humanitarian aid.

Of course, including gender and ensuring no-one gets left behind is important. However, and without sounding glib, doing this means management not getting left behind by, for example:

• Pointing out that exploring assumptions matter as much as, if not more than measuring indicators and the ‘system’ needs to be driven by questions defined by those who are its primary users, and they do not include external evaluators;
• Highlighting how, although numbers are important, they are arguably not as important as learning how, for example, the numbers of men and women or boys and girls came to be and how and how well they interact with  together.

Thanks again, and I hope the above helps,

Daniel