RE: Is this really an output? Addressing terminology differences between evaluators and project managers | Eval Forward

Dear Natalia et_al.,

Thank you for putting on the table an important challenge to both the evaluator and the manager of a development project. And I want to apologize for not being able to answer earlier; the situation in my country had taken over my mind and took all my time during the last 3 weeks. The question of clearly distinguishing an output from an outcome is of utmost importance for the development project manager as well as for the evaluator, as well as the project monitoring and evaluation staff. And I doubt that the problem is really a terminology problem, at least theoretically speaking. According to my modest experience, the problem has its origin in several factors that I will try to explain below:

  1. The weak link between project formulation, implementation, and monitoring and evaluation, of which the results framework (or logical framework) of a project is be the basis of this link. In this perspective, coherent and relevant indicators of the different types of results are formulated during the formulation of the project, even before the implementation of the first activity is launched. This weak link sometimes explains the difficulties in developing the ToR of an evaluation and therefore the difficulties that an evaluator may encounter in assessing the achievements and effects of a project, as mentioned by Natalia.
  2. The flagrant lack of skills and resources in project monitoring and evaluation, including monitoring, for various well-known and / or less well-known reasons. In some cases, managers prefer to conduct one or two ad hoc evaluations in the life of a project rather than having to support a follow-up service for the entire project duration, thinking that they will achieve the same purpose.
  3. The "too rigid" procedures in terms of monitoring and evaluation, adopted by some organizations, and confined in handbooks very often poorly developed - as evoked by Isha in this discussion. One of the reasons, in my humble opinion, is very often the focus on the specific content and the less importance attributed to the communicative dimension in the preparation of these handbooks. We may have sometimes mobilized a great resource person for the specific content, if he/she does not have the necessary communicative competence, we will get a very high quality specific content but a handbook that is practically useless.
  4. The apprehension, very often based on mistaken beliefs, made by some project managers on the project monitoring function, that yield a very little importance is given to the monitoring and evaluation staff . This somewhat explains the poor quality of the ToRs that Natalia is talking about in her contribution.
  5. The "voluntary" and "voluntarist" efforts of some enlightened practitioners who have sought at any cost – over the past two decades – to put a barrier between monitoring and evaluation. However, any development project necessarily needs the two "feet" of its monitoring and evaluation system in order to achieve the objectives assigned to it. Monitoring can never explain things exactly as the evaluation can do concerning some project happenings and the evaluation can not fill the gaps of the monitoring function. 

Having this said, a good training on the monitoring and evaluation of project staff, based on good logframe and result chain, can sometimes be the key to this problem. And to support this, I would like to share an experience I personally experienced in Sudan in 2003 on a project co-funded by IFAD and the Islamic Development Bank (IsDB) in North Kordofan State

I was contracted by IFAD to support the consolidation of the monitoring and evaluation system of this 7-year project while it was in the 4th year (first anomaly). The project was to deliver several outputs, including a 60 kilometre tarmac road between the State capital, El-Obeid, and the State second city, Bara, entirely financed by IsDB.

Locked up for 3 days with the entire project team, I was able to clearly see, through the indicators of effect proposed to me, that the project management team, including the principal responsible for monitoring and evaluation, was unable to clearly differentiate between the deliverable (the tarmac road) and the effects this deliverable could engender on its beneficiaries' living and income conditions. And slowly, my intervention and assistance made it possible for the project staff to start differentiating between a deliverable and its effect - as a development intervention - which can be perceptible only at the level of the social segments benefiting from a deliverable and not in the deliverable per se. I fully understand that the transformation of a stony road into a tarmac road is a change, but without the inclusion of the human dimension in our vision, it is difficult to pinpoint the development achieved. For proof, where can we perceive development of a new deliverable realized and closed for 3 years, for example, if human beings do not take advantage of it in order to change their living and income conditions (isn't it Hynda?). Thus, the project team members started, from the second day onwards, to differentiate things, suggesting better outcome indicators – completely different from output indicators, which served 3 years later to a good evaluation of the effects of the deliverable "tarmac road".

Thus, this little story highlights the necessary link that needs to be established between monitoring and evaluation from the start of a project – through mobilizing all necessary resources for the monitoring and evaluation system, including the necessary skills – so that evaluation can be done without much difficulty.

But even more importantly, although I am in favour of the evaluator "freedom of expression" (Isha), this necessary link between monitoring and evaluation will certainly lead to better ToRs for evaluation, guaranteeing this evaluator freedom within the framework defined by the project team. Without this link, too much of the evaluator's freedom of expression may incur a project at risk of receiving an evaluation report that is meaningless.

Sorry to have been a little long but the importance of the question asked by Natalia forced me to resort to certain details. I hope I have contributed a little bit to this discussion.

Mustapha