RE: Is this really an output? Addressing terminology differences between evaluators and project managers | Eval Forward

Dearest Natalia,

I am not surprised either and I have similar experiences with many organizations.

This is the one of the reasons why I decided to lobby for “freedom of speech for evaluators”. Let me explain what I mean.

First, we cannot be independent and bound to be under ToRs instructions; this is not how evaluation should be conducted. Evaluation is a lesson learning, gap-finding mission to eliminate obstacles and prepare to become a visionary leader, seeing things early, logically and to respond and share with rest of the stakeholders and guide them to take things forward.

I have done some fact-findings on this subject:

  1. In many organizations, the M&E entity or unit lacks evaluation knowledge at the field or ground level. They are leaning the art during the evaluation period from consultants.
  2. Most evaluators are bound to work as per the “Handbook” given by the organization and are unable to attend to any changes if required. Very uninformative.
  3. Most handbooks are more similar to curriculum for evaluation studies than “guidelines for evaluation”.
  4. A very few “Handbooks” are evaluation oriented, and instead focus on research approaches. Even the Kellogg foundation evaluation handbook does not differentiate the Researcher and Evaluator in https://cyc.brandeis.edu/pdfs/reports/EvaluationHandbook.pdf (see page 7).
  5. In some cases the definition of the Evaluation is questionable in both documents, Handbook and TORs. i.e. TOR and handbook are not compatible. The Handbooks give guidance on major programmes evaluations or end/post evaluations but lack guidance and examples in the field and ground challenges.
  6. Very few give templates for a TORs Terms of Reference
  7. Many ToRs are cut and paste
  8. Most manuals include standards for all evaluations including the questionnaires as well as target groups and instruction on methods of conducting the interviewers and on selection of target group identified by the programmes. Most of the time they are very biased.
  9. No questions are suggested to address the indirect outcome or impact of the programme but only direct expect-able answerable questions.

Overall sometimes I have asked the organization "why do you hire me, you can do it yourself" when they give and indicate everything on the methodology to conduct the evaluations.

Best

Isha

Independent evaluator

Sri Lanka