Principles for a healthy relationship between evaluators and experts

©Atul Loke

Principles for a healthy relationship between evaluators and experts

4 min.

Evaluations led by international development agencies need a combination of different types of expertise.

A common dilemma is striking a balance between team members who understand the technical subject and those who have expertise in evaluation methodologies and processes. In most evaluations, resources are scarce to hire big teams to cover all fields. Finding people who are experts in at least one discipline, have the right language skills and are skilled evaluators is challenging.

Here are some principles for creating a good working relationship between the roles of experts and evaluators in development evaluations based on my own experience as an evaluator, and on informal conversations with colleagues, notably in the last European Evaluation Society (EES) conference.

Manage expectations and reduce confusion

All evaluation actors should work to build a common understanding of the evaluation framework (objectives, approach, focus, methods, etc.). You will save a lot of time if, from the beginning, you spend time and effort to make sure you speak the same language and agree on roles and contributions. Do not assume that evaluation jargon is universal; other professions might have different definitions of the same words and concepts.

Ensure the evaluation team leader has the appropriate competencies to do the job

All evaluators, especially the team leader, should demonstrate competencies to do the job – a set of knowledge and skills that a practitioner needs. There is an ongoing debate on “professionalizing” evaluation practice in different contexts. None of the proposed frameworks appear to have been systematically derived or empirically validated through consensus built among diverse professionals across the field, but some are being developed, such as the development and adoption of competencies in Canada. Created as a key foundation of the Credentialed Evaluator designation under the auspices of the Canadian Evaluation Society (CES), five competencies domains were identified:

  • 1.0 Reflective Practice: Competencies focus on the fundamental norms and values underlying evaluation practice and awareness of one’s evaluation expertise and needs for growth.
  • 2.0 Technical Practice: Competencies focus on the specialized aspects of evaluation, such as design, data collection, analysis, interpretation, and reporting.
  • 3.0 Situational Practice: Competencies focus on the application of evaluative thinking in analyzing and attending to the unique interests, issues, and contextual circumstances in which evaluation skills are being applied.
  • 4.0 Management Practice: Competencies focus on the process of managing a project/evaluation, such as budgeting, coordinating resources, and supervising.
  • 5.0 Interpersonal Practice: Competencies focus on people skills, such as communication, negotiation, conflict resolution, collaboration, and diversity.

Make it easy for “accidental evaluators”

If you are an experienced evaluator and have to work with experts new to the evaluation job, including many team members, make the process clear and engage everyone from the beginning. Create templates, provide resources, share timely feedback and listen to expert ideas and concerns.

Make sure experts are aware of evaluation norms and standards, and follow ethical guidelines such as confidentiality, respect of views and opinion, etc. Unlike traditional professions, the evaluation community has yet to see a proper “code of conduct”. Nevertheless, established evaluators do adhere to norms, ethical values and standards; usually gathered in documents, either with a universal scope, such as the UN Norms and Standards for Evaluation or tailored to a regional context, such as Evaluation Standards for Latin America and the Caribbean of the Latin American and Caribbean Network of Monitoring, Evaluation and Systematization (ReLAC).

Fight biases and check the validity of findings

Subject experts may have an inclination towards specific methodologies, techniques, and practices in the technical area of study, which is a real threat to the validity of evaluation findings. Experts as evaluators tend to focus on the technical side more than other equally important aspects.

If it is not within your means as an evaluation team leader to check the quality of what experts deliver, ensure that their work is reviewed both by peers and the evaluand. The larger the spectrum of peer reviewers, the more insightful the feedback collected can be and the credibility of findings strengthened.

Nevertheless, as experts are established authorities in their field they can continue the advocacy work, use evaluation findings in their field and shape the research and development agenda. On the other hand, evaluators tend to move on to the next evaluation, reducing the potential impact.

What are your thoughts on working with experts in evaluations? Leave a comment below.

  • Dear Ibtissem,

    Thank you for posting a blog on this important issue. The complexity of development interventions requires that evaluation teams have various types of expertise. Yet, it is often quite difficult to secure right mix of expertise, and it is even more challenging to ensure that this expertise is effectively utilized during a short span of the evaluation process.

    Evaluation managers/leaders (hereinafter referred as Evaluation Team Leaders) need to make their thoughtful decisions on what expertise they need to obtain for the evaluation team, and what combination of expert knowledge and experience would work best for their evaluation.  In making these decisions and in leading the evaluation processes, Evaluation Leaders may indeed be guided by the five competency domains highlighted by Ibtissem  (or the Competencies for Canadian Evaluation Practice), which resonate with the 2016 UNEG Evaluation Competency Framework ( ).  

    The role of the evaluation leader in managing the teams of experts, is indeed strategic and challenging, and I would like to add just a few considerations in this regard, to stimulate further discussion on this important topic:

    Establishing strong leadership

    Evaluation Team Leaders need to effectively lead the evaluation team, coordinating effective engagement of each expert in the team, validating their individual inputs and undertaking quality controls at key milestones. Continuous communication with each team member throughout the process is essential to ensure that any questions are potential risks of drawbacks are timely and effectively addressed.

    Ensuring compliance with organizational standards

    The Evaluation Team leaders need to require compliance of each team members with applicable rules and regulations, and standards of conduct established by the organization, and establish quality parameters of expected deliverables. In collaboration with each team member, Evaluation Team Leader should ensure that there is a shared understanding of what is to be evaluated and how, clarifying how organizational standards and processes are to be applied, and what is expected from each individual team member.

    Focus, focus, focus

    Biases are unavoidable, yet their influence can be minimized by focusing on evaluation objective and clearly establishing the scope of evaluation analysis. Furthermore, it should be made clear that any expert inputs and viewpoints need to be validated and supported with solid and relevant evidence.

    Building relationships

    Teambuilding skills are essential, to ensure that team members support each other and works towards achieving common objective.

    Engaging external expert panels

    If at all possible, external panels of experts may be engaged to provide feedback on key thematic areas of your evaluation analysis. Recognized experts from counterpart organizations may assist by serving as high-level expert panel members, to review findings produced.

    These are just a few examples, and I anticipate continued discussion on this topic. 

    Kindest regards,

    Serdar Bayryyev