Позитивные изменения, Том 3 №1, 2023. Positive changes. Volume 3, Issue 1 (2023)

• Evaluation participants – the people who provide the information for the evaluation; these are usually employees or managers of the programs being evaluated (ASPPE, 2017).

Figure 1. Self-evaluation: three in one

ASPPE suggests that evaluation must be guided by the following principles:

1. Focus on practical use of the results.

2. Competence of the performers.

3. Appropriate methodology.

4. Transparency.

5. Safety.

6. Flexibility.

The “general case,” for which ASPPE principles were developed, is an external evaluation (see Figure 1).

A program (project) self-evaluation is the systematic collection of information about the activities of the program (project), its characteristics and outcomes, which is performed by the program (project) team to make judgments about the program (project), improve the effectiveness of the program (project), and/or inform decisions about future programming. The difference between this definition and the generic definition given at the beginning of the article is all in the italicized text: here, everything is done by the project team itself. The people implementing the project also set the evaluation task, plan the evaluation, collect the data (while also serving as important sources of information), analyze the data, formulate the evaluation results, and use them – all on their own. It turns out that in self-evaluation, the project team acts as the customer, the contractor, and the evaluation participants (see Figure 1).

Do the ASPPE evaluation principles apply in a self-evaluation situation?

We had a series of discussions on this issue as part of the 2022 PROOCENKU Club meetings[32 - Videos and materials from the meetings can be found on the Club’s website: http://www.proocenku.club/.]. They were attended by more than 40 people, mostly coming from non-profit organizations from many regions of Russia.

The outcome of these discussions can be summarized as follows:

1. The evaluation principles proposed by ASPPE are applicable to NGO self-evaluation.

2. Short definitions of these principles also apply to the self-evaluation situation.

3. However, recommendations for the use of these principles in self-evaluation practice require significant adjustment and simplification.

We have developed and discussed guidelines for a self-evaluation situation. This is how the Principles of Self-Evaluation of NGO Programs and Projects have appeared (PROOCENKU Alliance, 2023). These principles, with recommendations for their application, are presented below. I believe it can be useful to any NGO, regardless of its specialization.

Principle 1. Focus on practical use of the results.

The whole self-evaluation process should be focused on getting information that is useful to you.

Recommendations:

• You need to be as specific as possible as to why you are doing self-evaluation. Think carefully and discuss the scope of work for self-evaluation. Formulate the questions you want answered, and discuss who will use those answers and how.

• If it is not clear who and how will use the answers to any of the suggested questions, exclude these questions from the scope.

• Include only those questions where answers are not known or at least not obvious to you.

• See if there is a simpler way to answer these questions, rather than performing a self-evaluation.

• Try to keep the number of questions to a minimum. Leave only those of most importance to you.

• At the end of self-evaluation, hold a meeting to discuss the results and plan actions required to use those results (with timelines and responsibilities).

• Think about whether some of the results of your self-evaluation can be useful to someone else besides you.

Principle 2. Competence of the performers.

NGO employees who conduct self-evaluation must have the necessary and sufficient knowledge and skills to do so.

Recommendations:

• Self-evaluation is carried out according to certain rules. You don’t have to become an expert evaluator, but you do need to know the basics. To do this, you can read special literature or send some of your employees to a training.

• Keep in mind that it is always possible to consult with experts in areas where your own knowledge and skills are lacking. For example, your local university or professional evaluators association.

• Consider self-evaluation an opportunity to learn through practice. Discuss the experience and learn from it.

Principle 3. Appropriate methodology.

The choice of the general approach to self-evaluation and the methods of conducting it should be well justified, considering the limitations. Various evaluation methods must be used, following appropriate procedures and standards.

Recommendations:

• When planning self-evaluation, consider the limitations of available resources and time. Remember that you do not have all the necessary knowledge and skills for evaluation.

• It is important to have an understanding of the strengths and limitations of different approaches to project and program evaluation.

• When choosing self-evaluation methods and tools, it is better to do simple things right than to do complicated things wrong.

• If you have any doubts about the capabilities or appropriateness of a particular tool, you should consult with an expert or refrain from using that tool.

• Remember that the familiarity and prevalence of techniques do not guarantee that they will be appropriate for self-evaluation of a particular project in a given setting.

• Everyone can make mistakes, and it is important to learn from them and avoid their repeating.

Principle 4. Transparency.

Self-evaluation is a transparent process: all parties involved must be informed about the goals, methodology and intended use of its results.

Recommendations:

• Make sure that everyone involved in self-evaluation has sufficient information about the process and agrees to provide data voluntarily (respecting the principle of “informed consent”).

• Full results of the self-evaluation are only intended for you. For everyone else, they can either be partially open (at your discretion) or completely closed.

• You need to consider that you may have a bias towards your own project and be suffering from having lost the fresh perception of the project (“blurred view”): these issues should be openly discussed by the self-evaluation participants.

• Since you are performing the self-evaluation for yourself, not for someone else, you are interested in seeing your project for what it is. In this situation, embellishing simply makes no sense. Therefore, you should not view the self-evaluation situation as a potential conflict of interest or be concerned about it.