Позитивные изменения, Том 3 №1, 2023. Positive changes. Volume 3, Issue 1 (2023)

Another negative effect is aggression, abusive behavior and violence against competitors and referees. One example is Roman Shirokov, former captain of the Russian national soccer team, beating referee Nikita Danchenkov during the Moscow Celebrity Cup amateur tournament[78 - Armand, N. (2020). Roman Shirokov Sentenced to Compulsory Work for Beating a Referee. Retrieved from: https://sportrbc.ru/news/5fe5d56a9a7947e4123b977e. 25.12.2020. (accessed: 30.01.2023).]. Peter Dawson et al. (2022) studied the effect of abuse on match officials’ decision to leave their jobs. Based on surveys of soccer referees in France and the Netherlands, the authors investigated factors associated with verbal and physical abuse of referees, as well as the relationship of such abuse to referees’ intentions to leave the profession.

Sports can also have negative effects on relationships within the family. In particular, researchers highlight the high costs of playing sports to families (Kay, 2000; Dixon et al., 2008). Besides the financial obligations, the time costs are also increasing. Tess Kay (2000) notes that the need to exercise affects daily routines and causes changes in vacation plans and work schedules. Negative consequences can vary from increased stress and anxiety (O’Rourke et al., 2011) to decreased self-esteem and trauma (Kay, 2000). Feelings of jealousy and resentment or lack of time for other activities are also noted (C?tе, 1999). Tess Kay (2000) found that some children who did not compete felt jealous of their siblings who did participate in sports and felt that their relationship with their parents was not as strong. Finally, it has been shown that parents become less active as a result of their children’s participation in sports (Dixon, 2009).

The effects generated by sports projects can be classified into positive and negative, and by the direction of the impact – in the context of the social sphere, health and economy.

According to UEFA[79 - UEFA. (2019). The UEFA Grow SROI Model: Valuing the Impact of Football Participation in Europe. Retrieved from: https://www.knvb.nl/downloads/bestand/26276/uefa-sroi-final-report. (accessed 30.01.2023).], the negative effects of social sports can include injuries, doping use, corruption and the influence of gambling.

PROJECT EFFICIENCY ASSESSMENT

Several approaches can be used to evaluate the effectiveness of individual projects that use physical activity and sports to achieve social and economic effects, including expert assessment, project evaluation (and monitoring) and monetary evaluation, such as Social Return on Investment (SROI) evaluation. We will consider the similarities and differences between these approaches using the Girls on the Soccer Field project as an example. The project was implemented in Turkey in 2016. Social Return on Investment was calculated for it, and the corresponding report was certified by Social Value, an international network promoting the SROI approach[80 - Social Value UK. (2023). Retrieved from: https://socialvalueuk.org/. (accessed 30.01.2023).].

DESCRIPTION OF THE “GIRLS ON THE SOCCER FIELD” PROJECT

The Girls on the Soccer Field project was created by Melis Abacioglu and her company Actifit in 2013. The year before that, Melis and her friends had organized a soccer game for women after their male friends refused to play against them.

The project includes the establishment of a Soccer Academy – for five days, the girls participate in soccer training, as well as creative master classes in visual and theatrical arts and creativity development. In addition, a lecture on gender equality was held for the parents of female Academy participants, who were invited to attend their daughters’ training sessions.

Several approaches can be used to evaluate the effectiveness of individual projects, including expert assessment, project evaluation (and monitoring) and monetary evaluation.

Project goals:

• Through soccer, raise awareness of gender equality and promote greater participation of girls in sports;

• Develop the girls’ creative potential through creative workshops.

EVALUATION OF SOCIAL RETURN ON INVESTMENT

The assessment began with a stakeholder analysis. The following groups were identified: girls participating in the Academy, their parents, volunteer trainers, Actifit (the organization implementing the project), donors who supported the project financially, the foundation that conducted a presentation on gender equality issues for the parents.

The next step in the evaluation process was data collection, which started immediately after the completion of the Academy. The data was mainly collected through survey. Questionnaires were developed for each stakeholder group. These included open-ended questions about participation in the various project activities and the results obtained, for example: “What did you gain from participating in the project? How did the project change your life? Were there any negative consequences of your participation in the project?”

Based on the data collected from all stakeholder groups, their contribution to the project was determined. For example, the girls, their parents, and the volunteer coaches contributed their time; the donors contributed money and sports equipment.

In addition, based on the analysis of the completed questionnaires, the positive and negative results obtained were determined, as well as the relationship between the project activities and results. It is worth noting that this analysis divided the results into expected (planned) and unplanned ones. In case of the girls, for example, the project produced three types of results. First, they became more confident and convinced about the possibility of gender equality. According to the researchers’ analysis, the following chain of changes led to the first result: through soccer training, girls realized that soccer was not just a game for boys, as they had previously thought; besides, soccer training allowed them to appreciate the positive effects of sports, which gave them self-confidence and created awareness of gender equality opportunities. Second, the girls developed imagination and creativity. Third, it was easier for girls to communicate with friends and parents. The authors consider the latter an unplanned outcome of the report, as it was not part of the project goals.

The project results for parents included increased awareness of gender discrimination and improved mutual understanding, as the parents started to communicate more with their daughters. Volunteer coaches have developed communication skills, gained coaching experience, and noted successes in personal growth through participation in the project.

Actifit, which prepared and implemented the project, was able to engage donors, partners and volunteers. Therefore, one of the positive results of the project for the company was saving its own financial resources and employee labor. Additionally, the company was able to make the project more sustainable. Finally, successful operation of the Soccer Academy in Eskisehir was a case study that Actifit can use to further promote its “Girls on the Soccer Field” program. Employee motivation associated with participation in this program has also increased. The negative result of the program was that three employees of the company spent 28 days of their time working on the project.

The next step was monetizing the contributions made by the stakeholders and the results obtained. Only financial and in-kind contribution by Actifit and the donors was taken into account during the monetary evaluation. The report authors decided not to include a monetary evaluation of the time spent on participation in the project by children, parents, and volunteer coaches.

In monetizing the results, the authors used data on the cost of paid services, which would enable the project participants to obtain similar results otherwise. For example, there is a paid soccer section in Eskisehir. The assumption was made that a week of paid soccer lessons would give the girls the same result in terms of developing selfconfidence and understanding the possibility of gender equality, so the cost of this week was used as a “proxy” in estimating the cost of the outcome.

Outside of the project, parents could learn to understand their daughters better and communicate with them more if they took a class with a family psychologist, so the cost of such a course was a monetary estimate for this outcome.

Successful operation of the Soccer Academy in Eskisehir was a case study that Actifit can use to further promote its “Girls on the Soccer Field” program.

Since no one lectures on gender inequality in Eskisehir on a fee basis, it is curious that raising parents’ awareness of gender discrimination has received zero monetary value.

The project SROI was then calculated based on the monetary estimates obtained, both for the project as a whole and for each individual group of stakeholders. The overall SROI for the project was 12.45 – that is, for every Turkish Lira (TL) invested in the project, 12.45 TL worth of social effect was produced. Of these 12.45 TL, the girls’ results accounted for 20 % (2.43 TL) and the parents’ results accounted for 38 % (4.68 TL).

PROJECT EVALUATION

There are several different but similar approaches to evaluating projects. We prefer the one specifically described by the term ‘evaluation’.

This approach features a set of analytical activities that are conducted to provide project's decision-makers with the information they need regarding the project. The decision-makers in this case are usually either project managers or donors.

This approach assumes that a monitoring and evaluation system is developed as early as at the project planning phase. The system is based on the results of stakeholder analysis and a description of the project mechanism, which explains how the implementation of project activities will lead to the achievement of expected results (this description is commonly referred to as the theory of change or logical framework of the project).

Accordingly, using this approach, a stakeholder analysis of the “Girls on the Soccer Field” project would have had to be conducted as early as the planning stage. After that, it would be necessary to build a change map or a logical framework that would explain how the project activities would eventually translate to the progress towards its goals. Building such a map could also help to specify the goals (final expected results) of the project. For example, one of the goals is formulated as follows: “Through soccer, raise awareness of gender equality and promote greater participation of girls in sports.” In developing a change map, one would need to decide what chain of changes connects a presentation on gender equality for girls’ parents to this goal: for example, more informed parents would support their daughters’ desire to play sports. At this point, one may deem it reasonable that the results of the presentation would be better placed in a separate chain of changes and isolated as a separate project goal.

Based on the map of expected changes, the project authors determine what information will be needed to track the progress of project implementation and achievement of its goals. The next step is to decide who, when and how will collect, analyze and use this information.

The key task of evaluation is to help project managers understand whether the intended mechanism for achieving change is actually being implemented. To manage the process of change, it is very important to know where there are failures in the process. Accordingly, the monitoring and evaluation system is designed to identify these failures. In addition, project management now pays a lot of attention to unplanned results, whether positive or negative. This is why data collection for project evaluation could be much the same as in the SROI evaluation described above. But the analysis would have a clear focus on the degree to which the planned results are being achieved, and which of the unplanned results are positive and negative.

It should be noted that any SROI calculation or another form of monetary evaluation can be incorporated in the project evaluation process if decision makers need this information.

EXPERT ASSESSMENT

Expert assessment is generally used when the social results of a project cannot be measured, for whatever reason. It is most commonly used by donor organizations to assess project applications. Expert assessment is a judgment about the value of a project, about how good it is. When conducting an expert assessment, the client makes a list of criteria – the properties that a project must have, and the experts must answer how well the proposed project meets these criteria, based on the description provided in the project application and their own knowledge and experience. For example, if donors to the “Girls on the Soccer Field” project were to make a decision about funding based on expert assessment, they could use criteria such as whether the project is consistent with the strategy, whether the project is effective (able to achieve the expected results), whether the project is economically viable (uses the cheapest way to achieve the expected results) or there are more cost-effective options.

Since achieving the expected results is the main expectation of the project, the expert assessment can focus on a detailed analysis of the expected chains of changes. In this case, experts are presented with a detailed map of changes prepared by the project authors and are asked to assess the feasibility of all the links between the results, so any possible gaps can be found.

For example, when planning the “Girls on the Soccer Field” project, its authors might have planned the following chain of changes:

• Girls who, prior to the project, thought that soccer was a boys-only game take a full training course at the Academy (five days, three hours a day);

• As a result, they successfully learn the basic skills of the game;

• The girls successfully apply the new skills in practice games;

• The girls get satisfaction and enjoyment from the game;

• The girls understand that the idea that only boys are capable of playing soccer was false;

• Getting rid of one of the gender stereotypes helps girls see that gender equality is indeed possible.

During the project evaluation, the experts will need to answer how likely it is that completing one step in the chain will result in a successful implementation of the next step. For example, if the girls take a full course of training at the Academy (15 hours), what is the likelihood that they will learn the basic skills of the game?

CONCLUSION

Global experience shows that projects implemented in the sphere of physical culture and popular sports can create both significant positive social changes and negative effects. The study created a classification of these effects, which will help organizations implementing and supporting social sport projects to identify and evaluate them. The use of different types of assessment enables identification and measurement of the changes created at all stages of the project, which, in turn, helps to manage the change process better. In addition, the assessment is important to justify the amount of funding and the choice of strategic development areas, which is especially relevant in the current context, when all social initiatives can no longer count on the previous level of support from budgetary and non-budgetary sources.

REFERENCES

1. Gureeva, E. A. & Solntsev, I. V. (2014). Assessment of the Economic Effect of Major Sporting Events. MOSCOW: Ru-Science Publishers. 92 pages.

2. Allport, G. W. (1954). The nature of prejudice. MA: Addison-Wesley.