The changing face of monitoring and evaluation in the age of COVID-19: practitioners' field experiences from Zimbabwe.

The emergence of COVID-19 as a global pandemic presented a novel challenge to monitoring and evaluation in the humanitarian and development sectors. The measures taken to contain the spread of COVID-19 disrupted the traditional way of doing business in both the programming and monitoring and evaluation sectors. In particular, restrictions on movement in order to reduce the spread of the virus meant that monitoring and evaluation work had to transform from the traditional approaches. This study sought to investigate how monitoring and evaluation practice has evolved under the COVID-19 pandemic in Zimbabwe. The main objective of the study was to document lessons learned from different organisations and practitioners and to share best practice. The study deployed an online survey using Kobotoolbox and reached 171 respondents. A Webinar session with six presentations and discussions with programming, monitoring and evaluation practitioners in Zimbabwe was held to share experiences and lessons learned. This was followed up by key informant interviews with selected stakeholders. The study revealed a general shift from conventional monitoring and evaluation to COVID-19 tailored approaches which include deprioritisation of face to face data collection and increased remote data collection mechanisms, maximum utilisation of secondary data, limiting data collection to essential and critical data, simplifying the data collection methods and, rethinking sampling designs to promote inclusion. The study makes several recommendations for best practice and learning.

Introduction

The emergence of COVID-19 as a global pandemic presented a novel challenge to monitoring and evaluation in the humanitarian and development sectors. The measures taken to contain the spread of COVID-19 disrupted the traditional way of doing business in both the programming and monitoring and evaluation sectors. In particular, restrictions on movement in order to reduce the spread of the virus meant that monitoring and evaluation work had to transform from traditional approaches with more functions becoming performed virtually. This trend was observed in other sectors in including medicine, where telemedicine was being promoted, i.e., the delivery of healthcare services using information and communication technologies where distance was a factor (Ghosh et al., 2020). The early screening of COVID-19 respiratory symptoms was mostly conveniently conducted through this method utilising smartphone-based applications and the telephone (Hollander and Carr, 2020). Similarly, Humanitarian and Development agencies were reviewing the way monitoring and evaluation was conducted in line with the need for social distancing (United States Agency for international Development (USAID), 2020; United Nations Children's Fund (UNICEF), 2020; United Kingdom Agency for International Development (UKAID), 2020). By May 2020, a number of international humanitarian and development organisations including USAID and UNICEF had started to develop guidance documents on implementing monitoring and evaluation under the COVID-19 pandemic (UNICEF, 2020; USAID, 2020). COVID-19 has also led to a change in the way programming has been done by different agencies as some agencies had to adjust programs to incorporate COVID-19 needs (Rogers, 2020).

Impact of corona virus

COVID-19 has had a huge impact on all nations across the globe and is likely to continue until a cure is found or the efficacy of vaccines is ascertained in the long-term. Mohamedbhai (2020) argues that, ‘COVID-19 will leave no sector in any country in the world unaffected, and its consequences will be felt for years to come’. In explaining the gravity of the impact of this pandemic, Morrow-Howell et al. (2020) state that, ‘A series of challenges have arisen in the wake of the COVID-19 pandemic, including economic setbacks and adverse health and well-being effects.

COVID-19 (also known as the coronavirus disease 2019) was first reported from Wuhan, China, on December 31, 2019 (Wilder-Smith et al, 2020; World Health Organization, 2020). Zimbabwe reported its first COVID-19 case on the 21st of March 2020 being a case of a male adult who had recently travelled from the United States. Data from China indicated that older adults, particularly those with serious underlying health conditions, were at higher risk for severe COVID -19–associated illness and death compared to younger persons (Centre for Disease Control, 2020). The global reaction by most governments was to implement national lockdowns where the movement of citizens for non-essential services was restricted in order to contain the spread of the virus (Ghosh et al., 2020). The response mechanisms put in place by governments and organisations to control this pandemic had an impact on programming as well as monitoring and evaluation of social programs globally.

The programme planning, implementation, monitoring and evaluation units and activities within different organisations were not spared. The pandemic created a crisis worldwide. The International Labour Organisation (ILO) (2020) acknowledged that, ‘The crisis has already transformed into an economic and labour market shock, impacting not only supply (production of goods and services) but also demand (consumption and investment)’. The response priorities for Development agencies were to protect the health and well-being of both program clients and staff by taking various pathways through which both programming and monitoring and evaluation are conducted. The long-term impacts of the virus on patients remain not yet fully understood. Currently, data indicate that most people who contract COVID-19 will see a full recovery, but the long-term effects of the illness are not fully understood, particularly for patients who need more intensive care (Schumaker, 2020).

Response measures

García-Peñalvo (2020) asserts that the COVID-19 pandemic has imposed a demand to learn from unexpected events and past crises. Organisations worldwide have come up with a number of strategies to mitigate the impacts of COVID-19. ILO (2020) states that, ‘Preparedness at all levels is essential to mitigate impacts and increase resilience, protecting jobs, enterprises and livelihoods’. Song and Karako (2020) propound that, ‘Rapidly gathering and sharing scientific information about COVID-19 is an effective way to improve public responsiveness’. The sharing of information can be useful to all organisations beyond the health, sector. Programming, monitoring and evaluation platforms for sharing of experiences can be very worthwhile in improving practice in this age of Covid19 and beyond. Collaborative efforts can also assist in responding to this pandemic and mitigating the impacts. Given the impacts of COVID-19, humanitarian and development agencies are having to adapt their activities quickly to respond to the threat of the pandemic and keep both staff and clients safe (Macfarlan, 2020; García-Peñalvo, 2020).

History of remote monitoring and evaluation

A number of organisations rolled out guidance documents to their partners to assist them on adjusting monitoring and evaluation procedures under the pressure of COVID-19, particularly emphasizing the need for remote monitoring. The War Child Canada and the Women's Refugee Commission (2020) and UNICEF (2020) recommended a number of changes to the monitoring and evaluation systems of their partners. Remote monitoring and evaluation was proffered as a necessary response to the pandemic. The guidance document produced by War Child Canada and the Women's Refugee Commission (2020) details the definition of remote monitoring and the methods and tools to be used in remote monitoring and evaluation. Remote monitoring and evaluation, as proffered by War Child Canada and the Women's Refugee Commission (2020) and USAID (2020) amongst others, are entirely new methods. They have been under trial in the medical field for cardiovascular related medical monitoring over the years (e.g. Burri et al., 2011). Although this guidance document was drafted for organisations focusing on gender-based violence, much of what it discusses also relates to programs in general in the development and humanitarian sectors.

It is important to note that the COVID-19 pandemic aggravated already bad circumstances in programs such as those addressing GBV and food security. In general, extended lockdowns have meant that groups vulnerable to abuse such as women and children have had to endure longer periods locked up with perpetrators of abuse and thus being exposed to greater violence (Berman, 2020). Therefore, monitoring and evaluation became more important in order to keep track of trends. The document defines remote monitoring as ‘… the use of methods to review project progress data from locations separate from project sites’ (War Child Canada and the Women's Refugee Commission, 2020:1). It is important to note that remote monitoring is not a new issue arising out of the COVID-19 pandemic. This approach to monitoring and evaluation has been used over the years in conflict areas and other crisis situations where reaching project sites was difficult (War Child Canada and the Women's Refugee Commission, 2020). The COVID-19 pandemic has simply magnified the need for this approach.

According to War Child Canada and the Women's Refugee Commission (2020), there are a number of advantages when data is collected and managed remotely. Some of the advantages include the enhanced safety of both staff and program beneficiaries because the need to travel and meet physically is reduced. Furthermore, when remote monitoring is utilised the geographic areas reached may be larger than when physical contact is required. The elimination or reduction of travel costs can also potentially reduce expenses related to travel and subsistence. In general data becomes available in much shorter time frames. The War Child Canada and the Women's Refugee Commission (2020) has also argued that remote data collection could encourage more participation as it promotes honesty due to anonymity.

The USAID (2020) recommends that where agencies are using remote monitoring, it is helpful to develop a community of practice that can share resources and information. Different agencies can share research instruments so that data collected by one organisation can be useable by another organisation with programming in the same geographical area. Agencies can also share contacts of resource persons who can assist with resource monitoring activities and the supply of required monitoring information. In the case of Ethiopia, various agencies came together to constitute a community of practice. These agencies included DFID (Department for International Development), World Bank, IFPRI (International Food Policy Research Institute), and other NGO (Non-governmental Organisation) partners. As the USAID (2020) indicates, changing approaches to monitoring and evaluation under COVID-19 will of necessity require an adaptation of the monitoring and evaluation plan which drives monitoring and evaluation activities. The use of satellite-based earth observations data on agricultural programmes and flood damage monitoring has been considered and tried with some level of success in different programs (Whitcraft et al., 2015; Liu et al., 2002).

Research questions

The following research questions guided this study:

How has the COVID-19 pandemic changed monitoring and evaluation practice in Zimbabwe?

What mechanisms and tools have organisations and practitioners adopted to respond to the challenges and opportunities of conducting monitoring and evaluation under COVID-19?

To what extent have the new approaches of conducting monitoring and evaluation been successful?

What best practice lessons can be drawn for conducting monitoring and evaluation under the age of COVID-19 going into the future?

Methodology

Survey questionnaire

The study utilised a mixed methods approach for data gathering and analysis. An online survey questionnaire was developed using Kobotoolbox which is one of the freely available online platforms used to host online research projects. The questionnaire collected data on the experiences of monitoring and evaluation practitioners about the effects of COVID-19 on their usual monitoring and evaluation operations. Questions were asked on how the COVID-19 pandemic had changed the monitoring and evaluation function in different organisations. The survey questionnaire also sought to understand what cost dimensions had been brought about by the need to adapt to the challenges brought about by the COVID-19 global pandemic.

A total of 171 respondents were reached by the questionnaire which was deployed online. The questionnaire was deployed by sharing the online link to known monitoring and evaluation practitioners on email, and through social media groups particularly WhatsApp groups where monitoring and evaluation practitioners share information. These groups were further asked to share the survey link with other groups with monitoring and evaluation practitioners within Zimbabwe. Other stakeholders in monitoring and evaluation were welcome to respond to the questionnaire. These included program managers, Agency Directors, Government Heads of Department, and students in the Monitoring and Evaluation degree programs at Lupane State University who were also monitoring and evaluation practitioners. The survey targeted respondents from across all the country's ten provinces. It targeted practitioners from a variety of organisations including government departments, international NGOs, National NGOs and Community Based Organisations. We targeted 200 hundred responds. However, we managed to reach a figure of 171, giving an 86% response rate. Respondents self-selected to respond if they were monitoring and evaluation practitioners. We consider this number significant given the range of organisations represented in the table below. The main weakness of the sample was that we could not establish the number of Monitoring and Evaluation practitioners in Zimbabwe to use as a sampling frame. Secondly, we may have missed other social media groups with monitoring and evaluation professionals that we were not aware of (see Figure 1; Table 1).

Figure 1

Chart showing respondents according to occupation (n = 171). Source: Survey data.

Table 1

Respondents’ classification by type of organisation.

Type of Organisation	Frequency
International Non-governmental Organisation	61
Local Nationwide Non-governmental Organisation	45
Government Department	31
Community Based Non-governmental Organisation	29
United Nations Organisation	5

Key informant interviews

Five key informant interviews were held with monitoring and evaluation managers from selected international non-governmental organisations in Zimbabwe. The interviews were held over skype and on telephone. The main objective of the interviews was to understand how specific monitoring and evaluation systems were evolving under the COVID-19 pandemic. Respondents were asked to explain how they had transformed their monitoring and evaluation systems and what opportunities and challenges they faced in that transformation. We targeted major international organisations that were also funding several local partner NGOs in Zimbabwe because these organisations have a major influence on the development discourse and practice due to the number of partners that they fund. They direct the adoption and use of new methods and techniques in programs.

Online webinar

The online survey was followed up by a Webinar that was hosted by the Centre for Evaluation Science at Lupane State University. Six leading monitoring and evaluation practitioners from international organisations operating in Zimbabwe were tasked to present lessons from their organisations. These organisations included UNICEF (United Nations Children's Fund), International Rescue Committee (IRC), Trocaire and Plan International, Academics, the Ministry of Health and Child Care, Ministry of Social Services and the Zimbabwe Evaluation Association. The purpose of the Webinar was to share practical experiences on how organisations and practitioners were adapting monitoring and evaluation practice given the new constraints imposed by the COVID-19 pandemic induced lockdowns and social distancing. Presenters were asked to present their experiences explaining how the COVID-19 pandemic had affected their regular monitoring and evaluation activities, how they had adapted and with what levels of success, and what best practice lessons they thought should be shared with other organisations to ensure an effective transition to meet the new challenges. Participants in the Webinar were asked to post questions to the presenters and to each other on the chat facility. Data was gathered from presentations and from the chats in the chat facility which were both saved.

Data analysis

Data analysis was conducted separately for qualitative and quantitative data. Quantitative data were downloaded from kobo toolbox into Microsoft Excel in a table format. We used pivot tables to run basic frequencies and crosstabulations of variables of interest to this study. The frequency tables were converted into bar charts and column charts for ease of visual interpretation. Qualitative data was thematically analysed. Key informant interviews and the Webinar discussions were first transcribed. Afterwards the researchers read through the transcripts to identify emerging themes and patterns in the discussions.

Findings

The study established that in all organisations that were surveyed, the conventional practice of monitoring and evaluation was under evolution due to the challenges posed by the COVID-19 pandemic. Respondents in in-depth interviews and at the online webinar indicated that conventional monitoring and evaluation practice had been characterised by site visits for field monitoring and face to face data collection as well as physical data verification, validation and quality assurance. Trainings and program reviews had usually been done in workshop format and Monitoring and Evaluation technical working group meetings being normal for most organisations. Traditional practice has also been largely paper based. However, the COVID-19 pandemic brought about new circumstances and challenges for development practitioners in program implementation, monitoring and evaluation, which have necessitated a departure from doing business in the traditional manner.

Respondents from across the broad spectrum of monitoring and evaluation practitioners indicated that restrictions on movement and social interaction and other associated measures, which had come with government's efforts to curtail the spread of COVID-19, had directly challenged the traditional monitoring and evaluation methodologies. Contact and field travelling had been seriously reduced. Program staff had to work from home in most cases. Timeous reporting had become difficult or impossible using conventional means. Data collection and related evaluation preparatory activities involving direct person-to-person contact were being highly discouraged. As a result, non-critical in-person household visits had been suspended for most organisations. Some monitoring and evaluation exercises had to be cancelled or postponed (see Figure 2).

Figure 2

Respondents perceptions about the impact of COVID-19 (n = 171). Source: Survey Data.

For organisations that have health practitioners who interact with patients, the effects of the COVID-19 pandemic had been particularly observable. The Ministry of Health and Child Care Department of Monitoring and Evaluation indicated that it had been forced in some instances to cancel some support visits to critical areas due to lack of Personal Protective Equipment (PPEs). As a result of these new Monitoring and evaluation requirements including PPEs, monitoring and evaluation budgets had shot up significantly for some organisations as discussed later. Where a single vehicle would have been needed to transport members of staff in the past, and additional vehicle would now be required in order to adhere to the social distancing requirements. As a result of these increased expenses the Ministry of Health and Child Care was looking into viable ways of increasing and enhancing remote monitoring and data collection. As one webinar participant pointed out;

In the Ministry of Health and Child Care, our activities have been severely constrained. A lot of the work that we do requires us to physically visit sites to monitor performance. However, at the present moment, the number of people who can go on field visits is very limited because we cannot travel in large numbers in a car like we used to do. A car going on monitoring visits can only carry half of the number of passengers that it used to carry before COVID-19 (Webinar Participant from the Ministry of Health and Child Care, June 2020).

The fast-paced changes in the field of monitoring and evaluation under COVID-19 have come with cost implications. The survey sought to establish from respondents how monitoring and evaluation costs had evolved under COVID-19 in their organisations. Although most respondents indicated that costs had increased, some respondents indicated no change or a reduction of costs. Those that noted increased costs pointed out that where field visits were necessary, more vehicles were now required to carry the same number of people in order to observe social distancing. Agencies involved with health delivery services noted that their personal interacting with clients needed to have full Personal Protective Equipment (PPEs) which substantially increased the cost of M&E and programs in general. Respondents that noted a decrease in costs attributed that to the fact that monitoring and evaluation activities in their organisations had temporarily stopped completely as they figured out how to proceed. This meant that there were no expenses being incurred towards M&E activities. It may be possible that when this resumed, there would also be an increase in costs. As one respondent noted, costs were reduced because fieldwork had been cancelled and that is where a huge aspect of the M&E budget was directed. The cancellation of monitoring and evaluation workshops was also a major factor in reducing M&E costs. One key informant interviewee indicated that (see Figure 3);

Figure 3

Respondents' perceptions about the cost effects of COVID-19 on M&E activities (n = 171). Source: Survey data.

What we have realized in our organisation is that we need more connectivity going into the future, so what we are doing is to increase our budgets to ensure that we take care of the necessary technology such as tablets and phones some of which will go to field participants who assist with monitoring so that they are able to call, send photos or videos which we can use for remote monitoring. But we also notice that since more people are working remotely from home, we must increase the budget for data (Key informant interview with an International NGO Employee).

Adaptation methods

All organisations surveyed under this study indicated that they were resorting to an increased application of technology to sustain monitoring and evaluation activities without compromising on COVID-19 infection prevention protocols. The levels of application of technology also varied from one organisation to another depending on the type of data being collected, the expertise of staff members in using relevant technology, as well as other local contexts such as connectivity levels between implementing agencies and the program client populations. The technology adopted varied from entry level technology such as SMS messages to clients and stakeholders to telephone interviews, skype calls, WhatsApp voice notes, photos sent on WhatsApp and to zoom meetings. Many organisations indicated that instead of conducting face to face interviews, they were now encouraging that surveys should be based on technologies like telephone interviews, SMSs, voice prompt messages, WhatsApp calls and chats and photos from program sites. Most of these technologies were easy to use because they are largely accessible amongst Zimbabwean populations. Phone interviews were reportedly popular especially in collecting data for high frequency indicators. In other circumstances, organisations with adequate expertise had adopted more advanced remote monitoring tools. Among other tools, UNICEF reported that the organisation had resorted to use of Rapidpro, Call Centre and Google sheets for remote monitoring. The Ministry of Health and Child Care had also adopted an Electronic Monitoring tool for monitoring its programs and the U- reporting to capture client's satisfaction levels. The ministry of health was receiving support from partners in its effort to digitalise the monitoring and evaluation systems. Other organisations like IRC continued to use and adapt applications like Kobo toolbox, ODK and COMMCARE for data collection and GIS for remote sensing (see Figure 4).

Figure 4

Respondents' perceptions about training needs to meet new M&E demands under COVID-19 (n = 171). Source: Survey Data.

Presentations done by academics and the subsequent discussions from stakeholders showed that GIS and Remote sensing was a tool that had been underutilised in monitoring and evaluation to date. The Webinar discussions pointed to the fact that GIS technology, satellite maps, GPS and drone technology, were areas that were yet to be fully explored by organisations. These technologies were particularly useful for remote monitoring. It was reported that the Bulawayo City Council Health Department was already utilising drone technology for crowd monitoring during mealie-meal and bank queues. The use of drone technology ensured that public health monitors did not need to enter into the crowds to collect data for visual impressions. This would help to prevent direct contact between data collectors and the general public. Drones allowed the This encompasses use of drone technologies, satellite maps, GPS and other different types of mapping which can be integrated into other remote data collection tools. All this has potential to yield valuable monitoring and evaluation information with the use of very minimal personnel.

Several other adjustments were being made by monitoring and evaluation departments. Unless absolutely unavoidable, most organisations suspended face to face meetings and resorted to online meetings using forums like ZOOM, Microsoft Teams and Skype for meetings. Secondary data has become an important source to be exploited to the maximum. However, it should be noted that face to face interactions had not been totally abandoned. In some critical instances, site visits were still held, and one on one interviews conducted. In such instances where person to person meetings cannot be avoided, experience from participating organisations shows that the meetings are held under strict observance of COVID-19 guidelines of social distancing. Fewer people are allowed to attend such meetings which are held in the open spaces or spacious and well-ventilated facilities that permit social distancing. Organisations that had to proceed with such meetings also ensured the provision of supplies for hand hygiene for the protection of field staff and program clients. As one Webinar participant noted, not all activities could be delivered virtually,

The challenge is that interacting with clients virtually just is not the same as seeing them face to face. Some of the concepts we need to explain are too difficult to explain online. And then there is the problem that not all of them have access to gadgets that can be used online. Some clients live in rural areas where internet connectivity is next to zero (Webinar Participant, June 2020).

In some instances, it was reported that personnel like the Village Health Workers or Community Care Workers (CCWs) were being enrolled by different organisations to conduct data collection on their behalf. Organisations that could not access project sites sought to establish connections with community embedded professionals who could collect data on their behalf and give feedback. Those organisations that used third party monitoring highlighted the need to conduct trainings amongst third party monitors as they were not familiar with project goals and objectives. Their capacitation has been prioritised as they are an important source of community data in areas which cannot be reached physically. These observations are in tandem with the War Child Canada and the Women's Refugee Commission's (2020) advice that in cases where community focal persons and groups are to be used, it is recommended that a community scan should be done to find out if required persons or groups may already exist which are working with other organisations. This helps to prevent a duplication of groups which could lead to competition. Memoranda of Understanding with such groups or individuals can assist to clarify their roles and responsibilities. It is also recommended that there should be multiple contacts within the site or group to enable triangulation of data. Finally, the provision of training to the focal persons or groups cannot be over-emphasised. They need to have an appreciation of project goals and objectives, data quality requirements, ethical issues in data collection and other necessary project aspects.

Emerging challenges from the new monitoring and evaluation methods

The application of improvised monitoring and evaluation methods has not come without new challenges. It was noted from the Webinar discussions and in-depth interviews that there were significant ethical issues concerning the new remote monitoring processes being implemented. In particular, ethical considerations of confidentiality, consent and doing no harm were challenging to address when using remote data collection. For example, asking third party monitors to send photos as proof that particular activities were being carried out at the sites tended to interfere with issues of confidentiality and anonymity as program clients had to be photographed. This however could be overcome by asking the respondents for consent before being photographed.

The extensive use of technology in the new thrust to monitoring and evaluation has also raised a new type of challenges with regards to participatory monitoring and evaluation principles and the question of inclusivity. The principle of ‘no-one left behind’ as espoused in the Sustainable Development Goals was being undermined by the increasing use of technology to connect with program clients and stakeholders.

Emerging evidence shows that most program clients targeted by development and humanitarian projects tend to be the poorest communities who do not readily have access to modern technologies like smart phones and internet connectivity. This means that the new monitoring and evaluation thrust will increasingly leave behind those communities which cannot afford the technology used in monitoring and evaluation as more monitoring and evaluation activities become remotely controlled. The move to a more technology-based practice of monitoring and evaluation will become exclusionist in approach. This defeats the gains of participatory methodologies that have been promoted in the past. Besides the challenges of access to technology, connectivity was noted by respondents to be another factor that could contribute the exclusionary nature of the new monitoring and evaluation approaches. Some rural areas are poorly connected to the mobile phone grids and thus making it impossible to reach them by internet and phone. In such communities the key interlocutors may not have access to even the most basic of the technologies. Reaching out to them was noted to be a huge challenge. The principle of living no one behind is therefore under threat especially when dealing with marginalised groups like women with low literacy, children, child head households and people living with disabilities and communities living in marginal locations. Some respondents indicated that one challenge they were facing was gender related where women mostly did not have exclusive access to their own phones. These communities and groups would need to be capacitated with the requisite technology and connectivity costs if the monitoring and evaluation practice is to continue being inclusive and participatory. This kind of intervention, however, has the potential of increasing the M&E budgets for organisations. One respondent indicated that in their organisation the monitoring and evaluation budget since the COVID-19 pandemic had evolved to include a significant amount to purchase tablets and smart phones to enable remote monitoring and evaluation.

It is pertinent to note that capacity to adopt new monitoring and evaluation technologies was not only a challenge for program client populations, but also for evaluators. Evaluators may not have been used to a technology intensive practice requiring new instrument designs and new ways of thinking, and technicalities related to remote data collection. This gap was observed by respondents’ in many organisations. This observation meant that training institutions needed to urgently review their curricular to pay more attention to training on remote monitoring and evaluation technologies.

Practitioners also expressed concern on the issue of data quality, validity and integrity. These could be quite easily compromised in the absence of physical data verifications which had become limited or impossible in some settings due to restrictions connected with the COVID-19 pandemic. This had been further compounded by the need to reduce and keep sample sizes and contact points at the barest minimum. It was generally agreed by Webinar participants that in order to avoid complications in data collection, it was important to keep the data collection instruments simple and focused on the minimum required information. Collecting large quantities of information that never got used was seen as a luxury that could no longer be afforded given the spiralling costs of monitoring and evaluation. One key informant interview pointed out that,

The COVID-19 pandemic has taught us a lot of things including the fact that many times we collect data that we do not really need. Under the lockdown, we are using phones, so we have learned to ask only the critical questions and not redundant issues. We are learning to be efficient to save time and resources (Key Informant Interview with International NGO).

Lastly, it was noted that the safety of respondents, especially women, can potentially be compromised through social media and telephone-based monitoring and evaluation. In cases where the safety of the respondents is at stake such as when sensitive information is being collected, phone interviews may not be the best approach as the evaluator cannot see the environment that the respondent is responding from.

Conclusion

In conclusion, with regards to question 1, ‘How has the COVID-19 pandemic changed monitoring and evaluation practice in Zimbabwe?’, the study established that monitoring and evaluation was evolving under COVID-19 to become more virtually based as opposed to physical and face-to-face visits. Concerning question 2, ‘What mechanisms and tools have organisations and practitioners adopted to respond to the challenges and opportunities of conducting monitoring and evaluation under COVID-19?’, the study established that most organisations were utilising online platforms such as zoom, go-to-meeting, WhatsApp and telephone calls amongst other. Beneficiaries were also reporting with photographs and videos remotely. Concerning question 3, ‘To what extent have the new approaches of conducting monitoring and evaluation been successful?’, the study noted that the new approaches had generally been successful in ensuring that projects continued to function. However, some interactions were noted to be problematic when executed on virtual space. Some communities did not have adequate connectivity required for virtual interaction. Lastly, with regards to question 4, ‘What best practice lessons can be drawn for conducting monitoring and evaluation under the age of COVID-19 going into the future?’, it was established that organisations and institutions need to remodel their training and skills development programs to include online and remote skills literacy including GIS, kobo toolbox and other software.

We conclude that the COVID-19 pandemic has significantly disrupted and changed the traditional ways of conducting monitoring and evaluation globally. The need to protect the health of both agency staff, program clients and stakeholders and the global enforcement of lockdowns and social distancing has meant that monitoring and evaluation is evolving away from face to face interactions with respondents. It is projected from the findings of this study that remote monitoring and evaluation techniques will gain more ground in the foreseeable future even if COVID-19 becomes treatable. Agencies need to reskill their staff members in monitoring and evaluation departments to be able to meet the demands of monitoring and evaluation under COVID-19. However, we note that these changes will not affect monitoring and evaluation only. Programming will also need to evolve to meet the same challenges.

Recommendations

M&E systems and processes must be kept simple. We recommend that where available, practitioners should use technologies like phone calls, SMS, WhatsApp and Facebook. The needs of the decision makers must be prioritised to ensure they are getting timely and quality information for evidence based decision making.

Instead of a glut of data that sometimes do not get used, organisations are encouraged to limit their data collection to essential and critical data for project implementation and monitoring.

It is important that organisations continue to adhere to research ethics even when doing remote monitoring. Informed consent should be adhered to, although there is no physical contact with the potential participants. Some organisations reported that they had incorporated ethical requirements in their adapted online tools.

Adapted tools and methodologies should be resilient so they can be able to serve the ever changing context and environment. They must not be imposed, but developed in a participatory manner with the communities and all relevant stake holders.

Gender and protection considerations need prioritised considering the precarious conditions that women and children in our communities find themselves. The gender dimensions of access and ownership must also be considered in the evolution process.

Enhanced accountability to the communities can be facilitated through establishment or strengthening available Feedback and Reporting Mechanisms to ensure that they are responsive.

Declarations

Author contribution statement

Thulani Dube: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Siphilisiwe B. Ncube, Simon Mlotshwa, Getrude N. Matsika, Nelly Maonde: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

The authors do not have permission to share data.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.