Responsive neglected tropical disease programme delivery: listening and responding to the views and perceptions of beneficiaries.

In neglected tropical disease (NTD) programmes, beneficiary feedback mechanisms (BFMs) can be utilised to enhance programme quality and strengthen health outcomes by promoting the participation and empowerment of local stakeholders. This article reflects on the experiences of the Ascend programme in embedding a BFM in the Democratic Republic of Congo and Nigeria where key NTD data collection tools were adapted and prioritised across various elements of the NTD programme. Findings suggest that listening and responding to the needs of beneficiaries and building upon existing systems within NTD programmes is highly valuable in informing the planning and delivery of NTD activities.

Introduction

As the drive towards the sustainable control and elimination of neglected tropical diseases (NTDs) begins to show positive results, guided by the World Health Organization (WHO) roadmap[1] and the continuous support from local and international funders, pharmaceutical companies, academia and government stakeholders across the globe, the big question remains: To what extent are NTD programmes listening to communities and learning from local perspectives to improve programme quality delivery as a pathway to promoting community ownership and programme accountability?

Beneficiary feedback mechanisms (BFMs) are tools used to enhance programme quality and to strengthen health outcomes by collecting feedback at regular intervals[2] and can be an effective tool to use within large-scale health programmes to help promote the participation and empowerment of local stakeholders.[3] It is a cross-cutting theme within the WHO's roadmap 2030[1] agenda through strengthening efforts of accountability and equity—and can further drive progress towards the Sustainable Development Goal targets.

A systemised approach to beneficiary feedback can further inform the planning and delivery of NTD services as it enables programmes to listen, adapt and improve in response to feedback provided, thereby ensuring that activities are responding to and addressing the needs and requirements of beneficiaries. There is recognition from donors such as the Foreign, Commonwealth and Development Office (FCDO) that beneficiary feedback can foster more efficient outcomes for programmes and enhance the transparency and safety of programmes for beneficiaries.[4] The integration of feedback into existing national systems can promote person-centred healthcare service delivery[5] thereby elevating health systems’ strengthening and ownership. Additionally, soliciting feedback from frontline health staff during planning and implementation stages can reinforce health outcomes[2] and creating channels between local community members and community health staff can help reflect on operations within project evaluations.[6] In summary, effective BFMs ensure upstream and downstream accountability whereby programmes are accountable to both providers and receivers of support. Valuing the voices of those receiving support deserves a greater emphasis within programmes[7] as it can strengthen adaptive programming and improve the quality of services.[8]

As such, the Ascend West and Central Africa programme (hereafter called ‘Ascend’, or ‘the programme’[9]) embedded a systematised BFM from a pilot study in the Democratic Republic of Congo (DRC) and Nigeria to listen and respond to beneficiaries in the delivery of NTD services.

This commentary will reflect on the experiences of Ascend's efforts specifically in soliciting feedback from intermediary beneficiaries of NTD services (target recipients of capacity building) such as NTD coordinators, health workers, nurses, local government workers and community drug distributors (CDDs). It will also drive forward discussion on the need to embed beneficiary feedback into existing systems and processes to strengthen accountability and equity across the NTD landscape.

Mainstreaming the BFM within Ascend

An in-depth mapping of feedback points, tools and processes was undertaken to review key aspects of the programme's approach to beneficiary feedback. This was done to identify further opportunities for integrating feedback loops into existing processes and to build on existing strengths. Ultimately, the intention is for beneficiary feedback loops to be embedded in national health system processes to strengthen the national system, encourage local ownership, increase sustainability, increase value for money and eventually strengthen upstream and downstream accountability. The mapping was conducted through the review of relevant programme documents including programme/national monitoring and evaluation (M&E) tools and processes, key informant interviews and meetings with stakeholders such as Ministries of Health (MoH), non-governmental development organisation partners and other Ascend stakeholders in both Nigeria and DRC.

The mapping exercise identified several points of interaction with various programme participants in Ascend and therefore identified opportunities to adapt existing tools and processes to gather feedback on satisfaction with the services provided. This led to the classification of beneficiaries within the NTD programme into two main categories: end beneficiaries (individuals receiving treatment and surgery) and intermediary beneficiaries (individuals providing treatment and surgeries, such as CDDs, health workers and supervisors).

As part of this process, a suite of beneficiary feedback tools/questions was developed, and existing NTD tools in which to embed these were prioritised. A set of beneficiary feedback principles and considerations was developed as a guide to embedding BFMs across various elements of NTD programmes.

Some of these tools developed focused on end beneficiary feedback and included the provision of interviews following mass drug administration (MDA) implementation for recipients of MDA through various mobile data applications (Commcare, Dimagi, Inc.; Microsoft Form, Microsoft; and Open Data Kit [ODK], Get ODK Inc.), including open-ended questions on satisfaction, dissatisfaction and areas for MDA improvement to recipients of MDA through independent Coverage Evaluation Surveys (CES), interviewing hydrocele patients through a hydrocele tracker mobile data application and interviewing staff involved in hydrocele surgery.

The tool most developed and scaled up, however, which yielded the most valuable data, was the integration of soliciting feedback during routine MDA training from provincial/state NTD coordinators, nurses, health workers, local government workers, CDDs and case finders. Table 1 summarises these key activities.

Table 1.

Summary of training feedback tools and response in Nigeria and DRC

BFM touch point	Target beneficiary	Mode of data collection	Response and scale
Training feedback from participants at province and district level in DRC.	Provincial NTD coordinators, nurses, health workers.	Self-administered paper-based survey (transferred into ODK).[10]	301 responses across 11 provinces.
Training feedback from participants at state level in Nigeria.	State and local government NTD coordinators, nurses, health workers.	Self-administered survey hosted on Microsoft Forms.[11]	843 responses across 7 states.
Training feedback from participants at local government area (LGA) level in Nigeria.	CDDs.	Survey hosted on Commcare app, administered by independent supervisors.[12]	2128 responses across 6 states.

Feedback from training comprised a mixture of close questions on a Likert scale centred on satisfaction with various aspects of the programme such as training framework, topics covered, the instructors’ level of content knowledge and the quality of teaching. It also included open questions to gather suggestions for improvement. The interviews were done with intermediary beneficiary participants present at the organised training workshops as well as during MDA supervision. The choice of digital platform and the degree of utilisation was based on existing monitoring systems in both countries. It was apparent that if data collection can be integrated into existing activities that are already funded and compatible, then this is more cost-effective than more vertical/independent data collection approaches.

Main lessons learnt

While there is no singular best way to strengthen beneficiary feedback within NTD programmes, this is one case study where the programme tried to mainstream beneficiary feedback within NTD programme implementation. The pilot provided useful insights into the opinions of beneficiaries within the programme and the future use and refinement of beneficiary feedback tools on NTD programmes. For example, in Nigeria, 97% of CDDs providing feedback said the training had equipped them to do their job, citing various reasons for satisfaction, including useful training content, having practical demonstrations and being grateful for the hands-on experience in using the reporting tool.

While the participants at state level training expressed the introduction of electronic methods for pre- and post-training evaluation as a key element of satisfaction, they also suggested that the programme should provide mobile phone access for all participants. Furthermore, the LGA coordinators requested to be allowed to develop their own plans in advance of meetings rather than in groups at the meeting.

The main theme for improvement that emerged from intermediary beneficiaries across both countries was the request to increase the duration of training, to advance the notification of training schedules and for the programme to provide hard copies of training materials for reference.

Reflections to take forward

The digitisation of the tools made it possible to quickly identify specific districts where specific elements of the training (such as the venue or refreshments) were poor, and adequate steps could be taken to address the feedback and close the loop. For example, a suggestion to improve refreshments emanated from 3 out of >100 districts where training feedback was received in Nigeria—so the programme was able to isolate these locations to take actions for improvement. Despite the benefits of digitisation of the feedback tools, there was also a constraint with limited access to mobile phones, variable internet connectivity and low literacy, especially among the CDDs.

Many people that did not respond to qualitative questions did respond to quantitative questions. Open-ended questions can generate insightful feedback leading to programme adaptation, but the amount of time and skill required to collect and analyse data makes it expensive and time-consuming. While close-ended questions were easier for the programmes to collect and analyse, they may not generate an adequate representative sample to quantify the survey. This remains a valid problem, especially as programmes are inclined to balance time and resources with benefit.

The main limitation of the pilot study was the inability to generate adequate and representative feedback from end beneficiaries. However, the approach adopted from the outset was geared towards sustainability and ownership by the MoH and partners in both countries as a key focus. In the future, it will be useful to conduct a quality survey to determine what satisfaction criteria are important to consider and also to test other ways of collecting data from beneficiaries, especially end beneficiaries, through approaches such as adapting CES with a representative sample to collect feedback, utilising telephone lines and operating suggestions boxes where possible. Sightsavers have since piloted a BFM by encouraging MoH and partners in Cameroon and DRC to embed beneficiary feedback questions into CES to obtain independent feedback from a large number of end beneficiaries. This will continue to be prioritised as a tool.

In DRC and Nigeria, some of the feedback provided was beyond the scope of Ascend objectives. This is to be expected and implies that there is a need for continuous sensitisation of beneficiaries to better understand the feedback process and to ensure that the feedback provided is relevant to the programme in question.

Recommendations

Based on the experience from the Ascend programme, several key principles (as shown in Table 2) emerged that helped to guide the strengthening of beneficiary feedback loops from design to implementation. Depending on the context of implementation and the nature of future NTD programmes, adaptations could be made.

Table 2.

Recommendations to strengthen mechanisms for collecting beneficiary feedback in neglected tropical disease programmes

Theme	Recommendation
Sustainability	Consider the importance of integrating within existing national systems, working closely with MoH to create ownership and foster a culture of feedback, reflection and learning at all levels for a strengthened health system.
Integration	Optimise opportunities within current feedback points, for instance, adding in a small number of specific questions for beneficiary feedback on services within existing M&E tools such as surveys, supervision checklists and training evaluation forms. These are all cost-effective and potential pathways to sustainability.
Ethics	Ensure compliance with responsible data guidance/legislation and any ethics approval requirements in programming. It is vital that consent is gained before any feedback is sought, as well as being clear about how feedback loop will be used, reported and stored.
Leave no one behind	Consider how to engage with hard-to-reach groups in each country's context (such as prisoners, refugees, pastoralists, persons with disabilities, etc.). Multiple feedback channels should be created to ensure wider access.
Digital/mobile technology	Consider employing the use of digital/mobile technology. This presents a huge opportunity but there is a need to assess context, access and feasibility of integration. Paper-based systems of collecting feedback are considered to be one of the barriers to closing the feedback loop, especially when excess feedback is obtained in paper form. Increased digitisation of feedback, where possible, will enhance the efficiency of the analyses of feedback gathered and encourage real-time adaptation and uptake of the feedback.
Co-design	Beneficiaries should be engaged in the design phase of feedback mechanisms and programmes. Research indicates the importance of beneficiaries co-designing any feedback channel and having input into their preferred way of providing feedback. This is illustrated in the Participatory Guide for Planning Equitable Mass Administration of Medicines (PGP) to tackle NTDs developed by the COUNTDOWN Project.[a]
Objectivity/bias	Consideration of who is capturing the feedback, and how a relationship with varying power dynamics potentially influences information provided, are critical to reduce the impact of biases. Opportunities for independent collections and analysis of feedback should be prioritised.
Contextualisation	There is no one size fits all, and as such tools should be locally contextualised. This includes consideration of appropriate language, literacy level, avoiding unnecessary jargon, simplifying and/or explaining terms wherever possible and translation into local languages (and pre-testing of that translation) to aid comprehension and help ensure inclusion.
Awareness creation	Continuously sensitising communities on the importance of feedback loops will help to further streamline the quality and relevance of the responses received. This is important to ensure confidence in the feedback mechanism and that it will be used to improve the quality-of-service delivery. A culture of feedback can be fostered and strengthened when robust mechanisms are in place and they are fully embedded into NTD programming and service provision. Raising stakeholder/community awareness on the availability of feedback channels and examples of previous feedback that have led to adaptive programming will help in reinforcing the value of feedback, further stimulating its uptake.
Closing the feedback loop	The full closure of the feedback loop requires a system for analysing, reviewing and adapting to feedback, then communicating a response back to the beneficiaries. One of the barriers to closing the feedback loop noted in the deep-dive analysis of existing beneficiary feedback approaches in Ascend was the extensive use of hard copy forms that are challenging to collectively analyse and utilise. This should be a key consideration for any additional or adapted feedback channels.

COUNTDOWN. Participatory Guide for Planning Equitable Mass Administration of Medicines (PGP). 2021. https://countdown.lstmed.ac.uk/sites/default/files/centre/Countdown%20PGP_0.pdf [accessed October 28, 2021].

Overall, equity and accountability within NTD programmes (which are cross-cutting themes within the WHO's roadmap 2030) can be partially addressed through systemising approaches to beneficiary feedback. Certainly, finding new ways of listening and responding to the needs of beneficiaries and building upon existing systems within NTD programmes is paramount to informing the planning and delivery of NTD activities in moving towards elimination agendas.