From effectiveness to sustainability: understanding the impact of CARE's Community Score Card© social accountability approach in Ntcheu, Malawi.

We evaluated the sustainability of CARE's Community Score Card© (CSC) social accountability approach in Ntcheu, Malawi, approximately 2.5 years after the end of formal intervention activities. Using a cross-sectional, exploratory design, we conducted 41 focus groups with members of Community Health Advisory Groups (CHAGs) and youth groups and 19 semi-structured interviews with local and district government officials, project staff, and national stakeholders to understand how and in what form CSC activities are continuing. Focus groups and interviews were audio-recorded, transcribed and translated into English. Thematic coding was done using Dedoose software. Most groups were continuing to meet and implement the CSC, although some made modifications. CHAGs, youth and local government officials all attributed their continued implementation to the value that they saw in the process that allows marginalized groups within the community, including women and youth, a safe space for sharing their ideas and issues and the initial results this generated. However, lack of access to resources for implementation and challenges in convening and facilitating the interface meeting phase created barriers to continued sustainability. The CSC is sustainable by communities 2.5 years after the end of formal intervention activities. For future interventions, health systems and non-governmental organizations should plan for a transition phase with periodic refresher trainings and a small fund to support implementation, such as refreshments and transportation, to increase the likelihood of community-driven sustainability.

The Community Score Card© (CSC) was sustained by communities over 2 years after the end of the formal intervention period of the Maternal Health Alliance (MHAP) project.

Experiencing successes with the CSC during the formal intervention period motivated community and youth groups to continue using the process on their own.

Community and youth groups and government stakeholders appreciated the transparency and accountability created by the CSC process as well as the safe spaces for dialogue.

Sustainability of a social accountability approach, like the CSC, is possible, but planning for a transition phase, including refresher trainings and funds to support implementation, between the formal intervention period and community-driven sustainability is critical.

Introduction

Social accountability seeks to engage citizens, especially those from marginalized or vulnerable groups within the community, in a process of participatory governance to enhance the quality and equity of government services. Evidence supporting the effectiveness of social accountability approaches has been growing over the past decade (Björkman and Svensson, 2009; Bjorkman Nyqvist , Gonçalves, 2014; Touchton and Wampler, 2014). While the theory underlying social accountability suggests that these approaches should inherently be sustainable, exploring this question of sustainability in applied settings is pertinent in order to fully understand the power of social accountability to promote participation by citizens and accountability among local government officials, especially in the health sector.

A systematic review of sustainability among health programs in sub-Saharan Africa found that the focus is often on what was sustained; very few studies address how or among whom programs are sustained (Iwelunmor ). Another review, of the effects of community participation on maternal and newborn health outcomes, emphasizes the need for qualitative research to understand why programs are successful or not (Marston ). Here, we help to fill both these gaps by utilizing qualitative research to understand the process of sustainability—specifically how and among whom activities were sustained—for a large social accountability intervention in Malawi.

Malawi is a small, landlocked country in southeast Africa, with a population of nearly 16 million people. Women comprise 51% of the population, of which 45% are of reproductive age (15–49) (Malawi, National Statistical Office, Macro International, Institute for Resource Development, 2011). Despite progress, high rates of maternal and neonatal mortality remain a challenge, recent estimates of which are 439 per 100 000 live births and 23 per 1000 live births, respectively (National Statistics Office and ICF, 2017). The majority of Malawians (85%) live in rural areas, such as Ntcheu district, in the Central Region where this intervention took place. The District has a population of 474 464 (Malawi, National Statistical Office, Macro International, Institute for Resource Development, 2011).

The Community Score Card© (CSC), originally developed by CARE International in Malawi (hereafter ‘CARE Malawi’) in 2002, is a citizen-driven, social accountability approach that supports the assessment, monitoring and evaluation of the delivery of local government services, especially in the health sector (CARE Malawi, 2013). The CSC brings together community members, service providers and local government officials for a facilitated process of issue identification, prioritization and action planning. Importantly, it gathers feedback from both service users and providers to improve the communication between the two (CARE Malawi, 2013). The CSC’s emphasis on bringing together these groups sets it apart from other social accountability approaches that are more focused solely on the community side of the equation (Fox, 2015; Hernández ).

CARE Malawi implemented the CSC in Ntcheu as part of the Maternal Health Alliance Project (MHAP), an effort to help improve maternal and infant health outcomes in the area from 2012 to 2015. A cluster randomized trial of MHAP showed positive effects on community empowerment, service utilization, satisfaction with services and improved service delivery and coverage among frontline health workers. Women in the treatment areas reported an increase in Community Health Worker (CHW) home visits during pregnancy that was 20% greater and 6% greater after pregnancy than that of the control communities, a 57% higher current use of modern contraceptives and an increase in service satisfaction 16% greater (Gullo ). Women’s active participation in the CSC intervention was positively associated with governance measures such as joint monitoring and transparency, collective action and availability of community help (Gullo ). Health workers also reported positive outcomes as a result of the CSC, such as increased coverage and provision of comprehensive antenatal care (ANC) counselling as well as better record-keeping (Gullo ). Furthermore, there were improvements in important qualitative indicators that were locally identified by CSC participants. Both participating community members and health workers reported improvement in the relationship between health workers and communities, the reception of clients at health facilities, the commitment of health workers, the level of male involvement in maternal and newborn health and family planning and the accessibility of information (Gullo ).

While the CSC is designed with sustainability in mind by engaging local and district government officials from the beginning and working to embed the process in the district-level implementation plans that occur regularly throughout Malawi, CARE Malawi sought to assess more intentionally the tool’s sustainability after the implementing organization, in this case, CARE Malawi itself, stepped back from providing direct implementation support. We were interested in understanding why, how, where and in what form the CSC processes had continued; what factors facilitate or impede the sustainability of the CSC and what outcomes have been achieved or sustained. We aimed to generate evidence about the process of sustainability that health systems, non-governmental organizations and others can use to more effectively prime future social accountability interventions for sustainability. Therefore, this study aimed to explore and understand the process of uptake, implementation and continuation of the CSC by various partners in Ntcheu district, Malawi, in order to evaluate the sustainability of the CSC’s approach.

Methods

The maternal health alliance project and the CSC intervention

MHAP employed the CSC, a social accountability approach that includes five phases, as the intervention strategy. Phase 1 of the CSC involves planning and preparation and entails identifying issues, as well as securing cooperation and buy-in, from all participating groups, including community and government stakeholders; understanding the context and barriers encountered by service providers and users and determining the geographic scope of the initiative. During Phases 2 and 3, the CSC is facilitated separately with community members and service providers. These groups meet separately to identify and prioritize issues faced within their communities and workplaces, respectively. The identified issues are organized into indicators, which are then scored separately by the same community members and service providers. Phase 4 consists of the ‘interface’ meeting where community members and service providers, along with local government officials, come together to share their identified and prioritized issues and how they scored these issues to generate solutions and to develop joint action plans to implement these solutions. Phase 5 involves the implementation of the joint action plan, as well as monitoring and evaluating progress on the indicators. A complete cycle of Phases 1–5 is considered a ‘round’ of the CSC. After the first round, the CSC process, from Phase 2 to Phase 5, is repeated every 6 months (see Figure 1). Further details of the intervention are provided elsewhere (CARE Malawi, 2013).

Figure 1.

CARE’s CSC process

MHAP was implemented from 2012 to 2015 in partnership with the Government of Malawi’s Ministry of Health. A cluster-randomized control trial of the CSC was conducted as part of the project. During MHAP, CARE Malawi trained existing community health action groups (CHAGs) within the 10 health facility catchment areas in the intervention arm to co-lead the CSC process. In addition, over the course of the project, youth groups in Ntcheu began independently implementing the CSC in their communities, as did members of the local government District Health Management Team in order to integrate greater accountability measures into their family planning and maternal and child health programs. CHAGs comprised most of the intervention groups because those were targeted during the implementation of MHAP as opposed to the youth groups that arose organically. CHAGs and youth groups ranged from approximately 15 to 20 members each. During MHAP, CARE resourced the implementation of the CSC process, e.g. travel costs, facility rentals and supplies like stationary, but did not fund action items. Instead, CARE helped facilitate discussions around who could help fund and execute the prioritized actions. CARE supported up to six rounds of the CSC per intervention catchment area during MHAP. While the five-phased CSC process was implemented identically regardless of the group leading the approach, the action plans varied by group depending on prioritized issues and identified action items.

Design

This study utilized an exploratory, cross-sectional design comparing the sustainability of various partner-led approaches (i.e. local government, youth and CHAGs) of CSC implementation. We used qualitative methods—focus group discussions, semi-structured interviews and key informant interviews—to collect data, which allowed for in-depth exploration and better understanding of the continued uptake and implementation of the CSC by the various partners.

Sampling and data collection

Using purposive sampling across 10 health centre catchment areas, we identified a mix of CHAGs, youth groups and frontline health workers still actively implementing the CSC, those using a modified version and those no longer using it. We conducted 33 focus groups with CHAGs (n = 14 active groups, 10 mixed/partially active groups and 9 passive/non-active groups) and 8 focus groups with youth, representing over 95% of all the groups known to have been implementing the CSC at the conclusion of MHAP. Focus group discussions ranged in size from 8 to 18 participants. In addition, we conducted 13 semi-structured interviews with district and local officials and 6 key informant interviews with staff and national stakeholders for a total of 467 participants (n = 191 male, n = 276 female). Prior to the start of data collection, facilitators, interviewers and note-takers contracted from a local research consulting firm participated in a week-long training focused on both the CSC process and the principles of qualitative data collection. Focus group discussions took anywhere from 60 to 90 minutes; the interviews required between 45 and 60 minutes. Verbal informed consent was obtained prior to the start of the focus groups and interviews. The focus groups were conducted in the local language, Chichewa, transcribed and then translated into English in preparation for analysis. Interviews were conducted in either English or Chichewa depending on the preference of the interviewee. This study was reviewed and approved by the institutional review board of Malawi’s National Commission for Science and Technology.

Data analysis

Focus groups and interviews were audio-recorded, transcribed in Chichewa and translated into English. An initial codebook with a priori codes was developed in alignment with the research questions and then augmented and refined with emergent codes during the analysis. The qualitative data were coded and analysed using the Dedoose software. TS and ASK led the coding and checked for consistency across coders and cases. Thematic analysis was used to identify major themes arising from the data that aligned with predetermined research questions, and supporting quotes were selected to help illustrate those themes (Babbie, 2015). Analysis summaries by theme were drafted and shared among the authors, discussed, discrepancies resolved and then themes finalized. The final analytic themes were presented and discussed during a dissemination workshop in Lilongwe, Malawi, in December 2019 that included participants in the CHAG and youth focus groups, local leaders from Ntcheu and national stakeholders. This dissemination meeting provided an opportunity for the triangulation and validation of the findings. These final analytic themes are presented here.

Results

All key stakeholder groups could describe the general process of the CSC more than 2 years after the end of the formal intervention, which shows a level of knowledge retention, even if the terminology they used and their distinction between the various phases were not as precise as the model CARE originally developed. Overall, most groups, regardless of partner type, were still actively implementing the CSC process in some form. The main modifications from the original MHAP-supported implementation of the process include some groups (1) adapting the interface phase to meet with stakeholders individually or focus on internal accountability and prioritization and (2) adjusting the timeframe such that the rounds of the CSC extend longer than 6 months.

CHAGs

The CHAGs, even those who were no longer actively implementing the CSC, perceived the process as developing a sense of ownership and responsibility within communities and of transparency and accountability with local health service providers and government officials. The CSC gave CHAGs a sense of empowerment to address the issues of concern in their communities and of control over community assets and resources to do so. In exploring the motivation for the CHAGs’ sustained implementation of the CSC, it primarily came back to participants’ perceived value of the process in terms of creating a sense of ownership and a sense of achievement in affecting change in their communities.

In addition ( …) scorecard (clears the throat) creates a sense of ownership (…) and responsibility (…) over what you have or over what someone has. That is why now we rarely have cases of maternal mortality; it is because people have a sense of ownership and responsibility (…) to follow up on and address issues that affect them…That is what I can say in short [Nzama (study location)]

…it is a way to ensure that no one is being denied of their rights [Gowa (study location)]

CHAGs viewed the CSC as being able to build this sense of empowerment and ownership through the spaces it created for all voices within the community to be represented, freely expressed and heard. CHAGs highlighted how breaking into specific subgroups, such as men and women, during the second phase of the CSC process helps create this space where all voices could be shared. Providing all community members a safe space in which to generate and share their concerns first among similar constituents before sharing collectively drives CHAGs to continue using the CSC. The process has enabled them to prioritize a broad range of issues since they have a chance to surface first in the smaller groups, which are seen as safer spaces for discussion.

So, that [CSC] process helped people to have a better understanding on the issues discussed. Because when it’s just one person relaying the messages to a group of people, some accept the messages but others don’t; but because the approach was participatory where everyone got to speak their mind, it helped people to change. (Nzama)

The reason we do this is that there are some other issues that maybe sensitive … So, if you mix the youths with the elderly they may not be able to speak freely. For example, a woman will be like, should I speak this in front of Jane? No, it’s for elderly people. So, they fail to what? Express themselves freely… So, the reason we put them in different groups is that, so that people are free to say whatever they think (Kapeni)

CHAGs that continued with the CSC process reported that it helped them establish collaboration between service providers, government workers and community members. In addition, it provided a framework for generating solutions to problems. The process helped to uncover and prioritize problems within communities as well as identify those individuals and groups that these problems affected. Noticing these changes in relationships between community members and service providers brought on by the CSC was an often-mentioned driver encouraging CHAGs to continue using the process.

We help each other on what to do. We build a relationship with the hospital staff so that we should approach them when we notice their flaws to help each other. That is what happens (Nsiyaludzu_Tcheza).

… the other thing that pleases us is knowing that we have the ability to request the health center personnel and the community to attend our meetings and actually have them come, it gives us a sense of pride in our work. (Gowa).

Members of the CHAGs perceived significant changes in their communities—both health and non-health-related—as a result of implementing the CSC process. Some of the health-related changes included a perceived decrease in maternal deaths and an increase in institutional deliveries, an increase in use of family planning and, importantly, an increase in men’s supportive involvement in their partners’ reproductive health, including accompanying their wives to antenatal care visits, delivery and family planning consultations. In addition, CHAG members perceived a decrease in early marriages and an increase in girls staying in school. Seeing outcomes that CHAG members attribute to the work of the CSC process motivates continued use of the process.

When we started working with [local government], we no longer have maternal deaths in our community. If we say maternal deaths have dropped it will mean they still take place but since we started learning this, we have not experienced any maternal death. We have seen that [CSC process] has really benefited us.

… Pregnant women go to the hospital few days before due date of delivery the day the woman realize that she is pregnant she goes to the hospital to know how the baby is developing and are able to follow good safe motherhood [sic], while in the past women were not aware of these things… (Kapeni)

This made us involve the youth so that they could be helped and we could help them on how the can easily and freely access services, so that they can protect themselves; to help them have good health and a better future. Because of this, the youth have been greatly helped … (Nzama).

Youth

Similar to the CHAGs, participants in youth groups reported a mix of continued activity level around the CSC. While fewer youth groups were engaged in the CSC process during MHAP as compared to CHAGs, nearly all the youth groups are still actively using the CSC process, with some adaptation, more than 2 years later. For example, some youth groups have found it challenging to host and facilitate the interface meetings with a variety of stakeholder groups and so adapted the process to focus on identifying and prioritizing issues of importance to them and generating action plans for themselves to address these issues as best as they can.

Youth group members perceived that the CSC process has allowed them to gain a more active role in their communities and with local development projects. Furthermore, it has helped them gain respect within their communities and be recognized as an important stakeholder group. Experiencing this change in their role and a growing voice in their communities is a key driver in the youth sustaining the CSC. The structure of the CSC process in which groups are broken out into smaller, more homogenous, subgroups for initial brainstorming and issue generation has provided the youth a safe space to generate and share ideas. It has enhanced their ability to speak freely and have their voices heard. In their eyes, the CSC has improved relationships among various constituencies in the local community, especially among health providers and village leaders and has provided a mechanism for accountability. Achieving this ability to speak freely and have their voices heard within their communities motivates the youth to continue participating in the process, something that those who participated in the dissemination meeting in Lilongwe emphasized to the national stakeholders attending.

…this scorecard it helps us to be transparent and open to one another on a group when discussing health issues and it has helped to find the problems which without scorecard it couldn’t have been possible [sic] (Nsiyaludzu)

My views are that the issue of score card is very good because we don’t blame anyone, or point fingers at someone and blame them for not doing a good job or causing bad results, no we just work on abolishing the problem until things get better without fighting. (Yesaya-Kasinje)

When their sector receives poor marks during the scoring, the responsible duty bearer begins to suspect foul play about the research. In scorecard, we do not mention names, rather we identify them by sectors. … It is just fortunate that the voting and scoring happen in open (Kasinje)

scorecard is an accountability tool used to monitor developmental projects in the community … it is an approach that brings together donors and beneficiaries (citizens and duty bearers). It fosters promotion of transparency and accountability between the two. It is a good approach and is applicable everywhere (Kasinje)

Like the CHAGs, the youth group members perceived reproductive health-related benefits as well as other benefits arising from the CSC process, which motivated them to continue. The youth viewed the CSC as helping increase the use of family planning and decrease teenage pregnancies, as well as decrease early marriages and increase the proportion of girls staying in school. Furthermore, the youth felt that the CSC has helped them uncover corruption and address environmental concerns in their communities, such as deforestation.

We enlighten [our group members] of their health and the dangers of sexually transmitted diseases and unwanted pregnancies. Just to add on that, we also had condoms for those people who could not manage to abstain so that they should at least use protection. (Nsiyaludzu)

In addition, unique to the youth groups, they saw a benefit of the CSC in its ability to create greater self-reliance among themselves through the emphasis of the process on skills, education and advocacy. They view the CSC process as having helped them achieve greater independence along with a greater voice in community affairs. Overall, they perceive the CSC helping empower and enlighten them which has motivated continued use of the process.

The objective of this group is to teach young people how they can protect themselves from risk factors, but also to enable them to be self-reliant. (Chifwiri-Kasinje)

The objective of this youth club is empower young people to take part in the community development activities. At the same time, empowering the youth to become self-reliant. (Manjanja-Kasinje)

Local government officials

Local government officials, including frontline health workers, viewed the CSC as an opportunity for CHAGs and the youth to take more responsibility within the community. They see these groups as being the main ones responsible for implementing the CSC, whereas they perceive the local government as having a supporting role, especially during the interface meeting and action phases. However, local government officials lamented that their staff are not always available to participate in these stages, a frustration similarly expressed by the CHAGs and the youth and cited as a challenge to sustainability.

As with CHAGs and youth groups, local government officials perceived the CSC as a process through which to amplify the voices of a wider range of stakeholders within the community. Local government viewed community members’ continued, elevated participation, engagement, and involvement in local decision making processes as a result of the CSC which motivated them to continue supporting the process. Officials viewed this increased involvement as critical for more projects to succeed at the local level.

Aah, what has worked well in CHAG ( …) is the issue of involvement. These people are involved and they are able to know what they are doing, and they are able to generate issues surrounding problems affecting the community. We can also say the CHAGs act as a bridge (women voice in the nearby distance) they work as a bridge so they are mostly in touch with the health facility … (Bwanje)

There has also been involved relationship between the community and the health workers [sic]. At first there was no effective communication between the health workers and the community; community members could throw stone and hurl insults as they passed by the facility premises ( …) the doctors could even be physically harassed right at the facility. But these days you can’t hear such things happening. There has been a significant change compared to what used to happen … (Bwanje)

Local government officials noted some of the same health-related benefits resulting from the CSC process as other stakeholders. In particular, they highlighted the increase in men’s involvement in partners’ reproductive health, men accompanying their wives to antenatal care and deliveries, attending family planning counselling and being supportive of women’s nutritional needs and rest during pregnancy. They also perceived a decrease in maternal deaths, as the CHAGs did. Finally, local government officials noted improvements in the transparency and accountability of local development projects. For example, communities have worked together with local government officials to build infrastructure, such as latrines and classrooms. These perceptions of success and effectiveness serve as drivers to motivate continued support of and involvement in the CSC.

… Right now we have service level agreement which came about after the complaints, we are very grateful for that… (Chigodi)

At first, we only had four village clinics but as of now we have ten. Showing that we are making progress and we have six additional village clinics which are helping the under-five in the communities. This addressed the long-distance issue which is also a benefit. The cases of women giving birth at home or on their way to the hospital are very rare. (Nsiyaludzu)

… now [Members of Parliament] are more open in the way they manage the [community development fund]. When they start projects, they make sure that they complete them because now community members are able to track how the funds flow. We are the ones who are always close to the community members. (Community Development Assistant)

Facilitators of sustained implementation

Participants from across the various stakeholders identified several common themes that helped motivate a desire to sustain the implementation of the CSC process after CARE stepped back from direct facilitation. CHAGs, youth groups and local government officials all noted that the process provided a safe space for groups to share their ideas and express their concerns, a space that was critical but had not been available before. The youth, in particular, felt that the CSC process had helped them gain an important and active role in community development and had earned them respect among other stakeholders. This opportunity to be engaged motivates CHAGs and youth groups to continue using the CSC process. Even representatives from other non-governmental organizations not engaged in the youth-led process noted how the CSC increased the involvement of the youth and allowed other important constituent groups within the community to contribute.

thus through scorecard, when youths are found where there are elders they would fail to speak what is troubling them [sic ]

…to open up, but when women are on their own men on their own youths on their own they are able to open up in their groups and contribute (Champiti).

CHAGs and youth expressed that the successes they saw using the CSC process in making changes helped motivate them to continue the process. Youth felt that the process helped them identify problems, negotiate differences between constituents and track progress. CHAGs emphasized the importance of the training that CARE provided in the process to their ability to sustain it.

… because the Score Card also facilitates transparency, and then we see how they trained us we saw that we should also train others, and why we have participated even though CARE stopped, there is no other support but we can help our community change. (Kasinje)

Those groups that did continue with the CSC process credited the CSC’s structure along with the transparency and accountability it creates for that sustainability. Local government officials expressed that the interface meetings (Phase 4 of the CSC process) were particularly important for developing that accountability, yet these can also be a challenge to implement. All stakeholder groups also identified engagement across community members and local officials as essential.

… the thing is that when the communities sit down and then they talk, they come up with the solutions that would work, not things that are just made up to say just to end the story then a let’s just say this, but because it becomes like a routine to them to say we meet and we talk and we come up with solutions, so because they know next time we are going to meet and then we are going to deliberate on the same issues so if we are going to promise things that we are not going to do then it won’t work, so they just promise those things that they know will happen. (Assistant Environmental Health Officer)

Challenges to sustained implementation

CHAGs, youth groups and local government officials all noted similar, important challenges to sustaining the implementation of the CSC process after the NGO steps back from facilitating direct implementation. These challenges included difficulty incentivizing continued participation by group members, lacking resources for basic supplies to implement the process and difficulty implementing the interface meetings.

Facilitation of the interface meeting by CHAGs and youth groups presents another important challenge to sustained implementation. CARE, or any implementing NGO, can be seen as a neutral third party with the authority to facilitate an interface meeting bringing together community members, health care providers and local government officials and creating a safe space for all to share their views. Without the NGO in this facilitation role, some groups found it hard to bring all the various stakeholders together and were unclear who was responsible for what or had the authority to facilitate interface meetings. Both CHAGs and youth groups reported challenges with convening and facilitating interface meetings once CARE stepped back from direct implementation.

… Some of the challenges are to do with (…) in this area I have not seen MPs taking part in the scorecard process. This is a challenge in the sense that as an MP, they are connected to influential people and they also have their ways of get community members active; that’s why it’s [their lack of involvement] a challenge. (Bwanje)

Next, some groups continuing with the CSC, including both CHAGs and youth, struggled to attain the necessary resources once the NGO stepped back, which they felt affected both meeting attendance and execution of the action items planned during the process. Without the small incentives provided by the NGO for attendance, such as refreshments, some groups saw their members’ attendance at meetings decline. These groups wanted to be able to offer attendees a small gesture of appreciation like refreshments for participating, but they could not afford to do so. In addition to not having money for incentives, some group members lacked money for transportation to and from meetings, which limited the frequency of members’ attendance and thus their ability to participate and scope of influence. Other groups struggled to find the money necessary to implement the components of their action plans. While CARE had not resourced action items during the initial MHAP project, they had helped facilitate discussions around how resources could be mobilized to address certain action items. The local government also saw the lack of financial resources for implementing the CSC process and, particularly, the action plans as a challenge.

… he is talking about materials that are required when we doing our work; when we are doing scorecard, we are supposed to document everything. When we have agreed on something, we need to leave that with someone representing the people there; we also have to keep a record of the same. So, for that to happen, we require stationary to record all the information that is generated. We all need to write down whatever we discuss during preparation, the roles that have been assigned to each member (…) we just refer to that when we meet to do the scorecard. In short, stationary… (Nzama)

The most important resource we are lacking is transport as we have said. There are two kinds of transport, one which can be done through picking up and dropping a person off to the place. Another involves purchasing bicycles to be used as a form of transport to such places. There is also another way, which includes providing us transport money which can be used to board public transport. All these can help us without any complications, with much emphasis on the bicycles. (Nsiyaludzu_Tcheza)

Third, other NGOs working in the district started to introduce their own social accountability approaches, which are often similar to, but not the same as, the CSC process. The differences in these various social accountability processes and tools caused some confusion among community members. Despite these challenges, most CHAGs that had initially been trained by CARE during the intervention were still actively utilizing the CSC almost 2.5 years after the formal end of the intervention, and youth groups were still using the process amongst themselves to prioritize and score issues and make action plans even when they had difficulty convening other stakeholder groups for an interface meeting.

Many of the above challenges faced by the still active CHAGs contributed to other CHAGs’ decision to suspend the implementation of the CSC. Some of these CHAGs are still together as a group but not actively implementing the CSC due to the barriers they encountered. Their reasons for not continuing the CSC process were related to the logistics of implementation and not a concern about the approach itself. In fact, most of the non-active CHAGs expressed a desire to resume the CSC process if resources or support returned.

Alright …The most difficult part was getting health center personnel to attend the meetings…and to get them to understand what the community is asking for, that was the hardest part…it was tough (Gowa)

Sometimes we were facing challenges for instance, when we called for a meeting only few people were attending. The problems that we faced were like; absenteeism of people, when we called or meeting people were neglecting by not coming for the meetings because they wanted to see people from the Organizations. It would do us good if this thing could be a launched program in our community. (Biriwiri)

Recommendations

Stakeholders recommended periodic refresher trainings and a dedicated source of funds to implement the process and action items as critical to sustainability. CHAGs, youth and local government officials all recommended ongoing financial support for the continued implementation of the CSC, particularly, for meeting materials and supplies, incentives to encourage member attendance and transportation.

There is need for refresher trainings, because the groups received training sometime back …there is need for refresher trainings, for CAG and the health workers mmm…so that when…they are reaching out to people they should at least be confident in what they are doing. Because when you take too long you tend to forget. Umm, transport as I said and refreshments as I said, allowances these things are needed to perform the task well (cross talk) (Manjawira)

I think it’s the issue of support. Of course, we started doing it on our on as we have, but we would need financial support here and there, especially in the area of refreshments and stationary (Bwanje)

In addition, funds that CHAGs and youth could own or oversee and utilize for implementing identified action items were vitally important. All these stakeholder groups also recommended establishing a process for continued refresher trainings around the CSC. In addition, CHAG members recommended having a way to be identified as CSC members when serving in the community, i.e. branding. Youth groups were particularly vocal in their endorsement of the CSC by recommending that the CSC approach should be adopted nationwide throughout Malawi. Local government officials recommended better coordination and consensus among local NGOs using various social accountability approaches.

… Just to add on the same, if we are given gifts like t-shirts, wrappers and poster when visitors come in our community they [community members] wouldn’t find it difficult to identify us… (Mlangeni)

Best recommendation I would give is that when other NGOs come in the same areas to use a similar tool (scorecard) for instance we had ONSE [another NGO project], they should involve the community to find out what they already know and who in that community was involved in previous projects so that when forming their own CAGs for example, they could be mixing community members with experience and those without, so that these can complement each other. (Community Development Assistant)

Discussion

Our findings show a relatively high level of sustainability among groups continuing the CSC process on their own approximately 2.5 years after the termination of formal intervention activities and the NGO stepping back from facilitating direct implementation of the CSC. Most partner groups, from CHAGs to youth to local government, were still utilizing the CSC process, although some were doing so in a modified fashion. Importantly, even the groups that had ceased using the CSC were interested in restarting the process if the supporting resources to do so became available. While most evaluations of the sustainability of health interventions in sub-Saharan Africa look at the period 6–12 months after the end of the intervention (Iwelunmor ), we present the evidence for the sustainability of the CSC as a social accountability approach more than 2 years after the formal intervention given how many of the groups were still using the CSC process on their own without the support of an external implementing organization.

Common drivers of sustainability across partner groups include the perceived effectiveness of the CSC process in creating a safe space in which community members would share ideas and express themselves freely and the perceived value of the process in creating a structure for transparency and accountability within communities and between stakeholders. Experiencing successes with the CSC during the initial, NGO-supported implementation phase motivated groups to continue using the process on their own. For the youth groups, in particular, the CSC process built their agency—elevating their visibility, respect and responsibility within communities and causing others to recognize them as important stakeholders in the effort to improve maternal, newborn and reproductive health. These identified drivers of CSC sustainability are consistent with a meta-analysis of social accountability outcomes (Fox, 2015), which found that such interventions are more likely to be effective when they create environments that allow active voice and representation from historically marginalized and vulnerable segments of the population while simultaneously working to bolster public accountability.

These drivers of sustainability also align with findings from elsewhere around the critical components of social accountability. For example, a case study in Orissa, India, highlighted the ability of social accountability to sensitize community leaders and health providers to women’s needs and to provide an intermediary or structure through which poor and marginalized women can express their demands and have others pay attention to these demands (Papp ). In the Democratic Republic of Congo, social accountability was introduced to provide women with a previously missing structure or mechanism through which to express their concerns about and expectations of local health services in a collective manner instead of individually (Mafuta ). The ability of the CSC process to create these collective structures and levers to modify power, especially for marginalized and vulnerable groups in the community, such as women and youth, are some of the very things that were major contributing factors to its sustainability.

From a health worker perspective, the CSC process created some of these same structures and levers for wielding power, structures and levers that often are not accessible to frontline health workers at the local level. A realist review of social accountability interventions in low- and middle-income countries notes that interventions are more likely to be successful when providers are willing to respond to societal pressures, are receptive to citizen demands, view citizen groups as legitimate and see themselves as having an active role in change (Lodenstein ). In line with strategic approaches to social accountability (Fox, 2015), the CSC scales up collective action on both the demand side—the community members—and the supply side—the health service providers. The CSC is intentional about including frontline health providers as equal partners in the process and having community members and providers work together with local government officials to create action plans. This active involvement from both the client and provider sides of the local health services has been instrumental in the success and sustainability of the CSC.

Resources are a challenge for sustainability. A systematic review found that a key facilitator to the sustainability of health interventions in sub-Saharan Africa is working within existing resources (Iwelunmor ), yet, after the formal intervention ended, these groups had very few of their own resources available to dedicate to the CSC. Lack of resources was a repeated theme across all stakeholder groups. Preparing groups ahead of time to sustain the CSC process within the constraints of their own resources is crucial for sustainability. For future interventions, health systems and NGOs supporting social accountability approaches should plan for a transition phase with periodic refresher trainings and a fund to support implementation to increase the likelihood of community-driven sustainability.

Importantly, sustainability is a dynamic, not static, process (Iwelunmor ). Findings presented here are from May–July 2018, approximately 2.5 years after the end of the formal intervention. Seeing the success and results from the CSC process during the formal intervention period was a motivator for many groups to continue on their own, so it is important the initial intervention period be sufficiently long for these successes to occur. The original MHAP intervention was supported for 3 years before the groups assumed responsibility for continuing on their own. The sense of empowerment and self-reliance created by the CSC process also contributed to groups’ desires to continue.

Fidelity is a key question for implementation science (Rabin and Brownson, 2017) and for the CSC. While nearly all groups reported continuing the initial phases of the CSC—identifying and prioritizing issues—the interface phase of bringing different stakeholder groups together to share issues and create a joint action plan has been more challenging to sustain. Some groups found that other stakeholders were less likely to respond to their invitation to attend an interface meeting once the NGO stepped back, making it more difficult to generate the intended discussion across stakeholders. This suggests the importance of preparing community groups to be more at the forefront throughout the entire CSC process and exposing them to the tactics of engagement at the onset of the intervention, thus allowing more time to learn as this skillset takes time to acquire. The question of fidelity also arises when multiple NGOs are supporting different versions of social accountability approaches within the same or proximate communities. Having different versions being implemented simultaneously might cause confusion among community members and frustration among service providers if they feel they are having to respond to and participate in conflicting or duplicative processes. If one version is being actively supported by an NGO with resource investment, then it is possible that community members may gravitate towards participating in that process and decide it is not worth them continuing their existing efforts on their own. Furthermore, reviews (Fox, 2015; Lodenstein ) have shown varying levels of effectiveness for social accountability interventions depending on the approach and philosophy used.

Limitations

This was a cross-sectional study capturing perspectives on the CSC at one point in time approximately 2.5 years after the NGO stepped back from facilitating the direct implementation of the intervention. Since sustainability is a dynamic, ever-shifting process, these data only provide a snapshot into that process. Furthermore, those who agreed to participate in focus groups and interviews about the CSC over 2 years after the end of the intervention may have been more inclined to support the CSC and be invested in the sustainability of the process. We did, however, purposively invite those from groups that are no longer actively using the CSC to participate in interviews. Importantly, their perspectives on the facilitators and barriers of sustaining the process were similar to those who are still actively engaged in using the CSC, and our sample represented nearly 95% of all groups known to be implementing the CSC at the end of MHAP, the formal intervention period. Given that the interviews were conducted in Chichewa and translated into English for analysis, some of the original meaning of certain phrases and words may have been lost. However, both Chichewa and English interview transcripts were read and reviewed by local data collectors and supervisors fluent in both Chichewa and English; discrepancies were discussed and revised.

Conclusion

These findings around the sustainability of the CSC, coupled with the robust evidence of effectiveness (Gullo ; 2018; 2020), indicate that this approach supports long-term change in accountability at the community level and builds voice and transparency among various actors. Findings from this study show that the CSC is sustainable when planned with that intention. Key elements of the CSC process that were so valuable to stakeholders that they contributed to their desire and motivation to sustain the process include government buy-in; enabling women, marginalized groups and service providers the space and confidence to voice their views; bringing all stakeholders together to develop a shared action plan; following up to ensure actions are implemented; holding each other mutually accountable and working together to influence decision makers. A transition phase with intermittent refresher trainings and an allocated pot of financial resources between a formal intervention and expecting communities to continue independently may enhance continued implementation. The structure of the CSC process helps build this sustainability and addresses concerns seen in other social accountability formats/contexts. Programs using the CSC process and aiming for sustainability should plan for an intermediate or transition phase where NGO still provides periodic refresher trainings and co-facilitation in order to prepare for independence. Programs should also plan (and start building towards) an implementation and action fund for the groups to use as they transition into sustainable independence.