Program sustainability post PEPFAR direct service support in the Western Cape, South Africa.

BACKGROUND: Public health practitioners have little guidance around how to plan for the sustainability of donor sponsored programs after the donor withdraws. The literature is broad and provides no consensus on a definition of sustainability. This study used a mixed-methods methodology to assess program sustainability factors to inform donor-funded programs.
METHODS: This study examined 61 health facilities in the Western Cape, South Africa, supported by four PEPFAR-funded non-governmental organizations from 2007 to 2012. Retention in care (RIC) was used to determine health facility performance. Sustainability was measured by comparing RIC during PEPFAR direct service (20072012), to RIC in the post PEPFAR period (2013 to 2015). Forty-three semi-structured in-depth interviews were conducted with key informants. The qualitative data were used to examine how predictor variables were operationalized at a health facility and NGO level.
RESULTS: Our qualitative results suggest the following lessons for the sustainability of future programs: Sufficient and stable resources (i.e., financial, human resources, technical expertise, equipment, physical space)Investment in organizations that understand the local context and have strong relationships with local government.Strong leadership at a health facility levelJoint planning/coordination and formalized skill transferLocal positive perceived value of the programPartnerships.
CONCLUSION: Sustainability is complex, context dependent, and is reliant on various processes and outcomes. This study suggests additional health facility and community level staff should be employed in the health system to ensure RIC sustainability. Sustainability requires joint donor coordination with experienced local organizations with strong managers before during and after program implementation. If the program is as large as the South African HIV effort some dedicated additional resources in the long term would be required.

Introduction

Over the last twenty years, new sources of donor funding from private foundations, philanthropists and the private sector have significantly expanded the field of HIV/AIDS care. Global funding for HIV increased annually from $1.2 billion in 2002 to $8.6 billion in 2014 though there was a significant plateau of global HIV funding, following 2008, due to the global financial crisis [1]. The increased funding resulted in a small decrease in the incidence of HIV globally. At the same time the increasing number of PLHIV necessitated [2] that low and middle income countries (LMIC) augment their domestic HIV programs. In 2012 UNAIDS reported the main source of global HIV funding came from domestic resources [3]. Additionally in 2012 the, World Banks’ re-classification of country income levels [4] had a negative influence on the flow of donor funding, especially with the delineation of middle income countries (MICs) into lower and upper. Their criteria has been critiqued for being based on aggregate income levels, rather than social inequality [5].

Due to these changes in global donor funding there has been increased interest in transitions or graduations, when large donor funded programs decrease funding or exit a country, requiring the local government to take financial responsibility for their health programs. Transitions have been described as a “new art,” [6] which is “complex” [7].

Recently additional research has emerged on PEPFAR, the Global Fund and other donor transitions globally. PEPFAR transition literature from South Africa, Nigeria and Uganda have highlighted decreased access and reduced quality of care, preventative and community outreach services and retention in care after the withdraw of PEPFAR funding [7-10]. An evaluation in Nigeria found, post transition, a decrease in access to laboratory services which affected viral load testing (92% to 64%; p = 0.02), staff shortages due to a lack of incentives to retain staff (80% to 20%; p<0.01), and reductions in tracing systems for HIV patients (100% to 44%; p<0.01) and community testing services (84% to 64%; p<0.01) [7].

Program sustainability

The central premise of transition directly relates to sustainability and the long-lasting effects of donor funds. How to nurture the continuation of effective program benefits, especially after donors leave, should be a priority for the public health community just as much as implementing new programs. If efforts to scale up and sustain effective health investments are not prioritized, donors are constantly re-inventing the wheel, wasting scarce resources and time [11-13]. Also, there is a moral imperative to sustain programs that are effective. This is particularly true for chronic diseases such as HIV/AIDS.

Very little is known about what happens to programs or their outcomes when donor funding terminates. The literature estimates at least 40–50% of social programs collapse within a year after funding ends [14, 15]. Additionally, Cekan found that very few (1%) development projects are evaluated post donor funding [16].

The research on sustainability is broad, and the quality of the research methods used is generally poor. There is no clear agreement on a definition, little analysis on sustaining programs in a complex health system, and only a handful of lessons learned about large donor transitions have been reported. Local governments are left to sustain donor instituted programs as best they can or to let them expire from lack of funding or attention [6]. Wickremasinghe et al. [17] highlight that to achieve country ownership, strong relationships and engagement with government, in the design, implementation and evaluation are key. Ultimately, new programs need to be embedded within the local health system for government to adopt them [17].

Most donors set the program priorities and control the rules of the donor/grantee playing field, which includes defining sustainability. From the early days of international health Pan-American Health Organization (PAHO) and Rockefeller Foundation have equated sustainability with financial sustainability [18]. More recently, PEPFAR’s HIV/AIDS Sustainability Index and Dashboard (SID) focuses mainly on national level policies and financial sustainability [19].

There have been various international program design initiatives to increase the effectiveness and sustainability of international aid. In the mid-1990’s, the sector wide approach (SWAp), was introduced into international development circles. SWAp was a mechanism specifically targeted at health initiatives which intended to shift the decision making of the health budgets back to host governments, instead of international health donors. Donor funds would be put in a common fund, and local government would coordinate, plan, monitor the budget all health funding based on local priorities [20]. In theory, this would be more cost effective, increase sustainability and reduce duplication by donors and host governments [21]. In practice, countries implemented SWAp differently which made it difficult to measure, additionally there were other strategies introduced to increase the effectiveness of aid. To date these strategies, include the Paris Declaration and Accra Agenda for Action and Busan Partnership for Effective Develop of Co-operation of to improve the coordination of aid effectiveness, have shown few tangible effects on health outcomes [22]. The US government and Global Fund opt-ed out of SWAp, while increasing disease specific funding, directed at non-governmental organizations (NGOs). Research from Uganda [20], Mozambique [23] concluded SWAp received a small percentage of health funding, as PEPFAR and the Global Fund funding dramatically took over the international health funding scene.

The donor community has equated sustainability with financial capacity. Though consistent financial support is a key component of sustainability, we would argue along with others [6, 24, 25] this definition needs refinement. It is important to understand program sustainability to ensure that scarce health system resources, in addition to funding, are effectively used.

Transition in South Africa

South Africa is the country with the greatest number of people living with HIV globally (7.5 million) and with 4.1 million adults on treatment, [26] South Africa is running the largest HIV treatment program globally [10]. From 2004 to 2018, the United States President’s Emergency Fund for AIDS Relief (PEPFAR) invested $5.9 billion into the South African HIV/AIDS response [27]. Most PEPFAR funds in South Africa were distributed to NGOs that supported state health facilities to strengthen HIV/AIDS care and treatment programs. During the initial stages of PEPFAR, the majority of funds supported the distribution of antiretroviral treatment (ART) [28].Over the years, there have been various changes to PEPFAR’s leadership and strategy in South Africa. In 2012, there was a planned transition from service delivery to health systems strengthening, a gradual budget decrease and handover of the HIV program to the South African government (SAG). At this time, a study in Durban, South Africa estimated that 20% of clients were lost to follow-up by care and treatment programs in South Africa [29] mainly due to the poor treatment in government health facilities[8]. Based on Cloete’s estimate, [30] Kavanagh approximates the PEPFAR transition affected 50,000 to 200,000 people living with HIV (PLHIV) [10]. One of the main critiques was that the PEPFAR transition focused solely on care and treatment with no plans for other PEPFAR funded activities (i.e. prevention). Others found that at the national level there was a lack of PEPFAR leadership and a lack of clear guidance and communication around the pace of the budget decrease [31].

This high loss of clients, resulting in a lack of adherence to treatment regimens and consequent possible increase of drug resistant strains of the virus was a major concern. No formal evaluation of the PEPFAR transition in South Africa was ever undertaken; therefore, it is unclear what happened to the thousands of clients on treatment and to staff, and NGOs formerly funded by PEPFAR, or to HIV outcomes, such as ART retention and mortality.

Western Cape transition

The Western Cape is distinct from other South African provinces. With a concentration of tertiary health services and prominent internationally renowned HIV experts, it has some of the best health outcomes in South Africa [32, 33]. Compared to other provinces, the Western Cape also has the lowest HIV prevalence at 7.8% (2012). Historically, the Western Cape has the oldest health system focused on white urban populations, and governed by strong leadership [34]. Governed under the political opposition party, the AIDS program, specifically the PMTCT program was the first of its’ kind in South Africa [35].

Partially due to the availability of resources and strong leadership to make critical decisions and provide guidance, the Western Cape Government Health took the initiative to begin the PEPFAR transition process earlier than other provinces [36]. Over the course of two years (2011–2012) a memorandum of understanding was developed, a detailed database was created, staff cadres and salaries were aligned to government staffing norms and policies and hospitals and district staff were consulted. This process resulted in 40% (n = 78) of PEPFAR clinical and administrative posts being absorbed by government or 13% of all the Western Cape PEPFAR posts [36]. This paper aims to assess how the PEPFAR program in the Western Cape province of South Africa withdrew to identify factors associated with sustained performance.

Methods

All participants provided written informed consent. Ethical approval was received from Boston University Medical Campus Institutional Review Board (Protocol Number: H-37238) and the Biomedical Science Research Ethics Committee, University of the Western Cape, Cape Town, South Africa (BM18/5/2).

We evaluated PEPFAR program outcomes that were sustained following the withdrawal of funding for direct service support (2007–2012) and the factors that led to program sustainability. Health facilities were characterized by their ability to sustain HIV program outcomes post PEPFAR funding for direct service support and the organizational (i.e. health facility and NGO), programmatic, and contextual factors that led to sustainability were analyzed.

A mixed-methods approach was used to examine health facilities supported by four local PEPFAR treatment NGOs from 2007 to 2012. This paper reports on the qualitative results of the study. Quantitative results will be reported elsewhere. Financial sustainability is a key element to achieving program sustainability, but it is not the only factor, therefore the focus of this study was on the non-financial characteristics of sustainability.

PEPFAR intended to terminate direct service support in 2012/2013 in South Africa. This study used this planned direct service end date as the break point of our analysis. We refer to the period during direct service (2007–2012) as, “PEPFAR direct service” and after direct service (2013–2015) as “post PEPFAR direct service” (Fig 1). Retention in Care (RIC) was used to measure health facility performance. RIC and mortality are key indicators that demonstrate the long term sustainability of the ART program [37]. The study used the same definition the Western Cape Government Health (WCGH) uses for RIC which is: patients on first line treatment + second line treatment + third line treatment + patients who stopped ART, divided by (total number of patients on treatment–total transferred out). Sustainability was measured by comparing RIC during PEPFAR direct service 2007 to 2012, to RIC in the post PEPFAR period 2013 to 2015.

Fig 1

PEPFAR strategy timeline South Africa.

RIC is key to achieving the global 90-90-90 U.N goals: 90% of all people living with HIV will know their HIV status, 90% of people with diagnosed HIV infection will receive sustained ART and 90% of all people receiving ART are virally suppressed. RIC is currently used as the main indicator to achieve the second 90: 90% of people with diagnosed HIV infection will receive sustained ART [38]. The 90-90-90 goals have also coincided with the leveling of donor funding globally and the “transition” away from large global donors [39].

Study sample

This study examined health facilities supported by four local PEPFAR treatment NGOs from 2007–2012: (1) Kheth’impilo (KI), (2) Anova Health Institute (3) Right to Care, (4) TB, HIV/AIDS, Treatment Support and Integrated Therapy (that’sit). Since the interviews did not ask subjects about personal health details or collect protected health information, this study was declared exempt from human subjects review by Boston University Medical Campus Institutional Review Board and the Biomedical Science Research Ethics Committee at the University of the Western Cape. Informed consent was received from each participant.

These NGOs were the four PEPFAR supported HIV care and treatment organizations working in the Western Cape from 2007 to 2012. Right to Care’s timelines were slightly later, from 2009 to 2015. Each of the NGOs worked in a specific geographic region. The four NGOs under study supported 100 primary health care facilities in the Western Cape with PEPFAR funds between 2007 and 2012. A description of the health facilities and their outcomes can be found in Table 1. This study excluded tertiary and district hospitals due to the history of the PEPFAR program, which began in tertiary facilities and offered patients access to HIV specialists. Due to these differences, we only included facilities from a PHC (Primary Health Care) level.

Table 1

Summary of qualitative sample by health facility characteristics and outcomes.

Facility Characteristics			Facility Outcomes
Geography	Urban	9 (40.9%)	Overall RIC (2007–2015)	Low (<59.9%)	12 (54.5%)
	Rural	13 (59%)		High (>60%)	10 (45.5%
ART Patient Volume	Low (≤ 700)	8 (36.4%)	Sustainability	Poor (< -5.0%)	14 (63.6%)
	Medium (700.9 ≤ 2999.9)	6 (27.3%)		Sustained (-4.9% to 4.9%)	7 (31.8%)
	High (>3,000)	8 (36.4%)		Improved (>5.0%)	1 (4.5%)
Government ownership	CoCT	6 (27.3%)
	WCGH	15 (68.2%)
	Combined	1 (4.5%)
NGO Support	Anova	5 (22.7%)
	Right to Care	4 (18.2%)
	Kheth’impilo	5 (22.7%)
	that’sit	5 (22.7%)
	Kheth’impilo/ Anova	3 (13.6%)

The quantitative data was used to select the qualitative sample. Purposive sampling was used not to select a representative sample, but a broad distribution of health facilities to examine different contexts and better understand sustainability factors. Six health facility characteristics (geographic area, PEPFAR NGO, ART patient volume, government ownership, sustainability of retention in care (RIC) and RIC at 12 and 24 months) were used to investigate sustainability.

Data collection

In total, 43 in-depth interviews were conducted across a five-month period (October 28, 2018 to April 3, 2019). The interviews were conducted with health facility managers from 20 primary health care facilities, one clinical nurse practitioner who is a high-level practitioner just below a doctor and one lower-level staff nurse (Table 2). Fourteen key informant interviews were conducted with eight government officials and five NGO program managers (Two participants were interviewed from one NGO.). To gain clarity and a better understanding of the themes in the first set of interviews, a second round of follow up interviews were conducted with eight existing study participants.

Table 2

In-depth interviews.

5	NGO
	4 x NGO Program Directors
	1 x NGO Provincial Assistant Manager
22	Health Facility
	20 x Health Facility Manager or Operational Manager
	1 x Clinical Nurse Practitioners Nurses
	1x Staff Nurse
8	Government
	6 x Provincial Government Officials
	2 x District Government Officials
Total First Interviews: 35
Second Interviews
3	NGO
2	Provincial Government
3	Health Facility Managers
Second Interviews Total: 8
Grand Total: 43

To guide the semi-structured in-depth interviews, interview guides and information sheets were developed. Interview guides were validated with two health facility managers in KwaZulu-Natal. Three different interview guides were developed specific to each participant category (Table 2).

Data analysis

The interview recordings were transcribed, coded and themes were identified using a grounded theory and a thematic analysis using Nvivo 12 Pro [40]. Thematic analysis allows for theories to emerge from the data without trying to fit “preconceived ideas and theories” into the data, grounding the analysis in the data. This inductive process allowed for the observation of repeated patterns to allow for theories to emerge organically from the data. The sustainability factors guided the themes to be analyzed, but the analysis allowed for additional themes to emerge. The analyses used a flexible analytic approach which allows the investigator to move back and forth between the data and analysis to connect emerging themes [41, 42].

Results

Our qualitative results found nine key factors that lead to program sustainability. The factors associated with sustainability focus on people (e.g. health facility leadership, skilled staff, stable human resources, perceived value), relationships (long standing presence, partnerships) systems (donor coordination and formalized skills transfer) and additional resources (financial, human resources, technical expertise, equipment, physical space). These factors were integrated with the broader transition and sustainability literature to produce a framework to maximize program sustainability outlined below (Tables 3–8).

Table 3

Prior to program launch.

Actions: National Level
Donor	Before a funding announcement is put out donors need to work with national and provincial level stakeholders (i.e Ministry of Health, National Treasury, AIDS Councils) to understand local needs and gaps.
Grantee	Provincial government to work with facilities and communities to understand local needs.
Donor	Prioritize funding, local gaps and innovation.
Donor	Prioritize funding organizations that have a record of accomplishment in the geographical area.
Donor/Grantee	High level commitment
Grantee	Ideally established donor coordination system which communicates with all levels of governments

Table 8

Post transition (3–5 years after the end of the program).

Provincial and National Level
Donor/Grantee	• Monitor sustainability indicators
	• Conduct post-evaluation of program using a time series analysis
	• Share insights and lessons learned with all stakeholders

Joint planning

The donor and local government at every level of government (health facility, district, and provincial level) and NGO need to plan together throughout the life of the donor funded program. The main goal of the donor should be to fill needed gaps and let government lead the planning process. To ensure the planning process is authentic, the donor needs to have the skills to fill gaps, while respecting the local governments needs. Additionally, it helps if local government understands their own health system needs and has an established donor coordination system at a provincial level. Funding for a liaison person at the provincial level to coordinate transition activities was highlighted as a key to sustainability. This person would be responsible for ensuring transparency of donor funded activities and work with government to ensure the program is integrated into the local health system.

Long-standing presence

When an NGO has an established office in the geographic region, they understand the context, local policy and have strong relationships with government, which builds trust and results in more sustainable outcomes. These relationships and trust led to post donor funding opportunities for formerly funded PEPFAR NGOs. The most important PEPFAR outcome the WCGH wanted to sustain was the transfer of skills. We found this transfer must be formalized at both a centralized (i.e. provincial level) and decentralized level (i.e. health facility level). The centralized level of government should decide the human resources that are essential, which should be followed up with adequate financial resources. At a de-centralized level, a strong health facility manager is required to ensure the skills of donor funded staff person are transferred to local staff to sustain skill sets. One health facility manager ensured six months before the PEPFAR staff member left, they mentored and trained a local staff member in their job responsibilities.

Partnerships

One of the main factors, which led to sustained infrastructure, resources and improved donor coordination was due to donor/grantee partnerships. The donor and grantee were committed to providing resources toward a common goal—controlling the HIV epidemic. This commitment played out in several ways. In some instances, the PEPFAR NGO built a pharmacy and the local government created pharmacy posts to manage the pharmacy. In another example, a medical cart was funded by the PEPFAR NGO and commodities were stocked by local government.

“I know it was not so difficult to get things (PEPFAR direct service) but now (post PEPFAR) it’s difficult. You need to write a motivation first to get a table or chair and say there is an underspending.”(NGO Program Director, Rural)

Local government understood their service delivery gaps and required PEPFAR expertise post PEPFAR direct service support. Valued and trusted for their expertise, the district hired a former PEPFAR funded medical doctor to provide Nurse Initiated Management of Anti-retroviral Therapy (NIMART)mentorship and an NGO to assist with facility management trainings.

“I think, look, before it was never just about people coming in and doing the work for us. There was that transferring of skills. There was an ongoing process. There were relationships being built, when the mentors, from the different NGOs would come in, they would not just focus on their teams. They would look at; they would work with the (local government) team.” (Provincial Government Official, Urban)

Human resources

In the Western Cape (unlike some other areas of South Africa), PEPFAR was able to plan and formalize the transition of human resource posts from PEPFAR to local government. The retention of these posts and the relationships created led to a greater number of skilled staff being retained in the local health system. The PEPFAR funded NGO found investing in lower-level cadre of health facility staff (i.e. nurses, data capturers) was more sustainable, because they were from the local communities they were working in and less likely to leave the geographic area. Additionally, human resource stability was key to sustaining health facility outcomes. A government official mentioned they witnessed a decrease in health facility outcomes when there was human resource instability, when PEPFAR changed strategies to health systems strengthening support.

“It’s not that I’ve read any evidence on this but just what we pick up in the system. You see that slump (in the data) and you see people pick it up and pull it together and move forward.”…..“when you pull it (donor funding) out you will see a dip, but at some point the team that stays behind develops a sense of resilience.”(Provincial Health Official, Urban)

Strong facility manager

Sustainability was the result of dedication and the extra time that health facility leaders invested in coordinating donor funded activities at a facility level. A strong health facility manager needed to have a number of characteristics including time management and organizational skills, is motivated, has good communication with donor and local staff, able to manage stress, as well as plan for the future. Part of the role of the health facility manager included taking ownership of donor staff to ensure they are used effectively. High performing/high sustainability health facility managers did not wait for PEPFAR NGOs to define their staff members’ roles, they placed PEPFAR funded staff into gaps, integrated them into staff meetings and fired those who were too challenging.

“I always believe that if the staff [emphasis] are happy you get more out of them, than when they are not, so when you go into a facility, you look at first your staffing issues before you actually look at the patient issues, ‘cause patient issues can always sort but once you sorted your staffing problems and when they are seen as problems, you can sort that out and they are willing to sort out your patient issues for you.”(Health Facility Manager, Urban)

We observed there was more motivation by local health facility mangers to take ownership of the PEPFAR program when they felt the donor program was beneficial to them and the communities they served. Therefore, perceived value by the grantee was a factor that led to program outcome sustainability.

“And it also improved your other services because although they were only employed for ARVs, I trained them in Integrated Management of Childhood Illness(IMCI). Yeah so and that, that was one of the things I always did with the (PEPFAR) NGOs is that the person that’s employed with us but remember I’m going to train them in the capacity of them, so that every patient, they will see to all the other needs of the patient.” (Health Facility Manager, Urban)

Additional resources

Many of the top performing health facilities were provided with additional PEPFAR NGO support, including equipment (e.g. lactose meters, scales, computers) and infrastructure (e.g. extra counselling rooms, gardens) and with multiple PEPFAR posts from the human resource transition.

“But look, bottom line is, there is more that can be done with resources and hands-on deck. Definitely!” (Provincial Government Official, Urban)

Discussion

Our results add useful new insights to the current broad transition and sustainability literature. The planning and transition of donor funded activities should be led by the local government at a centralized level: either the provincial or district level. Congruent with the Avahan studies in India, the institutionalization of specific donor program components—mainly budgets, reporting systems and staff structures—are important from the beginning of the program [43]. Effective aid, and sustainability are reliant on alignment with the countries health and development priorities [44]. It is important for government and the NGO to be included into the design and implementation of the program since they will ultimately own the program when the donor pulls away [45]. Beracochea succinctly states, “Effective aid is by design, not by default”[45].

While the literature highlights the importance of leadership, our study specifies the qualities which a leader should display. Long tenure does not equate with leadership. Although we found that health facility managers stayed in their position for an average of 13 years, and on average had worked for local government for 23 years, long tenure was no guarantee of strong leadership and management skills. The best performing health facilities (high RIC and sustainability scores) were led by a health facility manager who had an understanding of the health facility’s needs, set clear roles and responsibilities with donor and local staff, had empathy for patients and cared for the wellbeing of their staff, while creating a strong motivated teams.

There are contextual factors that likely assisted sustainability in this study. This study was based in a province that is politically and financially stable. Additionally, PEPFAR funded an abundance of HIV research, which assisted the grantee and donor in understanding the HIV epidemic in the Western Cape. A better understanding of context will assist large donors when transitioning programs to local government to help ensure program gains are retained post donor funding. This study period also coincided with a time of change in South Africa, when the political support for ART access increased, national HIV budgets increased substantially, AIDS activism was noteworthy and when task-shifting and ART treatment guidelines became more receptive to placing more patients on treatment. These positive changes at a local level supported PEPFAR program’s goals, and the sustainability of the HIV program.

PEPFAR has been criticized for its vertical or single disease approach that tends to weaken the capacity of the local health system [46]. Some research has shown that vertical programs can improve health outcomes [47, 48], although the impact of vertical programs are not as effective as developing local policy or improving the local health system [49]. Others have argued that vertical programs displace funds from other more significant disease burdens [24, 50], increase the brain drain from the public system to donor NGOs who tend to pay higher salaries [24], and adds more work to overworked health care workers. The main concern is that if the public-sector workforce and infrastructure are undermined many African countries they will not reach their 90-90-90 goals [51]. In 2005, the Mozambique Ministry of Health and Health Alliance International took a health systems strengthening approaching, integrating ART services into the existing primary health care system. This “diagonal” approached proved successful, increasing HIV testing rates, reducing loss to follow up and expand HIV services geographically [52]. This approach also strengthened the PHC system, including laboratory and pharmacy services.

Though vertical programs have been criticized for creating parallel health systems, we found that the vertical PEPFAR support was not necessarily a barrier to sustained outcomes in the control of a priority disease. The lesson for future donors is the need to integrate their programs into existing local health structures for program outcomes to be sustainable. Practically this means local governments can place donor-funded vertical program staff into the health system but ensure HIV testing referrals and lab services are integrated in the public health system. The manager at the donor/grantee interface needs to take ownership of the donor program to ensure the donor support is streamlined and efficient for all facility staff and patients.

Donors and local government need to jointly create a sustainability or a phase out plan for every donor-funded activity. It is important to note that not every program activity needs to be sustained. The key question to ask is, Is the sustainability of the outcomes relevant to the objectives of the intervention or activity?[53]. Donors need to be especially careful about phasing out human resources in smaller health facilities that will struggle to maintain program outcomes because they are usually absorbed into other services in the health facility. “We found post PEPFAR direct service, larger health facilities could allow former PEPFAR trained staff to continue to work in the HIV program or were able to sustain PEPFAR’s vertical approach, while in smaller health facilities, PEPFAR trained staff were absorbed into other health services since there were fewer staff.” Strong leaders at the lowest level of the grantee (i.e health facility), plus retention of community health workers, administrators and data capturers were key to ensuring that positive health outcomes were not lost post donor funding.

Conclusion

The results of this sustainability study provide concrete guidance for donors, NGO’s, philanthropists, and local governments about how to channel donor funding to improve health outcomes. The results of this research can be integrated into program plans to maximize the sustainability of program outcomes. These policy recommendations set the sustainability factors within the context of transition to provide further guidance for donor transitions. To ensure the sustainability of outcomes of future transitions, the PEPFAR transition should have been formally evaluated by PEPFAR to ensure learnings could be applied to other countries going through a similar process.

The Western Cape PEPFAR program was able to transfer and sustain skilled health facility workers via the formal transition, sustain HIV expertise, maintain infrastructure and ensure a strong HIV program. In part, this was due to the strong and stable leadership in the province, formalized skill transfer at a centralized and de-centralized levels, and an abundance of HIV research on the Western Cape. Donors also need to be careful when phasing out human resources in small health facilities, because their specialized skill set will be lost then they are used in other areas of the facility. Though not the focus of this study, the ability of the local government to finance the majority of the HIV program budget was one of the key sustainability components. While research and HIV expertise were not initially defined as sustainability factors, the deep understanding if the Western Cape HIV epidemic and support in policy forums by HIV experts, played a significant role in building a strong HIV program.

This study was unable to identify a single predictor of sustainability. This was not surprising as sustainability is complex, dependent on the context, and relies on various processes and outcomes. What was clear is that future disease specific donor funded programs need to be intentionally integrated into health systems or use a diagonal program approach. If global efforts are going to make progress towards the 90-90-90 HIV goals, donors and local governments need to strategically plan for sustainability from the beginning of any donor funded program, while integrating external investments within local health programs and structures. While we have the tools to end the HIV/AIDS epidemic, global funding that would have supported the 90-90-90 goals has been withdrawn, which has undermined these efforts.

Our study outcomes can be generalized in planning for program sustainability. The following tables (Tables 3–8) provide a checklist for donors and grantees at each phase of a program.

Health facility interview guide.

(DOCX)

Click here for additional data file.

6 Jan 2021

PONE-D-20-37215

Program Sustainability Post PEPFAR Direct Service Support in the

Western Cape, South Africa

PLOS ONE

Dear Dr. Chiliza-

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.   I have carefully reviewed the manuscript and the reviewer comments, and please provide a major revision, taking the comments into account, and and resubmit your manuscript for reconsideration. Please note all reviewers felt this is an important and timely topic.  Please take the reviewer comments into account and set out your responses to us in detail. Please note that this is not a guarantee of publication. Revisions will be sent through a secondary review process before a final decision is made.

Please submit your revised manuscript by 5 March 2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Melissa Sharer

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information, or include a citation if it has been published previously.

3. In the Methods, please discuss whether and how the questionnaire was validated and/or pre-tested. If these did not occur, please provide the rationale for not doing so.

4. Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure.

5. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1 in your text; if accepted, production will need this reference to link the reader to the Table.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript is well written and the data and findings are presented in a clear fashion. My primary concerns with the submission is the time period in which the data is drawn. While the comparison RIC data is for the period immediately after transition, the data is five years old. There have been significant changes in the PEPFAR program since the time in which the study is conducted. At a minimum the authors should be using more recent data. Also, RIC is but one indicator which could and should be used to assess sustainability. Was there consideration given to other clinical indicators associated with the cascade, particularly HST data and viral load/suppression data?

Reviewer #2: PLOS-One_Review

Program Sustainability Post PEPFAR Direct Service Support in the Western Cape, South Africa

December 2020

This article provides a timely and important inquiry into the sustainability of donor-supported global health programs. This is an important topic that should have far greater attention paid to it in the global health literature. There are several aspects of the paper that require further development, including more extensive engagement with the global health literature, greater attention to the context of the Western Cape province, and relative to their proposed structure of program development, consideration of the role that community-based HIV/AIDS activists have played in South African HIV/AIDS politics and policy across the institutional levels of the state.

Thus, while the paper provides an interesting perspective on an important topic, it requires major revision before it can be considered for publication. As such, my recommendation for this paper is for it be revised and resubmitted for review.

First and foremost, there is insufficient engagement with influential approaches to program design in the field of global health. While these approaches have been primarily framed around patient-centered health outcomes rather than sustainability, since the authors focus on retention in care (RIC) as the mechanism through which sustainability is defined, the analysis is focused on similar parameters. With respect to approaches that the authors should review and engage with, Partners in Health (PiH) have advocated for an accompaniment approach to program development that involves extended engagement with across governmental sectors, civil society, and communities. In addition, Health Alliance International (HAI) has advocated for ‘diagonal’ approaches to global donor funding that involve coordination and engagement with local governmental and civil society actors and organizations to increase the impact of donor aid on communities and produce sustainable interventions. These are but two of many approaches to sustainability for global health programs that the authors would do well to consider as part of their analysis. The analysis is also quite shallow relative to the history of global/international health, which is a bit troubling. In short, the authors are not contextualizing their findings relative to important developments/histories in the field, which limits the impact and significance of their findings.

In addition, there is very little attention paid to the contextualization of this case study. I find the research to be valuable, but there is very little attention to precisely how and why the Western Cape stands out relative to other regions in South Africa relative to HIV/AIDS treatment. I have commented on this at length below and will refer the authors to my input there, but this point also links to their recommendations for program design, particularly their focus on facility managers. Taking individual leadership as a key factor without contextualizing the conditions within which facility managers operate diminishes the potential contribution. My own experience with facility managers in the Western Cape was that those who work closely with the community and HIV/AIDS activists also found success, but these sort of ties and shared governance did not find their place in the analysis or the program design recommendations the authors provide.

Regarding the parameters for the analysis, I appreciate that the authors have made the decision to publish their quantitative data separately. However, to exclude the financial data on sustainability from this paper entirely is quite limiting. This sort of data would provide the necessary context for understanding why some clinics were able to undertake the transition better than others, which remains unclear in the paper. It would also strengthen the comparative value of the paper, as we don’t know the size of Pepfar programs relative to public health budgets in the Western Cape, which would seem critical to contextualizing why the province was able to absorb former Pepfar-funded NGO staff members into the public health system. It may be the case that other societies that are Pepfar recipients would be able to undertake similar measures, but we don’t have the necessary information to undertake these sort of comparative exercises.

While it is likely beyond the scope of this paper, I would encourage the authors to think more critically about the broader power dynamics within which their research is situated. In thinking about “transitions” away from global donor funding, it is not only Pepfar that is noteworthy, but also the Global Fund. In short, this is a very important topic that they are engaging with, the implications of which are very much a life and death matter for many around the world, even more so as the COVID-19 pandemic brings with it economic contraction.

Since many recipient countries remain caught in a situation where conditionalities associated with debt repayment mean that they cannot increase health spending without commensurate increases in GDP, the looming crisis of “transitions” in global health funding mean that many societies will be facing declining levels of donor support along with shrinking health budgets. While a drug-resistant HIV epidemic is one possible entailment of this, generalized increases in mortality, under-nutrition, and suffering are also likely.

All of which is to say, I know that I have been quite critical with my review, but I have done so because I see the potential impact of this line of research and would like for the paper to reach its full potential. As such, please take my comments and critique in the constructive spirit in which they have been given, as I believe that this is an important topic that needs far greater attention.

Comments

Line 62

Retention in Care (RIC) – define acronym with first use

Line 105 – 108

It is true that many countries began to augment transnational donor funding during this time, but it is also important to note that the 2008 financial crisis produced a levelling-off in donor funding, and that with access to HIV/AIDS treatment and small decreases in HIV incidence, that the number of PLHIV continued to increase during this time, necessitating that recipient governments augment their HIV/AIDS programs.

Line 108-109

It would also be important to note that the World Bank recategorized country income levels according to new criteria during this time, which led to restrictions on donor flows, such as with the delineation of middle-income countries (MICs). That these criteria are based on aggregate income levels and do not take into account levels of social inequality has been an important critique of this process.

Line 140

A bit more background on why the donor community defines sustainability along financial lines is a very important issue to contextualize. From the roots of early international health programs led by the Pan-American Health Organization (PAHO), which were funded by the Rockefeller Foundation, to the rise of selective primary healthcare and cost-recovery amid structural adjustment in the 1980s, there is a clear thread whereby donors and countries that are able to exert authority within international institutions express power by defining program sustainability and impact according to criteria that they set, one of which is financial sustainability.

Line 150 – 152

Please see James Pfeiffer’s research on Mozambique on how Pepfar-funded interventions interact with public health systems, as it is more complex than these programs “working inside” public health clinics, day hospitals, etc.

Line 163-164

This is a good point, but it is also important to note that the emphasis on access to treatment was also based on the logic of treatment as prevention (TasP), or that people on HIV/AIDS treatment with undetectable viral loads could not transmit the virus to other people.

Line 169

On the lack of formal analysis of the transition in Pepfar programmatic leadership: who was to funded this? Who should have dedicated staff to examine this? There is an implicit critique here but it is left undefined, leading the reader to assume that the South African government should have done so. Is this the authors’ position? If so, please define.

Line 174

A bit more context here on why the Western Cape was distinct is important. The concentration of tertiary services and expertise is significant, but that is informed by the uneven historical development of health services around white, urban populations that began during the colonial era, continued during apartheid, and has not been resolved during the post-apartheid era. Also, HIV/AIDS programs were developed earlier in the province under the guidance of Fareed Abdullah and his team, particularly with PMTCT, which national government intervened to stop during the dark days of the Mbeki era. But support from the Global Fund in the province, which came earlier than the rest of the country, set up the Western Cape as an early success story and a province that has continued to exhibit stronger relative program management than others. Also, I believe that a similar process had already been undertaken with a Global Fund grant in the late 2000s, so there would have been institutional knowledge on how to manage the transition of donor programs from the government to an NGO.

Line 194

I hope that the financial aspects of the transition are not left out entirely, as that would weaken the robustness of the analysis considerably.

Line 200

In terms of defining RIC, a bit more would be helpful. I’m assuming that you are referring to people living with HIV/AIDS (PLHIV) who were adhering to treatment that were lost to follow up (LTFU)/did not adhere after the transition on Pepfar programs? Clarifying this would be helpful.

Line 216

Define primary healthcare (PHC) for first use

Line 232

Of what level were the staff nurses? Clinic Nurse Practitioners would be the assumption, but please define that for the reader.

Line 253

Please define modified grounded theory

Line 274

Defining the donor as an NGO is confusing here, since Pepfar is a donor program funded by the US gov’t. If you are referring to a primary recipient organization (PRO) that is working with secondary recipient organization (SRO), then that needs to be defined. Also, Pepfar, the Global Fund, and several other major global health programs are public-private partnerships (PPPs), so thinking through how you define these relationships is important, as simply labeling the donor and NGO limits the applicability and impact of your findings. Also, is the grantee always local government? This section needs to be thought through much more.

Line 277

Were there any parameters for defining what the coordinating position would be enabled to do/oversee? This seems a bit general and undefined since it is the key point in the section.

Line 280

Donor funded organization? Seems like there is a missing word here

Line 286 - 288

This is an important point that you are making, but it is not sufficiently contextualized. What you are observing is that the responsibility for a successful transition in skill transfer is decentralized and falls to the level of a facility manager. It would lead one to assume that the better capacitated facilities would therefore be better suited to have successful transfers. Since the areas with highest HIV prevalence and greatest need for skills transfer (the peri-urban townships) also tend to have over-burdened heal facilities, this is a critical point that should be further contextualized.

Line 303

Define NIMART for the reader

Line 314 – 31

Again, this had already been done by the WCDoH previously, with the transition of clinic management for the HIV/TB clinics established by Médecins sans Frontières (MSF) in Khayelitsha. There were some bumps in the road with these transitions, and it was a learning process. So, again, the success here is not surprising but the result of previous experience in navigating precisely this sort of transition.

Line 331

Again, the fact that the extra labor associated with ensuring a sustainable transition is being transferred to the facility level is really an important point. It is not being managed by the donor, provincial health, or city health, but by the facility manager. There would be a huge range of outcomes to be expected then, which would depend not only on the personal attributes of the facility manager, but the extent to which the facility is fully staffed, operational, etc. so that the manager has the ability to focus on the transition. Also, I would assume that CNPs would play a critical role in facilitating this transition, as they often have the clearest understanding of staffing needs, shortages, and areas where increased efficiencies can be achieved. Their labor, however, is often rendered invisible in this process, which is problematic.

Line 352

Define IMCI

Line 391

It is helpful that you are addressing the contextual factors here, but this is insufficient to frame your findings, which are quite particular to the Western Cape.

Line 396

It would be helpful to mention the role of HIV/AIDS activism in producing this change, as this was critical to enabling the shifts you identify

Line 400

There is a long-standing debate on the limits of vertical, disease-specific interventions and their lack of sustainability. Proponents of using “vertical” interventions to strengthen the broader health system (horizontal) interventions have advocated for doing precisely what you advise here, to create “diagonal” programs that use vertical funding streams for health systems strengthening. Health Alliance International (HAI) has done work that has modeled this approach in Mozambique is one of the most significant examples of the potential impact and success of this approach in the global health. In general, I would recommend linking your case study and discussion of findings to the global health literature, as your case study is constructed as a stand-alone example, when in reality it is part of a broader conversation on how best to channel donor funding to improve health outcomes.

Line 411

Is it because facilities are smaller or that they may be struggling to meet the level of need in communities with high burdens of disease?

Line 422

Does it make sense to mention the 90-90-90 goals for the first time in the conclusion? If this is the aim of the paper, then it would make sense to introduce this goal (which we are projected to miss significantly by the way) earlier in the paper.

Line 437

Who are national stakeholders? Does they include civil society and PLHIV or HIV/AIDS activists?

Also, shouldn’t provincial government work with facilities and communities to understand local needs?

Line 452

What are local champions? Who defines needs? What is the role of the community in this process?

Line 461

It might be useful to include the provincial treasury in the key stakeholders meetings, since presumably they will need to plan several years in advance if a transition will create greater budgetary demands for the health sector. The medium term expenditure framework (MTEF) requires such advance planning for budgetary processes.

Line 465

I would include the community or some proxy thereof in the final box in this section. The role of HIV/AIDS activists as counsellors and mentors who were also consulted by the WCDoH early on in the development of HIV/AIDS programs was vital to their success.

Line 471

Who is funding the staffing/capacity required to develop the transition plan? Is this being done by external consultants?

Line 472

The recommendation to have the skills transfer managed at the provincial level is contradicted by your evidence, which showed that facility managers oversaw this process.

Line 474

Given that the entire focus of your paper is to emphasize the importance and lack of research on how care is affected by a donor transition, I am very surprised that there is not a post-transition phase for research or monitoring/evaluation. Your proposed flow of program transition would therefore reproduce the precise issue that your paper aims to rectify.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Theodore Powers

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PLOS-One_Post-Pepfar Services_Review.docx

Click here for additional data file.

15 Mar 2021

Dear Plos One,

Please find my responses to the reviewers below.

I would also like to thank Professor Powers for a thorough and detailed review of my work. I really appreciate the time you spent reviewing it.

Thank you.

Jessica Chiliza

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information, or include a citation if it has been published previously.

3. In the Methods, please discuss whether and how the questionnaire was validated and/or pre-tested. If these did not occur, please provide the rationale for not doing so.

I did not develop a questionnaire, but I validated the interview guide with two Health Facility Managers. I noted this on lines 294-295. I also have attached the three different interview guides I used with government leaders, NGO managers and health facility staff.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

4. Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure.

Noted and updated. Thank you.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

5. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 1 in your text; if accepted, production will need this reference to link the reader to the Table.

Noted and edited. Thank you.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

________________________________________

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

________________________________________

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript is well written and the data and findings are presented in a clear fashion. My primary concerns with the submission is the time period in which the data is drawn. While the comparison RIC data is for the period immediately after transition, the data is five years old. There have been significant changes in the PEPFAR program since the time in which the study is conducted. At a minimum the authors should be using more recent data. Also, RIC is but one indicator which could and should be used to assess sustainability. Was there consideration given to other clinical indicators associated with the cascade, particularly HST data and viral load/suppression data?

At the time of this study (2018-2019) the most recent RIC data at a facility level available from the Western Cape Department of Health (WCDoH) was used. While other indicators of facility performance were considered the availability and comprehensive nature of the WCDoH data lent itself to characterizing facility performance. Clinical data is not easily available at a central source that allows facility specific aggregation.

The indicators used to measure sustainability needed to be outcomes of the PEPFAR program. Initially we considered using 4 indicators (proportion of new HIV cases identified, the proportion of people on treatment, proportion of people who continued treatment, mortality data and viral loads). When we initially analyzed all four indicators we realized simplifying to focus on the most sensitive measure of performance, which was RIC, would allow us to characterize facility performance to guide the selection of the most widely performing facilities for the interviews.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Reviewer #2: PLOS-One_Review

Program Sustainability Post PEPFAR Direct Service Support in the Western Cape, South Africa

December 2020

This article provides a timely and important inquiry into the sustainability of donor-supported global health programs. This is an important topic that should have far greater attention paid to it in the global health literature. There are several aspects of the paper that require further development, including more extensive engagement with the global health literature, greater attention to the context of the Western Cape province, and relative to their proposed structure of program development, consideration of the role that community-based HIV/AIDS activists have played in South African HIV/AIDS politics and policy across the institutional levels of the state.

Thus, while the paper provides an interesting perspective on an important topic, it requires major revision before it can be considered for publication. As such, my recommendation for this paper is for it be revised and resubmitted for review.

First and foremost, there is insufficient engagement with influential approaches to program design in the field of global health. While these approaches have been primarily framed around patient-centered health outcomes rather than sustainability, since the authors focus on retention in care (RIC) as the mechanism through which sustainability is defined, the analysis is focused on similar parameters. With respect to approaches that the authors should review and engage with, Partners in Health (PiH) have advocated for an accompaniment approach to program development that involves extended engagement with across governmental sectors, civil society, and communities. In addition, Health Alliance International (HAI) has advocated for ‘diagonal’ approaches to global donor funding that involve coordination and engagement with local governmental and civil society actors and organizations to increase the impact of donor aid on communities and produce sustainable interventions.

Thank you for these references. My understanding from the PIH accompaniment approach is that this was focused on patient adherence to ART, which does not relate to program sustainability. While these examples clearly provide assistance, improving continuity of care for HIV defaulters our research was focused on program sustainability not individual continuity of care.

(Mukherjee JS, Barry D, Weatherford RD, Desai IK, Farmer PE. Community-Based ART Programs: Sustaining Adherence and Follow-up. Curr HIV/AIDS Rep. 2016;13(6):359-366. doi:10.1007/s11904-016-0335-7)

Yes, HAI has advocated for SWAp and various other approaches to coordinating programs and asking for broader health system support, but vertical programing budgets have outweighed the Ministry of Health’s attempts at changing these practices. I have added a section on this the discussion.

These are but two of many approaches to sustainability for global health programs that the authors would do well to consider as part of their analysis. The analysis is also quite shallow relative to the history of global/international health, which is a bit troubling. In short, the authors are not contextualizing their findings relative to important developments/histories in the field, which limits the impact and significance of their findings.

Thank you for this comment. I added a few paragraphs in the Program Sustainability section.

In addition, there is very little attention paid to the contextualization of this case study. I find the research to be valuable, but there is very little attention to precisely how and why the Western Cape stands out relative to other regions in South Africa relative to HIV/AIDS treatment.

I added a paragraph about how the Western Cape is different from the rest of South Africa. Please refer to lines 209-216.

I have commented on this at length below and will refer the authors to my input there, but this point also links to their recommendations for program design, particularly their focus on facility managers. Taking individual leadership as a key factor without contextualizing the conditions within which facility managers operate diminishes the potential contribution. My own experience with facility managers in the Western Cape was that those who work closely with the community and HIV/AIDS activists also found success, but these sort of ties and shared governance did not find their place in the analysis or the program design recommendations the authors provide.

I regret that I did not formally enquire about the ties between activism and closeness with the community. I found some facility managers were skeptical of HIV activists and others were HIV activists themselves.

Regarding the parameters for the analysis, I appreciate that the authors have made the decision to publish their quantitative data separately. However, to exclude the financial data on sustainability from this paper entirely is quite limiting. This sort of data would provide the necessary context for understanding why some clinics were able to undertake the transition better than others, which remains unclear in the paper. It would also strengthen the comparative value of the paper, as we don’t know the size of Pepfar programs relative to public health budgets in the Western Cape, which would seem critical to contextualizing why the province was able to absorb former Pepfar-funded NGO staff members into the public health system. It may be the case that other societies that are Pepfar recipients would be able to undertake similar measures, but we don’t have the necessary information to undertake these sort of comparative exercises.

I completely agree with you. In my original proposal we included the financial data for individual NGO’s and different health facility levels but when we started collecting information it was impossible and would have totally changed the thesis and for that reason left out. It remains a valuable suggestion. I hope someone who has an accounting background would take this up. It is surprising it has not been conducted by PEPFAR. It would be invaluable. We thought we could do it but the focus was not on financial sustainability but on what makes for sustainable outcomes.

While it is likely beyond the scope of this paper, I would encourage the authors to think more critically about the broader power dynamics within which their research is situated. In thinking about “transitions” away from global donor funding, it is not only Pepfar that is noteworthy, but also the Global Fund. In short, this is a very important topic that they are engaging with, the implications of which are very much a life and death matter for many around the world, even more so as the COVID-19 pandemic brings with it economic contraction.

Since many recipient countries remain caught in a situation where conditionalities associated with debt repayment mean that they cannot increase health spending without commensurate increases in GDP, the looming crisis of “transitions” in global health funding mean that many societies will be facing declining levels of donor support along with shrinking health budgets. While a drug-resistant HIV epidemic is one possible entailment of this, generalized increases in mortality, under-nutrition, and suffering are also likely.

All of which is to say, I know that I have been quite critical with my review, but I have done so because I see the potential impact of this line of research and would like for the paper to reach its full potential. As such, please take my comments and critique in the constructive spirit in which they have been given, as I believe that this is an important topic that needs far greater attention.

Comments

Line 62

Retention in Care (RIC) – define acronym with first use

Noted and edited. ______________________________________________________________________

Line 105 – 108

It is true that many countries began to augment transnational donor funding during this time, but it is also important to note that the 2008 financial crisis produced a levelling-off in donor funding, and that with access to HIV/AIDS treatment and small decreases in HIV incidence, that the number of PLHIV continued to increase during this time, necessitating that recipient governments augment their HIV/AIDS programs.

Noted and edited.

______________________________________________________________________

Line 108-109

It would also be important to note that the World Bank recategorized country income levels according to new criteria during this time, which led to restrictions on donor flows, such as with the delineation of middle-income countries (MICs). That these criteria are based on aggregate income levels and do not take into account levels of social inequality has been an important critique of this process.

Noted and edited.

___________________________________________________________________

Line 140

A bit more background on why the donor community defines sustainability along financial lines is a very important issue to contextualize. From the roots of early international health programs led by the Pan-American Health Organization (PAHO), which were funded by the Rockefeller Foundation, to the rise of selective primary healthcare and cost-recovery amid structural adjustment in the 1980s, there is a clear thread whereby donors and countries that are able to exert authority within international institutions express power by defining program sustainability and impact according to criteria that they set, one of which is financial sustainability.

Thank you for this comment. I added a section which provides more context.

______________________________________________________________________

Line 150 – 152

Please see James Pfeiffer’s research on Mozambique on how Pepfar-funded interventions interact with public health systems, as it is more complex than these programs “working inside” public health clinics, day hospitals, etc.

Thank you for this comment. I added a paragraph which explains PEPFAR’s influence on the health system.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 163-164

This is a good point, but it is also important to note that the emphasis on access to treatment was also based on the logic of treatment as prevention (TasP), or that people on HIV/AIDS treatment with undetectable viral loads could not transmit the virus to other people.

This concept came later. While this was the rationale for the TB programs there was limited evidence until much later and used in the PEPFAR program.

______________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬_______________________________

Line 169

On the lack of formal analysis of the transition in Pepfar programmatic leadership: who was to funded this? Who should have dedicated staff to examine this? There is an implicit critique here but it is left undefined, leading the reader to assume that the South African government should have done so. Is this the authors’ position? If so, please define.

This is a good point that is addressed in the Conclusion section.

______________________________________________________________________

Line 174

A bit more context here on why the Western Cape was distinct is important. The concentration of tertiary services and expertise is significant, but that is informed by the uneven historical development of health services around white, urban populations that began during the colonial era, continued during apartheid, and has not been resolved during the post-apartheid era. Also, HIV/AIDS programs were developed earlier in the province under the guidance of Fareed Abdullah and his team, particularly with PMTCT, which national government intervened to stop during the dark days of the Mbeki era. But support from the Global Fund in the province, which came earlier than the rest of the country, set up the Western Cape as an early success story and a province that has continued to exhibit stronger relative program management than others. Also, I believe that a similar process had already been undertaken with a Global Fund grant in the late 2000s, so there would have been institutional knowledge on how to manage the transition of donor programs from the government to an NGO.

In your last sentence I assume you mean transition from NGO to government?

I consulted with the WCDoH, who said there was never a formal transfer of Global Fund human resources or programming over to government. Global Funded human resources applied for WCDoH posts when they were advertised.

______________________________________________________________________

Line 194

I hope that the financial aspects of the transition are not left out entirely, as that would weaken the robustness of the analysis considerably.

We analyzed PEPFAR expenditure figures for South Africa from 2007-2015.

While this allowed us to see the dip in funding from 2012-2014 in 2015 PEPFAR funding increased again when the Global AIDS Coordinator decided to fund direct service again.

Figure 9: PEPFAR and SAG Expenditure HIV and TB

2007-2015 (ZAR)*

Source:

2007-2010: South African Consolidated HIV and TB Spending Assessment 2007/8-2009/10

2011-2014: South African HIV and TB Investment Case, Reference Report

2014-2016: Consolidated HIV and TB Spending Assessment 2014/15-2016/17

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 200

In terms of defining RIC, a bit more would be helpful. I’m assuming that you are referring to people living with HIV/AIDS (PLHIV) who were adhering to treatment that were lost to follow up (LTFU)/did not adhere after the transition on Pepfar programs? Clarifying this would be helpful.

The definition of RIC used in this study is also used by the WCDoH. RIC was calculated per health facility per year among adults (age >15),

First line + Second line + Third Line + Clients stopped ART / (Total on treatment – Total transferred out)

“Total on treatment” includes the HIV clients who transferred into the health facility, via a formal or silent transfer. Silent transfers were considered new ART initiates, in the absence of a patient tracking system. Mortality dropped out of the RIC calculation. A decision was made based on a sensitivity analysis that an alternative RIC definition would not significantly change the outcome.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 216

Define primary healthcare (PHC) for first use

I edited PHC. Thank you.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 232

Of what level were the staff nurses? Clinic Nurse Practitioners would be the assumption, but please define that for the reader.

The level of nurse was 1 staff nurse and 1 clinical nurse practitioner. I noted these distinctions in the paper. See lines 287-289.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 253

Please define modified grounded theory

This was also a misunderstanding on my part. It should just be grounded theory.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 274

Defining the donor as an NGO is confusing here, since Pepfar is a donor program funded by the US gov’t. If you are referring to a primary recipient organization (PRO) that is working with secondary recipient organization (SRO), then that needs to be defined. Also, Pepfar, the Global Fund, and several other major global health programs are public-private partnerships (PPPs), so thinking through how you define these relationships is important, as simply labeling the donor and NGO limits the applicability and impact of your findings. Also, is the grantee always local government? This section needs to be thought through much more.

Thank you for this comment. I edited my paper referring to either “local government, NGO or donor (ie.PEPFAR).

¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 277

Were there any parameters for defining what the coordinating position would be enabled to do/oversee? This seems a bit general and undefined since it is the key point in the section.

This person would be responsible for ensuring the transparency of donor funded activities and work with government to ensure the program is integrated into the local health system. I clarified this in the paper.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 280

Donor funded organization? Seems like there is a missing word here

This has been edited.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 286 - 288

This is an important point that you are making, but it is not sufficiently contextualized. What you are observing is that the responsibility for a successful transition in skill transfer is decentralized and falls to the level of a facility manager. It would lead one to assume that the better capacitated facilities would therefore be better suited to have successful transfers. Since the areas with highest HIV prevalence and greatest need for skills transfer (the peri-urban townships) also tend to have over-burdened heal facilities, this is a critical point that should be further contextualized.

To clarify, my point is that the skills transfer needs to both centralized at a provincial level and decentralized at a facility level. The provincial level is needed so that essential donor funded staff are transferred to the public health system, which needs coordination in terms of adequate budgeting. At a health facility level, Health Facility Managers should ensure that donor funded staff who are being let go train local health facility staff in their job responsibilities.

I clarified this point in the paper.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 303

Define NIMART for the reader

Edited. Thank you

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 314 – 31

Again, this had already been done by the WCDoH previously, with the transition of clinic management for the HIV/TB clinics established by Médecins sans Frontières (MSF) in Khayelitsha. There were some bumps in the road with these transitions, and it was a learning process. So, again, the success here is not surprising but the result of previous experience in navigating precisely this sort of transition.

My understanding is the MSF work in Khayelitsha Site B for the past 20 years. They have worked in the same health facility. I know that they have conducted research on transitioning patients into adult ART treatment programs. I am aware of their work on adherence clubs, which was later adapted and scaled up throughout the province.

The WCDoH did not have experience transitioning programs and human resources as vast as PEPFAR. PEPFAR was very different in that they also were not transparent with their work so the WCDoH was not aware of the vast amount of programming in the province. PEPFAR was supporting 435 human resources in the Western Cape in 2012. So yes, the WCDoH may have had some experience transitioning some patients into the health system, but it was not at the scale of the PEPFAR experience.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 331

Again, the fact that the extra labor associated with ensuring a sustainable transition is being transferred to the facility level is really an important point. It is not being managed by the donor, provincial health, or city health, but by the facility manager. There would be a huge range of outcomes to be expected then, which would depend not only on the personal attributes of the facility manager, but the extent to which the facility is fully staffed, operational, etc. so that the manager has the ability to focus on the transition. Also, I would assume that CNPs would play a critical role in facilitating this transition, as they often have the clearest understanding of staffing needs, shortages, and areas where increased efficiencies can be achieved. Their labor, however, is often rendered invisible in this process, which is problematic.

These are all very good points and are highlighted in the Conclusion section.

_____________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬_______________________________¬¬¬¬¬_________________

Line 352

Define IMCI

Noted and edited.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 391

It is helpful that you are addressing the contextual factors here, but this is insufficient to frame your findings, which are quite particular to the Western Cape.

Point taken this will be noted in the conclusion ___________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬_______________________________

Line 396

It would be helpful to mention the role of HIV/AIDS activism in producing this change, as this was critical to enabling the shifts you identify.

Good point added to sentence about political support.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 400

There is a long-standing debate on the limits of vertical, disease-specific interventions and their lack of sustainability. Proponents of using “vertical” interventions to strengthen the broader health system (horizontal) interventions have advocated for doing precisely what you advise here, to create “diagonal” programs that use vertical funding streams for health systems strengthening. Health Alliance International (HAI) has done work that has modeled this approach in Mozambique is one of the most significant examples of the potential impact and success of this approach in the global health. In general, I would recommend linking your case study and discussion of findings to the global health literature, as your case study is constructed as a stand-alone example, when in reality it is part of a broader conversation on how best to channel donor funding to improve health outcomes.

As the reviewer states this is a long running debate and will be addressed again in the conclusion

_____________________________________________________________________

Line 411

Is it because facilities are smaller or that they may be struggling to meet the level of need in communities with high burdens of disease?

The reason for this is usually smaller health facilities with few staff are not able to let health staff just focus on one disease. So the donor funded HIV staff will usually get absorbed into the facilities and end up being a generalist, attending to all patients. This point was added to the conclusion.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 422

Does it make sense to mention the 90-90-90 goals for the first time in the conclusion? If this is the aim of the paper, then it would make sense to introduce this goal (which we are projected to miss significantly by the way) earlier in the paper.

I added a section at the beginning (line 260-264) regarding the 90-90-90 goals.

____________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬_______________________________

Line 437

Who are national stakeholders? Does they include civil society and PLHIV or HIV/AIDS activists?

Also, shouldn’t provincial government work with facilities and communities to understand local needs?

I defined these points in the paper.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 452

What are local champions? Who defines needs? What is the role of the community in this process?

I defined these points in the paper in Table 4 .

_____________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬_______________________________

Line 461

It might be useful to include the provincial treasury in the key stakeholders meetings, since presumably they will need to plan several years in advance if a transition will create greater budgetary demands for the health sector. The medium term expenditure framework (MTEF) requires such advance planning for budgetary processes.

I defined these points in the paper (see Table 5).

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 465

I would include the community or some proxy thereof in the final box in this section. The role of HIV/AIDS activists as counsellors and mentors who were also consulted by the WCDoH early on in the development of HIV/AIDS programs was vital to their success.

Thank you. I added this detail to the paper.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 471

Who is funding the staffing/capacity required to develop the transition plan? Is this being done by external consultants?

Thank you for this comment. I clarified this point in the paper.

_______________________________________¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬-_______________________________

Line 472

The recommendation to have the skills transfer managed at the provincial level is contradicted by your evidence, which showed that facility managers oversaw this process.

This is not what I was conveying. Please see my comment from lines 286 – 288 above

Line 474

Given that the entire focus of your paper is to emphasize the importance and lack of research on how care is affected by a donor transition, I am very surprised that there is not a post-transition phase for research or monitoring/evaluation. Your proposed flow of program transition would therefore reproduce the precise issue that your paper aims to rectify.

This is a very good point. I have added a section that includes the post-transition time period (see Table 8).

________________________________________

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Theodore Powers

Submitted filename: Responses to Reviewers .docx

Click here for additional data file.

7 Apr 2021

PONE-D-20-37215R1

Program Sustainability Post PEPFAR Direct Service Support in the

Western Cape, South Africa

PLOS ONE

Dear Jessica Chiliza,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel your submission has been strengthened and requires minor revision.  Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.  We appreciate your efforts to deepen our community's understanding of the process of sustainability as it aligns with external/PEPFAR funding in the context of South Africa.

Please submit your revised manuscript by 21 April 2021. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Melissa Sharer

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: N/A

Reviewer #2: The authors have significantly strengthened the paper through revision, and my recommendation is that the article be accepted for publication pending minor revisions, which are detailed below.

Line 75

This study suggests

Line 101

In 2008, or following 2008?

Line 111

Missing word after “increased”

Line 119

PEPFAR misspelled. Also, “after PEPFAR withdrawal” or “after the withdrawal of PEPFAR funding”

Line 122

Insert “and reductions in” prior to “tracing systems”

Line 171

It looks like there is an incomplete sentence here with “The donor community capacity”

Paragraph starting on Line 197

This is an excellent addition to the argument.

Section starting on Line 203

This is very helpful for situating the particularity of the Western Cape.

Paragraph starting on Line 253

It might be useful to signal that the 90-90-90 rhetoric emanating from the UN has coincided with the leveling off of donor funding and the “transition” processes initiated by PEPFAR and the Global Fund. In short, while we have the tools to “end HIV/AIDS”, funding that would have otherwise supported this approach has been withdrawn, which has undermined this program. Perhaps this would fit best in the conclusion, but it would be worth mentioning, as this critical dynamic seems to be lost on many.

Line 350

Consider rewording to: “understand the context, and local policy, and have…”

Line 351

Consider rewording to: “to the context, which builds trust and results in more”

Line 356

Consider rewording to: “and at a decentralized level.”

Line 450

Consider rewording to: “centralized level: either the provincial or district level.”

Line 455

Consider rewording to: “important for government and the NGO to be”

Line 467

This interview excerpt has already been used (Line 419). Please delete one of these so that there is not repetition.

Line 491

Consider rewording to: “the main concern is that if the public-sector workforce and infrastructure are undermined”

Line 511

Start new paragraph with: “Donors”

Line 546

Consider rewording to: “need to be intentionally”

Line 554

Under “Grantee”: “Ideally have an Establish a donor coordination”

Line 577

Perhaps consider including a wider array of inputs on the transition planning process. Certainly, while the donors will appreciate the objectivity of an external consultant, this is a critical moment in ensuring the long-term sustainability of the program, and there should therefore be a clear and thorough consultation process that involves the full range of stakeholders.

**********

16 Apr 2021

Responses to Reviewers #2

Dear Plos One,

Please find my responses in Bold to the reviewers.

Thank you.

Jessica Chiliza

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Reference #39 was added to include the decrease in global HIV funding as 90-90-90 targets were implemented.

Additional Editor Comments (if provided):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

________________________________________

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

________________________________________

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

________________________________________

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

________________________________________

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: N/A

Reviewer #2: The authors have significantly strengthened the paper through revision, and my recommendation is that the article be accepted for publication pending minor revisions, which are detailed below.

Line 75

This study suggests

Noted and edited.

Line 101

In 2008, or following 2008?

Noted and edited.

Line 111

Missing word after “increased”

Noted and edited.

Line 119

PEPFAR misspelled. Also, “after PEPFAR withdrawal” or “after the withdrawal of PEPFAR funding”

Noted and edited.

Line 122

Insert “and reductions in” prior to “tracing systems”

Noted and edited.

Line 171

It looks like there is an incomplete sentence here with “The donor community capacity”

Noted. It looks part of my sentence was missing. Thank you.

Paragraph starting on Line 197

This is an excellent addition to the argument.

Thank you.

Section starting on Line 203

This is very helpful for situating the particularity of the Western Cape.

Thank you.

Paragraph starting on Line 253

It might be useful to signal that the 90-90-90 rhetoric emanating from the UN has coincided with the leveling off of donor funding and the “transition” processes initiated by PEPFAR and the Global Fund. In short, while we have the tools to “end HIV/AIDS”, funding that would have otherwise supported this approach has been withdrawn, which has undermined this program. Perhaps this would fit best in the conclusion, but it would be worth mentioning, as this critical dynamic seems to be lost on many.

Thank you. I added a sentence on line 253 and in the conclusion with regard to this decrease in funding.

Line 350

Consider rewording to: “understand the context, and local policy, and have…”

Noted and edited.

Line 351

Consider rewording to: “to the context, which builds trust and results in more”

Thank you. I edited the whole sentence (lines 350 and 351) to the following:

“When an NGO has an established office in the geographic region, they understand the context, local policy and have strong relationships with government , which builds trust and results in more sustainable outcomes.”

Line 356

Consider rewording to: “and at a decentralized level.”

Noted and edited.

Line 450

Consider rewording to: “centralized level: either the provincial or district level.”

Noted and edited.

Line 455

Consider rewording to: “important for government and the NGO to be”

Noted and edited.

Line 467

This interview excerpt has already been used (Line 419). Please delete one of these so that there is not repetition.

Oh my! Thank you very much for picking this up. Noted and Edited.

Line 491

Consider rewording to: “the main concern is that if the public-sector workforce and infrastructure are undermined.”

Noted and edited.

Line 511

Start new paragraph with: “Donors”

Noted and edited.

Line 546

Consider rewording to: “need to be intentionally”

Noted and edited.

Line 554

Under “Grantee”: “Ideally have an Establish a donor coordination”

Edited. Thank you!

Line 577

Perhaps consider including a wider array of inputs on the transition planning process. Certainly, while the donors will appreciate the objectivity of an external consultant, this is a critical moment in ensuring the long-term sustainability of the program, and there should therefore be a clear and thorough consultation process that involves the full range of stakeholders.

Thank you for this comment. I have added a few additional inputs.

________________________________________

Submitted filename: Response to Reviewers #2.docx

Click here for additional data file.

23 Apr 2021

Program Sustainability Post PEPFAR Direct Service Support in the

Western Cape, South Africa

PONE-D-20-37215R2

Dear Dr. Chiliza,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Melissa Sharer, PhD MPH MSW

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

14 May 2021

PONE-D-20-37215R2

Program Sustainability Post PEPFAR Direct Service Support in the Western Cape, South Africa

Dear Dr. Chiliza:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Melissa Sharer

Academic Editor

PLOS ONE

Table 4

Beginning of the program.

Provincial Level
Donor	Respect the needs and opinions of the grantee.
Donor	Has the skills to fill the needs of the grantee.
Donor/Grantee	Recognize it will take extra time to coordinate donor funds.
Donor/Grantee	Understand the importance of human resource stability since it affects outcomes
Donor/Grantee	Transparency of program activities and resources including budgets.
Grantee	Needs local champions to keep motivation high.
Grantee	Characteristics of leader who is based at the lowest donor/grantee interface• Takes ownership of program staff and communicates clear roles and responsibilities• Empathy for patients and staff• Creates strong teams• Able to motivate/incentivize• Uses data to make decisions• Plans for the future• Good communication skills• Understands needs of facility

Table 5

Continued: Beginning of the program.

Provincial Level
Donor/Grantee	Consultation with a wide range of stakeholders to coordinate donor funded program (i.e. Provincial treasury, civil society, leaders from provincial, district, sub-district and health facility, HIV activists).
Donor/Grantee	Donor-funded liaisons are placed in national and provincial offices to assist with program implementation and coordination.
Donor/Grantee	Develop a program roadmap with clear timelines. Define and communicate overall goals, outcomes and coordination processes of donor-funded program.
Donor/Grantee	Develop a program implementation plan with all stakeholders. Define sustainability requirements. Not every activity must be sustained.• Align donor salaries with local salaries• Cost the program• Prioritize the funding of extra resources and human resources in smaller clinics.• Consider program beneficiaries and transience of different types of staff.• Consider the importance of community health workers and administrative support
Donor/Grantee	Develop an M&E plan for the program
	Align donor program indicators and staffing structures with local system.

Table 6

Mid-term.

Provincial Level
Donor/Grantee	All stakeholders discuss policy, budget, program, donor, local contextual changes and challenges facing the program
Donor/Grantee	Look for ways to create partnerships between government and/NGO, and between NGOs.
Donor/Grantee	Continuation of coordination meetings with grantee at lowest grantee/donor interface• NGO and health facility• Donor-funded staff and local staff• Provincial level government officials• HIV/AIDS activists and community leaders

Table 7

Transition period (final 2–5 years before program transitioned).

National Level
Donor/Grantee	Official transition plan developed by consultants with input from a wide range of stakeholders and funded by the donor
	• High level plan• Implementation plan
Donor/Grantee	Political commitment to the transition, which includes a financial commitment.
Provincial Level
Donor/Grantee	Grantee leads review process of program outputs and outcomes to assess program effectiveness
Donor/Grantee	If patients are moving from NGO care to the public system, develop a tracking system to monitor progress.
Donor/Grantee	• Formalize the skills transfer, which should be coordinated at centralized and decentralized levels.• Prepare the public health system to absorb donor funded activities and staff
Donor/Grantee	Review M&E data.
Donor/Grantee	Clear communication with all stakeholders regarding M&E updates and budget timelines
Grantee	Develop local transition plan.
	• Stakeholders should decide what they can realistically sustain within their budgets.• Review all donor funded activities• Possibility to use a staggered approach to absorb donor- funded resources
Donor	Provide capacity and technical assistance where needed