Introducing health facility accreditation in Liberia.

In recent years, dozens of countries have introduced accreditation and other quality improvement initiatives. A great deal of information is available regarding best practices in high- and middle-income countries; however, little is available to guide developing nations seeking to introduce an accreditation programme. This paper describes the outputs and lessons learned in the first year of establishing an accreditation programme in Liberia, a developing nation in West Africa that in 2003 emerged from a brutal 14-year civil war. The Liberian experience of developing and implementing a government-sponsored, widespread accreditation programme may provide insight to other low-income and post-conflict countries seeking a way to drive rapid, system-wide reform in the health system, even with limited infrastructure and extremely challenging conditions.

Introduction

For more than a decade, the World Health Organisation (WHO) has urged states in Africa to establish national quality improvement programmes as part of health sector reforms (WHO 2003). Experience with quality improvement demonstrates its potential positive influence on both management and clinical outcomes in low-income settings (Berwick 2004, Bradley et al. 2008, Campbell et al. 2008, Groene ). Furthermore, accreditation programmes may increase equity across health systems by fostering incremental improvements in quality (Montagu 2003). Nevertheless, most quality improvement efforts described in the literature focus on changes in an individual health facility or in a set of facilities (WHO 2003, Kebede ), rather than system-wide reform.

A potential method of fostering sustained facility-level improvements and larger system-level change is the establishment of an accreditation process for health facilities. Although accreditation programmes take a variety of forms, accreditation is typically a formal process of assessing the degree to which health facilities meet predetermined standards pertaining to quality and availability of services (Rooney and van Ostenberg 1999). Assessments are usually undertaken by an independent body, such as a non-governmental organisation (NGO) or industry-sponsored organisation (Rooney and van Ostenberg 1999, Montagu 2003), and the process provides a mechanism by which health care facilities can target areas for improvement. While accreditation is strongly linked to facility-level quality improvement efforts (Rooney and van Ostenberg 1999), it can also facilitate the monitoring of trends and evaluation of broader health systems.

Successful accreditation systems have existed for decades in high- and middle-income countries; however, the establishment of accreditation systems in low-income countries has been slow, particularly in sub-Saharan Africa. Two low-income sub-Saharan countries, Zambia and Uganda, have developed or begun to develop hospital accreditation systems and published findings (Bouchet , Luboga and Barnhart 2008). The Zambia accreditation system was developed in 2001 by the Ministry of Health with technical support from the USAID Quality Assurance Project (Bouchet ). Although the accreditation reported significant improvements in scores and significantly increased compliance with standards in the majority of the assessment's functional areas (Bukonda ), it operated for only 1 year as it was found to be too resource-intensive to sustain, costing nearly US$10,000 per hospital to implement. The Ugandan accreditation process is now developing standards and has not yet been implemented. Therefore, while accreditation is a common element in sustaining high-quality care in middle- and high-income countries, there are few published accounts or models for how best to implement accreditation in low-income settings.

Accordingly, this paper describes the process and early outcomes of establishing an accreditation programme for health facilities in Liberia, a post-conflict, sub-Saharan country with 437 open health facilities and a population of 3.4 million people as of 2009. The Liberian accreditation system was designed and implemented by the Ministry of Health and Social Welfare of Liberia (MOHSW) in collaboration with the Clinton HIV/AIDS Initiative (CHAI) as part of broader efforts to implement a Basic Package of Health Services (BPHS). The system was developed not to evaluate quality, but rather to determine the degree to which all health facilities are meeting the required BPHS clinical standards as well as management standards necessary to provide those services. The programme includes primary and secondary facilities (health clinics and health centres) in addition to tertiary hospitals. Although the accreditation process tracks BPHS implementation at the facility, county and national levels, it does not explicitly track quality improvements or measure quality of care. However, the accreditation process evaluates the existence of services and critical management systems necessary to provide quality services and can therefore shed light on areas in need of improvement.

Due to the extremely dire state of the health system resulting from the 14-year civil conflict in Liberia (1989–2003), the accreditation programme was designed to drive rapid system-wide improvements in service provision. An understanding of the innovations and challenges in the development and early implementation of health facility accreditation in Liberia may be helpful to other low-income countries considering accreditation as a component of health system reform.

Setting

The prolonged civil conflict in Liberia severely disrupted the nation's health care system. Trained clinicians fled the country, looting and fighting destroyed health facility infrastructure, supply chains were cut off, and medical training was interrupted for extended periods of time. Under-five mortality climbed to 235 per 1000 (WHO 2006), while maternal mortality rose to 994 per 100,000 live births (Liberia Institute of Statistics and Geo-Information Services (LISGIS) (Liberia) et al. 2008). With the democratic election of President Ellen Johnson Sirleaf in 2005, Liberia ushered in a new era of stability and optimism. In early 2007, the MOHSW released a new National Health Policy and National Health Plan (2007–2011), including the BPHS, which defined standardised services to be offered at health facilities of all levels. All facilities, including those supported by NGOs, are now required to provide full BPHS services free of charge. Furthermore, the MOHSW prioritised implementation of the BPHS at 40% of functional government-owned health facilities (called Fast Track Facilities) in each of Liberia's 15 counties; 70% will be targeted for BPHS implementation by December 2010.

The problem

Fragmented health system

As recently as mid-2007, the MOHSW did not have basic information about health care in Liberia. For example, the number, location and qualifications of government-employed health care workers remained unknown, as did the location and operating status of many health facilities. The accreditation process brought to light additional information, identifying 437 open health facilities in late 2008, of which 349 are government-owned. As of 2009, more than 70% of the government health facilities are operated on behalf of the MOHSW by faith-based organisations or 1 of 15 international and local NGOs. Despite widespread support from the NGO community for BPHS implementation at government-owned facilities, the provision of health services remained severely fragmented and inconsistent.

Logistical challenges to Basic Package of Health Services (BPHS) implementation and monitoring

The extreme logistical challenges in Liberia made implementing the BPHS and monitoring progress towards achieving national targets very difficult. After 14 years of conflict and more than 20 years of instability, human resources for service provision and management were severely limited, and infrastructure for communication and transport was extremely poor. There were no landline communications in the country, and due to an extremely limited electrical grid that provides power only in parts of central Monrovia, almost all electricity was sourced by diesel generators. Most health facilities were accessible only by rough dirt roads, while some were isolated for months at a time due to washed out roads and flooded rivers such that they could only be reached on foot. Many health facilities were outside cellular phone coverage areas and did not communicate regularly with their County Health Team (CHT) or the central MOHSW. Technology infrastructure and skills, both within central MOHSW and particularly outside the capital, were extremely limited; the majority of data collection and reporting were paper-based as of 2007–2008, with delivery that required timely travel between Monrovia and the counties.

Response: Basic Package of Health Services (BPHS) accreditation process

Purpose and design

The MOHSW developed a compulsory BPHS accreditation process to assess the current status of health care provision and to monitor BPHS implementation. Accreditation standards were developed in 2007–2008 with support from CHAI and faculty at the Yale School of Public Health, based on standards used in other developing countries and the systems needed to implement the BPHS. Distinct standard sets were developed for hospitals, health centres and health clinics.

Nine assessment categories were created, each of which is weighted to calculate the overall facility score. The Health Services category, those services required under Liberia's BPHS, is weighted most heavily at 40%, while the remaining 60% is evenly distributed throughout the following areas: Human Resources and Facility Operations, Pharmacy, Dispensary and Storeroom, Drugs and Supplies, Laboratory/ Diagnostics, Equipment, Communicable Disease & Infection Control, Medical Records and Guidelines and Infrastructure. See Figure 1 for more detail.

Figure 1.

Accreditation categories for assessment of Basic Package of Health Services implementation in Liberia, 2009.

In June 2008, the process was piloted at 184 priority facilities selected by CHTs (Fast Track Facilities, representing 40% of facilities in each county); all open health facilities, both government and private, were assessed in January-February 2009.

All items in the assessment tools were given a score value of 1 or 0 (for standards being met or unmet). See http://www.yale.edu/ghli/documents/LiberiaMOHSWAccreditationStandards.pdf for a listing of BPHS standards. For each assessment category, the percentage of standards met in that facility was calculated. These were then weighted according to a standardised weighting system (see Figure 1) and summed across all assessment categories to obtain the overall accreditation score of the facility. Facilities that received an overall accreditation score of 75% or higher were considered functional, and received a Bronze 1/2 Star. Facilities achieving 85% received a Silver 1 Star, while facilities meeting all of the required standards received Gold 2 Star certification. Figure 2 shows a sample of the summary reports provided to each implementing partner and CHT.

Figure 2.

Accreditation report card for Bong County, Liberia (March 2009).

Performing accreditation assessments

Pre-accreditation assessments were conducted at the 184 Fast Track Facilities in order to provide early feedback on BPHS implementation, as well as to pilot the accreditation process and tools. Based on the pilot and feedback from assessment team members and implementing partners, the assessment process and tools were refined to improve the timing and flow of conducting an assessment and to ensure a standardised long-term scoring system. In January and February 2009, all 437 open health facilities in the country (government and private) underwent accreditation assessments. Over 3 weeks, 15 two-person teams (comprised of MOHSW clinicians and staff donated by partner organisations) conducted site visits and completed assessments based on observations and interviews with staff and patients. Assessors completed assessment tools but did not calculate facility scores; this was done at the central level. On average, hospital assessments required a full day. Health centre and clinic assessments required half-a-day to complete. The assessment tool was created using PenDragon Forms (http://www.pendragon-software.com) and was completed on Palm Treo 680 Personal Digital Assistants (PDAs). Teams used one PDA and a back-up paper copy of the tool based on the BPHS standards to record the assessment data. Data were subsequently uploaded from the field to a central server at the MOHSW via the phone network. The use of wireless data transfer over the phone network eliminated the need for data entry, which prevented potential entry errors and allowed central support staff to troubleshoot and monitor data quality in real time, rather than waiting for teams to report problems or return to the central MOHSW with incomplete data. Each team carried a portable global positioning system (GPS) unit, which was used to plot the location of each facility.

Initial effects of introducing accreditation

Availability of information

The accreditation process resulted in unprecedented amounts of comprehensive, timely and detailed information about the health facilities. These data, which highlight strengths and weaknesses in the delivery system, have the potential to facilitate more evidence-based decision-making and priority setting at all levels of the health system. For example, a major purchase of laboratory equipment was made in February 2009 based on data collected through the accreditation process; this was prioritised due to the relatively low scores in this category across all facilities. The increased availability of information also fostered increased accountability, as needs, priorities and responsibilities can be more clearly defined and monitored at the facility, county and central levels.

Shared understanding of Ministry of Health and Social Welfare of Liberia (MOHSW) priorities and oversight

Through the accreditation process, the MOHSW was able to define and communicate the standards for individual health facilities as outlined in the National Health Plan. This enabled NGOs and health workers at all levels to better understand the criteria against which the MOHSW would judge their performance, and provided a new forum through which these stakeholders may voice opinions and suggestions for overall health system reform. The accreditation process created broad recognition that health facilities, CHTs and implementing partners would be benchmarked against pre-defined and published accreditation levels, which has fostered healthy competition among NGO partners and CHTs to make rapid and significant improvements. Furthermore, all CHTs, facilities and partners receiving an overall Bronze, Silver or Gold rating received certificates of achievement and public accolades by the MOHSW. The CHTs were also formally invited to share their best practices with MOHSW officials in strategy meetings.

The introduction of accreditation has increased these stakeholders’ confidence in the MOHSWs commitment to improve the health system. At the first accreditation assessment, some health facilities reported that supervisors from the central MOHSW had not been on-site in more than 20 years. After years of operating without government oversight or sufficient resources, assessment teams reported that facility staff welcomed the accreditation process as a signal that the government was re-engaged and actively making efforts to improve health delivery systems. Following the June 2008 assessments at select facilities, CHT members shifted available funds and human resources to address priorities identified in the accreditation reports and more forcefully requested that the MOHSW address problems extending beyond the county level (i.e., national supply chains, major infrastructure improvements, etc.), thus demonstrating their support for the process.

In addition, the MOHSW now requires that NGOs achieve at least 85 out of a possible 100% accreditation score prior to transitioning the facility over to MOHSW management. Although it is not yet clear whether there will be concrete repercussions for failure to do so, the accreditation process has enabled the MOHSW to better manage NGO assistance, which has in turn increased NGO accountability and solidified the MOHSW's leadership and authority in a previously fragmented health sector.

Performance-based financing

The accreditation data have contributed to a pilot of performance-based contracting by the MOHSW for NGOs operating government-owned health facilities. In 2009, the MOHSW piloted a pay-for-performance model of contracting in which NGOs running government health facilities will be measured against 15 indicators, one of which is the facility accreditation score. NGOs received a bonus of 0.5% of the contract value for every percent increase in a facility's overall accreditation score, up to 5% of the total value of their contract. Overall, the accreditation scores increased during the pilot period of pay-for-performance contracting. Furthermore, the performance-based contracting has been expanded in Liberia, and the accreditation score was retained as an additional indicator of performance as part of these contracting efforts. The MOHSW plans to extend additional performance-based contracts if this pilot programme proves to be a success.

Lessons learned in the first year of the accreditation process

Benefits of government-initiated processes

Accreditation is typically conducted by industry-sponsored organisations; however, in developing countries where there is a very limited health industry, governmental organisations may be needed to initiate accreditation processes. This is consistent with WHO reports suggesting that most recent quality improvement programmes in developing countries have been government-driven (Berwick 2004). In Liberia, the early results from a government-operated accreditation programme are promising, particularly with regards to implementing priority government programmes such as the BPHS. Government ownership of accreditation strengthened the credibility of the overall process, and facilitated more widespread assessments than would otherwise be possible through a voluntary or NGO-operated accreditation programme. Until there are other organisations with resources to conduct comprehensive, detailed assessments of health facilities in Liberia, MOHSW ownership will be critical to maintaining financial and political support for the process. As the health system develops and the emphasis of accreditation shifts from service provision to quality improvement, it may be more reasonable to transfer ownership to an autonomous NGO or private organisation.

Role of branding the accreditation process

In 2009, branding was used to identify assessment team members and accreditation levels, which strengthened the credibility of the process and helped to foster a culture of healthy competition among facilities and CHTs. During assessments, each team member wore an official MOHSW photo identification card and a vest with ‘BPHS Accreditation Team’ printed on the back, both of which identified him or her as an official MOHSW representative. This branding alerted patients and facility staff to the assessment, which further publicised the MOHSW's efforts to rebuild and improve the health sector. Additionally, branding logos were created to visually represent accreditation scores. Gold, Silver and Bronze logos were used to acknowledge facilities, counties and partners that reached the respective accreditation levels. As accreditation continues, an effort will be made to educate patients about the accreditation levels and provide facilities with widely visible logos to not only motivate staff to reach full accreditation, but also to promote patient awareness and choice.

Assessment team members

It became clear following the first assessments in June 2008 that clinicians were more effective assessors than non-clinicians, as they were familiar with the expected standards for clinical care. In most cases, the MOHSW also selected their own employees as assessors, which has the potential to increase the impact of the accreditation programme: nearly half of the assessors in January and February 2009 were both clinicians and members of CHTs, most of whom were responsible for overseeing the management of primary and secondary health care facilities throughout their county. One of the principal components of an accreditation system is the evaluation of standards by professional peers (Montagu 2003), further supporting the MOHSWs decision to rely on CHT members as assessors. As a result of their participation as assessors in counties other than their own, these health system managers gained a detailed understanding of the assessment tools, the overall accreditation process and cross-county challenges. This knowledge and perspective will be shared with other CHT members and potentially translated into targeted initiatives and more innovative solutions at the county and facility levels to meet accreditation standards.

Stakeholder engagement

Early and frequent communication with all stakeholders, from individual facilities to NGO partners and CHTs, has been crucial to the success of the process. Input from key stakeholders into the accreditation standards and assessment tools was a key component in gaining NGO and CHT support for the accreditation process. Prior to accreditation assessments, the MOHSW conducted extensive communication campaigns to inform stakeholders of the upcoming process. Meetings were held with private facility owners, CHTs and NGO partners to share copies of the assessment tools and schedules, address questions or concerns, and provide background information on the purpose of accreditation. NGO partners and owners of private facilities needed time to understand the accreditation process and review the assessment tools before lending their support.

The MOHSW actively and repeatedly solicited feedback on the accreditation standards and assessment tools from CHTs and NGOs. The opportunity to provide input allowed stakeholders to voice concerns over the level of standards and the implications of not being accredited. This in turn enabled the MOHSW to address their reservations and gain support for the accreditation process. In addition, press releases were published in several newspapers in the capital and public service announcements were aired on radio throughout the country. This widespread communication informed the public of MOHSW efforts to improve the health sector and further motivated NGO partners, CHTs and health facility staff to support the process.

Stakeholder support at the national and county levels greatly improved the ease of conducting assessments both logistically and politically. NGOs loaned vehicles and drivers to the MOHSW for the duration of assessments, and a few organisations also provided staff as assessors. Politically, the implementation of the accreditation programme was a signal to NGOs and private facility owners that the MOHSW is capable of monitoring facilities and holding organisations accountable for their commitments to the Liberian people.

Logistical challenges and planning

Logistical preparations need to begin months in advance of assessments. Wherever possible, facility staff should be informed of the exact date of the assessment to ensure that they are present and prepared; health facilities that for various reasons did not receive communication in advance of the June 2008 assessments were less prepared to provide the necessary information to assessors. However, few teams were able to follow the planned assessment schedules. Extremely poor road conditions made travel times substantially longer than anticipated – in some cases teams walked for over 2 hours to reach a facility – and lack of information at the central level about the location of facilities resulted in unrealistic scheduling. As a result, despite efforts to communicate plans, in some cases the staff at health facilities had no knowledge of the accreditation process when assessment teams arrived. Efforts were made to establish a database of contact information for the facility managers who can be reached by cellular phone. In addition, the MOHSW will establish more realistic schedules for the next assessments by liaising more closely with CHTs and district-level health staff. The use of GPS units in the January 2009 assessments will support this effort and allow for the creation of detailed maps of all facilities.

Pilot assessments and use of technology

The use of PDAs greatly improved the assessment process and demonstrated that even in extremely resource-constrained settings, advanced technology can have a high impact with a relatively low cost. PDAs have been successfully used in other developing and post-conflict countries to facilitate easier data collection and rapid data entry (d'Harcourt and Mulumba 2008). The PDA tools were piloted in Liberia in early December 2008, which was critical to improving the flow and decreasing the duration of assessments. Assessment team members quickly picked up the skills required to use PDAs (training lasted approximately 6 hours), even without extensive computer experience or prior use of the survey software. The PDAs allowed teams to send data directly to the central MOHSW server, which dramatically reduced the time spent on data entry and facilitated easier data analysis and automatic report generation. Though the development of the PDA assessment tool required outside technical assistance, the MOHSW now has more than 30 staff trained in PDA use, and efforts are under way to fully transfer management of the accreditation process to the MOHSW. Due to the initial success of using PDAs, the MOHSW is now exploring other ways in which they might facilitate data collection, routine reporting and other surveys.

Conclusion

The BPHS accreditation process has allowed the Liberian MOHSW to gather unprecedented amounts of information about the current status of service provision. Furthermore, it has galvanised support for BPHS implementation and provided facilities, CHTs and NGO partners with concrete feedback on areas in which improvement is required. At the central level, data are being analysed to identify commonalities between high-performing facilities in order to more strategically recommend targeted, high-impact improvements at the facility and county levels.

The implementation of this accreditation programme has a number of mechanisms by which it influenced changes in health facilities. These mechanisms included engagement of stakeholders in the reform process; involvement of key MOHSW officials in not only the policies, but also the procedures of implementation; and attention to implementation logistics including the adoption of information technologies to facilitate data capture and analysis. As noted by Pawson and Tilley (1997), programme outcomes depend on both context and mechanisms. Although the experience studied here was specific to the Liberian context, the mechanisms identified as important may provide guidance to other low-income countries seeking to improve the performance of their health delivery systems.

It should be noted, however, that the long-term outcomes of the introduction of system-wide accreditation in Liberia remain to be seen. The next round of accreditation (January 2010) will show whether targeted reporting and simplified, standardised scoring have resulted in buy-in and systematic improvements, but it is not yet clear whether the process will lead to the achievement of national BPHS implementation targets and eventually significant improvements in quality of care at the facility level. Currently, many health facility staff are under-qualified for their positions and have little or no management training. The system is overburdened by the shortage of health professionals in the country. Implementing needed systems to meet all accreditation standards (e.g., medical records, procurement, human resource management, budgeting and financial management) may take years to accomplish. Within the next year, the MOHSW will need to determine the longer-term goals of the accreditation process, particularly with regards to ensuring continued progression in service provision and quality of care beyond the current BPHS requirements.

The lessons learned through the establishment of an accreditation programme in Liberia may be useful to other nations seeking to develop a programme through which to monitor service delivery during a period of rapid transformation. These lessons are perhaps most relevant in settings where the government is seeking to assert greater control over a fragmented health system, particularly where there is a need to both ensure minimum standards and introduce more comprehensive requirements that will set a high bar for quality health care in the future.

Acknowledgements

This work was supported by the Clinton Health Access Initiative (CHAI) through funding from the Norwegian Ministry of Foreign Affairs. Funds for the first year of the accreditation process were provided by the Government of Liberia and the Global Alliance for Vaccines and Immunization (GAVI). We thank Guthrie Gray-Lobe for his creation of the accreditation reporting forms.