Clinical Referral Laboratories in Rwanda.

OBJECTIVES: We investigated the quality system performance in Rwandan referral laboratories to determine their progress toward accreditation.
METHODS: We conducted audits across five laboratories in 2017, using the Stepwise Laboratory Quality Improvement Process Towards Accreditation checklist. Laboratories were scored based on the World Health Organization grading scale (0-5 stars scale) and compared with earlier audits.
RESULTS: Between 2012 and 2017, only one laboratory progressed (from four to five stars). Four of the five laboratories decreased to one (three laboratories) and zero (one laboratory) stars from four and three stars. Management reviews, evaluation, audits, documents, records, and identification of nonconformities showed a low performance.
CONCLUSIONS: Four of five laboratories are not moving toward accreditation. However, this target is still achievable by energizing responsibilities of stakeholders and monitoring and evaluation. This would be possible because of the ability that laboratories showed in earlier audits, coupled with existing health policy that enables sustainable quality health care in Rwanda. © American Society for Clinical Pathology, 2018.

Clinical laboratories are vital in health systems and play a critical role in the quality of health care.[1-3] Accurate and reliable laboratory test results are key factors in guiding decision making and evidence-based actions for effective prevention, control, and surveillance of diseases as well as patient management.[1,2,4-8]

Laboratories in low-income countries, especially in sub-Saharan Africa, face a challenge of insufficient resources for quality improvement, despite their crucial role in supporting the health system.[9-11] In addition, the region has the lowest rate of international accredited laboratories and the highest burden of disease.[11-14] This situation creates an imbalance in terms of demand and provision of reliable laboratory services.[15]

The compliance with standards of the quality of health care is verified through the international accreditation of health facilities, including clinical laboratories.[5,16,17] To respond to the need of quality laboratory services as well as working toward international accreditation in low-income countries, a World Health Organization (WHO)–Africa region stepwise approach has been adopted as an effective strategy to enhance health care quality.[8,18,19] The approach is formed by the Strengthening Laboratory Management Towards Accreditation (SLMTA) program. The program is based on a series of trainings and mentorship of laboratory staff in laboratory quality management system improvement. The progress in quality improvement is measured using a Stepwise Laboratory Quality Improvement Process Towards Accreditation (SLIPTA), a checklist tailored on International Organization for Standardization (ISO) 15189.[19-22]

Since December 2017, 49 countries, including Rwanda, have implemented the SLMTA program,[23] and by 2014, 617 laboratories from 47 countries were enrolled in the program.[24] Published data of baseline and exit audits from these countries are encouraging.[24-26] Here, the baseline and exit audits are performed before and after (respectively) a series of training in SLMTA and mentorship in quality improvement projects implementation, whereas follow-up audits are performed after this period. In addition to the quality improvement registered between baseline and exit audits, laboratories are recommended to continuously apply laboratory quality systems to achieve international accreditation.[27] To be eligible for international accreditation, a laboratory in the SLMTA program is required to score 95% or more out of 275 points on the SLIPTA checklist audit.[22,28] This score is equivalent to five stars on the WHO grading scale of 0 (<55%) to five stars (≥95%), and the awarded score in this WHO system is valid for 2 years.[28]

The Ministry of Health of Rwanda started a process of quality improvement for its health facilities to achieve their accreditation.[29,30] On the front line of the process, there were four national referral hospitals alongside their respective laboratories, as well as the national reference laboratory, with the mandate of supporting their subordinate ones. Therefore, the quality performance of these hospitals and laboratories will influence those under their supervision. In Rwanda, the SLMTA program started in January 2010 with a baseline audit of these five clinical referral laboratories as a first cohort.[25] This was followed by exit and follow-up audits, respectively, in 2011 and 2012.[25] Then, progressively, other laboratories were enrolled with the ultimate goal of covering all hospital laboratories in Rwanda.[25,31] These five clinical referral laboratories demonstrated quality system improvement between baseline and exit as well as in the follow-up audits.[25] However, no available data after 2012 evaluate quality improvement toward accreditation. Such information is needed to evaluate the progress and guide new strategies for sustainability and further improvement.

The aim of our study, 7 years after the start of the program and 5 years after the follow-up audit, was to assess the level of quality performance of the five clinical referral laboratories in Rwanda. The study would determine whether they are progressing toward international accreditation or if they are declining. The findings of the study will inform policy and practice on strengths and weaknesses to work on toward laboratory accreditation and quality of health care improvement.

Materials and Methods

Study Design

To determine the quality performance of laboratories, we performed a cross-sectional systemic audit using the SLIPTA checklist (version 2015)[22] across all five clinical referral laboratories, and their performances were compared with previous audits held in 2010, 2011, and 2012.

Settings

The five clinical referral laboratories (laboratories 1-5) are composed of four laboratories belonging to national referral and teaching hospitals and one national reference laboratory (NRL). The NRL supervises the four and performs quality control of laboratory tests from district laboratories (especially human immunodeficiency virus, malaria, and tuberculosis tests), as well as disease surveillance and outbreak investigation.

Data Collection

Audits were conducted in April and August 2017 by authors qualified and certified by the WHO, US Centers for Disease Control and Prevention (CDC), and African Society of Laboratory Medicine (ASLM) in SLMTA and use of the SLIPTA checklist. We conducted the audits in teams of three persons for each laboratory, but no one audited his own institution. Prior to the audit, the team met for 1 day to review the tool and agree on the approach of assessment. We visited laboratories and discussed with laboratory leadership and staff to inform them on the research. We assessed each laboratory over 3 days through observation of processes and operations as well as working environment through reviewing documents and records, following specimens through the laboratory, interviewing relevant staff for additional clarifications, and other procedures as described in the tool.[22]

Analysis

We scored each laboratory with the WHO grading scale of zero to five stars for a percentage of performance (<55%, 55% to <65%, ≥65% to <75%, ≥75% to <85%, ≥85% to <95%, and ≥95%, respectively).

The overall performance of each laboratory was calculated by summing the performance of 12 different SLIPTA sections, which are based on 12 quality system essentials, and the percentage was calculated for comparison. The performance in the 12 sections was compared within each laboratory and between all laboratories. The trend of performance since 2010 was made using the 2017 score as well as the scores of 2010, 2011, and 2012. Charts were made with GraphPad Prism 7 (GraphPad Software, La Jolla, CA) and Microsoft Excel, version 2016 (Microsoft, Redmond, WA).

Ethical Considerations

The research proposal was presented, ethical clearance (No. 0059/RNEC/2017) was obtained from the Rwanda National Ethics Committee, and approval was acquired from the Ministry of Health and Ministry of Education prior to its implementation. Additional permissions were requested from leadership of institutions to which the evaluated laboratories belonged. The research project was presented and discussed in study sites prior to data collection, and feedback was provided at the end of the assessment.

Results

The findings are displayed in three sections: the overall performance of the laboratories in 2017, the quality system performance in 12 quality system essentials, and the evolution of quality performance of laboratories from 2010 to 2017.

Overall Laboratory Performance in 2017

All five laboratories were enrolled in the SLMTA program as the first cohort in Rwanda. The cohort was audited with the SLIPTA tool in 2010 (baseline), 2011 (exit), and 2012 (follow-up), and we performed our audit in 2017 using the SLIPTA tool. The findings of the 2017 evaluation showed that the laboratory that performed best scored 95.3%, which corresponds to five stars on the WHO grading scale. The laboratory that showed the lowest performance scored 52%, which is equivalent to zero stars. The remaining three laboratories reached the one-star level Figure 1.

Figure 1

Overall quality performance of referral laboratories in Rwanda assessed in 2017 using the Stepwise Laboratory Quality Improvement Process Towards Accreditation checklist.

Performance in 12 Quality System Essentials

For quality system essentials, our assessment found that certain determinants influenced the overall result. Management reviews, evaluations and audits, documents and records, and identification of nonconformities showed a low performance in the laboratory quality system in four laboratories. The facility and biosafety, purchasing and inventory, client management, and customer services as well as occurrence and incident management had a positive trend in almost all of the laboratories. One laboratory showed good performance in all quality system essentials Figure 2.

Figure 2

Quality performance in 12 quality system essentials of referral laboratories in Rwanda assessed using the Stepwise Laboratory Quality Improvement Process Towards Accreditation checklist in 2017.

Evolution of Quality Performance From 2010 to 2017

When comparing the performance of the latest follow-up in 2012 and our evaluation in 2017, the current findings indicated progress of only one of the laboratories, which scored four stars in 2012 and currently scored five stars. The remaining four laboratories declined in their performance, as three of them scored one star and one scored zero stars, while they scored four and three stars in 2012, respectively Figure 3.

Figure 3

Evaluation of quality performance of referral laboratories in Rwanda, 2010 to 2017.

Discussion

The Health Sector Policy and the Health Sector Strategic Plan of Rwanda emphasize the quality of health care as their major priority.[29,30] The accreditation of health facilities, including laboratories, is highlighted as the strategy to achieve the desired quality health care status.[29,30] Therefore, the Ministry of Health enrolled in the first cohort of SLMTA in 2010 its five clinical referral laboratories as a process toward accreditation and quality improvement.

This study showed the quality system performance of the laboratories in 2017, 7 years since 2010, when the program started, and 5 years since the last audit in 2012. The 2017 audits showed that one laboratory scored zero stars and three others one star, whereas one laboratory scored five stars. When comparing with the previous audits, we showed that 5 years after the latest stepwise approach toward accreditation, most of the Rwandan clinical referral laboratories (four of five) had decreased in quality performance. While the general performance decreased for the four laboratories, some common patterns like management review, document and records, identification of nonconformities, and evaluation and audits critically affected the performance of the laboratories.[26] These patterns are cornerstones of the entire laboratory quality improvement system. The good or poor performance in these cornerstones affects positively or negatively the quality system, respectively. The fact that most of these laboratories were affected systematically in the same quality system essentials suggests system problems that need to be identified and addressed.

The 2011 exit audit, which marked the end of SLMTA trainings, had shown improvement in quality performance in all laboratories compared with the baseline of 2010.[25] The follow-up audit performed in 2012 also showed continuous improvement in all five laboratories.[25] The audit of 2012 for five laboratories was planned as a follow-up of the 2011 one and after 12 months of addressing identified gaps. Here, we showed that this dynamic coordinated action of identifying gaps and planning how to address them led to positive results.

The continuous quality improvement score of the high-performing laboratory could be explained by the fact that the laboratory is enrolled in the East Africa Public Health Laboratory Networking Project, which provides financial support and regular audits that helped the laboratory to identify and address gaps.[31] The four other laboratories had each evolved on their own since 2012. Two laboratories (laboratory 2 and laboratory 3) had only occasional SLMTA/SLIPTA activities, whereas the two remaining laboratories (laboratory 1 and laboratory 4) had no SLMTA/SLIPTA-related activity in the same period. Reevaluating and applying lessons learned in the circle of planning as well as effective monitoring are the gold standards to achieve the desired quality of health care. Our findings corroborated with this principle by the fact that most of the assessed laboratories were declining in their quality performance, and insufficient follow-up was one of the contributing factors. Further research exploring factors regulating the observed quality performance status of clinical referral laboratories in Rwanda may find additional factors.

Rwanda is well known to be effective and efficient in implementing sociodevelopment programs, especially health-related reforms to improve population health outcomes.[32-35] Therefore, it should be possible for these laboratories to achieve and sustain international accreditation and then mentor lower-level laboratories. This possibility is based on laboratory ability demonstrated at the beginning of SLMTA program and the political commitment of the country. It requires a redefinition of responsibilities among national stakeholders and establishment of regular audits as well as an effective monitoring and evaluation framework.

The clinical referral laboratories of Rwanda may not be an isolated case among more than 617 laboratories enrolled in SLMTA in different countries.[24] While a national coordination and effective monitoring and evaluation mechanisms are recommended at the country level, such regional monitoring needs to be streamlined for effective synergy toward international accreditation and quality of health care as a target of the third sustainable development goal, with “no one left behind.”[36,37]

Even if the findings from this study can provide some insight for other countries with a similar context, the results cannot be generalized. Furthermore, readers of this article could also keep in mind that the audit of 2017 used the 2015 SLIPTA version based on ISO 15189:2012, while previous audits used the version of SLIPTA aligned to ISO 15189:2007. However, the difference in these two versions is too minor (17 points), which cannot explain the observed difference in quality performance while principles remained the same.

Conclusions

This study demonstrated that almost all assessed laboratories decreased in quality performance and, therefore, did not achieve the level of accreditation. Our findings highlighted that management reviews, evaluation and audits, documents and records, and identification of nonconformities constituted the bottlenecks that severely affected the quality system of the laboratories. In addition, the study showed the need for regular follow-up, periodic retraining, and external audits to support laboratory quality improvement in low-income countries. Based on the political will of Rwanda and the ability demonstrated by the assessed laboratories in earlier audits, their international accreditation is possible by energizing the responsibilities of national stakeholders and setting up a monitoring and evaluation mechanism.