A novel community health worker tool outperforms WHO clinical staging for assessment of antiretroviral therapy eligibility in a resource-limited setting.

The accuracy of a novel community health worker antiretroviral therapy eligibility assessment tool was examined in community members in Blantyre, Malawi. Nurses independently performed World Health Organization (WHO) staging and CD4 counts. One hundred ten (55.6%) of 198 HIV-positive participants had a CD4 count of <350 cells per cubic millimeter. The community health worker tool significantly outperformed WHO clinical staging in identifying CD4 count of <350 cells per cubic millimeter in terms of sensitivity (41% vs. 19%), positive predictive value (75% vs. 68%), negative predictive values (53% vs. 47%), and area under the receiver-operator curve (0.62 vs. 0.54; P = 0.017). Reliance on WHO staging is likely to result in missed and delayed antiretroviral therapy initiation.

INTRODUCTION

The 2010 World Health Organization (WHO) guidelines recommend that HIV-infected adults with a CD4 count of <350 cells per cubic millimeter be initiated onto antiretroviral therapy (ART).1 Availability of CD4 count measurement remains low in much of sub-Saharan Africa. For example, in Malawi, only 12% of HIV care facilities have functional on-site capacity for CD4 count measurement.2 Where facilities for CD4 count are not available, individuals assessed, as being in WHO clinical stage 3 or 4, should receive ART.1

The WHO clinical staging system for HIV was introduced in 19903 and was based on previous highly specific WHO case definitions for AIDS that were used in the era before the availability of reliable and rapid HIV diagnostic tests.4,5 Although the WHO clinical staging system has good prognostic value in predicting death6 and response to ART,7 it was not originally designed to be used for as screening tool for ART eligibility. Previous studies8–11 have shown the sensitivity of nurse-performed WHO stage 3 or 4 assessment in identifying CD4 count of <350 cells per cubic millimeter to be low (between 18% and 65%), potentially resulting in missed and delayed ART initiation.12

We previously found that eligibility assessments were a frequent cause of dropout before ART initiation,13 with health workers finding WHO staging overly complex and time consuming, and individuals reporting that completion of CD4 count measurement required multiple facility visits.14,15 With task shifting of HIV testing to community health workers (CHWs)16 and moves toward “test-and-treat,”17 eligibility assessments should be completed immediately following HIV diagnosis to maximize opportunities for linkage to ART. We therefore developed a brief CHW tool for assessing ART eligibility and compared its accuracy with that of with the current WHO clinical staging system.

METHODS

Study Design

An accuracy study nested within a cluster-randomized trial that was investigating community-based approaches to HIV diagnosis and treatment.18

ART Eligibility Guidelines and Development of Novel ART Eligibility Screening Tool

National ART eligibility guidelines in Malawi matched WHO recommendations (ie, CD4 count of <350 cells/mm3 or in WHO stage 3 or 4).1,19

The CHW tool was designed based on the WHO “Ask, Look, Classify, Act” approach incorporated within guidelines for the Integrated Management of Adult and Adolescent Illness.20 Questions were constructed for inclusion in the CHW tool from existing components of the WHO clinical staging system21; with additional validated questions that have previously assessed to be important predictors of ART eligibility (self-rated general health14,22) and advanced immunosuppression (self-rated skin or hair change).23–25 All questions included had to be easily understood and performed by CHWs and not require further laboratory investigations. The CHW tool classifies HIV-positive patients into 2 groups: those who are ART eligible (analogous to WHO stage 3 or 4) and those who are not ART eligible (analogous to WHO stage 1 or 2). Individuals who responded positively to any of the following items were classified as ART eligible:

reports fever or weight loss or diarrhea for all of last month,21

enough weight loss to affect fit of clothing,21

sore mouth or swallowing,21

treated for TB or any other infection (excluding malaria and “flu”) in the last year,21

self-rated change in skin or hair,23–25

unable to carry out daily duties (work, go to school, do housework),21 and

self-rated general health poor or fair,14

Study Site and Population

Adult (≥16 years) residents of 3 high-density urban townships in the north west of Blantyre, Malawi, were provided with access to home HIV self-testing through CHWs. CHWs were adult residents selected by a participatory approach and who received training in HIV testing and counseling (HTC). CHWs performed the novel ART eligibility assessment tool for individuals who disclosed a positive self-test result. HIV-positive individuals received a home visit from a nurse who independently performed confirmatory HTC, TB screening (symptom screen and sputum collection for smear and culture if positive), WHO clinical staging (without reference to the result of the CHW tool assessment), and drew blood for CD4 count. ART eligible participants (on national criteria) received offer of home or facility ART initiation and noneligible participants were referred to pre-ART clinics at local health centers.

Laboratory Methods

Blood samples were analyzed using the Partec Cyflow SL-3 platform (Partec, Görlitz, Germany).

Statistical Methods

Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for both the CHW tool and WHO clinical staging using CD4 count of <350 cells per cubic millimeter as the referent. As WHO has recently proposed expanding ART eligibility to include individuals with CD4 count of <500 cells per cubic millimeter,3,26 we repeated analysis with this criteria as the referent. A sample size of 191 patients was required to estimate sensitivity and specificity with a precision of ±10% at the P = 0.05 level.27 Statistical analysis was done using Stata 12.1 (Statacorp, College Station, TX).

Ethical Approval

The research ethics committees of the College of Medicine of Malawi, Liverpool School of Tropical Medicine, and London School of Hygiene and Tropical Medicine granted ethical approval for the parent cluster-randomized trial. All participants gave written informed consent to participate or witnessed thumbprint consent if illiterate.

RESULTS

Baseline Characteristics

Between February and November 2012, 198 of 206 HIV-positive adult community members were recruited to the study (96.1% participation). Men comprised 42% (84/198) of participants and were younger, had a higher level of schooling, and were more likely to have salaried employment than women (Table 1).

TABLE 1

Characteristics of Study Participants

Outcomes of ART Eligibility Assessments

A total of 31 of 198 (15.6%) individuals were assessed to be in WHO stage 3 or 4 by nurses, with men significantly more likely to be in the more advanced WHO stage 3 or 4 than women [19/84 (22.6%) vs. 12/114 (10.5%) respectively; P = 0.021]. Men also had a significantly lower median CD4 count at assessment compared with women [262 cells/mm3 (interquartile range, IQR: 148–373) vs. 374 cells/mm3 (IQR: 195–512); P = 0.002]. A total of 110 of 198 (55.6%) participants had a CD4 count below the Malawian national ART eligibility threshold of <350 cells per cubic millimeter.

Performance of CHW Tool in Identifying CD4 count of <350 Cells per Cubic Millimeter

One participant did not have the CHW tool completed by the CHW, leaving 197 participants who were analyzed with complete data for outcome of CHW tool, CD4 count, and WHO clinical staging assessment.

The sensitivity of the CHW tool (41.3%; 95% CI: 31.9% to 51.1%) was substantially higher than that of nurse-performed WHO clinical stage 3 or 4 assessment (19.1%, 95% CI: 12.2% to 27.7%) in identifying CD4 count of <350 cells per cubic millimeter (Table 2). The specificity of the CHW tool was 83.0% (95% CI: 73.4% to 90.1%) compared with 88.6% (95% CI: 80.1% to 94.4%) for WHO clinical stage 3 or 4 assessment. The area under the curve (AUC) was significantly higher for the CHW tool (0.62; 95% CI: 0.56 to 0.68) compared with WHO clinical stage 3 or 4 assessment (0.54; 95% CI: 0.49 to 0.59; P = 0.017).

TABLE 2

Accuracy of CHW Tool and WHO Clinical Stage 3 or 4 for Predicting CD4 count of <350 and <500 Cells per Cubic Millimeter

Unexpectedly, there were differences in test performance when results were stratified by gender. The sensitivity and specificity of the CHW tool were higher among men (45.6% and 92.3%, respectively) than among women (36.5% and 79.0%, respectively). For both men and women, however, the PPV (92.9% vs. 78.9% and 59.4% vs. 50.0%, respectively) and NPV (43.6% vs. 33.8% and 59.8% vs. 54.9%, respectively) of the CHW tool was higher than that of WHO clinical staging. The largest AUC was observed for men identified as ART eligible by the CHW tool (AUC = 0.69), which was significantly higher than for WHO stage 3 or 4 assessed men (AUC = 0.55; P = 0.016). For women, although the AUC for the CHW tool (AUC = 0.58) was greater than for WHO stage 3 or 4 assessment (AUC = 0.51), the difference was relatively small and did not reach statistical significance (P = 0.104).

Using CD4 count of <500 cells per cubic millimeter as the referent, the CHW tool similarly outperformed WHO clinical staging assessment in terms of sensitivity, PPV, NPV, and AUC (Table 2), although performance of both tools was worse than at the CD4 count of <350 cell per cubic millimeter cutoff.

DISCUSSION

The main finding from this study was that our brief CHW tool significantly outperformed the current WHO clinical staging system for identifying ART eligible individuals with CD4 count of <350 cells per cubic millimeter. The ease with which we were able to improve on the current WHO staging system emphasizes the need for improved tools that can be used by community and facility-based health workers to ensure “same-day, same-site” eligibility assessments and prompt linkage to ART. Moreover, the low sensitivity and high risk of misclassification when using the standard WHO clinical staging system in this and other studies8–11 questions its appropriateness as an ART eligibility assessment tool, especially with increasing CD4 count thresholds.28

The CHW tool was found to be significantly more accurate than WHO clinical staging assessment in terms of sensitivity, PPV, and NPV. In contrast to the WHO clinical staging system, which, strictly followed, requires nurses or doctors to undertake a complex and time-consuming assessment,14 our CHW tool was easily performed in the community and could be used in settings without sophisticated laboratories. The poor performance of nurse WHO clinical staging system in this and other studies8–11 is therefore not a factor of the capability of nurses. Instead it is a reflection of the inherent limitations of the WHO screening tool for identifying ART eligibility, a task for which it was never originally intended.

The CD4 count is the gold standard for determining ART eligibility, but access to laboratory measurement is problematic in most resource-limited settings.29 Point of care CD4 count measurement has the potential to negate some of these issues and improve retention in pre-ART care,30 but it is not yet widely available. Where CD4 count is not available at the point of HIV diagnosis, improved eligibility assessment tools such as ours could allow an immediate decision on ART initiation to be made, facilitating “test-and-treat” delivery of ART.

The accuracy of the CHW tool was noted to be higher among men than women. This may reflect the relatively more advanced immunosuppression of men compared with women in the study, consistent with findings from HIV care programmes in sub-Saharan Africa showing that men initiate ART later than women.28 Women may have also been less likely to report symptoms included in the tool. Future, larger studies should examine the performance of CHWs tool among men and women across CD4 count strata.

Our CHW tool was designed pragmatically to combine ease of use by CHWs with questions that were hoped would have a high discriminatory power. Component parts of the tool were not captured and so we were unable to assess the contribution of individual items to the tool performance. As such, it is possible that the number of items could be reduced, or the sensitivity and specificity improved beyond that reported here. Future research should attempt to refine the tool by examining the contribution of the included items to the test performance and evaluating any modifications.

In conclusion, this study has shown that a novel screening tool performed by CHWs can distinguish ART eligible from ART ineligible HIV-positive individuals. This could be added to HTC counselors' posttest evaluations, allowing same-day, same-site HIV diagnosis and ART eligibility assessment. The accuracy of the CHW tool significantly outperformed WHO staging by a nurse and the ease with which we were able to improve on the current WHO clinical staging system shows the urgent need for improved ART eligibility assessment tools.