Validation of a screening tool to identify older children living with HIV in primary care facilities in high HIV prevalence settings.

OBJECTIVE: We previously proposed a simple tool consisting of five items to screen for risk of HIV infection in adolescents (10-19 years) in Zimbabwe. The objective of this study is to validate the performance of this screening tool in children aged 6-15 years attending primary healthcare facilities in Zimbabwe.
METHODS: Children who had not been previously tested for HIV underwent testing with caregiver consent. The screening tool was modified to include four of the original five items to be appropriate for the younger age range, and was administered. A receiver operator characteristic analysis was conducted to determine a suitable cut-off score. The sensitivity, specificity and predictive value of the modified tool were assessed against the HIV test result.
RESULTS: A total of 9568 children, median age 9 (interquartile, IQR: 7-11) years and 4971 (52%) men, underwent HIV testing. HIV prevalence was 4.7% (95% confidence interval, CI:4.2-5.1%) and increased from 1.4% among those scoring zero on the tool to 63.6% among those scoring four (P < 0.001). Using a score of not less than one as the cut-off for HIV testing, the tool had a sensitivity of 80.4% (95% CI:76.5-84.0%), specificity of 66.3% (95% CI:65.3-67.2%), positive predictive value of 10.4% and a negative predictive value of 98.6%. The number needed to screen to identify one child living with HIV would drop from 22 to 10 if this screening tool was used.
CONCLUSION: The screening tool is a simple and sensitive method to identify children living with HIV in this setting. It can be used by lay healthcare workers and help prioritize limited resources.

Introduction

HIV testing is the first step towards accessing HIV prevention and care services. The coverage of antiretroviral therapy (ART) is disproportionately lower in children (30%) than in adults (43%) in sub-Saharan Africa, the key reason for this disparity being the low HIV testing rates in children [1]. Delayed diagnosis and treatment are associated with poor outcomes, and adolescents are the only age-group in which HIV-related deaths are continuing to increase, predominantly as a result of delayed diagnosis and institution of ART among children [2,3].

The World Health Organization (WHO) recommends provider-initiated HIV testing and counselling (PITC) for all individuals attending any healthcare facility in high HIV prevalence settings [4]. In recognition of the substantial gap in HIV testing in older children, the WHO developed specific HIV testing and counselling guidelines for adolescents in 2013, recommending PITC for adolescents in high HIV prevalence settings [5]. This is important as optimal implementation and universal coverage of prevention of mother-to-child transmission programmes in many high HIV prevalence settings has yet to be achieved [6-8]. Moreover, coverage of HIV testing in infancy remains poor with almost half of the high priority countries reporting coverage of less than 20% [9]. Only four countries – Namibia, South Africa, Swaziland and Zambia – provide early infant diagnosis to more than 50% of the children born to women living with HIV [9-11]. Thus with the current levels of programme performance a significant number of children living with HIV will reach older childhood without being diagnosed.

Whereas PITC for adolescents and children is recommended, the extent to which it is implemented in primary healthcare facilities is variable [12]. The few studies that have systematically reported on PITC in children and adolescents show low rates of HIV testing being offered by healthcare providers to children [13,14]. Vertically acquired HIV is likely to be responsible for the majority of symptomatic HIV infections in this age group in Southern Africa [15-17]. The lack of implementation of PITC in this age-group may be explained by a combination of factors such as limited awareness of the burden of vertically acquired HIV in older children, resulting in healthcare workers selectively offering HIV testing to those presenting with symptoms suggestive of HIV rather than to all children. Many of the long-term survivors of mother-to-child HIV transmission present to primary healthcare services with conditions such as chronic upper respiratory tract and skin infections that are also common among children who are not living with HIV [17,18]. HIV testing is thus offered when children present with the typical HIV-associated manifestations when HIV infection is at a more advanced stage [19]. Failure to consider long-term survival following vertical HIV infection therefore results in missed opportunities for timely diagnosis, and increases the risk of mortality and development of severe, irreversible long-term complications [20]. In addition, due to verticality of programmes, donor-driven priorities and workforce and logistic constraints (such as limited supply of HIV testing kits), HIV testing is prioritized in specific groups such as individuals diagnosed with tuberculosis and pregnant women [13,14].

Given resource constraints and the relatively low prevalence of HIV among older children and adolescents compared with adults, an initial screening tool to identify those at risk of being HIV-infected and then offering testing to those who screen positive could reduce the numbers that would need to undergo HIV testing and increase the yield. Such a screening tool would require sufficient sensitivity and specificity and be relatively cheap to administer.

We previously developed a simple screening tool for adolescence aged 10–19 years to identify those at risk of being HIV-infected in primary healthcare clinics (PHCs) in Harare, Zimbabwe. Fourteen socio-demographic and clinical variables defined a priori were considered in the development of the screening tool. The final screening tool consisted of five items, and using a cut-point of two provided sensitivity of 74% and specificity of 80% [21]. The screening tool can be administered by a lay healthcare worker, with an affirmative response to any two or more items prompting an offer of HIV testing. This study aimed to validate a modified screening tool in a large independent population, and to assess its performance under routine conditions in seven PHCs in Harare, Zimbabwe.

Methods

The study was conducted from January 2013 to December 2014 in seven PHCs in south-western Harare, Zimbabwe, as described previously [13]. In brief, all children aged 6–15 years who attended a PHC for any reason were offered HIV testing. HIV testing was carried out with guardian consent and child assent, as per national guidelines [22]. Those who had a documented HIV test result from the past 6 months, were known to be HIV-positive, were attending without a caregiver (unless an emancipated minor), required immediate hospitalization or were moribund were not offered HIV testing and were excluded from this study. The caregiver or child was asked to respond to four screening items asked by lay healthcare assistants regardless of whether the child underwent HIV testing and regardless of the HIV test result.

The following screening items were used:

Has the child been admitted to hospital before?

Does the child have recurring skin problems?

Are one or both parents of the child deceased?

Has the child had poor health in the last 3 months?

The original screening tool included a fifth item enquiring about symptoms of sexual transmitted infections. In view of the younger age group considered in the current study, this item was deemed inappropriate. Age, sex, clinic location and HIV test result were collected on standardized forms.

Data were entered into an ACCESS database using Cardiff TELEFORM Intelligent Character Optical Mark Recognition Software (Version 10.7) and analysed using STATA (Version 12.1). The analysis only included participants with HIV test results and excluded participants with missing data for any of the screening items. Screening items were coded one for positive and zero for negative responses. The total score was calculated as the sum of the numerical values of the four screening items (minimum 0, maximum 4). The maximum screening score was four and the minimum was zero. χ2 and Student's t-tests were used to compare categorical data. Logistic regression was used to investigate the association between HIV status, individual screening items and screening scores, with a random effect model and adjustment using a variance inflation factor to account for clinic-level clustering [23]. To determine the optimal cut-off for the screening tool for HIV infection, a receiver operating characteristic curve (ROC) was plotted. The area under the ROC curve (AUC) and corresponding sensitivity, specificity, positive predictive (PPV) and negative predictive values (NPV), number needed to test to identify one child living with HIV (NNT), and reduction in NNT by using the screening tool were determined for different score levels. Sensitivity and specificity was also calculated for each of the screening items separately. A similar analysis was conducted stratified by age and sex of participants.

The study was approved by Medical Research Council of Zimbabwe and Ethical Review Committee of The London School, Hygiene and Tropical Medicine and Biomedical Research and Training Institute. Consent procedures for HIV testing of participants followed the Zimbabwe National Guidelines for HIV Testing and Counselling and in Children [22].

Results

A total of 12 057 children aged 6 to 15 years were eligible for PITC, of whom 9655 (80%) underwent HIV testing (Fig. 1). Of the 9655 participants, 87 were excluded because they did not respond to all the screening tool items and/or had some demographic information missing leaving 9568 for analysis. There was little evidence of a difference in age and sex between those who underwent HIV testing and those who did not. However, those who did not test had a higher proportion of missing answers (Fig. 1) and lower prevalence of affirmative answers to each of the screening items and a lower proportion of the screening tool score (Table 1).

Fig. 1

Selection of the validation study sample.

Table 1

Baseline characteristics stratified by whether HIV tested or not and by HIV.

			HIV testing			HIV status
Characteristic			Not tested (N = 2253)	Tested (N = 9568)	P value	HIV-positive (N = 445)	HIV-negative (N = 9123)	P value
Age
	Median age (years)		9 (IQR 7–11)	9 (IQR 7–11)	0.76	11 (IQR 8–13)	9 (IQR 7–11)	<0.001
Sex	Male		1188 (52.8%)	4971 (52.0%)	0.50	211 (47.4%)	4760 (51.6%)	0.05
HIV positive	Yes			445 (4.7%)
Screening items	Been admitted to hospital before	Yes	179 (8.0%)	926 (9.7%)	0.01	76 (17.1%)	850 (9.3%)	<0.001
	Had recurring skin problems	Yes	220 (9.8%)	1453 (15.2%)	<0.001	149 (33.5%)	1304 (14.3%)	<0.001
	One or both parents deceased	Yes	217 (9.6%)	1372 (14.3%)	<0.001	238 (53.5%)	1134 (12.4%)	<0.001
	Had poor health in the last 3 months	Yes	97 (4.3%)	767 (8.0%)	<0.001	145 (32.6%)	622 (6.8%)	<0.001
Screening tool score	0		1706 (75.8)	6133 (64.1%)	<0.001	87 (19.5%)	6046 (66.1%	<0.001
	1		418 (18.6%)	2536 (26.5%)	<0.001	178 (40.0%)	2358 (25.9%)	<0.001
	2		94 (4.2%)	737 (7.7%)	<0.001	124 (27.9%)	613 (6.7%)	<0.001
	3		29 (1.3%)	140 (1.4%)	0.53	42 (9.4%)	98 (1.1%)	<0.001
	4		5 (0.2%)	22 (0.2%)	0.94	14 (3.2%)	8 (0.1%)	<0.001
	≥1		546 (24.2%)	3435 (35.9%)	<0.001	358 (80.5%)	3077 (33.7%)	<0.001

IQR, interquartile.

Of the 9568 participants, 4971 (52%) were men, the median age was 9 years (interquartile, IQR 7–11) and HIV prevalence was 4.7% (95% confidence interval, CI:4.2–5.1%) (Table 1). Children aged 6–9 years had a significantly lower HIV prevalence than those aged not less than 10 (3.2 vs. 6.4%, P < 0.001). An affirmative answer to each screening question was associated with being HIV-positive (Table 1), and the HIV prevalence increased significantly with an increase in score from 1.4% among those scoring 0 on the screening tool to 63.6% among those scoring four (P < 0.001). The sensitivity and specificity in detecting HIV status varied for the different screening items, with orphanhood having the highest sensitivity in predicting HIV infection (53.5%, 95%CI 48.7–58.2%) and poor health having the highest specificity (93.2%, 95% CI:92.6–93.7%) (Table 2).

Table 2

Sensitivity and specificity, positive predictive value, negative predictive value, number needed to test to identify 1 HIV-infected after application of screening tool, number needed to test to identify 1 HIV-infected adolescent misclassified by the screening tool as not at HIV risk and reduction in NNT+ by using screening tool.a

	Sn (%) (95% CI)	Sp (%) (95% CI)	PPV (95% CI)	NPV (95% CI)	NNT+ tool (95% CI)	NNT– tool (95% CI)	Reduction in NNT+ compared with universal testing (%)
Screening question
Been admitted to hospital before	17.1 (13.7–20.9)	90.7 (90.1–91.3)	8.2 (6.5–10.2)	95.7 (95.3–96.1)	12 (10.0–15.5)	23 (21.3–26.0)	45
Had recurring skin problems	33.5 (29.1–38.1)	85.7 (85.0–86.4)	10.3 (8.7–11.9)	96.4 (95.9–96.7)	10 (8.5–11.5)	27 (24.7–30.9)	56
One or both parents deceased	53.5 (48.7–58.2)	87.6 (86.9–88.2)	17.3 (15.4–19.5)	97.5 (97.1–97.8)	6 (5.2–6.5)	40 (34.9–45.7)	74
Had poor health in the last 3 months	32.6 (28.2–37.2)	93.2 (92.6–93.7)	18.9 (16.2–21.6)	96.6 (96.2–97.0)	5 (4.6–6.2)	29 (26.4–33.0)	76
Score
Screening tool score ≥1	80.4 (76.5–84.0)	66.3 (65.3–67.2)	10.4 (9.4–11.5)	98.6 (98.3–98.9)	10 (8.7–10.6)	70 (58.3–89.1)	56
Screening tool score ≥2	40.4 (35.9–45.2)	92.1 (91.5–92.7)	19.3 (16.2–22.7)	97.0 (96.5–97.4)	5 (4.4–5.7)	33 (29.2–37.1)	77
Screening tool score ≥3	12.6 (9.7–16.0)	98.8 (98.6–99.0)	34.6 (27.3–42.4)	95.9 (95.4–96.3)	3 (2.4–3.7)	24 (22.0–26.8)	87
Screening tool score ≥4	3.2 (1.7–5.2)	99.9 (99.8–100)	63.6 (40.7–82.8)	95.5 (95.0–95.9)	2 (1.2–2.3)	22 (20.3–24.4)	93

aNPV, negative predictive value; PPV, positive predictive value; Sn, sensitivity; Sp, specificity; NNT+ tool, number needed to test to identify 1 HIV-infected after application of screening tool; NNT-tool, number needed to test to identify 1 HIV-infected adolescent misclassified by the screening tool as not at HIV risk.

Optimal sensitivity and specificity was achieved using a screening score threshold of not less than one (AUC = 0.73, 95% CI 0.72–0.75) (Fig. 2). Using a cut-off score of not less than one to screen for HIV testing had a sensitivity of 80.4% (95%CI 76.5–84.0) and specificity of 66.3% (95% CI:65.3–67.2%) (Table 2). Adjustment for clinic-level clustering with variance adjustment yielded slightly wider confidence intervals for the specificity only [adjusted sensitivity: 80.4% (95% CI:77.0–83.9%), specificity: 66.3% (95% CI:58.0–74.5%)] with no change to the actual estimates. With an HIV prevalence of 5% in this population, the screening tool had PPV of 10.4% (95% CI:9.4–11.5%) and a NPV of 98.6% (95% CI:98.3–98.9%). Using a score of not less than one as the cut-off for the screening tool, the number needed to test to diagnose one child living with HIV was 10 (95% CI:8.7–10.6), and there would be a 56% reduction in the number needed to test to diagnose one child living with HIV if the screening tool was used compared with universal HIV testing (Table 2). However, it is also important to note that using the algorithm, an individual would be falsely classified as ’not at risk’ for every 70 children screened (Table 2).

Fig. 2

Receiver operating characteristic curve for each cut-off for the proposed screening tool.a

aFor each cut-off, sensitivity, specificity and area under the curve is indicated.

The screening items maintained their sensitivity and specificity and remained strong predictors of HIV infection after adjustment for clinic-level clustering. Sex stratified analysis did not yield significant difference in either sensitivity (83 vs. 79%, P = 0.31) or specificity (66 vs. 67%, P = 0.41). However, age-stratified analysis revealed higher sensitivity of the screening tool among those aged at least 10 years compared with children aged 6–9 years (84.4 vs. 73.2%, P = 0.003), whereas specificity remained comparable by age group (65.4 vs. 67.0%, P = 0.11). After applying the screening tool the number needed to test to diagnose one child with HIV infection was 15 (95% CI:12.6–17.8) in the 6–9 year olds compared with seven (95% CI:6.3–7.9) in the 10–15 year olds.

Discussion

This study tested the external validity of a simple four item tool to screen for the risk of HIV infection in older children and adolescents (aged 6–15 years) in a routine healthcare setting. The screening tool showed a high sensitivity and specificity for identifying older children and adolescents living with HIV attending primary healthcare facilities. Using this screening tool the number needed to test to diagnose one child living with HIV would be reduced from 22 when testing unselectively to 10 when targeted testing is introduced. The screening tool is designed to identify mainly older children and adolescents infected through mother-to-child HIV transmission. Previous paediatric screening tools to identify children such as the integrated management of childhood illness/HIV screening tool, have tended to focus diagnosis of younger children with HIV infection, are less evidence-based and have had lower sensitivity and specificity than this screening tool [24-26].

The burden of undiagnosed HIV among older children remains high and universal access to treatment in children cannot be achieved without significant scale-up of HIV testing [18]. However, the much lower prevalence of HIV infection in this age-group compared with that in adults means that, in the context of resource constraints blanket PITC testing strategy may be considered lower priority and less cost-effective in this age-group [16]. The use of a simple screening tool to identify those at highest risk would increase the yield of HIV infection, and reduce the cost of HIV testing especially amongst an age-group where the burden of undiagnosed HIV infection is high but the overall prevalence is low.

The screening tool consists of simple screening items and can be asked by lay health workers, with minimal training required. Implementation of the screening tool at scale might therefore be feasible and may be a sustainable alternative in the context of constrained resources, including shortage of staff and testing kits [13,27]. As well as identifying individuals at high risk of being HIV-infected, the screening tool may serve to increase awareness among healthcare providers of the need to consider HIV infection in older children and adolescents who do present with the classic HIV disease manifestations. Healthcare workers may be reluctant to discuss HIV testing with older children and younger adolescents as this raises uncomfortable questions about the source of infection [13,28]. The use of a screening tool may prompt this process in an age-group for whom alternative testing services are often not available. A further advantage could be normalization of HIV testing for this age-group in primary care.

The performance of the screening tool, specifically the NPV and the PPV, depends on the prevalence of HIV and the prevalence of the factors that constitute the screening tool. This is context-specific and the prevalence of the factors will likely change over time. For example, as ART becomes more widespread, the prevalence of AIDS orphans may decline. The sensitivity and specificity of the screening tool may decrease for children attending the health-facility repeatedly, and performance of the screening tool will be influenced by whether attendees to primary care are rescreened on each visit. Whereas we report data on number of visits rather than on individuals in this study, participants were not re-screened or re-tested if they had undergone screening or HIV testing previously.

Strengths of this study include the large number of participants resulting in narrow confidence intervals around sensitivity and specificity estimates. Further, the data were systematically and prospectively collected over a prolonged period of time demonstrating sustainability. The study was conducted in routine healthcare settings with the screening tool being administered by lay healthcare workers, making the findings more generalizable. The limitations are that 20% of eligible primary care attendees did not undergo testing, and the HIV status of participants who did not undergo testing was not available; the prevalence of an affirmative response to the screening tool items were lower than those among participants who did undergo HIV testing.

We stress that PITC of all health-facility attendees remains the optimum strategy and should be implemented whenever possible. The screening tool will have a false negativity rate and some children with HIV will be falsely classified as not at risk. The use of a screening tool offers an alternative approach taking into account the structural and other barriers to HIV testing in this age-group. Ultimately, the goal is to identify and treat children living with HIV timely and a screening approach provides a possible pragmatic approach to address some of the barriers to achieve this goal.

Acknowledgements

R.A.F. and T.B. designed the study. E.D. supervised data collection. T.B. analysed the data. T.B., R.A.F. and K.K. wrote the first draft of the manuscript. All authors contributed to the writing of the manuscript.

The authors would like to thank the nurses and counsellors at the six study clinics for their assistance with the study.

The study is funded by the Wellcome Trust through an Intermediate Fellowship awarded to R.A.F. (Grant no 095878/Z/11Z). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflicts of interest

The authors have no conflicts of interest.