An Empiric HIV Risk Scoring Tool to Predict HIV-1 Acquisition in African Women.

OBJECTIVE: To develop and validate an HIV risk assessment tool to predict HIV acquisition among African women.
DESIGN: Data were analyzed from 3 randomized trials of biomedical HIV prevention interventions among African women (VOICE, HPTN 035, and FEM-PrEP).
METHODS: We implemented standard methods for the development of clinical prediction rules to generate a risk-scoring tool to predict HIV acquisition over the course of 1 year. Performance of the score was assessed through internal and external validations.
RESULTS: The final risk score resulting from multivariable modeling included age, married/living with a partner, partner provides financial or material support, partner has other partners, alcohol use, detection of a curable sexually transmitted infection, and herpes simplex virus 2 serostatus. Point values for each factor ranged from 0 to 2, with a maximum possible total score of 11. Scores ≥5 were associated with HIV incidence >5 per 100 person-years and identified 91% of incident HIV infections from among only 64% of women. The area under the curve (AUC) for predictive ability of the score was 0.71 (95% confidence interval [CI]: 0.68 to 0.74), indicating good predictive ability. Risk score performance was generally similar with internal cross-validation (AUC = 0.69; 95% CI: 0.66 to 0.73) and external validation in HPTN 035 (AUC = 0.70; 95% CI: 0.65 to 0.75) and FEM-PrEP (AUC = 0.58; 95% CI: 0.51 to 0.65).
CONCLUSIONS: A discrete set of characteristics that can be easily assessed in clinical and research settings was predictive of HIV acquisition over 1 year. The use of a validated risk score could improve efficiency of recruitment into HIV prevention research and inform scale-up of HIV prevention strategies in women at highest risk.

INTRODUCTION

Globally, women account for more than half of all new HIV infections, with the greatest burden among African women.[1] Several recently completed HIV prevention trials in women have reported high incidence rates, 4%–5% per year overall and up to 10% at some sites, despite the provision of comprehensive HIV risk reduction services.[2-5] Given this persistent high HIV incidence, there is an urgent need to develop and deliver effective novel prevention interventions to women at high risk of acquiring HIV.

Risk assessment tools generated using prediction models are commonly used in clinical practice to simplify the identification of individuals at increased risk for a particular health outcome. Empiric HIV risk assessment tools have been developed for several populations, including African heterosexual HIV serodiscordant couples[6] and men who have sex with men (MSM) in the United States.[7,8] However, despite >20 years of research to define individual correlates of HIV acquisition in African women, an effective, validated tool to identify African women at highest risk of HIV acquisition has not been developed. Such tools could be used in HIV prevention research settings to inform study designs and improve recruitment efficiency, and guide the implementation of HIV prevention activities, such as pre-exposure prophylaxis (PrEP) by prioritizing those at highest risk.[9,10] Using data from several recently completed HIV biomedical prevention trials, we developed and validated a risk score to predict HIV acquisition in the next year among African women.

METHODS

Study Populations

We used data from 3 clinical trials of biomedical HIV prevention interventions to assess the relationship between demographic, behavioral, partnership, and clinical characteristics assessed at enrollment and HIV-1 acquisition risk in the year after assessment.

VOICE (Derivation Cohort)

Between September 2009 and June 2011, 5029 women from South Africa, Uganda, and Zimbabwe were enrolled into the VOICE study, a randomized, double-blind, placebo-controlled trial that assessed the safety and effectiveness of daily oral tenofovir disoproxil fumarate (TDF), oral TDF/emtricitabine (FTC), and 1% vaginal tenofovir gel as PrEP for HIV prevention (NCT00705679).[11] Healthy women were eligible to participate in the trial if they were 18–45 years old, HIV uninfected, sexually active within the last 3 months, not pregnant or breastfeeding, willing to use an effective method of contraception, hepatitis B negative, and had no evidence of abnormal hepatic or renal function. Planned follow-up was for a minimum of 12 months; however, the TDF and tenofovir arms were discontinued early for futility. Participants in the TDF/FTC arm and its corresponding placebo comparator continued participation until planned study exit (August 2012). In intent to treat analyses, HIV incidence in the intervention arms did not differ from that in the respective placebo arms.

HIV Prevention Trials Network 035 (Validation Cohort)

HIV Prevention Trials Network (HPTN) 035 was a randomized placebo-controlled trial that assessed the safety and effectiveness of BufferGel and 0.5% PRO2000 microbicidal gels for HIV prevention in women (NCT00074425).[5] Between February 2005 and January 2009, 3101 women from Malawi, South Africa, United States, Zimbabwe, and Zambia were enrolled and randomized to receive BufferGel, 0.5% PRO2000 gel, hydroxyethylcellulose placebo gel, or condoms alone (no study product); US participants were excluded from the present analysis. Participants who were at least 18 years old, HIV uninfected, sexually active within the last 3 months, and not pregnant were enrolled and followed for a minimum of 12 months. Neither microbicide significantly reduced the incidence of HIV compared with either control arm.

FEM-PrEP (Validation Cohort)

Between June 2009 and April 2011, 2120 women from Kenya, South Africa, and Tanzania were enrolled into the FEM-PrEP trial, a randomized, double-blind, placebo-controlled trial that assessed the safety and effectiveness of daily oral TDF/FTC for HIV prevention (NCT00625404).[3] Eligible women were 18–35 years old, HIV uninfected, not pregnant or breastfeeding, willing to use an effective contraceptive method at enrollment, and at high risk for becoming HIV infected (defined by the study protocol as having one or more vaginal sex acts in the last 2 weeks or more than one sex partner in the previous month). Women were excluded if they tested positive for hepatitis B or had evidence of abnormal hepatic or renal function. Participants were followed for up to 1 year. In April 2011, the trial was stopped early because of futility.

Protection of Human Subjects

All studies provided participants with comprehensive HIV prevention services during the trial, including risk reduction counseling, free condoms, and treatment of sexually transmitted infections (STIs). Participants provided written informed consent, and study protocols were approved by applicable local and national ethical and regulatory authorities.

HIV Testing and Laboratory Methods

Participants in all 3 studies underwent regular HIV testing using standard algorithms, which included HIV rapid testing using Determine HIV 1/2 (Abbott Diagnostic Division, Hoofddorp, the Netherlands), OraQuick (Orasure Technologies, Bethlehem, PA), Uni-Gold Recombigen HIV test (Trinity Biotech, Wicklow, Ireland), SD Bioline HIV 1/2 (Standard Diagnostics Inc., Suwon, Korea). In VOICE and HPTN 035, confirmatory testing was performed on samples with any positive HIV-1 rapid result using Western blot (Genetics Systems HIV-1 Western Blot kit; BioRad Laboratories, Hercules, CA). Participants in VOICE and FEM-PrEP received monthly HIV testing, whereas HPTN 035 participants underwent quarterly testing.

Risk Score Development and Validation

Our objective was to develop a simple risk score based on a discrete set of characteristics that could be easily assessed in variety of settings to identify African women at highest risk of HIV acquisition within the following year. We employed methods frequently used to develop prediction rules, which have been used to develop HIV prediction risk scores for other populations.[6,8] We conducted internal and external validations to ensure the robust performance of the score. Participant follow-up was censored at 1 year because we hypothesized that characteristics assessed at enrollment would be predictive up to 1 year from assessment; in addition, prevention strategies typically emphasize that HIV risk be reassessed at least annually (as recommended for PrEP[12]).

Baseline characteristics assessed in VOICE considered for the risk score included demographic, behavioral, clinical, and self-reported male partner characteristics that could be easily collected in clinical and research settings. All characteristics were parameterized as categorical variables. Age was dichotomized as less than 25 versus 25 years or greater as this categorization is frequently used in HIV prevention policy and programmatic settings.[13] We analyzed the relationship between these characteristics and HIV acquisition using univariate Cox proportional hazards among participants who had complete data. Potential predictors that were significantly associated with HIV acquisition in univariate models (P < 0.05) were evaluated in a multivariable model. To identify the combination of factors that best predicted HIV risk, we used forward and backward stepwise Cox proportional hazards model that evaluated the inclusion or exclusion of potential predictors at each step. All models were stratified by study site. The model with the lowest Akaike[14] information criterion was chosen as the final model for the risk score. Individual predictors included in the final model were assigned a score by dividing the coefficient for the predictor in the final model by the lowest coefficient among all predictors in the model and rounding to the nearest integer. The sum of the values for each predictor represented the total score for each participant, and the HIV incidence for each total score category was calculated. The predictive ability of the total score and each predictor was assessed by calculating area under the receiver operating characteristic curve.[15] The score was internally validated using 10-fold cross-validation, and the area under the curve (AUC) for the final model was compared with the mean AUC of the 10 different models. Additional performance characteristics (sensitivity, specificity, positive predictive value, and negative predictive value) were calculated using risk score cut-points that corresponded to an HIV incidence in the risk score category of approximately ≥3% and ≥5%. Incidence curves were generated to assess cumulative HIV incidence by risk score cut-point.

The risk score was externally validated by separately applying it to the HPTN 035 and FEM-PrEP study populations. Because the interventions evaluated in HPTN 035 and FEM-PrEP did not significantly reduce HIV infection risk, the score was applied to all enrolled participants with complete data for key predictors. We calculated the HIV incidence within each score category and determined the AUC (as described above) to assess the performance of the risk score in the 2 study populations. There were slight differences in the availability of baseline data for each study; therefore, the full risk score was adapted to accommodate these differences. The performance of the adapted risk scores was reassessed in VOICE using the methods described above and then applied to the HPTN 035 and FEM-PrEP populations for external validation.

Although diagnostics for STI detection, including herpes simplex virus type 2 (HSV-2), are available in many resource-limited settings, they may not be accessible in all settings where assessments of HIV risk are likely to occur. With those settings in mind, we planned a priori to assess the performance of a modified risk score that excluded laboratory-based testing using the same methods described above. All analyses were conducted using Stata v12.1 (StataCorp, College Station, TX) and R v2.15.1 (The R Project for Statistical Computing).

RESULTS

Among 5029 women enrolled in VOICE, 22 had acute HIV infection at baseline, 38 never returned for follow-up, and 142 had incomplete baseline predictor data and were excluded from this analysis. Baseline characteristics of the remaining 4834 women are presented in Table 1. Most women were young (<25 years old), not married or living with a partner, and were from South Africa. Most women (91%) reported vaginal sex in the 4 weeks before enrollment, with 1047 (22%) reporting unprotected sex in the last 7 days. Anal sex in the 3 months before enrollment was reported by 842 (17%) participants. There were 263 seroconversions during 4348 person-years of follow-up [HIV incidence = 6.05%; 95% confidence interval (CI): 5.36 to 6.83]. HIV incidence was highest in South Africa (7.34%), followed by Uganda (2.07%) and Zimbabwe (0.50%).

TABLE 1.

Participant Characteristics of Women in VOICE, HPTN 035, and FEM-PrEP*

Risk Score Models

More than 20 baseline characteristics were evaluated for possible inclusion in the risk score, and results from univariate, multivariable, and stepwise modeling are presented in Table 2. In the stepwise analysis, the following factors were retained in the final full prediction model: participant age, married or living with husband/primary partner, any alcohol use in the past 3 months, partner provides financial or material support, partner has other sexual partners, curable STI detected at baseline (C. trachomatis, N. gonorrhoeae, T. vaginalis, or syphilis), and HSV-2 serostatus. Although the number of live births was associated with HIV acquisition in the univariate analysis, it was not an independent predictor in multivariable analysis. Because the final full model included 2 laboratory-based factors that could not be evaluated in many clinical settings, we constructed a model for a modified risk score that excluded covariates for any curable STI and HSV-2 serostatus (Table 2). The effect estimates for each of the factors in modified risk score model were similar to those in the full risk score model.

TABLE 2.

Analysis of Select Baseline Predictors and Calculation of the Primary and Modified Risk Score

The total risk score for each participant was calculated by taking the sum of the values for each predictor, with maximum scores of 11 and 8 for the full and modified scores, respectively (Table 2). The HIV incidence by risk score value for the full and modified risk scores is presented in Figure 1. For the full risk score, we observed a sharp increase in HIV incidence among participants with risk scores ≥5. Risk score sensitivity and specificity at this cutoff were 91% and 38%, respectively (Fig. 2). A similar trend was observed for the modified risk score that excluded assessment of any curable STI and HSV-2 serostatus (Fig. 1B).

FIGURE 1.

HIV incidence and 95% CIs by risk score using the full risk score (A) and modified risk score that excludes variables for any curable STI at baseline and HSV-2 serostatus (B).

FIGURE 2.

Cumulative incidence curves of HIV acquisition stratified by HIV risk score cut-point. PPV: positive predictive value; NPV: negative predictive value.

Risk Score Validation

Receiver operating characteristic curves for the full and modified scores and also the individual categorical variables included in each score are presented in Supplemental Digital Content Figure 1, http://links.lww.com/QAI/A797. The AUC for the full score to correctly predict HIV acquisition was 0.71 (95% CI: 0.68 to 0.74), and the combination of factors included in the score performed better than each individual factor alone (Supplemental Digital Content Figure 1a, http://links.lww.com/QAI/A797). The mean AUC based on 10-fold internal cross-validation was similar to the AUC using the full data set (AUC = 0.69; 95% CI: 0.66 to 0.72), demonstrating the robustness of the full risk score within the VOICE data set. The AUC for the modified score was lower than for the full score (AUC = 0.68; 95% CI: 0.65 to 0.71); however, similar to the full score, the modified score performed better than each individual factors included in the score (Supplemental Digital Content Figure 1b, http://links.lww.com/QAI/A797), and the mean AUC from 10 cross-validation supported the robust generalizability of the score across the VOICE data set (AUC = 0.67; 95% CI: 0.64 to 0.70).

Data from HPTN 035 and FEM-PrEP were used to perform external validations of the full risk score. In HPTN 035, data were not collected on alcohol use at baseline; therefore, to accommodate these missing data, we re-evaluated the performance of the full risk score when alcohol use was not included in the model and then applied this version of the score to women who participated in HPTN 035 (maximum score = 10). The performance characteristics of the full score with alcohol use excluded were similar to those of the full score (Supplemental Digital Content Figure 2, http://links.lww.com/QAI/A797). Among 2848 women enrolled in HPTN 035 who had complete data for key baseline characteristics, there were 98 HIV seroconversions during 2903 person-years of follow-up (HIV incidence = 3.38%). HIV incidence by risk score is presented in Figure 3. When applied to HPTN 035 participants, the risk score had an AUC of 0.70 (95% CI: 0.65 to 0.75). A score cutoff of ≥3 had a sensitivity and specificity of 84% and 46%, respectively (Fig. 2). Results were generally similar for external validation of the modified risk score that did not include any curable STI, HSV-2 serostatus, or alcohol use (see Supplemental Digital Content Figure 3, http://links.lww.com/QAI/A797).

FIGURE 3.

HIV incidence and 95% CIs by risk score for HPTN 035 (A) and FEM-PrEP (B). HPTN 035 score includes variables for age, marital/cohabiting status, partner provides financial support, partner has other partners, any curable STI at baseline, and HSV-2 status. Point values for each variable are the same as in the full risk score. FEM-PrEP score includes variables for age (<25 = 1 point), marital/cohabiting status (no = 1 point), partner has other partners (yes or do not know = 1 point), and any curable STI at baseline (yes = 1 point).

In FEM-PrEP, data were not collected on alcohol use and whether partners provided financial or material support. In addition, HSV-2 testing was performed only for participants enrolled at the Pretoria, South Africa site. We similarly adapted the full risk score from VOICE for validation in FEM-PrEP by re-evaluating the score after excluding alcohol use, partner providing financial support, and HSV-2 serostatus (see Supplemental Digital Content Figure 2, http://links.lww.com/QAI/A797 for performance characteristics of this score in VOICE). Among 1804 women enrolled in FEM-PrEP with complete data for the remaining risk score characteristics, there were 59 HIV seroconversions during 1231 person-years of follow-up (HIV incidence = 4.79%). This modified score had a maximum value of 4 and an AUC = 0.58 (95% CI: 0.51 to 0.65) (Fig. 3). Using a cutoff of ≥2, the risk score had a sensitivity and specificity of 83% and 31%, respectively (Fig. 2). We re-evaluated this score including HSV-2 status and assessed its performance among the subgroup of women with data on baseline HSV-2 serostatus. Among 628 women with HSV-2 testing at baseline, the risk score AUC was 0.63 (see Supplemental Digital Content Figure 3, http://links.lww.com/QAI/A797).

DISCUSSION

In this analysis that included data from women enrolled in 3 HIV prevention trials across Eastern and Southern Africa, we identified a discrete set of easily measured characteristics that predicted HIV infection in the year after risk assessment. A risk score that included self-reported demographic, partnership, and behavioral factors and also biologic factors (detection of curable STIs and HSV-2 serostatus) had the highest predictive ability; however, a modified score that excluded biologic factors also performed well. Many of the factors included in the full risk score have demonstrated associations with HIV acquisition, including younger age, marital/cohabitation status, alcohol use, detection of curable STIs, and HSV-2 seropositivity[1,16-27]; nevertheless, the combination of factors we defined predicted incident HIV better than any one factor alone. Thus, by focusing on a combination of factors rather than individual factors, our scoring tool achieves good predictive ability and can be easily implemented in a variety of settings to identify women at high risk of acquiring HIV in the next year who will most benefit from priority access to HIV prevention interventions.

Our analysis adds to the growing body of the risk assessment literature that provides specific tools to predict HIV acquisition in different populations.[6-8,28,29] Our risk scores for African women performed well and had prediction performance characteristics similar to those of 3 other risk scores developed for HIV serodiscordant couples and MSM using similar methods (AUCs ranging from 0.67 to 0.74).[6-8] Our score was designed with community, research, and public health settings in mind, with the hope that the score would have robust applicability across these settings. In community settings, such as at voluntary counseling and testing centers, the risk score could be used to help counsel women about their HIV risk. In research settings, it could be used as a screening tool to improve recruitment efficiency by targeting enrollment of high-risk women. Finally, the risk score could be used by public health professionals and policy makers to assist in the prioritization of women for scale-up of new HIV prevention interventions, such as PrEP.

Use of population-specific validated risk scores allows for efficient and cost-effective implementation of HIV prevention interventions by targeting those who are most likely to benefit from PrEP and other novel HIV prevention interventions.[29-31] A PrEP demonstration project conducted among heterosexual serodiscordant couples in Africa is using an empiric HIV risk score to guide enrollment to target those at highest risk of HIV acquisition[9]. Based on the risk score, eligible couples would have had an expected incidence of >5% in the absence of HIV prevention interventions.[32] Similarly, a risk score for MSM is recommended when conducting assessments for potential PrEP initiation among MSM in the United States.[7,10] We assessed the performance of the risk score at different cutoffs in the context of potentially using a risk score cutoff to target PrEP implementation efforts for women. In both a higher incidence population (VOICE) and a slightly lower incidence population (HPTN 035), having a risk score of 5 or above for the full risk score was associated with an HIV incidence of ∼5 per 100 person-years or greater and had good sensitivity. A risk score of 3 was associated with incidences between ∼2 and 4 per 100 person-years or greater, depending on population (VOICE versus HPTN 035) and also had good sensitivity although more limited specificity. The selection of a risk score cutoff may depend on the resources available for targeted implementation of HIV prevention methods.

In many settings, women seek health care independent of their partner; therefore, it is important to evaluate relationships between self-reported partner characteristics and HIV acquisition risk. Male partner HIV status, viral load, and circumcision status are central factors related to HIV transmission potential.[6,33-37] Despite the absence of definitive information on partner HIV status, several participant-reported male partner characteristics were predictive of HIV acquisition and included in the final risk score models. The question of whether primary partners provided financial or material support was assessed as we hypothesized that women who reported receiving financial or material support would be at increased risk of HIV.[38,39] However, the opposite association was observed. Women engaged in more stable partnerships could be more likely to report that their partner provides financial or material support; therefore, this question may be a proxy for partnership stability rather than an indicator of higher risk transactional sex.

Hormonal contraceptive use has been associated with increased HIV acquisition risk in several studies.[40-42] Independent of a potential biological relationship between contraceptive use and HIV susceptibility, hormonal contraceptive use in a risk assessment tool could operate as a surrogate marker of HIV risk because of higher rates of unprotected sex compared with women not using hormonal contraception.[43,44] In VOICE and FEM-PrEP, use of a highly effective method of contraception was an enrollment criterion; thus, there were not sufficient numbers of women participating in these studies who were not using hormonal contraception to effectively assess hormonal contraceptive use versus nonuse as a predictor of HIV acquisition. Near universal hormonal contraceptive use in these populations may somewhat limit the generalizability of these findings; however, family planning clinics are one potential location for delivery of HIV prevention services, such as PrEP, and our data would be highly applicable in those settings.

The development and validation of our risk score were based on data from 3 recent prospective studies that enrolled women from 7 African countries. Although our analysis includes geographic diversity, our population consisted of women participating in biomedical HIV prevention trials, which may not be representative of all African women at high risk of HIV acquisition. Further evaluation of the risk score is needed to assess its performance among women in the broader population. Because most participants enrolled into VOICE were from South Africa, we attempted to improve the geographic diversity of the analysis by validating the score using data from HPTN 035 and FEM-PrEP. Still, 65% of all participants in the analysis were from South Africa, and our findings may be driven by characteristics of the epidemic that are specific to that region. Data collection instruments were similar between the studies; however, some of the factors assessed in VOICE and included in the final risk score were not assessed in HPTN 035 and FEM-PrEP. As a result, we were unable to fully perform external validation of the most predictive scores. Adaptations of the full and modified scores were generated instead, and these scores performed similarly in HPTN 035 but had more modest predictive ability in FEM-PrEP.

To have the greatest impact on reducing new HIV infections, high-risk individuals should be targeted for access to existing effective prevention interventions, including PrEP, and also for roll-out of future prevention modalities. Use of screening tools, including the risk score presented here, could improve efficiency in identifying high-risk women and enable priority access to highly effective HIV prevention methods to those who would benefit most.