HIV surveillance based on routine testing data from antenatal clinics in Malawi (2011-2018): measuring and adjusting for bias from imperfect testing coverage.

OBJECTIVE: The use of routinely collected data from prevention of mother-to-child transmission programs (ANC-RT) has been proposed to monitor HIV epidemic trends. This poses several challenges for surveillance, one of them being that women may opt-out of testing and/or test stock-outs may result in inconsistent service availability. In this study, we sought to empirically quantify the relationship between imperfect HIV testing coverage and HIV prevalence among pregnant women from ANC-RT data.
DESIGN: We used reports from the ANC Register of all antenatal care (ANC) sites in Malawi (2011-2018), including 49 244 monthly observations, from 764 facilities, totaling 4 375 777 women.
METHODS: Binomial logistic regression models with facility-level fixed effects and marginal standardization were used to assess the effect of testing coverage on HIV prevalence.
RESULTS: Testing coverage increased from 78 to 98% over 2011-2018. We estimated that, had testing coverage been perfect, prevalence would have been 0.4% point lower (95% CI 0.3-0.5%) than the 7.9% observed prevalence, a relative overestimation of 6%. Bias in HIV prevalence was the highest in 2012, when testing coverage was lowest (72%), resulting in a relative overestimation of HIV prevalence of 15% (95% CI 12-17%). Overall, adjustments for imperfect testing coverage led to a subtler decline in HIV prevalence over 2011--2018.
CONCLUSION: Malawi achieved high coverage of routine HIV testing in recent years. Nevertheless, imperfect testing coverage can lead to overestimation of HIV prevalence among pregnant women when coverage is suboptimal. ANC-RT data should be carefully evaluated for changes in testing coverage and completeness when used to monitor epidemic trends.

Introduction

Quality surveillance systems are at the core of HIV prevention and control efforts worldwide. In high HIV-burden countries where a noticeable proportion of the heterosexual population is living with HIV, sentinel surveillance has historically been conducted among pregnant women attending antenatal care (ANC-SS) [1]. Despite well characterized limitations [1-5], such data are an important input for tracking HIV epidemics in sub-Saharan Africa and generating model-based estimates of national and subnational HIV prevalence, incidence, and AIDS-mortality [6].

More recently, the use of routinely collected data from prevention of mother-to-child transmission (PMTCT) programs (ANC-RT) has been proposed to monitor epidemic trends [7,8]. WHO defines, such program data as ‘data routinely generated through ANC and PMTCT service delivery, and routinely recorded in standard site data tools’ [9]. Transitioning from ANC-SS to ANC-RT could have several advantages. First, this move would adhere to current ethical standards [10,11], as women could freely opt-out of testing, be offered pretest and posttest counselling, have their test results returned to them, and those testing positive could be referred to PMTCT and appropriate care and treatment services [7,12]. Second, it would alleviate the financial and logistical burden of conducting ANC-SS while offering the opportunity to strengthen PMTCT activities nationwide. Finally, it would expand the representativeness, granularity, and geographical coverage of previous surveillance systems that were based on a convenience sample of sites [9].

Use of ANC-RT data for surveillance poses unique challenges, however. First, historical trends in HIV prevalence from PMTCT programs are not available and statistical models are needed for appraisal of HIV epidemic trends when transitioning from ANC-SS to ANC-RT [13]. Second, the quality of HIV testing during ANC and data collection systems may vary between sites [14]. Third, women may opt-out of HIV testing and staffing shortages or test kit stock-outs may interrupt services, leading to reduced HIV status ascertainment rates. Depending on who gets ascertained, this could lead to over-estimation or under-estimation of HIV prevalence among this group of women [15]. Prevalence may be overestimated if known HIV-positive women are overrepresented when testing services are disrupted; if women perceived at higher risk are prioritized for testing when test kits are in short supply; and/or if HIV-negative women are more likely to opt-out of testing. Alternatively, HIV prevalence could be underestimated if known HIV-positive women are omitted from facility records or if women who opt-out are more likely to be living with HIV than those that accept HIV testing during ANC.

To limit the potential for selection bias, WHO recommends at least 90% HIV testing coverage at all sites [7,12]. This uptake level was defined based on simulations of prevalence ratios among women not receiving HIV testing (as compared with those who do) and level of uptake of HIV testing [7]. These simulations did not account for women self-reporting their known HIV status, however, and implicitly assumed that HIV prevalence is higher among women opting out of HIV testing.

Using detailed PMTCT data from Malawi (2011–2018), we aim to empirically estimate the impact of suboptimal coverage and uptake of HIV testing of pregnant women at ANC on estimates of HIV prevalence. The implications of suboptimal coverage for monitoring trends are discussed and general recommendations are provided regarding use of such routine data for HIV surveillance.

Methods

PMTCT has been a long-standing public health priority in Malawi [16,17]. In 2011, ‘Option B+’ was introduced – recommending immediate lifelong antiretroviral therapy (ART) initiation for all HIV-positive pregnant women – and PMTCT activities were fully integrated into maternal and child health services [18]. Both total fertility and HIV prevalence are high: total fertility was 4.4 children and 10.8% of women aged 15–49 years were living with HIV in 2015–2016 [19]. Coverage and uptake of ANC is near universal in Malawi, with 98.2% of women aged 15–49 years who had a live birth having attended at least one ANC visit [19].

Data sources

Data from the ‘Department of HIV and AIDS Management Information System’ (DHA-MIS) were used for this analysis. This electronic database has complete service records from all 764 health facilities with HIV services in all 29 districts in Malawi. The DHA-MIS includes monthly ANC facility reports that are based on manual aggregation from paper registers at most facilities and where HIV is ascertained at all ANC visits (see Figure S1 for an excerpt of the Malawi ANC Clinic Register). PMTCT and ART data is considered of high quality in Malawi because of the quarterly supportive supervision system for all facilities. Supervision visits include a systematic review of primary patient records and a quality audit of service reports. We based our analysis on monthly facility-level data collected between July 2011 and June 2018.

Measures

HIV status during ANC visits is recorded in five exhaustive and mutually exclusive categories. The first two categories are for known HIV status: either a documented positive test (or evidence that the women is on antiretroviral therapy) or a documented negative test in the 3 months prior to the first ANC visit. If the status is unknown, a new HIV test should be performed. Its outcome is then recorded as either a ‘new positive’ or ‘new negative’ result. In instances where the women's HIV status is not ascertained, this information should be collected in the last field. In this article, we measured coverage of HIV testing as follows:

where Cov is the measured coverage of HIV testing at facility i during month t; Total ANC is the total number of pregnant women presenting for ANC for that month and facility; and Not Tested is the number of women for which the HIV status was not ascertained. Our measure of HIV prevalence (Prv) at facility i at time t considers as the numerator the number of women with a documented prior test (Previous Pos) and the number of new positive tests performed as part of ANC (New Pos). The denominator consists of all women who had their HIV status ascertained (previous negative, previous positive, and all newly tested).

Statistical analyses

Our analyses of the impact of imperfect HIV testing coverage during ANC visits on estimates of HIV prevalence among pregnant women is based on the following two main assumptions. First, we posit that there are secular trends in HIV prevalence from 2011 to 2018. Second, after adjusting for these potential time trends, the variations in facility-level HIV prevalence estimates are explained by changes in HIV testing coverage and some random error. Potential confounding of the relationship between HIV prevalence and testing coverage by unobserved facility/catchment area characteristics can be accounted for by including facility-level fixed effect and we fitted models with and without these fixed effects. Fixed effects can control for any measured or unmeasured time-invariant facility-level confounders and we, therefore, preferred them to a random effects specification. Specifically, our chosen binomial logistic regression model takes the following form:

where Prv is the observed HIV prevalence, α is the intercept, γ(Cov) is a cubic b-spline for coverage of HIV testing at ANC with two equidistant knots (i.e. at coverage of 33 and 66%), δ is the vector of coefficient for time (Quarter, where time is categorized as yearly quarters to provide a less parametric specification than assuming a linear trend), and ωi is the vector of coefficients for the facility fixed-effects (Facility).

The preceding binomial regression model produces effect size measures on the log odds ratio scale but their interpretation would be more intuitive if these were reported on the risk difference scale. To this end, we used marginal standardization to calculate absolute bias. This was implemented using the following formula for risk differences (RD):

where the first fraction is a weighted average of expected prevalence estimates at the observed coverage level ϕ and where other covariates are also set to their observed values, defined in Z. The second fraction in the equation defines the contrast. In this case, Ψ is the alternative coverage level, here defined at 100% (i.e.Ψ = 100%). The resulting RD estimate considers that facilities with a higher number of tested ANC attendees contribute more to the overall prevalence estimates, and re-weight monthly facility-level observations through the denominators of the two fractions of this RD. For example, the denominator considers that facilities that previously had low testing coverage (ϕ) will see their weight increase because of the now perfect testing coverage (Ψ) (i.e. more women are being tested). Effect sizes estimates on the relative risk scale are calculated using the same equation as above, but dividing instead of subtracting the two estimates. The 95% confidence intervals (95% CI) are estimated using 1000 replicates of a block-of-block bootstrap where the resampling unit is defined as the facility to account for the within-facility serial correlation structure.

Additional analyses were carried out to examine how the impact of imperfect HIV testing affects the validity of ANC-RT data. First, we assessed the WHO recommendation of a minimum benchmark of 90% HIV testing coverage at all sites. We did so by estimating the bias in HIV prevalence that would be observed if all facilities had 90% coverage (i.e.,ϕ = 90%) as compared with perfect coverage. We also investigated whether biases in prevalence, if any, arise only because some women present at ANC already diagnosed (previous positive) and/or with a recent HIV-negative test (previous negative). To investigate this, we fitted the same regression model as described in the preceding paragraphs, while selectively excluding those with already known status from the numerators and denominators of both the prevalence and coverage estimates. If biases result entirely from women who already knew their HIV-positive status, the effect of imperfect coverage on HIV prevalence should be null.

All analyses were carried out using the R statistical software [20].

Results

Over the 2011–2018 period, 764 facilities reported information on 4 375 777 women. After removing observations for months in which no women presented at ANC facilities (93 facilities did not record any women attending ANC), a total of 49 244 monthly facility records were used for our analyses. ANC attendance was relatively stable over time but decreased slightly from 636 483 women in 2012 to 574 802 in 2017. The average HIV testing coverage was 86.9% between 2011 and 2018 (median was 97.1%); 67.0% of monthly facility reports showed 90% testing coverage or higher (Figure S2). This pattern varied in time, however, with coverage increasing from 72.1% in 2012 to 97.9% in 2018 (Table 1).

Table 1

HIV testing coverage, observed HIV prevalence, predicted HIV prevalence with perfect testing coverage, and absolute and relative bias because of imperfect HIV testing coverage in Malawi, by calendar year and overall (2011–2018).

	Testing Coveragea			Without facility-level fixed effects			With facility-level fixed effects
Period	Mean	Median (IQR)	Observed HIV prevalence	Predicted prevalence at 100% coverage	Absolute bias (95%CI)	Relative bias (95%CI)	Predicted prevalence at 100% coverage	Absolute bias (95%CI)	Relative bias (95%CI)
2011b	77.6%	85.9% (65.6–97.7%)	8.5%	8.3%	0.2% (−0.4 to 0.6%)	2% (−4 to 8%)	7.7%	0.8% (0.7–0.9%)	11% (9 to 12%)
2012	72.1%	80.8% (54.3–96.7%)	8.8%	8.6%	0.2% (−0.3 to 0.7%)	2% (−4 to 9%)	7.7%	1.1% (0.9–1.3%)	15% (12–17%)
2013	80.1%	89.2% (69.2–98.5%)	8.0%	7.9%	0.1% (−0.3 to 0.6%)	2% (−4 to 8%)	7.4%	0.7% (0.5–0.8%)	9% (7–11%
2014	85.7%	93.8% (79.0- 99.6%)	7.7%	7.6%	0.1% (−0.3 to 0.5%)	1% (−3 to 7%)	7.3%	0.4% (0.3–0.5%)	6% (4–7%)
2015	91.1%	97.6% (88.7–100.0%)	7.8%	7.7%	0.1% (−0.2 to 0.4%)	1% (−3 to 5%)	7.5%	0.3% (0.2–0.3%)	4% (2–4%)
2016	94.5%	99.2% (95.0–100.0%)	7.6%	7.5%	0.1% (−0.1 to 0.3%)	1% (−2 to 4%)	7.4%	0.2% (0.2–0.3%)	3% (2–4%)
2017	97.3%	100.0% (98.4–100.0%)	7.6%	7.5%	0.1% (−0.1 to 0.2%)	1% (−1 to 2%)	7.4%	0.1% (0.1–0.2%)	2% (1–2%)
2018b	97.9%	100.0% (99.2–100.0%)	7.4%	7.4%	0% (−0.1 to 0.1%)	0% (−1 to 2%)	7.3%	0.1% (0.1–0.1%)	1% (1–2%)
2011–18	86.9%	97.1% (82.1–100.0%)	7.9%	7.8%	0.1% (−0.3 to 0.4%)	1% (−3 to 5%)	7.4%	0.4% (0.3–0.5%)	6% (5–7%)

95% CI, 95% confidence intervals; IQR, interquartile range.

aMean, median, and IQR are weighted by the number of women attending antenatal care.

bInformation for only part of the year is available: in 2011 from July onwards and until May for the year 2018.

Observed HIV prevalence among women attending ANC in Malawi was 7.9% over the study period (Table 1). It decreased from a high of 8.8% in 2012 to 7.4% in 2018: a 1.4% point difference over 6 years. This reduction in observed HIV prevalence, as measured through ANC-RT, could be confounded by the concomitant increases in HIV testing coverage.

Impact of imperfect HIV testing coverage on prevalence estimates

Results from two models are presented: a model without facility-level fixed effect and a model that includes them. The model without facility-level fixed effects suggests that predicted HIV prevalence under the perfect coverage scenario would be 0.1% point lower (95% CI −0.3 to 0.4%) than the observed HIV prevalence over 2011–2017 but the confidence interval of this difference is wide (Table 1). Examining the results stratified by year, this difference was also highest for the year 2012, when testing coverage was lowest at 72%.

Accounting for potential unobserved confounders related to facility-level/catchment area characteristics (i.e. public/private facilities, geographic area, etc.) with fixed-effects resulted in a reduction of predicted HIV prevalence at perfect coverage (Table 1). Over the study period, HIV prevalence would have been 0.4% point lower (95% CI 0.3--0.5%), at 7.4%, had coverage been 100% at all time and for all sites. The relative overestimation of the observed HIV prevalence at imperfect coverage was 6% overall. Again, this bias was highest in 2012 because of the lowest testing coverage. For that year, the observed HIV prevalence overestimates the true HIV prevalence by 1.1% point – HIV prevalence was overestimated by a factor of 15%. Examining results stratified by region and year, we found that HIV prevalence could have been overestimated by as much as 18% (95% CI 13–24%) and 17% (95% CI 14–20%) in the Northern and Central regions in 2012, respectively (Table S1). The fact that the inclusion of facility-level fixed effects reduced predicted HIV prevalence estimates under the perfect testing coverage scenario suggests that HIV prevalence was higher among women attending facilities with higher testing coverage. For this reason, calculating HIV prevalence including facilities that have high testing coverage (i.e. >90%) could bias estimates: this approach would de facto select facilities with higher HIV prevalence. Indeed, HIV prevalence among pregnant women in Malawi would have been estimated at 7.8% over 2011–2018 if only observations with a minimum of 90% coverage are selected, as compared with 7.4% using the regression-based model with facility-level fixed effects.

The relationship between testing coverage and HIV prevalence is not linear (Fig. 1). As coverage approaches zero, HIV prevalence increases dramatically. When testing coverage is above 20%, a small but consistent decrease in HIV prevalence is observed. For example, if all facilities had testing coverage of 90% at all time over the study period, the observed HIV prevalence would have been 7.8% whereas the predicted prevalence with perfect coverage would have been 7.5% (difference of 0.3% point; 95% CI 0.1--0.5%), leading to a relative 4% overestimation (95% CI 2–6%). Our results were robust to the chosen functional form for the effect of testing coverage on HIV prevalence: spline specification with one knot (at 33--50% coverage), with two knots placed differently (at 25 and 50%; at 25 and 60%), with three equidistant knots, four equidistant knots, and five equidistant knots all produced similar results.

Fig. 1

Relationship between routine HIV testing coverage during antenatal care and predicted HIV prevalence in Malawi (solid line) with 95% confidence interval (shaded area).

The plot displays the estimated cubic b-spline with two equidistant knots. (Coefficients of the splines are obtained from the model that includes facility-level fixed effects.) The dots on the graph are the monthly observations of HIV prevalence at antenatal care. The size of the dots is proportional to the number of women attending antenatal care.

Our results so far suggest that imperfect coverage of routine HIV testing could lead to overestimation of ANC HIV prevalence. To test if this bias is explained by already diagnosed women being selectively included in the ANC prevalence estimates – which could be the case if HIV test kits are stocked-out and only the status of already diagnosed women can be recorded – we replicated our analyses selectively excluding women with known status from both prevalence and coverage estimates. Doing so, the observed proportion of women testing HIV positive was 4.4% (95% CI 4.3–4.9%) over 2011–2018. Under a scenario of perfect testing coverage, however, the proportion testing positive would have been 0.14% point lower (95% CI 0.1–0.2%). This small 3% (95% CI 2–5%) relative overestimation is also lower than the 6% relative overestimation estimated when those women are included. This suggest that bias in ANC HIV prevalence is the results of both selective inclusion of already diagnosed pregnant women and selective testing of pregnant women living with HIV when testing coverage is imperfect.

Discussion

Malawi has achieved high coverage of routine HIV testing in recent years, with more than 95% of pregnant women presenting at ANC having their HIV status ascertained in 2017. However, coverage was previously suboptimal and our analyses demonstrate that imperfect testing coverage can lead to an overestimation of national HIV prevalence. For example, we found that observed prevalence was 15% higher in 2012 because of lower ascertainment rates: in absolute terms, the measured prevalence was 1.1% points higher than the actual prevalence that would have been observed if the 28% of ANC women with unknown status had their HIV status ascertained. Our results also have implication for interpreting the observed HIV decline among women presenting at ANC. From 2011 to 2018, there was an observed 13% relative reduction in HIV prevalence (from 8.5 to 7.4%). However, if we adjust for imperfect testing coverage, which was more common in the early 2010s, the HIV prevalence decline was much subtler: it decreased from 7.7 to 7.3%, a 5% relative reduction. Important longitudinal changes in HIV testing coverage could thus potentially bias observed epidemic trends.

The WHO recommends that selection bias, defined as the percentage relative change in HIV prevalence among all pregnant women and those being tested for HIV at ANC, be less than 10% [7]. In Malawi, we found that the upper limit of the confidence interval of this relative bias was below this 10% threshold for all years where testing coverage was above 85%. This supports the WHO recommendation that, as a general standard, uptake of HIV testing of 90% or greater may be considered high [7]. However, even if all facility had a 90% coverage, the observed HIV prevalence could be affected by a small relative bias of 4%.

Our results suggest that pregnant women living with HIV in Malawi are more likely to have their HIV status ascertained when testing coverage is imperfect. This contrasts with serosurveys where individuals who opt-out of HIV testing are found to be more likely to be living with the virus [21-25]. These studies targeted the general population in nonclinical settings; however, and not specifically pregnant women attending ANC. This important difference might explain the observed discrepancy. Contrary to general population surveys, where HIV stigma could affect survey nonresponse [26,27], pregnant women attending ANC might be motivated to test for HIV because their unborn child has the potential to benefit from PMTCT. These results are consistent with previous studies that found higher HIV prevalence using ANC-RT data as compared with ANC-SS [28-30]. However, this does not explain the specific mechanisms through which pregnant women living with HIV are more likely to be tested. We hypothesize that women with high risk of HIV acquisition (e.g. a sexual partner living with HIV) and/or symptoms resulting from a concurrent (or previous) sexually transmitted infection could make them more likely to have their HIV status ascertained.

Inclusion of facility-level fixed effects in the regression models increased the effect size estimates for coverage, suggesting that not controlling for facility biased the association between testing coverage and HIV prevalence towards the null. This implies that the catchment populations of facilities with better HIV testing coverage also tend to have higher HIV prevalence. For example, facilities providing ANC services in urban areas might be less likely to experience stock-outs of HIV tests and attended by populations where HIV prevalence is higher. As the relationship between HIV testing coverage and HIV prevalence could be confounded by such characteristics, simple HIV prevalence adjustments based on inclusion/exclusion of observations with suboptimal testing coverage could introduce further biases. For this reason, managers of PMTCT programs should ensure high coverage of HIV testing and carefully assess potential biases in ANC-RT HIV prevalence estimates when coverage is imperfect for monitoring epidemic trends.

Some limitations need to be acknowledged. First, we did not address the issue that testing accuracy could be lower for routine HIV testing of pregnant women than when performed as part of ANC sentinel surveillance. Further, the quality of routine testing for ANC-RT and data collection can also vary between facilities and over time [11]. Issues of testing accuracy have the potential to further bias prevalence estimates based on ANC-RT. Evidence from several countries suggest that estimates from ANC-SS and ANC-RT are generally consistent and/or their agreement is improving with time [28,31-35]. The proportion of women testing positive in the ANC sentinel surveillance that also tested positive during routine ANC testing ranged between 76 and 98% in these studies. Such discrepancies also highlight a potentially high number of false negative results in ANC-RT. Testing quality should be monitored in Malawi and elsewhere to minimize such bias. Further, HIV prevalence and HIV testing acceptance have been shown to increase with age in serosurveys of the general population [22]. We did not analyze potential age-specific biases resulting from imperfect coverage as this level of data stratification was not available. The magnitude and, potentially, direction of bias could differ between age groups, however. As monitoring of HIV epidemics increasingly uses age-stratified information, future studies should examine if these biases are constant within age groups.

Strengths of our study include its large sample size of more than 4.3 million pregnant women attending ANC, from 764 unique health facilities, over a period of more than 5 years, with important variation in testing coverage. In addition, we conducted detailed statistical analyses and our results were robust to different spline specifications for testing coverage. Further, our inferences are based on Malawi's monitoring system for HIV surveillance, where completeness and reporting quality are of high standards. Although this clearly supports the internal validity of our findings, surveillance systems in other countries might face further challenges in interpreting epidemic trends if data completeness is substandard.

In conclusion, we provide empirical estimates demonstrating that imperfect coverage of routine HIV testing of pregnant women can bias HIV prevalence estimates. These findings call for ANC-RT data to be carefully evaluated and appropriately adjusted for changes in testing coverage and completeness when used to monitor epidemic trends. At a minimum, countries should examine longitudinal variations in testing coverage, potentially excluding temporal periods where testing coverage is low, before interpreting changes in HIV prevalence. Our recommendations may also be relevant for surveillance of other health conditions, such as syphilis, that are also routinely tested for at ANC and for which testing coverage might also be imperfect. As countries move programs forward to eliminate mother-to-child transmission, ANC-RT surveillance should provide many benefits, such as reduced costs, improved geographical coverage, and strengthen overall surveillance and health systems [7]. These benefits should not be outweighed by the limitations outlined in this article if the transition from ANC-SS to ANC-RT is supported by careful bias assessment and ongoing HIV testing and data quality monitoring.

Declarations

Ethics

All analyses were performed on routinely collected ANC data that has been anonymized, de-identified, and aggregated at the facility level at monthly intervals. Ethics approval was obtained from McGill University's Faculty of Medicine Institutional Review Board.

Data availability

The data used for this study can be downloaded from the Malawi Ministry of Health website (www.hiv.health.gov.mw).

Acknowledgements

Funding: M.M.G.'s research program is funded by a career award from the Fonds de recherche du Québec – Santé. J.W.E. was supported by NIH R03-AI125001 and UNAIDS.

Authors’ contributions: A.J., J.W.E., and M.M.G. conceived and designed the study. A.J., T.K., and A.M. obtained and administered the databases. M.M.G. performed the analyses and A.J., A.M., J.W.E., T.K., and M.M.G. interpreted them. M.M.G. drafted the manuscript and all authors critically reviewed it for important intellectual content. All authors approved the final version.

Conflicts of interest

There are no conflicts of interest.