Virological measures and factors associated with outcomes, and missing outcome data in HIV clinical trials: a methodological study.

BACKGROUND: To evaluate the definition of HIV virological outcomes in the literature and factors associated with outcomes and missing outcome data.
METHODS: We conducted a methodological review of HIV RCTs using a search (2009-2019) of PubMed, Embase and the Cochrane Central Register of Controlled Trials.Only full-text, peer-reviewed, randomised controlled trials (RCTs) that measured virological outcomes in people living with HIV, and published in English were included.We extracted study details and outcomes. We used logistic regression to identify factors associated with a viral threshold ≤50 copies/mL and linear regression to identify factors associated with missing outcome data.
RESULTS: Our search yielded 5847 articles; 180 were included. A virological outcome was the primary outcome in 73.5% of studies. 89 studies (49.4%) used virological success. The remaining used change in viral load (VL) (33 studies, 18.3%); virological failure (59 studies, 32.8%); or virological rebound (9 studies, 5.0%). 96 studies (53.3%) set the threshold at ≤50 copies/mL; and 33.1% used multiple measures.Compared with government and privately funded studies, RCTs with industry funding (adjusted OR 6.39; 95% CI 2.15 to 19.00; p<0.01) were significantly associated with higher odds of using a VL threshold of ≤50 copies/mL. Publication year, intervention type, income level and number of patients were not associated with a threshold of ≤50 copies/mL. Trials with pharmacological interventions had less missing data (β=-11.04; 95% CI -20.02 to -1.87; p=0.02). DISCUSSION: Country source of funding was associated with VL threshold choice and studies with pharmacological interventions had less missing data, which may in part explain heterogeneous virological outcomes across studies. Multiple measures of VL were not associated with missing data. The development of formal guidelines on virological outcome reporting in RCTs is needed. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

To the best of our knowledge, this is the first study to investigate the range of thresholds used in HIV clinical trials and the factors associated with these thresholds.

We conducted a comprehensive and exhaustive search for HIV trials with virological outcomes.

In both regression models, candidate covariates were selected a priori based on methodological plausibility and previous research.

Only peer-reviewed randomised controlled trials consisting of people living with HIV and at least one virological outcome were included.

Only studies published in English and as full text were included.

Background

HIV viral load (VL) is the measure of HIV RNA detected in a blood sample reported as copies per mL.1 It is a surrogate measure of treatment response and is considered an important prognostic indicator of disease progression. In non-compliant patients, HIV VL monitoring offers an opportunity to address poor adherence and prevent adverse events such as resistance mutations.2–6 Antiretroviral therapy (ART) is effective for preventing viral replication, consequently reducing VL and the probability of developing resistant mutations.1 Achieving an undetectable HIV VL is also important from a public health standpoint, as there is a strong positive correlation between risk of transmission and VL.7 For some patients, VL testing may reinforce medication adherence. Additionally, VL can be used to identify ineffective treatment regimens and aid in regimen changes.2 3 Likewise, VL is the primary predictor of the risk of heterosexual, homosexual and mother-to-child transmission in patients with HIV.7

Despite the importance of VL as a prognostic indicator and endpoint in trials, the threshold used to define virological failure or success varies throughout the HIV literature. For example, the WHO defines virological failure as a plasma VL above 1000 copies/mL in two consecutive VL measurements following 3 months of ART, while the British HIV association defines failure as single confirmed measurement of plasma VL above 200 copies/mL.4 5 Other studies define virological failure as a VL above 400 copies/mL, with or without consecutive measurements.6 8 Furthermore, recent studies in high-income settings have defined virological failure as two consecutive VL values of greater than 50 copies/mL, or as a consistent elevation of greater than 50 copies/mL at predefined points in time, such as 24 or 48 weeks after the initiation of treatment.1 9 10 Advancements in therapy which promote viral suppression to ≤50 copies/mL in most patients, including those with multidrug resistance, may have contributed to lower VL thresholds.10–12 Additionally, with the availability of more sensitive instruments, the VL threshold which is considered undetectable varies depending on the instrument employed.

Guidelines recommend a regimen change in the absence of virological success. This becomes problematic in resource-limited settings if viral thresholds are set too low, as clinicians may be limited to a single, more costly second-line treatment regimen.1 13 In theory, a significant decline of VL should be associated with clinical benefit, but the appropriate decline is hard to interpret when studies use different thresholds, and the clinical significance is unclear.1 13 Thus, it is important to establish whether a given viral threshold is the product of advanced diagnostic testing or is clinically significant.13

Further, in HIV clinical trials, the most common primary endpoint is to maintain virological success over time. Variations in definitions of virological success or failure may preclude the comparison of trials and prevent pooling of trial data.9 Likewise, missing data in HIV clinical trials threatens internal validity, introducing error and decreasing the power of the trials.14 This is likely to occur if multiple measurements are needed to confirm outcomes. As such, we sought to explore the association between several measures of virological outcomes and levels of missing data in HIV clinical trials.

The aim of this paper is to review the VL thresholds used to define virological failure and the trial characteristics associated with lower thresholds. We hypothesised that higher income countries, industry-funded studies and more recent clinical trials would be associated with lower VL thresholds; and that multiple measures of virological outcomes would lead to more missing data in trials.

Methods

Design

We conducted a methodological study of HIV randomised controlled trials (RCTs) in the past 10 years (2009–2019). We searched three databases for RCTs: PubMed, Embase and the Cochrane Central Register of Controlled Trials. Our search strategy included keywords: HIV, ART, randomised trials and VL (online supplemental appendix 1). The process of title and abstract screening, full-text review and study selection were completed independently by two reviewers, with disagreement resolved by the senior author.

Selection criteria

Only peer-reviewed RCTs consisting of patients with HIV and at least one virological outcome were included. We excluded papers not published in English or without accessible full texts. We also excluded nested studies, studies that were parts of larger studies and secondary analyses of primary data.

Data extraction

Data were extracted by one of three reviewers and verified by a second independent reviewer. Bibliographical information (author, year, journal and country); income level of study site (using World Development Indicators database defined by Gross National Income per capita); study design; intervention details of treatment and control group participants; number of study sites; sample size; percentage with missing virological outcome data; source of funding (industry, private or government)15; length of follow-up; virological outcome definition (success, failure, change in VL as defined by the trial); number of time points for VL measurement; threshold used to define the virological outcome; and percentage of missing data were extracted. Study screening was completed on Rayyan QCRI16 and data collection was completed and managed using Distiller SR.17 Agreement was measured using the Kappa statistic.18 Disagreements were resolved by a third reviewer.

Analysis

Data were summarised as counts and percentages, mean (SD) or median (quartile 1; quartile 3) as appropriate. We performed multivariable logistic regression to determine factors associated with lower VL thresholds. In the logistic regression model, the dependent variable was binary for VL threshold (>50 copies/mL vs ≤50 copies/mL). Studies that did not use VL thresholds were excluded from the logistic analysis of VL cut-off. Predictor variables included study characteristics such as year of publication, source of funding (government or private vs industry), number of study sites, income level of study site (high vs other), intervention type (any trial using pharmacological interventions between study groups vs non-pharmaceutical intervention (eg, behavioural intervention)) and number of patients randomised. Crude OR, adjusted OR (aOR), corresponding 95% CIs and p values were reported. Model fit was evaluated using Akaike’s information criterion (AIC).

Multivariable linear regression was used to determine which factors are associated with higher percentages of missing data. In both regression models, candidate covariates were selected a priori based on methodological plausibility and previous research. The first model in both regression analyses included prespecified covariates and the second (reduced) model included a subset of covariates which met statistical significance. The threshold for statistical significance was set at α=0.05. Data were analysed using Stata, V.16.19

Results

Our search retrieved 5847 studies, of which 5205 were excluded for not reporting virological outcomes (n=2882); not being on HIV (n=1413); not being an RCT (n=892) or being a duplicate (n=18). During full-text assessment of the remaining 642 studies, 462 were excluded for lacking accessible full texts (n=270) or failing to meet eligibility criteria (n=192) due to not being RCTs (n=152), not having virological outcomes (n=37) or not including people with HIV (n=3). Our screening process is outlined in figure 1. A total of 180 studies were included and the median (quartile 1; quartile 3) year of publication was 2016 (2015; 2017). The characteristics of the included studies are reported in table 1. A virological outcome was the reported as the primary outcome in 130 (73.5%) of the included studies. Virological success was the most commonly reported outcome (89 studies, 49.4%), followed by virological failure (59 studies, 32.8%), change in VL (33 studies, 18.3%) and virological rebound (9 studies, 5.0%). More than half (90 studies, 54.4%) of the studies came from high-income countries, and most trials investigated pharmaceutical interventions (151 studies, 83.9%). About half of the included studies used a VL threshold of ≤50 copies/mL (96 studies, 53.3%). The remaining studies had widely variable thresholds for VL, ranging from 75 copies/mL to over 1000 copies/mL. Cohen’s kappa coefficient between the reviewers for the value of the threshold, including missing data, was 0.91, indicating high agreement.18

Figure 1

Flow of studies. RCT, randomised controlled trial.

Table 1

Characteristics of included studies (2009–2019)

Variable	N (%)
Overall	180 (100)
Year of publication: median (q1; q3)	2016 (2015; 2017)
Income level of study site
High	98 (54.4)
Upper middle	11 (6.1)
Lower middle	18 (10)
Low	10 (5.6)
Mixed (studies with multiple sites in different income levels)	43 (23.9)
Funding
Industry	54 (39.4)
Government or private	83 (60.6)
Intervention type
Pharmacological	151 (83.9)
Non-pharmacological	29 (16.1)
Number of sites
Single centre	40 (22.4)
Multicentre	139 (77.7)
Number randomised: median (q1; q3)	243 (101; 491)
Per cent missing: median (q1; q3)**	9.0 (3.1; 16.0)
Virological outcome as primary outcome	130 (73.5)
Outcome type††
Change in VL	33 (18.3)
Virological success	89 (49.4)
Virological failure	59 (32.8)
Virological rebound	9 (5.0)
Time elapsed before reaching VL threshold (yes)	16 (9.0)
Other	6 (3.3)
Thresholds used (copies/mL)††
1000 or greater	13 (7.3)
500	3 (1.7)
400	16 (8.9)
200	15 (8.3)
150	1 (0.6)
100	1 (0.6)
75	1 (0.6)
≤50	96 (53.3)
No threshold used	32 (17.8)
Number of time points required for confirmation of virological outcome
1	119 (66.9)
>1	59 (33.1)

*Percentage of missing data for virological outcomes in all trials.

†Study may have more than one outcome.

‡Study may have more than one viral threshold.

Q, quartile; VL, viral load.

Factors associated with a viral threshold of 50 copies/mL or less

In univariable analyses, trials with pharmaceutical interventions (OR 6.39; 95% CI 2.15 to 19.00; p<0.01), trials with industrial source of funding (OR 7.70; 95% CI 2.68 to 22.10; p<0.01) and studies from high-income countries (OR 3.74; 95% CI 1.83 to 7.63; p<0.01) were more likely to use a viral threshold of ≤50 copies/mL (table 2).

Table 2

Univariable and multivariable logistic regression analyses to detect factors associated with viral threshold of 50 or less

VL threshold ≤50	Univariable analysis			Model 1: multivariable analysis			Model 2: multivariable analysis
	OR	95% CI	P value	aOR	95% CI	P value	aOR	95% CI	P value
Year of publication	0.96	0.75 to 1.22	0.72	0.91	0.64 to 1.30	0.63	–	–	–
Source of funding (industry)*	7.70	2.68 to 22.10	<0.01	5.66	1.77 to 18.09	<0.01	6.39	2.18 to 18.74	<0.01
Number of sites (multicentre)†	1.73	0.77 to 3.90	0.12	1.29	0.40 to 4.17	0.67	–	–	–
Income level (high)‡	3.74	1.83 to 7.63	<0.01	2.47	0.92 to 6.67	0.07	2.46	0.976 to 6.20	0.06
Intervention type (pharmaceutical)§	6.39	2.15 to 19.00	<0.01	2.98	0.71 to 12.63	0.14	–	–	–
Number of patients randomised¶	0.81	0.41 to 1.59	0.54	0.75	0.26 to 2.17	0.60	–	–	–

P<0.05 indicates statistical significance. aOR: adjusted OR.

Model 1: Multivariable analysis: Pseudo R2=0.19; p=0.0002.

Studies that did not use VL thresholds were excluded.

*Government and private source.

†Single centre.

‡Mixed, middle or low income.

§Non-pharmaceutical intervention.

¶Less than 250 randomised.

aOR, adjusted OR; VL, viral load.

In the first model, including all covariates, the multivariable analyses demonstrated that industry-funded studies (aOR 5.66; 95% CI 1.77 to 18.09; p<0.01) were associated with lower thresholds. However, in the second multivariable logistic regression model, which included only covariates that were significant in the first model, only industry funded studies were at high odds of having VL thresholds of ≤50 copies per mL (aOR 6.39; 95% CI 2.15 to 19.00; p<0.01). The logistic regression model including all the covariates appeared to have a similar fit to the reduced model (AIC=1.112; AIC=1.078, respectively). The results of both univariable and multivariable logistic regression models for factors associated with virological thresholds of ≤50 copies/mL are included in table 2.

Factors associated with percentage of missing outcome data

In univariable analysis, none of the selected covariates showed a significant association with missing outcome data (table 3).

Table 3

Univariable and multivariable linear regression analyses to detect factors associated with percentage of missing outcome data

Percentage of missing data	Univariable analysis			Multivariable analysis
	Coef.	95% CI	P value	Coef.	95% CI	P value
Year of publication	0.56	−0.96 to 2.10	0.47	0.06	−1.94 to 2.07	0.95
Source of funding (industry)*	−1.56	−6.78 to 3.66	0.56	3.57	−2.73 to 9.88	0.26
Intervention type (pharmaceutical)†	−1.74	−7.25 to 3.77	0.53	−11.04	−20.02 to −1.87	0.02
Number of sites (multicentre)‡	0.16	−4.75 to 4.79	1.00	4.93	−2.03 to 11.89	0.16
Income level (high)§	−1.25	−5.32 to 2.81	0.54	−2.10	−8.08 to 3.88	0.49
Outcome type (virological failure)¶	1.98	−2.32 to 6.29	0.37	3.25	−2.40 to 8.90	0.26
Viral threshold (≤50 copies/mL)	−4.08	−8.85 to 0.76	0.10	−2.22	−8.77 to 4.32	0.50
Length of follow-up (weeks)	0.09	−0.17 to 0.36	0.50	−0.11	−0.53 to 0.32	0.62
Number of patients randomised (more than 250)**	1.42	−2.63 to 5.47	0.49	−3.54	−9.82 to 2.73	0.27
Number of viral load measurement time points (more than one)††	−0.91	−5.38 to 3.56	0.69	2.57	−3.54 to 8.68	0.41

Multivariable model: R2=0.10, p=0.38.

Bold value is statistically significant (ie P<0.05).

*Government and private.

†Non-pharmaceutical.

‡Single centre.

§Mixed, middle or low income.

¶Outcome type: not virological failure.

**Less than 250 randomised patients.

††One-time time point.

Coef, Coefficient.

After adjusting for other covariates, trials with pharmaceutical interventions were associated with 11% less missing outcome data compared with trials that involved non-pharmaceutical interventions (95% CI −20.02 to −1.87; p=0.02).

Discussion

In this methodological review, we have shown that studies use different measures to define virological outcomes in HIV clinical trials. The use of VL thresholds ≤50 copies/mL appears to be associated with industry-funded studies and studies conducted in high-income countries. To the best of our knowledge, this is the most up to date and comprehensive review of HIV RCT viral thresholds, highlighting the importance of formal guidance for appropriate thresholds in HIV trials.

The variability in definitions of viral threshold in the included trials is concerning, given the clinical and research implications of VL. If universal formal guidance for appropriate thresholds in HIV trials is not developed, differing endpoints will compromise pooling of study results. Clinically, threshold guidelines may persuade practitioners to maintain or switch treatment regimens. The introduction of ultrasensitive assays (<5 copies/mL) may also lead to increased incidence of detectable viraemia or viral blips, followed by consistent undetectable measurements. These viral blips are difficult for clinicians to interpret as their clinical aetiology and prognostic significance continues to be questioned in the literature.19 20 Some studies have pointed to a potential decrease in drug bioavailability or reduced compliance leading to resistance, while others point to random biological and statistical variation with no clinical implications.21 Thus, if viral thresholds are set too low, unnecessary regimen changes may be made, which would be problematic in resource-limited settings. Conversely, viral thresholds which are set too high may result in patient harm.22

In this study, RCTs funded by industry were more likely to have a viral threshold of ≤50 copies/mL. The literature shows that sources of funding play a large role in the design and reporting of many RCTs, with HIV RCTs being no exception.23–26 Industry studies operate with larger budgets than government-funded or privately-funded studies, enabling them to run more sensitive tests using better equipment.27–30 This is in line with our findings.

Funding may also be linked to the country’s income level. We did not find an association between income level of the country in which the study was conducted and viral thresholds of ≤50 copies/mL. Although, differing guidelines in higher-income countries affect resources, laboratory capabilities, health priorities, research standards and pharmaceutical manufacturing capacity, sensitive VL testing equipment is beginning to become more affordable and accessible to all populations.26 30 31

Despite the increasing sensitivity of VL testing equipment and health administrators advocating for lower and more sensitive VL thresholds, there was no association between the year of publication and a viral threshold of ≤50 copies/mL.32–36 This may suggest that access to sensitive diagnostic technology may not have changed over time. However, as many studies did not report the year in which the trial was conducted, publication year was used as a surrogate to trial date, and may not accurately reflect the relationship between trial year and VL threshold.

Even though the number of VL measurements were not associated with missing data, we found that trials with pharmaceutical interventions were associated with lower levels of missing data. As pharmaceutical interventions are often less complex in design than non-pharmaceutical interventions and may represent the only opportunity for care in low-income settings, subjects are more likely to attend visits.37–39 Further, these studies tend to have a more robust and formalised data collection system, with increased financial incentive to maintain low levels of missing data or use enhanced strategies to prevent loss in follow-up.40

Our findings should be interpreted with caution as our study sample only included articles with full-text availability. Furthermore, studies that were not published in English, conference studies and grey literature were not included, which may limit the generalisability of our findings.

Future studies should continue to evaluate VL thresholds both to establish and implement formal guidelines regarding VL thresholds, especially in the context of research, but also to re-evaluate the role of VL assays and optimise their clinical value. With the development of technology such as ultrasensitive assays (PCR undetectable at <5 copies/mL), and application of dynamic models, researchers and clinicians alike must be wary of targeting lower thresholds with no added clinical benefit. Further research can help delineate the threshold at which clinical benefits are certain. As technology develops and countries obtain more sensitive assays, perhaps the only practical recommendation would be for trials to report a variety of thresholds to allow for trials to be compared and pooled.

Conclusion

Virological outcomes are inconsistent across HIV RCTs, with differences explained in part by country income and source of funding. To advance HIV research, the development and implementation of formal guidelines on reporting VL thresholds in HIV RCTs is warranted.