A systematic review of whether COVID-19 randomized controlled trials reported on demographic and clinical characteristics.

PURPOSE: We aim to assess the reporting of key patient-level demographic and clinical characteristics among COVID-19 related randomized controlled trials (RCTs).
METHODS: We queried English-language articles from PubMed, Web of Science, clinicaltrials.gov, and the CDC library of gray literature databases using keywords of "coronavirus," "covid," "clinical trial" and "randomized controlled trial" from January 2020 to June 2021. From the search, we conducted an initial review to rule-out duplicate entries, identify those that met inclusion criteria (i.e., had results), and exclude those that did not meet the definition of an RCT. Lastly, we abstracted the demographic and clinical characteristics reported on within each RCT.
RESULTS: From the initial 43 627 manuscripts, our final eligible manuscripts consisted of 149 RCTs described in 137 articles. Most of the RCTs (113/149) studied potential treatments, while fewer studied vaccines (29), prophylaxis strategies (5), and interventions to prevent transmission among those infected (2). Study populations ranged from 10 to 38 206 participants (median = 100, IQR: 60-300). All 149 RCTs reported on age, 147 on sex, 50 on race, and 110 on the prevalence of at least one comorbidity. No RCTs reported on income, urban versus rural residence, or other indicators of socioeconomic status (SES).
CONCLUSIONS: Limited reporting on race and other markers of SES make it difficult to draw conclusions about specific external target populations without making strong assumptions that treatment effects are homogenous. These findings highlight the need for more robust reporting on the clinical and demographic profiles of patients enrolled in COVID-19 related RCTs.

We assessed whether and how COVID‐19 related randomized controlled trials (RCTs) described demographic and clinical characteristics of their participants.

In this systematic review of all 149 COVID‐19 related RCTs published through June 2021, 100% reported on age, 99% on sex, 34% on race, and 74% on the prevalence of at least one comorbidity, with the two most frequently reported being diabetes (65%) and hypertension (54%), and 0% on SES indices.

The findings highlight the need for more robust reporting on the clinical and demographic profiles of COVID‐19‐related trial populations and creation of policies that ensure characteristics are adequately and consistently reported in publications.

INTRODUCTION

Researchers worldwide have been searching for ways to fight coronavirus disease 2019 (COVID‐19) since December 2019. While in‐vitro work and observational studies in humans can help identify potential interventions and treatments, randomized controlled trials (RCT) are generally considered the gold standard for assessing efficacy of treatment or prophylaxis. Unfortunately, enrolling participants in RCTs evaluating treatments for pandemics like COVID‐19 is difficult. , Because RCT participation is often linked to a desire to receive the best possible treatment, those at the highest risk of complications from the disease may be less willing to risk being randomized to not receive the study treatment when it is available through compassionate use, as in COVID‐19. , COVID‐19 treatment trial participants may under‐represent those with higher absolute risk of mortality.

Unfortunately, many treatments with beneficial effects in some individuals have no effect (or harmful effect) in others. If a treatment is equally beneficial for everyone on the relative scale, the number needed to treat will then vary across populations with differing baseline risks of the outcome. , Given that case fatality rates from COVID‐19 vary markedly as a function of sex, age, race/ethnicity, and respiratory and cardiovascular comorbidities, , , treatment effect estimates from RCTs may not reflect overall benefits and harms in a real‐world population. Further, depletion of the risk pool and relationships between socioeconomic status (SES) and infection risk may shift the characteristics of the infected population over time, meaning that early RCT results may fail to generalize to later real‐world populations.

RCT participation among minority groups has long been a concern in the United States. Since 1993, the NIH Revitalization Act requires minorities to be included in all NIH‐funded research and a 2017 NIH policy revision required trials to investigate treatment response among racial/ethnic groups and sex. , , Despite these policies, those with poor access to healthcare and lower SES have been, historically, underrepresented in RCTs and early work has already documented this trend continuing in COVID‐19 RCTs.

As a result, it is important to understand the patterns of participation in RCTs aimed at treating COVID‐19 or preventing the spread of COVID‐19 within and across communities. Before these patterns can be described, however, we must identify whether COVID‐19 RCTs are reporting on key demographic and clinical characteristics. To this end, we conducted a systematic review of COVID‐19 RCTs by time period and intervention types, and the extent to which these RCTs reported on key patient demographic and clinical characteristics.

METHODS

Search strategy

A protocol‐based search of electronic English‐language databases, including PubMed, Web of Science, the searchable CDC library of gray literature, and clinicaltrials.gov was performed to identify studies; the review was registered with PROSPERO on November 1, 2020. Within the three databases, we queried the title, keywords, and abstract of the article for Medical Subject Headings terms: “coronavirus,” “covid,” “clinical trial” and “randomized controlled trial.” Additionally, we used clinicaltrials.gov to identify COVID‐19 related trials that were recorded as “completed” with published results. We limited results to English language articles, publication year of 2020 or later, and (on PubMed) publication types including clinical trials, interventional clinical trials, and RCTs. Articles were arbitrarily pulled four times across stages of the pandemic, on October 31, 2020 (time period 1), January 31, 2021 (time period 2), March 31, 2021 (time period 3), and June 30, 2021 (time period 4). Our protocol initially required “trial” to be in the title of the article. For the second, third, and fourth time periods, the protocol was modified, and “trial” was no longer required to be part of the title in PubMed. This change captured more COVID‐19 vaccination studies that were not being picked up by our original search criteria. This change did not add any results to the first time period.

A modified preferred reporting items for systematic reviews and meta‐analyses (PRISMA) flowchart, presented in Figure 1, summarizes the selection of studies. Our search yielded 43 627 manuscripts, and two authors (J.P., M.W.C.) conducted a preliminary review to rule out duplicates and those that did not meet our definition of RCTs (i.e., patients randomized between at least two treatment arms and followed for outcomes), resulting in 914 titles. These were then sequentially examined from titles to abstracts to the paper text to determine whether the article met the inclusion and exclusion criteria. Seven hundred and seventy‐seven titles were excluded because they did not include final results, or the manuscript was not a true RCT (e.g., single armed intervention trials). Full‐text screening identified 137 manuscripts that met study criteria and 137 studies were included in the final review. Further, because some studies consisted of multiple trials, results were available from 149 trials across the 137 manuscripts (e.g., a vaccine study conducted with different doses in three age groups contributed one manuscript but three trials). Our goal was to examine all RCTs with published results to provide a complete census of current practice in COVID‐19 trial reporting. As such, we did not limit our analysis to high‐quality RCTs, as determined by standardized criteria.

FIGURE 1

Preferred reporting items for systematic reviews and meta‐analyses (PRISMA) study diagram. The flow diagram depicts the flow of information through the different phases of the systematic review

Data extraction

Two types of data were extracted from all articles for each trial: basic publication information and study characteristics. Basic publication information included study ID, trial identifier (national clinical trial number), title, lead author, region/country of research, and publication date. Study characteristics included study type (treatment, prophylaxis [e.g., hydroxychloroquine in healthy participants], vaccine, and transmission), type of treatment, primary outcome, main quantitative results, population type (e.g., hospitalization and symptomatic status of patients), total number of participants, whether they had a pregnancy exclusion criterion and if they did not, whether they collected information on the participation of pregnant women, lower and upper age limits, and enrollment start and end date.

After collecting basic data, we identified whether the trial reported on key demographic and clinical characteristics. Demographic characteristics included race, ethnicity, sex, age, income, rural/urban status, and any reporting of other SES‐related measures. Race was defined as a social category based on nationality, phenotype, or other markers of social difference whereas ethnicity is rooted in social meanings and sense of surrounding. , This differentiation is important because if studies are conducted in mono‐racial countries, ethnicity may be a better marker of segregation and historical conflict. We also examined clinical factors associated with higher COVID‐19 related mortality, including smoking status, body mass index (BMI), and use of oxygen therapy.

We further assessed reporting of several comorbidities. Categories of these comorbid conditions were determined after the first time period where the study team documented reported comorbidities. We consolidated similar comorbidities across studies into one category, and if the category was reported in at least two studies, we examined it. After this consolidation process, the final set of comorbidities included hypertension, diabetes, alcohol use disorder, general heart disease, heart failure, asthma, kidney disease, liver disease, rheumatic diseases, general respiratory disease, malignancy/cancer, chronic obstructive pulmonary disease (COPD), COPD or asthma, cerebrovascular disease, tuberculosis, hyperlipidemia, thyroid disease, obesity (as a condition), HIV/AIDS, and immunocompromised state. The full list is provided at our GitHub page.

Two reviewers (J.P., M.W.C.) completed data abstraction for the first time period on October 31, 2020, which focused on citation information and study characteristics. There were no major disagreements found and subsequent pulls were abstracted by one reviewer (J.P.). A second reviewer (M.W.C.) checked the data for every ambiguous field highlighted by the first reviewer (J.P.) to assess quality of data abstraction. Discrepancies were resolved through discussion.

Descriptive analysis

Study characteristics, including demographic and clinical characteristics, were synthesized using summary statistics. We compared proportions of studies falling into various categories (studies including US participants, different study types, and hospitalization), the proportion of studies reporting on key demographic variables (including age, sex, race, and BMI), and proportions of studies reporting on specific types of comorbidities (limited to the mostly commonly reported, including hypertension, diabetes, COPD, and obesity). We repeated these analyses stratified by time period, study type, and country (studies including US participants vs. studies without US participants). All data and models used in this study are publicly available via Data S1 and our GitHub page.

RESULTS

Overall results

Characteristics of manuscripts identified through the systematic review process are presented in Table 1. Manuscripts were published between April 2020 to June 2021, and were conducted in North America (26/137 studies, 19%), South America (22/137 studies, 16%), Asia (42/137 studies, 31%), Europe (19/137 studies, 14%), Middle East (26/137 studies, 19%), Africa (7/137 studies, 5%), Oceania (4/137 studies, 3%), and across multiple regions (13/137 studies, 9%). The three countries where the most RCTs were conducted were China (31), the United States (23), and Iran (22).

TABLE 1

Characteristics of identified RCTs through the systematic review process

Characteristic, demographic, or comorbidity	Period 1	Period 2	Period 3	Period 4	Cumulative
	N (%)	N (%)	N (%)	N (%)	N (%)
Total count of trials	61	31	21	36	149
Trial characteristics
Hospitalized	47 (77%)	17 (55%)	13 (62%)	25 (69%)	102 (68%)
United States participants	11 (18%)	7 (23%)	1 (5%)	4 (11%)	23 (15%)
Trial type
Treatment	50 (82%)	18 (58%)	17 (81%)	28 (78%)	113 (76%)
Vaccine	8 (13%)	11 (35%)	3 (14%)	7 (19%)	29 (19%)
Prophylaxis	3 (5%)	1 (3%)	1 (5%)	0	5 (3%)
Anti‐transmission	0	1 (3%)	0	1 (3%)	2 (1%)
Reported demographics
Age	61 (100%)	31 (100%)	21 (100%)	36 (100%)	149 (100%)
Sex	59 (97%)	31 (100%)	21 (100%)	36 (100%)	147 (99%)
Race	14 (23%)	18 (58%)	6 (29%)	12 (33%)	50 (34%)
Body mass index (BMI)	26 (43%)	17 (55%)	5 (24%)	14 (39%)	62 (42%)
Reported clinical characteristics
≥1 Comorbidity	48 (79%)	25 (81%)	18 (86%)	19 (53%)	110 (74%)
Smoke	17 (28%)	13 (42%)	5 (24%)	8 (22%)	43 (29%)
Oxygen therapy	22 (36%)	4 (13%)	2 (10%)	2 (6%)	30 (20%)
Diabetes	42 (69%)	23 (74%)	14 (67%)	18 (50%)	97 (65%)
Hypertension	38 (62%)	15 (48%)	12 (57%)	15 (42%)	80 (54%)
General heart disease	26 (43%)	20 (65%)	10 (48%)	7 (19%)	63 (42%)
Asthma	14 (23%)	6 (19%)	7 (33%)	4 (11%)	31 (21%)
Kidney disease	11 (18%)	7 (23%)	4 (19%)	5 (6%)	27 (18%)
Chronic obstructive pulmonary disease (COPD)	11 (18%)	6 (19%)	6 (29%)	3 (8%)	26 (17%)
General respiratory disease	9 (15%)	11 (35%)	3 (14%)	3 (8%)	26 (17%)
Malignancy/cancer	14 (23%)	6 (19%)	3 (14%)	2 (6%)	25 (17%)
Obesity	6 (10%)	9 (29%)	3 (14%)	2 (6%)	20 (13%)
Liver disease	9 (15%)	1 (3%)	6 (29%)	3 (8%)	19 (13%)
Immunocompromised	9 (15%)	0	2 (10%)	3 (8%)	14 (9%)
Hyperlipidemia	5 (8%)	2 (6%)	3 (14%)	3 (8%)	13 (9%)
Cerebrovascular disease	7 (11%)	0	3 (14%)	1 (3%)	11 (7%)
Heart failure	2 (3%)	3 (10%)	4 (19%)	0	9 (6%)
Thyroid disease	3 (5%)	1 (3%)	4 (19%)	0	8 (5%)
Tuberculosis	5 (8%)	0	2 (10%)	1 (3%)	8 (5%)
HIV/AIDS	3 (5%)	1 (3%)	1 (5%)	3 (8%)	8 (5%)
Rheumatic diseases	3 (5%)	0	1 (5%)	1 (3%)	5 (3%)
COPD or asthma	2 (3%)	0	1 (5%)	2 (6%)	5 (3%)
Alcohol use disorder	2 (3%)	0	0	0	2 (1%)

Abbreviation: RCTs, randomized controlled trials.

Figure 2 shows the frequency of various study characteristics among the RCTs. One hundred and thirteen studied potential treatments for infection, 29 studied vaccines, 5 studied prophylaxis strategies, and 2 studied interventions to prevent transmission among infected individuals. Study samples ranged from 10 to 38 206 participants (median = 100, IQR: 60–300). Figure 3 show proportions of RCTs reporting various demographic characteristics: 149 trials reported on age, 147 reported on sex, and 50 reported on race of participants. Among the 50 trials reporting race, only 20% indicated how race was ascertained (self‐report vs. hospital files). In addition, 110 reported the proportion of participants with at least one health comorbidity and 30 on use of oxygen therapy. Finally, Figure 4 shows the proportion of RCTs reporting on various types of comorbidities; while studies frequently reported on diabetes (n = 97) and hypertension (n = 80), they less frequently reported on asthma (n = 31) or COPD (n = 26). Pregnant women were explicitly excluded at baseline from 123 trials and only 7 trials reported on pregnancy. Two trials reported on the type of job participants held and another two trials reported on marital status; none reported on other indicators of SES.

FIGURE 2

Frequency of basic study characteristics among the randomized controlled trials (RCTs) examined

FIGURE 3

Frequency of reporting on various demographic characteristics among the randomized controlled trials (RCTs) examined

FIGURE 4

Frequency of reporting on various types of comorbidities among the randomized controlled trials (RCTs) examined

Stratification by study type

Vaccines

US participants were included in 24% of vaccine trials, 67% of prophylaxis trials, and 14% of treatment trials. Vaccine trials reported race (72%) and BMI (76%) at a higher rate than non‐vaccine trials (race = 24%, BMI = 33%) but reporting of comorbidities in vaccine trials was low (41%) compared to non‐vaccine trials (82%). There was low reporting on specific comorbidities including hypertension (3%) and diabetes (28%). Smoking status was reported in 29% of trials. Vaccine trials generally had a larger number of participants ranging from 25 to 38 206 participants (median = 224, IQR: 92–474) compared to non‐vaccine trials (median = 91, IQR: 58–199). Notably, only manuscripts on vaccines reported on multiple trials (29 trials within 17 manuscripts), with many reporting trials testing the same vaccine with different doses or age groups.

Treatment

Hospitalized patients were included in 89% of treatment RCTs while no prophylaxis or vaccine RCTs included hospitalized patients. Treatment RCTs were the least likely to report race (22%) compared to vaccine RCTs (72%) but were more likely to report the prevalence of at least one comorbidity (82%) and smoking habits (30%) compared to other study trials. There was also frequent reporting on several specific comorbidities including hypertension (85%) and diabetes (95%), but much lower data on other comorbidities including COPD (27%) and obesity (19%). Enrolled participants ranged from 10 to 16 442 participants (median = 93, IQR: 58–214).

Figures S1–S3 include full results stratified by study type (focusing on treatment and vaccine, given the low frequency of prophylaxis and anti‐transmission studies). There were too few prophylaxis and anti‐transmission studies for meaningful patterns to emerge.

Stratification by time period

Time period 1 identified 57 manuscripts, followed by 26, 20, and 34 manuscripts from time period 2, 3, and 4, respectively. This translated to 61, 31, 21, and 36 trials from time period 1, 2, 3, and 4, respectively. Treatment trials were abundant during the first and fourth time period at 82% and 78%, respectively. As a result, a larger proportion of trials included hospitalized patients during the first and fourth time period at 77% and 69%, respectively. The second time period had the highest proportion of vaccine trials at 35%.

Stratification by participants in the United States versus outside of United States

US participants

There were 23 trials from 20 manuscripts including participants from the United States. Among the 20 manuscripts, 12 reported on potential treatments, 4 studied vaccines, and 4 studied prophylaxis strategies. Studies including US participants were more likely to report race (81% in the United States vs. 23% outside the United States), ethnicity (65% vs. 4%), and smoking status (50% vs. 27%).

Non‐US participants

There were 117 manuscripts that did not include any US participants. Of these, 101 studied potential treatments, 13 studied vaccines, 2 studied prophylaxis strategies, and 1 studied transmission. Studies that did not include US participants were less likely to report on race (23%), ethnicity (3%), and smoking status (24%), but highly likely to report on comorbidities (74%).

DISCUSSION

COVID‐19 has inspired a wide array of systematic reviews , , , ; however, our large‐scale systematic review is the first to provide a synthesis of all existing COVID‐19 related RCTs with published findings and their reporting of key demographic and clinical characteristics by time period and study types through June 2021. While simple demographic statistics (e.g., age) and comorbidities (e.g., hypertension) were frequently reported on, more complex and sensitive demographics (e.g., race) and comorbidities (e.g., respiratory illness) were much less common. This is similar to the findings of Heidari et al. who concluded that only 2% of trials failed to report information on age and sex demographics. These findings highlight the need for more robust reporting on complex demographics and clinical characteristics to allow assessing of the generalizability of RCT findings to specific target populations. During the stratified portion of the review, there were several patterns, including differences in how each study type reported on comorbidities; differences in frequency of reporting on race between trials with US participants and those without; and differences between time periods.

It is surprising that more studies reported on hypertension (n = 80) and diabetes (n = 97) than conditions like COPD (n = 26), immunosuppression (n = 14) or cerebrovascular disease (n = 11), despite all of these being identified as major predictors of mortality among COVID‐19 patients. The most likely explanation for these discrepancies is that blood pressure and blood sugar levels are routinely taken, while spirometry or more detailed medical histories may be difficult to obtain. This problem is likely compounded in severely ill patients, who may be completely unresponsive. Further, the results of He et al. suggest that some chronic conditions such as cancer, heart failure, hypertension, chronic kidney disease, and COPD were eligibility criteria in 2.76%–8.42% of studies; such exclusion and inclusion criteria would not likely appear in the tables within published manuscripts. Unfortunately, these low rates of reporting make it difficult to understand the generalizability of these trials to specific target populations.

There were low rates of reporting on race (34%). The most likely reason is because many studies were conducted in countries like Iran or China, where race reflects a different social construct from that in Western countries and is not necessarily recognized as a potential effect modifier. Still, even if these studies gathered race information, race constructs would not translate between countries. Hence, a universal categorization of race is unlikely. Notably, in studies in which US participants were included, only 22% US‐based studies failed to report the distribution of patients enrolled by race. In more multi‐racial countries, RCTs were more likely to include data on patient‐reported or provider‐reported race. Finally, the lack of a universally accepted “COVID‐19 RCT study population reporting checklist” due to the rapid onset and severity of the pandemic may add to a lack uniformity in reporting across RCTs.

We also found that among the 137 manuscripts, only two included other indicators of SES, with two reporting of occupation type; none reported income. This makes it difficult to understand whether there is modification of treatment or vaccine effects by income or social status. Trial research teams should consult social scientists and epidemiologists for better ways to incorporate information on race and SES into the RCT data collection.

We also stratified our analysis based on intervention type. While there were too few prophylaxis and anti‐transmission studies for meaningful patterns to emerge, we observed patterns within the treatment and vaccination studies. Vaccine trials did not include hospitalized patients, while treatment trials included both hospitalized and non‐hospitalized patients. This explains the larger median and maximum population size in vaccine studies and the lower reporting of comorbidities. Treatment trials were also more likely to report on comorbidities than vaccine trials, indicating that treatments may vary by clinical characteristics. Alternately, comorbidities used to assess prognosis could have been more available in hospitalized settings.

This review has several limitations. First, the lag time of trial reporting and publication meant that key emerging factors (like COVID‐19 variants) could not be reported upon by the original trials of our investigation. Second, it is possible that some trials collected more detailed comorbidity and demographic data, and simply did not report it when publishing their findings; if this is the case, future work using that data may help clarify the representativeness of the trial or potential differences in treatment effects. Third, we were limited to trials that had published their results in English within the searched databases. Trials that were not able to publish (or have not yet published) their findings, most commonly in clinicaltrials.gov, may have collected more or less data than those that reached the academic community. Finally, we did not collect information on the trial sponsor, funding sources, or funding totals; more specific sources of funding may be correlated with more complete reporting of baseline demographics.

Researchers are just beginning to translate the findings of COVID‐19 RCTs to real‐world target populations. Future studies can leverage the extracted data on sex, age, race, and comorbidity distributions to directly compare the studies that reported on these factors to specific real‐world populations and understand major gaps in generalizability. The template used here may also help understand the reporting of demographic and clinical characteristics in bodies of evidence unrelated to COVID‐19.

CONCLUSION

While COVID‐19 related RCTs always or nearly always report readily available demographic and clinical characteristics, the lack of reporting on race, SES, and respiratory‐related conditions will make it difficult for clinicians, regulators, and public health practitioners to understand how well the findings from RCTs will translate to real‐world target populations.

FUNDING INFORMATION

This work was funded by PCORI award ME‐2017C‐9937; all statements in this presentation are solely those of the authors and not PCORI, its Board of Governors, or its Methodology Committee.

CONFLICT OF INTEREST

Dr Lund's spouse is a full‐time employee of GlaxoSmithKline and owns stock in the company. Dr. Stürmer receives salary support from the Center for Pharmacoepidemiology (current members are GlaxoSmithKline, UCB BioSciences, Takeda Pharmaceutical Company, AbbVie, and Boehringer Ingelheim; Merck was a former member), and owns stock in Novartis, Roche, and Novo Nordisk. Other authors declare no conflict of interest.

ETHICS STATEMENT

This systematic review did not require any ethical approval.

FIGURE S1 Distribution of geography and healthcare setting stratified by study type among the RCTs examined

FIGURE S2: Frequency of reporting on various demographic characteristics stratified by study type among the RCTs examined

FIGURE S3: Frequency of reporting on various types of comorbidities stratified by study type among the RCTs examined

Click here for additional data file.