Attention for sex in COVID-19 trials: a review of regulatory dossiers.

An under-representation of women and a lack of sex-specific analyses in COVID-19 trials has been suggested. However, the higher number of men than women who are severely affected by COVID-19 and the restricted information in scientific publications may have biased these suggestions. Therefore, we evaluated sex proportionality and sex-specific efficacy and safety data in trials of COVID-19 treatments and vaccines using both publicly available regulatory documents and confidential documents used by regulators in their review of medicinal products. Included were two treatments (ie, remdesivir and dexamethasone) and four vaccines (ie, BNT162b2 mRNA (BioNTech/Pfizer), mRNA-1273 (Moderna), ChAdOx1-S (AstraZeneca) and Ad26.COV2-S (Janssen)) that received marketing authorisation by the European Commission at the time of the study conduct. An under-representation of women was shown in three of the nine data sets for one treatment (ie, remdesivir), but the proportion of women included was representative in each of the data sets for the other five products. This indicates that there is no structural under-representation of women in the COVID-19 trials. Currently, sex-specific efficacy data are available for five of the six assessed products and sex-specific safety data are available for half of the products only. It is important that this information will also be made available for the other products. There are only small differences in efficacy and safety between men and women which are likely to be of limited clinical relevance. Sex-specific efficacy information can generally be found in the publicly available regulatory documents other than the Summary of Product Characteristics, for which more awareness might be required. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Sex differences have been shown in the COVID-19 pandemic and are important to examine.

A lack of consideration for women in clinical trials of COVID-19 treatments and vaccines has been suggested, both in terms of a lower number of women than men included in the trials and a lack of sex-specific analyses of the favourable and unfavourable effects.

Based on information from regulatory documents, we found no structural under-representation of women in the trials of the assessed medicinal products for COVID-19.

Sex-specific efficacy and safety information is available for respectively five and three of the six assessed COVID-19 products.

It is important that this information will also be made available for the other products.

Observed differences in efficacy and safety were small and likely of limited clinical relevance.

More awareness to sex-specific information that generally can be found in European Public Assessment Reports and clinical data on the European Medicines Agency website might be required.

Introduction

Sex differences, with men generally having a higher risk than women of COVID-19-related hospitalisation, intensive care unit (ICU) admission and death, have been observed during the COVID-19 pandemic.1–4 Gender-related behavioural factors (eg, smoking) could be part of the explanation for such differences, but biological (ie, sex) differences are also likely to play a role. There are known sex differences in the different steps from virus entry to adaptive immune response that may impact the response to the SARS-CoV-2 infection as well as the disease progression.1 5 In addition to response to the virus, men and women may respond differently to treatments and vaccines. For instance, women generally have higher overall antibody levels and report more adverse events (AEs) to vaccines than men.4 6 7 Attention to sex in clinical trials in general and in those of COVID-19 trials as a specific case is therefore considered necessary.8–13

A ‘sex gap’ in clinical trials of COVID-19 treatments and vaccines has been suggested, however, both in terms of a lower number of women than men included in the trials as well as a lack of sex-specific analyses of the favourable and unfavourable effects.14–16 There are two important factors to note that may have biased these conclusions. First, there is generally no adjustment for the higher number of men than women who are severely affected by COVID-19. Due to this higher occurrence of severe COVID-19 disease among men, clinical trials testing the effects of treatments in this population are logically more likely to include a higher number of men. This also explains the finding of a previous study showing that COVID-19 drug trials are less likely to have a good sex balance than the COVID-19 vaccine studies in which the general population is targeted.14 The participants in phase III clinical trials especially should be representative of the population that would use the product after its market approval.17

Second, the conclusions are based on reviews of scientific publications that contain much less information than the full dossiers that pharmaceutical companies submit to regulatory authorities to support a marketing authorisation application. A recent study using the information in the marketing authorisation application dossiers of medicines for various diseases presented a more positive view with sex-specific information on efficacy and safety available for each of the assessed products.18 This information, however, may not end up in the documents that are used to inform clinical practice. The Summary of Product Characteristics (SmPCs) provides the basis of information for European healthcare professionals on how to use a medicine safely and effectively. More detailed information on the clinical programme and benefit-risk assessment is available in the European Public Assessment Reports (EPARs).19 Since 2015, clinical data submitted by pharmaceutical companies to support their marketing applications are also being made publicly available on the European Medicines Agency (EMA) website.20

Given the suggested limited attention to women in the evaluation of COVID-19 treatments and vaccines and the above-mentioned limitations, we conducted a review of sex proportionality and sex-specific efficacy and safety data in publicly available regulatory documents and confidential documents used by regulators in their review of COVID-19 medicinal products.

Review process of the regulatory documents of COVID-19 medicinal products

We included products for the treatment and prevention of COVID-19 that received marketing authorisation by the European Commission as indicated on the EMA website21 before June 2021. This resulted in the inclusion of six medicinal products, respectively, two treatments (ie, remdesivir and dexamethasone) and four vaccines (ie, BNT162b2 mRNA (BioNTech/Pfizer), mRNA-1273 (Moderna), ChAdOx1-S (AstraZeneca) and Ad26.COV2-S (Janssen)) (table 1).

Table 1

Summary of the included medicinal products

Product name	Treatment/vaccine	Studies contributing to the main efficacy and safety data sets	Efficacy population	Safety population	SmPC	EPAR	Additional information
remdesivir	Treatment	NIAID-ACTT(1) GS-US 540 5776 (adaptive trial);GS-US-540–5773 (phase III);GS-US-540–5774 (phase III);CO-US-540–5758 (phase III)	All subjects in the adaptive trial	All subjects in the phase III trials	*	†	Confidential—MAA dossiers
dexamethasone	Treatment	RECOVERY adaptive platform trial	All subjects	All subjects	‡	§	Public—EMA website
BNT162b2 mRNA (BioNTech/Pfizer)	Vaccine	C4951001 (phase I/II/III)	Subjects without evidence of infection before vaccination phase I/II/III	All subjects in the phase II/III study	¶	**	Public—EMA website
mRNA-1273 (Moderna)	Vaccine	mRNA-1273-P301 (phase III)	Subjects in the phase III study 14 days after the second injection in the PP set	All subjects in the phase III study	††	‡‡	Public—EMA website
ChAdOx1-S (AstraZeneca)	Vaccine	Study COV001 (phase I/II);Study COV002 (phase II/III);Study COV003 (phase II/III);Study COV005 (phase I/II)	Subjects receiving SD/SD in both phase II/III studies	All subjects in all studies	§§	¶¶	Confidential—MAA dossiers
Ad26.COV2-S (Janssen)	Vaccine	VAC31518COV3001 (phase III)	Subjects in the PP set of the phase III study	All subjects in the phase III study	***	†††	Confidential—MAA dossiers

*https://www.ema.europa.eu/en/documents/product-information/veklury-epar-product-information_en.pdf.

†https://www.ema.europa.eu/en/documents/assessment-report/veklury-epar-public-assessment-report_en.pdf.

‡https://www.ema.europa.eu/en/documents/other/dexamethasone-covid19-article-53-procedure-proposals-product-information_en.pdf.

§https://www.ema.europa.eu/en/documents/other/dexamethasone-covid19-article-53-procedure-assessment-report_en.pdf.

¶https://www.ema.europa.eu/en/documents/product-information/comirnaty-epar-product-information_en.pdf.

**https://www.ema.europa.eu/en/documents/assessment-report/comirnaty-epar-public-assessment-report_en.pdf.

††https://www.ema.europa.eu/en/documents/product-information/spikevax-previously-covid-19-vaccine-moderna-epar-product-information_en.pdf.

‡‡https://www.ema.europa.eu/en/documents/assessment-report/covid-19-vaccine-moderna-epar-public-assessment-report_en.pdf.

§§https://www.ema.europa.eu/en/documents/product-information/vaxzevria-previously-covid-19-vaccine-astrazeneca-epar-product-information_en.pdf.

¶¶https://www.ema.europa.eu/en/documents/assessment-report/vaxzevria-previously-covid-19-vaccine-astrazeneca-epar-public-assessment-report_en.pdf.

***https://www.ema.europa.eu/en/documents/product-information/covid-19-vaccine-janssen-epar-product-information_en.pdf.

†††https://www.ema.europa.eu/en/documents/assessment-report/covid-19-vaccine-janssen-epar-public-assessment-report_en.pdf

EMA, European Medicines Agency; EPAR, European Public Assessment Report; MAA, marketing authorisation application; PP, per protocol; SmPC, Summary of Product Characteristics.

Publicly available regulatory information was extracted from the SmPCs, EPARs and the clinical data available on the EMA website. SmPCs and EPARs can be found on the EMA web page of a specific treatment or vaccine of interest and were available for each of the six included COVID-19 products. The clinical data on the EMA website are available after creating a free account. In these clinical data, we searched for information in the documents in the sections ‘clinical overview’ and ‘clinical study reports’. The clinical data on the EMA website were available for dexamethasone, BioNTech/Pfizer and Moderna. For the other products, that is, remdesivir, AstraZeneca and Janssen, we obtained additional information from the clinical study reports (module 5) and the summary information (module 2) of the confidential marketing authorisation application dossiers available at the Dutch Medicines Evaluation Board and EMA.

The following search terms were used in the screening of the documents: sex, gender, male, women, sub group, subgroup and sub-group. In the EPARs, the additional search terms ‘table’ and ‘figure’ were used since these documents sometimes include screenshots of tables and figures which cannot be screened using the search function.

Sex proportionality

Sex proportionality was assessed separately for trial participants receiving the treatment or vaccine, receiving standard of care or placebo, and per efficacy and safety data set in the main study or studies (table 1) and was determined by calculating the participation to prevalence ratio (PPR).22 For the calculation of the PPR, the percentage of women in the trial was divided by the percentage of women in the target population. For the treatments, the percentage of women hospitalised due to COVID-19 was used as the population reference for which we calculated the average of the percentages per country provided on the website of the Global Health 50/50 initiative,23 which is 43.8%. For the vaccines, we assumed that 50% of the population were women. A PPR <0.8 or >1.2 was considered respectively an under-representation or over-representation of women in the trial compared with the representation of women in the target population.22

In the efficacy and safety data sets of each of the medicinal products, both women and men were included. The numbers and/or percentages of included men and/or women were found in publicly available documents (table 2; online supplemental table 1). In total, we evaluated 27 data sets; 9 for remdesivir, 2 for dexamethasone and 4 for each of the vaccines. A proportional representation of women in each of the assessed data sets was shown for five of the six assessed products, that is, dexamethasone, BioNTech/Pfizer, Moderna, AstraZeneca and Janssen (figure 1). An under-representation of women was shown in three of the nine included data sets of remdesivir. PPRs of these three data sets ranged from 0.742 to 0.797.

Table 2

Availability of sex-specific information in the publicly available documents

	Sex proportionality	Sex-specific efficacy information	Sex-specific numerical efficacy data	Sex-specific safety information	Sex-specific numerical safety data
remdesivir	+	–	+	–	–
dexamethasone	+	+	+	–	–
BNT162b2 mRNA (BioNTech/Pfizer)	+	+	+	+	+
mRNA-1273 (Moderna)	+	+	+	–	–
ChAdOx1-S (AstraZeneca)	+	–	–	–	–
Ad26.COV2-S (Janssen)	+	+	+	+	+

Figure 1

Participation to prevalence ratios (PPR) of the data sets used for treatments remdesivir and dexamethasone, and vaccines BNT162b2 mRNA (BioNTech/Pfizer), mRNA-1273 (Moderna), ChAdOx1-S (AstraZeneca) and Ad26.COV2-S (Janssen).

Sex-specific efficacy data

Sex-specific efficacy information and numerical results were assessed for the main data set and population (table 1) and was available for five of the six assessed products, that is, remdesivir, dexamethasone, BioNTech/Pfizer, Moderna and Janssen. For these products, information was found in the publicly available documents with the SmPCs providing some general information for most products and the EPARs and clinical data on the EMA website providing sex-specific numerical efficacy data (table 2; online supplemental table 1).

The sex-specific numerical efficacy data show that the effects of the treatments and the vaccines are similar between women and men (figure 2). For remdesivir, the recovery rate ratio was 1.31 (95% CI 1.07 to 1.59) and 1.38 (95% CI 1.05 to 1.81), respectively, for men and women (figure 2A). For dexamethasone, the rate ratio was 0.80 (95% CI 0.71 to 0.91) for men and 0.90 (95% CI 0.74 to 1.09) for women (figure 2B). The effectiveness of the vaccines ranged from >60% for Janssen to >90% for BioNTech/Pfizer and Moderna among both women and men (figure 2C).

Figure 2

Sex-specific efficacy results of (A) treatment remdesivir, (B) treatment dexamethasone, and (C) vaccines BNT162b2 mRNA (BioNTech/Pfizer), mRNA-1273 (Moderna) and Ad26.COV2-S (Janssen). CI, confidence interval; RR, rate ratio; RRR, recovery rate ratio; VE, Vaccine effectiveness.

Sex-specific safety data

For sex-specific safety information and numerical results (table 1), we focused on any AEs, and on serious adverse events (SAEs) in the different arms of the trials, that is, in the active treatment or vaccine group and the comparator or placebo group. The safety information was available for three of the six products, that is, remdesivir, BioNTech/Pfizer and Janssen. None of the SmPCs of these products contained sex-specific safety information but publicly available information was found in the EPARs and clinical data on the EMA website for BioNTech/Pfizer and Janssen (table 2; online supplemental table 1). For remdesivir, the information was obtained from the confidential marketing authorisation application dossiers. Sex-specific numerical data of both AEs and SAEs was available for remdesivir and BioNTech/Pfizer whereas for Janssen, sex-specific numerical data of only SAEs was available.

We tested the potential sex differences in any AEs and SAEs for those receiving the treatment or vaccine and for those receiving the comparator or placebo using Pearson χ2 tests. We did not adjust for multiple testing to identify any possibly relevant sex differences, so p values <0.05 were considered statistically significant. With regard to remdesivir, women generally more often experienced any AEs than men whereas men generally more often experienced any SAEs in both the treatment and standard of care arms. A statistically significant difference, however, was only observed for the analysis of any AEs at the assessment of 5 days of treatment (59.7% of the women vs 44.7% of the men, p=0.042). For BioNTech/Pfizer, women more often experienced any AEs than men which was statistically significant in both the vaccine group (29.5% vs 24.5%, p<0.001) and the placebo group (13.2% vs 11.9%, p=0.011). For Janssen, no statistically significant differences between men and women were shown (table 3). The number of patients experiencing an SAE is very small, however, for any product, and the results of the analysis should be interpreted with caution.

Table 3

Percentage of trial participants with any AEs or SAEs of remdesivir, BNT162b2 mRNA (BioNTech/Pfizer) and Ad26.COV2-S (Janssen)

	Women	Men	P value
remdesivir (study 5774)*
Any AEs
RDV 5 days	59.7	44.7	0.042
RDV 10 days	56.0	54.2	0.810
SOC	49.3	42.4	0.340
Any SAEs
RDV 5 days	2.6	5.3	0.367
RDV 10 days	2.7	4.2	0.569
SOC	5.3	11.2	0.160
BioNTech/Pfizer
Any AEs
Vaccine	29.5	24.5	<0.001
Placebo	13.2	11.9	0.011
Any SAEs
Vaccine	0.5	0.6	0.385
Placebo	0.4	0.5	0.582
Janssen
Any SAEs
≥1 SAE vaccine	0.4	0.4	0.353
≥1 SAE placebo	0.6	0.7	0.492

*Data from the confidential marketing authorisation application dossier.

AE, adverse event; RDV, remdesivir; SAE, serious adverse event; SOC, standard of care.

Conclusion

The COVID-19 pandemic presented an exceptional global situation with an urgent need for therapies to curb the disease. Extensive funding was made available to develop new treatments and vaccines, as were many already approved medicines tested for their efficacy in treating COVID-19. The unprecedented public health crisis made flexible and fast regulatory processes for evaluating and approving medicinal products necessary. Consequently, the EMA used a rolling review process; that is, reviewing data sets as soon as these became available. Most of the products reviewed in our study received conditional marketing authorisations, implying that additional data would be provided after approval.24 Nevertheless, regulators applied a thorough review process and judged the benefit-risk balance to be positive at the time of approval. They considered that in view of the unmet need in this pandemic, the benefit to public health of the immediate availability on the market outweighed the risk inherent in the fact that additional data would still be required.25 This procedure is also used for other diseases with unmet medical need (eg, rare/orphan diseases, advanced cancer). The public, however, questioned whether sufficient knowledge was available at the time of approval due to these fast approvals. While we acknowledge that rather large randomised controlled trials (RCTs) were underpinning the conditional approvals, one specific concern was that there might have been too little attention for women in COVID-19 trials. Our review showed that women were generally well represented in these trials. An under-representation of women was shown in three of the nine data sets for one treatment (ie, remdesivir), but the proportion of women included was representative in each of the data sets for the other five products. No sex-specific safety information was available for dexamethasone and Moderna, and sex-specific efficacy and safety information was lacking for AstraZeneca. For this latter vaccine, the EPAR reported that subgroup comparisons including sex, on both efficacy and safety outcomes, will still be explored (online supplemental table 1). These data will be made available in a final full report. The sex-specific data of efficacy and safety available for, respectively, five and three products showed no pattern of relevant differences between men and women.

A limitation of our review is the inclusion of two treatments only. At the time of the study, however, remdesivir and dexamethasone were the only products formally authorised by the European Commission for the treatment of COVID-19 although several other treatments could be used as emergency products. The results of two reviewed treatments are not generalisable to other COVID-19 treatments. Also, the findings of the four included vaccines may not be generalisable to any future vaccine. We adjusted our proportionality analyses for the sex distribution in the target population using the average percentage of hospitalised patients with COVID-19 in various countries worldwide as provided by the Global Health 50/50 data. The percentage of hospitalised patients with COVID-19, however, widely differs across countries and only half of the countries tracked by the sex, gender and COVID-19 project of Global Health 50/50 have provided sex-specific information.23 26 Also, the percentages are different when considering the proportion of women admitted to the ICU. The reliability of the PPR depends on the reliability of the reference data used in the calculations. Furthermore, we did not assess the availability of sex-specific information on other relevant parameters, such as vaccine immunogenicity. In addition, it should be noted that we assessed the sex proportionality in the data sets of the main efficacy and safety studies only. Sex proportionality may be different in other studies, in earlier trial phases and in animal studies.18

Our findings are not in line with previous studies suggesting a sex gap in trials of COVID-19 treatments and vaccines.14–16 This discrepancy can be explained by the use of the percentage of women in the actual target population to determine the sex proportionality in our study as well as the use of different data sources between our study and previous studies, that is, regulatory documents versus scientific publications. For example, regarding the inclusion of women in the trials for the treatment with remdesivir and dexamethasone, the first explanation is plausible; the percentage of women hospitalised due to COVID-19 is lower than the percentage of men23 which results in a lower percentage of women expected to be included in the trials of medicines for the treatment of these patients. Regarding the second explanation, in spite of the increasing attention to sex and gender, exemplified by, for instance, the introduction of the Sex and Gender Equity in Research guidelines to stimulate a more systematic approach of reporting sex and gender in research papers,27 publications on COVID-19 and in various other fields28–30 often still lack sex-specific information. It is reassuring that our review shows that sex-specific information was collected and is available in the documents used in the evaluation of most of the COVID-19 products for marketing approval. This aligns with a previous study showing that sex-specific information is also available in these documents of medicinal products for other diseases.18 More attention to sex-specific safety information of the COVID-19 medicinal products is, however, appropriate since this information was at this stage available for half of the products only. The finding of the previous review of marketing authorisation application dossiers showing that sex-specific safety information was available for all of the assessed other medicinal products18 highlights the importance regulators give to the presentation of sex-specific information. Our review showed that not all this information is available yet for the COVID-19 products which may be explained by the public health emergency. Considering the relevance, it is expected that eventually for all licensed COVID-19 products, sex-specific data will be made available.

Although the EMA considers the SmPCs as the main source of product information for healthcare professionals, our study showed that SmPCs provide only some general sex-specific efficacy information and no sex-specific safety information. This is in line with regulators’ review guidelines that emphasise the importance to assess consistency of effects across subgroups, but that when consistency is observed in subgroups, such as in women and men, it is usually not necessary to document this in the SmPC.31 Whether information about such consistency in subgroups should still be structurally reported in SmPCs is a matter of debate. On the other hand, our review showed that the available sex-specific efficacy and safety information is publicly available for most of the assessed products in other documents, that is, the EPARs and clinical data on the EMA website. It is recommended to communicate widely about the availability of sex-specific information in these other publicly available sources to, for example, healthcare professionals, policy makers, patients and researchers.

Based on the data presented in our study, there is no clear reason to assume that there are relevant sex differences in the efficacy and safety of the evaluated COVID-19 products. For the BioNTech/Pfizer vaccine, a higher rate of AEs is observed in women compared with men; however, this was observed both in the vaccine group and the placebo group, suggesting that this is not a vaccine effect per se. With regard to efficacy, small differences are observed, suggesting higher efficacy in men; however, these are not significant and may be due to chance. Other publications suggest, however, that there are indications of possible sex differences. For instance, a meta-analysis combining clinical trials of different types of COVID-19 vaccines concludes that although effective in both men and women, the overall efficacy might be higher in men.32 This meta-analysis does not address the relevance of the observed differences, and as can be observed in figure 2, the clinical relevance of the observed differences, which were small and not significant in individual trials, may be debated. The rare events of anaphylaxis after vaccination with BioNTech/Pfizer and Moderna,33 both mRNA vaccines, and thrombosis combined with thrombocytopenia after vaccination with AstraZeneca34 and Janssen,35 both adenoviral vector vaccines, were (initially) mostly seen in women. Various factors may explain the observed difference between the results from the RCTs presented in our study and the additional follow-up ‘real world’ studies. A first factor relates to the size and representativeness of the included population. Although tens of thousands of people were included in the phase III clinical trials, they are still a small and selected sample of the overall population. For instance, as is the standard for clinical trials prior to the marketing authorisation of medicines, COVID-19 vaccine trials have excluded pregnant and lactating women,36 and these trials did show an under-representation of the elderly.37 A second factor relates to methodological and statistical aspects. For instance, subgroup analyses are likely to be underpowered since the trials are usually powered for testing the overall effects only.38 Moreover, besides sex, also several other subgroups are relevant when evaluating the treatment effects in COVID-19, such as age groups37 and groups based on different body mass index levels.39 To complicate matters further, there may be value in assessing interactions between subgroups, for example, between sex and age,40 as well as considering other sex-specific endpoints, for example, menstrual period changes,41 and causes of differences between men and women in treatment and vaccine effects. An example of the latter is assessing sex-related (eg, genes, hormones) and gender-related (eg, diet) explanations for differences in vaccine effects on immunogenicity.42–44

It is clear that it will not be possible to answer all the questions before a treatment or vaccine is marketed. As nicely summarised by Eichler et al45: ‘RCTs can only answer a minuscule fraction of the near-infinite number of questions about subpopulations, interactions, treatment settings, effects, etc., that are relevant to patients and healthcare professionals at the point of care.’ Further analyses and postmarketing studies using real-world data will be required to answer additional questions. Besides the full clinical reports, the EMA intends to also publish the individual, anonymised patient data of the marketed products in the future,46 making an extra effort for the development of medicinal products in COVID-19.47 Such data can be used to further examine specific research questions or make more personalised predictions on the effects of a treatment or vaccine.

To summarise, our review of regulatory documents showed that there is no structural under-representation of women in the trials of the assessed COVID-19 products. Sex-specific efficacy and safety information is available for respectively five and three of the six assessed products. It is important that this information will also be made available for the other products. Most of the information can be found in the publicly available EPARs and clinical data on the EMA website. More awareness of these data sources might be required. The available data do not suggest that there are clinically relevant differences in efficacy and safety between men and women but limitations of clinical trials in general require further assessment of possible sex differences.