Integrating sex and gender in studies of cardiac resynchronization therapy: a systematic review.

AIMS: To examine the prevalence, temporal changes, and impact of the National Institute of Health (NIH) Sex as a Biological Variable (SABV) policy on sex and gender reporting and analysis in cardiac resynchronization therapy (CRT) cohort studies. METHODS AND
RESULTS: We searched MEDLINE, EMBASE, and Web of Science for cohort studies reporting the effectiveness and safety of CRT in heart failure patients from January 2000 to June 2020, with no language restrictions. Segmented regression analysis was used for policy analysis. We included 253 studies. Fourteen per cent considered sex in the study design. Outcome data disaggregated by sex were only reported in 17% of the studies. Of the studies with statistical models (n = 173), 57% were adjusted for sex. Sixty-eight per cent of those reported an effect size for sex on the outcome. Sex-stratified analyses were conducted in 13% of the studies. Temporal analysis shows an increase in sex reporting in background, statistical models, study design, and discussion. Besides statistical models, NIH SABV policy analysis showed no significant change in the reporting of sex in study sections. Gender was not reported or analysed in any study.
CONCLUSIONS: There is a need to improve the study design, analysis, and completeness of reporting of sex in CRT cohort studies. Inadequate sex integration in study design and analysis may potentially hinder progress in understanding sex disparities in CRT. Deficiencies in the integration of sex in studies could be overcome by implementing guidance that already exists.

Introduction

There is important evidence for sex differences in the effectiveness and safety of cardiac resynchronization therapy (CRT). The reasons remain unclear due to the underrepresentation of women in trials, but they are likely to be multi‐faceted. Evidence shows that they may be due to differences in physiology, disease presentation, and gender‐related behaviours. , These uncertainties could be elucidated by improved reporting and integration of sex and gender in studies.

Sex and gender have been poorly analysed for many years, in all fields of medical research, with few studies reporting outcomes disaggregated by sex and gender. , To encourage sex integration in research, the National Institutes of Health (NIH) released a funding policy in 2015, Sex as a Biological Variable (SABV), calling onto scientists to consider sex in the reporting, study design, and analysis in human and animal studies. Other global efforts are also in place to promote sex and gender in research. , In addition, the principles of sex inclusion and analysis have been prominently endorsed by cardiovascular academic societies and leading journals. , In this study, we define sex as biological attributes that differentiate male from female individuals and gender as social constructs, identities, and roles set by the society.

For post market approval of CRT, cohort studies are commonly used to evaluate long‐term clinical outcomes, such as mortality, understanding prognostic factors and determinants, assessing practice variations and providing insight into real‐world health effects, especially in populations that are under‐represented in RCTs, such as women, minorities and older patients. Sex and gender integration in those studies could help understand the observed sex differences in response. However, there is limited information on the extent of sex and gender integration in CRT cohort studies. Thus, we assessed all the CRT cohort studies that reported clinical or safety outcomes among heart failure patients treated with CRT over the past two decades to determine the prevalence and temporal trends of sex and gender reporting and analysis.

Methods

Study protocol

This methodological study was conducted as part of a systematic review of sex differences in the clinical and safety outcomes of CRT as assessed in cohort studies (PROSPERO ID: CRD42020204804).

Searching

We developed a search strategy in collaboration with a librarian scientist for cohort studies reporting clinical and safety outcomes for CRT using combinations of both text‐words and MeSH headings to capture heart failure, cardiac devices, and cohort studies (Supporting information Methods 1). We validated the search strategy by searching a set of 10 studies identified in the preliminary search. MEDLINE, EMBASE, and Web of Science were searched from January 2000 to June 2020 with no language restrictions.

Eligibility criteria

We included cohort studies with (comparative cohorts) or without control groups (single cohorts) that reported clinical or safety outcomes among patients with heart failure for CRT with no restrictions on publication language or outcomes. Non‐English language studies were translated using Google Translate. Studies evaluating the efficacy of implantable cardioverter‐defibrillator (ICD)s and pacemakers were not eligible. Reviews, randomized trials, cross‐sectional studies, case–control studies, conference papers, editorials, and letters were also excluded. Studies that were limited to single sex were also excluded from our assessment. Studies were not considered as CRT studies if they reported outcomes in patients with multiple devices, but less than 50% of the study population received CRT, and data were not disaggregated by sex as a guarantee that most of the patients assessed received CRT. We included studies that assessed participants receiving a device upgrade from ICD to CRT‐D, but other device upgrades were excluded. Studies developing clinical prediction rules as their approach to analysing sex are different from studies that report clinical and safety outcomes of CRT. Such studies tend to focus on the performance of the model, rather than the effects of individual predictors. Studies that were limited to single sex were also excluded from our assessment.

Screening and data extraction form

All the articles were independently screened for inclusion and information extracted by two review authors, and discrepancies were resolved by consensus. We extracted the following study characteristics from each study: publication year, journal classification [quartile ranking based on impact factor as per Scientific Journal Rankings (SJR) where Q1 occupies the top 25% of the journals and Q4 occupies 75% to 100% ], sample size, geographic region, type of CRT, availability of control group, and use of the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

Table details the definitions of sex and gender considerations in the study sections; informed by the SAGER guidelines, guidance provided by the Cochrane and Campbell Sex and Gender Methods Group and a CIHR grant‐approved project for developing health equity extension for the STROBE reporting guidelines. , , We defined the reporting or analysis of gender as the reporting or analysis of how differences between men and women and gender‐diverse individuals as it relates to gender constructs, identities and roles.

Table 1

Sex and gender considerations

Study sections	Sex/gender considerations
Title/abstract	Were sex/gender term used in the title or abstract?
Objective of research question	Was the assessment of sex/gender differences an objective of the study?
Background	Did the background discuss why sex/gender differences may be expected?
Study design and analysis plan	Did the authors have strategies to balance across sex/gender in the study design? Did the authors reference sex/gender in the study eligibility criteria? Were there any efforts to address potential bias across sex/gender? Did the authors plan to stratify results by sex/gender?
Findings and analysis	Was sex/gender described in flow of participants? Were reasons for non‐participation reported separately for each sex/gender? Did the authors report the percentage of participants included in the study disaggregated by sex/gender? Did the authors report missing data of participant disaggregated by sex/gender? Did the authors report follow up time disaggregated by sex/gender? Were summary measures of outcome data reported separately for each sex/gender?
Statistical model plan and analysis	If the authors planned to use a statistical model, did they plan to control for sex/gender in analysis? If a statistical model was used in the analysis, was sex/gender adjusted for in the analysis? Were statistical models conducted separately by sex/gender?
Discussion	Did the authors discuss limitations pertaining to sex/gender? Did the authors discuss results pertaining to sex/gender? Did the authors consider sex/gender when discussing external validity to the general population?

Sex/gender corresponds to sex or gender. Variables were captured for sex and gender separately.

Statistical models were classified into either prediction models or association models because we hypothesized that prediction models would more likely report the covariates included in the model compared with association models. Prediction models were defined as models that aimed to identify predictors that contributed to the prediction of the outcome, while models aimed to determine the aetiology of disease were categorized as association models.

Statistical analysis

We present the study characteristics and prevalence of sex and gender considerations in the studies as proportions. Temporal patterns were evaluated using the Cochrane‐Armitage trend test via the DescTools package. We used segmented regression to assess whether there was a change in the reporting and analysis of sex after 2015; the date when the NIH SABV policy on sex as a biologic consideration was published. The NIH SABV date was used as a marker of possible change in considering sex and gender, because the policy from this influential funding organization and world leader in biomedical research received a great amount of attention and support in the scientific community on the importance of analysis by sex. , NIH is also a very influential funding organization and a world leader in biomedical research. A detailed explanation of the segmented regression model is in Supporting information Methods 3. All analyses were performed using R software and statistical significance was determined using a two‐tail significance level of 0.05 Figure 1.

Figure 1

Sex in cardiac resynchronization therapy (CRT) cohort studies reporting clinical and safety outcomes. (A) ‘Title/abstract’ includes the descriptive reporting of sex. (B) Presents the percentage of descriptive reporting of participant sex. *P ≤ 0.05. ***P ≤ 0.001. ****P ≤ 0.0001.

Results

Study selection and characteristics

Of 7518 identified studies, we included 253 studies that met our eligibility criteria (Figure ). Table summarizes the characteristics of the included studies. Most CRT studies were conducted in Europe (57%), of prospective study design (52%) and included participants receiving a variety of cardiac devices (83%) (i.e. devices other than CRT). The majority were published in Q1 classified journals (62%) (i.e. top 25% of all the journals), but only 2% reported the use of the STROBE guidelines for reporting.

Table 2

Characteristics of included studies

	n = 253 (%)
Publication year
2000–2010	90 (36)
2011–2020	163 (64)
Journal classification
Q1 a	156 (62)
Q2–Q4 a	93 (37)
Not yet assigned	4 (1)
Objective to assess sex differences
Primary	23 (9)
Secondary	5 (2)
Not reported	28 (89)
Cohort study type
Comparative	126 (50)
Single	127 (50)
Cohort study design
Prospective	131 (52)
Retrospective	122 (48)
Continent
Europe	143 (57)
North America	55 (22)
Asia	38 (15)
South America	8 (3)
Oceania	6 (2)
International	2 (0.8)
Africa	1 (0.4)
Sample size
<500	170 (67)
≥500	83 (33)
Age of participants
<65	97 (39)
≥65	155 (61)
NR	1 (0.4)
Type of cardiac device
CRT‐D only	27 (11)
CRT‐P only	17 (7)
Combination of devices	209 (83)
Prevention of sudden cardiac arrest (SCA)
Primary only	53 (21)
Primary and secondary	53 (21)
Not specified	147 (58)
Report use of STROBE guidelines	6 (2)

Q: Quartile; Q1 occupies the top 25% of the journals, while Q4 occupies 75% to 100%.

CRT‐D, cardiac resynchronization therapy‐defibrillator; CRT‐P, cardiac resynchronization therapy‐pacemaker; STROBE, The Strengthening the Reporting of Observational Studies in Epidemiology.

Reporting in study sections

Figure shows that 101 studies (40%) considered sex in the statistical model plan and analysis. In addition, sex was poorly reported in the background of the studies (14%) and the study design and analysis plan (16%). Less than half (40%) of the studies reported the proportion of both male and female individuals in their study characteristics (Figure ). Exploratory analysis shows that in all the study sections, retrospective studies considered sex more than prospective studies (Table ).

Figure 2

Impact of NIH Sex as a Biological Variable (SABV) policy on the reporting of sex in the study sections of cardiac resynchronization therapy (CRT) cohort studies reporting clinical and safety outcomes. Studies published in year 2020 were excluded because we did not capture all the studies published that year. Dotted lines at 2015 represent the release of National Institutes of Health (NIH) policy on including sex as a biological variable in reporting, study design and analysis. Segmented regression analysis was used to assess the impact of the NIH SABV policy on the reporting of sex considerations. ‘Title/abstract’ corresponds to title or abstract.

Over the past two decades, there has been an increase in the proportion of CRT studies that considered sex in the background (P = 0.0065), study design and analysis plan (P = 0.0014), statistical models (P < 0.001) and the discussion (P < 0.001) (Figure ). However, there was no significant change in the trend of reporting in the title/abstract (P = 0.4244) or the findings (P = 0.0784). Evaluation of the NIH SABV policy's impact on the consideration of sex indicates a significant change in the trend of considerations only in the statistical models (Figure ).

Study design and analysis plan

Fifteen studies (6%) planned to balance for differences across sex in the study design (Table ). Two studies attempted to address potential biases across sex. A priori planning to stratify results by sex was reported in 21 studies (8%).

Table 3

Sex in the study sections of cardiac resynchronization therapy (CRT) cohort studies reporting clinical and safety outcomes

	n = 253 (%)
Study design and analysis plan
Strategies to balance across sex in study design	15 (6)
Eligibility criteria mention sex	5 (2)
Address potential bias across sex	3 (1)
Plan to stratify results by sex	21 (8)
Findings and analysis
Sex of participants described from the process of examining eligibility to analysis	2 (1)
Reasons for non‐participation reported across sex	—
Missing data disaggregated by sex	2 (1)
Follow‐up time disaggregated by sex	5 (2)
Report at least one outcome disaggregated by sex	17 (7)
Report all outcomes disaggregated by sex	25 (10)
Interpretation of findings
Limitations of study pertaining to sex	15 (6)
Interpreted results while considering sex	59 (23)
Generalizability across sex	21 (8)

Findings and analysis

Only two studies (1%) described sex in the flow of participants: from patient recruitment to study eligibility and inclusion in analysis (Table ). Information on missing data disaggregated by sex was also reported in only two studies (1%). Five studies (2%) reported the follow time separately for each sex. Twenty‐five (10%) studies reported all their outcomes stratified by sex.

Trend analysis shows significant improvements in the adjustment for sex in reporting of outcomes disaggregated by sex (Figure ). Segmented regression shows that NIH SABV policy had no significant impact on the trend of disaggregating outcome data by sex (Figure ).

Statistical model plan and analysis

Of the included studies, 173 studies (69%) reported using statistical models in their analysis (Table ). Even though 69% of those adjusted for sex in their analysis, only 37% planned to control for sex a priori in their model. Of the studies that adjusted for sex in their statistical modes, 60% reported an effect size for sex. Twenty‐three studies (13%) analysed male and female participants separately.

Table 4

Sex in the statistical analysis models of cardiac resynchronization therapy (CRT) cohort studies reporting the clinical and safety outcomes

	Prediction model n = 94 (%)	Association model n = 79 (%)	P value a	All models N = 173 (%)
Planned to control for sex in statistical model	24 (26)	38 (48)	0.003	63 (37)
Report separate models by sex	5 (5)	18 (47)	0.002	23 (13)
Adjusted for sex	46 (49)	53 (67)	0.024	99 (57)
Reported effect size	37 (80)	30 (57)	0.021	67 (68)

Proportions calculated from 173 studies that included statistical models in their studies. Proportion of studies that reported an effect size calculated from the number of studies that adjusted for sex.

Fisher's exact test.

Fifty‐four per cent of the studies used prediction models in their analysis. Studies employing prediction models and association models differed significantly in terms of their planning to control for sex in their statistical model (P = 0.003), reporting of separate models by sex (P = 0.002), adjusting of models for sex (P = 0.024), and reporting of effect sizes for sex (P = 0.021).

Temporal analysis shows that CRT studies are increasingly incorporating sex in their statistical models by adjusting for it (P < 0.001) or assessing for interactions (P < 0.01) ( ). However, NIH SABV policy analysis shows no significant change in the trend of reporting of sex consideration in statistical models ( ).

Interpretation of findings

Fifty‐nine studies (23%) interpreted their results while accounting for sex (Table ). Discussion about the generalizability of the study results across sex was reported in 8%, while 6% of the studies discussed the study limitations pertaining to sex.

Analysis and reporting of gender in study sections

We assessed studies for explaining differences in risk due to gender identity, roles, and norms in the background; considering gender domains in study design, analysing the impact of gender on outcomes in the results; and discussing the findings in relation to gender identity, roles, and norms. However, gender was not reported or analysed in all the included studies.

Discussion

This systematic review of reporting and analysis of sex and gender in CRT studies reporting clinical and safety outcomes shows that sex reporting in all aspects of studies is improving over time but remains poorly integrated in the study design and analysis. There was no significant change in the trend of sex considerations after the introduction of the NIH SABV policy. In this study, we focused on the reporting and analysis of sex and gender, but this does not dimmish the importance of other patient characteristics such as ethnicity and age, which have been previously shown to differentially impact treatment quality and health outcomes. , Such characteristics could be assessed for CRT in future studies.

Considering the known sex differences in aetiology, prognosis, and disease presentation in heart failure, well‐thought‐out transparent research on the impact of sex and gender in health is needed to understand sex differences in response to CRT. Systematic reviews of landmark trials have shown that women benefit from CRT more than men. , , The reasons for the differences in response are uncertain, with evidence suggesting that sex may independently predict the response, while other evidence suggesting that sex may act as surrogate for height and QRS duration. The AdaptResponse trial, the largest randomized CRT trial with the largest proportion of women enrolled to date (41%), found that QRS duration was shorter in women regardless of height. This demonstrates how the underrepresentation of women in cardiovascular trials impedes the understanding of the observed differences.

Cohort studies could be utilized to overcome this underrepresentation as they represent data collected during routine care. The effect estimates from cohort studies may not accurately correspond to estimates from RCTs, but tend to coincide when assessing long‐term outcomes such as all‐cause mortality. Moreover, as guidelines are moving towards producing sex‐specific recommendations, cohort studies will be ideal for evaluating the adherence to the guideline recommendations and real‐world effectiveness of CRT.

Inadequate reporting threatens research reproducibility and diminishes the quality of evidence and impedes the understanding of sex differences. As an indicator for utilizing guidance for transparent reporting, we evaluated the proportion of CRT studies that reported the use of STROBE guidelines. Even though STROBE is the second most cited reporting guideline, less than 10 CRT studies reported the explicit use of the reporting guideline. This may be due to unclear reporting of guideline use as indicated by Caulley et al.

Inadequate integration of sex and gender in the study design may have impeded the efforts in place to understand these differences. The importance of transparent reporting and study design in clinical advances has been demonstrated in both basic science and clinical research to achieve accurate representation of the population. Methodological approaches to help improve the quality of evidence for sex differences such as sex‐specific eligibility criteria and stratified sampling or participant matching – should be considered.

Only 13% of the studies reported independent statistical models for men and women. Depending on a single statistical model to identify sex differences overstates the value of statistical significance and inhibits the application of sex‐specific definitions for clinical characteristics. , In addition, two out of 17 studies that conducted subgroup analyses specified that they planned to conduct such analyses. Hence, the credibility of the subgroups is doubtful, as well as the studies' ability to detect an interaction effect. Furthermore, our findings in CRT studies are comparable with an assessment of sex and gender in anaesthesia studies, indicating that adjustment for sex as a covariate in models is common in research, but reporting findings by sex is lacking.

We defined gender considerations as the reporting or the analysis of the influence of social constructs, identities, and roles set by the society on the findings. Gender was not considered in any of the included studies. The direct and indirect contribution of gender in CV disease manifestation, diagnosis, and treatment has been documented in the literature. For the analysis of gender, there is no standard approach for measuring gender, but validated tools have been developed. The GENESIS‐PRAXY (Gender and Sex Determinants of Cardiovascular Disease: From Bench to Beyond‐Premature Acute Coronary Syndrome) is a tool that provides a composite measure of gender identity, constructs and roles. The application of this tool on young patients with acute coronary syndrome has shown that adverse cardiovascular outcomes are more likely to occur in individuals with personality traits and social roles traditionally ascribed to women. Hence, the lack of gender analysis may hinder ability of scientific research to improve lives of men, women, and gender diverse people.

Sex reporting and analysis has improved over time in all the study sections, which is consistent with what was found in other studies. , However, we did not detect an effect for the NIH SABV policy, published on 9 June 2015, on the trend of sex reporting in all the study sections, except for statistical models. The lack of effect could be due to the following reasons.First, the NIH SABV policy exerts minimal influence on the rates of sex and gender reporting and analysis. Second, SABV may have not manifested in the cardiovascular literature. Third, we cannot differentiate if the effect is solely due to the NIH SABV policy or a combination of funding policies have preceded and succeeded it such as the CIHR's federal policy to account for sex in research in 2009, the European Association of Science Editors (EASE) gender policy mandate in 2013 and the European Commission funded GENDER‐NET Plus in 2017. In contrast, one could speculate that NIH funded studies would report sex better than non‐NIH funded studies. However, a study analysing sex bias in neuroscience studies published in 2017 (n = 1800) found that there little to no evidence that sex bias and omission differed by reported NIH funding status.

Limitations

One inherent weakness of this study is that we only assessed what was reported in the articles. Sex and gender considerations may have been implemented but not reported, which may have led to an underestimation of their prevalence. In addition, the findings of this study may not be generalizable to other types of studies within or outside the cardiovascular field. We also did not use other methods of searching such as manual reference search or citation search to identify studies not captured by our electronic search; therefore, there is a chance that some studies were missed. However, our large study sample mitigates this limitation with little possible changes in reporting prevalence and temporal change.

Conclusions

Our investigation of sex reporting and analysis in CRT cohort studies reporting clinical and safety outcomes over the past 20 years demonstrates that sex is suboptimally reported and inadequately analysed. Improvements in the consideration of sex were observed over time, but the NIH SABV policy only impacted sex considerations in statistical models. In addition, gender considerations were absent from the studies. Funders and journal editors can drive authors to consider sex and gender in reporting and analysis by developing policies to encourage or enforce reporting and analysis of sex and gender. This will result in improved reporting of sex and gender and scientific transparency, leading to better healthcare for all.

Conflict of interest

None declared.

Funding

This research is supported by the Graduate Studentship Program at the Bruyere Research Institute.

Figure S1. PRISMA chart of included studies.

Table S1. Sex considerations in cardiac resynchronization therapy studies stratified by study design.

Figure S2. Temporal trends in sex considerations in Cardiac Resynchronization Therapy (CRT) cohort studies reporting clinical and safety outcomes. Studies published in year 2020 were excluded since we did not capture all of the studies published that year. Two sided Cochrane‐Armitage test was used for trend analysis.

Figure S3. Temporal trends in sex considerations reported in statistical models and outcome data of Cardiac Resynchronization Therapy (CRT) cohort studies reporting clinical and safety outcomes. Proportions for sex considerations in statistical models were calculated from the number of studies that reported the use of statistical models published per year (n = 173). Studies published in year 2020 were excluded since we did not capture all of the studies published that year. Two sided Cochrane‐Armitage test was used for trend analysis. n.s = non‐significant. * P ≤ 0.05. **P ≤ 0.01. **** P ≤ 0.0001.

Figure S4. Impact of NIH Sex as a Biological Variable (SABV) policy on the reporting of sex considerations in statistical models and outcome data of Cardiac Resynchronization Therapy (CRT) cohort studies reporting clinical and safety outcomes. Studies published in year 2020 were excluded since we did not capture all the studies published that year. Dotted line at 2015 represent the release of the National Institutes of Health (NIH) policy on including sex as a biological variable in reporting, study design and analysis. Segmented regression analysis was used to assess the impact of the NIH policy on the reporting of sex considerations.

Click here for additional data file.