Chemotherapy effectiveness in trial-underrepresented groups with early breast cancer: A retrospective cohort study.

BACKGROUND: Adjuvant chemotherapy in early stage breast cancer has been shown to reduce mortality in a large meta-analysis of over 100 randomised trials. However, these trials largely excluded patients aged 70 years and over or with higher levels of comorbidity. There is therefore uncertainty about whether the effectiveness of adjuvant chemotherapy generalises to these groups, hindering patient and clinician decision-making. This study utilises administrative healthcare data-real world data (RWD)-and econometric methods for causal analysis to estimate treatment effectiveness in these trial-underrepresented groups. METHODS AND
FINDINGS: Women with early breast cancer aged 70 years and over and those under 70 years with a high level of comorbidity were identified and their records extracted from Scottish Cancer Registry (2001-2015) data linked to other routine health records. A high level of comorbidity was defined as scoring 1 or more on the Charlson comorbidity index, being in the top decile of inpatient stays, and/or having 5 or more visits to specific outpatient clinics, all within the 5 years preceding breast cancer diagnosis. Propensity score matching (PSM) and instrumental variable (IV) analysis, previously identified as feasible and valid in this setting, were used in conjunction with Cox regression to estimate hazard ratios for death from breast cancer and death from all causes. The analysis adjusts for age, clinical prognostic factors, and socioeconomic deprivation; the IV method may also adjust for unmeasured confounding factors. Cohorts of 9,653 and 7,965 were identified for women aged 70 years and over and those with high comorbidity, respectively. In the ≥70/high comorbidity cohorts, median follow-up was 5.17/6.53 years and there were 1,935/740 deaths from breast cancer. For women aged 70 years and over, the PSM-estimated HR was 0.73 (95% CI 0.64-0.95), while for women with high comorbidity it was 0.67 (95% CI 0.51-0.86). This translates to a mean predicted benefit in terms of overall survival at 10 years of approximately3% (percentage points) and 4%, respectively. A limitation of this analysis is that use of observational data means uncertainty remains both from sampling uncertainty and from potential bias from residual confounding.
CONCLUSIONS: The results of this study, as RWD, should be interpreted with caution and in the context of existing and emerging randomised data. The relative effectiveness of adjuvant chemotherapy in reducing mortality in patients with early stage breast cancer appears to be generalisable to the selected trial-underrepresented groups.

Introduction

The use of adjuvant chemotherapy after surgical treatment of early breast cancer is a major contributor to the reduction in mortality from breast cancer over the last 3 decades. A global collaboration of trialists published a definitive individual patient data meta-analysis of more than 100,000 women with breast cancer, concluding that chemotherapy reduces the risk of dying from breast cancer by about a third [1,2]. However, the clinical trials upon which this evidence relies were performed in highly selected patient populations including few patients older than 70 years or patients with other significant health conditions. In routine clinical practice there are many patients who would never have been included in those trials due to advanced age, comorbidity, or frailty. The decision of whether or not to administer or undergo adjuvant chemotherapy is informed solely by evidence from the ‘trial eligible’ patient population. Therefore, decisions are based on the assumption that the estimated treatment effect is generalisable, despite differences in personal characteristics.

A lack of evidence of generalisability of clinical trial results (the problem of external validity) has been recognised as a major barrier to translating research findings into changes in clinical practice [3]. Generalisability of experimental findings to the populations seen in clinical practice can be maximised by designing trials to be pragmatic, having wide inclusion criteria for patients, recruiting patients into trials from a variety of typical clinical settings, reducing differences between trial protocols and clinical practice, and obtaining data on relevant outcomes or adverse events [4].

Attempts to address a perceived lack of generalisability in the evidence base for adjuvant chemotherapy in patients aged 70 years and over by conducting further randomised controlled trials have failed due to poor recruitment [5,6]. In the pilot phase of the ACTION trial, a lack of equipoise on the part of both clinicians and patients was noted as the major reason for an unwillingness to participate in randomisation. The persistent lack of direct trial evidence for women aged 70 years and over engenders considerable uncertainty about the balance of patient benefit and harm from chemotherapy, which may lead to suboptimal treatment decisions and unwarranted variation in practice. When randomised studies are infeasible, as may be the case here, alternative methods using observational data may represent the best available source of evidence on treatment effectiveness. Observational data from routine sources have the potential to enhance external validity but at a cost of additional potential bias arising from the research design [7]. A lack of randomisation means that unaccounted for differences between patients who receive treatment and those who do not may bias results, a feature called residual confounding.

Prior analysis of Scottish Cancer Registry data demonstrated that several real world evidence (RWE) methods utilising available routine data from otherwise healthy women under 70 years are feasible and may give comparable results to randomised data in estimating the effectiveness of chemotherapy in early stage breast cancer [8]. Hazard ratios for breast cancer mortality in the trial-represented population were concordant between RWE and a randomised trial meta-analysis. However, results for all-cause mortality were less concordant, indicating a greater potential for bias in relation to this outcome [8].

This study aims to estimate the effectiveness of adjuvant chemotherapy for early stage breast cancer in reducing mortality for women aged 70 years and over and for women with a high level of comorbidity using real world data (RWD). The estimates are presented for consideration alongside available evidence from the trial-represented population.

Methods

A retrospective cohort study design was used. All records of women with primary invasive breast cancer (ICD-10 C50) diagnosed from 1 January 2001 to 31 December 2015 were retrieved from the Scottish Cancer Registry. Linkage to routine outpatient and inpatient records (ISD Scotland datasets SMR00 and SMR01, 1996 to 2017) was achieved using each patient’s uniquely identifying Community Health Index (CHI) number. Selection and linkage was provided by ISD Scotland. Use of these anonymous data in this research project was reviewed and approved by the NHS Scotland Public Benefit and Privacy Panel. Follow-up of vital status was available to April 2017. Women with first breast cancer were identified based on the first chronological record of diagnosis code ICD-10 C50 for the unique patient identifier. In the case of multiple simultaneous records, the record with the most complete data was selected. If completeness was identical then the record with the worse prognosis (PREDICT score) was selected. When records were identical in all extracted variables, the duplicate records were deleted.

Exclusion criteria included male sex, advanced cancer (clinical M stage = 1), no recorded surgery, or recorded neoadjuvant therapy (chemotherapy or hormone therapy prior to surgery). PREDICT (version 2) prognostic scores [9] were estimated for all patients with complete input data. The prognostic algorithm has previously been shown to be well calibrated in this population [10]. Further details of the dataset and variables are available in [8].

The ≥70 patient group was selected based on age at date of diagnosis being 70 years or greater. Based on clinical expert opinion, high comorbidity was defined as meeting 1 or more of 3 conditions: (1) a score of 1 or more on the Charlson comorbidity index [11] based on inpatient records from the previous 5 years, (2) total inpatient bed days in the previous 5 years in the top decile (6 or more) of the full cohort, and/or (3) 5 or more outpatient visits to respiratory, cardiology, or rheumatology specialties in the previous 5 years. Women aged 70 years and over were excluded from the high comorbidity group. A sensitivity analysis selected women aged 70 years and over excluding those with high comorbidity.

Statistical analysis

The choice of statistical analysis was determined based on prior assessment of the feasibility and validity of a range of econometric methods in the trial-represented population of cases from the same registry [8]. The study proposal for data analysis is included in S1 Text. The plan originally included only the regression discontinuity design but was later expanded to include regression adjustment, propensity score matching (PSM), and instrumental variable (IV) designs.

Two RWE designs—PSM and IV—were used to obtain estimates of adjuvant chemotherapy effectiveness. PSM was conducted using a propensity score as constructed in [8] with 1:1 nearest-neighbour matching within callipers, without replacement. Propensity scores were estimated using probit regression with the following explanatory variables: PREDICT 10-year probability of mortality, age at diagnosis, number of positive lymph nodes, pathological tumour size, tumour histological grade, mode of detection, estrogen receptor (ER) status, HER2 status, hormone therapy use, radiotherapy use, year of diagnosis, Scottish Index of Multiple Deprivation (SIMD) quintile, Charlson comorbidity index, log total inpatient bed days (in the 5 years prior to diagnosis), and log total outpatient visits (in the 5 years prior to diagnosis). Interactions of other clinical prognostic factors with ER status were also included. Two versions of the IV approach were estimated using (1) PREDICT chemotherapy benefit score and (2) PREDICT chemotherapy benefit score interacted with a post-2010 dummy variable. Two-stage residual inclusion was used to implement the IV approach as this is suitable when both the treatment and the outcomes are limited dependent variables [12]. Confidence intervals were calculated by simple bootstrap with 1,000 replications. Full details and justification for the selection of these RWE designs and specifications as the most suitable means of providing estimates of adjuvant chemotherapy are available in [8] and S2 Text. The analyses were repeated for the outcomes of breast cancer mortality and all-cause mortality. Breast cancer mortality was defined as a breast cancer code recorded as the primary cause of death or 1 of the 3 contributing causes of death in the death certificate.

Directly comparable estimates of adjuvant chemotherapy effectiveness are taken from the Early Breast Cancer Trialists’ Collaborative Group (EBCTCG) meta-analysis. The EBCTCG meta-analysis estimated that the HR for mortality from breast cancer for newer anthracycline regimens versus placebo was 0.71 (95% CI 0.62 to 0.83) [2]. The corresponding HR for all-cause mortality was 0.83 (95% CI 0.73 to 0.94).

HRs were estimated for comparison with trial meta-analysis reports, but HR is very limited for informing clinical decisions. Therefore, we have applied the estimated HR in a recalibrated version of the PREDICT model to produce an estimate of survival benefits (absolute risk reductions) over 10 years for all women in the sample for each group. This analysis was added following reviewer comments to aid clinical interpretation. Recalibration replaced the coefficient for chemotherapy use with a coefficient corresponding to the PSM-estimated HR for each group in this study. Expected benefit estimates were stratified in 5-year age bands for women aged 70 years and over, while women with comorbidities were stratified into 3 groups in 10-year age bands from 40 to 69 years. For the high comorbidity group, an additional hazard of non-breast-cancer death (HR ranging from 1–5) was applied to reflect how additional mortality from specific comorbidities might impact chemotherapy benefits (a method similar to that previously used in the [now defunct] Adjuvant Online! decision tool, in which the clinician could specify an additional risk of mortality [13]). Mean benefit from chemotherapy and the proportion of women with benefit at or above the guideline thresholds of 3% and 5% [14] were calculated.

Results

A total of 9,653 and 7,965 eligible patient records were identified for the ≥70 group and the high comorbidity group, respectively (Fig 1). Patient characteristics in the ≥70 and high comorbidity groups are displayed in Table 1.

Fig 1

Patient record selection.

Table 1

Summary statistics of trial-underrepresented group samples from the Scottish Cancer Registry, 2001–2015.

Characteristic	≥70 years	High comorbidity
Total number of patients	9,653	7,965
Total time at risk (years)	56,864	57,094
Median follow-up (years)	5.17	6.53
Number of breast cancer deaths	1,935	740
Number of other deaths	2,018	648
Five-year survival rate	74.7%	87.2%
Median age at diagnosis, years	76	60
Age	76.65 (0.05)	58.75 (0.14)
Tumour size (mm)	24.68 (0.16)	19.79 (0.21)
Inpatient days (in the 5 years prior)	4.99 (0.15)	13.39 (0.45)
PREDICT benefit score	3.02 (0.02)	2.68 (0.04)
Outpatient visits (in the 5 years prior)	7.22 (0.10)	12.68 (0.25)

Data given as mean (SD) unless otherwise indicated. Additional summary statistics available in S1 Table.

The results of PSM sample balance tests and IV first-stage results are available in S2 and S3 Tables. Good balance was demonstrated for both of the matched samples, with no clinically important differences in important covariates between treated and untreated groups. First-stage results for the IV analysis indicated statistically significant effects of the proposed instruments on the probability of receiving the treatment (≥70 IV1: 0.49 [95% CI 0.37–0.6], P < 0.001; IV2: 0.14 [95% CI 0.04–0.23], P = 0.006; high comorbidity IV1: 0.41 [95% CI 0.26–0.56], P < 0.001; IV2: 0.1 [95% CI 0.02–0.18], P = 0.012), an important assumption of the method. The estimated hazard ratios for death in women aged 70 years and over and women with high comorbidity are displayed in Table 2.

Table 2

Hazard ratios for death from breast cancer and all causes in women aged 70 years and over and women with high comorbidity.

Real world evidence method	N	Breast cancer death			Death from any cause
		Deaths	HR	95% CI	Deaths	HR	95% CI
Reference1			0.71	0.62–0.83		0.83	0.73–0.94
Women aged ≥70 years
PSM	1,298	431	0.78	0.64–0.95	568	0.71	0.60–0.85
IV1	9,653	1,935	0.57	0.42–0.74	3,953	0.61	0.49–0.74
IV2	9,653	1,935	0.57	0.42–0.73	3,953	0.63	0.50–0.76
Women with high comorbidity
PSM	2,034	254	0.67	0.51–0.86	421	0.67	0.56–0.82
IV1	7,965	740	0.68	0.42–1.10	1,388	0.92	0.63–1.33
IV2	7,965	740	0.59	0.37–0.99	1,388	0.82	0.58–1.22

1Early Breast Cancer Trialists’ Collaborative Group meta-analysis of newer anthracycline-containing regimens versus placebo [2].

IV, instrumental variable; PSM, propensity score matching.

Breast cancer mortality hazard ratios for women aged 70 years and over and those with high comorbidity are consistent with trial meta-analysis estimates. There was a closer match with the PSM estimates and somewhat lower HRs reported using the IV method. The confidence intervals indicate that sufficient uncertainty remains such that identical HRs between any of these methods and the trial meta-analysis cannot be ruled out at conventional thresholds, i.e., there was no strong evidence against generalisability. Overall results indicate a beneficial effect for chemotherapy at conventional statistical thresholds with the exception of IV1 for breast cancer death and IV1 and IV2 for all deaths in the high comorbidity group. Results were not sensitive to excluding women with high comorbidity from the ≥70 group (S4–S6 Tables).

Tables 3 and 4 translate these results into clinically meaningful estimates of overall survival benefit using a recalibrated version of the PREDICT model. For women aged aged 70 years and over, predicted benefits over 10 years average around a 3-percentage-point increase in survival. Benefits of 3% and above were estimated for approximately 40% of women of any age, while benefits of 5% and above were estimated for approximately 20% of women up to the age of 85 years. Over the age of 85 years, there are few women with predicted benefits at or above the 3% threshold and almost none at or above the 5% threshold. The actual proportion of women aged 70 years and over who received chemotherapy was 10.2%.

Table 3

Predicted survival benefit with chemotherapy for women aged 70 years and over.

Age	N	Predicted 10-year survival without chemo (%)	Predicted 10-year survival with chemo (%)	Mean absolute mortality benefit (%)*	Proportion with ≥3% benefit (%)	Proportion with ≥5% benefit (%)	Observed actual percent that received adjuvant chemo
70–74	3,955	57.9	62.x	2.9	39.6	19.2	18.2
75–79	3,131	45.8	49.6	3.1	44.2	18.9	7.3
80–84	1,784	32.x	35.4	3.x	46.4	17.3	1.5
85–99	681	17.3	20.4	3.x	48.8	9.7	0.6
90–95	95	6.x	8.6	2.7	43.2	0	0

*Percentage point improvement in survival at 10 years.

chemo, chemotherapy.

Table 4

Predicted survival benefit with chemotherapy for women aged under 70 years with high comorbidity.

Age and additional hazard of non-BC death	N	Predicted 10-year survival without chemo (%)	Predicted 10-year survival with chemo (%)	Mean absolute mortality benefit	Proportion with ≥3% benefit (%)	Proportion with ≥5% benefit (%)	Observed actual percent that received adjuvant chemo
Age 40–49 years	673						62.2
1		73.4	81.3	5.1	61.7	41.5
1.5		72.3	80.1	5.1	61.4	41.2
2.5		70.1	77.7	5.x	60.3	40.3
5		64.9	72.2	4.7	59.7	38.6
Age 50–59 years	2,991						36.3
1		79.4	84.1	3.4	37.4	24.5
1.5		77.1	81.7	3.3	37.1	23.9
2.5		72.7	77.1	3.2	36.6	23.2
5		62.8	67.x	3.1	35.8	21.3
Age 60–69 years	3,793						23.1
1		73.2	77.2	3.3	36.4	22.5
1.5		68.4	72.3	3.2	35.7	21.9
2.5		59.8	63.5	3.x	35.1	20.6
5		42.9	46.3	2.9	34.2	18.8

BC, breast cancer; chemo, chemotherapy.

In younger age groups with high comorbidity, the benefits are commensurably greater due to less age-related competing risk of mortality from non-breast-cancer causes. Predicted survival benefit from chemotherapy is related to the assumed increased hazard of mortality from other causes, but only weakly. Even with a relatively large additional risk of mortality from comorbidity (HR = 5), the mean absolute benefit and proportion of women at or above the 3% and 5% thresholds are largely preserved. The actual proportion of women under 70 years with high comorbidity who received chemotherapy was 32.1%.

Discussion

These observational data suggest that the relative effectiveness of adjuvant chemotherapy in reducing mortality in women with early stage breast cancer appears similar for trial-underrepresented groups (aged 70 years and over and high comorbidity) and trial-eligible groups. If one accepts that the additional assumptions required by RWE methods are met in this case, then this would imply that estimates of treatment effectiveness among trial-eligible patients are generalisable to these trial-underrepresented groups. These results are also in agreement with previous studies using observational data from the Surveillance, Epidemiology, and End Results (SEER) database for women aged 65 years and above [15].

Limitations

The main limitation relating to this analysis is that, despite use of the more robust RWE methods for causal inference, there remains potential for bias in these results from residual confounding [7]. Residual confounding may arise from lack of data regarding chemotherapy regimens, limited use of the full dimensionality of the comorbidity data, or from as yet unknown confounding variables. For this reason, these results are not a direct substitute for evidence from a high-quality randomised trial. Data included in this study are from a single country, which may limit generalisability to other settings.

Strengths

A strength of this study is the use of a large population-based cohort of patients with high-quality outcome determination and covariate information obtained from data linkage. Previous studies using SEER data did not have access to the same range of covariates. To our knowledge, no previous study has tested the feasibility of alternative methods of analysis to the same extent as was undertaken in this research. Use of a population-based cohort enhances external validity, allowing better assessment of effectiveness in clinical practice as opposed to efficacy in a highly selected trial population.

How this study can inform future research

An important issue that this study informs is whether or not additional randomised trials are needed in the trial-underrepresented groups we have identified. Our results suggest that trials conducted within these groups with a no-chemotherapy arm are probably neither necessary nor desirable. A beneficial treatment effect, consistent with the reported trial meta-analysis, was observed in both groups. While previous attempts to conduct trials in women aged 70 years and over proved infeasible due to lack of recruitment, our results may change the clinical community’s interpretation of the existing evidence and willingness to recruit such women to trials comparing 2 forms of therapy, although patients’ perception of the risks and benefits and willingness to participate may remain a barrier. Also, some additional randomised data may become available in more selected populations. Randomisation is being conducted among patients classified into specific risk groups in ongoing studies [16,17], and standard adjuvant chemotherapy is a control arm in trials of new targeted therapies [18,19]. These trials will provide some new randomised data in the ≥70 age group, as the age restrictions are more relaxed in most of these trials than was the case in previous studies. However, the total number of patients randomised in this group may still be small.

A recommendation for future research is that the RWE estimates produced here should be replicated using comparable routinely collected data from other regions or countries. This is an important step in validating the results and could help to identify any biases arising from measurement or selection in these specific routine datasets. As RWD are not gathered solely for the purposes of research, the measurement and recording of some variables may be suboptimal in any given RWD [20]. One advantage of RWD is that replication is relatively inexpensive and does not raise additional ethical concerns related to equipoise, in contrast to replication of a randomised trial. Following replication of our results, a careful synthesis of both observational and trial data results should be attempted. This must reflect all the available data as well as current beliefs about treatment effect generalisability in this context. An important future direction for RWE will be to undertake more specific exploratory analysis to look for predictors of outcome within trial-underrepresented groups. This type of analysis will be useful to inform targeted data collection within specific underserved or at-risk groups.

How this study can inform clinical practice

This analysis can inform treatment guidelines and patient information in trial-underrepresented groups—as exemplified in our presentation of expected benefits in terms of absolute survival using a recalibrated decision tool. Additional methodological development is also needed to support this type of evidence synthesis. First, more clarity is needed around the best methods for synthesis of observational and randomised data [21]. Second, methods must also address the generalisability of effectiveness estimates beyond the trial population, based on both extrapolation from existing data and prior beliefs based on other knowledge of the topic of study.

Evidence from observational data using RWE methods supports the generalisability of treatment effectiveness estimates for adjuvant chemotherapy for early breast cancer to women aged 70 years and over and those with high levels of comorbidity. The results of this study, as with all RWE, should be interpreted with appropriate caution and in the context of existing and emerging randomised data.

(DOCX)

Click here for additional data file.

Additional summary statistics of trial-underrepresented group samples.

(DOCX)

Click here for additional data file.

PSM sample covariate balance for women aged 70 years and over and women with high comorbidity.

(DOCX)

Click here for additional data file.

First-stage regression results for each specification for women aged 70 years and over and women with high comorbidity.

(DOCX)

Click here for additional data file.

PSM estimates of adjuvant chemotherapy effectiveness for women aged 70 years and over excluding those with high comorbidity.

(DOCX)

Click here for additional data file.

First-stage regression results for each specification for women aged 70 years and over excluding those with high comorbidity.

(DOCX)

Click here for additional data file.

IV results for women aged 70 years and over excluding those with high comorbidity.

(DOCX)

Click here for additional data file.

Chemotherapy effectiveness in trial-underrepresented groups with early breast cancer: A retrospective cohort study—relevant sections of study proposal for data analysis.

(DOCX)

Click here for additional data file.

Details of PSM and IV methods.

(DOCX)

Click here for additional data file.

30 Aug 2019

Dear Dr. Gray,

Thank you very much for submitting your manuscript "Real-world evidence for chemotherapy effectiveness in trial under-represented groups with early breast cancer" (PMEDICINE-D-19-02078) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Sep 20 2019 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see http://journals.plos.org/plosmedicine/s/submission-guidelines#loc-methods.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Adya Misra,

Senior Editor

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

Please to provide a url and accession number needed to apply for data access.

Please include a reporting checklist such as RECORD so we can suggest that.

http://www.equator-network.org/reporting-guidelines/record/ and provide the completed checklist as supplementary information. Please do not use page numbers as these are likely to change.

The foornotes should be moved into the main text.

Please clarify where you received ethical approval, the committee should be listed in the Methods section.

Please revise your title according to PLOS Medicine's style. Your title must be nondeclarative and not a question. It should begin with main concept if possible. "Effect of" should be used only if causality can be inferred, i.e., for an RCT. Please place the study design ("A randomized controlled trial," "A retrospective study," "A modelling study," etc.) in the subtitle (ie, after a colon).

References-Please use the "Vancouver" style for reference formatting, and see our website for other reference guidelines https://journals.plos.org/plosmedicine/s/submission-guidelines#loc-references

Abstract-please clarify what is included in “high level of comorbidities”

Abstract-please combine methods and findings into a single sub-heading

Abstract-please include a sentence describing the limitations of your study at the end of the methods and findings section

Abstract-please provide further demographics of women included in this study within the methods/results

Abstract-Please quantify the main results (with 95% CIs and p values).

Abstract-Please include the important dependent variables that are adjusted for in the analyses.

Abstract-Please include the actual amounts and/or absolute risk(s) of relevant outcomes (including NNT or NNH where appropriate), not just relative risks or correlation coefficients. (example for absolute risks: PMID: 28399126)

"Did your study have a prospective protocol or analysis plan? Please state this (either way) early in the Methods section.

a) If a prospective analysis plan (from your funding proposal, IRB or other ethics committee submission, study protocol, or other planning document written before analyzing the data) was used in designing the study, please include the relevant prospectively written document with your revised manuscript as a Supporting Information file to be published alongside your study, and cite it in the Methods section. A legend for this file should be included at the end of your manuscript.

b) If no such document exists, please make sure that the Methods section transparently describes when analyses were planned, and when/why any data-driven changes to analyses took place.

c) In either case, changes in the analysis—including those made in response to peer review comments—should be identified as such in the Methods section of the paper, with rationale."

At this stage, we ask that you include a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary

Introduction-please simplify the last sentence in the first paragraph “When a patient is not themselves part of that population then decisions are based

on the assumption that the estimated treatment effect is generalisable, despite the differences

in personal characteristics”

Introduction-please provide a reference for “When randomised studies are infeasible, as in this case, alternative methods using observational data may represent the best available source of evidence for treatment effectiveness” or consider toning down

Introduction-please provide a reference for “Hazard ratios for breast cancer mortality in

the trial-represented population were concordant between RWE and a randomised trial metaanalysis. However, results for all-cause mortality were less concordant, indicating a greater potential for bias in relation to this outcome”

Introduction- please provide a reference for “Clinical expert opinion suggests that the predominant chemotherapy regimens in use in Scotland in this time period were anthracycline containing (CEF/CAF) with cumulative doses similar to those described in the more recent randomised studies” and for “The EBCTCG metaanalysis

estimated that the HR for mortality from breast cancer for newer anthracycline

regimens versus placebo was 0.71 (0.62 to 0.83) (2). The corresponding HR for all-cause

mortality was 0.83 (0.73 to 0.94)”

Table 1- please provide units of measurement for Tumour size

Results-Please clarify the sentence “Good balance was demonstrated for both of the matched samples”

Results-Please provide a p-value and 95% CI for “First-stage results for IV indicated statistically significant effects of the proposed instruments on the probability of receiving the treatment, an important assumption of the method” and “For breast cancer mortality hazard ratios for women aged over 70 were very similar to trial meta-analysis estimates for PSM and somewhat lower using IV”

Results- Please provide further details regarding “Results in relation to all-cause

mortality report lower hazard ratios compared to the corresponding trial meta-analysis

estimates, with the exception of IV1 and IV2 estimates in the high comorbidity group”.

Results-please explain “degree of bias in all-mortality results may vary between over 70 and

high comorbidity groups reflecting differences in patient selection into treatment group”

Comments from the reviewers:

Reviewer #1: This article reports that hazard ratios for breast-cancer specific survival observed using real world data (Scottish registry) in elderly/frail populations are somehow similar to that noted in randomized trials.

General comment:

Two aspects can be distinguished in that paper:

- The real-world data analysis "exercise", using different tools (propensity score matching and instrumental variable analyses), applied to a large dataset that has been linked to (some of the) patient characteristics. A very brief reminder on the principles/differences between these two analyses would be welcome for non-experts in the introduction or discussion, as the PLoS Medicine audience is much broader than real world data experts. Note that the discussion is focusing exclusively on the potential role of real-world evidence (with much emphasis), with no discussion on the medical relevance of the findings.

- The overall interest of these results for breast oncologists, which is very limited. There is no clinical data available in the main manuscript, no subgroup analysis... The bottom line of this study is that, in (the few?) patients that were considered as fit enough to receive chemotherapy, the observed hazard ratio for breast cancer-specific survival is apparently similar to that in randomized trials. However, this patient population has a much shorter lifespan, so the impact of chemotherapy on overall survival remains limited and cannot be the same than the one observed in younger/healthier patients (that are included in trials). So, it is satisfactory to know that Scottish oncologists were right when prescribing adjuvant chemotherapy in some patients from a highly selected elderly/frail population, but I don't see many other uses of these results. More clinically relevant results would be, e.g. the number needed to treat, classified by age (or by number/type of comorbidities) and focusing on OS rather than BCSS.

Reviewer #2: This paper investigates whether results on chemotherapy effectiveness for treating early breast cancer, derived from randomised trials, are generalisable to patients that are under-represented in these trials.

It tests this by deriving analogous hazard ratios from real world data using statistical methods that the authors have previously shown to be feasible in this context.

This contributes to an important area of research, aiming to determine the most effective individually appropriate and consistently administered treatment decisions for older women with early breast cancer who may also have comorbidities.

On the whole it is straightforward and clearly presented, particularly as the technicalities of the methods used are referenced elsewhere (in an earlier paper by the same authors) rather than described in detail.

This paper is something of a corollary to that earlier paper but nevertheless makes an valid contribution in respect to the trial unrepresented groups.

There are a few places where the meaning is not clear and I would suggest that the discussion in particular should be made to read in a more clear and focussed way.

Methods:

Complete case analysis was used, but there is no discussion of the possible implications of this. Was any consideration given to using multiple imputation, for instance, to complete the missing data?

Results and discussion:

I think the description and discussion of the results could be sharpened up considerably.

Phrases such as "very similar ... somewhat lower ... even more closely aligned" are a bit vague and possibly subjective if there is no predefined criteria of accuracy.

Simply stating the actual differences might be better. Discussing the confidence intervals, as is done, is more objective but needs to be done better for the high-comorbidity group.

There is also the point that by using 3 methods you increase the chance of at least one of them agreeing with the trial results. I say this in particular with reference to the authors' comment that the IV methods are more successful in the high-comorbidity group.

I would suggest it is more important that the three methods all clearly show a beneficial treatment effect in the case of the over 70 group, as shown by the confidence intervals.

For the high comorbidity group this becomes slightly more marginal, with one CI bordering on including unity and another clearly doing so. This should be noted and discussed.

I think this will still allow for the methods used to be proposed as potentially useful for assessing generalisability and will point more clearly to the need for future work to consolidate these methods.

Detailed comments:

p3 para2

Should this be "A lack of evidence of generalisability ..."

p4 last sentence of introduction

I'm not sure what this actually means in practice. Maybe just make the sentence simpler.

Table 2 and Table 3

Adding a row to both of these tables with the EBCTCG meta analysis HR values would make comparison easy and help the reader.

p8 sentence 3: "Point estimates ..."

Comparison is being made between the analysis for the two groups and also between the estimates for the high comorbidity group and trial values. It is not entirely clear therefore which of these the "greater uncertainty" refers to.

p8 final sentence of Results: "It should be noted ..."

I can make nothing of this sentence and it should be rewritten more clearly.

p8 first sentence of Results

".. appears similar .." is rather vague and non-committal

next sentence

Does it imply they are generalisable or rather "not provide evidence against generalisability"?

p9 "Our results suggest ... "

This sentence presumably relies on the fact that, regardless of the exact HR values, the results (nearly) all suggest a beneficial treatment effect. If so, make this explicit.

p9 "Although trials in ... "

Unclear what this sentence is saying

p9 I dislike the use of "has been/will be"

p10 "This analysis should ultimately ..."

Use of "should" makes it sound unconvincing

p10 second para

This whole paragraph is poorly expressed. Especially the second sentence.

p11 Final paragraph

My judgment would be that this mostly supports the generalisability (of the fact that chemotherapy is effective) to these groups but the exact level of benefit is no so clearly generalisable.

Reviewer #3: This paper uses propensity matching and instrumental variables to test the effectiveness of adjuvant chemotherapy using routinely collected data. Overall, I found the paper interesting and easy to follow; however, I have some concerns especially around the lack of details in the methods and results.

Major comments:

* I realise that similar methods were used previously (reference # 6 by the same authors); however, the current manuscript would benefit from the inclusion of additional details in the statistical analysis section including the propensity score calculation, the analysis models, the prognostic ability of the used instrumental variable, etc. I have included more specific comments related to this issue below.

* I found the results section a bit too succinct. I believe additional results (additional outcomes, graphics, inclusion of tables currently in the supplement) would make the manuscript more interesting (see suggestions below).

* Please include relevant checklists in the supplement. See https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001885

Minor comments:

* Consider including the main estimates together with their confidence intervals in the abstract

* Consider using "routinely-collected data" instead of "real-world data"

* In the method section, please clarify what period served as the "baseline" to derive covariates and variables included in the propensity score calculation. Did the baseline period consist of the five years preceding the cancer diagnosis? For women with a diagnosis in 2001, I then assume that the linked data went as far back as 1996?

* If I understand correctly, the group of women over 70 could include women with high comorbidities but the high-comorbidity group excluded women over 70. Please confirm/clarify.

* Please clarify how potential controls for PSM were identified (i.e. eligibility). Please also clarify what the PREDICT chemotherapy benefit score consist of. For completeness, please also indicate the dependent variable used in the propensity score probit model (presumably treatment with adjuvant chemotherapy).

* Please clarify how the covariates included in the propensity score calculation were selected and what other variables were available (but not selected) in the linked data. Have the authors checked the balance after matching between other variables (those not included in the propensity score calculation)? Were the variables included in the propensity score calculation also used as covariates for the model analysing the outcome (principle of double-robustness?

* In footnote 2 on Page 5, "chemotherapy use" appears in the list of explanatory variables included in the propensity model. I would have thought that chemotherapy use would be the dependent variable, not an explanatory variable. Please correct/clarify.

* The last paragraph of the "statistical analysis" section which starts with "clinical expert opinion suggests…" appears to relate to background information rather than methods. I would suggest moving it to the introduction.

* In the methods, please explain how the data was analysed including the models used, the methods for censoring data, any covariate adjustment, handling of missing data, potential adjustment for the within matched pair correlations, software used, etc..

* Please clarify/report on the prognostic ability of the PREDICT score.

* Please consider including the tables showing the balance between the matched group in the main manuscript instead of in the supplement. It might also be interesting to see the balance (or lack thereof) before matching. In the "balance table" please clarify what "P NH: m1=m2" and "SMD" mean and how they were calculated e.g. 2-sample t-test.

* Figure 1. Please add information to describe the selection of matched controls.

* Please consider combining tables 2 and 3. Please make sure the number of decimals is consistent.

* Please consider adding Kaplan-Meier plots

* The last paragraph of the results placed between Table 3 and the discussion appears to be more about discussing and interpreting the results that about the reporting the results. Please consider integrating this paragraph into the discussion itself.

* I would like to see more discussion around the potential for residual confounding including reference to the variables available in the linked data but not included in the matching as well as variables not available at all.

Laurent Billot

Any attachments provided with reviews can be seen via the following link:

[LINK]

13 Nov 2019

Dear Dr. Gray,

Thank you very much for re-submitting your manuscript "Real-world evidence for chemotherapy effectiveness in trial under-represented groups with early breast cancer; a retrospective cohort study" (PMEDICINE-D-19-02078R1) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor and it was also seen again by xxx reviewers. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

Our publications team (plosmedicine@plos.org) will be in touch shortly about the production requirements for your paper, and the link and deadline for resubmission. DO NOT RESUBMIT BEFORE YOU'VE RECEIVED THE PRODUCTION REQUIREMENTS.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We expect to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.

We look forward to receiving the revised manuscript by Nov 20 2019 11:59PM.

Sincerely,

Adya Misra, PhD

Senior Editor

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

D-19-02078

Title- please remove "real world evidence"

DAS- please revise to “anonymously” Please to provide a url and accession number needed to apply for data access

Did your study have a prospective protocol or analysis plan? Please state this (either way) early in the Methods section.

a) If a prospective analysis plan (from your funding proposal, IRB or other ethics committee submission, study protocol, or other planning document written before analyzing the data) was used in designing the study, please include the relevant prospectively written document with your revised manuscript as a Supporting Information file to be published alongside your study, and cite it in the Methods section. A legend for this file should be included at the end of your manuscript.

b) If no such document exists, please make sure that the Methods section transparently describes when analyses were planned, and when/why any data-driven changes to analyses took place.

c) In either case, changes in the analysis—including those made in response to peer review comments—should be identified as such in the Methods section of the paper, with rationale."

At this stage, we ask that you include a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary

Abstract- Background refers to early stage breast cancer and the methods and findings section refers to primary. If both are the same, could we use consistent language.

Square brackets for all references

Footnotes- should be incorporated into main body of text

Discussion lines 22-24 this needs to be revised and toned down since the introduction states problems with recruitment of women over 70 were due to physicians and patients

Discussion requires a subsection outlining the limitations of the study

Discussion section requires toning down of the conclusions as this is a retrospective cohort study not a trial. Please provide a brief explanation of how this study design allows you to infer the "effectiveness" versus "efficacy" of chemotherapy.

RECORD checklist should be provided as a stand-alone SI file and page numbers should be removed as they are likely to change during publication

Comments from Reviewers:

Reviewer #3: I am comfortable with the responses and revisions. However, I am slightly confused by the new analyses reporting predicted survival benefits (Tables 3 and 4). In Table 3, the header of columns 3 and 4 state "Mean predicted 10-year survival ... (%)". Are these numbers showing the proportion of women expected to survive at 10 years? If so, I believe the word "mean" should not be there. If it is indeed a mean/average, then I do not understand what these numbers represent. For transparency, I would suggest indicating that these analyses were done post-hoc. The authors might add that they were conducted in response to the initial peer-review.

-Laurent Billot

Any attachments provided with reviews can be seen via the following link:

[LINK]

26 Nov 2019

Submitted filename: Requests from Editors 2 responses.docx

Click here for additional data file.

26 Nov 2019

Dear Dr Gray,

On behalf of my colleagues and the academic editor, Dr. Steven Shapiro, I am delighted to inform you that your manuscript entitled "Chemotherapy effectiveness in trial under-represented groups with early breast cancer: a retrospective cohort study" (PMEDICINE-D-19-02078R2) has been accepted for publication in PLOS Medicine. The publication date for your manuscript will be December 31, 2019.

PRODUCTION PROCESS

Before publication you will see the copyedited word document (in around 1-2 weeks from now) and a PDF galley proof shortly after that. The copyeditor will be in touch shortly before sending you the copyedited Word document. We will make some revisions at the copyediting stage to conform to our general style, and for clarification. When you receive this version you should check and revise it very carefully, including figures, tables, references, and supporting information, because corrections at the next stage (proofs) will be strictly limited to (1) errors in author names or affiliations, (2) errors of scientific fact that would cause misunderstandings to readers, and (3) printer's (introduced) errors.

If you are likely to be away when either this document or the proof is sent, please ensure we have contact information of a second person, as we will need you to respond quickly at each point.

PRESS

A selection of our articles each week are press released by the journal. You will be contacted nearer the time if we are press releasing your article in order to approve the content and check the contact information for journalists is correct. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact.

PROFILE INFORMATION

Now that your manuscript has been accepted, please log into EM and update your profile. Go to https://www.editorialmanager.com/pmedicine, log in, and click on the "Update My Information" link at the top of the page. Please update your user information to ensure an efficient production and billing process.

Thank you again for submitting the manuscript to PLOS Medicine. We look forward to publishing it.

Best wishes,

Adya Misra, PhD

Senior Editor

PLOS Medicine

plosmedicine.org