BioPETsurv: Methodology and open source software to evaluate biomarkers for prognostic enrichment of time-to-event clinical trials.

Biomarkers can be used to enrich a clinical trial for patients at higher risk for an outcome, a strategy termed "prognostic enrichment." Methodology is needed to evaluate biomarkers for prognostic enrichment of trials with time-to-event endpoints such as survival. Key considerations when considering prognostic enrichment include: clinical trial sample size; the number of patients one must screen to enroll the trial; and total patient screening costs and total per-patient trial costs. The Biomarker Prognostic Enrichment Tool for Survival Outcomes (BioPETsurv) is a suite of methods for estimating these elements to evaluate a prognostic enrichment biomarker and/or plan a prognostically enriched clinical trial with a time-to-event primary endpoint. BioPETsurv allows investigators to analyze data on a candidate biomarker and potentially censored survival times. Alternatively, BioPETsurv can simulate data to match a particular clinical setting. BioPETsurv's data simulator enables investigators to explore the potential utility of a prognostic enrichment biomarker for their clinical setting. Results demonstrate that both modestly prognostic and strongly prognostic biomarkers can improve trial metrics such as reducing sample size or trial costs. In addition to the quantitative analysis provided by BioPETsurv, investigators should consider the generalizability of trial results and evaluate the ethics of trial eligibility criteria. BioPETsurv is freely available as a package for the R statistical computing platform, and as a webtool at www.prognosticenrichment.com/surv.

Introduction

Biomarkers are used for various purposes across research and clinical contexts. In a clinical trial of an intervention intended to prevent or delay some unwanted clinical event, a biomarker may be useful for “prognostic enrichment” of the trial [1-5]. A prognostically enriched trial uses a biomarker to enroll only patients at relatively higher risk of the unwanted clinical event. Since study power depends on observing events, running a trial in an enriched study population can allow for a smaller trial compared to an unenriched trial [6, 7]. Moreover, it may be more ethically acceptable to test an intervention only on patients at high risk for the clinical event, and ethically preferable to test on a smaller study sample. Prognostic enrichment can produce greater efficiency in evaluating new interventions, potentially benefiting patients, sponsors, and public health.

There is a substantial literature on biomarkers that are predictive of treatment efficacy [8-12], also referred to as treatment-selection biomarkers [13-16]. In contrast, little has been written about evaluating biomarkers for prognostic enrichment [1, 2]. As noted by Temple [1], prognostic enrichment is well-established in cardiovascular disease, where it is common that interventions are first studied in individuals who are at high risk. CONSENSUS, the first trial of enalapril, enrolled only very high-risk patients (6-month mortality of 44%). CONSENSUS demonstrated efficacy of enalapril with only 253 patients. Subsequent trials in groups with less severe disease needed to be much larger.

In nephrology, a trial for patient with autosomal dominant polycystic kidney disease (ADPKD) enriched for those at greater risk of a substantial decline in renal function [17]. Total Kidney Volume (TKV), measured at baseline, was used in combination with patient age and estimated glomerular filtration rate (eGFR) to identify high risk patients. Without TKV, it was determined that 13 patients would need to be screened to enroll 11 patients to observe one event. With TKV, 25 patients would need to be screened to enroll 9 patients and observe one event. The FDA qualified TKV as a prognostic biomarker for use in clinical trials for ADPKD on August 31, 2015 [18]. The PRIORITY trial in patients with type 2 diabetes enriched for patients at high-risk of the primary endpoint, confirmed microalbuminuria, which occurred in 28% of participants classified as high-risk and only 9% of those classified as low-risk [19]. Although the trial did not establish that spironolactone is efficacious for the primary endpoint, without enrichment a sample size 3–4 times as large would have been needed and many more patients would have been exposed to a therapy that has side effects.

Despite prognostic enrichment being well-established in cardiology and employed in other clinical areas, little has been written about how to evaluate a biomarker for prognostic enrichment or to consider the trade-offs of an enriched vs. unenriched trial [4, 6]. For trials with a binary primary outcome, our group previously published the Biomarker Prognostic Enrichment Tool (BioPET) [7]. We identified key considerations for evaluating a biomarker for prognostic enrichment, including: clinical trial sample size; number of patients to screen to enroll the trial; total patient screening costs and the total of per-patient costs for patients in the trial. BioPET includes methods and graphical devices to evaluate a biomarker on these dimensions for trials with a binary outcome, but cannot be used for trials with a time-to-event outcome such as survival. Compared to trials with binary outcomes, trials with time-to-event outcomes can utilize more information in the data and accommodate the partial information available in censored outcomes. This article describes new methods and open source software, BioPETsurv, for such trials.

As a motivating example, consider the population of patients with a hospitalized episode of acute kidney injury [20] and a hypothetical intervention intended to prevent or delay the onset of chronic kidney disease. A randomized trial will compare the hazard for chronic kidney disease in a treatment group and a control group. As a proof-of-principle illustration, in this article we use synthetic data that mimic an existing cohort [20] to illustrate BioPETsurv for prognostic enrichment in this setting (Example 1).

BioPETsurv accommodates two trial designs. The first design is a fixed-duration trial ‒ the observation period is the same for all patients. The second design has an accrual period plus a follow-up period. For example, there may be a 1-year accrual period and a 3-year follow-up period, so that the observation period varies between 3 and 4 years for study participants.

BioPETsurv can be used to evaluate a biomarker and (possibly right-censored) survival data on a sample of patients. Alternatively, investigators can specify some essential features such as the event rate without enrichment and the prognostic capacity of the biomarker in terms of a hazard ratio. BioPETsurv can simulate biomarker and survival data matching these specifications, allowing investigators to explore prognostic enrichment for their clinical setting.

In this article, “biomarker” can refer to either a single measured characteristic or a “composite biomarker” [2] combining multiple prognostic factors [7]. For simplicity, we use "survival" for any time-to-event variable.

Methods

Without loss of generality, assume that patients with higher levels of the biomarker tend to experience the unwanted clinical outcome sooner. For a binary outcome, the area under the ROC curve (AUC) summarizes the discrimination performance of a biomarker. For a survival outcome, BioPETsurv displays the Kaplan-Meier survival curves for the entire patient population and for enriched subsets.

A continuous biomarker can, in principle, be used for a low or high level of enrichment of a trial. The level of enrichment is the threshold (percentile in the biomarker) above which patients are eligible for the trial. For example, excluding patients from the trial below the 10th percentile in the biomarker would be a low level of enrichment; requiring patients in the trial to be in the top quartile would be a high level of enrichment. Based on the level of enrichment, the prognostic strength of the biomarker, and the length of the trial, BioPETsurv estimates the expected event rate absent intervention. The expected event rate together with statistical testing specifications (e.g., power) and the treatment effect to detect determine the trial sample size. The total number of patients screened to enroll the trial depends on the trial sample size and the level of enrichment. For example, a trial with a 50% level of enrichment requires, on average, 2 patients to be evaluated to identify one eligible for the trial. Under the assumption that patients express interest in enrolling in the trial at a constant rate over time, `total number of patients screened' is a proxy for the calendar time to enroll the trial [7].

For cost analysis, BioPETsurv allows the cost for a patient in the trial to be either constant, or depend on the time the patient is in the trial before the primary endpoint. The latter may be realistic if the endpoint is death. The cost of screening, such as assay costs or patient work-up, must also be specified. Based on these user-specified costs, BioPETsurv calculates total trial cost for each enrichment level.

A key element in prognostic enrichment is the time-specific event rate by the end of the trial in enriched subgroups, which must be estimated. This can be done using Kaplan-Meier methods in subgroups. Alternatively, the nearest neighbor estimation method [4] allows the censoring process to depend on the biomarker and guarantees monotone estimated Receiver Operating Characteristic curves for time-specified outcomes. BioPETsurv offers both methods for fixed-duration trials and uses Kaplan-Meier methods for trials with an accrual period plus a follow-up period.

Fixed-duration trials

Given type I error rate α, power 1-β, and treatment hazard ratio HR, the number of events needed [21] is . For a given enrichment level and trial length, let be estimated survival; the event rate is in the control arm and in the treatment arm. Let be the sample size in one arm of a trial planned to have equal sample size in each arm. Then , so total N is . Let C1 be the cost for a patient in the trial and C2 the cost of screening (such as assay cost). For enrichment at quantile t (patients with biomarker below quantile t are excluded), total cost is . Let . We calculate the standard deviation (SD) from the delta method, and . We treat N, which comes from a standard sample size formula, as fixed; variability comes from .

Trials with an accrual period and a follow-up period

Let a and f be the accrual and follow-up time respectively. The only difference from a fixed-duration trial is in estimating the event rates, and , when participants are followed for different periods of time. Following [21], from Simpson’s rule and , with standard errors estimated by bootstrapping. All other trial characteristics follow as for fixed-duration trials.

Simulating data to allow investigators to explore prognostic enrichment

To allow investigators to explore prognostic enrichment without data on survival and the biomarker for a sample of patients, BioPETsurv can simulate data to mimic specific clinical parameters. Survival is simulated from a Weibull distribution with user-specified shape parameter k, which allows hazards to be constant, increasing, or decreasing. The user also specifies the survival probability p at time T, which are used to solve for the Weibull scale parameter a. We expect investigators will specify p based on knowledge of overall survival in the patient population. The data simulator takes p as the survival probability for individuals with mean biomarker level. The survival curve for this group is similar to the overall survival curve. The prognostic strength of the biomarker is specified by the hazard ratio for a 1 standard deviation difference in the biomarker. Without loss of generality, the biomarker X is mean-centered so that X = 0 is the baseline group. Given Weibull shape and scale parameters, baseline hazards are and under proportional hazards an individual with biomarker X has hazard , which corresponds to a Weibull distribution with the same shape parameter k and scale parameter . Biomarker data are simulated to have either a symmetric (normal) or right-skewed (lognormal) shape (user-specified). Based on biomarker value X = x, survival time is simulated from the appropriate Weibull distribution but censored at time T. The joint distribution of simulated biomarker and survival times is used for prognostic enrichment analysis.

Results

Example 1: A modestly prognostic biomarker and fixed-duration trial

Fig 1 and Table 1 show an example using BioPETsurv to evaluate a biomarker that is modestly prognostic of the event, with HR 1.2 corresponding to one standard deviation difference in the biomarker. The trial will be either 36 or 48 months. Fig 1A shows estimated survival curves for screening threshold 0% (top curve), i.e., for all patients (no enrichment). The plot shows that events accumulate more quickly in enriched subpopulations of patients, showing more quickly decreasing survival curves for enrichment levels 25%, 50%, and 75% (meaning that patients with biomarker below the 25th, 50th, or 75th percentile are excluded). Two vertical lines indicate the candidate trial lengths, 36 and 48 months. In all other panels in Fig 1, the horizontal axis is the level of enrichment, with 0% representing an unenriched trial. Fig 1B shows the estimated event rate as a function of the level of enrichment for both candidate trial lengths. Based on these event rates and specifying 90% power to detect treatment hazard ratio 0.8 (two-sided testing, α = 0.05), Fig 1C shows the sample size for each trial duration. As expected, the longer trial requires fewer patients than the shorter trial. Fig 1D shows the number of patients needed to screen to enroll the trial. With this modestly prognostic biomarker, the screening total increases with higher enrichment, although the increase is modest below 50% enrichment. Fig 1E and 1F display the cost analysis, with per-patient costs of $4000 (36-month trial) and $5000 (48-month trial). The screening cost was set at $500. In this example, with less than 25% enrichment an enriched trial is anticipated to be more expensive than an unenriched trial because the decrease in sample size is not enough to offset the cost of screening. The cost analysis shows cost savings for higher levels of enrichment.

Fig 1

BioPETsurv analysis of a modestly prognostic biomarker for a fixed-duration 36-month or 48-month trial.

Investigators are considering the biomarker for enrichment of either a 36-month or 48-month trial and specified 90% power to detect a hazard ratio of 0.8 using two-sided testing and α = 0.05. For cost analysis, the cost of screening was $500 and the cost of one patient in the trial was $4000 (36-month trial) and $5000 (48-month trial). The biomarker in this example has HR≈1.2 corresponding to a 1 SD difference in the marker.

Table 1

BioPETsurv analysis of a modestly prognostic biomarker (Example 1).

Screening Threshold	Event Rate (%)		Sample Size		Total Screened		Reduction in Total Cost (%)
	36 mo	48 mo	36 mo	48 mo	36 mo	48 mo	36 mo	48 mo
0%	14	18	6819	5221	6819	5221	0	0
5%	14	18	6726	5102	7080	5371	-12	-8
10%	14	18	6605	5045	7339	5606	-10	-7
15%	14	19	6441	4879	7578	5740	-8	-4
20%	15	20	6155	4668	7694	5835	-4	-1
25%	15	20	6014	4542	8019	6056	-3	1
30%	16	21	5825	4408	8322	6298	-1	4
35%	16	22	5643	4317	8682	6642	1	5
40%	17	22	5540	4221	9234	7035	2	6
45%	18	23	5299	4013	9635	7297	5	9
50%	19	25	4949	3733	9898	7466	9	14
55%	19	25	4845	3648	10767	8107	9	15
60%	21	27	4407	3414	11018	8536	15	18
65%	22	29	4141	3189	11832	9112	18	21
70%	23	29	3956	3147	13187	10491	18	20
75%	24	29	3905	3179	15620	12716	14	15
80%	27	33	3445	2821	17226	14106	18	19
85%	32	39	2880	2373	19201	15821	23	24
90%	37	45	2497	2012	24971	20121	18	23

In 36 months the clinical event occurs in 13% +/- 1% of patients without intervention; and 18% +/- 1% in 48 months. Sample size calculations reflect 90% power to detect 0.8 treatment hazard ratio using two-sided hypothesis testing and α = 0.05. The cost of screening is $500/patient and the per patient trial cost is $4000 (36-month trial) or $5000 (48-month trial). Screening Threshold is the proportion of patients who will be screened out of the trial based on biomarker level. Event Rate is the estimated rate of the clinical event in the enriched study population not receiving the intervention. Sample Size is the trial sample size calculated based on the event rate and statistical testing specifications. Total Screened is the total number of patients who would need to be screened to enroll the trial, which depends on the sample size and level of enrichment (screening threshold). Total Cost summarizes patient-related costs of different levels of enrichment, specifically the cost of biomarker-based screening and the cost of having a patient in a trial. Results show the potential for the biomarker to allow substantially smaller trial sample size and cost savings, but impose a greater burden on the total number of patients to screen to enroll the trial. These results are displayed in Fig 1, which also displays standard error estimates.

Example 2: Simulating data for a highly prognostic biomarker and a trial with accrual period and follow-up period

We illustrate the BioPETsurv data simulator. We set simulation parameters to mimic the clinical setting of Example 1 but anticipating a more highly prognostic biomarker. We simulated biomarker and survival data for n = 5,000 patients with event rate 18% at 48 months. We specified constant hazards, and a hazard ratio 2.8 corresponding to 1 standard deviation difference in the biomarker, which we simulated as normally distributed. Fig 2 and Table 2 give prognostic enrichment analysis using the simulated data and planning a trial with a 12-month accrual period and a 36-month follow-up. We again specified 90% power to detect 0.8 treatment hazard ratio, two-sided testing, and α = 0.05.

Fig 2

BioPETsurv analysis of simulated biomarker for a trial with a 12-month accrual period and 36-month follow-up period.

Investigators are planning a trial with a 12-month accrual period plus a 36-month follow-up period, and anticipate having a marker with HR≈2.8 corresponding to a 1 standard deviation difference in the marker. The BioPETsurv data simulator generated data for a normally distributed biomarker with this prognostic strength. Sample size calculations specified 90% power to detect a treatment hazard ratio of 0.8 using two-sided testing and α = 0.05. For cost analysis, patient screening cost was $300 and the cost of a patient in a trial was $100/month before the clinical endpoint. Numeric results are in Table 2.

Table 2

BioPETsurv analysis of simulated biomarker for a trial with a 12-month accrual period and 36-month follow-up period (Example 2).

Screening Threshold	Event Rate (%)	Sample Size	Total Screened	Reduction in Total Cost (%)
0%	17	5394	5394	0
5%	18	5146	5417	-3
10%	19	4937	5486	1
15%	20	4720	5553	6
20%	21	4517	5647	10
25%	22	4311	5748	14
30%	22	4155	5936	17
35%	24	3905	6008	22
40%	25	3706	6177	26
45%	26	3518	6397	30
50%	28	3320	6640	34
55%	29	3131	6958	38
60%	31	2935	7338	41
65%	34	2741	7832	45
70%	36	2553	8511	48
75%	39	2360	9440	50
80%	43	2153	10766	53
85%	48	1916	12774	55
90%	54	1670	16701	54

Investigators are planning a trial with a 12-month accrucal period plus a 36-month follow-up period, and anticipate having a biomarker with HR≈2.8 corresponding to a 1 standard devfiation difference in the marker. The BioPETsurv data simulator generated data for a normally distributed biomarker with this prognostic strength. Sample size calculatations specified 90% power to detect a treatment hazard ratio of 0.8 using two-sided testing and α = 0.05. For cost analysis, patient screening cost was set to $300 and the cost of a patient in a trial was set to $100/month before the clinical endpoint. Results are displayed in Fig 2.

Fig 2A shows estimated survival curves for no enrichment, and 25%, 50%, and 75% enrichment. Compared to Fig 1A, there is greater separation between the curves because the biomarker here is more prognostic. Fig 2B shows the average event rate for each level of enrichment (the average accounts for the variable length of patient follow-up). Sample size decreases steadily with greater enrichment (Fig 2C). The total number of patients to screen to enroll the trial is gradually increasing for lower levels of enrichment but increases rapidly at high levels of enrichment (Fig 2D). With $300 screening cost and a patient in the trial costing $100/month before the clinical event, there is potential for substantial savings from prognostic enrichment (Fig 2E and 2F).

Discussion

In this work we demonstrated a comprehensive framework for evaluating a biomarker for prognostic enrichment of a clinical trial with a survival endpoint. In both Examples 1 and 2, total trial costs are nearly monotone decreasing with greater levels of enrichment, but this will not always be true. For example, one can use the data simulator with the following specifications: $100 screening cost, $1000 cost per patient in the trial, 50% survival at 10 years and a trial planned for 5 years for 90% power to detect a treatment hazard ratio of 0.8. If the biomarker is highly prognostic (effect size 2.0), the total trial cost is U-shaped with a minimum at about the 75% enrichment level (that is, the trial only enrolls patients in the highest quartile of the biomarker). See S1 Fig and S1 Table. On the other hand, If the biomarker is weakly prognostic (e.g., effect size 1.2), total cost is monotone increasing with the level of enrichment. See S2 Fig and S2 Table. That is, at a 1:10 ratio of per patient screening and trial costs, it is not cost-effective to use prognostic enrichment at any level with the weakly prognostic biomarker.

Interestingly, the number of patients screened can be either an increasing or decreasing function of the level of enrichment. In both Examples 1 and 2 it was increasing. However, for highly prognostic biomarkers, the number of patients screened can be decreasing because the trial sample size drops precipitously and more than compensates for the additional patients who must be screened for an enriched trial (see [7] for examples).

The BioPETsurv data simulator requires specification of the biomarker distribution, the biomarker hazard ratio, the trend in hazards over time (increasing, constant, or decreasing), and the event rate without enrichment. These elements are realistic for area specialists to identify. The simulation methodology incorporates proportional hazards. As with any data simulation, results will be accurate only to the extent that the assumptions of the simulation hold.

When considering a prognostic enrichment strategy, investigators must consider multiple, sometimes conflicting goals: trial sample size, number of patients to screen for eligibility, and cost. BioPETsurv is useful for several types of questions in this arena. First, investigators with data on a prognostic biomarker can use BioPETsurv to evaluate that biomarker for its utility for prognostic enrichment for their clinical setting. Second, investigators with a prognostic biomarker who are planning a trial can use BioPETsurv to decide whether, and to what extent, to use the biomarker to plan and implement an enriched trial. Third, investigators working in a particular clinical setting can use BioPETsurv's simulation functionality to explore the prognostic capacity that would be needed in order for a biomarker to be useful for prognostic enrichment; results then inform biomarker research and development in that area [22-25]. BioPETsurv uses metrics that align with key trial elements. Together with important considerations around generalizability and ethics, BioPETsurv facilitates a comprehensive evaluation of competing dimensions in trial planning and the evaluation of prognostic enrichment biomarkers.

(TIF)

Click here for additional data file.

(TIF)

Click here for additional data file.

(DOCX)

Click here for additional data file.

(DOCX)

Click here for additional data file.

19 Jun 2020

PONE-D-20-12411

BioPETsurv:  Methodology and open source softwre to evaluate biomarkers for prognostic enrichment of time-to-event clinical trials

PLOS ONE

Dear Dr. Parikh,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 02 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Qin Liu

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Competing Interests section:

"SGC and CRP are members of the advisory board of RenalytixAI and own equity in the same. In the past 3 years, SGC has received consulting fees from Goldfinch Bio, CHF Solutions, Quark Biopharma, Janssen Pharmaceuticals, Takeda Pharmaceuticals, and Relypsa."

We note that one or more of the authors are employed by a commercial company: RenalytixAI, Goldfinch Bio, CHF Solutions, Quark Biopharma, Janssen Pharmaceuticals, Takeda Pharmaceuticals, and Relypsa.

2.1. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

2.2. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

3. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Please see attached.

Please see attached.

Please see attached.

Please see attached.

Please see attached.

Please see attached.

Please see attached.

Please see attached.

Please see attached.

Please see attached.

Reviewer #2: The study introduced a methodology and software called BioPETsurv. The work allows the end-users to plan for a trial with prognostic enrichment, in which the primary endpoint is time-to-event. Through simulation, some key operating characteristics, such as the number of patients to screen for eligibility and trial cost, can be generated to assist the trial planning. The free software in an R package: BioPETsurv will be helpful for many end-users. However, the overall scientific presentation should be improved to assist in comprehending. A few suggestions are listed below

1. Introduction

a. Line 46-56: Elaborate more about the history, concept, and definition of prognostic enrichment or a prognostically enriched trail by providing real-life examples.

b. Line 57-60: Elaborate what the predictive biomarker is and add references; why evaluating biomarkers for prognostic enrichment is of importance; When you say ‘sensible analyses,’ do you mean ‘sensitivity analyses’? If not, please clarify.

c. Line 61-64: Elaborate more about why those considerations would be meaningful in trial designs.

d. Line 71-76: for the motivating example, please provide more clinical background and design considerations in the trial (reference #8). Did the trial stratify by predictive biomarker? How did a prognostic enrichment design implement? For what purpose?

e. Line 82-83: BioPETsurv is a tool to help design a trial that aims to evaluate a predictive biomarker with a time-to-event primary endpoint, or BioPETsurv itself can help evaluate biomarkers?

f. Line 88-90: Should the biomarker here be a prognostics biomarker or a predictive biomarker? The two definitions are different, and there are many works of literature to delineate them.

2. Method section

a. Line 93-95: consider revising as ‘For a binary outcome, the area under the curve (AUC) by the ROC analysis summarizes the performance of discrimination of a biomarker.’

b. Line 95-97: KM curves for the time-to-event outcome are not analogous to the ROC curve for a binary outcome. However, for a time-to-event outcome, the researcher can generate time-specific AUC (PMID: 28388943)

c. Line 98: what is the definition of the level of enrichment?

d. Line 105: Not sure what the sentence means? Please revise.

e. Line 112: should it be ‘time-specific event rate’?

f. Line 121: what is the definition of the treatment hazard ratio? The HR for treated arm vs. control arm with enrichment level = 0? Or is it the HR corresponding to one standard deviation difference in the biomarker?

g. Line 129: what is the interpretation of the function of SD?

h. Line 146: in practice, how will time T be decided?

i. Line149-150: the sentence (‘but we have seen that ….’) is hard to understand, please revise.

3. Results section

a. It will be helpful to illustrate the example in the context of motivating case study (reference #8). What is the objective/purpose of design to including enrichment consideration? Why were the parameters of simulation set up at a specific value? What will preliminary data be essential for the simulation to be useful?

b. Figures 1 and 2, it is helpful to show the pattern change by the level of enrichment, but the clinical trial sample size and the number of patients screened could be too high to be realistic.

c. In a real trial design, how to reasonably determine those parameters for simulation?

d. I hope these tutorial examples can help future users to design their next trial.

4. Discussion Section

a. What is the novelty of the proposed study? Under what circumstance, the end-users should consider using the proposed approach.

b. Any other similar studies, if yes, what are the advantages and disadvantages of the current one.

c. Any other considerations the end-users should be aware of in order to generate the simulation that closely mimics the reality.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Jiangtao Gou

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PONE-D-20-12411-report.pdf

Click here for additional data file.

10 Jul 2020

Reviewer #1

Authors developed an R package BioPETsurv (released in January 2020) and an R shiny app available at http://prognosticenrichment.com/surv/ as a biomarker prognostic enrichment tool for survival analysis. The R package BioPETsurv is an extension of another package BioPET (released in July 2018). With these tools, clinical trialists can develop a protocol on a trial with prognostic enrichment faster and more effectively. This package also allows investigators to explore prognostic enrichment with simulated data. The total cost is computed as a linear combination of the cost for a patient in the trial and that of screening. The sample size estimations for fixed-duration trials and trials with an accrual period and a follow-up period follow the formulas in Schoenfeld (1983, Biometrics), Sample-size formula for the proportional-hazards regression model. This manuscript is well-structured and well written, and the documents for the R package and shiny app are also well-written. I only have a few comments below.

Comments

1. Under the settings in Table 1 and 2, the Reduction in Total Cost is generally an increasing function of the Screening Threshold. Authors can add additional situation where the Reduction in Total Cost is not a monotonic function of the Screening Threshold. For example, with effect size 2, Cost of screening a patient to determine trial eligibility 100 and Cost of running a patient through the trial 100, the maximum Reduction in Total Cost is achieved around Screening

Threshold = 75%.

Response: Thank you. We have added this example to the Discussion, where we have also added additional material of the different types of results one can see with trial metrics. In the interest of keeping our paper succinct, encouraging readers to explore our webtool, and to avoid repetition with material in Kerr et al (Clinical Trials, 2017), we did not add figures and tables for these examples to this paper. Readers can generate results themselves using our webtool. However, we would be happy to extend our paper to add more extensive results on additional examples at the direction of the editors.

2. Authors can discuss the general relation between the Reduction in Total Cost and the Screening Threshold. For example, when the relation is monotonic, when it is not.

Response: Thank you. As mentioned above, we have extended the Discussion section to describe the behavior of trial metrics.

3. The resolution of Figure 1 and 2 is quite low. I guess the reason is that the ggplot figures were first imbedded into MS Word and then were converted to pdf. Please make sure to have high resolution images for the final submission.

Response: Thank you. We have uploaded higher resolution figures with the revision.

4. There is a typo on the shiny app: “eligiblity” should be spelled as “eligibility”.

Response: Thank you very much for alerting us of this typo, which we have corrected.

Reviewer #2: The study introduced a methodology and software called BioPETsurv. The work allows the end-users to plan for a trial with prognostic enrichment, in which the primary endpoint is time-to-event. Through simulation, some key operating characteristics, such as the number of patients to screen for eligibility and trial cost, can be generated to assist the trial planning. The free software in an R package: BioPETsurv will be helpful for many end-users. However, the overall scientific presentation should be improved to assist in comprehending. A few suggestions are listed below

1. Introduction

a. Line 46-56: Elaborate more about the history, concept, and definition of prognostic enrichment or a prognostically enriched trail by providing real-life examples.

Response: Thank you. In the revised article we describe the CONSENSUS trial, which Temple (2010) considers a classic example of a trial using prognostic enrichment, and two more recent examples from nephrology.

b. Line 57-60: Elaborate what the predictive biomarker is and add references; why evaluating biomarkers for prognostic enrichment is of importance; When you say ‘sensible analyses,’ do you mean ‘sensitivity analyses’? If not, please clarify.

Response: Thank you. We have elaborated on that predictive biomarkers are useful for treatment selection because, by definition, they predict treatment benefit. We added 9 references on predictive biomarkers.

We meant "sensible analyses" and not "sensitivity analyses" – in using this term we intended to acknowledge that some of the ideas presented in our paper have appeared elsewhere. Our methodology brings these principles together in a unified, comprehensive framework, and our open source software facilitates the systematic application of these principles in practice. For clarity we have removed the term "sensible analyses" and re-written the relevant section of the Introduction.

c. Line 61-64: Elaborate more about why those considerations would be meaningful in trial designs.

Response: We believe most readers of this article will have be familiar with the challenges and constraints in designing a clinical trial. Due to the expense and difficulty of recruiting patients to a trial as well as the ethical preference to experiment on fewer rather than more patients, we believe it is clear that trialists are keenly interested in minimizing the sample size needed for a trial they are planning. In the interest of brevity, we have not expanded this sentence as the reviewer suggests, but would be happy to do so if directed by the editor.

d. Line 71-76: for the motivating example, please provide more clinical background and design considerations in the trial (reference #8). Did the trial stratify by predictive biomarker? How did a prognostic enrichment design implement? For what purpose?

Response: The reference (reference #8 in the first submission) is an observational study. There is general interest in trials of interventions to help prevent chronic kidney disease (CKD), but CKD is sufficiently uncommon such that large sample sizes would be required to run an adequately powered trial. Therefore, this is an example where a prognostic enrichment strategy could make a clinical trial more efficient and less expensive by testing treatment efficacy using fewer trial subjects (which is ethically preferable in itself). The reference does not describe an actual trial so we are unable to make the changes that the reviewer describes. In the revision, we have added the word "hypothetical" to make clear that we do not describe an actual trial.

e. Line 82-83: BioPETsurv is a tool to help design a trial that aims to evaluate a predictive biomarker with a time-to-event primary endpoint, or BioPETsurv itself can help evaluate biomarkers?

Response: Thank you. The reviewer is correct that BioPETsurv can be used for different precise purposes:

1. Investigators with data on a prognostic biomarker can use BioPETsurv to evaluate that biomarker for its utility for prognostic enrichment for their clinical setting.

2. Investigators with a prognostic biomarker who are planning a trial can use BioPETsurv to decide whether, and to what extent, to use the biomarker to plan and run an enriched trial.

3. Investigators working in a particular clinical setting can ask the question: what prognostic capacity would I need in order for a biomarker to be useful for prognostic enrichment? BioPETsurv's simulation functionality allows investigators to explore this question.

The Discussion of the revised submission is extensively revised, and better identifies and delineates these related, but distinct, goals. We are grateful to the reviewer for noting the ambiguity in our original submission.

f. Line 88-90: Should the biomarker here be a prognostics biomarker or a predictive biomarker? The two definitions are different, and there are many works of literature to delineate them.

Response: We have changed "predictors" to "prognostic factors" in this sentence to avoid confusion between prognostic and predictive biomarkers.

2. Method section

a. Line 93-95: consider revising as ‘For a binary outcome, the area under the curve (AUC) by the ROC analysis summarizes the performance of discrimination of a biomarker.’

Response: We have revised the sentence similar to the reviewer's suggestion.

b. Line 95-97: KM curves for the time-to-event outcome are not analogous to the ROC curve for a binary outcome. However, for a time-to-event outcome, the researcher can generate time-specific AUC (PMID: 28388943)

Response: Thank you. We agree with the reviewer that KM curves are not truly analogous to ROC curves. However, KM curves are able to communicate the prognostic capacity of a biomarker across a continuous range of time points. We have revised this sentence to remove the misleading statement that KM curves are analogous to ROC curves.

c. Line 98: what is the definition of the level of enrichment?

Response: Thank you. We have added the following sentences to make this clear: " A continuous biomarker can, in principle, be used for a low or high level of enrichment of a trial. The level of enrichment is the threshold (percentile in the biomarker) above which patients are eligible for the trial. Excluding patients from the trial below the 10th percentile in the biomarker would be a low level of enrichment; requiring patients in the trial to be in the top quartile would be a high level of enrichment."

d. Line 105: Not sure what the sentence means? Please revise.

Response: Thank you. This sentence refers to the "total number of patients screened to enroll the trial", which is described in the immediately preceding sentence. We have revised the confusing sentence to state: "Under the assumption that patients express interest in enrolling in the trial at a constant rate over time, `total number of patients screened' is a proxy for the calendar time to enroll the trial".

e. Line 112: should it be ‘time-specific event rate’?

Response: Yes, thank you. We have made this revision.

f. Line 121: what is the definition of the treatment hazard ratio? The HR for treated arm vs. control arm with enrichment level = 0? Or is it the HR corresponding to one standard deviation difference in the biomarker?

Response: The treatment hazard ratio (HR) is the treated arm vs. control arm. Since the biomarker is prognostic and not predictive, the HR is the same regardless of enrichment level – it is unnecessary to specify enrichment level = 0.

g. Line 129: what is the interpretation of the function of SD?

Response: It is the standard deviation. We have clarified this in the text.

h. Line 146: in practice, how will time T be decided?

Response: In practice, T will be decided according to the application. For example, if a treatment is intended to prevent relapse for cancer patients in remission, T will be shorter for cancers where relapse normally occurs soon after remission and T will be longer for cancers where relapse is typically more delayed.

i. Line149-150: the sentence (‘but we have seen that ….’) is hard to understand, please revise.

Response: Thank you. The paper now states: " The data simulator takes p as the survival probability for individuals with mean biomarker level. The survival curve for this group is similar to the overall survival curve.".

3. Results section

a. It will be helpful to illustrate the example in the context of motivating case study (reference #8). What is the objective/purpose of design to including enrichment consideration? Why were the parameters of simulation set up at a specific value? What will preliminary data be essential for the simulation to be useful?

Response: Please note that Example 1 does not illustrate the BioPETsurv data simulator. Because of proprietary data issues we were not able to analyze the data from the actual study and instead generated (outside of BioPET) synthetic data similar to the real data.

b. Figures 1 and 2, it is helpful to show the pattern change by the level of enrichment, but the clinical trial sample size and the number of patients screened could be too high to be realistic.

Response: We agree there is no guarantee a prognostic enrichment biomarker will make trial sample size and number of patients to screen realistic. This is precisely why it is so important to assess a biomarker for its capacity to be useful for prognostic enrichment. In Example 1, it is conceivable that a 7,000 patient trial (36-month trial with no enrichment) would be unrealistic but

a 4,000 patient trial (36-month trial with 75% enrichment) would be realistic if enough patients were available for such a high level of enrichment.

c. In a real trial design, how to reasonably determine those parameters for simulation?

Response: In developing BioPETsurv, we carefully considered the simulation parameters and believe these are parameters investigators will be comfortable specifying for the disease areas in which they specialize. For example, in cancer 5-year survival rates are routinely reported, which naturally translates to a survival rate and a time horizon to specify for the simulator.

d. I hope these tutorial examples can help future users to design their next trial.

Response: Thank you.

4. Discussion Section

a. What is the novelty of the proposed study? Under what circumstance, the end-users should consider using the proposed approach.

Response: We have revised the Discussion extensively to more clearly describe the circumstances in which BioPETsurv will be useful.

b. Any other similar studies, if yes, what are the advantages and disadvantages of the current one.

Response: We have not been able to identify any other publications (other than the original BioPET paper) that presents a similar, comprehensive framework for prognostic enrichment biomarkers.

c. Any other considerations the end-users should be aware of in order to generate the simulation that closely mimics the reality.

Response: We are uncertain what the reviewer has in mind here. We believe the simulation inputs and the BioPET outputs are adequately described for the audience of this paper. In addition, we include a paragraph in the Discussion that summarizes the required inputs and the assumptions of the simulator.

Submitted filename: ResponseToReviewers20200704.docx

Click here for additional data file.

18 Aug 2020

PONE-D-20-12411R1

BioPETsurv: Methodology and open source software to evaluate biomarkers for prognostic enrichment of time-to-event clinical trials

PLOS ONE

Dear Dr. Parikh,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please consider adding illustrations in supplement about additional situation where the Reduction in Total Cost is not a monotonic function of the Screening Threshold.

Please submit your revised manuscript by Oct 02 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Qin Liu

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Please see attached.

Limit 100 to 20000 Characters

Limit 100 to 20000 Characters

Limit 100 to 20000 Characters

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Jiangtao Gou

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PONE-D-20-12411_R1-report.pdf

Click here for additional data file.

2 Sep 2020

Reviewers' comments

Reviewer #1 and #2 agreed that our article is technically sound piece of research with data that support the conclusions, and that our analysis is appropriate and rigorous. Reviewer #1 had one additional comment:

Reviewer #1: Authors have edited this manuscript successfully. I only have one comment.

Authors have added additional descriptions in Discussion for the previous comment: “Under the settings in Table 1 and 2, the Reduction in Total Cost is generally an increasing function of the Screening Threshold. Authors can add additional situation where the Reduction in Total Cost is not a monotonic function of the Screening Threshold. For example, with effect size 2, Cost of screening a patient to determine trial eligibility 100 and Cost of running a patient through the trial 100, the maximum Reduction in Total Cost is achieved around Screening Threshold = 75%.” In order to keep the manuscript succinct, authors decided not to include additional illustrations. However, readers will be benefited from these additional illustrations. Therefore, authors may include these part into supplement material.

Response: Our revised submission includes Supplementary Tables S1 and S2 and Supplementary Figures S1 and S2 to illustrate these additional situations.

Submitted filename: ReviewResponse.docx

Click here for additional data file.

8 Sep 2020

BioPETsurv: Methodology and open source software to evaluate biomarkers for prognostic enrichment of time-to-event clinical trials

PONE-D-20-12411R2

Dear Dr. Parikh,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Qin Liu

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Please see attached.

Please see attached.

Please see attached.

Please see attached.

Please see attached.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Jiangtao Gou

Submitted filename: PONE-D-20-12411_R2-report.pdf

Click here for additional data file.

10 Sep 2020

PONE-D-20-12411R2

BioPETsurv:  Methodology and open source software to evaluate biomarkers for prognostic enrichment of time-to-event clinical trials

Dear Dr. Parikh:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Qin Liu

Academic Editor

PLOS ONE