Mapping expanded prostate cancer index composite to EQ5D utilities to inform economic evaluations in prostate cancer: Secondary analysis of NRG/RTOG 0415.

PURPOSE: The Expanded Prostate Cancer Index Composite (EPIC) is the most commonly used patient reported outcome (PRO) tool in prostate cancer (PC) clinical trials, but health utilities associated with the different health states assessed with this tool are unknown, limiting our ability to perform cost-utility analyses. This study aimed to map EPIC tool to EuroQoL-5D-3L (EQ5D) to generate EQ5D health utilities. METHODS AND MATERIALS: This is a secondary analysis of a prospective, randomized non-inferiority clinical trial, conducted between 04/2006 and 12/2009 at cancer centers across the United States, Canada, and Switzerland. Eligible patients included men >18 years with a known diagnosis of low-risk PC. Patient HRQoL data were collected using EPIC and health utilities were obtained using EQ5D. Data were divided into an estimation sample (n = 765, 70%) and a validation sample (n = 327, 30%). The mapping algorithms that capture the relationship between the instruments were estimated using ordinary least squares (OLS), Tobit, and two-part models. Five-fold cross-validation (in-sample) was used to compare the predictive performance of the estimated models. Final models were selected based on root mean square error (RMSE).
RESULTS: A total of 565 patients in the estimation sample had complete information on both EPIC and EQ5D questionnaires at baseline. Mean observed EQ5D utility was 0.90±0.13 (range: 0.28-1) with 55% of patients in full health. OLS models outperformed their counterpart Tobit and two-part models for all pre-determined model specifications. The best model fit was: "EQ5D utility = 0.248541 + 0.000748*(Urinary Function) + 0.001134*(Urinary Bother) + 0.000968*(Hormonal Function) + 0.004404*(Hormonal Bother)- 0.376487*(Zubrod) + 0.003562*(Urinary Function*Zubrod)"; RMSE was 0.10462.
CONCLUSIONS: This is the first study to identify a comprehensive set of mapping algorithms to generate EQ5D utilities from EPIC domain/ sub-domain scores. The study results will help estimate quality-adjusted life-years in PC economic evaluations.

Introduction

Treatment of localized prostate cancer (PC) continues to be a major focus of public health policy debate. Patients can choose from a wide range of management options, ranging from radical prostatectomy, radiation therapy, or active surveillance [1, 2]. Survival rates do not differ significantly between the different approaches, making treatment decision-making a complex and individualized process [3, 4].

Given the high global burden of PC, there have been calls for cost-effectiveness evaluations to better understand the economic implications of PC management. Cost-effectiveness analyses (CEAs) allow for the comparison of alternative treatment options in terms of incremental costs relative to quality-adjusted life-years (QALY) gained following treatment [5]. However, such evaluations are highly dependent on our ability to not only accurately model probabilities of experiencing cancer recurrence, overall survival, and treatment side effects over time, but also our ability to accurately calculate ‘utility’ values associated with the range of health states that can be experienced by a patient following PC treatment. Utility values are a measure of how patients view the overall quality of their life, with ‘0’ (corresponding to death) to ‘1’ (corresponding to perfect health) [6]. The results of previous PC CEAs have been sensitive to the utility values attached to health states captured in the trials informing them, underscoring the need for reliable and valid utilities [7, 8].

Utilities necessary for economic evaluations can be directly elicited in trials through use of a preference-based measure (PBM) [5, 9]. However, many trials do not collect a PBM, and instead include one or more patient-reported outcome measures (PROMs), which do not have established utility values. For example, the Expanded Prostate Cancer Index Composite (EPIC), one of the most commonly used PRO tools in prostate cancer clinical trials (including a pivotal trial comparing surgery to radiation and active surveillance [10, 11], as well as an ongoing study comparing protons to photons [12]) does not have associated utility values.

Utility mapping involves development and use of a statistical model or algorithm that links the outcomes from a PROM and a PBM to generate health utility values [5, 13–15]. Although clinical trials now often incorporate health utility estimation in their design, studies conducted in the past remain part of the evidence base as comparators for the evaluation of new technologies and have not always included a PBM [16-18]. Therefore, when utility information is not collected in a study, mapping has been proposed as an alternative solution and recommended as the second-best option after direct utility estimation for economic evaluations of interventions. The objective of this study is to map EPIC to health utilities that can be applied to future PC CEAs.

Methods

This mapping study followed methodological guidance issued by National Institute for Health and Care Excellence (NICE), and reporting standards guidance outlined in the 2015 MAPS (MApping onto Preference-based measures reporting Standards) statement and 2017 International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force Report [13–15, 19]. A battery of regression model specifications were tested to identify a set of mapping algorithms with and without demographic and clinical covariates.

Data source

The data for this study came from a previously published international multicenter, open-label randomized clinical trial (RCT) of patients with low-risk PC. This trial used a non-inferiority design to determine whether the efficacy of a hypo-fractionated treatment schedule was not worse than a conventional schedule in men with low-risk PC. The results of this trial showed no significant differences in outcomes between the two treatment modalities. Bruner et al examined the HRQoL outcomes in this trial and reported no clinically significant between-arm differences in EPIC domain scores and EQ-5D index and VAS scores through 5 years following the completion of radiation [20]. This data source was chosen for our mapping study as it collects data on both HRQoL measures of interest in PC patients undergoing treatment.

The Institutional Review Board approval was sought and received from the University Of Maryland School Of Medicine and NRG Oncology.

Sample selection

The study sample consisted of patients who had complete information on both EPIC and EQ5D at baseline. A 70% random sample was extracted from the 1,092 analyzable patients from the trial to create the estimation cohort and the remainder 30% sample was used as a validation cohort, to predict the performance of the estimated mapping algorithms. In addition to the HRQoL data, demographic characteristics and clinical covariates were also extracted.

Outcome measures

EuroQol-5D-3L

The EQ5D questionnaire is a generic PBM, recommended by NICE for use in economic evaluations and asks respondents to describe their health in five dimensions (mobility, self-care, usual activities, pain/ discomfort, and anxiety/ depression), each of which can be at one of three severity levels (1: no problems/ 2: some or moderate problems/ 3: extreme problems) [14, 15]. Two hundred forty-three combinations can be described in this way (35 combinations). Additionally, health states corresponding to unconsciousness and immediate death are also included in the valuation process [21]. The EQ-5D tariffs for our study were obtained using the US valuation of EQ-5D health states performed by Shaw et al. in a sample of 4,048 civilian noninstitutionalized English- and Spanish-speaking adults, aged 18 and older, who resided in the United States (50 states plus the District of Columbia) in 2002 [22].

Expanded Prostate Cancer Index Composite (EPIC)

EPIC is a comprehensive instrument designed to evaluate patient function and bother after PC treatment [3]. EPIC has been validated in men with localized PC who underwent surgery, external beam radiation, or brachytherapy with or without hormonal adjuvants. EPIC is sensitive to specific HRQoL effects of these therapies and to HRQoL effects of cancer progression [3, 23]. EPIC assesses the disease-specific aspects of PC and its therapies and is comprised of four summary domains (Urinary, Bowel, Sexual and Hormonal). In addition, each Domain Summary Score has measurable Function Subscale and Bother Subscale components. Response options for each EPIC item form a Likert scale and multi-item scale scores are transformed linearly to a 0–100 scale, with higher scores representing better HRQoL [3].

Conceptual overlap

Pearson’s correlation coefficients were used to determine the degree of conceptual overlap between EPIC domain and sub-domain scores and EQ5D index score [24, 25].

Model development

Linear regression is the most common approach to derive mapping function [13-15]. To account for the anticipated bimodal distribution of EQ5D for our study population, other functional forms were also explored [26]. Specifically, Tobit and two-part models were estimated to account for a significant proportion of patients in full health. The Tobit model assumes that the EQ5D utility data is censored at 1 and that the true value has a normal distribution whose mean is given by a linear combination of the covariates. Two-part models model the probability of being in full health using a logistic regression, and then model the remainder of the distribution using a OLS regression model [27].

For each of the functional forms, multiple model specifications were estimated (S1 Table). Separate sets of models with EPIC domains (group 1), EPIC sub-domains (group 2), EPIC domains with demographic characteristics (group 3), EPIC sub-domains with demographic characteristics (group 4), EPIC domains with demographic characteristics and clinical covariates (group 5), and finally, EPIC sub-domains with demographic characteristics and clinical covariates (group 6) were chosen to accommodate different possible combinations of variables in EPIC datasets available to researchers. Higher second and third order polynomials for domain scores, subdomain scores, and age were explored to examine non-linear relationships; interaction terms for race and Zubrod performance status were also explored. No further covariates were explored in an effort to be able to use the mapping algorithms in a wide range of datasets. Along with the full models specified in S1 Table, reduced models were also estimated using stepwise selection (forward selection; significance level of 0.25 required for entry and to remain in the model) in order to identify parsimonious models with high predictive ability.

Assessing model performance

The 70% random sample (n = 765) was used for estimation and internal validation of the mapping algorithms. Five-fold cross-validation was employed for estimation and internal validation [28, 29]. In 5-fold cross-validation, the data are split into 5 equal parts and the model is fitted on 4 parts with the 5th being held out for validation. The fitted model of the 4 selected parts is used to compute the predicted residual sum of squares on the 5th omitted part, and this process is repeated for each of the 5 parts. The sum of the 5 predicted residual sums of squares is obtained for each fitted model and is the estimate of the prediction error. Indices such as the absolute mean of the residuals or errors (MAE), and square root of the mean of the residual sum of squares (RMSE) are used to determine model performance. RMSE, a measure of individual prediction error, attaches relatively higher weights to large errors, making it an ideal metric when large errors are undesirable. This study used RMSE for identifying the candidate algorithms from each of the six groups of model specifications in S1 Table. Models with lower RMSE values represent higher predictive ability. A prediction model usually performs better with the data that were used in its development. Therefore, it is critical to evaluate how well the model works in other datasets. In absence of an external dataset, validation was performed by scoring the remaining 30% random sample (n = 327) using the candidate algorithms identified using the 5-fold cross validation in the 70% estimation sample.

Results

Descriptive statistics

The study cohort comprised of patients who consented to QOL collection and had complete baseline data on EPIC domains/subdomains as well as EQ5D dimensions. For models with EPIC domains as the primary independent variables, 565 patients in the 70% estimation sample and 232 patients in the 30% validation sample consented and had complete baseline data on EPIC domains and EQ5D. For models with EPIC sub-domains as the primary independent variables, 507 patients in the 70% estimation sample and 213 patients in the 30% validation sample consented and had complete baseline EPIC sub-domain data and EQ5D. Patient characteristics for each of these cohorts are summarized in Table 1. EQ5D distribution was highly skewed with >50% patients in full health in each cohort; distribution plots revealed a bimodal distribution peaking at full health and at health utility value of 0.8 (S1 Fig). Table 2 summarizes the mean EPIC domain/ sub-domain scores in the estimation cohort and validation cohort.

Table 1

Baseline characteristics of patients with complete EPIC domain and subdomain data.

Characteristic	Complete EPIC domain data		Complete EPIC sub-domain data
	Estimation Cohort (n = 565)	Validation Cohort (n = 232)	Estimation Cohort (n = 507)	Validation Cohort (n = 213)
Continuous Variables (mean ± SD)
Age	66.4±7.3	66.2±7.7	66.4±7.2	66.2±7.8
Baseline PSA	5.6±2.1	5.5±2.2	5.5±2.1	5.5±2.2
Categorical Variables (n (%))
Baseline PSA
<4	115 (20.3)	45 (19.4)	104 (20.5)	42 (19.7)
≥4	450 (79.7)	187 (80.6)	403 (79.5)	171 (80.3)
Race
White	466 (82.5)	179 (77.2)	421 (83.0)	163 (76.5)
Other	99 (17.5)	53 (22.8)	86 (17.0)	50 (23.5)
Zubrod
0	530 (93.8)	211 (90.9)	477 (94.1)	195 (91.5)
1	35 (6.2)	21 (9.1)	30 (5.9)	18 (8.5)
EQ5D
1	310 (54.9)	120 (51.7)	284 (56.0)	114 (53.5)
<1
	255 (45.1)	112 (48.3)	223 (44.0)	99 (46.5)

Table 2

EPIC domain and sub-domain scores and EQ5D scores at all study time-points.

Characteristic	Score (Mean±SD)
EPIC domains	Estimation Cohort (n = 565)	Validation Cohort (n = 232)
Urinary	87.5±12.1	86.5±12.5
Bowel	93.4±9.3	92.7±9.2
Sexual	49.6±26.3	50.4±26.6
Hormonal	91.0±11.0	90.5±11.8
EQ5D	0.9±0.1	0.9±0.1
EQ5D –median (IQR)	1 (0.83, 1)	1 (0.82, 1)
EPIC sub-domains	Estimation Cohort (n = 507)	Validation Cohort (n = 213)
Urinary Function	93.3±10.7	92.9±11.8
Urinary Bother	84.0±14.8	82.7±14.8
Urinary Irritation	86.8±12.6	85.6±12.1
Urinary Incontinence	91.6±14.0	91.3±14.8
Bowel Function	93.2±8.5	92.3±9.4
Bowel Bother	94.6±9.6	93.4±10.7
Sexual Function	43.7±26.9	45.1±27.5
Sexual Bother	64.0±32.9	64.9±32.4
Hormonal Function	88.7±13.6	88.7±13.5
Hormonal Bother	93.0±10.3	92.0±10.3
EQ5D	0.9±0.1	0.9±0.1
EQ5D –median (IQR)	1 (0.83, 1)	1 (0.83, 1)

IQR = Inter-Quartile Range

Pearson’s correlations between EQ5D and EPIC domains/ sub-domains showed evidence of conceptual overlap between the two measures. In the estimation cohort for models with EPIC domains, moderate correlations were found between EQ5D utility and urinary (r = 0.38), bowel (r = 0.34) and hormonal (r = 0.55) domains of EPIC; sexual domain was weakly correlated (r = 0.18) with EQ5D utility. In the estimation cohort for models with EPIC sub-domains, low to moderate correlations were found between EQ5D and urinary function (r = 0.31), urinary bother (r = 0.36), urinary irritation (r = 0.36), urinary incontinence (r = 0.27), bowel function (r = 0.30), bowel bother (r = 0.32), hormonal function (r = 0.43), hormonal bother (r = 0.53), sexual function (r = 0.17), and sexual bother (r = 0.16).

Mapping EPIC to EQ5D utilities

OLS, Tobit, and two-part models were estimated for all the model specifications in S1 Table, resulting in 144 unique full regression models. The best performing models for each of these regression types across the six groups of independent variables are presented in Table 3.

Table 3

Performance of full models in internal (5-fold cross-validation) and validation sets.

#	Model Specifications		EQ5D Index Scores			RMSE		Overall Rank
	Available Data	Regression Model	Mean ± SD	Minimum	Maximum	5-Fold Cross-Validation	Validation
	Actual EQ5D Data	-	0.90±0.13	0.28	1.00	-		-
1	EPIC Domains	OLS (1a)	0.90±0.08	0.51	0.99	0.10819	0.122668	9
		Tobit (1b)	0.95±0.09	0.37	1.00	0.12476	-	17
		2-Part (1a)	0.90±0.08	0.55	0.98	0.11016	-	11
2	EPIC Sub-Domains	OLS (2c)	0.91±0.08	0.33	1.01	0.10450	0.113311	2
		Tobit (2b)	0.95±0.09	0.34	1.00	0.12395	-	14
		2-Part (2a)	0.91±0.08	0.44	0.98	0.10484	-	4
3	EPIC Domains, Age, Race	OLS (3d)	0.90±0.08	0.43	1.01	0.10818	0.124491	8
		Tobit (3j)	0.95±0.09	0.46	1.00	0.12447	-	16
		2-Part (3a)	0.90±0.08	0.54	0.99	0.11017	-	12
4	EPIC Sub-Domains, Age, Race	OLS (4j)	0.91±0.08	0.33	1.01	0.10456		3
		Tobit (4g)	0.95±0.10	0.27	1.00	0.12477	-	18
		2-Part (4a)	0.90±0.08	0.50	0.99	0.10801	-	6
5	EPIC Domains, Age, Race, Zubrod, PSA	OLS (5g)	0.90±0.08	0.35	0.99	0.10615	0.122175	5
		Tobit (5j)	0.94±0.09	0.39	1.00	0.12276	-	13
		2-Part (5a)	0.90±0.08	0.26	0.99	0.10838	-	10
6	EPIC Sub-Domains, Age, Race, Zubrod, PSA	OLS (6i)	0.91±0.08	0.36	0.99	0.10429	0.110482	1
		Tobit (6g)	0.95±0.10	0.33	1.00	0.12407	-	15
		2-Part (6a)	0.90±0.08	0.51	0.99	0.10814	-	7

The OLS models outperformed the other model types in all six model specification groups. The best performing full model was an OLS model with EPIC sub-domains, age, race, Zubrod performance status, and baseline PSA levels (model 6i) with an RMSE of 0.10429:

Predicted EQ5D = 2.922434 + 0.003627*Urinary Function + 0.004125*Urinary Bother – 0.003625*Urinary irritation – 0.002242*Urinary Incontinence – 0.0000058476*Bowel Function – 0.000690*Bowel Bother + 0.000589*Sexual Function – 0.000244*Sexual Bother + 0.000721*Hormonal Function + 0.004691*Hormonal Bother – 0.126445*Age + 0.001997*(Age)2 – 0.000010336*(Age)3 + 0.009922*Race(other) – 0.456669*Zubrod + 0.016593*Urinary Function*Zubrod + 0.008613*Urinary Bother*Zubrod – 0.011*Urinary Irritation*Zubrod – 0.011342*Urinary Incontinence*Zubrod + 0.000711*Bowel Function*Zubrod + 0.003675*Bowel Bother*Zubrod – 0.001631*Sexual Function*Zubrod + 0.00008517*Sexual Bother*Zubrod – 0.000201*Hormonal Function*Zubrod – 0.002221*Hormonal Bother*Zubrod + 0.000332*PSA(≥4)

Reduced models for all six model specification groups were estimated to identify parsimonious models with high predictive ability (Table 4). For the reduced models, only OLS functional form was tested as OLS full models outperformed other model types. The best performing reduced model had an RMSE of 0.10462:

Table 4

Performance of reduced models in internal (5-fold cross-validation) and validation sets.

#	Model Specifications		EQ5D Index Scores			RMSE		Overall Rank
	Available Data	Regression Model	Mean ± SD	Minimum	Maximum	5-Fold Cross-Validation	Validation
	Actual EQ5D Data	-	0.90±0.13	0.28	1.0	-	-	-
1	EPIC Domains	U H	0.90±0.08	0.51	0.98	0.10810	0.123367	5
2	EPIC Sub-Domains	UF UB HF HB	0.90±0.07	0.46	0.98	0.10631	0.113095	2
3	EPIC Domains, Age, Race	U H	0.90±0.08	0.51	0.98	0.10810	0.123367	6
4	EPIC Sub-Domains, Age, Race	UF UB HF HB	0.90±0.07	0.46	0.98	0.10631	0.113095	3
5	EPIC Domains, Age, Race, Zubrod, PSA	U H Zubrod U*Zubrod	0.90±0.08	0.40	0.98	0.10654	0.123662	4
6	EPIC Sub-Domains, Age, Race, Zubrod, PSA	UF UB HF HB Zubrod UF*Zubrod	0.90±0.08	0.37	0.97	0.10462	0.114714	1

Predicted EQ5D = 0.248541 + 0.000748*Urinary Function + 0.001134*Urinary Bother + 0.000968*Hormonal Function + 0.004404*Hormonal Bother – 0.376487 *Zubrod + 0.003562*Urinary Function*Zubrod

The candidate full and reduced models for the remaining specifications are presented in S2 Table. Validation using these candidate models resulted in slightly higher RMSE values compared to 5-fold cross-validation, but the results remained consistent with the 5-fold cross-validation (Tables 3 and 4). S2 Fig presents the plot of predicted vs. observed EQ5D utilities for the best performing models in each group. The EQ5D utilities appear to be under-predicted at higher health states and over-predicted for lower health states. However, the mean predicted EQ5D utilities were very similar to the observed EQ5D utilities.

Discussion

This study identified a set of algorithms that map EPIC, a disease-specific HRQoL instrument in PC, to EQ5D, a generic preference-based instrument, using data from a randomized clinical trial. While there is considerable variation in the methodologies of mapping studies, a majority have employed some form of direct mapping strategy [16]. This mapping study followed the guidance from NICE and ISPOR task force and explored several functional forms and specifications to find the most straightforward model with highest predictive performance [13-15].

Tobit and two-part models were tested as their assumptions were compatible with the bimodal distribution of EQ5D utilities. However, they were outperformed by their counterpart OLS models for every model specification tested. Previous mapping studies have reported similar findings, where OLS regression provided better predictive ability than theoretically more robust regression procedures [16, 30, 31]. Separate algorithms were estimated using EPIC domains or subdomains data alone, and in combination with demographic covariates only or both demographic and clinical covariates, resulting in six unique sets of model specifications. Best-performing models for each of these sets were identified, so that researchers can use a model depending on the level of data at their disposal, thus, increasing the generalizability of this mapping exercise. In addition to the full models, reduced models were also estimated to identify parsimonious models with high predictive ability. Addition of demographic variables did not improve the predictive ability of the models; however, clinical covariates, specifically Zubrod performance status, improved the predictive performance. This was observed in both full and reduced models, where addition of clinical covariates resulted in lower RMSE values. Generally, models with EPIC sub-domains exhibited better predictive performance compared to their counterpart models with EPIC domains.

There are several strengths of this study that are worth mentioning. To the best of our knowledge, this is the first study to map EPIC to obtain health utilities for patients with PC. Bremner at al. mapped Prostate Cancer Index (PCI) to Patient-Oriented Prostate Utility Scale (PORPUS-U) utilities to incorporate historically collected HRQoL data in longitudinal datasets such as CaPSURE in economic evaluations [7]. EPIC is a more comprehensive instrument that evolved from PCI and is the most widely used PC specific HRQoL instrument in trials and clinical practice [23]. The algorithms identified in this study will allow incorporation of a vast body of evidence on comparative effectiveness of PC treatments in future economic evaluations. EQ5D is the recommended PBM by HTA bodies such as NICE, and considerable differences exist, even between utilities derived from different generic PBMs. Inconsistencies in the choice of PBMs in mapping studies would make comparisons across treatments and disease areas difficult. Unlike Bremnen et al., EQ5D, a generic PBM, was chosen in order to make comparisons across disease areas possible.

Mapping algorithms perform best when the target population has characteristics similar to the source population. While the trial sample does not represent every PC patient, a large proportion of patients with PC fall in this category. Patients with low-risk PC, as in this sample, tend to have high performance status and high EQ5D scores with minimal variability which may differ substantially from high-risk patients. Thus, caution should be exercised in extrapolating these algorithms to patients with high-risk PC. Future analyses could build on this work and identify best performing models for patients with high-risk PC.

As with any mapping study, this study has some limitations that merit discussion. Validation of candidate models in the 30% sample resulted in slightly higher RMSE values than those observed in the estimation cohort. This was expected as prediction models usually perform better with the data that were used in its development. However, models with lower RMSE values in the 5-fold cross-validation also had lower RMSE values in the validation set, supporting the robust predictive performance of the candidate algorithms in external datasets. While the health utilities for milder health states were under-predicted and worse health states were over-predicted, the mean predicted utilities at the population level were very similar to the observed mean utilities. These mapping algorithms are best suited to predict mean utilities and may not predict individual level EQ5D utilities with high degree of accuracy. Finally, inclusion in the estimation sample required complete data on EPIC domains/sub-domains along with EQ5D utilities. While differences may exist between patients who completed the questionnaires versus those who did not, the objective of our regression models was prediction and not estimation, therefore, risk of bias is minimal with using this subset of patients. There is considerable heterogeneity in the data sources that have been used in mapping studies; future studies should compare the impact of these differences on the resulting algorithms.

In conclusion, HRQoL measures can be descriptive (generic, or condition-specific) or preference-based (health utility measures) [32, 33]. It is often not feasible to include all these types of instruments in a given study, as this can be a costly and time-consuming endeavor. These studies however, form an important part of the evidence base for the effectiveness of an intervention. Mapping EPIC to EQ5D utilities bridges an important outcomes gap, allowing incorporation of a vast body of literature measuring descriptive HRQoL data in PC patients in the healthcare decision-making process.

Distribution plot of EQ5D in the estimation cohorts.

A. Patients with Complete Epic Domain Data (N = 565). B. Patients with Complete Epic Sub-Domain Data (N = 507).

(ZIP)

Click here for additional data file.

Plot of observed vs. predicted EQ5D utilities for candidate full models.

(ZIP)

Click here for additional data file.

Bland-Altman plot for full candidate model using EPIC sub-domain data.

(TIF)

Click here for additional data file.

Model specifications.

(DOCX)

Click here for additional data file.

Candidate mapping algorithms and external validation results in the 30% sample.

(DOCX)

Click here for additional data file.

Baseline characteristics of patients with complete EPIC domain data.

(DOCX)

Click here for additional data file.

EQ5D and EPIC domain scores for patients with complete EPIC domain data.

(DOCX)

Click here for additional data file.

EPIC sub-domain scores for patients with complete EPIC sub-domain data.

(DOCX)

Click here for additional data file.

Baseline characteristics of patients included vs not included in complete EPIC sub-domain analysis.

(DOCX)

Click here for additional data file.

16 Dec 2020

PONE-D-20-34900

Mapping Expanded Prostate Cancer Index Composite to EQ5D Utilities to Inform Economic Evaluations in Prostate Cancer: Secondary Analysis of NRG/RTOG 0415

PLOS ONE

Dear Dr. Mishra,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jan 30 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Michael E. O'Callaghan

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Within the Methods section, please provide additional details the methodology used for the selection of the published international multicentred, open-label randomised clinical trials.

3. Thank you for stating the following in the Competing Interests section:

"Drs. Khairnar, Albert, Bentzen, Bruner, Chen, Currey, Dayes, DeNittis, Horwitz, Lee, Michalski, Mullins, Palumbo, Pisansky, Seaward, Shah, Shaya, and Villalonga have nothing to disclose. Dr. Feng reports personal fees from Janssen Oncology, Sanofi, Bayer, Celgene, and Blue Earth Diagnostics, grants from Zenith Epigenetics, and other from PFS Genomics, outside the submitted work; Dr. Malone reports personal fees from Sanofi, and honoraria from Amgen, Abbvie, Astellas, Janssen, Tersara, Astra Zeneca, Knight Therapeutics, and Bayer, outside the submitted work; Dr. Mishra reports grants from American Society of Radiation Oncology (ASTRO), during the conduct of the study and other from Varian Medical Systems, outside the submitted work; Dr. Sandler reports grants from ACR/NRG Oncology, during the conduct of the study; personal fees from Janssen, other from Radiogel, outside the submitted work; Dr. Pugh reports other from Millennium, other from Pfizer, outside the submitted work. "

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

4. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Mapping prostate cancer index

General comments:

* replace everywhere "external validation" by "test sample", this is the sample for internal validation.

Refer to the first sample as the "estimation sample", this is the equivalent to a training sample in data science also called the hold out sample

external validation implies the use of an independent external data sample which is not the case here

(see https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets)

* replace everywhere in the text EQ5D by EQ-5D-3L as this is the version of the PBM questionnaire

METHODS

Sample Selection :

* Shortly describe the sample and the original data plus QoL results of the original article

* correct the naming of the samples (see above)

Outcome measures :

* specify which value set (Tariff) was used to value the EQ-5D-3L data

* I am not sure what you mean by "not part of the descriptive system"; death is anchored at zero and states worse than death can take negative values up to a lower limit , depending on the country specific Tariff used .

Model Development :

* line 160: this is wrong; OLS assumption implies that the errors (conditional on the explanatory variables) is normally distributed. This allows inference of the coefficients and tests of significance based on Normal Theory.

*Put the detailed list of the model specifications in an appendix

* line 173: higher second and third order polynomials

* which stepwise variable selection was used ? It seems it was a simple forward selection method based on the p-value. what was the criterion used to include/retain a variable ?

(see Forward selection is a type of stepwise regression which begins with an empty model and adds in variables one by one. In each forward step, you add the one variable that gives the single best improvement to your model.Sep 19, 2017

* line 166 the exact two-step method should be more detailed

was logistic regression used for the full health, what was its accuracy ?

how was the goodness of fit of the combined parts estimated utilities then further assessed ?

Were there any U values = 1 or higher resulting form the OLS regression part ? how were these dealth with ?

Forward Selection: Definition - Statistics How To

www.statisticshowto.com › forward-selection

Stepwise regression - Wikipedia "en.wikipedia.org › wiki › Stepwise_regression"

see Variable Selection www.biostat.jhsph.edu › ~iruczins › teaching

and for a discussion and limitations of the different methods

Loann. D. Desboulets , A Review on Variable Selection in Regression Analysis Econometrics, 2018

The results of the forward regresson methods should be confrmed by another selection method, especially since the n/p (observations/parameters) ratio is rather low in the regression incorporating interactions and power variables.

A LASSO type or similar method would be useful in this situation. It can be applied to the full most detailed equation including all subscales and interactions/power variables.

Assessing Model Performance :

The choice of only the RMSE as accuracy criterion (Goodness of fit) is not to be recommended , it should be complemented by other criteria as well including MAE, estimated utility values >1 and <0, etc...

I would urge the authors to also present a Bland-Altman plot of the results of their best fitting model (with 95% confidence intervals and minimally clinically important limits as well for EQ-5D utilities +- 0.08)

Also the multiple comparison problem given the hughe number of regressions performed should be discussed/adressed .

Five-fold cross-validation was used , I guess this was on the test sample ? how were the regression results then combined ? give some more details about the exact procedure followed to allow replication of your methodology by others.

RESULTS

Descriptive Statistics :

* Include the results of a statistical comparison test of the variables between the different samples in table 2 and table 3 to assess their similarity of the samples

* given the highly bimodal nature of the observed utilities non-parametric summary measures (medians, IQR, etc..) and test statistics should be preferred added to the tables

Mapping results :

* show first the tables of performance and selection of the best fitting equations

then show the detailed equation of the best fitting one(s)

A likelihood ratio test should be performed to compare the reduced equation and the full equation of the predicted 5EQ-5D as these are nested. If the H0 of equality is not rejected (in the testing sample) then the full equation can be dropped

* present the regression coefficients with their 95% CI and present aso the variance-covariance matrix of the regression parameters

DISCUSSION

line 316 which generic PBM did Bremen used ? specify

how bad was the underprediction of low observed utilities ? where was the utility threshold ?

how bad was the overprediction for observed high utilities ? where was the utility threshold ?

What was the variance of the estimates compaed to the observed variance of utilities for different values of utilities (low, average ,high, perfect health) or per quartile?

line 338: your risk of bias is linked to whether the censoring and non-response to the QoL questionnaires was truly random otherwise there is a risk of "survival or response" bias. Nothing tells you that the non responders had the same mapping coefficients as those of the completers so this could potentially alter the regression coefficients.

Reviewer #2: This paper has examined three econometric models for estimating EuroQol- 5 Dimension (EQ-5D) utility scores from the Expanded Prostate Cancer Index Composite (EPIC) to calculate quality adjusted life years for cost-utility analysis. The paper uses robust methods that should act as an aid for utility estimation within future economic evaluations of interventions using the Expanded Prostate Cancer Index Composite in Prostate Cancer. As such, it has the potential to act as a beneficial addition to the mapping literature. This article is well written, and the authors have carefully followed standard mapping methodology.

Major comments:

1. Abstract Page 3, Line 51: The authors state that the lack of health utilities associated with the different health states assessed with the EPIC are unknown, therefore limiting the ability to perform cost-effectiveness evaluations. Can the authors edit this and use cost-utility analysis (CUA) and not cost-effectiveness analysis (CEA) as the form of economic evaluation which allows for the comparison of alternative treatment options in terms of incremental costs relative to quality-adjusted life-years (QALY) gained following treatment is a cost-utility analysis.

2. Abstract Page 3, Line 52: The authors use the term "utility weights". This term is used in valuation studies when generating population preference weights or scoring algorithms and not mapping algorithms. The authors should correct this and use utility scores or utilities instead.

3. Page 6: The authors present mapping as though the reader might already know what it is. Can the authors provide a more detailed definition of what mapping is.

4. Page 8 Line 162: Several other estimators have been applied in the mapping literature, including Fractional Logistic regression (FLOGIT); Censored Least Absolute Deviations (CLAD) regression; Generalized Additive Models; and finite mixture models. There are critics of the Tobit estimators, for example, but why haven't finite mixture models been applied?

5. Details of ethics committee approvals should be provided.

6. Model selection should not be based solely upon the criteria, such as the predictive accuracy of on root mean square error (RMSE), laid out on page 12. The paper would be strengthened by a formal and staged selection process employed to choose between the models, including the BIC, AIC (for models for which the likelihood can be computed), misspecification tests, comparisons of conditional means or other similarly informative measures. These should dictate both the choice of covariates as well as the selection across different models.

7. Page 15, When assessing model performance: the errors should also be reported across subsets of the EQ5D utility score range as this is useful for indicating whether or not there is systematic bias in the predictions.

8. External validation is the preferred method for ascertaining the predictive accuracy of a mapping model. The authors of this paper use in-sample validation methods. Can the authors provide a detailed explanation of what a "five-fold cross-validation" is and how the in-sample validation datasets were generated? Secondly, how did they ensure that 'overfitting' was not an issue in the validation exercise? Thirdly, can the authors comment on how adequate five-fold cross-validation is as opposed to say ten-fold validation which has been in several mapping studies.

Minor comments:

1. Figure S1A: Please correctly label x-axis EQ5D and not EQ5D0

2. Page 8 Line 154: Please correct HRQOL to HRQoL

3. The paper does not seem to fully get across that mapping is a second-best solution and that having original data collected from relevant populations is a better solution. For the uninitiated, they may believe that EPIC data collected from patients with PC can be converted to EQ5D utilities "with a high level of accuracy". Hence, there is no need to collect original utility data.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

11 Feb 2021

Response to Reviewer’s Comments

Dear Reviewers,

We thank you for your insightful feedback. We have incorporated it where possible and provided justifications for our approach where needed. We’ve summarized the responses to your comments below.

Reviewer #1: Mapping prostate cancer index

General comments:

1. Replace everywhere "external validation" by "test sample", this is the sample for internal validation. Refer to the first sample as the "estimation sample", this is the equivalent to a training sample in data science also called the hold out sample. External validation implies the use of an independent external data sample which is not the case here. (see https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets).

Answer: We agree that we performed an added internal validation step in absence of an external dataset to perform external validation. We’ve made the suggested edit wherever it applies. We have replaced ‘external validation’ with ‘validation’ and have referred to the internal validation using the estimation cohort as ‘5-fold cross-validation’.

2. Replace everywhere in the text EQ5D by EQ-5D-3L as this is the version of the PBM questionnaire

Answer: To address this, we use EQ-5D-3L (EQ5D) the first time it is referenced and EQ5D subsequently.

METHODS

Sample Selection:

3. Shortly describe the sample and the original data plus QOL results of the original article

Answer: RTOG 0415 (Lee et al.) is a non-inferiority trial to determine whether the efficacy of a hypo-fractionated treatment schedule was not worse than a conventional schedule in men with low-risk PC. Using QOL data from this trial, Bruner et al reported no clinically significant between-arm differences in EPIC domain scores and EQ-5D index and VAS scores through 5 years following the completion of radiation. Taken together with the reporting by Lee et al, treatment with HRT is non-inferior to CRT in terms of disease free survival and prostate cancer-specific and general QOL, providing evidence to affirm that HRT is the standard of care in men with low-risk prostate cancer.

We added the following text in the manuscript to summarize the key take away from the trial: “The results of the trial showed no significant differences in outcomes between the two treatment modalities.”

4. Correct the naming of the samples (see above):

Answer: Addressed. Appropriate changes were made wherever applicable.

Outcome measures:

5. Specify which value set (Tariff) was used to value the EQ-5D-3L data

Answer: We computed the U.S. preference-weighted index score, see: Shaw JW, Johnson JA, Coons SJ. US valuation of the EQ-5D health states: development and testing of the D1 valuation model. Med Care. 2005 Mar;43(3):203-20.

The following sentence was added to the manuscript: “The EQ-5D tariffs for our study were obtained using the US valuation of EQ-5D health states performed by Shaw et al. in a sample of 4,048 civilian noninstitutionalized English- and Spanish-speaking adults, aged 18 and older, who resided in the United States (50 states plus the District of Columbia) in 2002.”

6. I am not sure what you mean by "not part of the descriptive system"; death is anchored at zero and states worse than death can take negative values up to a lower limit, depending on the country specific Tariff used.

Answer: Addressed. We deleted “not part of the descriptive system’ from the text to avoid any confusion.

Model Development:

7. Line 160: this is wrong; OLS assumption implies that the errors (conditional on the explanatory variables) is normally distributed. This allows inference of the coefficients and tests of significance based on Normal Theory.

Answer: Addressed. We removed the incorrect statement.

8. Put the detailed list of the model specifications in an appendix

Answer: Addressed. Moved the table of specifications from the manuscript to the appendix.

9. Line 173: higher second and third order polynomials

Answer: Addressed. Added “second and third”.

10. Which stepwise variable selection was used? It seems it was a simple forward selection method based on the p-value. What was the criterion used to include/retain a variable?

(See Forward selection is a type of stepwise regression which begins with an empty model and adds in variables one by one. In each forward step, you add the one variable that gives the single best improvement to your model.

Answer: We used the forward selection method in which we began with an empty model and added variables one after the other. The p-value for entry was set at 0.25 and the p-value for retention was kept at 0.25. We’ve added text to the manuscript to reflect this.

11. Line 166 the exact two-step method should be more detailed. Was logistic regression used for the full health, what was its accuracy? How was the goodness of fit of the combined parts estimated utilities then further assessed? Were there any U values = 1 or higher resulting from the OLS regression part? How were these dealt with?

Answer: We have added text to the manuscript to reflect that a logistic regression was conducted to identify people in full health. As several models were run, the accuracy varied across models but around 80% of the patients who were full health were correctly classified.

The final model selection was done based on how well the utilities were predicted in both parts put together and was assessed using RMSEs and MAEs.

There were some cases where the predicted utilities were over 1; these potentially contributed to higher RMSE values for two-part models.

12. Forward Selection: Definition - Statistics How To

www.statisticshowto.com › forward-selection

Stepwise regression - Wikipedia "en.wikipedia.org › wiki › Stepwise_regression"

see Variable Selection www.biostat.jhsph.edu › ~iruczins › teaching

and for a discussion and limitations of the different methods

Loann. D. Desboulets , A Review on Variable Selection in Regression Analysis Econometrics, 2018

Answer: We thank the reviewer for sharing these useful resources.

13. The results of the forward regresson methods should be confrmed by another selection method, especially since the n/p (observations/parameters) ratio is rather low in the regression incorporating interactions and power variables.

A LASSO type or similar method would be useful in this situation. It can be applied to the full most detailed equation including all subscales and interactions/power variables.

Answer: We thank the reviewer for their insights about this method to obtain simpler models. We did not consider applying LASSO regularization to our models, as we examined models ranging from just a few parameters to a large number of parameters.

Assessing Model Performance:

14. The choice of only the RMSE as accuracy criterion (Goodness of fit) is not to be recommended, it should be complemented by other criteria as well including MAE, estimated utility values >1 and <0, etc.

Answer: While both RMSE and MAE have been used to compare predictive performance of mapping algorithms, RMSE penalizes larger errors more than MAE, making it a more appropriate metric to assess overall performance of mapping algorithms. However, we also computed the MAEs for the models and the results were mostly consistent with the RMSEs. Algorithms in this study rarely yielded predicted utilities higher than 1, and no action in that regard was needed as a result.

15. I would urge the authors to also present a Bland-Altman plot of the results of their best fitting model (with 95% confidence intervals and minimally clinically important limits as well for EQ-5D utilities +- 0.08)

Answer: Addressed. Bland-Altman plot for the best fitting model is provided in the supporting material. The green lines reflect the MCID of 0.08 for EQ5D suggested by the reviewer.

16. Also the multiple comparison problem given the huge number of regressions performed should be discussed/addressed.

Answer: In this analysis, no multiplicity corrections were taken into account since this process concerned building the most appropriate model as opposed to interpretation of the results of the model. When building and comparing different models, inflation of the type I error was irrelevant since we were not assessing the significance of any particular variable but the overall fit of the model.

17. Five-fold cross-validation was used, I guess this was on the test sample? How were the regression results then combined? Give some more details about the exact procedure followed to allow replication of your methodology by others.

Answer: For the OLS models, the PROC GLMSELECT procedure was used and 5-fold cross-validation was performed using “CV Method = block (5)” option. For the Tobit and Two-part models, SAS macros were used to split the sample, run the regressions in training sets, score the test sets, and combine the estimates. The SAS code for this analysis can be provided upon request.

RESULTS

Descriptive Statistics:

18. Include the results of a statistical comparison test of the variables between the different samples in table 2 and table 3 to assess their similarity of the samples

Answer: Addressed. A table comparing these samples is submitted as supporting material.

19. Given the highly bimodal nature of the observed utilities non-parametric summary measures (medians, IQR, etc.) and test statistics should be preferred added to the tables

Answer: Addressed. We report median EQ5D and IQR in the summary tables

Mapping results:

20. Show first the tables of performance and selection of the best fitting equations

then show the detailed equation of the best fitting one(s)

Answer: Addressed. Moved the equation after the table of performance. All other candidate models are shared in the supporting material.

21. Present the regression coefficients with their 95% CI and present also the variance-covariance matrix of the regression parameters

Answer: We have presented the regression coefficients for all candidate models (in manuscript text and supporting material).

DISCUSSION

22. Line 316 which generic PBM did Bremen used? Specify. How bad was the under-prediction of low observed utilities? Where was the utility threshold? How bad was the over-prediction for observed high utilities? Where was the utility threshold? What was the variance of the estimates compared to the observed variance of utilities for different values of utilities (low, average, high, perfect health) or per quartile?

Answer: Bremnan et al did not use a genetic PBM. Instead, they used a prostate cancer specific instrument, named PORPUS-U that measures health utilities. We did not report the other details around the performance of their algorithms as the instruments used in both studies are different. We merely wanted to bring to readers’ attention that this is the first study to map EPIC to obtain EQ5D utilities in our knowledge and that prior mapping studies in prostate cancer have used different instruments. The choice of a disease-specific PBM in the study by Bremnan et al. study makes the results harder to generalize across different therapeutic areas. Moreover, the algorithm has limited application as the most frequently employed PROM in clinical trials in prostate cancer is EPIC, while they mapped PCI, an older questionnaire that EPIC evolved from.

23. Line 338: Your risk of bias is linked to whether the censoring and non-response to the QoL questionnaires was truly random otherwise there is a risk of "survival or response" bias. Nothing tells you that the non responders had the same mapping coefficients as those of the completers so this could potentially alter the regression coefficients.

Answer: We investigated if differences exist between characteristics of responders and non-responders to gain insights into whether mapping coefficients between these patients would be different. Variables that differed between patients with missing and completed assessments:

• Baseline: none

• 6 months: RT modality actually received (83.5% with completed EPICs received IMRT vs. 73.5% with missing EPICs)

• 12 months: age (60.6% with completed EPICs were >65 vs. 49.2% with missing EPICs) and planned RT modality (stratification factor; 81.9% with completed EPICs planned for IMRT vs. 75.2% with missing EPICs)

• 24 months: None were seen

• 60 months: race (83.7% with completed EPICs were white vs. 75.9% with missing EPICs) and ethnicity (98.6% with completed EPICs were not Hispanic vs. 94.6% with missing EPICs) and planned RT modality (stratification factor; 77.0% with completed EPICs planned for IMRT vs. 82.9% with missing EPICs)

As very few differences were seen between the responders and non-responders, the risk of response bias was considered to be low.

Reviewer #2:

This paper has examined three econometric models for estimating EuroQol- 5 Dimension (EQ-5D) utility scores from the Expanded Prostate Cancer Index Composite (EPIC) to calculate quality adjusted life years for cost-utility analysis. The paper uses robust methods that should act as an aid for utility estimation within future economic evaluations of interventions using the Expanded Prostate Cancer Index Composite in Prostate Cancer. As such, it has the potential to act as a beneficial addition to the mapping literature. This article is well written, and the authors have carefully followed standard mapping methodology.

Response: We thank the reviewer for their insightful feedback. We have incorporated the feedback where applicable and provided clarification for the concerns raised in the review. The responses to each comment are summarized below.

Major comments:

1. Abstract Page 3, Line 51: The authors state that the lack of health utilities associated with the different health states assessed with the EPIC are unknown, therefore limiting the ability to perform cost-effectiveness evaluations. Can the authors edit this and use cost-utility analysis (CUA) and not cost-effectiveness analysis (CEA) as the form of economic evaluation which allows for the comparison of alternative treatment options in terms of incremental costs relative to quality-adjusted life-years (QALY) gained following treatment is a cost-utility analysis.

Answer: Thank you for the comment. We’ve addressed this and replaced ‘cost-effectiveness evaluations’ with ‘cost-utility analyses’ in the abstract.

2. Abstract Page 3, Line 52: The authors use the term "utility weights". This term is used in valuation studies when generating population preference weights or scoring algorithms and not mapping algorithms. The authors should correct this and use utility scores or utilities instead.

Answer: Thank you for the comment. We’ve addressed this as well.

3. Page 6: The authors present mapping as though the reader might already know what it is. Can the authors provide a more detailed definition of what mapping is?

Answer: The following sentence was added to provide more insights about the mapping process: “Utility mapping involves development and use of a statistical model or algorithm that links the outcomes from a PROM and a PBM to generate health utility values.”

4. Page 8 Line 162: Several other estimators have been applied in the mapping literature, including Fractional Logistic regression (FLOGIT); Censored Least Absolute Deviations (CLAD) regression; Generalized Additive Models; and finite mixture models. There are critics of the Tobit estimators, for example, but why haven't finite mixture models been applied?

Answer: In our study, the OLS models performed quite well with low errors overall both in the 5-fold cross-validation sample as well as in the 30% validation sample. Tobit and two-part models performed poorly compared to OLS models for each specification. These results were consistent with several other mapping studies that have found OLS models better than other more robust regression procedures in predicting health utilities. Therefore, additional model types were not explored.

5. Details of ethics committee approvals should be provided.

Answer: The following sentence was added to reflect the approvals sought in conducting this study: “The Institutional Review Board approval was sought and received from the University Of Maryland School Of Medicine and NRG Oncology”

6. Model selection should not be based solely upon the criteria, such as the predictive accuracy of on root mean square error (RMSE), laid out on page 12. The paper would be strengthened by a formal and staged selection process employed to choose between the models, including the BIC, AIC (for models for which the likelihood can be computed), misspecification tests, comparisons of conditional means or other similarly informative measures. These should dictate both the choice of covariates as well as the selection across different models.

Answer: We explored AIC and MAEs in addition to RMSE. MAEs do not penalize large errors like RMSE does, making RMSE a better indicator of predictive accuracy. The comparison of AICs for the tested model specifications provided results similar to the RMSEs, justifying the choice of RMSE as an indicator of predictive accuracy in our study.

7. Page 15: When assessing model performance: the errors should also be reported across subsets of the EQ5D utility score range as this is useful for indicating whether or not there is systematic bias in the predictions.

Answer: Based on the feedback from reviewer 1, we produced a Bland-Altman plot for the best performing model that shows the level of agreement between the observed and predicted utilities. Health utilities in our study wee under-predicted for patients in full health and over-predicted for those in more sever health states. This is a limitation of regression-based mapping and we have highlighted this in the discussion section. Additionally, we provide the plot of observed vs. predicted utilities, which show how accurate the prediction was (closer to the regression line indicating better prediction).

8. External validation is the preferred method for ascertaining the predictive accuracy of a mapping model. The authors of this paper use in-sample validation methods. Can the authors provide a detailed explanation of what a "five-fold cross-validation" is and how the in-sample validation datasets were generated? Secondly, how did they ensure that 'overfitting' was not an issue in the validation exercise? Thirdly, can the authors comment on how adequate five-fold cross-validation is as opposed to say ten-fold validation which has been in several mapping studies.

Answer: The following text is added in the manuscript to describe 5-fold-cross-validation: “In 5-fold cross-validation, the data are split into 5 equal parts and the model is fitted on 4 parts with the 5th being held out for validation. The fitted model of the 4 selected parts is used to compute the predicted residual sum of squares on the 5th omitted part, and this process is repeated for each of the 5 parts. The sum of the 5 predicted residual sums of squares is obtained for each fitted model and is the estimate of the prediction error. Indices such as the absolute mean of the residuals or errors (MAE), and square root of the mean of the residual sum of squares (RMSE) are used to determine model performance.” K-fold cross-validation ensures that we select the algorithm with the least errors on the training set as well as the test set, thus minimizing the risk of under-fitting or overfitting. The choice of k in k-fold cross-validation is somewhat arbitrary. While 10-fold CV has been found to result in models with relatively low bias and modest variance, 5-fold CV has also been used in several studies. On the other hand, some studies have used the leave one out cross-validation (LOOCV), where k = n. When selecting the number of folds in a CV exercise, one must balance the efficiency gains in terms of low bias, and the increase in run time and variance of the estimates as the number of folds increases. With a 5-fold CV approach, we had sufficient number of data points in our training sets (n ~ 400), increasing our confidence in this approach.

Minor comments:

1. Figure S1A: Please correctly label x-axis EQ5D and not EQ5D0

Answer: Addressed.

2. Page 8 Line 154: Please correct HRQOL to HRQoL

Answer: Addressed.

3. The paper does not seem to fully get across that mapping is a second-best solution and that having original data collected from relevant populations is a better solution. For the uninitiated, they may believe that EPIC data collected from patients with PC can be converted to EQ5D utilities "with a high level of accuracy". Hence, there is no need to collect original utility data.

Answer: The following sentence was added to the Introduction section to convey that mapping should be considered as an alternative only when direct estimation of utilities is not conducted: “Therefore, when utility information is not collected in a study, mapping has been proposed as an alternative solution and recommended as the second-best option after direct utility estimation for economic evaluations of interventions.”

Submitted filename: Response to Reviewer Comments.docx

Click here for additional data file.

12 Mar 2021

Mapping Expanded Prostate Cancer Index Composite to EQ5D Utilities to Inform Economic Evaluations in Prostate Cancer: Secondary Analysis of NRG/RTOG 0415

PONE-D-20-34900R1

Dear Dr. Mishra,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

A/Prof Michael E. O'Callaghan

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: The authors have addressed the points raised in my previous review, and publication of the manuscript as is recommended.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

5 Apr 2021

PONE-D-20-34900R1

Mapping Expanded Prostate Cancer Index Composite to EQ5D utilities to inform economic evaluations in Prostate Cancer: Secondary analysis of NRG/RTOG 0415

Dear Dr. Mishra:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Associate Professor Michael E. O'Callaghan

Academic Editor

PLOS ONE