Construct Validity of the Questionnaire Quality From the Patients Perspective Adapted for Surgical Prostate Cancer Patients.

Patient-reported experience measures (PREMs) are important to capture the patients' voice. No such measure is routinely used for evaluation after robotic-assisted radical prostatectomy for prostate cancer. The aim of this study was to adapt the short version of the PREM questionnaire quality from the patients' perspective (QPP), and assess the construct validity of this version. Quality from the patients' perspective assesses 4 dimensions of quality of care. Involving discussion with user representatives, the QPP short version was adapted by adding 7 context-specific questions based on items from the Expanded Prostate Cancer Index Composite for Clinical Practice. This short version was answered on smartphone or tablet by 265 patients. We used exploratory factor analysis to assess dimensionality. For comparison with previous publications of the QPP, the analysis was repeated after mean imputation of missing values. The factor analysis identified 7 factors among the 30 analyzed items included in the analysis, explaining 64.9% of the variance. After imputation of missing, 2 factors explained 48.6% of the variance. None of these analysis captured the 4 dimensions of the QPP.

Background

Prostate cancer is the most common cancer among men, and in Norway, the lifetime prevalence is about 15% (1). The natural history of prostate cancer varies from low-risk disease with good prognosis without treatment, to high-risk disease with rapid progression (2). Hence, treatment is individualized, and management varies according to patient characteristics (age, comorbidity), patient preferences, and disease factors (tumor stage and histologic grade). To avoid overtreatment and adverse effects, patients with low-risk disease are normally observed. Patients with intermediate and high-risk disease and a life expectancy over 10 years are offered treatment with curative intention. According to current guidelines, standard curative or radical treatment is either external radiation therapy or surgery, currently most often in the form of robotic-assisted radical prostatectomy (RARP) (3). Both these treatment may reduce quality of life (4). After surgery, the most common long-term adverse effects are urinary incontinence and erectile dysfunction.

Patient-reported outcome measures (PROMs) are essential to evaluate treatment outcomes and to assure good quality of care. Several questionnaires have been developed for this purpose. Until recently, no standard PROM questionnaire has been used for patients with prostate cancer, but the International Consortium for Health Outcomes recommends Expanded Prostate Cancer Index Composite (EPIC) (5). However, to capture the patients’ voice and meet the criteria for quality of care defined by the World Health Organization (WHO) (6), it is necessary to combine PROMs with patient-reported experience measures (PREMs). Despite current recommendations (7), systematic collection of PROMs and PREMs is often missing in clinical registries (8) and PREMs are not routine for the evaluation of prostate cancer treatment (9). For surgical prostate cancer patients, a PROM measure would be urinary adverse effects, while a PREM measure is how they experience the information about adverse effects and involvement in decisions about their treatment.

Choosing a proper PREM questionnaire to evaluate prostate cancer care is a challenge, as there are few available instruments and no gold standard (10,11). The Swedish questionnaire quality from the patients’ perspective (QPP) is a PREM questionnaire that has been tested and validated in other settings and patient groups (12,13). A short version has been translated to Norwegian. The aim of this study was to adapt the QPP short version specifically for patients with prostate cancer treated with RARP and to assess the construct validity of this adapted version.

Methods

Setting and Routines

The Urological Department at Innlandet Hospital Trust treats about 200 patients with RARP each year. Patients referred for surgery receive oral and written information about the procedure and possible adverse effects, and if they consent, they are enrolled in a local quality database (14). The database includes clinical data and PROMs assessed by the Expanded Prostate Index Composite for Clinical Practice (EPIC-CP) (15). In August 2017, an adapted short version of the QPP was included in the quality database. Prior to discharge, the patients completed the QPP using a tablet or smartphone. They were encouraged to complete the questionnaire without assistance. Eligible patients (treated with RARP, Norwegian speaking, providing informed consent) were consecutively recruited to participate in this study from August 2017 to June 2019.

Questionnaire

The original QPP is based on qualitative patients’ interviews, aiming to identify the aspects of care that matter the most to the them (12). A short form was later developed (13).

The QPP is intended to assess 4 dimensions of quality: (a) the caregivers’ medical/technical competence; (b) identity-oriented approach toward the patients; (c) the organization’s physical-technical conditions; and (d) sociocultural approach (12). The caregivers’ medical/technical competence and identity-oriented approach are person-related dimensions. The former includes the perceived quality of the involved health personnel’s competence and the latter assesses whether the patients feel sufficiently informed about planned treatment and adverse effects. The physical–technical dimension concerns whether up-to-date equipment is available, and the sociocultural approach dimension assesses whether the health care unit is constructed for and oriented to the patients rather than for and to its staff.

The QPP differs from other PREM questionnaires in that it assesses the subjective importance of all items (10). The patients are presented multiple statements on the quality of various aspects of care, for which they are asked to respond on a Likert scale from “do not agree at all” (1) to “totally agree” (4). For each item, they are asked to report its perceived importance on a scale from “little or no importance” (1) to “very high importance” (4). “Not applicable” options are also available.

To adapt the QPP short version to prostate cancer patients, 7 context-specific questions based on items from EPIC-CP were added. Expanded Prostate Cancer Index Composite for Clinical Practice assesses urinary and sexual adverse effects (15). The context-specific questions assess the patients perceived quality and importance of the information given about these adverse effects and the help they received to better cope with them. To ensure that the additional items were understandable and relevant, the adapted version was tested on 5 user representatives previously treated with RARP. The process involved discussion with each representative and presentation of QPP for the whole group. The final QPP version included 30 items assessing perceived quality and the same 30 questions about the subjective importance of these items.

Statistical Analysis

Patient characteristics were described as means, SD, minimum and maximum values for continuous variables, and as frequencies and percentages for categorical variables. The QPP items were described by frequencies and percentages, yet to allow comparison with other studies, also means and SDs were calculated. Responses “not applicable” were considered as structurally missing values. The pattern of other missing values was explored by creating a dummy variable for missing values for each item, and running a multiple logistic regression model with a dummy variable as outcome and patient characteristics as explanatory variables (16). When any of the considered characteristics were significantly associated the dummy variable, missing data were assumed not to be missing conditionally at random, which is usually considered a prerequisite for analyzing data sets with missing values.

As new items were added to the short version of QPP, the absence of a hypothesis precluded a confirmatory factor analysis for assessing the dimensionality of the questionnaire. Exploratory factor analysis was therefore applied for the adapted QPP. For extraction of factors, 3 methods were employed, principal factors, principal-component factors, and iterated principal factor method. The number of factors was assessed by applying the Kaiser’s criterion of eigenvalues equal to or larger than 1.0, followed by parallel analysis. The extracted factors were further rotated by varimax and promax methods for easier interpretation.

A matrix of Spearman’s correlations was employed as input for exploratory factor analysis, an appropriate approach for skewed ordinal data. Additionally, and entirely for comparison to other studies, the same analysis was performed with missing values, also those structurally missing, replaced by the mean values of existing items.

Internal consistency of the identified factors was assessed with Cronbach’s α, where the values close to 1 indicate good internal consistency of the scale (17). As Cronbach’s α is sensitive to deviations from normality, omega coefficient was presented as well (18). The analyses were performed in SPSS v26 and Stata/SE v16.1.

Results

From August 2017 to June 2019, 361 patients gave their consent to inclusion in the quality assurance database and to participate in the present study. Of these, 265 (73.4%) completed the adapted short version of QPP. Their mean age was 66 years (38-79, SD = 6.5 years). Twenty-six percent had 9 years of obligatory school, 38% had high school education, and 33% an academic degree.

Overall, the patients rated the quality of care as good (Table 1). The proportion of missing items ranged from 3.4% (9/265) to 9.8% (26/265), the lowest percentage for “I receive examination and treatment within an acceptable waiting time,” and the highest for “I receive help for sexual adverse effects.” The proportion of items considered “not applicable” ranged from none to 34.7% (92/265). The item most often considered as “not applicable” was “I receive help for sexual adverse effects.” The reported subjective importance for the items showed similar pattern (Table 1).

Table 1.

Distributions of Scores for Each Item.a

	Mean (SD)	Missing, n (%)	Score 0, n (%)	Score 1, n (%)	Score 2, n (%)	Score 3, n (%)	Score 4, n (%)
A: Items addressing perceived reality
I receive good information about examination and treatment	3.8 (0.5)	10 (3.8)	1 (0.4)	0	2 (0.8)	37 (14.0)	215 (81.1)
I receive good information about results of examination and treatment	3.7 (0.5)	14 (5.3)	0	2 (0.8)	3 (1.1)	55 (20.8)	191 (77.2)
I receive good information about how to take care of myself	3.6 (0.7)	14 (5.3)	3 (1.1)	0	6 (2.3)	69 (26.0)	173 (65.3)
I receive good information about which doctors who are responsible	3.6 (0.7)	15 (5.7)	1 (0.4)	6 (2.3)	15 (5.7)	44 (16.6)	184 (69.4)
I receive good information about which nurses who are responsible	3.4 (0.9)	14 (5.3)	2 (0.8)	11 (4.2)	28 (10.6)	54 (20.4)	156 (58.9)
I receive good information about my illness	3.6 (0.6)	16 (6.0)	0	1 (0.4)	9 (3.4)	68 (25.7)	171 (64.5)
I receive good information about adverse effects	3.6 (0.7)	14 (5.3)	3 (1.1)	2 (0.8)	11 (4.2)	74 (27.4)	161 (60.8)
I receive good information about urinary adverse effects	3.5 (0.9)	18 (6.8)	8 (3.0)	2 (0.8)	8 (3.0)	59 (22.3)	170 (64.2)
I receive good information about bowel adverse effects	3.0 (1.2)	14 (5.3)	16 (6.0)	12 (4.5)	47 (17.7)	70 (26.4)	106 (40.0)
I receive good information about sexual adverse effects	3.4 (1.0)	16 (6.0)	12 (4.5)	0	17 (6.4)	64 (24.2)	156 (58.9)
I receive the best possible medical care	3.9 (0.3)	13 (4.9)	0	0	1 (0.4)	21 (7.9)	230 (86.8)
I receive help for pain	3.9 (0.5)	11 (4.2)	2 (0.8)	0	2 (0.8)	26 (9.8)	224 (84.5)
I receive examination and treatment within an acceptable waiting time	3.7 (0.6)	9 (3.4)	1 (0.4)	2 (0.8)	11 (4.2)	47 (17.7)	195 (73.6)
I receive help for urinary adverse effects	2.9 (1.6)	17 (6.4)	51 (19.2)	2 (0.8)	8 (3.0)	42 (15.8)	145 (54.7)
I receive help for bowel adverse effects	2.5 (1.6)	16 (6.0)	66 (24.9)	4 (1.5)	29 (10.9)	51 (19.2)	99 (37.4)
I receive help for sexual adverse effects	1.9 (1.7)	26 (9.8)	92 (34.7)	6 (2.3)	30 (11.3)	49 (18.5)	62 (23.4)
I have access to necessary equipment	3.6 (1.0)	14 (5.3)	14 (5.3)	1 (0.4)	4 (1.5)	35 (13.2)	197 (74.3)
I have access to necessary urinary pads	3.6 (1.0)	12 (4.5)	17 (6.4)	0	4 (1.5)	17 (6.4)	215 (81.1)
Doctors seem to understand how I experience my situation	3.8 (0.5)	14 (5.3)	0	1 (0.4)	6 (2.3)	45 (17.0)	199 (75.1)
Doctors are respectful toward me	3.9 (0.3)	12 (4.5)	0	0	1 (0.4)	18 (6.8)	234 (88.3)
Doctors show empathy for me	3.8 (0.4)	13 (4.9)	0	0	2 (0.8)	44 (16.6)	206 (77.7)
Nurses seem to understand how I experience my situation	3.9 (0.3)	12 (4.5)	0	0	1 (0.4)	17 (6.4)	235 (88.7)
Nurses are respectful toward me	3.9 (0.3)	11 (4.2)	1 (0.4)	0	0	12 (4.5)	241 (90.1)
Nurses show empathy for me	3.9 (0.3)	17 (6.4)	0	0	0	17 (6.4)	231 (87.2)
I have good opportunity to participate in the decisions regarding my  treatment	3.5 (0.9)	10 (3.8)	10 (3.8)	0	18 (6.8)	59 (22.3)	168 (63.4)
My relatives and friends are treated with respect	3.0 (1.7)	21 (7.9)	55 (20.9)	0	4 (1.5)	15 (5.7)	170 (64.2)
There is a pleasant and secure atmosphere on the ward	3.8 (0.6)	12 (4.5)	3 (1.1)	1 (0.4)	1 (0.4)	27 (10.2)	221 (83.4)
My care is determined by my own request rather than staff procedures	3.4 (1.0)	16 (6.0)	8 (3.0)	6 (2.3)	21 (7.9)	63 (23.8)	151 (57.0)
I have the opportunity to talk the doctors undisturbed	2.0 (1.9)	14 (5.3)	106 (40.0)	9 (3.4)	9 (3.4)	22 (8.3)	105 (39.6)
I have the opportunity to talk to the nurses undisturbed	2.0 (1.9)	19 (7.2)	111 (41.9)	4 (1.5)	8 (3.0)	19 (7.2)	104 (39.2)
B: Items addressing subjective importance
I receive good information about examination and treatment	3.8 (0.4)	30 (11.3)	0	0	2 (0.8)	38 (14.3)	195 (73.6)
I receive good information about results of examination and treatment	3.8 (0.4)	28 (10.6)	0	0	1 (0.4)	49 (18.5)	187 (70.6)
I receive good information about how to take care of myself	3.6 (0.7)	32 (12.1)	3 (1.1)	0	4 (1.5)	65 (24.5)	161 (60.8)
I receive good information about which doctors who are responsible	3.6 (0.7)	27 (10.2)	2 (0.8)	4 (1.5)	11 (4.2)	50 (18.9)	171 (64.5)
I receive good information about which nurses who are responsible	3.4 (0.9)	27 (10.2)	3 (1.1)	7 (2.6)	22 (8.3)	64 (24.2)	142 (53.6)
I receive good information about my illness	3.7 (0.6)	30 (11.3)	0	2 (0.8)	4 (1.5)	64 (24.2)	165 (62.3)
I receive good information about adverse effects	3.6 (0.8)	31 (11.7)	4 (1.5)	0	13 (4.9)	52 (19.6)	165 (62.3)
I receive good information about urinary adverse effects	3.5 (0.9)	31 (11.7)	9 (3.4)	2 (0.8)	10 (3.8)	46 (17.4)	167 (63.0)
I receive good information about bowel adverse effects	3.1 (1.2)	36 (13.6)	16 (6.0)	4 (1.5)	29 (10.9)	66 (24.9)	114 (43.0)
I receive good information about sexual adverse effects	3.3 (1.1)	38 (14.3)	15 (5.7)	3 (1.1)	16 (6.0)	57 (21.5)	136 (51.3)
I receive the best possible medical care	3.9 (0.3)	38 (14.3)	0	0	1 (0.4)	19 (7.2)	207 (78.1)
I receive help for pain	3.8 (0.6)	33 (12.5)	2 (0.8)	0	5 (1.9)	25 (9.4)	200 (75.5)
I receive examination and treatment within an acceptable waiting time	3.8 (0.6)	32 (12.1)	1 (0.4)	1 (0.4)	8 (3.0)	33 (12.5)	190 (71.7)
I receive help for urinary adverse effects	2.9 (1.6)	37 (14.0)	47 (17.7)	1 (0.4)	6 (2.3)	40 (15.1)	134 (50.6)
I receive help for bowel adverse effects	2.5 (1.7)	38 (14.3)	61 (23.0)	2 (0.8)	15 (5.7)	47 (17.7)	102 (38.5)
I receive help for sexual adverse effects	2.1 (1.7)	48 (18.1)	77 (29.1)	3 (1.1)	22 (8.3)	56 (21.1)	59 (22.3)
I have access to necessary equipment	3.6 (1.0)	37 (14.0)	14 (5.3)	0	1 (0.4)	41 (15.5)	172 (64.5)
I have access to necessary urinary pads	3.6 (1.0)	42 (15.8)	15 (5.7)	0	2 (0.8)	18 (6.8)	188 (70.9)
Doctors seem to understand how I experience my situation	3.8 (0.5)	33 (12.5)	0	1 (0.4)	3 (1.1)	35 (13.2)	193 (72.8)
Doctors are respectful toward me	3.9 (0.2)	37 (14.0)	0	0	0	13 (4.9)	215 (81.1)
Doctors show empathy for me	3.9 (0.4)	33 (12.5)	0	0	1 (0.4)	32 (12.1)	199 (75.1)
Nurses seem to understand how I experience my situation	3.9 (0.3)	33 (12.5)	0	0	1 (0.4)	15 (5.7)	216 (81.5)
Nurses are respectful toward me	3.9 (0.2)	34 (12.8)	0	0	0	13 (4.9)	218 (82.3)
Nurses show empathy for me	3.9 (0.3)	34 (12.8)	0	0	0	17 (6.4)	214 (80.8)
I have good opportunity to participate in the decisions regarding my  treatment	3.5 (0.9)	37 (14.0)	8 (3.0)	0	11 (4.2)	50 (18.9)	159 (60.0)
My relatives and friends are treated with respect	3.1 (1.6)	42 (15.8)	46 (17.4)	0	4 (1.5)	17 (6.4)	156 (58.9)
There is a pleasant and secure atmosphere on the ward	3.9 (0.5)	41 (15.5)	2 (0.8)	1 (0.4)	0	21 (7.9)	200 (75.5)
My care is determined by my own request rather than staff procedures	3.4 (0.9)	40 (15.1)	8 (3.0)	2 (0.8)	18 (6.8)	54 (20.4)	143 (54.0)
I have the opportunity to talk the doctors undisturbed	2.0 (1.9)	42 (15.8)	93 (35.1)	8 (3.0)	10 (3.8)	23 (8.6)	89 (33.6)
I have the opportunity to talk to the nurses undisturbed	2.0 (1.9)	41 (15.5)	98 (37.0)	6 (2.3)	11 (4.2)	19 (7.2)	90 (34.0)

a The table includes 30 items covering perceived reality followed by the same 30 items for subjective importance. Data are presented as mean scores, SD, frequencies and percentages. The 4-point Likert scale for items addressing perceived reality ranges from 1: Do not agree to 4: Totally agree. The scale for items addressing subjective importance ranges from 1: Little or no importance to 4: Very high importance. For all items 0 = not applicable.

After starting to include patients, we discovered that some patients received a questionnaire with incorrect answering options for subjective importance. Instead of presenting the options ranging from “little or no importance” to “very high importance,” they were presented with options ranging from “do not agree at all” to “totally agree.” Consequently, the main analyses were based on data for the 30 items covering perceived quality.

For several items, the patterns of missing values showed that these values were not likely to be “missing conditionally at random” (data not presented), making most imputation methods questionable. Spearman’s correlations for the 30 items assessing perceived quality showed no clear pattern (Table 2). The exploratory factor analysis with the principal factor extraction method resulted in 19 factors. As most of these factors contained only weakly loading items, this solution was not explored further.

Table 2.

Spearman’s Correlations among 30 Items Representing Perceived Reality Calculated on All Available Pairs of Items.

	3	5	7	9	11	13	15	17	19	21	23	25	27	29	31	33	35	37	39	41	43	45	47	49	51	53	55	57	59
1. I receive good information about examination and treatment	0.6	0.3	0.3	0.3	0.4	0.4	0.4	0.3	0.5	0.4	0.3	0.3	0.5	0.3	0.3	0.4	0.3	0.5	0.5	0.4	0.4	0.4	0.4	0.4	0.4	0.3	0.3	0.4	0.2
3. I receive good information about results of examination and treatment		0.4	0.4	0.4	0.6	0.4	0.5	0.4	0.4	0.3	0.3	0.3	0.5	0.4	0.4	0.4	0.3	0.4	0.3	0.4	0.3	0.3	0.3	0.4	0.4	0.2	0.4	0.4	0.2
5. I receive good information about how to take care of myself			0.3	0.4	0.4	0.4	0.4	0.3	0.4	0.2	0.2	0.2	0.4	0.3	0.3	0.3	0.2	0.3	0.2	0.3	0.3	0.3	0.3	0.4	0.3	0.4	0.2	0.3	0.2
7. I receive good information about which doctors who are responsible				0.5	0.3	0.3	0.4	0.3	0.2	0.3	0.1	0.2	0.3	0.2	0.3	0.3	0.3	0.4	0.3	0.5	0.3	0.3	0.3	0.4	0.4	0.3	0.3	0.4	0.3
9. I receive good information about which nurses who are responsible					0.4	0.2	0.3	0.4	0.2	0.3	0.2	0.2	0.4	0.4	0.3	0.3	0.2	0.3	0.3	0.3	0.4	0.3	0.3	0.4	0.3	0.2	0.3	0.3	0.3
11. I receive good information about my illness						0.5	0.5	0.4	0.5	0.3	0.3	0.3	0.4	0.4	0.5	0.4	0.2	0.3	0.3	0.3	0.2	0.3	0.3	0.3	0.3	0.3	0.3	0.4	0.3
13. I receive good information about adverse effects							0.5	0.3	0.5	0.3	0.2	0.3	0.4	0.4	0.4	0.3	0.1	0.3	0.3	0.3	0.2	0.3	0.3	0.4	0.3	0.3	0.3	0.3	0.3
15. I receive good information about urinary adverse effects								0.4	0.4	0.3	0.3	0.3	0.4	0.4	0.4	0.4	0.3	0.3	0.2	0.3	0.3	0.3	0.3	0.4	0.3	0.3	0.2	0.3	0.3
17. I receive good information about bowel adverse effects									0.3	0.2	0.2	0.3	0.4	0.6	0.4	0.3	0.1	0.3	0.1	0.3	0.2	0.1	0.1	0.4	0.1	0.2	0.2	0.2	0.3
19. I receive good information about sexual adverse effects										0.3	0.2	0.2	0.4	0.4	0.5	0.4	0.3	0.3	0.3	0.3	0.2	0.3	0.3	0.3	0.2	0.2	0.2	0.2	0.1
21. I receive the best possible medical care											0.5	0.3	0.2	0.2	0.1	0.3	0.4	0.4	0.4	0.4	0.5	0.5	0.6	0.3	0.5	0.4	0.3	0.2	0.1
23. I receive help for pain												0.3	0.4	0.3	0.2	0.3	0.4	0.2	0.2	0.2	0.4	0.3	0.4	0.2	0.4	0.4	0.3	0.3	0.2
25. I receive examination and treatment within an acceptable waiting time													0.3	0.3	0.2	0.3	0.2	0.2	0.2	0.3	0.2	0.2	0.2	0.3	0.2	0.3	0.3	0.1	0.0
27. I receive help for urinary adverse effects														0.6	0.4	0.4	0.3	0.4	0.3	0.3	0.3	0.2	0.3	0.4	0.2	0.2	0.2	0.3	0.2
29. I receive help for bowel adverse effects															0.6	0.3	0.2	0.4	0.2	0.3	0.3	0.2	0.2	0.3	0.1	0.2	0.3	0.3	0.2
31. I receive help for sexual adverse effects																0.4	0.2	0.3	0.2	0.3	0.1	0.2	0.1	0.3	0.1	0.2	0.4	0.4	0.3
33. I have access to necessary equipment																	0.5	0.3	0.3	0.3	0.3	0.4	0.4	0.4	0.4	0.3	0.3	0.4	0.3
35. I have access to necessary urinary pads																		0.3	0.5	0.4	0.5	0.5	0.4	0.2	0.5	0.4	0.2	0.2	0.1
37. Doctors seem to understand how I experience my situation																			0.6	0.6	0.5	0.4	0.5	0.5	0.4	0.4	0.4	0.4	0.4
39. Doctors are respectful toward me																				0.6	0.5	0.6	0.5	0.3	0.5	0.4	0.3	0.3	0.1
41. Doctors show empathy for me																					0.4	0.4	0.5	0.5	0.5	0.3	0.4	0.3	0.2
43. Nurses seem to understand how I experience my situation																						0.7	0.7	0.3	0.5	0.4	0.3	0.4	0.2
45. Nurses are respectful toward me																							0.7	0.4	0.6	0.5	0.3	0.4	0.3
47. Nurses show empathy for me																								0.4	0.6	0.4	0.4	0.4	0.3
49. I have good opportunity to participate in the decisions regarding my treatment																									0.3	0.4	0.5	0.3	0.4
51. My relatives and friends are treated with respect																										0.6	0.3	0.4	0.3
53. There is a pleasant and secure atmosphere on the ward																											0.4	0.3	0.1
55. My care is determined by my own request rather than staff procedures																												0.3	0.4
57. I have the opportunity to talk the doctors undisturbed																													0.7
59. I have the opportunity to talk to the nurses undisturbed

The principal component factor method identified 7 factors (Table 3), with the same structure obtained by applying promax and varimax rotation, that explained 64.9% of variation of the scale. Factor-1, consisting of 7 items, explained 35.5% of the variance (Cronbach’s α 0.86 and ω 0.90). Factor-2 consisted of 9 items but explained only 9.2% of the variance. The remaining factors contained 2 to 3 items and explained little of the variance.

Table 3.

Results of Exploratory Factor Analyses.a

	Principal-component factors method, varimax rotation							Iterated principal factor method, varimax rotation
	F1	F2	F3	F4	F5	F6	F7	F1	F2	F3	F4
Proportion of variance explained	35.5	9.2	5.3	4.0	3.8	3.7	3.4	35.0	8.7	5.1	3.5
Cronbach’s α	0.86	0.86	0.73	0.76	0.85	0.60	0.62	0.90	0.87	0.85	0.74
Omega	0.90	0.90	0.79	0.70	0.61	0.74	0.68	0.90	0.88	0.62	0.67
1. I receive good information about examination and treatment	0.33	0.52	0.20	0.39	0.01	−0.02	0.13	0.50	0.38	0.06	0.23
3. I receive good information about results of examination and treatment	0.19	0.60	0.22	0.21	0.08	0.03	0.36	0.62	0.28	0.16	0.13
5. I receive good information about how to take care of myself	0.15	0.53	0.13	0.00	0.09	0.14	0.35	0.52	0.23	0.16	0.05
7. I receive good information about which doctors who are responsible	0.18	0.26	0.02	0.26	0.15	0.18	0.67	0.34	0.31	0.26	0.22
9. I receive good information about which nurses who are responsible	0.18	0.12	0.45	0.09	0.17	0.05	0.62	0.43	0.19	0.24	0.21
11. I receive good information about my illness	0.14	0.68	0.23	0.07	0.12	0.05	0.20	0.66	0.22	0.17	0.03
13. I receive good information about adverse effects	0.08	0.70	0.07	0.13	0.06	0.32	0.01	0.58	0.21	0.11	0.08
15. I receive good information about urinary adverse effects	0.18	0.65	0.17	−0.03	0.13	0.08	0.22	0.60	0.25	0.18	−0.03
17. I receive good information about bowel adverse effects	−0.01	0.23	0.64	−0.03	0.13	0.22	0.31	0.58	−0.02	0.18	0.19
19. I receive good information about sexual adverse effects	0.18	0.70	0.22	0.19	−0.01	0.09	−0.28	0.58	0.23	−0.03	0.08
21. I receive the best possible medical care	0.69	0.22	0.11	0.03	−0.11	0.19	0.10	0.26	0.65	−0.03	0.06
23. I receive help for pain	0.61	0.18	0.36	−0.30	−0.03	0.20	−0.03	0.33	0.44	0.01	−0.03
25. I receive examination and treatment within an acceptable waiting time	0.24	0.30	0.25	−0.06	−0.33	0.37	0.25	0.42	0.27	−0.09	0.08
27. I receive help for urinary adverse effects	0.19	0.40	0.63	0.24	0.01	−0.10	0.12	0.61	0.14	0.03	0.26
29. I receive help for bowel adverse effects	0.08	0.27	0.80	0.14	0.07	0.15	0.01	0.65	0.01	0.05	0.31
31. I receive help for sexual adverse effects	−0.05	0.49	0.48	0.10	0.26	0.27	−0.09	0.63	−0.02	0.20	0.18
33. I have access to necessary equipment	0.38	0.52	0.13	0.01	0.21	0.07	0.13	0.47	0.39	0.21	0.02
35. I have access to necessary urinary pads	0.68	0.16	0.11	0.15	−0.02	−0.03	0.05	0.18	0.59	0.00	0.13
37. Doctors seem to understand how I experience my situation	0.33	0.15	0.28	0.59	0.24	0.25	0.09	0.29	0.35	0.23	0.62
39. Doctors are respectful toward me	0.56	0.14	0.08	0.62	−0.03	0.04	0.08	0.15	0.60	−0.02	0.48
41. Doctors show empathy for me	0.28	0.22	0.07	0.63	0.02	0.35	0.24	0.28	0.41	0.11	0.53
43. Nurses seem to understand how I experience my situation	0.76	0.01	0.14	0.23	0.17	0.06	0.11	0.10	0.70	0.16	0.24
45. Nurses are respectful toward me	0.74	0.14	−0.07	0.29	0.25	0.12	0.03	0.07	0.76	0.22	0.22
47. Nurses show empathy for me	0.75	0.16	0.04	0.25	0.20	0.13	0.05	0.15	0.75	0.18	0.21
49. I have good opportunity to participate in the decisions regarding my treatment	0.14	0.26	0.21	0.32	0.21	0.51	0.22	0.41	0.25	0.26	0.37
51. My relatives and friends are treated with respect	0.69	0.25	−0.19	0.08	0.14	0.23	0.30	0.16	0.76	0.21	0.03
53. There is a pleasant and secure atmosphere on the ward	0.51	0.18	0.00	−0.03	0.00	0.58	0.15	0.25	0.53	0.09	0.13
55. My care is determined by my own request rather than staff procedures	0.18	0.12	0.20	0.26	0.20	0.68	0.02	0.30	0.27	0.21	0.33
57. I have the opportunity to talk the doctors undisturbed	0.27	0.27	0.10	0.09	0.75	0.04	0.22	0.27	0.27	0.67	0.12
59. I have the opportunity to talk to the nurses undisturbed	0.09	0.15	0.15	0.03	0.87	0.18	0.10	0.20	0.07	0.89	0.13

a Factor loadings in bold face indicate factor for each item.

Iterated principal factor method with varimax rotation resulted in 4 factors with loadings higher than when promax rotation was used. The 4 factors explained 52.2% of the variance (Table 3) and showed quite good internal consistency, but the structure was difficult to interpret. Factor 1 was largest and consisted of 17 items, while factor 3 and factor 4 only contained 2 items.

In all analyses, the Kaiser’s criterion agreed well with the results of parallel analysis. The exploratory factor analysis on items with missing values imputed by the average of the existing items for each patient, gave a completely different factor structure. The model identified only 2 factors, with factor 1 consisting of 19 items contributing with 49.0% of explained variance, while factor 2 included the remaining 11 items but explained only 11.1% of the variance (results not shown).

Discussion

We adapted a short version of the QPP questionnaire for patients treated with RARP and assessed its construct validity. However, we did not identify the 4 previously described dimensions of the QPP questionnaire.

Our main analysis (principal component factor method) identified 7 factors. Factor-1 included items from 3 dimensions: the caregivers’ medical/technical competence, their identity-oriented approach toward the patients, and the organization’s physical–technical conditions. Factor 2 included items from the identity-oriented dimension and items that addresses the caregivers’ medical/technical competence. In addition, items from the identity-oriented dimension were presented in 4 different factors, and closely related items in the same dimension loaded on different factors. Items in the sociocultural approach dimension loaded on 3 different factors, and although all items addressing the caregivers’ medical/technical competence loaded on the same factor (factor 1), this factor also included items from 2 other dimensions. Thus, we found no clear pattern among items and were not able to differentiate 1 dimension from another in our data set. Analysis with iterated principal factor method identified 4 factors, but no clear structure was found.

To enable comparisons with former studies on the QPP, we performed an exploratory factor analysis after imputing missing values with the mean of existing items on each patient. This resulted in a 2-factor solution, which was very different from those found in our main analyses. This might indicate that mean imputation affects the correlation structure in a considerable way. Furthermore, the structure of the 2-factor solution did not coincide with those reported in other patient groups (12,13,19).

Adding new items may explain that our results are inconsistent with the original QPP dimensions. We believe, however, that this discrepancy relates to different statistical approach. We have chosen to follow recommendations stating that the first step in assessing the dimensionality of a questionnaire is to explore the correlation matrix among items and to perform factor analysis on all items simultaneously as it is important to test if items in 1 dimension do not load on others (20,21). In other studies on the QPP, the lack of correlation matrix among the items limits the insight into the structure of the questionnaire and without factor analysis performed on all items simultaneously (12), it is not possible to assess cross-loadings, which might have revealed a different structure. Furthermore, Cronbach’s α, as used in former QPP publications is not sufficient to assess dimensionality (12,13). Cronbach’s α cannot be regarded as a measure of unidimensionality (22) but rather assumes that items constitute a single dimension and measure its internal consistency.

The distribution of answers on the items assessing perceived quality of care was highly skewed. A highly skewed distribution and a ceiling effect seem both to be a general, problematic characteristic of most existing PREM instruments (11). Whether this is the case for QPP when applied in other settings is not clear. Many previous studies have reported descriptive statistics for single items as means and SD only (19), which does not give an adequate picture of the data distribution. Moreover, we observed a high proportion of missing items and items considered not applicable, which implies difficulties when assessing construct validity. The management of missing values and the background for imputation methods are scarcely described in other studies of QPP (12,19).

From a clinical point of view, our results indicate that the patients were generally satisfied with their care. However, we intended to find a PREM questionnaire for surgical prostate cancer patients that provides relevant feedback and identifies areas in need of improvement, and the modified QPP cannot be recommended for routine use in its present form. Its clinical value is restricted by the ceiling effect, and we will thus continue our pursuit for a clinically useful PREM questionnaire for RARP patients. We will consider a questionnaire with fewer items to reduce the burden for the patients and improve the response rate.

A limitation of this study is that some patients received a questionnaire with incorrect response options for their perceived importance of the different items, which limited the analysis to data for perceived quality of care.

Conclusions

We were not able to identify the previously described dimensions of the QPP in our cohort of surgical prostate cancer patients. In its present form, QPP has limited clinical value to assess how patients with prostate cancer experience their care after RARP.