Validation of a patient reported outcome questionnaire for assessing success of endoscopic prostatectomy.

PURPOSE: Several international committees involved in establishing standards of care have recommended that patients undergoing surgery for bladder outlet obstruction should be assessed with patient reported outcomes (PRO). The Patient Global Impression of Improvement (PGI-I) is an instrument designed to measure a patients interpretation of symptom changes following intervention. The objective of this study was to validate the PGI-I as a PRO assessment following surgery for bladder outflow obstruction (BOO) in men with benign prostatic hyperplasia (BPH).
METHODS: Men undergoing photoselective vaporisation of the prostate were followed prospectively. Pre- and postoperative International Prostate Symptom Score (IPSS), Quality of life (QoL) index, peak urinary flow (Qmax), and postvoid residual (PVR) assessments were done. The PGI-I was conducted and correlated at 3 months postoperatively to changes in IPSS, QoL, Qmax, and PVR.
RESULTS: One hundred and sixty-six consecutive patients were included. Following surgery, IPSS and QoL improved by 11 and 2.4 points (P<0.0001). PGI-I was found to correlate with postoperative changes in IPSS and QoL (Pearson correlation, 0.47 and 0.58, respectively; P<0.0001).
CONCLUSIONS: This is the first study to validate the PGI-I as a PRO measure to surgery for BOO. This suggests a potential for the PGI-I to be used to assess surgical therapies for BPH and may be a valuable addition for measuring outcomes in clinical trials evaluating surgical interventions for BPH.

INTRODUCTION

Lower urinary tract symptoms (LUTS) is a common condition in men with a prevalence of bothersome symptoms reported in 30%–50% [1,2]. LUTS suggestive of benign prostatic hypertrophy (BPH) is associated with a lower level of overall health related quality of life (HRQL) [3-5]. HRQL decreases as severity of LUTS increase [4,5]. Currently treatment success for this condition is limited to comparing symptoms scores and isolated quality of life (QoL) indexes. Several international committees in charge of establishing standards for measuring outcomes following intervention for LUTS however have recommended documenting the patients, self-reported, impact of treatment [6,7]. A patient’s perspective of clinical impact and treatment benefits can be recorded in questionnaires called patient reported outcomes (PROs).

Currently, there are a range of validated questionnaires assessing symptoms and QoL for men with LUTS, the most recognised of which is the International Prostate Symptom Score (IPSS) also called the American Urological Association (AUA) BPH symptom score questionnaire. This questionnaire includes five questions on voiding symptoms (nocturia, frequency, etc.) and a single question on QoL. These questionnaires are aimed at assessing a patient’s symptoms at a particular point in time. Changes in scores overtime are used to assess outcomes following intervention but they are not designed to assess the patient’s perception of changes in symptoms postintervention.

PRO assessments are designed specifically to assess a patient’s perception of changes following treatment. A global index is a PRO assessment, which ranks patients change in symptom following intervention, in a way that is easy to use, compared and interpret. A global scale gives an overall appraisal of a patient’s perception of change following intervention. Currently, no global assessment instrument has been validated for assessing outcomes in the management of LUTS. The Patient Global Impression of Improvement (PGI-I) index is a possible candidate for this role. The PGI-I scale was originally modelled after psycho-pharmacological scales described in 1976 (Clinical Global Impression) [8]. The Clinical Global Impression-Improvement (CGI-I) scale is a 7-point scale that requires the clinician to assess how much the patient’s illness has improved or worsened relative baseline. The PGI-I scale used is the same but is completed by the patient (Table 1). It has been validated for use in female patients following intervention for both urinary incontinence and prolapse [9,10]. It has also been demonstrated to have excellent test-retest reliability [10]. Yalcin and Bump [9] altered the stem of the questionnaire for their patients with stress urinary incontinence but maintaining the response options. This study used their version of the PGI-I.

Table 1.

Patient Global Impression of Improvement

Check the one number that best describes how your urinary tract condition is now compared with how it was before your operation
1	Very much better
2	Much better
3	A little better
4	No change
5	A little worse
6	Much worse
7	Very much worse

The aim of this study was to validate the use of the PGI-I in men following surgical treatment of LUTS by correlating it with other outcome assessment measures.

MATERIALS AND METHODS

This project was part of a prospective, longitudinal, observational study which recruited men with significant LUTS whom underwent photoselective vaporization of the prostate (PVP), using the 120 W Greenlight laser (American Medical Systems, Minnetonka, MN, USA).

The indications for surgery were consistent with those described by both the European Association of Urology and AUA guidelines [11,12]. Once identified as candidates for surgery, the men had their flow rate (Qmax) and postvoid residuals (PVRs) measured by uroflowmetry and bladder scanner. Their preoperative IPSS and QoL scores were also recorded at this time. Preoperatively, prostate size was determined by transrectal ultrasound (TRUS) assessment using the ellipsoid method.

The men subsequently underwent greenlight laser prostatectomy using the 120 W lithium triborate laser.

Men were reviewed at both 6 weeks and 3 months. It was at the 3 month review, that the men were reassessed with the same measures but with the addition of the PGI-I. Three months was chosen as the best assessment point for two reasons. Firstly, it is at 3 months when postoperative, healing symptoms would have settled for the vast majority of men (dysuria, urgency, frequency, etc.), and men can be regarded as having reached treatment baseline. Secondly it is close enough for most men to maintain reasonable recall of their preoperative symptoms and more accurately assess how their symptoms have changed with treatment.

The IPSS, QoL, Qmax, and PVR results were compared from baseline to the 3-month follow-up using paired t-test and Wilcoxon signed rank test. The validity of the PGI-I was assessed by correlating the PGI-I response to changes in the other assessment tools. Pearson coefficient was used for correlations. Statistical significance was concluded when P≤0.05. Statistical analysis was performed with SPSS Statstics GradPac ver. 18 (IBM Co., Amonk, NY, USA).

RESULTS

One hundred sixty-six consecutive patients who underwent PVP were included. Thirty-two patients were excluded due to incomplete follow-up. Incomplete follow-up occurred in 19 patients because the PGI-I score was not recorded (reasons unclear), in a further 12 patients because they did not attend follow-up and one died prior to follow-up.

Twenty-four patients were in acute retention preoperative, with 19 having an indwelling catheter and 5 doing intermittent self-catheterisation. When presenting changes in flow rate and PVR and when correlating these results with PGI-I, patients in acute retention were excluded.

Mean age at surgery was 67 years (Table 2). Mean TRUS volume was 45 mL. The average preoperative IPSS score was 20 (range, 2–35), with 96% of patients having a score ≥8. Mean preoperative QoL score, Qmax, and PVR were 4.4, 8 mL/sec, and 169 mL, respectively (Table 2).

Table 2.

Demographics and preoperative results for patients undergoing PVP

Variable	Mean±SD
Age (yr)	67±9
IPSS score	20±7
QoL index	4.4±1
Qmax (mL/sec)	8±3.5
PVR (mL)	169±161
PV (mL)	45±33

PVP, photoselective vaporisation of the prostate; SD, standard deviation; IPSS, International Prostate Symptom Score; QoL, quality of life; Qmax, peak urinary flow; PVR, postvoid residual; PV, prostate volume.

At 3-month postoperative review the IPSS and QoL scores improved by an average of 11 and 2.4, to means of 9 and 2 respectively (P<0.0001) (Table 3). Qmax significant improved by an average of 14 mL/sec (P<0.0001), while PVR decreased to a mean of 36 mL.

Table 3.

Average change in results for patients 3 months post PVP

Variable	Preoperative mean	Postoperative mean	Mean±SEM	P-value
IPSS	20	9	11±0.70	<0.0001
QoL index	4.4	2	2.4±0.17	<0.0001
Qmax (mL/sec)	8	22	−14±0.96	<0.0001
PVR (mL)	169	36	133±16	<0.0001

PVP, photoselective vaporisation of the prostate; SEM, standard error of the mean; IPSS, International Prostate Symptom Score; QoL, quality of life; Qmax, peak urinary flow; PVR, postvoid residual.

The average PGI-I score was 2 (much better). The highest score was 6 (much worse). Overall 96% responded 3 or better (Table 4). The other 4% reported their symptoms were unchanged or worse. The patient who responded with a score of 6 reported this outcome despite showing improvement in both IPSS and Qmax.

Table 4.

PGI-I scores post PVP

PGI-I score	No. (%)
1	59 (44)
2	53 (40)
3	17 (13)
4	2 (2)
5	1 (1)
6	1 (1)

PGI-I, Patient Global Impression of Improvement; PVP, photoselective vaporisation of the prostate.

The mean change in IPSS score was 11 (standard deviation [SD], 8), QoL score was 2.4 (SD, 1.9), Qmax was 14 mL/sec (SD, 10), and PVR was 133 mL (SD, 160). All of these changes were statistically significant (Table 3).

Pearson coefficient was used to correlate changes in results to the PGI-I score. The PGI-I score correlated strongly with changes in IPSS and QoL, with Pearson correlations of 0.47 and 0.58, respectively (P<0.0001). A less but still significant correlation was observed with PVR, with a Pearson correlation of 0.21 (P=0.041) (Table 5). It did not correlate with changes in Qmax (P=0.56).

Table 5.

Correlation of PGI-I to change in IPSS, QoL, Qmax, and PVR 3 months following PVP

Variable	Mean change	Pearson correlation	P-value
IPSS	11	0.47	<0.0001
QoL index	2.4	0.58	<0.0001
Qmax (mL/sec)	−14	−0.06	0.560
PVR (mL)	133	0.21	0.041

PGI-I, Patient Global Impression of Improvement; IPSS, International Prostate Symptom Score; QoL, quality of life; Qmax, peak urinary flow; PVR, postvoid residual.

Changes in Qmax did not correlate with changes in QoL (P=0.91) but showed a weak correlation with changes in IPSS (Pearson coefficient=0.19, P=0.53). Changes in PVR did correlate with changes in QoL (Pearson coefficient=0.21, P=0.036) and changes in IPSS (Pearson coefficient=0.24, P=0.019).

DISCUSSION

After an operation, a patient will form an opinion as to whether or not the procedure was a success, that is, did it ‘cure’ or improve their symptoms. Unfortunately there is a paucity of data documenting these opinions following surgical intervention for LUTS, especially for PVP. This absence of information results directly from the fact that we have no reliable way of recording this information.

A surgeon assesses if treatment for BPH was successfully by measuring common symptom and sign parameters, for example flow rate and IPSS scores. Failure to record improvement in these parameters is usually regarded as treatment failure or a result from the development of treatment complications. Recognised complications following PVP include urethral stricture, bladder neck contractor and detrusor overactivity, all of which frequently impact on standard postoperative assessment tools.

A patient impression of success however may also be influenced by other, unmeasured factors. These include expectations regarding outcome, which can be shaped by previous personal experiences, experiences of friends and relatives and also the attitude of the clinician. It may also be influenced by the development of new symptoms not asked about in common questionnaires (incontinence and sexual dysfunction). Also long-term follow-up studies have revealed that patients expect surgery to help them achieve a specific goal and reaching this goal impacts on their opinion of successful treatment [13,14].

Consequently the concept of ‘cure’ should not be limited to a surgeon assessment of changes in symptom scores and objective measures. The patient’s self-reported impression of treatment success is equally as important. PRO assessments or PRO questionnaires, aim at documenting a patient’s impression of treatment success therefore give insight into whether their goal for treatment was achieved.

QoL assessment indications a patient’s current global status at a particular point in time. Improvement in QoL is frequently used as an indicator of effective treatment however there is little evidence to clarify what constitutes a significant or meaningful change in QoL scores [15]. For example, improvement in QoL, may come simply from a patient becoming more accepting of their urinary condition rather than any change in voiding behaviour occurring.

A PRO is an objective assessment of the patient’s subjective experience. PROs represent a clinical review of symptom impact and treatment benefit from a patient’s perspective. This can be measured independent of changes in symptoms scores and QoL indexes. The advantage of using a global index is that it provides a single best measure of change from the individual’s perspective. A global index can assess clinically meaningful change by taking into account more information than that reflected in the IPSS scores and QoL Index.

PROs are not just beneficial for the individual patient. Global assessment of treatment outcomes can be used for quality assurance. Clinical governance is a systematic approach to maintaining and improving the quality of patient care within a health system. PROs can be used to efficiently assess patient outcomes and rapidly signal if a deviation is occurring. Since these questionnaires are nonspecific, they have the ability to incorporate all factors which may be influencing patient perceived outcomes. When deviations are identified, early intervention can occur to help identify the cause and minimise the impact on patient outcomes, thus maintaining appropriate quality of care.

Unfortunately, there is currently no validated, easy to administer, PRO to assess outcome following surgery for BPH in patients with LUTS. The PGI-I was identified as a possible candidate.

The PGI-I scale was originally modelled after the 1976 psycho-pharmacological scale used in psychiatry called the CGI-I. The CGI-I is a 7-point scale, completed by the clinician, who rates how much they perceive a patient’s illness has changed following intervention. Later, the perspective of the CGI-I was to that of the patient, making it the PGI-I scale. This modified scale was then adopted to assess patient outcomes in other areas, including outcomes following hearing aid fitting and to evaluate the efficacy of therapy in the management of neuropathic pain.

In 2003, Yalcin and Bump [9] modified the stem question, making it suitable for patients with lower urinary tract conditions. They then established its validity in assessing female patients following surgery for stress urinary incontinence. They found the PGI-I to be a simple to administer and interpret questionnaire. Their PGI-I had the potential to be utilised as an alternative way of examining patient expectations following treatment for LUTS and allows objective assessment of a subjective phenomenon.

This study was aimed at assessing the validity of the PGI-I as a potential PRO for men undergoing PVP for LUTs. In order to assess the suitability of a certain PRO to assess outcomes following a particular treatment, it is critical to correlate it to known outcome success measures. The validity of the PGI-I for assessing outcomes following PVP for BPH therefore was tested by correlating it to the urinary parameters usually measured following surgery.

This study found that the PGI-I did correlate significantly with subjective urinary measures, particularly with improvement in IPSS and QoL scores. The mean improvement in IPSS was 11, and this correlated with a PGI-I score of 2: much improved. The mean QoL score also improved an average of 2 points, with most patients now scoring a total of 2: mostly satisfied with their urinary condition. Therefore an improvement in overall symptoms correlated significantly with patients reporting a positive outcome.

The PGI-I however did not correlate particularly well with changes in objective measures, in this case Qmax and PVR. This was observed despite the fact that improvement in Qmax and PVR did correlated with improvement in IPSS score. This is possibly a reflection of the fact that the improvement in Qmax and PVR is more likely to improve some of the urinary parameters measured by the IPSS (straining and intermittency) compared to others (nocturia and frequency). This means that improvement in Qmax and PVR can lead to an overall reduction in IPSS score by decreasing some urinary parameters while others remain unchanged. If the unchanged parameters (nocturia and frequency) are the ones that were the most bothersome to the patient then, understandably, this will be reflected in the PGI-I score. It would be valuable to see if there is a relationship between individual measure in the IPSS questionnaire and the PGI-I. Unfortunately, in this study only the overall IPSS score was recorded, therefore it is not possible to make this comparison. This study however does highlight the need for a PRO measure in addition to changes in symptom score.

This study was also not able to assess if the PGI-I score correlated with development of or changes with, urinary or other parameters not measured in the IPSS score. These parameters include incontinence, dysuria and erectile function. It was interesting to observe that the one patient who responded with a PGI-I score of 6 (much worse) had a measured improvement in both IPSS and Qmax results. This suggests that the patient experienced a deterioration or development of symptoms not recorded in the IPSS questionnaire. This is an area that warrants investigation in future studies.

In conclusion, PROs following treatment intervention for LUTs due to BPH are proving to be an important, independent measure of treatment success. Currently there is little data to guide us toward an appropriate PRO to use in patients undergoing endoscopic prostatectomy. The PGI-I, a known PRO, has been previously validated for use in women following surgery for prolapse and stress urinary incontinence. This study helps support the use of the PGI-I as an assessment tool following PVP for BPH. We successfully demonstrated its validity in this setting by identifying its strong correlation with changes in the IPSS score and QoL index. In addition, since the PGI-I is easy to administer and interpret, it has the potential to be a valuable addition, not only in clinical practice, but also in trials, to help compare interventions in the management of LUTS.