Analysis of rating appropriateness and patient outcomes in cataract surgery.

PURPOSE: Using the RAND/UCLA methodology to create the appropriateness criteria, we assessed that the appropriate ratings in cataract surgery can be a better prognosis of outcomes in postoperative 12 months than uncertain or inappropriate ratings. In addition, we identified the degree of the appropriate rating surgery associated with the outcome changes in postoperative 12 months.
MATERIALS AND METHODS: The patients in this study were followed up prospectively in preoperative and postoperative 12 months periods. The 20 ophthalmologists in 14 hospitals were asked to refer about 20 patients who were scheduled to undergo cataract surgery from March and June of 1997. A multiple regression analysis was used to identify the degree of the appropriate surgery associated with the changes of outcomes. The outcomes were designed as the clinical and functional outcomes (visual acuity, visual function, satisfaction with vision, and satisfaction with overall care).
RESULTS: The outcome changes of vision acuity (p < 0.001), vision function-14 (p < 0.001), and symptom score (p < 0.006) were significantly different between four appropriateness ratings (crucial, appropriate, uncertain, and inappropriate). There was a trend that the appropriate rating surgeries were related to the successful change of the vision function (2.29, p = 0.015) and satisfaction with vision (3.84, p = 0.014) in 12 month postoperative period.
CONCLUSION: The crucial or appropriate rating surgeries may indicate better outcomes than uncertain or inappropriate rating surgeries do. The appropriate rating surgeries were more closely related to functional outcome vision function, VF-14 and subjective outcome (satisfaction with vision) in postoperative 12 months than inappropriate rating surgeries.

INTRODUCTION

"Appropriateness of care" is the catch phrase of researchers who argue that identifying inappropriate care could lead to cost savings for the health care system. The appropriate consortium proposed a system that includes assessments of both overuse and underuse of services and evaluates care for acute and chronic conditions for all age groups in the population. In 1980s, the RAND/UCLA appropriateness methods (RAM) were developed to deal with the deficiencies of outcome data by asking experts to provide an assessment of the surgery's appropriateness after they have reviewed the available information.1-5 The RAM acts as a powerful decision making tool whether a procedure should or should not have been performed. Criteria for the appropriateness of seven services including cataract surgery, have been applied in published studies.4,6

Cataract surgery is one of the most frequently used surgical procedures in Korea: 235,340 surgeries were performed at a cost of 227 billion won in 2007,7 and 1.8 million surgeries were performed in US Medicare beneficiaries who were not enrolled in health maintenance organizations in 2004.8

The Agency for Health Care Policy and Research (now called the Agency for Healthcare Research and Quality) is mandated by law to investigate the "outcomes, effectiveness, and appropriateness" of health care services in 1994.1 "Outcomes research" is distinguished by its focus on using functional status, patient preferences, and other patient-centered information in evaluating the impact of health services.9,10 The US National Cataract Patient Outcomes Research Team (PORT) studied variations in preoperative, intraoperative, and postoperative cataract surgical management, and the result showed that the identified variation in treatment modalities partially affected patient outcomes.2,11,12

Since the researches on the relation between the appropriateness and outcomes research are limited, we undertook this study to assess whether cataract surgeries ratings as appropriate (crucial and appropriate) were associated more with changes in postoperative outcomes, such as visual acuity, visual function-14, symptom score, and satisfaction with overall, than uncertain or inappropriate cataract surgeries. These evidences contribute to validity of the RAM. Thus, the cataract patient's outcome differentiations were assessed by the four appropriateness ratings between preoperative and postoperative periods, and the degree of an appropriate surgery associated with the outcome changes was identified in postoperative 12 months. This study was based on the previous results which rated cataract surgery in Korea by the RAM.13

MATERIALS AND METHODS

Subjects

The methodology for the cataract appropriateness ratings has previously been published in detail,13,14 and we summarized these methods in the following text.

The case studies of 389 patients who underwent cataract surgery for either one or both eyes were examined. The surgeries were performed by 20 ophthalmologists who were practicing at 14 Universities or General hospitals from March to June in 1997. Two hundred eighty-one (72.2%) patients were followed up in postoperative 12 months period: 2 patients died and 106 patients changed the contact number or address. Also, 59 patients lacked of information, such as visual acuity, to fit any of the indications in the RAND/UCLA appropriateness ratings. Finally, 222 patients (57.1%) could be assessed for appropriateness by the indications. The exclusion criteria of the study populations were patients who had undergone cataract surgery or who had a combined procedure involving glaucoma, corneal, or viteo-retinal surgery. Deaf or confused patients were excluded. In this study, the influence of 14 institutions was not significantly associated with these study results (data not shown).

Patients were interviewed and clinical data were obtained. The trained surveyors visited the hospitals to interview the patients with cataract surgery scheduled. Patients were recruited consecutively until 30 patients had been enrolled from each practice. Patients were informed that, if they were enrolled, they would be asked to undergo a 30-40 minute interview before their cataract surgery and in postoperative 12 months after their cataract surgery. Also, the clinical data were collected from self-entered abstraction forms by ophthalmologist. The clinical data included visual acuity, comorbidity, surgical methods, and so on. To verify the clinical data, we requested a copy of the patient clinical records from ophthalmologists. Each abstract was then reviewed by a physician expert in the particular procedure.

Data collection

Appropriateness Methods1,13

Appropriateness ratings in this study are defined as follows: 'crucial/necessity', 'appropriate', 'uncertain', and 'inappropriate'.

The Rand Corporation's Health Sciences Program used literature analysis and assessment by expert panels to evaluate the appropriateness or inappropriateness of performing procedures in a wide variety of specified clinical situations. An expert panel, after performing an extensive review of the literature, rated 2,905 clinical scenarios. The final list of clinical situations or 'indications' was divided into four chapters: 1) unilateral cataract without other ocular pathology, 2) bilateral cataracts without other ocular pathology, 3) unilateral cataract with other ocular disease and 4) bilateral cataracts with other ocular disease. Each indication included visual acuity in the eye which was to undergo surgery and in the contralateral eye, and the extent of the impairment of visual function.15,16 Visual Function was characterized by specific impairments, described by the following: 'patient experiences visual impairment from glare'; 'patient expresses difficulty with recreation', 'watching television, or reading due to vision'; 'patient experiences employment limitations due to vision'; 'patient expresses difficulty with Activities of Daily Living (ADLs)17 believed in part due to cataract' and 'nonspecified visual impairment and no impairment'. The ADLs included 'basic' activities, such as bathing, eating, dressing, shopping, and light housework. All functional impairments recorded in the patient record were entered into the study database.

The panel that rated the indications for cataract surgery was composed of nine physicians (5 ophthalmologists, 1 geriatrician, 1 family practitioner, and 2 internists). The panelists performed three separate rounds of indication ratings. In each round, each clinical scenario was rated on a scale from 1 to 9. The 'appropriate' indications had a median panel rating of 7, 8, or 9, which implied that the expected benefit of the procedure exceeded the risk by a sufficiently wide margin, which justified the risk. 'Inappropriate' meant that the indication had a median panel rating of 1, 2, or 3, signifying that the risks were deemed to exceed the benefits. The 'uncertain' indications were for procedures which had either a median rating of 4, 5, or 6, or substantial disagreement among the panel members pertaining to the benefits and risks (i.e., three or more panelists rated the indication as appropriate and three or more rated it as inappropriate). At the end of round 3, 43% of the indications were rated appropriate: 36% inappropriate: 17% uncertain due to a median rating of 4, 5, or 6, and 4% uncertain due to disagreement.16 The entire list of indications was published in 1993.4

These scenarios, called indication in which cataract surgery might be performed, were ranked on a scale from 1 to 9; from 'inappropriate' to 'appropriate' respectively. Subsequently, clinical data of 222 cataract surgeries at 14 institutions were abstracted and entered into a computerized study database. Next, each surgery, in which sufficient data were available from the medical records, was assigned to a specific indication, by using the method that linked the patient data to the clinical scenarios. Appropriateness classification was based on the panel's rating of the indications, independently reviewed by the panel.

Vision function-14

The interview included the vision function-14 (VF-14) test - a reliable and valid function impairment test for patients with cataracts.2,15 The VF-14 is an index that measures the amount of difficulties patients have in performing 14 daily vision-dependent activities such as driving day and night, reading small print, watching television, and doing fine handwork. For each of the 14 items addressed by the index, a score of four was assigned to a patient who reported 'no difficulty' with the activities; a score of 3, 2, and 1 were assigned to a patient who reported 'a little', 'a moderate amount', and 'a great deal' of difficulties, respectively. Finally, a score of zero was assigned to a patient who was 'unable to do' the activi-ties because of his/her vision.

A patient's scores on all the activities, performed with no difficulties and not performed due to visual impairment, were averaged; yielding an average score between 0 and 4. The average score was then multiplied by 25, producing a possible final score, ranging between 0 (minimum vision) and 100 (maximum vision ability).

Visual acuity

Snellen18 visual acuity was obtained from the clinical records, and was assessed by the logarithms of minimum angles of resolution (LogMAR).1 Visual acuity of 20/1000 was assigned to a patient who could only count fingers, 20/2000 to a patient who could detect only hand motion, and 20/4000 to a patient who had light perception only or no light perception.

Satisfaction with vision

Patients were asked questions on the overall amount of satisfaction with their vision. They replied that they were 'very satisfied', 'moderately satisfied', 'moderately dissatisfied', and 'very dissatisfied' with their vision, scoring 100, 66.7, 33.3, and 0, respectively.

Symptoms with vision

Patients were asked whether they were bothered by any of the six symptoms that are commonly reported by cataract patients: double or distorted vision, halos or glare, blurry vision, disturbing brightness, color distortion, and worsening vision. Scores of 3, 2, and 1 were assigned to a patient according to the severity of the symptoms: 'very bothered', 'somewhat bothered', and 'slightly bothered', respectively. A score of 0 was assigned to a patient who did not have any of the symptoms or was not at all bothered by them. The patient's scores for each of the six symptoms were then summed, resulting in a score between 0 (no symptoms or not at all bothered by any of the symptom) and 18 (very bothered by all 6 symptoms).

Satisfaction with overall care

Satisfaction with Overall Care was divided into three areas: eleven questions relating to interpersonal care, eleven questions relating to physician explanations, and fourteen questions relating to satisfaction with hospital services. In total, there were 36 questions. The replies were rated as 'extremely satisfied', 'very satisfied', 'satisfied', 'moderately satisfied', and 'dissatisfied', to which the scores 100, 75, 50, 25, and 0 were given, respectively.

Provider characteristics

For ophthalmologists, years of experience were classified into two categories: 1-10 years and 11 years or longer. The annual volume of surgery was rated as 5-200 per year and 201 or more per year.

Subjective and relative health status

A patient's subjective and relative health status were measured by 5 levels: 'extremely best', 'very best', 'best', 'moderate', and 'worse'. The scores of each status were 100, 75, 50, 25, and 0, respectively.

Statistical analysis

This study was designed to compare the mean differences in cataract patient's outcomes between the appropriateness ratings in the preoperative and postoperative 12 months and to identify the degree of the appropriate surgery associated with the outcome changes in the postoperative 12 months.

The ANOVA and Duncan's multiple range tests were used to analysis the outcome changes between the four appropriateness ratings (crucial, appropriate, uncertain and inappropriate ratings). Multiple regression analysis was used to identify the degree of the appropriate surgery. The dependent variables were the changes of the visual acuity, VF-14, symptom scores, satisfaction with vision, and satisfaction with overall care. The independent variables included the appropriate surgery.

RESULTS

The results of the preoperative characteristics were statistically significant in the age (p = 0.016), education (p = 0.011) as patients' characteristics, operated eye LogMAR visual acuity (VA; p < 0.001), VF-14 (p < 0.001), symptom score (p = 0.003) as clinical and functional outcomes, other ocular disease (p = 0.041), and operated eye side VA (p < 0.001) between the four appropriateness ratings of crucial, appropriate, uncertain and inappropriate (Table 1). The appropriate rating surgeries (crucial or appropriate) showed operated eye VA, symptom score, VF-14, operated both eye and older age worse than uncertain and inappropriateness rating surgeries. Especially in the operated eye VA and symptom score, the crucial rating surgeries were significantly different from the other three categories.

Table 1

Comparisons of Preoperative Characteristics by Appropriateness Ratings

SD, standard deviation; Marital status-others, including separated and unmarried, VA, LogMAR visual acuity; VF-14, visual function-14; ECCE, extracapsular cataract extraction.

*Duncan's test: significant with uncertain and inappropriate ratings.

†Duncan's test: significant with appropriate, uncertain, and inappropriate.

The ANOVA test was used to assess the mean differences of the cataract patients' outcomes by the appropriateness ratings in the postoperative 12 months (Table 2). The outcome changes of vision acuity (p < 0.001), VF-14 (p < 0.001), and symptom score (p = 0.006) were statistically significant between the four appropriateness ratings. In rating group comparison analysis, the appropriate rating surgeries (crucial or appropriate) showed a trend of the outcome changes in both the operated eye VA and VF-14, compared with the uncertain or inappropriate rated surgeries. In the symptom score, the crucial rated surgeries were shown to be significantly different from the other three categories.

Table 2

Changes of Outcome between Preoperative and Postoperative Period of 12 Months (Mean ± SD)

SD, standard deviation; VA, LogMAR visual acuity; VF-14, visual function-14.

*Duncan's test: significant with uncertain and inappropriate.

Regarding the outcome changes between the preoperative and postoperative 12 months (Table 3), we defined the clinical outcome (VA), functional outcome (VF-14), cognitive outcome (symptom score), and subjective outcome (satisfaction with vision).

Table 3

Multiple Regression Analysis of the Association between Patient and Ophthalmologist Characteristics, Preoperative Visual Outcomes, Appropriate Surgeries, and 12 Postoperative Outcome Changes

VA, LogMAR visual acuity; VF-14, visual function-14, ECCE, extracapsular cataract extraction.

*p < 0.001, †p < 0.01, ‡p < 0.05.

The change of the VF-14 was related to the appropriate surgery (2.29, p = 0.015), and the preoperative VF-14 (-0.90, p < 0.001). The change of satisfaction with vision was associated with appropriate surgery (3.84, p = 0.014), preoperative satisfaction with vision (-1.04, p < 0.001), satisfaction with overall care (0.25, p = 0.001), and marital status (9.67, p < 0.006).

The change of the symptom score was related to preoperative symptom score (1.03, p < 0.001), and that of the visual acuity was related to the preoperative visual acuity (0.88, p < 0.001), other ocular disease (-0.22, p = 0.008), and surgical methods (-0.33, p = 0.006).

DISCUSSION

In elective surgery with high demand and limited resources, appropriateness and outcomes of health care interventions are important issues for the patients' quality of life.19 Using the RAND/UCLA appropriateness methods (RAM), we found that the outcome changes in the postoperative 12 months was associated with appropriateness rating surgeries more than inappropriateness rating surgeries, and identified the indicators of better outcomes.

The previous study,13 on which this study was based, rated the cataract surgeries as 'crucial', 'appropriate', 'uncertain', and 'inappropriate' according to the RAND/UCLA cataract surgery ratings in Korean surgical cases. Cataract procedures, defined as 'appropriate' surgery (77.0%) were classified as 'crucial (30.6%)' and 'appropriate (46.4%)'; those defined as 'inappropriate' surgery (23.0%), were classified as 'uncertain (15.3%)' and 'inappropriate (7.7%)'.

The "Cataract Patient Outcomes Research Team (PORT)" assessed variations in preoperative, short-term, and long-term outcomes of surgeries.2,12 The PORT ultimately pursued the pure outcomes rather than the clinical outcomes.11 To assess the patients' pure outcomes in this study, we followed long-term outcomes of patients for their stable visual status up to 12 months after surgery.

The present result demonstrated that preoperative appropriateness ratings (crucial or appropriate) significantly changed postoperative 12 month outcomes such as vision acuity (p < 0.001), VF-14 (p < 0.001), and symptom score (p = 0.003). Tobacman et al.16 examined the association between preoperative cataract appropriateness and measurement of visual acuity in postoperative period of 2 to 4 months, and found that 89% of the appropriately rated surgery patients had their visual acuity improved after surgery, whereas uncertain (68%) and inappropriate (36%) rated surgery patients were observed. In a prospective study, Hemingway et al.20 reported a strong relationship between median appropriateness score and clinical outcomes for Coranary artery bypass graft (CABG) and percutaneous translunminal coronary angioplasty (PTCA) These results demonstrate significant relationships between appropriateness ratings and postoperative surgical outcomes. Therefore, the RAM helps to clearly distinguish beneficial situations for decision making.

In multiple regression analysis, the appropriately rated surgeries (crucial and appropriate) were shown to be associated with VF-14 as the functional outcome and satisfaction with vision as the cognitive outcome between the preoperative and postoperative 12 months periods.

Previous researches have identified outcome indicators of the cataract surgery without appropriate rating categories. Applegate et al.17 reported that the most appropriate indication for surgical procedures was a visual function disability which was attributable to the presence of cataracts. Schein et al.21 reported that the lower level VF-14 was the predictor of better surgical outcomes, which were defined as improvements in one or more of the measures such as visual acuity, symptom score, or VF-14 in 4 months after surgery. Positive outcome is indicated by improvement in visual acuity and vision-related quality of life.2 Reduced self-rated health and health related quality of life (HRQOL) have both been associated with visual impairment.22 The outcomes eventually pursue the whole patient outcome such as self-rated health and HRQOL. These results indicate the need to consider VF-14 and satisfaction with vision as indicators for the appropriate rating surgery.

This study has a limitation in application because the study was conducted only at university hospitals and general hospitals. The modification of RAM for Korean situation is, therefore, suggested in future studies. The appropriateness rating method can be applied either prospectively or retrospectively in recent health care systems. Prospectively, appropriateness rating measures can be computerized, so that physicians and patients could be accessible during decision making. Retrospective use can identify problem indications or physicians with patterns of inappropriate care.

In summary, lower level of vision acuity, VF-14 and symptom score in preoperative times were the indicators of appropriate cataract surgeries. Appropriate rating surgeries (crucial and appropriate) were closely related to the outcomes about functional outcome (VF-14) and the subjective outcome (satisfaction with vision) in postoperative 12 months more than uncertain or inappropriate rating surgeries.