Estimating the risk of developing glaucoma.

The issue of risk assessment in glaucoma has received increasing attention in the past few years since the publication of results from the Ocular Hypertension Treatment Study. Predictive models have been developed in order to estimate the risk that patients with ocular hypertension will develop glaucoma if left untreated. The purpose of this article is to review issues on the development and validation of predictive models to estimate risk of glaucoma development. Current models are reviewed and details about their development and validation are provided.

WHY RISK CALCULATORS ARE NECESSARY?

Risk calculators or predictive models are an essential part of health care. They have been continuously developed in several areas of medicine and their use has significantly contributed to the management of many disorders. The identification of risk factors for development or progression of disease is a fundamental component in the construction of these models. The concept of risk factor became widespread in medicine ever since the landmark Framingham Heart Study first reported in the early 1960s that cigarette smoking, elevated blood cholesterol, and high blood pressure were predictors of the likelihood of dying from heart disease. Framingham investigators then started developing predictive models to evaluate the global risk of cardiovascular disease based on the summation of all major risk factors [1-3]. Risk assessment and prevention has contributed significantly to reduce mortality from cardiovascular disease and its successful implementation in this scenario has stimulated its application to several other areas.

Recently, the concept of risk assessment has also been applied to ophthalmology, more specifically, for assessment of the risk of development of glaucoma in patients with ocular hypertension. It is estimated that approximately 8% of adults over the age of 40 years in the United States have ocular hypertension [4]. While ocular hypertension is a common finding, clinicians do not know which patients to treat or which patients to monitor without treatment. In 2002, the publication of the results of the Ocular Hypertension Treatment Study (OHTS) stimulated a reassessment of the ways in which to evaluate and manage patients with ocular hypertension [5, 6]. Since the OHTS publication, several strategies for risk assessment in ocular hypertension have been proposed and some have been successfully implemented in clinical practice. Several predictive models (or risk calculators) have been proposed and their use in clinical practice is likely to provide a more objective and evidence-based approach to the management of patients with ocular hypertension.

Although the information on individual risk factors may already help clinicians in management decisions, it is frequently difficult to integrate the information on the several risk factors and provide a global assessment for a particular patient. In that situation, predictive models or risk calculators may benefit clinicians in providing a more objective assessment of risk. Mansberger et al. [7] performed a survey of ophthalmologists to estimate their ability to predict the risk of glaucoma development in ocular hypertensive patients [7]. Ophthalmologists had the benefit of an oral review and written handouts summarizing the OHTS results. They found that ophthalmologists tended to underestimate the risk when compared to the actual risk found by a risk calculator. Ophthalmologists also had a large range of predictions, sometimes differing from the actual risk by 40%, illustrating the need for a more standardized method for risk assessment. In another study, Boland and colleagues demonstrated that the use of a risk calculator changed recommendations performed by clinicians and increased the measure of confidence in their recommendations, while also decreasing inconsistency [8].

DEVELOPMENT OF A PREDICTIVE MODEL TO ESTIMATE THE RISK OF GLAUCOMA DEVELOPMENT

The development of predictive models requires a series of complex steps which initially involve the acquisition and analysis of data from one or multiple longitudinal studies that have carefully followed patients over time. A critical step is the identification of the risk factors associated with the outcome one wants to predict. A few large, prospective, longitudinal studies have provided evidence with regard to the risk factors for conversion from ocular hypertension to glaucoma. From these studies, two were randomized clinical trials, The Ocular Hypertension Treatment Study (OHTS) [5], and the European Glaucoma Prevention Study (EGPS) [9]. These two studies have provided the basis for development and validation of the prediction models for glaucoma development available today. Both studies have evaluated a large number of predictive factors for their potential association with the risk of converting to glaucoma. When pooled analyses of the OHTS and EGPS data were conducted, only 5 baseline factors were identified as significantly associated with the risk of converting to glaucoma: age, intraocular pressure, central corneal thickness, the measurement of the vertical cup/disc ratio of the optic nerve and the visual field index pattern standard deviation (PSD) (Fig. ) [10].

Several other factors were not found to be statistically significantly related to the risk of conversion to glaucoma in the OHTS/EGPS pooled dataset, such as diabetes mellitus, history of heart disease and race, among others. It is important to emphasize, however, that even for the OHTS/EGPS combined dataset; the power of the study was probably not enough to detect a significant predictive value for many of the evaluated risk factors. Also, methodological weaknesses precluded a better investigation of the real value of potential risk factors, such as positive family history of glaucoma. As no relatives of the study subjects were examined, investigators had to rely on self-reported family history with its potential inaccuracy. It is likely that this contributed to the lack of association between family history and risk of glaucoma development as reported by these investigations.

After the information on risk factors is obtained, statistical methods can be applied to develop models for prediction of outcome using one or more explanatory variables. In 2005, we published the results on the development of a risk calculator to assess the risk of an ocular hypertensive patient to develop glaucoma [11]. The risk calculator was derived based on the results published by the OHTS [5, 12] and incorporated the variables that were described by that study as being significantly associated with the risk of developing glaucoma over time. The risk calculator was designed to estimate the chance of an ocular hypertensive patient to develop glaucoma if left untreated for 5 years. To simplify the use of the risk calculator, a point system and an electronic version of the calculator were made available for clinicians.

A predictive model that is derived from a particular dataset is not guaranteed to work on a different group of patients. In fact, the performance of regression models (or risk calculators) used as diagnostic or prediction tools is generally better on the dataset on which the model has been constructed (derivation set) compared to the performance of the same model on new data. Therefore, before risk calculators can be successfully incorporated into clinical practice they need to be validated on different populations. By validation we mean establishing that the risk calculator works satisfactorily for patients other than those from whose data the model was derived. Along with the steps involved in the development of the risk calculator, we also presented the results of its validation on an independent population of 126 patients with ocular hypertension who were followed as part of a prospective longitudinal study conducted at the University of California San Diego (DIGS – Diagnostic Innovations in Glaucoma Study).

Several steps were taken to validate the OHTS-derived model. In the first step, the importance of the prognostic variables that had been previously identified by the OHTS study was evaluated on the new data set (DIGS data set). All the variables had similar performance, except for diabetes mellitus, which was not significantly associated with the risk of developing glaucoma in the DIGS data. Subsequently, the predictive performance of the model was investigated on the new data set. The ability of the OHTS-derived risk calculator to discriminate DIGS subjects who developed glaucoma from those who did not was reasonably good with a c-index of approximately 0.7. The c-index is a measure of the discriminating ability of a model (similar to the area under the Receiver Operating Characteristic [ROC] curve) and a c-index of 0.7 indicates that, in approximately 70% of the cases, the model allocated a higher predicted probability for a subject who actually developed glaucoma than for a subject who did not. The closer the c-index gets to 1, the better the discriminating ability of the model. The values of c-index found for the OHTS-derived risk calculator when applied to DIGS subjects were similar to those found when risk models such as the Framingham coronary prediction scores are used to predict coronary heart disease events [2, 13]. D’Agostino et al. reported c-indexes ranging from 0.63 to 0.83 when the Framingham functions were applied to 6 different cohorts of patients [2].

The OHTS-derived risk calculator also had a good calibration when applied to the DIGS data set. Checking calibration is another important step in validating a predictive model. A reliable or well-calibrated model will give predicted probabilities that agree numerically with the actual outcomes. For example, let’s consider a group of 100 ocular hypertensive patients. If the model assigns an average probability of 12% for conversion to glaucoma for this group of subjects, it is expected that approximately 12 subjects will convert to glaucoma over time. That is, for a well-calibrated model, the predicted probabilities of conversion to glaucoma will agree closely with the observed probabilities of conversion. The OHTS-derived risk calculator performed well on the DIGS data set. For patients in whom the model predicted a high chance of converting to glaucoma, there was a high observed conversion rate; whereas for patients in whom the model predicted a low conversion rate, there was a low observed conversion rate.

In 2007, OHTS and EGPS investigators published results of the development and validation of a risk calculator for glaucoma based on the analysis of the combined OHTS/EGPS dataset [10]. The results were similar to the predictive model published in 2005, and the risk calculator contained the 5 variables significantly associated with the risk of glaucoma conversion: age, IOP, CCT, PSD and vertical cup/disc ratio. The risk model from the pooled OHTS/EGPS sample of over 1,100 ocular hypertension patients demonstrated excellent fit with a c-statistic of 0.74 and good calibration. The OHTS/EGPS risk calculator is available on the web at .

LIMITATIONS OF PREDICTIVE MODELS

The use of predictive models in clinical practice has several limitations. Predictive models are based on restricted populations of patients that were selected based on strict inclusion and exclusion criteria and that may not be representative of all patients seen at everyday clinical settings. Use of these models should be restricted to those patients who are similar to the ones included in the studies used to develop and/or validate it. It is also important to emphasize that although predictive models can provide a more objective evaluation of risk, their use does not replace the judgment of a clinician when making management decisions. For example, current risk calculators to estimate risk of glaucoma development do not include important information to guide treatment such as medical health status and life expectancy, patient’s willingness to treatment, costs of medications and overall effect of treatment on quality of life. Also, it is important to emphasize that current risk calculators for glaucoma have been designed to estimate the risk of development of the earliest signs of disease, which do not necessarily have an impact on the quality of vision of the patient. Finally, as more evidence regarding risk factors for disease development and progression accumulates, newer and better refined predictive models will be developed that should replace current existing ones.