The unruptured intracranial aneurysm treatment score: a multidisciplinary consensus.

OBJECTIVE: We endeavored to develop an unruptured intracranial aneurysm (UIA) treatment score (UIATS) model that includes and quantifies key factors involved in clinical decision-making in the management of UIAs and to assess agreement for this model among specialists in UIA management and research.
METHODS: An international multidisciplinary (neurosurgery, neuroradiology, neurology, clinical epidemiology) group of 69 specialists was convened to develop and validate the UIATS model using a Delphi consensus. For internal (39 panel members involved in identification of relevant features) and external validation (30 independent external reviewers), 30 selected UIA cases were used to analyze agreement with UIATS management recommendations based on a 5-point Likert scale (5 indicating strong agreement). Interrater agreement (IRA) was assessed with standardized coefficients of dispersion (vr*) (vr* = 0 indicating excellent agreement and vr* = 1 indicating poor agreement).
RESULTS: The UIATS accounts for 29 key factors in UIA management. Agreement with UIATS (mean Likert scores) was 4.2 (95% confidence interval [CI] 4.1-4.3) per reviewer for both reviewer cohorts; agreement per case was 4.3 (95% CI 4.1-4.4) for panel members and 4.5 (95% CI 4.3-4.6) for external reviewers (p = 0.017). Mean Likert scores were 4.2 (95% CI 4.1-4.3) for interventional reviewers (n = 56) and 4.1 (95% CI 3.9-4.4) for noninterventional reviewers (n = 12) (p = 0.290). Overall IRA (vr*) for both cohorts was 0.026 (95% CI 0.019-0.033).
CONCLUSIONS: This novel UIA decision guidance study captures an excellent consensus among highly informed individuals on UIA management, irrespective of their underlying specialty. Clinicians can use the UIATS as a comprehensive mechanism for indicating how a large group of specialists might manage an individual patient with a UIA.

Unruptured intracranial aneurysms (UIAs) are prevalent in 3% of the adult population and are increasingly detected due to more frequent cranial imaging.[1,2] Previous cohort studies found that only a small proportion of UIAs rupture, causing subarachnoid hemorrhage (SAH) with a significant case fatality rate.[3-9] Although partially biased by selection of specific subgroups of UIAs, these studies found that especially small UIAs generally have a low risk of rupture. However, because small UIAs are so highly prevalent, most instances of SAH are caused by rupture of small aneurysms, but it remains unclear which small UIAs are prone to rupture so that preventive treatment could be considered. Case-control studies have suggested factors (e.g., UIA morphology, family history, smoking, or hypertension) that may increase the risk of UIA rupture, but uncertainties remain.[10-13] This is further complicated by the fact that there are only a few established risk factors for complications during aneurysm repair (e.g., patient age), even though numerous factors have been suggested.[14,15] Thus, data with varying levels of evidence must be taken into account when counseling patients with UIAs, which may lead to high levels of variation in the management of UIA among clinicians.[16,17] Our aim was to (1) develop a UIA treatment score (UIATS) that explicitly summarizes and quantifies recently reported consensus data on factors UIA specialists consider for appropriate management of UIAs, and (2) assess agreement with management recommendations based on the UIATS among specialists who were and were not involved in its development.[18]

METHODS

Delphi consensus.

A multidisciplinary group of 69 cerebrovascular specialists consisting of 43 neurosurgeons (11 of whom were trained in both neurosurgical and endovascular aneurysm repair), 14 interventional neuroradiologists, 11 neurologists, and 1 clinical epidemiologist was convened to participate in a Delphi consensus on contemporary UIA management (figure 1 and figure e-1 on the Neurology® Web site at Neurology.org). The data from the first 4 rounds of the 7-step Delphi consensus process derived from Web-based surveys (SurveyMonkey, Palo Alto, CA) and identified and rated the most relevant features used to assess and manage UIAs.[18] These data were used to develop and then to validate the UIATS model in rounds 5–7. The consensus group consisted of 2 cohorts: 39 of the 69 specialists (panel members) participated in the first 6 rounds to develop and validate the scoring model, whereas the remaining 30 specialists (external reviewers) who did not participate in the development of the scoring model reviewed the cases and UIATS-derived management recommendations for external validation. The characteristics of the 39 panel members have been previously reported.[18] The 30 external reviewers consisted of 16 neurosurgeons (7 of whom had dual training in endovascular and surgical aneurysm repair), 7 interventional neuroradiologists, and 7 neurologists. Two authors (N.E. and K.B.) who did not participate in the Delphi process designed the survey rounds, compiled the representative cases, and developed the UIATS with input from selected panel members. Definitions (see below) and preferred categories (e.g., for aneurysm size and patient age range) to estimate relevance for risk of UIA rupture or treatment were determined based on mean frequencies of agreement for results in rounds 2, 3, and 6. A participant frequency of more than 70% was considered sufficient to complete a round or to accept a definition or category.

Figure 1

Study flow of the Delphi consensus process

Participant frequencies for each round are given in parentheses. The panel member group consisted of 28 neurosurgeons (5 of whom were dually trained in endovascular and microsurgical aneurysm repair), 7 interventional neuroradiologists, 3 neurologists, and 1 clinical epidemiologist. The external reviewer group consisted of 15 neurosurgeons (7 of whom were dually trained in endovascular and microsurgical aneurysm repair), 7 interventional neuroradiologists, and 8 neurologists. CV = cerebrovascular; UIATS = unruptured intracranial aneurysm treatment score.

The development and validation of the UIATS model are described in the e-Methods section (tables e-1 and e-2).

Definitions.

The UIATS model does not apply to any nonsaccular aneurysms such as infective, traumatic, fusiform, or dissecting aneurysms; flow-related aneurysms associated with arteriovenous malformations; the rare saccular aneurysms associated with specific diseases/entities (e.g., collagen disorders, dwarfism, moyamoya syndrome); and aneurysms in patients younger than 18 years, since such aneurysms are rare, may have a distinct natural history, and may markedly alter the complexity of the score. The following definitions for individual items are presented in order of appearance in the UIATS model.

Patient domain.

Familial aneurysms were considered to be present in families in which 2 or more first-degree relatives were diagnosed with a UIA or SAH previously.[19] Current cigarette smoking was defined as a risk factor for adults who had smoked 100 cigarettes in their lifetime and smoked cigarettes every day (daily) or some days (nondaily) at the time of clinical presentation.[20] Hypertension, whether treated or untreated, was defined as systolic blood pressure greater than 140 mm Hg and/or diastolic blood pressure greater than 90 mm Hg.[21] Current drug or alcohol use was defined as a risk factor in those with recent (within 1 year of clinical presentation) cocaine or amphetamine exposure or heavy alcohol consumption (>300 g ethanol weekly). Clinical or radiologic signs of mass effect were defined as any symptoms or findings indicative of a global or focal space-occupying effect from a UIA, such as progressive headaches without SAH, nausea, vomiting, focal neurologic or radiologic findings (midline shift and/or herniation, edema, and cranial nerve compression). Cranial nerve deficits were defined as any neurologic deficits from compression of a cranial nerve, e.g., visual disturbances, oculomotor dysfunction for anterior circulation UIAs, and/or lower cranial nerve deficits (IX, X, or XI) for posterior circulation UIAs. Chronic or malignant disease was defined as any cardiovascular, pulmonary, renal, gastrointestinal, musculoskeletal, or CNS disorder of nonmalignant or malignant etiology that impaired the patient's life expectancy. Neurocognitive disorders were defined as Alzheimer disease or other etiologies for dementia that result in impairment of the patient's ability to live alone. Concomitant coagulopathies were defined as hemophilia and von Willebrand disease. Concomitant thrombophilic diseases were defined as factor V Leiden mutation, antiphospholipid antibody syndrome, antithrombin III deficiency, or protein C/S deficiency. Psychiatric disorders were defined as clinical depression, bipolar affective disorder, schizoaffective disorder, or obsessive-compulsive disorder resulting in impairment of the patient's ability to live alone.

Aneurysm domain.

Aneurysm size was defined as the greatest aneurysm diameter, measured using 3-dimensional reconstruction of the catheter angiograms. Aneurysm lobulation was defined as irregular daughter sac–like protrusion(s) of the aneurysm wall on the 3-dimensional angiographic reconstruction images. Aspect ratio was defined as the ratio of aneurysm dome dimension and neck width.[13] Size ratio was defined as largest aneurysm diameter divided by parent artery diameter.[22]

Treatment domain.

In addition to all measures listed under conservative management, treatment (aneurysm repair) was defined as any type of surgical or endovascular repair of a UIA. Conservative management was defined as clinical and/or radiologic follow-up and any medical treatment (e.g., for hypertension or epilepsy) or modification of lifestyle risk factors (e.g., cessation of smoking, cessation of alcohol or drug consumption). In case of effective normalization of hypertension or effective cessation of lifestyle risk factors for at least 1 year at follow-up, such risk factors do not need to be further considered at the subsequent evaluation using the UIATS. A complex aneurysm was defined as an aneurysm of any size that also had any of the following features: wide neck (greater than the diameter of the parent artery), significant lobulations, calcifications, intra-aneurysm thrombus, proximal vessel tortuosity/stenosis, branch artery incorporated into the neck or aneurysm sac, and very small aneurysm diameter (<3 mm).

Statistical analysis.

Wherever relevant, significance was accepted at the level of p < 0.05. A Wilcoxon signed-rank test for related samples was used to compare agreement in round 5 with agreement in round 6 to identify relevant changes in agreement following adjustment of the UIATS after round 5. Significant differences in the agreement with the UIATS between the panel members and the external reviewers per case, per reviewer, and by specialty (interventional [neurosurgery, interventional neuroradiology, or both] vs noninterventional [neurology or clinical epidemiology]) were then identified using the Mann-Whitney U test based on mean Likert scores. Means and 95% confidence intervals (CIs) were chosen instead of medians to illustrate variability in agreement per case, per reviewer, and by specialty. Standardized quartile coefficients of dispersion (vr*) were calculated to determine the degree of interrater agreement for every case.[18] vr* Values approaching 0 correspond to a high degree of interrater agreement, whereas vr* values approaching 1 correspond to a low degree of interrater agreement. The Pearson' product-moment correlation coefficient was calculated to analyze the potential association between the level of agreement among panel members or external reviewers and the score magnitude. The score magnitude, defined as the absolute difference between the UIATS numerical values supporting “aneurysm repair” and “conservative management” for each case (a small score magnitude suggests a less definitive UIATS recommendation), was used to analyze the relation of the strength of a UIATS-derived recommendation and the level of agreement among the reviewers.

RESULTS

The study flow and participant frequencies during the Delphi consensus process are given in figure 1. The UIATS model was developed based on the data from rounds 1–4 in 3 domains (patient-, aneurysm-, and treatment-related), comprising 13 different categories and 29 different features (figure 2).

Figure 2

The unruptured intracranial aneurysm treatment score

The unruptured intracranial aneurysm treatment score (UIATS) model includes and quantifies the key factors for clinical decision-making in the management of unruptured intracranial aneurysms (UIAs), developed based on relevance rating data from Delphi consensus rounds 1–4.[18] To calculate a management recommendation for a UIA, the number of points corresponding to each patient-, aneurysm-, or treatment-related feature on both management columns of the scoring form (“in favor of UIA repair” and “in favor of UIA conservative management”) are added up. This will lead to 2 numerical values, 1 favoring aneurysm repair (surgical or endovascular), and 1 favoring conservative management. The definitions for each category and factor are found in the Methods section. For cases with a score difference of 3 points or more, the direction, i.e., the difference between the calculated numerical values on each side of the recommendation columns, will suggest an individual management recommendation (i.e., aneurysm repair or conservative management). For cases that have similar aneurysm treatment and conservative management scores (±2 point difference or less), the recommendation is “not definitive” and either management approach could be supported, as additional factors apart from those used in the development of UIATS may be considered in making a final decision regarding the management recommendation and long-term follow-up. For cases with multiple aneurysms, every aneurysm must be evaluated separately, which will then also result in separate recommendations for each aneurysm. *The minimal intervention-related risk is always added as a constant factor (5 points). AComA = anterior communicating artery; BasA = basilar artery; BP = blood pressure; multiple = multiple selection category; PComA = posterior communicating artery; SAH = subarachnoid hemorrhage; single = single selection category.

The applicability of the preliminary UIATS model was initially tested in round 5. Mean agreement with UIATS-derived recommendations based on Likert scores (5 indicating strong agreement and 1 indicating strong disagreement) was 3.7 (95% CI 3.6–3.8) per panel member and 3.7 (95% CI 3.3–4.1) per case. After adjustment of the UIATS for age and aneurysm size, mean agreement with treatment recommendations based on the final UIATS model per reviewer was 4.2 (95% CI 4.1–4.3) for the panel members (p < 0.001 compared to round 5) and 4.2 (95% CI 4.1–4.3) for the external reviewers. Mean agreement per case was 4.3 (95% CI 4.1–4.4) for panel members (p < 0.01 compared to round 5) and 4.5 (95% CI 4.3–4.6) for external reviewers (figure 3A). Agreement per case was higher among external reviewers than among panel members (p = 0.017, Mann-Whitney U test). Dichotomized overall agreement (agree vs disagree) with treatment recommendations based on the UIATS was 95% among panel members and 94% among external reviewers. Interrater agreement per case was high for both reviewers' cohorts (vr* = 0.023, 95% CI 0.018–0.027 for panel members; vr* = 0.028, 95% CI 0.022–0.034 for external reviewers). Despite an unbalanced distribution of specialties among the consensus group, there was no difference (p = 0.290, Mann-Whitney U test) in mean Likert scores per reviewer between interventional (4.2, 95% CI 4.1–4.3 for neurosurgery, interventional neuroradiology, or both, n = 56) and noninterventional (4.1, 95% CI 3.9–4.4 for neurology and clinical epidemiology, n = 12) specialties or between individual specialties (p = 0.325, Mann-Whitney U test). There was a distinct correlation between the score magnitude of the recommendation (difference in points for interventional and conservative management) and the level of agreement, suggesting that the clearer the recommendation of the UIATS, the higher the level of agreement among the reviewers (figure 3B). A representative set of UIATS examples and corresponding UIATS recommendations resulting in the highest and lowest overall agreement are illustrated in figures 4 and 5, respectively.

Figure 3

Validation of the UIATS

(A) Agreement with unruptured intracranial aneurysm treatment score (UIATS)-derived recommendations per case and per rater. Means for Likert scores (y-axis) are illustrated for each case (dots) and for each rater (circles) among panel members and the blinded external reviewers (x-axis). A Likert score of 5 indicates strong agreement; 4 indicates agreement; 3 indicates neutrality; 2 indicates disagreement; and 1 indicates strong disagreement. Since means for Likert scores did not fall below a score of 3, the y-axis scale does not show disagreement and strong disagreement. Compared to agreement of panel members with UIATS-derived treatment recommendations for each case, the mean Likert score (y-axis) was significantly higher among the external reviewers, who were completely blinded to the score raw data and the survey development and design (*indicates p = 0.017). (B) Correlation of Likert scores and UIATS characteristics. The level of agreement (y-axis) between panel members and external reviewers was correlated (Pearson product-moment) with the UIATS differences between aneurysm repair and conservative management for every UIATS treatment recommendation (x-axis). The score magnitude correlated significantly with agreement among the reviewers, independently for panel members (r2 = 0.323, p = 0.002; solid line) and external reviewers (r2 = 0.399, p < 0.001; dotted line).

UIATS case recommendation resulting in the highest agreement among the internal and external reviewer cohorts

Corresponding scores in accordance with unruptured intracranial aneurysm treatment score (UIATS) features are given in parentheses. A catheter angiogram of a 47-year-old woman (3 points favoring treatment for patient age 41–60 years) who previously underwent cranial MRI for chronic headaches with (A) posterior-anterior and (B) lateral projection as well as (C) posterior-anterior and (D) lateral 3-dimensional reconstructions is shown. This incidental irregularly shaped (3 points favoring treatment for irregular morphology) anterior communicating artery aneurysm (arrow, 2 points favoring treatment for aneurysm location) had a maximum diameter of 7.6 mm (2 points favoring treatment for aneurysm diameter) with a neck diameter of 3.5 mm. Aspect and size ratios were calculated to be 2.1 and 3.8, respectively (1 point favoring treatment for aspect or size ratio greater than 1.6 or 3.0, respectively). Her medical history included arterial hypertension (2 points favoring treatment for risk factor hypertension) but no other chronic comorbidities. The resulting scoring based on the UIATS was 13 points in favor of aneurysm repair and 7 points in favor of conservative management (1 point for patient age 41–60 years, 1 point for aneurysm size 6–10 mm, and 5 points for the constant intervention-related risk). The resulting UIATS recommendation was “aneurysm repair.” Overall agreement with this UIATS recommendation was 4.73 (95% confidence interval 4.62–4.85) for both reviewer cohorts.

UIATS case recommendation resulting in the lowest agreement among the internal and external reviewer cohorts

Corresponding scores in accordance with unruptured intracranial aneurysm treatment score (UIATS) features are given in parentheses. A catheter angiogram of a 25-year-old woman (4 points favoring treatment for patient age younger than 40 years) who previously underwent MRI for persistent headaches, vertigo, and occasional bitemporal vision disturbances (due to a migraine aura) with (A) posterior-anterior and (B) lateral projection as well as (C) posterior-anterior and (D) lateral 3-dimensional reconstructions is shown. This left-sided wide-necked paraophthalmic internal carotid artery aneurysm had a maximum diameter of 3 mm with a neck diameter of 2.7 mm (3 points favoring conservative management for aneurysm complexity due to wide neck). The patient did not report any additional risk factors or comorbidities in her past medical history. The resulting scores based on the UIATS were 4 points in favor of aneurysm repair and 8 points (including 5 points favoring conservative management for the constant intervention-related risk) in favor of conservative management. The resulting UIATS recommendation was “conservative management.” Overall agreement with this UIATS recommendation was 3.7 (95% confidence interval 3.44–3.96) for both reviewer cohorts.

DISCUSSION

The key finding of our consensus project among a large and diverse group of multidisciplinary cerebrovascular specialists is that we were able to develop a comprehensive scoring model for management recommendations for UIAs. Importantly, this model is in high agreement with current UIA decision-making in a selected UIA patient population, evidenced by the high agreement and interrater agreement with UIATS-derived recommendations among the specialist group. The level of agreement with management recommendations based on the UIATS model is independent from the underlying professional background of the specialists, i.e., interventional or noninterventional, and is even higher among specialists who were not involved in the development of the scoring model.

The UIATS is neither a prognostic study nor a predictive model for UIA rupture, as it is derived from consensus on contemporary practice of UIA management among cerebrovascular specialists using the Delphi method and only indirectly from published data. In that sense it differs from the recently developed PHASES score, a model based on prospectively collected data from 6 cohort studies on risk of UIA rupture that provides absolute risks of rupture for the initial 5 years after aneurysm detection using 6 easily retrievable baseline characteristics (patient geographical location and age, aneurysm size and location, presence of arterial hypertension, and previous SAH from a different aneurysm).[23] However, some subgroups in the PHASES score were underrepresented, such as patients with familial aneurysms, or may have been underrepresented, such as young smokers. Thus, the score may not apply to all patients. Also, the score holds true for only the initial 5 years after UIA detection because of limited long-term follow-up data, and many patients have a predicted life expectancy of more than 5 years. Moreover, several aneurysm characteristics found or suggested to be risk factors for rupture in case-control studies could not be included in the PHASES score. Finally, the PHASES score was developed to predict the risk of rupture, whereas when a clinician recommends repair of a UIA, the risk of intervention must also be taken into account, which is not done in the PHASES score. Thus, many uncertainties due to varying levels of evidence remain, which have to be accounted for in clinical practice when consulting patients with UIAs.

The UIATS model was designed to address these uncertainties and thus to potentially harmonize the high level of variation among clinicians about the individual management of patients with UIA. The UIATS was developed partially based on data but also using consensus among specialists in the field. The merit of this model is that it accounts for many different factors that often influence clinical decisions but that were not well studied in previous observational studies, e.g., young age or long life expectancy, coexistent modifiable or nonmodifiable risk factors, coexistent morbidities, morphologic UIA features, or relevance of clinical symptoms related to UIAs.[5,10,11,13,24,25] Admittedly, the UIATS model requires more baseline characteristics, than the PHASES score, which makes its application slightly more time consuming. However, our results highlight that specialists in UIA research and treatment account for these factors in their decision-making and that these specialists have a high level of agreement about how to handle these factors for which data are currently incomplete or lacking. We used the Delphi method to scientifically obtain consensus and subsequently to systematically categorize potential factors for contemporary decision-making on the appropriate management of UIAs. Studies from other medical fields have used the Delphi method to systematically reach consensus or develop treatment scores on similar controversial or complex subjects among professionally and/or geographically dispersed specialists.[26-28] We emphasize that data derived from consensus among specialists cannot replace evidence but rather can complement it, specifically in areas where there is uncertainty. If more observational data become available and predictors for aneurysm rupture and treatment complications become more sophisticated, the UIATS model could be adjusted. However, until such data become available, our proposed scoring model constitutes an organized and objective means of capturing the best consensus possible on UIA management as a complement to existing UIA rupture risk prediction models.

Our study has several limitations. Although we included a wide variety of multidisciplinary specialists from various predominantly high-income Western countries with different health care systems, we cannot claim that this “population” of specialists is representative of the general “community of UIA specialists or experts,” if such a group could be defined. Second, one important premise in the construction of the UIATS is that the management of UIAs constitutes a comparison between the risk of rupture vs the risks of treatment in that particular patient. Some have argued that these 2 “quantities,” one being a risk event rate and the other being a one-time risk, cannot validly be compared.[29] However, in daily practice, this is what clinicians do, i.e., compare the risk of aneurysm rupture with the risk of prophylactic aneurysm repair. Third, we used a binary decision scenario (treat or not treat) rather than making distinctions between different possible treatment modalities, mainly to propose a unifying scoring model that would not divide the specialist group. Fourth, we used pooled data from meta-analyses to incorporate treatment risk percentages into our model. However, given the lack of more robust data on risk factors for UIA treatment complications, it is unclear whether the individual treatment risk in a patient with a UIA may be distinctly lower or higher depending on the surgeon's or interventionalist's experience, the treatment modality, and the aneurysm complexity. Lastly, the series of cases we validated with the UIATS covered a wide range of patients and aneurysms but may not entirely reflect the complete spectrum of patients and aneurysms seen in clinical practice. As an example, in the validation data set, 13 of 30 UIAs were larger than 7 mm, which may not reflect the actual UIA size distribution in the general population. In addition, the presentation of the actual score values to the external reviewers during the validation may have introduced some bias if these external experts were somewhat uncertain and therefore more willing to agree with the proposed recommendations. However, given the expertise of the external reviewers we do not feel this has influenced the results to a large extent.

This multidisciplinary consensus project has resulted in the UIATS model, which captures contemporary and multifactorial decision-making by specialists on UIA management. By applying this scoring model, clinicians can appreciate what highly informed individuals in the cerebrovascular field would advise in a particular patient based on current data and uncertainties. Since the validity of the UIATS model was tested based on theoretical rather than empirical data, its applicability and clinical accuracy remain to be prospectively explored in patients with UIA. Thus, the current treatment recommendations may change if new data from such studies or future observational UIA studies become available. Ultimately, individual patient factors beyond those accounted for in the present scoring model may also alter the individual management of a patient with a UIA.