Evaluation of pure-tone audiometric protocols in vestibular schwannoma screening.

The objective was to evaluate the pure-tone audiogram-based screening protocols in VS diagnostics. We retrospectively analyzed presenting symptoms, pure tone audiometry and MRI finding from 246 VS patients and 442 controls were collected to test screening protocols (AAO-HNS, AMCLASS-A/B, Charing Cross, Cueva, DOH, Nashville, Oxford, Rule3000, Schlauch, Seattle, Sunderland) for sensitivity and specificity. Results were pooled with data from five other studies, and analysis of sensitivity, specificity and positive likelihood ratio (LR+) for each protocol was performed. Our results show that protocols with significantly higher sensitivity (AMCLASS-A/B, Nashville) show also significantly lowest specificity, and tend to have low association (positive likelihood ratio, LR+) to the VS. The highest LR+ was found for protocols AAO-HNS, Rule3000 and Seattle. In conclusions, knowing their properties, screening protocols are simple decision-making tools in VS diagnostic. To use the advantage of the highest sensitivity, protocols AMCLASS-A + B or Nashville can be of choice. For more reasonable approach, applying the protocols with high LR+ (AAO-HNS, Rule3000, Seattle) may reduce the overall number of MRI scans at expense of only few primarily undiagnosed VS.

Introduction

Vestibular schwannoma (VS) is a rare benign tumor of internal auditory canal that is usually searched for as a possible cause of unilateral hearing loss. Contrast-enhanced magnetic resonance imaging (MRI) has high sensitivity for retrocochlear pathologies, and is currently considered the gold standard in VS diagnostics. Indications for MRI imaging include patient’s symptoms and especially audiological findings, as the most common symptom for VS is the asymmetric sensorineural hearing loss. Asymmetry in hearing, however, is very widely present in population also without a link to retrocochlear pathology (Pittman and Stelmachowicz, 2003; Urben et al., 1999). Thus, yield of incidental VS proved by MRI imaging is low, 2–8% (Cueva, 2004; Dawes and Jeannon, 1998; Saliba et al., 2011; Urben et al., 1999; in our dataset 3.5% - unpublished data). A number of audiometric screening protocols have been proposed to quantify the interaural asymmetry relevant for VS suspicion and indication for MRI imaging (Committee on Hearing, 1995; Cueva, 2004; Dawes and Jeannon, 1998; Hunter et al., 1999; Mangham, 1991; Margolis and Saly 2009; Neary et al., 1996; Obholzer et al., 2004; Saliba et al., 2009, 2011; Schlauch and Levine, 1995; Sheppard et al., 1996; Urben et al., 1999; Welling et al., 1990). Considerable effort has been spent to evaluate protocols on a single cohort level in order to identify the protocol with optimal sensitivity and specificity, however, with various results.

Recently, the congress of neurological surgeons recommended to perform MRI screening on patients with≥10 dB of interaural difference at 2 or more contiguous frequencies (AMCLASS-A; Margolis and Saly, 2009; Sweeny et al., 2018) or ≥15 dB at 1 frequency (AMCLASS-B; Margolis and Saly, 2009; Sweeny et al., 2018). Moreover, Saliba et al. (2009, 2011) showed the interaural difference ≥15 dB at 3 kHz (Rule3000) to be the most valuable in positive likelihood ratio (LR+) for VS. Based on these studies Rule3000 for more selective indication for MRI was alternatively recommended (Sweeny et al., 2018).

In the current study, audiograms of 246 VS and 442 control patients were analyzed for interaural asymmetry according to 14 published screening protocols: AMCLASS-A (Audiogram Classification System-A), AMCLASS-B, Charing Cross, DOH (Department of Health), Sunderland, Nashville, RULE3000, Mangham (≥10 dB), Schlauch, Oxford, Seattle, AAO-HNS (American Academy of Otorhinolaryngology-Head and Neck Surgery), Cueva, 3 × 15dB (Committee on Hearing, 1995; Cueva, 2004; Dawes and Jeannon, 1998; Hunter et al., 1999; Mangham, 1991; Margolis and Saly 2009; Neary et al., 1996; Obholzer et al., 2004; Saliba et al., 2009, 2011; Schlauch and Levine, 1995; Sheppard et al., 1996; Urben et al., 1999; Welling et al., 1990). Further, gained results were added to findings from five other studies (Cheng and Wareing, 2012; Gimsing, 2010; Nouraei et al., 2007; Obholzer et al., 2004; Saliba et al., 2011) and analysis of sensitivity, specificity, and LR+ of each protocol was performed.

Patients and methods

The study was approved by the institutional research board of the Helsinki University Hospital. Medical charts form 699 patients (257 VS cases and 442 controls) were reviewed. Patients were examined for VS suspicion due to unilateral hearing loss, tinnitus (not pulsatile) or vertigo in years 2001–2012 at Department of Otorhinolaryngology of Helsinki University Hospital, Helsinki, Finland. Patients did not have any other obvious ear pathology that could impair hearing ability asymmetrically. MRI imaging with contrast showed VS in 257 cases. In 4 patients VS was found bilaterally and in 7 patients VS was located at the side of asymptomatic (better) ear (these are as such incidental findings sui generis). These 11 cases were excluded from the study, leaving 246 suitable VS patients for the cohort. The control group for audiometric screening protocols sensitivity/specificity testing consisted of 442 patients with symptoms typical for VS (asymmetric hearing loss, vertigo, tinnitus), and in whom MRI imaging was negative for retrocochlear pathology (these patients were examined in years 2011–2012). In all the patients pure tone hearing thresholds for air and bone conduction for both ears at frequencies 125 Hz, 250 Hz, 500 Hz, 1 kHz, 2 kHz, 3 kHz, 4 kHz, 6 kHz and 8 kHz (125 Hz – 4 kHz for bone conduction) were measured. If a patient was unable to register tested frequency at maximal intensity due to a deep hearing loss, threshold was set 5 dB higher than the last tested intensity level to allow further analysis of the audiogram. Average hearing threshold values (AHT) were calculated as the average of thresholds in the frequency ranges according to the definition of the screening protocol, as noticed in text and tables. Interaural difference of hearing loss was calculated as the difference in AHT between the symptomatic and the better ear. Hearing loss was defined as asymmetric, if interaural difference was ≥10 dB in two neighbouring frequencies (protocol AMCLASS-A;Margolis and Saly, 2008) or ≥15 dB at any single frequency (AMCLASS-B; Margolis and Saly, 2008). Asymmetry as defined by all other protocols is greater interaural difference than for AMCLASS-A or AMCLASS-B. Positive likelihood ratio (LR+) was calculated according to the formula: sensitivity/ (1-specificity). Data in bar graphs are displayed as mean ± SEM. Data was analyzed using GraphPad Prism 5.03 (San Diego, CA, USA) software. To compare differences between groups Student’s t-test was used and differences were marked as significant for p < 0.05.

Results

Demography and tumor size

Medical charts from 246 patients with MRI proven VS were reviewed. There was a slight female to male predominance (1.16:1), the mean age was 56 ± 14 years (mean ± standard deviation, SD; range 16–86). Distribution of VS incidence within age groups at the time of diagnosis is displayed in Fig. 1. In the control group (442 patients) mean age (57 ± 15 y, range 16–88) and sex distribution (female to male ratio 1.24:1) were similar to VS group.

Fig. 1

Representation of age groups at the time of diagnosis. Total numbers on the left x-axis, relative representation on the right.

In this VS patient cohort the largest tumor dimension varied from very small to giant (1 mm–45 mm; median 13 mm; mean 15 mm). Tumor size distribution among cases was as follows: ≤10 mm, 96 tumors; ≤20 mm, 99 tumors; ≤30 mm, 30 tumors; ˃30 mm, 21 tumors.

VS patient symptoms

Among 246 VS patients asymmetric hearing loss (AMCLASS-A or AMCLASS-B, see Patients and methods) was found in 240 (97.5%) cases. Out of these 240 patients 137 (57%) reported additionally at least one of the following symptoms: tinnitus (40%), vertigo and sensations of instability (25%), sudden onset of sensorineural hearing loss (2.5%), facial nerve paresis (0.8%) or hemiageusia (0.8%). In 6 VS patients (2.5% of all VS) hearing was without signs of asymmetry, but all these patients presented with either tinnitus, or vertigo, or had facial nerve paresis (data not shown).

Evaluation of screening protocols

In order to determine which patients with asymmetrically impaired sensorineural hearing should undergo MRI,a number of audiometric screening protocols have been developed (Table 1). Individual screening protocols evaluate different patterns of asymmetry. Recent publications on evaluation of screening protocols for VS diagnostics were reviewed, and the results from this study were added to the existing data (Table 2 and Table 3). So far the statistical properties of the protocols have been analyzed within a single group of patients (Cheng and Wareing, 2012; Gimsing, 2010; Mangham, 1991; Nouraei et al., 2007; Obholzer et al., 2004; Saliba et al., 2011; Schlauch and Levine, 1995). We asked, whether it would be possible to determine a superior protocol in sensitivity, specificity, and/or likelihood ratio using the reviewed data. We concentrated on protocols with sensitivity and specificity values reported from 3 or more published studies (including the current study): Nashville, Sunderland, DOH, Seattle and Oxford (6 studies); Charing Cross (5 studies); AAO-HNS and Cueva (4 studies); Schlauch, Rule3000, AMCLASS-A and AMCLASS-B (3 studies). As noted in Table 2, Table 3, the variability of reported sensitivity and especially specificity values is rather high, and, as generally expected, there seems to be an inverse correlation between the sensitivity and the specificity of any given protocol (Fig. 2).

Table 1

Audiometric protocols used in screening for vestibular schwannoma.

Author	Protocol	Protocol definition
Welling et al. (1990).	Nashville	≥15 dB at any single frequency 0.5–4 kHz
Neary et al. (1996).	DOH	≥20 dB at any single frequency 0.5–4 kHz
Margolis and Saly (2008)	AMCLASS-B	≥15 dB at any single frequency
Saliba et al. (2009), 2011	Rule3000	≥15 dB at 3 kHz
Mangham (1991).	Mangham	≥10 dB in AHT (1–8 kHz)
Schlauch and Levine (1995).	Schlauch	≥20 dB in AHT (1–8 kHz) in men, ≥ 20 dB at 4 kHz in women
Sheppard et al. (1996).	Oxford	≥15 dB in AHT (0.5–8 kHz)
Hunter et al. (1999).	Seattle	≥15 dB in AHT (1–8 kHz)
Urben et al. (1999); Com.on Hear., 1995.	AAO-HNS	≥15 dB in AHT (0.5–3 kHz)
Dawes and Jeannon (1998).	Sunderland	≥20 dB in 2 consecutive frequencies
Margolis and Saly (2008).	AMCLASS-A	≥10 dB in 2 consecutive frequencies
Cueva (2004).	Cueva	≥15 dB in 2 octave frequencies
Obholzer et al. (2004).	3 × 15dB	≥15 dB in 3 consecutive frequencies
Obholzer et al. (2004).	Charing Cross	≥15 dB in 2 consecutive frequencies if better ear AHT (0.25–8 kHz) ≤ 30 dB≥20 dB in 2 consecutive frequencies if better ear AHT (0.25–8 kHz) > 30 dB

AbbreviationsAAO-HNS, American Academy of Otorhinolaryngology-Head and Neck Surgery; AMCLASS, Audiogram Classification System; AHT, average hearing threshold; DOH, Department of Health UK; Com. on hear., Committee on Hearing.

Table 2

Sensitivity (%) of screening protocols according to the published studies.

	Obholzer et al. (2004).	Nouraei et al. (2007).	Gimsing (2010).	Saliba et al. (2011).	Cheng and Wareing (2012).	Current study	Mean (SEM)/weighted mean
n (VS)	36	129	199	84	131a	246
AMCLASS-B	100				88	97	95 (3.6)/94
AMCLASS-A				93	93	94	93.3 (0.3)/93
Nashville	100	97	93	93	88	89	93.3 (1.9)/92
DOH	97	95	88	87	83	89	89.8 (2.1)/89
Cueva	97			81	86	91	88.8 (3.4)/88
Sunderland	97	95	93	74	83	84	87.7 (3.6)/87
Charing Cross	97	92	76		84	87	87.2 (3.7)/85
Seattle	92	91	80	92	86	80	86.8 (2.3)/84
AAO-HNS	92			90	87	72	85 (4.5)/80
Oxford	86	82	68	93	86	77	82 (3.5)/79
Rule3000				73	88	82	81 (4.4)/82
Schlauch			71		82	67	73.3 (4.5)/72
Mangham			(≥5 dB) 92		(≥10 dB) 92	(≥10 dB) 87
3 × 15dB	92					83

Mean values with SEM/weighted mean.

Cheng and Wareing included 131 cerebellopontine angle tumors: 123 VSs and 8 meningiomas.

Table 3

Specificity (%) of screening protocols according to the published studies.

	Obholzer et al. (2004).	Nouraei et al. (2007).	Gimsing (2010).	Saliba et al. (2011).	Cheng and Wareing (2012).	Current study	Mean (SEM)/weighted mean
n (controls)	100	129	225	148	1620	442
AAO-HNS	62			54	65	60	60.2 (2.3)/63
Rule3000				76	57	44	59 (9.3)/56
Schlauch			52		63	59	58 (3.2)/61
Seattle	62	61	50	44	60	49	54.3 (3.1)/57
Charing Cross	49	53	50		66	34	50.4 (5.1)/58
Oxford	35	40	46	43	61	55	46.7 (4)/56
Cueva	47			60	48	27	45.5 (6.8)/45
DOH	37	26	47	59	63	32	44 (6.1.)/54
Sunderlanda	15	15	48	70	61	45	42.3 (9.4)/54
Nashville	36	39	46	43	52	30	41 (3.2)/46
AMCLASS-B	27				45	11	27.7 (9.8)/37
AMCLASS-A				26	32	17	25 (4.6)/29
Mangham			(≥5 dB) 47		(≥10 dB) 44	(≥10 dB) 32
3 × 15dB	65					36

Mean values with SEM/weighted mean.

In the study of Obholzer and Nourai protocol Sunderland included also clinical criteria – unilateral tinnitus and symptoms typical for Menière disease. That might result in a lower specificity comparing to the other studies.

Fig. 2

Sensitivity (gray bars, values in Table 2) and specificity (white bars, values in Table 3) of the tested screening protocols. Mean values with SEM. For statistic comparison see Table 4. (CC, Charing Cross protocol).

Table 4

Statistical comparison of sensitivity and specificity between screening protocols..

To find out a superior screening protocol in sensitivity, specificity, and/or positive likelihood ratio, data from the published studies and the current study were pooled. Sensitivity and specificity of each protocol was compared separately with each other (t-test; Table 4). Nashville protocol was more sensitive than three other protocols (Rule3000, Oxford, Schlauch). Sensitivity of AMCLASS-A protocol was significantly higher than Schlauch and Rule3000. The least sensitive protocol was Schlauch protocol that was significantly lower in sensitivity than seven other protocols.

Regarding specificity, none of the protocols was constantly superior to the others. AMCLASS-A protocol was statistically less specific than seven other protocols. AMCLASS-B and Nashville protocols were less specific than three other protocols. In conclusion, Nashville and AMCLASS-A tend to be more sensitive than some other protocols, but it comes at the expense of specificity, especially in the case of AMCLASS-A.

To assess the diagnostic value of a tested protocol, positive likelihood ratio (LR+) was calculated for each protocol using sensitivity and specificity values from individual cohort studies. Mean LR + values for each protocol were compared with each other by t-test (Table 5). The highest mean LR+ was found for Rule3000 (2.18 ± 0.46). However, due to high standard error there was no significant difference when compared to the other protocols. On the other hand, AAO-HNS (2.17 ± 0.17) protocol had significantly higher LR + than AMCLASS-A (1.25 ± 0.07), AMCLASS-B (1.35 ± 0.15), Oxford (1.58 ± 0.14) and Nashville (1.6 ± 0.08). Efficacy of AMCLASS-A (1.25 ± 0.07) was significantly lower than Nashville (1.6 ± 0.08), AAO-HNS (2.17 ± 0.17) and Seattle (1.95 ± 0.16).

Table 5

Statistical comparison of positive likelihood ratio (LR+) between protocols..

Discussion

In our patient cohort the incidence of vestibular schwannoma (VS) reached its peak in the age group of 50–69 y with the mean age 56, which is in line with previous observations on VS patients’ demography (Stangerup and Caye-Thomasen, 2012).

Screening protocols based on audiogram have been designed to simplify the decision making for further investigation in patients with asymmetric hearing loss. The advantages of audiometric screening protocols are their simplicity, objectivity, easiness to apply, and they cause no additional costs. Recent studies have tested screening protocols for their sensitivity and specificity, however, with variable results (Cheng and Wareing, 2012; Gimsing, 2010; Mangham, 1991; Nouraei et al., 2007; Obholzer et al., 2004; Saliba et al., 2011; Schlauch and Levine, 1995). Published data was pooled together with results from this study and sensitivity, specificity, and positive likelihood ratio (LR+) between the most frequently tested protocols (AAO-HNS, AMCLASS-A, AMCLASS-B, DOH, Charing Cross, Cueva, Nashville, Oxford, Rule3000, Seattle, Schlauch, Sunderland) was compared. All protocols showed sensitivity higher than 80% apart from Schlauch (73%), and most protocols were statistically more sensitive than Schlauch (Table 2, Table 4). For the most sensitive protocols (AMCLASS-B, AMCLASS-A, Nashville) the sensitivity comes with significantly lower specificity when compared with the other protocols (Table 3, Table 4). LR + takes into account both sensitivity and specificity, and is used to examine the protocol’s usefulness in detecting a given condition. In this study LR+ was calculated for each protocol and dataset, and mean LR + values were compared between protocols (Table 5). In the studies of Saliba et al. (2009, 2011) there was a tendency for Rule3000 protocol to have the highest LR+. However, thus far Rule3000 has been examined in only three datasets and due to high variability (in particular specificity) the comparison to other protocols did not yield significant differences. The second best performing protocol in LR+ was AAO-HNS, that was significantly better than some other protocols with higher sensitivity, but far worse specificity (AMCLASS-A, AMCLASS-B, Nashville).

Because yield of VS proven by MRI is very low, about 2–8% (Cueva, 2004; Dawes and Jeannon, 1998; Saliba et al., 2011; Urben et al., 1999; in our dataset 3.5% - unpublished data), there is a significant number of negative MRI scans, and thus using a protocol of higher specificity can result in considerable cost saving with only few undetected VS (due to the protocol’s lower sensitivity). Low diagnostic rate allows us to use a very simplified example of comparison between a high sensitivity – low specificity protocol (e.g. AMCLASS-A) and lower sensitivity – higher specificity protocol (e.g. AAO-HNS, Rule3000). If VS yield in MRI is 3%, there is 97% negative waste. Using AMCLASS-A (specificity 25%), 73% of all patients is marked as positive, but the scan is negative (false positive result of the screening). Using AAO-HNS or Rule3000 (specificity about 60%), the false positive rate is only 39%. In this simplified example that means, that using a protocol with high specificity the rate of MRI scans could be reduced by 34% with an expense of very few primarily undetected VS. As lately the congress of neurosurgeons (Sweeney et al., 2018) recommended, if we look for the best sensitivity regardless high false positive waste, AMCLASS-A + B protocols could be the choice. On the other hand, if missing of few VSs is acceptable and rather cost saving approach is preferred, for example Rule3000, AAO-HNS or Seattle protocols could be an option.

Retrospective evaluation of screening protocols raises problems especially with the specificity interpretation. Whereas analyzing sensitivity in a group of MRI proven VS audiograms is definite (protocol condition fulfilled or not), specificity is usually obtained from much more heterogeneous group of VS negative audiograms and strongly depends on the initially chosencriteria for MRI imaging (used screening protocol, inclusion of different types of tinnitus, Menière’s diseases etc.). If MRI was indicated according to more conservative criteria (more profound asymmetry) and MRI is negative, post hoc analysis of protocols with less demanding criteria would result in lower specificity values (protocol marks vast majority of cases as false positive and cannot distinguish true negative cases any more, as seen for example in AMCLASS-A, AMCLASS-B, Nashville). Thus, specificity might be underestimated and should be carefully interpreted. Another pitfall in audiometric screening for VS is that unilateral hearing loss is a very common but non-specific symptom in population (up to 21% (Urben et al., 1999) and rises to 38% in 60 year-old adults (Pittman and Stelmachowicz, 2003) which is also the age of the highest VS incidence), and VS as a cause of hearing asymmetry is a rare condition. This fact, again, is limiting screening protocol specificity and resulting in a high false positive rate.

To avoid undesirable excess in MRI scans screening protocols may be combined in clinical decision-making with other audiometric examination, e.g. speech discrimination or auditory brainstem responses (but see Cueva, 2004). The presence of another vestibulocochlear symptom may support VS suspicion, but it is not a decisive factor as asymmetric hearing is the only symptom in almost half of the VS cases. Anyhow, the choice of the most suitable audiometric screening protocol is still more or less based on experience and preferences of the individual ENT center.

Furthermore, there is a group of VS patients with very little ear symptoms. In our patient cohort all the screening protocols missed 7 cases with symmetric hearing and, naturally, 7 additional patients with VS on the better ear side (otherwise excluded from this study). The presence of additional vestibulocochlear symptoms may lead to VS suspicion and diagnostic MRI. However, in most of these cases, VS is diagnosed as an incidental finding within wider interdisciplinary examination.

Analysis of audiometric screening protocols for VS showed, that protocols are statistically comparable in sensitivity ranging from 81 to 95% (apart from Schlauch). There was clearly higher variation in specificity (25–60%). The highest positive likelihood ratio (LR+) was observed for protocols Rule3000, AAO-HNS and Seattle. Using these protocols seems to offer cost-effectiveness without compromising sensitivity. Knowing their limits, audiometric screening protocols are very clear and easy-to-implement tools in VS screening. The choice of protocol, however, depends on preferences of the individual diagnostic center.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.