Validity and Reliability of the Vestibular/Ocular Motor Screening and Associations With Common Concussion Screening Tools.

BACKGROUND: Sustaining a concussion commonly results in vestibular impairments that may be associated with balance deficits. To screen for vestibular impairments after a concussion, the Vestibular/Ocular Motor Screening (VOMS) tool was developed. The relationship between the VOMS and other concussion screening tools, such as the Balance Error Scoring System (BESS) and King-Devick (K-D), have not been explored. HYPOTHESES: (1) VOMS would provide reliable results and not provoke symptoms in healthy adolescents and (2) VOMS test items would measure related aspects of vestibular function that are not measured through the BESS or K-D. STUDY
DESIGN: Cross-sectional, descriptive. LEVEL OF EVIDENCE: Level 4.
METHODS: A total of 105 healthy adolescents (53 male, 52 female; mean age, 15.4 years) completed the VOMS, BESS, and K-D tests. A subsample of 21 adolescents (16 male, 5 female; mean age, 15.5 years) completed the VOMS twice.
RESULTS: The median total symptom score for all 7 VOMS items was 0 (0-5). The majority of the individual VOMS test items total symptom scores demonstrated a significant correlation with each other ( rs = 0.25-0.66, P < 0.02). The individual VOMS items did not demonstrate a significant relationship to the BESS or K-D. VOMS items demonstrated high agreement in total symptom scores between testing trials, with near point convergence (NPC) distance demonstrating an intraclass correlation coefficient (ICC) of 0.95 (95% CI, 0.89-0.98; P < 0.001). The MDC95 (minimal detectable change with 95 confidence) for NPC distance was 4 cm.
CONCLUSION: The VOMS did not provoke vestibular symptoms in healthy adolescents. The VOMS items measured unique aspects of vestibular function other than those measured by the BESS or K-D with good reliability. CLINICAL RELEVANCE: Clinicians should consider implementing the VOMS as part of a comprehensive concussion assessment if vestibular impairment is suspected. If NPC distance is measured twice, a difference of >4 cm would be considered real change outside of measurement error.

Approximately a quarter of a million children received medical attention for sports or recreational concussions in 2009.[8,16,23] A concussion is a complex pathology and may cause numerous symptoms, including vestibular, balance, and visual ocular abnormalities. In an outpatient pediatric sports medicine clinic, 81% of youth athletes diagnosed with concussion demonstrated abnormal gaze stability and impaired tandem gait, indicating vestibular abnormality.[10] Balance abnormalities occur in 51% of high school football players after a concussion.[22] In a separate study, 69% of youth athletes who sustained a concussion demonstrated at least 1 visual abnormality.[24] Adolescents with vestibular abnormalities secondary to a concussion take longer to return to school and to sport,[10] while those with visual impairments have worse neurocognitive performance.[12,15,33]

The integration of the vestibular, vision, and somatosensory systems is essential for the maintenance of balance. The vestibular system is intricately related to the balance system through the vestibulospinal reflex (VSR) and to the visual system through the vestibulo-ocular reflex (VOR).[19] The VSR promotes appropriate motor responses in the extremities for the maintenance of balance while the VOR provides connections to the ocular muscles in order to keep clear vision with head movement.[19] Besides the VOR, additional visual functions such as near point convergence (NPC), smooth pursuit, and saccades are used to maintain clear vision and focus when the head is not moving.[19]

Clinical assessment tools examine the vestibular, balance, and visual ocular systems after a concussion. The Balance Error Scoring System (BESS) was developed to examine static standing balance (VSR) in athletes.[18] The BESS is currently recommended as part of a comprehensive concussion assessment.[7,26] Although the BESS is easy to complete with established validity and reliability,[5] it has some measurement challenges, specifically ceiling and learning effects.[28,36] The King-Devick (K-D) test is a quick and easy visual ocular assessment that requires a participant to rapidly identify and correctly name variably spaced single digits on 3 separate cards.[12,13] The K-D measures saccadic eye movement as well as language and concentration.[12,13,15] An increase in the time taken to complete the K-D compared with baseline is indicative of a concussion with sensitivity of 86% and specificity of 90%.[12-14]

After a concussion, vestibular impairments can occur without concomitant balance and saccadic abnormalities; therefore, the Vestibular/Ocular Motor Screening (VOMS) was developed to evaluate the vestibular/ocular motor system (Table 1). The VOMS measures 4 specific patient symptoms after each vestibular/ocular test, as well as the NPC distance.[27] Provoking symptoms by at least 2 points on a 10-point Likert-type scale on any VOMS test items or an NPC distance ≥5 cm may differentiate athletes who have sustained a concussion (area under curve = 0.64-0.78, P < 0.01).[27] Since the VOMS requires visual/ocular movement, individuals without a history of concussion and/or vestibular dysfunction may become symptomatic with testing or have abnormal NPC; therefore, investigating the provocation of symptoms and NPC in a group of healthy, nonsymptomatic adolescents is important. Furthermore, it is unclear if the VOMS total symptom score or NPC distance is associated with the BESS and the K-D. Additionally, investigating test-retest reliability of the VOMS provides psychometric evidence for the VOMS in clinical practice. The purpose of this investigation was 4-fold: (1) to report symptom provocation response characteristics of the VOMS test items and the NPC distance in a cohort of adolescents, (2) to examine the relationship between the VOMS and the BESS and the VOMS and K-D, (3) to examine the test-retest reliability of the VOMS, and (4) to determine minimal detectable change (MDC) for NPC distance.

Table 1.

Vestibular/Ocular Motor Screening (VOMS) for concussion[27]

VOMS Test	Headache[a]	Dizziness[a]	Nausea[a]	Fogginess[a]	Total Symptom Score[b]
Baseline symptoms
Smooth pursuit
Horizontal saccades
Vertical saccades
Near point convergence Measure 1: __________ Measure 2: __________ Measure 3: __________
Horizontal VOR
Vertical VOR
Visual motion sensitivity

VOR, vestibulo-ocular reflex.

Provocation of symptoms is rated on a scale from 0 to 10, with 0 being no symptoms and 10 being severe symptoms.

Total symptom score = change in headache from baseline + change in dizziness from baseline + change in nausea from baseline + change in fogginess from baseline for each of the VOMS test items.

Methods

The study protocol was approved by the institutional review board at the University of Michigan-Flint and by the school board of each of the participating schools.

Participants

A cross-sectional sample of 9th- through 12th-grade students was obtained from 3 high schools in the Midwest region. The participants’ parents signed the informed consent and participants provided assent to participate in the testing. In order to screen for any musculoskeletal or neurological impairments that would affect their performance on the selected measures, participants with self-reported history of concussion or history of low back or lower extremity injury within 3 months prior to the date of testing were excluded from the study. The 3-month cutoff was used as a reasonable timeframe for recovery from neuromusculoskeletal injury and has been used in studies seeking to report reference values in adolescents.[1,3] The sample consisted of 105 participants (53 male, 52 female), mean age 15.4 (SD = 1.2) years. To examine the test-retest reliability of the VOMS test, a subsample (selected by availability) of 21 participants (16 male, 5 female), mean age 15.5 (SD = 1.2) years, completed the VOMS for a second time after 60 minutes (Table 2). To estimate the sample size needed for reliability, β = 0.20, α = 0.05, and a desired intraclass correlation coefficient (ICC) range of 0.70 to 0.90 was used to estimate the sample size needed for reliability.[37] An ICC of 0.90 has been suggested as an ideal value for measures used in clinical practice, and 0.70 was determined to be minimally acceptable for this study.[30]

Table 2.

Demographic characteristics of sample

	Entire Sample (N = 105)	Reliability Sample (N = 21)
Gender, n (%)
Male	53 (50.5)	16 (76.2)
Female	52 (49.5)	5 (23.8)
Height (inches), mean (SD)	67.5 (4.8)	70.0 (3.8)
Weight (lb), mean (SD)	156.8 (30.5)	169.7 (40.2)
Age (y), mean (SD)	15.4 (1.2)	15.5 (1.2)

Instrumentation

Vestibular/Ocular Motor Screening Assessment

Participants completed the VOMS as outlined in the original protocol and tested 7 items, including smooth pursuit, saccades (horizontal and vertical), VOR (horizontal and vertical), vision motion sensitivity testing, and NPC distance (3 trials).[27] Participants were asked the intensity, using a scale from 0 (none) to 10 (severe), of 4 possible symptoms (headache, dizziness, nausea, and fogginess) at baseline and after each of the 7 VOMS tests (Table 1). Any change in each of the 4 symptoms from baseline was reported for each of the 7 VOMS items and used for test-retest reliability analysis. The total symptom score was calculated by summating any change in 4 symptoms from baseline for each of the 7 VOMS tests and used for correlation analysis. When completing NPC distance, participants were instructed to focus on a 12-point font letter placed on the end of tongue depressor as it was moved closer toward their nose, and to inform the researcher when their vision became double. Once the participant indicated the presence of double vision, the distance from the tip of the participant’s nose to the tongue depressor was measured. The average of the 3 trials for NPC distance (centimeters) was calculated and used in analysis. The internal consistency of the VOMS is high (Cronbach’s α = 0.92).[27]

Balance Error Scoring System

The BESS consists of 3 stance positions (feet together, tandem, and single-leg stance), completed on both firm and foam surfaces, with hands placed on hips, eyes closed, and held for 20 seconds.[18] Errors—defined as opening the eyes, lifting the hands off the hips, stepping, falling out of position, lifting forefoot or heel, abducting the hip by more than 30°, or failing to return to the test position in more than 5 seconds—were counted during each of the positions. The maximum errors in each of the 6 trials were 10, and the maximum possible number of errors in total were 60. The BESS has been reported to have acceptable test-retest reliability (ICC = 0.70-0.74) and high intrarater reliability (ICC = 0.98) in young athletes.[3,34,35] The BESS has demonstrated validity in identifying balance deficits in individuals with concussion.[17,25]

King-Devick Test

The standardized instructions of the K-D test were used.[12,13] Participants were asked to read out loud single digits found on 3 cards, from left to right and top to bottom, as quickly as possible without making any mistakes. The times for all 3 cards were summed to give a total time score. Each participant completed the K-D twice, with the fastest total time of the 2 trials without errors used in analyses. The K-D has demonstrated good test-retest reliability ranging from an ICC of 0.86 to 0.97.[1,12,21]

Procedures

After demographic information was obtained, participants completed the VOMS, BESS, and K-D. The participants completed the assessments at separate testing stations within a larger open area where other testing was being completed. The order of the testing was pseudorandom based on station availability. The VOMS testing was completed by a licensed clinician with certification in vestibular rehabilitation, and the BESS and K-D testing were completed by licensed clinicians and graduate student research assistants. During the planning phases of the study, training was conducted for all testers. After training, formal reliability testing was completed. The interrater reliability for K-D was ICC3,1 = 0.99 (0.97-0.99) and for BESS was ICC3,1 = 0.92 (0.87-0.94).

Statistical Analysis

Descriptive statistics were used to describe the demographic characteristics of the sample and the performance characteristics of the VOMS, BESS, and K-D testing. Correlation analysis, using the Spearman’s rho (rs) was completed to examine the relationships between the total symptom score for each of the 7 VOMS test items, including NPC distance as compared with the BESS error score, and the K-D best trial time. The false discovery rate was used to account for multiple correlations. The false discovery rate is the expected proportion of erroneous rejections among all rejections.[6] Associations between variables were considered good to excellent (rs > 0.75), moderate to good (rs = 0.50-0.75), fair (rs = 0.25-0.50), and little to none (rs < 0.25).[30] When exploring the symptom provocation data of each of the 7 VOMS test items in the test-retest reliability subsample, a lack of variability was observed; therefore, traditional reliability measures (eg, Cohen’s kappa) could not be used. To further investigate the data, the percentage of agreement between the 2 trials was calculated. To examine for test-retest reliability of the NPC distance, ICC and its 95% confidence intervals were calculated. ICCs are commonly interpreted as those >0.80 as good reliability, between 0.60 and 0.79 as moderate, and those <0.60 as poor reliability.[30] The standard errors of the measurement (SEM) values were calculated using the following formula: . The MDC95 was calculated using the following formula: . For all analyses, the Statistical Package for the Social Science (version 21) was used (IBM Corp).

Results

The median total symptom provocation change for each of the 7 VOMS tests was 0 (range, 0-5; Table 3). The median NPC distance was 3.0 cm (range, 0-20.7 cm). No significant differences were observed between adolescents with history of concussion (ie, >3 months) and those without history of concussion, t(103) = −0.407, P = 0.613.

Table 3.

Performance characteristics of the Vestibular/Ocular Motor Screening (VOMS) (N = 105)

VOMS Test Items	Total Symptom Score
	Median	Range	Frequency (Total Symptom Score <2 or NPC <5); n (%)
Smooth pursuit	0	0-2	102 (95.2)
Horizontal saccade	0	0-3	99 (94.3)
Vertical saccade	0	0-5	98 (93.3)
Convergence	0	0-4	100 (95.2)
Horizontal VOR	0	0-5	94 (89.5)
Vertical VOR	0	0-5	98 (93.3)
Visual motion sensitivity	0	0-3	95 (90.5)
NPC distance (cm)	3.0	0-20.7	79 (75.2)

NPC, near point convergence; VOR, vestibulo-ocular reflex.

Fair to good relationships were found between the VOMS total symptom change scores in 90% (19 of 21) of the comparisons (rs = 0.25-0.66, P ≤ 0.02; Table 4). NPC distance was only found to be significantly related to horizontal VOR (rs = 0.26, P = 0.02; Table 4) and vertical VOR (rs = 0.25, P = 0.02; Table 4). No statistically significant relationships were found between any of the VOMS items to the BESS error score or K-D score (rs = −0.03-0.18, P > 0.05; Table 4).

Table 4.

Interitem correlations for Vestibular/Ocular Motor Screening total symptom scores and NPC distance, BESS, and K-D (N = 105)

	Horizontal Saccade	Vertical Saccade	NPC, Symptom	Horizontal VOR	Vertical VOR	VMS	NPC, Distance	BESS	K-D
Smooth pursuit	0.64[a]	0.52[a]	0.52[a]	0.12	0.22	0.30[a]	−0.05	−0.03	0.08
Horizontal saccade		0.64[a]	0.50[a]	0.40[a]	0.27[a]	0.48[a]	−0.05	0.11	0.10
Vertical saccade			0.58[a]	0.36[a]	0.44[a]	0.48[a]	−0.16	0.18	0.10
NPC, symptoms				0.32[a]	0.57[a]	0.61[a]	0.04	0.15	0.07
Horizontal VOR					0.62[a]	0.46[a]	0.26[a]	0.09	0.03
Vertical VOR						0.66[a]	0.25[a]	0.12	0.03
VMS							0.14	0.02	0.15
NPC distance, cm								0.11	−0.03

BESS, Balance Error Scoring System; K-D, King-Devick; NPC, near point convergence; VMS, visual motion sensitivity; VOR, vestibulo-ocular reflex.

Correlation is significant at the adjusted α = 0.0242.

The percentage of agreement in symptom changes between VOMS trial 1 and VOMS trial 2 was 100% for 18 out of the 28 tests (64.3%; Table 5). The NPC distance demonstrated good reliability with ICC = 0.95 (95% CI, 0.89-0.98; P < 0.001). The SEM for NPC distance was 1.5 cm and the MDC95 was calculated as 4 cm.

Table 5.

Level of agreement between Vestibular Ocular/Motor Screening (VOMS) test items and report of provocation of symptoms between trial 1 and trial 2 (N = 21)

VOMS Test Items	Change in Symptoms Trial 1/Trial 2
	0, n (%)	1, n (%)
Smooth pursuit
Headache change	21 (100)
Dizziness change	19 (90)	2 (10)
Nausea change	21 (100)
Fogginess change	20 (95)	1 (5)
Horizontal saccade
Headache change	21 (100)
Dizziness change	21 (100)
Nausea change	21 (100)
Fogginess change	19 (90)	2 (10)
Vertical saccade
Headache change	21 (100)
Dizziness change	21 (100)
Nausea change	21 (100)
Fogginess change	20 (95)	1 (5)
Near point convergence, symptoms
Headache change	20 (95)	1 (5)
Dizziness change	20 (95)	1 (5)
Nausea change	21 (100)
Fogginess change	21 (100)
Horizontal VOR
Headache change	21 (100)
Dizziness change[a]	18 (85)	2 (10)
Nausea change	21 (100)
Fogginess change	20 (95)	1 (5)
Vertical VOR
Headache change	20 (95)	1 (5)
Dizziness change	21 (100)
Nausea change	21 (100)
Fogginess change	21 (100)
Visual motion sensitivity
Headache change	21 (100)
Dizziness change	19 (90)	2 (10)
Nausea change	21 (100)
Fogginess change	21 (100)

VOR, vestibulo-ocular reflex.

One subject had a symptom change of 2.

Discussion

Vestibular/ocular assessment in the presence of vestibular abnormalities is anticipated to provoke symptoms. As expected, the participants in this study, with no reported history of vestibular dysfunction, did not become symptomatic with vestibular/ocular movement. However, 25% (n = 26) of the participants in this study demonstrated an NPC distance of ≥5 cm. Abnormal NPC of ≥5 cm is one measure of convergence insufficiency, the inability of the eyes to work together when completing near work.[31] The prevalence of convergence insufficiency in the current sample is comparable to the range of prevalence reported in the elsewhere (2%-33%).[31] In youth athletes after a concussion, convergence insufficiency has been reported to be 45%.[27] An NPC distance abnormality measured after concussion may be attributed to a preexisting NPC abnormality or concussion. Screening for NPC distance in youth athletes preseason would provide more confidence in the diagnostic utility of NPC assessment after a suspected concussion.

In a cohort of adolescents, the VOMS measured related, but not identical, aspects of vestibular function that were not otherwise assessed by the BESS and K-D. The fair-to-good relationship among the VOMS items in the current sample (rs = 0.25-0.66) was lower than the relationship observed among the VOMS items in patients with concussion (rs = 0.44-0.88).[27] These differences may be explained by the concomitant presence of multiple symptoms after concussion or cross interference of symptom reporting.[9] The 2 test items that were not related to smooth pursuit were horizontal and vertical VOR; however, horizontal and vertical VOR were the only 2 test items that were related to NPC distance. The order of testing was completed consistently with horizontal and vertical VOR tested immediately after NPC distance. Randomizing the testing would allow for further exploration of this potential relationship between NPC distance and horizontal and vertical VOR.

The performance of the BESS (median errors = 22) and K-D (median = 43.3 seconds) for participants in this study is comparable to the reported reference values for the BESS (range, 17-20)[3,20] and K-D (44.5 seconds).[11,14] The VOMS total symptom scores and NPC distance did not correlate with the BESS total error score and the K-D in this sample of adolescents. The 3 tests used in this study appear to be measuring different constructs. The VOMS is specifically designed to invoke vestibular symptoms that may be subtle and not experienced at rest. The assessment of balance impairments is a tool in concussion assessment; however, most athletes who experience dizziness after a concussion do not report associated balance impairments.[22] K-D is used preseason, with any worsening of score indicative of sustaining a concussion, but it measures saccadic eye movement and does not measure the provocation of symptoms. Understanding the relationship of vestibular, balance, and visual ocular testing allows clinicians to make appropriate assessment decisions. The VOMS is easy to implement and does not require a baseline. Because the VOMS specifically looks to provoke symptoms that can occur after a concussion, it may be used to screen for vestibular symptoms after a concussion that are not otherwise captured by the BESS or K-D test. Vestibular impairments are commonly overlooked despite their high prevalence and prognostic utility to determine a protracted recovery.[10,22] The median time for referral to vestibular rehabilitation ranges from 53 to 61 days after a concussion,[32] and once identified, vestibular impairments improve with rehabilitation.[2,4]

The individual symptoms provoked by each of the 7 VOMS test items demonstrated a high level of agreement between trials in the subsample of participants. Similar to previous studies, NPC distance demonstrated good reliability (ICC = 0.95-0.98).[29] Nonetheless, the minimal detectable change for the NPC distance was 4 cm, which is equivalent to 80% of the conventional cutoff to determine normal NPC distance (5 cm). Once an initial NPC distance is established, a change of 4 cm is needed to confirm a true change beyond the expected measurement error. Therefore, the confidence in the diagnostic utility of NPC distance in diagnosing convergence insufficiency needs to be supported by other findings such as symptom provocation during testing or observation of the insufficiency by the examiner.

There are some limitations to this study. The participants were adolescents and the results may not be generalizable to younger or older athletes. A convenience sample was used, and with no randomization of testing order. The VOMS tests use subjective reporting, which inherently includes bias and a potential fear that subjects may not want to report symptoms in order to prevent suspicion of a concussion. The VOMS was tested without prior exertion (eg, games, practices); therefore, clinicians who perform the VOMS in athletes after exertion must exercise caution in interpreting possible VOMS abnormalities.

Conclusion

In a group of healthy adolescents, the VOMS test items did not provoke common vestibular symptoms. The VOMS test items measured associated, but not duplicate, aspects of vestibular ocular function that were not otherwise captured by the BESS or K-D. Recognizing similarities and differences in common assessment tools allows clinicians to implement preseason and postconcussion testing and make appropriate clinical decisions.