Aden Kittel1, Paul Larkin2, Nathan Elsworthy3, Michael Spittle4. 1. Institute for Health and Sport, Victoria University, Australia. Electronic address: Aden.Kittel@live.vu.edu.au. 2. Institute for Health and Sport, Victoria University, Australia; Maribyrnong Sports Academy, Australia. 3. School of Health, Medical and Applied Sciences, Central Queensland University, Australia. 4. Institute for Health and Sport, Victoria University, Australia.
Abstract
OBJECTIVES: To examine the reliability, construct validity and ecological validity of 360° VR and match broadcast footage for off-field decision-making assessment in Australian football umpires. DESIGN: Validation assessments with test re-test reliability. METHODS: Two video-based tests of 60 clips each were developed to assess Australian football umpire decision-making, including 360° video of small-sided Australian football games and match broadcast footage of AFL games. Elite (n = 13) and amateur (n = 15) umpires participated in two testing sessions, in a randomised, counterbalanced design. Test re-test reliability was assessed using Cohen's Kappa for individual clips and Intraclass Correlation Coefficients for test scores. Video tests were assessed for construct validity. Ecological validity of the decision-making processes was assessed for each method. RESULTS: 31 clips met the minimum Kappa criteria for the 360° VR test and 28 clips for match broadcast. Results indicated strong reliability for the 360° VR (ICC = 0.89) and match broadcast (ICC = 0.89) tests. For both video modes, elite umpires performed significantly better in decision-making accuracy than amateur (p < 0.05). For ecological validity of the decision-making processes, 360° VR was rated significantly higher than match broadcast vision (p < 0.05) overall. CONCLUSIONS: This is the first study to examine the reliability and validity of 360° VR footage as an off-field decision-making assessment tool in sport. As match broadcast vision is commonly used to assess decision-making in athletes and officials, results suggest that 360° VR is also an appropriate assessment tool. Although both video modes demonstrate similar reliability and construct validity, 360° VR was considered more specific to in-game decision-making processes, suggesting stronger ecological validity.
OBJECTIVES: To examine the reliability, construct validity and ecological validity of 360° VR and match broadcast footage for off-field decision-making assessment in Australian football umpires. DESIGN: Validation assessments with test re-test reliability. METHODS: Two video-based tests of 60 clips each were developed to assess Australian football umpire decision-making, including 360° video of small-sided Australian football games and match broadcast footage of AFL games. Elite (n = 13) and amateur (n = 15) umpires participated in two testing sessions, in a randomised, counterbalanced design. Test re-test reliability was assessed using Cohen's Kappa for individual clips and Intraclass Correlation Coefficients for test scores. Video tests were assessed for construct validity. Ecological validity of the decision-making processes was assessed for each method. RESULTS: 31 clips met the minimum Kappa criteria for the 360° VR test and 28 clips for match broadcast. Results indicated strong reliability for the 360° VR (ICC = 0.89) and match broadcast (ICC = 0.89) tests. For both video modes, elite umpires performed significantly better in decision-making accuracy than amateur (p < 0.05). For ecological validity of the decision-making processes, 360° VR was rated significantly higher than match broadcast vision (p < 0.05) overall. CONCLUSIONS: This is the first study to examine the reliability and validity of 360° VR footage as an off-field decision-making assessment tool in sport. As match broadcast vision is commonly used to assess decision-making in athletes and officials, results suggest that 360° VR is also an appropriate assessment tool. Although both video modes demonstrate similar reliability and construct validity, 360° VR was considered more specific to in-game decision-making processes, suggesting stronger ecological validity.