Visuo-spatial ability in colonoscopy simulator training.

Visuo-spatial ability is associated with a quality of performance in a variety of surgical and medical skills. However, visuo-spatial ability is typically assessed using Visualization tests only, which led to an incomplete understanding of the involvement of visuo-spatial ability in these skills. To remedy this situation, the current study investigated the role of a broad range of visuo-spatial factors in colonoscopy simulator training. Fifteen medical trainees (no clinical experience in colonoscopy) participated in two psycho-metric test sessions to assess four visuo-spatial ability factors. Next, participants trained flexible endoscope manipulation, and navigation to the cecum on the GI Mentor II simulator, for four sessions within 1 week. Visualization, and to a lesser degree Spatial relations were the only visuo-spatial ability factors to correlate with colonoscopy simulator performance. Visualization additionally covaried with learning rate for time on task on both simulator tasks. High Visualization ability indicated faster exercise completion. Similar to other endoscopic procedures, performance in colonoscopy is positively associated with Visualization, a visuo-spatial ability factor characterized by the ability to mentally manipulate complex visuo-spatial stimuli. The complexity of the visuo-spatial mental transformations required to successfully perform colonoscopy is likely responsible for the challenging nature of this technique, and should inform training- and assessment design. Long term training studies, as well as studies investigating the nature of visuo-spatial complexity in this domain are needed to better understand the role of visuo-spatial ability in colonoscopy, and other endoscopic techniques.

Introduction

General introduction

Visuo-spatial ability is associated with quality of performance in a variety of surgical and medical skills (Hegarty et al. 2007). However, visuo-spatial ability is typically assessed using Visualization tests only. Since Visualization is but one of a number of factors that together make up visuo-spatial ability, more comprehensive studies are needed to assess the role of other aspects of visuo-spatial ability in medical skills. The study presented here is the first to investigate how a broad set of four critical visuo-spatial ability factors correspond to achievement in colonoscopy simulator training. This can inform both simulator software design and the design of tests specifically aimed at predicting an individual’s success in acquiring skill in endoscopic surgery.

Visuo-spatial ability in surgery

A leading model of human cognitive abilities is John B. Carroll’s Three Stratum Theory (Carroll 1993). This three-tiered model, based on a re-analysis of over 400 datasets, holds that there is a first-tier general intelligence factor (g) that is defined as the shared factor loadings of second-tier factors such as verbal intelligence, analytical intelligence, and visuo-spatial ability. Each of these second-tier factors in turn is defined as the shared factor loadings of a number of third-tier factors. Third-tier factors are the ones that are measured by specific tests, e.g. Visualization (nested within the second-tier factor of visuo-spatial ability) is commonly measured with Vandenberg and Kuse’s well-known Mental Rotation Test (Vandenberg and Kuse 1973).

Visuo-spatial ability refers to the human cognitive ability to form, retrieve, and manipulate mental models of a visual and spatial nature (Lohman 1979a). Carroll identifies five main third-tier factors nested within visuo-spatial ability, namely Visualization, Spatial relations, Speed of closure, Flexibility of closure, and Perceptual speed.

Visualization is the ability to manipulate complex mental representations of a visuo-spatial nature. Most research uses Visualization as a proxy for visuo-spatial ability. The relation between Visualization and surgical proficiency is consequently relatively well charted. Visualization correlates with Time on task for a number of laparoscopic tasks (Keehner et al. 2006; Risucci et al. 2000, 2001), with quality of laparoscopic surgery as measured by rating scales (Keehner et al. 2004; Schueneman et al. 1983; Wanzel et al. 2002, 2003), and with quality of laparoscopic surgery as measured by simulator training outcomes (Eyal and Tendick 2001; Hedman et al. 2006). Additionally, Risucci (2002) found that surgeons score higher on Visualization tests than a comparable normative sample from the general population.

Spatial relations is the ability to quickly manipulate simple mental representations of a visuo-spatial nature. This factor is thought to be similar to Visualization, but with more emphasis on speed, in relatively simple tasks. Ritter et al. (2006) found a correlation between Spatial relations and both duration of training and number of trials on a flexible endoscope simulator task (colonoscopy). Westman et al. (2006), Haluck et al. (2002), and Eyal and Tendick (2001) all found a significant correlation between quality in a number of surgical simulator training tasks and Spatial relations. Westman et al. found this for a colonoscopy simulator task, Haluck et al. and Eyal & Tendick for a laparoscopy simulator task.

Speed of closure refers to the ability to match incomplete stimuli to memory representations of the corresponding complete stimuli. Wanzel et al. (2002, 2003), Risucci et al. (2000, 2001), and Risucci (2002) use Speed of closure tests in their research. Risucci et al. studied laparoscopic simulator training tasks, Wanzel et al. studied non-endoscopic procedures. Only in Risucci’s 2001 study a low, but significant correlation between Speed of closure and speed on several simulator dexterity drills was found. This factor seems to contribute little to surgical skill.

Flexibility of closure is the ability to identify given patterns in a cluttered visual environment. Steele et al. (1992), and Gibbons et al. (1986) found a significant correlation between Flexibility of closure and ratings of both quality of anastomoses on the porcine model and operating room performance. Schueneman et al. (1984) used Flexibility of closure tests as well, but found no significant correlations with operating room performance as measured by rating scales. Such contradicting results make it hard to assess the contribution of Flexibility of closure to surgical skill and surgical learning.

Perceptual speed refers to the ability to quickly identify a given shape from a number of alternatives. Perceptual speed has not been included in surgical training studies yet, as far as we know.

Concluding, there is good evidence for the involvement of Visualization and Spatial relations in surgical training, and some evidence against the involvement of Speed of closure in such training. Flexibility of closure and Perceptual speed remain little researched in this context. Specifically for colonoscopy training (studied here) only one study is known to us that investigates the relation between achievement in such training and visuo-spatial ability (by proxy of a Spatial relations test). The present study aims to develop a more complete understanding of the involvement of the various aspects of visuo-spatial ability in the development of endoscopic skill. Surgical curricula in an age of simulator technology need such knowledge to optimize both efficiency and effectiveness.

Materials and methods

Participants

Fifteen medical trainees, five male and ten female, participated in this study. None of the participants reported clinical experience in colonoscopy. All participants were between 21 and 29 years of age, with a mean age of 25. All reported normal or corrected to normal vision. Prior to signing up, potential participants were informed on the nature of the study and the activities involved. They were also informed that all gathered data would be processed anonymously. All participants signed an ‘informed consent’ form.

Study protocol

Prior to performing on the simulator, participants filled out a demographics questionnaire. Participants’ visuo-spatial abilities were assessed in two sessions by a specially assembled test-battery (as described in paragraph 2.3). Both test-battery and questionnaire were administered in group sessions. In the next stage, participants individually received a general introduction into colonoscopy, VR endoscopy simulators, and how to utilize the colonoscopy simulator instruments by means of a video prior to performing the first session, to ensure all participants would have a comparable basic level of knowledge and understanding. After this introduction, initial performance in basic colonoscopy skills was assessed using the GI mentor II simulator. Finally, all participants received colonoscopy training on this same simulator. In total, the training consisted of four simulator sessions performed within one week. Time needed to conclude one session was on average 45 min.

Psychometric assessment

Visuo-spatial ability was tested during two psychometric assessment sessions. Each session included four different tests measuring the visuo-spatial ability factors of Visualization, Spatial relations, Flexibility of closure, and Perceptual speed. To measure these abilities, two tests for each factor were administered, on separate occasions. This was done to compensate for intra-individual factors that might influence test results (e.g. lack of sleep). The test battery was administered in groups, in the period preceding the simulator training sessions. Example items of representative tests measuring these factors are given in Fig. 1. A complete list of the tests used in this study can be found in “Appendix”.

Fig. 1

Example items of the four paper-and-pencil tests that were used to measure relevant factors of participants’ visuo-spatial ability. All tests require the participant to identify a target figure shown to the left from a row of similar figures to the right. ‘A’ shows an example item from the Mental Rotation Test, measuring Visualization. ‘B’ shows an item from the Cards test, measuring Spatial relations. ‘C’, Hidden Objects measures Fluency of Closure. With items such as shown in ‘D’, Perceptual speed is measured

Colonoscopy simulator training

Next, participants trained in flexible endoscope manipulation and navigation to the cecum (a pouch that marks the beginning of the large intestine) on the GI Mentor II simulator. Training consisted of four 45-min sessions within 1 week (one per day). Each session involved two different tasks. The EndoBubble exercise was combined with the task to reach the cecum in various VR-colonoscopy cases to provide training in, and objective assessment of, skills in basic camera navigation, instrument aiming, and bimanual instrument control (Buzink et al. 2007).

In the EndoBubble exercise the participant navigated through a virtual tube as quickly as possible and popped as many balloons as possible with a virtual needle mounted on the tip of the flexible endoscope, without touching the wall. This task involved camera navigation, aiming, and bimanual instrument control in an abstracted environment. Performance measures for this task were total time (Time), number of popped balloons (Popped balloons), and number of times the wall was touched (Wall touches).

In the VR-colonoscopy exercises, the task was to navigate the camera through a virtual representation of a winding colon with peristaltic movement to the cecum as quickly as possible, with as little patient discomfort as possible. In this task, the participant also occasionally needed to apply torque to the endoscope shaft, or was confronted with looping of the colon. This system provides the following performance measures: total time (Time), percentage of time the virtual patient was in pain (Pain), percentage of time spent with a clear view (Clear view, camera is aligned with motion path), and the number of times the camera lost its alignment with the motion path (Lost lumen).

Data reduction

For the EndoBubbles task, Time was found to be the only useful training outcome measure. Popped balloons showed a ceiling effect (minus four instances where one balloon was missed, all twenty balloons were always popped). The distribution of Wall touches showed this measure to not accurately reflect damage incidents, as single incidents would result in a widely different number of wall touching instances. These latter two variables where not analyzed.

For the VR-Colonoscopy task variable Pain again single incidents would lead to very different values, so this variable was left out of the analysis. Time, Clear view, and Lost lumen were analyzed.

The visuo-spatial test battery consisted of two tests for each of the four factors of interest in this study. For each individual, mean values of each of those four pairs of tests were calculated. No significant deviations from the normal distribution were found according to a Kolmogorov–Smirnov test, allowing the use of parametric tests.

Results

Means and standard deviations for the Endobubbles performance variable Time and for the VR-Colonoscopy performance variables Time, Clear view, and Lost lumen are shown in Fig. 2. To asses whether learning took place, repeated measures ANCOVAs were run for all performance variables. Session was used as repeating measure. The four cognitive ability factors described in Sect. “Visuo-spatial ability in surgery” were used in these analyses as covariables, to assess the correlation of these covariables on learning. Effects for Time, for both tasks, were highly significant, F(1, 14) = 41.1, p < .001 for the EndoBubbles task, and F(1, 14) = 10.3, p < .001 for the VR-Colonoscopy task, confirming that participants became faster on both tasks. A significant between subjects effect for the covariable Visualization was also found for Time, for both tasks, F(1, 14) = 10.7, p < .01 for the EndoBubbles task, and F(1,14) = 8.6, p < .02 for the VR-Colonoscopy task. This means that for both tasks participants of high visualization ability improved faster on Time compared to participants of low visualization ability. Post-hoc effect sizes were estimated for both Time variables, and were found to be large (f² = .5). No learning was found for the VR-Colonoscopy variables Clear view, and Lost lumen.

Fig. 2

Means and standard deviations for the analyzed performance variables, for all sessions and both tasks. Top left EndoBubbles task (time on task), top right VR-colonoscopy task (time on task), bottom left VR-Colonoscopy task (percentage of time spent with clear view), bottom right VR-Colonoscopy task (number of times lumen was lost from view)

Additionally, to assess the effect of the four cognitive ability factors on Time for both tasks, Clear view, and Lost lumen, correlations were calculated for these variables (Table 1). For Time and Clear view means were derived over of all sessions. For Lost lumen, all session values were added. In this way, the number of observations was quadrupled. For Time for both simulator tasks, only Visualization correlated significantly, and negatively, with performance (i.e., the better participants scored on the Visualization tests, the faster they performed the simulator tasks). For the VR-Colonoscopy task variables Clear view and Lost lumen, both Visualization and Spatial relations correlated significantly with performance, negatively for Lost lumen, and positively for Clear view. Again this indicates better performance correlates with higher cognitive ability scores. Correlations for the VR-Colonoscopy performance variables are reported in Table 2.

Table 1

Correlations between the four visuo-spatial ability factors and Time on task for both colonoscopy simulator training tasks

Performance variables	Visualization	Spatial relations	Speed of closure	Perceptual speed
Participants (n = 15)
Time EndoBubbles	−.61*	.25	−.18	−.03
Time VR-Colonoscopy	−.69**	−.30	−.31	.27
Clear view	.43	.44	.23	.25
Lost lumen	−.65**	−.60**	−.34	.02

* Significant at the .05 level; ** significant at the .01 level

Table 2

Correlations for all analyzed VR-Colonoscopy performance variables

Performance variables	Clear view	Lost lumen
Participants (n = 15)
Time	−.38	.81**
Clear view		−.71**

** Significant at the .01 level

Discussion

Visualization, as measured by Vandenberg and Kuse’s mental rotation test (Vandenberg and Kuse 1978) and Guay’s visualization of viewpoints test (Guay and Mc Daniels 1976), was found to be important for performance on colonoscopy simulator training, in this study correlating with performance on Time for both tasks, and with Clear view and Lost lumen for the VR-Colonoscopy task. Visualization also covaried with learning rate for both simulator training tasks, as assessed by repeated measures analyses for Time. Stefanidis et al. (2006) report a similar finding for two cognitive ability factors that are somewhat visuo-spatial in nature, but strictly speaking fall in the domains of Memory and Reasoning. The importance of the visuo-spatial factor of Visualization for colonoscopy learning rate represents a novel find, indicating fewer training sessions for high scorers on this ability to attain the same level of skill when compared to low scorers. Given the relationship between Visualization and endoscopic proficiency (outlined in the introduction), this finding likely generalizes to learning rate in other endoscopic domains.

Contrasting Visualization to other visuo-spatial ability factors, the defining characteristic of Visualization is high stimulus complexity (Vandenberg and Kuse 1978; Hegarty and Waller 2005). The ability to mentally manipulate complex visuo-spatial structures is critical to early colonoscopy training performance. The specific cognitive speed and memory demands of other visuo-spatial factors are less involved in this performance. Future work in the construction of visuo-spatial tests aimed at predicting endoscopic performance should benefit from including requirements of stimulus complexity. As visuo-spatial complexity is not well understood, general psychological research in this area is needed, as well as task analyses aimed at finding out what makes colonoscopy (and other endoscopic techniques) visuo-spatially complex. Such studies can, and should, inform endoscopic training.

To a lesser degree, the visuo-spatial ability factor Spatial relations was also involved in simulator performance, on the Lost lumen variable of the VR-Colonoscopy task. Spatial relations tests are known from factor analytic studies to partially load on Visualization (Carroll 1993). The lesser, but disctict involvement of Spatial relations thus reinforces the above argument on the role of visuo-spatial complexity.

Of interest to note is that of the variables Clear view and Lost lumen, causally related and in this study highly correlated, only Lost lumen significantly correlated with Visualization and Spatial relations. This suggests Lost lumen represents a more challenging skill, and may thus be the better measure for skilled performance. The significant correlation between Time and Lost lumen, but not Time and Clear view further shows Lost lumen to be a more relevant measure to train to, as a better performance on Lost lumen is more likely to result in Time gains. ‘Better safe than sorry’ obviously applies!

The lack of a damage measure that is both valid and usable is worrying, and further development of such measures should be prioritized by developers of colonoscopy simulator systems.

A drawback of this study is that only early learning was studied, long term studies are needed to assess the role of visuo-spatial ability in different learning stages. In many professions that demand highly skilled visuo-motor performance, typically automation of these skills takes place over time, perhaps reducing the demands on the higher cognitive abilities of its practitioners (sports, music) (Rasmussen 1987). Surgery differs in that there is a large degree of unpredictability with each new case. Automation will still take place, but the role of visuo-spatial ability is likely to stay important. This is corroborated by Keehner et al. (2006) who find general reasoning demands taper off after early training, but visuo-spatial ability remains important.

An additional question is why we should be interested in visuo-spatial ability over other cognitive abilities in the context of surgical training. Surely endoscopic skills also demand high reasoning ability, memory ability, etc., from its practitioners! We would like to speculate that by the time students of medicine start their residencies, their prior education already pretty much has functioned as a filter for those other relevant abilities. In contrast, visuo-spatial ability only starts playing a major role during one’s residency, but has up to that point pretty much flown ‘under the radar’, so to speak. If this were the case, a major improvement over current educational practice would be the inclusion early in the medical curriculum of assignments that can only be successfully concluded if visuo-spatial demands are met.