Driving Trail Making Test part B: a variant of the TMT-B.

[Purpose] The Trail Making Test part B (TMT-B) is used in evaluating driving abilities and includes testing for the executive function. A driving simulator version of this test (DTMT-B) was developed to measure drivers' executive abilities in three-dimensional space. The purpose of the present study was to assess the validity of the DTMT-B for driving assessment. [Subjects] Thirty stroke patients and 65 healthy subjects were recruited. [Methods] Participants performed the TMT-B and DTMT-B. The DTMT-B was run on a driving simulator in which the individual performed a task on virtual roads connecting the lettered and numbered TMT-B points by simulated driving instead of connecting them with lines as in the paper or computerized TMT-B. Intra-class correlation coefficients (ICCs) were used to assess validities. Significant correlations were found between the TMT-B and DTMT-B.
[Results] Participants performed the TMT-B and DTMT-B. Intra-class correlation coefficients (ICCs) were used to assess validities. Significant correlations were found between the TMT-B and DTMT-B.
[Conclusion] The results suggest that the DTMT-B may be useful as part of driver screening assessment using a driving simulator for stroke patients and that it may also be used to assess the executive functions for healthy people.

INTRODUCTION

Driving is a complex task requiring appropriate reactions to different situations on the road1) and depends on intact sensory and motor functions2). Driving performance is adversely affected by many reasons like visual, neurological, and age-related deficits3). Therefore, formulating a fitness to drive recommendation using psychological and neurological tests is important in keeping potentially unsafe drivers with neurological deficits off the roads4). Various off-road tests to assess an individual’s fitness to drive include Trail Making Test, Stroke Driver Screening Assessment, Useful Field of View test, Cognitive Behavioral Driver’s Inventory, DriveAble, Clock Drawing Test, DriveSafe/DriveAware, computerized sensorimotor and cognitive tests4). Among these tests, the Trail Making Test part B (TMT-B) has been demonstrated, by several researchers, to be a good neuropsychological test for assessing fitness to drive in a laboratory environment5).

The Trail Making Test (TMT) consists of part A and part B in some neuropsychological tools used in clinical assessment. TMT-B is known to be a simple but effective tool for assessment of cognitive abilities like executive function, visuo-conceptual function, visuo-motor tracking and attention6). Betz and Fisher7) suggested that TMT-B be used as a short test for predicting future traffic accidents or driving test failures and occasionally in the evaluation of elderly drivers. Many studies have regarded the TMT-B as an easy and short test with ability to predict fitness to drive in the elderly or in cognitively impaired drivers7,8,9). These studies reported significant association between drivers’ involvement in road traffic accidents and their poor performance on the TMT-B. Other studies have suggested that elderly drivers who perform poorly on the TMT-B had a higher risk of involvement in traffic accidents9,10,11). Although the TMT-B test may be a simple and useful measure of executive function as a predictor of an individual’s fitness to drive, it does not give a clear picture of the function and is not an easy test to administer.

Simulators offer many advantages over conventional computerized neuropsychological tests. This includes their ability to present virtual environments or create realistic situations. They also give a clear picture of the executive function and ensure uniform testing conditions. We, therefore, developed a new test called the Driving TMT-B (DTMT-B) that presented a virtual environment in three-dimensional spaces to test the executive functions of drivers. This study aimed to examine the validity of the DTMT-B as a predictor of an individual’s fitness to drive.

SUBJECTS AND METHODS

Thirty stroke patients (right hemiplegia, 13; left hemiplegia, 17; age, 56.10, ± 9.40, mean ± standard deviation) and 65 healthy control subjects (age, 43.45 ± 15.40) participated in this study. The Stroke Participants’ sitting balance were normal and their brunstrom stage of arm and hand were 2.8 (SD=1.49) and 2.77 (SD=1.52), respectively (Table 1). The participants were inpatients and outpatients recruited at the Korean National Rehabilitation Center. Patients who (1) had visual neglect or apraxia, (2) had limited range of motion, (3) had seizure or epilepsy, (4) could not sit unaided, (5) were pregnant, (6) had psychiatric disease, or (7) had ataxia, were excluded from the study.

Table 1.

Demographic characteristics of study participants and task completion times

	All(N=95)	Stroke patients(N=30)	Healthy participants(N=65)
Male, n (%)	53 (55.8)	20 (66.7)	33 (50.8)
Female, n (%)	42 (44.2)	10 (33.3)	32 (49.2)
Age (years)	47.44 (14.95)	56.10 (9.40)	43.45 (15.40)
Education (years)	13.57 (3.73)	11.37 (4.00)	14.58 (3.14)
KMMSE Score	28.17 (2.35)	26.40 (3.21)	28.98 (1.14)
CTMT-B in seconds	47.77 (3.55)	85.80 (30.22)	30.22 (13.83)
MAS		1.60 (1.30)
Brunstrom Stage (Arm)		2.80 (1.49)
Brunstrom Stage (Hand)		2.77 (1.52)

All values except gender are represented as Mean (standard deviation). KMMSE: Korean Mini-Mental Status Examination, CTMT-B: Computerized Trail Making Test B (sub-item of CPAD), MAS: Modified Ashworth Scale

All participants were provided a detailed description of the study and informed consent was obtained from all patients prior to participation. Human subject ethical approval was obtained from the Institutional Review Board of Korea National Rehabilitation Center (NRCIRB-2012-01-004) prior to conducting the study.

Participants were screened by using the Korean Mini-Mental Status Examination (KMMSE)12) before enrollment. All participants were briefly interviewed to obtain demographic information and neurological history.

The computerized version of Trail Making Test part B (CTMT-B) emulates the paper and pencil version of the test on a computer screen. The Cognitive Perceptual Assessment for Driving (CPAD) developed by the Korea National Rehabilitation Center has one such CTMT-B embedded as a subtest. The CPAD is a computer-based assessment tool to assess the driving capacity of people with neurological deficits13). The CTMT-B presents 25 items including numbers and letters randomly scattered on an LCD screen. Participants are required to click alternately on numbers and letters in numerical and alphabetical order respectively. A total task completion time over 2 minutes was regarded as failing the CTMT-B. According to Park et al.13), the subjects’ CTMT-B scores were associated with driving performance such as car crash, vehicle positioning, and lane changes.

Traditionally, the TMT-B has been used as a measure of the executive function as well as of processing speed and visual scan14). It imposes strong cognitive demands.

The Korean version of TMT-B15) was used for the paper-and-pencil version of the test. The test required participants to connect a set of numbers and letters positioned randomly on an 8.5 by 11 inch sheet of paper with pencil lines. Participants were asked to alternate between numbers and letters while following numerical or alphabetical order respectively. Primary end point of the test was the time taken for completion of the task16). During the test, errors made by the participants were pointed out by the examiners and participants were redirected to the last correct selection and asked to continue the task from that point. The stopwatch used to measure the performance time continued even when participants made errors. A total task completion time exceeding 5 minutes was regarded as failing the TMT-B15).

DTMT-B is a driver screening assessment performed with a driving simulator incorporating a modified version of the TMT-B. The simulator set up consists of a table, two LED screens, a small steering wheel, an accelerator and a brake. All subjects were explained the task clearly and were allowed to perform in a practice session before the actual DTMT-B test. Participants performed a simulated driving on a virtual road of 2,674 m presented on one LED screen, while a map showing a moving point indicating the current location as well as the final destination were projected on the second LED screen. The virtual roads were in effect the TMT points and this task expected participants to drive along them rather than connect them with lines as in the paper or computerized TMT. Similar to the TMT-B, the DTMT-B required connecting, that is, driving alternately through the locations marked by numbers and letters in numerical and alphabetical order respectively. The driving speed was fixed at 40 km/h and was determined as optimal based on a pilot test conducted prior to the present study. When participants made errors, the computer program indicated the error and relocated the car to the last correct place. Errors made by a participant reflected in their total score as total task completion time. A total task completion time of over 15 min was regarded as failing the DTMT-B (Fig. 1).

Fig. 1.

Driving tools for Driving Trail Making Test-B (DTMT-B)

The dependent measures were the times required to complete the CTMT-B, the TMT-B and the DTMT-B respectively. For convergent validity, our study performed correlation analyses among the DTMT-B, the KMMSE, and the CTMT-B using the Pearson’s product correlation method. In the validation study, intra-class correlation coefficients (ICCs) were used for each task. ICCs values were graded as “poor to moderate” (less than 0.75) good (greater than 0.75) and “excellent” (greater than 0.90)17, 18).

RESULTS

Thirty stroke patients (right hemiplegia, 13; left hemiplegia, 17; age, 56.10 ± 9.40 years) and 65 healthy control subjects (age, 43.45 ± 15.40 years) participated in this study. The stroke participants’ sitting balance were normal and their brunstrom stage of arm and hand were 2.8 (SD=1.49) and 2.77 (SD=1.52) respectively (Table 1). The stroke group’s task completion time was significantly longer than that of healthy participants (Table 2, p<0.001). Significant correlations were found between TMT-B and DTMT-B (Table 3, r = 0.732, p < 0.001). The DTMT-B had moderate correlation with the KMMSE(r=–0.501, p<0.001). The ICC values were 0.66 (95% CI =0.465–0.763, p<0.001) among all participants, 0.574 (95% CI=0.106–0.797, p=0.012) among stroke patients, and 0.499 (95% CI=0.178–0.694, p=0.003) among healthy controls (Table 4).

Table 2.

Comparison between stroke patients and healthy participants

Measures	Stroke patientsMean (SD)	Healthy participantMean (SD)	p value
TMT-B (seconds)	233.10 (83.12)	108.38 (66.78)	<0.001
DTMT-B (seconds)	723.03 (246.38)	416.25 (187.97)	<0.001

TMT-B: Trail making test part B, DTMT-B: driving simulator running a modified version of the TMT-B

Table 3.

Pearson correlation coefficients between tests (N = 95)

	TMT-B	CTMT-B	KMMSE
DTMT-B	0.732*	0.729*	−0.501*

TMT-B: Trail making test part B, CTMT-B: Computerized Trail Making Test B (sub-item of CPAD), KMMSE: Korean Mini-Mental Status Examination, DTMT-B: driving simulator running a modified version of the TMT-B. * p<0.001

Table 4.

Interclass correlation coefficients between TMT-B and DTMT-B across groups

	ICCs	95% Confidence Intervals
		Lower	Upper
All participants
TMT-B vs. DTMT-B	0.644	0.465	0.763	*
Stroke patients
TMT-B vs. DTMT-B	0.574	0.106	0.797	*
Healthy controls
TMT-B vs. DTMT-B	0.499	0.178	0.694	*

TMT-B: Trail making test part B, DTMT-B: driving simulator running a modified version of the TMT-B, ICC: Interclass correlation coefficient, CI: confidence interval. *: p<0.05

DISCUSSION

This study found that the newly developed test DTMT-B test, which used a driving simulator, was a reliable instrument for evaluating cognitive abilities like executive function. The DTMT-B was demonstrated to be a good tool for evaluating cognitive abilities in a more realistic environment of driving than done by other variations of the TMT-B test. Many researchers have used driving simulators in the assessment of driving performance in individuals with various diseases and in elderly drivers because of the advantages offered by them9, 19,20,21). Moreover, many studies have found a correlation between driving simulator training and improvement in driving skills22, 23). In one of these studies, Roenker et al.23) reported that elderly drivers improved their driving performance skills. Gamache, Lavalliere, Tremblay, Simoneau and Teasdale reported a case study in which driving skills significantly improved in a patient with total brain injury (TBI) after 1 year of simulator training. This study also found an improvement in the patient’s driving skills and safety in real driving situations24).

The ICCs between DTMT-B and TMT-B was reliable (ICC=0.644). ICC is defined as the correlation between one measurement on a goal (either a single rating or a mean of several ratings) and another measurement obtained on that goal. In our study the ICC of all groups was reliable though it was different between stroke patients and healthy participants respectively (ICC = 0.574, 0.499). Even though DTMT-B was not found to be a highly reliable tool, the ICCs between two groups were valid regarding the discrete discrepancy in their cognitive abilities. The DTMT-B was more difficult to perform and complete than the TMT-B. However, it required the subjects to identify the route path in the 3 dimensional environmental spaces, a task in which the executive function must be employed.

Wang et al.25) developed the Rout Map Recall Test (RMRT) and validated it against the TMT-B. Cronbach’s alpha of the RMRT was 0.752 (p<0.001) and its correlation with the Chinese version of the TMT-B was −0.732 (p<0.001).

Though our study is similar in terms of using the TMT-B for testing participants’ memory, attention, and executive function, we combined a driving simulator with the TMT-B. While Wang et al.25) used a 2-dimensional, planar simulation, we used a simulated 3-dimensional space.

Driving simulator tests are safe as they avoid the risks associated with real driving, such as traffic accidents. They also have higher face validity than pencil-and-paper tests, such as the TMT-B26, 27). The use of a driving simulator also offers the possibility to observe and evaluate clients’ driving abilities in a virtual reality version of real road situations. A variety of studies have used driving simulators for evaluating and training elderly or people with disabilities28,29,30,31).

This study analyzed the DTMT-B which combined a driving simulator with the TMT-B, and the CTMT-B, a computerized version of the TMT-B, for the safe evaluation of drivers’ cognitive functions. This study also analyzed the reliability of these tests by comparing scores obtained in the paper version of the TMT-B.

The TMT is a simple and short neuropsychological test that assesses cognitive abilities such as visuo-conceptual and visuomotor tracking, sustained attention, and task alteration ability6). Other researchers have suggested that the TMT-B assesses processing speed, visual and motor integration, symbol recognition and sequencing, executive function and in addition to other abilities7). Ricker and Axelrod32) developed an oral version of the TMT (OTMT) that had significant correlation with the TMT-B (r = 0.72). In addition, Messinis et al.33) reported that the Color Trail Test had a strong correlation with the Greek TMT. Our research produced results similar to those of previous studies. Correlations between DTMT-B and TMT-B, between DTMT-B and CTMT-B were very strong (r=0.732, 0.729 respectively). This suggests that assessment of executive function for fitness to drive recommendations could be evaluated using the simulator version of the TMT-B.

This study has some limitations. Firstly, the small size of 30 stroke patients and 65 healthy participants is small and the results may not be representative of a larger population. Secondly, age and education level may have considerably influenced the subject’ DTMT-B performance in the present study.

The present study proved that the DTMT-B was effective as part of driver screening assessment for stroke patients and healthy participants for assessment of cognitive performance related to the executive function. This study was successful in developing the DTMT-B as a 3-dimensional virtual environment to test the executive functions of drivers and examined its validity. The DTMT-B strongly and reliably correlated with the TMT-B. The DTMT-B may be useful as part of driver screening assessment to assess the executive functions of stroke patients as well as healthy people.