Cut-off point for the trail making test to predict unsafe driving after stroke.

[Purpose] This study examined the cut-off point of the Trail Making Test in predicting the risk of unsafe driving in stroke patients.
[Subjects and Methods] A total of 81 stroke patients with a driver's license participated in this study. The DriveABLE Cognitive Assessment Tool, Trail Making Test-A, and Trail Making Test-B evaluations were conducted in all participants. All participants were classified into the safety or risk groups based on the DriveABLE Cognitive Assessment Tool evaluation results. The Trail Making Test results underwent a receiver operating characteristic analysis in each group.
[Results] The results of the receiver operating characteristic curve analysis showed that the cut-off point for Trail Making Test-A was 32 seconds and the cut-off point for Trail Making Test-B was 79 seconds. The positive predictive values of the Trail Making Test-A and Trail Making Test-B were 98.3% and 98.3%, respectively, and the negative predictive values of the Trail Making Test-A and Trail Making Test-B were 81.0% and 73.9%, respectively.
[Conclusion] The Trail Making Test is a useful tool for predicting the risk of unsafe driving in stroke patients. This tool is expected to be used more actively for screening stroke drivers with respect to their cognitive function.

INTRODUCTION

Driving cessation causes mobility decline, social isolation, and depression after stroke1). This neurological disease is associated with physical, cognitive, perceptual, and sensory dysfunctions2). Cognitive function is a particularly important element for driving safety in stroke patients3). Therefore, the evaluation system should test cognitive impairment, which is an invisible risk factor for unsafe driving4).

The on-road test is considered the “gold-standard” of testing driving function5). It is, however, difficult to widely use in the overall stroke patient population because of time-related and financial problems3). Alternatively, a simulator test system is used, but it is difficult to fully predict cognitive problems associated with driving6). Thus, driving-related cognitive assessment has been used as a modality for forecasting risk before driving7).

The most commonly used cognitive tests are TMT-A (Trail Making Test-A), TMT-B (Trail Making Test-B), UFOV (Useful Field of View Test), and MMSE-K (Mini Mental State Examination-K)7). The TMT measures attention, memory, sequencing, decision-making, and automatic thinking8, 9). Driving behavior of stroke drivers is influenced through sensory input, core cognitive processing, and higher-order processing. Core cognitive processing includes attention, perception, and memory while higher-order processing includes decision-making, planning, and automatic thinking3). Therefore, the TMT is a very useful evaluation to measure both core cognitive processing and higher-order processing7).

The TMT is known as a useful assessment tool that can predict unsafe driving in stroke and elderly drivers7, 10, 11). Additionally, if studies that can predict criterion validity are performed, it can be clearly used to predict the risk in a clinical trial because it provides a reference point for predicting the risk of unsafe driving. As an advantage, the TMT can determine the level of a patient’s risk of unsafe driving before a detailed driving assessment, such as the DCAT (The DriveABLE Cognitive Assessment Tool) for stroke and elderly drivers, is performed. Hence, studies have been performed that assessed the cut-off point for the TMT’s ability to predict the risk of unsafe driving in older drivers12, 13). In addition, a study that predicts the cut-off point for stroke drivers should be performed. This would make the TMT very useful for predicting the risk of unsafe driving in stroke drivers.

The purpose of this study is to identify the cut-off point to predict the risk of unsafe driving in stroke patients through the TMT evaluation.

SUBJECTS AND METHODS

Eighty-one subjects with stroke participated in the study. The study participants were receiving rehabilitation treatment at the K University rehabilitation center in Korea. Participants with a driver’s license, no visual problems, and no history of seizures or epilepsy within the last six months were included. All the subjects provided written informed consent according to the ethical principles of the Declaration of Helsinki. Table 1 presents the general and driving related characteristics of the participants.

Table 1.

General characteristics of the participants (n=81)

Characteristics		N (%)/mean ± SD
Gender	Male	64 (79.0)
	Female	17 (21.0)
Age (years)		56.22 ± 10.86
Disease period (months)		43.41 ± 51.84
TMT-A (seconds)		77.07 ± 79.39
TMT-B (seconds)		155.26 ± 135.79
Education	Illiterate	1 (1.2)
	Preschool	10 (12.3)
	Middle school	13 (16.0)
	High school	23 (28.4)
	Above high school	34 (42.1)
Type of stroke	Infarction	45 (55.6)
	Hemorrhage	36 (44.4)
Affected side	Right	42 (51.9)
	Left	39 (48.1)
Past driving experience	<5 years	14 (17.3)
	≥5 years	4 (4.9)
	≥10 years	63 (77.8)

The study period was from July 2013 to November 2014. The study was carried out in three stages. In the first stage, the DCAT, TMT-A, and TMT-B evaluations were conducted in all participants. In the second stage, the participants were classified into the safety or risk group based on the evaluation results of the DCAT. In the final stage, TMT-A and TMT-B results underwent a receiver operating characteristic (ROC) analysis in each group.

The DCAT is an in-office driving assessment system for predicting the driving risk in the on-road driving assessment14). It assesses the possibility of failing in the on-road driving evaluation and informs the central computer that analyzes the results of memory, attention, judgment, response time, decision making, and judgment of emergency situation tests. The DORE (The DriveABLE On-Road Evaluation) refers to this measured value15). In this study, patients were classified as safe or unsafe drivers. The TMT is divided into the A and B types. It measures the response time, attention, memory, sequencing, decision making, automatic thinking, etc. and is a cognitive test that has a high association with stroke drivers7).

PASW Statistics Version 18 (IBM Corporation, Armonk, NY, USA) and MedCalc Version 16.1 (LIONBRIDGE Inc., LA, USA) were used for statistical analysis. An ROC curve analysis and an area under the curve (AUC) estimate were used to assess the cut-off points for the TMT in stroke drivers. A p value <0.05 was considered statistically significant.

RESULTS

The result of the ROC curve analysis showed that the cut-off point for the TMT-A was 32 seconds. This score was located at the intersection point of sensitivity of 0.937 and specificity of 0.944. The Youden index of the score was 0.881 for the highest score. The cut-off point for the TMT-B was 79 seconds. This score was located at the intersection point of sensitivity of 0.905 and specificity of 0.944. The Youden index of the score was 0.849 for the highest score (Table 2). The AUC for the TMT-A was 0.978 and it was close to 1. The AUC for the TMT-B was 0.956 and it was close to 1 (Table 3).

Table 2.

Comparison of the cognitive and driving functions between SDG and USDG

TMT-A score	Sensitivity	95% CI	Specificity	95% CI	Youden index	+LR	−LR	PPV	NPV
30	0.968	0.890–0.996	0.833	0.586–0.964	0.802	5.81	0.04	95.300	88.200
31	0.937	0.845–0.982	0.889	0.653–0.986	0.825	8.43	0.07	96.700	80.000
32	0.937	0.845–0.982	0.944	0.727–0.999	0.881	16.86	0.07	98.300	81.000
42	0.762	0.638–0.860	0.944	0.727–0.999	0.706	13.71	0.25	98.000	53.100
TMT-B score	Sensitivity	95% CI	Specificity	95% CI	Youden index	+LR	−LR	PPV	NPV
77	0.921	0.824–0.974	0.889	0.653–0.986	0.810	8.29	0.09	96.700	76.200
78	0.905	0.804–0.964	0.889	0.653–0.986	0.794	8.14	0.11	96.600	72.700
79	0.905	0.804–0.964	0.944	0.727–0.999	0.849	16.29	0.10	98.300	73.900
106	0.603	0.472–0.724	0.944	0.727–0.999	0.548	10.86	0.42	97.400	40.500

TMT: trail making test, CI: confidence interval, +LR: positive likelihood ratio, −LR: negative likelihood ratio, PPV: positive predictive value, NPV: negative predictive value

Table 3.

AUCs for TMT-A and TMT-B

	AUC	95% CI	SE
TMT-A	0.978*	0.918–0.998	0.0149
TMT-B	0.956*	0.886–0.989	0.0252

*p<0.01, TMT: trail making test, CI: confidence interval, SE: standard error

DISCUSSION

The purpose of this research was to validate the predictive validity of the TMT for predicting the risk of unsafe driving in stroke patients and to determine the cut-off points. The results of the ROC curve analysis showed that the cut-off point was determined to the highest point of the Youden Index and the intersection point of sensitivity and specificity. These points were 32 seconds for the TMT-A and 79 seconds for the TMT-B. In a past study of patients with cognitive impairment, the cut-off points were 39.5 seconds for the TMT-A and 180 seconds for the TMT-B12). In another study of elderly drivers, the cut-off point for the TMT-B was 106.7 seconds12). The cut-off point for the TMT-B is different. This is considered to be because the TMT-B more strictly measures cognitive function than the TMT-A.

For the TMT-A, the PPV (positive predictive value) for accurately determining a safe driver was 98.3%, and the NPV (negative predictive value) for accurately determining an unsafe driver was 81.0%. For the TMT-B, the PPV was 98.3% and the NPV was 73.9%. A study predicted the driving risk of patients with cognitive impairment through the TMT evaluation, and it was observed that the PPV and NPV of the TMT-A were 77% and 62%, respectively, and the PPV and NPV of the TMT-B were 50% and 88%, respectively12). The values obtained in this study are high compared with those of previous studies. Stroke patients mainly use their non-affected side, which may also be assessed with TMT evaluation. Because this study was performed in specific subjects, as opposed to previous studies, the results are likely to be highly predictable.

These findings were confirmed in a prior study that investigated elderly drivers. In a research using the TMT to predict the risk of unsafe driving in older drivers, the sensitivities of the TMT-A and TMT-B were 73% and 77%, respectively, and the specificities of the TMT-A and TMT-B were 68% and 77%, respectively12). In comparison, the sensitivity and specificity of the TMT-A and TMT-B in this study were confirmed to be relatively high, i.e. 90.0% or higher. Unlike the elderly, stroke drivers can compensate for the loss of function in the affected side. Because of these features, it is thought that the predictive value of the assessment with the TMT is higher.

The AUC is the area under the ROC curve and a value closer to 1 is indicative of a correct diagnostic tool16). An AUC ≥0.9 indicates a very accurate tool; an AUC ≥0.7 indicates a moderately accurate tool; an AUC ≥0.5 indicates a marginally accurate tool; and an AUC ≤0.5 indicates a tool without discrimination17). In this study, AUC values for the TMT-A and TMT-B were 0.978 and 0.956, respectively. Hence, the TMT is a very accurate tool to predict the risk of unsafe driving in stroke drivers. However, this study did not determine the cut-off point for the TMT error values. Further studies on this issue are needed.