Comparison of the SIMARD MD to Clinical Impression in Assessing Fitness to Drive in Patients with Cognitive Impairment.

BACKGROUND: The assessment of fitness to drive in patients with cognitive impairment is complex. The SIMARD MD was developed to assist with assessing fitness to drive. This study compares the clinical decision made by a geriatrician regarding driving with the SIMARD MD score.
METHODS: Patients with a diagnosis of mild dementia or mild cognitive impairment, who had a SIMARD MD test, were included in the sample. A retrospective chart review was completed to gather diagnosis, driving status, and cognitive and functional information.
RESULTS: Sixty-three patients were identified and 57 met the inclusion criteria. The mean age was 77.1 years (SD 8.9). The most common diagnosis was Alzheimer's disease in 22 (38.6%) patients. The mean MMSE score was 24.9 (SD 3.34) and the mean MoCA was 19.9 (SD 3.58). The mean SIMARD MD score was 37.2 (SD 19.54). Twenty-four patients had a SIMARD MD score ≤ 30, twenty-eight between 31-70, and five scored > 70. The SIMARD MD scores did not differ significantly compared to the clinical decision (ANOVA p value = 0.14).
CONCLUSIONS: There was no association between the SIMARD MD scores and the geriatricians' clinical decision regarding fitness to drive in persons with mild dementia or mild cognitive impairment.

INTRODUCTION

Driving is a complex task involving cognition and physical dexterity. It is widely recognized that cognitive impairment may affect fitness to drive.( The Canadian Medical Association (CMA) has published guidelines (now in its 8th edition) to assist physicians with the assessment of driving capacity.( The guidelines state that moderate-to-severe dementia is a contraindication to driving. The CMA uses The Third Canadian Consensus Conference on Dementia guidelines( when describing patients with moderate and severe dementia as those who have an inability to independently perform multiple instrumental activities of daily living or any basic activity of daily living. Another method of assessing the stage of dementia can be accomplished using the Clinical Dementia Rating Scale,( with a score of 2 signaling a moderate stage and a score of 3 a severe stage. The CMA guideline is congruent with The Canadian Council of Motor Transport Association (CCMTA) Medical Standards for Drivers that also states that persons with moderate or severe dementia are “ineligible for any class of licence.”(

For physicians, determining fitness to drive becomes more complicated in patients with a diagnosis of mild cognitive impairment (MCI) or mild dementia. In this case, the CMA and CCMTA agree that these patients need to be evaluated on an individual basis to determine driving competence. The CMA guidelines state that currently there is no screening tool, which when used on its own, can accurately predict driving safety with acceptable specificity and sensitivity.( They recommend that, when fitness to drive is in question, the patient should be referred for a comprehensive off- and on-road test.

In many physician practices, assessment of fitness to drive in persons with dementia is based on clinical judgment using diagnostic information, driving history, and a functional history of their instrumental and basic activities of daily living, all of which are supplemented by bedside cognitive tests such as the Mini-Mental Status Examination (MMSE)( and the Montreal Cognitive Assessment (MoCA).( The importance of clinical judgment is supported by the American Academy of Neurology.( Using this usual approach to assess fitness to drive, physicians can: 1) choose to allow the patient to continue to drive, 2) ask the patient to voluntarily stop driving, 3) write a letter to the transportation authorities, suggesting the patient’s licence be revoked, or 4) request that the patient complete a driving assessment, either performed by a governmental agency or a third party. New Brunswick residents requiring a formal driving assessment are referred to the Department of Public Safety (DPS) to undergo a written and on-road driving assessment, which is the same driving assessment that any citizen of any age would complete. DriveABLE (developed at the Neurocognitive Research Unit of the Northern Alberta Regional Geriatric Program) is an example of a third-party driving assessment, but it was not available in New Brunswick at the time of this study. By law, physicians in New Brunswick must report drivers who are unfit to drive, but this is not the case for all Canadian provinces.(

Two comprehensive literature reviews suggest there is inadequate evidence to confidently support the use of a single screening tool to predict driving fitness in cognitively impaired individuals, citing lack of cut-off scores for specific tests and wide study design variability.( A recent systematic review of the Trails B test found that 32 of 47 studies found a positive correlation between Trails B scores and driving fitness; however, the remaining 15 studies did not find an association.( A publication in the Canadian Geriatrics Journal of CME cautions physicians who assume that one screening tool is adequate, and provides tips to evaluate driving screening tools.( Similarly, another publication provides a “Driving and Dementia Toolkit” for health professionals that can be used as a resource.(

In an attempt to develop an easy bedside tool to assess driving capacity in patients with cognitive impairment, the Screen for the Identification of the Cognitively Impaired Medically At-Risk Driver a Modification of the DemTect( (SIMARD MD) was developed.( The original DemTect was developed as a screening tool for MCI and early dementia, and included a number of subtests.( The SIMARD MD uses three of the six DemTect subtests: the Supermarket Task, Repeat of the Word List, and the Number Conversion test.( Based on the results of a regression analysis, these subtests were chosen as they were found to correlate best with driving ability using the DriveABLE On-Road Evaluation (DORE).(

The SIMARD MD has a maximum score of 130 points, with higher scores reflecting higher cognitive function.( Participants receive a score that classifies them into one of three categories (≤ 30 points) “high probability of failing a driving assessment,” (31–70 points) “referral for a driving assessment recommended,” and, (> 70 points) “low probability of failing a driving assessment.” In the study by Dobbs and Schnopflocher, there were two phases—the initial study and the validation study.( Both studies used a sample of individuals with cognitive impairment with or without dementia and a control group consisting of older, healthy, community-dwelling individuals.( The individuals with cognitive impairment were patients of community-based general practitioners who were referred to the study. The Dobbs study found that 86% percent of those who were predicted to fail the on-road driving test based on their SIMARD MD score subsequently failed in the initial study; similarly, 80% in the validation study.( Eighty-four percent of those predicted to pass the on-road test based in their SIMARD MD score subsequently passed in the initial study; 87% in the validation study.(

Bédard et al.( recently published a review of the SIMARD MD comparing its score to other cognitive measures and found that patients with a post-secondary education scored higher on the SIMARD MD, suggesting an educational bias. Similarly they found scores were significantly lower for older patients, suggesting a bias towards age, as well.(

Although research on the accuracy of clinical impression notes variability between health care providers,( the authors of this study chose to use it as a comparison to the SIMARD MD, as it is the current standard of care used by geriatricians in New Brunswick.

The purpose of this study is to determine whether there is an association between the clinical decision made by a geriatrician regarding fitness to drive in persons with mild dementia and MCI, compared to the SIMARD MD score.

METHODS

The study participants were patients seen by a geriatrician in an outpatient Geriatric Assessment Clinic or Memory Clinic between January and June of 2012, resulting in a convenience sample. Four geriatricians participated in this study, and they completed the SIMARD MD with their patients after they had made a clinical decision regarding driving fitness using their usual practice methods. Usual practice included examining diagnostic information, results of cognitive tests (MMSE, MoCA), driving history, caregiver concerns, and the patient’s functional status. Ethics approval for this study was obtained from the Horizon Health Network Research Ethics Board and the University of New Brunswick Research Ethics Board.

Patients included in this study had a diagnosis of mild dementia or MCI. The authors defined patients with mild cognitive impairment as those who had cognitive decline that did not impact their ability to perform basic or instrumental activities of daily living. The authors defined patients with mild dementia as those who were independent with their activities of daily living, but required some level of assistance with one or more instrumental activities of daily living. Patients excluded were those who did not have dementia, who had age-associated memory impairment or cognitive impairment secondary to a reversible cause, and those who never drove.

The charts were reviewed retrospectively, using a data collection form designed for this study. The information collected included demographic data, diagnostic information, functional status, driving history, and the results of any cognitive tests completed (MMSE and MoCA). Demographic data included age, gender, place of residence, and the diagnosis included the type of cognitive impairment. Functional status was based on general questions to the patient and caregiver during the regular clinical encounter and was recorded by noting the patient’s ability to perform his or her own activities of daily living and instrumental activities of daily living. When available, a detailed driving history was recorded, which included previous near-miss accidents, motor vehicle crashes, patient and family concerns of driving safety, and whether or not the patient had ever been referred for a driving test (along with the result).

The clinical decision made by the geriatrician was recorded as: continued to drive, asked to voluntarily stop driving, referred to Department of Public Safety for an on-road test, or revoked licence (although revoking a licence is ultimately the decision of the DPS).

The data were entered into SPSS 15.0, Minitab 16 and R 2.15.1, and they were used to analyze the data. The data were summarized using descriptive statistics. An ANOVA was carried out to compare the mean SIMARD MD, MMSE, and MoCA scores across the four clinical decision categories. All scores were square-root transformed prior to the analysis in order to satisfy the ANOVA assumptions. Fisher’s exact test of independence was used to test whether the SIMARD MD scores (grouped into the three designated SIMARD MD categories) were associated with the clinical decision made by the geriatrician.

RESULTS

Sixty-three patients had a SIMARD MD completed. Fifty-seven met our inclusion criteria. Of the six patients that were excluded, one had cognitive impairment related to depression that later reversed with treatment, one had age-associated memory impairment, three did not have dementia, and one never drove.

The mean age of the sample was 77.1 years, with a mean MMSE score of 24.9 (Table 1). All of the patients in the sample were independent in their basic activities of daily living. The majority of the sample was male and lived at home. The two most common diagnoses were Alzheimer’s disease and MCI (Figure 1).

TABLE 1.

Sample characteristics (N = 57)

Mean age in years (SD)	77.1 (8.9)
Mean MMSE (SD) n=53	24.9 (3.3)
Mean MoCA© (SD) n=39	19.9 (3.6)
Place of Residence:
Home	87.7%
Apartment	7.0%
Assisted Living	3.5%
Special Care Home	1.8%
Married	78.9%
Single	5.3%
Widowed	10.5%
Divorced	5.3%
Male	68.4%

Figure 1.

Cognitive diagnosis

Twenty-four patients (42.1%) had a score on the SIMARD MD of ≤ 30, indicating that these patients had a high probability of failing an on-road test. Only 8.8% scored > 70, suggesting that they had a high probability of passing an on-road test (Table 2).

TABLE 2.

SIMARD MD scores compared with clinical decision

	Still Driving	Asked to Voluntarily Stop Driving	Referred to Department of Public Safety	Revoked Licence	Total
≤30	8	9	1	6	24
31–70	13	4	7	4	28
>70	3	0	0	2	5
Total	24	13	8	12	57

p = 0.037.

The distribution of SIMARD MD, MMSE, and MoCA scores compared to the clinical decisions are depicted graphically in Figures 2A, 2B, and 2C. The overall mean SIMARD MD score was 37.2 (SD 19.5). The mean SIMARD MD (p value 0.14) and MoCA scores (p value 0.293) were not significantly different across the four clinical decision categories (Table 3). The mean MMSE scores, however, differed (p value 0.005) and further analysis, using Tukey’s multiple comparison procedure, revealed that the difference was due to the MMSE scores being significantly lower in the “Asked to Voluntarily Stop Driving” category when compared to each of the other categories. Pairwise comparisons of MMSE scores in the “Still Driving,” “Referred to DPS”, and “Revoked Licence” were not, however, significant.

Figure 2.

Clinical decision compared with (a) SIMARD MD score, (b) MMSE score, and (c) MoCA score

TABLE 3.

SIMARD MD, MMSE, and MoCA score compared to clinical decision

	Still Driving	Asked to Voluntarily Stop Driving	Referred to Department of Public Safety	Revoked Licence	p value
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
SIMARD MD	40.9 (20.1)	26.7 (12.4)	39.8 (12.8)	39.3 (25.6)	p=0.14
MMSE	25.8 (3.2)	22.1 (2.3)	25.8 (3.1)	25.6 (3.5)	p=0.005
MoCA	20.8 (2.8)	18.7 (3.1)	19.4 (2.7)	18.5 (5.6)	p=0.293

When comparing the association between the SIMARD MD and age, the Pearson correlation was −0.357 (p value = 0.006), suggesting a negative linear relationship. This suggests that lower SIMARD MD scores are associated with older patients. In addition when comparing the SIMARD MD and MMSE scores, there was a positive linear relationship (Pearson correlation = 0.658, p value < 0.001). This indicates that higher SIMARD MD scores were associated with higher scores on the MMSE.

There was no significant association between SIMARD MD scores (classified into the four categories) and the clinical impression made by the geriatrician (p value 0.073) (Table 2). When the categories “Asked to Voluntarily Stop Driving” and “Referred to Department of Public Safety” were combined, the association was also not significant (p value 0.28).

Of the patients who scored ≤ 30, eight continued to drive. Six had a diagnosis of Alzheimer’s disease, one had MCI, and one had cognitive impairment not yet diagnosed. They were all independent with their activities of daily living and none had had a documented near-miss or motor-vehicle crash. In two of the eight patients, the spouse stated that the patient needed more help with navigation on unfamiliar routes, but did not express concerns about the patient’s driving safety. Three of the eight patients had previously been referred for a driving assessment with the DPS and all three had passed.

Of the five patients who scored > 70, two had their licences revoked by the geriatrician. One of the patients had a diagnosis of vascular dementia and the other, frontal lobe dementia. Both were independent in their activities of daily living, but both had significant concerns raised by the family about their driving safety, and one had had a motor-vehicle crash. One had had a driving assessment completed outside of New Brunswick by DriveABLE and had failed this test.

When the sample was divided into patients with dementia of any type versus MCI (Table 4), 71.4% of the MCI group fell into the indeterminate range on the SIMARD MD, indicating that a large number of those with MCI would have been referred for an on-road driving test. This was not reflected in the clinical decision made by the geriatrician, where only one out of the 14 patients with MCI was referred for an on-road driving test.

TABLE 4.

Mild cognitive impairment versus all types of dementia

	MCI n=14	Dementia n=43
Age in years (SD)	76.1 (5.6)	77.4 (9.8)
Male	78.6%	65.1%
SIMARD MD	2 scored < 31	22 scored <31
	10 scored 31–70	18 scored 31–70
	2 scored >70	3 scored >70
MMSE (SD)	26.9 (3.3) n=13	24.2 (3.1) n=40
MoCA (SD)	21.3 (2.9) n=11	19.3 (3.7) n=28
Driving Decision	12 Driving	12 Driving
	1 Asked to Voluntarily Stop Driving	12 Asked to Voluntarily Stop Driving
	1 Referred to DPS	7 Referred to DPS
	0 Revoked Licence	12 Revoked Licence

Within the entire sample, 20 (35.0%) completed an on-road driving test. In those who scored ≤ 30, eight underwent a driver’s test and six passed. In those scoring 31–70 on the SIMARD MD, 11 had a driver’s test and six passed (Table 5).

TABLE 5.

Results of patients who had a driving test

SIMARD MD Score	Pass	Fail	Patient Did Not Complete Test	Result Unknown	Total
≤30	6	0	1	1	8
31–70	6	2	1	2	11
>70	0	1	0	0	1
Total	12	3	2	3	20

DISCUSSION

The results of this retrospective review found that the SIMARD MD was not associated with the clinical decision made by a geriatrician performed in the usual manner in patients with mild dementia or mild cognitive impairment. Similarly, the MoCA scores did not differ between clinical decision categories. The MMSE scores were lower for the patients who were classified as “Asked to Voluntarily Stop Driving”; however, there was no significant difference, for instance, in MMSE scores between patients assessed as “Still Driving” and “Revoked Licence.”

In a paper by Bédard et al. the authors concluded that age and education were correlated with SIMARD MD scores.( This study also found that age does affect the SIMARD MD, with lower scores in those who are older. Although we did not collect information on education in our study, this would be an interesting area to pursue in the future.

This study supports findings from the literature that a single assessment may not be ideal for assessing fitness to drive.( A recent publication reviewing an in office driving assessment tool that is endorsed by the American Medical Association known as the Assessment of Driving-Related Skills (ADReS) also found that it was not a test that should be recommended on its own to predict the need for an on-road driving assessment.( Although one, easy-to-administer, bedside tool, that can accurately predict driving fitness would be ideal, current research has focused on combining existing tools, with or without patient behavioural traits, to predict driving fitness.( In a recently published abstract, researchers surveyed dementia experts to create a list of patient-related factors, as well as cognitive screening tools that, when combined in particular scenarios, would warrant the physician to report the patient to the provincial driving authority and or request an on-road driving assessment.( From this research, Rapaport and colleagues have proposed a “Dementia and Driving: Decision Making Worksheet”, which uses a weighted scale for each of the eight items including: history of a crash, caregiver concern, abnormal clock drawing, a long Trails test (> 4.5 minutes), a low MoCA score (< 20), patient slowness, patient behavioural change, and low patient insight.( Another publication comparing a battery of tests to on-road driving performance in participants with a diagnosis of dementia found that, through a stepwise multiple logistic regression analysis, the combination of tools that was most useful in predicting failure on the on-road test included: the Eight-item Informant Interview to Differentiate Aging and Dementia, the Clock-Drawing Test, and the Trail-Making Test Part A or the Snellgove Maze Test.( It was suggested that the combination of these tests could be performed together in less than 10 minutes in any clinician’s office.(

Another finding of this study was similar to the Bédard et al. paper,( which showed a high number of patients who fell into the indeterminate range (49.1%). According to the SIMARD MD guidelines, this would warrant a referral for an on-road driving assessment. This is also consistent with the SIMARD MD initial and validation study findings, where 49.2% of participants in the initial study and 50.4% in the validation study fell into the indeterminate range.( These statistics suggest that a large percentage of patients who have a SIMARD MD completed by their physician would subsequently be recommended to take an on-road driving test.

A review by Hogan and Bédard concluded that it could not support the routine use of the SIMARD MD at present.( Two areas of concern identified by the review were the relatively high false-positive and false-negative rates.( In the SIMARD MD paper, 84% of those predicted to pass subsequently passed and 86% of those predicted to fail subsequently failed, translating into a 16% false-positive and 14% false-negative rate.( In the context of our study, by replacing the geriatrician’s clinical assessment for the DORE, 60% of those predicted to pass continued to drive, while 63% of those predicted to fail by the SIMARD MD had their licences revoked or volunteered to stop driving, translating into a 40% false-positive and 37% false-negative rate. The false-positive and false-negative rates in our study were high.

This study had several limitations. The design of our study was a retrospective chart review and we were limited to using a small convenience sample. While the overall result could be subject to a larger-than-desired probability of a type II error, informal calculations indicate that, at least for large-effect sizes, our sample size should have been sufficient to pick up a difference.

Another limitation of our study was the use of clinical decision-making as the standard by which to compare the SIMARD MD. One study demonstrated that a physician’s accuracy to predict driving ability based on clinical opinion is less than ideal.( In this study, participants were assessed for fitness to drive based on information provided by a chart review and a patient interview, including a clinical dementia rating, MMSE score, and history of previous motor vehicle crashes and traffic violations, as well as a general physical exam. With this information, physicians then rated participants as safe, unsafe or marginally safe to drive. Physician ratings were compared to the results of on-road testing and it was found that general practitioners accurately rated 62%–64% of the participants, while dementia specialists correctly categorized 72%–78% of patients.( Interestingly, a recent survey of specialists found that 95.6% of geriatricians felt that assessing fitness to drive was an important part of their practice, but only 30.4% were confident in their ability to evaluate driving.(

A final potential area for bias in this study was the specialty clinic setting that used the clinical opinions of four female geriatricians. A recent survey of physicians found that females were more likely to report unsafe driving, as compared to their male colleagues, and that geriatricians and cognitive neurologists were more likely to report unsafe driving than family physicians and geriatric psychiatrists.( This may limit the generalizability of the study.

This study used information gathered during usual clinical care in an outpatient setting. The data collected were limited to what was available on the chart, and the diagnoses of dementia and mild cognitive impairment were made using usual clinical judgment. Data about instrumental or basic activities of daily living were collected based on subjective information provided by the patient and/or family. While this method may call into question the accuracy of the information collected, it can also be argued that this represents a real clinical environment when specific diagnostic and functional tools are not always used.

Before adoption of a new tool into clinical practice, it is important to know that independent research groups have validated it and that it is truly measuring what it was intended to measure. This study, like the Bédard et al. study,( does not support the use of the SIMARD MD as a single bedside test to assess fitness to drive in persons with mild dementia or mild cognitive impairment.

CONCLUSION

There was no statistically significant association found in this study between the score on the SIMARD MD and the clinical driving decision made by the geriatrician in an outpatient setting in persons with mild dementia or mild cognitive impairment. Based on our findings, we would not recommend that the SIMARD MD be used exclusively to assist physicians in deciding fitness to drive in patients with mild dementia or MCI. More research is needed with this tool before it should be recommended for widespread use in a clinical setting. Comparison of this tool to more comprehensive approaches to the assessment of driving should also be considered.