The Ottawa 3DY Predicts Mortality in a Prospective Cohort Study.

Background: The Ottawa 3DY (O3DY) is a simple measure of cognition.
Objectives: 1) To determine if the O3DY predicts mortality; and 2) To compare the discrimination of the O3DY to the Mini-Mental State Examination (MMSE) and Modified MMSE (3MS).
Methods: Analyses of a population based cohort study of 1,751 participants aged 65+; conducted in 1991/2 with follow-up over five years. The O3DY, age, sex, education, comorbid conditions, the MMSE, and the 3MS were measured: 4.5% of the participants had missing data for the O3DY; 42.8% were considered as positive (one or more errors), and 52.7% were considered as negative (no errors). Logistic regression models were constructed with the outcome of death at time 2. A Receiver Operator Curve (ROC) was constructed and the Area Under the ROC (AUROC) was calculated using a c-statistic.
Results: The unadjusted odds ratio (OR) and 95% confidence interval (CI) for mortality was 1.96 (1.56, 2.47); and the adjusted OR was 1.33 (1.02, 1.72). The AUROC was 0.66 for the 3MS, 0.65 for the MMSE, and 0.60 for the O3DY. Conclusions: The O3DY predicts mortality over a long time frame, although the discrimination is less than that of longer measures of cognition.

INTRODUCTION

Impaired cognition is common in older adults, and predicts multiple adverse outcomes. Measures of cognition which have previously been reported to predict these adverse outcomes are: clinical diagnoses of dementia,( mild cognitive impairment (MCI),( cognitive impairment no dementia (CIND);( detailed neuropsychiatric tests,( and poor performance on cognitive status tests which require in person assessments( [such as the Folstein Mini-Mental Status Examination (MMSE) and the modified MMSE (3MS)]. Most older adults also value cognition highly. While current guidelines caution against routine population-based screening for cognitive impairment,( a high clinical index of suspicion is appropriate. In certain settings, such as those presenting with memory complaints, as part of comprehensive geriatric assessment, and in high prevalence settings, case finding of cognitive impairment may be appropriate. A wide variety of instruments are available to assess cognition. There has long been interest in simpler measures of cognition which can be rapidly completed in clinical encounters. With the advent of the Covid-19 pandemic, interest has increased in the use of brief cognitive measures which minimize contact time, and which can be completed without the use of potential fomites (such as pencils and paper). Some of these measures could also potentially be administered remotely by telephone or via teleconferencing software. The evidence regarding remote assessments is evolving rapidly. Currently, it appears that virtual assessments may be acceptable to some patients and caregivers,( and that some measures—such as the Telephone Interview for Cognitive Status (TICS)(—have acceptable diagnostic accuracy compared to in-person instruments, though clearly more research is needed.

The Ottawa 3DY (O3DY) is a simple test of cognition derived from items of the 3MS which do not require pen and paper, and are short and easy to administer (date, day of the week, spelling WORLD backwards, and the year).( It has been validated in the Canadian Study of Health and Aging (CSHA) data,( and subsequently studied in emergency department settings, where is has good sensitivity for detecting cognitive impairment, as well as acceptable specificity for detecting delirium(—with one study showing moderate sensitivity and specificity.( It has been studied in French as well as English.( The O3DY has not been shown to predict adverse outcomes over long time frames.

We have conducted an analysis of the Manitoba Study of Health and Aging (MSHA), a population-based cohort study, to determine if the O3DY predicts adverse outcomes over a five-year time frame. The specific objectives are:

To determine if the O3DY predicts mortality over a five-year period after adjusting for confounding factors; and

To compare the discrimination of the O3DY to the MMSE and 3MS in predicting death.

METHODS

The MSHA is population-based cohort study conducted in Manitoba, in conjunction with the CSHA.( The CSHA was a large epidemiological cohort study of those over 65 years old to determine the prevalence, incidence, and outcomes of dementia. Manitoba expanded the CSHA sampling frame to include all regions in the province, including rural areas, and supplementary questions were added. The original sampling frame was from a list provided by Manitoba Health, the provincial ministry of health. Because health-care coverage is universal in Manitoba, this represents a comprehensive representative sampling frame.( The sampling was stratified by region, with representation from the entire province; 56% lived in Winnipeg. The sampling frame was determined to oversample the oldest population, with a sample ratio of 2 for those aged 75 to 85 and 2.5 for those over 85, relative to those aged 65–75. Persons residing in institutions (nursing homes and chronic care hospitals) did not undergo the screening interview. Initially, 2,890 persons were selected, of whom 1,751 were enrolled. The initial time of assessment was 1991/2 with follow-up in 1996/7.

At both time points, participants who were living in the community were interviewed in their own homes by trained interviewers. Data gathered included age, sex, educational level (years of education), and a list of chronic conditions, including self-reported memory. The Modified Mini-Mental State Examination (3MS)( was used as the screening test for cognitive impairment and dementia. The 3MS extends the MMSE( to include additional items on date and place of birth, animal naming, similarities, and a second recall task, and expands the total score from 30 points to 100 points. The 3MS was conducted in a manner that allowed calculation of the MMSE.( The O3DY can be derived from the 3MS.

Death was ascertained at an interim contact at two years and at follow-up at five years. Data on death were considered at the time of the time 2 screening interview, at the time 2 clinical interview, and at the end of the entire time 2 period in 1997. We consider death at the time of the time 2 clinical interview. There were no missing data for death by 1997. Long-term care use—defined as nursing home residence or living in a chronic hospital—was ascertained at the time 2 screening interview. There were seven people missing data on long-term care use. There were 100 participants who entered long-term care and subsequently died whom we considered within the “dead” category, and 111 participants living in long-term care at the time 2 screening interview. Those who were waiting for long-term care in acute hospitals, and those who had been accepted for long-term care and were waiting in the community, were considered to be living in the community. For those participants who were still in the community, the assessment process and interview questions were similar to those at time 1.

The study adheres to the Declaration of Helskinki, and consent was obtained from the participants, or an appropriate proxy. The original study, as well as these analyses, received ethical approval from the Research Ethics Board of the Bannatyne Campus of the University of Manitoba.

Statistical Analyses

We calculated the O3DY from the 3MS. The primary outcome was death at the end of the follow-up period. The secondary endpoint was death or long-term care admission. We conducted bivariate analyses with the O3DY as a binary variable. For categorical variables, we used chi-square tests, and for continuous variables, we used Student’s t-tests (assuming unequal variance). As a supplementary analysis, we considered the O3DY as a categorical variable (0 to 4 points). We also examined each item (orientation to date, day of the week, and year, as well as the ability to spell WORLD backwards) as a predictor variable. We then constructed logistic regression models with death as our primary outcome and the O3DY as the variable of interest. We included potential confounding variables as covariates in the logistic regression model. Finally, we constructed Receiver Operator Curves (ROC) for the effect of the O3DY on mortality, and calculated a c-statistic for the area under the ROC for the O3DY, the MMSE, and the 3MS. For the secondary outcome of long-term care use, we did not construct regression models. All analyses were conducted in SPSS Version 25 (IBM SPSS Statistics, Armonk, NY).

RESULTS

There were 1,751 participants who were enrolled in 1991/2. Of these, 79 were excluded due to missing data; 77 of these were missing data on spelling WORLD backwards. Most of the participants with missing data had low levels of literacy and were unable to spell WORLD forwards. Overall, 4.5% of the participants had missing data for the O3DY; 42.8% were considered positive (errors), and 52.7% were considered negative (no errors). The characteristics of the sample are shown in Table 1. Those who scored positive on the O3DY were older, had lower educational attainment, and were more likely to be male. Unsurprisingly, they also had lower mean scores on the overall MMSE and 3MS. When considering each item, 70.4% spelled WORLD backwards correctly, 96.1% accurately identified the year, 94.2% accurately identified the day of the week, and 76.1% accurately identified the date. If the O3DY was considered as a categorical score, 0.7% scored 0; 2.3% scored 1; 9.3% scored 2; 30.6% scored 3 and 52.7% scored 4; with 4.5% missing.

TABLE 1

Baseline characteristics of the sample

	O3DT Positive (N=750)	O3DY Negative (N=922)	O3DY Missing (N=79)	Total Sample (N=1751)
Age (Mean Years, SD)	77.7 (7.2)a	74.7 (6.6)a	79.2 (8.1)a	76.2 (7.1)
Sex (% female)	397 (52.9%)a	591 (64.1%)a	37 (46.8%)a	1025 (58.5%)
Education (Mean Years, SD)	8.6 (3.4)a	10.3 (3.2)a	4.2 (3.6)a	9.3 (3.6)
Hypertension (%)	240 (32.0%)	317 (34.5%)	30 (38.0%)	587 (33.6%)
Heart Disease (%)	225 (30.0%)	272 (29.5%)	29 (36.7%)	520 (30.0%)
Stroke (%)	57 (7.6%)a	54 (5.9%)a	9 (11.4%)a	120 (6.9%)
Diabetes (%)	74 (9.9%)a	65 (7.1%)a	12 (15.4%)a	151 (8.6%)
Cancer (%)	53 (7.1%)	64 (7.0%)	4 (5.1%)	121 (6.9%)
SML (%)	252 (33.8%)a	177 (19.2%)a	27 (34.6%)a	456 (26.1%)
3MS (Mean, SD)	80.7 (11.2)a	90.5 (6.6)a	68.3 (16.3)a	85.3 (11.1)
MMSE (Mean, SD)	24.6 (3.3)a	28.1 (1.6)a	19.5 (4.4)a	26.2 (3.5)

Denotes p .05

O3DY = Ottawa 3DY; SML = subjective memory loss, 3MS = Modified Mini-Mental State Examination; MMSE = Mini-Mental State Examination.

Over the course of the study, 417 of the participants died: 29.6% of those who scored positive vs. 17.7% who scored negative and 40.5% of those with a missing 03DY (p < .001, chi square test.) The effect on mortality appeared to be a gradient across the score of the O3DY, if we considered the score as a categorical variable; as did the effect on long-term care admission (Figure 1).

FIGURE 1

Ottawa 3DY score and outcomes five years later

In logistic regression models (Table 2), the O3DY predicted mortality, even after adjusting for a variety of potential confounding factors. Male sex, older age, and self-reported heart disease, stroke, diabetes and cancer also predicted death. The lack of an effect of education should be viewed with caution, since in this data set, education, and cognition are correlated, and the effect of education may be mediated through higher cognitive test scores.

TABLE 2

Results of logistic regression models for five year mortality; the odds ratios (ORs) and 95 per cent confidence intervals (95% CI) are shown

	OR (95%CI)	OR (95% CI)	OR (95% CI)
O3DY (ref=negative)	1.96 (1.56, 2.47)	1.38 (1.07, 1.78)	1.33 (1.02, 1.72)
Age (per year)		1.11 (1.09, 1.13)	1.10 (1.08, 1.13)
Sex (ref=women)		1.74 (1.36, 2.22)	1.70 (1.32, 2.20)
Education (per year)		0.98 (0.94. 1.01)	0.98 (0.94, 1.02)
Hypertension (ref=no)			0.92 (0.71, 1.22)
Heart Disease (ref=no)			1.51 (1.15, 1.97)
Stroke (ref=no)			1.49 (0.95, 2.34)
Diabetes (ref=no)			1.73 (1.14, 2.62)
Cancer(ref=no)			1.70 (1.10, 2.64)
SML (ref=no)			1.24 (0.95, 1.63)

We considered the O3DY as a continuous score from 0 to 4, and constructed ROC curves for the O3DY, the MMSE, and the 3MS. All three scores predicted mortality with a moderate degree of discrimination; the c-statistic for the 3MS was 0.66; the MMSE was 0.65; and the O3DY was 0.60. Both the 3MS and the MMSE offered superior discrimination compared to the 03DY.

DISCUSSION

We have analyzed a prospective population-based cohort study and found that the O3DY predicts mortality, even after adjusting for potential confounding factors. This is a simple, easily administered test which can be conducted without physical contact, which may be useful in some clinical settings. There remain some advantages to considering more detailed cognitive tests in other settings. First, the percent of the population who scored positive on the O3DY was quite high. The intent of the O3DY is to identify those who require more in-depth assessment, and this may be a high percentage of individuals in a general population of older adults, and would likely be even higher in clinical settings. Second, the discrimination of the 03DY is less than the MMSE or the 3MS. This may be due to the expanded scoring within the normal range, which may be prognostically important. None of the tests used alone is discriminative enough to be used alone to predict death. On the other hand, all the cognitive scores are surprisingly accurate predictors of five-year mortality, given their brief and simple nature.

There are some limitations to our study. First, there is some overlap of participants in the MSHA with the CSHA from which the O3DY was derived. This overlap is one-quarter of the total number of participants who participated in both studies. Second, the data set is old. While the association between cognition and death has not likely changed in the intervening time, the baseline mortality rate in Canada has fallen since the time of the study. Third, Manitoba in general, and the participants in the MSHA in particular, had lower educational attainment than the Canadian population and the CSHA participants. Since the time of the MSHA, educational attainment has increased. This may limit generalizability. On the other hand, more modern populations may have fewer missing items as literacy has increased. Fourth, while there were no missing variables for vital status at time 2, there were missing items in our data set—notably on spelling WORLD. These participants had a high mortality rate, and data were not missing at random. Strengths of the study include a large representative sample of older adults using standardized measures gathered in a systematic manner.

Our findings are important for several reasons. First, our findings underscore the importance of cognition as an important prognostic marker.( Despite being recognized in Hippocrates time,( many currently used prognostic indicators do not consider cognition. Second, even brief bedside measures of cognition predict adverse outcomes. While more detailed measures offer more information, even these three basic measures alone offer surprisingly accurate information. More studies in both population-based studies and in clinical samples are warranted to determine if our findings are consistent in other places and times and in other cultures. We do not advocate using cognitive test scores alone to predict death, as their discriminative accuracy is modest. But cognition—even crudely measured—should be one consideration in predicting risk for future mortality.

The O3DY should be studied further for use in the Covid-19 era. As well, its accuracy if administered by telephone or video link as well, as in primary care settings, should be studied. This would allow brief cognitive assessments to be conducted while potentially minimizing viral transmission risk.