Registration of attentional function as a predictor of incident delirium (the RAPID study).

INTRODUCTION: Older adults undergoing elective surgery have a high risk of developing postoperative delirium (POD). Validated models predicting POD are scarce. This study investigated whether preoperative impairment of attentional function predicts POD in older adults without previously diagnosed cognitive impairment.
METHODS: In this prospective cohort study we recruited patients aged ≥70 years preceding major elective surgery. Preoperatively a visual vigilance test was administered to determine intra-individual reaction-time variability. Postoperatively, presence of delirium was screened daily.
RESULTS: We recruited 152 patients, 25 (16.4%) developed POD. Intra-individual reaction-time variability was not significantly different between patients with or without POD (0.18 ± 0.08 ms vs 0.22 ± 0.11 ms; P = 0.087). Receiver operating characteristic analyses indicated a poor accuracy for POD (area under the curve 0.609 ± 0.63). Except for surgery duration, no clinically significant between-group differences were found for secondary outcome parameters. DISCUSSION: Preoperative intra-individual reaction time variability does not predict the incidence of POD in older patients undergoing major elective surgery.

INTRODUCTION

The risk of developing postoperative delirium (POD) after elective surgery among older adults ranges from 3.6% to 50.0% , and is associated with an increase in length of hospital stay, complication rates, mortality, and higher health‐care expenditures. , , , , Although an array of risk factors for POD has been identified, validated models predicting POD in individual patients are scarce, , , , , while such measures might have the potential for a personalized approach aimed at tailor‐made POD risk reduction in the perioperative period.

Moreover, studies tend to focus exclusively on demographic characteristics and comorbid conditions, whereas risk assessment of POD by using individual pathophysiological markers involved in the evolution of POD are absent. Previous research suggests that impairment of attentional function might serve as an early and specific individual predictor of incident POD, even in previously cognitively undisturbed patients. , , Preoperatively administered attention‐based cognitive tasks, including sustained visual attention, have been used as part of neuropsychological test batteries for the prediction of subsequent POD. , Fluctuations and altered attention are hallmarks of delirium, and so it makes sense that pre‐surgery markers of attention and variations in attention would be predictive for POD, perhaps from a cognitive reserve prospective. However, only one study, performed by Lowery et al., has prospectively assessed preoperative intra‐individual reaction‐time variability as a sole predictor of POD. This study was limited to older (age ≥70 years) patients undergoing elective hip and knee replacement, and it showed significantly higher preoperative intra‐individual reaction‐time variability among patients developing POD. No attempts to reproduce these findings in other surgical populations have been published to date, limiting the external validity and practical implementation of these observations.  We hypothesized that intra‐individual reaction time variability is predictive for POD in older adults, without previously diagnosed cognitive impairment, undergoing elective major surgery. A clinically easy‐to‐use test, measuring attentional function, might potentially help to preoperatively identify individual patients at an increased risk of developing POD, and thus create a target for future prevention.

METHODS

Study design and ethics

The Registration of Attentional Function as a Predictor of Incident Delirium (RAPID) study was a single center, observational, prospective cohort study. The study was approved by the local medical ethics committee of the Amsterdam University Medical Centers, location AMC (METC AMC, the Netherlands; protocol number NL47720.018.014). Written informed consent was obtained from all patients before the start of the study. The study was performed according to the Declaration of Helsinki. The trial was registered with ClinicalTrials.gov, number NCT03988179.

Study population and inclusion criteria

Subjects were recruited among patients visiting the outpatient clinic of the Department of Anesthesiology of the Amsterdam University Medical Center (UMC), Location Academic Medical Centre (AMC) in Amsterdam for preoperative assessment preceding elective surgery. Patients aged 70 years or older who were scheduled for elective major surgery were screened for eligibility between April 2014 and May 2015. Exclusion criteria were cognitive impairment, a language barrier hindering informed consent or instructions, and/or a serious functional disability of the dominant hand (eg, palsy, amputation, arthrodesis).

HIGHLIGHT

The incidence of postoperative delirium (POD) in older adults without previously diagnosed cognitive impairment undergoing elective major surgery is ≈15%.

Prediction models for POD based on individual patients’ pathophysiological markers are promising.

Preoperative intra‐individual reaction time variability does not predict the incidence of POD.

RESEARCH IN CONTEXT

Systematic review: The authors reviewed current literature using PubMed and Medline and titles and abstracts were screened and cross‐referenced for eligibility. While much is written regarding postoperative delirium (POD), validated models predicting POD by using individual pathophysiological markers are not yet as widely studied. Relevant recent publications describing such models are appropriately cited.

Interpretation: Validated models predicting POD in individual patients are scarce and tend to be limited to demographic characteristics and comorbid conditions. Based on previous literature we hypothesized that intra‐individual reaction time variability is predictive for POD in older adults, without previously diagnosed cognitive impairment, undergoing elective major surgery. This hypothesis is consistent with previous studies.

Future directions: Even though prediction models for POD based on individual patients’ pathophysiological markers are promising, this study shows that preoperative intra‐individual reaction time variability does not predict the incidence of POD in older patients undergoing major elective surgery.

Measurements

After obtaining written informed consent from eligible patients, a short preoperative test of attentional function was administered at the outpatient clinic of the Department of Anesthesiology of the Amsterdam UMC—Location AMC (n = 45/29.6%) or at the ward, at latest the day before surgery (n = 107/70.4%). When testing was done at the outpatient clinic there was an average of 13 days between testing and surgery. Testing on the ward was done the night before surgery. We developed a visual vigilance test based on the Digit Vigilance test, a test originally derived from the Continuous Performance Test by Rosvold et al., and which was presented on a tablet computer. The Digit Vigilance test was first used in patients with delirium by Lowery et al. Our test was not specifically validated because it's a very commonly used test from the Cognitive Drug Research computerized assessment system, which is validated as an evaluation tool for assessing attentional performance in the general population and among people with dementia. , , ,

Stimuli consisted of a one‐digit number (0 to 9) presented in the middle of the screen. For each subject the application randomly selected a specific number as the target stimulus. Patients were instructed to keep their dominant trigger finger close to the screen and press the touch‐screen button, located at the bottom of the screen, as fast as possible whenever their target appeared. In case another stimulus appeared, the patient was instructed to withhold action. The stimulus disappeared after the button was activated, or if no reaction was given within 3 seconds. Stimuli were randomly presented with a 1‐, 2‐, 3‐, 4‐, or 5‐second inter‐stimuli‐interval blank, which randomly varied in blocks of five. After an eight‐stimuli practice trial, the test started and a total of 50 stimuli were presented with a 50% target prevalence rate. Full test duration was ≈5 minutes. During the test, reaction time and the accuracy of response were registered.

Relevant demographic data were obtained. Preoperative use of anti‐cholinergic medication was evaluated using the anti‐cholinergic burden scale (ACB). Visual acuity was measured using the standardized Snellen‐test. Preoperative cognitive functioning was determined using the Mini‐Mental State Examination (MMSE). Preceding cognitive decline was assessed using the 16‐item short version of the Informant Questionnaire on Cognitive Decline (IQCODE‐N). When results on the MMSE or the IQCODE‐N were suggestive of presence of dementia (<24 points, or >3.4 points, respectively), the Clinical Dementia Rating scale was administered, and patients with a subsequent score of one were excluded from participation. The presence of depressive symptoms was determined using the short version of the Geriatric Depression Scale (GDS‐15). , Activities of daily living (ADL) were measured using the Barthel Index (BI). Finally, the Lawton instrumental activities of daily living (IADL) scale was used to determine IADL. All tests, including the preoperative test of attentional function, were administered in the same preoperative sitting.

After admission to the hospital, and until 5 days post‐operatively or discharge, whichever came first, the presence of POD was screened daily by a rotating group of blinded assessors (trained research assistants) using the Confusion Assessment Method (CAM). Type of surgery, operating times, perioperative anesthetic data, and total days spend in our closed intensive care unit (ICU) were obtained for all patients. In the ICU, family could be present day and night if so desired. Postoperative complications and the use of benzodiazepines, neuroleptics, opioids, and anti‐cholinergic drugs according to the ACB were registered. Pain was assessed pre‐ and postoperatively using the numeric rating scale.

Outcome parameters

The primary outcome was between‐group difference of preoperative intra‐individual reaction‐time variability in POD versus non‐POD patients. As secondary outcomes we assessed differences in preoperative individual accuracy of reaction‐time response and the sensitivity and specificity of a combined index (ratio) of preoperative intra‐individual reaction‐time variability and accuracy of response in predicting POD. Finally, we determined between‐group differences in baseline characteristics and ACB, MMSE, IQCODE‐N, GDS‐15, BI, and Lawton IADL scale scores.

Sample size calculation

The study by Lowery et al. examining the difference in preoperative mean intra‐individual reaction‐time variability between POD and non‐POD patients found a 4‐ to 5‐fold increased risk of POD when subjects scored one standard deviation (SD) above the sample means. To be able to find a difference in mean preoperative reaction‐time variability between both groups with an effect‐size of 0.9 (Cohen's d) at a significance level of 0.05 (ß 0.8) in this study, 21 subjects had to be included in each group.

Previous studies examining the incidence of POD in older adults undergoing elective surgery show a risk of POD ranging from 7% to 27% among preoperatively cognitively normal patients. , , , , In this study, we estimated the risk of developing POD to be 15%. To include at least 21 patients developing POD, a total sample size of 140 patients would be needed.

Statistics

Patients were divided into two groups. The first group included patients who did not develop POD (non‐POD); the second group included patients who developed POD for at least 1 day. Pairwise comparisons were performed for all baseline characteristics and preoperative tests. In case of normally distributed data the Student t test was performed and expressed as means with SD. In non‐normally distributed data, either the Mann‐Whitney U or chi‐square test was performed, and data are expressed as medians with interquartile ranges. To evaluate the sensitivity and specificity of the visual vigilance test, a receiver operating characteristic (ROC) curve was plotted.

All statistical analyses were performed using SPSS software (version 25.0; IBM Corp., Armonk, New York, USA).

RESULTS

Study cohort and patient characteristics

A total of 320 patients were invited for the study, of which 172 patients provided informed consent. Of these, 20 patients were excluded and 152 patients were included in the study. The reasons for exclusion are listed in Figure 1. Male subjects had a mean age of 75.4 ± 4.5 years and female subjects had a mean age of 76.44 ± 4.41 years (Table 1). Reasons for surgery were cardiothoracic surgery (55.9%), major abdominal surgery (31.6%), major pelvic surgery (9.2%), or major orthopedic surgery (3.3%) and 80 patients spent at least one night in the ICU (52.6%). A total of 25 patients developed POD (16.4%). No patients were lost to follow‐up.

FIGURE 1

Reasons for exclusion. CDR, Clinical Dementia Rating scale; IQCODE, Informant Questionnaire on Cognitive Decline; MMSE, Mini‐Mental State Examination

TABLE 1

Baseline characteristics

	No postoperative delirium (n = 127)	Postoperative delirium (n = 25)	P (two tailed)
Age [mean ± SD]	75.4 ± 4.5	76.4 ± 4.4	P = 0.277
Sex (male n, %)	72 (57.7%)	15 (60.0%)	P = 0.760
Type of surgery (n, %)
Cardiothoracic surgery	70 (55.1%)	15 (60.0%)
Major abdominal surgery	40 (31.5%)	8 (32.0%)	P = 0.797
Major pelvic surgery	13 (10.2%)	1 (4.0%)
Major orthopedic surgery	4 (3.1%)	1 (4.0%)
Anesthesia type (n, %)
General anesthesia:	102 (80.3%)	18 (72.0%)
Inhalation anesthetics	97	18	P = 0.355
TIVA	5	0
General anesthesia + epidural	23 (18.1%) 21	7 (28.0%)	P = 0.320
Inhalation anesthetics	21	7
TIVA	2	0
Spinal anesthesia + sedation	2 (1.6%)	0	P = 0.531
Duration of surgery [mean ± SD]
Total operating time	225.7 ± 92.4 min	268.2 ± 114.5	P = 0.046a
Cardiothoracic surgery	222.6 ± 59.4 min	258 ± 66.6 min	P = 0.044a
Major abdominal surgery	265.1 ± 123.1 min	321.8 ± 163.7 min	P = 0.265
Major pelvic surgery	137.3 ± 40.2 min	160 min	P = 0.598
Major orthopedic surgery	149 ± 92.2 min	99 min	P = 0.661
Days in ICU (median ± IQR)	1.00 ± 2 days	1.00 ± 2 days	P = 0.282
Mean perioperative glucose (median + IQR)	7.66 ± 2.04 mmol/l	7.16 ± 1.95 mmol/l	P = 0.185
Δ perioperative glucose (median ± IQR)	1.14 ± 1.97 mmol/l	0.79 ± 1.74 mmol/l	P = 0.267
Alcohol use (n,%)	55 (43.3%)	14 (56.0%)	P = 0.246
Tobacco use (n,%)	11 (8.7%)	4 (16.0%)	P = 0.262
Diabetes (n,%)	23 (18.4%)	2 (7.4%)	P = 0.082
ASA classification (n, %)
ASA I	8 (6 3%)	1 (40%)	P = 0.15
ASA II	49 (38.6%)	6 (24.0%)
ASA III	70 (55.1%)	18 (72.0%)

ASA, American Society of Anesthesiology; ICU, intensive care unit; IQR, interquartile ranges; SD, standard deviation; TIVA, total intravenous anesthesia.

Reaction time

There were no sex differences between men and women for preoperative intra‐individual reaction time (957.7 ± 260.8 ms vs 1042.2 ± 287.3 ms; P = 0.06), preoperative intra‐individual reaction time SD (190.2 ± 111.4 ms vs 188.2 ± 104.4 ms; P = 0.915), and ratio (0.19 ± 0.09 ms vs 0.18 ± 0.08 ms; P = 0.281).

No statistically significant differences were found in patient errors and omissions (0.43 ± 0.913 vs 0.6 ± 0.957; P = 0.193) between groups. Preoperative intra‐individual reaction time (997.0 ± 283.5 ms vs 977.7 ± 229.9 ms; P = 0.899), preoperative intra‐individual reaction time SD (181.9 ± 99.5 ms vs 227.2 ± 140.6 ms; P = 0.197), and ratio (0.18 ± 0.08 ms vs 0.22 ± 0.11 ms; P = 0.087) showed no statistically significant differences between groups (Table 2).

TABLE 2

Mean intra‐individual reaction time, standard deviation, and ratio

	No postoperative delirium (median ± IQR)	Postoperative delirium (median ± IQR)	P (two tailed)
Preoperative intra‐individual reaction time	997.0 ± 283.5ms	977.7 ± 229.9 ms	P = 0.899
Preoperative intra‐individual reaction time standard deviation	181.9 ± 99.5ms	227.2 ± 140.6 ms	P = 0.197
Ratio	0.18 ± 0.08 ms	0.22 ± 0.11 ms	P = 0.087

IQR, Interquartile ranges.

To assess the sensitivity and specificity of this predictive test, a ROC curve was plotted. The area under the curve was 0.609 ± 0.63 (95% confidence interval [CI] = 0.485 to 0.732; P = 0.087), indicating a poor predictive test for POD (Figure 2). As the predictive value of the test is most important, a cut‐off point of 0.155 ms was used favoring sensitivity over specificity, instead of using Youden's index to maximize sensitivity and specificity. For this cut‐off value the sensitivity is 0.720 (72%; 95% CI = 54.4% to 89.6%) with an associated specificity of 0.441 (44.1%; 95% CI = 35.5% to 52.7%). Further analyses of the test characteristics, using a cut‐off value of 0.155 ms, show the test has a positive likelihood ratio of 1.29 and a negative likelihood ratio of 0.64. The positive predictive value of the test is 20.2% with a negative predictive value of 88.9% (Table 3).

FIGURE 2

Receiver operating characteristic curve

TABLE 3

Test characteristics

Ratio	No postoperative delirium	Postoperative delirium
>0.155ms	71	18	89	LR+ = 1.29
				PPV = 20.2%
<0.155ms	56	7	63	LR− = 0.64
				NPV = 88.9%
	25	127	152
	Sensitivity 72%		Specificity 44.1%
	95% CI 54.4% to 89.6%		95% CI 35.5% to 52.7%

Cut‐off value of 0.155 ms.

Secondary outcome parameters

Statistically significant between‐group differences were found for the preoperative IQCODE score (3.08 ± 0.19 vs 3.10 ± 0.31; P = 0.038) and the total (225.7 ± 92.4 minutes vs 268.2 ± 114.5 minutes, P = 0.046) and cardiothoracic operating times (222.6 ± 59.4 minutes vs 258 ± 66.6 minutes, P = 0.044). There were no statistically significant differences between POD and non‐POD patients for visual acuity (P = 0.536), education level (P = 0.739), preoperative test scores (Table 4), and other baseline characteristics (Table 1). A sensitivity analysis was performed with and without cardiothoracic surgery, which did not yield any statistically significant differences (P = 0.797 vs P = 0.784). Perioperative glucose measurements (Table 1) and the use of pre‐ and postoperative opioids (P = 0.876), benzodiazepines (P = 0.168), and anti‐cholinergic medication (P = 0.147) were similar in both groups.

TABLE 4

Test and questionnaire scores

	No postoperative delirium (median ± IQR)	Postoperative delirium (median ± IQR)	P (two tailed)
MMSE score [median ± IQR]	27.80 ± 1.62	27.36 ± 1.75	P = 0.252
IQCODE score [median ± IQR]	3.08 ± 0.19	3.10 ± 0.31	P = 0.038
GDS score [median ± IQR]	1.86 ± 2.01	2.28 ± 1.75	P = 0.093
BI score [median ± IQR]	19.63 ± 1.27	19.36 ± 1.75	P = 0.932
IADL score [median ± IQR]	10.10 ± 2.38	10.00 ± 2.77	P = 0.577
Postoperative ACB score [median ± IQR]	1.10 ± 1.06	1.20 ± 1.14	P = 0.895

ACB, anti‐cholinergic burden scale; BI, Barthel Index; GDS, Geriatric Depression Scale; IADL, Lawton IADL scale; IQCODE, Informant Questionnaire on Cognitive Decline; IQR, interquartile range; MMSE, Mini‐Mental State Examination.

DISCUSSION

In this prospective cohort study, pre‐operative intra‐individual reaction time, reaction time variability, and ratio did not differ between older adults with and without POD undergoing major surgery. In addition, no relevant differences were found when studying confounding factors, including ACB, preoperative cognitive decline, depressive symptoms or impairment in ADL.

Previous research has shown that complex attention, tested as part of a neuropsychological test battery, differed significantly between POD and non‐POD patients. However, this study was not powered to predict POD based on specific neuropsychological test results. Only one study, performed by Lowery et al., prospectively studied preoperative intra‐individual reaction‐time in relation to POD. Our findings are in contrast with this study, which showed a 4‐ to 5‐fold greater risk for developing POD when patients had increased reaction times or fluctuations in attention.14 The visual vigilance test used in our study was comparable to the test administered by Lowery et al., although we focused solely on digital vigilance. Both these studies studied slightly different techniques and tasks, and both reported significant results. However, it seems unlikely that slight variances explain the very noticeable differences in POD risk for attentional function. The researchers chose to pursue this simple test, consisting of a simple task, in the hopes of finding a truly easy‐to‐use tool to predict POD, especially in light of the earlier results of these then recently released studies. Perhaps measuring more cognitive functions would have resulted in a different outcome.

We used a relatively young population and perhaps an older population would have behaved differently. Furthermore, we did not take sociodemographic factors into account as a possible confounder, though we did look at education levels and found no between‐group differences.

Approximately half of the population underwent cardiothoracic surgery. These patients are known to be prone to neurocognitive sequelae, which may occur independently of POD. Because we used a preoperative test, instead of both pre‐ and postoperative tests which could be affected by these neurocognitive sequelae rather than by POD, we decided to include both cardiac and non‐cardiac surgeries in the same cohort. In the cardiothoracic surgeries we did find a statistically significant longer operating time; however, our statistical analyses and sensitivity analysis showed a similar incidence of POD in both cardiac and non‐cardiac surgeries. Therefore, it appears unlikely that longer operating time significantly impacted the development of POD in (these) patients.

Another possible explanation why previous research did find significant results may be that differences in anti‐cholinergic burden, cognitive decline, depressive symptoms, or impairment in ADL were present in the population, but not measured. We determined potentially confounding factors for POD , such as comorbidities (eg, American Society of Anesthesiology physical status ), cognitive reserve, functional status, educational levels, anti‐cholinergic medication use, or depression and found no relevant baseline differences.

We report a POD incidence of 25/152 (16.4%), which is within the reported range in the literature , , , , and similar to the rate reported by Lowery et al. of 14/98 (14.7%). It is recommended to perform the CAM daily on all patients with multiple risk factors and it has been shown that even recovery room delirium is a strong predictor for future POD. We assessed POD from day 0 to 5, whereas this previous study conducted POD assessment on day 3 to 7. Nine of our patients were diagnosed with POD within the first 3 days postoperatively, while POD subsided before day 3 in 6 of 25 patients, which would therefore not have been included in the POD group if POD assessment was done only from day 3 to 7. However, due to fluctuation in mental status inherent in delirium, performing the CAM once daily may be insufficient to detect all cases.

Some limitations should be noted when interpreting our results. The visual vigilance test that was used was rather short compared to tests of attentional function used in formal neuropsychological testing. As such, more subtle differences in our population without cognitive impairment cannot be ruled out. To control for confounding variables within and between groups in preoperative status, all participants could have been evaluated within the same number of hours prior to surgery. Also, our test used a 50% prevalence target rate, perhaps one out of every four would have demonstrated some differences. Furthermore, comfort with computer‐based testing, or lack thereof, may have impacted data.

Our sample size was calculated on the assumption of an effect size of 0.9 (Cohen's d). This was based on previous research; nevertheless, this is a very large effect size and perhaps a smaller effect size and consequent bigger sample size could have produced different results. Therefore, a smaller effect size would be preferable in subsequent studies. Post hoc power analysis revealed that a minimal effect size of 0.617 could have been detected. Thus, from our results we cannot exclude the possibility that smaller differences (ie, effect size <0.617) may be predictive for POD, although one could question its clinical utility if this would be the case.

Similar to the previous research, no clinically significant between‐group differences were found regarding previously described risk factors for POD , such as diabetes, age, functional status, pre‐ and postoperative medication use, or perioperative glucose levels. However, these risk factors stem from a larger database and pooled data analyses, and our study was not designed (and therefore not powered) to detect an influence of these predictors on POD. , We did however perform the largest prospective study so far on preoperative intra‐individual reaction time variability and POD, and the first in patients undergoing major surgery while accounting for possible confounders.

This study does not support our hypothesis that preoperative intra‐individual reaction time variability is an independent predictor for the development of POD. Therefore, we cannot endorse routinely administering attentional assessments by this test during the preoperative evaluation of older adults. The visual vigilance test had both a low sensitivity and specificity and is therefore not recommended to assess preoperative risk for developing POD. There still remains a need for further research on this subject to find a usable tool to assist others in identifying those who are at risk for POD.