Assessment of a digital game as a neuropsychological test for postoperative cognitive dysfunction.

OBJECTIVE: Postoperative cognitive dysfunction may result from worsening in a condition of previous impairment. It causes greater difficulty in recovery, longer hospital stays, and consequent delay in returning to work activities. Digital games have a potential neuromodulatory and rehabilitation effect. In this study, a digital game was used as a neuropsychological test to assess postoperative cognitive dysfunction, with preoperative patient performance as control.
METHODS: It was a non-controlled study, with patients selected among candidates for elective non-cardiac surgery, evaluated in the pre- and postoperative periods. The digital game used has six phases developed to evaluate selective attention, alternating attention, visuoperception, inhibitory control, short-term memory, and long-term memory. The digital game takes about 25 minutes. Scores are the sum of correct answers in each cognitive domain. Statistical analysis compared these cognitive functions pre- and post-surgery using a generalized linear mixed model (ANCOVA).
RESULTS: Sixty patients were evaluated, 40% male and 60% female, with a mean age of 52.7 ± 13.5 years. Except for visuoperception, a reduction in post-surgery scores was found in all phases of the digital game.
CONCLUSION: The digital game was able to detect decline in several cognitive functions postoperatively. As its completion is faster than in conventional tests on paper, this digital game may be a potentially recommended tool for assessing patients, especially the elderly and in the early postoperative period.

Introduction

Since the 1950’s, some patients have reported not feeling the same after being submitted to a variety of surgeries.1, 2 These clinical reports encouraged the investigation of neuropsychological tests to assess neuropsychological cognitive function after cardiac surgery. Postoperative Cognitive Deficit (POCD) has a multifactorial etiology and has been consistently reported in elderly patients, for periods from 7 days up to 3 months postoperatively.4, 5, 6

Environmental, educational, or even emotional factors as predictors of a likely diagnosis of POCD, in addition to age combined with existing methodological difficulties, make it challenging to design investigational studies.7, 8

Current neuropsychological tests can present limitations by not being specific to inpatient populations. Their performance requires a lot of time for application, with possible fatigue or increased anxiety of patients preoperatively. Rasmussen defined this dysfunction of intellectual functioning as an impairment of cognitive functions, such as memory, attention, executive function or language, for which neuropsychological tests are more accurate diagnostic tools.10, 11, 12, 13

The literature registers differences among methods used to diagnose dysfunctions and their prognosis. However, inconsistencies are common, given the variety of tests, interval between assessment sessions, statistical analyses employed, and boundaries between neuropsychological deficit and POCD.1, 10, 11, 13, 14, 15 Anesthesia protocols, follow-up, time elapsed after surgery, and sensitivity of tests add to the difficulties in interpretation.

POCD in the elderly seems to result from worsening of a previous impairment that may have been missed even after certain neuropsychological tests.13, 17 The condition leads to more difficult clinical recovery, longer hospital stay and consequent delay to return to work. Prolonged hospitalizations put stress on healthcare as a whole, in addition to being associated with hazards related to individuals with cognitive dysfunctions in risk settings.

Digital games have a potential neuromodulation and rehabilitation effect. Studies from the 1980’s have shown these effects and adjustments of this technology, making rehabilitation a more intuitive process for patients.19, 20, 21, 22 New interaction dynamics with already known stimuli and digital reality help learning and rehabilitation.23, 24 Regardless of being used for rehabilitation, performance in a digital game developed as an alternative to neuropsychological tests on “pen and paper” can be an efficacious tool for detecting POCD.

The objective of the present study was to use a digital game as a neuropsychological test to assess postoperative cognitive dysfunction in the function domains: memory (short and long term); attention (selective and alternating); executive (inhibitory control and visuoperception).

Methods

The study protocol was approved by the Ethics Committee of Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (CAPPesq – research Project nº 14086 CAAE: 49463315.5.1001.0068) and is registered on Clinicaltrials.gov by number NCT02551952.

After surgery indication was confirmed, participants were invited to take part and sign the Consent Form before taking tests. Data were collected between September 2018 and October 2019.

Participants were selected among inpatients scheduled for surgery at the Central Institute of Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo, assessed in the pre- and postoperative periods. A team of trained interns – Psychology students – helped with data collection, which occurred at the hospital rooms of inpatients scheduled for elective general surgery, urology, gastrointestinal, gynecology, and head and neck procedures.

Patients who were candidates for non-cardiac elective surgery and with ages between 20 and 80 years were included. We excluded patients to be submitted to cardiac surgery because delirium and cognitive dysfunction frequently occur after cardiac surgery and are associated with a high mortality risk. We also excluded patients with incapacity to use a digital game, upper limb mobility limitation, significant mental impairment which compromised understanding instructions on how to use the game, significant impaired visual acuity without corrective lens, and deaf patients due to the auditive stimuli present in the game. It was, therefore, a study without a control group, and with a dependent, pre-, and post-intervention design.

We used the patent registered MentalPlus® test. It is a digital game developed to assess and stimulate neuropsychological attention, memory, and executive functions, regarding planning, strategy search, flexibility of thought, and inhibitory control (resistance to distractor stimulus and impulse control). The test takes roughly 25 minutes from beginning to end through all game phases, and assesses selective attention (ATSTOTAL), alternating attention (ATATOTAL), visuoperception (VISUOP), inhibitory control (FECIA), short-term memory (MCPTOTAL) and long-term memory (MLPTOTAL). Performance scores are the sum of correct hits at each phase and are displayed right after the end of the game. A video explaining game functioning can be seen on: https://www.youtube.com/watch?v=aJqvYb7jeHA [Accessed on 29 Dec 2019].

Statistical analysis to compare cognitive functions before and after surgery used GLMM – generalized linear mixed model, IBM SPSS Statistics, version 26, assuming 5% as significance level, probability with normal distribution and identity link function. Controls for age, gender, and time of interview (pre- or post-surgery) were performed for fixed effects, and study subject for random effect.

We added analysis equivalent to mixed ANCOVA to check if reduction in scores attributed to cognitive functions were influenced by duration of surgery. The role of the covariable is assumed by the preoperative measurement, that worked as baseline control, while the postoperative measurement worked as the outcome variable. The factor was duration of surgery, which we classified into three categories: Short – less than 120 minutes; Intermediate – between 120 and 240 minutes; Long – more than 240 minutes.

So, three groups of independent patients were set up, with each one of them measured twice to consider the mixed design, simultaneously among participants and intra-participant. ANCOVA is an alternative to the dependent t-test, with the advantage of requiring a smaller sample for the test; the power of an ANCOVA with (1-ρ²)n is equal to a t-test with a sample of n individuals. Another advantage over the t-test is allowing for controls, given in this study, a perfectly reasonable assumption is that age and gender may also be related to change in cognitive performance. The R version 4.0.3 (2020-10-10) of mixed ANCOVA was implemented, using its equivalent with a generalized linear model function with

Before applying ANCOVA, both its two main assumptions should also be checked. They are dissociation between factor and covariable, provided byand equal inclinations of regression lines, by

Results

One hundred and twelve patients signed the informed consent and participated in the preoperative data collecting. Of these, 52 patients refused repeating the game in the postoperative period or had had their surgery cancelled and were excluded from the study. Sixty patients were studied.

Of the participants, 40% were male and 60%, female, with a mean age of 52.7 years. The characteristics of the sample are shown in more detail on Table 1. The comparison between number of correct hits on the six phases of the digital game obtained before and after surgery showed a reduction in the postoperative score in selective and alternating attention, inhibitory control and short- and long-term memory. No significant change was found for visuoperception (Table 2). The interval between pre- and post-surgery assessments ranged from 1 to 12 days, depending on the recovery status of the patients: 65.5% were reassessed up to the second postoperative day, 27.6% between the third and seventh days, and 6.9% on the second week.

Table 1

Characteristics of the sample: gender, age, years of schooling, type of surgery, and type of anesthesia.

Gender	Men (%)	Women (%)	Total
n	24 (40%)	36 (60%)	60
Age	Mean	S.D.
n	52.7	13.5
Schooling	1–6	7–9	10–12	> 12 years
n	19	12	16	12
Type of surgery	Gastrointestinal	General surgery	Gynecology	Others
n	26	9	11	14
Anesthesiaa	General	Epidural	Spinal	Sedation
n	54	10	9	6

The sum of anesthesia procedures is above 60 because more than one procedure may have been necessary for the same patient.

Table 2

Pre- and post-surgery digital game scores described as mean ± standard deviation.

	Pre-surgery	Post-surgery	Statistics
ATSTOTAL	25.67 ± 3.08	18.07 ± 4.45	F(1;38) = 188.246 p < 0.001
ATATOTAL	23.47 ± 3.88	20.40 ± 4.57	F(1;48) = 42.338 p < 0.001
VISUOP	30.07 ± 13.44	31.20 ± 10.79	F(1;7) = 0.213 p = 0.659
FECIA	24.57 ± 3.77	17.78 ± 5.49	F(1;31) = 98.137 p < 0.001
MCPTOTAL	7.92 ± 2.35	4.87 ± 2.24	F(1;55) = 179.034 p < 0.001
MLPTOTAL	8.43 ± 2.35	6.15 ± 2.36	F(1;37) = 88.935 p < 0.001

Statistics analysis performed using a generalized linear mixed model – GLMM (IBM SPSS Statistics, version 26), considering level of significance of 5%, normal distribution probability and link identity function, controlling for age, gender, and time of assessment (pre- or post-surgery) as fixed effects and, study subject as random effect.

ATSTOTAL, Selective Attention; ATATOTAL, Alternating Attention; VISUOP, Visuoperception; FECIA, Inhibitory Control; MCPTOTAL, Short-Term Memory; MLPTOTAL, Long-Term Memory.

We obtained data only for 49 patients related to the influence of the duration of surgery. We found reduction in all scores except for visuoperception, albeit the reduction was similar regardless of duration of surgery (short, intermediate, and long) (Table 3).

Table 3

Effect of surgery duration (Short, less than 120 minutes; Intermediate, between 120 and 240 minutes; Long, more than 240 minutes) on digital game scores.

	Time	Pre	Post	n	Diss	Slope	Statistics
ATSTOTAL	Short	26.36 ± 2.20	19.91 ± 2.12	11	F(2.44) = 0.034, p = 0.967	F(2.41) = 1.123, p = 0.335	F(2.43) = 2.760, p = 0.188
	Int.	25.59 ± 3.63	18.05 ± 4.57	22
	Long	25.88 ± 2.45	17.25 ± 3.91	16
ATATOTAL	Short	24.36 ± 3.11	22.00 ± 2.00	11	F(2.44) = 0.537, p = 0.588	F(2.41) = 0.463, p = 0.633	F(2.43) = 21.942, p = 0.924
	Int.	23.55 ± 4.09	20.05 ± 5.24	22
	Long	22.88 ± 3.83	20.31 ± 4.42	16
VISUOP	Short	29.55 ± 11.24	28.00 ± 5.69	11	F(2.44) = 0.034, p = 0.967	F(2.41) = 3.208, p = 0.051	F(2.43) = 36.872, p = 0.467
	Int.	33.18 ± 16.64	33.41 ± 12.42	22
	Long	32.31 ± 12.89	31.81 ± 12.3	16
FECIA	Short	22.18 ± 2.99	17.00 ± 4.20	11	F(2.44) = 4.323, p = 0.019	F(2.41) = 0.678, p = 0.513	F(2.43) = 12.441, p = 0.983
	Int.	25.50 ± 3.07	18.77 ± 5.37	22
	Long	24.38 ± 4.16	18.00 ± 5.57	16
MCPTOTAL	Short	5.18 ± 2.36	4.91 ± 2.51	11	F(2.44) = 1.417, p = 0.253	F(2.41) = 0.048, p = 0.953	F(2.43) = 2.125, p = 0.987
	Int.	6.27 ± 2.16	5.00 ± 2.27	22
	Long	5.81 ± 2.43	5.06 ± 3.19	16
MLPTOTAL	Short	8.18 ± 2.52	6.55 ± 2.84	11	F(2.44) = 0.451, p = 0.64	F(2.41) = 0.630, p = 0.537	F(2.43) = 21.026, p = 0.766
	Int.	8.36 ± 2.42	6.41 ± 2.28	22
	Long	8.62 ± 2.58	6.12 ± 2.50	16

Statistics analysis performed using ANCOVA (R version 4.0.3) assuming 5% level of significance, dissociation between pre-operative and duration of surgery and homogeneity of regression slopes for each factor, controlling for age and gender.

ATSTOTAL, Selective Attention; ATATOTAL, Alternating Attention; VISUOP, Visuoperception; FECIA, Inhibitory Control; MCPTOTAL, Short-Term Memory; MLPTOTAL, Long-Term Memory; Int., Intermediate.

Discussion

The digital game presented decreased scores at several phases of the game, possibly due to worsening of cognitive performance of patients during the postoperative. Although mere clinical observation may suggest diagnosis of POCD, technology can help document this observation, making it better to pinpoint cognitive impairment in attention (selective and alternating), executive functions (inhibitory control and search strategy), and memory (short- and long-term) domains. Additionally, we observed that the digital game did not detect change in the visuoperception function. Such results suggest that the digital game can detect changes in POCD, given that some level of cognitive dysfunction is expected after anesthesia and surgery.14, 28

Interestingly, we did not find a discriminated reduction in scores associated with duration of surgery. Besides the likely insufficient sample, other possible explanations can be raised. One can imagine that the surgery or type of anesthesia itself would be enough to cause POCD, regardless of duration. We were not able to check these hypotheses with the data available because the description of the procedures varied greatly and all patients received general anesthesia, whether combined to other techniques or not.

Beside procedures, many other variables can influence results of these assessments, including preoperatory anxiety and socioeconomic status, characteristics that are difficult to measure. Notably in university hospital environments, anxiety is high and, in a study assessing a sample obtained from a population similar to the present study, the lack of information about the surgery was also considered a source of anxiety among patients.

Neuropsychological tests commonly used require time and effort to be administered, while their digital counterparts are more straightforward, which is important for patients with already impaired health. Even if the results obtained from both methods revealed only modest agreement, one can argue that both methods are not required to have equivalent measurement units, as long as they point to POCD in the same direction. Still, the need to identify high-risk patients and the potential to inform and begin routine assessments, as well as implementing techniques to reduce this risk offer support to this study, notably toward the possibility of non-pharmacological multidisciplinary interventions.

One of the most important cognitive functions is attention, as it is considered the foundation for the remainder functions, because by attention focus, we acquire information, register and elaborate strategies to solve problems. In the present study, the observers suggest that long-term memory (MLPTOTAL) registered the highest difficulty for patients at the digital game employed, which seems to be reflected by the decrease in postoperative scores compared to preoperative scores. Another greatly affected domain was inhibitory control, but this phase of the game requires short-term memory and motor skill functions simultaneously. Memory is considered the most affected cognitive domain in POCD,15, 35 and our results suggest that the digital game can detect its impairment.

Our study has some limitations. The power to generalize results is low, given the exploratory design to test the instrument as a non-control study. The sample size is medium, and the results show a moderately relative variability. The moment the game was administered postoperatively could not be strictly controlled as it depended on a patient’s postoperative clinical status and on the routine of each ward.

Regarding the number of dropouts, the possible rationale is the fact that, as the present study was part of a larger project in which data were collected not only for the digital game, but also by traditional methods (tests on paper) that required some hours of interaction, patients would give up, because of the time and effort required from the participants in postoperative recovery with malaise, impossibility due to patient status at the time of data collecting, or, as mentioned, surgery cancellation after pre collecting. Therefore, an assessment of practical application and effectiveness of the instrument is required for future studies, using the digital game only, and consequently demanding less from participants.

We controlled for gender and age to eliminate their potential influence on statistical results, but it was not possible to control for the effects of the different anesthesia or sedation protocols because they were homogeneous regarding general anesthesia and unbalanced regarding other anesthesia procedures (e.g., sedation, epidural, or spinal). Maybe anesthesia time, type of surgical procedure and surgery time scheduled may influence taking into account different circadian rhythms, which deserve further studies, as they could not be ascertained by the data obtained.

Nevertheless, the results presented by this study contribute to introduce this digital cognitive assessment tool, developed specifically for research in hospital and clinical scenarios. Digital game scores are consonant with cognitive reductions. A digital game can be considered an alternative tool with ample potential, notably for reducing time for neuropsychological assessment, its standardization and because it is self-explanatory.

Conflicts of interest

The authors declare no conflicts of interest.