An Individual Cognitive Stimulation Therapy App for People with Dementia and Carers: Results from a Feasibility Randomized Controlled Trial (RCT).

BACKGROUND: There is a lack of digital resources that support the cognition and quality of life (QoL) of people with dementia. The individual cognitive stimulation therapy application (iCST app) aims to provide cognitive stimulation and social interaction to people with dementia and carers through interactive touch-screen technology. This study set out to determine the feasibility of conducting a full-scale, randomized controlled trial (RCT) with the iCST app.
METHODS: This was a single blind, feasibility RCT including people with mild to moderate dementia and their carers. Multiple trial components were assessed including recruitment and retention rates, intervention fidelity and usability, and acceptability of the outcome assessments which included measures of cognition and QoL. A sample of the intervention group was invited to a semi-structured post-trial interview to examine the experience of using the iCST app.
RESULTS: Sixty-one dyads were randomised to the iCST app (n = 31) or treatment-as-usual (TAU) control group (n = 30) for 11 weeks. In the iCST app group, 77% used the intervention for 20 minutes or more each week. Carers using the iCST app rated their QoL better at follow-up 2 compared to the TAU control group (EQ-5D, MD = 7.69, 95% CI = 2.32-13.06, p = 0.006). No significant differences were found on the other outcome measures.
CONCLUSION: The iCST app was deemed usable and enjoyable. Most participants completed the activities more quickly than anticipated and did not have enough activities to continue using the app frequently. Expansion of the iCST app is needed to maintain engagement for longer. Findings indicate that computerised cognitive stimulation can be beneficial, and a large-scale RCT is feasible with modifications to trial components. The results are relevant to researchers, software developers, policy-makers, people with dementia and carers who are looking to be involved in such interventions. TRIAL REGISTRATION: ClinicalTrials.gov, NCT03282877. Registered on 19 July 2017.

Introduction

Background

People with dementia can face difficulties in staying mentally stimulated and can experience a reduced quality of life (QoL) following their diagnosis.1 Given the anticipated rise in the global prevalence of dementia,1 there is an urgent need for innovative resources to support people with dementia in their daily lives. Existing resources such as Cognitive Stimulation Therapy (CST) have proven to be helpful. CST is a non-pharmacological, group treatment for people with dementia which has shown to benefit cognition and QoL.2 The individual version of CST (iCST) is typically delivered by a carer at home and can improve the relationship quality between the person with dementia and carer, and the QoL of carers.3

In terms of innovation, digital technology has the potential to support people with dementia through a range of assistive tools, eg, reminder and tracking devices, social robots, and touch-screen applications (apps).4 In addition, computerised cognitive interventions can lead to improvements in cognition.5 A qualitative study by Asghar, Cang and Yu6 found that people with dementia mainly use assistive technology for health monitoring and socialisation. Participants deemed assistive technology as a positive addition to their daily lives and stressed the importance of suitable designs and functionalities for future technologies.6 However, despite these encouraging findings, there is a lack of digital resources for post-diagnostic support, and a lack of touch-screen apps for enjoyment and leisure targeted towards people with dementia.7 Moreover, a systematic review found that compared to studies focusing on computerised cognitive training and rehabilitation, the field of computerised cognitive stimulation is relatively underdeveloped.5 More research is needed in the field given that another systematic review found that the effects of cognitive stimulation can be maximised through the added use of computers as their content and platforms can be cognitively stimulating by themselves.8 More recently, as a result of the COVID-19 pandemic there is a pressing need for accessing a form of CST at home. CST groups have been offered via Zoom: a video conferencing software.9 In addition, researchers have developed an iCST app which can be used together by people with dementia and carers.10 The iCST app has some advantages over traditional CST. For instance, it can improve accessibility to a form of CST for people who are unable to attend groups and support people with dementia to stay mentally stimulated in the safety of their homes. Furthermore, the iCST app can be used on touch-screen tablets and previous research has shown that touch-screen tablets can be highly intuitive and well-designed touch-screen interventions can improve the well-being of people with dementia.11 This platform combined with the use of interactive technology (eg, audio-visual stimuli), based on the principles and content of CST and iCST may produce additional benefits in terms of cognition and QoL for people with dementia. The involvement of a carer can also have a positive impact as a previous iCST trial showed benefits in the relationship quality between the person with dementia and carer.3 In addition, previous research suggests that people with dementia may have concerns regarding the use of assistive technology leading to social isolation.12 Having the carer as part of the iCST app itself will promote social interaction which may alleviate concerns regarding social isolation.

The development of the iCST app took place through multiple collaborations with people with dementia with a special focus on appropriate design. It took place in iterative stages following various frameworks including the Medical Research Council Framework for developing complex interventions and the Centre for eHealth Research roadmap.13,14 The final stage of development includes this feasibility RCT in order to gain insights in the usefulness, usability, potential effectiveness, and feasibility of conducting a full-scale RCT with the app.

Aim

The aim of this study was to evaluate the feasibility of conducting a full-scale RCT with the iCST app compared to a treatment as usual (TAU) control group.

Materials and Methods

This study was designed as a multi-centre, single blind, feasibility RCT with participants allocated on a 1:1 ratio to the experimental group (completing two to three, 30-minute iCST app activities per week at home) or TAU control group for 11 weeks. It was registered with ClinicalTrials.gov on 19 July 2017 (registration number: NCT03282877). The study complies with the Declaration of Helsinki15 and received ethical approval from the Yorkshire and the Humber – Bradford Leeds Research Ethics Committee (REC number 17/YH/0405) and the Health Research Authority in March 2018. A study protocol containing full details of the methodology including the design, procedure, and intervention was published in Rai, Schneider and Orrell.16

Participants

Recruitment took place in five NHS study sites in the East Midlands from November 2018 to April 2019 and from July 2019 to March 2020. Potential participants were identified from a variety of settings including primary and secondary care, memory clinics, support groups, Join Dementia Research (JDR: an online register), and social media. In addition, each participating study site referred to their own database of people with dementia and carers who had previously expressed interest in taking part in research.

The sample included people with mild to moderate dementia and their informal carers (relatives or friends) who were recruited as dyads to the study. People with dementia were eligible if they were at least 50 years of age; met the Diagnostic and Statistical Manual of Mental Disorders (DSM IV) criteria for dementia of any type; scored 10 or above on the Mini Mental State Examination (MMSE)17 or 16 or above on the Montreal Cognitive Assessment (MoCA);18 had some ability to communicate and understand in English (eg, ability to give informed consent); were able to see and hear well enough to participate; and did not have a major physical illness or disability affecting their participation. Carers were eligible if they were at least 21 years of age; had the ability to speak and understand English; were able to see and hear well enough to participate; and did not have a major physical illness or disability affecting their participation. In addition, either participant needed to have access to their own touch-screen tablet (with software version 10 for iOS and version 4.4.2. for Android). Dyads were excluded in the case of concurrent participation in any other interventional study.

Research staff members at each study site checked the eligibility of referrals received from staff at the recruitment sources. Eligible participants were sent a participant information sheet (PIS) containing full details about the study. If still interested in participating, the dyad was recruited into the trial and a date was set for the baseline assessment and consenting. Dyads provided written informed consent and completed a baseline assessment during the first in-home visit prior to randomization. A first follow-up (FU1) took place five weeks post-baseline followed by a second follow-up (FU2) 11 weeks post-baseline. All participants were given a £10 Apple or Google Play store voucher at FU2 in order for them to download the iCST app once it had been released on the app stores. A sample of the experimental group was invited for a semi-structured post-trial interview after completion of the study to gain insights in the acceptability of the iCST app including the overall experience of using the app.

Sample Size

The proposed sample size was 60 dyads leading to 30 dyads per treatment arm. This was based on a previous audit of trials registered in the Clinical Research Network (CRN) database in the United Kingdom. This audit found that most feasibility and pilot trials had a median of 30 or 36 participants per arm and the researchers recommend an upper limit of 60 participants for a feasibility trial.19

Randomisation

Randomisation took place after consent and the baseline assessment using an online, central randomisation service called Sealed Envelope (). Block randomisation was employed with block sizes of four to six (randomly varied and generated by Sealed Envelope). This technique is frequently used in clinical trials to minimise bias and to allocate an equal number of participants to each treatment arm by sequencing participant assignments by block. This method is especially useful for small sample sizes.20

The researcher at the local study site performed each randomisation using the participant identification code of the person with dementia. Dyads were informed of their allocation outcome over the telephone and, if necessary, a visit was arranged for dyads in the experimental group to install the iCST app.

Blinding

The trial included both blinded and unblinded researchers at each local site. It was not possible to blind the participants to their treatment arm as the iCST app demands active participation from the subject. The baseline assessment could be performed by either a blinded or unblinded researcher. However, FU1 and FU2 were completed only by the blinded researcher who was unaware of the randomisation outcome for each dyad. Details in case of allocation disclosure were recorded by the visiting researcher. The unblinded researcher performed the randomisation, communicated the outcome with the participants, and for the experimental group, installed the iCST app, provided weekly telephone support calls, and completed the usability and acceptability questionnaire at the end of the study. Furthermore, the unblinded researcher was not informed about the results of the assessments.

Intervention – iCST App

Full details of the intervention can be found in the published study protocol.16 The iCST app is a one-to-one, carer-led, home-based programme of structured cognitive stimulation for people with dementia but delivered on a touch-screen tablet. Its content was adapted from the paper-based iCST manual including the principles, themes, and activities.3 Participants in the experimental group used a third version of the iCST app prototype over 11 weeks post-baseline as this would be the point where all activities would be completed at least once. This version consisted of 21 activities which include a mix of game-like, interactive features such as audio-visual stimuli, and discussion questions. All activities were completed together by the person with dementia and the carer which helped provide opportunities for social interaction: a key element of paper-based CST and iCST. Some examples of activities include: “Sounds”, “The Price is Right”, “Arts”, and “My Life”. Additional features of the app included a home screen with a short introduction and key tips, and choice of two levels with level 2 containing more challenging content or different questions than level 1. It was recommended that participants use the app for two or three times a week and to use it for 30 minutes each time (eg, two or three activities a week). This was based on development work with the app and previous CST and iCST research.3,10 Participants were free to spend more time on the app if they wished, and this was recorded during the weekly telephone calls.

Training, Adherence and Support

Unblinded researchers were responsible for installing the iCST app on the devices of the dyads in the experimental group through an in-home visit. The researcher explained how the app worked using a short, supplementary document containing instructions with screenshots of the app. Furthermore, all dyads received weekly telephone support calls from the unblinded researcher in order to monitor adherence but also to track overall progress and any challenges and/or technical difficulties with using the iCST app. Questions were related to general experience, average amount of activities completed in a week, average amount of time spent per activity, enjoyment, and any likes/dislikes. Any reasons for not being able to use the iCST app over the week were also recorded. In addition to self-reported data, usage was also monitored through back-end tracking using analytics.

TAU Control Group

The control group consisted of a TAU control group and did not receive any additional interventions. Therefore, the effects of the iCST app were compared with the natural progression of people with dementia under conditions of usual care. The treatments and services which were already available to people with dementia and their carers randomised to the TAU control group, may have differed between and within recruitment sites. The visiting researcher recorded any current participation with CST groups and/or use of acetylcholinesterase inhibitors (AchEIs) at the baseline assessment. Randomisation ensured that both the experimental and TAU control group contained an equal number of participants who took medication and who were exposed to a form of CST.

Outcomes

Feasibility Outcomes

This trial investigated key feasibility aspects including the rates of screening, recruitment, randomisation, and retention using logs.13 Acceptability of the outcome measures was evaluated by assessing the completion rates, and the acceptability and fidelity of the iCST app was evaluated through weekly telephone support calls, analytics, and a usability and acceptability questionnaire. Based on a benchmark set in previous feasibility trials involving psychological treatments, it was expected that >75% of the participants in the experimental group would need to complete the recommended minimum of 2 activities on average every week for the iCST app to be considered feasible.21,22

Clinical Outcomes

Clinical outcomes for the person with dementia included cognition using the Alzheimer’s Disease Assessment Scale-Cognition (ADAS-Cog)23 QoL using the Quality of Life – Alzheimer’s Disease (QoL-AD) and EuroQoL five dimensions (EQ-5D) questionnaires,24,25 dyadic relationship quality using the Quality of the Carer Patient Relationship (QCPR) questionnaire,26 depression using the Cornell Scale for Depression in Dementia (CSDD),27 behavioural disturbances using the Neuropsychiatric Inventory (NPI),28 and functional abilities using the Bristol Activities of Daily Living Scale (BADLS).29 Outcomes for the carers included QoL using the EQ-5D,25 anxiety and depression using the Hospital Anxiety and Depression Scale (HADS),30 and dyadic relationship quality using the QCPR.26 In addition, both people with dementia and carers completed two technology-related scales; one for self-efficacy beliefs in computer/tablet use measured at baseline only with the Computer User Self-Efficacy (CUSE) scale,31 and the other for the usability and acceptability of the iCST app measured at FU2 only with the Questionnaire of Usability and Acceptability (CUA).32 All assessments took place in the homes of the participants. The data was stored in a secure cabinet at the University of Nottingham and it was entered manually into SPSS version 25 for Windows which was used for all the analyses.

Informed Consent

People with mild to moderate dementia were expected to be able to give informed consent for participation provided that appropriate care was taken in explaining the research and sufficient time was allowed for them to reach a decision. Written informed consent was taken at baseline from both the person with dementia and carer. Consent forms were signed and dated by the participant and the researcher before they entered the study.

Post-Trial Interviews

A total of three dyads in the experimental group participated in a joint, semi-structured interview upon completion of the study. The purpose of the interviews was to gain additional information on the layout and content of the iCST app, the overall experience of using it as a dyad, and any practicalities surrounding its use in daily life. All interviews took place in the home of the participants and written informed consent from both participants was obtained prior to data collection. The person with dementia and carer were interviewed together in order for them to better reflect on the experience of using the app together. The interview lasted approximately 45 to 60 minutes. The data were audio-recorded and transcribed.

Statistical Analyses

Full details of the statistical analyses can be found in the published study protocol.16 Key feasibility outcomes are reported through frequencies including the number of participants screened, recruited, randomised, and retained. Adherence to the intervention was assessed by calculating the average number of iCST app activities completed by the dyad as logged in the weekly telephone calls and through anonymous back-end tracking using analytics. The usability and acceptability of the iCST app were investigated by examining data from the weekly telephone calls, post-trial interviews, and by calculating scores on the CUA. Data from the post-trial interviews were summarised and grouped according to the discussion topic. It was not coded nor analysed thematically with specialised software considering the small sample size of participants partaking in the interviews and difficulty in reaching data saturation.33

Outcome measures were assessed for appropriateness by calculating missing data rates within the measures and across the assessments. Outcome analyses included descriptive statistics computed for each group and outcome measure including means, standard deviations, 95% confidence intervals and effect sizes.34,35 In order to compare the outcomes on each of the questionnaires between the two groups, an analysis of covariance (ANCOVA) was undertaken. All analyses were based on the intention to treat principle in that all available data was included in the analyses. Missing data was only imputed using pro-rating if fewer than 20% of cases were missing on any given measure.3

Results

Recruitment and Retention

A total of 463 dyads were approached or referred across the five sites and, of these, 61 dyads were recruited to the study (see Figure 1). In many cases, the lack of having a correct device for the study led to dyads being ineligible. Participants were most often referred from or approached through dementia support groups (n = 164), JDR (n = 88), the site’s own research database (n = 84), or clinicians working in CMHTs or memory assessment services (n = 74). For 53 dyads, the referral source was listed as “other” eg, through leaflets advertising the study. Unblinding occurred for 11 dyads in the group using the iCST app (by unblinding the visiting researcher at FU1) and one dyad in the control group.

Figure 1

CONSORT flow diagram: participant flow through feasibility trial. *One dyad who did not receive the intervention in the experimental group was subsequently withdrawn from the study.

Demographic Information

Table 1 shows the demographic information of people with dementia and carers. Tests for homogeneity showed no differences in distributions on any of the characteristics between the iCST app and TAU control group. The majority of people with dementia were not involved in any CST groups at the time of the study (n = 51). Of the people with dementia participating in CST groups, six were in the control group and three were in the experimental group. Carers were more experienced with using technology than people with dementia (see Table 1).

Table 1

Demographics of People with Dementia and Carers.

	Total (n = 61)	iCST App (n = 31)	TAU (n = 30)	Tests for Homogeneity – p value
Characteristic
Person with dementia
Age in years: mean, SD	73, 7.67 (range: 50–89)	74.03, 6.83	71.81, 8.52	0.16
Male (%)	42 (69)	22 (71)	20 (67)	0.72
Ethnicity white (%)	59 (97)	29 (93.5)	30 (100)	0.37
Relationship with carer: married (%)	51 (84)	27 (87)	24 (80)	0.39
Lives with spouse/partner (%)	52 (85)	27 (87)	25 (83)	0.94
Educationa: no qualifications or School Leaver O-Levels/GCE (%)	25 (41)	15 (48)	10 (33)	0.51
Taking AchEI medication (%)	43 (70.5)	22 (71)	21 (70)	0.89
Experience with technology: quite a lot or more	13 (22)	9 (29)	4 (13)	0.08
Carer
Age in years: mean, SD	66.89, 10.72 (range: 27–83)	68.52, 8.96	65.15, 12.28	0.15
Female (%)	47 (77)	26 (84)	21 (70)	0.20
Ethnicity white (%)	60 (98)	31 (100)	29 (97)	n/ab
Educationa: School Leaver O- Levels/GCE or Higher Education (BSc/BA) (%)	33 (54)	17 (55)	16 (53)	0.63
Experience with technology: quite a lot or more (%)	21 (35)	10 (33)	11 (37)	0.65

Notes: aEducation categories included no qualifications, school leaver – O levels/GCE, school lever – A levels, higher education (BSc/BA), postgraduate education (MSc/MA/PhD), and other. bAll carers were white British with one missing value in the control group. Hence, no variation detected between the two groups.

The iPad was the most familiar type of device with 43 dyads having one (72%), and 17 dyads owned an Android tablet including: Samsung, Lenovo and Amazon Kindle (28%).

Feasibility and Acceptability of the iCST App

Four dyads were either withdrawn early due to COVID-19 or did not receive their allocated intervention due to device incompatibility. Therefore, telephone data were available for 26 dyads and of these, 46% (n = 12) were able to complete 21 or more activities over 11 weeks, 23% (n = 6) completed 14 to 21 activities, 27% (n = 7) completed 7 to 14 activities, and 4% (n = 1) completed six activities during the study. Some dyads reported doing more than one activity on one day and dividing their time across several activities (eg, 10 minutes on “Word Search” and 20 minutes on “The Price is Right”). On average, dyads spent 38 minutes per activity which is slightly more than the 30 minutes specified.

Analytics data available for 18 dyads showed the most popular activities included “Word Search” and “Sudoku” whereas discussion-based activities were less popular, and “Being Active” and “Being Creative” were visited the least. Data showed that 66% of the dyads spent between 20 and 60 minutes on the app per week and 11% were able to spend 60 minutes per week or more on the app. Frequent reasons for not being able to complete iCST app activities included: holidays or family commitments, time constraints, and ill health. One dyad reported doing substitute activities while on holiday which were inspired by the iCST app such as dancing, and word search puzzle books. Furthermore, some technical difficulties caused issues, eg, the app stopped working for 5 days on Apple devices.

The acceptability of the iCST app was shown through comments from the weekly telephone calls and the post-trial interviews.

Content

Most dyads found the app to be engaging and fun, and the majority of the content was deemed appropriate. When discussing the activity “Globe Trotter”, one carer commented:

A lot of the places we’d been to. So that was nice to sort of talk about that then when we’d been there. – Carer, Interview 1.

There was a consensus about the need for more content as dyads would make their way through the app relatively quickly and did some activities multiple times leading to repetition of the content. Not all activities were relevant depending on individual interests (eg, need for more culturally relevant content) and for people with very mild dementia, the app was often too easy and lacked challenge. Though “Being Active” and “Being Creative” were the least popular activities, one dyad used these as an inspiration to go dancing and another dyad visited a crafts store. Lastly, one person with dementia felt it was important to engage in a variety of activities despite not liking some of them:

Sometimes it (the app) has to have the ones that you do not actually like to see if that actually challenges you to do it. – Person with dementia, Interview 1.

Design

People Were Positive About the Design of the App

I think it is very well presented. – Person with dementia, Interview 2.

One carer found the navigation to be difficult while the majority found the app easy to use after getting used to it:

It was quite quick you know, precise once you clicked on welcome, which activities to choose. In that respect, that was good. – Carer, Interview 1.

There was no consensus on the use of colours and some people with dementia mentioned they did not find the colour scheme as important as other features of the app. Text and images were deemed appropriate with one carer suggesting to change the logo to make it more relevant to the purpose of the app and improve association.

Potential Benefits

One carer reflected on the need for the iCST app and went on to say she found it very useful as it made cognitive stimulation more accessible. Dyads often reported enjoyment and spending time together while using the app:

It certainly uplifted me (…) Not only could I do this but it was actually pleasant to do this together. – Person with dementia, Interview 2.

The app was also used in different contexts with one dyad taking the tablet with them on holidays and other dyads using the app with their grandchildren or independently at times. Other observed benefits for the person with dementia included increased concentration and memory (eg, through remembering news headlines), and being able to engage in deeper conversations. Some carers noticed increased confidence in the person they were caring for related to their own abilities or willingness to try new things (eg, using technology, taking up maths tuition, becoming aware of their language skills):

I have seen an awakening in (…), it’s like he has come to life, cognitive-wise and interested. Realising he can do things he struggled with. – Carer, Interview 2.

Some benefits of the iCST app compared to paper-based iCST included its accessibility, speed, and modern feel.

Improvements and Updates to the iCST App

In addition, to the need for more-tailored content, some dyads commented that there was a need for another, intermediate level to better accommodate the different levels of abilities. For some activities, there was a need for more guidance and clarity in some questions and additional feedback on exact correct and incorrect answers as the app was deemed to be a bit ambiguous which could lead to frustration (eg, “Food” and “Odd One Out”).

Regarding the layout and when choosing a new activity, some dyads would like to see a list of activities on the screen rather than swiping through three activities at a time. There was another suggestion to make switching between levels and activities more straightforward. Lastly, a few bugs needed to be resolved, eg, missing map from “Globe Trotter”, low volume of the musical instruments in “Sounds”. Other minor technical difficulties were reported which were often related to slow connection or need for a software update on the tablet.

Outcome Data

Acceptability of Outcome Measures

The acceptability of the outcome measures was assessed through investigating the number of completed questionnaires and the number of questionnaires with missing data. For people with dementia, a total of 44 questionnaires at baseline, 44 at FU1, and 42 at FU2 did not have any missing data. For carers, 58 questionnaires were completed in full at baseline, 49 at FU1, and 52 at FU2. Some participants reported that the assessments were lengthy and found the QCPR and CUSE in particular to be confusing due to the wording of some of the questions. In addition, the CUSE was often not completed if the participant did not have any or little experience with using technology leading to substantial missing data on this measure for people with dementia (n = 37) and carers (n = 24). Furthermore, the CSDD included considerable missing data as it included items that participants were “unable to evaluate”.

Outcome Scores

For people with dementia, participants in the experimental group scored higher (M = 126.81, SD = 26.62) on the CUSE than participants in the control group (M = 109.10, SD = 27.84) and an independent samples t-test did not show a significant difference between the two groups (t(40) = −2.11, p = 0.41, two-tailed). On average, the scores of people with dementia were lower than the scores of carers, but not significantly so. For carers, the scores between the two treatment groups did not differ greatly (M = 132.59, SD = 30.49 for the experimental group, M = 131.41, SD = 28.82 for the control group).

Tables 2 and 3 show the results of the ANCOVA tests at FU1 and FU2 for people with dementia and carers after adjusting for baseline outcomes. For the person with dementia, the analyses at FU1 and FU2 did not show any significant differences between the iCST app and TAU control group on any of the outcome measures. The estimated, adjusted means on the QoL-AD and EQ-5D at FU1 were higher for participants in the iCST app group (MD = 3.12, SMD = 0.40 and MD = 7.00, SMD = 0.37 respectively) which could be a sign of improvement on these quality of life measures. However, these differences were smaller at FU2 (MD = 1.70, SMD = 0.28 for QoL-AD, and MD = −0.14, SMD = 0.01 for EQ-5D). Mean differences on the remainder of the measures were small. For the carers, analyses showed a significant difference at the 5% level on the EQ-5D at FU2 between the iCST app and TAU control group with a higher estimated, adjusted mean for people in the iCST app group (MD = 7.69, 95% CI = 2.32–13.06, SMD = 0.53, p = 0.006). This is potentially indicative of the effectiveness of the iCST app on the QoL of carers, however, considering the small sample size and therefore large confidence interval, these results should be considered with caution.

Table 2

Adjusted Means for Outcome Measures for People with Dementia and Carers in the iCST App and TAU Control Group at FU1

FU1	iCST App	TAU	MD	Difference CI (95%)	Effect Size (SMD)	p value
Person with dementia
ADAS-Cog	16.58	17.94	−1.36	−3.71 to 1.00	0.15	0.253
QoL-AD	37.92	34.80	3.12	−0.48 to 6.73	0.40	0.088
CSDD	4.95	4.93	0.02	−1.46 to 1.51	0.00	0.972
EQ-5D	80.12	73.12	7.00	−2.07 to 16.06	0.37	0.127
QCPR	58.68	58.77	−0.09	−2.44 to 2.25	0.01	0.938
QoL-AD [P]	34.39	34.77	−0.38	−2.35 to 1.59	0.07	0.703
CSDD [P]	4.91	4.02	0.89	−0.74 to 2.53	0.27	0.278
BADLS [P]	9.55	9.55	0.00	−2.04 to 2.05	0.00	0.996
NPI total [P]	12.76	12.29	0.47	−4.74 to 5.66	0.03	0.859
Carer
EQ-5D	88.71	85.22	3.49	−1.61 to 8.57	0.29	0.176
HADS-Anxiety	5.36	5.03	0.33	−1.05 to 1.71	0.08	0.634
HADS-Depression	2.56	3.18	−0.62	−1.70 to 0.47	0.18	0.257
QCPR	56.05	58.07	−2.02	−5.40 to 1.37	0.25	0.236

Abbreviations: MD, mean difference; CI, confidence interval; SMD, standardised mean difference; [P], proxy rated measure.

Table 3

Adjusted Means for Outcome Measures for People with Dementia and Carers in the iCST App and TAU Control Group at FU2

FU2	iCST App	TAU	MD	Difference CI (95%)	Effect Size (SMD)	p value
Person with dementia
ADAS-Cog	17.00	17.48	−0.48	−3.37 to 2.40	0.05	0.735
QoL-AD	38.20	36.50	1.70	−0.45 to 3.86	0.28	0.119
CSDD	4.94	4.87	0.07	−1.76 to 1.92	0.01	0.933
EQ-5D	77.83	77.97	−0.14	−8.61 to 8.33	0.01	0.974
QCPR	58.05	58.88	−0.83	−3.76 to 2.11	0.11	0.574
QoL-AD [P]	33.25	33.13	0.12	−2.36 to 2.61	0.02	0.921
CSDD [P]	5.86	5.03	0.83	−1.35 to 3.01	0.18	0.446
BADLS [P]	9.57	10.58	−1.01	−3.04 to 1.03	0.13	0.324
NPI total [P]	11.86	10.91	0.95	−4.69 to 6.60	0.07	0.736
Carer
EQ-5Da	86.58	78.89	7.69	2.32 to 13.06	0.53	0.006
HADS-Anxiety	5.28	5.72	−0.44	−2.04 to 1.15	0.09	0.575
HADS-Depression	3.28	3.14	0.14	−0.95 to 1.23	0.04	0.798
QCPR	57.42	57.40	0.02	−2.93 to 2.97	0.00	0.990

Note: aSignificant result at p < 0.05.

Abbreviations: MD, mean difference; CI, confidence interval; SMD, standardised mean difference; [P], proxy rated measure.

The remainder of the outcomes at FU1 and FU2 did not show any significant differences between the iCST app and the TAU control group.

Adverse Events

One serious adverse event, which occurred in the control group, was reported. This included a hospitalisation of a person with dementia due to a broken hip, and was unrelated to the study. Two other adverse events were reported, both of which occurred in the experimental group. This led to the withdrawal of both dyads. One dyad was withdrawn due to a death in the family. The other dyad was withdrawn due to a study related issue. For this dyad, completing the questionnaires at FU1 in a room apart from the carer had led to some distress for the person with dementia later in the day. In order to prevent this from occurring again and after discussing this with the carer, the research team decided to withdraw the dyad from the study. A total of five dyads were withdrawn due to concerns over COVID-19.

Discussion

This study set out to evaluate the feasibility of conducting a full-scale RCT with the iCST app compared to a TAU control group. The study was designed as a feasibility RCT in order to be better informed about its appropriateness for a larger-scale study in terms of screening, recruitment, randomisation, retention, feasibility and acceptability of the iCST app, and outcome measures. Data collection was supported by a mixed methods approach where quantitative data from questionnaires and analytics was complemented by qualitative data from telephone calls and interviews with people with dementia and carers.

Study Findings

Screening, Recruitment, Randomisation, and Retention Rates

A total of 61 dyads were recruited by five study sites. The technology-related inclusion criteria were the biggest challenge, eg, some participants did not have a compatible touch-screen tablet for accessing the iCST app. For a proportion of the referrals, the reason for exclusion was unknown which will need to be better monitored in a future study. Furthermore, the involvement of a study partner, in this case a carer, may have led to an additional barrier towards recruitment as study participation then relies on the willingness of both the person with dementia and the carer.36 Some strategies to improve recruitment for a future study include more regular visits or phone calls to recruiting sites, advertisements in newspapers or on the radio, and modifying the inclusion criteria (eg, compatibility of the iCST app with more devices).37

Block randomisation was appropriate for the study and the allocation outcome was acceptable to participants given there were no drop-outs as a consequence of having been randomised to either of the two groups. The attrition rate (13%) was low for the study as only 8 dyads out of 61 dropped out of the study for reasons unrelated to the intervention.

Feasibility and Acceptability of the iCST App

Although 58% of the dyads reported being able to complete two or more activities per week which would amount to 60 minutes or more spent on the iCST app per week, analytics showed that only 11% were able to do so. This means that the previous benchmark set to determine the feasibility of the iCST app (>75% of dyads completing two or more activities per week) has not been met. Completion times most likely differed across dyads, eg, one dyad may complete Past Events in 30 minutes while others may only take 10 minutes. Challenges related to low levels of adherence were also reported in previous iCST research where less than half of the participants in the iCST group completed at least two activities per week and a smaller proportion did not complete any activities.3 However, as this included participant reported data, actual levels of adherence may have differed. This study aimed to remedy this by verifying the telephone data through analytics and therefore tracking the time spent on the iCST app. In the previous iCST study, reasons for low adherence included difficulties with fitting iCST in the daily routine due to a lack of time, poor health or the activities being too easy.3 This is similar to findings from this study where low adherence to the app was most often detected among people with very mild dementia. These participants found the content too easy and would spend less time on the app as they completed the activities relatively quickly. More tailored and appropriate content for its users may increase adherence to the iCST app.

In terms of usability and acceptability, carers rated the iCST app better than people with dementia.38 Despite giving a positive rating to its usability and most notably its design, roughly half of the people with dementia and carers were less willing to use it frequently and people with dementia gave a low rating to the usefulness of the app. Usefulness of the iCST app may have been comprised by the lack of relevance and range of the activities for some people with dementia. Carers judged the app to be more useful than people with dementia.38

Appropriateness of Outcome Measures

The majority of the outcome measures were acceptable to most participants; however, some participants found the assessments too lengthy which at times led to fatigue. The CSDD may not be appropriate for assessing symptoms of depression as participants found some areas difficult to evaluate leading to less meaningful data. This is also the case for the CUSE which was included to investigate any differences in computer user self-efficacy between the two groups. Given the length of the assessments and that data on both the CUSE and CSDD was often missing, a future study could potentially reduce the amount of outcome measures or modify the current selection. The Geriatric Depression Scale-15, which has been used in the previous large-scale iCST study, may be a suitable alternative to measure depression among people with dementia rather than the CSDD.39

Outcome Data

Considering the small sample size of the study and therefore lack of statistical power to detect effectiveness, no definite conclusions can be drawn from the results which should be interpreted with caution. However, potential signs of improvements can be identified which can be relevant for future research. For people with dementia, there were no significant differences on any of the outcome measures between the iCST app and TAU control group which is in accordance with previous, paper-based iCST research.3 In terms of computerised cognitive stimulation, Astell, Smith, Potter and Preston-Jones40 conducted a study in which they investigated the effectiveness of a group-based, computerised reminiscence and conversation tool (CIRCA). Similar to the findings of previous, paper-based CST research,2 results from Astell, Smith, Potter and Preston-Jones40 showed significant improvements in the cognition and QoL of people with dementia following the intervention. This potentially suggests that elements such as a group setting or a structured approach towards delivery may be essential in obtaining benefits on cognition and QoL.

For carers, a significant difference with a medium to large effect size in favour of participants in the iCST app group was found for QoL (EQ-5D). This is in accordance with results from the previous paper-based iCST study which also found significant improvements for the carer’s QoL on the same outcome measure.3 This suggests that a carer-led cognitive stimulation programme may be helpful to the carer him/herself. A systematic review by Tyack and Camic11 also found that touch-screen interventions can have a positive impact on the well-being of carers which could be explained by a decrease in burden and improvement in the quality of the relationship with the person they are caring for. However, for this study, the possibility of a type II error needs to be considered since it was underpowered for a full trial and some effect sizes suggested that a larger sample size may have found significant results in more domains.

In the post-trial interviews, two dyads mentioned that the iCST app had helped the person with dementia to feel more confident in their cognitive abilities and their abilities to use technology. For one dyad, this increase in confidence subsequently led the person with dementia to engage with other cognitive activities which he had previously not done. Asghar, Cang and Yu6 also found that assistive technology could encourage people with dementia to undertake activities that they were previously unable or reluctant to. Similarly, Tyack and Camic11 found similar findings whereby mastery of a touch-screen intervention for people with dementia led to increased confidence in own abilities, feelings of empowerment, and pride.

Both quantitative and qualitative findings suggest that the iCST app may be as effective as paper-based iCST but the app may have certain advantages over the paper-based version. For instance, the iCST app allows for improved monitoring of adherence, and a broader scope for updates and new activities. In addition, interactive, touch-screen technology may be better placed to promote engagement as all participants in the iCST app group actively completed a proportion of the activities whereas, for paper-based iCST, the RCT found that 22% of the participating dyads did not complete any activities.3

Strengths and Limitations

A strength of this study was that it allowed for the comprehensive investigation of multiple aspects related to the study process and the intervention to better prepare for a full-scale trial. The combination of different types of data from multiple sources with feedback from both people with dementia and carers, supported data triangulation and helped to increase the validity of the data. For instance, data from telephone calls was supported both by analytics and post-trial interviews. The addition of analytics in particular provided valuable insights in the adherence to the iCST app. Furthermore, this study had a relatively low attrition rate leading to minimal data loss.

In terms of limitations, the sample was mainly made up of white British participants which led to the underrepresentation of other minority ethnic groups in the study. The iCST app was only compatible with certain touch-screen tablets and software versions which provided an additional challenge to recruitment for all study sites. In addition, the relatively low adherence to the iCST app and variance in its use were limitations of the study. Technical difficulties also impacted adherence negatively and in a full-scale trial it is advisable to use a more stable version of the app. Lastly, the interviews included a small sample of dyads who only had a positive experience with using the app. This led to a lack of insights from dyads who found the app to be less useful and there is a need for a more in-depth qualitative evaluation of the app in a larger trial.

Recommendations for a Full-Scale RCT

Based on the current findings, it is recommended to conduct a full-scale trial with the iCST app but with the necessary modifications. Table 4 includes the Acceptance Checklist for Clinical Effectiveness Pilot Trials (ACCEPT) which consists of several trial components ranging from trial design and interventions to randomisation and data procedures.41 It describes how the various trial components have been monitored in this study, what the outcomes are in terms of recommendations for a full-scale RCT and how these can be achieved. It is recommended to make amendments to the majority of the trial components including: sample, intervention, participants, blinding, data, research governance, data analysis and trial management. The design, consent procedures, randomisation process, and Health & Safety regulations were deemed appropriate.

Table 4

ACCEPT for a Full-Scale RCT with the iCST App41

Component of Trial		Monitoring Methods	Amend?	Outcomes
Trial design		Reviewed suitability of and adherence to research protocol.	No	Trial design and related components are appropriate.
Sample size		Tested assumptions within protocol on: number of sites; recruitment rates; retention rates; and SD of primary outcomes.	Yes	Revision necessary in terms of sample size calculation; recruiting capacity; trial period; and funding.
Interventions	Clinical governance	Assessed compliance with formal training in intervention through contact with local PIs.	Yes	Enhance formal training and supervision of local researchers and/or research nurses at each site eg, by additional training visits and/or catch-ups.
	Intervention fidelity	Measured & assessed adherence to intervention through weekly telephone calls and analytics.	Yes	Enhance supervision of intervention using identifiable analytics. Extend the iCST app with more relevant activities and provide more guidance in its use eg, through the involvement of a formal carer.
Participants	Recruitment strategy	Assessed participant flow per recruitment source.	Yes	Refine recruitment strategy eg, by promoting engagement within recruitment sources (eg, memory clinics) and include other sources such as the Alzheimer’s Society.
	Eligibility criteria	Assessed reasons for ineligible participants and any barriers to recruitment.	Yes	Refine eligibility criteria eg, by making the iCST app compatible with a maximum number of devices.
Consent procedures	Participant information sheets (PIS)	Monitored PIS distribution and emergence of questions related to the PIS through contact with PIs at local site.	No	PIS are appropriate.
	Taking informed consent	Monitored consent documentation and appropriateness of forms through contact with PIs at local site.	No	Consent process and accompanying forms are appropriate.
Randomisation process		Checked randomisation procedures including use of Sealed Envelope, randomisation sequences and accessibility by researchers.	No	Randomisation procedure and training of research team are appropriate.
Blinding		Checked occurrences of unblinding by participants and whether unblinded researchers can keep other researchers blind.	Yes	Extend blinding procedures, eg, by checking whether blinded assessors can predict individual allocations.
Data	Data collection	Assessed adherence to assessments and weekly telephone calls/questionnaires.	Yes	Refine schedules to reduce assessment burden and modify outcome measure selection. Enhance training of research team in data collection tools such as outcome measures to minimise errors and missing data.
	Data quality	Tested missing data procedures listed within the analysis plan.	Yes	Refine missing data procedures in case of assessments missing in full eg, through statistical analyses.
	Data management	Tested suitability of trial database, storage of data, related procedures and software.	Yes	Refine trial database and data monitoring procedures considering the amount of data in larger trial.
Research Governance	Research protocol adherence	Tested adherence to research protocol as widely as possible through regular contact with local PIs.	Yes	Enable quality assurance officer (QAO) to test adherence as widely as possible. Refine protocol to enhance quality assurance plan and training of team.
	Adverse events (AE)	Assessed occurrences and severity of AEs, and reporting procedures.	Yes	Refine AE reporting and assessment procedures through the addition of a QAO.
	Health & Safety (H&S)	Monitored H&S procedures, eg, during installation and assessment visits.	No	Refinement to H&S procedures not necessary.
Data analysis		Tested an analysis plan on the obtained data.	Yes	Refine analysis plan to address research aims in full in terms of effectiveness on outcomes.
Trial management		Reviewed role descriptions of research team including at local sites.	Yes	Extension of research team will be necessary through a Trial Steering Committee and a Data Monitoring Committee. Refine roles eg, depending on workloads.

Notes: Adapted from Charlesworth G, Burnell K, Hoe J et al. Acceptance checklist for clinical effectiveness pilot trials: a systematic approach. BMC Med Res Methodol 13, 78 (2013). Creative Commons Attribution License ()41.

As the iCST app was used with a high level of flexibility, this may have led to the lack of any potential signs of effectiveness for people with dementia. In order to better understand the impact of the iCST app, it may need to be offered using a more structured approach. The iCST app could be offered by a formal carer as part of the care routine which has now been done with paper-based iCST leading to potential benefits in terms of cognitive functioning.42 Participants in a future study could also be given more guidance on its use to promote a more standardised approach to the app. This would need to be supported with more sophisticated adherence measures such as the use of analytics which can be linked to the individual user in combination with regular telephone calls. By standardising the “dose” across the participants, a future study may provide different study findings. As some participants in this study completed the iCST app quicker than anticipated, extending the iCST app with more content would allow them to spend more time on it and may also provide different results.

Conclusion

This study gave insights in the feasibility of conducting a full-scale RCT with the iCST app compared to a TAU control group. In terms of the study process, recruitment proved to be challenging due to a lack of eligible participants, randomisation measures were adequate, attrition was low, and some inadequate outcome measures were identified for which alternatives were found. In terms of the intervention, adherence to the iCST app and variability in its use were additional challenges. Participants largely judged the iCST app to be usable and most found it enjoyable. Lastly, there are some promising findings in terms of benefits for the QoL of carers following the use of the iCST app. It is recommended to conduct a full-scale RCT with the necessary modifications which include an increase in capacity to better support a larger sample size, recruitment and study monitoring, a more structured and guided approach towards offering the iCST app supported by adherence monitoring, an extension to the iCST app in terms of content, and an increase in device compatibility to ensure the iCST app can be accessed on more touch-screen tablets or other devices. These modifications will help to create a more suitable version of the intervention and will strengthen the design of a full-scale RCT to better understand the effectiveness and impact of the iCST app.