Short Term Economic Evaluation of the Digital Platform "Support, Monitoring and Reminder Technology for Mild Dementia" (SMART4MD) for People with Mild Cognitive Impairment and their Informal Caregivers.

BACKGROUND: A randomized controlled trial of the SMART4MD tablet application was conducted for persons with mild cognitive impairment (PwMCI) and their informal caregivers to improve or maintain quality of life.
OBJECTIVE: The objective was to conduct economic evaluation of SMART4MD compared to standard care in Sweden from a healthcare provider perspective based on a 6-month follow-up period.
METHODS: Three hundred forty-five dyads were enrolled: 173 dyads in the intervention group and 172 in standard care. The primary outcome measures for PwMCI and informal caregivers were quality-adjusted life years (QALY). The results are presented as incremental cost-effectiveness ratios, and confidence intervals are calculated using non-parametric bootstrap procedure.
RESULTS: For PwMCI, the mean difference in total costs between intervention and standard care was € 12 (95% CI: -2090 to 2115) (US$ = € 1.19) and the mean QALY change was -0.004 (95% CI: -0.009 to 0.002). For informal caregivers, the cost difference was - € 539 (95% CI: -2624 to 1545) and 0.003 (95% CI: -0.002 to 0.008) for QALY. The difference in cost and QALY for PwMCI and informal caregivers combined was -€ 527 (95% CI: -3621 to 2568) and -0.001 (95% CI: -0.008 to 0.006). Although generally insignificant differences, this indicates that SMART4MD, compared to standard care was: 1) more costly and less effective for PwMCI, 2) less costly and more effective for informal caregivers, and 3) less costly and less effective for PwMCI and informal caregivers combined.
CONCLUSION: The cost-effectiveness of SMART4MD over 6 months is inconclusive, although the intervention might be more beneficial for informal caregivers than PwMCI.

Trial registration: ClinicalTrials. rials.gov: NCT03325699

INTRODUCTION

Dementia now ranks as one of the leading causes of death [1] and disability [2] in the world, and the global costs of dementia have exceeded $1 trillion per annum [3]. In Sweden, approximately 158,000 persons live with dementia with an incidence rate of 24,000 per year, leading to a societal cost of € 7.2 billion per year [4].

Informal caregiving is an essential part of the care provided to people living with dementia (PwD), especially to allow them to remain in their own homes for as long as possible. Caregiving in the home environment is promoted to reduce the strain on healthcare systems [5], although this might make informal caregivers vulnerable to emotional stress and poor mental health [6]. Since the cure for dementia is yet to be discovered, policies aim to support persons living well with dementia [7] and delay the disease progression [7]. The likelihood of developing dementia is high among persons with mild cognitive impairment (PwMCI) [8, 9], although measures can be taken to delay the onset and learn strategies to maintain daily activities even after the onset of dementia. Therefore, targeting PwMCI and their informal caregivers with well-designed interventions might be an efficient way to manage the burden of dementia while at the same time improving well-being.

The use of information and communication technologies, including mobile phones, to remind elderly persons of healthcare appointments and other important events has been effective [10]. Further, the use of smartphone/tablet applications (app) in healthcare (mHealth) has opened an opportunity to support PwMCI’s self-care and habituating them to a technology that can be even more useful in the later stage of the disease progression. Moreover, mHealth might also be useful to ease the burden of informal caregivers, as the well-being of informal caregivers is important due to his/her central role in the care chain. The Support, Monitoring and Reminder Technology for Mild Dementia (SMART4MD) intervention try to accomplish all this (ClinicalTrials.gov: NCT03325699). The intervention was conducted in three European countries (Sweden, Spain, and Belgium) from December 2017 to September 2020 as a pragmatic randomized controlled trial (RCT) targeting PwMCI and their informal caregivers to improve and/or maintain quality of life [11].

Informed and transparent decision-making is important to allocate efficiently scarce healthcare resources. Therefore, before deciding whether to implement such solutions, decision-makers need to know if the intervention is worth the use of society’s scarce resources, i.e., whether it is cost-effective or not. Economic evaluations are central to national reimbursement decisions of new health technologies in many high-income countries, such as Sweden.

Therefore, the objective is to study if the intervention provided in the SMART4MD trial is cost-effective compared to standard care with a short-term (6-month) follow-up in Sweden.

METHODS

This economic evaluation was conducted following the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guideline [12] (see Supplementary Material) and using data from the SMART4MD trial with intention-to-treat analysis.

The SMART4MD trial

SMART4MD is a health app for tablets specifically designed for people living with mild dementia developed by Healthbit Ltd (healthbit.com), incorporating input through interviews from PwMCI, their informal caregivers, and healthcare professionals. Some important features of the app include reminders (e.g., medication, healthcare appointments), cognitive support (e.g., cognitive-stimulating games and photographs), and an optional function where health information can be shared with family and informal caregivers (e.g., day-to-day health status, specific health problems and quality of life). The purpose of the app was to improve or maintain quality of life of PwMCI and their informal caregiver. The app could help PwMCI to maintain a routine in their daily life, create and consolidate a habit that would facilitate the individual’s management of the disease in the later stages of the disease progression, and reduce stress. Each dyad in the intervention group received a 1.5-h training session on how to use the app, led by a research nurse. Detailed information on the SMART4MD app and the trial has been published elsewhere [11].

For this study, we used the data collected in the Swedish site. Participants were recruited from primary care, secondary care services (memory clinics), outpatient clinics, day hospitals, specialist mental health care unit, geriatric medicine unit, and neurology services unit in Region Blekinge in Southeast Sweden. The trial was conducted from December 2017 to September 2020 at a research clinic at Blekinge Institute of Technology, Karlskrona, Sweden. Participants were eligible if they were aged ≥55, scored 20–28 on the Mini-Mental State Examination (MMSE), and were in charge of taking their medications, had no functional disability that could hinder the use of the app, had Wi-Fi in their home, had an informal caregiver, and experienced cognitive decline for more than 6 months. Participants were excluded if they scored above 11 on the Geriatric Depression Scale [13] and were diagnosed with a terminal illness with a life expectancy of fewer than three years assessed by the research nurse based on information obtained at baseline visit and from the participants’ medical journals.

Three hundred forty-five dyads consisting of PwMCI and his/her main informal caregiver participated in the trial, randomized to intervention (n = 173) and control group (n = 172). Participants in the intervention group were provided with the SMART4MD app on tablets along with standard care for a period of 6 months. The app was supposed to be used daily by the PwMCI with the help of the informal caregiver when needed. The control group received standard care only, mainly provided by their primary healthcare center. In Sweden, the standard care for older adults generally includes at least a yearly visit to a physician for routine check-ups and renewal of drug prescriptions. Standard care reflects the normal healthcare utilization outside the trial and varies between participants due to several factors, such as general health, co-morbidities, and health-seeking behavior.

The study was performed in compliance with the ethical guidelines of the Declaration of Helsinki (World Medical Association, 2013). The SMART4MD study has been approved by the Regional Ethical Review Board in Sweden (LU No. 650-00 and No. 744-00). The older adults were provided both written and oral information about the trial and its content before written informed consent was obtained from each participant entering the trial.

Cost measures

Even though the societal perspective is the gold standard in economic evaluations, the current study is performed from the healthcare provider perspective due to the non-availability of data on important societal cost categories, such as social care and participants’ out-of-pocket expenses. In Sweden, municipalities provide community care, social services, and home support services, whereas regional councils are responsible for providing healthcare services. It should therefore be noted that the healthcare provider perspective does not include care provided by the municipalities. All costs were estimated in Swedish kronor (SEK) in 2018 price year and converted to Euro (€) using 10.23SEK/€ (US$ = € 1.19) exchange rate [14].

Since no specific diagnosis-related healthcare utilization was targeted, all types of healthcare visits were measured for both in- and outpatient care. Outpatient care includes both primary care and specialized outpatient clinics, and we thus include all healthcare utilization. Costs associated with the initiation of the project (i.e., advertisement, printed letters, follow-up calls and follow-up visits) and providing the intervention (i.e., cost of tablets, data administrator time, screening, and training of PwMCI) were not included in the analyses as these were considered research costs.

Diagnosis-related groups (DRG) is a patient classification system used to reimburse the healthcare provider based on the expected resource intensity of care. DRGs are assigned based on principal diagnosis, secondary diagnosis, surgical procedure performed, comorbidities and complications, and patient’s age, sex, and discharge status [15]. Data on healthcare utilization, including DRG-based cost per episode were obtained from the Blekinge regional council healthcare registers [16, 17]. These DRG-based costs are calculated by the healthcare provider (Region Blekinge). In case of missing cost values, we used the average cost of a visit in primary care and specialized outpatient clinics in Blekinge retrieved from Blekinge regional council healthcare registers. Since all specialized outpatient clinics have a different average cost per patient per visit (Supplementary Table 1), we used a visit’s average cost for each specialized outpatient clinic.

Effect measures

We used quality-adjusted life-years (QALY) measured by the EQ-5D-3L questionnaire [18] using the Swedish experience-based tariff as the main outcome [19]. We used the area-under-the-curve approach to estimate the change in QALY [20].

We also used the Quality of Life-Alzheimer’s Disease (QoL-AD) [21] and MMSE score [22] to measure quality of life and cognitive change over time. Both PwMCI and their caregiver completed the QoL-AD in parallel, i.e., the informal caregiver proxy-reported on behalf of the PwMCI. The reports are combined into a weighted composite score by giving twice the weight to the patient’s assessment than the caregiver’s assessment following Logsdon et al. [23]. MMSE is a standardized instrument to assess cognitive impairment [22] based on mental abilities such as memory, attention, and language. The maximum score of MMSE is 30 points, where a higher score means a better performance.

The main effect measure for informal caregivers was the QALY, measured as above. The secondary outcome was the carer burden, evaluated using the short form Zarit Burden Interview (ZBI-12) [24]. This self-reported 12-item scale covers the extent to which caring affects informal caregivers’ health and social life. As a lower score indicates a better outcome, we have reversed the ZBI score to have a more intuitive interpretation in line with the other effect measures.

The missing health outcome values at 6-month were treated as missing in base case statistical analysis.

Analysis of cost-effectiveness

Cost-effectiveness was estimated as incremental cost-effectiveness ratio (ICER) and Net Monetary Benefit (NMB). ICER is a ratio of the difference in average costs per participant to the difference in average health benefits per participant of the intervention group compared to the control group. Sampling uncertainty was assessed using 5,000 bootstrap re-samples to estimate ICERs for PwMCI, informal caregivers and for PwMCI and informal caregivers combined.

The point estimates and the bootstrapped ICERs are graphically presented on a four-quadrant cost-effectiveness (CE) plane where the x-axis measures effect differences and the y-axis measures cost differences between intervention and control groups. Both the southeast (SE) and northwest (NW) quadrants of the CE-plane explicitly indicate whether an intervention is cost-effective (dominant if the ICER is located in SE) or not (dominated if the ICER is located in NW) compared to its comparator. However, cost-effectiveness is more challenging to determine if the ICER is located either in the southwest (SW) or northeast (NE) quadrant. The SW quadrant shows the possibility of an intervention being less effective and less costly compared to its comparator, and the cost-effectiveness decision thus depends on the willingness to accept, i.e., how much savings is required for society to be willing to forgo a health benefit. The NE quadrant reflects an intervention that is more effective and more costly than its comparator and cost-effectiveness depends on society’s willingness-to-pay (WTP), i.e., how much the society is willing to pay to receive an additional health benefit [25].

We also estimated the cost-effectiveness acceptability curve (CEAC) to present the likelihood of the intervention being cost-effective compared to standard care for a range of WTP thresholds. Although Sweden does not follow a specific threshold, the Swedish National Board of Health and Welfare regards costs above 500,000 SEK (€ 48,876) as high cost per QALY gained [26]. We, therefore, use this as the WTP threshold in this study.

We also present the results as a net monetary benefit (NMB) which shows the value of an intervention in monetary terms when the value of the benefit (QALY) is known. NMB is calculated as ((incremental benefit × WTP) –incremental cost). A positive NMB indicates that the intervention is cost-effective compared with the alternative. NMB has several advantages over ICER, including being linear, having simple sampling distribution and being more stable than the ICER when the differences in effectiveness between the intervention and comparator is small [27].

All health outcome measures were adjusted for baseline values using multivariable ordinary least square regression. Statistical analyses were performed to assess the statistical differences between intervention and control groups (inter-group analyses: independent t-test for continuous variables and chi-square for categorical variables) and statistical differences within intervention and control groups (intra-group analyses: paired t-test). The analyses were executed using Stata/SE 15.1 (StataCorp LP, College Station, TX, USA) and SPSS (IBM Corp. Released 2020. IBM SPSS Statistics for Macintosh, Version 27.0. Armonk, NY: IBM Corp). Since the time period for analyses was less than 1 year, we did not discount costs or effects.

Sensitivity and subgroup analyses

Several sensitivity and subgroup analyses were performed to capture uncertainties around the base case estimates. All these analyses were performed for 1) PwMCI, 2) caregivers, and 3) dyads, but only with the main outcome measure –QALY.

Complete case analysis: Missing health outcome data is a common problem associated with RCTs. A sensitivity analysis was performed where participants lost-to-follow-up were removed from all analyses.

Multiple imputations for missing QALYs: We imputed missing information on EQ-5D-3L index score for both groups using multiple imputations [28]. We generated ten different data sets for each group in SPSS using Markov chain Monte Carlo method. The variables used in the models are age, sex, intervention/control group and reasons for dropout. We also used baseline EQ-5D-3L index score while imputing the follow-up EQ-5D-3L index score. After that, the multiple datasets were analyzed and pooled estimates were computed following the methods described by Rubin [29].

Using the UK tariff for QALY estimation: The UK tariff [30] is widely used in the scientific literature and increases comparability with other studies.

Intervention cost: The intervention included the use of tablets and a training session held by the research nurse on how to use the app. These costs are not expected to fall on the healthcare provider in case of wide implementation and are not included in the base case estimate. Therefore, this sensitivity analysis includes these costs, using the price of the tablets including mobile data and the salary of research nurse, as a cost of the intervention.

Removing zero healthcare cost: Given their age and health condition, PwMCI in the trial were expected to have healthcare contacts during the trial period. As a sensitivity analysis, we excluded PwMCI as well as informal caregivers with zero healthcare cost as potential outliers.

Removing top 5% cost outliers: The top 5% (cost≥€ 25,741) were removed from the analysis to exclude the high-cost outliers.

Subgroup analysis –sex: Due to the difference in health-seeking behavior between men and women, we stratified the results based on sex.

Subgroup analysis –age: Results are stratified based on age (up to 70 versus 70 + years of age) as familiarity and use of tablets might differ with age.

Subgroup analysis –MMSE: Results are stratified based on MMSE score at baseline (≤26 versus > 26) [31] as the level of cognitive decline might affect the outcome of the intervention.

RESULTS

Baseline characteristics of the PwMCI and informal caregivers showed no significant differences between intervention and control groups (Table 1). After 6 months, 48 dyads (14%) dropped out from the trial and one dyad was missing due to incomplete data (Supplementary Figure 1). The reasons for dropping out are presented in Supplementary Table 2. In addition, both intervention and control groups lost one dyad due to cognitive reasons. The only statistically significant differences at baseline between those dyads that do and do not drop out of the study are noted in the control group (dropouts had worse ‘EQ-5D-3L index score’, were more likely to have secondary education and being single, while the informal caregiver was about 10 years younger) (Supplementary Tables 3-4).

Table 1

Baseline and 6-month follow-up characteristics of PwMCI and informal caregivers

Characteristics	PwMCI				Informal caregiver
	Baseline		Follow-up		Baseline		Follow-up
	Intervention (n = 173)	Control (n = 172)	Intervention (n = 138)	Control (n = 158)	Intervention (n = 173)	Control (n = 172)	Intervention (n = 138)	Control (n = 158)
Agea	76.13 (5.06)	76.31 (5.18)	75.98 (5.20)	76.35 (5.23)	70.02 (10.45)	69.24 (11.35)	69.82 (10.11)	70.01 (10.70)
Gender, n (%)
Male	97 (56)	103 (60)	80 (58)	98 (62)	57 (33)	53 (31)	42 (30)	46 (29)
Female	76 (44)	69 (40)	58 (42)	60 (38)	116 (67)	119 (69)	96 (70)	112 (71)
Education, n (%)
Elementary School	57 (33)	64 (37)	47 (34)	61 (39)	44 (26)	36 (21)	40 (29)	35 (22)
Secondary School	57 (33)	38 (22)	47 (34)	30 (19)	64 (37)	60 (35)	48 (35)	55 (35)
Higher Education	58 (34)	70 (41)	43 (32)	67 (42)	63 (37)	75 (44)	48 (35)	67 (43)
Civil status, n (%)
Single	46 (27)	42 (24)	38 (28)	35 (22)	23 (13)	20 (12)	18 (13)	18 (11)
Married/living together	127 (73)	130 (76)	100 (72)	123 (78)	150 (87)	152 (88)	120 (87)	140 (89)
QoL-ADa	40.55 (5.35)	40.89 (5.41)	39.83 (4.66)	39.71 (5.13)	39.40 (5.95)	39.16 (5.92)	38.36 (5.37)	38.03 (6.14)
EQ-5D-3L index scorea	0.894 (0.08)	0.886 (0.09)	0.869 (0.099)	0.876 (0.101)	0.90 (0.08)	0.89 (0.09)	0.88 (0.10)	0.87 (0.12)
MMSE scorea	26.53 (1.79)	26.78 (1.64)	27.69 (2.18)	27.42 (2.46)
ZBIa,b					43.53 (6.12)	43.13 (6.61)	43.32 (6.45)	43.20 (6.49)

MMSE: Mini-Mental State Examination; PwMCI: person with mild cognitive impairment; QoL-AD: quality of life in Alzheimer’s disease; ZBI: Zarit Caregiver Burden Inventory; n = number; % = percentage. aMean (standard deviation); bn = 171 for control group at baseline. Significance levels: p < 0.05*, 0.01** and 0.001***.

The total 6-month cost per PwMCI in the intervention and control group was on average € 8,188 and € 8,175 per person, respectively (Table 2). The greatest share of healthcare costs in both groups was related to outpatient care (75%). For informal caregivers, the total 6-month cost for intervention and control groups were on average € 6,050 and € 6,589, respectively (Table 2). Informal caregivers in the intervention group had higher outpatient and lower inpatient costs compared to the control group. The total 6-month cost for the dyads was lower for the intervention group (€ 14,238) compared to the control group (€ 14,764). None of the cost differences between the groups was statistically significant.

Table 2

Number of healthcare visits and related cost (€) for the participants (mean and standard error)

	PwMCI		Informal Caregiver
	Intervention (n = 173)	Control (n = 172)	Intervention (n = 173)	Control (n = 172)
Outpatient care visits	24.16 (1.91)	23.56 (1.96)	17 (1.57)	16.77 (1.58)
Inpatient admissions	0.31 (0.05)	0.38 (0.06)	0.28 (0.06)	0.35 (0.06)
Inpatient days	2.36 (0.76)	2.01 (0.49)	2.79 (1.15)	3.86 (1.59)
Outpatient care cost	6,155 (532)	6,148 (510)	4,346 (469)	4,222 (405)
Inpatient care cost	2,033 (417)	2,027 (385)	1,704 (403)	2,367 (478)
Total cost	8,188 (762)	8,175 (751)	6,050 (763)	6,589 (742)

Independent sample t-test is used to assess the statistical differences between intervention and control group (inter-group [between groups] analysis). No statistically significant differences were found.

In both groups, PwMCI experienced a statistically significant loss in quality of life over the 6 months period. However, the MMSE score improved in both groups, statistically significantly more so in the intervention group (Table 3). Like PwMCI, a decline in quality of life (EQ-5D-3L index score) over the study period was observed for informal caregivers in both groups, although only statistically significant in the control group (Table 3).

Table 3

Baseline to 6-month change in health effects within (intra-group differences) intervention and control group and change in health effects between (inter-group differences) intervention and control groups

	Intervention (n = 138)	Control (n = 158)	Inter-group difference	p (inter-group differences)
PwMCI
EQ-5D-3L index scorea	–0.03 (0.008)	–0.01 (0.007)	–0.01 (0.01)	0.1759
Composite QoL-ADa,b	–0.76 (0.27)	–1.27 (0.28)	0.51 (0.40)	0.2027
MMSE scorea	1.06 (0.16)	0.60 (0.14)	0.46 (0.21)	0.0331
Informal Caregiver
EQ-5D-3L index scorea	–0.01 (0.007)	–0.02 (0.007)	0.01 (0.01)	0.2686
ZBIa,c	–0.26 (0.46)	–0.15 (0.37)	–0.11 (0.58)	0.8439
Dyads (PwMCI plus Informal Caregiver)
EQ-5D-3L index scorea	–0.04 (0.01)	–0.04 (0.01)	–0.003 (0.01)	0.8205

MMSE: Mini-Mental State Examination; PwMCI: person with mild cognitive impairment; QoL-AD: quality of life in Alzheimer’s disease; ZBI: Zarit Caregiver Burden Inventory. Bold means significant intra-group (within group) differences at 5% using paired t-test. Independent t-test is used to assess the statistical differences between intervention and control group (inter-group [between groups] analysis). No statistically significant differences were found between intervention and control group except for MMSE using independent t-test. aMean (standard error); bn = 137 for intervention & n = 157 for control group; cn = 157 for control group.

The inverse ZBI score showed an insignificant increase in caregivers’ burden in both groups after 6 months. For the dyads, the quality of life (EQ-5D-3L index score) was reduced over the study period. Table 4 shows the difference in health effects between the intervention and control groups while controlling for baseline differences. No significant differences were noted

Table 4

Differences in pooled mean cost and health effects with 95% CI, ICERs, and NMB (€)

Effectiveness measures	Sample sizea		Cost difference		Effect difference		ICERs	NMB
	Intervention	Control	ΔC	Bootstrap 95% CI	ΔE	Bootstrap 95% CI
PwMCI
mean QALY change	173/138	172/158	12	–2090.33 to 2115.28	–0.00358	–0.009 to 0.002	Dominated	–187
Adjusted QoL-AD	173/173	172/171	12	–2090.33 to 2115.28	0.3322	–0.42 to 1.08	36
MMSE adjusted	173/173	172/172	12	–2090.33 to 2115.28	0.2100	–0.12 to 0.54	57
Informal Caregiver
mean QALY change	173/138	172/157	–539	–2623.78 to 1545.42	0.0028	–0.002 to 0.008	Dominant	676
ZBI (adjusted)	173/173	172/171	–539	–2623.78 to 1545.42	0.23	–0.72 to 1.18	Dominant
Dyads (PwMCI plus Informal Caregiver)
mean QALY change	173/138	172/157	–527	–3621.48 to 2568.06	–0.00083	–0.008 to 0.006	634 940	486

ICER: incremental cost-effectiveness ratio; MMSE: Mini-Mental State Examination; NMB: net monetary benefit; PwMCI: person with mild cognitive impairment; QALY: quality-adjusted life years; QoL-AD: quality of life in Alzheimer’s disease; ZBI: Zarit Caregiver Burden Inventory. Incremental effect with positive value represent improved outcomes. We reversed ZBI scores in order to obtain this. While dominance may be demonstrated, no significant differences in incremental costs and effects were found (Independent t-test). Adjustments were made on 6-month values of effectiveness measures by regressing them on baseline values. aNumber of participants available for cost estimation first, followed by number of participants available for health effects.

Cost-effectiveness analysis

For PwMCI, the intervention was dominated by standard care as the intervention group had higher costs (€ 12) and lower QALY compared to the control group. This was also shown with the negative NMB (–€ 187) (Table 4 and Supplementary Table 5). The CE plane showed that incremental CE-pairs were spread in all four quadrants with 47% in the northwest quadrant (more costly and less effective). Given a WTP of € 48,876 per QALY, the CEAC indicated that the intervention had less than 50% probability of being cost-effective (Fig. 1). However, as there was a gain in QoL-AD and MMSE scores in the intervention group compared to the control group, the ICERs for these outcome measures were € 36 and € 57 per unit gain in QoL-AD and MMSE, respectively.

Fig. 1

CE-plane from healthcare provider perspective and CEAC indicates probability of the SMART4MD being cost-effective at different values (€) of willingness-to-pay per QALY gain.

For informal caregivers, the intervention dominated standard care, i.e., the intervention was less costly and more effective in terms of QALY, with a NMB of € 676 (Table 4 and Supplementary Table 5). Sixty percent of the CE-pairs were in the southeast quadrant (less costly and more effective) followed by 26% in the northeast quadrant (more costly and more effective) in the CE-plane. The CEAC indicated that the intervention had 70% probability of being cost-effective at WTP of € 48 876 per QALY for the caregivers (Fig. 1). Moreover, the intervention also dominated standard care in terms of ZBI.

Combining PwMCI and informal caregiver indicated that the intervention group had lower costs and lower QALY than the control group. This means that the intervention can be considered cost-effective if the society’s willingness-to-accept a QALY loss was lower than the estimated ICER of € 634,940. Presenting these results in terms of the NMB (€ 486) indicated that intervention was cost-effective at the WTP of € 48,876 (Table 4 and Supplementary Table 5). Thirty-seven percent of the CE-pairs were in the southwest quadrant (less costly and less effective) in the CE-plane and the intervention had 60% probability of being cost-effective at € 48,876 WTP.

The results of the sensitivity and subgroup analyses are reported in Table 5. The only significant difference between intervention and control group was found for imputed QALY change for PwMCI. Although numerical changes in the costs and effects were noted across analyses, the general base case results hold. The exceptions were the results stratified for men and above 70 years of age for PwMCI, where the intervention appeared to be less costly and less effective as compared to base case results (intervention dominated by standard care).

Table 5

Sensitivity analyses from healthcare provider perspective in ICERs

No.	Scenarios	Sample sizea		Changes in cost (Bootstrap 95% CI)	Changes in effect (QALY) (Bootstrap 95% CI)	ICER (€)
		Intervention	Control
PwMCI
	Base case	173/138	172/158	12 (–2090 to 2115)	–0.00358 (–0.009 to 0.002)	Dominated
1	Complete case	138/138	158/158	27 (–2078 to 2131)	–0.00358 (–0.009 to 0.002)	Dominated
2	Imputed QALY change	173/173	172/172	12 (–2090 to 2115)	–0.0052 (–0.01 to –0.0005)*	Dominated
3	UK tariff	173/138	172/158	12 (–2090 to 2115)	0.000025 (–0.01 to 0.01)	489 796
4	Intervention cost	173/138	172/158	185 (–1917 to 2288)	–0.00358 (–0.009 to 0.002)	Dominated
5	Removing zero healthcare cost	164/130	165/153	115 (–2018 to 2248)	–0.00359 (–0.009 to 0.002)	Dominated
6	Removing high-cost outliers	165/133	162/151	132 (–1234 to 1498)	–0.00377 (–0.009 to 0.001)	Dominated
7a	Men only	97/80	103/98	–347 (–3232 to 2539)	–0.00428 (–0.0099 to 0.001)	81 075
7b	Women only	76/58	69/60	704 (–2353 to 3761)	–0.00258 (–0.01 to 0.007)	Dominated
8a	Age≤70	21/19	22/21	2275 (–3695 to 8246)	0.00550 (–0.0031 to 0.01)	413 636
8b	Age > 70	152/119	150/137	–321 (2560 to 1917)	–0.004999 (–0.01 to 0.0007)	64 213
9a	MMSE≤26	67/52	54/48	–460 (–3917 to 2996)	–0.000507 (–0.01 to 0.009)	907 477
9b	MMSE > 26	106/86	118/110	138 (–2452 to 2729)	–0.0048 (–0.01 to 0.001)	Dominated
Informal Caregiver
	Base case	173/138	172/157	–539 (–2623 to 1545)	0.0028 (–0.002 to 0.008)	Dominant
1	Complete case	138/138	157/157	–527 (–2806 to 1752)	0.0028 (–0.002 to 0.008)	Dominant
2	Imputed QALY change	173/173	172/171	–539 (–2623 to 1545)	0.0012 (–0.003 to 0.005)	Dominant
3	Removing zero healthcare cost	153/122	151/137	–665 (–2912 to 1583)	0.0025 (–0.003 to 0.008)	Dominant
4	Removing high-cost outliers	165/132	163/149	–404 (–1672 to 863)	0.0016 (–0.003 to 0.006)	Dominant
5	UK tariff	173/138	172/157	–539 (–2623 to 1545)	–0.0047 (–0.02 to 0.008)	114 681
6a	Men only	57/42	53/46	–165 (–3748 to 3419)	0.00177 (–0.005 to 0.009)	Dominant
6b	Women only	116/96	119/111	–732 (–3313 to 1849)	0.00321 (–0.003 to 0.0095)	Dominant
7a	Age≤70	75/63	76/68	–518 (–2694 to 1658)	0.00699 (0.002 to 0.01)	Dominant
7b	Age > 70	98/75	96/89	–613 (–3814 to 2588)	–0.000807 (–0.009 to 0.007)	759 603
8a	ZBI = 48 (No burden)	42/42	47/47	2177 (–2744 to 7098)	–0.000084 (–0.009 to 0.009)	Dominated
8b	ZBI < 48 (Some burden)	96/96	111/110	–1650 (–4179 to 879)	0.0039987 (–0.002 to 0.0098)	Dominant
Dyads (PwMCI plus Informal Caregiver)
	Base case	173/138	172/157	–527 (–3621 to 2568)	–0.00083 (–0.008 to 0.006)	634 940
1	Complete case	138/138	157/157	–450 (–3682 to 2782)	–0.00083 (–0.008 to 0.006)	542 169
2	Imputed QALY change	173/173	172/171	–527 (–3621 to 2568)	–0.004 (–0.01 to 0.002)	131 750
3	Intervention cost	173/138	172/157	–354 (–3449 to 2741)	–0.00083 (–0.008 to 0.006)	426 506
4	Removing zero healthcare cost	172/137	172/157	–444 (–3514 to 2627)	–0.00091 (–0.008 to 0.006)	487 912
5	Removing high-cost outliers	171/138	170/157	–770 (–3574 to 2034)	–0.0011 (–0.008 to 0.006)	700 000
6	UK tariff	173/138	172/157	–527 (–3621 to 2568)	–0.00495 (–0.02 to 0.01)	106 465
7a	Men only	151/119	155/143	–402 (–3715 to 2912)	–0.00202 (–0.0096 to 0.006)	199 010
8b	Women only	170/135	171/157	–638 (–3712 to 2437)	0.0000088 (–0.007 to 0.007)	Dominant
9a	Age≤70	84/71	81/72	–887 (–4677 to 2903)	0.00598 (–0.002 to 0.01)	Dominant
9b	Age > 70	161/127	155/140	–1211 (–4461 to 2040)	–0.0025 (–0.01 to 0.005)	484 400

ICER: incremental cost-effectiveness ratio; MMSE: Mini-Mental State Examination; PwMCI: person with mild cognitive impairment; QALY: quality-adjusted life years; ZBI: Zarit Caregiver Burden Inventory. aNumber of participants available for cost estimation first, followed by number of participants available for health effects. Incremental effect with positive value represent improved outcomes. We reversed ZBI scores in order to obtain this. Significance levels: p < 0.05*, 0.01** and 0.001***.

DISCUSSION

We performed an economic evaluation of the SMART4MD trial and focused on both PwMCI and informal caregivers from a healthcare provider perspective with 6-month duration. We found no significant differences in effects and costs between the intervention and standard care groups. Therefore, the results should be interpreted carefully since there does not seem to be any noticeable difference between the two approaches in the short run.

Looking at the numerical differences, we find that standard care dominated the intervention for PwMCI when QALY was used as the outcome measure. This finding is supported by a sensitivity analysis where imputed values of QALY were used (Table 5). In contrast, the intervention dominated standard care for informal caregivers. This implies that the SMART4MD intervention was more beneficial to the caregivers than the PwMCI. This is further enhanced by the results using care burden (ZBI) as the outcome measure. When combining PwMCI and caregivers, the intervention reduced both the costs and QALY, indicating that resources could be saved at the expense of loss of quality of life in the range of the present WTP threshold. The CEAC curve showed that the intervention has < 50%, 70%, and 60% probability of being cost-effective for the PwMCI, caregivers and dyads respectively at € 48,876 WTP, indicating a likelihood to be considered good value for money for caregivers and dyads.

We have not included the intervention cost in the base case analysis, even though the trial distributed tablets and provided training on using the app. This is because the use of smartphones or tablets is common, and it can be assumed that PwMCI has access to a smartphone or tablet and know how to use an app. Thus, it is considered that in the case of a potential implementation of the SMART4MD app, these costs would not fall on the healthcare provider. Indeed, the idea is that healthcare professionals will describe the benefits and encourage PwMCI and their informal caregivers to use this app. Still, we conducted a sensitivity analysis to test this assumption (Table 5). The results showed that this had no impact on the conclusions of the study.

In contrast, considering other outcome measures besides QALY for PwMCI, the ICER were € 36 per QoL-AD gain and € 57 per MMSE score gain, respectively. It is not clear whether these values can be considered as cost-effective as there is a lack of established cost-effectiveness thresholds for QoL-AD and MMSE. However, the ICERs are very low, and the societal value of a one-unit change in these instruments is likely much higher. Previously, a 1-3-point decrease in MMSE in one year and an improvement of 3 points on QoL-AD have been considered clinically meaningful differences [32, 33].

In comparing the different outcome measures, it should be noted that QALY measured by the EQ-5D-3L instrument might not be the most appropriate way to measure quality of life for PwD. For example, the EQ-5D-3L lacks attributes to capture cognition [34] and relationships with caregivers and social support, which are considered important factors [35]. It is thus possible that the QoL-AD and MMSE measures indicate a more effective intervention than what the results based on the QALY would imply. In addition, the use of other health-related measures such as SF-6D or DEMQOL might be interesting but was unavailable for the current study.

We failed to find any noticeable differences between the intervention and control groups in the current study. This is likely due to the short period (6 months) of the trial, as 6 months could be argued to be too short for health effects to manifest. Furthermore, the participants are at an early stage of the disease progression and might still largely cope with the cognitive decline, especially in partnership with an informal caregiver. Therefore, it is perhaps not surprising that the informal caregivers show the largest increase in quality of life since some of their responsibilities are covered by the app. Although the app was intended for PwMCI, we do not know exactly how the app was used. It is possible that the app reminded informal caregivers about medications and doctors’ appointments related to the PwMCI, which could have reduced their stress to memorize PwMCI’s schedule. Furthermore, SMART4MD might yield additional benefits when PwMCI has more than one informal caregiver, i.e., the effect of SMART4MD might be underestimated.

Finally, one central purpose of introducing the app to PwMCI was to create and consolidate a habit that would facilitate the individual’s disease management in the later stages of the disease progression. Therefore, a 6-month follow-up period could be considered too short of capturing this effect. However, it should be noted that due to the rapid progression of dementia diseases, a 6-month follow-up period is common in the field [36]. All this may imply that the current study under-estimates the cost-effectiveness of the SMART4MD intervention because one may expect greater positive effects with a longer follow-up. Among the drop out participants from the intervention group, most participants stated that they do not want to participate in the trial (Supplementary Table 2). One possible reason behind the dropout and insignificant effectiveness of the trial might be that the older adults faced difficulties in using the SMART4MD app. It might also be possible that the participants did not realize the current benefits of using the app as revealed in other studies based on SMART4MD trial [37-39]. Therefore, they are missing out on potential long-term benefits of developing a habit that could be useful in the later stages of the disease progression. Unfortunately, this study lacked information on the actual use of the app.

Published evidence on the cost-effectiveness of different mHealth interventions targeting older adults with chronic conditions in a home-care setting is inconclusive [40]. For instance, mHealth interventions were reported not cost-effective compared to reference care for patients with chronic obstructive pulmonary disease in the UK [41] and in Denmark [42]. However, the use of a smartphone/tablet for older adults diagnosed with Parkinson’s disease [43] and cardiac rehabilitation [42] was cost-effective. In addition, automated diabetes remote monitoring and management systems have been cost-effective compared to usual care [44]. Thus, there is a lack of comprehensive evidence regarding the use of mHealth for older adults diagnosed with chronic conditions in general and with mild cognitive impairment or dementia in particular. Furthermore, due to differences between interventions, for example, in terms of learning curve for healthcare professionals and/or participants [41, 42], level of engagement of healthcare professionals [42], and participants’ adherence rates [43] it is also unlikely that cost-effectiveness of specific mHealth intervention are broadly generalizable within the field of mHealth.

With an increasing number of PwD in ageing societies, cost-effective interventions that can facilitate ageing in the home environment and reduce the burden on informal caregivers are of great importance. However, economic evaluations of mHealth interventions for older adults in-home care settings rarely include informal caregivers in the analyses [40]. Moreover, no economic evaluation has previously been conducted on mHealth interventions designed for PwMCI or PwD and their informal caregivers to the best of our knowledge. These are the main strengths of the current study. In addition to that, it is based on a pragmatic RCT [45]. However, the short follow-up is a limitation and positive effects might become visible with a longer follow-up period, given continuous use of the app. Another limitation is the absence of data on medication, which is an important part of the healthcare provider perspective. Also, we did not account for baseline cost differences between the intervention and control group while estimating follow-up costs due to lack of data. This might have an effect on result as baseline healthcare utilization might influence the follow-up healthcare utilization of the participants [46].

The sample size was calculated based on clinical outcome measures and not with respect to cost-effectiveness analyses. This can be seen in the cost data where the statistical variation is too large to make a meaningful interpretation of the cost difference. This indicates that the trial is underpowered for cost calculations, leading to difficulties establishing differences in costs between treatment groups. Therefore, we conducted a post-hoc power calculation for cost-effectiveness [47] using SMART4MD trial data on dyads’ cost and outcome (QALY). Based on an assumed willingness-to-pay threshold in the range of € 100–€ 48,876, a sample needed in each group ranged from 11,339–13,722 dyads. This is far from the actual sample size of 173/172. Therefore, the cost result presented in the current study should not necessarily be considered a negative result but rather that it is difficult to draw meaningful conclusions due to an underpowered trial in terms of cost-effectiveness. However, significant difference was not found in terms of the primary outcome of the trial, i.e., QoL-AD [11], even though the trial was powered based on QoL-AD.

Due to a lack of data on the actual use of the app, our analysis is confined to studying the presence of the app, which provides PwMCI an opportunity to use it. This also means that we lack data on any behavioral change connected to the use of the app. Therefore, the focus of the current study was on the actual sought-after achievement, an improvement in health-related quality of life, as an improvement in QoL would indicate a behavioral change. Given the results, further studies are now needed to investigate if the intervention failed to change the participants’ behaviors or if the behavioral change is without effect on the main outcome measure (health-related quality of life). Lastly, we could not perform the analysis from a societal perspective as we lacked important cost categories (e.g., cost for the social support, cost of informal care and participants’ out-of-pocket expenses).

Conclusion

The cost-effectiveness of SMART4MD is inconclusive and should be interpreted cautiously. Some lessons learned in this study are to consider power calculation following guidelines for sample size requirement for economic evaluation and collect information on who in the dyad uses the app and to what extent. This would allow for focusing on behavior change in the analysis and thereby better isolate the true effect of the app.

Click here for additional data file.