Growing pains: Lessons learned from a failed mobile mindfulness clinical trial for patients with complex care needs.

This paper discusses lessons learned from a failed clinical trial investigating the use of a mobile application (app) to deliver a mindfulness intervention to middle-aged and older adults receiving services at a rehabilitation hospital in Ontario, Canada. A randomized controlled trial with 82 participants was planned, with the experimental group receiving access to a mindfulness app and a wait-list control group receiving access to the app after 4 weeks; however, the study could not be completed due to low recruitment rates. This implementation failure was considered from the perspective of the PARIHS framework. More specifically, Three key recruitment challenges were identified, and recommendations for future research provided. Firstly, the increasingly complex care needs of the study population appeared to influence eligibility; it would be beneficial for future research to consider adopting strategies to better understand the needs of the target population. Secondly, participants' stage of care and readiness of change likely negatively influenced compliance and retention in this study, and should be assessed in future research. Finally, a lack of clinician integration into the research team negatively impacted recruitment in this study; future studies should consider integrating direct service providers into the research team as this may increase buy-in and referral rates. The challenges and recommendations outlined can inform design and implementation of future studies in this area.

Introduction

Mindfulness is an adaptive psychological mechanism that is known to play a role in protecting middle-aged and older adults from the negative effects of stress on their mental health [1]. Mindfulness-based stress reduction (MBSR), an intervention that trains individuals in mindfulness techniques through meditation and gentle mindful movement, has been the subject of much research among the aging population, and is known to have positive psychosocial outcomes [2]. Although most MBSR interventions employ standard 8-week group training delivered face-to-face, mindfulness training is increasingly being delivered online using technological platforms such as websites and mobile applications, in an effort to improve accessibility, flexibility, and cost-effectiveness [3]. Given these benefits, the use of technology to deliver these interventions for the aging population may be a valuable resource to support aging in place [[4], [5], [6], [7]].

The present study intended to investigate the use of a mobile app to deliver a mindfulness intervention for middle-aged and older adults with complex medical conditions undergoing rehabilitation through a small-scale clinical trial. Patients in this populations often have multimorbidity and complex care needs [8] and are particularly vulnerable to stress resulting from a combination of physical challenges, psychosocial issues and potential side-effects of treatment [9]. Furthermore, they are at a stage of life where financial concerns are often paramount [10,11]. As such, they may benefit from interventions to help cope with these stressors in a cost-effective manner, while transitioning away from hospital care. The primary aims of the study were to determine the feasibility of using a mobile mindfulness app for middle-aged and older patients with multimorbidity undergoing rehabilitation, its efficacy in relieving symptoms of stress, improving quality of life and reducing cost of treatment.

The study failed due to low recruitment rates. Consequently, this paper begins with a brief description of the intervention and the sample, before focusing on the recruitment process and the challenges faced during the study to support other researchers studying technology-enabled interventions with aging patients with complex care needs. Considering the increasingly common multimorbidity of older adults admitted to inpatient rehabilitation settings [[12], [13], [14]] and that technology offers a cost-effective method for reaching more patients [[15], [16], [17]], disseminating such lessons is crucial for advancement in this area.

Methods

Study design and sample

The study aimed to employ a randomized controlled design with 1:1 equal allocation to experimental or control group. The required sample size after accounting for attrition was estimated to be 82 (41 each in the experimental and control groups). Recruitment commenced in April 2019 after receiving approval from the Institutional Research Ethics Board and registration on the clinical trials registry (Registration Number NCT03908918). The sample was drawn from patients receiving services at a rehabilitation hospital in Ontario, Canada, and meeting the following inclusion criteria: (1) men and women over the age of 40 (2) admitted as in-patients (3) with access to a smart phone with data connection (4) willingness to give time for mindfulness practice (5) sufficient ability to speak and read English (6) willingness to be randomized into experimental or control groups and complete all assessments. Due to delays in obtaining ethics approval, outpatients were also recruited from the start of the study. Participants with cognitive or mental impairments (<6 on the Brief Screen for Cognitive Impairment) [18], those admitted to the hospital's long-term or palliative care and those using an existing smartphone-app based mindfulness practice of one or more sessions per week were excluded.

Recruitment procedure and materials

Participants were primarily recruited through referrals from members of the allied health team at the hospital who acted as “Clinical Champions”. Clinical Champions were identified and informed about the study through three “Lunch and Learn” sessions held by the research team and brief study introductions at relevant clinician meetings. They were also provided with recruitment materials to assist in identifying participants meeting the inclusion criteria, including an information flyer and a consent to contact form. Clinicians who identified potentially eligible patients from the unit/clinic census provided them with the flyer and obtained consent for the research team to contact them. The signed consent to contact form was sent to the Study Coordinator. The research team also used supplementary recruitment strategies, including weekly reminder emails to clinical champions for recruitment and reminders/updates at clinician meetings.

Referred patients were then contacted and given a choice of completing an eligibility screening and the study consent process either through an online survey hosted by SurveyMonkey or in person with the study coordinator. Consenting participants completed the baseline measures and were randomized into experimental or control groups immediately after. Block randomization was used to ensure equal sample sizes and gender balance.

Intervention and follow-up

The app used in this study was developed by Mobio Interactive Inc. (MI), a mobile technology development company, incorporated in Toronto, Ontario, Canada. Am Mindfulness is a next-generation version of Wildflowers, an app which has demonstrated efficacy in a previous randomized controlled trial [19]. Am Mindfulness incorporates Wildflowers curriculum with improved UX/UI for elderly patients and is available for free download from the Apple and Google app stores with some free and paid features. It offers several mindfulness-based activities, including a customized library of pre-recorded guided meditations that aim to ameliorate psychosocial issues such as stress, anxiety, sadness.

Participants in the experimental group were provided with a link to the Am Mindfulness app immediately after randomization and completion of baseline assessments. The Study Coordinator also conducted an orientation tutorial to assist participants with downloading and using the app features. Participants were asked to perform one 10-min lesson on the app for at least four days a week, by following a program plan (a core curriculum of 8 lessons and accompanying meditations and a larger library of other meditations). The company provided daily data on the frequency of participants’ usage of the app, which was used to monitor participation and send reminder notifications when necessary. Participants in the wait-list control group received treatment as usual and were provided access to the app and the orientation tutorial after the 4-week period. All study participants received one-year free subscription to the Am Mindfulness app as compensation for their participation.

Outcome measures

The study originally intended to use the cost of treatment for all participants as a primary outcome measure. This was to be determined through an analysis of patient records to calculate the length of stay and the cost of clinical services provided to patients during their stay. This was not calculated, as meaningful analysis would not have been possible due to insufficient recruitment. The other outcome measures used included a series of psychometric instruments to assess different psychosocial constructs: the Acceptance and Action Questionnaire-II (for psychological flexibility) [20], the PROMIS-57 Profile (for health-related quality of life) and the NIH Toolbox Perceived Stress Fixed Form [21]. Additionally, referring clinicians were asked to fill out a single-question survey before and after the intervention to measure clinician-reported quality of patient appointment time. This data was also excluded as the clinicians in the patients’ circle of care changed over the course of their time in the study, particularly when inpatients moved to outpatient care after discharge.

Description of sample

Recruitment was conducted over six months (April–September 2019). During this period, Clinical Champions referred 19 patients, of whom two did not meet the inclusion criteria and six either did not respond, or indicated that they were not interested in participating. The other 11 participants were recruited and included six inpatients and five outpatients, with a mean age of 62 years (SD = 8.85).

Seven participants were randomized into the experimental group, out of whom two registered and used the app throughout the study period and completed the follow-up assessments. The remaining participants in the experimental group either did not register the app or use it frequently enough, despite repeated attempts to follow up, and therefore could not be post-tested. Four participants were randomized into the control group, out of whom three completed the follow-up assessments. The other participant in the control group could not be post-tested as she did not respond to emails/phone calls. Therefore, only five participants (two in the experimental and three in the control group) completed the study. Table 1 outlines the characteristics of all participants enrolled in the study.

Table 1

Participant characteristics.

Participant No.	Gender	Age	Admission status at enrollment	Group	App usage
1	Female	55	Inpatient	Experimental	Not registered/used
2	Female	68	Inpatient	Control	Not registered/useda
3	Male	65	Inpatient	Experimental	Not registered/used
4	Female	61	Inpatient	Experimental	Used regularly
5	Female	49	Inpatient	Experimental	Not registered/used
6	Female	66	Outpatient	Control	Not registered/useda
7	Female	69	Outpatient	Experimental	Not registered/used
8	Female	68	Inpatient	Experimental	Not registered/used
9	Female	59	Outpatient	Control	Used regularlya
10	Male	45	Outpatient	Experimental	Used regularly
11	Male	73	Outpatient	Control	Used regularlya

Refers to frequency of usage over one month after the study period.

Discussion

Despite sustained efforts of the research team, we could not reach our goal of 82 participants and were unable to conduct a conclusive analysis of the effectiveness of this intervention. Throughout, and following study completion, the research team met to discuss and attempt to remediate potential pitfalls in the study design and implementation. The Promoting Action on Research Implementation in Health Services (PARIHS) framework, with its dimensions of: 1) evidence, 2) context, and 3) facilitation was used to conceptualize these failures post-hoc [25]. Briefly, the PARIHS framework proposes that for the implementation of a novel treatment strategy to be effective, there needs to be: 1) a strong evidence base that is applicable to the proposed patient population and their context; 2) a good understanding of the relevance and fit of the treatment strategy for the wider organizational context where it will be implemented; 3) appropriate facilitation of the treatment strategy (for a more detailed description of the PARHIS framework, see. Kitson and colleagues [25]).

Using the PARHIS framework, our implementation failure, including challenges to recruitment identified, remedial actions attempted, and lessons learned are discussed here. Our reflections may offer insights into factors contributing to recruitment difficulties in studies aiming to better understand the healthcare needs of patients with complex care needs living with multimorbidity. We also outline some recommendations for much needed future studies in this domain.

Evidence

Research evidence

The present study was designed based on previous RCTs investigating the use of the Am app (previously Wildflowers) on other populations [19]. Although the app was adapted to suit the needs of the multimorbid older adults with complex care needs, it had not been previously tested with this population. It would be valuable for future studies to consider conducting a preliminary feasibility study to gain a better understanding of the target population prior to initiating an RCT. An added qualitative component, such as in-depth interviews with a small number of participants using the app, would also be useful in gaining an understanding of user perspectives and experiences with the app, as well as their needs.

Patient experience: stage of care and readiness for change

Study participants’ stage of care may also have influenced recruitment. The project initially aimed to study the use of the Am Mindfulness app among inpatients undergoing rehabilitation and was later expanded to include outpatients in an effort to improve recruitment. However, it was observed that participants who registered and used the app regularly and completed the follow-up assessments were mostly outpatients, whereas those who either did not register the app or did not use it regularly after registering were mostly inpatients. Inpatients who were recruited appeared preoccupied with ongoing health concerns and were frequently occupied in rehabilitation sessions. As a result, they may not have been in the frame of mind to focus on this type of intervention. One inpatient reportedly did not have access to an appropriate device to use the app at the hospital. Most of the recruited inpatients were discharged prior to completing the study intervention period and the research team experienced difficulties in following up with them after discharge, particularly if they were not scheduled for outpatient services. It was relatively easier to follow up with outpatients, who visited the hospital regularly for appointments, through assistance from their referring or treating clinician. Outpatients also seemed more responsive to follow-up emails and phone calls and appeared to be interested in using the app.

These observations, considered in conjunction with research showing the importance of patients' readiness for change in predicting a range of treatment outcomes [[26], [27], [28], [29]], suggest that introducing the intervention at a different stage (either among healthy older adults before they require medical care/rehabilitation or older adults undergoing outpatient rehabilitation) may have improved participant recruitment and retention. Other studies exploring the use of technology with a geriatric population have reported similar observations [30]. These findings emphasize the need for future studies to carefully consider the location of patients in their trajectory of care or recovery and to time the administration of their intervention accordingly. For example, this could be done through assessment of patients’ readiness for change in a pilot study.

Although this was not measured, it is likely that the study may have increased clinician and patient awareness of the benefits of mindfulness-based approaches as a therapeutic modality. It may have been useful to include an outcome measure to assess clinician and patient attitudes to mindfulness before and after the study for a more definitive understanding and as a possible proxy for assessing readiness for change. Introducing mindfulness-based interventions or resources in the inpatient setting may serve to plant a seed and flag patients who may potentially benefit from these interventions through outpatient care after their return to community living.

Context

Outer context: increasingly complex care needs of the study population

Despite the research team's efforts to identify and implement effective recruitment strategies, the total number of referrals for eligible participants received during the six-month study period was considerably lower than expected. The target population for this study included older adults undergoing rehabilitation, with chronic conditions and complex healthcare needs. A growing number of these patients are commonly designated as alternate-level-of-care (ALC) in Ontario, indicating that they no longer require hospital care but are experiencing delayed discharge as they are unable to access care at home, long-term care or assisted living [22,23]. The Ontario Hospital Association (2019) reports that as of September 2019, the number of ALC patients was at a record high of 5372 [24]. The study setting, like other hospitals across Ontario, had also witnessed a rise in the number of ALC patients coinciding with the data collection period. Many of these patients would not have met the eligibility criteria for this study by virtue of the greater likelihood of cognitive impairments among them, possibly contributing to the reduced number of referrals.

Inner context: A Recent merger

Between the time that the study was proposed and when data collection began, the hospital underwent a merger. Research has shown that staff may experience occupational uncertainty during mergers [31]; this may have impacted staff morale and engagement, which in turn may have reduced clinicians’ capacity to engage with the research study.

Facilitation

Integration of the research team into the organization and incentivization of stakeholder buy-in

Another factor worth considering with respect to the low referral rate is the recruitment strategy employed, which greatly relied on clinician referrals as none of the research team members were involved in direct clinical service provision. Although the research team spent a lot of time increasing awareness about the study among clinicians through frequent meetings and reminders, it is possible that clinicians may have had other ongoing responsibilities and commitments that could have influenced the number of referrals received. The use of the consent to contact form to obtain consent for the research team to contact the patient also often delayed recruitment. Alternative recruitment strategies with more direct involvement of the research team could be explored in future studies. For example, it might be valuable to consider engaging interested clinicians as direct members of the research team, as these clinicians are already engrained in the organization and have an established network. Furthermore, given the low referral rate, the use of qualitative interviews to explore clinicians’ attitudes towards and experiences of identifying and referring patients may have been worthwhile. Indeed, considering the limited research available on patients living with multimorbidity, future researchers could incorporate this qualitative component as a form of methodological auditing to help to develop standards for investigators interested in working with this under researched population.

Conclusion

To conclude, this study taught us much about the need for developing tools and strategies for conducting research on patients with complex care needs. Although we initially thought a cost-effective, mobile-delivered, intervention would be optimal for patients with limited resources, undergoing a highly stressful period in their lives, due to difficulties with recruitment and engagement, we were forced to reconsider our approach. This led us to think deeply about, and outline the reasons for failure associated with our study; this research has proved invaluable for us as we consider the design of future studies at the hospital, and we hope that it will also benefit researchers interested in conducting research in the area of multimorbidity.

Clinical trial registry name

ClinicalTrials.gov PRS.

URL

https://clinicaltrials.gov/

Registration number

NCT03908918.

Funding

This study was funded through the SPARK Program at the Centre for Aging and Brain Health Innovation (CABHI). Funding for this program is provided by the through the Ministry of Research Innovation and Science, as well as the through the and the Baycrest Foundation. The funding source had no involvement in study design, collection, analysis and interpretation of data, writing the manuscript or in the decision to submit the manuscript for publication.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.