Promoting Problem Solving About Health Management: A Mixed-Methods Pilot Evaluation of a Digital Health Intervention for Older Adults With Pre-Frailty and Frailty.

Background: There is a need for interventions to promote health management of older adults with pre-frailty and frailty. Technology poses promising solutions, but questions exist about effective delivery.
Objectives: We present the results of a mixed-methods pilot evaluation of Virtual Online Communities for Older Adults (VOCALE), an 8-week intervention conducted in the northwestern United States, in which participants shared health-related experiences and applied problem solving skills in a Facebook group.
Methods: We performed a mixed-methods process evaluation, integrating quantitative and qualitative data, to characterize the intervention and its effects. We focus on four areas: health-related measures (health literacy and self-efficacy), participation, problem solving skills enacted, and subjective feedback.
Results: Eight older adults with pre-frailty and frailty (age = 82.7 ± 6.6 years) completed the study. There was an upward trend in health literacy and health self-efficacy post-intervention. Participants posted at least two times per week. Content analysis of 210 posts showed participants were able to apply the problem solving skills taught, and exit interviews showed participants' increased awareness of the need to manage health, and enjoyment in learning about others.
Conclusion: This mixed-methods evaluation provides insight into feasibility and design considerations for online interventions to promote health management among vulnerable older adults.

Introduction

Frailty is a potentially modifiable geriatric syndrome in which older adults experience symptoms including weight loss, fatigue, weakness, slowness, and inactivity (Clegg et al., 2013; Fried et al., 2001). Frailty shares similarities with common long-term conditions in that it can be ameliorated but not cured, is costly, and is progressive (Harrison et al., 2015). Of people older than 85 years, 25% to 50% are estimated to be pre-frail and frail (Collard et al., 2012). These persons are susceptible to functional loss and reduced quality of life, independent of other risk factors (Clegg et al., 2013; Zaslavsky et al., 2016). Moreover, there is a relationship between frailty, poor self-management, and poor health (Cramm et al., 2014). Historically, primary care responses to frailty were mostly reactive to acute events such as falls; however, of late, calls for more proactive, integrative, person-centered and community-based responses have emerged (Turner & Clegg, 2014). In this study, we embrace the health promotion approach and present an intervention intended to target older adults’ self-management practices by improving their problem-solving skills.

Interventions that address self-management and support older adults can be costly when delivered in a traditional in-person format; technology is increasingly considered as a vehicle for expanding information access. Pew reports have shown that older adults increasingly adopt newer technologies, with 42% of adults ages 65 and older reporting smartphone ownership, and 67% stating internet usage (Anderson & Perrin, 2017). Older adults are also increasingly using social media such as Facebook and Twitter (Tennant et al., 2015).

Older adults can have difficulty with interfaces due to aging-related changes such as movement control, perception, and cognition (Lee & Coughlin, 2015; Lee et al., 2013). However, despite potential barriers to adoption, most older adults harbor positive views of technology and are willing to engage with it (Betts et al., 2019; Mitzner et al., 2010). Moreover, use of online interventions among older adults can be associated with increased social activity, decreased loneliness, increased perceived social support, improved self-competence, and enhanced wellbeing (Czaja et al., 2018; Morton et al., 2018).

Although online support group interventions have been successfully used in several geriatric and non-geriatric populations, such as patients with breast and prostate cancer (Ihrig et al., 2020; McCaughan et al., 2017) and patients with dementia and their caregivers (Howe et al., 2020), to our knowledge, online peer-to-peer support interventions have yet to be tested in the context of frailty. As people with frailty have distinct informational, social and health-management needs, they might be uniquely poised to benefit from accessing relevant health information and from interacting with others with similar health issues through online group interventions. We present results from a pilot evaluation of Virtual Online Communities for Aging-Life Experiences (VOCALE), an online discussion group to facilitate problem solving and adoption of health management strategies among older adults with frailty.

VOCALE incorporates elements of problem-solving therapy (PST), a cognitive-behavioral intervention focused on adaptive problem-solving attitudes and skills (Nezu et al., 2010). PST has been employed with different populations including adolescents (Eskin et al., 2008), family caregivers (Washington et al., 2018), and older adults (Kiosses & Alexopoulos, 2014); and with different conditions including depression (Kirkham et al., 2016), diabetes (Fitzpatrick et al., 2013), and frailty (Chan et al., 2012, 2017).

Internet-based PST is limited, though one form of PST, self-examination therapy, has been incorporated into an Internet-based format (Kleiboer et al., 2015; Warmerdam et al., 2010). Online versions of PST often involve one-on-one interactions with a coach through multiple modalities including email and videoconference (Kleiboer et al., 2015; Wade et al., 2015). In this study, we employed a novel online, group version of PST based on the ADAPT (Attitude, Define, Alternatives, Predict, and Try out) framework (D’Zurilla & Nezu, 2006). This version differs from other forms of online PST in its group delivery; participants work through didactic content in an online discussion forum. We performed a mixed-methods process evaluation, including both quantitative and qualitative data, to examine the ways in which the intervention affected participants. We focused on four aspects: (1) health-related measures; (2) participation; (3) problem solving strategies; and (4) participant feedback.

Methods

Design

VOCALE is a moderated online discussion group designed to facilitate health management and problem solving. The intervention has been developed iteratively over multiple rounds, facilitating refinement on participant feedback. The first-round pilot testing results have been reported in (Teng et al., 2019). The second-round pilot testing results are reported herein. All study procedures were approved by the University of Washington Institutional Review Board.

Intervention

The VOCALE intervention is a moderated Facebook online discussion group comprised of two main sections: exploring health issues (3 weeks) and PST (5 weeks). Facebook was chosen due to its high penetrance among older adults.

Exploring health issues

In our first-round pilot study, participants were encouraged to share aging-related experiences and management strategies with other participants by engaging in weekly discussions on health-related topics, including sleep, physical activity, fatigue, and slowness, selected from extant frailty literature, over the course of 10 weeks. We selected the most common topics from that pilot study—sleep, physical activity, and pain—to use in the first 3 weeks of this second-round pilot study. Each week, we presented a discussion prompt on the featured topic (Figure 1a), and participants were asked to respond below the discussion prompt.

Figure 1.

Sample weekly discussion prompt: (a) week 3 Pain and (b) week 6 Define the Problem.

Problem solving therapy

At the end of the third week, participants completed the Social Problem Solving Inventory—Revised (hereafter referred to as the “Problem Solving Inventory,” D’Zurilla et al., 2002), a 25-item scale instrument comprised of Likert-type items and five dimensions: positive attitude, rationale problem-solving skills, negative attitude, impulsivity/carelessness, and avoidance. In this study, we administered the Problem Solving Inventory prior to presenting didactic content on problem solving skills to help participants understand how they tend to approach problems. Rather than being used as a diagnostic measure, in this study the Problem Solving Inventory was used to stimulate reflection. In the fourth week, participants were asked to share their thoughts on the Problem Solving Inventory results.

In the four remaining weeks, we asked participants to practice each skill from the ADAPT framework. Each week, participants worked on one skill from the framework (Table 1). “Predict” and “Try out” were combined into 1 week to keep the intervention a suitable length.

Table 1.

Problem Solving Skills Based on the ADAPT Framework.*

Code	Definition and example
Attitude	Definition: Individuals adopt a positive, optimistic attitude toward the problem and their problem-solving ability.
	Example: To me using the cane is a badge of courage, a symbol that you have conquered your fear of embarrassment or shame and have understood and accepted why you need its assistance. What fun to select that fashionable assistant! [ID 203]
Define	Definition: Individuals define the problem by obtaining relevant facts, identifying obstacles, and specifying realistic goals.
	Example: Sally’s problem is the pain she suffers. She has not yet found help to relieve the pain either by finding remedy of its source or a means of stopping the pain. [ID 218]
Alternatives	Definition: Individuals generate a variety of alternatives to overcome identified obstacles and achieve goals.
	Example: Perhaps Sally could find a Financial advisor who specializes in senior finances. [ID 211]
Predict	Definition: Individuals predict the positive and negative consequences of each alternative and choose the one(s) that has/have the highest chance of achieving the goals, while minimizing costs and maximizing benefits.
	Example: The personal effects for Sally could give her a sense of accomplishment and those around her will be more satisfied, as well. [ID 211]
Try Out	Definition: Individuals try out the solution and monitor its effects.There is no example for this skill because it was not covered as a separate skill in the intervention.

Definitions adapted from D’Zurilla and Nezu (2010).

We introduced a persona named Sally Walker, whose characteristics were developed from symptoms and health management challenges that participants shared in the first-round pilot study. The idea for employing a fictional character was based on the concept of personas, which are prototypical representations of target users used to help ground design work (Pruitt & Adlin, 2010). The sample from the first and second rounds were similar in terms of baseline characteristics (e.g., age, gender, race/ethnicity, education level, familiarity with computers).

Participants were asked to apply the skills that they learned to assist Sally. For instance, to facilitate the practice of the target skill of “defining problems,” in the Define week, we used the following prompts to guide their discussion of the persona’s story: What are the problems? Why is solving these problems important for Sally? What are her potential obstacles to solving the problems? The moderator would then assess whether the participants employed the target skill of defining problems by reviewing their comments to the prompts. If a participant skipped this “defining problem” step while jumping ahead and offering solutions directly, the moderator would gently ask the participants to clarify what the problems were. A screenshot of the description of Sally and the prompt for Week 6, Define the Problem, are presented in Figure 1b. All of the materials used in the study are available upon request.

Participants were expected to post at least twice a week, and they received compensation for their participation. If a participant did not respond within the first 4 days of publication of the week’s discussion prompt, the moderator would send an email, reminding the participant that his or her feedback and perspectives were valuable. Participants were able to respond to any comments under existing discussion posts. However, if a participant created a new post that was not a response to an existing post, the moderator would need to approve the post before it would be displayed.

Discussion was facilitated by a moderator who was a member of the research team (predoctoral graduate student). The moderator maintained a neutral stance, endeavored to make participants feel acknowledged, and encouraged them to contribute to the discussion, both through responding to the posted topic, as well as by interacting with one another. The moderator also answered questions and provided clarification as needed, but did not provide any advice. The moderator and the rest of the study team also monitored the discussion regularly (several times a day). The study team was instructed to watch for circumstances in which intervention might be necessary (e.g., conflicts and/or potential participant discomfort due to the nature of conversation), but ultimately none was needed.

Participants

We employed multiple recruitment methods, including recruiting through retirement communities, public libraries, a university study recruitment website, electronic flyers, and print flyers in retirement communities and community centers, to recruit a target sample of 8 to 12 older adults who experience pre-frailty or frailty (defined later), from an urban area in the northwestern United States. The sample size was kept at approximately 10 to facilitate a more intimate discussion.

Those who learned of our study through print or electronic flyers were provided a project description and were asked to contact the study team if interested. Those who lived in retirement communities were invited to attend a brief in-person presentation and Q&A session in which attendees were informed of the study goals, procedure, and structure. The study inclusion criteria were: individuals were at least 65 years of age, had Internet access, and met at least one criterion from the short Women’s Health Initiative (sWHI) frailty measure (Zaslavsky et al., 2017). The sWHI is a frailty measure employing the following self-report criteria: physical activity, fatigue, weight loss, and physical function. People meeting at least one criterion are judged to be pre-frail and those meeting two or three are judged to be frail. The sWHI measure has been compared with Fried’s CHS frailty phenotype measure for prediction of health outcomes; the measure performed well and was described as “practical for use in settings with limited resources” (Zaslavsky et al., 2017). Despite its nomenclature, the sWHI frailty measure has also been used in prior research involving samples with both men and women (Strandberg et al., 2018; Teng et al., 2019).

Data Collection

Prospective participants were screened through an electronic survey. Research team members followed up with eligible participants to schedule in-person data collection and training sessions where participants were introduced to the study, signed the consent form, completed baseline questionnaires, and received one-on-one Facebook training sessions to ensure that they were capable of posting in the group. Training sessions also consisted of informing participants group rules and online conduct. Participants were also encouraged not to disclose any information that they did not wish others to know. Training materials were available for download from the Facebook group page. At the end of the study, we administered questionnaires and conducted a semi-structured interview asking participants about their experiences.

Measures

At study enrollment, participants completed a demographics questionnaire and a 4-m gait speed test to assess mobility limitations. Comfort level with computers was assessed with a single Likert-type item with the following levels: very uncomfortable, somewhat uncomfortable, neutral, somewhat comfortable, very comfortable. Gait speed data were collected following a standardized protocol (Karpman et al., 2014). At baseline and the end of the study, we measured health literacy using the abbreviated version of the Test of Functional Health Literacy in Adults (S-TOFHLA), a widely used measure of health literacy with good reliability and validity (Baker et al., 1999). As lower health literacy might be seen as a risk factor for frailty (Huang et al., 2018), health literacy was a variable of interest in this study. To measure chronic disease management self-efficacy, we employed the 6-item Self-Efficacy for Managing Chronic Disease scale (SEMCD, Lorig et al., 2001), a widely used measure of self-efficacy that has been demonstrated to have high internal consistency reliability (Ritter & Lorig, 2014). As the SEMCD is specific to chronic disease, we also employed a 5-item general measure of health self-efficacy (Lee et al., 2008).

We also employed a questionnaire on participants’ health information behavior, and health management strategies based on extant literature (Chen, 2012; Edwards et al., 2012). At the end of the study, we collected data on users’ perceptions of their experience using the User Engagement Scale (O’Brien & Toms, 2013). As the last two measures were used to provide additional context on participants’ routines and user experience and out of the scope of the current evaluation, they will not be discussed further in this article.

Data Analysis

In our evaluation, we focused on four aspects, to provide a holistic assessment of how participants responded to the intervention. First, we presented descriptive statistics to characterize the sample, and compared changes in two health-related measures, health literacy and self-efficacy, before and after the 8-week intervention. We did not perform pairwise inferential tests due to the small sample size. We also report the number of posts that participants made.

The remaining three parts of the process evaluation (participation, enactment of problem solving skills, and participants’ subjective perceptions of the study) involved qualitative analysis of two data types: the discussion content and exit interviews conducted with participants. First, we describe our data preparation procedures, and then we will proceed to describe the analysis procedure. Discussion board content was manually extracted from the Facebook group at the end of the study. The exit interviews were recorded and fully transcribed. The average interview time was 22:51 (SD = 6:45). All content was imported and analyzed using the qualitative data analysis software ATLAS.ti (v8) (ATLAS.ti, 2020).

We analyzed the online discussion content using quantitative content analysis, which has been defined as the “systematic and replicable examination of symbols of communication, which have been assigned numeric values according to valid measurement rules” (Riffe et al., 2014, p. 19). Communications are coded and frequencies can be used for comparisons and/or statistical analyses (Strijbos et al., 2006).

In this study, we employed quantitative content analysis to characterize how and to what extent participants engaged with the study through discursive acts, as well as the extent to which they enacted the problem solving skills from the ADAPT framework. There is extant literature employing content analysis to characterize discursive acts in online support group discussions and online therapeutic communications (e.g., Chen, 2014; Holländare et al., 2016; Marra et al., 2004); this literature informed the development of our coding scheme. Examples of discursive acts include “Advice or Suggestion” and “Sharing.” A full list of all discursive acts, along with definitions and examples, appear in the Results section. The unit of analysis was the post; a single post could be judged as exhibiting more than one discursive act.

We also examined the extent to which participants learned the skills that are part of the ADAPT framework (attitude, define, alternatives, predict, and try out). Each post from the last 4 weeks, in which the skills are intended to be enacted, was classified by the coding scheme in Table 1. It is possible that a post exhibited more than one skill.

Last, we performed qualitative analysis of the exit interviews to characterize participants’ perceptions of the intervention. The transcripts were analyzed using a general inductive analytic method (Thomas, 2006). One coder developed an initial coding scheme based on the literature and the data, and then a second coder independently applied the codes to the data. The coders resolved disagreements by consensus and iteratively refined the coding scheme. As the goal of this qualitative evaluation was to examine whether participants were engaged in the intervention and felt that the intervention provided worthwhile content, we report three major themes from the exit interviews: satisfaction and participation, contributions to learning, and general feedback.

Results

Sample

We enrolled 13 older adults in the study. Three dropped out before the intervention began due to a prolonged recruitment, from November 2018 to early April 2019, and two left during the study due to unexpected health issues unrelated to the study. This resulted in an overall attrition of 39%, and attrition of 20% during the study. We provide sample characteristics for the eight participants that completed the study (Table 2). The mean age was 82.7 (SD = 6.6) years. The sample was primarily female (87%, n = 7), White (87.5%, n = 7), and had a high level of education (62.5% had a baccalaureate degree or above). There was a distribution of income levels, and all were somewhat or very familiar with computers.

Table 2.

Sample Characteristics (n = 8).

Characteristic	Statistic	n
Age, M/SD/range	82.7/6.6/69~92	8
Weight (lb), M/SD/range	167.7/29.6/132~209	7
Height, range	4′11"–5′7"	8
Gait speed (m/s), M/SD/range	0.99/0.21/0.65~1.2	8
	%	n
Sex
Male	12	1
Female	88	7
Race
White	87.5	7
Other	12.5	1
Education
High school diploma or G.E.D.	25	2
Vocational or Associate degree	12.5	1
Baccalaureate degree	12.5	1
Master’s degree	37.5	3
Doctoral degree (PhD, MD, JD, etc.)	12.5	1
Income
$20,000–$39,999	37.5	3
$40,000–$59,999	25	2
$80,000–$99,999	25	2
Don’t know/prefer not to answer	12.5	1
Comfort level with computers
Somewhat comfortable	37.5	3
Very comfortable	62.5	5
Facebook user status at baseline
Yes	87.5	7
No	12.5	1
Frailty classification
Pre-frail (1)	37.5	3
Frail (2+)	62.5	5

Health-Related Measures

We examined changes in health-related measures. Average baseline health literacy was 32.4 (SD = 6.2). There was a positive change in health literacy as measured through S-TOFHLA (M = 2.9, SD = 6.3). We also examined two types of self-efficacy, general health self-efficacy, and chronic disease self-efficacy. There was a positive change in the former (M = 1.6, SD = 2.7) but not the latter (M = 0, SD = 10.2) (Table 3).

Table 3.

Baseline, Post-Study, and Change in Health Literacy and Self-Efficacy.

	Baseline (M/SD/range, n)	Post-study (M/SD/range, n)	Change (M/SD/range, n)
S-TOFHLA	32.4/6.2/18~36 (n = 8)	35.3/0.9/34~36 (n = 8)	2.9/6.3/−1~17 (n = 8)
General health self-efficacy	19.8/3.4/13~24 (n = 8)	21.4/2.8/16~25 (n = 8)	1.6/2.7/−2~6 (n = 8)
Chronic disease management self-efficacy	44.7/12.2/29~58 (n = 7)	42.8/8.3/34~53 (n = 6)	0/10.2/−18~10 (n = 6)

Participation

Over the 8-week intervention, study participants authored 210 discussion posts. One did not participate in the discussions due to a hospitalization at the start of the study. After hospital discharge the participant did not express interest to drop out, but also did not respond to our reminders to participate. Hence this person was still considered a study participant and included in the analyses. The mean posts per participant (including this person) was 2.88 (SD = 0.70) each week (Figure 2). Participants were particularly engaged in Week 4, when we presented them with the Problem Solving Inventory results and asked them to reflect upon whether the results helped them to understand their own problem-solving orientations.

Figure 2.

Participation by week.

We performed a qualitative analysis of discursive acts to characterize the ways in which participants interacted with one another. The most common code was “Sharing,” followed by “Opinion or Discussion,” illustrating that participants engaged extensively with the weekly topics by presenting their own views and experiences (Table 4). There was also interaction in terms of participants asking and answering questions, affirming others’ contributions, giving advice, and agreeing with others.

Table 4.

Discursive Acts by Participants.

Code Name	Definition and example	Number
Advice	Poster is providing advice.	31
	Example: We don’t have a bathtub anymore, so I put my legs on a heated cherry pit bag. [ID 211]
Affirmation	Affirming a statement made by, or complimenting, the previous poster.	34
	Example: I am especially impressed with the comments of my colleagues herein in regard to the evaluations of the last survey. [ID 207]
Agreement	Agreeing with the previous poster.	27
	Example: I agree with [ID 207] that empathy is important in our lives and in our own happiness as well.
Answering	Providing an answer to a previously asked question.	41
	Example: well, personally, I find pain to be deterrent to clear thinking as well as mobility. . . [ID 218]
Asking	Asking a question.	40
	Example: how DO you “empty your mind”? I’ve tried meditation, but that doesn’t work for me either! [ID 217]
Clarification	Providing clarification. This discursive act usually occurred in the context of the moderator providing clarification on the weekly topic or other aspects of the intervention and/or discussion platform.	0
Empathy	Poster expresses sympathy or understanding for the person’s situation (e.g., they may say that they have similar issues).	5
	Example: sounds like the same problems I have. . . [ID 217]
Helpful	Poster comments that a previous contribution was useful/helpful.	2
	Example: I just now read the chapter suggested by [moderator name] and have a better understanding of the connection between problem-solving coping and stressors in everyday living and our general health. It was most helpful. [ID 216]
Interesting	Poster comments that intervention content is interesting.	7
	Example: It was an interesting questionnaire. [ID 203]
Negative feedback	Poster provides negative feedback.	4
	Example: Question one. “What did I get?” I have a positive attitude. . . Then the next column headed Rationale said same thing. I question why 2 columns are exactly the same. How is that helpful for the person reading this. [ID 215]
Opinion (opinion or discussion)	Poster offers their opinion or continues the discussion.	50
	Example: The job of a care giver is one of the most emotionally draining of all duties. [ID 207]
Sharing	Poster shares their experiences, an information source, or something else related to the discussion.	110
	Example: I consider myself fortunate as I sleep soundly through the night—except for one or two obligatory bathroom trips. . . [ID 218]
Surprise	Poster expresses surprise.	1
	Example: On one analysis of my responses I was very surprised that there was a negativity in my reasoning. I am famously known for being devoid of being negative and therefore was curious about what my responses triggered in that result. [ID 207]

Problem Solving Therapy

To evaluate the PST component of the intervention, we performed qualitative analysis of problem-solving skills enacted. We employed a stacked bar chart to depict the problem solving skills exhibited (Figure 3). In successive weeks, participants were instructed to focus on the persona’s Attitude (green), Define the problem (yellow), focus on Alternatives (blue), and then Predict consequences and Try out solutions (dark orange). In Figure 3, each color family represents one skill. Example quotations for each skill type are in Table 1.

Figure 3.

Frequency of problem solving skills exhibited by week.

Participants generally followed the prescribed week’s activity, but there was a tendency for participants to provide solutions to problems in all weeks. Participants had difficulties following the instructions for the Week 6 activity, “Define the Problem.” As participants tended to exercise other skills while presenting alternatives, quotes which exhibited a different skill, in addition to alternatives, have been depicted in shades of blue.

Participant Feedback

We performed qualitative analysis of the exit interviews of the participants who completed the study and participated in the online discussions (n = 7).

Participation

Participants spent varying amounts of time on the intervention. Many participants noted their participation took ten minutes or less per session, though one participant spent up to 30 minutes per session and two said it depended on how much there was to comment on.

Motivators of participation included: feeling encouraged by the moderator, curiosity, and enjoyment from interacting with others. Participants also observed that they and/or others seemed to engage more over time, and many commented that this might be due to increasing comfort with other participants over time. Participants mentioned different aspects of the intervention that facilitated participation: the in-person training, the specificity of the prompts, and dealing with the persona as opposed to one’s own problems. Obstacles to participation included not remembering to participate and the feeling that it was the same people participating all of the time.

Contributions to learning

We asked participants to reflect upon whether and how the VOCALE intervention contributed to their understanding of the body and problem solving skills. With regard to the former, some participants felt that participating in VOCALE gave them insight into other people’s problems. They also expressed a greater appreciation for the importance of exercise and awareness of the need to be proactive in health management.

With regard to problem solving, participants mentioned effects they experienced, including that the discussion affected participants’ behavior, either in terms of provoking a more active approach to addressing health issues, or considering the use of the skills in their own lives:

My husband had a Neurology appointment. . . we were trying to focus on how we can improve several things. And I kept thinking about [sic] discussion we’d had and maybe how I could follow through with him at home so that probably made me more, it was more on my radar because we’d just been having that discussion.—ID 211

And what other people are think [sic], which I find very helpful. . . sometimes you do get ideas on how to. . . I do remember somebody did say a couple things about pain management. . . I’ll probably ask my doctor, who I’ll be seeing it a couple of weeks. It did motivate me to make another appointment with my doctor. . .—ID 217

Some participants remarked that they enjoyed talking and discussing the Problem Solving Inventory, and one participant commented that they found the exercise on thinking through consequences helpful: “Because of the way the presentation was worded where we had to link cause and effect and it certainly made you think” (ID 203).

Feedback

Participants provided feedback on the benefits that they saw in participating, as well as areas for improvement. In relation to benefits, they said the experience was interesting, informative, they liked reading others’ contributions, and they enjoyed learning about others. They also liked the specificity of the content, felt the intervention made them think, and found the experience enjoyable. The intervention also helped them to put things in perspective:

I think it was interesting. . . it kinda puts these things in perspective; maybe you are not the only one with some of these issues or thoughts or whatever it is. It was positive as far as I’m concerned.—ID 208

Congruent with the increased comfort and participation noted by participants, one participant observed that her appreciation of the intervention grew over time:

At first. . . I was really not into it. . . I’m thinking that by the time we got to Sally’s experience, there was so much input from the participants that I found interesting, that I started appreciating it more. And they were just practical, like what worked for me. Like, good advice. And it was obvious that it may not work for everybody. It was just what worked for me and I appreciated that.—ID 216

We also learned of areas for improvement. Some participants reported being confused about what they were being asked to do in the beginning, and wanting a clearer explanation upfront:

I think we were confused in the beginning. Not completely understanding what it was you wanted us to do. . . I think more of an upfront explanation would have helped.—ID 203

Some participants also said that they had trouble with the persona:

… this fictional person didn’t respond. You know, you would make suggestions, but we got no feedback from that person. . . You know, it was kind of like talking to a blank wall you were just coming up with these suggestions and didn’t know whether they got there or what the person thought.—ID 217

I would have cared more if it was one of the real people.—ID 218

Despite these issues, participants’ felt there was value to the open-ended part of the intervention in which they shared health management experiences on selected topics, as well as the problem-solving component. When asked what parts of the intervention were most useful, one participant answered:

Well, for sure, it would be that problem solving. . . I was real interested in the responses that people were giving or how to sleep, how to get to sleep or that piqued my interest because as a caregiver, I have to be up and down all night and so, once I’m awake I really find it difficult to go to sleep again. . .—ID 211

Discussion

In this study, we presented a novel method for delivering PST in an online environment to improve health management in people living with frailty over an 8-week intervention. We employed a mixed-methods process evaluation. First, we examined health-related measures. There was an improvement in health literacy and health self-efficacy. As health-related measures on their own do not provide a holistic picture of the effect of the intervention, we employed other measures. Study participants stayed engaged throughout the intervention, posting at least twice a week. In terms of the PST skills, the qualitative analysis showed that participants were able to practice the skills being taught. Participants’ subjective feedback included that they saw various benefits, including finding the intervention interesting, informative, and enjoyable. As qualitative analysis of the online discussion indicated, participants particularly enjoyed sharing experiences and opinions. They also identified areas for intervention improvement, such as a need for clearer explanations at times and difficulties relating to the persona.

The mixed-methods evaluation afforded a more holistic evaluation of the intervention. At least in this limited sample, we were able to see that some health-related measures improved post-intervention, and that participants successfully practiced the skills, and we were further able to contextualize participants’ subjective experiences of the intervention. These, in particular, could inform future improvements to the intervention. For example, participants’ feedback from the exit interviews showed that the sustained interaction among participants over time did result in increased familiarity, deepening of the experience, and a deepened appreciation of both other participants’ contributions and the intervention. We were also able to see that there were weeks in which the didactic content was less successful. For example, in Week 6 we asked participants to “define the problems” that Sally, the persona, experiences. We had many fewer instances of the focal skill being enacted successfully as compared to the other weeks (Figure 3). Moreover, some participants expressed concern the two sections (first 3 weeks of health issue exploration and last 5 weeks of PST) seemed disconnected, and they were surprised when the problem-solving component was introduced. In the future, it might be helpful to provide a roadmap of all the weeks, as well as provide a clearer rationale for the importance of improving problem solving skills.

In this study, participants expressed greater appreciation of physical activity and the need to be more proactive about self-management, which could lead to a healthier lifestyle. Previous research has reported healthy lifestyles have been associated with less decline in physical, psychological, cognitive, and social domains in old age (Visser et al., 2019). The exit interviews indicated that participants enjoyed learning about others and felt the shared content might be useful in their own lives. This finding illustrates the utility of a group format for the intervention, as opposed to an individual format, such as participants receiving educational content in the form of a booklet or through individual therapy sessions.

There is limited extant research on online communities or social networking interventions involving frail older adults. One prior study involved the use of personal online health communities comprised of online health community for frail older people, their informal caregivers, and their providers (Makai et al., 2014). Predictors of usage included more functional problems and providers, sufficient computer skills, and an interest in playing an active role in care. Though the intended purpose of the community in their study, coordination of care, was different from ours, there may be factors in common that motivate study participation. For example, future work could examine potential associations between functional problems and interest in health management.

A sense of community developed with participants over time, suggesting that a digital health intervention could enhance the sense of connectedness experienced by frail and pre-frail older adults. These findings are consistent with extant work arguing that technology access enhances social connectivity and reduces loneliness among older adults (Czaja et al., 2018). Qualitative studies of frailty have reported that a sense of connectedness and belonging are important to maintain (Ebrahimi et al., 2013; Nicholson et al., 2013); interventions such as this one could contribute to that sense of connectedness. Moreover, as social isolation and loneliness are risk factors for frailty progression, mitigating social isolation is of paramount importance (Gale et al., 2018).

Self-efficacy can play a role in frailty. For example, higher coping self-efficacy has been associated with decreased odds of pre-frailty and frailty (Hladek et al., 2020). Similarly, health literacy has also been associated with pre-frailty and frailty, and higher health literacy is more likely to be observed in non-frail individuals than in more frail counterparts (Huang et al., 2018; Shirooka et al., 2017). As such, the potential effect of the VOCALE intervention on health literacy is of interest. Though our pilot evaluation involved a small sample, we observed positive changes in health literacy and general health self-efficacy, suggesting that there is potential to improve these important psychological measures in this population. The average baseline health literacy in this study was in the upper range of what is considered adequate (22–36) on the S-TOFHLA (Baker et al., 1999), suggesting that participants had a limited room for improvement.

Previous research of technology use among the “oldest old” has argued that adoption can be based on a complex interplay of factors, including social, attitudinal, physical, digital literacy, and usability (Neves et al., 2015). Though we encountered issues with confusion in our first pilot study (Teng et al., 2019), we revised our materials and enhanced our training, and subsequently did not encounter issues in this study. In addition, our training materials feature large text and clear iconography to enhance clarity, and we incorporated a substantive amount of training, which is consistent with design guidelines for older adults (Neves et al., 2015). The mean age of participants in this study was over 80, suggesting that an online group intervention is possible for at least a subset of the “oldest old” (i.e., ages 80+). However, the samples of our pilot studies have had relatively high self-reported comfort with computers; more research would be needed to examine whether this intervention would be appropriate for older adults with less familiarity with computers.

Limitations

This study had various limitations. First, the sample was relatively homogeneous on certain dimensions. For example, participants were primarily female, White, and had a high degree of familiarity with computers. Second, the sample size of this pilot study was kept small to keep the group size small. While this afforded the advantage of facilitating an intimate discussion, group size also limited the sample size, and there is a need for larger scale studies. Other limitations include the use of a single persona. A few participants said they had trouble relating to the persona. Future research could incorporate multiple personas. A third limitation is that participants were not asked to explicitly apply the skills that they learn to their own situations; integration of didactic content to encourage participants to practice the skills in their own lives remains as part of future work.

Conclusion

We presented a pilot study of a novel digital health intervention, VOCALE, and illustrated the potential for technology to assist older adults with frailty to problem solve about health management. A mixed-methods process evaluation showed that there was an improvement in health literacy and health-related self-efficacy, and participants stayed engaged throughout the intervention. Qualitative analysis of discussion and interview data showed that participants were able to apply the problem solving skills taught, had greater appreciation for being proactive about health management, and enjoyed learning about others’ experiences. Though there is a need to further refine the intervention content and evaluate the study on a larger scale, our pilot evaluation suggests an online intervention that enables participants to practice problem solving skills in a group setting can facilitate self-management among older adults with pre-frailty and frailty. This article also provided examples of ways to improve the usability of a technological intervention for older adults with pre-frailty and frailty.