Design, deployment, and usability of a mobile system for cardiovascular health monitoring within the electronic Framingham Heart Study.

Background: The electronic Framingham Heart Study (eFHS) is an ongoing nested study, which includes FHS study participants, examining associations between health data from mobile devices with cardiovascular risk factors and disease. Objective: To describe application (app) design, report user characteristics, and describe usability and survey response rates.
Methods: Eligible FHS participants were consented and offered a smartwatch (Apple Watch), a digital blood pressure (BP) cuff, and the eFHS smartphone app for administering surveys remotely. We assessed usability of the new app using 2 domains (functionality, aesthetics) of the Mobile App Rating Scale (MARS) and assessed survey completion rates at baseline and 3 months.
Results: A total of 196 participants were recruited using the enhanced eFHS app. Of these, 97 (49.5%) completed the MARS instrument. Average age of participants was 53 ± 9 years, 51.5% were women, and 93.8% were white. Eighty-six percent of participants completed at least 1 measure on the baseline survey, and 50% completed the 3-month assessment. Overall subjective score of the app was 4.2 ± 0.7 on a scale from 1 to 5 stars. Of those who shared their health data with others, 46% shared their BP and 7.7% shared their physical activity with a health care provider.
Conclusion: Participants rated the new, enhanced eFHS app positively overall. Mobile app survey completion rates were high, consistent with positive in-app ratings from participants. These mobile data collection modalities offer clinicians new opportunities to engage in conversations about health behaviors.

Introduction

A working group from the National Heart, Lung, and Blood Institute (NHLBI) on Epidemiology and Population Science recommended launching digital cardiovascular disease (CVD) epidemiology and recognized the need to evaluate the usability of, and adherence to, digital and mobile health technologies for digital phenotyping., Although large electronic cohorts and clinical trials, including the Apple Heart Study, have generally engaged younger, healthy volunteers, several large cohort studies, including the National Institutes of Health “All of Us” Program, Google Verily’s Project Baseline, and the Framingham Heart Study (FHS), have launched efforts to engage community-dwelling middle-aged participants with, or at risk for, CVD in mobile or digital health research.3, 4, 5, 6, 7

The main objective of the electronic Framingham Heart Study (eFHS) is to add new mobile and digital measures into the FHS and relate digital markers to risk factors and outcomes by leveraging the FHS’s ongoing, deep CVD risk factor phenotyping and multiple measures in cohorts of middle-aged participants over time. The importance of remote monitoring through mobile technology is further highlighted by the increasing reliance on telemedicine in the era of the coronavirus disease 2019 (COVID-19) pandemic and illustrates the need for more robust research on the process of incorporating and implementing these technologies into longitudinal studies. To accomplish this objective, we developed a custom smartphone application (eFHS app) and began deployment of a digital blood pressure (BP) cuff as well as a smartwatch (Apple Watch; Apple, Inc., Cupertino, CA). Informed by feedback from FHS participants and staff, a new version of the eFHS app (running on the MyDataHelps™ [MDH] app; CareEvolution, Inc., Ann Arbor, MI) was released in January 2019, which added a novel dashboard to facilitate participants’ understanding of their BP and physical activity step data and personalized push notifications to promote adherence to app usage. In addition, we developed usability surveys sent through the app to better understand the user experience.

In this article, we describe the new dashboard design and the methodology for its deployment; report the characteristics of the new eFHS app users; and describe usability survey responses and survey response rates over the course of the first 3 months of study deployment. We also report on the settings in which participants shared their device-collected health information, which can highlight potential new avenues for providers to engage their patients in conversations about health behaviors.

Methods

Study population and setting

The eFHS study recruited from 3 cohorts embedded within the FHS that were originally enrolled from 2002 to 2005: the Third Generation cohort (n = 4095); the Omni 2 cohort (n = 410; multiethnic cohort); and the New Offspring Spouse cohort (n = 103). Protocols for eFHS and the original FHS have been previously detailed. Briefly, eligible study FHS participants were invited to participate in eFHS as part of their routine third research examination at the FHS Research Center to occur from 2016 to 2019. eFHS began enrollment on June 20, 2016, with the smartphone app and a digital BP cuff for weekly home BP monitoring. Inclusion criteria included proficiency in English, ownership of a compatible iOS or Android operating system (beginning October 30, 2017) device, residence in the United States, and willingness to provide permissions for notifications and data sharing with the Research Center. Informed consent for participation was included in the overall consent for FHS’s Examination 3 as well as in a separate electronic document within the eFHS app. These documents are publicly available through the FHS Web site. The eFHS protocol was reviewed and approved by the FHS Executive Committee and the Institutional Review Board at the Boston University Medical Center. Data presented in this study will be available from the Biologic Specimen and Data Repository Information Coordinating Center (BioLINCC) (https://biolincc.nhlbi.nih.gov/home/).

Study smartphone app

The initial version of the study app (eFHS app) was created using Apple’s ResearchKit through a partnership with CareEvolution. This app was designed to serve as a means of delivering all study surveys electronically to participants as well as acting as a channel of communication from the study team through notifications and messages. Participants downloaded the app from the Apple App Store or Google Play Store by following written instructions or receiving direct assistance at the Research Center. After registering for the study and logging in, all participants provided electronic consent. Upon first-time use, participants were prompted to complete a baseline survey with 9 sections that assessed a variety of demographic, psychosocial, and medical characteristics (Supplemental Table 2). Additional surveys were deployed via the app at 3, 6, 9, and 12 months after baseline to assess changes in health status and behavior. This app also included periodic notifications for overdue surveys, reminder messages for complying with study device use (BP cuff and smartwatch to collect heart rate, daily steps, and BP), and notifications of new available surveys.

In January 2019, the eFHS updated its study by switching over to using the MDH app to enhance participant experience by allowing for additional functionalities and further integration for passive transmission of sensor data with digital health devices provided by the study. The enhanced eFHS app included a new dashboard that was developed to return meaningful device data to participants to improve participant engagement and adherence (Figure 1). The eFHS dashboard detailed health data collected by study devices and allowed for goal-oriented customization to motivate users to take concrete measures to improve their health, such as the ability to set daily step goals. Although the MDH app works for both iOS and Android operating systems, the study devices described here were only iOS compatible, and the study dashboard was targeted toward participants who received a BP cuff or smartwatch. Surveys within the app all remained unchanged. Research Center staff deployed the new eFHS MDH app and devices, and they trained participants on the use of all system components during their regularly scheduled FHS Research Center examinations. All eFHS participants after MDH app deployment were enrolled using MDH. Existing eFHS users of the prior app version were migrated to MDH in the first half of 2020.

Figure 1

Screenshots of the new electronic Framingham Heart Study (eFHS) app home screen and dashboard showing blood pressure (American Heart Association guidelines) and activity (daily steps).

An important aspect of the eFHS app is its personalized notification feature to encourage study survey engagement available in both the eFHS app and the MDH app. As participants reach study timepoints when additional surveys were deployed (at months 3, 6, 9, and 12) (Supplemental Table 2), push notifications were used to inform participants of the availability of these surveys as well as completion of any overdue surveys (“Reminder: You have surveys to complete”; or “You have surveys due today. Please open the eFHS app and complete them.”)

Measures

BP and physical activity

In addition to downloading the study app, participants were offered 2 digital health devices at the time of enrollment: the Nokia-Withings digital BP cuff (Model BP-801, Withings, Issy-les-Moulineaux, France), a validated, Food and Drug Administration–approved home BP monitor; and the Apple Watch Series 0 for monitoring physical activity in steps. All study participants were instructed on the use of both devices at the time of enrollment and were provided technical assistance as needed, including syncing the Apple Watch to their personal phones and taking a sample BP measurement while at the Research Center. Participants were asked at study onset to take their BP once per week at approximately the same time every week. Participants were provided with written instructions for BP cuff use and proper home monitoring techniques, and were instructed to contact their providers when they obtained abnormally high BP values. Written instructions regarding use, charging, and maintenance of the Apple Watch were provided to participants, and they were asked to wear the watch daily.

Mobile App Rating Scale

The Mobile App Rating Scale (MARS) is a validated instrument designed to assess the functionality as well as overall quality of mobile apps across several domains, including engagement, functionality, aesthetics, information, and overall subjective quality. All items use a 5-point Likert-like scale, and each domain is designed to be scored separately by averaging each item within each domain. Additionally, there are three subjective quality domain questions. The MARS also includes a supplement in which app-specific questions can be added at the investigators’ discretion if applicable to their research purpose. The scores of all domains can be averaged for an overall quality measure, and all scores range from a minimum of 1 to a maximum of 5, with a higher number indicating greater usability.

The MARS instrument was modified for use in the eFHS study in order to streamline the study survey and focus only on domains relevant to the MDH app, as well as to reduce the burden of technical verbiage to be more suitable for the present study cohort. Specifically, only the functionality and aesthetics domain subscales were assessed among the four standard MARS domains. In addition, we used 1 subjective quality measure question for star ratings, and an app-specific supplement on intention and health behavior was applied for BP and physical activity if the participant received the corresponding digital health device (BP cuff for BP measurement or Apple Watch for physical activity). Finally, 2 investigator-generated questions were appended at the end of the MARS assessment, asking participants whether or not they had shown their BP or step count data to anyone (yes/no); and, if so, who was the individual (family, friends, health care provider, or other). The full instrument can be found in the Supplement.

Survey adherence

Completing a survey was defined as completion of 75% or more of the questions in the survey within a prespecified timeframe. The baseline survey must be completed within 89 days of enrollment, and the 3-month survey must be completed between 90 and 179 days after enrollment. Nine different measures were included in the baseline survey, whereas only 1 instrument was deployed at the 3-month survey. We defined survey adherence at baseline in 2 ways: (1) proportion of participants who completed at least 1 instrument; and (2) proportion of participants who completed all instruments.

Statistical analysis

Descriptive data on participant characteristics, responses to individual MARS elements, and data sharing with others are presented. Additionally, those who completed the MARS instrument were compared to participants who had not completed the MARS from among the MDH users. All continuous variables were compared using the Student t test. Categorical variables were compared using the Pearson χ2 test, and all P values are reported. Two-sided P <.05 was considered statistically significant. All statistical analyses were performed using R Version 4.0 (The R Foundation; https://www.r-project.org/).

Results

Participant demographics

Since January 2019, 196 participants were recruited into the eFHS study as users of the MDH app; of these participants, 188 received the MARS instrument. The average age of this cohort was 53 ± 9 years, more than one-half of the participants were women, and the majority of participants were white. The prevalence of diabetes was approximately 1 in 10 participants, hypertension was seen in about one-third of participants, and the prevalence of any cardiovascular disease was very low (Table 1). Fifty participants in the cohort received a BP cuff, whereas 62 participants accepted an Apple Watch. There were no significant demographic or clinical differences between people who completed the MARS versus those who did not (Supplemental Table 1).

Table 1

Demographic and clinical characteristics of participants who completed MARS (n = 97)

Age (y)	52.7 ± 9.4
Female	50 (52)
White	91 (94)
Body mass index (kg/m2)	29.4 ± 6.8
Systolic blood pressure (mm Hg)	120 ± 14
Diastolic blood pressure (mm Hg)	76 ± 9
Current smoking	5 (5)
Diabetes mellitus	8 (9)
Hypertension	29 (30)
Prevalent cardiovascular disease	2 (2)
Physical activity index	33 [5]
Highest education level achieved
Completed high school or less	12 (12)
Completed some college	19 (20)
Bachelor’s degree	35 (36)
Graduate or professional degree	31 (32)
Married, living as married, living with partner	69 (73)
Self-reported health as excellent	16 (17)
Employed full time	67 (70)
Device used
Apple Watch	62 (64)
Blood pressure cuff	50 (52)

Values are given as n, mean ± SD, n (%), or median [interquartile range].

MARS = Mobile App Rating Scale.

MARS

A total of 97 participants (51.6%) completed the MARS instrument. Participant responses to the individual MARS domains are presented in Figure 2. Participants endorsed high ratings across the different domains, with all mean ± SD values ranging from 4.05 ± 0.77 in visual appeal to 4.66 ± 0.59 in performance on a scale ranging from 1 to 5 (where 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; and 5 = strongly agree). The overall impression of the app was rated to be 4.18 ± 0.74 by study participants.

Figure 2

Participant responses to elements of the Mobile App Rating Scale (Supplemental Table 3). Values are given as mean ± SD.

Participants also rated the MDH platform with respect to a dashboard associated with either the Apple Watch or BP cuff using the app-specific supplement of the MARS instrument. Mean responses are detailed in Figure 3. Values are given on a Likert-like scale ranging from 1 to 4, where 1 = strongly disagree; 2 = disagree; 3 = agree; and 4 = strongly agree. Participant responses are high across all survey elements. Notably, the 2 values with means below 3 include belief that using the app would encourage conversations with a health care provider around physical activity, and the belief that the MDH app would decrease BP. The smartwatch was perceived to be significantly more likely to increase physical activity than the BP cuff was to decrease BP, although use of the BP cuff was perceived to be significantly more likely to encourage conversation with a provider than the smartwatch (P <.05 for both).

Figure 3

Participant responses to health behavior questions about blood pressure (BP) or physical activity (PA). Full questions are given in the Supplement. The scale for these questions ranges from 1 to 4, where 1 = strongly disagree; 2 = disagree; 3 = agree; and 4 = strongly agree (Supplemental Table 4).

Additionally, 26 of the 50 participants (52%) who received a BP cuff responded that they shared their BP data with someone. Of these 26 participants, 12 (46%) showed their health care provider their BP measurements (Table 2). About one-third of participants (20/62) who received the Apple Watch chose to share their physical activity data, but only 2 of these 20 participants (7.7%) shared this information with their health care providers.

Table 2

Sharing of blood pressure and physical activity data

	Blood pressure (n = 50)	Physical activity (n = 62)
With anyone	26 (52.0)	20 (32.3)
With family	20 (40.0)	20 (32.3)
With friends	5 (10.0)	7 (11.3)
With health care provider	12 (24.0)	2 (3.2)

Values are given as n (%).

From the total 196 MDH users, 86% of participants (n = 164) adhered to at least 1 baseline survey, whereas 63% (n = 121) completed all baseline surveys within the allotted response window. Adherence decreased to 50% at the 3-month time period. Additional survey questionnaires planned for 6 and 9 months are not yet deployed at the time of writing of this manuscript.

Discussion

We described the deployment and usability evaluation of a system of mobile health devices, including a smartphone app, a smartwatch, and a digital BP cuff, within an established cohort of FHS participants. The 4 main findings are as follows. (1) Functionality, aesthetics, and overall rating of the app were highly rated by participants. (2) Participants endorsed that use of the system would positively impact their health behaviors. (3) Over 50% of study participants shared their BP with someone, and more than one-third shared their physical activity data. Participants were more likely to engage in conversations with their medical provider regarding their BP as compared to physical activity. (4) Adherence to survey response was high initially but decreased moderately over 3 months.

Given the significant heterogeneity in scope and quality of commercially available health apps, previous work has unsurprisingly found that BP and physical activity monitoring apps encompass a wide range of MARS scores., Interestingly, studies utilizing the MARS for BP and physical activity monitoring apps have almost universally reported functionality as consistently the highest scoring domain, followed by aesthetics,12, 13, 14, 15 a phenomenon we observed as well in the present study. This trend may indicate a specific effort by health app developers in prioritizing these 2 particular domains during the development process, potentially at a detriment to other facets of the app, as studies have demonstrated that there can be significant disparity between the functionality and aesthetic subscales and the other MARS domains., Given that the primary function of the MDH app was data collection, the engagement and information quality domains were not deemed relevant and thus were not assessed in the present study.

In addition to the dashboard to encourage participant engagement, a core functionality of the app includes data collection through a myriad of survey instruments deployed at various timepoints. The ubiquity of personal smart devices has enabled new modalities for questionnaire-based data collection through app-based survey administration, and studies in recent years have aimed to optimize this mode of data collection and to understand its validity compared to traditional methods.16, 17, 18 However, longitudinal adherence to mobile self-administered surveys is relatively underexplored, and the rate of decline in survey adherence differs dramatically depending on the study participants and context of survey deployment.19, 20, 21 We observed a decrease in survey completion from baseline to the 3-month timepoint despite implementing reminder prompts through push notifications; however, this drop is less drastic than seen in other similar e-cohorts., This finding highlights a potential weakness in exclusively mobile-based studies in similar populations. More active efforts to understand participant engagement patterns in future studies may be crucial to retaining a satisfactory response rate.

Health care engagement

Despite the overall high usability scores and general positive health behavior change endorsed by participants, they were less likely to respond that the app dashboard and BP cuff would result in a decrease in their BP compared to the same question about the smartwatch and an increase in their physical activity. However, measurement of BP was reported to encourage conversation with health care providers, which was reflected in the observation that nearly one-half of participants who showed their BP data to anyone did so in a health care context. Interestingly, the reverse pattern was observed with regard to physical activity. Participants expressed more confidence in the smartwatch and dashboard being able to actually increase their physical activity. However, use of the system did not encourage conversation by participants with their providers regarding physical activity, so only a small fraction of participants showed these data to their providers. This is likely because BP is widely perceived as being a more “traditionally medical” vital sign than physical activity and more often is associated with routine procedure in a clinical setting. Conversely, although physical activity monitoring was less likely to spark conversation with a health care provider, it was perceived to be a much more modifiable health metric than BP. This may be so because participants have direct control over physical activity, whereas they may find it more difficult to visualize the less explicit connection between risk factor modification and downstream changes in BP. Additionally, given that average BP in the cohort was in the optimal range, participants may not have perceived a need to alter their BP values.

This observed optimism toward increasing physical activity mediated by digital technologies can potentially be harnessed by health care providers as an opportunity to emphasize the important role exercise plays in overall health. As physical activity has been well established to be inextricably linked to a multitude of important health outcomes, it is crucial for providers to create a clinical environment in which patients not only feel comfortable discussing their physical activity with their health care providers but are actively encouraged to do so. Indeed, evidence suggests that patient–provider communications regarding physical activity may be limited even in populations that would significantly benefit from increased physical activity., The rapidly growing popularity of mobile and digital health devices for physical activity monitoring may prove to be a useful avenue in which providers can begin these important discussions with their patients.

Study strengths and limitations

Our study has numerous strengths. The study sample is derived from one of the most scientifically rigorous and longest standing longitudinal studies focused on cardiovascular health, and it deploys a novel, multifaceted mobile health system in this cohort. Additionally, we used a validated and widely used measure of app quality in assessing study outcomes. However, our study should be considered within the context of several limitations. A large portion of participants were well educated and healthy, which may influence perceptions of the health technologies used in the study and potentially have inflated the measured usability ratings. Additionally, all participants are geographically located in central Massachusetts, and most participants were white and middle-aged, so the generalizability of our results to other geographical areas, races/ethnicities, and ages is uncertain. Furthermore, the in-person enrollment procedures as well as the potentially established longitudinal relationships that FHS participants may have with the study could be a potential source of bias, and our observed adherence and usability metrics may be slightly higher as a result. Finally, the engagement and information subscales of the MARS were not assessed as a part of the study as they were not major foci of the MDH app, and including these questions may have provided further interesting insights into general perceptions of the app. The study design is observational, and the data solicited by the MARS were self-reported, so we cannot exclude misclassification of the responses, and we cannot establish causal associations.

Conclusion

A mobile-based system including a health app, digital BP cuff, and smartwatch activity tracker can be successfully deployed in a healthy adult cohort as a feasible way to monitor cardiovascular health and collect device-based outcomes. These new modalities of data collection highlight a potential opportunity for providers to better engage patients in conversations about health behaviors, especially physical activity.

Funding Sources

The electronic Framingham Heart Study (eFHS) is supported by grants from National Heart, Lung, and Blood Institute (R01HL141434) and the (number 74624). The Framingham Heart Study is supported by contract from NHLBI (PI VSR 75N92019D00031). Eric Ding’s time is supported by grant F30HL149335 from the National Heart, Lung, and Blood Institute. Dr McManus’s time is supported by grants R01HL126911, R01HL137734, R01HL137794, R01HL135219, R01HL136660, U54HL143541, and 1U01HL146382 from the National Heart, Lung and Blood Institute. Dr Benjamin’s time is supported by grants R01HL092577, 2U54HL120163, R01AG066010, 1R01AG066914, and AF AHA_18SFRN34110082.

Disclosures

Dr Schramm is an employee of CareEvolution, Inc. Dr McManus received honorary, speaking/consulting fees, or grants from Flexcon, Rose Consulting, , Pfizer, Boston Biomedical Associates, Samsung, Phillips, Mobile Sense, CareEvolution, Flexcon, Boehringer Ingelheim, Biotronik, Otsuka Pharmaceuticals, and Sanofi; and declares financial support for serving on the Steering Committee for the GUARD-AF study (NCT04126486) and Advisory Committee for the Fitbit Heart Study (NCT04176926). Dr Benjamin was an uncompensated member for MyHeartLab Steering Committee, a PI-initiated study from Samsung to UCSF (PI: Jeffrey Olgin, MD). Dr Murabito was a guest speaker/consultant for Merck Research Laboratories. Apple was not involved in the study design, analysis, interpretation, or reporting of study results. The Apple Watches were provided to Boston University by the Apple, Inc., at no cost to the study.

Ethics Statement

The authors designed the study, and gathered and analyzed the data according to the Helsinki Declaration guidelines on human research. The research protocol used in this study was reviewed and approved by the institutional review board.

Patient Consent

All patients provided written informed consent.

Authorship

All authors attest they meet the current ICMJE criteria for authorship.

Disclaimers

Given his role as Editor-in-Chief, David McManus had no involvement in the peer review of this article and has no access to information regarding its peer review. Full responsibility for the editorial process for this article was delegated to Hamid Ghanbari. Given their roles as Associate Editors, Belinda Borrelli and Chunyu Liu had no involvement in the peer review of this article and have no access to information regarding its peer review.