Use of Mobile Devices to Measure Outcomes in Clinical Research, 2010-2016: A Systematic Literature Review.

BACKGROUND: The use of mobile devices in clinical research has advanced substantially in recent years due to the rapid pace of technology development. With an overall aim of informing the future use of mobile devices in interventional clinical research to measure primary outcomes, we conducted a systematic review of the use of and clinical outcomes measured by mobile devices (mobile outcomes) in observational and interventional clinical research.
METHOD: We conducted a PubMed search using a range of search terms to retrieve peer-reviewed articles on clinical research published between January 2010 and May 2016 in which mobile devices were used to measure study outcomes. We screened each publication for specific inclusion and exclusion criteria. We then identified and qualitatively summarized the use of mobile outcome assessments in clinical research, including the type and design of the study, therapeutic focus, type of mobile device(s) used, and specific mobile outcomes reported.
RESULTS: The search retrieved 2,530 potential articles of interest. After screening, 88 publications remained. Twenty-five percent of the publications (n = 22) described mobile outcomes used in interventional research, and the rest (n = 66) described observational clinical research. Thirteen therapeutic areas were represented. Five categories of mobile devices were identified: (1) inertial sensors, (2) biosensors, (3) pressure sensors and walkways, (4) medication adherence monitors, and (5) location monitors; inertial sensors/accelerometers were most common (reported in 86% of the publications). Among the variety of mobile outcomes, various assessments of physical activity were most common (reported in 74% of the publications). Other mobile outcomes included assessments of sleep, mobility, and pill adherence, as well as biomarkers assessed using a mobile device, including cardiac measures, glucose, gastric reflux, respiratory measures, and intensity of head-related injury.
CONCLUSION: Mobile devices are being widely used in clinical research to assess outcomes, although their use in interventional research to assess therapeutic effectiveness is limited. For mobile devices to be used more frequently in pivotal interventional research - such as trials informing regulatory decision-making - more focus should be placed on: (1) consolidating the evidence supporting the clinical meaningfulness of specific mobile outcomes, and (2) standardizing the use of mobile devices in clinical research to measure specific mobile outcomes (e.g., data capture frequencies, placement of device). To that aim, this manuscript offers a broad overview of the various mobile outcome assessments currently used in observational and interventional research, and categorizes and consolidates this information for researchers interested in using mobile devices to assess outcomes in interventional research.

Introduction

Assessments of clinical outcomes that are meaningful to patients and that can accurately and reliably measure the potential therapeutic effects of an intervention are needed [1]. Advances in mobile devices, such as wearables and other remote sensors, may provide opportunities to develop new, valuable clinical outcome assessments which may help to accelerate the development of new treatments for patients. Mobile devices offer the potential to collect objective data from research participants with greater frequency than conventional data collection methods (e.g., paper diaries/surveys or clinician/staff observations not using mobile technology), as well as the opportunity to collect data outside of structured research settings, during activities of daily living. Outcome assessments that are made using a mobile device (mobile outcomes) include new ways of measuring traditional clinical outcomes and biomarkers [2, 3], as well as completely novel outcomes that would not be possible without the use of a mobile device. The use of mobile devices in clinical research may provide opportunities to assess disease burden and therapeutic effectiveness in ways that are sensitive, reliable, and relevant to patients' daily lives. Mobile devices may also decrease the burden of trial participation among both patients and research staff, and expand access to patients who typically do not have opportunities to participate in research.

The Clinical Trials Transformation Initiative (CTTI) is a public-private partnership co-founded by the US Food and Drug Administration and Duke University whose mission is to develop and drive the adoption of practices that will increase the quality and efficiency of clinical trials. CTTI observed that while the use of mobile devices in clinical research has increased in recent years, given technological advances, the integration of mobile devices into interventional research – specifically, randomized controlled trials (RCTs) – appears to have evolved at a much slower rate. Given the potential of mobile devices to improve clinical outcome assessments, CTTI aims to inform the development of new mobile outcome assessments for use in future clinical research – particularly in pivotal RCTs and trials to inform regulatory decision-making – by systematically describing recent uses of mobile devices in clinical research. Through such a review, we hope to describe the current state of the field and indicate where efforts to develop and include mobile outcome assessments for use in clinical trials have been concentrated to date. To the best of our knowledge, there has been no other effort to systematically consolidate the available peer-reviewed literature reporting the use of mobile outcome assessments in clinical research across various therapeutic areas.

Methods

We conducted a systematic search of peer-reviewed literature indexed in PubMed and published between January 2010 and May 2016. For the purpose of this review, we chose not to limit the scope of our search to any single therapeutic area or study design, assuming that all study designs (observational or interventional) could inform our aim. The search terms and inclusion and exclusion criteria used for identifying publications were developed in collaboration with a medical librarian and a multidisciplinary research team, including representatives from the US Food and Drug Administration, academia, the pharmaceutical industry, patient advocacy organizations, and mobile device experts [4]. Appendix 1 provides a com plete list of the search terms.

Publications were selected for inclusion if they met all the following criteria: (1) the study focused on a stated therapeutic area or health condition; (2) the study used a mobile device to measure and record study outcomes outside of a research clinic setting (i.e., remote data capture); (3) the mobile device collected objective data; and (4) the study assessed the effect of an assigned intervention (i.e., interventional trials) or monitored exposures and health conditions of participants (i.e., observational studies). Studies that solely examined feasibility or measured only subjective data (e.g., patient-reported outcomes [PROs]) were excluded, as were meta-analyses.

Three steps were taken to assess the relevance of each publication identified in the search. First, two trained analysts independently reviewed the titles of all publications and identified those that they believed did not meet the inclusion criteria. Publications were excluded if both analysts independently determined that the publication was not relevant. Second, two analysts independently applied the inclusion/exclusion criteria to the remaining publications by reviewing the abstracts; differences in the reviewers' assessment of eligibility were resolved by a third analyst. Third, for the publications that remained, two analysts reviewed the full text of the publication for final confirmation of eligibility.

To organize and extract the relevant information from the final publications, we used NVivo, a qualitative data analysis software program [5]. We identified and extracted the following information from each publication: (1) the design (i.e., interventional trial vs. observational study) [6] and type of clinical research (e.g., treatment, prevention, epidemiological) [7]; (2) therapeutic conditions under investigation; (3) mobile device(s) used; (4) mobile outcome assessments and conventionally measured outcome assessments reported; (5) placement of the device; (6) sampling rate; (7) whether the mobile outcome assessment was used to measure a primary, co-primary (where the outcome was one of several deemed necessary to measure an intervention effect or change over time in the study), secondary, or exploratory endpoint; and (8) overall study objectives. Next, we applied current descriptions of outcome assessments (i.e., biomarkers, performance outcomes, observer-reported outcomes, clinician-reported outcomes, and PROs) [2, 3] to all assessments reported in the publications. We then grouped the mobile outcomes according to how they were used in the research – e.g., whether the outcome was used as an assessment of users' physical activity, sleep, or respiration. Online supplementary materials for this publication summarize the context of use of the various mobile outcome assessments (for all online suppl. material, see www.karger.com/doi/10.1159/000486347).

Results

Screening

Our initial search (Appendix 1) retrieved 2,530 references (Fig. 1). We excluded just over a third of the retrieved publications (n = 942) after title screening and another 78% (n = 1,241) after abstract screening. The excluded publications predominately reported: (1) early-phase studies of validity and reliability of the device or (2) clinic-based studies (i.e., wearable or sensor devices were not used for remote data capture). A total of 104 publications were included in the full document review. Upon further review, we excluded 16 additional publications on the basis of our inclusion and exclusion criteria. Data were extracted from the remaining 88 publications.

Fig. 1

Flow diagram of the review process.

Data Extraction

Only a quarter of the publications reviewed (n = 22; 25%) described interventional trials. All interventional trials were RCTs and included prevention trials (n = 5) [8, 9, 10, 11, 12], health-related quality-of-life trials (n = 11) [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23] (which are RCTs that focus on managing the burden of chronic illness and coping with symptoms), and treatment trials (n = 6) [24, 25, 26, 27, 28, 29] (which focus on assessing the safety and efficacy of a new medication or medical device). The remaining publications (n = 66) described observational clinical research studies where exposures were unassigned and outcomes were captured using mobile devices. The observational studies included epidemiological studies (n = 37), quality-of-life studies (n = 17), prevention studies (n = 6), diagnostic studies (n = 3), 1 screening study, 1 genetic study, and 1 expanded access study (see online suppl. material Type and Design of Clinical Research Studies Using Mobile Outcome Assessments).

Thirteen different therapeutic areas were identified in the review (Table 1). The most frequently cited areas of study were cardiology (n = 19), diabetes (n = 13), sleep (n = 10), obesity (n = 9), and geriatrics (n = 9), all together comprising over half (68%) of the 88 publications. These categories were not exclusive, as some studies investigated multiple related therapeutic areas (e.g., diabetes and obesity [30], diabetes and myocardial infarctions [31], nutrient deficiency and sleep [32]).

Table 1

Therapeutic areas and technologies by study design

	Interventional trials (n = 22)		Observational studies (n = 66)
	n (%)	Ref.	n (%)	Ref.
Therapeutic area
Cardiology	3 (14)	17, 19, 25	16 (24)	31, 37–40, 53–63
Diabetes	5 (28)	11, 12, 24, 26, 28	8 (12)	30, 31, 41, 64–68
Sleep	3 (14)	15, 20, 21	7 (11)	32, 34–36, 69–71
Obesity	0 (0)	-	9 (14)	30, 72–79
Geriatrics	0 (0)	-	9 (14)	80–88
Neurology	1 (5)	13	3 (5)	44, 46, 89
Reproductive and peripartum health	2 (9)	8, 10	2 (3)	90, 91
Orthopedics	1 (5)	22	3 (5)	92–94
Pulmonology	0 (0)	-	3 (5)	95–97
Arthritis	1 (5)	16	2 (3)	93, 98
Psychology	0 (0)	-	3 (5)	78, 99, 100
Cancer	3 (14)	14, 18, 23	0 (0)	-
Nephrology	0 (0)	-	2 (3)	33, 101
Gastroenterology	1 (5)	29	1 (2)	43
Nutrition	1 (5)	27	1 (2)	32
Device
Wearable inertial sensor/accelerometer	16 (73)	8, 9, 12, 14–25, 27	59 (89)	30, 32–36, 44, 46, 53–103
Biosensor	6 (28)	11, 12, 24, 26, 28, 29	7 (11)	31, 33, 37, 39–41, 43
Continuous glucose monitor	5 (23)	11, 12, 24, 26, 28	1 (2)	41
Electrocardiograph	0 (0)	-	2 (3)	31, 40
Ingestible pH monitor	1 (5)	29	1 (2)	43
Ambulatory blood pressure monitor	0 (0)	-	1 (2)	39
Implantable cardioverter-defibrillator	0 (0)	-	1 (2)	37
Heart rate monitor	0 (0)	-	1 (2)	33
Pressure sensor and instrumented walkways	1 (5)	13	1 (2)	38
Medication adherence monitor	1 (5)	10	0 (0)	-
Geolocation monitor	0 (0)	-	1 (2)	33
Global Positioning System	0 (0)	-	1 (2)	33
Altimeter	0 (0)	-	1 (2)	33

Five different mobile device categories were identified (Table 1). The overwhelming majority of the publications (86%; n = 75) used inertial (motion) sensors to capture mobile outcomes. Inertial sensors include accelerometers and gyroscopes and are used to measure a body's acceleration and angular rate of motion. Biosensors were the next most common type of device identified (15%; n = 13). These included continuous glucose monitors (CGMs), ambulatory electrocardiographs, ingestible pH monitors, ambulatory blood pressure monitors, implantable cardioverter defibrillators, and heart rate monitors. Other mobile devices used were pressure sensors and instrumented walkways, medication adherence monitors, and geolocation monitors. Some studies used multiple devices to measure outcomes (e.g., a CGM and accelerometer [12, 24], a heart rate monitor and accelerometer with geo location monitoring [33]).

Two types of mobile outcome were identified in the publications: mobile performance outcomes and biomarkers. Mobile outcomes were often captured multiple times per day and in free-living conditions. The most common were mobile performance outcomes of users' physical activity, which were identified in 74% of the publications (n = 65). Other mobile performance outcomes included measurements of users' sleep, mobility, and pill adherence. Mobile biomarkers included measures of cardiac, glucose, gastric reflux, and respiration outcomes, as well as the intensity of head-related injury. Mobile outcome measurements were used in 15 publications to assess primary or co-primary endpoints in RCTs (Table 2). In each of these studies, other conventionally measured outcome assessments, including observed and self-reported outcome measures, were captured.

Table 2

Use of mobile outcome assessments in RCTs

	Primary endpoint (n = 10)		Co-primary endpoint (n = 5)		Secondary endpoint (n = 9)		Exploratory endpoint(n = 3)		Other (n = 2)
	n (%)	Ref.	n (%)	Ref.	n (%)	Ref.	n (%)	Ref.	n (%)	Ref.
Therapeutic area
Cardiology	1 (10)	19	1 (20)	17	2 (22)	19, 25	0 (0)	-	0 (0)	-
Diabetes	3 (30)	11, 24, 26	2 (40)	12, 28	3 (33)	11, 26, 28	0 (0)	-	1 (50)	26
Sleep	0 (0)	-	1 (20)	21	2 (22)	15, 20	0 (0)	-	0 (0)	-
Neurology	1 (10)	13	0 (0)	-	0 (0)	-	0 (0)	-	0 (0)	-
Reproductive and	2 (20)	8, 10	0 (0)	-	0 (0)	-	0 (0)	-	0 (0)	-
peripartum health Orthopedics	1 (10)	22	0 (0)	-	1 (11)	22	0 (0)	-	0 (0)	-
Pediatrics	0 (0)	-	0 (0)	-	0 (0)	-	1 (33)	9	0 (0)	-
Arthritis	0 (0)	-	0 (0)	-	1 (11)	16	0 (0)	-	0 (0)	-
Cancer	0 (0)	-	1 (20)	23	0 (0)	-	2 (66)	14, 18	1 (50)	14
Gastroenterology	1 (10)	29	0 (0)	-	0 (0)	-	0 (0)	-	0 (0)	-
Nutrition	1 (10)	27	0 (0)	-	0 (0)	-	0 (0)	-	0 (0)	-
Mobile outcome category
Performance outcome	6 (60)	8, 10, 13, 19, 22, 27	3 (60)	17, 21, 23	6 (67)	15, 16, 19, 20, 22, 25	3 (100)	9, 14, 18	1 (50)	14
Biomarkers	4 (40)	11, 24, 26, 29	2 (40)	12, 28	3 (33)	11, 26, 28	0 (0)		1 (50)	26
Device
Wearable inertial sensor/accelerometer	5 (50)	8, 19, 22, 24, 27	4 (80)	12, 17, 21, 23	6 (67)	15, 16, 19, 20, 22, 25	3 (100)	9, 14,	1 (50)	14
Biosensor	4 (40)	11, 24, 26, 29	2 (40)	12, 28	3 (33)	11, 26, 28	0 (0)	18	1 (50)	26
Continuous glucose monitor	3 (30)	11, 24, 26	2 (40)	12, 28	3 (33)	11, 26, 28	0 (0)	-	1 (50)	26
Ingestible pH monitor	1 (10)	29	0 (0)	-	0 (0)	-	0 (0)	-	0 (0)	-
Pressure sensor and instrumented walkways	1 (10)	13	0 (0)	-	0 (0)	-	0 (0)	-	0 (0)	-
Medication adherence monitor	1 (10)	10	0 (0)	-	0 (0)	-	0 (0)	-	0 (0)	-

Mobile Performance Outcomes

Physical Activity

Mobile outcome assessments of physical activity included measurements of device users' activity intensity, duration, and frequency (Table 3). Each of these assessments used inertial sensors, although these devices were used in a variety of study contexts (see online suppl. Table Mobile Physical Activity Outcomes). The placement of wearable devices on users' bodies was dependent on the device and intended physical activity; however, over half (n = 38) of the publications reporting physical activity-related outcomes noted using waist-worn devices (see online suppl. Table Mobile Physical Activity Outcomes for a full list of device placements for various physical activity assessments). Wearable inertial sensors were also placed on users' wrist, leg, foot, arm, base of the spine, and head. Of the publications that specified the frequency with which mobile physical activity outcomes were collected (n = 22), over half (n = 12) sampled in 60-s epochs (see online suppl. Table Mobile Physical Activity Outcomes for a full list of sampling frequencies reported). Thirteen publications described the use of mobile devices to collect objective physical activity data in RCTs, and 52 publications described their use in observational studies (see online suppl. material Use of Mobile Outcomes in Clinical Research).

Table 3

Mobile PA outcomes

Mobileoutcomes	RCTs (n = 13),n (%)	Observational studies (n = 52),n (%)	Total publications (n = 65),n (%)
Intensity	9 (69)	40 (77)	49 (75)
Accelerometer counts	6 (46)	23 (44)	29 (45)
Average counts per day	6 (46)	22 (42)	28 (43)
Total counts per day	0 (0)	2 (4)	2 (3)
Average counts between symptom reporting	1 (8)	0 (0)	1 (2)
Average counts during data collection period	1 (8)	0 (0)	1 (2)
Step counts	1 (8)	20 (38)	21 (32)
Average daily step count	0 (0)	16 (31)	16 (25)
Total steps per data collection period	1 (8)	3 (6)	4 (6)
Average steps per bout of continuous steps	0 (0)	3 (6)	3 (5)
Cadence (steps per minute)	0 (0)	2 (4)	2 (3)
METs	0 (0)	10 (19)	10 (15)
METs per minute	0 (0)	9 (17)	9 (14)
METs per sustained bout of MVPA (>10 min)	0 (0)	2 (4)	2 (3)
METs per hour	0 (0)	1 (2)	1 (2)
Daily caloric (energy) expenditure (kcal per day)	2 (15)	7 (13)	9 (14)
Daily peak and low activity counts and ratios (e.g., 1- and 2-h peak counts per day, sedentary-to-light activity ratio)	0 (0)	7 (13)	7 (11)
Duration	6 (46)	39 (75)	45 (69)
Minutes per day of activity of varying intensity	4 (31)	32 (62)	36 (55)
Minutes per day of MVPA	4 (31)	29 (56)	33 (51)
Minutes per day sedentary	2 (15)	24 (46)	26 (40)
Minutes per day of light intensity	2 (15)	19 (37)	21 (32)
Minutes per day of all intensity	2 (15)	9 (17)	11 (17)
Minutes per bout of MVPA	0 (0)	8 (15)	8 (12)
Minutes per day doing various activities (e.g., lying down, sitting, standing, moving, shuffling, walking)	1 (8)	5 (10)	6 (9)
Cumulative daily minutes per bout of sedentariness	1 (8)	3 (6)	4 (6)
Percent of daily time doing various activities (e.g., wearing device; inactivity;
sitting; low, medium, high step activity)	0 (0)	3 (6)	3 (5)
Percent of daily time in sustained (e.g., >1 min,≥10 min) MVPA	0 (0)	1 (2)	1 (2)
Cumulative daily minutes per bout (lasting 1 h or more) of MVPA	0 (0)	1 (2)	1 (2)
Frequency	1 (8)	14 (27)	15 (23)
Number of daily bouts of varying activity	1 (8)	10 (19)	11 (17)
Number of moderate- or vigorous-intensity bouts	0 (0)	4 (8)	4 (6)
Number of daily sedentary bouts of varying duration (e.g.,≤30, >30, >60 min)	0 (0)	3 (6)	3 (5)
Number of sedentary breaks	0 (0)	3 (6)	3 (5)
Number of sit-to-stand transitions	0 (0)	2 (4)	2 (3)
Number of walking bouts of varying intensity	0 (0)	2 (4)	2 (3)
Number of daily upright events	1 (8)	0 (0)	1 (2)
Description of physical activity patterns	0 (0)	3 (6)	3 (5)
Intermittent or real-world nature of activity intensity	0 (0)	1 (2)	1 (2)
Staggering of daily physical activity (1- and 2-h peak-to-average-activity counts)	0 (0)	1 (2)	1 (2)
Diurnal profile of physical activity (average steps per hour across different times of day)	0 (0)	1 (2)	1 (2)
Variance of time between bouts	0 (0)	1 (2)	1 (2)
Other physical activity endpoints	0 (0)	6(12)	6 (9)
Proportion of population meeting guidelines	0 (0)	6 (12)	6 (9)
Healthy People 2010 PA recommendations	0 (0)	1 (2)	1 (2)
2010 WHO guidelines (achieving at least 2.5 h of MVPA per week)	0 (0)	1 (2)	1 (2)
WHO recommendations (average of≥60 min of MVPA per day)	0 (0)	1 (2)	1 (2)
UK PA Guidelines	0 (0)	1 (2)	1 (2)
ACSM higher limit recommendation for PA	0 (0)	1 (2)	1 (2)
ACSM lower limit recommendation for PA (67 METs min a day)	0 (0)	1 (2)	1 (2)
AHA and ACSM minimum recommendations of PA	0 (0)	1 (2)	1 (2)
National Association for Sport and Physical Education PA recommendations	0 (0)	1 (2)	1 (2)

PA, physical activity; METs, metabolic equivalents; MVPA, moderate-to-vigorous PA; ACSM, American College of Sports Medicine; AHA, American Heart Association.

Among the RCTs, mobile physical activity outcomes were used in a wide range of study contexts. For example, they were used in quality-of-life RCTs among patients with arthritis [16], cancer [14, 18, 23], various forms of heart disease [17, 19], Parkinson's disease [13], hip fractures [22], and insomnia [20]. Other RCTs included 2 prevention trials to increase physical activity among adolescents [9] and postpartum women [8], 1 phase II trial of a counseling intervention to reduce sedentary time among stroke survivors [25], and 1 phase III trial of the effects of nutrient supplements among 18-month-old children [27]. The mobile physical activity outcomes were used as primary or co-primary endpoints in 7 of these trials, as secondary endpoints in 4 trials, and as exploratory endpoints in 3 trials (Table 2).

Similarly, a wide range of observational studies used mobile outcomes of physical activity. A list of these studies and the context of use of the mobile outcomes measured can be found in the online supplementary material (online suppl. Table Mobile Physical Activity Outcomes).

Sleep

Mobile outcomes of participants' sleep performance included measurements of duration of rest, sleep efficiency (i.e., percent of time in bed spent sleeping), wakefulness after sleep onset, sleep latency (i.e., the amount of time after recorded bedtime and sleep onset), and personal light exposure (Table 4). Three publications reported the use of mobile sleep outcomes in RCTs, while another 8 publications reported mobile sleep outcomes in observational studies (see online suppl. material Use of Mobile Outcomes in Clinical Research).

Table 4

Mobile sleep outcomes

Mobile outcomes	RCTs (n = 3),n (%)	Observational studies (n = 8),n (%)	Total publications (n = 11),n (%)
Duration of rest	3 (100)	7 (88)	10 (91)
Total sleep time	3 (100)	3 (38)	6 (55)
Total time in bed	2 (67)	0 (0)	2 (18)
Bed time	1 (33)	0 (0)	1 (9)
Average nap time	0 (0)	1 (13)	1 (9)
Sleep period time	0 (0)	1 (13)	1 (9)
Longest sleep period	0 (0)	1 (13)	1 (9)
True sleep time	0 (0)	1 (13)	1 (9)
Sleep duration	0 (0)	1 (13)	1 (9)
Hours with rapid shallow breathing	0 (0)	1 (13)	1 (9)
Time asleep	0 (0)	1 (13)	1 (9)
Sleep efficiency percentage (e.g., actual sleep time/total sleep duration or time in bed)	2 (67)	4 (50)	6 (55)
Wakefulness after sleep onset	1 (33)	5 (63)	6 (55)
Time awake after sleep onset	1 (33)	4 (50)	5 (45)
Number of awakenings after sleep onset	0 (0)	3 (38)	3 (27)
Sleep onset latency	2 (67)	2 (25)	5 (36)
Activity level during sleep	0 (0)	2 (25)	2 (18)
Sleep activity level	0 (0)	1 (13)	1 (9)
Movement rate	0 (0)	1 (13)	1 (9)
Standard deviation of movement rate	0 (0)	1 (13)	1 (9)
Patterns of behavior during sleep	0 (0)	1 (13)	1 (9)
Personal light exposure	0 (0)	1 (13)	1 (9)
Circadian illuminance	0 (0)	1 (13)	1 (9)
Photopic illuminance	0 (0)	1 (13)	1 (9)
Circadian stimulus	0 (0)	1 (13)	1 (9)

The 3 RCTs were all quality-of-life studies that assessed the impact of an intervention on participants' sleep quality. The participants in 2 of these RCTs wore Actiwatch devices (Philips Respironics, Bend, OR, USA) on their wrists [15, 21], while the participants in the other RCT wore a SenseWear Armband (SensorMedics Italia, Milan, Italy) placed on their nondominant upper arm [20]. In 1 of the RCTs, the Actiwatch device was used to measure users' sleep performance as a co-primary study endpoint [21] (Table 2). In the other 2 RCTs, mobile sleep outcomes were used to assess secondary endpoints [15, 20] (Table 2). See online supplementary Table Mobile Sleep Outcomes for a full list of observational studies.

Inertial sensors were used in the majority of sleep-related observational studies to measure sleep quality, and their outcomes were compared or combined with conventional measurements of sleep, including self-reported and direct observation (see online suppl. Table Mobile Sleep Outcomes). One study, however, compared a standard biomarker for sleep outcome (i.e., levels of melatonin in saliva samples) to a mobile sleep outcome using the Daysimeter-D inertial sensor (Lighting Research Center, Troy, NY, USA) [34]. The Daysimeter-D is a small device that combines inertial sensing and a light meter to measure ambient levels of light. The participants in this study wore goggles mounted with the Daysimeter-D device to measure activity as well as light exposure [34]. The wearable devices used in other observational studies measuring sleep were primarily placed on the users' nondominant wrist or arm. Children in 1 observational study wore the ActiGraph GT3X+ accelerometer (ActiGraph, Pensacola, FL, USA) on their waists to collect daytime as well as sleep-time activity [35]. Mobile outcomes measuring sleep performance were used to measure primary or co-primary study endpoints in 3 of these observational studies [32, 35, 36]. A full list of how these outcomes were used can be found in online supplementary Table Mobile Sleep Outcomes.

Mobility

Assessment of mobile device users' mobility included objective measurements of gross motor activity, including walking speed, upright time, and quality of gait (Table 5). Two publications reported mobile performance outcomes of users' mobility in RCTs, and 3 other publications reported their use in observational studies (see online suppl. Table Mobile Mobility Outcomes). The 2 RCTs were both quality-of-life studies. In one of these RCTs, patients with Parkinson's disease used a wearable inertial sensor (CuPiD system) placed on their ankles and a portable, pressure-sensing, instrumented walkway placed on the floor (PKMAS Walkway; ProtoKinetics, Havertown, PA, USA) to assess a primary endpoint [13] (Table 2). In the other RCT, geriatric patients used inertial sensors (activPAL; PAL Technologies Ltd., Glasgow, UK) placed on their thigh after surgery for hip fractures to assess a secondary endpoint [22]. See online suppl. Table Mobile Mobility Outcomes for more information on the placement of mobile devices to assess mobility performance and for information on observational studies using mobile outcome assessments of mobility.

Table 5

Mobile mobility outcomes

Mobile outcomes	RCTs (n = 2),n (%)	Observational studies (n = 3),n (%)	Total publications (n = 5),n (%)
Walking speed	1 (50)	2 (67)	3 (60)
Upright time (standing and walking) over 24 h	1 (50)	0 (0)	1 (20)
Gait quality	0 (0)	1 (33)	1 (20)
Smoothness of gait	0 (0)	1 (33)	1 (20)
Stride regularity (gait rhythm and consistency)	0 (0)	1 (33)	1 (20)
Width of dominant peak in power spectrum	0 (0)	1 (33)	1 (20)

Adherence

One publication [10] reported an RCT using a mobile medication adherence monitor (SIMpill®, London, UK) to assess the mean number of pills missed as a means of measuring patient adherence to an oral contraceptive pill (see suppl. Table Mobile Adherence Outcomes for a list of contexts of use). The adherence monitor is an electronic pillbox that records the time and date of accessing the pills contained within the device. The study, which investigated the impact of daily text message reminders on contraceptive pill adherence, used the mobile outcome to assess the study's primary endpoint.

Mobile Biomarkers

Cardiac Biomarkers

Mobile assessment of cardiac biomarkers included continuous monitoring and measurement of patients' heart rate, daytime and nighttime pulse pressure, occurrence of atrial fibrillation, heart rate turbulence, and T-wave analyses (Table 6). All of the publications reporting on the use of mobile cardiac biomarkers were observational and included epidemiological (n = 2), prevention (n = 2), diagnostic (n = 1), and genetic (n = 1) studies. Devices used included ambulatory heart rate monitors, electrocardiographs, ambulatory blood pressure monitor, implantable cardioverter-defibrillators, and pressure sensors. Mobile cardiac biomarkers were used to measure study outcomes among patients with atrial fibrillation [37], heart failure [38], hypertension [39], and myocardial infarctions [31], as well as patients who had undergone kidney transplant surgery [33]. The mobile cardiac biomarkers were used to measure either primary or co-primary study endpoints in 4 observational studies [33, 37, 39, 40] and exploratory endpoints in 2 observational studies [31, 38] (see online suppl. Table Mobile Cardiac Biomarkers for a list of contexts of use of cardiac biomarkers).

Table 6

Mobile cardiac biomarkers

Mobile outcomes	RCTs(n = 0),n (%)	Observational studies (n = 6),n (%)	Total publications (n = 6),n (%)
Heart rate (bpm)	0 (0)	3 (50)	3 (50)
Standard deviation of heart rate	0 (0)	1 (17)	1 (17)
Daytime pulse pressure (mm Hg)	0 (0)	1 (17)	1 (17)
Nighttime pulse pressure (mm Hg)	0 (0)	1 (17)	1 (17)
Occurrence of atrial fibrillation (for >6 min or 6 h)	0 (0)	1 (17)	1 (17)
Heart rate turbulence	0 (0)	1 (17)	1 (17)
Turbulence onset	0 (0)	1 (17)	1 (17)
Turbulence slope	0 (0)	1 (17)	1 (17)
T-wave analyses	0 (0)	1 (17)	1 (17)
QT interval	0 (0)	1 (17)	1 (17)
Fully automatic biomarkers of the T wave named biGaussian function	0 (0)	1 (17)	1 (17)
3D markers derived from a principal component analysis on the T wave	0 (0)	1 (17)	1 (17)

bpm, beats per minute.

Glucose Biomarkers

Mobile biomarkers of users' glucose were measured by continuous glucose monitoring using wearable CGMs. Mobile biomarkers included remote monitoring of average glucose levels, as well as the time in range, number of severe hypoglycemic episodes, glucose level variability, and minimum and maximum glycemic values (Table 7). Mobile glucose biomarkers were used as primary outcomes in 5 RCTs (Table 2). Each of the RCTs assessed the use of closed-loop sensor-augmented insulin pump therapies – or artificial pancreases – to manage glycemic variability and reduce the time outside of the “normal” glucose range among patients with type 1 diabetes. Mobile glucose biomarkers were also used to measure secondary study endpoints in 3 other trials (Table 2) and 1 observational study [41]. See online supplementary Table Mobile Glucose Biomarkers for a full list of contexts of use of glucose biomarkers.

Table 7

Mobile glucose biomarkers

Mobileoutcomes	RCTs (n = 5),n (%)	Observational studies (n = 1),n (%)	Total publications (n = 6),n (%)
Average glucoselevel	5 (100)	1 (100)	6 (100)
Meanglucoselevelasmeasured with the use of CGMs	4 (80)	1 (100)	5 (83)
Median glucose level	1 (20)	0 (0)	1 (17)
Time in range	4 (80)	1 (100)	5 (83)
Percentage of time in“normal” blood glucose range (3.9–10 mmol/L or 70–180 mg/dL)	4 (80)	1 (100)	5 (83)
Over 24 h	3 (60)	1 (100)	4 (67)
During nighttime and evening hours	1 (20)	0 (0)	1 (17)
During intervention period	1 (20)	0 (0)	1 (17)
Percentage of time spent in hyperglycemic range (>10 mmol/L or 180 mg/dL)	3 (60)	1 (100)	4 (67)
Percentage of time spent in hypoglycemic range (<3.9 mmol/L or 70 mg/dL)	3 (60)	1 (100)	4 (67)
Mean percent of time with a low glucose level	1 (20)	0 (0)	1 (17)
Time in tight glucose target range (4.4–7.8 mmol/L)	1 (20)	0 (0)	1 (17)
Percentage of time spent in severe hypoglycemic range (<55 mg/dL)	1 (20)	0 (0)	1 (17)
Percentage of time in mild hyperglycemic range (180–250 mg/dL)	1 (20)	0 (0)	1 (17)
Number of episodes of severe hypoglycemia	2 (40)	1 (100)	3 (50)
Glucose level variability	2 (40)	1 (100)	3 (50)
Standard deviation of blood glucose	2 (40)	1 (100)	3 (50)
Minimum and maximum glycemic values	1 (20)	0 (0)	1 (17)

CGMs, continuous glucose monitors.

Gastric Reflux Biomarkers

Mobile outcomes of gastric reflux biomarkers included continuous real-world monitoring and measurement of the percent of time with a gastric pH <4, the users' DeMeester score [42], and the total number of acid episodes (Table 8). One publication reported the use of mobile gastric reflux biomarkers in a phase IV RCT investigating appropriate treatment dosing [29], and another publication reported their use in an observational study investigating the diagnosis of gastroesophageal reflux disease [43]. Both of these studies used the Bravo pH capsule monitoring device (Medtronic, Minneapolis, MN, USA), which was attached to the patients' esophageal mucosa by a clinician. The observational study reported sampling users' pH levels every 6 s [43]. The online supplementary Table Mobile Gastric Reflux Biomarkers provides more details on the contexts of use of these mobile outcomes.

Table 8

Mobile gastric reflux biomarkers

Mobile outcomes	RCTs (n = 1),n (%)	Observational studies (n = 1),n (%)	Total publications (n = 2),n (%)
Percent of total time with a pH <4 over various time periods (24 h, 48 h)	1 (100)	1 (100)	2 (100)
DeMeester score	1 (100)	1 (100)	2 (100)
Total number of acid episodes	0 (0)	1 (100)	1 (50)

Respiration Biomarkers

One publication reported measuring mobile outcomes of respiration, including the rate and standard deviation of respiration, by placing a pressure sensor under patients' mattresses (see online suppl. Table Mobile Respiration Biomarkers) [38]. This mobile biomarker was incorporated as an exploratory endpoint in an observational cardiology study to assess physiological patterns of patients with heart failure in the home environment and to determine if specific patterns correlate with hospital readmissions [38].

Intensity of Head-Related Injury Biomarkers

One publication [44] used an inertial sensor (X2 Biosystems Inc., Seattle, WA, USA) placed in the mouth guard of rugby players to measure the magnitude and frequency of head impacts. Specific measurements included the linear and rotational acceleration of the head after impact, impact location, and frequency and duration (measured in milliseconds) of the impact (see online suppl. Table Mobile Head-Related Injury Intensity Biomarkers for further information on the context of use) sampled at 1,000 Hz (i.e., 1,000 times per second). Response biomarkers [3] were interpreted from the exposure (i.e., head impact) measures using previously published thresholds for injury tolerance levels for concussion, total impact frequency burden, and head impact severity.

Discussion

This systematic review describes recent use of mobile devices in clinical research. We found that mobile devices are being used across a variety of therapeutic areas, but they are currently more commonly used in observational than interventional research. Because the use of mobile devices in any type of clinical research can inform how these devices could be used in future interventional research, we have chosen to include information about their use in observational research in this review.

The majority of publications reported using mobile outcomes – including continuous and remote monitoring of users' performance and specific biomarkers – to inform primary or co-primary study endpoints. Mobile devices provided new ways to assess clinical outcomes and biomarkers at higher frequency, outside of structured research settings, during activities of daily living, and with greater objectivity, given the technologies' ability to monitor patients with minimal self- or observer input. The uses of inertial sensors/accelerometers were reported in a large proportion of the reviewed publications to remotely capture users' physical activity, sleep, and mobility.

Given the broad scope of our search terms, we identified and summarized a variety of mobile biomarkers (e.g., continuous glucose monitoring, ambulatory blood pressure monitoring, continuous pH monitoring). In the publications we reviewed, the biomarkers may have also had multiple applications in the clinical studies. For instance, they may have been used for prognostic or predictive purposes and/or for monitoring safety [3]. During this review, we noted that applying current definitions for conventionally measured outcomes to mobile outcomes was difficult, as current category definitions [2, 3] may not adequately reflect the novelty of mobile outcome assessments. Biomarkers are currently defined as assessments of biological processes, such as histological, biochemical, or radiographic measurements, that reflect the physiological effects of disease progression or therapeutic intervention. It is often noted that they are not direct assessments of how a patient feels, functions, or survives [2, 3, 45]. Other clinical outcome assessments, including clinician-reported outcomes, observer-reported outcomes, PROs, and performance outcomes, can provide more direct assessment of meaningful health aspects [2]. Performance outcomes are quantifications of patient performance in a specified task instructed by a health care professional, while each of the other outcome assessments are based upon observations originating from specific observers, i.e., health care professionals, patients, or someone other than the patient or a health care professional [3]. Unfortunately, these definitions do not take into account the novelty of mobile device-based measurements, which include measurements collected while users engage in activities related to their daily living without researcher supervision, nor the high frequency of data capture of mobile sensors, some of which have the capability of sampling at several hundred times per second – rates that for all intents and purposes may be considered “continuous” during the measurement epoch. It is likely that, by objectively measuring day-to-day patient activity and more acute fluctuations in biological markers, mobile outcomes may be able to more directly assess meaningful patient health outcomes and thus provide a more complete overall picture of disease burden and therapeutic effect. For the purposes of this review, we placed mobile outcomes into existing categories for purposes of comparison; however, it may be necessary to include new categories (or modify current definitions) of clinical outcome assessments in order to accommodate novel measures using mobile devices and advance their use in interventional research.

Another factor that may be hindering the use of mobile devices in interventional research is the lack of standardization. We found that studies investigated a wide range of variables within specific mobile outcomes. As an example, studies included in our review captured a wide variety of mobile performance outcomes used to measure the intensity, duration, and frequency of users' physical activity. Further, studies used an array of variables to assess their endpoints, including variations in sampling rates, placement of the device, and technologies. In particular, there is a wide array of inertial sensors on the market that have varying proprietary standards for reporting physical activity outcomes. If this trend continues, the lack of standardization will make interpreting and comparing results across studies and across therapeutic areas more difficult, thereby inhibiting the acceptance and greater use of mobile outcomes in regulatory interventional research.

In our review we did not attempt to identify the intended use of specific biomarkers (e.g., prognostic biomarkers, predictive biomarkers, or safety biomarkers [3]) but recognize that not all biomarkers are used ultimately to assess research outcomes. Identifying the intended use of these measurements would provide greater understanding of how to appropriately use them in clinical research. Additionally, in identifying and categorizing mobile outcomes in clinical research, we attempted to only identify measurements used to assess a clinical outcome (e.g., severity of head impact), rather than to measure an exposure (e.g., frequency of head impacts). However, given the variety of new technologies used in clinical research, this can be difficult to distinguish. For example, some technologies have the capability of capturing a wide range of data at one time, including data that could be used to identify exposures as well as track outcomes.

Our review has several limitations. First, a number of studies may have been excluded from the final analysis due to our interpretation of the published research methods. As a result, some mobile outcomes related to specific therapeutic or disease conditions were not summarized in our review. For example, in our original search we retrieved over 96 references that were related to Parkinson's disease. The vast majority of these studies were excluded from our final review because the mobile outcomes (e.g., freezing of gait, brady­kinesia, postural sway, tremor, etc.) were not used in the context of a clinical research study (one of our inclusion criteria). In all, over 70% of the 96 Parkinson's disease-related references retrieved in our initial literature search focused on the development of mobile devices and pertinent algorithms to collect clinical outcomes related to Parkinson's disease, and all but 2 [13, 46] of the remaining references described studies conducted solely in controlled clinic-based environments (one of our exclusion criteria). This demonstrates the vast amount of effort that has gone into the development and refinement of mobile outcome assessments for this disease condition and the wealth of scientific evidence that supports the use and application of mobile technologies in future Parkinson's disease clinical research studies. There are a number of recommended literature reviews that focus on identifying and consolidating the evidence supporting these Parkinson's disease-related outcomes [47, 48, 49, 50], including the specific technologies used in these trials [47, 49, 51], and the validation processes used to ensure accurate and reliable measurements [52].

Second, during the screening process we identified numerous studies that did not meet our inclusion criteria, but these studies suggest that the use of mobile devices in clinical research is rich with early-stage studies (e.g., validation and feasibility studies) to develop new mobile outcomes, validate the analytical operability of technologies, and determine the feasibility of applying these new outcomes and technologies in clinical trials. Validations within these studies include comparisons of healthy subjects and patients with target conditions to assess predictive capabilities of mobile outcomes, comparisons of mobile outcomes with conventionally measured outcomes, studies refining algorithms to interpret mobile outcomes, and studies clarifying the link between mobile outcomes and clinically meaningful endpoints. These developments suggest that the use of mobile devices in clinical trials is likely to see significant growth in the near future.

Finally, this review is limited to studies indexed in PubMed. Anecdotally, we are aware of dozens of studies using mobile devices to measure clinical outcomes and biomarkers conducted by industry sponsors and device manufacturers that have not been published in the peer-reviewed literature. Therefore, this review likely underreports the use of mobile outcomes in clinical research.

Conclusion

Mobile devices are being widely used in clinical research, although their use in interventional research to assess therapeutic effectiveness is limited. For mobile devices to be used more frequently in regulatory interventional research, it is important to emphasize validating, or consolidating, evidence on the clinical meaningfulness of the mobile outcome assessments identified in this review. The wealth of peer-reviewed publications reporting observational research using mobile outcome assessments indicates that such efforts are already underway. To further support that aim, CTTI has developed recommendations and tools that may be helpful for selecting appropriate mobile outcomes as future clinical trial endpoints. We refer readers to CTTI's full set of recommendations and tools for additional information (https://www.ctti-clinicaltrials.org/projects/novel-endpoints) [4].

Statement of Ethics

The authors have no ethical conflicts to disclose.

Disclosure Statement

The authors declare no conflicts of interest.

Funding Sources

Funding for this article was made possible, in part, by the US Food and Drug Administration through grant R18FD005292 and cooperative agreement U19FD003800. Views expressed in publications do not necessarily reflect the official policies of the US Department of Health and Human Services, nor does any mention of trade names, commercial practices, or organization imply endorsement by the US Government. Partial funding was also provided by pooled membership fees and in-kind contributions from CTTI's member organizations.

Appendix 1

Layer	Search terms	Results, n
#1	Search “Wireless Technology”[MeSH] OR “Mobile Applications”[MeSH] OR “Mobile Application”[tiab] OR “Mobile Applications”[tiab] OR “Mobile Apps”[tiab] OR “Mobile App”[tiab] OR “Portable Electronic Apps”[tiab] OR “Portable Electronic Applications”[tiab] OR “Portable Software Application”[tiab] OR “remote technologies”[tiab] OR “remote technology”[tiab] OR “smartphone”[tiab] OR “mobile tech”[tiab] OR “mobile technologies”[tiab] OR “mobile technology”[tiab] OR “mobile device”[tiab] OR “mobile devices”[tiab] OR “wearable technologies”[tiab] OR “wearable technology”[tiab] OR “wearable device”[tiab] OR “wearable devices”[tiab] OR “accelerometer”[tiab] OR “accelerometers”[tiab] OR “Monitoring, Physiologic”[MeSH] OR “Accelerometry”[MeSH] OR “Accelerometry”[tiab] OR “actigraphy”[tiab] OR “biosensor”[tiab] OR “biometric”[tiab] OR “ResearchKit”[tiab] OR “HealthKit”[tiab] OR “healthpatch”[tiab] OR “biochip”[tiab] OR “mobile health data”[tiab] OR “mobile data”[tiab] OR “implant device”[tiab] OR “implant devices”[tiab] OR “implant technology”[tiab] OR “implant technologies”[tiab] OR “ingestible device”[tiab] OR “ingestible devices”[tiab] OR “ingestible technology”[tiab] OR “ingestible technologies”[tiab] OR ((“sensor”[tiab] OR “sensors”[tiab]) AND (“data”[tiab] OR “wireless”[tiab] OR “wearable”[tiab])) OR “Equistasi”[tiab]
#2	Search wear[tiab] OR worn[tiab] OR wearable[tiab] OR wearing[tiab] OR “Signal Processing, Computer-Assisted”[MeSH] OR ((Wireless[tiab] OR remote[tiab]) AND (monitor[tiab] OR monitors[tiab] OR monitoring[tiab] OR diagnosis[tiab] OR diagnose[tiab]))	108,556
#3	#1 AND #2	13,234
#4	Search (randomized controlled trial[pt] OR controlled clinical trial[pt] OR randomized[tiab] OR randomised[tiab] OR randomization[tiab] OR randomisation[tiab] OR placebo[tiab] OR drug therapy[sh] OR randomly[tiab] OR trial[tiab] OR groups[tiab] OR Clinical trial[pt] OR “clinical trial”[tiab] OR “clinical trials”[tiab] OR “evaluation studies”[Publication Type] OR “evaluation studies as topic”[MeSH Terms] OR “evaluation study”[tiab] OR evaluation studies[tiab] OR “intervention study”[tiab] OR “intervention studies”[tiab] OR “case-control studies”[MeSH Terms] OR “case-control”[tiab] OR “cohort studies”[MeSH Terms] OR cohort[tiab] OR “longitudinal studies”[MeSH Terms] OR “longitudinal”[tiab] OR longitudinally[tiab] OR “prospective”[tiab] OR prospectively[tiab] OR “retrospective studies”[MeSH Terms] OR “retrospective”[tiab] OR “follow up”[tiab] OR “comparative study”[Publication Type] OR “comparative study”[tiab] OR “clinical experiments”[tiab] OR “clinical experiment”[tiab]	7,195,858
#5	#3 AND #4	5,534
#6	#5 NOT (Editorial[ptyp] OR Letter[ptyp] OR Case Reports[ptyp] OR “a case study”[ti] OR “: case study”[ti] OR “case study: ” OR Comment[ptyp]) NOT (animals[mh] NOT humans[mh]) AND (“2010/01/01”[PDat] : “3000/12/31”[PDat]) AND English[lang]	2,530