Patient-generated health data management and quality challenges in remote patient monitoring.

BACKGROUND: Patient-Generated Health Data (PGHD) in remote monitoring programs is a promising source of precise, personalized data, encouraged by expanding growth in the health technologies market. However, PGHD utilization in clinical settings is low. One of the critical challenges that impedes confident clinical use of PGHD is that these data are not managed according to any recognized approach for data quality assurance.
OBJECTIVE: This article aims to identify the PGHD management and quality challenges that such an approach must address, as these are expressed by key PGHD stakeholder groups.
MATERIALS AND METHODS: In-depth interviews were conducted with 20 experts who have experience in the use of PGHD in remote patient monitoring, including: healthcare providers, health information professionals within clinical settings, and commercial providers of remote monitoring solutions. Participants were asked to describe PGHD management processes in the remote monitoring programs in which they are involved, and to express their perspectives on PGHD quality challenges during the data management stages.
RESULTS: The remote monitoring programs in the study did not follow clear PGHD management or quality assurance approach. Participants were not fully aware of all the considerations of PGHD quality. Digital health literacy, wearable accuracy, difficulty in data interpretation, and lack of PGHD integration with electronic medical record systems were among the key challenges identified that impact PGHD quality.
CONCLUSION: Co-development of PGHD quality guidelines with relevant stakeholders, including patients, is needed to ensure that quality remote monitoring data from wearables is available for use in more precise and personalized patient care.

BACKGROUND AND SIGNIFICANCE

Patient-Generated Health Data (PGHD) are any health-related data generated by a patient, such as biometric data, symptoms, lifestyle choices, and treatment history. These data are distinguished from health data within clinical settings in two key ways: patients, not healthcare providers, take the primary responsibility for collecting the data; and patients may choose how and with whom they share the data.

The popularity and ubiquity of mobile health technologies is increasing the production of PGHD. Advanced tools such as wearable sensors, mobile apps, web portals, and home monitoring devices enable patients to collect more data about general or specific aspects of their health and increase the potential for PGHD to be useful for healthcare providers in remote patient monitoring (RPM) interventions.

There is published evidence of the range of potential benefits that PGHD from RPM bring to individualizing clinical care. For example, PGHD may help in reduction of hospital re-admission and clinical visits, facilitate timely advice, supplement clinical data captured in in-patient care, deliver a more comprehensive picture of a patient’s status,, and support more personalized treatment planning.

Such evidence that PGHD can make care more personalized and precise engenders positive attitudes toward clinical adoption. In a 2018 Fitbit study of healthcare providers, administrative decision-makers, and IT professionals, 79% of participants saw value in PGHD to supplement consultations and 72% agreed that PGHD is essential to make better decisions about patient care. However, interventions often are initiated on a small scale for a short time; establishing RPM programs based on PGHD as a systematic part of routine clinical care is still challenging. A 2018 survey of over 20 000 healthcare consumers in 28 countries revealed nonsignificant adoption of PGHD.

Reliability concerns hinder expansion of PGHD collection and use as part of routine clinical practices. Healthcare providers often lack confidence that PGHD will support them to make appropriate decisions efficiently, about how to provide individualized care. Given that PGHD are collected by nonprofessionals, in nonstandard ways, outside the controlled environment of the clinic or laboratory. Also, PGHD comprise large amounts of raw data that need to be processed, cleaned, analyzed, and managed before their fitness for use in decision support can be assured. To be trusted for use in shared decision-making by a patient and their healthcare provider, PGHD must be systematically assured to be accurate, complete, accessible, and understandable—necessarily in a multistakeholder and collaborative process.

No known strategy exists to address the requirement for PGHD quality to occur both outside and inside the clinical care setting. This is despite established guidelines on how conventional clinical data quality is managed in healthcare. Technical and operational obstacles that impact PGHD quality include interoperability,,, integration with current electronic medical record (EMR) systems,,, data governance,, co-interpretation, and timeliness-related challenges. Despite the potential of PGHD to support clinical decisions that more precisely match the individual patient’s situation, there are still few studies specifically investigating PGHD quality which perceived that various challenges affect different aspects of PGHD quality which require further efforts,

Hence our research aims to identify what data quality aspects need more consideration at each stage of data management, that is, when PGHD flows from a patient to their healthcare provider. We explored perspectives of those who deal with PGHD in RPM for its primary use purpose, namely patient care, as to understand how different stakeholders consider data quality. Among the questions we have asked of these stakeholders: How do PGHD management processes work in current RPM programs? What are the PGHD quality challenges? And what potential solutions are feasible?

Understanding PGHD stakeholders’ insights can provide a basis for further work to develop guidance concerning quality assurance of PGHD in RPM. The current study focuses on medical grade and consumer wearables, because they are the starting point of PGHD collection, continuously capturing volumes of data during individualized personal activity, and subsequently connecting automatically or manually to platforms, such as mobile apps and web portals.

METHODS

Our study design follows qualitative research principles suggested for eHealth studies. Data were captured through in-depth, open-ended in-person and phone interviews to address descriptive and explanatory questions (eg, what, how, and why). This study received human research ethics approval from the University of Melbourne (Ethics ID: 1850996).

Participants

Interviewees were drawn from three groups of experts who are concerned with the clinical use of PGHD in diverse RPM programs. We included two groups of participants in primary, secondary, and tertiary care settings of the Australian States of Victoria and New South Wales, working with PGHD in any clinical condition, whether the data are generated from a wearable approved for medical use, or from a consumer market wearable. The participant groups include healthcare professionals (“CPs”) such as doctors, nurses, and allied health practitioners; and health information professionals (“IPs”), for example chief information officers, health information managers, and health IT managers. The third group of participants was commercial RPM solution providers (“SPs”), such as wearable manufacturers, representatives of PGHD integration services, and consultants working in this area. This participant group was expanded from Australia to the United Kingdom and United States, due to relevant technology initiatives and recent policies toward incorporating PGHD into routine clinical practice there.,

Snowball sampling was used to recruit participants. We did not invite further participants once saturation occurred, that is, no new information was forthcoming from additional interviewees.

Because we sought to identify PGHD management and quality challenges from professional and industry perspectives, we deliberately excluded patients and their caregivers from this study. However, it lays the groundwork for our next research stage, with patients’ involvement, to co-design guidelines for managing PGHD quality. In addition, we emphasized those groups interested in PGHD’s primary use, that is, patient care; likewise, input from secondary users of PGHD, such as health researchers and payers, will be sought later.

Data collection

Participants were invited to comment on two topics using a protocol comprised of open-ended questions:

The existing PGHD management process in RPM—We asked participants to explain data flow from the point when a patient wears the device and starts collecting data to the point when the healthcare provider reviews the data for decision making. As a prompt to elicit this information we adapted the mHealth Evaluation, Reporting, and Assessment checklist including elements such as infrastructure, technology platform, interoperability, and data security.

The challenges related to PGHD quality—We asked participants to comment on the needs and concerns in relation to providing safe individualized care. We used as a prompt, a comprehensive clinical data quality guideline developed by the Australian Capital Territory including criteria such as accuracy (data are free from errors), accessibility (clear ways for authorized users to access data), consistency (uniformity of data throughout different management stages), interpretation (the form of data presentation to help draw out the key message), relevancy (usefulness of data in the related clinical context), timeliness (data availability in the time needed), and the institutional environment (sociotechnical issues in the clinical setting that may affect data quality).

To ensure consistency, one researcher conducted all interviews.

Data analysis

Interviews were audio recorded and transcribed verbatim using NVivo software version 11. The interview transcripts were thematically analyzed according to the two major topics and within those, were analyzed at further levels of granularity, as follows. Thematic descriptions were coded as topic 1 (PGHD management stages) or topic 2 (PGHD quality aspects). Then, coded findings were sorted into perspectives about PGHD quality, according to whether these came from a CP, IP, or SP stakeholder. Further, the coded results were analyzed to identify gaps or silences, on the topics of PGHD management stages and quality aspects. One researcher analyzed the findings, which were subsequently checked by three expert researchers to ensure agreement.

RESULTS

Twenty participants were interviewed (Table 1). They were involved in RPM interventions across three health conditions, in one primary care setting (diabetes), one secondary care setting (cardiac arrhythmia), and six tertiary care settings (four diabetes and two sleep disorders). The wearables prescribed were:

Table 1.

Participants

Participants	Profession	ID
Healthcare professionals (CPs)	Endocrinologist	CP1, CP2
	Diabetes educator	CP3, CP4, CP5, CP6
	Cardiology technician	CP7
	Sleep technician	CP8, CP9
Health information professionals (IPs)	Chief information officer	IP1, IP2
	Health informatician	IP3
	Health IT manager	IP4
RPM solution providers (SPs)	Wearable manufacturer	SP1
	PGHD integration service provider	SP2, SP3, SP4
	RPM consultant	SP5, SP6, SP7

Diabetes: medical wearables were continuous glucose monitoring (CGM) devices and insulin pumps; consumer wearables were various types of fitness trackers.

Cardiac arrhythmia: the medical wearable was an event monitoring wearable.

Sleep disorder: the medical wearable was a sleep monitoring/activity watch.

(No consumer wearables were used in cardiac arrhythmia or sleep disorders).

PGHD management stages

PGHD management is often outsourced to the wearable manufacturer which include three overall stages: (1) data collection; (2) data processing; and (3) information use for clinical decision making.

PGHD management was similar in remote monitoring of cardiac arrhythmia and of sleep disorders. Data were collected automatically, then flowed automatically from the wearable to the manufacturer’s platform, analyzed there, and subsequently downloaded by the healthcare provider during the patient consultation. In both conditions, data were not designed for a patient to view or interact with during the wearable use period.

Conversely, CPs of diabetes RPM mentioned frequent interaction between the patients and their wearables (Figure 1). PGHD collection generally included two parts: automatic sensor recording, and manual entry of a patient’s meals and activity data into a connected device (eg, receiver or mobile app). Patients received real-time information displayed on the connected device. The collected data were then transferred automatically to the wearable manufacturer’s platform for analysis. Afterward, a report with information about the patient’s status was available for review by both the healthcare provider and the patient.

Figure 1.

PGHD management in diabetes remote monitoring.

Besides CGM wearables, people with diabetes may use consumer devices such as fitness trackers for monitoring their general wellness. However, according to endocrinologists and diabetes educators, CP1–CP6, PGHD from consumer devices were not considered an essential part of diabetes RPM and did not undergo data management processes equivalent to those applied to medical wearable data. CPs themselves might not be inclined to review such data and provide feedback.

PGHD quality challenges

The three groups of participants had different perspectives about PGHD quality challenges at each stage of PGHD management, with gaps in their views on PGHD quality. We paraphrased their points of view about which data quality aspect mattered at each stage of PGHD management, as follows in Table 2 (CPs), Table 3 (IPs), and Table 4 (SPs).

Table 2.

Healthcare providers’ (CPs) perspectives on data quality challenges during PGHD management stages

1. PGHD collection	2. PGHD processing	3. PGHD use for patient care
Accuracy Sometimes CGM read higher values than the actual measurement (CP1, CP2, CP3, CP4) Patient’s illness or dehydration affect the sensor functionality during data recording (CP3, CP5) Healthcare providers and CGM manufacturers disagree on calibration frequency (CP3) Potentials for errors in manual data entry into CGM (CP6) Wrong application of CGM on body results in inaccurate data (CP1) There is a lot burden on patients using CGMs (CP1)	Consistency It is difficult to normalize data from different wearables into one platform (CP6)	Accessibility When patients change CGMs, it is difficult for healthcare providers to access new data (CP4) Lack of organisational ethics approach for data access (CP1) Lack of IT staff to help healthcare providers in accessing reports (CP4, CP5)
Institutional environment Lack of knowledge and guidelines in clinical settings for patients on how to use CGMs properly (CP3) Lack of frequent follow-up to ensure that patients use event monitoring wearables correctly and provide complete data (CP7)	Institutional environment No possibility for further analysis on the report data (CP3) The current infrastructure in healthcare settings does not allow integration of CGM data with EMRs (CP1, CP6)	Interpretability A large amount of data is displayed in the report which needs cleaning and editing (CP5, CP8) Report formats of sleep wearable are not user-friendly (CP8)
Interpretability When patients do not understand the represented data, they are less motivated to use the wearable continuously (CP3, CP7)	Relevancy Different departments have different criteria for what information are relevant to be presented in the reports (CP8)	Relevancy Healthcare providers spend a lot of time on prioritizing the most useful information from the large amount of data presented in the reports (CP6)
		Timeliness Difficulty in data sharing between visits in public care systems, for example where patients see different clinicians (CP2, CP6) Data are not available to healthcare providers on a real-time basis (CP1, CP3, CP4, CP5, CP9)

Table 3.

Health information professionals’ (IPs) perspectives on data quality challenges during PGHD management stages

1. PGHD collection	2. PGHD processing	3. PGHD use for patient care
Accessibility There are challenges in ensuring cybersecurity of wearables to be safe from hackers’ access (IP1)	Institutional environment The current IT infrastructure in healthcare settings, lack of funding and expertise do not allow integration of CGM data with EMRs (IP2, IP3, IP4)	Interpretability Healthcare providers and wearable manufacturers have different views on report formats (IP1)

Table 4.

RPM solution providers’ (SPs) perspectives on data quality challenges during PGHD management stages

1. PGHD collection	2. PGHD processing	3. PGHD use for patient care
Accuracy No automation of contextual data which can improve data accuracy and reduce burden on patients and improve their engagement (SP3)	Institutional environment Lack of PGHD incorporation into current workflows and lack of interfaces reduce PGHD values (SP1, SP6, SP7)	Consistency Various inconsistent reports that do not talk to each other will not provide clear picture of patient status (SP4, SP7)
Consistency Inconsistent data collection from different wearables leads to inconsistency in data presented on reports (SP7)		Interpretability Interpretation of different reports from different wearables is difficult for healthcare providers (SP5) Data are not presented with context (SP7)
		Timeliness Retrospective data access reduces the clinical value of PGHD (SP2, SP4)

Healthcare providers addressed data accuracy at the level of data collection which could be influenced by the wearable design or functionality as well as errors in manual data entry.

Unreliable data collection might be related to lack of support and/or advice for patients on how to collect data, or from lack of guidelines in the healthcare setting. An endocrinologist mentioned that “We have real limited number of staff per patients. We cannot have anybody contact [us] at any time of the day for advice” [CP1]. Likewise, the style of data presentation via the wearable components such as the receiver or mobile app could impact a patient’s understanding of their data and thus their decisions about self-care. This was specific to diabetes RPM; in the other two conditions, data were not available to patients.

After collection, PGHD underwent processing on the wearable manufacturer’s platform. This report might be analyzed by clinical technicians and educators before being used by a clinical specialist. At this stage, PGHD from different wearable platforms were reported to be inconsistent and of unfamiliar structure to healthcare providers. It usually happens when a patient changed to a new version of a wearable: “They are being updated very quickly as soon as new device comes out and it is constantly changing how that is communicated with healthcare professionals. They [patients] just hope healthcare professionals keep up with that because they are very confused by them a lot in most cases” [CP6]. Moreover, reports usually could not be analyzed further, because they were not integrated with EMR systems. “Sometimes it might get scanned in. It does not go in directly. There is no facility for it to be dumped in directly” [CP1].

Depending on the clinical context, CPs had to identify the most relevant information in a patient’s reports. CP6 pointed out the burden of discovering the most relevant information at PGHD processing stage: “I use my judgment a lot as to what information is important. What I am going to focus on and prioritize and too much information is just information overload.”

One main challenge at the third stage was to access processed PGHD. This is largely due to nonstandard access to reports from disparate platforms. “I have got doctors coming to the clinic all the time saying I cannot see anything. I say you are in the wrong account, that is the other device, because they just get used to something and it is changed again” [CP5].

Also, CPs were concerned about missing important patterns when data were not available as soon as they were collected. If a patient received feedback only weeks afterward, during a consultation, they might fail to change behavior, or it could weaken their motivation to collect data. CP8 said: “The introduction of daily data sharing from devices to constantly upload is a good advantage because the clinician can identify quickly when something goes wrong or if they suspect that there is an issue rather than waiting for the data collection period to end and then discover that there was an issue and therefore put some more burden on the health system because the patient will then have to come back.”

IPs in clinical settings had no involvement apart from assisting healthcare providers to access information via the wearable manufacturer’s portal. IPs expressed concerns about accessibility, interpretation, and the lack of infrastructure in healthcare settings. IP1 addressed unauthorized access to PGHD: “We need to make sure that the security is surrounding home devices and their data, interfacing that into the EMR is airtight, so that no one can interfere with the devices and treatment that we might prescribe.” Similar to CPs, IPs considered it important to integrate PGHD into the EMR systems, to normalize and standardize PGHD in structures that comply with clinical document formats. IP2 stated: “We need to inform industry what is required. So, industry currently is doing what they believe is the correct thing to do but we need to inform them what would be clinically appropriate.”

Similar to CPs, SPs mentioned accuracy challenges at the PGHD collection stage.

“The contextual data can be automated too. It shouldn't all burden the patient, that leads to errors. You need a really simple application to make it easy for them. It's not easy for a patient and they don't understand why they're doing it and they're going to disengage” [SP3].

SPs addressed data inconsistency challenges at both collection and data use stages. “The data is unstructured, it's big and it's messy. What we know is that people are using different devices and they're using them inconsistently. They may start using something and then they may move on to something different. But consistency is a big issue. That control factor is just not there” [SP7].

The challenges of PGHD interpretation at data use stage was also addressed: “When we get biometric data back it's fairly difficult to identify a story because you need to understand biometrics in context. So, if someone is more active or less active, they need to give you a sense of what they were doing and why was that or how that impacted their quality of life” [SP7].

DISCUSSION

Overall, these findings point to certain PGHD quality aspects requiring improvement at each stage of data management process, from various perspectives. Also, the findings indicate a lack of cohesion among participants on PGHD management and its quality challenges.

Gaps in PGHD management and potential solutions

A key challenge in PGHD management is that data do not flow into EMR systems, creating no opportunity for data storage and linking with clinical data, or for further analysis during a patient’s care journey.

The results show that despite the existence of defined clinical data quality management protocols, PGHD management appears not to undergo the same level of audit as it would if managed in a clinical setting. The three participant groups highlighted various socio-technical obstacles to PGHD management that hinder its integration with EMR systems, consistent with the research literature.,,, These studies have identified several barriers to integration of PGHD into clinical care, namely: poor interoperability, lack of interfaces for healthcare providers, and lack of incorporation into workflows and accessibility at the point of care.

The possibility to use PGHD to make clinical care more personalized and precise has been demonstrated in two pilot programs administered by the Office of National Coordinator for Health IT using interfaces to integrate PGHD with the current clinical system and workflow. Also, recently developed standards for personal connected health systems offer the potential to improve interoperability of PGHD through Fast Healthcare Interoperability Resources.

Lack of expert human resources in RPM programs emerged as an institutional challenge affecting PGHD quality according to our study. Although the IPs were not actively involved in PGHD management, SPs (SP2, SP3, SP4, and SP6) suggested that health information professionals may have a crucial role in cleansing, normalizing, and standardizing data to present information in a meaningful way to healthcare providers. “We will need EMR analysts to understand what device data you want to send, and where do you want to put [the data] into the record. There is a little bit of limitation on the number of staff that we have available to help inform an informatics agenda within the hospital” [IP1].

Combining wearables data management with health coaching has been considered elsewhere as a means to overcome barriers of PGHD integration into clinical care. Our study found that cardiac technicians and sleep technicians, as well as diabetes educators, undertook a coaching role, but they lacked digital health literacy.

Implications for ensuring PGHD quality

Views on PGHD quality requirements at the various stages of data management differed among our participants, and in some cases their views differed from external research findings and policies.

PGHD accessibility

IPs considered PGHD accessibility issues during data collection and transmission, and in relation to cybersecurity, whereas CPs considered accessibility issues to occur at the data use phase, reflecting their challenges to access PGHD. In our study, CPs working with PGHD in cardiac and sleep disorders intentionally disabled data view by their patients, to prevent behavior changes that might impact PGHD collection and analysis. However, studies in similar RPM have shown that accessing data by patients has enabled them to engage more in their health management and to help in timely diagnoses. For example, in detecting atrial fibrillation, users received notifications of irregular pulses accompanied by advice to schedule a clinical consultation. A recent FDA policy recommends wearable manufacturers to share patient data with patients upon their request.

PGHD accuracy

Both CPs and SPs highlighted the importance of PGHD accuracy at data collection stage. Accuracy issues were the main reason why people with diabetes stopped using CGM wearables in one study. Our findings show that whether data were captured from a medical or consumer wearable, these data could be subjected to human or technological errors leading to some form of inaccuracy.

Some wearables, such as CGM, require more user interaction than others, such as setting change, calibration, and manual data entry. CGM wearables design is complex; therefore, proper training is essential, as articulated by the study participants and this aligns with other diabetes management studies.

PGHD consistency and interpretation

CPs addressed consistency challenges during the data processing while SPs discussed data inconsistency at the beginning of PGHD collection. CP3, CP4, and CP6 said that they did not limit patients to using a specific model of wearable device for CGM data collection, “There are number of devices and we really leave it up to them to decide what device they want. We do not recommend one over the other” [CP4].

However different platforms present reports in different formats. All SPs suggested allowing PGHD collection from various wearables while providing specified interfaces to connect these data to EMRs to present data in a single format. IP1 suggested that only HL7 compliant devices could ensure consistency in collected data and ultimately facilitate information interpretation for healthcare providers.

All three participant groups were concerned with problems in report interpretation in the use of PGHD for decision-making by healthcare providers. This need for better integration with EMRs and standardised presentation formats is raised elsewhere, both for clinicians and for patients to co-interpret data.

PGHD relevancy

Data relevancy was considered by CPs only in the present study. CPs stated challenges with data overload in reports which would make it difficult to prioritize information. CP3, CP7, and CP8, raised the need for guidelines to prioritize the most relevant and essential data for patient care among the vast amount of information that appears in the reports. SP4 and SP7 mentioned that the lack of contextualization reduced the meaningfulness of report information.

PGHD timeliness

Immediate access to data for both patients and healthcare providers could improve detecting behavioral patterns that impact clinical measurements and improve self-care. This was raised by CPs and SPs. One SP participant gave an example of their pilot program in which data were transferred on a near real-time basis and which assisted in discovering habits that would not have been identified otherwise: “I'm noticing a spike in your blood glucose every morning but I'm not seeing any eating habits changing, I'm not seeing activity changes. So, what's causing this spike? Well it came out of my conversation that every night he would have just a handful popcorn or handful of chips; he was eating when he is watching TV or watching the game with the kids. It was an absent-minded habit but because of the data, because of that conversation, and the time they have to connect from that point, he was able to stop that behavior, lowered his A1C, and today his A1C is down nearly two points and he's lost 50 pounds. So that's the way in which you can personalize the treatment throughout the time and prioritize patient data” [SP3].

LIMITATIONS

This study does not directly channel the patient voice. Further work will contain wider group of PGHD stakeholders in reviewing our findings from this study and in gathering their perspectives on PGHD. The full spectrum of consumer and medical wearables is not represented among the participants we interviewed, however some of the most widespread devices were considered. Moreover, the current study concentrated on primary use of PGHD and thus excluded secondary users of these data for other purposes, such as payers, data scientists, and researchers. Further stages of this project will involve these groups of stakeholders in PGHD quality.

CONCLUSION

The rise in PGHD production through the proliferation of health wearables is becoming an essential part of personalizing patient-centered care and could play an important role in providing data to inform precision medicine initiatives. PGHD can augment in-patient clinical data collection and provide a more comprehensive view of a patient’s status, showing environmental and behavioral factors affecting a patient’s specific health condition. However, to achieve this will require improvements in data management, as well as in quality assurance of the data as they flow.

To establish PGHD in RPM as an integral part of personalized, precision healthcare, we need more effective collaboration among PGHD stakeholders, a wider appreciation of human factors, device design factors, and readiness factors in healthcare organizations at present, and implementation of systematic PGHD management. This move will be aided by shared guidance on good practice. This study is part of a larger project that will develop practical guidance that can be trialed in healthcare settings with this aim.

FUNDING STATEMENT

This project is funded through a University of Melbourne PhD scholarship.

CONTRIBUTORSHIP STATEMENT

The corresponding author has a substantial contribution to the conception and design of the work; as well as the acquisition, analysis, and interpretation of data; and in drafting the article and revising it critically for important intellectual content. The other three authors contribute to the conception and design of the work, revising it critically for important intellectual content, and final approval of the version to be published.

COMPETING INTERESTS STATEMENT

The authors have no competing interests to declare.