Measuring the outcomes of using person-generated health data: a case study of developing a PROM item bank.

INTRODUCTION: Patient-reported outcome measures (PROMs) allow patients to self-report the status of their health condition or experience independently. A key area for PROMs to contribute in building the evidence base is in understanding the effects of using person-generated health data (PGHD), and using PROMs to measure outcomes of using PGHD has been suggested in the literature. Key considerations inherent in the stroke rehabilitation context makes the measurement of PGHD outcomes in home-based poststroke rehabilitation, which uses body-tracking technologies, an important use case.
OBJECTIVE: This paper describes the development of a preliminary item bank of a PROM-PGHD for Kinect-based stroke rehabilitation systems (K-SRS), or PROM-PGHD for K-SRS.
METHODS: The authors designed a method to develop PROMs of using PGHD, or PROM-PGHD. The PROM-PGHD Development Method was designed by augmenting a key PROM development process, the Qualitative Item Review, and follows PROM development best practice. It has five steps, namely, literature review; binning and winnowing; initial item revision; eliciting patient input and final item Revision.
RESULTS: A preliminary item bank of the PROM-PGHD for K-SRS is presented. This is the result of implementing the first three steps of the PROM-PGHD Development Method within the domains of interest, that is, stroke and Kinect-based simulated rehabilitation.
CONCLUSIONS: This paper has set out a case study of our method, showing what needs to be done to ensure that the PROM-PGHD items are suited to the health condition and technology category. We described it as a case study because we argue that it is possible for the PROM-PGHD method to be used by others to measure effects of PGHD utilisation in other cases of health conditions and technology categories. Hence, it offers generalisability and has broader clinical relevance for evidence-based practice with PGHD. This paper is the first to offer a case study of developing a PROM-PGHD. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Patient-reported outcome measures (PROMs) offer a standardised approach to evaluating and improving healthcare services by enabling patients to contribute to more precise evaluation of the effects of various health interventions; and they contribute to improving the evidence base in various areas of clinical care.

Utilisation of person-generated health data (PGHD) by patients promotes participatory health, as it has been suggested to increase their engagement, improve health management coordination with their care providers and increase their sense of social support and connectedness.

Measuring PGHD outcomes in home-based poststroke rehabilitation, which uses body-tracking technologies, is an important use case due to key considerations inherent in the stroke rehabilitation context.

This paper has demonstrated a case study of our PROM-PGHD Development Method, using Kinect-based stroke rehabilitation systems (K-SRS) as the case study, resulting in a preliminary item bank of PROM-PGHD for K-SRS.

The PROM-PGHD method may be used by others to measure effects of PGHD utilisation in other cases of health conditions and technology categories, and therefore has broader clinical relevance for evidence-based practice with PGHD.

Introduction

Patient-reported outcome measures (PROMs) allow patients to self-report the status of their health condition or experience independently.1–3 As such, PROMs enable patients to contribute to more precise evaluation of the effects of various health interventions, and they contribute to improving the evidence base in various areas of clinical care.4 PROMs offer a standardised approach to evaluating and improving healthcare services, and this is highlighted by key national projects to develop suites of PROMs for various health conditions in the USA, Europe and Australia.5

A key area for PROMs to contribute in building the evidence base is in understanding the effects of using person-generated health data (PGHD).6 PGHD are created, recorded and analysed by people, who are monitoring their health outside of a clinical care setting. They include health, wellness and other biometric data produced from technologies such as mobile applications, activity tracking devices and simulated rehabilitation technologies.7–9 Utilisation of PGHD by patients promotes participatory health, as it has been suggested to increase their engagement, improve health management coordination with their care providers and increase their sense of social support and connectedness.10–15 When patients better understand their illness, it may make them more active in improving their health behaviour.16

Using PROMs to measure outcomes of using PGHD has been suggested.9 A PROM of using PGHD, or PROM-PGHD, would allow patients to directly self-report their health outcomes or status as result of accessing and using their own PGHD. This may generate a deeper understanding of how PGHDs may impact patients’ health status and quality of life, and has significance in an era of increasing remote wearable and mobile patient monitoring. Similar to PROMs being used as a complement to other health outcomes indicators,5 PROM-PGHDs could also be used to complement existing patient-reported, and clinician-reported, outcome measures. This could contribute to a more accurate and comprehensive assessment of patients’ experiences of using PGHD from existing and new health information technologies. Consequently, PROM-PGHDs may offer a deeper understanding of the health outcomes and related impacts of those technologies.

Measuring PGHD outcomes in home-based poststroke rehabilitation, which uses body-tracking technologies, is an important use case due to key considerations inherent in the stroke rehabilitation context.8 These may include its high cost over a long period of time; difficulties in access to therapy17; the complexity of care required18 19 and the need for patients to undertake frequent, repetitive movement exercises appropriate to their condition to support improved health outcomes.20 21 Therefore, more convenient, practical and effective options for patients are needed, which technology interventions may provide. Simulated rehabilitation systems, for example, using Kinect (Microsoft, Redmond, Washington, USA) provide patients with simulated activities of daily living.22 Stroke therapy benefits from such systems have been previously reported.22–25

As patients use such systems, they produce PGHD in the form of therapeutic progress data. Those PGHD have the potential to be used by clinicians and by patients themselves to monitor and evaluate patients’ recovery more consistently.22 24 Similar to how PROMs allow for a more holistic evaluation of the effects of various health services and interventions,4 a PROM-PGHD for simulated poststroke rehabilitation technologies could provide a more precise assessment of those systems, and also increase understanding of how those systems impact the health status of patients.6

Objective

This paper describes the development of a preliminary item bank of a PROM-PGHD for Kinect-based stroke rehabilitation systems (K-SRS), or PROM-PGHD for K-SRS.

Methods

In response to the lack of a systematic way for patients to measure and self-report health effects they experience from using their PGHD—whether those effects are positive, negative or nil—the authors designed a method to develop PROMs of using PGHD, or PROM-PGHD. The PROM-PGHD Development Method was designed by augmenting a key PROM development process, the Qualitative Item Review,26 and follows PROM development best practice. Table 1 presents the steps of the PROM-PGHD Development Method.

Table 1

Activities of the PROM-PGHD Development Method

#	Step name	Activities
1	Literature review to identify existing items	This is a search of the literature surrounding established PROMs within the target health condition, and relevant self-reported measures within the target technology category. This considers the socio-technical context of the target domains, and serves as the foundation for building proposed outcome measure items. Items that represent the range of domain-relevant experiences are identified.
2	Binning and winnowing	Binning involves categorising the selected items according to the effects of PGHD utilisation that they could measure. Winnowing excludes items that would not be able to measure effects of using PGHD. It also excludes items based on QIR criteria: items that are too narrow, disease-specific, redundant or confusing.26
3	Item revision process	In this step, retained items are revised according to QIR criteria to ensure: consistency of item response options; similarity in wording contexts; concise and simple items; item independence from other questions and that items encourage use of available response options to reduce cognitive burden on respondents.26 Additionally for PROM-PGHD, some item terminologies may be revised to better match the target health condition and technology category, to make them more specific to the target domains.
4	Focus groups and interviews with target patient cohort	This step ensures that patient input is elicited in the development of item banks. It enables the developers to understand the vocabulary and thinking processes of the target group, and to gather feedback on individual items. It is aimed to bridge relevant gaps between the current items and the target domain or concepts to be measured. This may also highlight other measurement areas expressed by patients that are not covered in the preliminary item bank.
5	Final item revisions	Based on patient input gathered from the previous step, the items are revised once more. They are tested with the Lexile Analyser (MetaMetrics, Durham, North Carolina) to assess their readability; to catch items that may still be difficult to read. After the revisions are completed, field testing on the items may begin, in order to understand their quantitative characteristics.

PGHD, person-generated health data; PROMs, patient-reported outcome measures; QIR, Qualitative Item Review.

This paper describes the use of the first three steps of the method, to develop a preliminary item bank of a PROM-PGHD for K-SRS. Validation and subsequent revision of the preliminary PROM-PGHD through the methods’ last two steps, that is, eliciting patient input,6 and final item revision, are reported elsewhere. The following sections are organised accordingly.

Step 1: literature review

In this step, a literature review relating to the two domains of the focus area, the health condition and the technology category, was conducted. This is to consider the socio-technical system context of the focus area, necessary for designing a PROM-PGHD that is appropriate for the needs of the patient cohort. Based on the objectives of this paper, the target health condition of this paper is stroke, and the technology category is Kinect-based simulated rehabilitation. Through the literature review, existing PROM items within the domains of interest were identified.

A consideration in this step is that PROMs are not generally found in the literature reporting on research in the technology category. However, this research may report patient experience or satisfaction with a health intervention measured in ways that are similar to PROMs.27 Hence, a variety of measures of satisfaction or experience from the literature in the technology category are included in the identification of PROMs for PGHD.

The authors conducted an extensive literature review, which examined the extent of PGHD utilisation in 41 included studies of Kinect-based simulated rehabilitation systems for stroke; full details appear here.8 The review identified existing PROMs within the poststroke health condition, and self-reported measures within the Kinect-based simulated rehabilitation technology category. These are listed in table 2.

Table 2

PROMs identified through the first step of the PROM-PGHD Development Method, the literature review

Study	PROM or similar instrument
Bird et al, 201639	Borg rating of perceived exertion scale.Visual Analogue Scale (pain and fatigue).5-point Likert scales on enjoyment, and on perceived benefit.
Proffitt and Lange, 201540	Game Experience Questionnaire from IBM.System Usability Questionnaire.Activities-Specific Balance Confidence Scale.Stroke-Specific Quality of Life Scale.
Song and Park, 201541	Beck Depression Inventory.Relationship Change Scale.
Allen et al, 201342	Mixed Reality Experience Questionnaire.
Kizony et al, 201343	Motor Activity Log.
Kairy et al, 201644	Stroke Impact Scale (quality of life).Personal log of exercise time, feelings (motivation/appreciation), other services received and adverse events.

PGHD, person-generated health data; PROMs, patient-reported outcome measures.

The end of step one resulted in a range of possible PROMs or similar instruments to capture both the post-stroke and the simulated rehabilitation domains of interest. The individual items of the PROMs were analysed for appropriate ‘binning’ or ‘winnowing’ as described in the next step.

Step 2: binning and winnowing

In this step, individual items of the identified PROMs were assessed for inclusion to the preliminary PROM-PGHD, and then categorised in a process called ‘binning’, explained next. ‘Winnowing’ is the process of assessing the PROM items and determining whether they should be ‘winnowed’, or removed. Many of the items were removed as they could not be used to measure the effects of using PGHD. The criteria for winnowing items are as follows: (1) item content was inconsistent with the PROM-PGHD objective of measuring effects of using PGHD; (2) the item content was too specific to be applicable elsewhere, for example, it was too disease-specific or (3) items were redundant or confusing.

Table 3 shows examples of items that were removed from the winnowing process, and the reasons for their removal. Meanwhile, table 4 shows the list of PROM items that were retained after winnowing, and their reasons for inclusion.

Table 3

Examples of items removed from the winnowing process

PROM	Removed item	Reason for removal
Game experience questionnaire	Overall, I am satisfied with the ease of completing the tasks in this scenario.	Inconsistent with the objective of measuring effects of PGHD utilisation.
Beck depression inventory	I have not noticed any recent change in my interest in sex.	Too narrow or specific.
Stroke-specific quality of life scale	Did you have trouble walking?	Disease-specific.
System usability questionnaire	I found the product very awkward to use.	Inconsistent with the objective of measuring effects of PGHD utilisation.
Relationship change scale	Within the last 4 weeks, I feel my friend views me as a satisfactory friend: (1) much less; (2) less; (3) no change; (4) more; (5) much more.	Too narrow or specific.

PGHD, person-generated health data; PROMs, patient-reported outcome measures.

Table 4

Alignment of identified PROM items with PGHD effects, and reasons for their inclusion

Included PROM item/s	Reason for inclusion	Alignment with PGHD effects
Game Experience questionnairepoststudy items: The system gave error messages that clearly told me how to fix my problems. Whenever I made a mistake using the system, I could recover easily and quickly. Response options: scale of 1 (strongly agree) to 7 (strongly disagree), and N/A.	These items could rate whether/how patients correct an action or behaviour, depending on the ease of resolving issues with the way PGHD is provided to them.	Effect 1:Influence health-related behavioural or attitude changes in patients.
Beck depression inventory:A (Mood)0 I do not feel sad.1 I feel blue or sad.B (Pessimism)0 I am not particularly pessimistic or discouraged about my future.1a I feel discouraged about my future.C (Sense of failure)0 I do not feel like a failure.1 I feel I have failed more than the average person.G (Self-hate)0 I do not feel disappointed in myself.1a I am disappointed in myself.Response options: among a list of options, tick a statement if true.	These items could rate whether/how patients’ attitude may be affected by access to their PGHD.
Stroke-specific quality of life scale, mood items: I felt hopeless about my future. I was discouraged about my future. I had little confidence in myself. Response options: strongly agree, moderately agree, neither agree nor disagree, moderately disagree or strongly disagree.
Game experience questionnairepoststudy items: The information (such as online help, onscreen messages and other documentation) provided with this system is clear. It was easy to find the information I needed. The information provided for the system was easy to understand. The information was effective in helping me complete the tasks and scenarios. After-scenario items:3. Overall, I am satisfied with the support information (online help, messages, documentation) when completing the tasks.Response options: scale of 1 (strongly agree) to 7 (strongly disagree), and N/A.	These items could rate whether/how patients completed tasks depending on the quality of PGHD provided.	Effect 2:Influence patient management of their own care, due to changes in feelings about their health status.
System usability questionnaire: I found the product unnecessarily complex. I think that I would need the support of a technical person to be able to use this product. Response options: scale of 1 (strongly disagree) to 5 (strongly agree).	These items could rate whether/how patients’ understanding of PGHD was affected by the interface to and presentation of their data.
Relationship change scale:1. Within the last 4 weeks, my satisfaction with myself as a person has become: (1) much less; (2) less; (3) unchanged; (4) greater; (5) much greater.	This item could rate the personal satisfaction of the patient after accessing PGHD.
Game Experience Questionnairepoststudy item:18. This system has all the functions and capabilities I expect it to have.Response option: scale of 1 (strongly agree) to 7 (strongly disagree), and N/A.	This item could rate whether/how patients’ personal preferences for access to their data were supported by the system.	Effect 4:Facilitate personal care goals.

N/A, not available; PGHD, person-generated health data; PROMs, patient-reported outcome measures.

The retained items after winnowing were categorised in a process called ‘binning’. Binning, a term used in statistics to mean grouping items together, is the process of aligning the retained PROM items with reported effects on patients who have used PGHD. As an efficient way of targeting reported effects for this purpose, articles from a major journal special issue on PGHD7 were analysed inductively28; to categorise ways used in them to describe reported effects of using PGHD. The derived themes are a representative sample of PGHD utilisation effects from a variety of health information technologies, for different health conditions. These effects are listed below:

Influence health-related behavioural or attitude changes in patients.29

Influence patient management of their own care, due to changes in feelings about their health status.30

Influence interest in their care processes.14 30 31

Facilitate personal care goals.30 32

Influence relationship with care providers.10 11 30 31

The PROM items retained after winnowing were matched against these categories of effects of PGHD utilisation. Table 4 shows the PROMs identified from step 1, and their outcome measure items, with corresponding response options that were retained after the winnowing process. It also lists the reasons for the items’ inclusion. The ‘Reason for inclusion’ column describes why the items may be appropriate in measuring self-reported outcomes of patients’ utilisation of PGHD. The final column shows the alignment of the retained items with the thematically derived PGHD utilisation effects after the binning process. Only effects 1, 2 and 4 had items binned, or aligned with them.

PGHD has been used to describe data that have been generated and recorded by people, and interpreted by them,7 that is, people are accessing and using their own health information. Thus, PROM items that use the terms data and information both were included.

Step 3: item revision

The retained PROM items from step 2 were selected from pre-existing PROMs identified in the literature; as such they were not worded consistently, and their response options differed. To ensure that the resulting PROM-PGHD can be presented as one coherent test, and to reduce cognitive burden on respondents, the retained PROM items were revised in this step.

They were revised, where necessary, to better match the target health condition and technology category. Further revisions may also occur to ensure that the different PROM item response options are consistent; their content has similar wording; are concise and simple; are able to stand alone separately from the other questions and are worded to encourage use of available response options.26

A consequence of collecting several existing items from PROM instruments is the resulting variability of response options present.26 However, there is a lack of empirical evidence that any particular set of response options is better than others. Optimal response options may vary based on the individual items in question.26 Thus, to ensure that the resulting PROM-PGHD is capable of measuring the experiences of future respondents within the target domains, some response options of the retained items also were revised to include additional response types. This is to provide patients with varying response options to comment on in the fourth step. The fourth step is the process of eliciting patient input on the preliminary PROM-PGHD item bank, which is out of scope for this paper, and is reported elsewhere.

Table 5 shows the included PROM items from table 4. It then depicts any revisions conducted on the PROM items and their corresponding response options. Furthermore, it provides the reason/s for the revision.

Table 5

Revision of identified items

Included PROM item/s	Revised PROM item/s Changes are italicised	Reason for revision/s
Game experience questionnairepoststudy items: The system gave error messages that clearly told me how to fix my problems. Whenever I made a mistake using the system, I could recover easily and quickly. Response options: scale of 1 (strongly agree) to 7 (strongly disagree), and N/A.	9. The system gave error messages that clearly told me how to fix my posture or movement.Response option: always, often, sometimes, rarely or never.10. Whenever I made a movement mistake, I could recover easily and quickly.Response option: without any difficulty, with a little difficulty, with some difficulty, with much difficulty or unable to do.	Item contents and response options revised:Item contents changed to better reflect target health condition and technology category.Response options changed to match contents better, from levels of agreement to levels of frequency and capability. Chosen scales follow best practice uniform response options for frequency and capability.26
Beck depression inventory:A (Mood)0 I do not feel sad.1 I feel blue or sad.B (Pessimism)0 I am not particularly pessimistic or discouraged about my future.1a I feel discouraged about my future.C (Sense of failure)0 I do not feel like a failure.1 I feel I have failed more than the average person.G (Self-hate)0 I do not feel disappointed in myself.1a I am disappointed in myself.Response options: among a list of options, tick a statement if true.	A (Mood)0 I felt okay. 1 I felt blue or sad.B (Pessimism)0 I was fairly upbeat about my progress.1a I felt pessimistic about my progress.C (Sense of failure)0 I felt like I was succeeding fairly well. 1 I felt that I had failed more than the average personG (Self-hate)0 I felt satisfied with myself.1a I was disappointed in myself.Response options: among a list of options, tick a statement if true.	Item contents revised:Item contents changed to better reflect rehabilitation context of the health condition.Changed from present to past tense for consistency with other items.
Stroke-specific quality of life scale, mood items: I felt hopeless about my future. I was discouraged about my future. I had little confidence in myself. Response options: strongly agree, moderately agree, neither agree nor disagree, moderately disagree or strongly disagree.	1a. I felt hopeless about my progress. 1b. I felt hopeful about my progress.2a. I was discouraged about my progress. 2b. I was encouraged about my progress. 3a. I had little confidence in myself. 3b. I was pretty confident in myself.Response options: among a list of options, tick a statement if true.	Item contents and response options revised:These items, similar with the Beck Depression Inventory items, could indicate how patients’ access to their PGHD may affect their mood.Item contents changed to better reflect stroke rehabilitation context.Response options changed for consistency in presenting these items with items of the Beck Depression Inventory.
Game experience questionnairepoststudy items: The information (such as online help, onscreen messages and other documentation) provided with this system is clear. It was easy to find the information I needed. The information provided for the system was easy to understand. The information was effective in helping me complete the tasks and scenarios. Response options: scale of 1 (strongly agree) to 7 (strongly disagree), and N/A.	The online help, onscreen messages and other documentation that that this system gave me made it clear to me what to do. It was easy for me to find the information I needed from the system. The information provided by the system was easy for me to understand. The information from the system helped me to complete the tasks and scenarios. Response option: scale of 1 (strongly agree) to 6 (strongly disagree), and N/A.	Item response options revised:Numeric response options revised to six, from seven, to be consistent with best practice uniform response options for rating scales.26 This also reduces cognitive burden on respondents.
System usability questionnaire: I found the product unnecessarily complex. I think that I would need the support of a technical person to be able to use this product. Response options: scale of 1 (strongly disagree) to 5 (strongly agree).	I found that the data about me were more complex than I needed. I thought that I would need the support of a technical person to be able to understand my data. Response options: among a list of options, tick a statement if true.	Item contents and response options revised:Item contents revised to reflect patient access to data.Response option type was varied to elicit patient input on acceptability of response formats.26
Game experience questionnaireafter-scenario items:3. Overall, I am satisfied with the support information (online help, messages, documentation) when completing the tasks.Response options: scale of 1 (strongly agree) to 7 (strongly disagree), and N/A.	3. Overall, I was satisfied with the way the information (online help, messages, documentation) supported me when completing the tasks. Response option: among a list of options, tick if true.	Item contents and response options revised:Item contents changed to past tense, for consistency with other items.Response option type was varied to elicit patient input on acceptability of response formats.26
Relationship change scale:1. Within the last 4 weeks, my satisfaction with myself as a person has become: (1) much less; (2) less; (3) unchanged; (4) greater; (5) much greater.	1. Within the last 4 weeks, my satisfaction with myself as a person has become: (1) much less; (2) somewhat less; (3) unchanged; (4) somewhat more; (5) much more.	Item response option revised:Response option improved for clarity.
Game experience questionnairepoststudy item:18. This system has all the functions and capabilities I expect it to have.Response option: scale of 1 (strongly agree) to 7 (strongly disagree), and N/A.	18. This system has shown me all rehabilitation information I expect to have about myself. Response option: true or false.	Item content and response option revised:Item changed to better reflect rehabilitation context.Response option type was varied to elicit patient input on acceptability of response formats.26

N/A, not available; PGHD, person-generated health data; PROMs, patient-reported outcome measures.

The final thing to do in this step is to group the revised items according to their alignment with a PGHD effect. For this item bank, they are also numbered as a group according to their response options, that is, true/false statements, rating scales and multiple choice questions. The result is a preliminary item bank of a PROM-PGHD for K-SRS.

Results

Figure 1 presents the preliminary item bank of the PROM-PGHD for K-SRS. This is the result of implementing the first three steps of the PROM-PGHD Development Method within the domains of interest, that is, stroke and Kinect-based simulated rehabilitation.

Figure 1

Preliminary PROM-PGHD item bank for K-SRS. This figure presents the preliminary item bank of the PROM-PGHD for K-SRS. This is the result of implementing the first three steps of the PROM-PGHD Development Method within the domains of interest, that is, stroke and Kinect-based simulated rehabilitation. The items were first grouped according to the PGHD effects they aligned with, and then grouped further according to their response types. K-SRS, Kinect-based stroke rehabilitation systems; N/A, not available; PGHD, person-generated health data; PROMs, patient-reported outcome measures.

The items were categorised into the three PGHD effects that they were aligned with, represented as ‘sections’. The items under the second section (aligned with the second PGHD utilisation effect) could indicate how PGHD influences patient decisions on the management of their own care, due to how they felt about their health status. Thus, this effect was rephrased as the ‘self-management of care’ section. The item under the third section, aligned with the fourth effect, could indicate whether patients have sufficient access to their PGHD in a way they prefer, to facilitate personalised self-care strategies. Hence, this effect was rephrased as the ‘personalisation’ section.

Nine items fell under the first section on ‘behavioural or attitude changes’, eight items under the second section on ‘self-management of care’ and one item under the third section ‘personalisation’. All items under each section were then grouped according to their response types, to improve the flow of items when read.

Discussion

This paper has demonstrated a case study of our PROM-PGHD Development Method, using K-SRS as the case study. The result is a preliminary item bank of PROM-PGHD for K-SRS. It demonstrated the implementation of each step of the PROM-PGHD development process.

The next step is to present the developed item bank to patients, to elicit their input on the items themselves and to discuss their experiences with accessing and using PGHD. This is to gather concepts that may not have been covered by the current item bank, and is the fourth step of the PROM-PGHD method. This step is reported elsewhere.6

An interesting consideration discovered in this case study is the necessity of including, in the identification of PROMs, measures from the technology category to self-report satisfaction or experience. This is to ensure coverage of relevant items within the socio-technical context of the domains being measured. It is still a disciplined approach to the selection of outcome measures within the technology category, as the process identifies—through the literature review—those measures that have been used in a K-SRS setting. This is a unique and valuable aspect to the PROM-PGHD development process that future developers of a PROM-PGHD will need to consider.

Using PROMs to measure outcomes of using PGHD has been suggested.9 PROMs allow for a more holistic evaluation of the effects of various health services and interventions.4 Similarly, a PROM-PGHD would allow for a more precise, patient-centred assessment of such systems; and may increase understanding on how they could impact the health status of patients. It promotes participatory health within the K-SRS domain as it recognises the value of the patient experience in the assessment and evaluation of PGHD, and the technologies that produce them.33 PROMs may be used to understand the impact healthcare services have on the status and quality of life of patients.2 Similarly, it is hoped that the item bank would, in the future, assist clinicians in selecting appropriate K-SRS based on PGHD utilisation effects on patients; and for patients to understand how certain K-SRS could affect their management of their own health. Moreover, the method’s applicability for a variety of health conditions and technology categories make it broadly relevant for evidence-based practice in clinical work with PGHD.

This paper is the first to offer a case study of developing a PROM-PGHD for a target health condition and technology category. While there are studies that present the development of PROMs of a health condition,3 34 and measures to self-report experience or satisfaction with health technologies,27 35 36 there have been no studies presenting the development of a PROM of using PGHD.

Limitations

It was necessary to revise the content of existing PROM items identified from the literature review, due to the collection of several, existing items from PROM instruments, resulting in a variety of response options present.26 Moreover, to elicit patient input on acceptability of response formats, some of the response option types were revised. While revisions of the response options for uniformity is considered minor and unlikely to alter the items substantially,26 we recognise that changing the content of the items may introduce changes to the items’ function. This revision is essential, however, to develop a preliminary item bank to measure self-reported outcomes of PGHD utilisation. This process still follows best practice of searching the literature for existing concepts and items, and eliciting patient input.1 26 37 38 We believe this is preferable to starting the item development completely from scratch. Nonetheless, the suitability of the items is expected to be improved in the next step, where patient perspectives on their PGHD utilisation experience are gathered.

As described, the method used to identify the PGHD utilisation effects in step 2 (binning and winnowing) was an inductive thematic analysis of a recent, authoritative source (JAMIA special issue on PGHD7). The special issue compiled a range of applications and effects of PGHD across a variety of health conditions and technology categories. However, the list may not have covered all effects reported in the literature. The effects that should be measured however, will be verified and/or supplemented by patients in the next step. Open-ended questions will be asked of the patients around their experience of accessing and using PGHD from a K-SRS, to elicit any effects that may not have been covered in the initial list in step 2. The authors have already reported on one such discussion with patients here.6

While the preliminary item bank of the PROM-PGHD for K-SRS was organised as described for the purpose of presentation here, this is not its final, or complete form. The objective of this paper was to describe a formal process of developing a preliminary item bank, which could then be presented to patients to elicit their input on the items’ readability, appropriateness of wording and relevance to their experiences of accessing and using their PGHD in a K-SRS. Moreover, because the items were categorised according to the effects they align with, the resulting item bank seems unbalanced in terms of the number of items under each section. The third section in particular has only one item. It should also be noted that only PGHD utilisation effects 1, 2, and 4 had items binned or aligned with them. Consequently, effects 3 and 5, which measure changes in patient engagement with formal care, were not represented in the items identified from the literature. This will be a key area of enquiry, which will be explored in the fourth step of the PROM-PGHD method.

Conclusion

This paper has set out a case study of our method, showing what needs to be done to ensure that the PROM-PGHD items are suited to the health condition and technology category. We described it as a case study because we argue that it is possible for the PROM-PGHD method to be used by others to measure effects of PGHD utilisation in other cases of health conditions and technology categories. Hence, it offers generalisability and has broader clinical relevance for evidence-based practice with PGHD.