EULAR points to consider for the development, evaluation and implementation of mobile health applications aiding self-management in people living with rheumatic and musculoskeletal diseases.

Background: Mobile health applications (apps) are available to enable people with rheumatic and musculoskeletal diseases (RMDs) to better self-manage their health. However, guidance on the development and evaluation of such apps is lacking.
Objectives: The objective of this EULAR task force was to establish points to consider (PtC) for the development, evaluation and implementation of apps for self-management of RMDs.
Methods: A systematic literature review of app content and development strategies was conducted, followed by patient focus group and an online survey. Based on this information and along with task force expert opinion, PtC were formulated in a face-to-face meeting by a multidisciplinary task force panel of experts, including two patient research partners. The level of agreement among the panel in regard to each PtC was established by anonymous online voting.
Results: Three overarching principles and 10 PtC were formulated. Three PtC are related to patient safety, considered as a critical issue by the panel. Three are related to relevance of the content and functionalities. The requirement for transparency around app development and funding sources, along with involvement of relevant health professionals, were also raised. Ease of app access across ages and abilities was highlighted, in addition to considering the cost benefit of apps from the outset. The level of agreement was from 8.8 to 9.9 out of 10.
Conclusion: These EULAR PtC provide guidance on important aspects that should be considered for the development, evaluation and implementation of existing and new apps. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

There is unmet need to standardise the development of mHealth apps for self-management in rheumatic and musculoskeletal diseases.

This work aimed at developing guidance in for the development, evaluation and implementation of self-management apps in rheumatology.

These points to consider will serve as a basis for the evaluation and endorsement of existing apps or the development of future apps aiding self-management by people living with rheumatic and musculoskeletal diseases.

Introduction

Mobile health (mHealth) technologies have the capacity to transform the mode and quality of healthcare, allowing people to take a more proactive role in their health and well-being.1 This is increasingly achieved using smart phone technology (sometimes linked to wearable sensors) that gathers health-related data via medical application interfaces.2 By enabling people to access and share their health information, mHealth has the capacity to empower individuals to take a more active role in self-managing their health and well-being.3 The number of available mobile health applications (apps) has exponentially grown over the past few years and so has the number of users.4 Apps are designed for a wide range of users, from healthy individuals to people living with long-term conditions including, but not limited to, rheumatic and musculoskeletal diseases (RMDs),5 psychiatric,6 respiratory,7 cardiovascular diseases8 and diabetes mellitus.9

The use of such apps is becoming more and more popular among people living with long-term conditions. Aside from the increasing popularity of apps among people with long-term conditions, mHealth apps can support disease assessment by health professionals and enhance doctor–patient interactions. For example, they have been useful tools to facilitate the calculation of disease activity scores, and monitor patients through the collection of clinical and laboratory parameters.10 A survey performed among people living with RMDs (429 respondents) revealed that around 50% were aware of the existence of such apps.11 This corresponds with previous works performed in different specialties. For example, a previous survey among people living with type 1 diabetes mellitus found that 40% of respondents reported being aware of self-management apps.12 Alongside the value of apps in patient self-empowerment, they can also facilitate patient communication with health professionals,13 improve treatment adherence,14 provide condition-specific education and enable remote condition monitoring.15 16

Recent studies assessing the quality of rheumatoid arthritis apps showed that most apps were not achieving high-quality scores and that data on funding and origin were frequently unavailable.17 18 These studies raise concerns and highlight the need for benchmarking the quality of mHealth apps to ensure patient safety, among other aspects. Scientific bodies, such as the EULAR, should contribute to quality control measures of existing apps, fulfilling this unmet need. Therefore, the aim of this project was to develop EULAR points to consider (PtC) for the development, evaluation and implementation of apps aiding self-management of RMDs.

Methods

The task force (TF) was led by two convenors (AN and FB) and two methodologists (EN and LG) and the fellow (AN) according to the 2014 updated EULAR Standardised Operating Procedures.19 This was modified with a single face-to-face meeting to take place, with part of the budget dedicated to focus group realisation. The TF consisted of 19 members from 10 countries across Europe. The panel included 15 rheumatologists, 2 patient research partners and 2 health professionals (occupational therapists). The TF also included two representatives from the EMerging EUlar NETwork.

Preparatory TF work: systematic literature review and mixed-methods study

A systematic literature review (SLR) was conducted by the fellow (AN) under the supervision of the methodologists.20 The SLR objective was to obtain detailed information on existing mHealth apps to aid self-management among people living with RMDs, focusing on content and development methods. Due to the peculiarities of the topic, and the importance of patient input in this domain, a second part of this work focused on a mixed-methods study (including a qualitative and a quantitative approach) to obtain an in-depth understanding of the topic through direct patient feedback. We first conducted a patient focus group in the UK, the findings of which informed the design of an international patient survey. This survey provided wider insights into the needs, views, experiences and preferences in mHealth apps to aid self-management among people living with RMDs.11 This preparatory material was used as a basis for the formulation of the draft recommendation list by the TF steering group (fellow, convenor, comethodologists).

Face-to-face meeting

The results of the SLR and mixed-methods study were sent to the TF before the face-to-face meeting and were subsequently presented to the TF during the face-to-face meeting in November 2018. This formed the basis of in-depth discussions by the TF. The presented evidence was followed by the presentation of preliminary draft recommendations to the TF one by one, along with the corresponding evidence. During the meeting, the draft recommendations were reformulated, and some themes were merged. Discussions culminated in the formulation of a definition for mHealth apps, overarching principles and PtC. Consensus on the final wording of overarching principles and PtC was considered final if >75% of the TF members voted in favour of the PtC at the first round, >67% at the second >50% at the third round.

Level of agreement

The Oxford Levels of Evidence from Ia (Evidence from meta-analysis of randomised controlled trials) to IV (Evidence from expert committee reports or opinions or clinical experience of respected authorities, or both) were applied for each PtC.21 Subsequent strength of statement was assigned ranging from A (directly based on level I evidence) to D (directly based on level IV evidence or extrapolated recommendations from level I, II or III evidence). The percentage of agreement during the meeting was recorded for the overarching principles, through a vote by hand raise.

Subsequently, members of the TF were asked, by email, to provide their level of agreement for each PtC, through anonymous voting on an 11-point Likert scale from 0 to 10 (0: completely disagree, 10: completely agree). The mean and SD of the level of agreement with each statement were calculated.

Results

Target population

These PtC are intended to assist in evaluating the quality of existing apps, while guiding the development, evaluation and implementation of future apps aiding self-management among people living with RMDs.

The target audience are apps users, including patients, parents/carers, health professionals, rheumatologists, patient organisations, scientific societies, app developers and regulatory agencies.

Overarching principles

A definition of apps and three overarching principles were formulated (table 1). The overarching principles were formulated based on the aspects felt as the most important by the TF.

Table 1

Three overarching principles and 10 points to consider (PTC) for the development, evaluation and implementation of apps to aid self-management among people living with rheumatic and musculoskeletal diseases (RMDs)

Overarching principles	Agreement (%)
Apps* for self-management support the health, well-being and empowerment of people living with RMDs.	100
Apps* require an overarching conceptual framework, which defines the target population and purpose of the app.	100
User privacy and safety are fundamental considerations for all apps* aimed at people living with RMDs.	100
PtC	Oxford level of evidence	Strength of statement	Level of agreementMean (SD)
1. The information content in self-management apps should be up to date, scientifically justifiable, user acceptable and evidence based where applicable.	Level 5	D	9.8 (0.4)
2. Apps should be relevant and tailored to the individual needs of people with RMDs.	Level 5	D	9.7 (0.5)
3. The design, development and validation of self-management apps should involve people with RMDs and relevant healthcare providers.	Level 5	D	9.8 (0.6)
4. There should be transparency on an app’s developer, funding source, content validation process, version updates and data ownership.	Level 5	D	9.9 (0.3)
5. Data collection as part of apps must adhere to all applicable regulatory frameworks, particularly data protection.	Level 5	D	9.9 (0.3)
6. Apps must not result in physical or emotional harm to people with RMDs.	Level 5	D	9.3 (1)
7. Apps could facilitate patient–healthcare provider communication and contribute to electronic health records or research.	Level 5	D	9.4 (0.9)
8. App design should consider accessibility of people with RMDs across ages and abilities.	Level 5	D	9.4 (0.9)
9. If a social network is an important component of an app, structures should be in place to ensure appropriate content moderation.	Level 5	D	9.5 (0.6)
10. The rheumatology community should consider the cost-benefit balance of apps before endorsement and/or promotion.	Level 5	D	8.9 (1.3)

*An app is a small programme that can be downloaded and installed on a mobile device. For the purpose of these PtC, the definition takes a focus on self-management of RMDs.

PtC, points to consider.

Apps for self-management have the clear mission of supporting health, well-being and empowerment of people living with RMDs.2 22 Those developing apps should follow a step-by-step framework in line with their aim and target population.23 24 This framework should be defined beforehand and should guide the development and evaluation phases, by involving relevant stakeholders and collecting relevant data in the pilot phase. This process should then be implemented in a validation phase25 in a collaborative approach with all relevant stakeholders.26

More importantly, it was felt as fundamental by the TF to emphasise the need of ensuring patient safety and data security.27 28 Patient safety is described as making no physical or emotional harm to patients.29 Data security refers to protection of data against unauthorised use. Those two concepts form the basis of regulations in the European Union (EU) and USA.30

Agreement was 100% by vote during the meeting for the three overarching principles.

Points to consider

Ten PtC were formulated. The level of evidence as well as strength of statement and level of agreement are provided for each overarching principles and PtC in table 1. The overall level of evidence ranged from 4 to 5, with a subsequent strength of statement from C to D. The level of agreement was high, with full agreement among the TF for overarching principles, and a mean ranging from 8.9 to 9.9 for each of the 10 PtC.

For these PtC, an app was defined as follows: an app is a small programme that can be downloaded and installed on a mobile device. For the purpose of these PtC, the definition takes a focus on self-management of RMDs.

PtC 1: the information content in self-management apps should be up to date, scientifically justifiable, user acceptable and evidence based where applicable

A ‘user’ is defined as a person living with RMDs and/or their families/carers. The TF emphasises that any medical information provided to people living with RMDs should be scientifically accurate and evidence based; however, this should be balanced to ensure that information can be understood and interpreted by people living with RMDs. Any medical content provided by self-management apps should be validated by appropriate experts.31 32 A previous review of existing rheumatology apps highlighted that the source of medical information was lacking in 40% of the screened apps.18 The TF emphasised the importance of citing sources whenever possible.

PtC 2: apps should be relevant and tailored to the individual needs of people with RMDs

Apps need to be tailored to the needs of people living with RMDs, in addition to encompassing different aspects of their conditions in a relevant manner. This principle should be applicable for both condition-specific and non-specific apps. The scope of apps should be clearly defined, and features included in apps should be relevant to the target audience. Patient priorities, for example, pain, function and fatigue, should be taken into account when designing apps.33 34

PtC 3: the design, development and validation of self-management apps should involve people with RMDs and relevant healthcare providers

The SLR informing this work highlighted that health professionals and patients were rarely involved in every stage of app development. Patients were usually involved in the test phase, but less frequently in the design phase. Moreover, when in place, the design phase rarely included a qualitative phase. Since patients are the target users, app development should be patient centred and driven directly by the needs and priorities of people living with RMDs. Moreover, as medical content is provided and medical data are usually collected in such apps, healthcare providers, in particular rheumatologists, should be involved in the development phase.31 32 Healthcare providers are defined as any physicians or other health professionals (eg, nurses, physiotherapists and occupational therapists) involved in the care of people living with RMDs.35

PtC 4: there should be transparency on an app’s developer, funding source, content validation process, version updates and data ownership

Important information such as the developer, funding source(s), advertisement and promotion, conflict of interest or date of last update were missing from the description of a significant number of apps.18 The TF considered that such information should be made publicly available for any app.

PtC 5: data collection as part of an app must adhere to all applicable regulatory frameworks, particularly data protection

Applicable regulations and ethical principles should be followed if medical-related data are collected as part of apps, to ensure that appropriate data protection regulations are adhered to, while promoting patient safety.30 The latter refers to physical or emotional harm, and includes data protection. Electronic data protection is extremely topical, with the recent implementation of regulations on a European level with the EU’s General Data Protection Regulation, as well as the existing 1996 US Health Insurance Portability and Accountability Act.36 37 Moreover, generic regulation and/or guidance pertaining to mobile apps have been developed on a national level in some EULAR countries, such as the 38UK and Spain.39

PtC 6: apps must not result in physical or emotional harm to people with RMDs

According to WHO, patient safety is the absence of preventable harm to a patient during the process of healthcare and reduction of risk of unnecessary harm associated with healthcare to an acceptable minimum.40 This concept was strongly perceived as crucial by all TF members and was therefore included in the overarching principles in addition to PtC 6. The content and functionalities of apps should consider the well-being of people living with RMDs as a priority. Content that could potentially cause emotional distress or suicidal ideation must be avoided. Appropriate signposting to relevant support organisations for people should also be provided when presenting potentially sensitive information which could elicit emotional and/or suicidal thoughts.

PtC 7: apps could facilitate patient–healthcare provider communication and contribute to electronic health records or research

The sustainability of apps depends on their perceived need and continued use by people living with RMDs, as well as on the interaction around results connected with the physician or other healthcare providers. The lack of feedback on data collected by apps seems to affect app use and its cessation in diabetes.41 In addition to demonstrating the capacity of apps to enhance patient–healthcare provider communication to people living RMDs, there is also a need for health professionals and regulators to acknowledge the use and capacity of apps to enhance patient–healthcare provider communication.42

PtC 8: app design should consider accessibility of people with RMDs across ages and abilities

The accessibility of apps and their ease of use is an important point towards their implementation and sustainability. Age limit was discussed as these PtC do not specifically include the use of apps for children and young people, since apps should be age and developmentally appropriate. The TF agreed on the importance of accessibility of apps, including the visual display and functionality (screen monitor, buttons, text boxes, literacy…).43 Indeed, app design should follow the principle of universal design, and should be usable regardless of previous experience of mobile device use.25 44

PtC 9: if a social network is an important component of an app, structures should be in place to ensure appropriate content moderation

Like other forms of social media, if apps allow public communication between users, they should include a moderation component (eg, someone to moderate live communication/interaction on communication platforms, thus avoiding and/or removing inappropriate or harmful content).45 46 These structures have been defined in these PtC as a system in place for the app that can be a person or an automatic system that ensures appropriate content in the discussion.47 The TF emphasised that the app developer should be responsible for ensuring such structures exist and are enforced.

PtC 10: the rheumatology community should consider the cost–benefit balance of apps before endorsement and/or promotion

Financial affordability was discussed during the TF meeting. In our survey, the majority of people living with RMDs did not wish to pay for the apps.11 It was agreed by the TF that cost should be limited as much as possible, to remain affordable. Some discussion occurred on cost benefit. To date, only a few apps demonstrated their efficacy in randomised controlled trials, and to our knowledge, none focused specifically on the self-management of RMDs. It is expected that more studies will be published in the near future.48 49 The TF recommends that in that case, the development, promotion and purchase of apps should take cost–benefit balance into account.

Further steps

As part of this TF, areas of uncertainty pertaining to apps were discussed, defining directions for future research. These include a review and evaluation of existing app for self-management of RMDs against these EULAR PtC. The planification of a workshop with relevant stakeholders, including rheumatologists/health professionals, people living with RMDs and app developers, would also be helpful in order to define further collaborative projects. Finally, the development of an EULAR self-management app for people living with RMDs following these PtC could be of interest.

Discussion

These PtC for the development, evaluation and implementation of mHealth apps aiding self-management by people living with RMDs were informed by published evidence and mixed-method approaches involving direct patient feedback. The scope of these PtC is focused and limited to apps, as opposed to the more general concept of ehealth/digital health.50 The overall goal of this work was to enhance the process of app development, providing further elements for app standardisation, while ensuring safety and appropriateness for key stakeholders. In an era of exponential growth in apps and in the absence of specific regulatory frameworks, the need for benchmarking the quality of mHealth apps to ensure patient safety is crucial. Three overarching principles and 10 generic PtC for apps aimed at people living with RMDs have been developed which can be further used to: (1) measure the quality of existing mHealth apps and (2) inform the development of future mHealth apps.

During the meeting, the TF agreed on the principal importance of patient safety and data protection. eHealth data protection is a major concern and regulatory guidelines are in place to ensure that data is collected and stored in safe and compliant manners.36 37 However, the status of mHealth apps regarding this particular issue is still unclear. Indeed, mHealth apps need to have a ‘medical purpose’ to fall under EU legislation, presenting some uncertainty as to whether the criterion of the intended medical use includes apps to promote self-management.26 Professional organisations could play a role in regulations around mHealth apps. These PtC include guidance on those important aspects, building on the Consolidated Standards of Reporting Trials (CONSORT-EHEALTH) criteria for reporting studies involving web-based and mHealth interventions.51 This is a checklist instrument that was constructed as an extension of the CONSORT statement. The CONSORT-EHEALTH reporting guidelines provide the basis for evaluating the validity and applicability of eHealth trials and should improve the reporting of findings.

The present work benefits from having incorporated a mixed-methods study to obtain direct patient feedback, owing to the lack of evidence on the topic.50 This ensured a more comprehensive picture was obtained, involving key stakeholders, namely patients. The research agenda as well as PtC takes this into account, and it is expected that these PtC will stimulate further research and publications on this topic. On the other hand, the mHealth field is rapidly moving with an increasing number of apps being accessible on the market based on user demand, making some of these PtC difficult to apply. Unfortunately, representation from app developers and regulatory agencies were lacking as part of the TF. Further work arising from this will include the organisation of a workshop with app developers in order to gather their input on these PtC. Future implementation steps may include an analysis of the PtC impact on newly developed apps. Further ahead, a review of existing apps against these PtC should be conducted in order to assess their impact on the design and content. As part of the dissemination phase, we will also disseminate an electronic survey on agreement and feasibility among a larger panel of stakeholders, including rheumatologists/health professionals, people living with RMDs and app developers. Moreover, an update on these PtC will most likely to be required within the next 5 years, accounting for the rapidly advancing area. It is suggested that such meetings should include app developers and/or regulators where appropriate.

It is anticipated that EULAR, national societies, patient organisations, app developers and regulatory agencies will use these PtC as a basis for the evaluation and endorsement of existing apps or the development of future apps aiding self-management by people living with RMDs.