Proposing a standardized, step-by-step model for creating post-traumatic stress disorder (PTSD) related mobile mental health apps in a framework based on technical and medical norms.

Background: Prevalence of post-traumatic stress disorder (PTSD) is a problem all over the world. There are high barriers for entry into formal psychotherapy, which results in a lack of mental health care for a significant part of the population. Mobile mental Health (mMHealth) applications (apps) seem to be a promising new development for countering this lack of care, building on the success of mHealth (Mobile Health) apps in general. Unfortunately, the overall quality of such apps stands in high contrast to their number. The aim of this manuscript is to propose a standard for creating PTSD-related mMHealth apps incorporating a process for evaluation and assessment of their usability and impact.
Methods: This is done by first defining each step of the process and its relation to the other steps. The steps themselves, divided into those concerned with development, evaluation and implementation, are bound to the established medical and technical norms pertaining to them. Existing protocols from recent literature have been integrated into these steps.
Results: As a result, a comprehensive model covering the process of creating, assessing and implementing an mMHealth app from start to finish was developed. The model may be adapted to other disorders or specialized for certain symptoms of PTSD.
Conclusion: Adopting such a model could result in a 'blueprint' for creating mMHealth apps in a standardized way, thereby facilitating the testing and comparing of such apps.

Current situation and unmet needs

Post-traumatic stress disorder (PTSD) is prevalent all over the world (Kessler et al., 2017). Yet effective treatment or even any kind of treatment seems to be out of reach for at least half of the afflicted population and only a minority receives speciality mental health care (Kessler et al., 2017; Koenen et al., 2017). This situation stems partly from a general lack of access to health care but is also common where such access exists, as cases of PTSD are often overlooked or misdiagnosed (Lu et al., 2017). Even when patients are correctly diagnosed with PTSD, there might be a shortage of available therapists specializing in psychotraumatology, leading to long periods of waiting (Mojtabai et al., 2011). All of this results in a high barrier for entry into appropriate mental health care for patients with PTSD. It underlines the need for supplementary low-threshold, high-coverage means of assessing and supporting the mental health of this population by either supporting professional healthcare or bridging the possibly substantial gap before therapy can begin.

In recent years, it has become apparent that the spread and acceptance of smartphone technology could lay the foundations for such a system (Kao & Liebovitz, 2017; Olff, 2015). Health applications (apps) refer to all (mobile) software aimed at positively influencing physical and mental well-being. Medical apps are a subset of those apps with a pronounced medical aspect (e.g. detection, prevention, monitoring, treatment or alleviation of certain diseases or disorders) and have to conform to certain norms, while for medical device apps even stricter norms apply (Gregor-Haack, 2018). While the number of publicly available mobile health apps (mHealth apps) and especially mobile mental health apps (mMHealth apps) has skyrocketed (Luxton, McCann, Bush, Mishkind, & Reger, 2011; Rathbone & Prescott, 2017), their overall quality is limited (Bakker, Kazantzis, Rickwood, & Rickard, 2016; Van Ameringen, Turna, Khalesi, Pullia, & Patterson, 2017).

Before an app is released and made available to the public a rigorous process of evaluation and assessment has to take place. If the app is to be qualified as a medical device, it is not only important to apply all the norms concerning such devices in development, but also to establish that the app has the desired effect (Grundy, Wang, & Bero, 2016). For this, appropriate methods of evaluation, either a randomized controlled trial (RCT) as the golden standard or alternative supporting evaluation methodologies such as iterative participatory research and single case designs have to be performed (Nicholas, Boydell, & Christensen, 2016). The same holds for apps not classified as medical devices, as this classification impacts only the technical norms that have to be fulfilled, not the requirements of evidence-based treatment.

Although there are some examples of mMHealth apps that are firmly grounded on evidence-based medicine (Donker et al., 2013; Kuhn et al., 2017; Mohr et al., 2017; Jason E Owen et al., 2018; Possemato et al., 2016; van der Meer, Bakker, Schrieken, Hoofwijk, & Olff, 2017; Wang, Varma, & Prosperi, 2018), many of the apps lack validation through RCTs (Van Ameringen et al., 2017; but also refer to section ‘evaluation and assessment‘ below). This may cause the apps to be useless or even actively harmful, for example, when they medicalize normal mental health states or convince the user to not seek professional help (Parker et al., 2018). This is a structural problem for all research into mMHealth apps. It has its roots in the clash between the highly regulated nature of medical devices and the, by comparison, rather unregulated app market (Anthes, 2016; Martínez-Pérez, De La Torre-Díez, & López-Coronado, 2013; Wang et al., 2018; Zhao, Freeman, & Li, 2016).

The lack of rigorous evaluation of the effectiveness of these apps (Grundy et al., 2016; Van Ameringen et al., 2017) has led to calls for the development of a universal standard for the creation of mMHealth apps. These calls are becoming increasingly louder especially from the clinical side (Bakker et al., 2016; Choi et al., 2018) as well as from developing industry’s side (GSMA, 2016). A number of different national and international recommendations and norms pertaining to app creation and evaluation exist. Examples include MARS (mobile app rating scale) (Stoyanov et al., 2015) and uMARS (user-MARS) (Stoyanov, Hides, Kavanagh, & Wilson, 2016), as well as the European mHealth subgroup’s suggestions for future work (EU Member States mHealth Subgroup, 2017). Olff (2015) contains many helpful observations about the development and evaluation of mMHealth apps. Existing norms include, but are not limited to, the EU- and IEC-norms concerned with the creation of apps and medical devices (esp. IEC, 2005, 2014, 2016). While the recommendations and norms go into great detail about the specifics, there is as of yet no accepted abstract model of how to apply them in a way that enables a repeatable creation process and easy testing and comparing of the products. Apps that follow a standardized model of development and evaluation would be easily comparable, thus making it easier to disseminate information about the quality of those apps to the public. Moreover, as a blueprint, it can easily be conveyed to apps for tablet PCs and for comparable browser-based web applications. The purpose of this tutorial is to propose such a model for evidence-based PTSD-related mMHealth apps.

Creating an mMHealth app: step by step

The proposed model is comprised of a number of steps, which cover the process of creating such an app from idea to dissemination and beyond. Figure 1 contains a graphical representation of the different steps and how they relate to one another. The model is based on the requirements of international standards (most importantly IEC 82304-1:2016) and the ISR (Information system research framework) method (Cronholm & Göbel, 2016; Hevner, March, Park, & Ram, 2004; Schnall, Bakken, Brown, Carballo-Dieguez, & Iribarren, 2016) but is adapted for mental health, more specifically for PTSD (see section ‘Specifying for PTSD’ below). It is important to note that every step of the model requires constant documentation of all the different approaches and solutions by the app’s creators (both mental healthcare professionals and information technicians, respectively) to facilitate singling out points of improvement and make the whole process replicable for others wishing to evaluate or adapt it. In the following sections, each step of the model will be discussed.

Figure 1.

Blueprint for creating a PTSD-related mMHealth app to standardize development processes for better comparability and testability.

Defining healthcare goals according to patient needs

The first necessary step in creating an mMHealth app is to define its goals according to the needs of the target population. This includes, for example, the decision whether the primary purpose of the app is assessment like ‘Smart Assessment on your Mobile’ (“SAM”, van der Meer et al., 2017), symptom tracking and self-management of symptoms like the SPIRIT app (Bauer et al., 2017) and PTSD Coach (Kuhn et al., 2014), treatment as the CBTi Coach (Kuhn et al., 2016), or treatment assistance like PE Coach (Kuhn et al., 2015), or a combination of those, each resulting in radically different medical and technical requirements for the app. Ideally, this step is informed by previous research into how to convert the usual methods of assessment, tracking and treatment into their digital counterparts effectively. A number of reviews about currently released mMHealth apps exist and should be taken into consideration when designing new interventions (e.g., Radovic et al., 2016). More specifically, there has been a recent review of PTSD-related mMHealth apps for iOS and Android that compares the different apps in regard to their usability, validation and integration into clinical PTSD-treatment (Rodriguez-Paras et al., 2017).

This first step will also decide if the proposed app is to be considered a medical device, with all the additional requirements and norms that follow from it. Even though there are recent studies that show a promising potential for online therapy to reach the effectiveness of face-to-face therapy in trauma-specific treatment (Kuester, Niemeyer, & Knaevelsrud, 2016), evidence suggests that mMHealth apps should only be used as a supplement for professional therapy, not instead of it (Keen & Roberts, 2017; Possemato et al., 2016). This has to be made abundantly clear to the user (Price et al., 2014).

Another important question to address is whether the app should act as an isolated program or be connected to various online databases for the purpose of receiving and sending information, like, e.g. the SPIRIT program (Bauer, Hodsdon, Bechtel, & Fortney, 2018), which has a big impact on which norms to apply concerning data security and integrity. Although the implementation of this has technical challenges, deciding whether the app needs such a connection is a medical consideration based on the desired functionalities of the app.

Collecting information on the needs of the patients via surveys and focus groups can be a central aspect of this step, often deciding the very basic direction that the development of the app takes. Valuable advice on this is to be found, for example, in Avis, van Mierlo, Fournier, & Ball (2015); Baskerville, Struik, & Dash (2018); and Harte et al. (2017).

Taking great care in first defining what the app should do from a medical perspective makes it easier to subsequently find concrete technical solutions to the desired capabilities. Establishing a direct line of communication between the teams working on the medical aspect of development and those concerning themselves with the technological aspect is of vital importance even in this first step as it helps to eliminate confusion over what is technically possible and what is required medically.

Establishing an appropriate IT structure

Once the medical goal is set, an appropriate IT architecture for the program has to be found in close collaboration between the technical and the clinical team. Different objectives call for different coding frameworks to be used. A variety of papers and textbooks cover topics of app development (Eisenman, 2018; Griffith, 2017; Singh & Phan, 2018). Innovative cross-platform app development frameworks (e.g. Ionic or React Native) are available as open source.

Nine principles for digital development were originally established by UNICEF in 2009 (UNICEF, 2014) and subsequently led to the creation of the Digital Principal Working Group in 2014. Their set of best practices informs the development of technology-enabled programs, calls for the use of open standards, open data, open source, and open innovation (Bauer et al., 2018; Digital Principal Working Group, 2018). An open approach to digital development can help to increase collaboration in the digital development community and avoid duplicating work that has already been done (Bauer et al., 2018). To minimize the technical complexity of creating an app, software frameworks have been developed that facilitate the process (e.g. PHIT, Eckhoff et al., 2015; Kizakevich et al., 2018). These frameworks can be understood as a kind of construction kit for software, providing a common platform and reusable content for both development and evaluation, thereby greatly reducing the cost of app development. The PHIT framework, for example, which facilitates mobile data collection and health intervention research, was used to convert a series of subjective data instruments like the Post-Traumatic Stress Disorder Checklist (PCL; Weathers et al., 2013) into a digital form via XML-coded logic.

To, for example, ensure interoperability for an app that regularly sends data to the regional Health Information System, it is important to make an effort to integrate the planned app into the existing IT structure of this system (Agarwal et al., 2016). If there is the purpose to exchange user data between connected systems, translation of clinical data into a clinical document architecture (CDA) file is recommended. Any data that is exchanged to external systems should conform to relevant data transfer standards like FHIR® (Fast Healthcare Interoperability Resources) created by the Health Level 7 International health-care standards organization. The base FHIR® specification is a platform specification (HL7, 2018).

If web capability is desired it is always recommended to utilize highly standardized protocols such as RESTful web services (Richardson & Ruby, 2008), an architectural style for designing network applications that specifies constraints including performance, scalability, and modifiability.

Also, a primary concern in this step should be the compliance with all applicable legislature concerning data security and product safety. It is critical to document the choices made in this step as well as the reasoning behind them. This facilitates not only modifications made to the app in later stages of development but also creates the transparency needed for effective assessment and evaluation of the app through independent studies.

Specifying for PTSD: functionalities and lay-out

In this step, functions are designed to fulfil the goal set in the beginning. If, for example, the goal is to make available certain exercises to combat dissociated mind states, those have to be chosen carefully according to the intended effects. This includes a thorough assessment of any potential risks of those exercises. If questionnaires are to be part of the app, they have to be re-examined for their efficacy when used digitally and then converted into a form compatible with mobile phone screens (Marcano Belisario et al., 2015; Price, Kuhn, Hoffman, Ruzek, & Acierno, 2015; Van Ameringen et al., 2017; van der Meer et al., 2017). Each intended function of the app has to be associated to one or more of the goals set at the beginning, just as every goal should be represented by the functions that are working to achieve that goal.

There has been a considerable success with apps using a variety of functionalities like mental health information and automated tailoring with reporting of thoughts, recommended activities, memory function log of past app use and links to crisis support services (Bakker et al., 2016).

The app PTSD Coach, for example, has four main goals: Giving information to the patients; tracking the symptoms of the patient; managing these symptoms and making additional support available. Part of managing the symptoms is grounding and resource activation. For this purpose, mindfulness-training exercises were converted into electronic form and made available in the app (Kuhn et al., 2017, 2016; Miner et al., 2016). The Dutch version of the app (SUPPORT Coach) includes a calendar function for the purpose of scheduling appointments and activities, while the Canadian version added a section concerned with information about ‘operational stress injuries’, meaning any psychological difficulties sustained while serving in the Canadian Armed Forces or the Canadian Royal Mounted Police (Kuhn et al., 2018). CoachPTBS, the German version, added a mood diary and contents from CBT-I Coach for insomnia (Kuhn et al., 2016) as well as tool to assess the levels of anxiety and depression (Kuhn et al., 2018).

Design choices also have to be considered very carefully, especially in relation to users suffering from PTSD. For example, this can include a ‘Get Help Immediately’ button being visible at all times on screen, or a simple and clear menu layout which can even be used in impaired states of mind.

Specifying for PTSD: applying medical norms

The preceding steps, specifications for PTSD and finding an appropriate IT-structure, ideally carried out simultaneously and in close cooperation, are pivotal for the process of applying the various national and international norms to the design process of the app. From the medical side, strict adherence to the categories and classifications of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5, American Psychiatric Association, 2013) and the International Classification of Diseases (ICD-10, World Health Organization, 1992), as well as ICD-11 in the near future (World Health Organization, 2018), and the guidelines of evidence-based treatment for PTSD (see Table 1) are a must.

Table 1.

Relevant guidelines for evidence-based treatment for PTSD to base mMHealth apps on.

● NICE Guidelines (NICE, 2005)● Institute of Medicine Guideline (Institute of Medicine, 2008)● Kapstadt-Konsensuskonferenz zur Behandlung der PTBS (Stein et al., 2009)● The ISTSS Expert Consensus Treatment Guidelines For Complex PTSD In Adults (Cloitre et al., 2012)● The Association of the Scientific Medical Societies in Germany (Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften e.V., AWMF) Guidelines (Flatten, 2013), currently under revision● Australian Guidelines for Acute Stress Disorder & Posttraumatic Stress Disorder (Forbes et al., 2013)● Management of Acute Stress, PTSD, and Bereavement – WHO recommendations (Tol, Barbui, & Van Ommeren, 2013)● Clinical Practice Guideline for the treatment of PTSD (American Psychological Association, 2017)● Va/DoD Clinical Practice Guideline for the Management of Posttraumatic Stress Disorder and Acute Stress Disorder (Management of Posttraumatic Stress Disorder Work Group, 2017)

(American Psychological Association, 2017; Cloitre et al., 2012; Forbes et al., 2013; Institute of Medicine, 2008; Management of Posttraumatic Stress Disorder Work Group, 2017; NICE, 2005; Stein et al., 2009; Tol et al., 2013)

Applying technical norms

There are a variety of recommended principles to fulfil while creating an mHealth app, such as the principles for digital development for technology-enabled service delivery programs (Bauer et al., 2018, see also Box 1).

Box 1.

Principles for digital development for technology-enabled service delivery programs (Bauer et al., 2018; Digital Principal Working Group, 2018).

1. Design with the user2. Understand the existing ecosystem3. Design for scale4. Build for sustainability5. Be data driven6. Use open standards, open data, open source, and open innovation7. Reuse and improve8. Address privacy and security9. Be collaborative

Design with the user (and not for them) refers to a design philosophy centred around partnering with the user throughout the project life cycle. Understanding the existing ecosystem essentially means being aware of the circumstances in which an app is released, such as databases that the information coming from the app is being fed to, or the health regulations of the targeted region. Designing for scale is achieved by planning for the app to be adopted by a large number of people. Sustainability is mostly concerned with securing uninterrupted funding and institutionalization of the product. Being data driven refers to basing the design decisions on data collected from all stages of development, as well as a rigorous method of collecting those data. An example of this is the methodical collection of feedback from beta-testers and incorporating the lessons learned into the final app (Bauer et al., 2018). Using open standards, open data, open source and open innovation is a fundamental principle when developing software, because using already existing tools saves time and money. The SPIRIT app, for instance, was developed using CommCare, an open source mobile data collection platform (Bauer et al., 2018). Reusing and improving on preliminary work such as already existing apps also saves scarce resources and makes it easy to avoid starting a project from scratch. The international versions of the PTSD Coach are good examples of this, as they reused an already existing app to transform it into localized versions (Kuhn et al., 2018). Addressing privacy and security means careful consideration of which data are collected and which data are acquired, used, stored and shared. Being collaborative is essential for all team efforts, but especially so when combining medical and technical expertise.

Fulfilling the technical norms required by national or international laws when creating an app and/or a medical device is a more complicated effort, highly dependent on the laws of the region where the app is to be released. In most developed countries there is a framework of legal and technical norms concerning most aspects of app development, such as safety regulations provided by the FDA, although there are also serious shortcomings to those frameworks which need to be addressed, like the rigidity of current regulatory solutions (N. P. Terry & Gunter, 2018). Adhering to official norms is mandatory when the intended purpose for the app is to be considered a medical device (see Box 2).

Box 2.

Norms concerning software products as medical device.

In the EU: The EU guideline MEDDEV 2.1/6 from 2016 details the requirements for qualifying an app as a medical product. Generally, this is the case if it is designed for the purpose of ‘diagnosis, prevention, monitoring, treatment or alleviation of disease’ (EEC 93/42 Art. 1.2a). As such, it is subject to all relevant EU norms for medical devices.In the US: Contrarily, the FDA utilizes a classification scheme for medical software devices that mostly revolves around listing examples of apps that are to be considered as medical devices (U.S. Department of Health and Human Services Food and Drug Administration, Center for Devices and Radiological Health, & Center for Biologics Evaluation and Research, 2015). The FDA intends to exercise enforcement discretion for these apps.International: Additionally, there are international norms harmonized by the EU and the US concerning the creation and evaluation of medical devices (IEC 62304, IEC, 2014) and health software products (IEC 82304-1, IEC, 2016). Any apps considered being a medical device should conform to these norms.

(U.S. Department of Health and Human Services Food and Drug Administration, Center for Devices and Radiological Health, & Center for Biologics Evaluation and Research, 2015)

In addition to that, some functions require adherence to more specific norms. For example, if the app incorporates a feature sending and receiving patient data from external servers, norms relating to data security and privacy apply (Dehling, Gao, Schneider, & Sunyaev, 2015; Parker et al., 2017). User rights have been strengthened by the recently released General Data Protection Regulation (EUR-Lex, 2018). This is even more important for patients with mental health issues.

All necessary information about data security, the imprint, a disclaimer and the terms of use are required to be integrated into the app.

Importance of interdisciplinary communication

Technical, legal and medical aspects of such projects are interdependent. Nevertheless, communication between the two sides is often challenging and can represent a real hurdle to the successful development of an mMHealth app. It is equally important for the medical team to be informed of the technical limitations as it is for the technical team to have a clear understanding of what the program should be able to do. Information on how to build effective interdisciplinary communication can be found, for example, in Kuziemsky et al. (2009)

Included in interdisciplinary communication is the effective communication with partners such as contracted app developers. It is highly important to use well-drafted contracts in which all desired functionalities are specified. Contracting technology transfer offices and involving legal support (e.g., from within the developer’s clinic or institute) can assist in those vital processes. Contracts should specifically include clauses about updating, bug finding and bug fixing even after the product is finished. This is recommended, as it gives responsibility for these important tasks to the people most familiar with the software and simplifies it to add additional features at a later date. Also, ownership and IP-issues of the to-be-created product need to be carefully arranged from the start of the project.

Using proven methods of design

When all the desired capabilities are formulated and the general architecture of the app is clear, the process of designing the app itself can begin. At this step, it is important to incorporate a system for centring the design process on the user. Several such methods for user-centred design (UCD) have been proposed. The design principles found in Bauer et al. (2018) (see Box 1) are one such example of focusing a PTSD-specific app on user-centred design, although the term itself is never mentioned. Giunti, Mylonopoulou, & Romero (2018) contain a more explicit description of UCD, which is characterized as a mixture of iterative involvement of the end user in the different phases of development, usage of idea generation techniques such as brainstorming, early prototyping and usability testing of the system. Schnall et al. (2016) use a very similar method of UCD in the context of an HIV-prevention app. Baskerville et al. (2018) employ the so-called 5-cycle model (listen, plan, do, act, study) which in effect results in a design process heavily involving the end user.

Of course, this is not an exhaustive list of design methods, all of which have significant differences, so it is recommended to choose carefully amongst them, as some lend themselves better to the use in mental health care than others. The paper by Rizzo et al. (2006), although not dealing strictly with mobile apps, is a very useful resource in determining the correct design choices in the context of PTSD.

Alpha and beta testing with and without patients

A central aspect of UCD is the extensive use of alpha and beta testing, when possible with members of the targeted population. This means handing out prototypes of the app to focus groups consisting of patients with PTSD, healthcare professionals and technical experts, collecting their impressions of the user experience and modifying the app accordingly. For this, different ways of measuring the usability of a given app have been proposed (e.g. Wakefield et al., 2017), like the System Usability Scale (SUS; Brooke, 1996) or the eHealth literacy scale (eHEALS; Norman & Skinner, 2006). In the first few steps, certain agility is necessary for the development environment so that changes made after the need became apparent in testing can be easily implemented. Failure in accommodating the expectations of the targeted users often results in the users’ lack of willingness to use the app (Giunti et al., 2018; Schnall et al., 2016). This would interfere with the goal of lowering the barrier of mental health interventions for PTSD through easily accessible smartphone apps.

Evaluation and assessment

Normally, the process of evaluation and assessment would take place before the release of the app. It is highly critical in determining the value of any given app (Kuester et al., 2016; Van Ameringen et al., 2017). During the evaluation process, a limited release that makes the app available only to the target group should be considered.

Currently, PTSD Coach (Hoffman et al., 2011) is the most rigorously evaluated PTSD specific app. A pilot RCT comparing the effects of using PTSD Coach in self-management (n = 10) versus assisted by a clinician (n = 10), showed improvements in PCL-scores after eight weeks in both groups (38% v. 70% of participants), with clinician-assisted use having, although only as a trend, greater results (Possemato et al., 2016).

The first larger RCT, a crossover design between waitlist group (n = 24) versus app users (n = 25), revealed significant reductions in PCL-Scores after 1 month, however, for both groups, and no between-group difference (Miner et al., 2016). Thus, a larger scale RCT was conducted, with an app group (n = 62), using PTSD Coach for 3 months this time, compared to waitlist control (n = 58). Consequently, clinical improvements in PTSD symptoms were greater in the app group than waitlist control. There were, however, no significant differences between the groups posttreatment. Additionally, the app seemed to also affect depressive symptoms and psychosocial functioning positively. (Kuhn et al., 2017).

There are currently competing protocols for the evaluation and assessment of mHealth apps, and adherence to one of them is strongly recommended (Quiñones, Rusu, & Rusu, 2018). The World Health Organization (WHO), for example, created the mHealth evidence reporting and assessment (mERA) checklist for this purpose (Agarwal et al., 2016), while the EU developed their own model for assessment of telemedicine (MAST, Kidholm, Clemensen, Caffery, & Smith, 2017; Kidholm et al., 2012)

The MAST criteria should ‘be used as a basis for decision making in EU and the European countries in decisions on use of telemedicine applications’ (Kidholm et al., 2010). During a multidisciplinary assessment, the criteria include seven focus domains such as safety, clinical effectiveness and patient perspectives. The Task Force E-Mental Health of the German Association for Psychiatry, Psychotherapy and Psychosomatics (DGPPN, Deutsche Gesellschaft für Psychiatrie und Psychotherapie, Psychosomatik und Nervenheilkunde) has developed an adaptation for psychiatric conditions (Klein et al., 2016). Although the MAST criteria are most relevant for matured telemedicine tools (that can be thoroughly tested in clinical trials), they can still be helpful during development. For instance, appreciation of patient (or user) perspectives, one of the MAST focus domains, eventually leading to user satisfaction, should be practised during the whole development process, for instance in the form of assessing the tool’s usability. The non-specific evaluation methods of the world of computer science also provide many valuable insights about statistics and heuristics (Quiñones et al., 2018).

There is growing evidence that apps are engaging, easy-to-use, and provide a relative advantage to traditional care without apps (Owen et al., 2018). Pilot studies on how the end users (e.g., clinicians and patients) are interacting with the app seem promising (Cernvall, Sveen, Bergh Johannesson, & Arnberg, 2018; Rodriguez-Paras et al., 2017). Kuhn and colleagues focused on subjective perceptions, such as user satisfaction and helpfulness (Kuhn et al., 2014) while Owen and colleagues expanded more on objective user engagement and common points of attrition in using the app (Owen et al., 2015). As mentioned earlier, first validation studies were also carried out to ensure that the PTSD-related apps’ purpose and specifications are met (Rodriguez-Paras et al., 2017). Although all these studies had positive results, most noted the need for additional testing to determine the app’s impact on clinical outcomes (Rodriguez-Paras et al., 2017).

Release of the app

After all desired functionalities are implemented and have been thoroughly tested, and accompanying documentation for the users has been put in place, the app can be released for public use. It is recommended to make the app available to the largest audience possible, which mostly means releasing it for both Android operating system and iOS. Choosing a straightforward name that directly hints at the purpose of the app is also an important factor, making it easier to find and encouraging potential users to try it out.

The release of an app itself, as well as maintenance, costs a certain amount of money, something that needs to be included in the financing plan. Except for making the app cost money, ways to secure funding over a longer period of time include finding fixed sponsors such as non-profits or state institutions, or including the app into a reimbursement plan with health insurance (Gregor-Haack, 2018). Having the app classified as a medical device makes this inclusion far more likely. Regular updates, bug fixes and engagement with the users greatly increase the chance of the app to stay available, as long as all conditions in the terms of use of the app store are met.

Localization

Differences in legislature, language and culture require localization of apps if they are to be introduced into another country. Though most of the work in this step consists of translation and cultural adaptation, it is also important to keep an eye on the continued efficacy of the localized app (Stara et al., 2018). If the languages have much in common and there are no major differences in the norms pertaining to the design of apps, re-evaluation, although still necessary, can be kept to a minimum. But when there are significant differences more impactful changes have to be made and it becomes crucial to re-evaluate the localized version of the app extensively (Bardosh, Murray, Khaemba, Smillie, & Lester, 2017).

Post-market communication

Post-market communication (PMC) is an important part of the life cycle of every serious app, among other things, involves keeping an overview over the number of downloaded apps and their subsequent reviews and feedback by the users, as well as employing this information to update the product (e.g., Owen et al., 2015). Thus, it consists of software maintenance, bug fixing and user support. This can be problematic because it creates a continuous workload and need for funding, even after the app is released and evaluation has been completed. Many mHealth and mMHealth apps, even if they are designed well, fail to adequately meet this requirement. For this reason, emphasizing the importance of proper PMC is the key to the development of long-term, stable apps. Another important consideration after release is the question of ‘scale-up’, that is how to disseminate not only the single app among a larger audience but also how to reach more people with mHealth solutions in general. Utilizing the ubiquitous data many of these apps generate is a big step forward in this direction (Tomlinson, Rotheram-Borus, Swartz, & Tsai, 2013).

If the app has a connection to Electronic Health Records (EHR), the seamless and secure integration into their data systems has to be assured continuously (Kao & Liebovitz, 2017).

Taken all together, the steps outlined above make up a standardized model for the creation, design and evaluation of a PTSD-related mMHealth app. It utilizes different existing protocols for individual steps, combining them in a coherent way with the intention of clarifying and streamlining the entire process. This model may be adapted for different mental health issues or specific symptoms of PTSD. The widespread adoption of such a ‘blueprint’ may counter the flood of non-evaluated low-quality mMHealth apps currently drowning the market and provide government agencies and medical professionals with a way to safely recommend apps that successfully follow the model. Moreover, it would bring the medical community one step closer to unlocking the enormous potential of using the universally prevalent smartphones for medical purposes. If successful, this model may be adapted for mental health problems other than PTSD, only requiring adjustment in the ‘specifying’ step.

Apps created following the same general outline would have the advantage of being easy to test and to compare. The tests themselves could become more standardized, making it easier and less expensive to assess new apps. If this process can be made streamlined enough, ‘prescription apps’ for mental health problems may become common occurrences in the future. ‘Prescription apps’ would have the additional benefit of covering at least a part of the expenses for development by costing an appropriate fee, while not reducing usage because the cost could be covered by health insurance (Terry, 2015). Although this is still in its early stages, a step in this direction was already taken by some health insurance companies, who offer certain apps (mostly concerning depression) for free to their customers (aerzteblatt.de, 2018). Recently, the idea of including apps that have demonstrated their worth in the catalogues of recognized and approved medical aides and appliances of the different countries has gained traction (e.g. Hilfsmittelverzeichnis in Germany by GKV-Spitzenverband, 2012).

While the value of a standardized model for the creation of mMHealth apps is clear, there remains a significant challenge in encouraging its adoption all over the world. We encourage the reader to consider this document as a starting point. The model will become more refined after more experience is gained with its application and when new regulations or technologies become available.

CDA	Clinical Document Architecture (CDA) is a popular, exible markup standard developed by Health Level 7 International (HL7) that defines the structure of certain medical records, such as discharge summaries and progress notes, as a way to better exchange this information between providers and patients.
FHIR	Fast Healthcare Interoperability Resources, is a draft standard describing data formats and elements (‘resources’) and an Application Programming Interface (API) for exchanging Electronic health records.
GSMA	Global System for Mobile Communications, it represents the interests of mobile network operators worldwide
HL7	Health Level 7; a standards body that defines interoperability and messaging standards for the health industry.
Interoperability	Ability of health information systems to work together within and across organizational boundaries in order to advance the health status of, and the effective delivery of healthcare for, individuals and communities
MAST	Model for Assessment of Telemedicine initiated by the European Commission
MEDDEV	European guideline for medical devices
mERA	mHealth Evidence Reporting and Assessment checklist by the WHO
mHealth apps	Mobile health applications (for smart devices)
mMHealth apps	Mobile mental health applications (for smart devices)
RESTful web services	Representational State Transfer (REST) is an architectural style for designing network applications that specifies constraints, such as the uniform interface, that if applied to a web service induce desirable properties, such as performance, scalability, and modifiability.
UCD	User-Centered Design revolves around integrating prospective users into the design process