Electronic clinical decision support algorithms incorporating point-of-care diagnostic tests in low-resource settings: a target product profile.

Health workers in low-resource settings often lack the support and tools to follow evidence-based clinical recommendations for diagnosing, treating and managing sick patients. Digital technologies, by combining patient health information and point-of-care diagnostics with evidence-based clinical protocols, can help improve the quality of care and the rational use of resources, and save patient lives. A growing number of electronic clinical decision support algorithms (CDSAs) on mobile devices are being developed and piloted without evidence of safety or impact. Here, we present a target product profile (TPP) for CDSAs aimed at guiding preventive or curative consultations in low-resource settings. This document will help align developer and implementer processes and product specifications with the needs of end users, in terms of quality, safety, performance and operational functionality. To identify the characteristics of CDSAs, a multidisciplinary group of experts (academia, industry and policy makers) with expertise in diagnostic and CDSA development and implementation in low-income and middle-income countries were convened to discuss a draft TPP. The TPP was finalised through a Delphi process to facilitate consensus building. An agreement greater than 75% was reached for all 40 TPP characteristics. In general, experts were in overwhelming agreement that, given that CDSAs provide patient management recommendations, the underlying clinical algorithms should be human-interpretable and evidence-based. Whenever possible, the algorithm's patient management output should take into account pretest disease probabilities and likelihood ratios of clinical and diagnostic predictors. In addition, validation processes should at a minimum show that CDSAs are implementing faithfully the evidence they are based on, and ideally the impact on patient health outcomes. In terms of operational needs, CDSAs should be designed to fit within clinic workflows and function in connectivity-challenged and high-volume settings. Data collected through the tool should conform to local patient privacy regulations and international data standards. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Clinical decision support tools deliver evidence-based recommendations to support clinical decision-making such as prescribing antibiotic treatment.

Although widely integrated in many healthcare activities in high-income countries, in low-resource settings handheld-based clinical decision support algorithms (CDSAs) are being developed and have shown to provide benefits to patients (outcome) as well as to the public (reduction of antibiotic overprescription).

To support end user curative and preventive consultations at point of care in low-resource settings, stakeholders defined CDSAs that are based on evidence, human-interpretable and customisable to the setting and its workflow, and importantly incorporate contextual data and diagnostic test performance in the patient management recommendation delivered.

CDSAs can help deliver quality health services by providing evidence-based recommendations during patient consultations.

The use of such tools, in combination with available resources, aims to improve patient outcome and rational use of diagnostics and therapeutics at point of care, and in the long term reduce economic hardship on patients and health systems.

Introduction

Health workers at primary healthcare (PHC) have the greatest challenge of ensuring appropriate care for their communities. One way to support health workers is through the provision of clinical guidance in the form of clinical practice guidelines or clinical decision support algorithms (CDSAs). Clinical practice guidelines are a set of recommendations on diagnostic and treatment modalities based on systematic review of evidence and assessment of the benefits and harms to patients. They aim to deliver the best care and rational clinical decisions to improve patient outcome. The WHO developed several simple clinical decision guidelines to support health workers in low-income and middle-income countries (LMICs) in the evaluation and management of clinical problems, with the ultimate goal of improving quality of care.1 These include the Integrated Management of Childhood Illness (IMCI) designed in the 1990s for common diseases affecting children younger than 5 years of age,2 which was later adapted for use in the community as the integrated Community Case Management guideline.3 Other guidelines have been developed and either take an integrated management approach similar to IMCI, such as the Integrated Management of Adolescent and Adult Illness,4 antenatal care (ANC) and family planning, or a disease-specific approach (ie, WHO’s guidelines for diagnosis, treatment, prevention and control of dengue5).

While the global health community awaits updated and more comprehensive clinical algorithms, the fourth industrial revolution6 is seeing developing economies ‘leapfrogging’7 mobile technologies across multiple sectors, including in health. As a result, a plethora of digital clinical decision support systems (CDSS) have been implemented in LMICs. CDSS is defined as ‘digitized job aids that combine an individual’s health information with the health worker’s knowledge and clinical protocols to assist health workers in making diagnosis and treatment decisions’.8 It is well recognised that adherence to standardised clinical practice guidelines is a measurement of quality of care,9 10 and that improving adherence to evidence-based guidelines is associated with better health outcomes.11 By digitalising clinical guidelines into electronic CDSAs, in its simplest form a decision tree, several groups have shown significant increase in health worker adherence to guidelines compared with routine care (ie, CDSA vs paper-based IMCI).12–14

However, today CDSAs carry limitations similar to the guidelines and still lack point-of-care (POC) data that could improve the accuracy of diagnoses and treatment recommendations. In terms of clinical guidelines related to febrile illness, the only diagnostic test widely used is a malaria rapid diagnostic test (mRDT), and while this inclusion was important in the context of ‘test-and-treat’15 it is limited to ruling in or ruling out one pathogen alone. Acute febrile illness can be caused by bacterial or viral respiratory infections, or seasonal, geographically localised pathogens like dengue. The inclusion of additional simple wet and digital diagnostics into adapted algorithms can reduce antimicrobial overprescriptions as well as improve the child’s health outcome.16 17 This was demonstrated with IMCI-based CDSAs, and it is postulated that similar benefits on outcome and antimicrobial use could be awarded to other use cases.

With the growing development of CDSAs, in particular those based on widely adapted guidelines that have the potential to reach thousands to millions of people, including vulnerable groups, it is important to ensure that new tools meet the needs of end users. Establishing a target product profile (TPP) for such a toolkit is an important step towards meeting this goal. TPPs include minimal and optimal definitions of the characteristics of a proposed health tool to address a defined public health need. Ideally the tool is designed to meet the majority of optimal characteristic criteria as feasible, while still meeting the minimal criteria.

To this end, here we present a TPP that was defined through consensus with stakeholders and experts. It aims to inform developers and implementers alike of the requirements for CDSAs aimed at guiding health workers during preventive or curative consultations with any type of patient (eg, children, adolescents, adults, pregnant women and so on) using, whenever feasible, available diagnostic tools.

Methods

Draft TPP

A draft TPP (version 0) for CDSAs combined with POC results was developed (by KGP, SD, RS, ZK, GLM, FGM) based on standard procedures at the Foundation for Innovative New Diagnostics (FIND) and WHO, where characteristics are defined as ‘minimal’ and ‘optimal’ criteria (online supplementary file 1). ‘Minimal’ is used to refer to the lowest acceptable output for a characteristic and ‘optimal’ for the ideal target of a characteristic. The comment sections in a TPP are used to explain or provide examples of the characteristics. As this TPP was for a multicomponent toolkit, CDSA and POCs, the structure of the draft TPP was adapted to cover the different components and for the purpose of clarity. These sections included general scope, scope of toolkit components, electronic clinical decision support algorithm, POC tools, app, data and procurement, and were also the sections used in the final TPP.

Expert meeting

To discuss selected aspects of the draft TPP (algorithm validation, performance, machine learning (ML), diagnostic data, disease prediction, clinical workflow and application functionality) (online supplementary file 1), a meeting on this subject was convened by FIND and WHO involving 39 experts from academic institutions, industry, and private and public sectors from 11 countries.18 Experts were selected based on their experience in relevant areas of work directly related to digital health (clinical decision support or other digital tools targeting resource-challenged settings) and/or diagnostics (wet and digital). Following the expert meeting, the TPP was updated based on expert feedback (version 1.0) (online supplementary file 2).

Delphi process

To facilitate consensus building for the TPP, we followed a Delphi-like process where TPP version 1.0 was reviewed through a three-part online survey. The first part of the survey collected professional information from the survey respondents, as well as experience with clinical practice/research, diagnostic test development, software development, implementation of healthcare programmes in LMICs and health information systems. The second part of the survey allowed for rating of both minimal and optimal criteria for all TPP characteristics. Agreement was scored on a Likert scale from 1 to 5, where 1 corresponds to fully disagree, 2 mostly disagree, 3 neither agree nor disagree, 4 mostly agree and 5 fully agree. Disagreement with a criterion was based on a rating from 1 to 3 and required a comment/suggestion by the survey respondent. To reach consensus, more than 75% of survey respondents should provide a rating of at least 4 (agreement) on the Likert scale. The third part of the survey provided an opportunity for respondents to share comments/feedback/ideas on the TPP if not covered in the characteristics section. After the first round of the survey, agreement for each characteristic was calculated as a percentage, and respondent comments, whether associated with agreement or disagreement, were reviewed. The TPP was revised based on reviewer feedback and sent for a second survey. The data were compiled as in the first round. If 75% agreement was achieved for all characteristics, the Delphi process was closed.

Patient and public involvement statement

At no point in this work were patients or the public involved in the design of this study or in the reporting and dissemination of the results.

Results

Expert consensus on the TPP through a Delphi process

In the first round, the online survey was sent to 77 experts (including workshop participants), and 28 responded to the survey (response rate, 36.4%), with some respondents submitting a coordinated response for their institution. Twenty-five per cent of the respondents were from LMICs (7 of 28); however, the majority of the respondents (22 of 28, 78%) had experience working in Africa and 57% (16 of 28) in South-East Asia. More than 85% (24 of 28) of the respondents had more than 5 years of experience in clinical practice and research, 50% (14 of 28) in health programme implementation in LMICs, 39% (11 of 28) in health information systems, 21% (6 of 28) in software development and 14% (4 of 28) in diagnostic product development (online supplementary file 3). Most participants work in international organisations (7 of 28, 25%) and academic institutions (10 of 28, 35.7%). In the first round, at least 50% agreement was reached for all criteria of all the characteristics, and feedback was taken into consideration to develop TPP version 1.1 for the second round of the Delphi survey (online supplementary file 4). In this new version, the characteristic ‘Encounter’ was removed as several participants expressed it was redundant with the characteristic ‘Workflow’. Therefore version 1.1 had 40 characteristics.

A second online survey was sent to the 28 respondents from round 1. In this round, 23 participants responded to the survey (response rate, 82%). Most participants were from academia (7 of 23, 30%), non-profit international organisations (7 of 23, 30%) and had a similar profile to respondents from round 1 (online supplementary file 3). In this round of review, at least 75% of the participants agreed with the minimal and optimal criteria for all TPP characteristics. On average, 91% of survey respondents agreed with the minimal and optimal criteria (figure 1) and the TPP was finalised (version 2).

Figure 1

Figure 1Expert consensus on TPP characteristics. Expert agreement is shown for each TPP characteristic minimal (Min) and optimal (Opt) criteria. Agreement was scored on a Likert scale from 1 to 5, 1 = fully disagree, 2 =mostly disagree, 3 =neither agree nor disagree, 4 =mostly agree and 5 =fully agree. Consensus was reached when more than 75% of survey respondents provided a rating of at least 4. CDS, clinical decision support; POC, point-of-care; API, application programming interface

Final TPP

General scope and Scope of toolkit components

This TPP defines a toolkit composed of CDSAs and POCs intended to support evidence-based clinical decisions made by health workers during a preventive or curative consultation, by capturing patient clinical and context-specific data, as well as diagnostic test results, to provide recommendations on diagnosis and patient management (including counselling, follow-up visits and so on). The clinical algorithm integrates POC results and is embedded in an app. POCs in this toolkit should be regulatory approved for use in the setting of implementation.

The experts agreed that the CDSA should first define its target: (1) patients (eg, age group), (2) medical problems (eg, syndrome or disease), (3) end users (eg, minimally trained health worker or trained clinician) and (4) level of care (eg, community or health facility) for which the clinical recommendations are meant and relevant. These specifications should be clearly communicated to the end user at the beginning of the algorithm. The health worker should immediately be warned if the information she or he enters in the system does not fit with these specifications. For example, if the scope of a CDSA is limited to patients aged 6 months to 5 years and the user enters 2 months of age, the CDSA logic should prompt an incongruence of the data entry and this should be appropriately reported back to the user. In cases where an app proposes different algorithms for different use cases (ie, ANC, management of childhood illness), the specifications shall be clearly available for each individual algorithm. For example, if a CDSA allows for consultations related to ANC for pregnant women as well as for neonates, the algorithm should request specific patient information to triage patient along different algorithm branches respective to the use case (table 1).

Table 1

Minimal and optimal target product profile characteristics focused on the general scope of the toolkit and toolkit component characteristics, as defined by expert consensus process

General Scope
Characteristics	Minimal requirements	Optimal requirements	Comments
Intended use	The toolkit, composed of an electronic clinical decision support algorithm(s) and POC diagnostic tests, is intended to increase evidence-based treatment decisions by capturing diagnostic test results, patient clinical data (eg, exposures, signs including vital signs) and context-specific data (eg, disease incidence, seasonality) to provide treatment and care recommendations
Target population	The algorithm shall define the target populationInclusion and exclusion criteria are used at the encounter to enrol the patient		The system can be modular, that is, composed of discrete algorithms, such that one can be used for a specific population based on predefined criteria
Setting (level of implementation in the healthcare system)	Defined by the algorithm		The system can be modular, that is, composed of discrete algorithms, such that one can be used for a specific healthcare setting based on the infrastructure, workforce knowledge and skills, and POC tools available
Targeted end user	Defined by the algorithm		The end user shall have the required training/skills to use the app appropriately
Scope of toolkit components
Algorithm access	The electronic clinical decision support algorithm is accessible through an app downloaded on compatible target devices		The app can be a web app, a native app or a hybrid app
Algorithm content	Built on:  Well-established clinical evidence based on WHO/international/local clinical care guidelines, peer-reviewed articles (systematic reviews, original clinical research) and clinical experience/practice  And/or appropriate clinical validation research*		*See FDA’s SaMD clinical evaluation for guidance20
Algorithm treatment recommendations	Therapeutic recommendations shall be compliant with national treatment guidelines and national EML, and support antimicrobial stewardship	Same as minimal, and evidence-based medicine to support optimal treatment recommendations	Recommendations support the appropriate selection, dosage and duration of antimicrobial and any other kind of treatment and management, causing the least harm to the patient
Compatible POC tools	POCs or other relevant medical devices prompted for use by the app shall be locally relevant, that is, recommended by EDL or relevant national equivalent or country programme	Same as minimal, plus emerging diagnostic tools and medical devices relevant to the algorithm and implementation setting
Regulated toolkit components	POC diagnostic tests and medical devices are regulatory approved and compliant with local regulations	Same as minimal, and if the software is a medical device the app shall also have regulatory approval
Compatible devices	The app is compatible with any device, including:  Smartphones  Tablets  Computers		Computers are included as it may be necessary for health facility supervisors to access data collected at the facility level to make informed decisions (ie, restocking medical supplies)
Compatible operating systems	OS agnostic	Same as minimal

EDL, Essential Diagnostics List; EML, Essential Medicines List; FDA, Food and Drug Administration; OS, operating system; POC, point-of-care; SaMD, software as a medical device.

Clinical decision support algorithms

Regarding the algorithm’s medical content, the working group agreed that it must be evidence-based. Whether newly developed or adapted from existing guidelines, the algorithm’s medical content should be developed following international standards for the development or adaptation of evidence-based clinical guidelines, to ensure it implements the best available evidence. This evidence can originate from well-established clinical guidelines such as WHO/international/local clinical care guidelines, or from peer-reviewed articles, clinical practice and/or validation research, and the level of evidence and the strength of recommendations in the target context should ideally be communicated. In line with this, recommendations should cause the least harm to the patient and to the community. For example, algorithms related to treatment and management of a patient suffering from infectious diseases should also support antimicrobial stewardship to avoid unnecessary side effects for the patient and the development of resistance in the community and at the global level. In addition, the algorithm should be validated both analytically and semantically to ensure that the algorithm output is accurate and reproducible, does not deviate from expert content/evidence, and that there are no interactions or conflicts in the logic.

Furthermore, important characteristics were content quality, ML. With the rise of ML also in this domain of CDSAs to analyse the data captured through algorithms and then for example augment algorithm diagnostic or prognostic accuracy, workshop experts thought it important to address this in the TPP. It was agreed that an algorithm that recommends treatment decisions and management of patients should be at a minimum human-interpretable, meaning one should understand how input data are processed into output data. Changes to the algorithm logic based on ML analyses should also be validated (table 2).

Table 2

Minimal and optimal target product profile characteristics focused on the electronic clinical decision support algorithm and POC tools, as defined by expert consensus process

Clinical decision support algorithm
Characteristics	Minimal requirements	Optimal requirements	Comments
Content transparency	The algorithm is ‘human interpretable’. The healthcare programme and end user can comprehend the algorithm decision-making processes	The healthcare programme and end user have access to underlying evidence and methodology used to develop the algorithm	Human-interpretable: a human can understand the choices taken by the model in the decision-making process, that is, how output variables are generated based on input variables. Visual representations (eg, decision trees, principal component analyses, protocol charts and so on) and performance metrics can be used to support content transparency
Quality control	The algorithm has been (1) analytically and (2) semantically tested:  Analytical verification: the algorithm output is accurate and reproducible  Semantical verification: the algorithm does not deviate from expert content/evidence and there are no interactions or conflicts in the logic		(1) and (2) answer the question ‘did I build the model right?’(See FDA’s SaMD clinical evaluation for current guidance20)
Algorithm validation	The algorithm has been validated. The level of validation will depend on the CDSS status as an SaMDRefer to upcoming rulings from regulatory bodies such as the FDA or the European Commission		Answers the question ‘did I build the right model?’(See FDA’s SaMD clinical evaluation for current guidance20)
Machine learning	None, the algorithm is static	ML is applied to generate data on the algorithm performance, improve content, inform healthcare system processes and so onChanges in the algorithm based on ML shall be validated
POC tool
POC data inputs	Any kind of data (qualitative data such as positive/negative/invalid lateral flow test results, and quantitative data such as data provided by haemoglobinometers and glucometers)
Disease likelihood	Based on: Pretest probability (in the absence of POC or POC performance data) Or POC positive/negative likelihood ratio	Based on:  Pretest probability  And POC positive/negative likelihood ratio	The test performance (eg, likelihood ratio) is known and performance data ideally previously determined through independent studies in relevant settingsThe test brand should ideally also be considered so as to account for changes between manufacturers
POC training	On-site training performed by local authority or implementer

CDSS, clinical decision support system; FDA, Food and Drug Administration; ML, machine learning; POC, point-of-care; SaMD, software as a medical device.

POC tools

POC tools are defined here as medical diagnostic resources that can be used by health workers immediately at point of care to measure clinical parameters (ie, vital signs, host biomarkers and pathogen biomarkers) or visualise a patient’s tissue. They vary in design complexity (lateral flow vs table-top instrument), reagent, ease of use and cost. These tools are often budgeted and resourced by ministries of health but may not always be available in facilities, in particular at lower levels of the healthcare system. Using these tools during patient consultations and incorporating their data into the algorithms are dependent on their availability and whether the CDSA was designed to compute those data into triage or recommendation outputs. Experts agreed that the post-test probability or the likelihood of a disease being present should be assessed based on the pretest probability—that is, the prevalence of the disease in the corresponding patient population; and the diagnostic positive/negative likelihood ratio—that is, how good a test predicts the presence or absence of disease, of all clinical information (ie, signs, symptoms and POC results) (table 2). Clinical diagnosis often lacks accuracy, and therefore diagnostic tests that have the proven added value for patient management should be integrated within algorithms if the setting allows for it. These tests can be wet diagnostics such as pathogen-specific rapid diagnostic tests or handheld devices such as haemoglobinometers, and do not exclude digital diagnostics such as digital thermometers.

App

An important characteristic in this section is system validation. Today, there is no standard process to validate and assess the performance accuracy of CDSAs. There is also a severe lack of evidence and peer-reviewed publications on both the development and the performance of these tools in LMICs. While some groups have performed randomised controlled trials (RCTs) to measure patient outcome and other parameters that impact patient health, such as the reduction of antibiotic overprescription,16 17 others have assessed the performance of their tool by measuring health worker adherence to guidelines.12–14 The level of validation ultimately depends on the type of algorithm. The US Food and Drug Administration has provided some guidance on the kind of evidence that needs to be provided.19 The TPP criteria are inspired by this guidance. The minimal criteria guarantee that the app has a CDSA that is based on evidence (clinical association) and that input data are processed correctly (analytical validation). The optimal criteria include clinical validation, in addition to clinical association and analytical validation. Through clinical validation, the app has been assessed for the intended outcome (ie, improved clinical outcome, better quality of care, rational use of resources, reduction of antibiotic overprescription and so on) (table 3).

Table 3

Minimal and optimal target product profile characteristics focused on app characteristics, as defined by expert consensus process

App
Characteristics	Minimal requirements	Optimal requirements	Comments
System validation	The app has been validated in the intended implementation setting. There is evidence demonstrating: Valid clinical association (clinical output based on input data is supported by well-established or novel evidence) And analytical validation (input data are processed correctly into expected output data)	The app has been validated in the intended setting. There is evidence demonstrating: Valid clinical association (clinical output based on input data is supported by well-established or novel evidence) And analytical validation (input data are processed correctly into expected output data) And clinical validation (clinical safety or other meaningful outcome relevant to the intended use)	There is evidence that the system is based on evidence and working to achieve the intended use for the intended setting(See FDA’s SaMD clinical evaluation for current guidance20)
System access (public API)	Publicly available API for data access protected by authentication and authorisation. At a minimum, technical standards are adhered to	Optimally, HIE/HL7 standards are adhered to
Context configuration	Language: UN official languages  Local time  Local weights and measures	Same as minimal, and the following:  Option to customise to local official language  Other country preferences	Language can be a major barrier to the proper use of the tool for patient management and can lead to errors and misinterpretation
Customisation	Changes to the app, such as updates to the list of medicines and POCs available in the setting, can be made by the healthcare programme. Validation of this change shall be provided	Same as minimal
User access rights	Appropriate data access is provided based on specific roles		Roles may include data manager (facility supervisor) or data entry person (nurse)
Expert support	None	Built-in access to online/remote expert advice to assist in patient consultation via SMS, audio call and video conferencing
App training	On-site training	Same as minimal, and remote training and/or remote ‘Train the Trainer’
Internet availability	Functions offline (allows for service delivery and key workflows)  Allows automatic resynchronisation	Same as minimal, and trigger alerts on user device when data have not been synchronised for a long time	Internet connection can be very unstable, therefore the tool should work in offline mode so as to not disrupt the workflow of the user
Clinical data entry	Manual entry by the operator	Same as minimal, plus automatic upload of digital data (eg, from biosensors, medical devices)	Optimal: this allows external integration of other app modules, built-in and third-party apps and devices
Patient management recommendation	Consultation data are summarised and actionable recommendations provided (eg, treatment, referral, home care or follow-up)	Same as minimal, and recommendations are integrated in EMRs and HIS
Navigation	Sequential: the user follows a strict sequence of data input to reach a final recommendation	Non-sequential: the user can move in any direction through an assessment and change input data to reach a final recommendation
Workflow requirements to enable time-delayed POC data input	Ability for a user to perform multiple, simultaneous consultations, with pause and resume capability, to allow clinical and laboratory data entry	Same as minimal, plus ability to disable simultaneous workflow feature in settings with minimally skilled workers	This is particularly relevant for implementation in settings where testing and clinical consultations are performed in different locations
Task management	Multiple algorithms can be supported simultaneously in one application against a common data set		Can accommodate task-shifting capability, that is, multiple consultations can be opened at a time and patient profiles can be accessed using preset user access rights
Follow-up	None	Ability to retrieve patient information using patient registration information	The optimal implies data recoverability, also covered in the Data section
System malfunction protection	System malfunctions are made clear to the user
Scalability	The app shall allow high transaction volumes with complex workflows to cover primary care workforce at a national scale
Updates and versioning	Processes are in place to control any app changes (including algorithm version updates) and provide the appropriate and correct update to the user

API, application programming interface; EMR, electronic medical record; FDA, Food and Drug Administration; HIE, Health Information Exchange; HIS, health information system; HL7, Health Level 7; POC, point-of-care; SaMD, software as a medical device; SMS, short message service; UN, United Nations.

In May 2017, a new regulation (EU 2017/745) on medical devices, to which CDSA pertains by definition,20 was adopted by the European Commission, which will be applied starting Spring 2020.21 This regulation includes a reinforcement of the rules on the clinical evidence that should be provided by developers and manufacturers, at least if based in Europe. While waiting for LMICs to have their own regulations, this new regulation will certainly impact development and validation by European developers.

Experts also called for application programming interface to allow other software to ‘talk to’ and interact with the tool through commonly known programming languages. The app should be protected by authorisation and authentication that, as a minimum requirement, adhere to usual technical standards, and optimally would adhere to the HIE (Health Information Exchange) and/or HL7 (Health Level Seven) international22 standards. It was noted that adherence to HL7 can be expensive; hence, this was proposed as an optimal rather than a minimal requirement. The app should also be designed to integrate with patient care pathways and health worker workflows with limited disruption. The app accordingly has pause and resume capability to allow time for diagnostic tests to be performed and the results entered in the algorithm. POCs such as mRDTs can require up to 20 min for results to be read, depending on the manufacturer.

Data

Digital tools have the potential to amass enormous amount of data. With CDSA, patient personal data are captured, stored and potentially used for other operational (ie, supply chain management) and algorithm development purposes. Many countries have personal data privacy and protection laws in place to protect citizens from the abuse of their personal information. In the European Union this is known as the General Data Protection Regulation (GDPR).23 However, there are still many countries around the world that do not have a national eHealth strategy or personal data legislation. This called for the need to address personal data handling with CDSA for security and accountability purposes. Experts agreed that the implementer, who would be considered the data controller, is responsible for the data and should comply with local data policies and legislations. No specific legislation was mentioned as this differs by country and region. In addition, the history of the processes that are applied to data as well as the origin of data should be documented (data provenance). The app should function under secure connectivity to avoid loss and corruption of sensitive data, and mitigate cyberattacks, whether data are at rest or in transmission (table 4).

Table 4

Minimal and optimal target product profile characteristics focused on data characteristics, as defined by expert consensus process

Data
Characteristics	Minimal requirements	Optimal requirements	Comments
Data capture	Text, numeric, image, audio, video	Same as minimal, and GPS, barcode and biometric
Data validation	The system validates data entry to prevent errors that diminish value of the data or the outcome
Data ownership	Ownership shall be determined by the healthcare programme		The healthcare programme is responsible for compliance with any country law, policy and regulation
Data storage	The healthcare programme shall be able to choose the destination of the app’s data	Same as minimal
Data recovery	Data can be recovered or the system can be re-established to the desired state in the event of interruption or failure
Data flow	The flow of data shall be determined by the healthcare programme	Same as minimal
Data reporting	Data export available from all target devices	Prebuilt data reporting, analytics and dashboards are available with the app	The level of data manipulation, aggregation and reporting should be sensitive to the device the app is running on, that is, the computer app can be rich in functionality, and the mobile app is optimised for data collection and exchange only
Data provenance	Included	Same as minimal	Provides origin and processes applied to output data. When data are downloaded or shared, the version of the model is tagged so it is always clear how the data were obtained
Data dictionary	Available, referencing standards used (eg, ICD, SNOMED)		Ensures indicators reported are uniform across different health programmes
Data security and privacy	The app operates under secure connectivity which meets data protection and regulations of individual countries to avoid loss and corruption of sensitive data, and mitigate cyberattacks, whether data are at rest or in transmission.Conforms to national privacy laws. Includes processes such as:  Two-factor authentication  Authorisation/access control  De-identified data  Data encryption		Encourages GDPR (should no national data security policies exist) to ensure a system that:  Preserves data integrity  Identifies and mitigates risks  Provides relevant parties security processes

GPDR, General Data Protection Regulation; GPS, global positioning system; ICD, International Classification of Diseases; SNOMED, Systematized Nomenclature of Medicine.

Discussion

In this work, our aim was to develop a TPP to enable the development of a toolkit that includes CDSAs that are based on evidence and designed to integrate the results of the diagnostics performed at point of care to guide rational clinical decisions. The use of such toolkit would aid in strengthening healthcare particularly at the community and primary care echelons, and help address health worker difficulties in differentiating diseases and syndromes without diagnostic tests and the aid of formalised diagnostic processes, which can be provided through algorithms.

In terms of TPP characteristics, ‘Data’ and ‘Validation’ emerged as key topics of discussion. Data generated from CDSA require storage and connectivity and are therefore vulnerable to abuse, loss or unlawful purposes. Although many countries have enacted personal data protection legislation, many fall short on enforcing these and others are still drafting bills. For example, in the African region, in 2017, 17 (out of 54) African countries had put in place comprehensive personal data protection legislation.24 Several regimes in the region are starting to draft legislation on processing and movement of personal data25; however, funding attrition and the lack of regulators cripple the process. That said, there are continent-wide initiatives that promise to set data protection framework, such as the EU’s GDPR. The African Network of Personal Data Protection Authorities, a cooperation of eight African countries, seeks to draft data protection laws, formulate opinions on specific issues and establish a consultative framework on data protection. There are also regional conventions for data protection and privacy that have been enacted, such as the Southern African Development Community Model Law26 for the south and the Act on Personal Data Protection within the Economic Community of West African States for the west.27 Unlike their southern and western counterparts, east African nations have not adopted similar regional frameworks, which can set cross-border and data portability limitations. In addition, some countries’ national laws fall short in setting the required safeguards for data privacy breach and data portability and lack bodies such as data protection authorities to enforce these legislations.24 This is just the African scenario, but every country and region will have their own set of policies. Therefore, the healthcare programme and data controllers have an important responsibility to manage the programme in compliance with local laws or, where lacking, consult with government authorities on which regulation to abide by to foremost protect patient rights. The regulation of CDSS is also a very contentious topic, and relevant groups will have to watch the landscape closely to determine whether their tool fits the definition of a software as a medical device and what data they will need to generate for regulatory approval.

In terms of ‘Validation’, the big question was how to ensure CDSAs are safe for use. Developers and implementers should aim for clinical association, analytical validation and clinical validation. However, clinical validation can require significant amount of financial resources and time. As an example, e-POCT (electronic point-of-care test) was developed in 2014 to incorporate the latest scientific evidence to expand the medical content of IMCI. Its safety and efficacy were determined in an RCT in Tanzanian outpatient clinics.17 However, in 2019, the tool has not yet been implemented at scale or evaluated in non-controlled settings. Efforts like e-POCT to optimise CDSA content and assess its impact on patient health outcomes are needed to grow the evidence for CDSA’s role in improving patient care. However, providing these tools to patients also demands that we put our efforts in studies to measure the effectiveness of CDSA while in the hands of the end user and in the setting of intended use. Indeed, when CDSAs are implemented as part of routine practice, there may be factors that affect the overall effectiveness and impact of the tool. These include lack of or poor health worker training, lack of connectivity, software malfunctions, and poor design, leading to low usability and acceptability of the tool by health workers. Cycles of evaluation and improvement can be sought to perfect these tools in intended settings. In addition, digital data obtained from CDSA validation can be used in turn to dynamically improve the tool itself, faster than development improvements in classic wet diagnostics. CDSAs also have the potential to serve as platforms to support the evaluation or development of new content or guidelines. These can be adapted more quickly to different geographies to conform to local policies or support their update.

The call for the development of PHC during the Declaration of Astana revived the global commitment to provide quality health services to all.28 Achieving universal health coverage will therefore require that digital health interventions are quality-assured and evidence-based. Indeed, the WHO has begun developing frameworks to create a common digital language and synthesising the evidence around emerging mobile-based digital health technologies through two recent publications.8 29 These resources are meant to guide the global health community in assessing digital interventions that will improve quality of care, meaning interventions that are safe, effective, timely, efficient, equitable and people-centred, and these include CDSAs.30

Despite best efforts to balance expertise and knowledge as part of the multidisciplinary group to build consensus around a multicomponent tool, it is possible that some bias was introduced in certain sections due to a lack of expert input. A further limitation is that this TPP does not address some classic diagnostic characteristics, namely specimen type, performance accuracy (ie, sensitivity and specificity), operating conditions (ie, humidity and temperature) and cost. Since the tool is intended for an unspecified number of use cases due to the vastness of public health issues faced in healthcare systems, it was deemed expansive to define the multitude of different performance combinations. Further, TPPs usually include a cost estimate, but we could not estimate the cost of such a toolkit because again the use case and the setting of use are not fixed. The use of this kit would likely require a service delivery model attuned to the country of implementation.

Conclusion

The management of patients in low-resource settings with complex epidemiology is extremely challenging without clinical algorithms and accurate and portable rapid diagnostic tests. The consequences of suboptimal quality of care can go even beyond the patient’s health to include public health considerations, such as the development of antimicrobial resistance due to overprescribing of medicines. The aim of the proposed TPP is to support developers and implementers of these toolkits that are designed to guide health workers throughout clinical assessments, to give them confidence in clinical decisions and actions, whether it is sending a patient to a hospital or not, prescribing or not prescribing antibiotics, or recommending rehydration at home.