Research protocol for the validation of a new portable technology for real-time continuous monitoring of Early Warning Score (EWS) in hospital practice and for an early-stage multistakeholder assessment.

INTRODUCTION: The real-time continuous monitoring of vital parameters in patients affected by multiple chronic conditions and/or COVID-19 can lead to several benefits to the Italian National Healthcare System (IT-NHS). The UBiquitous Integrated CARE (UBICARE) technology is a novel health digital platform at the validation stage in hospital setting. UBICARE might support the urgent need for digitalisation and early intervention, as well as minimise the face-to-face delivery of care in both hospital and community-based care settings. This research protocol aims to design an early-stage assessment of the multidimensional impact induced by UBICARE within the IT-NHS alongside technology validation in a hospital ward. METHODS AND ANALYSIS: The targeted patients will be medium/high-risk hypertensive individuals as an illustrative first example of how UBICARE might bring benefits to susceptible patients. A mixed-method study will be applied to incorporate to the validation study a multistakeholder perspective, including perceived patient experiences and preferences, and facilitate technology adoption. First, semistructured interviews will be undertaken with a variety of stakeholders including clinicians, health managers and policy-makers to capture views on the likely technology utility, economic sustainability, impact of adoption in hospital practice and alternative adoption scenarios. Second, a monocentric, non-randomised and non-comparative clinical study, supplemented by the administration of standardised usability questionnaires to patients and health professionals, will validate the use of UBICARE in hospital practice. Finally, the results of the previous stages will be discussed in a multidisciplinary-facilitated workshop with IT-NHS relevant stakeholders to reconcile stakeholders' perspectives. Limitations include a non-random recruitment strategy in the clinical study, small sample size of the key stakeholders and potential stakeholder recruitment bias introduced by the research technique. ETHICS AND DISSEMINATION: The Ethics Committee for Clinical Experimentation of Tuscany Region approved the protocol. Participation in this study is voluntary. Study results will be disseminated through peer-reviewed publications and academic conferences. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Multimethod approach assessing the impact of UBiquitous Integrated CARE (UBICARE) into the Italian National Healthcare System.

Hospital-based validation of the two UBICARE components: the real time-Early Warning Score system and the continuous non-invasive blood pressure module.

Generation of multidimensional and multidisciplinary evidence focusing on clinical, organisational and human factors based on multistakeholder view.

Patient and public engagement by incorporating the perceived user experiences and preferences into the early-stage assessment.

Small sample size leading to limitation in the generalisability of results.

Introduction

The continuous monitoring of vital parameters, especially for patients affected by multiple comorbidities or multiple chronic conditions with a greater need for early intervention and special care, can lead to several benefits in terms of health outcomes and the overall economic sustainability of the Italian National Healthcare System (IT-NHS). To achieve this goal, innovative, portable and affordable biomedical technologies that effectively support the automation of several tasks within existing clinical pathways are deemed as essential.1 Moreover, at the time of starting this study, COVID-19 is spreading rapidly across Europe and growing in severity. This pandemic shows the urgent need for exploiting innovation and digitalisation in prevention and care. It is necessary to accelerate the translation and adoption of new or novel portable technologies and telemedicine systems able to minimise the face-to-face delivery of care and support clinical decision-making within a community-based care.2

As outlined in figure 1, the UBiquitous Integrated Care (UBICARE) technology is a novel health digital platform able to detect and report real time the early signs of clinical deterioration using an integration of three systems: (1) a wearable monitoring system (UB1); (2) a digital health platform (UB2) and (3) a mobile monitoring system (UB3).

Figure 1

Architecture of the UBICARE technology. UBICARE, UBiquitous Integrated CARE.

The core of the wearable component (UB1) is a medical device (MD), CE marked, constituted by multiple sensors enabling the continuous acquisition of breath frequency, oxygen saturation, temperature, hearth rate, respiratory rate, ECG, as well as the discrete cuff measurements of the blood pressure (BP). Moreover, UB1 has been equipped by the following components whose validation in a hospital ward is one of the primary goals of this protocol:

A software dedicated to the estimates of the real-time-Early Warning Score (rt-EWS) and responsible for data transfer. The model adopted is the Modified EWS (MEWS) being most frequently validated and used in hospitals to identify clinical deterioration.3

A prototype sensor for the continuous non-invasive BP (cNIBP) monitoring.

The other two UBICARE systems (UB2, UB3) were designed to suit health professionals’ specific needs. While UB2 enables to triage patients, carry out admissions and discharges, register new data and access historical data, UB3 is dedicated to data display and management of notifications and alerts. Thus, health professionals can always access information and easily monitor the patients’ health status at the remote station and/or on the move (ie, via smartphone, tablet or other mobile devices).

Several wearable devices equipped with vital sign sensors have already been introduced in hospital settings,4 5 and particular emphasis was taken by devices for non-invasive continuous BP measurement. However, existing solutions offer a system that is fragmented, and from which is extremely difficult to obtain an overall view of the patients’ health condition. There is a need for an integrated system, which is easy-to-use, standardised and able to provide mobile features. The UBICARE technology can bring several benefits to patients and health professionals: (1) remote, continuous and real-time monitoring of vital signs enabling prevention, early intervention and special care; (2) overview of the historical patient data/health status that can be always consulted by health professionals; (3) alert system allowing early intervention in the event of rapid worsening of the patient’s health condition; (4) suitability in both hospital and community-based care settings. These unique characteristics lead UBICARE to be a promising clinical decision support tool to be also used in the care of COVID-19 patients, which are in isolation at home or in other facilities, such as sanitary hotels. The UBICARE technology system is currently in its final form (ie, at the eighth Technology Readiness Level) at the validation stage in hospital setting before launch into IT-NHS.

This protocol provides the design for the early-stage assessment of the multidimensional impact of UBICARE within the IT-NHS alongside technology validation in a hospital ward. The targeted patients for this early-stage assessment study are medium/high-risk hypertensive patients6 as an illustrative first example of how UBICARE might bring benefits to susceptible patients. To reach these aims, a monocentric, non-randomised and non-comparative clinical study will be conducted and a qualitative approach using a multistakeholder perspective will be adopted to understand and facilitate the potential adoption of such a new clinical decision support-tool into clinical routine practice. In the last decade, there has been an increasing use of multi-dimensional evaluation tools, such as Health Technology Assessment (HTA), to lead to effective and affordable biomedical technologies to public, patients and society more rapidly. HTA is the systematic evaluation of the clinical, health economic, societal, legal and ethical issues related to the introduction, dissemination and use of a medical technology7. It serves to generate and synthetize multi-disciplinary evidence to inform health policy, resource allocation and clinical decision-making.7 Recently, there has been an increasing understanding of the need to seek a broader approach in HTA by considering additional parameters of benefit to the traditional ones (i.e. clinical and economic aspects).8 Indeed, organisational needs and human factors (eg, usability and user experiences) are extremely relevant when assessing innovative MDs, especially to support their translation into clinical practice. Anticipating the evaluation of end-user preferences, including an early evaluation of the likely organisational change induced into existing pathways and processes, has been effective to increase the probability of a better technology acceptability into real world.7 Moreover, assessing organisational needs and end-user requirements as soon as possible before adoption into routine clinical practice can enhance device performance-related outcomes reduce human errors, training time and mental efforts.9 10

Objectives

Specifically, this research study protocol aims to:

Validate the two UBICARE components (ie, the rt-EWS system and the cNIBP module) in a hospital ward.

Understand key drivers and barriers to adoption in a hospital environment. This will be achieved by:

Identify key stakeholders involved in the adoption, use and management of UBICARE.

Map the current clinical pathway for hypertensive patients in a hospital setting, eventually include those affected by COVID-19 to support the delivery of emergency medicine, and understand the potential organisational change induced by UBICARE.

Explore acceptability/usability requirements of end-users (ie, patients and healthcare professionals).

Obtain feedback on the potential clinical utility and economic sustainability of UBICARE, and gather views on the export of the technology outside hospital in order to support a community-based care and manage the most risky patients eventually including those hit by COVID-19.

Methods

Study design

Brainstorming with the clinical leads and technology developers started in June 2019 and informed study design. A mixed-methods study will be undertaken in three stages (see table 1). Stakeholder recruitment started in early 2020 and semistructured interviews (stage 1) are ongoing. The clinical study (stage 2) was planned for the third quarter of 2020. However, there has been a delay due to the COVID-19 emergency. The multidisciplinary-facilitated workshop (stage 3) is planned for the second semester of 2021.

Table 1

Study outline

	Method	Key stakeholders	Patient public involvement	Evidence generation/assessment
Stage 1	Semistructured interview study	Physicians and health professionals; hospital administrative staff; health managers and policy makers	No	Clinical utility; organisational/economic impact of hospital adoption; potential scenarios of adoption
Stage 2	Clinical study supplemented by standardised questionnaires	End-users (ie, hospital health professionals and patients)	Yes	Accuracy; clinical benefits; usability/acceptability; end-user experience/ satisfaction
Stage 3	Facilitated group workshop and dissemination	Health professionals; health managers; policy-makers; representatives of patient groups/ organisations	Yes	Multidimensional impact of hospital adoption; barriers and facilitators to adoption; alternative adoption scenarios

The study description is reported in accordance with the GRIPP2-sf checklist.11

Stage 1 will consist of semistructured interviews with a variety of key stakeholders for technology translation into the IT-NHS. The semistructured interview study will capture stakeholders’ views on the likely technology utility, economic sustainability, impact of adoption in a hospital setting and alternative adoption scenarios. Patients and patient representatives will not be involved at this stage. In stage 2, a monocentric, non-randomised and non-comparative clinical study will validate the use of the two abovementioned UBICARE components in a hospital ward. The study will be supplemented by the administration of standardised and paper-based questionnaires to patients and hospital health professionals to gather their perceived user experience/satisfaction, usability and acceptance of UBICARE. In addition, patients will be asked to report their views about alternative scenarios for technology adoption. Finally, stage 3 will be a multidisciplinary-facilitated workshop with IT-NHS relevant stakeholders and patient representatives to discuss results of the previous study stages and reconcile stakeholders’ perspectives. Patients recruited in the clinical study, as well as representatives of patient organisations (eg, Italian Pulmonary Hypertension Patient Association), will be actively engaged in stages 2 and 3, respectively.

The content of the semistructured interviews (stage 1), as well as the supplementary questionnaires to the clinical study (stage 2) were validated by incorporating clinical, ethical and technical feedbacks from the project partners.

Stage 1: semistructured interviews

A semistructured interview study will be undertaken with key stakeholders for the management, adoption and use of UBICARE in the IT-NHS hospital setting to collect primary data. The study will include a variety of stakeholders allowing for a multi-dimensional prospective of the potential impact of UBICARE, but at the cost of a small sample size in each stakeholder group.

The study sample will include participants from each of the following stakeholder groups: (1) Primary care physicians; (2) Cardiologists; (3) Other specialists; (4) A&E doctors and nurses; (5) Members of the hospital administrative staff (including clinical engineers); (6) Health managers and policy-makers. Most clinicians and members of the hospital administrative staff will be invited to participate among those employed at Fondazione Toscana Gabriele Monasterio (FTGM) where the clinical study (stage 2) will be carried out. We will make every effort to recruit health professionals working in other hospital centres. In additional, we will consider a minimum number of two interviewees from each stakeholder group to set a feasible sample size, given time and resource. However, the final number of participants will be piloted by the interview results, as more interviews will be carried out until no new themes emerge. We are aware of the existence of potential recruitment bias introduced by the typology of research, as stakeholders with a constructive approach towards biomedical innovation may be more likely to be engaged in the study.

Stakeholders will be interviewed individually through semistructured interviews conducted face to face or remotely by phone. Interviews will last for approximately 30–40 min, will be digitally recorded, and accurately transcribed for text analysis. Stakeholders will be asked to provide an overview of current pathways and processes involved in managing hypertensive patients at the FTGM. Their views will be also captured about the potential impact of adopting UBICARE in a hospital setting employing the FTGM as an illustrative case study. Indeed, in the core part of the interview, participants will be asked a series of open questions regarding how the technology might be best utilised in the hospital routine practice, as well as its extension to home setting and other facilities (eg, sanitary hotels and residential care homes). Alternative adoption scenarios outside hospital settings will be explored in order to satisfy the urgent need for prevention, telemedicine and community-based care by monitoring the most vulnerable patients including those infected with COVID-19, who present increased risk for an unpredictable evolving of the disease. Participants will be encouraged to consider whether the device might replace existing MDs within the current clinical processes and pathways, or whether it would necessitate changes to fit with processes and pathways. To support this discussion, the researcher will ask professionals to think about the main activities, individuals, technologies and other aspects of the IT-NHS involved in managing a typical hypertensive patient. Finally, the professionals will be questioned about any unmet needs regarding the current pathways of hypertension, as well as the likely clinical utility, barriers for adoption, uptakes to these barriers, and alternative scenarios for the adoption of UBICARE. Participants will be also asked to reflect on, and provide feedback about, the potential economic impact of UBICARE on the IT-NHS sustainability (eg, in terms of potential time gained/taken when adopting the new technology) and the stakeholders likely to be most affected (eg, doctors, nurses, patients). A list of topics and reference questions (see online supplemental file 1) were developed starting from the EUnetHTA framework12 and will be used as a prompt track.

Stage 2: clinical study

A monocentric, non-randomised and non-comparative clinical study will be carried out at the FTGM, a monospecialty cardiac hospital in Pisa (Italy) over a period of 5 months.

The clinical staff involved in the study at the FTGM will have the responsibility to (1) inform patients; (2) collect their informed consent; (3) verify their eligibility for the study; (4) complete a paper-based case report form gathering demographic patient data and the values of the measurements undertaken in the hospital routine practice. Health professionals will be also asked to complete a short, standardised questionnaire regarding their likelihood of using and recommending UBICARE (see online supplemental file 2). The grade of perceived user experience, usability, acceptance and professional trust in UBICARE will be measured through an adapted version of the Unified Theory of Acceptance and Use of Technology (UTAUT) model on a 7-point Likert scale. The usability questionnaire is articulated in multiple sections corresponding to the five predictors of the end-user behaviour: (1) performance expectancy; (2) effort expectancy; (3) social influence; (4) facilitating conditions and (5) intention to use.13

Sample selection and eligibility

The target sample size will consist of 33 patients to ensure an adequate statistical power for the study purposes according to the International Protocol of the European Society of Hypertension14 and its revision.15 Moreover, patients falling under different BP categories within the BP range identified by15 will be consecutively recruited over a period of 5 months at the FTGM. More specifically, the number of patients recruited in each of the three Systolic BP and three Diastolic BP recruitment ranges must be from 10 to 12 individuals to guarantee a uniform distribution of BP values across a representative range, as shown by.15 The patients enrolled in the study must also meet the following inclusion criteria: (1) minimum age older than 25 years; (2) ability to provide informed consent; (3) able to carry out the procedures required by the protocol; (4) clinical stability; (5) hospitalisation for any pathologies, except from those listed in the exclusion criteria. Exclusion criteria concern concomitant and acute pathologies, such as arrhythmias or atrial fibrillation; acute renal failure; acute liver failure; pregnancy; skin injures making difficult wearing the technology. However, as we are facing the COVID-19 pandemic, even in its early phase, we are now considering to include also COVID-19 patients who show up at the FTGM with an increased need for continuous and systematic monitoring of vital parameters to support the delivery of emergency medicine.

Hospital-based validation of the two UBICARE components: the rt-EWS system and the cNIBP module

Hospital-based validation with the patients enrolled will be carried out to assess the accuracy of the two UBICARE components (ie, the rt-EWS system and the cNIBP module) compared with standard practice. During hospitalisation, patients will be asked to wear the wearable monitoring system (UB1) to conduct two separate tests: (1) the rt-EWS test, which will last for approximately 12 hours; (2) the cNIBP test, which will end after approximately 12 min. Therefore, for each UBICARE component, a different validation procedure will be carried out in hospital wards.

First, the validation of the rt-EWS system will be undertaken by comparing the measurements computed by UBICARE to the standard evaluations conducted by the FTGM staff (ie, gold standard). In this study, the minimum interval between the estimation of two consecutive MEWS scores computed by the UBICARE system will be set every 30 min. Moreover, the validation procedure (see online supplemental file 3) will be similar to the one used by Churpek et al to evaluate the accuracy of the MEWS score for detecting clinical deterioration in hospital wards.16

Second, the validation of the cNIBP module will be carried out by comparing the measurements computed by UBICARE to the measurements conducted by using aneroid or mercury sphygmomanometers as reference standards. The validation procedure will follow the validation requirements stated in the International Protocol of the European Society of Hypertension.15 The requirements, based on evidence from a large number of validation studies on BP measuring devices, are in terms of environment, subject enrolled, observer and BP category. The validation procedure (see online supplemental file 4) will be applied only to the recorded measurements, as well as require a supervisor and two independent observers.

Stage 3: multidisciplinary-facilitated workshop

Stage 3 will consist in a facilitated group workshop aiming to discuss the results of the previous study stages and reconcile stakeholders’ perspectives. The workshop will also focus on the discussion of the barriers and facilitators to adoption in a hospital setting, as well as alternative adoption scenarios into the IT-NHS. Participants will be asked to provide their views on the findings from stages 1 and 2, and further emergent themes will contribute to refine the results of the early-stage assessment study.

Influential stakeholder groups will be selected among those identified in stage 1, including patient representatives. Convenience sampling will be undertaken. A maximum number of 12 participants will allow for a round table discussion format. The facilitated workshop will be digitally recorded, and a text analysis will be undertaken in the same way as interviews (stage 1).

Patient public involvement

Patients and public will be actively engaged in stage 2 and stage 3 of this study.

In stage 2, the 33 patients enrolled in the clinical study will be invited to fill a lean and standardised questionnaire about their experience in the use of UBICARE (see online supplemental file 5). The questionnaire will be administered to the patient by the FTGM nursing staff at the end of two separate experiences with the new technology:

The rt-EWS test, which will last for approximately 12 hours.

The cNIBP module, which will end after approximately 12 min.

The questionnaire will ask patients to report their subjective experience with UBICARE and assess user satisfaction, perceived usability and technology acceptability. Patients will be also invited to report their views about potential barriers to adoption, uptakes to these barriers and alternative adoption scenarios for UBICARE. In addition to this, patients will have the opportunity to evaluate the potential use of UBICARE in a home setting, and report whether the availability of such a telemonitoring system at home might enhance daily emotional and psychological aspects (eg, chronic patients might feel relieved and more confident) resulting in a general improvement of quality of life. The patient questionnaire employed the UTAUT model13 and the Post Study System Usability Questionnaire (PSSUQ) standard17 18 to assess the grade of usability and user satisfaction, using a 7-point Likert scale. The literature documented the use of PSSUQ in case studies of innovative MDs.19

In stage 3, patient representatives of Italian associations for the management and care of hypertension will be invited to participate in a multidisciplinary-facilitated workshop using a round table discussion format. We will share and discuss the insights from the previous study stages to reconcile different stakeholders’ perspectives and further capture patient’s view.

Data management and analysis plan

The validation of the rt-EWS system will be investigated through receiver operating characteristic (ROC) curves and then estimating the area under the ROC curve (AUC) using the trapezoidal rule. Sensitivities and specificities of the cut points will be investigated with an AUC of at least 0.6 in compliance with Churpek et al.16

Interview scripts and questionnaire data will be managed and stored on the servers of Sant’Anna School of Advanced Studies in Pisa (Italy). The servers and storage platforms follow the European Directive 95/46/EC (General Data Protection Regulation).20 Patient data will be treated in a strictly confidential way and analysed in anonymous form. Indeed, no identification data will be saved in the survey questionnaires. The researchers of the Management and Health Laboratory of Sant’Anna School of Advanced Studies will be eventually able to contact patients only through the FTGM nursing staff to clarify any data discrepancies arose.

The researchers will proceed to analyse the collected data using three main analysis techniques: (1) process mapping; (2) thematic analysis of qualitative data and (3) descriptive statistics. Process Mapping will be employed to map out the clinical pathway of hypertension described by stakeholders. Two researchers with expertise in qualitative research will coanalyse the data independently to minimise bias. Initially, interview data will be coded based on the predetermined themes; it is expected that additional emergent themes will arise. If required, participants will be contacted to clarify issues in interpretation.

Interview data will be analysed with NVivo V.10.1.1 software (QSR International, Melbourne, Victoria, Australia). Statistical analyses will be performed using SPSS, V.19 for Windows (IBM). For statistical analyses, the CIs will be set at 95%. Descriptive statistics, such as frequency, mean, SD and percentages, will be used to describe patient demographics, and the grade of usability, acceptability and professional trust perceived by the end-users of UBICARE.

Ethics and dissemination

The protocol received the approval from the Ethical Committee for Clinical Experimentation of the Tuscany Region on 10 October 2019 (Prot: 2019/0027222/GEN).

The clinical study will be carried out in the scrupulous respect of the protocol and in compliance with the recommendations of Declaration of Helsinki and its revisions, as well as the ethical principles that inspire medical activity.

Participation in this study is voluntary for both patients and professionals. During stakeholder recruitment, all potential participants will make an informed decision about their participation and they have the right to withdraw at any time. Similarly, the FTGM health staff will allow patients to make an informed decision regarding their participation in this study. The right of the patient to refuse to participate, or withdraw at any time, will be respected without compromising the quality of clinical assistance received. The FTGM doctors can decide to suspend the procedure for a patient if there will be a suspicion of health risk, or in the interest of data quality produced by the study itself. No financial or other remuneration will be directly offered to participants.

The study results will be disseminated through peer-reviewed publications, academic conferences and formal presentations to patient representatives, health professionals and practitioners and policy-makers.

Limitations

This study design does have some limitations.

Patient recruitment in the clinical study will be decided and managed by the clinical investigator. We are aware that a non-random recruitment strategy may lead to potential confounders and risk of bias in the interpretation of the results of the clinical study.21

In addition to this, when facing with sparse, fragmented and multidimensional evidence reporting on the clinical translation of innovative MDs, the particular nature of the research may introduce several limitations that are common to qualitative techniques, such as sample size and recruitment bias.22 First, the small sample of key stakeholders for technology adoption may not be representative. Second, since study participation requires a time burden to attend interviews and group workshop, only the stakeholders with a positive attitude towards technology innovations may be more interested to spend time and be engaged. These limitations may be overcome by a random sampling approach that, unfortunately, would not be feasible in the present study.