A Collaborative Platform for Management of Chronic Diseases via Guideline-Driven Individualized Care Plans.

Older age is associated with an increased accumulation of multiple chronic conditions. The clinical management of patients suffering from multiple chronic conditions is very complex, disconnected and time-consuming with the traditional care settings. Integrated care is a means to address the growing demand for improved patient experience and health outcomes of multimorbid and long-term care patients. Care planning is a prevalent approach of integrated care, where the aim is to deliver more personalized and targeted care creating shared care plans by clearly articulating the role of each provider and patient in the care process. In this paper, we present a method and corresponding implementation of a semi-automatic care plan management tool, integrated with clinical decision support services which can seamlessly access and assess the electronic health records (EHRs) of the patient in comparison with evidence based clinical guidelines to suggest personalized recommendations for goals and interventions to be added to the individualized care plans. We also report the results of usability studies carried out in four pilot sites by patients and clinicians.

Introduction

A growing share of the population (15% in 2010) in OECD countries is over 65 and expected to reach 22% by 2030 [1]. Older age is associated with an increased accumulation of multiple chronic conditions (multi-morbidity), including a growing number of functional and cognitive impairments [2]. More than half of all older people have at least three chronic conditions, and a significant proportion has five or more [3]. Multi-morbidity creates diverse, and sometimes, contradictory needs, which challenge patients and the delivery of health services [4]. The clinical management of patients suffering from multiple chronic conditions is very complex, disconnected and time-consuming within the traditional care settings and, hence, currently those with chronic conditions and long-term care needs experience shortcomings and gaps in care provision.

Integrated care is seen as a means to transform health services to meet these challenges of 21st century, by addressing the growing demand for improved patient experience and health outcomes of multimorbid and long-term care patients [5]. Care planning is a central approach of integrated care, where the aim is to deliver more personalized and targeted care creating shared care plans that map care processes (care pathways) by clearly articulating the role of each provider and patient in the care process. In the state of the art practices, the multidisciplinary teams (MDT) meet face-to-face to discuss and revise the care plans of several patients at once, at regular time intervals; usually monthly. Individualized care plans are created by manually going over the standard steps of care pathways, i.e. template care plans which are documentation of the optimal management for typical, defined disease patterns. Although implementation of integrated care via these manual processes is already an enhancement over traditional fragmented care practices, we believe significant improvement can be achieved if intelligent collaborative tools can be developed to support both the care teams and also patients and their care givers. In this paper, we present the architecture and implementation of a coordinated care and cure delivery platform, as an integrated care management tool encompassing clinical decision support services for MDTs and patient empowerment tools for patients and their care givers (Fig. 1).

Fig. 1

The basic concepts of the C3-Cloud system.

In particular, we address the following bottlenecks in traditional care delivery mechanisms, through the automated tools and services we provide:

Traditionally, the adoption of clinical practice guidelines has been promoted for managing chronic conditions. However, although CPGs may contain some basic references, their scope is predominantly focused on single diseases, without sufficient consideration of co-morbidity and multimorbidity. Following more than one clinical guideline can result in inefficiencies for the patient and for the health system due to duplicated and inconveniently scheduled investigations and clinic visits and, more importantly, treatments that may adversely affect another condition [6,7]. Care providers are in need of clinical decision support services to detect and warn them about guideline conflicts, to select upon most suitable treatment options in the light of evidence based guidelines and to schedule and prioritize treatment activities. Our platform equips the MDTs with intelligent services to suggest personalized goals and interventions for the care plan of the patient based on the most recent context of the patient and evidence based guidelines. The coordinated care and cure delivery platform enables the MDTs to coordinate the execution and monitoring of the integrated care plans in close cooperation.

Managing multi-morbidity, through the current treatment methods, results in specialty silos involving multiple health and social care providers who are not effectively communicating and sharing information. As the number and complexity of health conditions increase over time and episodes of acute illness are superimposed, the type and number of care providers contributing to the care of individuals also increases. It becomes significantly more difficult to align and coordinate care across care teams and associated settings. This results in fragmented care, due to poor communication and information sharing. Without secure information exchange among the actors involved in health, social and informal care services and a process to reconcile potentially conflicting treatment plans, it is impossible to avoid redundant and potentially harmful interventions. Informed decision making also requires information to be shared between the regional/institutional Electronic Health Records (EHRs), Social Care Records (SCRs) and homecare services. C3-Cloud encompasses interoperability adapters that allow heterogeneous data sources to share their EHR data securely, and an online collaboration that offers MDT members a single coherent view (of that data). The interoperability architecture also enables the clinical decision support services to seamlessly access and assess the electronic health records (EHRs). In this way they can provide personalized recommendations for goals and interventions to be added to the individualized care plans of the patient.

Patients and their informal care givers including family members often do not have a Rvoice in the management of their own care. WHO reports that adherence to long-term therapy for chronic illnesses in developed countries averages only 50%, and is even lower in developing countries [8]. Adherence rates drop significantly in complex treatment and care regimes compared to simpler ones [8]; multi-morbidity together with the increased probability of poly-pharmacy reinforces non-adherence behaviour further [9]. Adherence to treatment regimes (lifestyle and drugs) necessitates behaviour change by the patients, which can be more difficult for the elderly. Patients and their informal care givers, including family members, need to receive complete information about the benefits and risks of treatments; and to be offered real opportunities for shared decision making, expressing preferences and engaging in self-management. Our solution provides a patient empowerment platform to ensure active participation of patients and their informal care givers to the management of their multi-morbid chronic conditions. Patient empowerment platform presents the care plan via quantitative and qualitative outcome goals and action items to the patients. It continuosly collects feedback from the patient to record their activities and problems they encounter, monitor risk factors through online patient-reported outcome measures questionnaires, and establish a two way communication between MDT members and the patients.

The paper is organized as follows: in Section 2, we briefly present existing research results in this field and explain how we complement and extend these work. Section 3 provides a detailed description of our architecture, elaborating the architectural choices and implementation strategies for each subcomponent. Section 4 reports the results of the usability studies that have been conducted. Finally, in Section 5 we present the planned future work, and elaborate on the advantages of our platform and conclude the paper.

Related Work

Clinical guidelines are used in the healthcare domain to improve the quality of care [10]. It has been demonstrated that clinical guidelines provided as real-time decision support systems improve patient care significantly [[11], [12], [13], [14]] and decrease undesired practice variability [15]. Yet, the success of clinical decision-support systems requires that they are seamlessly integrated with clinical workflows [16,17]. Several methodological approaches exist to implement clinical guidelines into operational practice. Narrative guidelines can be formalized via computer interpretable guideline representation languages such as Arden Syntax and PROforma [18]. These can be served as modular clinical decision support (CDS) services, that can be utilized by hospital information systems during patient treatment to provide alert and reminders about missing or contraindicating interventions (e.g., EBMeDS [19]). However, this does not directly support healthcare professionals to follow a standardized plan of care for a specific condition, as a clinical workflow. Clinical pathways are appropriate for that purpose; however clinical guidelines and care pathways are often viewed as separate entities, their synergistic potential remaining only partially exploited [20].

Clinical information systems have been built to automate care pathways to send reminders for providers to enable periodical assessments, diagnostic tests and treatments; data collection on process and outcome indicators for performance assessment; continuous monitoring of progress and information sharing, examples include InformaCare, Medix [21,22]. Yet, these do not include personalized clinical decision support in the light of clinical guidelines.

In this paper, we present an approach to effectively integrate clinical guidelines and care pathways: we show that it is possible to semi-automatically personalize care pathways to create individualized care plans, by automatically processing knowledge in clinical guidelines and patient's EHRs. In this way, this will enable following the recommendations of clinical guidelines as a clinical workflow executed via integrated care plans for addressing the demanding needs of patients suffering from long-term chronic conditions.

C3-Cloud System Architecture

The C3-Cloud project [23] aims to change the currently fragmented medical care provided for the patients suffering from multiple chronic conditions. It provides an ICT infrastructure for patients and multi-disciplinary care teams, to coordinate the integrated care for the patients in a patient centered fashion.

We have implemented a Coordinated Care and Cure Delivery Platform (C3DP) that allows collaborative creation and execution of personalized care plans for multi-morbid patients by a multidisciplinary care team (MDT) including GPs, specialists, study nurses, pharmacists, physiotherapists, geriatricians, nutritionists, social care and homecare workers. C3DP is the Web application for collaborative and personalized care plan management by the members of MDT. In the C3-Cloud architecture, C3DP sits at the top of the hierarchy and is directly integrated with all the other C3-Cloud components and indirectly with the local EHR/EMR systems of the pilot sites as presented in Fig. 2. All the patient data required for care planning are fetched from the C3-Cloud FHIR Repository, which is continuously fed with existing EHR data of the pilot sites via our interoperability architecture composed of the Technical and Semantic Interoperability Suites (TIS and SIS). With the help of Clinical Decision Support Modules (CDSM) automating multiple clinical guidelines, C3DP processes electronic health records of the individual patients and provides guidance to the multidisciplinary care team members for i) risk prediction and stratification, ii) personalized selection of treatment goals and interventions in the light of evidence based guidelines, iii) reconciliation of conflicting treatment options and iv) management of polypharmacy. Active patient involvement and treatment adherence is achieved through a Patient Empowerment Platform (PEP), ensuring patient needs are respected in decision making and taking into account preferences and psychosocial aspects. Finally, the Security and Privacy Suite (SPS) provides common security features for user authentication, authorization and audit logging to all of the other components.

Fig. 2

High level system architecture.

In order to ensure wide adoption, we have chosen to build a standards-based architecture, where widely accepted industry standards are chosen as building blocks of our implementation. In the following sub-sections, we first briefly present our interoperability infrastructure, then outline the details of the architecture sub-components which are integrated to implement an intelligent platform to support integrated care by enabling the personalization of care pathways as care plans dynamically.

C3-Cloud Interoperability Architecture

Aiming to orchestrate the care across multiple care givers and treatment sites, and automatically process patients' EHRs to be able to recommend personalized treatment goals and interventions, inevitably requires interoperability to exchange and seamlessly process medications, conditions, interventions, episodic treatment plans, preferences and patient reported data including sensor measurements. We have chosen to build our technical interoperability layer based on clinical resources and RESTful interfaces of the HL7 Fast Healthcare Interoperability Resources (FHIR) STU3 standards framework [24]. The C3DP accesses patient's most recent EHRs, through FHIR based interfaces implemented on top of the proprietary APIs provided by local EHR systems in our pilot sites.

Technical Interoperability Suite

The Technical Interoperability Suite (TIS) provides a standard-based data exchange protocol, in order to enable information exchange between local care systems and C3-Cloud components, such as C3DP. C3-Cloud is being piloted in three different pilot sites: a) Basque Health Service - Osakidetza [OSAKI], Spain; b) Region Jamtland Harjedalen [RJH], Sweden and c) South Warwickshire NHS Foundation Trust [SWFT], UK. The EHR API, data representation, and operational environment vary among local care systems, which hinders the processing of a unified record while creating the care plan. In order to provide maximum flexibility and extensibility, TIS is implemented as an extract, transform and load (ETL) software development kit (SDK). TIS utilizes the ETL model to pull patient data out of a local EHR system through its native API, convert the data into selected FHIR resources compliant with C3-Cloud profiles with support of the Semantic Interoperability Suite (SIS), and push the transformed FHIR data into the C3-Cloud FHIR repository. The core of TIS is an ETL engine, which is able to schedule and execute ETL tasks. TIS also provides an extensible library of functions, on top of which is easy to assemble an ETL task for integration with an EHR data source. TIS provides both a web-based user interface for system administrator to execute or schedule an ETL task, and a RESTful service API for other C3-Cloud components, such as C3DP, to trigger an immediate ETL action, so as to get the latest patient data.

A set of pipelines have been developed for addressing the diverse needs of the three different C3-Cloud pilot environments and the heteregeneous EHR system APIs provided. Fig. 3 summarises the patient data APIs that each pilot site exposes. The first pilot site (Basque Health Service - Osakidetza [OSAKI], Spain) provides two separate Web services: one for exposing medical summary of patient as HL7 CDA documents, and a second one to expose the lab results of the patients as a separate CDA document. The second pilot site (Region Jamtland Harjedalen [RJH], Sweden) provides 6 different RESTful services that expose patient data (patient demographics, diagnoses, lab results, medications, notes and encounters) via proprietary JSON documents. Finally, the third pilot site (South Warwickshire NHS Foundation Trust [SWFT], UK) provides patient data through two daily CSV exports; one from primary care system, and the second from secondary care and community care encounters. All the pipelines follow a similar pattern: retrieve patient data by patient identifiers; invoke SIS Structure Mapping Service to transform the data into FHIR resources; combine all resources into a FHIR transaction bundle; include an AuditEvent with timestamp in the transaction bundle; and commit the transaction bundle into the C3-Cloud FHIR Repository. If it is the first data import, i.e. the patient has not been created in the repository yet, TIS will add a FHIR Patient resource to the bundle, which includes C3-Cloud study identifier and evaluation group assignment information, and notify C3DP by sending a PatientCreated event through C3DP Event API. If an error occurs at any step of the pipeline execution, TIS logs the error in the database and presents it via the control panel.

Fig. 3

TIS interfaces to local EHR systems for fetching patient data.

Semantic Interoperability Suite

The Semantic Interoperability Suite (SIS) handles both structural mappings among different information models and resolves semantic mismatches due to the use of different terminology systems and different compositional aggregations, as used to represent the same clinical concept. Due to local implications of terminologies used, the SIS is developed in close relation with the pilot sites. Two different types of mappings are performed in the semantic interoperability suite: structural mappings and semantic mappings. Structural mappings are involved in the transformation between local pilot sites data in local format and FHIR resources data format used in C3-Cloud. Semantic mappings perform the transcoding between coding systems used in local sites and the C3-Cloud components.

The architecture of the SIS is provided in Fig. 4. SIS is articulated around two main sub-components: SIS Structural Mapper and SIS Semantic Mapper.

Fig. 4

Semantic interoperability suite architecture.

SIS Structural Mapper: The structural mapper of SIS is the internal SIS sub-component in charge of the generation of FHIR resources, which have to be filled with data provided in pilot site local format by TIS. To achieve its purpose, the structural mapper consists of pilot site dedicated local format mappers. These mappers provide precise mappings to create correspondence to every relevant data exported by the pilot site to its correct interpretation and place in FHIR resource. FHIR resources, mapped from pilot site data, are defined in the C3-Cloud data dictionary.

SIS Semantic Mapper: The semantic mapper of SIS is in charge of transcoding, using the HeTOP service [25], the vocabulary used to describe data exported by pilot site into standard codes that are used in the high-level components of C3-Cloud. A clinical concept mapping sheet is being maintained as the source of truth, which includes all the clinical concepts that are needed by the CDS services, in reference terminologies like SNOMED-CT, LOINC and WHO ATC, and all the local codes (e.g., Spanish and Swedish versions of ICD-10, completely local terminologies for laboratory tests) that are used by the pilot sites for these concepts. In total, 218 common clinical concepts including conditions, active ingredients of medications, procedures, lab results, vital signs, immunizations and family member history have been identified and bound to well-known terminology systems like SNOMED-CT, LOINC and ATC. It should be noted that this list includes not only leaf-level but also high-level concepts such as antihypertensive drugs (ATC:C02) and beta blockers (ATC:C07) or diabetes (SNOMED-CT:73211009); hence the number of leaf-level concepts in effect is much higher. These concepts have been mapped to 516 different codes from locally used terminology systems of three pilot sites. These local systems are composed of localized versions of international systems like Spanish and Swedish versions of ICD-10 and ATC in the case of Basque Country and Region Jamtland Harjedalen, and national systems like DBP codes for Spanish observations and READ codes for UK diagnoses. Mapping benefits from the implicit hierarchical relationship between high-level and leaf-level concepts in these local systems as well.

The Structural Mapper generates JSON encoded FHIR resources. The semantic mapping is based on a pre-filled registry containing, for each concept, the corresponding code(s) for each site's terminology, and the code used as reference by C3-Cloud. The registry is continuously updated via a dedicated service during the time of the project. Multiple codes can be specified for a single concept if the used terminology has several codes corresponding to the concept (narrower-than relation). Multiple terminologies are used as reference, in order to match each concept exactly. Both the Structural Mapper and Semantic Mapper provide a REST API for integration with other C3-Cloud components. An example mapping response, which is represented as an HL7 FHIR ConceptMap resource, is provided in Fig. 5.

Fig. 5

Semantic mapping example.

In this example, the request is to map 44,054,006 in SNOMED-CT terminology that is used as a reference for diagnoses in C3-Cloud to the terminology that is used by the Osakidetza pilot site, which is ICD-10 Spanish (ICD-10-SE) in this case. A JSON-encoded FHIR ConceptMap resource is provided as a response, describing the URIs of the input and output code systems (SNOMED-CT as the source and ICD-10-SE as the target), the input code and the corresponding code in the target system (ICD-10-SE code E11 for Diabetes mellitus type 2 in local language). It also shows the type of relation; equivalent in this case, meaning that the two concepts are identical.

FHIR Repository

The C3-Cloud FHIR Repository acts as the centralized data repository for existing clinical data of the patients and newly created care planning related data. It stores the data, which arrive from EHR systems via TIS and newly created or updated care plan data from other C3-Cloud components like C3DP and PEP, as HL7 FHIR resources. C3-Cloud FHIR Repository, onFHIR.io [26] is fully compliant to FHIR STU3 specification and implemented on top of MongoDB noSQL database. An authorized user or system can use native FHIR STU3 API, i.e. Restful interfaces to store/query/update/delete patient data. It is not possible to access any resource in the secured repository without first acquiring a valid access token. The authorization flow is fully compliant with the Smart App Authorization specification, which is based on the well-known OAuth 2.0 specification [27] as supported by C3-Cloud SPS module. Thanks to C3-Cloud FHIR Repository's automatic auditing functionality, audit trail records are kept for each access and manipulation of data as FHIR AuditEvent resources to ensure accountability. These audit resources are available from the same API for authorized users with administrator roles as any other FHIR resource.

Clinical Decision Support Services

The Clinical Decision Support (CDS) services are supporting modules of the C3-Cloud Coordinated Care and Cure Delivery Platform (C3DP). The CDS services enable the reconciliation of clinical guidelines for individual diseases, risk stratification, poly-pharmacy management and care plan goal setting and monitoring. As part of the C3DP platform, CDS services access, fuse and analyze patient data. This is accomplished in order to: perform risk assessment and stratification of candidate elderly people for inclusion in integrated care programmes; reconcile clinical guidelines for individual diseases to develop personalized care plans; detect and propose resolutions for guideline clashes; detect duplicate, unnecessary or contraindicating medications; and monitor and detect deviations from the outcome goals set in a patient's care plan. C3-Cloud focuses on elderly patients, who have at least 2 out of the following 4 chronic diseases:

Diabetes Mellitus type 2 (T2D)

Renal Failure (RF) (excluding Glomerular filtration rate (GFR) or estimated (eGFR) ≪ 30)

Heart Failure (HF) (including New York Heart Association (NYHA) Functional Classification I-II; excluding NYHA III-IV)

Depression (Dp) (mild/moderate conditions only)

The clinical expert group of the C3-Cloud project have examined the clinical literature, and have identified four NICE (National Institute for Health and Care Excellence) guidelines to be followed for the management of these four conditons as depicted in Table 1. NICE guidelines are already used in UK; the clinical experts from Spanish and Swedish pilot sites have reviewed them and provided the required local extensions. For example, while NICE guidelines suggest using’Atorvastatin 20 mg or 80 mg’ for lipid lowering, this recommendation has been modified as’Simvastatin 20–40 mg’ in Basque localization. Based on these selected NICE guidelines, CDS services have been implemented to support care planning for the 4 specific diseases and their combinations, as listed in Table 1.

Table 1

CDS Services implemented for each disease.

Diseases	Type 2 Diabetes	Renal failure	Heart failure	Depression
Selected Guidelines	NG28: Type 2 diabetes in adults [28]	CG182 Chronic kidney disease in adults [29]	CG108: Chronic heart failure in adults [30]	CG90: Depression n adults [31]
CDS Services Implemented	DM Blood pressure management	CKD referral	CHF vaccination	Depression assessment
	QRISK2 assesment and Lipid Management	CKD eGFR-control frequency	CHF Stability Review	Mild to moderate depression treatment
	HbA1c targets	CKD CVD prevention and treatment	CHF Diuretics Recommendation	Antidepressant treatment
	Blood glucose management	CKD blood pressure treatment	CHF Pharmacological Treatment
	Diabetic foot complication
	Diabetic nephropathy management
	Diabetic neuropathy management
	Diabetic retinopathy management
Generic Services for multiple diseases	Life style management
	Diet management
	Education

The clinical expert group examined these guidelines and have designed several flowcharts that can be used as a guidance to ease the development of care plans addressing the individual needs of patients [32]. As an example, 19 flowcharts have been designed by clinical experts covering the recommendations of NICE Type 2 diabetes in adults: management clinical guideline (NG28) [28]. These flowcharts constituted the basis for the CDS algorithms, which, in collaborations with the project engineers, have been implemented as a real-time executable CDS services. The ones that can provide computable suggestions have been identified and the inputs and possible outputs of these CDS services have been specified as FHIR resources. An example annotated flowchart in Fig. 6, depicts possible CDS recommendations about the required lipid lowering goals and interventions. The specifications of these CDS services have been validated once again by clinical experts. 7 different CDS services have been implemented automating the 19 flowcharts extracted from the NG 28 guidelines. These 7 CDS services implement a total of 80 different clinical rules, checking 108 different patient criteria, to recommend 119 different personalized goal and intervention suggestions.

Fig. 6

A sample flowchart for lipid managment CDS.

An uncritical combination of clinical guidelines for separate diseases when treating multi-morbid patients could have contradictions that would increase risk and in some cases even result in unfeasible treatment. There is a need for reconciliation to support clinicians in decision making in risk assessment, setting goals, choosing activities or pharmacologic treatment to include in a care plan of a multi-morbid patient. The reconciliation exercise we followed aimed to analyze the Disease-Disease, Disease-Drug and Drug-Disease interactions between the recommendations of given by the flowcharts designed to address the needs of single conditions. The final aim is to reconcile relevant recommendations for different chronic conditions by identifying the synergies, cautions and contradictions. The clinical expert groups have carefully analyzed the flowcharts identified for individual conditions, and checked the interactions in multi-morbid patients (patterns of double, triple or quadruple comorbidity). Potential conflicts were identified such as repetition, wrong sequence or overlaps of activities, contradictory goals, location inconsistencies alternative options, constraints, potential treatment synergies (either beneficial or harmful), outliers, and inconsistencies within pairs of pathways. The next step was to reconcile these recommendations to mitigate the conflict or inconsistency by modifiying them according to available knowledge (select, removal, merge, substitution, modify with extra input and output). Reconciled rules have been designed and integrated into existing flowcharts by modfying them where necessary. Altogether, 52 reconciled rules were defined: 50 rules for the two-diseases combination, 1 rule for three-diseases combination and 1 rule for four-diseases combination [33].

These guideline-based flowcharts and reconciliation rules are implemented as FHIR based the CDS Hooks services [34], with decision logic encoded in the Guideline Definition Language (GDL) version 2 [35]. GDL is a formal language used to express clinical rules and guidelines in a machine-readable format by leveraging semantically interoperable EHR standards. GDLv2 supports FHIR and CDS Hooks. The project uses the GDL2 Editor software to develop both the CDS guideline definitions and CDS Hooks services (Fig. 7).

Fig. 7

A screenshot of the diabetic foot problem guideline in GDL2 editor.

To be able to intelligently propose individualized goals and activity suggestions for the selected health concerns, C3DP is integrated with these external CDS services via CDS Hooks API. The electronic health records of the patient retrieved from local care systems, are passed as input to CDS services after the semantic mapping of the local codes into international code systems are handled as explained in Section 3.1. Within CDS logic, processing these patient specific diagnoses, lab results, medication data enables the selection of individualized goals and interventions for this specific patient. The response consists of textual recommendations communicated as information cards and computable recommendations communicated as suggestion cards in conformance to CDS Hooks API. In suggestion cards, the recommended goals and activities are represented as FHIR resources (such as MedicationRequest, Goal, Appointment resources) which can readily be included into the care plan model. The finalized personalized care plan can be shared back with the local EHR Systems by exporting the care plan as a FHIR CarePlan resource serialized as a JSON instance.

In addition to the guideline-based services, a RESTful service is developed to manage drug-drug and drug-disease interactions. The drug interaction service provides warning of potential adverse interactions between drugs, as well as a list of side-effects, for use by the rest of the C3-Cloud architecture. The service implements the interactions between drugs, as specified by the National Institute of Care Excellence's implementation of the British National Formulary (BNF) [36]. BNF is a pharmaceutical reference book, used by the UK NHS. The information provided by the service, is identification of potential adverse interaction between drugs, the effects of the interaction, the severity of the interaction, as well as the basis on which the interaction has been specified by the NICE-BNF. For example, Acarbose is a drug active ingredient Alpha-glucosidase inhibitor, commonly used by patients with type 2 diabetes, which reduces the effects of carbohydrates on blood sugar. Acarbose is listed as having a pharmacokinetic interaction with the active ingredient Digoxin used in patients with Congestive Heart Failure to improve quality of life and prevent hospitalisation. The interaction is listed as moderate in criticality, having an effect as decreasing the concentration of Digoxin. In addition to interactions, the service provides a list of side-effects for each substance, along with their frequency (i.e., common, uncommon and rare).

The information collected by the NICE BNF is encoded as a database defining the relationships between the main concepts e.g., interactions and drugs, and contains instances for each active ingredient. The current database contains 108,600 instances of interactions and 26,403 instances of side-effects for 1009 substances. The service is designed to receive a list of active ingredients of the drugs a patient may be taking. The list is checked against the database and returns a data object containing the interaction information as well as the list of side-effects. The information can be shown to the C3DP dashboard notifying the MDT members of potential risks.

Patient Empowerment Platform

The objective of the Patient Empowerment Platform (PEP) is to provide patients with access to the published care plan and its information and thus increase patient and informal caregiver participation in decision making [37,38]. PEP aims to provide computerised means to improve the interaction between patients and health professionals and to digitally collect relevant data and information to enable monitoring of care plan related activity status and progress. It directly interacts with C3DP to receive new and updated care plans, and to send back patient reported observations. It allows MDT members to assign questionnaires to patients to collect patient reported outcome measures, which are later filled in by the patient via PEP interfaces and shared back with C3DP through standard based interfaces based on HL7 FHIR. It, also, directly communicates with the supported set of sensor devices to record patient measurements. The core user functionalities and features provided to PEP users are:

Make published care plans available to the users.

Send reminders to patients to help them comply and stay on track with the interventions and activities included in the care plan.

Allow patients to actively collect data related to the care plan activities.

Allow health professionals and patients to communicate with each other using either messages or video appointments.

Provide patients with access to relevant self-management material.

Allow patients to provide feedback to MDT members about the care plan activities (such as reporting probable side effects of medications)

The PEP user application is a modern web application, which allows PEP Users (patients and their informal caregivers) to access all the functionality via web browsers (see Fig. 8). PEP is built on top of the Medixine Suite product. The Medixine Suite technology stack follows traditional logic for web-based services and consists of an OS (Microsoft Windows Server), data storage (Microsoft SQL Server), Web Server (IIS) and Programming platform (.NET). The database layer contains some supporting functionalities, but the main business logic is built into the application layer. The application logic is based on modern resource-based thinking and is accessed through a REST API that is structured around those resources. The core logic includes role-based access configuration for different operations, full audit trails of operations performed in the system, an application ecosystem model, dynamic and extensible data modelling tools, event subscription model for integrations and extensible support for multiple languages and cultures.

Fig. 8

Patient empowerment platform - care plan details screen.

Security and Privacy Suite

The Security and Privacy Suite (SPS) is responsible for authentication and authorisation of the care team members, while they are managing personalized care plans of patients and ensuring that all data exchanged within and across C3-Cloud software components is encrypted and properly auditable. In the C3-Cloud architecture, the patient's electronic health records received from the local EHR systems via the TIS, patient reported observations from the PEP, and the care plan of the patient managed through C3DP, are all managed in the C3-Cloud FHIR Repository. Hence, each of these client apps, i.e. TIS, PEP and C3DP needs to be authenticated and authorized to access (read, write, and update) patient data to the C3-Cloud FHIR Repository, via the functionalities provided by SPS. All such operations need to be logged for ensuring accountability via SPS. SPS enables authentication of the care team members into the C3-Cloud applications in two ways: i) via their already existing accounts (e.g., username-password) provided by the local authorities by integrating with the existing Identity Provider (IdP) systems of the pilot sites; and ii) by creating C3-Cloud specific user accounts for those users whose IdP's cannot be integrated with the SPS, for example, the social care workers. The SPS has three sub-components:

C3-Cloud SPS Server provides services for user registration, privacy policy management and endpoints defined in the OpenID Connect 1.0 standard to perform authentication and authorization (Authorization Endpoint, Token Endpoint, etc.). By implementing the OpenID Connect API, it serves C3-Cloud Identity Provider (IdP), which is the default IdP when the IdP of some users (e.g., social care workers) of the pilot sites cannot be integrated within the scope of the project. The SPS Server also manages the C3-Cloud Access Control Policy Store.

C3-Cloud SPS Manager is a web application for representing the functionalities of C3-Cloud SPS Server with the following user interfaces: single sign on UIs, policy management UI, client registration UI, user registration UI and audit viewer UI.

Audit Record Repository is a FHIR repository that maintains audit trail records implemented as FHIR AuditEvent resource. In C3-Cloud architecture, the C3-Cloud FHIR Repository is used as the Audit Record Repository. An extra instance of the same repository is not created for practical reasons.

Coordinated Care and Cure Delivery Platform (C3DP)

As depicted in Fig. 9, the Coordinated Care and Cure Delivery Platform (C3DP) is implemented as a Web application for collaborative and personalized care plan management by the members of a multidisciplinary team of care (MDT). All the patient data required for care planning are fetched from the C3-Cloud FHIR Repository, which is continuously updated with EHR data from the pilot sites via TIS and SIS. C3DP visualizes these data and helps the health professionals to easily manage the integrated care coordination process for multi-morbid elderly patients. C3DP implements the HL7 Care Plan DAM, and enables health professionals to design a care plan for a patient from scratch by selecting health concerns to be addressed from the EHR of the patient, and setting goals and activities to address the needs of this health concern. This process is formalized as a FHIR Care Plan resource, which consists of building blocks like Goal and different types of Activity resources (Fig. 10).

Fig. 9

Care plan summary screen of C3DP.

Fig. 10

Building blocks of a care plan.

The major functionalities enabled by the C3DP are:

Review of medical summar. C3DP provides a mechanism to review a complete view of patient's medical record. All the patient data that are provided by the local EHR/EMR systems, and also by the patient via Patient Empowerment Platform (PEP), including conditions, medications, allergies, lab results, vital signs, procedures and social history are presented to the care team members in a single location.

Preparation of an individualized care plan based on evidence based clinical guidelines. In addition to the addressed health concerns and risk factors, an integrated care plan is mainly composed of goals to manage the health concerns and the activities (i.e., interventions) to achieve the identified goals and improve the associated health concerns. In C3-Cloud, education materials for the empowerment of the patients are also part of the care plans, while they are treated separately from the rest of the activities. There are three ways of adding a goal/activity/education material to a care plan:

Manual entry from scratch: Care team members can, at any time, create a goal/activity/education material themselves via the C3DP user interface.

Recommendations from the CDS services: Personalized goal, activity and education material suggestions, provided by the CDS services according to patient data can be directly added to the care plan of a patient by the care team members, or after some modifications. Fig. 11 depicts a snapshot where personalized suggestions based on CDS recommendations are presented to MDT members.

Fig. 11

A snapshot from C3DP presenting personalized activity suggestions.

Transfer from the older care plan: When provided by the local systems of the pilot sites, it is possible to transfer existing goals and activities from a treatment plan of a patient into an integrated care plan during its initialisation.

Cross-check of all patient data that are needed as Input by the CDS services. The CDS services process patient EHR data to recommend personalized goal and interventions. However the missing or incomplete data can affect the correctness of a CDS recommendation. In order to address this challenge, all clinical concepts that may affect the CDS recommendation and result in adverse events are presented to the care team members along with contextual information that will help the user flag potential errors. For example, the lipid management CDS service checks the existence of three diseases (and some other parameters such as lab results and specific medications) in its decision tree: type 2 diabetes, chronic kidney disease and cardiovascular disease. The patient records retrieved from the local EHR system show that the patient has type 2 diabetes, but there is no information about the existence of chronic kidney disease and cardiovascular disease. The GP of the patient can declare that this patient has also a cardiovascular disease, which was somehow missing in the patient's EHR system records. Such newly provided patient data, via the C3DP interface, can be provided back to the original EHR/EMR systems.

Execution of a care plan. Integrated care planning is a continuous process. Ideally, an integrated care plan lives with the patient and is adjusted to the most recent patient context. It is updated during planned and unplanned encounters of the patient with health professionals and social care workers, and also with patient provided feedback via the Patient Empowerment Platform. All updates can be shared with the local EHR/EMR systems as well. Hence, execution of a care plan refers to the continuous follow-up and update of an integrated care plan. This can happen in a number of ways in C3-Cloud:

Updating the progress of goals and activities: The status of any goal or activity can be updated (e.g., a goal can be set as achieved or on-target) by a care team member. The patient can also provide feedback on their progress.

Re-execution of CDS services during planned and unplanned encounters: This is akin to the CDS service usage for the first time during initialization of a care plan. Relevant progress in the patient status is reflected in the recommendations of the CDS services.

Display of patient provided data: Patient and his informal care giver are active participants of the care planning process. Goals and activities are decided with his active involvement, and for an activity that is assigned to themselves, the patient is able to provide update via the Patient Empowerment Platform (PEP). Patient provided data includes questionnaire responses, medical device measurements (e.g., blood glucose, blood pressure), daily meal photographs and more. All patient provided data are matched with the corresponding care plan items and shown to the care team members.

Commenting on the care plan items: It is also possible to comment on specific goals and activities of a care plan, which are visible to the care team members.

Management of the care team. It is possible to invite new care team members to a care plan, during initialization or at any time. An invitation is subject to the confirmation of the invited care team member, who is informed via a notification in the system and an email depending on the preference of the pilot sites. The care team manager, who is always the GP of the patient in all 3 pilot sites of C3-Cloud, can also remove a professional from a care team, or a member may want to leave a care team. It is also possible for a health professional or social care worker to request joining an existing care team for a specific patient. Different roles can have different rights in the care team; for example, a nurse assistant or a social care worker can see a care plan but not modify it.

Communication among care team members and with the patient/informal care giver. C3DP has its own messaging module that enables safe messaging among all care team members, and also with the patients due to the integration between C3DP and PEP. HL7 FHIR Communication resource is used for messaging.

Dashboard view. Dashboard view enables a signed in care team member to quickly go over the important updates in the care plans of all her patients since the previous login, such as new messages received, awaiting appointments, new system notifications.

Patient provided data screen. This view collates all the patient provided data such as vital sign measurements, meal photos, feedback on the care plan and messages to the care team members in a single dashboard.

Activity calendar. It enables view and update of scheduled activities of a care team member on a calendar.

Real-time system notifications. Real-time system notifications are implemented for several events (e.g., for care plan update, new patient feedback, new message, invitation to a care team, etc.). When the user is already logged in to the system, such notifications are displayed in real time. It is also possible to access care team members via email for offline scenarios. SMS option was dropped by the pilot sites for real-time clinical notifications.

Usability Studies

To ensure an iterative and holistic approach, the evaluation and impact assessment of the C3-Cloud project has been split into four layers in accordance with the different stakeholder groups it effects and the different stages of development and deployment:

Evaluation layer 1 targeted C3-Cloud software component and application tests along defined protocols by making use of 5 health ICT experts from the University of Warwick, 26 patients and 22 MDT members in the pilot sites.

Evaluation layer 2 included a heuristic evaluation, a Nielsen walkthrough [?] and a questionnaire, in preparation for layer 3 evaluation. Layer 2 involves 5 health ICT experts from the University of Warwick, 27 patients and 20 MDT members in the 3 pilot sites.

Evaluation layer 3 will employ an exploratory technology trial that uses baseline and closure patient observations. Approximately 150 intervention patients and 52 MDT members will be involved in layer 3 evaluations by answering questionnaires and being involved in interviews.

Evaluation layer 4 will employ a predictive modelling tool to model the C3-Cloud impact when scaled up, using intervention and control patient data. Approximately 526 intervention patients and 62 members of the multidisciplinary teams will be involved in layer 4 evaluation by answering questionnaires and giving access to their anonymized EHR data. In addition, the data of 526 control patients will be used for data analysis.

Among these, we have completed the execution of first two layers, and evaluation layers 3 and 4 will be initiated in early 2019.

We aimed to ensure continuous, open feedback loops to the development team in evaluation layers 1 and 2 for the software improvement. In the first evaluation layer, we first carried out tests specified based on functional requirements of the software and performed application testing before deployment to ensure that all software components work well together. Results of this study is reported in [39]. In this section we will focus on reporting the usability studies realized in evaluation layer 2.

Usability testing is a process of tracking real users via a carefully designed protocol to test a system before and during deployment. This is useful to avoid technology-induced errors; identify issues and validate and improve the performance of a final product [40]. Kushniruk and colleagues argue that both commercial vendor based testing and in-situ testing are needed to ensure system usability [41].

The research objective of the presented approach is to identify and categorize early usability issues of the C3-Cloud components that are used by MDT members and patients. This objective served to answer the following research questions in accordance with the C3-Cloud research protocol [42].

How usable is the C3-Cloud application perceived by experts, patients and MDT members?

What usability issues can be identified that must be improved before deployment of the solution?

The following four methods were selected for our usability testing approach:

Method 1: A heuristic evaluation with health IT experts following the Nielsen walkthrough (Health ICT Experts). Heuristic evaluation is a usability engineering method which was comprehensively discussed by Jakob Nielsen in 1994 in his book, “Usability inspection methods” [?]. According to Nielsen, it is “a usability engineering method for finding the usability problems in a user interface design so that they can be attended to as part of an iterative design process”. In Nielsen walkthrough, multiple evaluators are involved (Nielsen recommends three to five) and the users are asked to discover the answer to given questions by using the system several times (at least twice) according to the storyboard.

Method 2: Spontaneous feedback gathering during the test sessions with MDT members and patients separetely (Patients and MDT members)

Method 3: Product reaction cards (Patients and MDT members)

Method 4: The QUIS7 questionnaire on user interaction satisfaction (Patients and MDT members).

Our usability testing approach is user-centric. The test sessions involved members of the MDT and patients from the target group spanning three pilot sites in the Basque Country (Spain), Region Jämtland Härjedalen (Sweden) and South Warwickshire (UK). Health ICT experts were recruited from the University of Warwick. All MDT members and patients were recruited among people in the prospective group of software users who speak English. For method 1, the health ICT experts had the chance to contact technical partners for any questions at any time. For methods 2–4 a language facilitator from each pilot site moderated each session and was available for any question that was raised from the participants. All participants received an introduction and a brief overview on the systems to clarify the test session objectives. The technical project partners developed a walkthrough for both the C3DP and the PEP. This walkthrough informed test participants about how to access the C3DP and the PEP and listed test user credentials for all participants. All participants received their own login credentials for the online High Level Component (HLC) demonstrators of the C3DP and the PEP. This was followed by step-by-step descriptions activities to be performed by all testers. Technical partners had ensured that all possible functionalities of the software were covered by the activities that the test participants followed. All test participants had followed these procedures for a two hour session.

The number of participants attended the test sessions of layer 2 evaluation are depicted in Table 2.

Table 2

Number of participants for the usability testing.

Pilot sites/Participant profiles	Health ICT Experts	Patients	MDT members
University of Warwick	5	–	–
South Warwickshire (SWFT)	–	13	12
Basque Country (BC)	–	2	6
Region Jämtland Härjedalen (RJH)	–	12	2
Total	5	27	20

Method 1- Heuristic Evaluation

Heuristic evaluation (HE) focuses on the interface of the system, in the C3-Cloud case the PEP and C3DP interfaces for patients and healthcare professionals respectively. It is performed by individual reviewers isolated from each other, and at the end of the process, results are collated and fed back to the developers. HE is a process that is part of the iterative development process of a system. HE can reveal a number of issues about the system. Examples of these include bad design that may lead the user making a slip or mistake, as well as design that may be seen to not appreciate the sensitivities of the user (e.g., system dialogues). All these issues are classified in a number of categories, which are the heuristic categories. Thirteen heuristics were considered in the C3-Cloud HE, including Visibility of system status, User controls and freedom, consistency and standards, Error recovery.

Five specialists, usability evaluation reviewers from the University of Warwick performed the heuristic evaluation, consisting of the following steps: (i) Reviewers attended a 30-minute session where the purpose of the evaluation, process and documentation were explained; (ii) Reviewers reviewed the PEP and C3DP manuals and walkthrough descriptions; (iii) Reviewers made a first structure-free evaluation of the interfaces; (iv) A second structured pass was done following the workflows in the manuals and comments were classified under each heuristic; (v) Based on the comments, reviewers completed a spreadsheet with common issues for each heuristic category, frequency and severity were combined to create an overall risk matrix that will prioritize modifications by the technical teams.

The results for C3DP and PEP are shown in Table 3, where table presents the distribution of % of usability errors, in each heuristic.

Table 3

Summary of overall usability issues.

Usability issues	Distribution of issues for C3DP (total = 196)	Distribution of issues for PEP (total = 67)
1. Visibility of System Status	13%	10%
2. Match Between the System and the Real World	19%	13%
3. User Control and Freedom	7%	7%
4. Consistency and Standards	20%	10%
5. Help Users Reconnize, Diagnose, and Recover from Errors	6%	7%
6. Error Prevention	9%	0%
7. Recognition Rather than Recall	3%	10%
8. Flexibility and Efficienct of Use	2%	3%
9. Aesthetic and Minimalist Design	2%	0%
10. Help and Documentation	4%	10%
11. Skills	3%	7%
12. Pleasurable and Respectful Interaction with the User	4%	9%
13. Privacy	4%	0%
14. Accessibility	5%	10%

Method 2- Spontaneous Feedback

The objectives of the test sessions were explained to all MDT members and patient participants and they were given an introduction in the softwares to be tested. Subsequently, they followed the activities shown in a two hour session. Any spontaneous feedback that was given during the test sessions in May 2018 was recorded and reported by the session moderator. Feedback was clustered for general feedback on the HLCs and feedback on specific functionalities (usability; care plan goals; care plan activities; terminology). The number of clustered comments recorded from the spontaneous feedback can be seen in Table 4. Duplicated feedback was not reported. Feedback was supported with screenshots when needed or useful. The software developers studied and prioritised the feedback using an internal issue tracking system. Prioritization was done for bugs, improvements, features and cosmetic changes. In total, 101 comments on the C3DP and 44 comments on the PEP platform were obtained by recording the spontaneous feedback of test participants. Testers were generally very positive about the C3-Cloud concept and experienced it as very promising, helpful and easy to use. MDT members have particularly liked the clinical focus of the platforms. The fact that the C3DP suggests goals and activities was experienced as being very positive and helpful. The software developers translated relevant issues into a tracking tool and collaborated with the projects' clinical reference group and the pilot sites to resolve open issues. Feedback responses were incorporated in respective activities of software development, software deployment at the pilot sites and training plans.

Table 4

Spontaneous feedback.

Category	C3DP Platform	PEP Platform
General feedback	11 comments	9 comments
Usability feedback	58 comments	28 comments
Care plan goals	5 comments	–
Care plan activities	17 comments	–
System terminology	10 comments	7 comments

Method 3- Product Reaction Cards

The “product reaction cards”1 is a fast and simple method used for an overall system evaluation. It allows the user to describe the system from a predefined set of 118 words (Fig. 12). This list includes positive words, together with negatives and neutral words. The main advantage of this approach is that it does not rely on a questionnaire or rating scales and users do not have to generate words themselves [44]. For the “product reaction cards” method, 48 people (22 MDTs and 26 Patients) from the three pilot sites: Osakidetza, RJH and SWFT; participated to the evaluation. The participants received login credentials for the online demonstrators of the C3DP (for MDTs) and the PEP (for Patients/ICGs) as well as training materials including a walkthrough that guided them through certain activities on the demonstrators The participants were asked to pick the words that best describe the C3-Cloud platform or how using the product made them feel. We limit the choice number to 5 words, as commonly used in such approach. At the end of the study, a scoring was made to identify the most commonly words by the participants to describe the system.

Fig. 12

Set of 118 words for the “product reaction cards” [43].

For the 22 participants of the MDT profile, 30% describe the system as “Collaborative”, 16% find it “Comprehensive”, 17% find it both “Empowering” and “Innovative” and 20% as “Time Consuming”. For the 26 participants of the patient profile, 25% describe the system as “Useful”, 21% find it both “Accessible and Convenient” and 17% find it Appealing and 16% find “Advanced”. In general, for all the 48 participants, the C3-Cloud system appears to be “Collaborative” at 23%, “Useful and Empowering” at 17%, “Innovative” at 15% and both “Complex” and “Comprehensive” at 14%. In this study, the “product reaction cards” method gives a good understanding of the user's experience. This method is easy and quick to conduct and permits to get user's experience with the system. When we collate the responses from all participants, results we obtained show that users pick either same card or a closely related card. Overall, this “product reaction cards” method is an early evaluation exercise. It permited to get first users' feedback and feelings. This helped the technical partners/component owners to further improve the C3-Cloud design, application and its implementation to better align with what the user's expectation.

Method 4- QUIS7

The Questionnaire for User Interaction Satisfaction (QUIS, 7th version) is a tool that measures attitude towards software interface factors: screen factors, terminology and system feedback, learning factors, system capabilities, technical manuals, on-line tutorials, multimedia, voice recognition, virtual environments, internet access and software installation. Respondends are asked to rate one or more questions for these categories on a 0–9 scale. The original QUIS7 questionnaire items were adapted to cater for the requirements of the software. Specific items were omitted as they were considered not useful for the evaluation at this stage of the project. In addition to the standard English and Spanish versions, the questionnaire was translated to Swedish for use at the RJH deployment site. After the test session participants finished the walkthrough, they were asked to fill in the QUIS7 online questionnaire anonymously. This early usability testing was performed with both MDT members for the C3DP platform and patients for the PEP platform. The results of the QUIS7 questionnaire are used for shaping the design and redesign of the platforms, detecting areas for usability improvement, and the comparative evaluation of the platform from its current status and later during the technology trial. While the QUIS7 follows a clear structure and helps identifying areas for improvement, it lacks detail and reasoning when certain aspects were rated less positive. Thus, method 2 (spontaneous feedback) complements this method for more specific insight.

For each QUIS7 category the mean rating per question was derived (see Fig. 13). In addition the mean rating, the standard deviation (STD), the distribution of ratings on a bar chart and a pie chart were derived per question for further detail (see Fig. 14). The bar-chart displayed only valid responses, while the pie chart shows all responses, including the percentage of non-responders. The bar-charts and pie-charts are colour coded orange for values from 0 to 4 and green for values from 5 to 9. The total number of valid patient responses for the C3DP usability testing is n = 20, all items that were rated lower than 6 in the mean; all items that had a STD larger than 2; all items that had a non-response rate of ≫15%. The total number of valid patient responses for the PEP usability testing is n = 26, all items that were rated lower than 5.9 in the mean; all items that had a STD larger than 2.3; all items that had a non-response rate of ≫20%.

Fig. 13

Learning (C3DP).

Fig. 14

Getting started is (difficult – easy) (C3DP).

Results

Usability studies held with MDT members and patients have shown that the proposed method is able to address the needs of care plan personalization via CDS services implementing clinical guidelines. Testers were generally very positive about the C3-Cloud concept and experienced it as very promising, helpful and easy to use. They have particularly liked how the tools were very clinically focused. The fact that C3DP suggests goals and activities was experienced as being very positive and helpful. Especially the two front-end facing components, C3DP and PEP, have benefited from feedback for improvements during the usability and application testing. Unstructured feedback, expressed through the think-aloud method by the test participants, has been particularly useful. Development teams have responded to the feedback received and incorporated them in relevant tasks, such as development, deployment and training.

Discussion and Conclusions

The C3-Cloud system has been co-designed and co-produced with the end-users from the very early stages of the project. In order to facilitate co-production, technical partners have first created user interface mock-ups (especially for C3DP) during the requirements analysis phase, and then created producing early prototypes starting from the architectural design phase, followed by several iterations and reviews by the end-users till the end of integration. This process has helped a lot in achieving easy adaptation and better acceptance of the end-users to the C3-Cloud solution. Usability studies held with the clinical experts with simulated patient data in evaluation layers 1 and 2 have shown that the proposed method is able to address the needs of care plan personalization via CDS services implementing clinical guidelines. Moreover, the recommendations from these usability studies have helped to improve further the user-facing components C3DP and PEP.

As the next step, the system will be operated and validated in real life within the scope of evaluation layers 3 and 4 to examine usability and acceptance of personalized care plans for chronic disease management in three pilot sites: Basque Country (Spain), Region of Jämtland Härjedalen (Sweden) and South Warwickshire NHS Foundation Trust (UK) via a 12 months pilot study to be carried out with approximately 62 health and social care workers including general practitioners, nurses and specialists and 526 patients using the C3-Cloud solution. The integrated solution has already been deployed to staging environments of all three pilot sites. Soon the final training activities with the actual users will take place and then pilots in operational environments will be started where patients will be directly involved.

Limitations, Challenges and Future Work

C3-Cloud is a very complex project with ambitious aims and developing a common solution for varying settings, restrictions and needs of three different pilot sites has been quite a challenging task. We have overcome these challenges by working closely with the end-users and their local IT teams, and loosely coupling the C3-Cloud end-user facing components with the local site EHR/EMR systems via a standards-based interoperability layer composed of an HL7 FHIR Repository and pilot site specific adapters transforming the data in local formats into FHIR resources compliant with the C3-Cloud profiles. It is a fact that, the interoperability layer requires manual activities especially for semantic mapping. Currently, the semantic interoperability challenges have been effectively dealt with by focusing on the specific data requirements of the targeted chronic diseases and associated CDS services implementing clinical guidelines, and validating all the relevant but limited set of terminology code mappings by clinical experts.

In addition to this, the reconciliation of the recommendations from multiple clinical guidelines had been a manual work that has been carried out by our clinical expert groups. As a future work we aim to explore semantic reasoning tools to semi-automatically detect possible clashes between the recommendations coming from different clinical guidelines.

Finally, although the data flow and transformation from the local EHR/EMR systems into the C3-Cloud solution is achieved completely, and the personalized care plans as the outcome of the C3-Cloud solution can be provided back to the local systems in a widely-used international standard (HL7 FHIR), it could not be possible to integrate the care plan goals and activities back to the local systems in the local formats due to legal and technical constraints. As a long term future work, this will be enabled for better exploitation of the system to third parties.