Improving health outcomes for young people with long term conditions: The role of digital communication in current and future patient-clinician communication for NHS providers of specialist clinical services for young people - LYNC study protocol.

BACKGROUND: Young people living with long term conditions are vulnerable to health service disengagement. This endangers their long term health. Studies report requests for digital forms of communication - email, text, social media - with their health care team. Digital clinical communication is troublesome for the UK NHS. AIM: In this article we aim to present the research protocol for evaluating the impacts and outcomes of digital clinical communications for young people living with long term conditions and provide critical analysis of their use, monitoring and evaluation by NHS providers (LYNC study: Long term conditions, Young people, Networked Communications).
METHODS: The research involves: (a) patient and public involvement activities with 16-24 year olds with and without long term health conditions; (b) six literature reviews; (c) case studies - the main empirical part of the study - and (d) synthesis and a consensus meeting. Case studies use a mixed methods design. Interviews and non-participant observation of practitioners and patients communicating in up to 20 specialist clinical settings will be combined with data, aggregated at the case level (non-identifiable patient data) on a range of clinical outcomes meaningful within the case and across cases. We will describe the use of digital clinical communication from the perspective of patients, clinical staff, support staff and managers, interviewing up to 15 young people and 15 staff per case study. Outcome data includes emergency admissions, A&E attendance and DNA (did not attend) rates. Case studies will be analysed to understand impacts of digital clinical communication on patient health outcomes, health care costs and consumption, ethics and patient safety.

Background

Young people living with long term conditions are vulnerable to health service disengagement and this endangers their long term adult health. This can be particularly problematic at the time of transition from paediatric to adult services. Poor transition can lead to disengagement from health services and poorer health outcomes.[1-3] For example, 35% of young renal transplant recipients lose their transplants by 36 months with a large peak of graft loss between the ages of 20 and 24 years;[4] The number of 20–24 year olds with diabetes having their HbA1c measured drops by >5% compared to the number of 10–19 year olds[5] and overall health outcomes compare poorly with those for an adult population.[6] With sickle cell disease, regular attendance at outpatient clinics and adherence to penicillin prophylaxis declines[7-9] in the context of 25% of sickle cell deaths in young people being linked to infection.[10]

Research suggests that service level factors that affect engagement with health care of young people with long term conditions include: poor patient–clinician communication; inflexible access to people and information; lack of person-centred health care and the need for continuity and relationship development.[11-13] Several studies report requests for the use of digital forms of communication – email, text and social media – with their health care team.[11,14]

The use of these methods of digital clinical communication is troublesome for the NHS. Most clinical professional bodies proffer caution in their use;[15] NHS Trusts lack information governance policies to regulate and monitor this clinical activity, and infrastructure to safeguard the use of digital clinical communications is locally determined. This context makes the real life evaluation of the role of digital clinical communication complex for clinicians, individual health care trusts and the NHS.

Evidence of the effects of digital clinical communication internationally, from systematic reviews across a range of long term conditions and across the lifespan, is equivocal, although no trials report poorer outcomes in the experimental digital communication arm. It is difficult to ascertain what contributes to positive effects on health outcomes where these are found, due to population heterogeneity and study quality. Several reviews found patient engagement with health care providers increased;[16-19] asynchronous communications were found to lead to greater improvements in glycaemic control and self-care outcomes and synchronous interventions were found to be more user friendly and more cost effective for patient and provider, with combined interventions leading to greatest quality of life improvements.[20] Negative impacts include increased depression, deteriorated parental relationships and information overload.[20] The evidence continues to have much uncertainty contained within it despite considerable research endeavour. However, the reviews highlight priority topics for future research to fill gaps in the evidence.

Eighteen published reviews were identified,[16-33] from which the following research recommendations were made: the importance of understanding what was important to patients, public and clinicians; costs associated with health care resource use by patients and health professional workload in meeting patient demand;[18,25] the information security, confidentiality and privacy issues related to digital clinical communication; the development of broader policy guidelines; the need to develop an evidence base across conditions and clinical contexts;[16,19,23,30] the need for a deeper understanding of these interventions, the moderators and mediators of change and the theoretical basis for assuming effectiveness and how all this links to desired outcomes such as A&E attendance, hospitalisations and clinical outcomes;[18,20,30] the need to explore any impact on the patient–clinician relationship, including how patients and clinicians negotiate health needs and health care. From a research design perspective, research involving a more diverse population and using qualitative methods was recommended. Our research tackles a number of the priority topics identified by these systematic reviews.

In this evaluation (LYNC study: Long term conditions, Young people, Networked Communications) of the use of digital communication between patients and clinicians on clinical matters in the UK NHS, we will study digital clinical communication technology as it is currently being used in its various technical forms and draw out results that are transferrable across technologies and across conditions. The digital communication ecosystem is rapidly changing[34] and this enables us to future proof our findings. Our patient and public involvement (PPI) activity with young people previously undertaken suggests that the comparison across different diseases of the use of digital clinical communication will provide important insights.

Aims, research question and objectives

The overall research question is: ‘What are the effects, impacts, costs and necessary safeguards for digital clinical communications for young people living with long term conditions and engaging with NHS providers?’ The research has two aims: (1) To evaluate the impacts and outcomes of digital clinical communications for young people living with a long term condition; (2) To provide a critical analysis of the use, monitoring and evaluation, of digital clinical communications by NHS providers.

The objectives are as follows:

To engage young people, including those with long term conditions, in the implementation of the research;

To evaluate and synthesise published evidence on the use of digital clinical communication by health professionals with young people with long term conditions;

To identify from the perspective of patients, clinicians, clinic support staff, clinical and IT managers and information governance specialists the issues, concerns, opportunities and solutions for the use of digital clinical communication in the NHS for a variety of clinical conditions;

To investigate the impact of digital clinical communications on health outcomes for young people with long term conditions and on their engagement with, and use of, health services;

To describe the cost of implementation and ongoing provision of digital clinical communication and how it varies across different clinical conditions, to understand the value of this service to patients and clinicians, to understand the cost of upscaling;

To recommend outcome measures for future cost effectiveness studies across disease areas;

To develop and disseminate guidance for NHS providers and commissioners on policy, procedures, service management and payback in return for investment and guidance on which clinical areas are most likely to benefit;

To consider the need for and design of future cost effectiveness research.

Study design

The research involves: (a) PPI activity; (b) literature review; (c) case studies – the main empirical part of the study – and (d) synthesis and a consensus meeting (see Figure 1). The design for the empirical case studies uses a mixed methods case study design. Qualitative data from interviews and non-participant observation of practitioners and patients communicating in up to 20 specialist clinical settings will be combined with quantitative data, aggregated at the case level (non-identifiable patient data), on a range of clinical outcomes meaningful within the case and across cases. We will seek to describe the use of digital clinical communication from the perspective of all the stakeholders in the clinic (patients/clinical staff/support staff/managers). Following the definition of a case study by Robert Yin[35] as ‘an empirical inquiry that investigates a contemporary phenomenon in depth and within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident’ (page 18), our ‘contemporary phenomenon’ for study is the use of digital clinical communication, and the ‘real-life context’ is the NHS, in particular specialist care provision for young people with long term conditions. Our initial propositions to focus our research[35] are: (a) young people between the ages of 16 and 24 years in 2014 are digital natives and use, or would use, digital clinical communication in preference to other means of communication with their clinical team as this fits with their day to day mode of communication; (b) digital clinical communication is used by clinicians to promote the engagement with health care of young people with long term conditions with the aim of improving their health outcome, even if it puts at risk other aspects of clinical service provision (e.g. record keeping). These propositions suggest where to look for evidence to answer our research questions: the young people and the clinical teams. Thus, our unit of analysis will be the young person with a long term condition in communication with their clinical teams (i.e. young person–communication–clinical team). This unit of analysis is embedded in the wider clinic, the technology through which the communication is conveyed, the NHS generally and contemporary society.[35]

Figure 1.

Flow diagram showing links between research activities.

Methods

Ethical approval was obtained from UK NRES Committee West Midlands – The Black Country, 13 March 2014; REC reference 14/WM/0066; IRAS project ID: 147967.

Public and patient involvement

This activity involves young people and aims to capture their ideas, views and concerns on the subject of our research and its conduct. We will recruit young people through schools and other educational institutions to engage in workshops (approximately 10 institutions for each workshop, each sending two young people). The young people gain an extra curricula experience that will broaden their knowledge of health and research and will contribute significantly to their personal statement or CV. Their educational institution gains an enriched curriculum for their students and the research team gains input from the young people. Not all the young people involved will have a long term condition themselves but many will be close to someone who does. By drawing on a general population of young people we avoid focusing on specific conditions. The young people will be trained during a one day workshop in research methods. They develop their own questions from our suggested topics and write a protocol for collecting data from peers (survey or interview usually using digital media). Within their educational institution they execute their mini-research project over a few weeks. At a second one day workshop the young people are trained in analysis, analyse their own data and write a report. Each set of workshops focuses on one of the following:

Development of propositions to inform case study design: Why do young people with long term conditions want to contact their clinical team digitally? Why do they choose to use a particular digital medium?

Recruitment and research design: Are we asking young people the right questions in the right way? Which young people would we miss out? What would young people like to ask health professionals about digital clinical communication? Which patient reported outcome measure is appropriate for use across a wide range of conditions (informed by literature review)?

Analysis and dissemination: Is our analysis capturing the messages and themes communicated by young people? Are young people saying what we think they are saying? What are the important messages from our research for clinicians, commissioners and policy makers?

Evidence from all the projects will be synthesised to inform study design.

Literature reviews

Using published peer reviewed research literature and grey literature we will seek to answer the following research questions concerning the UK NHS provision of digital clinical communication for young people with long term conditions:

What generic outcome measures are available to assess the impact of digital clinical communication?

How and for what purpose is this form of communication taking place (or not) in the UK?

What is the ethical, legal, policy and governance framework for digital clinical communication?

What is the evidence in the literature to support, challenge or add value to the case study findings?

Review 1

An initial search strategy will be developed for MEDLINE and adapted and refined for other databases. Keyword combinations and specific search terms will be used, focusing on the concepts of digital communication, ongoing patient/clinician interaction, specific technologies (e.g. text messaging) and systematic reviews.

The following electronic bibliographic databases will be searched: Cochrane Library (including Cochrane Systematic Reviews, DARE, CENTRAL, NHS EED, and HTA databases), MEDLINE, MEDLINE In-progress & Other Non-Indexed Citations, EMBASE, PsycINFO and Web of Science (including Science Citation Index and Conference Proceedings). Citations in eligible papers and previous reviews in the subject areas will be examined for additional papers that meet the inclusion criteria. Supplementary searches will be undertaken to find additional studies published since the systematic reviews, including scrutiny of references of included studies, citation searching, and searching relevant websites.

Papers will be selected for inclusion if they meet the following criteria:

All papers are systematic reviews and include at least one randomised controlled trial;

The paper must focus on any chronic health conditions (physical or mental health);

All papers should include at least one outcome measure that assesses the impact of digital clinical communication;

Outcomes could be assessed using either validated or non-validated scales;

The paper is published in English only;

The reviews include studies with primary data;

The research for the paper must include studying communication in both directions – patient to clinician and clinician to patient;

The types of technologies we will include are as follows: email, text messaging, social media and web based patient portals, and voice over internet protocol (VoIP) (e.g. Skype and Google Talk) which can simultaneously transmit voice and other media such as text and images;

Where the use of a digitally delivered intervention or the delivery of disease prevention and health promotion information forms part of ongoing patient–clinical team communication, they will be included.

Papers will be excluded if they:

Involve measures aimed at non-English speaking populations;

Are independent studies, case reports, case series, retrospective observational studies, editorials or comments;

Deliver therapeutic interventions via digital communication media such as cognitive behavioural therapy, psychotherapy or psychiatry;

Involve digital communication that solely involves the delivery of information on disease prevention and health promotion;

Involve technologies that solely provide a service that is the same as a telephone consultation

Involve solely one way communication;

Involve solely automated communication (e.g. automatic text messages as reminders);

Focus on health behaviours (e.g. smoking cessation, weight management);

Involve solely communication patient–patient (e.g. online support groups) or clinician–clinician.

The search will not be limited by date, populations, or health conditions. One reviewer will screen all identify bibliographic records for titles/abstracts and identified potential papers meeting the inclusion criteria. Two reviewers will independently screen a portion of the records and discuss any disagreements. One reviewer will extract relevant data on age groups, long term conditions, digital interventions, generic and condition-specific outcome measures, and validated generic outcome measures (scale names, descriptions and full references). Data extraction will be checked by a second researcher and disagreements resolved through discussion.

We will review the outcome measures identified by seeking evidence of the development or evaluation of the measures. We will exclude measures where we are unable to identify any evidence of reliability or validity. Using the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) checklist[36] we will assess each measure for the methodological quality of development studies, measurement properties, and interpretability and generalisability of the measurement results. We will summarise the findings in a form accessible to young people engaging in patient and public involvement activities.

Review 2

This exploratory literature review will identify reports of the use of digital clinical communication by specialist NHS providers in the UK. We will search both peer reviews and grey literature. The review will alert us to potential case study sites.

Review 3

We will collate relevant ethical, legal, policy and governance documents and summarise them to inform case study data collection.

Review 4

During analysis of case study data we will identify topics for up to six rapid scoping reviews. The reviews will aim to find evidence that supports or challenges or in some other way adds value or a wider dimension to the case study findings and places the case study findings in a wider research context. The exact inclusion and exclusion criteria for each of these reviews will be determined by the results of the case studies. For example, we may seek to evaluate clinical trial evidence for a specific type of participant (e.g. diabetes), intervention (e.g. mobile phones), outcomes (e.g. reduction in HbA1c). In this fast changing field it may be necessary to extend the existing review of factors that promote or inhibit the implementation of e-health systems.[37] If social media are found to be important we may extend our own theory based review of their use[38] and the recent systematic review.[39] Another possibility is a review of qualitative evidence. Each review will summarise the available literature from the previous five-year period in tables with a narrative synthesis and discussion of findings.

Case studies

Recruitment and sampling

Inclusion and exclusion criteria

We will study specialist clinics or clinical teams who provide mostly outreach services and that provide NHS health care to young people (aged 16–24 years) with long term physical/mental health conditions which currently or potentially have serious health implications for the young people and are expensive to treat now or in the future.

We will include asynchronous communication technologies such as email, text messaging, social media and web based patient portals. We will also include synchronous technologies such as VoIP (e.g. Skype and Google Talk) which can simultaneously transmit voice and other media such as text and images. Currently, these systems usually use the internet or mobile phone infrastructures with crossover between these infrastructures. If other digital communication technologies come into common use during the project we will include them if we identify clinics where they are used for clinical communication. We are not intending to include technologies that provide a service that is all but the same as a telephone consultation as there is a large body of evidence on clinical telephone consultations.

Our research is concerned with communication between patients and specialist (secondary or tertiary care) clinicians/clinic teams who have already been in contact with each other in the clinical setting. Our focus is on systems of communication where there is, or is potentially, communication in both directions – patient to clinician and clinician to patient. We are not including specifically the delivery of therapeutic interventions via digital communication media[40] such as cognitive behavioural therapy,[41] nor are we including digital communication that solely involves the delivery of information on disease prevention and health promotion.[42] Where the use of a digitally delivered intervention or the delivery of disease prevention and health promotion information forms part of ongoing patient–clinical team communication, they will be included.

Identification and sampling of study sites

Up to 60 sites will be identified from publically available literature, through posing questions on the internet aimed at young people with chronic conditions and through snowball sampling through relevant clinical networks. From these sites, up to 20 will be recruited. The lead clinician/manager of each potential clinical site will be contacted by letter and phone to explain the study and seek their agreement to an initial telephone interview. During the initial telephone interview we will ask about the nature of the clinical team and the use of digital clinical communication to inform our sampling. We will also seek agreement to participation in the study if their clinical team was to be sampled, and seek information on barriers to participation (e.g. upcoming move of clinic location). We will then sample from those that have agreed to participate. Using this approach we should achieve diversity of health condition, type of technology and degree of integration of the digital clinical communication within the routine work of the clinic, along with diversity of geographical location and regional or district specialist clinical teams. We will sample clinical sites so that each is different from the last on one or more of these criteria. Recruitment will stop when we have included a diversity of clinical teams and we reach data saturation. We will also consider the need to collect data to provide contrasts between clinical teams for analysis. For example, if a clinic is actively using digital clinical communication for advising patients on changing medication regimes, we would aim to recruit a clinical team that undertakes similar clinical activity but not using digital clinical communication.

Ethical issues

We will be asking health care staff to talk about practices that may be contravening NHS current information governance guidance. Health professionals are responding to the needs and demands of their young patients by using digital communication that does not necessarily meet the NHS information governance criteria. To collect rich data we will emphasise the confidentiality of the research data and that we are collecting data from many clinics so it will not be possible to identify specific clinics/staff from our research report. We will have an ethical protocol in place for considering breaches of information governance policy and professional standards. We do not expect to take action for activity that we find is common practice but will be alert to serious breaches of policy and professional standards. We will steer a careful path here as clinicians using digital clinical communication will be rich sources of data for the project. We will ensure transparency of ethical process.

Study site recruitment

A briefing meeting will be held with staff members, during which a timetable of data collection will be drawn up to maximise opportunity for data collection and minimise disruption to the clinic. Information about the study will be distributed to all clinic staff members and posters about the study will be displayed in the clinic for both staff and patients to see for the duration of the fieldwork.

Recruitment of clinic staff for interview and informed consent

During the clinic field work we will sample health professionals working in the clinical team who use or would potentially use digital clinical communication (nurses, doctors, professions allied to medicine) and administrative, managerial and technical staff who provide support, including IT managers and information governance specialists. Sampling will be purposive for diversity of experience and opinion about digital clinical communication within each case study. We will aim for data saturation for each clinic and expect to undertake individual interview with up to 15 clinic staff. In some clinics the number needed to reach saturation may be as low as four or five. Clinicians may decline to be interviewed despite the clinic being a case study site. For the collection of data about the processes of the clinic (e.g. patient flows, staff tasks), where convenient for the staff, we may interview several members of staff together. Written consent will be obtained at the beginning and confirmed at the end of each interview with clinicians.

Recruitment of patients for interview and informed consent

Patients with appointments to consult with a clinical team member will be identified and sent a study information sheet two weeks before their appointment. Those who agree to interview will be given either an appointment time for interview that fits with their clinic attendance or arrangements will be made for interview via telephone/Skype/email/Facebook etc., whichever is preferred by the patient. Consent will be taken before the interview commences. Where an interview is held over the telephone, consent will be taken verbally. Patients will be offered a thank you token of a £20 High Street voucher. During interview the patient will be asked about the study team interviewing their parent/carer/household member (e.g. girl/boyfriend or wife/husband). The patient will make the decision as to whether they are willing for the parent/carer/household member to be interviewed. If they are willing, they will be asked to pass a participant information sheet to the parent/carer/household member and to provide their contact details. If consent is given these interviews will usually be by telephone or email. Consent will be collected as for the patient. We are including patient/carer/household member interviews as these people may be involved in digital clinical communication, particularly for the lower ages in our study and where patients potentially become seriously ill.

Although phone and email interviews might not give as rich data as face to face, we will offer a choice to encourage participation. Interview length is likely to range from a 45 minute face to face interview after clinic through to one email exchange. Guided by clinic staff, we will purposively sample for current users, past users and non-users of digital clinical communication, patients from localities with low socioeconomic indicators and patients from ethnic minorities. As well as digital communication users, we will aim to recruit for interview patients who are or might be excluded from the use of digital communication media due to lack of resource, due to disabilities or because they do not want to use them. Where necessary, we will employ an interpreter to assist with communication at recruitment and for undertaking interviews with people unable to communicate in English. We will aim for a diversity of patients and data saturation within each clinic and expect to interview up to 15 patients or patient/parent/carer/household member dyads. Where patient and parent/carer/household member are both interviewed, we will interview them separately if they agree.

Data collection

Through our observation and exploring the perceptions of patients and clinical team members, we seek data on what happens in the clinic, why it happens and its impact. Within this realist approach,[43] data collection will be guided by existing theory concerning the implementation of innovation, in particular the comprehensive framework for implementation research (CFIR)[44] and normalisation process theory (NPT).[45,46] CFIR is a framework developed from extensive literature review and synthesis of theories that identifies five major domains for exploration when evaluating the implementation of change in health care organisations: the intervention, inner and outer setting, the individuals involved, and the process by which implementation is accomplished. NPT was developed from empirical studies and considers the actions of people when implementing a change or new technology into their working practices. It considers four constructs of actions: coherence, cognitive participation, collective action and reflexive monitoring. We will use these theories to sensitise us to the areas to explore in data collection. For example, in the CFIR the domain of intervention characteristics includes stakeholders’ perceptions of the advantage of implementation of the intervention, its adaptability, the potential for testing the intervention on a small scale, and its complexity. Similarly, NPT suggests questions such as whether the intervention fits within the overall goals of the organisation. However, we will not be constrained by these theories and will actively seek other relevant data.

Documentary analysis

With the assistance of each clinic’s lead clinician/manager, we will collate and summarise current policies and procedures. We will ask the clinical lead/manager to tell us about any reported incidents or adverse events related to digital clinical communication that have occurred in the previous three years.

Non-participant observation

The researcher will observe how the clinic functions and ask clarifying questions guided by an observation proforma. Clinic staff will be shadowed for up to two hours at any one time and up to four times during data collection to observe different types of clinic activity. Observation data will include: who uses digital clinical communication, where, when, why and for what purpose; frequency of digital clinical communications and the length of time spent dealing with these communications. Field notes will be taken. These notes will be reviewed and adjustments made to observation plans to ensure all aspects of relevant clinical team activity is observed.

Collection of impact data

The researcher will establish during observation how any use of digital clinical communication is being or could be evaluated for its intended objectives. If a clinic has evaluated their use of digital clinical communication, we will seek access to this evaluation. If not, with the clinic team, we will plan a retrospective evaluation using available data. If the purpose of the digital clinical communication was to improve concordance with treatment or monitoring regimens then we will seek data that reflects this (for example, a routinely used clinical indicator). If the purpose was to improve access to advice and support at the time of need, we will assess whether this is taking place. The evaluation plan will include: time frame (before and after initiation of use); data relevant to objectives (e.g. blood test results); time points (e.g. annual data); clinic denominator (young people recurrently in contact with clinic team). The following data will be collected where available for all clinical teams included in the case study for before and after initiation of the use of digital clinical communication (or for non-user clinics, over a similar period of time as for user clinics): DNA (did not attend) rates (excluding first appointments); emergency hospital admissions and accident and emergency department attendance rates. Where a clinic caters for adults we will limit the data to patients aged 16–24 years. Data will be extracted from existing clinical and administrative data.

Collection of economic data about the digital communication system

We aim to establish the direct cost involved with the development, implementation and day to day running of the technology used in the case study sites for digital clinical communication. We will determine, at each site, the extent to which the development, implementation and maintenance of technology has been managed internally, or commissioned from external specialists. For internally managed activity, we will identify staffing and equipment costs associated with these activities, and determine whether there were specific challenges or design features that were particularly costly to accommodate. We will also investigate costs associated with externally commissioned activities.

Semi-structured interview content (staff, patients and parents/carers)

Interviews will usually be brief (up to 45 minutes), audio recorded, and focused on the experience of using digital clinical communication. In advance, interviewees will be asked to bring to the interview examples of recent digital clinical communications (anonymised) and critical incidents as examples to inform interview discussion. These examples will not be given to the research team by health professionals. Patients and their parents/carers/household members may choose to give the researcher these examples.

In clinics where digital clinical communication is in use, interviews will cover the following:

Intended objectives of using digital clinical communication and whether or not they have been achieved;

Digital clinical communication actually used, why it was used and in what context;

Understanding of the nature of privacy and confidentiality in the context of digital clinical communication;

Understanding of the clinician’s duty of care and the patient–clinician relationship, including responsibility for care/self-care in this context;

Features of the digital clinical communication system, the content of the communication and any contextual factors that contribute to its successful/unsuccessful use;

Perceived risks (patient safety, ethics, data storage);

Costs and benefits (patient experience, staff work experience, unintended consequences, impact on other services, financial costs and savings, evaluated health outcomes);

Future implications of greater use of digital communications;

Need for or experience of training for using digital communication with patients/clinicians.

Within each interview we will also use a variation of the critical incident technique for both when digital clinical communication did and did not work well: tell me about a situation where the digital clinical communication did/did not work well for you; what happened (unfolding); what was the result (consequence); how did you cope (mitigation); what could have happened (worst credible effect).

To investigate the impact of digital clinical communication on staff workload, in interviews with staff we will attempt to capture ways in which digital communication has increased their workload, or allowed them to work more efficiently. We will ask participants to quantify this impact as far as possible.

The interview topics will be adapted for use in clinics where there is partial use or past use of digital clinical communication. Where it has not been used, interviews will seek to explore currently used processes of communication between clinicians and patients, attitudes to digital clinical communication and reasons for not using it and implications of greater use of digital communications including training needs. We expect to reach data saturation rapidly in these clinics.

The interviews will be an opportunity to expose to clinicians and young patients any generic measures identified in literature review, to ascertain to what extent the success or not of their digital service is captured by these outcome measures. If no generic measure has been found through literature review or if cumulative case study analysis identifies that existing measures are not viewed as adequate, then the interview and observational data will be used to develop items for a new generic scale to capture the impact of digital clinical communications.

We will investigate the value patients place on digital clinical communication using a willingness-to-accept approach.[47] Within the semi-structured interviews, patients will be asked to consider the hypothetical opportunity to receive payment (cash or vouchers) as an alternative to using the digital communication system, and indicate how much they would need to be offered to forego use of the system.

Data management and analysis

All qualitative data will be given an identifier, typed up/transcribed and during this process anonymised. NVivo software will be used to manage this data and for coding. Retrospective quantitative clinic data will be obtained in aggregate form. Throughout analysis we will use standard techniques for quality checking, including qualitative coding by independent researchers and investigation of outliers and non-standard responses. Qualitative analysis will be concurrent with data collection to ensure data collection ceases when data saturation is reached. We have developed analysis questions to ensure we meet the relevant study objectives. We describe below the analysis for each question. However, each approach to analysis will inform others as they all form part of the same case study.

Analysis questions: What works for whom, where, when and why?

Given the research gaps identified related to the need for generalisable evidence across disease areas, we will focus analysis on the commonalities across the health conditions such as communication about medication, communicating results of investigations, symptom reporting or health service navigation; what one review[20] called ‘the function of the communication’. We will follow Yin’s case study approach[35] combined with realist evaluation approach.[43] In the data, we will identify descriptions of actual events where digital clinical communication has been used. We will categorise these into configurations of context, mechanism and outcome, where context is proximal influences, mechanism is the digital communication and the interaction of the patient and clinician with that communication, and the outcome is proximal to the mechanism. (Illustrative example: context = young person experiencing psychosis + mechanism = SMS communication with mental health team aiming to maintain medication concordance > outcome: young person takes medication regularly.) From patient and public involvement activity, from published studies and theory and from early data collection, we will develop propositions that are possible explanations of a more distal outcome. To test each proposition, we will interrogate the categorised configurations of context–mechanism–outcome along with data about the wider patient and clinic context, quantitative measures of outcome such as health status or service use and qualitative data that can explain the outcome. We will seek to confirm or refute the propositions and seek rival explanations. This will continue until we are finding no new data/patterns of data (data saturation) and so no evidence for revising the explanations further. We will then develop logic models which bring together explanations as a chain of events. These identify a context, with a mechanism that through intermediate steps produces a final outcome. The aim is not to produce one logic model but a number of alternative models. (Illustrative example: context = mental health team working with young people in deprived inner city locality with high rates of admission for psychosis + mechanism = SMS messages to remind young people about medication and young people can report side effects > proximal outcome: young people are concordant with medication > subsequent outcome: fewer acute psychotic episodes > distal outcome: reduced rates of admission.) The construction of these logic models will draw on earlier analysis and on wider relevant research literature.

Analysis question: Using existing clinical data, what is the impact of digital clinical communications on the health status and behaviours of patients?

Data on how the impact on patients’ health status is currently evaluated in relation to the use of digital clinical communication will be extracted from the case study material and summarised for each clinic. The data extracted from routine clinic records for the planned clinic evaluations will be analysed using descriptive statistics, cross-tabulations, and statistical tests of association. To analyse the difference in DNA rates before and after implementation of the use of digital clinical communication we will report the proportion of DNA patients and calculate the difference in these proportions with the appropriate 95% confidence interval based on a delta-method standard error for the difference in proportions. We will also report the P-value from the test of the difference in binomial proportions.[48] We will also report similar analyses for rates of emergency hospitalisation and rates of accident and emergency department attendance.

We will compare rates between clinics to understand overall trends and any exceptions. For example, use of digital clinical communication may be associated with increased accident and emergency department attendances for one long term condition and not another. We will also compare study clinics with published data for the same condition and its management in the UK. Statistical modelling of each outcome (DNA, emergency hospitalisation and accident and emergency attendance) will be performed using multilevel mixed effects logistic regression using clinic level random effects and treating the before and after DNA data (and data at any additional time points) as repeated measures.[49] This will allow estimation of an intra-class correlation coefficient for the proportion of variance explained by the within clinic variance. The use of digital clinical communication will be included in the model as a binary covariate allowing estimation of an odds ratio for the specific outcome for the effect of digital communication.

Analysis questions: What value do patients place on digital clinical communication? What are the direct resource use implications for the NHS of implementing it? How does the direct resource use vary when used with different patient groups? What are the resource implications for scaling up in the NHS?

To understand the value patients place on digital clinical communication we use the willingness-to-accept approach and link the answers to data from patients (or patient/parent carers) on their experience with such communication, to better understand what aspects are highly valued. Data collected from the case study sites about cost, design features, staff work load and time will be combined with evidence from the literature to build up a picture of the costs associated with digital clinical communication, and the immediate benefits to patients and health care professionals. We will explore how these costs and benefits vary according to the design of the system and the disease area where it is used. We will also explore the extent to which costs are fixed or vary with size, in order to explore the impact of scaling up particular interventions to be available across the NHS.

Analysis questions: What concerns do patients and clinicians have about confidentiality in relation to digital clinical communication? How does it affect the patient/clinician relationship and the clinician’s duty of care? What regulatory framework is needed to reassure patients and clinicians regarding its use?

Given the research gaps identified in relation to privacy and data protection and the effect of digital clinical communication on the patient–clinician relationship, we will include an empirical ethical analysis of interview and observational data focusing on patients’ and clinicians’ views on the nature of confidentiality and privacy, clinical duty of care, and trust between patient and health care professional in the context of their experience of digital clinical communication. We will follow the method described by Ives and Draper for ‘normative policy oriented empirical ethics’.[50] This approach recognises the need for ethical policy (in this case policy on the use of digital clinical communication) to be informed by both a theoretical analysis of the ethical concerns and the moral intuitions of the relevant stakeholders. Analysis involves an iterative process of reflective equilibrium between the empirical data (intuitions of patients and clinicians on confidentiality and trust in the context of digital clinical communication) and theoretical analysis (ethical and legal discourse on confidentiality and duty of care).

Analysis question: What are the significant risks to patient safety associated with the use of digital clinical communication in the context of supporting young people with chronic disease?

The introduction of technology may change the way in which a service is delivered and used. It is important to identify proactively and to assess any potential threats to patient safety that may arise as a result of this. Such a risk assessment needs to consider both intended use scenarios as well as scenarios where the technology may be used in ways that may not have been intended (reasonably foreseeable misuse). In addition, credible failure scenarios (i.e. situations where use of the technology fails) need to be identified and their impact on patient safety assessed. The risk analysis will be informed by: (a) consideration of actual events through the study of incident reports from the participating organisations as far as these are available to the research team and (b) perceptions of staff and patients elicited through the semi-structured interviews using a variation of the critical incident technique (described above).

Analysis question: In future, how can its effectiveness be measured across health conditions?

We will undertake thematic analysis of qualitative data from the case studies specifically for aspects of impact. We will continue analysis of data until no new themes are being found. Themes will be summarised and compared with those covered by existing generic measures identified from the literature review or will be developed into a draft outcome measure for future development and testing beyond the scope of this study.

Consensus meeting

Research questions: What are the risks of the use of digital clinical communication to patients and to NHS specialist care providers? What policy and procedural changes are needed for gaining benefit and limiting harm? In which clinical areas is benefit most likely, and how is benefit most likely to be achieved? What future evaluation is needed and how should it be undertaken?

The results of each of our analyses will be interrogated to answer each of the above questions. These answers will then be presented at a consensus meeting along with a series of scenarios as illustration based on the study results. There are various designs for running consensus meetings.[51] We will ensure attendees have experience or insight relevant to the topic, and bring a range of views.[52] We will invite national stakeholders (approximately eight) representing various aspects of health care provision for young people, recruit young people (approximately eight) and representatives of specialist service providers (approximately eight). We will advertise for young people to attend among all those who engaged with patient and public involvement activities, as these young people will already have developed some insights into the issues. We will advertise for representatives of specialist service providers by contacting all providers who expressed an interest in the research (even if they did not become a study site). We will identify relevant national stakeholders including representatives of policy making bodies, professional associations and educational organisations. The format will be a modified form of the NIH Consensus Development Conference.[52] The form of consensus may be agreement about the multiple options available and the caveats that apply to different contexts. The meeting will run as follows: a series of short presentations on the different aspects of the study; discussion of the scenarios in small groups followed by plenary feedback and discussion; consideration of each of the research questions for this work package in different small groups; feedback to the whole group; continued discussion until consensus is reached. The meeting will be chaired by a patient/public representative from the project management group.

Discussion

This multi-method evaluation aims to understand the benefits, costs and consequences of the use of digital clinical communication in the UK NHS for people requiring specialist services for long term conditions. The study focuses on young people as, for this cohort, ability to use digital communication will not be a limiting factor. The evaluation will indicate where, when, how and with whom digital clinical communication is currently used successfully, risks and procedures for minimising the risks of innovation, the risks of not innovating and actual cost of provision including start up, continuation and upscaling. Health professionals work within their professional guidelines and health care provider information governance policy. This study will provide evidence from which these can be developed and refined to enable appropriate use of digital clinical communication. The study will also provide evidence for the successful implementation of digital clinical communication and suggest areas where further research on its use is needed.