Delivering outpatient virtual clinics during the COVID-19 pandemic: early evaluation of clinicians' experiences.

BACKGROUND: The COVID-19 pandemic has put health systems across the world under significant pressure. In March 2020, a national directive was issued by the National Health Service (NHS) England instructing trusts to scale back face-to-face outpatient appointments, and rapidly implement virtual clinics.
METHODS: A multidisciplinary team of change managers, analysts and clinicians were assembled to evaluate initial implementation of virtual clinics at Guy's and St Thomas' NHS Foundation Trust. In-depth interviews were conducted with clinicians who have delivered virtual clinics during the pandemic. An inductive thematic approach was used to analyse clinicians' early experiences and identify enablers for longer term sustainability.
RESULTS: Ninety-five clinicians from specialist services across the trust were interviewed between April and May 2020 to reflect on their experiences of delivering virtual clinics during Wave I COVID-19. Key reflections include the perceived benefits of virtual consultations to patients and clinicians; the limitations of virtual consultations compared with face-to-face consultations; and the key enablers that would optimise and sustain the delivery of virtual pathways longer term.
CONCLUSIONS: In response to the pandemic, outpatient services across the trust were rapidly redesigned and virtual clinics implemented. As a result, services have been able to sustain some level of service delivery. However, clinicians have identified challenges in delivering this model of care and highlighted enablers needed to sustaining the delivery of virtual clinics longer term, such as patient access to diagnostic tests and investigations closer to home. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

The coronavirus (SARS-CoV-2) pandemic was first identified in Wuhan, China, in December 2019. In the UK, the rate of infection and mortality of the virus has put unprecedented pressure on the National Health Service (NHS) and other health systems across the world.1 Consequently, face-to-face (F2F) outpatient appointments were scaled back and elective medical care deprioritised, as per directive from NHS England.2 The NHS Long Term Plan published in 2019 specified a need for service providers to reduce F2F appointments by one-third over the next 5 years.3 However, the onset of the pandemic necessitated the acceleration of this vision. In mid-March 2020, NHS trusts were required to instate models of telehealth over a matter of days as the default mechanism for delivering outpatient appointments. Such action was adopted nationally in an attempt to minimise transmission of the virus while maintaining care standards for patients with chronic health conditions and ensuring ongoing access to specialist care.4

Telehealth is defined by the WHO as:

the delivery of health care services, where distance is a critical factor, by all health professionals using information and communication technologies, for the exchange of valid information for diagnosis, treatment, and prevention of disease and injuries, research and evaluation, and for the continuing education of health care providers, in all the interests of advancing the health of individuals.5

Virtual or remote clinics are a form of telehealth and can be delivered synchronously, such as by telephone and video, or asynchronously, such as by email or patient portal messages.6 The terminology used to describe telehealth is continually evolving and is yet to be standardised, but typically implies the virtual presence of clinicians as service providers to patients, or clinician-to-clinician communications.7

Over the last decade, virtual clinics have been pitched as a convenient and cost-effective way to deliver care. Some studies have explored feasibility of virtual clinics in primary care,8 cardiac surgery9 and glaucoma services.10 However, a lack of high-quality evidence demonstrating the efficacy of virtual clinics, particularly in secondary care, has meant that the integration of virtual clinics in health systems has been slow.11 Implementation of virtual clinics that mediate desirable clinical outcomes, patient satisfaction rates and cost-effectiveness typically evolves through a structured approach of service transformation comprising stakeholder engagement, service planning and evaluation/audit. Like many other industries, COVID-19 has challenged healthcare systems to digitally transform over a short space of time. At the onset of the pandemic, healthcare providers were contending with short-term challenges, such as how to maintain outpatient care to complex patients with chronic health conditions. As healthcare organisations adjust to the new environment, clinicians are grappling with the longer term question: What should outpatient service provision look like over the course of the pandemic and beyond?

In March 2020, outpatient services across Guy’s and St Thomas’ NHS Foundation Trust began to reduce outpatient activity and reconfigure F2F appointments into virtual consultations (VC), as a direct response to COVID-19. Given the rapid pace of this transformation there was an acute need to evaluate the newly implemented virtual clinics from the perspectives of clinicians delivering them. An internal evaluation team was commissioned to deliver the evaluation. The aim of the evaluation was to review the feasibility and effectiveness of VCs compared with F2F consultations from the perspectives of clinicians.

Methods

Design

This was a qualitative analysis of in-depth interviews conducted with clinicians. The interviews were conducted as part of a trust-based evaluation of virtual clinics implemented as a strategic response to COVID-19. The evaluation was commissioned internally and approved on 20 April 2020.

Participants

Convenience sampling is the deliberate selection of a participant due to the qualities the participant possesses.12 A non-random sampling technique, it was used to capture perspectives from clinicians from a broad range of specialties across the trust. Clinicians were identified through the trust’s internal network and those who were willing to provide information about the implementation of virtual clinics in their specialty. Written consent to take part in the interviews was obtained from participants via email prior to the interview.

Data collection

Interviews lasted 30–40 min and were conducted over Skype or Microsoft Teams. Interviewers conducted the interviews using a purpose-designed interview schedule. A copy of the interview schedule can be found in online supplemental file 1. Interviews were conducted by coauthors SN, DC and BM. The interview schedule featured questions based on the following areas: (1) the enablers and limitations to delivering VCs, (2) optimisation and sustainability of virtual clinic pathways. Questions were open ended and collectively reviewed by the evaluation team. No repeat interviews were carried out. Emergent themes were fed back to the outpatient strategic team on a weekly basis to inform potential work streams arising from the evaluation to drive forward initiatives.

Patient and public involvement

The aim of this evaluation was to explore and capture the first-hand experiences of clinicians delivering virtual clinics. The interview schedule was reviewed by the evaluation team, but given our work focused on professionals’ experiences it was not relevant to involve patients in the study design.

Analysis

The data analysis was based on an inductive thematic approach as described by Javadi and Zarea.13 The aim of the analysis was to identify and group together similar experiences and perspectives relating to the delivery of VCs. This analysis comprised four steps: (1) familiarisation with the data; (2) generating initial codes; (3) searching for themes; (4) reviewing themes and defining and naming themes. The analysis aimed to identify commonality across transcripts allowing for themes to emerge from the qualitative data. To increase rigour, 20% of the transcripts were double coded. No changes were made as a result of double coding.

Results

A total of 110 clinicians were interviewed. Ninety-five interviews were successfully recorded and transcribed verbatim. The remaining 15 interviews did not record successfully, and therefore could not be transcribed. Transcripts were assigned identifiers for anonymisation. Clinicians interviewed included 55 consultants, 15 physiotherapists, 14 nurses and 11 ‘other’ healthcare professionals. Interviews were carried out from April to May 2020.

Results reported here correspond to the 95 interview transcripts. A summary of the themes discussed can be found in table 1. The majority of clinicians had indicated that virtual clinics set up in their respective services were a direct response to the COVID-19 pandemic (n=68) and only offered F2F consultations prior. A few services did have some existing virtual clinics in their service (n=13) or had explored the possibility of delivering virtual clinics prior to the pandemic (n=14).

Table 1

Summary of themes

Theme	Subtheme
Reflections of achievements	Enablers to redesign Benefits to clinicians Perceived patient benefits
Limitations of virtual consultations	Diagnostic tests Telephone and video consultations Surgical preassessment Challenges for clinicians Efficiency Technological Accessibility Environmental
Sustainable delivery of virtual consultations	Triage Diagnostic pathways Patient and clinician resources e-PROMs

e-PROM, electronic patient-reported outcome measure.

What are clinicians’ reflections on what has been achieved so far?

Enablers to redesign

All clinicians who were interviewed acknowledged the enormity of what has been achieved thus far in transforming outpatient services under unprecedented circumstances. Clinicians reported that the trust directive to reduce and reconfigure outpatient services was a significant enabler to this implementation. This facilitated the removal of barriers that would otherwise have impeded redesign. Patient compliance to VCs was attributed to the government directive to stay at home, as well as the pre-existing rapport clinicians have built with patients prior to the pandemic: ‘I see follow up patients virtually; by that I mean telephone and they are quite comfortable because they know me, that’s what they say, as opposed to new patients’ (consultant Ear, Nose and Throat surgeon).

Benefit to clinicians

Clinicians reported feeling empowered to redesign their respective services, and a sense of achievement: ‘I guess that was about the satisfaction of actually having managed to get it right for the patients’ (consultant haematologist). Many clinicians acknowledged the reduced exposure to COVID-19 for themselves as well as for patients when delivering virtual clinics, in addition to the possibility of flexible working: ‘I was able to maintain a pretty much full clinic throughout so I guess that’s demonstrated that you can work in an offsite way’ (consultant dermatologist). Clinicians observed that patients were generally understanding of the current limitations in care and this was observed through a more focused discussion in consultations.

Perceived patient benefits

Clinicians reported that the potential benefits of VCs for patients include the spared financial expense of travelling to the hospital, particularly for tertiary referrals who may live outside the SE London network. As well as the primary benefit of avoiding exposure to COVID-19, clinicians observed that virtual clinics are potentially less disruptive to school or work and patients do not have to wait in crowded waiting areas. An additional benefit noted was that virtual clinics are more accessible to patients who do not feel well enough to travel to hospital.

What are the limitations of VCs?

Diagnostic tests

One of the most frequently reported limitations of virtual clinics is the inability to conduct diagnostic tests. Most clinicians stated that they had postponed tests such as blood tests and X-rays. For patients where an updated blood test was essential, many clinicians have been arranging blood tests with General Practitioners (GPs) on a case-by-case basis. This, and the chasing of results, has been described as time consuming to set up. While clinicians felt that, overall, GPshave been accommodating to conducting these tests, some had refused citing a lack of commissioning agreements. Some services also require specialist tests that are only offered on-site requiring these patients to come in.

Telephone and video consultations

A majority of clinicians reported delivering appointments by telephone only (n=72). While clinicians acknowledged that a lot of the required information is transmissible over the phone, limitations of telephone consultations were also highlighted. These included the inability to read non-verbal cues, to detect safeguarding concerns and the inability to conduct a hands-on clinical examination: ‘A lot of what we do is looking and feeling joints to check if they are inflamed or not. I can’t do that over the phone’ (consultant rheumatologist) and ‘Later in pregnancy, it’s more appropriate that we see them because actually we need to examine a lot of them to pick up things’ (consultant gynaecologist). In F2F clinics, clinicians mentioned that they are typically aware of patients who have arrived and waiting. Twenty-one clinicians reported delivering virtual clinics by telephone as well as video, and two clinicians were delivering video consultations only. Reported benefits of video consultations include the increased ability to read non-verbal cues in addition to maintaining some level of patient contact that clinicians are used to. In some cases, clinicians found that offering patients video consultations potentially added an extra incentive for them, in that video consultations maintain more of the contact patients have with clinicians. Clinical examination conducted over video was reported to be suboptimal particularly in highly visual specialties such as dermatology, in addition to which some clinicians had medicolegal concerns about clinical examination over video.

Surgical preassessment

All 30 of the clinicians interviewed from a surgical specialty stated that a VC cannot substitute a traditional preassessment consultation for patients requiring surgery. The primary reason cited included the inability to conduct a physical examination as above: ‘You want to examine the heart, the lungs—you cannot do that based on what patients are telling me on the telephone, I need to see the patient physically, to be able to make a judgement’ (preassessment nurse, surgery). Additionally, clinicians stated that the prospect of taking electronic consent from patients poses an ethical dilemma from both parties and may have medicolegal implications. Importantly, surgeons stressed the importance of building trust with their patients. This was considered to be central to establishing an effective therapeutic relationship which cannot be achieved remotely and referenced in the quote: ‘I think you need to have that at least once initial hands on touchy examination to make that connection between the patient and the surgeon’ (consultant orthopaedic surgeon). Clinicians felt that establishing this relationship with the patient can be fundamental in optimising patient satisfaction, willingness to disclose sensitive information, compliance with follow-up attendance and treatment.

Challenges for clinicians

All clinicians described rapid redesign of their respective services: ‘We had to do it due to a crisis, we hadn’t really planned and prepared to do it properly’ (physiotherapist) and ‘All the clinicians triaged the list and were fairly ruthless to try and reduce the number of patients who have to come to the hospital’ (consultant nephrologist). While this reactive approach was described as a necessity given the time pressures, longer term service planning has been very challenging. Additionally, all clinicians acknowledged that the switch to an almost entirely virtual delivery has been a momentous change from standard practice. Sitting at a desk all day continuously doing telephone consultations was also perceived to be challenging. Many reported missing patient contact and hence some were keen to be redeployed to deliver front-line care.

In a typical outpatient clinic, clinicians would routinely be supported by administrative support and allied health professionals such as specialist nurses. In a virtual setting, this has been absent or more challenging to access particularly for clinicians working remotely: ‘The thing with clinics is regardless of whether virtual or face-to-face they require the back office support in order to make a clinic run smoothly and improve the efficiency’ (consultant urologist). Working in siloed conditions has also resulted in a feeling of isolation among some clinicians: ‘I did like the way things were, I liked face to face, I think the virtual clinics differ a little bit, more lonely’ (consultant gastroenterologist).

Efficiency

A summary of the factors reported to affect the length of a VC can be found in (table 2).

Table 2

Factors influencing the efficiency of a virtual consultation (VC)

	Factors leading to faster VC	Factors slowing down VC
Positive	Clinicians do not have to wait for patient to enter clinic room. Conversation is generally more focused. Patients are generally more well/not picking up other viruses during lockdown.
Negative	No review/discussion of diagnostic test as test postponed. Unable to assess disease activity with physical examination. Unable to take consent for treatment/surgery over the phone. Patient less inclined to share other problems, for example, psychological aspects.	Clinician has to check patient contact details on electronic patient record/incorrect contact details for patients. Clinician has to ring the same patient a few times/leave a message where number is withheld and patient does not answer first time. Unable to show patient a visual to aid explanation of result/condition, thus explanation takes longer. Case-by-case arrangement with GPs for blood tests takes a long time/as well as chasing up results. Poor internet connectivity at home. Time spent looking up patient notes before telephone call rather than while the patient is in front of clinician. Clinician has to check email for photo forwarded by administrator.

Technological

The most frequently reported issues by clinicians in delivering virtual clinics related to limited technological resources. All clinicians agreed that the migration to digital technologies to facilitate this delivery has not occurred in tandem with implementation: ‘I think that’s what our main difficulty is, just having the right equipment set up…’ (consultant orthopaedic surgeon). Given the acceleration towards virtual clinics, many clinicians felt telephone consultations were the only option under the time constraints. While some services are delivering video consultations, not all clinicians are equipped with trust-issued laptops. The majority of clinicians have had to use their own mobile phones for telephone clinics when working remotely which has given rise to a few issues: ‘I have to use my own number and there have been times where patients have called me back…if we did have a work phone then you could separate work and home a little bit better’ (consultant rheumatologist). Clinicians also reported some of the technical challenges working from home:

Doing it at home is extremely difficult on a laptop screen when in one place you need documentation about results, you need to get their complex history from previous notes, letters to GP and so on, and trying to display all of that on a single laptop screen is challenging. (Consultant dermatologist)

Accessibility

Clinicians who have delivered virtual clinics at other trusts mentioned that under non-COVID-19 circumstances the selection and stratification of patients suitable for a VC would be decided prospectively.

An interesting point raised by several clinicians was the observation that most of the patient benefits of virtual clinics are all convenience or efficiency related, and the limitations are mostly clinical:

I don’t think there’s any benefit to the patient of not having a face-to-face apart from perhaps travel time and cost. I know that because we are dealing with under-fives; I think there’s no substitute and it’s not in the parents or the child’s best interest not to be seen face-to-face. (Consultant paediatrician)

Environmental

Clinicians mostly indicated that they were working on-site as well as remotely, and described the space requirements for virtual clinics as a private room equipped with a computer and phone. Many clinicians felt that they have less control over the clinical environment than they would if the consultation was taking place F2F in a clinic room: ‘There’s the issue of patient confidentiality, so unless you’ve got a separate office where you can close the door, there may be other members of the household that will overhear conversations which is a bit of a worry’ (nurse consultant, dermatology). This has also been attributed to not being able to assess whether they have the full attention of the patient during consultation. Some clinicians working from home have found it quite challenging to find a private room to conduct virtual clinics, particularly those living in shared accommodation or who have children or because they are not set up to work remotely: ‘At home the only private space I can have is in my bedroom’ (physiotherapist). Others have felt uncomfortable conducting video consultations where patients can see into their home.

How can VCs be optimised and delivered sustainably?

A list of initiatives identified by clinicians that would promote sustainability of virtual clinics is summarised in table 3.

Table 3

Summary of factors that would sustain longer term delivery of virtual clinics as reported by clinicians

Accessibility	Establishing a model of stratification of patients who will be unable to access virtual consultations. Disease-specific model of stratification of patients who require a face-to-face appointment.
Complex/vulnerable patients	Multidisciplinary one-stop clinic. Jointly delivered video consultations. Flexibility with appointment intervals based on patient need.
Virtual triage	To replace walk-in outpatient services. To signpost patients to appropriate services. To determine the level of clinical input required, for example, advice; video/telephone consultation.
Diagnostic tests	To establish a community phlebotomy pathway. Providing equipment for diagnostic testing at home (eg, for height/weight, blood pressure).
Clinic resources	Trust-issued laptop and mobile phone/headset. Private clinic room/office space.
PROMs	Collection of electronic patient-reported outcome measures.
Resources for patients	Library of information videos/tutorials. Clinical presence in online communities. Self-monitoring tutorials, for example, how to measure blood pressure; height and weight; pulse.
Resources for clinicians	A summary of each available digital platform and full list of functions. Tutorials offering information about communication tactics/effective communication in VCs. Tutorials that provides general guidance on delivering a telephone/video consultation optimally. Guidance on educating junior medical staff virtually.
Trust directive	To remain flexible and not micromanage service redesign.

PROM, patient-reported outcome measure; VC, virtual consultation.

Triage

Clinicians identified the need for services to adopt triage methods for systematic prioritisation of patients according to urgency. This system would allocate an appropriate level of clinical resource and sequence of care, for example, telephone or F2F appointment based on the initial condition of each patient. A telephone-based triage service was preferred by most. Benefits in accessibility of other means, such as email or platforms for patients to self-book into clinics, were mentioned. However, there were concerns that such triage methods may mean patients who require urgent care may not be attended to urgently.

Diagnostic pathways

All clinicians felt that establishing a diagnostic pathway for patients attending virtual clinics will be essential in enabling patients to progress on a virtual pathway. While clinicians mentioned some specialist tests can only be conducted on-site, such as scans or hearing tests, most blood tests could be conducted in the community or primary care. A community phlebotomy pathway did not yet exist but was frequently mentioned as something that could enable patients to progress on their pathway while streamlining processes for clinicians.

Patient and clinician resources

Thirty-eight (40%) clinicians felt that patients would benefit from having access to a ‘patient portal’, a centralised online platform that holds a library of information videos and resources. These resources could provide guidance to patients on VCs, such as how to log into an upcoming video consultation. Similarly, many clinicians suggested that an equivalent platform for clinicians would also be useful. Technological resources to deliver virtual clinics were highly requested, such as Trust laptops, mobile phones, headsets and an additional screen for when working remotely.

Electronic patient-reported outcome measures

Many clinicians discussed the need to incorporate electronic patient-reported outcome measures (PROMs) into virtual pathways. The mode of administration has conventionally been paper format in outpatient clinics. In line with the largely virtual delivery of outpatient services, many clinicians mentioned that PROMs collected remotely are a necessary component of specialist care. Clinicians stressed the benefits of electronic PROMs to collect data on the patient’s symptom burden, quality of life, and physical and social functions as integral to augmenting the clinician’s understanding of the patient’s health status. Importantly, collection and review of e-PROM data could also substitute the need for a telephone consultation in stable patients on long-term follow-up.

Discussion

All clinicians acknowledged the extent of transformation and changes in practice involved in delivering an alternative outpatient service at the start of the pandemic. They highlighted a number of barriers to virtual pathways that would be essential to overcome to ensure sustainability.

The evaluation was conducted during the first COVID-19-induced national lockdown, since which studies exploring the impact of virtual clinics in response to COVID-19 have started to emerge. Quinn and colleagues4 reported that virtual clinics have demonstrated increased accessibility to specialist care to diabetes as patients can attend their virtual appointment from home. They also observed increased attendance among patients who did not regularly attend their appointments prior to the pandemic.4 Most clinicians in our evaluation reported benefits to patients, including not having to travel to the hospital for their appointment. This is a significant benefit given in 2017/2018, spend on transport made up 14% of all weekly household expenditures, making it the largest category of spend.14

Early evaluation of virtual clinics conducted at the Royal National Orthopaedic Hospital reported high patient satisfaction scores (90/100) for telephone and video consultations. However, patients were more likely to consider using phone consultations again after a phone appointment (94% of patients) whereas patients were less likely to request a video consultation (44% of patients).15 The authors have posed these preferences to be fluid, given the pandemic is a situation which has led patients to undergo VC regardless of their actual preferences. Qualitative feedback from patients reported some technical errors with video consultations and lower confidence levels using technology. The majority of clinicians in our evaluation were conducting virtual clinics by telephone, citing they felt most information was transmissible over the phone.

A rapid review conducted by Murphy and colleagues describes the acceptability of telehealth as a model of care for geriatric outpatient consultations. Reported challenges include technical difficulties, communication issues (such as hearing loss) and the perception of VCs to be impersonal and intimidating to some patients.16 This supports concerns raised by clinicians in the present evaluation in relation to issues with confidentiality, suggesting that some patients may not be able to speak openly.

Another limitation of virtual clinics highlighted by clinicians in this evaluation is the inability to conduct a physical examination of the patient. This has also been reported elsewhere, for instance, in oral medicine where diagnosis of lesions is dependent on visual assessment.17 Rutherford and colleagues explored patient satisfaction with current care and attitudes to virtual clinics in a general surgical outpatient department, finding in a survey of 223 patients that 88% of patients were of the view that physical examination is an important part of a consultation.18

Access to technology among patients has been widely reported as a factor determining the uptake and sustainability. This has also been reported in our evaluation as applicable to clinicians, particularly those delivering virtual clinics from home. Clinicians are typically used in hospital clinics with dual-screen monitors and hospital-issued desktop PCs which they may not have access to working remotely.19

Prior to the pandemic, it was widely reported that uptake and engagement of virtual clinics and digital health services among patients was low.20 A significant reported enabler of rapidly redesigning outpatient pathways was the national and subsequent organisational directive requiring discretion for virtual clinics. Changes in legislation that supports deployment of virtual clinics in response to COVID-19 have also been described as an enabler within other healthcare systems.21

As highlighted by clinicians in this evaluation, many patients need blood tests, imaging and other investigations, and this is a significant risk to the sustainability of virtual clinics. Consideration as to where patients should have these investigations, such as in clinics away from hospitals managing patients positive for COVID-19, has been suggested.22 A retrospective cohort analysis of patients attending virtual cardiology clinics during March 2020 and F2F consultations in the preceding year showed a 15% reduction in investigations booked in the virtual clinics and a significantly higher discharge rate.23 Conversely, other studies have reported modest reductions in discharge rates after virtual clinic appointments when compared with F2F, a finding perhaps indicative of investigative constraints or lower confidence in the clinical decision-making and thus a reluctance to discharge without a formal review.24

Clinicians highlighted factors to consider as lockdown is gradually lifted. For example, the availability of patients may change as society shifts in and out of levels of lockdown. The sustainability of virtual clinics and monitoring when children are back to school is a further challenge.25 Some clinicians are also concerned that patients may perceive VCs as a temporary measure which may affect attendance. Clinicians are also anticipating frustration from patients who are unable to progress on a solely virtual pathway such as oral-maxillofacial surgery which has a focus on clinical examination and is therefore suitable for F2F consultations only.26 Most clinicians acknowledged that one of the enablers to implementing VCs is the pre-existing rapport built up with patients during previous F2F appointments.27

The long-term place of VCs as a substitute for F2F is unclear.28 Rapid demands of the pandemic meant that appointments were simply converted to telephone or video, as opposed to developing a model of VCs that integrated with existing pathways. As we recalibrate services, it is important to understand the factors which will determine the optimum setting for the individual patient. While evidence reporting clinical outcomes of VCs is continually emerging, there is, as yet, little exploring the risks that virtual clinics introduce. These include, but are not limited to, safeguarding concerns, exacerbation of the digital divide and accessibility of healthcare, adverse outcomes for patients who would otherwise have been physically examined and lengthening of patient pathways.

We acknowledge our limited sample size comprises a proportion of the workforce at the organisation. Our aim was an early evaluation of virtual clinics conducted during the first few months of COVID-19 in the UK. Further evaluation is required to capture progression and longer term implementation of virtual clinics. Since completion of this evaluation, the findings are being used to focus outpatient improvement work at the trust. Potential barriers identified by clinicians to the longer term sustainability of virtual clinics have been extracted. It is not yet clear which metrics should inform this, but they could include monitoring of conversion rates of virtual appointments to F2F, the average number of appointments patients had during and prior to the pandemic and patient experience of virtual versus F2F appointments as well as clinical outcome measures. All clinicians agreed that a hybrid model of F2F and virtual clinics will be essential to maintaining care standards for the patients. It is therefore critical to establish the right balance of virtual to F2F activity to ensure effective service delivery and capacity planning.

We recommended further evaluation to be conducted once virtual clinic pathways have become more established. Additionally, we recommend patient satisfaction to be explored among patients attending virtual clinics in comparison to those attending F2F. A health economics evaluation would also provide valuable information, particularly in relation to the cost-effectiveness of virtual clinics.

Conclusions

The pandemic has necessitated rapid redesign of outpatient services which demonstrated the adaptability of our health system. This evaluation aimed to capture first-hand experiences and perceptions of clinicians who have dramatically transformed outpatient services in order to meet patient need during unprecedented circumstances. To our knowledge, this is the first qualitative evaluation of virtual clinic implementation across an NHS hospital trust established as a consequence of COVID-19. This evaluation will help to inform healthcare systems who are introducing virtual clinics or exploring ways to sustain similar initiatives. It is important to emphasise that this evaluation is a snapshot of the initial response to COVID-19: further structured evaluation and service-specific clinical audits will be required to compare clinical outcomes of virtual to F2F consultations and pathways.