A permanent legacy of the pandemic? Outcomes of and staff views on the introduction of virtual clinics to an Irish oncology service.

BACKGROUND: Virtual clinics were introduced to our practice in March 2020. We aimed to assess outcomes from virtual clinics and to assess staff views on them and their barriers to implementation nationally.
METHODS: We prospectively assessed outcomes from 53 planned virtual consultations in a cancer centre oncology outpatient department (April-July 2020). Thirty-two oncologists completed an online survey.
RESULTS: Visit durations ranged from < 5 min (n = 2, 4%) to 30 + min/patient (n = 9, 20%) (median: 18 min (range 4-141, IQR 10-30 min)). Median time spent preparing for patients who did not attend (n = 6, 11%) was 15 min (range 9-15 min). Most patients were scheduled for routine follow-up (n = 41, 87%), with some planned for an early in-person visit (n = 3) or investigation (n = 3). Where bloods had been requested (n = 25), samples had often not been taken (n = 20, 80%) or results were unavailable (n = 3, 12%). Different plans may have been agreed with two patients (4%) had they attended in-person. Virtual visits were perceived as faster by most doctors in the online survey (n = 26, 84%), with some (n = 5, 16%) reporting a difference of 10 min per patient. Many (n = 13, 42%) arranged earlier follow-up appointments. Low satisfaction was associated with difficulty with patient assessment (81%) or communication (63%), resource limitation (48%), or poor access to results of investigations (40%). The majority (n = 21, 67%) do not feel their virtual clinic quality is as good as in-person.
CONCLUSIONS: If virtual clinics are to play a long-term role in oncology, it is essential to monitor clinic quality and plan visits proactively.

Introduction

Telemedicine involves the use of telephone or video calls to facilitate the care of patients not physically in the hospital. While it has become more commonly utilized in Irish healthcare as a response to the COVID-19 pandemic [1], it has been used in the international oncology community for some time. Pre-pandemic applications have enabled access to specialist treatment for patients in rural areas [2, 3], including allowing discussion of their cases at multidisciplinary tumour boards [2] and enrolment in clinical trials [4]. It has also enabled other aspects of cancer care, such as teledermatology reviews, which have been effective both in improving early skin cancer diagnostics at the primary care level [5] and obtaining specialist review for patients with chemotherapy-related skin toxicities after starting treatment [6]. Other studies have reported substantial time and travel savings for patients [7] and improvements in quality of life [8].

Barriers to uptake of teleoncology in Ireland have included low patient/staff computer literacy, lack of equipment or Internet connectivity, privacy and data protection issues, concerns regarding damage to the doctor-patient relationship, a lack of implementation models, and resistance to change [7]. Forced adaptations during the COVID-19 pandemic have helped to overcome some of these, and 2020 has seen significant increases in the use of telemedicine both internationally and in Ireland: over 20% of Irish adults have now had a “virtual” appointment, compared to only 4% pre-pandemic[1].

Because those with cancer are at high risk of negative outcomes from COVID-19 [9, 10] and of nosocomial infection [10], amongst other adaptations in our service [11, 12], some outpatient visits in our hospital were converted to virtual reviews, similar to the strategy used in other Irish units [13, 14]. Virtual reviews can reduce patient exposure from infected staff/other patients, staff exposure risk, and PPE use and facilitate waiting-room social distancing for those who must attend in-person, while providing continuity of care.

Though some centres have used video calling [15], 19% of our patients reported poor Internet access [12], and the hospital had limited video calling facilities; therefore, clinics were run on a telephone-only basis. Patients were screened by their oncology consultant for suitability for the virtual clinic and seen in-person if there was an anticipated language/communication barrier, or if the visit was likely to be complex, such as after disease progression.

The majority of both international [16] and Irish oncology patients report high satisfaction with virtual clinics and believed they could play a future role in their care [14]. Despite this, other patients report very low satisfaction, and in a previous study of the oncology patients in our hospital, 16% cited virtual clinics as the pandemic-related modification to their care that they found most difficult [12]. Despite the significant volume of literature relating to virtual clinics and telemedicine, few studies have assessed their impact at levels other than psychosocial issues, patient experience and quality of life [17], and the perceived benefits often related to convenience factors such as savings on travel times or parking payments [14] rather than to clinical outcomes. Although these factors are important, they do not necessarily translate to providing an appropriate standard of care. In other work, oncologists have worried that virtual clinics are reliant on patient knowledge of what was important to report [18]. Overuse of virtual clinics, while possibly reducing economic barriers, risks disadvantaging those who are less medically literate, or have cognitive/communication problems, and a patient who wishes to raise sensitive topics/ask for in-person care may not feel able to do this without strong self-advocacy skills. In pre-pandemic work in the USA, many oncologists reported concerns about appropriately assessing patients virtually without a physical examination [18], a concern that many of our colleagues had shared. Doctors seeing patients in a virtual primary care setting were less likely to have objective assessment data such as recent blood pressures[19], a problem which may also apply to our patients. We attempted to explore outcomes from patients seen in our virtual clinics, and staff experiences of virtual clinics within the national service, in order to identify potential targets for quality improvement.

Methods

The details and outcomes of 53 patients, scheduled to attend virtual clinics, in a tertiary Irish cancer centre between April and July 2020, were recorded by oncology doctors (n = 6). These virtual clinics replaced routine outpatient appointments. Patients were on surveillance following systemic anti-cancer therapy (n = 36, 68%), or were receiving hormonal therapy (n = 16, 30%). One patient was being contacted as a follow-up after a recent admission.

At 6-month follow-up, patient outcomes were assessed to identify disease relapses/recurrences. If patient records stopped abruptly, local newspapers and websites (rip.ie) were searched for an obituary.

In January 2021, an electronic survey was distributed to oncology doctors recruited primarily from a national WhatsApp group. Thirty-two responses were received (approximately 40% response rate). The survey contained demographic questions, qualitative opinion questions, and 5-point Likert scales (“strongly agree” to “strongly disagree” in response to test items). Likert responses of “neither agree nor disagree” were excluded from analysis.

A patient survey was planned, but data is not presented due to poor recruitment and sampling bias.

The study was conducted under the supervision of the Tallaght University Hospital/St. James’s Hospital Joint Research Ethics Committee.

Data was analysed with the Statistical Package for the Social Sciences (SPSS) Version 27 (IBM Corporation, Armonk, NY). Descriptive statistics are reported as median and range for quantitative variables or percentages and frequencies for categorical variables. Differences between groups were evaluated using independent sample t tests, chi-square analyses, and ANOVAs. A p value of < 0.05 was considered statistically significant.

Results

Virtual clinic outcomes

Participant characteristics are shown in Table 1.

Table 1

Patient demographics

		Scheduled (n = 53)	Attended (n = 47)
Age	(Years, median, range)	61 (22–84)	62 (22–84)
Cancer	Gynaecological	22 (40%)	18 (36%)
	Breast	8 (15%)	6 (13%)
	Testicular	7 (13%)	7 (15%)
	Prostate	6 (11%)	6 (13%)
	Renal	2 (4%)	2 (4%)
	Lymphoma	2 (4%)	2 (4%)
	Lung	3 (8%)	3 (8%)
	Gastrointestinal	2 (4%)	2 (4%)
	Sarcoma	1 (2%)	1 (2%)
Gender	Male	18 (34%)	17 (36%)
	Female	35 (66%)	30 (64%)
Time since diagnosis	(Months, median, range)	27 (2–170)	23 (2–170)
Disease status	No evidence of disease	34 (64%)	29 (62%)
	Hormonal therapy	16 (30%)	14 (28%)
	Subsequent recurrence/progression	14 (26%)	13 (28%)
Intended visit purpose	Essential clinic exam	19 (36%)	17 (36%)
	Results of tumour markers	25 (47%)	22 (47%)
	Results of radiology imaging	13 (25%)	12 (26%)
Clinic duration	(minutes, median, range)	18 (4–141)*	18 (4–141)*

*Times not recorded for 5 patients, times recorded of > 2 h with no explanation for long duration (n = 3) excluded from analysis. No significant differences between “scheduled” and “attended” for any field

Clinic appointments, including time to prepare the chart, request investigations, and dictate correspondence, had a median duration of 18 min (range 4–141 min, IQR 10–30 min). Visit durations ranged from under 5 min (n = 2, 4%), to 5–10 min (n = 12, 29%), to over 30 min per patient (n = 9, 20%). For the 6 patients who did not attend, a median time of 15 min (range 9–15 min) was spent preparing/attempting to contact them.

Most patients were scheduled for routine follow-up (n = 41, 87%), with a minority scheduled for non-routine follow-up (n = 6, 13%), such as an early exam/in-person visit (n = 3) or investigations due to new symptoms/concerns (n = 3).

Where bloods had been requested prior to the clinic (n = 25), samples had often not been taken (n = 20, 80%) or had been taken by the GP but no results were available (n = 3, 12%).

For most appointments, doctors were satisfied with the operation of the virtual clinic (n = 20 of 30 responses, 67%, non-responses = 23). Satisfaction was lowest in genitourinary cancers (HR 9.3, 95% CI 1.6–53.2, p = 0.01). There was a trend to association between low satisfaction and missing bloods results (p = 0.07). Doctor satisfaction was not predicted by any other factors.

At 6-month follow-up, 5 patients had died, and 14 (26%) had recurrences/progression of their disease. In most cases (n = 9, 64%), the outcomes would have been the same had they had an in-person visit (e.g. recurrence identified as a result of symptoms reported in the virtual clinic), while in other cases (n = 4, 29%), the outcomes would likely have been the same (e.g. asymptomatic radiological progression at subsequent visit). One patient had a pattern of alcohol misuse, of which we were not aware. Serology results were unavailable for another patient, and when results were received several weeks later, her CA-125 was > 2000 kU/L, having previously been normal, with disease recurrence later confirmed radiologically.

Doctor survey

The demographics of the 32 doctors completing the survey are described in Table 2.

Table 2

Surveyed doctor demographics

		N (%)
Medical experience†	< 5 years	6 (19%)
	5–7 years	13 (42%)
	8–10 years	10 (32%)
	10 + years	2 (6%)
Virtual clinic experience (% of outpatient encounters in 2020)	< 5%	1 (3%)
	5–20%	1 (3%)
	20–50%	12 (38%)
	> 50%	18 (56%)
Patient type†	Pre-screened, mostly simple	24 (77%)
	Not pre-screened, included complex cases	7 (23%)
Currently working†	Designated cancer centre	21 (68%)
	Other oncology unit	7 (23%)
	Outside Ireland*	3 (10%)
Visit times† (in comparison to a similar in-person clinic)	More than 10 min longer	0
	5–10 min longer	1 (3%)
	1–5 min longer	2 (7%)
	Similar	1 (3%)
	1–5 min shorter	14 (45%)
	5–10 min shorter	8 (26%)
	More than 10 min shorter	5 (16%)
“I brought patients for follow-up back sooner” †	Agree/strongly agree	13 (42%)
	Neither agree nor disagree	2 (6%)
	Disagree/strongly disagree	16 (52%)
“I could communicate well during the virtual visit” †	Agree/strongly agree	21 (67%)
	Neither agree nor disagree	5 (16%)
	Disagree/strongly disagree	5 (16%)
“Virtual clinics were better in quality overall as an outpatient visit” †	Agree/strongly agree	7 (23%)
	Neither agree nor disagree	3 (10%)
	Disagree/strongly disagree	21 (67%)

*These doctors had been employed in Ireland until the July 2020 changeover, so their views were included

†Missing responses = 1

Visit times were perceived as shorter by most doctors (n = 26, 84%), with some (n = 5, 16%) reporting their virtual clinic duration was typically shorter than in-person equivalents by over 10 min per patient, 42% (n = 13) arranged earlier follow-up appointments than they would typically schedule for patients seen in-person. Higher satisfaction was seen in those reporting shortest visit durations (5 min or more shorter than in-person equivalents, X2 (1, N = 31) = 7.1, p < 0.01, OR 15, 95% CI 2–144). Satisfied doctors were less likely to bring patients back early for follow-up (X2 (1, N = 31) = 6.5, p = 0.01, OR 0.6, 95% CI 0.4–0.9).

Several doctors reported frequent difficulty in contacting patients (n = 9, 28%). Others reported that patients had often not been informed of the virtual clinic prior to being contacted by the doctor (n = 8, 25%). This was associated with low satisfaction (X2 (1, N = 25) = 4.6, p = 0.03, OR 0.6, 95% CI 0.4–0.9). Higher satisfaction was seen in those who were usually able to contact patients (X2 (1, N = 25) = 17.3, p < 0.001, OR 4.5, 95% CI 1.5–15.3).

Doctor satisfaction was lower in those with difficulty with patient assessment (X2 (1, N = 31) = 15.7, p < 0.001, OR 0.02, 95% CI 0.001–0.231) or communication (X2 (1, N = 31) = 4.1, p = 0.04, OR 0.17, 95% CI 0.03–1.06), resource limitations (X2 (1, N = 31) = 8.5, p = 0.004, OR 0.6, 95% CI 0.4–0.9), or poor access to results of bloods (X2 (1, N = 23) = 5.3, p = 0.02, OR 0.6, 95% CI 0.4–1).

Many doctors (n = 14, 45%) believe virtual clinics required a lot of additional support from primary care teams, more than what is typically required for “in-person” clinics (of note, we do not believe respondents were disparaging primary care in any way by this statement; this is explored further in the discussion).

Other concerns such as patient privacy (n = 4), patient distractibility (n = 3), and patient resistance to virtual clinics (n = 5) were not major barriers, cited by less than 20% of doctors. Barriers are described in Table 3.

Table 3

Barriers to successful virtual clinics

		N (%)
Assessment	Any assessment issue	26 (81%)
	Performance status more difficult to assess	23 (74%)
	Lack of exam to assess treatment response	21 (68%)
	Lack of exam to assess drug toxicity	17 (55%)
	Lack of a collateral history from family	8 (26%)
Communication	Any communication issue	20 (63%)
	Difficulty contacting patients	9 (28%)
	Patient hearing	7 (23%)
	Need to repeat information to relatives	11 (35%)
	Need for translator	13 (42%)
Investigations	Difficulty accessing bloods/x-rays	12 (40%)
Resources	Any resource issue	15 (48%)
	No suitable physical space	7 (23%)
	Inadequate time scheduled	6 (19%)
	More frequent interruptions	7 (23%)

There was a trend towards higher satisfaction in those with most experience of virtual clinics. Of the doctors who had worked in centres where 50% or more of patients were seen virtually, 33% were satisfied (6 of 18), compared to only 8% otherwise (1 of 13) (p = 0.09). Satisfaction was not impacted by any other factors, including medical experience or patient complexity.

Despite these limitations, some doctors believe the virtual clinic is the same as (n = 3, 10%) or better than (n = 7, 23%) an in-person clinic in terms of quality, and most (n = 21, 68%) reported that they were able to communicate well.

The most popular improvement to virtual clinics was better access to external results, such as GP bloods (n = 24, 77%). Most respondents thought that patients should have no more than two consecutive virtual visits (n = 22, 71%), while few believed that video calling would be a useful addition (n = 10, 32%).

Qualitative responses are described in Box 1.

“[virtual clinics] will have a place if they are supported by the ability to see a patient promptly if there's an issue identified” (consultant, designated cancer centre, >15 years experience)

“...focus on oncology issues, rather than 20 mins with someone who had a T1 breast cancer in 2006 telling you about dietary intolerances!” (registrar, designated cancer centre, 5-7 years experience)

“should not be used for patients who don't like to attend, these can be the most difficult to assess over the phone” (registrar, designated cancer centre, 5-7 years experience)

“given 10 [virtual clinic patients to contact] while on annual leave, told they were “only virtuals” & wouldn’t take long” (registrar, designated cancer centre, 5-7 years experience)

Discussion

For the patients included in our study, the same outcomes would have been achieved in the virtual setting or in-person, with only two exceptions (4%). Most doctors reported communicating well in virtual clinics and being mostly satisfied with the assessed visits, but only 33% felt that virtual clinics were of similar/superior quality to in-person equivalents.

Almost all doctors (81%) reported some difficulty with patient assessment. This included a lack of physical examination, incomplete investigations, and judgements of performance status. In our selected cohort of patients, pre-screened as being suitable for a virtual visit, 13% were still called back for an early investigation/in-person visit, and 42% of our peers reported regularly doing this. In a bid to compensate for limited assessments, doctors might order more radiological imaging [20], exposing the patient to unnecessary testing and putting more stress on the healthcare system.

In addition to reduced qualitative assessments, many clinics were conducted without quantitative data. While patients often have not had recent tumour markers prior to clinic, in the face-to-face setting it is possible to obtain samples via hospital phlebotomy, and review the results the same day. If virtual clinic samples are processed in a patient’s local hospital, results may not be received for weeks, and though this will be partially resolved with a national laboratory information system, this is likely several years away still.

In our pre-selected cohort, the main visit purpose (review of tumour markers/physical exams) was often not achieved. Given that much of surveillance care is based on either physical examinations or monitoring of tumour markers, clinics conducted without these risk being superficial “box-ticking” exercises that provide only false reassurance. Doctors with more experience of virtual clinics reported higher satisfaction with them, which may suggest that higher-volume centres have better processes for managing virtual clinics, though an alternate explanation is that those with more exposure are more accustomed to the discomfort of making decisions without all relevant information.

Relatively little of the literature has examined clinic outcomes, or even considered how we should measure quality in telemedicine [17]. One framework [21] suggests measuring quality in terms of access to care, financial impact/cost, experience, and effectiveness. Some possible quality improvement metrics are suggested in Box 2. As most patients will likely receive hybrid care, it may be appropriate to apply some of these metrics to the department as a whole. Some key performance indicators for quality care have been outlined in other countries [22], and new standards have been introduced by ASCO very recently [23]. Many of these issues raised in our study, such as patient preparedness, and the need to proactively structure virtual visits, have been found internationally, and were discussed extensively at the ASCO annual meeting this year by Dr Bakitas and Dr Mulvey [24, 25]. Local guidelines, such as those from the Irish Medical Council[26], should also be followed.

Though our “did not attend” rate, at 11%, was consistent with national estimates for face-to-face outpatient services, patients being uncontactable in virtual clinics caused greater inefficiencies. Doctors spent significant time reviewing correspondence, laboratory, and radiological reports prior to contacting a patient, whereas in face-to-face clinics, this is not done until patients have arrived in the waiting room. Most doctors felt virtual clinics were faster, but some very long visit times were recorded in our review. As 42% of doctors are also bringing patients back for follow-up sooner, erosions are easily made into ‘time savings’.

Doctor satisfaction was lower where patients had not been expecting a call, and efforts should be made to advise patients of upcoming virtual clinics, both as common courtesy, to allow them to prepare questions and to minimize the wastage of staff time.

In many cases, virtual clinics were supplemented with additional general practitioner care. Almost half (45%) of the surveyed oncologists felt that virtual clinics relied on more GP support than in-person clinics. While many GPs were extremely obliging in providing care that would typically be a hospital responsibility (e.g. breast examinations, phlebotomy), its sustainability in the long term is doubtful without significantly greater primary care resourcing. Resourcing of virtual clinics was similarly an issue, with approximately 20% of doctors reporting they were not given a suitable physical space or time to conduct the virtual clinics in, or that they were interrupted more often. While virtual clinics have allowed us to work remotely during COVID-19-related self-isolation, there were reports of doctors being asked to call patients while on annual/sick leave.

Many hospitals are investing in video calling equipment—our study suggests that “videoclinics” would not be popular. Funding might be better used to allow three-party phone calls to include a translator or the patient’s family, the exclusion of whom were barriers for 40% and 35% of doctors, respectively.

Some caveats apply. Cancer subtypes were not representative; only two patients (4%) with gastrointestinal cancer were included, though these represent 32% of our patients[27]. Few patients in our study had multiple consecutive virtual visits, “doctor satisfaction” is recorded in only 56% of cases, and as our data was collected by a small number of doctors, with patients pre-screened for suitability, it may not generalize to other services.

Conclusions

Despite barriers, most doctors felt they could communicate well. While faster for many, time and cost savings are eroded as many doctors bring patients back for follow-up early. If telemedicine is to play a role in post-pandemic oncology services, successful implementation requires robust methods for ensuring patients have had necessary investigations before clinic, appropriate resourcing both in hospital and at primary care, careful selection so that only appropriate patients are seen virtually, and ongoing quality improvement to ensure that patients continue to receive high standards of care.

Access to care	• Patients should receive support to facilitate communication. If spending on translator services after the introduction of virtual clinics is lower, it may suggest that some patients are being disadvantaged   Monitor how far in advance of clinics patients are notified and if they receive a “text reminder” • Proactive visit planning and monitoring if virtual clinic lists are being screened for appropriateness. This could consider:   o When the patient was last seen in-person   o If they have previously indicated they do not want their care to be virtual   o If they have communication or technological barriers   o If the patient is likely to need care that can only be provided in-person   o If investigations have been requested, that the results are available • Monitor waiting times for those who need in-person follow-up after virtual clinics • Monitor referrals to the emergency department from virtual clinics
Financial impact/service efficiency	• Monitor “did not attend” rate: the HSE target for this is < 10%, but a lower target may be appropriate in the virtual setting, because of the extra burden on clinical staff in attempting to contact patients • High “did not attend” rates in those who consistently attend face-to-face appointments may suggest technological barriers play a role • Monitor virtual clinic start and end times, and impact on staff overtime pay
Experience	• Patient feedback, input from advocacy groups, patient-reported outcome measures • Monitor patient complaint rates, closely investigate those involving virtual clinics • Staff feedback: some centres have optimized workflow by assigning virtual reviews to registrars/nurse practitioners, while consultants see more complicated in-person patients. Feedback should be sought from those with recent high-volume experience • Assess if virtual clinics are being conducted in appropriate settings, with dedicated time and space
Effectiveness	• Audit against guidelines—where surveillance recommendations include tumour markers/physical examinations, are these being followed, or are radiological investigations requested more frequently than expected • Indicators suggesting sub-optimal assessments, e.g. dayward cancellations because of falling performance scores/toxicity that had not been detected in a recent virtual clinic • Indicators suggesting sub-optimal communication, e.g. chemotherapy deferrals because patients had not taken premedications, patients/families contacting secretaries post-clinic to gain clarification on information given • Long-term quality metrics, e.g. number of patients “lost to follow-up”, number of patients referred to smoking cessation/screening for clinical trials