Patient experience of virtual consultations in Oral Medicine during the COVID-19 pandemic.

OBJECTIVE: The COVID-19 pandemic resulted in a rapid shift to the use of virtual consultations in both primary and secondary care. The aim of this study was to assess patient experience of virtual consultations (telephone and video) in the Oral Medicine department during the first wave of the COVID-19 pandemic.
METHODS: A validated survey was developed with the Patient Experience Team in Guy's and St Thomas' NHS Foundation Trust. A combination of previously validated questions and newly validated psychometric questions were used to design the patient feedback questionnaire. The survey was administered to all patients following their virtual (telephone or video) consultation. Data were synthesised and electronically analysed. Qualitative data were thematically analysed.
RESULTS: A total of 115 surveys were completed. Over 82% rated their experience as good or very good and 69% preferred a virtual consultation for their next consultation. Thematic analysis of individual comments identified positive themes including convenience and positive/helpful clinical experience. Areas for development identified from thematic analysis included accessibility and clinical limitations in not undertaking a physical examination.
CONCLUSION: Overall, the patient experience of virtual consultations in Oral Medicine was positive.

INTRODUCTION

‘Teledentistry’ was first described in 1997 as ‘the practice of using video‐conferencing technologies to diagnose and provide advice about treatment over a distance’ (Jampani et al., 2011). This description has evolved to include the use of advancing information technology and electronic tools of communication for the exchange of medical or dental information. There has been slow adaptation of telemedicine to dentistry in light of the practical nature of the specialty (Mariño & Ghanim, 2013).

The COVID‐19 pandemic resulted in a rapid and sudden shift from traditional face‐to‐face consultations to the use of virtual consultations in both primary and secondary care. Following the outbreak of coronavirus in the UK, all routine dental care was suspended on the March 23, 2020. In an effort to reduce non‐essential contact and travel, a number of NHS Trusts employed digital technology to maintain a level of outpatient care and ensure patient care was not compromised for non‐COVID patients. Digital transformation is a key component of the NHS Long Term Plan first introduced in 2019 (Chapman & Middleton, 2019). The COVID‐19 pandemic accelerated many of the ambitions laid out in the plan and, in particular, expanded the use of virtual consultations, allowing the NHS to continue to provide essential services to patients.

There are few publications concerning patient feedback of virtual consultations in dentistry. A recent non‐validated patient satisfaction survey of Attend Anywhere video consultations, a NHS approved remote consultation platform, using a five‐point Likert scale found a 97% satisfaction rate amongst patients in the dental setting (Rahman et al., 2020). A common problem in adopting the Likert scale to patient satisfaction questionnaires (PSQs), occurs when a high proportion of participants select the maximum score available, referred to as the ‘ceiling effect’ resulting in doubt over the sensitivity of such PSQs. Patient experience questionnaires are a more appropriate alternative to patient satisfaction questionnaires in telehealth research. Patient satisfaction and experience are terms often incorrectly used interchangeably. Satisfaction pertains to patient expectations, and it is considered a subjective measure. Patient experience is in accordance with the patient's values, preferences and participation in healthcare decisions, may measure additional outcomes such as the degree of patient‐centred communication, treatment with respect, compassion and dignity (Langbecker et al., 2017).

Validity, in the context of a patient experience questionnaire, refers to the degree to which the data collection instrument accurately measures the patients’ experience. When deciding upon a validated instrument to assess patient experience, there are several factors to consider. A published tool does not imply that it has been well validated or psychometrically sound. Numerous instruments exist to assess patient satisfaction and/or experience in virtual consultations. A recent systematic review aiming to determine the content and construct validity covered by the available telehealth patient experience and communication tools identified twelve suitable tools. The instruments included were largely heterogeneous with respect to the type of validity being assessed. There is a paucity of validated instruments assessing patient experience of virtual consultations in dentistry (Weaver et al., 2020).

In the Oral Medicine setting, there are a number of scenarios in which virtual consultations can be helpful. This includes screening of patients to aid triage and clinic utilisation, as well as the review of patients with stable conditions, including recurrent oral ulceration, immunobullous disorders and chronic orofacial pain disorders. In addition, it allows for the delivery of investigation results, such as blood tests and biopsy results for simple mucosal lesions. High‐quality patient‐generated images provide optimisation for these virtual consultations. However, there are limitations, for example patients with oral potentially malignant disorders require meticulous examination to assess for mucosal changes and as such patients in this cohort are not amenable to a virtual consultation (Macken et al., 2020).

The aim of this study was to assess patient experience of virtual consultations (telephone and video) in the Oral Medicine department, Guy's & St Thomas' NHS Foundation Trust (GSTT), during the first wave of the COVID‐19 pandemic using a validated questionnaire developed with GSTT Patient Experience Team.

MATERIALS AND METHODOLOGY

The protocol for this service evaluation was registered with the Trust's Clinical Audit Department (Audit Number 11181). All patients attending virtual appointments in the Oral Medicine department at GSTT on the NHS Attend Anywhere virtual consultation platform and telephone consultations over a 5‐month period (May 1, 2020 to October 6, 2020) were included. The range of conditions treated during these virtual consultations were broad and included both new and review patients. Histopathology and radiological scan results were discussed, and repeat prescriptions were remotely prescribed and posted direct to the patient. No clinician experience information was obtained for this service evaluation exercise.

Staff training was provided by the hospital IT department on rollout of the virtual consultation process. Appropriate computers were installed with software, microphones and webcams to facilitate Attend Anywhere. Patients attending virtual consultations were notified of their appointment 3 weeks before their appointment by letter. A text message was also sent with a link to the Attend Anywhere virtual waiting room. A standard operating procedure (SOP) was developed by the Trust for conducting virtual consultations. At the start of the virtual consultation, patient details (address & DOB) were confirmed, and verbal consent was obtained to proceed. Clinical notes from the virtual consultation were recorded in the electronic dental record (EDR).

Survey development, design and validation

The patient experience questionnaire entitled ‘Your Virtual Appointment: Your Experience’ was developed by the GSTT Patient Experience Team (PEX). The survey consisted of 30 questions, answered anonymously and electronically by the participant. (Figure 1).

FIGURE 1

‘Your virtual appointment: your experience’: A summary of the patient of the patient experience questionnaire

The survey was designed prior to the COVID‐19 pandemic, with modifications made to reflect the increased use of virtual consultations during the pandemic. Two survey fieldwork days took place in April 2020. These days were organised the Outpatient Tactical Group (OTG) within GSTT. In attendance were members of the Patient Experience Team, clinicians of multiple grades from different specialties and patient advice and liaison services staff. The specialties involved in the initial phase included Dermatology, Allergy Services, Rheumatology and Clinical Genetics. Dental specialties, including Oral Medicine, were involved on the second fieldwork day. The following categories were considered:

Objectives: The primary objective of the survey was to assess patient experience of virtual consultations. Secondary objectives included gathering demographic details.

Order of Questions: Questions were ordered to follow a logical fashion.

Choice of questions: Questions included were from the GSTT survey question bank. This bank contains questions from national surveys (Picker) that have been conducted on behalf of the Trust. All questions have been validated in previous studies. Eleven questions were newly designed. During the design of these new questions, consideration was made to avoid double questioning, bias, leading questions and avoid the use of jargon. Validation of these new questions involved expert interviews and cognitive testing as well as, psychometric evaluation undertaken by exploratory factor analysis, reliability analysis, and construct validity assessment. A combination of open and closed questions were included. The source of questions included in the final questionnaires is summarised in Table 1.

TABLE 1

Summary of source of questions included in final survey

Question	Domain	Source
1) Type of Appointment:	Appointment	Picker Outpatient Survey 2007
2) Please tell us who your appointment was with	Appointment	Picker Outpatient Survey 2007
3) What was your appointment about?	Appointment	Newly created by the PEX team and OTG
4) If you contacted the service to book a video, telephone appointment, make an enquiry or speak to a healthcare professional, how easy was it to book online?	Patient experience/technology	Newly created by the PEX team and OTG
5) If your appointment was changed to a telephone/video appointment, were you informed about it before then?	Patient experience/technology	Newly created by the PEX team and OTG
6) What was your appointment about?	Appointment	Picker Outpatient Survey 2007
7) If you were given a specific time for your appointment, how long after the stated appointment time did your appointment start?	Patient experience/waiting	Emergency Department Survey 2008
8) How long was your virtual appointment?	Patient experience/appointments	Picker Outpatient Survey 2007
9) Did you feel this time was…?	Patient experience/appointments	New created by the PEX team and OTG
10) Did you feel you were able to get everything out of this appointment as you would in a face‐to‐face session?	Patient experience/technology	New created by the PEX team and OTG
11) Please tell us more…	Patient experience	New created by the PEX team and OTG
12) Did you have confidence and trust in the staff treating you?	Patient experience	Picker Outpatient Survey 2009
13) Did you feel involved as much as you wanted to be in decisions about your care and treatment during this appointment?	Patient experience	Picker Outpatient Survey 2009
14) How much information was given to you about your condition or treatment?	Patient experience	Ipsos MORI 2008 Quarterly telephone survey (Wave 1)
15) Did hospital staff tell you who to contact whether you were worried about your condition or treatment after your appointment ended?	Patient experience	Picker Outpatient Survey 2007
16) Did you have any concerns prior to your appointment?	Patient experience	Newly created by the PEX team and OTG
17) If yes, please tell us what they were …	Patient experience	Newly created by the PEX team and OTG
18) Did your appointment provide you with the information you required?	Patient experience	Modified from Ipsos MORI 2008 Quarterly telephone survey (Wave 1)
19) Overall, how would you rate your experience of this appointment?	Patient experience	Ipsos MORI 2008/09 Telephone survey (Wave 1)
20) Thinking about your appointment, would prefer to use this again for your next appointment (for the same/ similar condition)?	Technology	Newly created by the PEX team and OTG
21) Which of the following options below would you be happy to use whether you had to contact us again for the same/similar condition?	Technology	Newly created by the PEX team and OTG
22/23) Which of the following technology do you have regular access to, and how confident do you feel using this?	Technology	Newly created by the PEX team and OTG
24) Who was the main person answering the questions to this survey?	Demographics	Picker Young Patients Survey 2008
25) Are you male or female?	Demographics/equality & diversity	Picker Outpatient Survey 2007
26) In which age group are you?	Demographics/equality & diversity	Picker Outpatient Survey 2007
27) What is your sexual orientation?	Demographics/equality & diversity	Picker Inpatient Survey 2008
28) Which of these best describes your/your child's ethnic background?	Demographics/equality & diversity	Picker Inpatient Survey 2008
29) What is your religion?	Demographics/equality & diversity	Picker Inpatient Survey 2008
30) If you have a physical or mental disability or impairment, please tell us what it is:	Health status/equality & diversity	Picker Emergency Department Survey 2008

Responses: More open‐ended questions for example ‘Overall, how would you rate your experience of this appointment’ utilised a lengthened modified Likert scale as is illustrated in Figure 2, Question 19. More specific questions used a shortened scale, for example, ‘Did you feel you were able to get everything out of this appointment as you would in face‐to‐face session?', question 18, Figure 3. All answers were multiple choice. Some free text or white space questions (questions 11 and 17) were also included. The demographic‐related questions were single answer.

FIGURE 2

Summary of responses to Question 19: Overall, how would you rate your experience of this appointment?

FIGURE 3

Summary of responses to Question 12: Did you have confidence and trust in the staff treating you?

Length of questionnaire: It is generally acceptable that patient experience questionnaires should take no longer than 15–20 min, to avoid participant fatigue and the ‘ceiling effect’. (Rolstad et al., 2011)

Layout and format: A questionnaire template was used and included an opening paragraph, closing statement and data protection statement.

Taking all the above into account, the survey was developed with 30 questions and piloted on two separate patient groups. The pilot groups consisted of voluntary participants from the specialties included in the initial design. Each group consisted of 20 patients, and the demographics varied with regard to age, ethnicity and health status to ensure a heterogeneous sample and representative analysis. Minor adjustments were required following suggestions from these pilot groups. These included changes to the order of questions and the omission of two questions to allow the survey to be completed in the suggested time period.

Following adaptations, this final survey was sent to all dental patients following their virtual consultation. Participation was voluntary. No survey specific questions were included. Questionnaires were electronically reported directly via the patient experience portal. The responses were collated, analysed and then reported via the Civica Experience® cloud‐based software solution.

Qualitative data (comments entered in free text areas) were thematically analysed using the six‐stage process proposed by Braun & Clark (Braun & Clarke, 2014). The six stages were familiarisation, coding, generating themes, reviewing themes, defining and naming themes and finally summarising the analysis. This process was undertaken by two authors (CH and DB).

RESULTS

A total of 115 participants completed the survey. The response rate was 9.8%. (115/1173).

Demographic details

The largest proportion of respondents were in the 55–64 age group (32%) (37/115), were female (28%) (32/115) and White British (64%) (74/115). Twelve per cent (14/115) of the sample reported a mobility difficulty, while 4% (5/115) and 1% of the sample reported hearing and communication impairment, respectively. Of the 115 completed questionnaires, 94% (108/115) were completed by the patient themselves, 3% (4/115) were completed by a carer/family member and both the patient and carer/family member (4/115), respectively.

Survey results and thematic analysis

The majority of appointments were with a doctor or dentist (75%) (86/115); the remainder of consultations were with allied health professionals (e.g. clinical psychologists) and registered dental nurses. There was no stratification based on the grade of doctor/dentist. The largest proportion of consultations were review appointments (46%) (53/115), with the remaining being new patient consultations (22%) (25/115), pre‐planned telephone appointments (16%) (18/115) or the result of an online request to speak with a professional (4%) (5/115) and medication‐related queries (6%) (7/115). Six per cent (7/115) did not answer this question. The majority of consultations lasted less than 15 min (63.48%) (73/115), which most found this to be ‘about the right time’ (85.22%) (98/115). Less than 2% (2/115) felt they were not involved as much as they wanted to be in decisions about your care and treatment during their consultation. Almost 10% (11/115) felt that not enough information was given and 6% (7/115) felt the appointment did not provide them with the information required.

More respondents described their smartphone as ‘technology they have regular access to’ and felt more confident to use (81/115), when compared to telephones and other devices such as tablets and computers/laptops.

Thematic analysis of the comments identified positive themes, including convenience and positive / helpful clinical experience. Examples of comments that illustrate the convenience strand identified included:

‘effective and efficient for that stage of my treatment. It works well alongside face‐to‐face appointments. It makes better use of NHS resources’

‘it was good and a lot less onerous than travelling to hospital and waiting. COVID aside, a very efficient approach for the future’.

Examples of comments illustrating a positive clinical experience:

‘The appointment was great and I was able to discuss my symptoms and sent pictures, but I need to attend face to face for swabs to be completed’

‘I feel I had excellent treatment from Guy's and a rapid scan was arranged recently to check my issue. Excellent experience and thank you to all in these very challenging times’.

As is illustrated in Figure 2 and Figure 3, 78% (98/115) responded ‘yes, always’ when answering a question related to confidence of staff treating you and over 80% (96/115) rated the overall experience as good.

Areas for development identified from thematic analysis included accessibility and clinical limitations. A substantial number of comments identified the need for a face‐to‐face appointment, examples illustrating this include:

‘I think given the circumstances that the consultant did very well in trying to resolve my health issues. However I feel a video or face to face is probably easier to diagnose. I’m relieved I didn't have to go to an appt during this time but it's also difficult to describe fully the extent of how my condition affects me without being able to show someone.’

‘I'm seen at Guys Oral Medicine – require visual inspection of my condition as risk of pre cancer’.

While a large proportion of the sample felt they were able to get everything from the consultation and would prefer a virtual consultation for their next appointment when compared to a face‐to‐face appointment, there was still significant portion that answered no to these questions (13.91% (16/115) and 19.3% (22/115), respectively (Figures 4 and 5)).

FIGURE 4

Summary of responses to Question 10: Did you feel you were able to get everything out you would in face‐to‐face session?

FIGURE 5

Summary of responses to Question 20: Thinking about your appointment, would you prefer to use this again for your next appointment (for same/ similar condition)?

With regard to accessibility, several technical issues were identified varying from ‘the wrong link’ being sent and ‘the platform not recognising me’. Inherent scepticism of virtual consultations was also identified, ‘was a bit suspicious of a scam’. A large portion (22%) (22/115) were not informed by staff, letter, text or email if their appointment was changed to a virtual one (Figure 6).

FIGURE 6

Summary of responses to Question 5: If your appointment was changed to a telephone/video appointment was changed to a telephone appointment, were you informed about it before then?

DISCUSSION

As the COVID‐19 pandemic progressed, healthcare providers were forced to redesign care pathways to ensure the ongoing delivery of ‘business as usual’ services, including elective outpatient appointments, while minimising travel and person‐to‐person contact. Virtual consultations provide an opportunity to do this safely and effectively, while meeting social distancing requirements and minimising patient flow through hospitals.

There is a paucity of validated patient experience questionnaires for virtual consultations in dentistry. Several studies completed in the dental setting during the COVID‐19 pandemic, focus on patient satisfaction outcomes rather than experience, with the level of satisfaction generally reported to be high. The reported satisfaction in the medical environment during the COVID‐19 pandemic was also observably high amongst a wide spectrum of medical specialties. (Andrews et al., 2020) There are few validated questionnaires that can be used in telemedicine and those that do exist were not designed to be used in this context. In addition, the validity of these questionnaires has not been tested in the specific telemedicine setting (Weaver et al., 2020). To the best of the authors’ knowledge, this is the first study assessing patient experience of virtual consultations using a validated questionnaire tool in the dental setting, as well as in Oral Medicine. A recent study completed corroborates our findings in the Oral Surgery setting, commenting on positive patient experience; however, there was no commentary on the validation of the survey instrument used and a prospective approach utilised does not allow for direct comparison. In addition, majority of the questions in the survey tool focus on patient satisfaction (Ibraheim et al., 2021).

While the survey used was validated and utilised questions from previous validated questionnaires, a number of limitations were identified. All questions included in the questionnaire were either validated individually or as part of the original source survey. The final survey combined these questions in a new format, and the validity of the final questionnaire was assessed in the Trust prior to its use in our setting. However, we acknowledge the narrow setting for validation, including both the type of hospital, patient demographics and London setting. It would be useful to test the questionnaire in additional settings, in particular primary care, to ensure its overall validity.

Psychometric validity used in new question development introduces a degree of ambiguity. Validity evidence gathered, such as predictive capability and accuracy are difficult to combine meaningfully, as many of these constructs are fundamentally divergent. Psychometric analysis is still, however, the most widely used means of validity testing in survey development in patient‐reported experience measures due to reproducibility. An alternative method of validating involves eliciting the relative importance of candidate questionnaire items and when this method has been applied, differences in final content of survey was different when psychometric analysis was used in isolation. Questions that patients may consider important may be excluded using psychometric validity. (Sizmur et al., 2020) The survey was not designed exclusively for assessing virtual consultations in dentistry or Oral Medicine; the application of this survey in restorative and surgical dental specialties may require adaptation to reflect the difficulties in virtual consultations in these specialties. Finally, as this was undertaken as a service evaluation, data obtained were unidirectional and no staff feedback was obtained. It would be interesting to assess clinician feedback on the merits and disadvantages of remote consultation in further studies.

While the response rate was low (8.9%), the absolute number of completed questionnaires (115) was felt to be adequate to allow for analysis and meaningful conclusion to be drawn from the sample and reflect the scope of the specialty and accurately reflect patient experiences during the height of the pandemic in the United Kingdom and Europe. This unique time cannot be replicated, and as such, the evaluation and reporting of patient experiences during the emergence of COVID‐19 provide an invaluable insight despite the low response rate. As this is the first service evaluation of its kind, the results do still add to the evidence in this area. A number of positive themes were identified including convenience. Oral Medicine is typically delivered in the dental hospital setting in major cities and patients in more rural settings may have to travel long distances to attend clinical appointments, where possible virtual consultations are being encouraged for these patients for stable conditions where a face‐to‐face appointment may not be necessary. There is no replacement for the direct visual examination and this was identified in our thematic analysis, as a result there has been a reduction in virtual consultations where potentially malignant disorders are concerned and direct visual examination is of paramount importance. Accessibility to technology and technological issues were also identified and it is anticipated that the service changes proposed above will help overcome some of these, many of which may be applicable to departments that have encountered similar issues for example patient notification of appointments.

Although not specifically addressed in this study, virtual consultations offer potential advantages for healthcare providers at hospital and primary care level. These include improved cost‐effectiveness, the ability to extend access to specialty services and mitigation of staff shortages. In the context of COVID‐19, staff who fall in high‐risk categories and were self‐isolating due to clinical vulnerability, virtual consultations allowed this staff group to work remotely. Given the operative nature of dentistry as a specialty, a major disadvantage is the lack of clinical examination and the ability to perform treatment. This has particularly been the case for the restorative and surgical sub‐specialties. In the Oral Medicine setting, clinical examination is crucial to exclude precancerous and cancerous lesions. This was evidenced by several comments in the thematic analyses where participants identified the clinical limitations of virtual consultations. A substantial number of comments identified the need for a face‐to‐face appointment, and these comments were more frequently noted when potentially malignant disorders were considered. Examples of some of these comments are above. While a large proportion felt they were able to get everything from a virtual consultation and preferred this mode for their next appointment, a significant proportion did not (13.91% and 19.3% respectively), clinical limitations (and lack of direct visual examination) and difficulties with accessibility could account for this. This study reports the findings of a unidirectional service evaluation, and therefore, no clinical data were collected, such as diagnosis or reason for attending. Further studies examining patient experience linked to clinical information would be useful, especially in oral medicine, to identify which patient groups might benefit most from virtual appointments and those for which this option is limited.

In addition, there also may be a lack of appropriate technological services and appropriate training on how to use these services. There are also confounding issues surrounding the use of patient data and ethical issues regarding information governance (Abbas et al., 2020; Almazrooa et al., 2021; Aquilanti et al., 2020; Byrne & Watkinson, 2020; Crawford & Taylor, 2020; Ghai, 2020; Santana et al., 2020). A relatively recent systematic review, concluded on the emerging evidence to support the use of virtual consultations in dentistry, however, commented on the lack of conclusive evidence to make evidence‐based policy decisions. (Estai et al., 2018) A more recent systematic review completed during the pandemic also commented on the potential benefits of teledentistry; however, there were many limitations to this review, including a large amount of heterogeneity between studies included, and none of the studies included focused on patient experience. (Achmad et al., 2020).

From our analysis of responses, a number of service changes have been planned including encouraging conversation regarding clinic design and patient preference (face‐to‐face versus virtual), a review of the suitability of clinical problems to virtual clinics, communication skills training for virtual consultations, a review of the mechanisms by which patients are notified of appointments and to provide more information to patients regarding their appointment (e.g., timing).

In Oral Medicine, it is anticipated that face‐to‐face consultations will remain the gold standard for many patient consultations, especially when oral potentially malignant disorders are considered and direct visual examination is required, even when high‐quality patient‐generated images are available. This was reflected in our analysis where a number of participants identified the need for face‐to‐face clinical appointment. However, virtual consultations offer a cost effective and efficient alternative for the review of stable oral mucosal disorders and facial pain patients (Macken et al., 2020).

CONCLUSION

There is a lack of universally accepted patient experience questionnaires for virtual consultations in dentistry and specifically oral medicine. Current questionnaires focus on patient satisfaction rather than patient experience and therefore do not consider patient's values, preferences and participation in healthcare decisions. The overall patient experience assessed using this validated questionnaire indicates a positive patient experience of virtual consultations during the COVID‐19 pandemic. It is expected that virtual consultations will continue within the Trust long after the pandemic has ended.

CONFLICTS OF INTEREST

None to declare.

AUTHOR CONTRIBUTIONS

Vignesh Eswara Murthy: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Software; Supervision; Validation; Visualization; Writing‐original draft; Writing‐review & editing. Cameron Herbert: Data curation; Project administration. Davinder Bains: Conceptualization; Data curation; Project administration. Michael E Escudier: Writing‐review & editing. Barbara Carey: Conceptualization; Data curation; Investigation; Methodology; Project administration; Supervision; Writing‐original draft; Writing‐review & editing. Martyn Ormond: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Software; Supervision; Validation; Visualization; Writing‐original draft; Writing‐review & editing.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1111/odi.14006.