Accuracy of virtual assessment in hand trauma.

A virtual clinic was developed from an existing telemedicine system to manage hand trauma in the Queen Victoria Hospital, East Grinstead, UK, during the first wave of the COVID-19 pandemic. This study evaluates the accuracy of the assessments made and makes comparisons to the traditional face-to-face clinic. The accuracy of assessment was analysed by comparing diagnosis with findings at surgery. One hundred and eighty-nine virtual assessments conducted by telephone with photographic data or by video were compared to 129 face-to-face assessments conducted prior to the pandemic. There was no difference in the accuracy of virtual and face-to-face clinics for patients treated surgically (p=0.27); treatment was correctly predicted for 87% of the virtual group and 78% of the face-to-face group. However, fewer virtual assessments led to a surgical outcome (p=0.0064); 68% of the virtual group had surgical outcomes compared to 82% of the face-to-face group. Most face-to-face assessments were done by a specialty trainee compared to a range of clinicians in the virtual clinic. Accuracy of assessment among trainees was significantly associated with experience for the virtual (p=0.045) but not face-to-face clinics (p=0.94). Virtual assessment by video versus telephone plus photographs were similarly accurate. We conclude that virtual triage serves as a successful alternative to face-to-face appointments. It is robust and succeeds in reducing footfall to the hospital site in line with recent social distancing measures against COVID-19. We have shown that video conferencing triage is no better than telemedicine triage with telephone and photographs.

Introduction

The 2019 novel coronavirus (COVID-19) pandemic has led to unprecedented pressure on health services. In our hospital, this led to the postponement of non-urgent procedures, prioritisation of cancer and trauma surgery, and utilisation of the local private hospital to provide ‘clean’ and ‘dirty’ operating sites. Outpatient appointments largely became telephone or video consultations, and a dedicated virtual hand trauma clinic was set up. We aim to compare outcomes between this new virtual hand trauma clinic and the traditional face-to-face clinic, with a focus on the accuracy of assessment of face-to-face, video, and telephone triage.

The Queen Victoria Hospital (QVH) is a specialist centre for reconstructive surgery and rehabilitation, with departments in Plastic Surgery, Oral and Maxillofacial Surgery and Ophthalmology. It receives burns, hand and facial trauma referrals from the South East of England region. It has a small critical care unit, a burns unit and surgical wards.

Telemedicine has been used for over 20 years as part of the QVH trauma referral system; Telemedicine Referral Image Portal System (TRIPS). QVH was, therefore, streamlined towards a ‘virtual’ way of delivering care when the COVID-19 pandemic hit. It was initiated after a child was unnecessarily helicoptered to the QVH Burns Unit following an inaccurate referral. With TRIPS QVH developed a safe simple encrypted web-based system for the transfer of images; referring units sent photographs of injuries with basic details rather than just a telephone call.

It is well reported that telemedicine can improve patient care through cost savings and increased time efficiency., At QVH, TRIPS allows virtual ward rounds, MDTs, and remote assessment. It avoids unnecessary information in referrals and minimises data shared. At QVH, there was improved access for patients and earlier and more frequent senior clinician involvement. In addition, it created a central repository of referrals as part of the medical record and allowed an audit trail and teaching cases for virtual case management for trainees.

Telemedicine, including video conferencing, is not a new phenomenon. It is used where patient access is challenging because of patient circumstances or local geography. This has been facilitated by a jump in mobile telephone use over the last four decades with smartphone ownership in 80% of the population in developed countries and 82% in developing countries by 2017. Previous studies have shown that injury assessment is similar regardless of whether digital imaging or face-to-face consultation is used.

Establishment of virtual clinic

During the first UK COVID-19 lockdown, the virtual clinic was set up to triage hand trauma referrals. The team members were the pre-existing trauma coordinators, plastic surgery core trainees, specialty trainees, consultants, and pharmacists. Compared to the face-to-face clinics, a consultant was present throughout. In addition, the same small team worked solely on the virtual clinic rather than frequent the rotation of junior doctors seen on the face-to-face clinics. The trauma coordinator received referrals and booked patients in for the same day or next day virtual assessment depending on availability. Any urgent referrals were discussed and transferred for admission and treatment as appropriate. Patients were contacted by the virtual team via telephone or video call with access to the TRIPS photograph of the injury sent by the referring accident and emergency department (A+E). If the referral was triaged while the patient was still in the referring unit's A+E, a call to the referring clinician in A+E was made. Following virtual assessment there were broadly three outcomes: (1) admission to QVH, (2) day case procedure and (3) advice and discharge with virtual follow-up if necessary. A similar system existed prior to the COVID-19 pandemic; a trauma co-ordinator triaged referrals via telephone using accompanying TRIPS photographs to either direct admission, face-to-face trauma clinic or telephone advice only.

We aim to assess the accuracy of assessments made in the new virtual hand trauma clinic and compare them with assessments made in the traditional face-to-face clinic.

Materials and Methods

Evaluation study prior to virtual clinic set-up

In preliminary planning for the virtual clinic, a core trainee year two (CT2) and specialty trainees in years 5, 6 and 7 (ST5, ST6 and ST7) were asked to assess one day of trauma referrals using photographs from the TRIPS system. Each clinician was asked to predict what treatment was required and this was compared with the patient's operation note. The requirement for surgery was scored separately. The structures involved, level of operating surgeon, type of anaesthetic, all had to be predicted for a correct treatment prediction. These results established a baseline for senior clinicians in the clinic and allowed to be built teaching around knowledge gaps.

Assessment of virtual clinic

The accuracy of assessment in the virtual clinic was evaluated. Two weeks of face-to-face hand trauma clinic sessions from January 2020 prior to the first UK national lockdown were analysed with the outcome of assessment compared with the patient's operation note. The same analysis was carried out for two weeks of virtual clinic sessions occurring during the UK's first lockdown enabling comparison between virtual and face-to-face assessments. Injured structures involved all had to be correctly predicted to score as a correct treatment prediction. The chi-square test was used to measure the statistical significance of differences between groups.

Results

To assess the feasibility of a virtual clinic, the accuracy of assessment by trainees was evaluated. ST5, ST6 and ST7 grades had 100% success rate in predicting whether a patient needed an operation. CT2 grade had an 86% success rate. The specialty trainees more reliably predicted the structures injured and treatment required. The core trainee with the least experience in plastic surgery was least reliable, but still correctly predicted treatment in 71% of cases. Overall, 82% of treatments were correctly predicted.

Accuracy of assessments made in the virtual clinics: comparison with face-to-face clinics

One hundred and eighty-nine patients were seen in virtual clinics over a 2-week period during the first UK lockdown. Ninety patients were assessed by video call, and 72 via telephone. Twelve were contacted by phone, while in an A+E, four patients were seen face-to-face, and five were un-contactable. One hundred and twenty-nine face-to-face consultations in two weeks prior to the first national lockdown were similarly analysed of which nine did not attend the clinic or their notes were unavailable (Table 1).

Table 1

Type of assessment.

	Face-to-face clinic	Virtual clinic
Video		90
Telephone to patient		72
Call to A+E		12
Call to inpatient teams		4
Face-to-face		6
Unable to contact		5
Face-to-face	120
Cancelled on day	1
Did not attend	8
Total	129	189

We analysed whether consultation resulted in a surgical outcome or conservative treatment. A significantly lower (p=0.02) proportion (68%) of patients assessed in the virtual clinics were treated surgically compared to those assessed in the face-to-face clinics (82%) (Table 2).

Table 2

Surgical outcome from virtual and face-to-face assessments.

	Face-to-face clinic	Virtual clinic
Surgical	99	126
Non-surgical	21	58
DNA/notes not available	9	5

Of 126 patients that were assessed virtually who required an operation 109 (87%) had an accurate assessment, all structures injured were correctly predicted. Twelve were overestimated, and five had their injury underestimated. Despite this, no patient underwent an unnecessary procedure. Two patients initially managed with conservative management by the virtual clinic subsequently went on to have surgery. In three cases, the patient was booked for surgery by the virtual clinic, but the patient cancelled their surgery. Patients assessed virtually were either contacted by video or by telephone with supporting photographs. There was no significant difference between the accuracy of assessment by video or telephone (p=0.88).

In the face-to-face group, of 99 patients that required surgery 77 (78%) had an accurate assessment. Six were overestimated, and 13 were underestimated. Again, regardless of whether a patient's injury was over or underestimated, it did not result in an unnecessary operation. In one case of underestimation, there was a missed dislocation that required a return to theatre otherwise under/overestimates did not alter management.

There was no significant difference in the overall accuracy of assessments done in the pre-COVID-19 face-to-face clinics, and the virtual assessments done during the first UK lockdown (p=0.27) (Table 3).

Table 3

Accuracy of prediction of the extent of surgical procedure in those requiring surgery.

	Face-to-face clinic	Virtual clinic	Virtual video	Virtual telephone+photograph
Correct	77 (79%)	109 (87%)	46	50
Overestimate	6 (6%)	12 (10%)	8	4
Underestimate	13 (13%)	5 (4%)	1	4
Surgery cancelled on day	3

Clinicians from core trainees to consultants conducted both the virtual and face-to-face clinics; however, the majority of face-to-face assessments (61%) were performed by speciality trainees, and they performed more virtual assessments (42%) than any other group. The accuracy of the assessments of the various grades is illustrated in Figure 1. As expected, assessments by consultants were the most accurate. The overall accuracy (% correct) for the trainee grades was very similar; 80% in the face-to-face and 78% for the virtual clinics, but this dropped to 72% for core trainees in the virtual clinics. Accuracy was significantly associated with experience for the virtual assessment clinics (p=0.04), but not the face-to-face clinics possibly reflecting the difficulty of assessing hand injuries virtually and the amount of training and experience required to make an accurate assessment.

Figure 1

Proportion of correctly predicted treatments by training grade in face to face and virtual clinics.

Discussion

Telemedicine has been established at QVH for many years to enable the management of trauma referrals. The COVID-19 pandemic required UK hospitals to alter the management of patient flow, including switching outpatient clinics to telephone and video consultations as they faced social distancing measures and restrictions.

Our experience is unique as there was an existing telemedicine system in place that has been utilised during the pandemic and found to be extremely valuable. The set-up of the virtual hand trauma clinic allowed us to further explore the feasibility of assessment with video conferencing.

Our experience has shown that virtual assessment works well but that video calls are not essential when the alternative of a telephone call plus photographs is available. We discuss several potential pitfalls with video conferencing that we encountered, plus how staffing of the clinics may have contributed to our findings. Pitfalls included video conferencing not being suitable for all types of examination and are less useful if patients have dressings over their wounds that are difficult for patients to remove, bandwidth requirements, and the expectation that patients have internet and smartphone access can be problematic.

Video calls can be an inadequate means of assessment for certain areas of the body, e.g., intraoral or intimate areas, e.g., breast examination. However they are a reasonable way of assessing hand trauma as appearance and hand movements can be assessed. One obvious disadvantage of video conference calls compared to face to face examination is that the clinician cannot feel or palpate – an essential part of hand examination – nor meet the patient to get a feel for the ‘end of the bed examination’ and their general well being.

Hand trauma patients invariably had their wounds dressed at their initial visit to their local A+E. It was difficult for patients to take their dressings down and re-do them. Patients, therefore, had an assessment without the injury being visualised directly, and clinicians relied on the accompanying TRIPS photographs and assessing movement where possible. In comparison to face-to-face clinics, the injury was always seen and directly examined. Therefore, in these scenarios, a video consultation contributed less to the accuracy of the diagnosis. An ideal situation would be if the patient could be assessed virtually via video when the referring A+E is assessing the wound prior to dressing. Unfortunately, the current NHS system rarely allows for this flexibility. That said, our results show that video assessment does not improve accuracy and therefore may be unnecessary. One area where the video call worked very well was in reviewing elderly and frail patients and those who were shielding. We were able to liaise with carers and community nurses to be present at the call to aid in dressing changes and help with the logistics of setting up the call.

Technical issues were encountered in conducting video consultations. A huge amount of bandwidth is required to run multiple video calls from a clinic at once. Poor bandwidth gives a pixelated poor-quality picture making assessment impossible. In addition, the use of video technology excludes those who do not have internet or computer or smartphone access. Staffing differences between the virtual and face-to-face clinics likely contributed to the increased accuracy seen in virtual assessment. One key difference between the face-to-face clinics and the virtual clinics was that a consultant was always present on the virtual clinic to see patients and supervise other clinicians. In addition, the same small team was working solely on the virtual clinic rather than a frequent rotation of staff on the face-to-face clinics. This allowed for greater scrutiny of referrals by a dedicated team who had no other clinical duties to juggle such as ward work or on-call work. In addition, we believe there was a different expectation from the different clinic set-ups. In the face-to-face clinic, the main focus was to triage patients into whether the patient needed surgery or not. We postulate there was less focus on the exact detail of the injury, e.g., correctly assessing which tendons were injured. Although this lowered the accuracy in our evaluation and did not adversely affect patient outcome, patients who required surgery still had surgery. In the virtual clinic, clinicians tried to predict the exact nature of the injury to make an appropriate plan to ultimately make sure the patient pathway was correct, and patient journeys to the hospital were minimised in line with COVID-19 restrictions.

Also of note is the difference between the number of patients treated with surgery after an initial virtual (68%) or face-to-face (82%) assessment. This may be explained by the fact that patients may have been pre-screened in arranging for a face-to-face appointment by our trauma coordinator meaning that patients who obviously did not need surgery were not brought to the hospital. This highlights the benefits of our previous working practices in establishing the virtual clinic. It can be further explained by the risk-benefit consideration made by the assessing clinicians during the first wave for vulnerable patients between having surgery versus the risk of a hospital visit and potential exposure to COVID-19.

Going forward, QVH has been able to build on its experience of the virtual clinics. A new model at QVH post-first wave and lockdown are a modified, but similar version of that used pre-COVID-19, and highlights that we were already working effectively with a virtual set-up using telephone triage with the TRIPS system. This allowed the quick adaptation of pre-existing systems to expand the hand trauma triage service to using video conferencing. Given there is no benefit in terms of accuracy of assessment, we have now shown that a video consultation is not required if photographs of the injury are available; only telephone access is necessary. Having only telephone plus photograph triage is a simpler and accessible system and one that we continue to use predominantly run by our trauma coordinators.

One limitation of this study was that patients who were discharged from both the virtual and face-to-face clinics were not followed up, making us unable to know whether these patients were managed correctly.

Conclusion

We have discussed the results and challenges of the set-up of a virtual clinic introducing video conferencing to our practice to assess hand trauma in a fluctuating and evolving clinical environment due to the COVID-19 pandemic. Crucially, the virtual clinic successfully allowed patients to be remotely assessed. This allowed reduced footfall to our hospital meaning reduced unnecessary journeys and reduced risk of exposure to COVID-19. We found, as expected, senior trainees were better able to assess injuries and predict correct management but also that with training all grades have good accuracy. We have shown that virtual clinic assessments are accurate, and no patient underwent an unnecessary procedure as a result of a virtual assessment. We found using a telephone call plus photographs gave similar accuracy as a video call and therefore suggest video assessment although useful is not essential..

Although the COVID-19 pandemic has led to changes in practice, it is uncertain how many of these changes will persist once healthcare begins to return to normal. Our data show that hand trauma assessment through virtual assessment is one area that should continue as it provides accurate decision making without the need for face-to-face clinics. It also highlights the importance of embracing technology early and incorporating it into daily practice as we have done for over two decades. We are able to remind surgeons that outpatient appointments and triage can be done with success virtually with no detriment to patient care and that video conferencing is not essential when telephone and photographs are available.