Evaluation of telemedicine for new outpatient neurological consultations.

Objective: The COVID-19 pandemic has broadened the use of teleneurology, how this compares with face-to-face (F2F) clinics is unclear. This study compared virtual with F2F new neurological consultations.
Methods: We retrospectively evaluated new outpatient consultations in neurology clinics in Aberdeen Royal Infirmary. We compared sociodemographic data, time to consultation, time to diagnosis, the need for reassessment and re-investigation between traditional F2F and virtual clinics using the web-based Attend Anywhere platform or telephone into patients' own homes (or chosen location) without a trained assistant. We calculated the relative risk (RR) of the need for reassessment and re-investigation over 6-month periods by the suspected neurological diagnosis.
Results: 73% of consultations were virtual (Attend Anywhere or telephone) between June and October 2020, this was almost non-existent (<0.1%) in June-October 2019. We analysed 352 F2F (June-July 2019) and 225 virtual consultations (June-July 2020). Compared with F2F clinics, virtual clinics had a longer time to diagnosis (p=0.019), were more likely to be reassessed (RR: 2.2, 95% CI: 1.5 to 3.2; p<0.0001) and re-investigated (RR: 1.50, 95% CI: 0.88 to 2.54; p=0.133), this was likelier in those aged ≥60 years. Patients with headaches and suspected seizures were less likely to need reassessment or re-investigation following virtual clinics than multiple sclerosis and neuroinflammatory disorders, spinal cord disorders and functional neurological disorders.
Conclusion: This study demonstrates that virtual clinics have higher rates of reassessment and re-investigation than F2F clinics. As virtual clinics become a potential consultation alternative, this study should instruct the selection of patients for either consultation type. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

There are concerns that neurology outpatient assessments using virtual clinics have lower diagnostic accuracy compared with face-to-face clinics.

This retrospective study showed that virtual clinics had higher rates of reassessment and re-investigation and longer time to diagnosis. Virtual clinics can be applied better for headache and seizure consultations compared with other neurological diagnoses.

These differences should influence the choice of consultation. Larger prospective studies are needed to understand the acceptability, long-term outcomes and the wider economic and environmental significance of virtual clinics.

Introduction

Telemedicine is a form of medical practice using communication technology, in which the medical provider and the patient are in different locations.1 The use of telemedicine in neurology has gained interest over the years,2–7 because of its potential to improve access to specialist neurological care, reduce the need for patients to travel and reduce patients’ waiting time.2 8 9 Switching to remote consultation during the COVID-19 pandemic ensured continued care of neurological patients while limiting physical contact.10 This was a required change based on the government’s demand for neurologists and their patients in a relatively short time from what was an established standard of neurological care.11

This urgent contingency plan of using virtual clinics during the pandemic had to be made without consideration of patient satisfaction, diagnostic capabilities and the quality of care compared with the traditional face-to-face (F2F) consultations. Despite the growing evidence that teleneurology is feasible, beneficial, acceptable and cost-effective,12 13 its effectiveness across different neurology presentations has not been assessed.

The main concern with teleneurology is the lack of full objective neurological examination, which may affect the diagnostic accuracy and therapeutic recommendations.14 15 This is particularly so for the National Health Service (NHS) Attend Anywhere platform without a trained assistant. Although there has been some evaluation of video conferencing with far-end assistance, this may be different from Attend Anywhere.16 Other problems envisaged include difficulties discussing sensitive issues,17 ethical concerns, data security and privacy.18 The lack of studies and information on remote consultations could have had negative consequences on the quality of patient care and physician satisfaction. With the COVID-19 pandemic unlikely to end soon, teleneurology is likely to be an essential tool and even beyond the pandemic.7 19 20 This work evaluated the use of virtual consultations using Attend Anywhere and telephone compared with traditional F2F consultations for new neurological consultations with regard to the rates of reassessment and re-investigation. The main question this work aimed to answer is whether virtual consultations using Attend Anywhere and telephone without a trained assistant are as good as the traditional F2F consultations in assessing neurology outpatient.

Methodology

Study design and participants

We retrospectively evaluated virtual clinics (Attend Anywhere and telephone) against F2F clinics for a new neurological consultation. The subjects of interest were new patients attending the neurology outpatient department aged 16 years and above. Participants were assessed in the neurology department of Aberdeen Royal Infirmary, a tertiary hospital covering Grampian, Shetland and Orkney Islands. The average pre-COVID-19 weekly clinic attendance was 48.3 patients. During the COVID-19 pandemic, most neurology consultations became virtual, with an evolving mixture of the traditional F2F and virtual clinics. The video clinic in NHS Grampian is branded ‘Near Me’, this uses the Attend Anywhere software for transmission. Patients are sent a link, which they can use to join using a desktop computer, laptop, tablet or smartphone.7 The patient connects in from their own homes or chosen location without a trained assistant to help with the examination. Attend Anywhere was the default modality with the telephone used where the patient did not have access to suitable technology, where the Attend Anywhere connection failed or the patient requested a telephone consultation. Before the pandemic, a small number of virtual clinics were undertaken using video conferencing with a far-end assistant and none were undertaken using Attend Anywhere. Therefore, NHS Grampian may have been more able to adopt virtual consultations than other centres.7

Data collection methods

Data on new neurology consultations were collected from electronic health records. The number of patients seen during the periods June–October 2019 and 2020 was retrieved to assess the trends in the use of virtual clinics. Detailed information on the subjects was collected consecutively for virtual clinics during the COVID-19 period (June–July 2020) and F2F clinics during the pre-COVID-19 (June–July 2019) period. The period June 2020 onwards was chosen because the period April and May was a learning period giving time to get used to the technology and become competent in its use.

We acquired information on sociodemographic data such as age, gender, referral source and the referral priority (urgent or routine). We further retrieved detailed information on the time to consultation, time to a diagnosis, the type of diagnosis, number of investigation requests, the need to be reassessed or re-investigated, and whether treatment was offered, over 6-month periods. We defined reassessment as the need to be seen again F2F because of unsatisfactory or insufficient information and/or the need to perform a neurological examination to reach a diagnosis. Re-investigation was defined as the need to have further investigation(s) during or following a reassessment to reach a diagnosis. We did not include those who needed follow-up to assess disease progression or for continued care. The neurological diagnoses reached by the specialists were classified based on a modification of a previous study on neurology consultation patterns.21 All follow-up patients were excluded.

Statistical analyses

Data were entered and cleaned in Excel sheets (2016) and analysed using Stata V.15 (StataCorp 2017, Stata Statistical Software: Release 15; College Station, Texas, USA: StataCorp). We compared baseline characteristics between subjects seen as F2F and virtual clinics using mainly descriptive statistics and graphical methods. The mean±SD or for skewed data the median and IQR were used for summarising continuous variables and compared using the Student’s t-test or the Wilcoxon rank-sum test, as appropriate. A χ2 test or Fisher’s exact was used to compare categorical variables. We calculated the risk of reassessment and re-investigation by clinic type and diagnosis in a univariate analysis reporting the relative risk (RR) and their corresponding 95% confidence. As there were few missing data, our analyses were conducted without multiple imputations and reported where applicable. We assessed the confounding effect of age using a Mantel-Haenszel equation where applicable. A p value of <0.05 was considered statistically significant.

Results

Figure 1 shows the clinic attendance and recruitment process between the periods June and October 2019 (pre-COVID-19) and 2020 (COVID-19 period). A total of 1110 patients attended the neurology outpatient clinics in the 5 months in 2019, this was down by 24% in the same period in 2020, with no significant age or gender difference (online supplemental table 1). There was no statistically significant difference in the number of patients referred, but not seen in the clinic over these periods (p=0.251). In June 2020, due to the COVID-19 pandemic, 93% were virtual clinics and by September and October 2020, approximately three-fifths were virtual (figure 2). Overall, 618 (73%) patients in the 5 months of 2020 were consulted through virtual clinics and a small number predominantly covered Orkney and Shetland (<0.1%—video conferencing with far-end assistant) in 2019. Online supplemental figure 1 shows the pattern of utilisation of virtual clinics over 13 months. We had 395 and 272 subjects during the months June and July of 2019 and 2020 but excluded 11% from both years because they were not new cases.

Figure 1

Neurology clinic attendance and the recruitment process.

Figure 2

Distribution of virtual and non-virtual clinics in 2020.

Table 1 shows the differences between those recruited in 2019 and 2020. There was no age difference between the 2 years. More females attended the clinic in both years, but less in 2020 (p=0.071). The median days to the consultation were significantly less in 2020 (10%; p=0.025), with more urgent clinic requests in 2020 (9%; p=0.015). Most referrals were from the general practitioners (GPs) (~80%) and within the Grampian Health Board (~95%). Figure 3 shows the age distribution of those recruited.

Table 1

Number recruited in June and July 2019 and 2020

	2019	2020	P value
Recruited	352 (89.1)	242 (89.0)	0.954
Not recruited*	43 (10.9)	30 (11.0)
Age
Mean	50.4±17.7	49.5±15.8	0.533
Range	16–94	16–88
Median (IQR)	50 (37–64)	49 (38–61)
Gender
Female	228 (64.8)	139 (57.4)	0.071
Male	124 (35.2)	103 (42.6)
Health Board
Grampian	340 (96.6)	226 (93.4)	0.070
Others†	12 (3.4)	16 (6.6)
Referral source
GP	298 (84.7)	196 (81.0)	0.495
Consultants at this Health Board	52 (14.7)	44 (18.2)
Other health professionals	2 (0.6)	2 (0.8)
Time to consultation (days)‡
Mean±SD	102.9±51.9	88.1±70.7
Median (IQR)	120 (51–136)	108 (11–153)	0.025
Range	1–336	1–223
Referral priority
Routine	278 (79.0)	170 (70.3)	0.015
Urgent	74 (21.0)	72 (29.7)
Clinic type
Non-virtual	352 (100.0)	17 (7.0)	<0.001
Virtual	0 (0.0)	225 (93.0)

*Some were returning patients, attended Botox or MS return clinics.

†Fife, Shetland, Orkney, Forth Valley, Highland, Lanarkshire and Tayside.

‡Six missing data in 2019 and one missing in 2019.

GP, general practitioner; MS, multiple sclerosis.

Figure 3

Age distribution of subjects recruited by year.

The mirror bar chart (figure 4) illustrates the pattern of diagnoses between the 2 years (F2F vs virtual). Over the two periods, about one-third of diagnoses were headaches and facial pain, followed by functional neurological disorders (FNDs) (15.4%), neuromuscular and cranial neuropathies (8.5%) and epileptic seizures (8.3%) (online supplemental figure 2). There were no significant differences between the various diagnoses and clinic types.

Figure 4

Pattern of diagnoses between the 2 years in percentage (non-virtual 2019 vs virtual 2020).

Of those recruited in June and July 2020 only, 17 (7%) were F2F and were excluded in the analysis comparing the F2F cases in 2019 and virtual cases in 2020 (details shown in online supplemental table 2). Of the 225 virtual cases, the majority were Near-Me clinics, while 18 (8%) were telephone consultations. In general, there were no significant gender or median age differences between the two groups. Comparing the age groups, there was about a 13% increase in the age group 40–59 years and a 9% decrease in the age group ≥60 years in the virtual group compared with the F2F group (p=0.027). There was a non-significant lower median time to the consultations in the virtual group (p=0.152). The virtual clinics had more urgent cases (p=0.042).

The majority (~80%) of patients had a diagnosis in less than a week of consultation. The time to diagnosis was longer in the virtual clinics compared with the F2F clinics (p=0.019). Virtual clinic patients were twice as likely (RR: 2.2, 95% CI: 1.5 to 3.2; p<0.0001) to require reassessment compared with F2F clinics. After adjustment for the confounding effect of age, the RR differed in the various age groups, the risk of reassessment was 1.14 (95% CI: 0.11 to 11.59) in those less than 25 years and was 2.54 (95% CI: 1.31 to 4.94) in those aged ≥60 years (table 2). However, there was no statistically significant evidence that the apparent higher likelihood of reassessment was not due to the confounding effect of age (Mantel-Haenszel combined RR: 2.09, 95% CI: 1.44 to 3.05; p=0.880). There was no significant difference in the initial number of investigations requested between the F2F and the virtual clinics. Those who attended virtual clinics, however, were 50% more likely to require re-investigation, but this was not significant with a wide 95% CI (RR: 1.50, 95% CI: 0.88 to 2.54; p=0.133) (online supplemental table 2). Similarly, there is a higher RR for re-investigation for those above 60 years, but this did not meet statistical significance according to more refined age groups (Mantel-Haenszel combined RR: 1.42, 95% CI: 0.84 to 2.40; p=0.577) (table 2).

Table 2

The relative risk of reassessment and re-investigation comparing virtual against F2F clinics adjusting for the confounding effect of age

Age group	Relative risk (95% CI) and p value
	Reassessment	Re-investigation
16–24 years	1.14 (0.11 to 11.59)	–
25–39 years	2.04 (0.82 to 5.10)	0.55 (0.06 to 5.15)
40–59 years	1.94 (1.13 to 3.35)	1.35 (0.66 to 2.73)
60 years	2.54 (1.31 to 4.94)	1.86 (0.80 to 4.33)
Crude	2.17 (1.49 to 3.16)	1.50 (0.88 to 2.54)
Mantel-Haenszel combined	2.09 (1.44 to 3.05)	1.42 (0.84 to 2.40)
P value	0.880	0.577

The relative difference as a measure of confounding (percentage difference) is 3.43% for reassessment and 5.21% for re-investigation. This is less likely to suggest age as confounding.

F2F, face-to-face.

Online supplemental table 3 details the differences between those who required reassessment or not and associated factors in a univariate analysis. From the 2 years, 16.1% needed to be reassessed. There was no age or gender difference in reassessment. The risk of reassessments was two times higher among virtual clinics (RR: 2.17, 95% CI: 1.49 to 3.16; p<0.001). Those who required reassessment were almost nine times more likely to require re-investigation (RR: 8.61, 95% CI: 6.32 to 11.74; p<0.0001). Online supplemental table 4 shows the details of the factors associated with re-investigation. Those who required re-investigation were significantly older (p=0.006), but no significant gender difference. The likelihood of re-investigation was 50% higher for virtual clinic assessments (RR: 1.49, 95% CI: 0.88 to 2.54; p=0.133) and more than twice as high for telephone consultations (RR: 2.35, 95% CI: 0.91 to 6.10; p=0.088).

Table 3 depicts the risk of reassessment and re-investigation by the type of neurological diagnosis showed that the risk of reassessment was negative for two conditions. Those with headaches and facial pains and suspected seizures were about 80% (RR: 0.22, 95% CI: 0.11 to 0.43; p<0.0001) and 60% (RR: 0.37, 95% CI: 0.12 to 1.12; p=0.052), respectively, less likely to require reassessment. The likelihood for reassessment was more than twice as high for multiple sclerosis and neuroinflammatory disorders (RR: 2.18, 95% CI: 1.24 to 3.85; p=0.013) and spinal cord and radicular disorders (RR: 2.28, 95% CI: 1.36 to 3.84; p=0.004), 75% more for non-specific neurological disorder (RR: 1.75, 95% CI: 1.04 to 2.96; p=0.045) and about 40% more for FND (RR=1.41, 95% CI: 0.90 to 2.21; p=0.145). The risk for re-investigation was 87% less likely in those with headaches and facial pain (RR: 0.13, 95% CI: 0.04 to 0.41; p<0.001) and 78% less likely for epileptic seizures (RR: 0.22, 95% CI: 0.03 to 1.57; p=0.086). The likelihood of re-investigation was almost four times higher for multiple sclerosis and neuroinflammatory disorders (RR: 3.82, 95% CI: 2.00 to 7.32; p<0.001), twice for non-specific neurological disorder (RR: 1.99, 95% CI: 0.95 to 4.16; p=0.071) and about 40% higher (although not significant) for FND (RR: 1.37, 95% CI: 0.71 to 2.63). Compared with virtual clinics, there was no significant difference in the risk of reassessment for telephone consultations (RR: 1.17, 95% CI: 0.54 to 2.57; p=0.696), but more than twice the rate of re-investigation (RR: 2.35, 95% CI: 0.91 to 6.10; p=0.088). There was no difference in whether treatment was offered or not between the two groups.

Table 3

Univariate analysis of the risk of reassessment and re-investigation by neurological diagnosis

	Relative risk (95% CI) and p value
	Reassessment		Re-investigation
Negative likelihood
Headaches and facial pains	0.22 (0.11 to 0.43)	<0.001	0.13 (0.04 to 0.41)	<0.001
Epileptic seizures	0.37 (0.12 to 1.12)	0.052	0.22 (0.03 to 1.57)	0.086
Positive likelihood
Multiple sclerosis and neuroinflammatory disorders	2.18 (1.24 to 3.85)	0.013	3.82 (2.00 to 7.32)	<0.001
Spinal disorders and radiculopathies	2.28 (1.36 to 3.84)	0.004	1.14 (0.38 to 3.47)	0.810
Non-specific neurological symptoms	1.75 (1.04 to 2.96)	0.045	1.99 (0.95 to 4.16)	0.071
Functional neurological disorders	1.41 (0.90 to 2.21)	0.145	1.37 (0.71 to 2.63)	0.353
Neuromuscular and cranial neuropathies	1.45 (0.83 to 2.52)	0.208	1.81 (0.86 to 3.79)	0.123
Vestibular and cerebellar disorder	1.58 (0.66 to 3.76)	0.327	0.71 (0.10 to 4.79)	0.716
Movement disorders	1.27 (0.68 to 2.35)	0.461	1.04 (0.39 to 2.75)	0.941

A further subanalysis of the risk of reassessment of virtual clinics, excluding cases from 2019 (online supplemental table 5), showed that those who needed reassessment were significantly older compared with those who did not (median (IQR): 48 (37–60) years vs 54 (46–63) years; p=0.028). Those who required reassessment were 6.5 times more likely to require re-investigation.

Discussions

Our study shows that since the pandemic, about two-thirds of neurology outpatient appointments in our department were virtual clinics. Compared with non-virtual clinics, virtual clinics had a lower median time to consultation, longer time to diagnosis and increased likelihood for reassessment and re-investigation, more so among older subjects. Headaches and seizure disorders were the only two conditions less likely to be reassessed or re-investigated.

This study showed the rapid change from F2F to virtual clinics as the predominant modality for neurology consultation following the pandemic. The very few virtual clinics in 2019 reported in our study were video conferencing with a far-end assistant as Attend Anywhere was not available then. Video conferencing with a far-end assistant was sparingly used by a few consultants; this clinic could not continue during the pandemic as it requires health personnel at the far-end to perform directed neurological examinations.7 The shorter median time to consultations in the virtual clinics may have been due to the more urgent clinics. There was an edict to focus more on the urgent referrals during the pandemic.

Our study showed that the risk of reassessment and re-investigation was higher among those above 60 years of age. The reasons for this may be due to the presence of comorbidities, difficulty in accepting and using information technology or cognitive decline,22 23 but this may not always be the case and we should not assume that older persons cannot use technology.24 However, this should be considered when triaging older patients for remote consultation. Our finding of the higher risk of reassessment and re-investigation using virtual clinics (Attend Anywhere and telephone) compared with F2F shows that telemedicine may not be as good as F2F, and this is likely due to the absence of a detailed neurological examination. The study by Duncan et al16 showed that virtual clinics with a far-end directed examination by an experienced neurologist fared well compared with F2F. Similarly, studies from Northern Ireland in carefully selected subjects found good diagnostic accuracy for neurological conditions with appropriate management strategies using virtual clinics with far-end directed examination.25 26 A pilot study comparing F2F and telemedicine with a trained assistant for new neurological outpatient referrals in the same centre found no difference in diagnoses between the two modalities with only minor differences between the type and number of investigations requested.27 Our study showed no gender difference in the risk of reassessment or re-investigation comparing virtual and non-virtual clinics.

Importantly, our study assessed the risk of reassessment and re-investigation by the type of neurological diagnoses. We found that headaches and suspected seizures were the only conditions that did not show higher rates of reassessment and re-investigation in virtual clinics. Epilepsy and headache assessment is largely dictated by patient (and eyewitness) histories and so is less reliant on a detailed neurological examination and therefore more suitable and with less risk compared with conditions that may require an examination to reach a diagnosis.28 29 A study assessing the safety and efficacy of virtual consultations for primary headaches found no differences between telemedicine and F2F, with the estimated likelihood of missing one secondary headache being about 1 in 20 000.30 Another study showed that some form of neurological examination by video clinics compared favourably with F2F,31 but it does have its limitations as it may not pick subtle signs and is not able to completely replace F2F. Therefore, the crucial nature of neurological examination should not be underplayed, and patients should be carefully selected looking for red-flag signs. Our seizure referrals routinely have an ECG undertaken in the GP clinic rooms and forwarded to the clinic by the time of assessment. GPs may also refer patients with headache for an optician review if they suspect secondary causes of headaches. We did not assess the percentage of those who had these preliminary investigations in our study and whether this protocol had any benefit on the reassessment rate is unknown, as it is useful from a medicolegal perspective. Directed neurological examination using a trained assistant at the far-end could potentially improve diagnostic certainty when using Attend Anywhere, particularly for those where the travel distance is an issue. Patients with an established diagnosis and on treatment could easily be followed up using Attend Anywhere and telephone.

There are some important study limitations. First, the retrospective nature of data collection and the relatively short period of data collection. Our study method of reviewing case notes may have inaccurately estimated outcome measures. Second, the lack of multivariable analyses for the risk of reassessment and re-investigation is a limitation. Factors such as the presence of comorbidities, gender and the inherent differences between physicians could be confounders in the analysis. Although we controlled for age as a confounder, the Mantel-Haenszel test has its limitation. Third, the small sample size for some diagnoses would have made the analysis unstable. A bigger prospective cohort study controlling for these factors will be useful in the future. Fourth, we did not differentiate analysis between Attend Anywhere and the telephone consultations because of the small sample size of telephone consultations. Lastly, whether some of the differences in reassessment and re-investigation were due to the inherent differences required for the diagnosis of certain conditions or differences between clinicians’ practice and confidence is unknown and was not assessed in this study. Understanding some of the limitations and long-term implications will be important and therefore the need for randomised clinical trials.

Despite the limitations, this work outlines some of the opportunities and challenges to using Attend Anywhere and the telephone as a standard for all neurology consultations. Findings from this work should help inform triaging patients for clinics, establish patients’ stratification and decide on which consultation is best for which patient (table 4). There is a need for further studies assessing patients’ preferences and satisfaction, diagnostic accuracy and long-term outcomes, including mortality. Studies have suggested the positive economic impact of virtual clinics: substantial economic gains and productivity, freeing up additional appointment hours, reduced cost of attending appointments and do-not-attend rates, potentially reducing carbon emissions from reduced travel.32 An analysis estimated that up to half a million kilogram of CO2 emission will be reduced annually through reduced travel when the number of follow-ups is reduced by 15% in the West Midlands of the UK.33 We recommend increased access to and literacy in the use of computers and smartphones for the elderly, also providing support by carers and family.22

Table 4

Recommendation useful in triaging patients for neurology clinic type

	Favours VC	Favours F2F
Elderly patients (>60 years)	×	✓
Headaches	✓	×
Seizures	✓	×
MS and neuroinflammatory disorders	×	✓
FND	×	✓
Spinal disorder	×	✓
Neuromuscular and cranial neuropathies	×	✓
Gender	Unsure	Unsure

F2F, face-to-face; FND, functional neurological disorder; MS, multiple sclerosis; VC, virtual clinic.

Conclusion

This study evidenced that those seen by Attend Anywhere and telephone are more likely to be reassessed and re-investigated compared with F2F, particularly older subjects. Headaches and epileptic disorders are less likely to be reassessed or re-investigated compared with other neurological diagnoses. The findings from this study have implications for triaging patients for virtual assessment based on age and the diagnostic question.