Jenny McCleery1, Julian Laverty2, Terry J Quinn3. 1. Oxford Health NHS Foundation Trust, Banbury, UK. 2. Oxford Health NHS Foundation Trust, Oxford, UK. 3. Institute of Cardiovascular and Medical Sciences, University of Glasgow, Glasgow, UK.
Abstract
BACKGROUND: Many millions of people living with dementia around the world are not diagnosed, which has a negative impact both on their access to care and treatment and on rational service planning. Telehealth - the use of information and communication technology (ICT) to provide health services at a distance - may be a way to increase access to specialist assessment for people with suspected dementia, especially those living in remote or rural areas. It has also been much used during the COVID-19 pandemic. It is important to know whether diagnoses made using telehealth assessment are as accurate as those made in conventional, face-to-face clinical settings. OBJECTIVES: Primary objective: to assess the diagnostic accuracy of telehealth assessment for dementia and mild cognitive impairment. Secondary objectives: to identify the quality and quantity of the relevant research evidence; to identify sources of heterogeneity in the test accuracy data; to identify and synthesise any data on patient or clinician satisfaction, resource use, costs or feasibility of the telehealth assessment models in the included studies. SEARCH METHODS: We searched multiple databases and clinical trial registers on 4 November 2020 for published and 'grey' literature and registered trials. We applied no search filters and no language restrictions. We screened the retrieved citations in duplicate and assessed in duplicate the full texts of papers considered potentially relevant. SELECTION CRITERIA: We included in the review cross-sectional studies with 10 or more participants who had been referred to a specialist service for assessment of a suspected cognitive disorder. Within a period of one month or less, each participant had to undergo two clinical assessments designed to diagnose dementia or mild cognitive impairment (MCI): a telehealth assessment (the index test) and a conventional face-to-face assessment (the reference standard). The telehealth assessment could be informed by some data collected face-to-face, e.g. by nurses working in primary care, but all contact between the patient and the specialist clinician responsible for synthesising the information and making the diagnosis had to take place remotely using ICT. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted data from included studies. Data extracted covered study design, setting, participants, details of index test and reference standard, and results in the form of numbers of participants given diagnoses of dementia or MCI. Data were also sought on dementia subtype diagnoses and on quantitative measures of patient or clinician satisfaction, resource use, costs and feasibility. We assessed risk of bias and applicability of each included study using QUADAS-2. We entered the results into 2x2 tables in order to calculate the sensitivity and specificity of telehealth assessment for the diagnosis of all-cause dementia, MCI, and any cognitive syndrome (combining dementia and MCI). We presented the results of included studies narratively because there were too few studies to derive summary estimates of sensitivity and specificity. MAIN RESULTS: Three studies with 136 participants were eligible for inclusion. Two studies (20 and 100 participants) took place in community settings in Australia and one study (16 participants) was conducted in veterans' homes in the USA. Participants were referred from primary care with undiagnosed cognitive symptoms or were identified as being at high risk of having dementia on a screening test in the care homes. Dementia and MCI were target conditions in the larger study; the other studies targeted dementia diagnosis only. Only one small study used a 'pure' telehealth model, i.e. not involving any elements of face-to-face assessment. The studies were generally well-conducted. We considered two studies to be at high risk of incorporation bias because a substantial amount of information collected face-to-face by nurses was used to inform both index test and reference standard assessments. One study was at unclear risk of selection bias. For the diagnosis of all-cause dementia, sensitivity of telehealth assessment ranged from 0.80 to 1.00 and specificity from 0.80 to 1.00. We considered this to be very low-certainty evidence due to imprecision, inconsistency between studies and risk of bias. For the diagnosis of MCI, data were available from only one study (100 participants) giving a sensitivity of 0.71 (95% CI 0.54 to 0.84) and a specificity of 0.73 (95% CI 0.60 to 0.84). We considered this to be low-certainty evidence due to imprecision and risk of bias. For diagnosis of any cognitive syndrome (dementia or MCI), data from the same study gave a sensitivity of 0.97 (95% CI 0.91 to 0.99) and a specificity of 0.22 (95% CI 0.03 to 0.60). The majority of diagnostic disagreements concerned the distinction between MCI and dementia, occurring approximately equally in either direction. There was also a tendency for patients identified as cognitively healthy at face-to-face assessment to be diagnosed with MCI at telehealth assessment (but numbers were small). There were insufficient data to make any assessment of the accuracy of dementia subtype diagnosis. One study provided a small amount of data indicating a good level of clinician and especially patient satisfaction with the telehealth model. There were no data on resource use, costs or feasibility. AUTHORS' CONCLUSIONS: We found only very few eligible studies with a small number of participants. An important difference between the studies providing data for the analyses was whether the target condition was dementia only (two studies) or dementia and MCI (one study). The data suggest that telehealth assessment may be highly sensitive and specific for the diagnosis of all-cause dementia when assessed against a reference standard of conventional face-to-face assessment, but the estimates are imprecise due to small sample sizes and between-study heterogeneity, and may apply mainly to telehealth models which incorporate a considerable amount of face-to-face contact with healthcare professionals other than the doctor responsible for making the diagnosis. For the diagnosis of MCI by telehealth assessment, best estimates of both sensitivity and specificity were somewhat lower, but were based on a single study. Errors occurred at the cognitively healthy/MCI and the MCI/dementia boundaries. However, there is no evidence that diagnostic disagreements were more frequent than would be expected due to the known variation between clinicians' opinions when assigning a dementia diagnosis.
BACKGROUND: Many millions of people living with dementia around the world are not diagnosed, which has a negative impact both on their access to care and treatment and on rational service planning. Telehealth - the use of information and communication technology (ICT) to provide health services at a distance - may be a way to increase access to specialist assessment for people with suspected dementia, especially those living in remote or rural areas. It has also been much used during the COVID-19 pandemic. It is important to know whether diagnoses made using telehealth assessment are as accurate as those made in conventional, face-to-face clinical settings. OBJECTIVES: Primary objective: to assess the diagnostic accuracy of telehealth assessment for dementia and mild cognitive impairment. Secondary objectives: to identify the quality and quantity of the relevant research evidence; to identify sources of heterogeneity in the test accuracy data; to identify and synthesise any data on patient or clinician satisfaction, resource use, costs or feasibility of the telehealth assessment models in the included studies. SEARCH METHODS: We searched multiple databases and clinical trial registers on 4 November 2020 for published and 'grey' literature and registered trials. We applied no search filters and no language restrictions. We screened the retrieved citations in duplicate and assessed in duplicate the full texts of papers considered potentially relevant. SELECTION CRITERIA: We included in the review cross-sectional studies with 10 or more participants who had been referred to a specialist service for assessment of a suspected cognitive disorder. Within a period of one month or less, each participant had to undergo two clinical assessments designed to diagnose dementia or mild cognitive impairment (MCI): a telehealth assessment (the index test) and a conventional face-to-face assessment (the reference standard). The telehealth assessment could be informed by some data collected face-to-face, e.g. by nurses working in primary care, but all contact between the patient and the specialist clinician responsible for synthesising the information and making the diagnosis had to take place remotely using ICT. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted data from included studies. Data extracted covered study design, setting, participants, details of index test and reference standard, and results in the form of numbers of participants given diagnoses of dementia or MCI. Data were also sought on dementia subtype diagnoses and on quantitative measures of patient or clinician satisfaction, resource use, costs and feasibility. We assessed risk of bias and applicability of each included study using QUADAS-2. We entered the results into 2x2 tables in order to calculate the sensitivity and specificity of telehealth assessment for the diagnosis of all-cause dementia, MCI, and any cognitive syndrome (combining dementia and MCI). We presented the results of included studies narratively because there were too few studies to derive summary estimates of sensitivity and specificity. MAIN RESULTS: Three studies with 136 participants were eligible for inclusion. Two studies (20 and 100 participants) took place in community settings in Australia and one study (16 participants) was conducted in veterans' homes in the USA. Participants were referred from primary care with undiagnosed cognitive symptoms or were identified as being at high risk of having dementia on a screening test in the care homes. Dementia and MCI were target conditions in the larger study; the other studies targeted dementia diagnosis only. Only one small study used a 'pure' telehealth model, i.e. not involving any elements of face-to-face assessment. The studies were generally well-conducted. We considered two studies to be at high risk of incorporation bias because a substantial amount of information collected face-to-face by nurses was used to inform both index test and reference standard assessments. One study was at unclear risk of selection bias. For the diagnosis of all-cause dementia, sensitivity of telehealth assessment ranged from 0.80 to 1.00 and specificity from 0.80 to 1.00. We considered this to be very low-certainty evidence due to imprecision, inconsistency between studies and risk of bias. For the diagnosis of MCI, data were available from only one study (100 participants) giving a sensitivity of 0.71 (95% CI 0.54 to 0.84) and a specificity of 0.73 (95% CI 0.60 to 0.84). We considered this to be low-certainty evidence due to imprecision and risk of bias. For diagnosis of any cognitive syndrome (dementia or MCI), data from the same study gave a sensitivity of 0.97 (95% CI 0.91 to 0.99) and a specificity of 0.22 (95% CI 0.03 to 0.60). The majority of diagnostic disagreements concerned the distinction between MCI and dementia, occurring approximately equally in either direction. There was also a tendency for patients identified as cognitively healthy at face-to-face assessment to be diagnosed with MCI at telehealth assessment (but numbers were small). There were insufficient data to make any assessment of the accuracy of dementia subtype diagnosis. One study provided a small amount of data indicating a good level of clinician and especially patient satisfaction with the telehealth model. There were no data on resource use, costs or feasibility. AUTHORS' CONCLUSIONS: We found only very few eligible studies with a small number of participants. An important difference between the studies providing data for the analyses was whether the target condition was dementia only (two studies) or dementia and MCI (one study). The data suggest that telehealth assessment may be highly sensitive and specific for the diagnosis of all-cause dementia when assessed against a reference standard of conventional face-to-face assessment, but the estimates are imprecise due to small sample sizes and between-study heterogeneity, and may apply mainly to telehealth models which incorporate a considerable amount of face-to-face contact with healthcare professionals other than the doctor responsible for making the diagnosis. For the diagnosis of MCI by telehealth assessment, best estimates of both sensitivity and specificity were somewhat lower, but were based on a single study. Errors occurred at the cognitively healthy/MCI and the MCI/dementia boundaries. However, there is no evidence that diagnostic disagreements were more frequent than would be expected due to the known variation between clinicians' opinions when assigning a dementia diagnosis.
Authors: Suzanne C Freeman; Clareece R Kerby; Amit Patel; Nicola J Cooper; Terry Quinn; Alex J Sutton Journal: BMC Med Res Methodol Date: 2019-04-18 Impact factor: 4.615
Authors: W Annefloor van Enst; Eleanor Ochodo; Rob J P M Scholten; Lotty Hooft; Mariska M Leeflang Journal: BMC Med Res Methodol Date: 2014-05-23 Impact factor: 4.615
Authors: Lucy C Beishon; Emma Elliott; Tuuli M Hietamies; Riona Mc Ardle; Aoife O'Mahony; Amy R Elliott; Terry J Quinn Journal: Cochrane Database Syst Rev Date: 2022-04-08