Randomised parallel trial on the effectiveness and cost-effectiveness in screening gait disorder of silent cerebrovascular disease assisted by artificial intelligent system versus clinical doctors (ACCURATE-1): study protocol.

IntroductionSilent cerebrovascular disease (SCD), which is a common disease in the elderly, leads to cognitive decline, gait disorders, depression and urination dysfunction, and increases the risk of cerebrovascular events. Our study aims to compare the accuracy of the diagnosis of SCD-related gait disorders between the intelligent system and the clinician. Our team have developed an intelligent evaluation system for gait. This study will evaluate whether the intelligent system can help doctors make clinical decisions and predictions, which aids the early prevention and treatment of SCD. METHODS AND ANALYSIS: This study is a multi-centred, prospective, randomised and controlled trial.SCD subjects aged 60-85 years in Shanghai and Guizhou will be recruited continuously. All subjects will randomly be divided into a doctor with intelligence assistance group or a doctor group, at a 1:1 ratio. The doctor and intelligent assistant group will accept the intelligent system evaluation. The intelligent system obtains gait parameters by an Red-Green-Blue-depth camera and computer vision algorithm. The doctor group will accept the clinicians' routine treatment procedures. Meanwhile, all subjects will accept the panel's gait assessment and recognition rating scale as the gold standard. The primary outcome is the sensitivity of the intelligent system and clinicians to screen for gait disorders. The secondary outcomes include the healthcare costs and the incremental cost effectiveness ratio of intelligent systems and clinicians to screen for gait disorders. ETHICS AND DISSEMINATION: Approval was granted by the Ethics Committee of Zhongshan Hospital affiliated with Fudan University on 26 November 2019. The approval number is B2019-027(2) R. All subjects will sign an informed consent form before enrolment. Serious adverse events will be reported to the main researchers and ethics committees. The subjects' data will be kept strictly confidential. The results will be disseminated in peer-reviewed journals. TRIAL REGISTRATION NUMBER: NCT04457908. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Independent research and development of the intelligent gait evaluation system.

Compare the accuracy of diagnosing silent cerebrovascular disease-related gait disorders between the intelligent system and clinicians.

Evaluates the effectiveness and cost-effectiveness of the intelligent systems.

Enrol subjects both in economically developed areas and underdeveloped areas.

Follow-up will not be involved.

Introduction

Silent cerebrovascular disease (SCD) is very common in the elderly, and often incidentally found by cranial imaging.1 It presents as a lacunar infarct, white matter hyperintensities (WMH) and microhemorrhages on imaging. However, patients do not have acute symptoms. Reports on the prevalence rate of SCD varies, mainly due to the selection of different sample populations. Furthermore, there is a lack of relevant studies for people under 45 years of age. Approximately 25% of those over 80 years of age have SCD.2 Leary and Saver3 found that more than 11 million people in the USA were newly diagnosed with cerebral infarction or haemorrhage on imaging, but only 770 000 of them had clinical symptoms. SCD lacks the symptoms of an acute neurological impairment. Thus, it is often overlooked by patients and doctors. Nonetheless, it is also associated with chronic neurological impairments. Multiple studies showed that SCD can lead to cognitive decline, gait disorders, depression and urination dysfunction, and increase the risk of future cerebrovascular events.4–6 Debette7 et al assessed the incidence of stroke, dementia and death in 2229 community patients (mean age 62±9 years), and found that SCD patients had increased risk of stroke (HR: 2.84, 95% CI :1.32 to 6.10), and dementia (HR: 6.12, 95% CI :1.82 to 20.54), which were independent risk factors.

Stroke results in high medical costs.Reed8 et al analysed the hospitalisation cost for patients with cerebrovascular events in 137 community hospitals. Patients with SAH had the highest cost ($23 777, n=1124), followed by patients with intracerebral haemorrhage ($10 241, n=3139), ischaemic cerebral infarction ($5837, n=18 740) and transient ischaemic attack ($3350, n=7861). The length of stay was 11.5 days for intracerebral haemorrhage, 7.5 days for intracerebral haemorrhage, 5.9 days for ischaemic cerebral infarction, and 3.4 days for transient ischaemic attacks.

Early detection of subtle neurological impairment in SCD and standardised intervention can help improve patient prognosis and reduce costs. At present, the diagnosis of SCD mainly relies on the imaging and clinical expertise of doctors, which may be subjective and leads to misdiagnosis. Therefore, the use of an intelligent system for early quantitative evaluation of neurological damage in SCD can reduce the evaluation time and differences between individuals. It also consequently helps in the early prevention of SCD and guides diagnosis and treatment while reducing medical costs.

Methods and analysis

Study design

ACCURATE-1 is a multicentre, prospective, superiority, randomised parallel trial (figure 1)

Figure 1

Flow diagram of ACCURATE-1. SCD, silent cerebrovascular disease.

Subjects will be randomly divided into a doctor and intelligent assistant group, and a doctor group at a 1:1 ratio. The doctor and intelligent assistant group will accept the intelligent system evaluation, while the doctor group will accept the clinician’s routine treatment procedures. Meanwhile, all subjects will accept the panel’s gait assessment and cognitive scales as the gold standard.

Setting and timeline

The trial will be conducted in 14 hospitals in Shanghai and Guizhou, including secondary and tertiary hospitals. All staff members of the trial have been trained before the trial started. Recruitment of patients started at 25 September 2019. The trial was halted for more than a year due to the COVID-19 pandemic. Recruitment is ongoing now. The trial is scheduled to end in February 2022.

Participants

In this study, subjects with SCD aged 60–85 years in Shanghai and Guizhou will be recruited continuously. Subjects can refuse to participate or withdraw from the trial at any stage without discrimination or unfair treatment, and their treatment and rights will not be affected. All subjects who agree to attend our trial will sign an informed consent form. After recruitment, eligible subjects will be selected for the study according to the inclusion and exclusion criteria.

The inclusion criteria are as follows:

Aged 60–85 years.

Diagnosed with SCD, according to the 2016 statement issued by the American Heart Association (AHA) and American Stroke Association (ASA):

No clear previous history of stroke.

Cranial MRI shows at least one of the following finding within 1 year and Digital Imaging and Communications in Medicine (DICOM) data should be provided. (1) A lacunar infarct of vascular origin: subcortical round or ovoid fluid-rich lacunar lesion with a diameter of 3–15 mm, showing low central signal and irregular marginal high signal on T2-flair. The central signal is similar to the cerebrospinal fluid. Fazekas scores should be ≥2 points. (2) WMH of vascular origin: high signal on T2-flair in the white matter area (periventricular or subcortical). Fazekas scores should be ≥2 points. (3) Cerebral microbleeds: small, round, empty focus lesion on SWI or T2-weighted image, 2–10 mm in diameter. The number of microbleed lesions should be ≥5.

Conscious and able to complete cognitive assessment.

Able to stand and walk independently and complete gait assessment without assistance.

Sign the informed consent.

The exclusion criteria are as follows:

Intracranial lesions have been clearly diagnosed as a demyelination disease, leukodystrophy, intracranial space-occupying lesions, autoimmune encephalitis, etc.

Previously be diagnosed as Parkinson’s disease, normal pressure hydrocephalus, peripheral neuropathy, osteoarthritis.

Previously be diagnosed as Alzheimer’s disease, frontotemporal dementia, Lewy body dementia, etc.

Severe neurological diseases such as previous cerebral trauma, epilepsy and myelopathy, etc.

Cannot accomplish the cognitive assessment, such as severe visual or hearing impairment.

Cannot finish the gait assessment, such as severe cardiovascular disorder.

Study procedure

Appropriate subjects will be selected based on the inclusion and exclusion criteria. Clinical data will be collected by doctors based on patients’ demographics, medical history, neurological function assessment, laboratory examinations, imaging tests, quality of life, health service utilisation, socioeconomic status and medical and other social costs. The entire data collection process will be recorded only for data verification and monitoring.

Whether the subject’s cranial MRI meets the inclusion criteria will first be determined by trained doctors according to the Standards for Reporting Vascular Changes on Neuroimaging (STRIVE) standard. The committee of experts, including clinical radiology experts and image postprocessing technology personnel, will review the DICOM data. Subjects who do not pass the review will be excluded accordingly.

Subjects in the doctor and intelligent assistant group will be tested for the Time Up and Go9 (TUG) Test evaluated by the intelligent system. The accuracy of this system in screening abnormal gait performance is 90.14%.10 The intelligent system contains an RGB-depth camera, using to record the TUG tests that include walking video, two-dimensional colour images and scene depth images. The gait parameters in TUG test are obtained by computer vision algorithm and the data queue is established. The algorithm can track human motion in the video and identify the main joints in each frame to achieve pose estimation. Then, the previously extracted parameters are taken as input, and a machine learning-based classifier is used to filter abnormal gait.

Mini-Cog test11 will be used to screen subjects’ memory and executive function. Subjects will be asked to remember three unrelated words and immediately repeat these three words. Afterwards, they will be asked to draw a clock with 12 numbers and a pointer to 3:40, then recall the three words. Subjects will retell the sentences of the Mini-Mental State Examination12 (MMSE) and the Montreal Cognitive Assessment13 (MoCA). Subjects will be asked to repeat ‘44 stone lions’, ‘I only know Zhang Liang came to help today’ and ‘The cat always hid under the sofa when the dog was in the room’ in Chinese. The intelligent system will access the subjects’ gait characteristics (get up, turnaround time, stride length, step velocity, stride length, step width, etc.), language features (pronunciation, intonation, word order, wrong language, language fluency, etc.), and clock features (circle, number, pointer).

Subjects in the doctor group will undergo routine medical procedures. There is only one doctor in the doctor group of each centre. The doctor group is required to comprise of attending or resident physicians in neurology and/or attending/resident physicians receiving standardised training in neurology. The physician will register his/her professional qualifications, relevant knowledge training experience, educational background and working years. The physician will determine whether the subjects have gait disorders through routine medical procedures such as their present and previous medical history and physical examination data.

The video of the TUG test of all subjects (including the doctor group and the doctor and intelligent assistant group) will be evaluated by two specialists in movement disorders as the gold standard. Specialists will be blinded to the group allocation. They will classify the subjects’ gait as normal or abnormal. If the results are different, the opinion of the third expert will be included.

All subjects will be evaluated based on the following scales under the guidance of a trained doctor: (1) MMSE:12 evaluates time and place orientation, immediate and delayed memory, attention and computation, naming, retelling, listening comprehension, reading and expression and visual-spatial ability, with scores ranging from 0 to 30; (2) MoCA:13 evaluates visual space, executive function, naming, memory, attention, language, abstraction and orientation, scores ranging from 0 to 30; (3) Colour Word Test14 (CWT): evaluates semantic activation, dominant response inhibition, attention, working memory, information processing speed, etc.; (4) Digit Span Test15 (DST): evaluates immediate memory and attention; (5) Verbal Fluency Test16 (VFT): evaluates language capabilities; (6) TUG:9 evaluates the total time subjects will take to complete it, with the average value obtained after three repetitions; (7) 10- Metre Walk Test17 (10 MWT): the subjects will walk 10 m in a straight line at normal walking speed, while the time and number of steps required for the subject to complete the 10 MWT will be recorded, with the average value obtained after three repetitions; (8) Tinetti Performance-Oriented Mobility Assessment18 (TinettiPOMA): this includes balance and gait tests, with a maximum score of 28. A score between 19 and 24 indicates a risk of falling, while a score below 19 indicates a high risk of falling.19

All subjects will be evaluated using the 5-level version of EuroQol Five Dimensions Questionnaire20(EQ-5D-5L), which describes the health-related quality of life of the subjects.

Resources for health services are limited, especially in remote areas. It is important to evaluate the economics of our smart systems. Cost-effectiveness21 is used to assess how much it costs that intelligent systems and doctors to diagnose each gait disorder. To evaluate the cost-effectiveness from the healthcare system and the societal perspectives, we will collect the data of unit costs and utilisations of the equipment, medications and labour hours taken to deliver each individual diagnosis, as well as the additional cost of patients’ accommodations, transportation and productivity losses due to their disease. The labour hours taken will be collected through a questionnaire for staffs, while the equipment cost (intelligent system) will be amortised over its estimated lifespan. The medication and other patient costs will be collected using a patient questionnaire (table 1).

Table 1

Assessment of two groups

Assessment	Doctor and intelligent assistant	Doctor
Intelligent TUG test	×
Intelligent Mini-cog test	×
Intelligent sentence repetition test	×
Routine treatment procedure		×
Panel’s gait assessment	×	×
TUG	×	×
10MWT	×	×
TinettiPOMA	×	×
MMSE	×	×
MoCA	×	×
CWT	×	×
DST	×	×
VFT	×	×
EQ-5D	×	×
Number of falls	×	×
Utilisation and unit cost	×	×

* indicates that the assessment took place.

CWT, Colour Word Test; DST, Digit Span Test; Mini-Cog, Mini-Cognitive Assessment; MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment; 10 MWT, 10 Metre Walking Test; TinettiPOMA, Tinetti Performance-Oriented Mobility Assessment; TUG, Time Up and Go Test; VFT, Verbal Fluency Test.

Assessments

Outcome measures

The primary outcome is the sensitivity of the intelligent system and clinicians to screen for gait disorders.

The secondary outcomes are as follows: (1) the specificity and the Youden index22 (calculate as sensitivity plus specificity minus one) of the intelligent system and clinicians to screen for gait disorders; (2) the positive and negative predictive values of the intelligent system and clinicians at different levels to screen for gait disorders and (3) healthcare costs of intelligent systems and clinicians to screen for gait disorders, and the incremental cost effectiveness ratio21 will be estimated by cost per additional true case detected using an intelligent system vs clinicians.

Sample size

This study is a 1:1 superiority trial. Referring to the preliminary study of gait disorder in SCD and our group, we expect that the sensitivity of doctors and intelligent assistants will be 85%, while the sensitivity of the doctor group will be 68%. The power is 1−β=80%, with a significance level of α=0.05. According to our calculations, there should be 94 positive cases evaluated by the gold standard in each group. The expected shedding rate is 6%; therefore, each group required 100 positive cases. Considering that the positive rate of gait disorder in the population is approximately 20%, a total of 1000 subjects should be included. NCSS Statistical Software 2021 was used to calculate sample size. (https://www.ncss.com/)

There are 14 subcentres for the two regions in our study, including three secondary and three tertiary hospitals in Shanghai, and four secondary and four tertiary hospitals in Guizhou. The expected ratio of patients in secondary and tertiary hospitals is 1:2; in principle, no less than 30 subjects should be enrolled in each centre and 400 subjects for each region.

Randomisation

Stratified blocked randomisation will be used in this study. Stratification factors included regions (Shanghai and Guizhou), and hospital levels (secondary and tertiary hospitals). All subjects meeting the inclusion criteria are randomly divided into a doctor and intelligent assistant group and a doctor group at a 1:1 ratio through the central randomisation system.

Data analysis

The normality was tested with the Shapiro–Wilk test. Continuous data with a normal distribution are expressed as the mean±SD. Data with non-normal distribution are presented as medians with IQR ranges. A t-test or non-parametric test will be used to compare continuous data. Count data are expressed as frequency (%). For comparison of categorical variables, the χ2 test, Fisher’s exact probability test or Cochran–Mantel–Haenszel test will be used. Subgroup analyses will include region and hospital levels. An intention-to-treat analysis will be applied. Subjects who are randomly assigned to either the intelligent group or the doctor group will be analysed as such, regardless of whether they received intelligent assessment or not. A significant difference was considered to be statistically significant at p < 0.05. Statistical analyses were performed using the SAS V.9.4.

A cost-effectiveness analysis will be conducted from a healthcare system and a societal perspective; all the costs and diagnostic outcomes will be listed separately, then the incremental cost will be calculated per true case additionally detected by using the intelligent system vs the clinicians. We will explore the possibilities of conducting a long-term cost-effectiveness analysis using economic decision modelling based on future cost savings and health gains by using the intelligent system versus clinicians to screen for gain disorder.

Patient and public involvement

Each patient will voluntarily participate in the study and sign the informed consent. Each subcentre will recruit patients according to the inclusion criteria and competed for enrolment. Patients will not involve in the design of this study. Patients do not need to assess the burden of the intervention. The result of this study will be disseminated via peer-reviewed journals.

Discussion

Our study aims to compare the accuracy of the diagnosis of SCD-related gait disorders between the intelligent system and the clinician. Furthermore, we aim to evaluate the effectiveness and equity of intelligent systems to diagnose SCD-related gait disorders compared with clinicians.

Early identification of the characteristic gait of SCD is helpful for clinical diagnosis and treatment. Studies have found that the deterioration of neural gait disorder is often associated with impaired cognitive function, which can serve as a warning sign of dementia. Rosso23 et al reported that after a 14-year-follow-up, gait slowing was associated with cognitive impairment in the elderly population (OR per 0.1 s/y slowing 1.47; 95% CI, 1.04 to 2.07). After 9 years of follow-up, Dumurgier24 et al found that 296 of the 3663 subjects developed dementia, in which a decreased pace was associated with an increased risk of dementia, with a HR value reaching 3.39 for every 0.007 m/s decrease in pace (95% CI: 1.37 to 8.43). Therefore, early quantitative gait analysis will help in the early detection of cognitive impairment. Appropriate interventions are needed to improve patient outcomes and prognoses.

However, the assessment of gait and cognitive function mostly depends on the visual or scale method used by doctors. Due to the lack of a unified evaluation process, the results are relatively random and inconsistent. Therefore, using artificial intelligence to detect gait disorder not only reduces time and labour costs, but also avoids individual evaluation differences. There were some researches based on intelligent gait analysis with wearable devices. Ahad et al25 collected gait data using three sensors placed in a belt and backpack. They analysed 67 solution and found that the best result achieved 24.23% prediction error for gender estimation, and 5.39 mean absolute error for age. Qiu et al26 used inertial sensors to monitor the function of the body’s lower limbs and capture their movements to reconstruct a three-dimensional model. Our intelligent system is easy to operate and has low requirements on hardware and site. Meanwhile, to the best of our knowledge, this is the first study to analyse gait features in SCD based on an intelligent system.

Currently, no studies have explored the effectiveness of SCD screening in reducing adverse health events or cost-effectiveness.1 Although SCD may cause dementia and increase the incidence of stroke, the absolute risk is not high. Therefore, screening requires a low-cost and highly efficient test method. Artificial intelligence is a good choice. We will investigate the human, material and financial costs of physicians and artificial intelligence in different regions when assessing a patient’s neurological function. We hope that our intelligent system can reduce the cost of SCD screening and improve diagnosis in remote areas.