Retinal Imaging Techniques Based on Machine Learning Models in Recognition and Prediction of Mild Cognitive Impairment.

BACKGROUND AND
PURPOSE: Mild Cognitive Impairment (MCI) is thought to be the signal of many progressive diseases but is easily ignored. Therefore, a simple and easy screening method for recognizing and predicting MCI is urgently needed. The study aimed to establish machine learning models of retinal vascular features to categorize and predict MCI. PATIENTS AND METHODS: Subjects enrolled underwent cognitive function assessment and were divided into a normal group, an MCI group, and a dementia group, and fundus photography was performed. MATLAB 2019b was used for fundus image preprocessing and vascular segmentation. Via the Green channel, adaptive histogram equalization (AHE), image binarization, and median filtering, we obtained the original and segmentation retinal vessel images. Afterwards, the histogram of oriented gradient (HOG) was used for image feature extraction. Support vector machine (SVM) and extreme learning machine (ELM) were selected for training models in the fundus original images and fundus vascular segmentation images, respectively. Among the three cognitive groups, sensitivity, specificity, the receiver operating characteristic (ROC) curves, and the area under the curve (AUC) were used to evaluate and compare the predictive performance of the two models in the fundus original and vascular segmentation images, respectively.
RESULTS: A total of 86 eligible subjects were enrolled in the study. After a clinical cognitive assessment, the participants were divided into the normal group (N = 38), the MCI group (N = 26), and the dementia group (N = 22). A total of 332 qualified fundus images were adopted after screening. Comparing the models among the three groups showed that the SVM model had more advantages than the ELM model in the fundus original images and vascular segmentation images. Meanwhile, we found that the original images performed better than the segmentation images in the same prediction model. Among the three groups, the SVM model of the fundus original images had the best performance.
CONCLUSION: The establishment of a predictive model based on vascular-related feature extraction from fundus images has high recognition and prediction abilities for cognitive function and can be used as a screening method for MCI. CLINICAL TRIAL REGISTRATION: ChiCTR.org.cn (ChiCTR1900027404), Registered on Nov 12, 2019.

Introduction

Mild Cognitive Impairment

With the ageing of the population, the number of people with mild cognitive impairment (MCI) or dementia is expected to increase. There are over 55 million people worldwide living with dementia in 2020. This number will almost double every 20 years, reaching 78 million in 2030 and 139 million in 2050.1 Such growth will inevitably lead to a substantial increase in the economic and health burden on societies worldwide.

MCI, which is defined by the Diagnostic and Statistical Manual of Mental Disorders 5th Edition (DSM-V), refers to a clinical condition between normal ageing and dementia. Patients have a subjective and objective decline from a previous level of functioning in one or more of the six cognitive domains but their activities of daily living are not significantly affected and the problems are not severe enough to meet the diagnosis criteria for dementia.2 There are more than six main cognitive domains that could be affected potentially, such as learning and memory, complex attention, executive function, language, visuospatial and structural function, and social cognition. These domains are hierarchical in nature, with the bottom referring to more basic sensory and perceptual processes and the top referring to elements of executive functioning and cognitive control. They are not independent of each other and executive functioning exerts control over the utilization of more basic processes. Among the domains, learning and memory is considered to be the most vulnerable.3

In the past, MCI was thought simply to be a “precursor” phase of Alzheimer’s disease (AD).4,5 Current research has shown that MCI is not synonymous with AD and does not always progress to dementia; fortunately, some patients revert to the range of normal cognition.6 However, there are many diseases known to have a positive association with MCI, such as Parkinson’s disease, cerebrovascular diseases, and Huntington’s disease.7 Initially, these progressive diseases may appear as mild cognitive behavioural impairment, which is easily overlooked. In principle, the diagnosis of MCI is decided on the basis of clinical judgment whether the patients are absence or presence of cognitive impairment. Afterwards, specialists conduct the comprehensive neuropsychological test that can help to diagnose MCI and assess impaired cognitive domains further, which should take into account age and education of the patients. Therefore, universal screening for MCI is important to facilitate earlier diagnosis, and potential progression may be prevented or postponed with treating reversible causes and making lifestyle changes. Nowadays, the early diagnosis of MCI has been the focus of the recent research.

Machine Learning

In recent years, machine learning (ML) has become the hot spot in neurocognition study due to great advantages and wide applications in feature training and group classification. ML is an artificial intelligence instrument that predicts future data based on learning data from the past. There are two main types of ML: supervised learning and unsupervised learning. Supervised learning methods, such as neural networks (NNs), support vector machines (SVMs), K-nearest algorithms, linear and logistic regression, and random forest (RF) algorithms, build a model by learning from known classes (labelled training data). In contrast, unsupervised learning methods, including principal component analysis (PCA), are to classify to predict for unknown data (unlabelled training data).8

With advances in ML and computing techniques, developing computerized-aided diagnostics to assist specialists in the diagnosis and prediction of diseases has been recognized as an important area of medical research and development. As a branch of ML, deep learning (DL) is an algorithm of representational learning based on artificial neural networks. Simulating the structure and function of the nervous system of the human brain, DL learns representation information automatically by transforming the input data into multiple levels of abstraction. In the deeply stacked neural network, the output results of the upper neural layer serve as the input data of the next, and the final layer reveals the output diagnostic results. Each neuron in the layer has its own weight for each input connection that represents the strength for the particular connection and the bias value, which allows us to shift the activation function to the neuron with the weighted sum of the inputs and to control the trigger value of the activation function. In order to create the output for the neuron and introduce non-linearity to the neuron decision, one of the activation functions is applied to the neuron output.9 There are several types of deep neural networks, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs). CNNs are known to deal with image data, while RNNs are utilized for sequential input data.

As a clinical decision-making technology, DL can provide decision support to clinicians and improving the accuracy and efficiency of various diagnostic and treatment processes by extracting features directly from the raw data and generate an appropriate model for the target task.10 DL is also frequently applied to improve the sensitivity and specificity of disease detection and surveillance, increase objectivity in clinical decision making, significantly enhance identification and diagnostic imaging, and contribute to a wide range of screening programs.11

Retinal Imaging Techniques

Retinal imaging technique is commonly used in the field of neurology and ophthalmology. Currently, coloured digital imaging technique of retina captured by fundus camera is one of the standard methods to record the vascular appearance of the retina. Fundus vascular morphology can be observed directly by ophthalmoscope. Optical coherence tomography angiography (OCT-A) is a non-invasive imaging technique providing depth-resolved images of blood flow in the retina, choroid, and optic nerve.12 The three methods mentioned above have been applied in the clinical practice.

Literature has indicated that the retina and the central nervous system are homologous. Embryologically, the retina, the epitaxial part of the diencephalon, originates from the neural tube. During early development, the retina has a similar pattern of angiogenesis to the brain.13,14 Meanwhile, the retinal vasculature is the only part of an individual’s cerebral microcirculation that can be directly visualized.15 Therefore, the observation of changes in the retinal vasculature can be used as a window to evaluate the continuous changes in the central microcirculation.

Recently, a growing body of evidence has demonstrated that the nerve fiber and vascular pathologic changes of the retina and brain are significantly correlated.16 The study reported that retinal nerve fiber layer (RNFL) thinning was associated with cingulate cortex atrophy and episodic memory decline in old participants.17 Some evidence has also shown that the RNFL was thinner in the superior quadrant in patients with AD compared to the healthy groups, while the RNFL thickness was not differed significantly in the inferior, nasal, and temporal quadrants.18

In addition to the changes of nerve fiber layer thickness, there is a difference in vascular morphology with cognitive impairment. The researchers found that the arterial dilation and reaction amplitude of retinal vessels were decreased in the AD group in comparison with the control group, and it demonstrated that Alzheimer’s and MCI subjects were characterized by significant impairment of the retinal neurovascular coupling which was inversely correlated with the level of amyloid β in the cerebrospinal fluid.19 Recently, several clinical investigations have attempted to determine whether the retinal vasculature could be an imaging biomarker for cognitive impairment.20,21 They found that the amplitude of retinal venous pulsations correlated negatively with the neocortical Aβ scores while the amplitude of retinal arterial pulsations correlated positively with neocortical Aβ scores. Meanwhile, they also found that the retinal ganglion cell layer (RGCL) thickness was significantly lower in the clinical AD group.22 Studies have also shown that retinal vascular geometry changes, including retinal vascular calibre, retinal vascular tortuosity, retinal vascular fractal dimension,23 and retinal nerve layer thickness,20 are correlated with cognitive impairment.

Changes in the retinal vessels to a certain extent can reflect the extent of a lesion by fundus examination. Clinicians can extract specific features from fundus images to suggest reactions or changes from systemic diseases, such as diabetes, multiple sclerosis, and cognitive impairment, and to assist in making correct diagnoses and treatments.24

Retinal Vessel Segmentation Technology

Machine learning based retinal vessel segmentation technology for fundus images is one of the most important segmentation methods at present and is the basis of clinical auxiliary diagnosis systems and large-scale screening systems.25

Due to the interference of noise sources during image acquisition, transmission and storage, and the characteristics of blood vessels in fundus images, it is difficult to extract and segment blood vessels as areas of interest effectively. In order to extract the vascular components in fundus images and obtain the ideal segmentation effect, the contrast between the blood vessels and the background was increased, the greyscale of the background part of the vasculature consistent was made, and necessary image preprocessing was carried out before vascular segmentation. In the MATLAB environment, blood vessels in fundus image pretreatment technology as well as theoretical analysis and research combined with fundus image backgrounds are studied. Fundus images, by the red, green, and blue (RGB) channels, enhance image contrast, greyscale inversion and enhancement, and image filtering, realize fundus image preprocessing to obtain clear vascular eyeground veins from digital images and enhance vascular characteristics.26 Computer image processing technology is applied to the analysis and processing of coloured fundus images, and the blood vessels of the images are segmented and quantified. This approach greatly improves the utilization of medical resources, which has become an important direction in the research field of computer-aided diagnosis in recent years.

In this study, features from coloured retinal vascular images are extracted and analysed in a comprehensive manner based on ML to identify and predict MCI. This article aims to screen out populations at high risk for MCI as early as possible by ML models, to make clinical diagnoses in time, and to determine effective intervention means, which is of great significance for achieving the secondary prevention goals of early detection, early diagnosis, and early treatment, delaying the progression of the disease, and improving the lives of elderly individuals. Moreover, examining the pathophysiology of the retina in these earliest stages of dementia is scientifically valuable for the development of a screening measure that can be administered by point-of-care clinicians to large populations. It is important for the coming ageing of the population in 2050 as well.

Materials and Methods

Study Design

This study was a cross-sectional study. Eighty-six subjects were recruited from the outpatient department and the inpatient department of rehabilitation medicine at the Seventh Affiliated Hospital of Sun Yat-sen University from November 2019 to August 2020. All participants, between the age of 18 and 80 years, including stroke patients and non-stroke subjects, were provided with written consent for study inclusion and offered information for the case report form (CRF). The exclusion criteria were unstable vital signs, complications with severe chronic diseases which are difficult to complete clinical assessments, and contraindications for completing fundus photography.

The CRF was used to collect the demographic information and the clinical information of the subjects, such as age, sex, and time of education. All enrolled stroke patients conformed to the relevant diagnostic criteria for cerebral infarction and cerebral haemorrhage, which were formulated by MRI or CT examination, and they also completed the internal carotid and vertebral artery ultrasonography and heart ultrasonography simultaneously. Meanwhile, they all underwent cognitive function assessments, including the Mini-Mental State Examination scale (MMSE), Montreal Cognitive Assessment scale (MoCA), Clinical Dementia Rating scale (CDR), Physical Activities of Daily Living (PADL), Instrumental Activities of Daily Living (IADL), and fundus photography.

The study was approved by the ethic committee at the Seventh Affiliated Hospital of Sun Yat-sen University, where all data collection occurred, complies with the principles of the Declaration of Helsinki. The study has been registered in the China Clinical Trial Registry (ChiCTR1900027404).

Cognitive Impairment Grouping

MCI Diagnostic Criteria

In the 11th Revision of the International Classification of Diseases (ICD-11), the World Health Organization (WHO) adopted the definition MCI as ‘mild neurocognitive disorders’.27,28 It includes the following four main points: a) subjective experience of a decline from a previous level of cognitive functioning; b) accompanying objective evidence of impairment in performance on one or more cognitive domains relative to that expected given the individual’s age and general level of intellectual functioning, and that is not sufficiently severe to significantly interfere with independence in the person’s performance of activities of daily living; c) cognitive impairment that is not entirely attributable to normal ageing; and d) cognitive impairment that may be attributable to an underlying disease of the nervous system, a trauma, an infection, or other disease processes affecting specific areas of the brain, to chronic use of specific substances or medications, or to an etiology that may be undetermined. These points align with the DSM-5 diagnostic guidelines as well.2

Criteria for Grouping Cognitive Functions

The subjects were grouped based on the MCI diagnostic criteria and a clinical cognitive scale assessment. According to the clinical experts’ advice and the relative literature for analysis, the subjects who were divided into MCI group met the following requirements: the range of MMSE score was a loss of 3 from normal (normal standard according to educational level: illiterate group >19, primary group >22, junior secondary group and above >26); MoCA scale scores ranged from 19 to 26, the CDR score was 0.5, and PADL and IADL were not apparently abnormal. The rest of the subjects were classified into the normal group or cognitive impairment group.29–31 Afterwards, each cognitive group was divided into a training set and validation set at a ratio of 2:1 according to the random tables.

Personnel Training and Quality Monitoring

Researchers were organized to conduct training of clinical cognition scales before the study began and discussed problems and inconsistent opinions throughout the survey to ensure the identical evaluation criteria. During the evaluation, the investigators introduced the purpose and content of the study for participants and obtained informed consent and signatures from the subjects. Then, each subjects completed all the assessments and fundus photography in the study with the same day. The researchers completed the CRF within one week. Meanwhile, one person in charge and another person as quality controller of the study supervised the research process and controlled the quality of all the research data.

Fundus Image Acquisition, Image Preprocessing and Segmentation

RetiCam 3100, the fully automatic and free mydriatic fundus camera, was used for fundus imaging. Total four fundus images were taken, two of which belong to one eye with their centre on the optic disc and the macula, respectively.

MATLAB 2019b (MathWorks, USA) was used for image preprocessing and segmentation, as presented in Figure 1, via the Green channel, adaptive histogram equalization (AHE), image binarization, and median filtering, to obtain the original and segmentation retinal vessel images. The preprocessed fundus images were randomly horizontally translated, flipped, randomly enlarged, miscut transformed and filled to increase the diversity of the training images. Then, the images were normalized. Finally, the retinal vessel images were inputted for feature extraction and constructed machine learning models.32

Figure 1

Fundus images centred on the macula and optic disc. (A) Original fundus images centred on the macula; (B) adaptive histogram equalization (AHE) fundus images centred on the macula; (C) segmentation fundus images centred on the macula; (a) Original fundus images centred on the optic disc; (b) adaptive histogram equalization (AHE) fundus images centred on the optic disc; (c) segmentation fundus images centred on the optic disc.

AHE

Histogram equalization (HE) is a commonly used histogram method. The basic idea is to determine a mapping curve through the histogram of the grey distribution of the image, which is used to transform the grey level of the image to achieve the purpose of improving the image contrast. AHE is a computer image processing technique used to improve image contrast. Different from ordinary HE algorithms, the AHE algorithm calculates the local histogram of the image and then redistributes the brightness to change the image contrast.

Median Filtering

Median filtering based on the theory of order statistics of a nonlinear signal processing technology can effectively restrain noise, and it is the basic principle of the digital image or the value of the point in the sequence in which the values at various points in the field of a median instead. Then, the surrounding pixel values close to the real value are ordered to remove isolated noise points.

Extract Image Features and Train Machine Learning Models

Model Selection

The support vector machine (SVM) is a general type of feedforward network that is mainly used to solve the problem of data classification in pattern recognition fields. The SVM is based on nonlinear mapping theory to find the optimal hyperplane of feature space division to help in finding the key samples and eliminating the redundant samples with good robustness.33

Extreme learning machines (ELM) are learning algorithms for solving a single hidden layer feedforward neural network that does not need to adjust the hidden nodes. Its greatest advantage is that it is faster than the traditional learning algorithm on the premise of ensuring the learning accuracy.34

The above two models were selected for training in this study.

Extract Features and Model Training

Extracting image features and training machine learning models were implemented in Python 3.6 with OpenCV. In this study, feature extraction and model training were carried out on the original and segmentation images of fundus images, respectively. The workflow is presented in Figure 2.

Figure 2

The workflow of the study. (A) Flowchart of the study; (B) graphical abstracts of the study.

The histogram of oriented gradient (HOG) feature is a feature descriptor used for object detection in computer vision and image processing. It calculates and statistically analyses a histogram of gradient direction in the local area of the image to form the feature. The feature extraction process was as follows:

Gamma standardization: the image was greyed first, and then the gamma space was processed to reduce the sensitivity of the light.

Image gradient calculation: the gradient of the abscissa and ordinate directions of the image was calculated, and the gradient direction value of each pixel position was calculated accordingly. The derivation not only can capture contours and some textural information but also further attenuate the effect of lighting.

Histogram of gradient direction construction by cell units: the images were divided into multiple cell units of the same size, the gradient direction was divided into 9 bins, and the interval of each direction was 0–20 degrees.

Block normalization: the cells were grouped into large, spatially connected sections. In this way, the feature vectors of the cells within a block were connected in series to obtain the HOG feature of the block. The gradient intensity varied over a wide range due to local variations in illumination and variations in the foreground-background contrast. Therefore, this required the normalization of the gradient intensity.

HOG feature calculation: the last step was to collect all overlapping blocks in the detection window for HOG feature collection and combine them into the final feature vector for classification.

In this paper, the size of the normalized image was 80×80 pixels, the size of the block area was 40×40 pixels, the size of each cell was 8×8 pixels, and the pixel with sliding step length was 8×8 pixels. The gradient direction was divided into 9 bins. That is, the number of eigenvectors was [(80–8)/8+1]2*(2*2) *9=3600. Finally, the feature vector was input into the model for training as well as self-verification and verification.

Statistical Analysis

Python 3.6 was used for data processing. The result of the machine learning models was estimated using a ten-fold cross-validation approach. The receiver operating characteristic (ROC) curves of sensitivity and specificity and the area under the curve (AUC) were evaluated to identify and predict impaired cognitive function. Sensitivity was used to describe the proportion of actual cognitive impairment and judged as positive (sensitivity = true positive/(true positive + false negative)), and specificity was used to describe the proportion of actual non-cognitive impairment and judged as negative (specificity = true negative/(true negative + false positive)). This paper compared the sensitivity, specificity and AUC of the original retinal vascular fundus images and segmentation images for classifying and predicting cognitive impairment and compared the classification effects of different machine learning models. A two-sided t-test was used for statistical analysis, and P < 0.05 was considered statistically significant.

Results

Demographic Characteristics

A total of 86 eligible subjects were included in the study, with an average age of 46.37 ± 1.79 years. Of these patients, the following cases were noted: 25 with cerebral infarction, 13 with cerebral hemorrhage, and 48 with non-stroke subjects. After comprehensive cognitive assessment, the participants were divided into a normal group (N = 38), an MCI group (N = 26), and a dementia group (N = 22), and then each group was randomly divided into a training set and a validation set, as shown in Table 1. Significant differences were observed among the three groups in terms of age, educational status, MMSE, MoCA, CDR, PADL and IADL, but not sex. A total of 332 qualified fundus images were adopted after screening. Twelve images were removed due to poor exposure, over-exposure, blurriness, or out-of-focus.

Table 1

Baseline Characteristics of the Training and Validation Sets

Characteristics of Total	Normal Group	MCI Group	Dementia Group	P-value	F /χ2 -value
Age (years)	35.11 ± 1.85	55.59 ± 2.16	57.61 ± 3.21	<0.0001*	33.11
Sample size(M/F) (n)	38 (15/23)	26 (15/11)	22 (12/10)	0.2958	2.436
Cerebral infarction (M/F) (n)	0	12 (8/4)	13 (7/6)	0.5133	0.4274
Cerebral hemorrhage (M/F) (n)	0	7 (3/4)	6 (3/3)	0.7968	0.06633
Non-stroke(M/F) (n)	38 (15/23)	7 (4/3)	3 (2/1)	0.4885	1.433
Time of education(years)	14.72 ± 0.76	9.66 ± 0.98	9.58 ± 0.92	<0.0001*	12.60
MMSE (score)	29.43 ± 0.16	27.88 ± 0.40	21.67 ± 0.82	<0.0001*	43.64
MoCA (score)	27.61 ± 0.36	22.00 ± 0.77	14.50 ± 1.30	<0.0001*	79.42
CDR (score)	0.00 ± 0.00	0.50 ± 0.00	1.50 ± 0.15	<0.0001*	151.6
PADL (score)	5.97 ± 0.02	4.41 ± 0.51	1.61 ± 0.41	<0.0001*	48.31
IADL (score)	8.00 ± 0.00	6.50 ± 0.49	2.72 ± 0.55	<0.0001*	59.67
Number of fundus images (n)	149	93	90	-	-
Characteristics of Training Set	Normal Group	MCI Group	Dementia Group	P-value	F /χ2 -value
Age (years)	37.50 ± 2.46	56.60 ± 2.63	55.50 ± 3.21	<0.0001*	17.22
Sample size (M/F) (n)	26 (10/16)	17 (9/8)	15 (8/7)	0.5379	1.240
Cerebral infarction (M/F) (n)	0	8 (5/3)	9 (5/4)	0.7715	0.08433
Cerebral hemorrhage (M/F) (n)	0	5 (2/3)	4 (2/2)	0.7642	0.0900
Non-stroke(M/F) (n)	26 (10/16)	4 (2/2)	2 (1/1)	0.8741	0.2691
Time of education(years)	13.58 ± 0.84	8.63 ± 1.29	9.63 ± 1.12	0.0021*	7.013
MMSE (score)	29.38 ± 0.29	27.33 ± 0.74	24.33 ± 0.92	<0.0001*	17.96
MoCA (score)	27.63 ± 0.52	21.47 ± 1.10	17.00 ± 1.41	<0.0001*	35.09
CDR (score)	0.00 ± 0.00	0.50 ± 0.00	1.17 ± 0.11	<0.0001*	158.7
PADL (score)	5.96 ± 0.04	3.67 ± 0.67	1.75 ± 0.51	<0.0001*	27.59
IADL (score)	8.00 ± 0.00	5.80 ± 0.65	3.33 ± 0.67	<0.0001*	29.15
Number of fundus images (n)	102	66	56	-	-
Characteristics of Validation Set	Normal Group	MCI Group	Dementia Group	P-value	F /χ2 -value
Age (years)	30.33 ± 2.04	53.43 ± 3.93	61.83 ± 7.39	<0.0001*	18.49
Sample size (M/F) (n)	12 (5/7)	9 (6/3)	7 (4/3)	0.5117	1.340
Cerebral infarction (M/F) (n)	0	4 (3/1)	4 (2/2)	0.4652	0.5333
Cerebral hemorrhage (M/F) (n)	0	2 (1/1)	2 (1/1)	1.000	0.0000
Non-stroke(M/F) (n)	12 (5/7)	3 (2/1)	1 (1/0)	0.4346	1.667
Time of education(years)	17.00 ± 1.34	11.86 ± 1.06	9.50 ± 1.77	0.0031*	7.605
MMSE (score)	29.33 ± 0.22	28.29 ± 0.29	16.33 ± 1.41	<0.0001*	115.7
MoCA (score)	27.58 ± 0.34	23.14 ± 0.34	9.50 ± 1.06	<0.0001*	272.7
CDR (score)	0.00 ± 0.00	0.50 ± 0.00	2.17 ± 0.17	<0.0001*	251.2
PADL (score)	6.00 ± 0.00	6.00 ± 0.00	1.33 ± 0.71	<0.0001*	71.24
IADL (score)	8.00 ± 0.00	7.86 ± 0.14	1.50 ± 0.85	<0.0001*	98.57
Number of fundus images (n)	47	27	34	-	-

Notes: -: not applicable; *Statistically significant among the Normal group, MCI group, and Dementia group.

Abbreviations: F, female; M, male; MMSE, the Mini-Mental State Examination scale; MoCA, Montreal Cognitive Assessment scale; CDR, Clinical Dementia Rating scale; PADL, Physical Activities of Daily Living; IADL, Instrumental Activities of Daily Living.

Comparison of Each Cognitive Function Group Based on Models and the Fundus Original and Vascular Segmentation Images

The fundus original and vascular segmentation images of each group were selected to establish the SVM and ELM models, respectively. The model was self-validated and validated apart in the training set and validation set. The sensitivity, specificity and AUC results are shown in Table 2 and Figure 3.

Table 2

Sensitivity, Specificity and AUC for the SVM and ELM Models of the Fundus Original and Vascular Segmentation Images

Original Images							P-value
	SVM
	Normal Group		MCI Group		Dementia Group
	Self-Validation	Validation	Self-Validation	Validation	Self-Validation	Validation
AUC	0.87	0.85	0.88	0.87	0.90	0.86
Sensitivity	0.8488	0.7609	0.7941	0.6901	0.8684	0.9277
Specificity	0.7222	0.7500	0.8871	0.9844	0.8750	0.6964
							PS-E1=0.0012*
	ELM
	Normal Group		MCI Group		Dementia Group
	Self-Validation	Validation	Self-Validation	Validation	Self-Validation	Validation
AUC	0.82	0.81	0.86	0.83	0.85	0.84
Sensitivity	0.8488	0.7391	0.8159	0.6100	0.9737	0.7222
Specificity	0.6389	0.8571	0.7661	0.9844	0.5833	0.9107
Segmentation Images
	SVM
	Normal Group		MCI Group		Dementia Group
	Self-Validation	Validation	Self-Validation	Validation	Self-Validation	Validation
AUC	0.86	0.85	0.87	0.81	0.84	0.81
Sensitivity	0.7958	0.8913	0.7647	0.7248	0.8158	0.6111
Specificity	0.7917	0.7500	0.9113	0.8281	0.7750	0.9286
							PS-E2=0.0441*
	ELM
	Normal Group		MCI Group		Dementia Group
	Self-Validation	Validation	Self-Validation	Validation	Self-Validation	Validation
AUC	0.75	0.73	0.83	0.80	0.82	0.79
Sensitivity	0.6860	0.5217	0.6745	0.7500	0.6553	0.8933
Specificity	0.6806	0.8929	0.9032	0.7813	0.8750	0.6607
							PS-S=0.0388†
							PE-E=0.0052†

Notes: The PS-E1 and PS-E2 values were obtained, respectively, from the AUCs of the fundus original and vascular segmentation images between the SVM and ELM models. The PS-S value was obtained from the AUC of the SVM model between the fundus original and vascular segmentation images, respectively. The PE-E value was obtained from the AUC of the ELM model between the fundus original and vascular segmentation images, respectively. *Statistically significant between the SVM and ELM model; †Statistically significant between the fundus original and vascular segmentation images.

Abbreviations: MCI, Mild Cognitive Impairment; AUC, the area under the curve; SVM, support vector machine; ELM, extreme learning machine.

Figure 3

The ROC curves and areas under the curve for the SVM and ELM models of the fundus original and segmentation images in the training set and validation set. (A) The ROC curve and area under the curve for the SVM model of the fundus original images in the training set; (B) the ROC curve and area under the curve for the SVM model of the fundus original images in the validation set; (C) the ROC curve and area under the curve for the ELM model of the fundus original images in the training set; (D) the ROC curve and area under the curve for the ELM model of the fundus original images in the validation set; (a) The ROC curve and area under the curve for the SVM model of the fundus segmentation images in the training set; (b) the ROC curve and area under the curve for the SVM model of the fundus segmentation images in the validation set; (c) the ROC curve and area under the curve for the ELM model of the fundus segmentation images in the training set; (d) the ROC curve and area under the curve for the ELM model of the fundus segmentation images in the validation set.

In the fundus original images, the AUCs of the SVM model were 0.85, 0.87, and 0.86 for the normal group, MCI group, and dementia group, while the AUCs of the ELM model were 0.81, 0.83, and 0.84, suggesting that the fundus original vascular images achieved good predictive efficacy. Meanwhile, the AUCs of the fundus original images between the SVM and ELM models among three groups were compared, and the results showed that the difference was statistically significant (P = 0.0012), indicating that the SVM model had more advantages than the ELM model. Similar results were found in the fundus vascular segmentation images. The AUCs of the SVM model were 0.85, 0.81, and 0.81 for the three groups in the fundus vascular segmentation images, while the AUCs of the ELM model were 0.73, 0.80, and 0.79, suggesting that the fundus vascular segmentation images achieved good predictive efficacy as well. Also, the AUCs of the fundus vascular segmentation images between the two models were compared, and the results showed that the difference was statistically significant (P = 0.0441), demonstrating the SVM model was better than ELM model.

In addition, the fundus original and vascular segmentation images were compared between the same prediction models. For the SVM model, the AUCs of the fundus original images were 0.85, 0.87, and 0.86 for the normal group, MCI group, and dementia group, while the AUCs of the segmentation images were 0.85, 0.81, and 0.81, and the results showed that the difference was statistically significant (P = 0.0388), suggesting the fundus original images performed better than the vascular segmentation images. For the ELM model, the results indicated that the difference was statistically significant (P = 0.0052), suggesting the fundus original images performed better than the vascular segmentation images as well. In the fundus original images, the AUCs of the SVM model in the above three groups were 0.87, 0.88 and 0.90 in the training set and 0.85, 0.87, and 0.86 in the validation set, respectively, which was the best performance of all.

Discussion

Episodic memory loss is the most common early manifestation of dementia and is often overlooked. Later, terrain difficulties gradually emerge and are often accompanied by difficulties with multitasking. As the disease continues to progress, cognitive impairment becomes more severe and interferes with activities in daily life. Most dementia patients are identified in the middle and late period often by family members. Disease-modifying treatments that alter the underlying disease pathology or disease course are not yet available.35,36 Therefore, dementia seriously affects the quality of life of patients and their families and increases the social burden. At present, some studies have shown that early intervention, such as cognitive function training,37 motor function training38 and psychological and behavioural intervention,39 in patients with MCI can delay or prevent the progression of cognitive impairment and can even promote a reversion to a normal cognitive state in some patients. Therefore, early prediction and diagnosis of dementia is important for clinical decision-making and the prognosis of patients.

There are many clinical instruments available to screen for cognitive dysfunction. There are clinical assessments that follow the DSM-V criteria for specific tests in six cognitive domains, including learning and memory, language, executive function, complex attention, and social cognition. However, different environmental conditions, time periods, and qualified evaluators require specific tools to screen for cognitive impairment. The MMSE40 is still the most widely used in screening dementia, and the MoCA41 has better performance than the other dementia screening tests for MCI. Additionally, neuropsychological assessments,42 which include six main cognitive domains, may predict preclinical diseases, such as AD. Recently, various neuroimaging technologies, including structural magnetic resonance imaging (MRI), diffusion tensor imaging (DTI), arterial spin labelling (ASL), magnetic resonance spectroscopy (MRS), functional MRI (fMRI), and positron emission tomography (PET), have emerged and been applied in clinical practice. There is also growing evidence supporting the validity of MRI as a biomarker for MCI and dementia. Early identification and diagnosis of MCI or dementia can be achieved by detecting tau protein aggregation sites, atrophy of the temporal lobe and hippocampus, and changes in blood flow and metabolism.43,44 However, most clinical assessments of cognitive function are based on medical knowledge and MRI techniques are often used in hospitals. With the increasing degree of global ageing, it is difficult to effectively carry out large-scale screening.

Fundus photography is simple and easy to perform, cost-effective, and most hospitals have gradually included it in physical examination programs or popularized it in social health programs. In this study, a machine learning intelligent diagnostic model was combined with fundus photography technology based on the theoretical basis of the homology of retinal blood vessels and cerebral blood vessels during embryonic development. By extracting the features of blood vessels from the fundus images, we attempted to establish a prediction model to screen people at high risk of cognitive impairment. The study found that the models based on the vascular features of the fundus images had good recognition and prediction ability, and the normal group, the MCI group, and the dementia group showed good calibration and discrimination ability in the self-verification and verification set as well. Therefore, clinicians could refer to the predicted results and the patient’s situation to perform further targeted evaluation or examination as well as targeted cognitive function training.

In the fundus original and vascular segmentation images, we found that the SVM model had more advantages than the ELM model. Moreover, compared to fundus vascular segmentation images, we found that the original images had a better ability to recognize and predict cognitive impairment for the same prediction model. In this paper, taking monocular fundus photos centred on optic disc and the macula allowed us to extract vascular features from different angles and in a larger range, avoiding the loss of vascular information. The AHE algorithm and median filtering method used in this study for segmentation were more suitable for improving the local contrast of the image to obtain more image details and for removing noise to protect the edge of the signal from being blurred. Additionally, the method of pedestrian detection based on HOG feature combined with SVM classifiers was first proposed by French researcher Dalal in CVPR in 2005. Now, it has been widely and successfully used in image recognition, especially in pedestrian detection. Because quantization of position and direction space by the HOG algorithm can suppress the influence of translation and rotation to a certain extent and normalize it to offset the changes brought by illumination, the HOG algorithm was applied to extract features in this study. Compared with the segmentation images, original images performed better in the prediction of MCI, this result indicated that some information in the picture may be lost in the process of image segmentation. Alternatively, we need to expand the sample size further and improve image segmentation techniques and the performance of the models to verify this result.

Furthermore, there are some inconsistencies in the choice of cognitive assessment scales in the studies. MMSE is the most widely applied test for cognitive function screening throughout decades. However, it has been shown inadequate to detect MCI and clinical signs of dementia because of its ceiling effect.45 Then, MoCA was developed by Nasreddine et al in 2005 and has been shown to be a highly capable tracking tool for the recognition of normal cognitive function, MCI, and early-onset dementia. The sensitivity of MoCA to identify MCI was 90%, which is superior to MMSE when evaluating patients with MCI.30 But some studies have demonstrated that MoCA is not better than MMSE.46 Consequently, there is no consensus on which tool is more accurate in detecting cognitive function decline. CDR is a global assessment tool to evaluate both cognition and function. Also, it is a highly effective clinically based interview to distinguish between cognitive and functional decline in the spectrum of AD. In general, a CDR score of 0.5 is considered to be the definition of MCI.47 Studies have consistently shown that ADL is impaired in patients with cognitive impairment. Therefore, the impact of cognitive impairment on ADL, especially IADL, is being used as a criterion to distinguish between MCI and dementia.48 Combining with diagnostic criteria for MCI based on ICD-11, we used MMSE, MOCA, CDR, PADL, and IADL to comprehensively evaluate the cognitive and functional status of subjects, and then classified into a normal group, an MCI group, and a dementia group.

There are several limitations to our study that may represent a direction for further research. One important limitation is the lack of external validation for the model. To obtain a high level of clinical application evidence, multicentre validation with a larger sample size is needed. In addition, the studies showed that blood sugar,49 blood pressure,50 and language51 are associated with cognitive function. Whether those factors had good recognition and predictive ability for MCI, or whether they could improve the predictive performance of the model in combination with fundus vessel features, requires further exploration. Moreover, different diseases and different subtypes of the stroke may lead to different degrees and domains of cognitive impairment. In order to improve the early recognition and prediction of cognitive impairment in different diseases and different types of the same disease, further research on a large sample size is needed.

Conclusion

In the study, the establishment of a predictive model based on vascular-related feature extraction from fundus images by machine learning has high recognition and prediction abilities for cognitive function and can be used as a screening method for MCI. In conclusion, we believe this study gives a preliminary conclusion and provides a basis for further research in this field. Compared with the cognitive function assessment and imaging examination commonly used in clinical screening and diagnosis of dementia, the predictive model has greater advantages in operation and implementation, and is more suitable as a screening method for MCI or dementia in the community or physical examination centres. In future research, we will conduct a larger sample size study in multiple centres and try a variety of image segmentation and feature extraction methods to improve the recognition ability. In the process of further research, the performance of the prediction model will decline due to different factors, such as equipment and region. How to train the model with more stable and better performance through the transfer learning method is our next research direction.