Universal artificial intelligence platform for collaborative management of cataracts.

PURPOSE: To establish and validate a universal artificial intelligence (AI) platform for collaborative management of cataracts involving multilevel clinical scenarios and explored an AI-based medical referral pattern to improve collaborative efficiency and resource coverage.
METHODS: The training and validation datasets were derived from the Chinese Medical Alliance for Artificial Intelligence, covering multilevel healthcare facilities and capture modes. The datasets were labelled using a three-step strategy: (1) capture mode recognition; (2) cataract diagnosis as a normal lens, cataract or a postoperative eye and (3) detection of referable cataracts with respect to aetiology and severity. Moreover, we integrated the cataract AI agent with a real-world multilevel referral pattern involving self-monitoring at home, primary healthcare and specialised hospital services.
RESULTS: The universal AI platform and multilevel collaborative pattern showed robust diagnostic performance in three-step tasks: (1) capture mode recognition (area under the curve (AUC) 99.28%-99.71%), (2) cataract diagnosis (normal lens, cataract or postoperative eye with AUCs of 99.82%, 99.96% and 99.93% for mydriatic-slit lamp mode and AUCs >99% for other capture modes) and (3) detection of referable cataracts (AUCs >91% in all tests). In the real-world tertiary referral pattern, the agent suggested 30.3% of people be 'referred', substantially increasing the ophthalmologist-to-population service ratio by 10.2-fold compared with the traditional pattern.
CONCLUSIONS: The universal AI platform and multilevel collaborative pattern showed robust diagnostic performance and effective service for cataracts. The context of our AI-based medical referral pattern will be extended to other common disease conditions and resource-intensive situations. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

The current healthcare system is far from satisfactory for the management of common diseases, due to inadequate levels and imbalanced distribution of medical resources in low-income and middle-income countries.1 With the development of electronic medical records, digitised medical devices, wearable monitors and patient portals, telemedicine services show great potential to facilitate the evaluation, diagnosis and management of remote patients.2 3 However, the application of telemedicine in ophthalmology is currently in its infancy. Current teleophthalmology services are largely performed via ‘store and forward’ methods, which rely on the already overburdened specialists in hospitals to perform additional tasks.4 Therefore, a more efficient and effective pattern of collaboration among patients, primary healthcare providers and hospitals remains to be explored.

Artificial intelligence (AI) holds great promise for the improvement of teleophthalmology. In recent years, medical AI has moved from theory towards application in real clinical practice.5 6 The advantages of medical AI include reduction of medical costs and improvement of diagnostic and therapeutic efficiency.7 In April 2018, the US Food and Drug Administration approved the application of the first AI-based device, ‘IDx-DR’ to assist in the detection of certain diabetic eye diseases.8 However, IDx-DR falls short in accurately detecting complicated cases, which may lead to misdiagnosis and missed diagnoses.9 Therefore, it is imperative to enhance the capability of medical AI in data analysis and decision-making and to integrate current AI technology into primary healthcare services to improve patient coverage.

Cataracts are the leading cause of visual impairment worldwide, accounting for >50% of cases of blindness in low-income and middle-income countries.10 Most cataracts are related to age,11 and some are also associated with systemic diseases,12 trauma13 and congenital factors.14 With the global trend of population ageing, the prevalence of cataracts is expected to increase.15 By 2050, the number of cases of cataract blindness in China is projected to reach 20 million. However, the distribution of medical resources is far from satisfactory for cataract diagnosis and management, particularly in the primary medical facilities of low-income and middle-income countries.16 Early diagnosis and timely management of cataracts are essential for improving patient’s quality of life and reducing healthcare burdens.17 Our group has developed AI platforms for the management of congenital cataracts.18 Previous studies have also focused on the use of deep learning for the identification and grading of age-related cataracts.19 However, no universal AI agent is available for the management of cataracts that can recognise different capture modes, aetiologies and stages of treatment.

In this study, we established and validated a universal AI platform for the collaborative management of cataracts involving multilevel clinical scenarios. More importantly, we investigated an AI-based medical referral pattern to improve collaborative efficiency and medical resource coverage.

Methods

Dataset collection and labelling for AI agent training

The training set, which included 37 638 slit lamp photographs of normal lenses, cataracts of varying severity and aetiology and postoperative eyes, was derived from an ongoing national Chinese cataract screening programme by the Chinese Medical Alliance for Artificial Intelligence (CMAAI). The CMAAI is a union of medical institutions, computer science research groups and enterprises in the field of AI with the purpose of promoting the research and translational application of AI in medicine. The cataract AI agent was validated using the same screening programme as the CMAAI cohort between 2016 and 2017 and including external validation datasets from four additional multicentre cohorts from collaborating hospitals and community healthcare centres. The demographics and summary of the training and validation datasets were shown in online supplementary table S1.

After testing, the models trained with the training dataset were subjected to validation. The dataset used for training was not used for testing. The trained deep learning model was frozen prior to any validation procedures. The deep learning predictions with timestamps were verified and saved by an individual who was blinded to the expert panel labels to ensure that there was no information leakage or double-dipping when predictions were compared with classifications determined by the expert panel (figure 1).

Figure 1

Overall training pipeline for the cataract artificial intelligence (AI) agent. (A) The dataset included 37 638 images of 10 257 cases from the Chinese Medical Alliance for Artificial Intelligence (CMAAI) (30 132 images for agent training, 7506 images for the validation test). Each image was independently described and labelled by two experienced ophthalmologists, and a third ophthalmologist was consulted in case of disagreement. (B) All 37 638 images, accompanied by capture modes and diagnosis labels, were used to train the cataract AI agent. (C) The trained cataract AI agent was used to establish a multicentre validation system in conjunction with collaborating hospitals.

Images for which inclusion of the lens area had been validated were eligible for training. There were no specific requirements regarding imaging pixels or equipment. Each photograph was independently described and labelled by two experienced ophthalmologists (XH, WL and WY), and a third ophthalmologist (ZZ, WL) was consulted if disagreement arose between the initial ophthalmologists. The expert panel was blinded and had no access to the deep learning predictions. With respect to preprocessing, autocutting was employed to minimise noise around the lens, and autotransformation was conducted to save the image at a size of 224×224 pixels. A variety of slit lamps were used, including BQ-900, BX-900, OVS-II and PSL-Classic. This study was registered with ClinicalTrials.gov (identifier: NCT03623971).

Cataract diagnosis and management model for the AI agent

The cataract AI agent was designed to perform the following steps. In step 1, slit lamp photographs were classified into four separate capture modes: mydriatic-diffuse, mydriatic-slit lamp, non-mydriatic-diffuse, and non-mydriatic-slit lamp. In step 2, each photograph was diagnosed as a normal lens, cataract or a postoperative eye. In step 3, aetiological classification and cataract severity were considered to further subclassify each diagnosed photograph with respect to a management strategy of referral or follow-up. The logic flow used by the AI agent for diagnosis and management is presented in figure 2. For the images captured under mydriatic conditions, the pupil should be at least 5 mm. Visual axis opacification (VAO) in paediatric cataracts or posterior capsular opacification (PCO) was defined as opacity within the 3 mm diameter area from the visual axis. The severity of adult cataracts was evaluated primarily by the Lens Opacities Classification System II,20 nuclear grades (I~IV). A cataract with nuclear grading III or IV was defined as severe cataract or ‘referral’; otherwise, it was defined as mild cataract. If the primary evaluation decision was mild cataract with nuclear grading I~II, a secondary evaluation was performed to detect significant PCO or anterior capsular opacification (ACO), which were defined as referral conditions as well.

Figure 2

Logic flow for cataract diagnosis and management. The cataract artificial intelligence agent was designed to perform the following steps. In step 1, slit lamp photographs were classified into four separate capture modes: mydriatic-diffuse, mydriatic-slit lamp, non-mydriatic-diffuse and non-mydriatic-slit lamp. In step 2, each photograph was classified as a normal lens, a cataract or a postoperative eye. In step 3, aetiological classification and cataract severity were considered to further subclassify each photograph with respect to a management strategy of referral or follow-up. ACO, anterior capsular opacification; PCO, posterior capsular opacification; VAO, visual axis opacification.

Deep learning convolutional neural network for training and classification

ResNet was used for the image classification task in this project. Among all entrants, this algorithm exhibited the best performance on the ImageNet Large Scale Visual Recognition Challenge classification task in 2015.21 ImageNet is an image database built to measure and compare the progress of algorithms with respect to addressing image recognition problems.

The architecture of the ResNet used in this paper is depicted in online supplementary figure S1. It consists of 16 residual blocks. Each block is composed of three convolutional layers, which are 1×1, 3×3 and 1×1 convolutions with different numbers of channels. The 1×1 convolutions are responsible for reducing and increasing the dimensions of channels, and the 3×3 convolution is the main processing unit. Overall, the ResNet framework contains 50 convolutional layers and 2 pooling layers. Each pixel on each output channel is computed using the convolution between the three-dimensional kernel and the corresponding pixels across the three input channels. If the number of input channels is N, then the kernel will be N-dimensional.

Maximum pooling layers and batch normalisation22 layers were also incorporated into the extractor. The maximum pooling layers were used to down sample the image and obtain more abstract and global features, and batch normalisation was used to accelerate the training process. In accordance with the most widely used activation approach in the literature, all activations were rectified linear units. Stochastic gradient descent was used to train the network. In addition, data augmentation was performed to balance data from different categories before training via random rotation, translation, cropping and flipping (online supplementary figure S1).

ResNet applied in this study is a single task deep learning model. Different objectives (mode recognition, cataract diagnosis, severity evaluation) were trained separately with ResNet networks, one for each task. Pretrained weights were not used for the model training. The discriminating method of the score thresholds is not used, and the softmax layer is directly used to take the category corresponding to the maximum value.

The experimental environment was built using the Ubuntu 16.04.2 LTS 64-bit, Convolutional Architecture for Fast Feature Embedding (Caffe) framework and Compute Unified Device Architecture.

Statistical analyses

The indices used for evaluation were calculated using the formulas accuracy (ACC)=(TP+TN)/(TP+TN+ FP+FN), sensitivity (SEN)=TP/(TP+FN) and specificity (SPE)=TN/(TN+FP), where TP is true positive, TN is true negative, FP is false positive and FN is false negative. Asymptotic two-sided 95% CIs, adjusted for clustering by patients, were calculated and presented as proportions (sensitivity, specificity, positive predictive value and negative predictive value) and the area under the curve (AUC). Receiver operating characteristic curves were created using the R statistical package, V.3.2.4.

Results

Among the 37 638 images (18 819 eyes) in the CMAAI training and validation dataset, 20.5%, 44.7% and 34.7% had a normal lens, cataract and postoperative eye, respectively. Among the images with cataract diagnoses, the incidence of mild cataract (nuclear I–II) and severe cataract were 53.6% and 46.4%, respectively. The images for each capture mode represent 26.52% (mydriatic-diffuse), 26.52% (mydriatic-slit lamp), 26.52% (non-mydriatic-slit lamp) and 20.44% (non-mydriatic-diffuse) of the total numbers, respectively (online supplementary table S2).

The cataract AI agent was designed to perform the following steps. In step 1, the cataract AI agent distinguished among the four capture modes with AUCs of 99.36% for mydriatic-diffuse, 99.28% for mydriatic-slit lamp, 99.68% for non-mydriatic-diffuse and 99.71% for non-mydriatic-slit lamp (online supplementary figure S2).

In step 2, the agent determined diagnoses of a normal lens, cataract or postoperative eye with AUCs of 99.67%, 99.93% and 99.93%, respectively, for mydriatic-diffuse; 99.82%, 99.96% and 99.93%, respectively, for mydriatic-slit lamp; 99.26%, 99.19% and 98.99%, respectively, for non-mydriatic-diffuse and 99.30%, 99.38% and 99.74%, respectively, for non-mydriatic-slit lamp (figure 3 and online supplementary table S3).

Figure 3

Receiver operating characteristic curves and areas under the curve (AUCs) of the deep learning system for cataract diagnosis (cataract, normal or postoperative eyes). The datasets were trained and validated in separate capture modes: (A) mydriatic-diffuse images; (B) mydriatic-slit lamp images; (C) non-mydriatic-diffuse images; (D) non-mydriatic-slit lamp images.

In step 3, aetiological classification and cataract severity were considered to further subclassify specific diagnosed photographs with respect to a management strategy for referral or follow-up. For adult cataracts (aged >18 years), the agent estimated cataract severity with AUCs of 98.84% (mydriatic-diffuse), 99.15% (mydriatic-slit lamp), 93.28% (non-mydriatic-diffuse) and 98.38% (non-mydriatic-slit lamp). Among the mild adult cataracts (nuclear I~II), the AUC of detecting referable PCO/ACO was 94.88%. In paediatric cataracts (aged <18 years), the AUC of detecting referable VAO was 100.00%. Among postoperative eyes, the AUC for detecting referable PCO was 91.90% (figure 4 and online supplementary table S4). To provide patients with remote real-time monitoring of their disease conditions, we applied a Django framework (V.1.11.16, http://www.djangoproject.com/) to establish a web platform for cataract diagnosis. The web server for the diagnostic system is freely available at 20 592. After selecting an age range (aged <18 or ≥18 years) and diagnosis module (especially for PCO diagnosis of retro-illumination pictures), users can upload a new case by selecting an image from the file. The outputs, including capture modes, cataract diagnosis, severity evaluation and advice for referral, are presented on the website. To prevent misdiagnoses, doctors assess all normal cases weekly according to the general classification and communicate with patients if needed. Since the logical semantic can be updated according to the latest diagnostic guidelines, the diagnosis and treatment decisions of the platform can advance with time to meet the latest diagnostic criteria. For users who wish to test the web platform, we also provided 20 typical sample cases for download on the website.

Figure 4

Receiver operating characteristic curves and areas under the curve (AUCs) of the deep learning system for referable cataracts regarding disease severity and aetiology. (A) The deep learning system for adult cataract severity evaluation. according to the Emery nuclear grading system in current practice, mild cataract (non-referable) is defined as nuclear I–II, and severe cataract (referable) is defined as nuclear III–V. (B) The deep learning system for the detection of referable cataracts based on different aetiologies and diagnoses. Referable cataracts were defined as significant subcapsular opacification (PCO/ACO) in mild adult cataracts, VAO in paediatric cataracts or VAO) in postoperative eyes. ACO, anterior capsular opacification; PCO, posterior capsular opacification; VAO, visual axis opacification.

To integrate the cataract AI agent with real-world clinical practice, we established a novel tertiary healthcare referral pattern involving self-monitoring at home, primary healthcare and specialised hospital services. As shown in figure 5 (right panel), at level I, the basic level, information was collected from users’ mobile devices for self-monitoring. The information includes basic demographics items for registration, visual acuity (VA) and a brief case history, based on which, it screens for candidates with complaints of decreased VA or blurred vision as referral to level II; families also help to take photos of eye appearance and submit to the system. These photos were referred if necessary to assist the evaluation of ocular surface conditions. At level II, suspicious cases based on self-monitoring are referred to community-based healthcare facilities, where anterior segment images are obtained by slit lamp microscopes. The cataract AI agent provides a comprehensive evaluation by considering the diagnosis and referable conditions and then saves all of the obtained information in a database. At level III, if the AI agent decides that the cataract is a ‘referral’, a fast-track notification system was triggered, and notification is sent to the doctors for immediate confirmation. Patients are then informed that they should undergo a comprehensive examination according to the procedures of the CMAAI. Additionally, once a week, CMAAI doctors check all cases and confirm the results from the cataract AI agent.

Figure 5

Novel tertiary healthcare referral system based on the cataract artificial intelligence (AI) agent and comparison with the traditional healthcare system. In the left panel, the cataract clinic of Zhongshan Ophthalmic Center is used as an example of a traditional healthcare system for cataract management. Since there were 80 000 outpatients served by 20 specialists in the year 2017, 1 ophthalmologist can serve 4000 persons in a year. The right panel shows the operating mechanism of the novel tertiary referral system. At level I, the information including basic demographics items for registration, visual acuity (VA) and a brief case history were collected by users’ mobile device for self-monitoring. At level II, suspicious cases based on self-monitoring are referred to community-based healthcare facilities (3600/61 210; 5.9%), where anterior segment images are obtained by slit lamp microscopes. The cataract AI agent provides a comprehensive evaluation by considering the diagnosis and referable conditions and then saves all of the obtained information in a database. At level III, if the AI agent decides that the cataract is a ‘referral’, a fast-track notification system is triggered, and a notification is sent to the doctors for immediate confirmation. Patients (1090/3600, 30.3%) are then informed that they should undergo a comprehensive examination according to the procedures of the Chinese Medical Alliance for Artificial Intelligence. The pilot study was operated by three ophthalmologists for the 61 210 residents in Yuexiu District within half a year. Accordingly, 1 ophthalmologist can serve 40 806 persons in a year.

As an important link with the tertiary referral system, cataract AI ambulatory sites were established in four separate community healthcare centres (Baiyun Street, Zhuguang Street, Dongshan Street and Huanghuagang Street) in Yuexiu District, Guangzhou, China. None of the community healthcare centres had previously acquired any ophthalmic examination instruments or provided ophthalmic services. In each cataract AI ambulatory unit, ocular anterior segment images (in slit lamp and diffuse models) of residents were collected, together with information on VA and medical history. The data collected from the individual units were uploaded to the website-based cloud platform, and the AI diagnostic and referral decision was sent to the mobile terminal of each resident. The cataract AI ambulatory sites determined the diagnoses of normal, cataract and postoperative lens with AUCs of 94.35%, 95.96% and 99.64%, respectively, in non-mydriatic-slit lamp capture mode. Furthermore, the units estimated cataract severity with AUCs of 91.51%. The performance of the cataract AI unit as external validation in the real world is shown in table 1.

Table 1

Summary statistics for the diagnostic performance of the cataract AI ambulatory site in a real-world tertiary referral pattern

		AUC	ACC	SEN	SPE
Cataract diagnosis	Normal	94.35%(89.57%, 99.14%)	88.18%(84.20%, 91.46%)	71.25%(60.05%, 80.82%)	93.60%(89.81%, 96.30%）
	Cataract	95.96%(93.16%, 98.75%)	88.79%(84.88%, 91.98%)	92.00%(87.33%, 95.36%)	83.85%(76.37%, 89.71%)
	Postoperative	99.64%(98.10%, 100%)	98.18%(96.08%, 99.39%)	96.00%(86.29%, 99.51%)	98.57%(96.38%, 99.61%)
Severity evaluation	Severenuclear (III–V)	91.51%(86.13%, 96.88%)	79.50%(73.23%, 94.87%)	73.00%(63.20%, 81.39%)	86.00%(77.63%, 92.13%)
	Mildnuclear (I–II)	91.51%(85.64%, 97.37%)	79.50%(73.23%, 94.87%)	86.00%(77.63%, 92.13%)	73.00%(63.20%, 81.39%)

ACC=(TP+TN)/(TP+TN+FP+FN); SEN=TP/(TP+FN); SPE=TN/(TN+FP).

ACC, accuracy; AUC, area under the curve; FN, false negative; FP, false positive; SEN, sensitivity; SPE, specificity; TN, true negative; TP, true positive.

The novel tertiary referral system is compared with the traditional pattern of population coverage and medical resources in figure 5. In the left panel, the cataract clinic at Zhongshan Ophthalmic Center was taken as an example of the traditional healthcare system for cataract management. Since there were 80 000 outpatients served by 20 specialists in the year 2017,23 1 ophthalmologist can serve 4000 persons in a year. In the traditional healthcare system, ophthalmologists are exclusively in secondary or tertiary hospitals, whereas primary healthcare remains powerless to provide ophthalmic healthcare services to residents. The right panel shows the situation in the novel tertiary referral system. The pilot study was operated by three ophthalmologists for the 61 210 residents in Yuexiu District. Accordingly, 1 ophthalmologist can serve 40 806 persons in a year, achieving an ophthalmologist to population service ratio 10.2 times higher than the traditional pattern. During the half-year pilot study (January to June 2018), 3600 of the 61 210 (5.9%) residents received ophthalmic examinations in AI ambulatory units of community healthcare centres, and after double-checking the diagnoses of both AI and ophthalmologists, 1090 residents (1090/3600, 30.3%) were referred to ophthalmic clinics for further management.

Discussion

In this study, we established and validated a deep learning algorithm to achieve the collaborative management of cataracts using a three-step strategy: (1) capture mode recognition; (2) cataract diagnosis and (3) detection of referable cataracts with respect to aetiology and severity. The cataract AI agent achieved AUCs >99% for detecting the capture mode and cataract diagnoses in all tests. For the detection of referable cataracts, the cataract AI agent achieved AUCs >91%, even in the most difficult non-mydriatic-diffuse mode. This agent, which is developed via training and validation with the world’s largest photography database and collaborating hospitals’ datasets, is expected to improve the diagnosis and management of cataracts in multilevel collaborative systems.

Breakthroughs in AI, including applications in medicine and healthcare-related fields, have been rapidly achieved in recent years.24–26 In contrast to systemic diseases or other ocular disorders, cataracts hold promise for the management by AI agents considering their apparently uniform lesion areas and pathological bases (cloudy lens). Our group has recently developed an AI platform-CC Cruiser for the management of congenital cataracts.18 In the follow-up multicentre randomised controlled trial, CC-Cruiser exhibited comparable diagnosis accuracy, less time-consuming performance and achieved high level of patient satisfaction.27 Our previous studies indicated CC-Cruiser has the capacity to assist human doctors in clinical practice in its current state. However, the aetiology and phenotype of cataracts are variable, and an AI agent that focuses on a single specific cataract subtype cannot be applied to community-based healthcare services, where ophthalmology specialists are urgently needed. In this study, we applied different diagnostic and severity cataracts to evaluate the system for different purposes. For example, in adult patients (aged >18 years) with cataracts, the nuclear grading level was primarily evaluated to screen for the referable conditions (nuclear III–V). Among the mild adult cataracts, the subcapsular opacification (PCO or ACO) was evaluated to detect the referable ‘special’ cataract types other than the most common age-related cataracts. In paediatric patients (aged <18 years) with cataracts, the referable cataracts were defined with significant VAO considering the potential effect on the development of the visual system during this period. In addition to the management of cataract cases, the cataract AI agent is designed to detect postoperative eyes and referable PCO in the visual axis area. Taken together, the universal cataract AI agent is applicable in basic clinical practice for the management of cataracts and has the capacity to recognise different capture modes, aetiologies and stages of treatment.

Traditional cataract grading systems and previous studies with deep learning systems for cataract management have relied on a single capture mode of a slit lamp microscope28: mydriatic-slit lamp.29 30 The cycloplegics contributes to detection of peripheral cataract and is routinely used for presurgery evaluation in hospitals; while considering the risk of complications, cost and time-consuming factors, most community-based screening project would prefer capturing pictures with small pupil. Pictures in diffuse light help to observe the entire area of the lens opacification and are easier to capture for the slit lamp operator, while picture in silt lamp has the advantages of cataract nuclear grading and evaluation of the relationship between the lens plague with capsules. Thus, each of the four capture modes (mydriatic/non-mydriatic combined with diffuse/slit lamp) has the advantages and should be included in real-world cataract screening scenarios. Intriguingly, in this study, the non-mydriatic capture mode, even in diffuse conditions, showed cataract diagnostic performance comparable to that of the ‘gold-standard’ mydriatic-slit lamp mode, with AUC values >99% in all tests. For the detection of complicated referable conditions, the most difficult ‘non-mydriatic-diffuse’ mode still achieved an AUC >91%. These results suggest the feasibility of using the AI agent via a mobile application, even for the collection of images from patients at home.

This AI-based devices have been used with high accuracy in the detection of vision-threatening referable diabetes retinopathy (DR) in retinal images.31 The application of this technology took the lead to increase the efficiency and accessibility in real-world DR screening programmes.32 In contrast, as the leading cause of blindness worldwide, cataract has not been managed with clinically applicable platform using deep learning algorithms. Based on the AI platform in this study, we conducted a pilot study to evaluate its accessibility and efficiency in the real-world tertiary referral system. The result showed the agent suggested 30.3% of people be ‘referred’, substantially increasing the ophthalmologist-to-population service ratio by 10.2-fold compared with the traditional pattern. The collaborative platform and referral pattern could be extended to the management of other ophthalmic diseases, with updated user accessible mobile devices and automatic examination instruments. Further clinical trials of the AI agent will be conducted in subsequent community-based screenings in our next studies.