Recent developments on computer aided systems for diagnosis of diabetic retinopathy: a review.

Diabetes is a long-term condition in which the pancreas quits producing insulin or the body's insulin isn't utilised properly. One of the signs of diabetes is Diabetic Retinopathy. Diabetic retinopathy is the most prevalent type of diabetes, if remains unaddressed, diabetic retinopathy can affect all diabetics and become very serious, raising the chances of blindness. It is a chronic systemic condition that affects up to 80% of patients for more than ten years. Many researchers believe that if diabetes individuals are diagnosed early enough, they can be rescued from the condition in 90% of cases. Diabetes damages the capillaries, which are microscopic blood vessels in the retina. On images, blood vessel damage is usually noticeable. Therefore, in this study, several traditional, as well as deep learning-based approaches, are reviewed for the classification and detection of this particular diabetic-based eye disease known as diabetic retinopathy, and also the advantage of one approach over the other is also described. Along with the approaches, the dataset and the evaluation metrics useful for DR detection and classification are also discussed. The main finding of this study is to aware researchers about the different challenges occurs while detecting diabetic retinopathy using computer vision, deep learning techniques. Therefore, a purpose of this review paper is to sum up all the major aspects while detecting DR like lesion identification, classification and segmentation, security attacks on the deep learning models, proper categorization of datasets and evaluation metrics. As deep learning models are quite expensive and more prone to security attacks thus, in future it is advisable to develop a refined, reliable and robust model which overcomes all these aspects which are commonly found while designing deep learning models.

Introduction

Diabetic Retinopathy (DR) is a disorder that may occur in individuals who have diabetes specifically Type II diabetes. It causes when blood vessels in the retina swell, leak, or close off completely. Dry eyes, darker areas of vision, eye floaters, and difficulties detecting colors are all early signs of diabetic retinopathy (https://www.nei.nih.gov/learn-about-eye-health/eye-conditions-and-diseases/diabetic-retinopathy). Early detection of the DR is very important to prevent the complete blindness of the affected eye. Diabetic maculopathy is a disorder that can develop as a result of retinopathy in diabetics. Damage to the macula is known as maculopathy. It’s in charge of your clear vision, which you utilize for things like watching TV or reading. It can be harmed by diabetes, resulting in leaks (oedema) Fig. 1.

Fig. 1

a Healthy Retina Vision b DR Vision (http://www.vision-and-eye-health.com/diabetic-retinopathy.html)

There are many surveys already published in this study like Dimple Nagpal [126], et al., 2021 has published a review paper describing the approaches, datasets and performance measures of diabetic retinopathy. Anoop Balakrishnan and Subbian [81] has given an overview of different diabetic retinopathy detection techniques using machine learning in fundus images but fails to review even about some of the basic deep learning algorithms which are nowadays commonly used in the DR detection [13]. Moreover, Norah Asiri, et al., considers the different deep learning-based algorithms and also describes the different datasets used in the DR detection. Similarly, W Alyoubi et al., also discusses about the deep learning algorithms used in diabetic retinopathy and also categorizes the work into lesion based and vessel-based classification [9]. Furthermore, Javeria Amin et al., also discusses about the DR detection algorithms and datasets but now there is much more improvement in the technologies and algorithms in terms of space and time complexity therefore it is necessary to review the recent technologies as it will be helpful or beneficial for the researchers to carry out the fruitful research in the particular area of diabetic retinopathy [10].

Apart from these there are some more reviews in the field of diabetic retinopathy but these reviews either focuses on only the ML based techniques or discusses only the lesion based or classification based or segmentation based retinal features but, in this review, the equal focus on all the three types of retinal features are given. Not only these features are categorised into three classes but also further classified on the basis of their specification. The use of various feature selection and feature fusion methods are described which is generally ignored in the studies. The paper reviews various traditional and deep learning techniques of each of these screening systems along with their performance which clearly describes how one technique is better than other. Aside from the DR screening, there is lack of categorization of datasets means dividing the datasets on the basis of their availability i.e., either they are available publicly or privately. But in this paper, the clear classification of the public and private dataset is given. As dataset plays one of the major roles in any research therefore clear description of the dataset should be given. This will help the researcher to decide which dataset is to use as and when required and how to access that dataset. Moreover, there are very few papers which discusses about the research gaps in the present algorithms which are also described in this work. It also compared the traditional and deep learning techniques on the basis of retinal features.

Besides, the discussion of all these techniques in the literature, this study parallelly focuses on the security and robustness of the deployed algorithms against adversarial attacks which is a major concern nowadays. Therefore, discussion of all these things (all types of lesions, proper categorization of datasets and performance metrics, usage of different types of feature extraction and fusion techniques in the field of DR, analysing the research gaps, equal concern on security and robustness of the algorithm) in a single study is hard to find. Thus, this review is a furnished bundle of description of all aspects of diabetic retinopathy which will help the researchers in all aspects of their work.

Diabetic retinopathy stages

DR includes two types and four stages. The two types are Background or non-proliferative diabetic retinopathy (NPDR) and Proliferative retinopathy. Background retinopathy or NPDR is the earliest stage of DR that can lead to other eye problems, like macular edema and macular ischemia [146, 175]. NPDR has certain features from which it can be differentiated such as microaneurysms (MA), Haemorrhages, hard exudates, soft exudates. Figure 2 shows the different types of lesions in a retina.

Fig. 2

a Non-Proliferative DR b Proliferative DR c Normal Retina (https://www.retinamd.com/diseases-and-treatments/retinal-conditions-and-diseases/diabetic-retinopathy/)

Proliferative retinopathy is the serious stage in which the retina’s surface is covered in tiny blood vessels because it can grow without creating symptoms, it’s critical to get routine retinal screenings. Problems that occur in PDR are vitreous hemorrhages, neovascularization, detached retina. Based on these stages DR has various severity levels such as - No DR, Mild NPDR, Moderate NPDR, Severe NPDR, PDR https://www.healthline.com/health/diabetes/diabetic-retinopathy-stages. These severity can be seen in Fig. 3.

Fig. 3

Diabetic Retinopathy DR Stages [185]

Stage 1 mild nonproliferative diabetic retinopathy

Diabetic retinopathy in its initial phases is characterized by little patches of expansion in the blood vessels of the retina. These areas of swelling are known as microaneurysms.

Stage 2 moderate nonproliferative diabetic retinopathy

The retina’s blood supply becomes obstructed as small blood vessels swell, preventing regular feeding. As a result of this, the macula became clogged with blood and other fluids.

Stage 3 severe nonproliferative diabetic retinopathy

A greater number of blood vessels in the retina get clogged, leading to a significant reduction in blood circulation to this region. The body acquires signals to begin generating new blood vessels in the retina at this moment.

Stage 4 proliferative diabetic retinopathy

It is the most extensive stage of the disease when new cells form in the retina. Because these blood vessels are frequently fragile, there is a greater risk of fluid leakage. This results in several vision issues, including blurriness, a smaller field of vision, and even vision loss.

These specific stages of the DR along with their retinal findings are illustrated in Table 1.

Table 1

Diabetic Retinopathy Stages

DR level	Retinal Findings
Mild NPDR [133]	MAs Only
Moderate NPDR [195]	One or more haemorrhages, MAs, or any of the ones that follow Cotton wool spots Retinal hemorrhages Hard exudate Venous beading
Moderately Severe NPDR [128, 195]	One or more of the following 4 quadrants have mild intraretinal microvascular anomalies. 2–3 quadrants have severe retinal hemorrhages. Beading of the venous system in one or more quadrants
Severe NPDR [107, 128]	No symptoms of PDR and any of the following (4–2-1 rule) Each quadrant had severe intraretinal hemorrhages and microaneurysms. Two or more quadrants with distinct venous beading Considerable intraretinal microvascular abnormalities in 1 or more quadrants.
PDR [200]	One or both of following Neovascularization Vitreous/ preretinal hemorrhage

Motivation

The primary purpose of this study is to scrutinize different DR screening techniques based on various retinal features. After analyzing, based on their techniques, performance and dataset used it finds out the latest cutting-edge CAD technologies for the detection of healthy and unhealthy retinal features. Moreover, it studies the different datasets which are significant in the context of diabetic retinopathy. In addition to this, metrics for measuring performance that is used to assess DR detection techniques are elaborated. Therefore, this study provides better future directions which will help the subsequent researchers to improve their research in the area of DR detection.

Research gap

Despite the fact that a substantial degree of investigation has resulted in novice methods for the identification of DR. But some of them are not accepted by the health science. Therefore, to computerise the method, a number of research gaps must be filled. Some of the research gaps are defined below-

Selection of Database- To work with ML/DL models a large amount of data is required for training purpose. Therefore, one of the most important aspects of building an ML/AI project is gathering and preparing the dataset. The model cannot be well trained without the appropriate dataset.

AI techniques applicability- The relevance of AI models for future investigation in healthcare areas is critical, and smart health monitoring techniques should be used to diagnosis illness. These recognised photos can then be transferred to human professionals for review, decreasing their workload, lowering examination time, and preventing future problems by treating them quickly.

Selection of appropriate algorithm- The selected algorithm must be robust and reliable to different types of security attacks. The designed algorithm must be best in terms of time and space complexity.

Clear finding of the lesion type- The detection of the lesions in correct portion of the retina and discrimination of it is of the utmost importance. The proper categorization of the lesion defines the different types of severity levels of the disease and then the treatment is carried out accordingly.

Selection of performance measure- Suitable estimation measures are used to evaluate the model’s performance. It is one of the challenging tasks as it is used to assess the statistics quality of models.

Resistant to adversarial attack- There are different types of adversarial attacks which affects the functionality of the AI models Therefore, the model should be designed in such a way that

Robustness and Reliability of the model- Risk stratification, prediction, able to combat failure, should compute the level of certainty; all these aspects must be kept in mind to make a robust, reliable model.

Other strategies have been linked to improved retinal circulation visualisation. As a result, there is a strong need to discover result of the emergence, biological factors, and retinal indicators in order to thoroughly elucidate pathogenesis.

Paper organization

The remaining paper is organized as, Section 2 discusses the various publicly and privately available datasets, and also presents the performance parameters used to assess the effectiveness of the defined strategy, section 3 comprises different machine learning (ML) and deep learning (DL) ways of detecting DR. In the section 4 of the paper, different adversarial attacks on the artificial techniques and how it can be overcome are discussed, the observation based on the defined techniques is discussed in section 5, and the study is summed up including future directions in section 6. The bubble diagram below in Fig. 4 shows full taxonomy of the review paper.

Fig. 4

Taxonomy of the Review Pap7SimplePara>

In further sections of this study, will discuss the conventional and deep learning techniques/ approaches of the DR detection and also talk about the available datasets which will train the network such that it may perform better and gives the best results. Furthermore, to test the results different evaluation metrics are elaborated in the further section.

Database

The objective of databases is to verify the validity of information, and thereafter evaluate the findings of automatic DR screening with today’s technology. The data sets do not include any other information about the patient. The following shows several databases used for evaluating new algorithms for automated DR screening and analysis, a complete description of these databases is shown in Table 2.

Table 2

DR Dataset Specification

Ref.	Dataset	No. of Images	Resolution	Format	Training Sets	Test Sets	Task
Hoover, et al., 2000	STARE [74]	402 images	605*700	.ppm	–	–	Vessels extraction, Optic nerve
Staal, et al., 2004	DRIVE [171]	40 images 33 normal 7 abnormal	584*565	.jpeg	20	20	Vessels extraction
Budai, et al., 2013	HRF [24]	45 images 15- healthy eyes 15- DR eyes 15- glaucomatous eyes	3304 * 2336	.jpg	22	23	retinal vessel segmentation
Carmona, et al., 2008	DRIONS-DB [27]	110 images	600 * 400	.png	–	–	OD
Fraz, et al., 2012	CHASE_DB1 [53]	28 images	1280 × 960	.jpeg .png			vessel segmentation
Al Diri et al., 2008	REVIEW [5]	16 images	3584 × 2438 1360 × 1024 2160 × 1440 3300 × 2600	–	–	–	Vessels extraction
Kaggle	Kaggle APTOS [82]	88,702 images	Different image resolution	.jpeg	35,126	53,576	-No DR -Mild -Moderate -Severe -PDR
Kauppi, et al., 2006	DIARETDB0 [85]	130 images 20- Normal 110- Abnormal	Different image resolution	.txt	–	–	-MAs -SE -HE -HMs -neovascularization
Kälviäinen. Et al., 2007	DIARETDB1 [83]	89 images No DR- 27 Mild- 7 Moderate & Severe- 28 PDR 27	1500 X 1152 pixels	Image masks GT:.png	28	61	-MAs -SE -HE -HMs
Decencière, Etienne, et al., 2014	MESSIDOR [41]	1200 images 800- with pupil dilation 400- without dilation	1440*960, 2240*1488 or 2304*1536 pixels	.tiff Diagnosis- excel	–	–	-DR grading -Risk of Macular Edema
MESSIDOR, 2015	MESSIDOR2 [120]	1748 images	Different image resolution	1052 images- .png 690 images- .jpg	–	–	-DR grading -Risk of Macular Edema
Porwal, et al.,2018	IDRiD [141]	516 images	4288 × 2848	.jpg	413	103	-DR grading -Macular Edema
Li Tiao, et al.,2019	DDR [102]	13,673 images 6266- healthy 6257- DR	512 × 512 pixels	–	6835	4105	-MAs -SE -HE -HMs
Niemeijer, et al.,2009	ROC [130]	100 images	Different image resolution	.jpeg	50	50	Mas
Decenciere, et al.,2013	e-optha [40]	e-optha MA- 381 images 233- normal 148- MAs 47- EXs e-optha EX- 82 images	2544 × 1696 1440 × 9,601,504 × 1000 2048 × 1360	Imges- .jpeg GT:.png	–	–	-MAs -EXs
Giancardo, et al., 2012	HEI-MED [60]	169 images	2196 × 1958 pixels	.jpeg	–	–	-DR -EXs
Zhang, et al., 2010	ORIGA [206]	650 images 482- normal 168- glaucomatous	720 × 576	–	–	–	- OD - OC - Cup-to-disc ratio (CDR)
Sivaswamy, et al., 2014	DRISHTI-GS [168]	101 images 31- normal 70- glaucomatous	2896 × 1944	.png	50	51	OD segmentation -OC segmentation
Fumero, et al., 2011	RIM-ONE [56]	169 images	2144 × 1424	. bmp	–	–	Optic nerve
Derwin, et al., 2020	AGAR300 [42]	28 images	2448 × 3264	.jpeg	–	–	Mas
Tariq Kan, et al.,2020	ONHSD [90]	99 fundus images	640 × 480	–	–	–	Optic Nerve Head
Li ding, et al., 2020	PRIME-FP20 [45]	15	400 × 400	.tiff .png	–	–	retinal vessel segmentation
Jing Tian, et al.,2016	University of Miami OCT [180]	50 OCT of 10 different patients with	768 × 496	–	–	–	mild, non-proliferative diabetic retinopathy

Public database

The following are some of the publicly available datasets used for the DR detection-.

AGAR300

The AGAR300 dataset makes it easier for researchers to compare MA detection methods with digital fundus. The photographs were taken with a Fundus photography machine [42].

DDR

Dataset for Diabetic Retinopathy (DDR) For a single image, it enables multi-level annotations at the image, pixel, and bounding-box levels. Ten deep learning models were evaluated using DDR It used for DR detection, Lesion Segmentation, and detection task [102].

IDRiD

Indian Diabetic Retinopathy Image Dataset (IDRiD) was captured by a retinal expert at an Eye Clinic in Nanded, Maharashtra, India [141]. A Kowa VX-10 alpha digital fundus camera with a 50° FOV was used to capture the images.

Kaggle APTOS

Kaggle Asia Pacific Tele-Ophthalmology Society (APTOS) dataset is designed by EyePACS and is mainly designed for identifying eye diseases in rural areas as medical screening is difficult in these areas [82]. This dataset contains retinal fundus images classified into five categories (No DR, Mild, Moderate, Severe, Proliferative DR).

DRISHTI-GS

Drishti-GS is a dataset for validating OD segmentation, cup detection, and notching detection [168]. Aravind Eye Hospital in Madurai, India, gathered and annotated the pictures in the Drishti-GS collection. Because all of the patients whose eye pictures are included in this collection are Indians, this dataset represents a single population.

MESSIDOR

Methods to Evaluate Segmentation and Indexing techniques in the field of Retinal Ophthalmology (MESSIDOR) was a French Ministry of Research and Defense-funded research initiative [41]. To capture the images, three ophthalmologic departments use a color video 3CCD camera mounted on a Topcon TRC NW6 non-mydriatic retinograph with a 45° field of view.

MESSIDOR2

Diabetic patients were recruited from the Brest University Hospital’s, France Ophthalmology department to populate Messidor-Extension. This dataset consists of two-macula-centered eye fundus images (one per eye) [120].

e-ophtha

The E-Ophtha dataset was created by a telemedical network for diabetic retinopathy research financed by the French Research Agency [40]. E-Ophtha Ex and E-Ophtha MA are two forms of E-Ophtha datasets [51].

DRIVE

Digital Retinal Images for Vessel Extraction (DRIVE) introduced by Joes Staal et al. The images were captured with a Canon CR5 non-mydriatic 3CCD camera with a FOV of 45 degrees. In this dataset, there are one and two manual segmentations for each image in the training and testing sets, respectively [171].

HRF

High-Resolution Fundus (HRF) Image Database rooted by a joint research group that has created a fund to encourage an automatic segmentation algorithms comparison studies using retinal fundus pictures. In every picture, there are binary gold standard vessel segmentation images [24].

CHASE_DB1

Child Heart and Health Study in England (CHASE) are color fundus images collected from both left and right eyes of 14 school children. Two independent human experts annotate each image acquired at a 30-degree FOV [53].

HEI-MED

The Hamilton Eye Institute Macular Edema Dataset (HEI-MED) (formerly DMED) gathered as part of a telemedicine network for DR detection built by the Hamilton Eye Institute in partnership with the Université de Bourgogne [60]. All exudation sites, as well as other bright lesions on the fundus, such as cotton wool patches, drusens, or noticeably visible fluid, were discovered in this dataset. There was no discernible difference between the hard and soft exudates since this distinction is prone to mistakes and does not provide a clear clinical benefit for identification.

RIM-ONE

Retinal Image Database for Optic Nerve Evaluation (RIM-ONE) RIM-ONE is only dedicated to ONH segmentation. The retinographs were taken in the three hospitals, all of which are located in different parts of Spain [56].

INSPIRE

Stands for Iowa Normative Set for Processing Images of the Retina (INSPIRE). Currently, it consists of two sets.

INSPIRE-stereo

It’s the only healthcare stereo image dataset with realistic depth ground truth, and thus the only stereo image dataset with continuous, non-telemetry-based ground truth. It comprises 30 stereo color optic disc pictures [131].

INSPIRE

AVR includes 40 color images of the arteries and optic disc, as well as a reference standard for the arterio-venous ratio. The reference standard is based on the average of two experts’ evaluations of the images using IVAN (a semi-automated computer program developed by the University of Wisconsin, Madison, WI, USA) [206].

ORIGA

Online retinal fundus image database for glaucoma analysis and research (ORIGA) is an online resource that allows academics to exchange fundus photos and ground truths as standards for retinal image analysis findings and diagnosis [206]. It concentrates on OD and OC segmentation, as well as the cup-to-disc ratio (CDR) for glaucoma detection.

ROC

Retinopathy Online Challenge (ROC) was taken using a Topcon NW 100, a Topcon NW 200, or a Canon CR5-45NM, resulting in two different shaped FOVs [130].

DRIONS

Digital Retinal Images for Optic Nerve Segmentation Database (DRIONS-DB) is randomly selected from Ophthalmology Service at Miguel Servet Hospital, Saragossa (Spain). There are two sets of ground-truth optic disc annotations [27]. The first set is typically used for testing and training. The second group serves as the “human” baseline.

DIARETDB0

This data set is also known as “calibration level 0 fundus images”. Images were taken with a digital fundus device with a 50° FOV and unspecified settings. The images were collected in the Kuopio University Hospital, Finland [85].

DIARETDB1

This data set is also known as “calibration level 1 fundus images”. The photographs were annotated by four independent professionals [83]. These specialists defined the areas where MAs and HMs can be seen and created maps for each class of lesion.

STARE

The STructured Analysis of the Retina (STARE) Project is mainly a database for retinal vessel segmentation funded by the U.S. National Institutes of Health [74]. The Topcon fundus camera was used to capture all of the retinal fundus pictures in this data.

Private Datasets

Below are the private DR datasets which are collected from the private institutions or gathering patient details from private hospitals.

These databases (2.2.1–2.2.4) are gathered in association with Moorfields Eye Hospital from numerous population-based research. The following datasets are applied in work [6] for Automatic Optic Disc Localization.

HAPIEE

This dataset consists of a total of 1951 images of resolution 3072 × 2048 are collected from Lithuania, Europe. It divides the images into three classes labeled as normal, suspicious, and abnormal.

KENYA

This dataset consists total of 1125 images of resolution 850 × 565 are collected from Kenya, Africa.

PAMDI

This dataset consists of total 907 images of resolution 1024 × 1024 are collected from Italy, Europe. Similar, to HAPIEE it also has categorized the images into three classes normal, suspicious and abnormal.

KSSH

It comprises 67 images of 858 × 570 resolution gathered from Saudi Arabia, the Middle East in collaboration with King Faisal Specialist Hospital.

LaTIM

LaTIM (Laboratoire de Traitement de lInformation Médicale) database includes 36 colour eye fundus photos from Brest University Hospital. This work is done by [183] for MA detection. The images were saved in the tiff file format and have a resolution of 2240 × 1488 pixels. There is at least one MA in each of the 36 photos, for a total of 542 MAs.

UTHSC SA

It is collected from the University of Texas Health Science Center in San Antonio which’s why named UTHSC SA [3]. The images are 2048 × 2392 pixels in size taken from a Canon CF-60 ultraviolet retinal camera with a 60 field of vision. The size of the UTHSC SA images is 2048 × 2392 pixels.

Bejan Singh Eye Hospital database

The photos were acquired with a resolution of 2240 × 1488 pixels having JPEG file format. A collection of 100 DR-affected retinal images was gathered at Bejan Singh Eye Hospital in Nagarcoil, India, using a special fundus camera and scanned with a laser film scanner. It is used by [84] for exudate and optic disc detection.

SNDRSP

Singapore National DR Screening Program in 2013 and 2010, − 197,085 fundus pictures were recorded. This database was collected solely for the aim of researching DR and other eye problems [86].

LECHC DR

This dataset was received from Lotus Eye Care Hospital, India. Digital fundus pictures were recorded in this dataset is 122. Cannon’s non-mydriatic Zeiss digital fundus photos were used to capture the digital fundus photos [16]. There were 28 normal retinal scans in total, with 94 photos classified as DR images.

The both types of datasets i.e., private as well as public along with their sources are discussed in this section. Each and all datasets are elaborated along with all the necessary parameters like resolution, format of images, number of images and their division according to train, test and validate. Along with these parameters the task of each of the datasets like which dataset can be applied for segmentation, classification and other tasks. Thus, defining the datasets in this way will be helpful for the researcher to decide which dataset is to pick up and from where without wasting much time on this.

The arrangement of dataset in this review article is in this way, because rather than wasting time in selecting database, it is suggested that one should show their skills while preparing the dataset by applying appropriate preprocessing techniques for the further research so that better results are obtained.

Table 3 shows various performance measures used to evaluate the proposed algorithm. Performance metrics are based on probability estimates and it is very important to choose the performance metrics appropriately to measure the feasibility of the model.

Table 3

Performance Metrics

References	Performance Measure	Formulae	Description
Hao, et al., 2020 [194]	Error Rate	Error Rate=FP+FNTP+TN+FP+FN	The error rate is the average number of times we incorrectly predict the class of our target.
Vakili et al., 2020 [182], (https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173)	Accuracy (ACC)	ACC=TP+TNTP+TN+FP+FN	Rate of correct classifications
Powers et al., 2011& Goutte et al.,2005 [62, 142]	Sensitivity/ True Positive Rate/ Recall	Sen=TPTP+FN	quantifies the number of positive class predictions made out of all positive
Hao, et al., 2020 [194]	Specificity/ True Negative Rate	Spe=TNTN+FP	determines the percentage of correctly identified positives
Powers et al., 2011& Goutte et al.,2005 [62, 142]	Precision/ Positive Predictive Value	PPV=TPTP+FP	the number of positive class predictions that belong to the positive class.
Vakili et al., 2020 [182]	Area Under Curve (AUC)	AUC=∫∞−∞TPRTFPR^′TdT	(AUC) is a summary of the ROC curve and is an assessment of a classifier’s ability to discriminate between classes.
Hao, et al., 2020 [194], https://jonathan-hui.medium.com/map-mean-average-precision-for-object-detection-45c121a31173	False Positive Rate	FPR=FPTN+FP= 1-SP	The number of real negatives mistakenly predicted by the model determines the FPR.
Powers et al., 2011 [142]	Correlation Coefficient	CC=TP.TN−FP.FNTP+FPTP+FNTN+FPTN+FN	Correlation calculates the strength of the relationship between two variables. CC is used to determine the strength of the association between two variables.
Goutte et al.,2005 [62]	F-Score	F−Score=1+β^2Precision.Recallβ^2.Precision+Recall	It’s a way of measuring to check how accurate a predictor is on a certain dataset. It’s a criterion for evaluating binary classification methods that categorize examples as either “positive” or “negative.”
Furnkranz, et al., 2010	Mean-Squared Error (MSE) [57]	MSE=∑M,NI1mn−I2mn^2M∗N	The mean of the squared difference between the expected and observed parameters.
Korhonen et al., 2012	Peak Signal-To-Noise Ratio (PSNR) [95]	PSNR=10log_10R^2MSE	used to measure the quality of picture and video restoration after lossy compression
Tang, et al., 2015	Kappa Score [177]	OA = (A + D) EA = (((A + B)*(A + C)) + ((C + D)*(B + D)))/(A + B + C + D) Kappa = ((OA) – (EA)/((A + B + C + D) – (EA))	The kappa statistical measure of how well the instances categorized by the ML classifier matched the data labeled as ground truth while adjusting for the predicted accuracy of a random classifier.
Afroz, et al., 2014 & Thada et al., 2013	Dice Similarity Coefficient (DSC) [2, 178]	DSC=2TP2TP+FP+FN	DSC is the similarity quotient and has a value between 0 and 1.
Rezatofighi, et al., 2019	Intersection over Union (IoU) [156]	IoU=TPTP+FP+FN	IOU is a typical semantic image segmentation assessment metric that computes the IOU for every semantic class before averaging over all classes.

This is a very important step while designing any model. Therefore, all the appropriate metrics are discussed here in this section. The usage of these measures will be clear in the 3.1, 3.2, and 3.3 steps of section 3.

DR screening methods

Many researchers have continued to automate methods for detecting, classifying, and segmenting DRs based on their retinal features. These methods are described in this section. Figure 5 shows the DR screening techniques which discussed in this section in detail.

Fig. 5

DR Screening methods based on retinal features

Lesion detection

DR is defined by retinal lesions such as microaneurysms (MA), hard exudates (HE), and intraretinal hemorrhages, which are found in about 77–90% of people with diabetes who have had the disease for 15 years or longer [136]. This section highlights the research done to identify and classify different forms of DR lesions. The identification of different types of lesions can be seen in Fig. 6.

Fig. 6

Identification of different types of lesions [36, 147]

Microaneurysms

One of the first clinical signs of diabetes is retinal microaneurysm (MA) [11]. Retinopathy is a condition that affects the eyes (DR). Its identification is necessary to alleviate vision loss and monitoring DR [43]. Jingyu Du et al. [47], 2020, automatically detected MA from fundus images which involves candidate extraction and classification. This algorithm uses local cross-section transformation and multi-feature fusion [25]. The suggested local cross-section transformation increases descriptor differentiation by modulating the distinction amongst MAs and conflicting systems, which helps with classification and identification. Therefore, the method achieves better performance when experimented with one-ophtha, ROC training set, and DiaretDB1 datasets.

The technique to detect DR in color fundus images is developed by Shengchun Long et al., 2020. The machine learning-based directional local contrast (DLC) uses three different ways for classification [110]. The designed framework was first trained and evaluated on the e-ophtha MA database, and then on the DIARETDB1 database. The results of microaneurysm identification on the two datasets were compared to known algorithms on a lesion basis. AUC and FROC score on e-optha 0.87 and 0.86 and DIARETDB1 dataset are 0.374 and 0.210, respectively.

Yun Cheng, et al., 2020, For quick microaneurysm localization and extraction, random forest and canny edge detection algorithms are applied. in DR images [33]. The images from the MESSIDOR dataset are used for this purpose. Moreover, the sensitivity, specificity, and accuracy are evaluated in five groups which range from 85.32% to 92.34%, 97.79% to 99.54%, and 99.33% to 99.90% respectively. Thus, it can be observed that the algorithm leads to timely detection and achieves better accuracy.

Alessandro Bria et al., 2020, suggested a two-stage deep learning approach for dealing with the high-class imbalance observed during small lesion detector preparation [23]. Firstly, to deal with imbalanced data deep cascade (DC) model is defined and then CNN is used for data training. Thus, this DCNN approach is used to detect MAs and microcalcification. It has also been observed that the proposed DCNN is ten times faster than the widely used CNN approach. Thus, it can be concluded that the results of this study should be applied to a complete CAD scheme that involves stages for ad-hoc lesion postprocessing and/or false-positive elimination.

Turab Selçuk, Ahmet Alkan, et al., 2019, in this ant bee colony optimization algorithm is used in place of defined image processing techniques. Regardless of image comparison, the ant colony-based approach introduced in this study has a stable and higher efficiency [164]. As a result, it is obvious that the suggested method effectively detects microaneurysms even in photos of bad quality., making it easier for clinicians to diagnose them. Dice and Jaccard similarity index values were used to measure the correlations between microaneurysms that were segmented manually or automatically [164].

Noushin Eftekhari et al. [50], 2019, uses a deep learning technique by using two-step CNN. Due to the use of a two-stage CNN in this system, the MAs candidates for classification are chosen from a balanced dataset and descriptive portion of the picture where their composition is identical to MA, resulting in a reduction in training time. It has been observed that the experiments which are done on publicly available dataset ROC, E-Ophtha-MA are 0.3 higher as compared to other methods. On an average sensitivity on ROC, E-optha MA dataset is 0.769 and 0.771 respectively.

G. Indumathi and V. Sathananthavathi, 2019, This paper proposes an automated identification of microaneurysms in retinal images that contribute to diabetic retinopathy [77]. This study uses a rigorous preprocessing phase to improve the contrast between the MA candidates and the context. Since the final classification is focused on the accumulation of individual regression trees, it is inferred that a high degree of precision was obtained using 58 features.

Piotr Chudzik et al., 2018, developed a new patch-based fully CNN with batch normalization layers and a Dice loss feature. This method has less overhead as it works in only three computational steps while other methods complete in five steps. Moreover, the microaneurysm [36], effectively migrates information between datasets. The proposed approach immediately selects the most discriminative characteristics for MA identification and is resistant to improvements in image lighting or contrast.

Yuji Hatanaka et al., 2018, This article gives an MA detector that incorporates three different detector types: the double-ring filter, the form index based on the Hessian matrix, and the Gabor filter. The two-step DCNN and four-layer perceptron method is applied on the DIARETDB1 database for MA detection [72].

Wei Zhou et al. [208], 2017, Multi-feature Fusion Dictionary Learning (MFFDL) technique is introduced in this paper for the detection of microaneurysm in DR. This method combines the semantic relationships between multi-features and dictionary learning into a single system for MA identification [208]. The analyses are performed on the ROC training database, which is both normal and freely available. In contrast to cutting-edge techniques, the findings clearly show that the suggested system for MA detection has a higher average sensitivity.

Mrinal Haloi, 2015, A deep neural network (DNN) is employed to recognise MAs in this study [70]. This technique is different from other techniques in a way that it does not requires additional blood vessel extraction and pre-processing steps and thus useful for mass screening purposes. The model is implemented on the MESSIDOR dataset and has a sensitivity, specificity, accuracy, and AUC of 90%, 91%, 90%, and 0.989 respectively.

After studying so many traditional and deep learning techniques of microaneurysms it has been observed that tuning of model hyperparameters and ensemble learning leads to substantial performance improvement. The model gives better results if it is efficient in terms of both space and time complexity. Therefore, future researchers must give importance to all these aspects while designing an algorithm Table 4; Fig. 7.

Table 4

Different methods for microaneurysms detection

Literature	Year	Database	Methods	Sensitivity	Specificity	Accuracy	AUC
P.R. Wankhede, K. B. Khanchandani [193]	2020	DIARETDB1 E-optha MA	Pixel Intensity Rank Transform	DIARETDB1 98.79%, E-optha MA 94.59%	DIARETDB1 83.33% E-optha MA 96.56%	DIARETDB1 97.75% E-optha MA 95.80%	–
D. Jeba Derwin et al. [42]	2020	ROC Single real- time, AGAR300	Local Neighborhood Differential Coherence Pattern (LNDCP)	For ROC and AGAR300, FROC scores of 0.481 and 0.442 were attained, respectively.			–
Tania Melo et al. [119]	2020	ROC e-ophtha SCREEN-DR Messidor	Sliding band filter (SBF)	At the lesion level, e-ophtha- 64% SCREEN-DR- 81%	–	–	Train dataset ROC- 0.716 e-ophtha MA- 0.792 SCREEN-DR- 0.831
Shengchun Long et al. [110]	2019	e-ophtha MA and DIARETDB1	Directional Local Contrast (DLC)	FROC Score e-ophtha MA 0.374 DIARETDB1 0.210			e-ophtha MA 0.87 and DIARETDB1 0.86
Amrita Roy Chowdhury et al. [35]	2019	DIARETDB1, Teleoptha, Messidor	Naïve Bayes classifier, Random Forest classifier, K-means clustering	–	–	Random Forest classifier- 93.58% Naïve Bayes classifier- 83.63%	–
Shailesh Kumar, Basant Kumar [96]	2018	DIARETDB1	PCA, CLAHE, Averaging filter, SVM	96%	92%	–	–
Jose Ignacio Orlando et al. [134]	2018	DIARETDB1, e-optha, Messidor	CNN using handcrafted elements, Random Forest classifier	97.2%	–	–	93.4%
Diana Veiga et al. [183]	2018	LaTIM, e-ophtha, ROC	SVM	For average of ten false positive per image LaTIM- 62%, e-ophtha- 66%, ROC-32%	–	–	–
Baisheng Dai et al. [37]	2016	ROC DIARETDB1	Gradient Vector Analysis and Class Imbalance Classification	ROC- 0.433 at 1/8, 1/4, 1/2, 1, 2, 4 and 8 false positives per image DIARETDB1–0.321	–	–	–
Ruchir Srivastava et al. [170]	2015	DIARETDB1	Frangi-based filters	ROC- 97%

Fig. 7

a Normal Image b Image with MAs [119]

Exudate segmentation

When vision is harmed, it is due to exudates, which are fluid leaks in the blood vessels (EXs). Exudates are divided into two types: hard exudates (yellow spots in the retina) and soft exudates (regions of soft yellow or white with smudged edges) [43]. Both the exudates are clearly defined in Fig. 8.

Fig. 8

a Hard Exudate and b Soft Exudate in DR affected eye [87]

Yongshuo Zong et al. [211], 2020, suggested an automated U-net method for hard exudates segmentation is to help ophthalmologists recognize DR at an initial point. Further, to overcome the challenge of the small and imbalanced dataset, the SLIC superpixel algorithm is implemented to produce sample patches thus helpful in clinical diagnosis. Generally, the DCNN model is applied for detection purposes but n, et al., 2020, used a pre-trained CNN and transfer learning model for feature extraction. The experimental findings of the study reveal that the pretrained CNN-based system outperforms conventional strategies for exudate detection.

Parham Khojasteh, et al., 2019, applied supervised and unsupervised classifiers using various deep learning techniques for the detection of exudate [93]. This research is such that the result of a deep-learning method is determined by the parameters chosen. The tests were carried out on two public datasets DIARETDB1 and e-Ophtha. Moreover, it has been found that ResNet50 with SVM outperformed other networks, with performance and sensitivity of 98% and 0.99 respectively.

Khojasteh, et al., 2019, authors have developed a new color space for automatic exudate detection. Based on the research findings [94], this study presented a new color space for fundus images with three channels. Thus, this article shows that the selection of color space is a critical consideration when doing fundus image analysis.

Shengchun Long, et al. [109], 2019, created and tested an automated image processing method for HE detection that uses a complex support vector machine (SVM) classification is followed by threshold and fuzzy C-means clustering (FCM). This methodology is repeatable and achieves high precision for HE identification.

Kemal Adem, 2018, The technique applied in this paper shows a high-level enhancement using simple pre-processing techniques, OD segmentation using a CNN-based exudate detector to locate exudates in the retinal picture, and a circular Hough transform to eliminate the optical disc (OD) areas of the image automatically [1]. The proposed method is more efficient than those achieved using only CNN or image processing approaches.

Morphological feature extraction is used by Shilpa Joshi and P. T. Karule, 2018, for the diagnosis of the bright structure of hard exudates. Centered on the identification of rough exudates, it may be used for automated DR scanning and grading of retinopathy diseases [80].

Sanjeev Dubey, and Utkarsh Mittal, 2018, have proposed an algorithm that is beneficial when there is a large database for exudate detection [48]. This initiative has great potential because it integrates the majority of exudate pixels into its candidates and outputs individual pixels after classification. It mainly segments exudate in fundus images using computer vision and ML techniques.

Avula Benzamin, Chandan Chakraborty [18], 2018, uses a deep learning technique for hard exudate identification achieving an accuracy of 98%. The algorithm uses an eight-layer CNN model and distinguishes between the background and exudate patches.

Sreeparna Banerjee, Diptoneel Kayal, 2016, The paper outlines a procedure for detecting exudate using retinal fundus images and a range of morphological operations, normalized cut (NC), mean shift (MS), and Canny’s method [17]. This collaborative approach prevents over-segmentation while also reducing time complexity and clearly defining the exudates.

Exudates is also one of the early signs of DR. The presence of exudates in the retina should be diagnosed at early stages only. From the literature it has been gathered that exudate identification and segmentation is challenging due to the large intraclass variation and high interclass similarity. The review shows that the exudate detection is based on various standards such as region growing, adaptive and global thresholding, clustering and classification. Each of these techniques have their own advantages and disadvantages such as some of the methods shows false lesion. One more thing the researcher should focus on that to diagnose exudates, detection of edges in an image is also of equal importance as analysing proper boundary will shows better statistics. It has also seen from the literature that the further categorization of exudates into soft, circumscribed plaques of exudate and hard exudates is missing in many of the works, most of researcher has simply detected the exudates in an image without categorizing it further. Therefore, for future researcher it has been suggested that specification of exudates are necessary, as reliable recognition and categorization of it are of deep-rooted scrutiny in an automated diabetic retinopathy screening system. Some of the methods of exudate detection is given in Table 5.

Table 5

Different methods for diagnosis of exudate

Literature	Year	Database	Methods	Performance
Hui Wang et al. [190]	2020	e-optha, HEI-MED	Multi-feature joint representation, DCNN	AUC e-optha-0.9644, HEI-MED- 0.9323 F-score e-optha- 0.8929, HEI-MED- 0.9326
Nipon Theera-Umpon et al. [179]	2019	DiaRetDB1	Multilayer perceptron network (MLP), SVM, Hierarchical adaptive neuro-fuzzy inference system, CNN	AUC - 0.998
S. Karkuzhali, D. Manimegalai [84]	2019	DIARETDB0, DIARETDB1, MESSIDOR, DRIVE, STARE and Bejan Singh Eye Hospital	Inverse Surface Adaptive Thresholding Algorithm	Sensitivity 97.43%, 98.87%, 99.12%, 97.21%, 98.72%, and 96.63%, Specificity 91.56%, 92.31%, 90.21%, 90.14%, 89.58%, 92.56% Accuracy 99.34%, 99.67%, 98.34%, 98.87%, 99.13%, 98.34% for DIARETDB0, DIARETDB1, MESSIDOR, DRIVE, STARE, and Bejan Singh Eye Hospital databases respectively
Anoop Balakrishnan Kadan and Perumal Sankar Subbian [81]	2019	DIARETDB1 and DRIVE	Evolutionary Feature Selection, KNN	Accuracy- 99.34%
Juan Mo et al. [122]	2018	HEI-MED E-Ophtha EX	Cascaded Deep Residual Networks	HEI-MED Sensitivity- 0.9255 PPV- 0.8212 F-Score- 0.8499 E-Ophtha EX Sensitivity- 0.9227 PPV- 0.9100 F-Score- 0.9053
Shuang Yu et al. [201]	2017	E-Ophtha EX	DCNN	Accuracy 96.21%, Sensitivity 94.28%, Specificity 98.06%, F-Score 96.05%
M. Moazam Fraz [54]	2017	DIARETDB1, e-Ophtha EX, HEI-MED and Messidor	Ensemble Classifier of Bootstrapped Decision Trees	Accuracy- 0.8772, 0.8925, 0.9577, and 0.9836 and Area Under ROC- 0.9310, 0.9403, 0.9842, and 0.9961 for DIARETDB1, e-Ophtha EX, HEI-MED and Messidor respectively
Imani and Pourreza [75]	2016	DIARETDB1	Dynamic thresholding, morphological processing, false-positive removal	Sensitivity- 89.01% Specificity- 99.93%
Xiwei Zhang et al. [205]	2014	e-ophtha EX Messidor, DiaRetDB1 v2, and HEI-MED	Mathematical Morphology, random forest algorithm	AUC e-ophtha EX- 0.95 DiaRetDB1 v2–0.95, Messidor-0.93 HEI-MED- 0.94
Carla Agurto et al. [3]	2014	UTHSC SA, Messidor	Partial Least Squares (PLS) Multiscale Optimization	AUC UTHSC SA + Messidor- 0.962 UTHSC SA- 0.970 Messidor- 0.973

Hemorrhage detection

The initial symptom is the discovery of hemorrhages in the retina. As a result of DR, earlier bleeding detection can help to decrease blindness. The severity of the condition is determined by the frequency and shape of hemorrhages. The number of studies in the literature is an overview for hemorrhage detection Fig. 9.

Fig. 9

Retinal Hemorrhages Associated with High Altitude [112]

R. Murugan [125], 2019, To detect HE, this paper presents an improved motion pattern generation algorithm. The effectiveness of this approach is that it decreases the dimensional space relying on image resolutions, which speeds up HE identification. The proposed technique is implemented in MATLAB and experimented on publicly available MESSSIDOR dataset.

Wu [196] et al., 2019, The framework developed in this study is based on human visual features and 2D Gaussian fitting. The established method categorizes hemorrhages based on watershed segmentation and background approximation. According to the experimental data, the total average sensitivity, accuracy, and specificity for hemorrhage in the image stage were 100%, 95.42, and 82%, respectively.

He Zhao, et al., 2019, In this article, a supervised learning pipeline is presented, the center of which is the development of a synthetic fundus database using the proposed R-sGAN methodology [207]. Investigations on a range of fundus imaging datasets demonstrate that the proposed strategy is feasible.

N. Shobha Rani et al., 2019, The goals of this work are to extract blood vessel patterns and hemorrhages from green channel derived from RGB using linked object and to further identify the local binary pattern features as hemorrhage detected and non-hemorrhage detected photos from segmented objects, Sobel edge detection algorithms were used. The method is implemented on IDRiD database and generates an average accuracy of 92.31% [150].

Parham Khojasteh, et al., 2018, In the reported approach, an alternative method for reliably and concurrently detecting exudates, hemorrhages, and microaneurysms that use the probabilistic output from a CNN is suggested [92]. This method is based on image and patched-based analysis achieving a sensitivity of 0.84, 0.85, and 0.96 for detection of hemorrhages, microaneurysms, and exudates respectively when considering patch-based analysis which is better than the other reported studies.

Godlin Atlas L, Kumar Parasuraman, 2018, This study used classifier and segmentation methods to detect hemorrhages in retinal fundus photographs [61]. Firstly, preprocessing is done on images and useful features are extracted from them. Then, the ANFIS classifier is applied to distinguish the images as normal and hemorrhage-affected abnormal images. This novel segmentation method achieved an accuracy of 92.56% in the area growing with the GWO technique in comparison to the current protocol.

Salim Lahmiri, Amir Shmuel [99], 2017, this research aims to create a fully automatic a method for identifying retinal hemorrhages in images. A retinal image is analyzed with variational mode decomposition (VMD) is the proposed method’s initial step, which catches the high frequencies of the image. Finally, a predictor trained with all estimated texture features is used to distinguish between photographs of normal retinas with haemorrhages and photos of diseased retinas with haemorrhages. This method requires less time and easy to implement.

Di Xiao, et al., 2018, shows a novel hemorrhage detection approach that is built on rule-based and machine learning approaches [197]. In addition to identifying isolated hemorrhage areas, the authors concentrated on improving the identification of hemorrhages that are similar to or associated with retinal blood vessels. The presented technique achieved a sensitivity of 93.3% and specificity of 88%.

Dolly Sahu, Sachin Meshram, 2016, This system incorporated a new versatile method for automatically detecting hemorrhage [159]. This can be accomplished in three stages: removing noise from the fundus picture, removing vessels, removing the fovea, and detecting by taking shape, area, aspect ratio, density, and mean strength into account. This method scores a sensitivity and specificity of 87.71% and 94.62% respectively.

Inbarathi. R, Karthikeyan. R, 2014, uses a supervised SVM classifier to determine whether the retinal images are healthy and as hemorrhage-affected images. Further, the KNN classification for hemorrhage detection is shown in this proposed work. To perform the hemorrhage detection task Messidor dataset is used in this work [76].

Hemmorhage is a major disorder in the eye which leads to severe vision loss. The work in the literature firstly uses the edge detection methods for their accurate detection. The different morphological and filtering operations are applied to enhance the image appropriately for the accurate results. The detection of all these types of lesions precisely is very important for the classification of the DR. It has been observed that mostly ML or traditional methods are used in the literature for the hemorhage detection and a smaller number of images in the dataset is taken to check the accuracy of the model or algorithm. Therefore, it has been suggested to increase the number of images in the dataset and try to implement the DL algorithms to carry out this work. The various methods to detect hemorrhages along with their techniques are shown in Table 6.

Table 6

Methods to detect hemorrhages

Literature	Year	Database	Methods	Performance
Jun Wu et al. [196]	2019	DIARETDB	Two-dimensional Gaussian fitting, Watershed segmentation,	Hemorrhage value at the image level Sensitivity- 100% Specificity- 82% Accuracy- 95.42% Hemorrhage value at lesion level Sensitivity- 90.30% Positive Predictive- 94.01%
N.Shobha Rani et al. [150]	2019	IDRiD	Connected object and Sobel edge detection, Laplacian filtering followed by morphological bridging operation	Accuracy- 92.31%
Amrita Roy Chowdhury et al. [35]	2019	DIARETDB0 DIARETDB1 MESSIDOR Tele Optha	Random Forest	Accuracy- 93.58%
Sonali S. Gaikwad, Ramesh R. Manza [58]	2017	DIARETDB0 DIARETDB1	Template Matching, Morphological Opening, Intensity Transformation	Average Accuracy- 98.7% Sensitivity- 80%
Di Xiao et al. [197]	2017	DiaRetDB1 Local Database	Rule-Based, Random Forest	Sensitivity DiaRetDB1–93.3% Local Database- 88% Specificity DiaRetDB1–91.9% Local Database- 85.6%
Nishigandha G. Kurale and M.V. Vaidya [97]	2017	Messidor	Splat segmentation, watershed transform, SVM	Accuracy- 88% AUC- 0.89
Mark J. J. P. van Grinsven et al. [63]	2016	Kaggle MESSIDOR	CNN, Selective Data Sampling	AUC Kaggle- 0.894 MESSIDOR- 0.972
Priyakshi Bharali et al. [19]	2015	HRF, DIARETDB0, DIARETDB1, MESSIDOR and Local databases	Region Growing, Morphological Operations, Modified NICK’s Local Threshold Algorithm	Sensitivity- 97.3% Specificity- 98.92% Accuracy- 98.22%
Liye Guo et al. [65]	2015	Real-World Database	Multiclass discriminant analysis	Accuracy- 90.9%
Li Tang et al. [176]	2013	MESSIDOR	Splat Feature Classification	AUC Splat Level- 0.96 Image Level- 0.87

Choroidal neovascularization (CNV) detection

“Neovascularization” means “new blood vessels” therefore, the development of new blood vessels in the eye’s choroid layer is known as choroidal neovascularization [147]. It is mainly caused in a patient who is already suffering from age related macular degeneration (AMD). Wet AMD develops in these people when aberrant blood vessels develop into the retina and leak fluid, making the retina “wet.”. If CNV is left untreated it will lead to vision loss [34]. There are many works in the literature to find out the CNV using deep learning, machine learning techniques but the main concern of this work is to focus on the diabetic retinopathy-based lesions therefore, not going much deeper into this section it will review some of the work in the literature so that one may have the overview about this lesion type.

Venkatesan Rajinikanth et al. 2021 [147], uses machine learning scheme based binary classification along with Mayfly-Optimization Algorithm (MFA). The SVM-FG classifier achieves an accuracy of more than 92%. This method mainly uses OCT images.

Andreas Maunz et al. 2021 [118], uses both OCT and fluorescein angiography (FA) for CNV classification. The study shows that spectral domain- optical coherence tomography (SD-OCT) alone is better in terms of accuracy for the classification of CNV. This study emphasises the decreased requirement for FA and offers an automated option to physical image reading at baseline.

Kawther Taibouni et al. 2021 [172], uses a deep learning algorithm for the screening of CNV, this groundbreaking CNN-based tool will aid clinicians in the difficult chore of screening for neovascular problems.

Jie Wang et al. 2020 [191], The use of CNN in this paper is used to present a completely automated CNV identification and segmentation algorithm. The suggested technique should assist CNV detection and visualisation monitoring by allowing totally automated classification and segmentation.

Yu-Bai Chou et al. 2021 [34], uses a deep learning and ensemble stacking for CNV categorization. This research could provide an alternate way for constructing a multimodal DL framework, enhance its ability to differentiate between disorders, and clinical sciences more widely applicable in DL model construction.

These are some of the reviews of the literature for the categorization of CNV using ML- DL techniques. It has been seen that OCT images alone are enough for the detection of CNV. The new researchers should give much more emphasis on the image preprocessing and should apply some advanced deep learning algorithms for better results in terms of space and time complexity.

Classification

In this section, studies based on binary and multiclass classification are summarized.

Binary classification

The binary classification is simple to implement as it categorizes the images into DR or no DR. This is beneficial when the patients just want to know whether the eye is diseased or not. The work by M.Sakthi Sree devi [149], et al., 2021, is to segregate the features of DR using the retinal layers in an optical coherence tomography (OCT) image that is automatically segmented based on gradient detail. The Graph Cut method is applied to the images which are used to extract some of the major features from them and used to differentiate between the healthy or unhealthy DR images.

VíctorVives-Boix, and Daniel Ruiz-Fernández, 2021, Metaplasticity is intended to be implemented in convolutional neural networks i.e. metaplasticity in the backpropagation stage of CNN is used to diagnose DR in this work [184]. The dataset is chosen from Kaggle for the implementation of this work and it has been observed that Inception V3 with metaplasticity achieves better result scoring an accuracy, precision, recall, and F1 score of 95.56%, 98.9%, 90%, and, 94.24% respectively.

Gaurav Saxena, et al., 2020, The binary classification of the images is done using the models built on CNN and trained on publicly accessible datasets [163]. EyePACS dataset is used for the implementation of the model. The benchmark test dataset MESSIDOR 1 is used to gain sensitivity, specificity, and AUC 81.02%, 86.09%, and 0.92 respectively and MESSIDOR2 having sensitivity, specificity, and AUC 88.84%, 89.92%, and 0.958 respectively.

D. Jude Hemanth [73], et al., 2020, The authors use a hybrid structure for the DR detection. At first, images are enhanced using the two well-known techniques named Histogram Equalization (HE) and Contrast Limited Adaptive HE (CLAHE). Then, CNN classification is used for the diagnosis. The methods are evaluated and attain an accuracy, precision, recall, specificity, GMean, and FScore of 97%, 94%, 94%, 98%, 95%, and 94% respectively.

Revathy R, et al., 2020, In this study machine learning technique and extracts three features namely microaneurysms, exudate, hemorrhage and count their number occurrence in an image to classify them as normal and abnormal [155]. Furthermore, the combination of classifiers like random forest, k-NN, SVM, logistic regression, multilayer perceptron network known as hybrid classifiers is used for classification purposes.

Misgina Tsighe Hagos, Shri Kant, 2019, In this method, transfer learning using pre-trained Inception V3 model is used to divide DR into two groups with far less training data unlike previous DR classification methods [69]. The proposed model gains a better accuracy as compared to all the models which participated in the Kaggle challenge dataset. Our method may be applied to several other deep learning-based medical image recognition problems where classified training evidence is scarce.

Sabyasachi Chakraborty, et al., 2019, A supervised classification method is used in this paper using a customized ANN is presented to make a more precise diagnosis in the case of DR [29]. The features are extracted and given as the input to the classifier which classifies the result as DR and non-DR images. The model achieves an accuracy of 94.13% using the back propagation technique and can be effectively used in hospitals to detect the DR in patients.

Xianglong Zeng [204], et al., 2020, In this paper, transfer learning is used to build a unique CNN model with Siamese-like architecture. This architecture is different from the other models in a way that by using the pathophysiological association of both eyes, the algorithm takes binocular fundus pictures as input and estimates the probability of RDR for each eye. This selection of binocular fundus attains better results, a kappa score of 0.829 and AUC 0.951.

Navoneel Chakrabarty, and Subhrasankar Chatterjee [28], 2019, This article suggests an image processing improved hybrid DL-ML method for DR evaluation. Initially, images are pre-processed using thresholding, binarization, and grayscale conversion techniques. Then, these images are fed into a CNN-SVM-based hybrid system to classify the images as healthy and unhealthy. The study presented in this paper aims to save diabetic patients and assist them in being vigilant about their health issues.

Kele Xu, et al., 2017, in this research, the application of deep CNN methods for the effective classification of DR using color fundus images is developed and attained an accuracy of 94.5% [198]. In addition to this, a data augmentation mechanism for the suggested algorithm was provided, which increases the program’s performance.

Multiclass classification

In the multi-class classification, the images are categorized based on their severity levels specifically into four to five classes as no, mild, moderate, severe DR, PDR. This type of classification is much more beneficial than binary classification as it can diagnose the disease more specifically which gives better treatment.

Recep E. Hacisoftaoglu et al., 2020, The purpose of this research is to use a deep learning approach and the ResNet50 network to build an automated DR learning algorithm for smartphone-based retinal images. The paper uses the predefined DNN and retrained the network on different retinal datasets [68]. Thus, uses the concept of transfer learning and at last, the proposed model is tested on smartphone-based synthetic images to see how accurate smartphone-based retinal imaging systems are at detecting DR. The results show that the model gives better performance on validation data. On the independent research dataset, the suggested solution had a classification accuracy of 98.6%, with a sensitivity of 98.2% and a specificity of 99.1%, with an AUC of 0.9978.

The model defined by Abhishek Samanta [160] et al., 2020, works on a minimal dataset. Deep learning is implemented to address the classification of a 4-class problem in Diabetic Retinopathy. This model is robust and light, and it can perform excellently in small real-time environments with minimal processing resources, allowing the screening process to be sped up. Transfer learning on the pretrained DenseNet has proven to be highly successful on this dataset, and the training methodology used by us worked out great to achieve significantly high classification accuracy.

Yung-Hui Li [103], et al., 2019, produces a time-efficient model for the diagnosis of the DR. In contrast to the conventional DCNN strategy, the proposed technique substitutes the max-pooling layers with fractional max-pooling. Both of these DCNNs, each with a distinct number of layers, are equipped to generate more discriminative classification functionality. The proposed DR classifier divides DR phases into five groups, each labeled between zero and four. According to the experimental findings, the suggested technique will obtain a better detection rate. Additionally, an app named Deep Retina is developed in which ordinary citizens will take fundus photographs on their own and produce an instant result are determined by this algorithm. It can help with home nursing, remote medical treatment, and self-examination.

Nour Eldeen M. KhalifaIn, et al., 2019, in this study, augmentation procedures have been used to maximize the dataset images to be larger than the initial dataset that solves the overfitting issue. AlexNet, ResNet18, SqueezeNet, GoogleNet, VGG16, and VGG19 are the deep transfer learning models used in this article. These models are chosen as they have several layers that are less, thus, it is less complicated which reduces the training and time complexity [88].

The article by Tanapat Ratanapakorn, et al., 2019, detect the DR by classifying it into three classes i.e. normal eye, PDR, and NPDR, using the combination of digital image processing toolbox of MATLAB [151]. The experiments are performed on fundus images and achieve good accuracy. The model can be practically used in remote and rural areas for the screening of the disease.

Shaohua Wan, et al., 2018, In this study, CNN is used for DR detection, and transfer learning, and hyperparameter tuning methods are introduced for the classification of the DR images [187]. Kaggle dataset is used for training purposes. Data augmentation and normalization are done due to less number of images which helps in better training of the data. This method classifies the images into five classes. The classification accuracy obtained is 95.68%.

Parvathy En, Bharadwaja Kumar G, 2017, In this article, a method based on the CNN technique is designed for detecting DR and characterizing images based on disease severity level [139]. Since reliability and accuracy are critical in the healthcare sector, therefore, utilizing DL algorithms for image recognition can effectively solve these problems. The method will be applied in the machine linked to the fundus camera. When the camera captures an object, the machine sends it to the algorithm, which determines whether or not the specific patient has DR and, if so, the intensity level of the disease is determined.

Darshit Doshi [46] et al., 2016, The development and implementation of GPU-accelerated deep CNN to diagnose and determine optimal retinal images into five disease phases depending on seriousness is presented in this study. Further, the quadratic kappa metric is used for the evaluation of results. The three CNN models are combined to obtain better results. Therefore, after ensemble kappa score of 0.3996 is achieved.

T Chandrakumar, R. Kathirvel, et al.,2016, The authors used DCNN architecture as no external feature extraction is required. Moreover, many current supervising algorithms need further pre-processing or post-processing stages to distinguish the various stages of diabetic retinopathy [31]. This experimental study is done on DRIVE, STARE and Kaggle datasets achieving an accuracy of 94–96%.

In this section, two types of classification are discussed and it has been observed that multiclass classification is far better than the two class i.e., binary classification as it enhances the model by categorizing the disease (DR) on the basis of their severity levels. Both of these classifications along with their performance are discussed in Table 7. Many complications of the DR can be avoided by these techniques. The algorithm based on any of these classifications will give much accurate results if trained and tested on large number of images. The number of papers reviewed here shows that almost all the researchers have used Kaggle based APTOS dataset for multiclass classification which classifies the dataset into five classes. Therefore, it is suggested to the future researchers to experiment their algorithms to some other self-collected dataset.

Table 7

Different techniques for categorization of DR

Literature	Year	Database	Methods	Performance	Classification
Borys Tymchenko [181]	2020	Kaggle, APTOS	CNN	APTOS Dataset Quadratic Weighted Kappa score- 0.925 sensitivity and specificity- 0.99	Multiclass (5- Classes)
Alexandr Pak et al. [137]	2020	APTOS-2019	Compare DenseNet, ResNet with EfficientNet	ordinal regression DenseNet- 0.690, ResNet 150–0.708, ResNet 101–0.734, EfficientNet- 0.790	Multiclass (5- Classes)
Parshva Vora and Sudhir Shrestha [185]	2020	88,000 labeled images from Kaggle/ EyePacs	CNN and a k-fold cross-validation	Caffee 75.6% accuracy and 98% specificity	Multiclass
Nour Eldeen M. KhalifaIn et al. [88]	2019	APTOS-2019	Deep Transfer Learning Models	Among AlexNet, VGG16, ResNet 18, SqueezeNet, VGG19, Google Net Highest Accuracy- AlexNet 97.9% Highest Recall - VGG16 96.02% Highest Precision- AlexNet 96.23% Highest F1 Score- AlexNet 95.82%	Multiclass (5- Classes)
Yung-Hui Li et al. [103]	2019	Kaggle	DCNN	Accuracy 5-class - 86.17% Accuracy binary class – 91.05%	Multiclass (5- Classes) Binary class
Muhammad Mateen et al. [116]	2018	Kaggle	VGG-19 Architecture with PCA and SVD	Accuracy 92.21%, 98.34%, 97.96%, and 98.13% for FC7-PCA, FC7-SVD, FC8-PCA, and FC8-SVD, respectively	Multiclass (5- Classes)
Suvajit Dutta et al. [49]	2018	Kaggle	Back Propagation NN, DNN and CNN, Fuzzy C-means	Testing Accuracy BNN- 42 CNN (VGG16)- 78.3 DNN- 86.3	Multiclass
Zhiguang Wang et al. [188]	2017	Kaggle	DCNN for Discriminative Localization and Visual Explanation	On the validation set, Kappa scores- 0.70 for 256-pixel images, 0.80 for 512-pixel images and 0.81 for 768-pixel images	Multiclass (4- Classes)
Ramon Pires et al. [140]	2019	Training dataset- Kaggle Testing dataset- Messidor	Convolutional Neural Networks (CNN)	area under the ROC curve −98.2%	Binary Classification
M. T. Esfahan et al. [52]	2018	Kaggle	CNN based ResNet34	Accuracy- 85% Sensitivity- 86% F1 Score- 85%	Binary Classification
Gabriel Tozatto Zago et al. [203]	2020	Standard Diabetic Retinopathy Database, DIARETDB0, DIARETDB1, Kaggle, Messidor, Messidor1, IDRiD, DDR	Fully Patch CNN based ResNet34	AUC, Sensitivity Messidor- 0.912, 0.940, Kaggle- 0.764, 0.911, IDRiD- 0.818, 0.841, DDR- 0.848, 0.891, DIARETDB0–0.786, 0.821 Respectively	Binary Classification
Sudipta Dandapat et al. [38]	2021	Own Dataset	SVM, CNN, k-NN	Best Results SVM Accuracy-96.6% Sensitivity-0.66 Specificity- 0.95	Binary Classification
Quang H Nguyen et al. [129]	2020		CNN, VGG-16, VGG-19	sensitivity- 80%, accuracy- 82%, specificity- 82%, AUC- 0.904	Multiclass (4- Classes)
Supriya Mishra et al. [121]	2020	Kaggle (APTOS)	DL DenseNet Architecture	Accuracy- 0.9611 Kappa Score- 0.8981	Multiclass (5- Classes)
Víctor Vives-Boix, and Daniel Ruiz-Fernández [184]	2021	small diabetic retinopathy dataset of 3662 images from Kaggle	Synaptic metaplasticity in CNN	Accuracy- 95.56%, F1-score- 94.24%, Precision- 98.9%, and Recall- 90%	Binary Classification
Deepthi K Prasad [143]	2015	DIARETDB1	morphological operations, Haar wavelet transformation Back propagation neural network and one rule classifier	BPNN sensitivity- 93.3%, accuracy- 93.8%, specificity- 95.23% One rule sensitivity- 97.8%, accuracy- 97.75%, specificity- 97.5%	Binary Classification

Segmentation based

The different segmentation-based techniques to determine the abnormality in the eye are optic disc/cup segmentation and blood vessel segmentation. Both of them are overviewed in this section.

Optical disk/ optical cup segmentation

The circular area at the rear of the eye’s interior where the optic nerve links to the retina also called as the optic nerve head (ONH). The “cup” in the middle of the optic disc is usually very thin in contrast to the entire optic disc [91]. To separate the related sections of the retinal image and measure the cup-to-disk ratio, segmentation techniques such as optic disc and optic cup segmentation are used. One of the important stages in identifying DR the segmentation of the Optic Disc (OD) occurs automatically. As a result, OD pixels must be removed from the DR detection. The segmentation of optic disk/optic cup is shown in Fig. 10. Numerous methods for detecting the optic disc with varying performance characteristics such as accuracy, speed, and consistency have been introduced in the literature.

Fig. 10

Cropped region of interest (b) from original fundus image (a) [169]

YinghuaFu [55], et al.,2021, highlights an automated OD segmentation process in irregular fundus images that combines U-net with a model-driven probability bubble approach. The experiments are done on Kaggle, MESSIDOR, and NIVE database indicate that the suggested procedure effectively prevents the interference of bright lesions in abnormal fundus images thereby achieving a satisfactory OD segmentation. Moreover, when there is insufficient training data, the suggested solution makes use of the DL architecture and increases its accuracy by the model-driven location constraint.

XinYuan, et al.,2021, this analysis proposes a residual multi-scale CNN with a semantic extraction module [202]. It introduces and validated a novel W-Net, a recent fully CNN model which is capable of extracting global knowledge and bridging semantic holes in the convergence of deep and shallow semantic information. The study is carried out on four databases namely private dataset, REFUGE, DRISHTI-GS1, and RIM-ONEr3, and have an overlap error of 0.0511, 0.0684, 0.0540, 0.0492 in OC segmentation and 0.2547, 0.1777, 0.2332, and 0.2372 in OD segmentation, respectively.

Marzieh Mokhtari et al. [123], 2019, In this analysis, the symmetry of two eyes is investigated by measuring the local cup to disc ratio (CDR) from each B-scan using a fundus image fusion and OCT B-scans. To visualize the OCT details in fovea-ONH the two-step procedure is followed. After aligning the left and right pictures, OCT data is reported to their corresponding fundus images. The local CDRs are then calculated by dividing the totaled areas of cups by the totaled areas of discs in the corresponding local regions. Using this approach new index called local volumetric CDR (VCDR) is also added.

Zaka Ur Rehman et al., 2018, This paper describes a multi-parametric OD localization in retinal images [153]. The feature selection and classification are performed using Adaboost, Rusboost, SVM, and Random Forest algorithm. The findings show that the proposed approach is more resistant to the extremely variable nature of optic disc presence than other professional and intellectual methods, which fall short due to their dependence on a single function modality.

Luiz Carlos Rodrigues, MaurícioMarengoni [157], 2017, gives an algorithm to diagnose OD based on mathematical aspect and wavelet transform. Moreover, to extract, the retinal blood vessel from given image tubular characteristics along with a graph-based approach are used. The technique is designed in such a way that no pre and post-processing is required. Further, genetic algorithms are used for the optimization of the parameters. The approach is evaluated with an accuracy of 0.9465 after being tested on two databases.

Rafael Arnay, et al., 2017, In this study, the ant colony optimization technique is used for optic cup segmentation. The RIM-ONE dataset was used to validate the cup to disc ratio for glaucoma assessment, providing a cup segmentation overlap error of 24.3% and an AUC of 0.7957 [12].

Mohammad Alshayeji, et al. [8], 2017, This study applied a gravitational law-based edge detection method to retinal fundus images to detect optic discs. Furthermore, a novel filtering strategy known as candidate selection was presented, which can reduce the number of missing ODs. The introduced method was evaluated on STARE, DRIVE, DMED, and DiaRet database and have a detection rate of 95%, 100%, 92.90%, and 97.75% respectively.

Daniel D’ıaz-Pernil, et al., 2016, an automatic parallel network is implemented for OD identification [44]. The proposed method uses a hybrid technology by combining Hough transform with AGP-color segmentation and thus novel technology Hough circle cloud is introduced. The method achieves an accuracy of 99.63% after testing it on 129 retinal fundus images.

M. Partha Sarathi, et al. [162], 2016, To segment the OD, the blood vessel inpainting, region growing process, and ellipse fitting are used. The adaptive threshold makes the approach more adaptable to the design of the picture. When conducted on traditional research databases such as MESSIDOR and DRIVE, the proposed methodology yielded substantial results, with average overlapping ratios of 89% in MESSIDOR and 87% in DRIVE database.

Kevis-Kokitsi Maninis, et al. [114], 2016, uses a deep CNN model and DRIU which gives strength to CNN employs two sets of advanced layers are trained to tackle both retinal vascular and optic disc segmentation problems using a base network architecture. The qualitative and quantitative experiments were done for the OD detection using four publicly available datasets.

These are mostly used to check the symmetry between both the eyes, by calculating cup to disc ratio. It is very obvious to determine cup to disc ratio while segmenting optic disc and optic cup but calculating the accurate cup to disc ratio is a major concern. Most of the algorithms in the literature uses preprocessing techniques before detecting like background subtraction, noise removal, image conversion, dilation techniques on the collected images, applying filters and then the ML, DL techniques are applied for segmentation. This part is most useful as it is also helpful in detecting other eye diseases such glaucoma. Some of the methods of optic disk/optic cup together with their dataset and performance measure are shown in Table 8.

Table 8

Different methods for detection of optic disc and optic cup

Literature	Year	Database	Methods	Performance
Xuesheng Bian et al. [21]	2020	REFUGE Origa650	Generative Adversarial Learning Anatomy Guided Cascade Network	OD Segmentation- Dice Score- 93.31% IoU- 0.8763 OC Segmentation Dice Score - 88.04% IoU- 0.7914
Lei Wang et al. [189]	2019	CFI DIARETDB0 DIARETDB1 DRIONS-DB DRIVE MESSIDOR ORIGA	coarse-to-fine deep learning framework U-net model	Average Intersection over Union (IoU)- 89.1% Dice Similarity Coefficient (DSC)- 93.9%
Qing Liu et al. [106]	2019	ORIGA and DRISHTI	Spatial-Aware Joint Segmentation	Dice OD DRISHTI- 0.98 OC DRISHTI- 0.89
Mohammad A.U. Khan et al. [89]	2019	MESSIDOR	Vessel Convergence, Elliptical Symmetry	Accuracy- 93.5%
Sangita Bharkad [20]	2017	DRIVE, DIRATEDB0, DIRATEDB1 and DRIONS	Grayscale Morphological Dilation, Equiripple low pass FIR filter	OD Accuracies DRIVE, DRIONS- 100%, DIRATEDB0–96.92%, DIRATEDB1–98.98%, sensitivity and specificity are in the range of 74.60–87.07%, 99.39–99.61% on these four databases
Arunava Chakravarty, Jayanthi Sivaswamy [30]	2017	INSPIRE DRISHTI-GS1 Dataset-1 RIM-ONE v2 DRIONS MESSIDOR	Depth reconstruction, Conditional Random Field, Coupled sparse dictionary
Di Niu et al. [132]	2017	ORIGA MESSIDOR	Cascading Localization Method, CNN, saliency map	Classification Accuracy ORIGA- 99.87% MESSIDOR- 99.01% ORIGA+ MESSIDOR- 99.44% Localization Accuracy ORIGA- 99.33% MESSIDOR- 98.75% ORIGA+ MESSIDOR- 99.04%
Hanan S. Alghamdi et al. [6]	2016	DRIVE, DIARETDB1, STARE, MESSIDOR, HAPIEE, KENYA and PAMDI	Cascade Classifiers, CNN	OD Localization, Accuracy DRIVE-100%, DIARETDB1–98.88%, STARE-86.71%, MESSIDOR-99.20%, HAPIEE-98.36%, KENYA-99.53% PAMDI-98.13% OD Abnormality Detector Sensitivity HAPIEE-96.42%, PAMDI-94.54%, Specificity HAPIEE-86.00%, PAMDI-98.59% Accuracy HAPIEE-86.52%, PAMDI-97.76%
Sa’ed. Abed [7]	2016	DRIVE, DiaRetDB1, DMED and STARE	Background Subtraction-based Optic Disc Detection (BSODD), Swarm Intelligence Techniques	Accuracy DRIVE, DiaRetDB1–100%, DMED-98.82% and STARE-95%
M. Partha Sarathi et al. [162]	2016	DRIVE, MESSIDOR	In-painting Region growing Spline interpolation	Average Overlapping Ratio DRIVE- 87%, MESSIDOR- 89% Average OD Segmentation Accuracy- 91%
Ngan-Meng Tan et al. [174]	2015	DRIONS-DB, RIM-ONE v.3, DRISHTI-GS	U-Net CNN	Optic Disc IoU-0.89(DRIONS-DB, RIM-ONE v.3) Dice-0.94(DRIONS-DB), 0.95 (RIM-ONE v.3) Optic cup IoU-0.75(DRIONS-DB), 0.85 (RIM-ONE v.3) Dice-0.69(DRIONS-DB), 0.82 (RIM-ONE v.3)
Sohini Roychowdhury et al. [158]	2015	DRIVE, DIARETDB1, DIARETDB0, CHASE DB1, MESSIDOR and STARE	Gaussian Mixture Model classifier	OD segmentation success range 98.8–100% OD segmentation overlap score range 72–84%
Balazs Harangi, Andras Hajdu [71]	2015	DRIVE, DIARETDB1, DIARETDB0, MESSIDOR	Ensemble-based system Naïve Bayes (NB)	PPV NB model DRIVE-100%, DIARETDB0–96.15%, DIARETDB1–96.63% MESSIDOR-97.65% HNB model DRIVE-100%, DIARETDB0–98.46%, DIARETDB1–98.88% MESSIDOR- 98.33%

Blood vessel segmentation

Retinal blood vessel segmentation is a critical step in ophthalmic research. Due to the poor contrast and complex feature details of blood vessels, it is difficult to reliably segment small vessels. The central retinal artery and vein, along with their branches, are examples of retinal blood vessels. Blood Vessel Segmentation is effective to identify retinal diseases more accurately and computer-aided diagnostic (CAD) systems are one of the most difficult problems. The first phase of most computer-aided-diagnosis programs (CAD) is retinal blood vessel segmentation, which is used to diagnose ocular diseases like diabetic retinopathy (DR) as well as non-ocular diseases like hypertension, stroke, and cardiovascular diseases. The blood vessel segmentation is shown in Fig. 11.

Fig. 11

a Retinal Image b Blood Vessel Segmentation [161]

Seung Yeon Shin [167] et al.,2019, presents a novel deep learning-based vessel segmentation technique in which they integrated graph convolutional network into a unified CNN architecture that demonstrates the graphical structure of vessel form along with local appearance for vessel segmentation. The developed method can be used for the improvement of CNN based vessel segmentation model. The architecture passes through various modules like the GCNN module, inference module and at last network, training is carried out. The experiment is done on DRIVE, STARE retinal image, and coronary X-Ray angiography (CA-XRA) datasets.

Bidirectional Symmetric Cascade Network (BSCN) is a supervised approach proposed by Yanfei Guo [64] et al., in 2020 in which vessel contour labels of fixed diameter are used to supervise each layer in the network. Moreover, + Dense Dilated Convolution Module (DDCM) is applied to improve the multi-scale feature of a retinal blood vessel. Later, all the outputs of the DDCM are combined and an enhanced vessel segmentation is obtained as output. This module is developed on STARE, DRIVE HRF and CHASE_DB1 and achieves an accuracy of 0.9846, 0.9872, 0.9856, 0.9889 respectively.

Meng Li et al. 2017, introduce a supervised vessel segmentation approach [101]. This method is based on reinforcement local description which is more robust as it contains the details of intensity, shape, and edge of the vessel. The proposed technique works on the STARE dataset and achieves sensitivity, specificity, accuracy of 0.7843, 0.9837, 0.9690 respectively.

The problem of segmenting small blood vessels is overcome by Yuliang et al., 2020. The authors concentrate on deep learning width thus introduces enhanced retinal blood vessel segmentation structure (WA-Net) which segments the blood vessels more precisely [111]. As a result, it has therapeutic use in an automated diagnostic system and the ability to aid doctors to diagnose fundus diseases. The technique achieves a global accuracy of 95.66% on DRIVE and 96.45% on the STARE dataset.

Nasser Tamim, et al. [173], 2020, A supervised learning-based approach is suggested in this article, which uses a multi-layer perceptron neural network and a carefully chosen vector of features. Additionally, to improve segmentation, a post-processing procedure based on mathematical morphological operators is used. The suggested approach incorporates multiple elements that entirely contribute to its performance like the use of 24 selected features, selection of classifier (MLP network) and the way it is treated, applying post-processing and at last using five metrics to calculate the model performance. Thus, after going through all the stages it has been identified that the approach outperforms seven other related state-of-the-art approaches.

Changlu Guo et al. [66],2020, designed a Spatial Attention U-Net (SA-UNet) that introduces a spatial attention framework for adaptive function refinement that indicates the attention map along the spatial axis and multiplies it by the input feature map. In conjunction, to avoid overfitting, the proposed network uses hierarchical dropout convolutional blocks rather than U-Net’s convolutional blocks. This network works well to differentiate between both the blood vessel and the surrounding environment and can be applied for any other retinal vessel segmentation task as the retinal image has identical vascular system features.

Sambit S Mondal [124], et al.,2020, The authors in this paper have detected the retinal blood vessels using the GIFKCN method. Before applying the technique, the images are pre-processed using various approaches for pre-processing, which including Contrast Limited Adaptive Histogram Equalization (CLAHE), Gaussian Filter, Morphological operations. The work is done on the DRIVE database. At last, the results of the proposed model are compared with other models and it has been found that the introduced model has better accuracy, sensitivity, false-positive as 0.979, 0.989, 0.039 respectively.

Many methods and algorithms are used for blood vessel segmentation. Most of these methods aims to reduce overfitting and enhance model generalization and they also focuses in determining the thickness of the vessel as on the basis of the severity of the DR both thin and thick vessel are getting affected. Therefore, it is suggested that the diameter of the vessel should be precisely calculated while designing a model. The different blood vessel segmentation can be referred from Table 9.

Table 9

Methods for blood vessel segmentation

Literature	Year	Database	Methods	Sensitivity	Specificity	Accuracy	AUC
T. Jemima Jebaseeli et al. [78]	2019	STARE, DRIVE, HRF, REVIEW, and DRIONS	Tandem Pulse Coupled Neural Network Model and Deep Learning Based SVM	80.61%	99.54%	99.49%	–
Changlu Guo et al. [66]	2020	DRIVE, CHASE_DB1	Spatial Attention U-Net	DRIVE −0.8212 CHASE_DB1–0.8573	–	DRIVE-0.9698 CHASE_DB1–0.9755	DRIVE-0.9864 CHASE_DB1–0.9905
Nasser Tamim, et al. [173]	2020	DRIVE STARE CHASE_DB1	Hybrid Features and Multi-Layer Perceptron Neural Networks	DRIVE 0.7542 STARE 0.7806 CHASE_DB10.7585	DRIVE 0.9843 STARE 0.9825 CHASE_DB1 0.9846	DRIVE 0.9607 STARE 0.9632 CHASE_DB1 0.9577
Juntang Zhuang [209]	2018	DRIVE, CHASE_DB1	Laddernet: Multi-Path Networks Based on U-Net	DRIVE −0.7856 CHASE_DB1–0.7978	DRIVE-0.9810 CHASE_DB1–0.9818	DRIVE-0.9561 CHASE_DB1–0.9656	DRIVE-0.9793CHASE_DB1–0.9839
Maison et al. [113]	2018	DRIVE	Gaussian Filter	96.90%	82.10%	95.72%	–
F. Orujov et al. [135]	2020	DRIVE STARE CHASE_DB1	Fuzzy Based Image Edge Detection Algorithm	DRIVE 0.838 STARE 0.8342 CHASE_DB1 0.880	DRIVE 0.957 STARE 0.8806 CHASE_DB1 0.968	DRIVE 0.939 STARE 0.865 CHASE_DB1 0.950	–
Benzhi Chen et al. [32]	2020	3150 normal retinal images are collected	Multi-Scale Sparse Coding Based Learning (MSSCL) Algorithm				Mean AUC- 0.9918 Std AUC- 0.0028 Mean MAP- 0.8711 Std MAP- 0.0155
Ibrahim Atli, Osman Serdar Gedik [14]	2021	STARE, CHASE_DB1 and DRIVE	Sine-Net: A fully convolutional deep learning architecture	DRIVE 0.8260 STARE 0.6776 CHASE_DB1 0.7856	DRIVE 0.9824 STARE 0.9946 CHASE_DB1 0.7856	DRIVE 0.9685 STARE 0.9711 CHASE_DB1 0.9676	DRIVE 0.9852 STARE 0.9807 CHASE_DB1 0.982
Sonali Dasha, Manas Ranjan Senapati [39]	2020	DRIVE	combined approach of DWT, Tyler Coye and Gamma correction	[DWT (db1, sym1,coif1) + Tyler Coye] 0.7314 [DWT(db1, sym1,coif1) + gamma (.5, .6, .7, .8, .9, 1) + Tyler Coye] 0.7403	[DWT (db1, sym1,coif1) + Tyler Coye] 0.9891 [DWT(db1, sym1,coif1) + gamma (.5, .6, .7, .8, .9, 1) + Tyler Coye] 0.9905	[DWT(db1, sym1,coif1) + Tyler Coye] 0.949 [DWT(db1, sym1,coif1) + gamma (.5, .6, .7, .8, .9, 1) + Tyler Coye] 0.9661	–
Zhexin Jiang et al. [79]	2018	DRIVE, STARE, CHASE_DB1 and HRF	Fully Convolutional Network with Transfer Learning	Single Database Set DRIVE 0.7540, STARE 0.8352, CHASE_DB1 0.8640 Cross Database Set DRIVE 0.7121, STARE 0.7820, CHASE_DB1 0.7217	Single Database Set DRIVE 0.9825, STARE 0.9846, CHASE_DB1 0.9745 Cross Database Set DRIVE 0.9832, STARE 0.9798, CHASE_DB1 0.9770	Single Database Set DRIVE 0.9624, STARE 0.9734, CHASE_DB1 0.9968 Cross Database Set DRIVE 0.9593, STARE 0.9653, CHASE_DB1 0.9591	Single Database Set DRIVE 0.9810, STARE 0.9900, CHASE_DB1 0.9810 Cross Database Set DRIVE 0.9680, STARE 0.9870, CHASE_DB1 0.9580

Feature selection and feature fusion methods

By picking the most significant features and discarding superfluous attributes, feature selection enhances the ML, DL approaches experience and raises the prognostic capacity of these algorithms [148]. On the other side, feature fusion is the process of combining training picture feature vectors generated from the common weighted network layer with feature vectors consisting of other statistical information such that the suggested framework can use quite so many features for categorization [210]. Thus, feature selection and fusion of the selected features plays a major role to bring off better output in terms of accuracy and other performance measures. The various feature selection and feature fusion method are given in Table 10.

Table 10

Different types of feature selection and fusion methods

Reference	Method	Technique	Dataset	Performance	Advantage	Disadvantage
K Yazhini et al., 2020 [199]	Fusion based feature extraction	Gray level co-occurrence matrix and VGG-19 known as FM-GLCM-VGG19	Kaggle	Accuracy- 71.30% Sensitivity- 50.43% Specificity- 80.19%	This fusion based takes less time.	Model is validated on only one image file format. The combination of images should be taken to check the robustness of the model
S. Gayathri et al., 2020 [59]	Feature extraction and feature selection	Feature extraction SURF, BRISK, MR-MR feature selection and ranking, SVM, MLP, Naïve Bayes	IDRiD, MESSIDOR, DIARETDB0	Binary Classification Average Accuracy- 98.13% Multiclass Classification Average Accuracy- 95.13%	Robust and reliable. Trained and tested on different types of images Suitable for both binary as well as multiclass classification	The novel deep learning techniques should be taken into consideration for better results.
Zun Shen et al., 2021 [166]	Ensemble Learning	XGBoost, Stacking	Biochemical and Physical dataset	Average Accuracy- 83.95%	Prohibits data feature redundancy.	No appropriate technique for dimensionality reduction. Prediction accuracy is not up to the mark.
Farrukh Zia et al., 2021 [210]	Feature Selection and Feature Fusion	VGG and Inception V3	Kaggle	Accuracy – 96.4%	Comparatively takes less time. Different classification methods are applied for dataset testing.	Fitness function is missing while selecting features
Muhammad Mateen et al., 2020 [117]		Convolutional Neural Network	e-Ophtha DIARETDB1	e-Ophtha Accuracy- 98.43% DIARETDB1 Accuracy- 98.91%	Pretrained models with transfer learning improves the model accuracy. Combination of images are used for better training	Detection of other lesion such as hemorrhages, microaneurysms are missing in this study
Lakshmana Kumar Ramasamy et al., 2021 [148]	Feature Fusion	Ridgelet Transform, Sequential Minimal Optimization	DIARETDB1 KAGGLE	DIARETDB1 Accuracy- 97.05% Sensitivity- 98.87% Specificity- 95.24%	Taken large number of images in account Use of optimization improves results	Computation time complexity is high Deep features are missing
Zhuang Ai et al., 2021 [4]	Deep Ensemble Learning	DR-IIXRN	Kaggle	AUC- 95% Accuracy- 92% Recall Rate- 92%	Can provide specialists with more accurate disease grading in order to select the most appropriate treatment options	Testing of model which can provide specialists with more accurate disease grading in disease
Jyostna Devi Bodapati. Et al., 2020 [22]	Blended Learning	Deep Neural Network (DNN), ConvNet	Kaggle	Binary Classification Accuracy- 97.41% kappa score-94.82% Multiclass Classification accuracy-81.7% kappa score-71.1%	Faster as it uses blended features	Heterogeneity of images are missing.

Adversarial attacks

Cyber-security is the technique of safeguarding software applications from online threats, which are becoming increasingly common in the digital world. In AI, ML and Deep Learning field, adversarial attack, a methodology that tries to deceive models using false data, is becoming a significant concern [145]. The security and resilience of implemented algorithms must be assured due to the accelerated expansion of artificial intelligence (AI) and deep learning (DL) approaches [154].

Attacks on Ai techniques are frequently classified along three axes: influence on the classifier, security breach, and specificity. As shown in Fig. 12, implementing an adversarial attack entail acquiring an input (authentic) image and purposefully perturbing it with a noise, forcing the connectivity to misdiagnose the authentic image, perhaps leading to a massive classification error.

Fig. 12

Example of adversarial attack on an authentic image [186]

There are different categories of adversarial attacks. One of them is targeted and untargeted attack. In targeted attack, the goal is to make the model misclassify the intended targeted class by predicting the adversarial example [15]. On the other hand, untargeted attack does not have any targeted class to make the model misclassify, its goal is to simply make the model misguide with the help of adversarial example [98, 152].

It has been gathered from the research that the targeted attacks are much more successful as compared to untargeted attacks but they have a large time complexity and untargeted attacks take much less time comparatively. Figure 12 shows how the involvement of external noise affects an image and results in an attack.

Next classification of attacks are black box and white box attacks [186]. In a black-box attack, the attacker has no knowledge of the target model’s architecture or characteristics, and his or her only competence is to feed the target model the data they want and watch the target model categorize the results [67]. On the other hand, in white box attacks model’s attributes are accessible to attackers [108]. The Table 11 below shows the comparison and working of different adversarial attacks.

Table 11

Adversarial attack for image classification

Reference	Attack Type	Approach	Description	Pros	Cons
Hongchen Cao et al., 2021 [26]	Black Box	DL	In this research, black-box strategy for spoofing deep learning networks in apps by training alternative approaches is presented. To undertake blackbox adversarial assaults, the technique is tested on ten real-world deep-learning apps from Google Play.	• Consider Large dataset. • Real world apps are taken into consideration. • Success rate improves up to 38.97%	• Not suitable for obfuscated apps. • The experiment is only done on computer vision-based apps i.e., not widely acceptable
Sensen Guo, et al., 2021 [67]	Black Box	ML	proposes a machine learning based abnormal flow detector. The substitute model is trained with a comparable decision boundary and algorithm is used to create adversarial instances then examines if these examples can avoid the target models detection.	• Heterogeneous datasets are used • The method has high chances to bypass the detection of target model.	• Experiments on real network are missing
Nicolas Papernot et al., 2017 [138]	Black Box	ML	Synthetic data generation is used to craft misclassified examples. The solution of a work is a substantial step toward easing earlier attackers’ rigid preconceptions about adversarial abilities.	• Uses real world models such as hosted by two widely used sites i.e., Google and Amazon	• Did not focus on the other combinations of adversarial examples.
Hongying Liu et al., 2020 [108]	White Box	DL	We introduced the ADV-ReLU framework, a new universal adversarial example generation system that can be successfully incorporated into gradient-based white-box gradient-based algorithms.	• Robust Framework	• Some other dataset should also be used to test the framework
Yixiang Wang et al., [192]	White Box	DL	DIAA, an interpretable white-box AE attack strategy that investigates the use of the deep Taylor decomposition’s interpretable strategy for the evaluation of the most significant aspects is used.	• successfully attack both non-robust and robust systems with minimal perturbation • Tested on different datasets	• Not suitable for a smaller number of images.
Linfeng Ye [105]	White Box	DL	One step First order method for neural network attack. TVM framework is used to fasten the backword and forward propagation	• High success rate as compared to second order or multiclass first order • Faster than other proposed attacks	• Not much robust • Does not work well if multiple steps attacks are taken into account.
Gege Qi et al., 2021 [144]	Stabilized Medical Image	DL	image-based medical adversarial attack approach is proposed. The loss deviation and a loss stabilization functions are used	• Investigations on a variety of medical computer vision benchmarks, have shown that the suggested technique is stable. • Recent research (COVID 19) is taken into account.	• Only focuses on medical imaging dataset.
Jing Lin et al., 2022 [104]	Secure ML	ML	a distributed adversarial retraining approach is used. The proposed framework has used soft label and uses transferability which reduces its time complexity.	• Robust against adversarial attacks.	• Not suitable for black box attacks
Sheeba Lal et al., [100]	DR detection	ML, DL	Defensive model against noise, fusion technique on the retinal DR images.	• Different features are taken into consideration in order to improve the model’s sturdiness	• Limited to particular application
Jiawang Bai et al., 2021 [15]	Targeted Attack	DL	To achieve stealthiness, aim is to misdiagnose a certain instance into a target class without modifying it, but without significantly reducing the predictive performance of other samples. Because the parameters are stored as bits (i.e., 0 and 1) Therefore, the problem is formulated as binary integer programming	• Numerous tests show that our strategy is superior when it comes to attacking DNNs. • The defined technique is resistance to different parameters.	• The proposed is not superior than other defined methods in terms of time complexity.
Pradeep Rathore et al., 2020 [152]	Targeted, Untargeted, and Universal	DL	These attacks are studied on 54 multiclass UCR time series database	• It puts real-world implementations reliant on deep learning models in jeopardy.	• The experiment on multiple adversarial attacks is missing.
Hyun Kwon et al., 2018 [98]	Untargeted Attack	DL	This work uses random untargeted adversarial examples.	• It is open to steganography • Overcomes problem of pattern vulnerability.	• Experiments on medical imaging dataset is missing.

From the literature, it can be seen that deep neural network lacks in the area of adversarial examples. Therefore, research is going on different types of attacks (targeted, untargeted, black box, white box etc.) taking deep learning as a base. Thus, it is recommended to the researchers to test the performance of each of these attacks extensively in the DL environment. Comparative studies are done on machine learning model too, to check the performance of the algorithm in each of these technologies.

Discussion

Diabetic retinopathy can be detected with digital fundus imaging. The majority of clinical guidelines recommend that diabetic patients, specifically those with mild to moderate retinopathy, undergo this test throughout their lifetimes. As the number of patients is on a massive rise, the proper patient examination has become a cumbersome and contentious job for medical professionals. On the other hand, to give rapid and effective classification results, automated diagnosis systems for DR detection using fundus images have been identified.

This study comprises various image processing, DL, ML, based computer-based methods for DR screening. Moreover, all the features such as classification, segmentation, and lesion-based feature are taken into consideration for the proper diagnosis of the disease. Moreover, some aspects are mentioned from the development perspective of DR-based CAD approaches. Some of these approaches are not much reliable because of certain limitations like, as the cases are increasing it is necessary to know the level of severity so that the patient is getting treated accordingly but some of the studies are restricted to binary classification. Furthermore, most of the research used out-of-date ML and image processing methods based on enormous numbers of images, with no quantitative measures. Therefore, it has been observed that certain factors should be kept in mind while designing the models like the model should be able to categorise all five DR phases, the selection of dataset should be appropriate, the data should be well pre-processed before using it for classification, DME (diabetic macular edoema) is frequently examined because it is the most common cause of eyesight loss in diabetics.

Juxtaposition of traditional and deep learning techniques

The steps explain the differences between deep learning techniques and standard approaches.

The traditional approaches require manual extraction of features from the image by an expert which is not considered in DL based approach thus removing an overhead.

DL requires heavy computational resources as there are major advancements made in GPU capabilities & other related computational resources.

Deep learning networks’ skill to generalize can be enhanced by improving their size by growing each layer with a certain number of levels and units.

Moreover, Table 12 is used to justify the comparison between these two approaches. The table clearly shows that the deep learning-based approaches outperform the traditional state-of-the-art approach. The method used for comparison is OD OC, MAs, Exudates. The same performance metrics and datasets are analyzed for the juxtaposition of these approaches.

Table 12

Traditional and DL methods comparison based on their retinal features

Approach	Method	Feature	Database	Performance (%)
Traditional Approach	Morphological operations, local coarse segmentation [127]	Retinal Vessel	DRIVE	Accuracy = 87
Deep Learning	Spatial Attention U-Net [66]			Accuracy = 96.98
Traditional Approach	Ensemble Classifier of Bootstrapped Decision Trees [54]	Exudate	DIARETDB1	Accuracy = 87.72
Deep Learning	CNN, ResNet [93]			Accuracy = 98
Traditional Approach	Directional Local Contrast (DLC) [36]	Microaneurysms	DIARETDB1	AUC = 86
Deep Learning	Stacked Sparse Autoencoder (SSAE) [165]			AUC = 96.2
Traditional Approach	Random Forest [35]	Retinal Hemorrhages	DIARETDB0 DIARETDB1 MESSIDOR, HRF, Teleopthaa	Accuracy = 93.58
Deep Learning	3D CNN [115]			Accuracy = 97.71
Traditional Approach	In-painting Region growing Spline interpolation [162]	Optic Disc	MESSIDOR	Accuracy = 89
Deep Learning	Cascade Classifiers, CNN [6]			Accuracy = 99.20

Deep learning methodologies

Deep learning is essential for enhancing medical imaging performance. Deep learning advances have ushered in a new era in clinical imaging. It has piqued the interest of academics and researchers involved in designing medical imaging technologies based on historical data. However, combining deep learning and medical imaging poses numerous obstacles which should be overcome for better results. The key areas where deep learning faces obstacles are massive datasets for training, overfitting of data, fluctuations in input, data quality, context understanding, and enormous volume of data. Figs. 13 and 14 shows the complete evaluation of traditional and deep learning techniques by taking different performance measure into consideration.

Fig. 13

Blood Vessel Segmentation performance in traditional and DL-based methods using DRIVE and CHASE Datasets

Fig. 14

Performance of Lesion Based Techniques in traditional and DL-based methods using DIARETDB1 Dataset

Conclusion and future directions

Diabetic retinopathy is one of the major causes of vision loss. If not treated initially will change its stage from mild to severe DR thus leads to blindness. Therefore, in this survey, various strategies for detecting diabetic retinopathy are elaborated. The strategies can be classification-based (binary/multiclass), lesion-based (MAs hemorrhages, exudates), segmentation-based (OD/OC segmentation, blood vessel segmentation). Then the overview of different publicly and privately available datasets is discussed along with certain evaluation metrics. The comparison of traditional and DL approaches is done showing how DL methods outperform the state-of-art methods along with some challenges that deep learning faces while designing the models. This paper provides a complete overview of current state-of-the-art deep-learning-based algorithms for DR diagnosis, which will aid researchers in conducting additional research on the subject.

Although, deep learning assists to build improved procedures for DR detection and advanced state-of-the-art procedures forward, it’s indeed an unsolved problem that necessitates further investigation. There are few DL-based techniques, and advanced DL approaches that must be created to overcome this challenge. DL models are.

essentially black boxes that don’t lay out critical value interpretations that could confirm their utility in a real-world environment. The majority of the approaches in this evaluation offer no interpretation of their results. Moreover, certain aspects of this field require refinement, such as extraction of blood vessels is a challenging task, the ONH structure varies by subject, determining the OD boundary, as it has blurred edges. As a result, there is no single technique that can solve all of these issues. More efficient approaches for detecting DR-related retinal changes and structure are also required. Therefore, the future directions suggest that the model is required to design in such a way that it gives many accurate results, should be comparatively less expensive, and be able to overcome most of the challenges of the DL methods.