Artificial neural network in diagnostic cytology.

The artificial neural network (ANN) is a computer software design or model that simulates the biological neural network of the human brain. Instead of biological neurons, ANN is composed of many layers of nodes that carry the signal and process it to make the final decision. ANN is a modern technology that is widely used in different fields of science. The ANN is reshaping the medical system and the various areas of pathology. In this paper, the basic concept and applications of ANN in cytology have been discussed. In this paper, the various articles published on ANN in the field of cytology have been systemically reviewed. The ANN is relatively less used in cytology. After introducing convolutional neural network and whole slide scanners in the commercial market, it is now essential to have thorough knowledge in this field to start diagnostic application of ANN.

INTRODUCTION

The human brain is a highly complex organ and can interpret the most challenging visual images in a fraction of a second. The experienced cytologist often instantly recognizes the problematic cases of cytology. The ordinary computer can do intricate works under human guidance. However, a computer cannot decide on the difficult cases unless it is programmed to do the job. The human brain has specific essential characteristics, such as (a) it is highly nonlinear and can process parallel data, (b) it has learning capability, (c) it is adaptable to a changing environment, and (d) it can decide with confidence.

Moreover, the human brain is robust, and it does not stop working just because a few cells die. At the same time, the ordinary computer does not possess any such qualities. Here lies the importance of artificial neural network (ANN). The ANN is a computer software design that can simulate the human brain. ANN can perform many complicated and challenging functions.[

ANN has learning capacities, and therefore ANN can solve many complex nonlinear problems. In the last few decades, ANN is reshaping the medical system and the various fields of pathology. However, till now, there is little impact of ANN in cytology, and only a few scattered works are available in the literature.[

After introducing whole slide imaging (WSI), it is expected that ANN will also be introduced in the field of cytology in the routine laboratory within a few years. The present review discusses the basic concept, applications of ANN, and the future of WSI in cytology.

ANN

The ANN is a computer software design or model that simulates the biological neural network of the human brain. Instead of biological neurons, ANN comprises many layers of nodes that carry the signal and process it to make the final decision. The system can modify the internal structure depending on the particular situation.

The human brain comprises biological neurons, whereas the ANN is made of nodes [Figure 1].

Figure 1:

Schematic diagram comparing the biological neuron and the artificial neuron.

Biological neuron

The biological neuron has three main parts, (a) dendrites, (b) the cell body, and (c) axon. The dendrites of the neuronal body receive signals (input signals) through multiple axons of the other neurons through the synaptic gaps. The cell body processes the signals obtained by all the dendrites, and if the summation of all the signals exceeds the threshold value, it fires the next neuron through the axons.

Artificial neuron

In an ANN, there are multiple nodes instead of neurons. The node receives the input signals that simulate the signals of the dendrites of the biological neurons. The node processes the input signals considering the weightage of the individual input signal (w) and the intensity of the signal (i). Finally, the node activates the next neuron depending on the threshold value [Figure 2].

Figure 2:

The mathematical model of the artificial neural network. W indicates weightage of each input signal, and X means the intensity of the signal. When the summation of all the signal exceeds the threshold value, then it fires the next node. It is also known as activation.

The activation or firing of the next neuron commonly occurs by the sigmoid neuron unit function. It is the most commonly used nonlinear activation function. Here, the threshold of activation of the next neuron is a smooth transition like a sigmoid graph. It prevents the jump of the output value to fire the next neuron. The other popular nonlinear activation functions are hyperbolic tangent and “rectified linear unit.”

LEARNING OF THE ANN MODEL

The most crucial part of ANN is learning to do a specific function. Whenever the ANN learns to solve a particular problem, the strength of the interneuronal connection changes. The alteration of the neuronal connection is preserved as specific weightage in between those two neuronal connections. Therefore, the learning of ANN means changing the weightage of the connection. The learning is a dynamic process of (a) evaluating the output, (b) adapting weights, and (c) taking the new inputs. The learning of ANN may be of three main types as described below:

Unsupervised learning

In the case of unsupervised learning, there is no set of training data and no helping or supervision from the outside. Hence, there is no available information on the desired output. The “unsupervised learning” is used mainly in data clustering or reduce dimensionality compression.

Supervised learning: In “supervised learning,” the classified, labeled data are fed in the ANN system along with well-defined output. So each time, ANN receives direct feedback. The “supervised learning” is applied for the classification and regression, and prediction of disease outcome or future.

Reinforcement learning: Reinforcement learning is a relatively slow learning process due to “randomness” of weight distribution. Here, weight is distributed randomly to the neuronal connection, and each correct output is rewarded. It is just like a “teacher-training system” where the teacher scores the right action of the student.

The back-propagation model: It is one of the commonly used supervised learning algorithms. Multiple layers of nodes are present in the “back-propagation model.” At first, the nodes of the input layer receive the signals from the external environment. These signals are transferred to the nodes of the hidden layers. Ultimately, the signal is processed in the nodes of the output layer, and the final decision comes out from the system [Figure 3]. The hidden layer is also known as the “black box” because we cannot control the functions of this layer, and most of the nonlinear problems are solved in the black box. In each iteration of ANN, the error is calculated as the desired output (D) minus the actual output (Y). Depending on the size of the error, the weight in between the neuron is altered (Δ Wi). Therefore, the in case of input Ii, desired output D, actual output Y, change of weight Δ Wi, learning rate η, perceptron learning rule is Δ Wi = η * (D-Y).Ii

Figure 3:

Schematic diagram of the back propagation model. The model consists of multiple layers of input, hidden, and output nodes. The error is calculated as the difference between the desired output (d) and the actual output (y). Depending on the size of the error, the weight in between the neuron is altered (Δ Wi). The input = Ii, desired output = D, actual output = Y, change of weight = Δ Wi, learning rate = η.

THE DIFFERENT TYPES OF THE ANN

The various ANN models are available such as feed-forward neural network, radial basis neural network, recurrent neural network, convolution neural network (CNN), modular neural network, and back-propagation network.

CNN

At present, CNN is the most widely used ANN model. CNN is a type of deep neural network.[ It is a best-suited network for image recognition and classification.[ In CNN, we do not use the individual pixels of the whole image. Instead, a patch of the image is used as an input node. These patches of nodes again are transmitted to the next layers as another small set of patches. Here, the special mathematical operation “convolution” is used. A filter consists of a much smaller matrix (3 × 3–9 × 9) are used. The filter is moved over the images, and the 3 × 3 filter makes the 3 × 3 pixel image area in a single-pixel value. By the repeated convolution operation, the whole image is made as a simplified feature map. It is then used as an input layer [Figure 4].

Figure 4:

Schematic diagram of convolutional neural network.

APPLICATIONS OF ANN IN CYTOLOGY

Gynecological lesions

Cervical smear

ANN was 1st time used in cervical smear cytology by PAPNET system. PAPNET successfully identified the atypical cells in the cervical smear.[ It was one of the most commercially successful ANN systems in the field of cytology. A meta-analytical study[ showed that PAPNET detected a higher number of positive cases (>20%) than manual screening. The PAPNET system also identifies 33% abnormal cases in the false-negative cases of manually screened cervical cytology smear. PAPNET is predominantly semi-automated screening. Regarding primary screening by PAPNET, the studies showed that PAPNET is equally effective as primary screening in comparison to manual screening by conventional cytology.[

At present, the PAPNET system is out of the market.

Recently Sanyal et al. showed the successful application of CNN in cervical cytology to distinguish a different set of the squamous intraepithelial lesion.[ They considered that CNN is a promising tool in the screening of liquid-based cytology smear.

In an extensive review of machine learning, William et al.[ described that most neural networks have a 93.78% accuracy rate.

Non-gynecological lesions

Fine-needle aspiration cytology (FNAC)

Breast

Various workers have built ANN of breast lesions since 1991 [Table 1]. Dawson et al.[ applied Bayesian analysis and ANN to differentiate high and low-grade breast carcinomas. They used nuclear morphometry and nuclear textural features of breast tumors and successfully identified 70% of low-grade breast carcinoma.

Table 1:

Neural network of the breast.

Author and year	Study design	Number of cases	Comments
Dawson et al.[17]1991	Nuclear morphometry and textural features of both benign and carcinoma of the breast	Total of 35 breast carcinomas	They used both Bayesian analysis and ANN. They correctly grade low-grade carcinomas than high-grade carcinoma
Ravdin et al.[18]1992	Eight input layer, four hidden layers and one output layer (8-4-1). The eight input nodes were labeled as estrogen and progesterone receptor status, DNA index, S-phase fraction, tumor size, and the number of axillary lymph nodes involvement, patient’s age, and follow-up of the percentage of cases ANN was used to predict the clinical outcome of breast cancer patients	Total of 1008 breast carcinomas	ANN successfully identified low and high-risk patients
Markopoulos et al.[19]1997	Learning vector quantization (LVQ) neural network was used to discriminate benign and malignant breast tumors	Total of 100 case: (68 carcinomas and 32 benign lesions)	Back propagation ANN can identify 87.41% of the cells
Ohno-Machado et al.[20]1998	ANN and logistic regression analysis were done based on nine pathological features	Total of 460 carcinoma cases	Both ANN and logistic regression analysis showed equal performance. However, the weightage of individual features were different in these two systems
Einstein et al.[21]1998	Three-dimensional chromatin structure by fractal dimension was measured to apply in ANN to discriminate benign and malignant cases	Total of 19 benign 22 carcinomas	Neural network model correctly classified all the case
Dey et al.[4]2013	Both the qualitative cytological features and objective morphometric data were collected and applied in the back propagation ANN. A total of 34 input layers, 17 hidden layers and three output layers (34-17-3) network was designed	Sixty-four cases of histology proven breast lesions consisting of 20 fibroadenomas, 28 infiltrating ductal carcinomas, and 16 infiltrating lobular carcinomas	ANN program successfully classified all the cases of benign, and ILC cases and six of seven IDC cases
Subbaiah et al.[5]2014	The network was designed as s 41-20-1 (41 input nodes, 20 hidden nodes, 1 output node). Cytological features along with nuclear morphomeric, densitometric, and GLCM features were used as input nodes	Total of 112 cases. Fibroadenomas 52 cases and 60 cases of IDC	ANN model correctly identified all cases of fibroadenomas and infiltrating carcinomas in the test set

ANN: Artificial neural network

Ravdin et al.[ successfully differentiate low and high-risk breast carcinomas with the help of ANN. The ANN model was designed as eight input layers, four hidden layers, and one output layer (8-4-1). The eight input nodes were labeled as Hormone receptor status (estrogen and progesterone), DNA index, synthetic-phase fraction, tumor size, and the number of axillary lymph nodes involvement, patient’s age, and follow-up of the percentage of cases that have relapsed. They compared the results of ANN with the Cox Regression Modeling and noted that ANN was equally helpful to predict the high- and low-risk group of patients.

Markopoulos et al.[ correctly identified 87.41% of the cells of breast tumors with the help of back propagation (BP) ANN. They emphasized that ANN and image morphometric data have great potential in diagnosing breast carcinoma in FNAC material.

In 1998, Ohno-Machado et al.[ applied both the ANN model and logistic regression analysis in 460 cases of breast lesions to classify benign and malignant lesions. Both ANN and logistic regression analysis showed equal performance. However, the weightage of individual features was different in these two systems. In the same year, Einstein et al.[ made an ANN model based on the spectral fractal dimension of the nuclei of the cells. They observed a significant difference in the fractal properties of benign and malignant nuclei and commented that ANN can utilize these features to classify the tumors.

Dey et al.,[ in the year 2013, built an ANN model in breast carcinomas based on the cytological features and nuclear morphometric data. The ANN model was made as 34-17-3 (34 input layers, 17 hidden layers, and three output layers). The ANN was trained with the help of BP online training until the network error comes to 0.076482. They used a logistic function for the network activation. ANN was successful in classifying both ductal and lobular carcinomas of the breast. After 1 year the same group[ used nuclear textural data, nuclear optical density, and morphometry to build an ANN to differentiate benign and malignant breast lesions on FNAC material.

THYROID

Several researchers tried to make an ANN from cytology material of thyroid lesions [Table 2].[ In 2004, Ippolito et al.[ made a feed-forward ANN from the cytologic and clinical data to detect the benign and malignant thyroid nodules. They noted that ANN was highly successful in this matter. Cochand-Priollet et al.[ in 2004 used an image analyzer to extract 25 nuclear morphometric features. It was noted that only four morphometric features were essential to differentiate benign from malignant thyroid lesions. They incorporated those features in the ANN model and successfully differentiated benign from malignant thyroid lesions.

Table 2:

Neural network of the thyroid.

Author and year	Number of cases	Study design	Comments
Ippolito et al.[22]2004	453 cases	A Feed-forward artificial neural network with 15-15-1 (input-hidden- output) design. The cytological features and clinical data were used as input nodes	ANN model can discriminate with higher sensitivity and specificity between benign and malignant nodules
Cochand-Priollet et al.[23]2005	157 cases	The nuclear morphometric features were extracted from the nuclei and four important features were selected as input( roundness factor, standard deviation of the histogram, maximum value of the cooccurrence matrix, and mean value of the differences in histogram)The ANN model was used to distinguish benign and malignant thyroid tumors	ANN successfully discriminated benign and malignant lesions of thyroid
Shapiro et al.[24]2007	197 cases	The cytologic features, nuclear morphometric data and chromatin texture analysis data were used to make the ANN model that can distinguish follicular adenoma and carcinoma	In 90% cases, ANN successfully identified the different types of follicular tumor
Varlatzidou et al.[25]2011	335 cases	Size, shape, and texture of nuclei were used to make an ANN model to distinguish benign and malignant lesions of thyroid	ANN correctly identified the benign and malignant lesions
Savala et al.[7]2018	57 cases	A back propagation ANN was designed as 31-5-1 (31 Input nodes. 5 Hidden nodes and 1 Output node)The cytological features and nuclear morphometric features were used as input nodes. ANN model was applied to distinguish follicular adenoma and carcinoma	ANN model successfully distinguished all cases of FA and FC
Sanyal et al.[26]2018	87 cases	Convolutional neural network model was applied on the microphotographs of papillary carcinoma of the thyroid, and other non-papillary thyroid	The model showed good 90.48% sensitivity, and 83.33% specificity

Shapiro et al.,[ in the year 2007, attempted to differentiate follicular adenoma from follicular carcinoma of the thyroid on cytology material. They developed an ANN model based on the cytological features, nuclear morphometric data, and chromatin texture data of thyroid follicular cells. In 90% of cases, ANN successfully classified follicular tumors.

Varlatzidou et al.,[ in 2011, applied to learn vector quantizer (LVQ) ANN to differentiate benign and malignant thyroid tumors. They studied 335 FNAC cases and extracted the morphometric data and texture of 100 nuclei per case to make an ANN model. The ANN successfully identified the benign and malignant cells both at the individual cellular level and in case-based level.

In 2018, Savala et al.[ built an ANN model and successfully differentiated follicular adenoma and carcinoma on FNAC smear. They designed the model as 31-5-1 (Input nodes- Hidden nodes-Output node). The cytological features, along with morphometric data of thyroid follicular cells, were used for the input nodes. In this ANN model, the hyperbolic function was employed to activate the input nodes, and logistic was used in the output activation function. They used the “online back propagation method” to train the ANN model. At least 500 iterations were done to keep as the error as low as 0.000792.

ANN model successfully distinguished all cases of follicular adenoma and carcinomas.

In 2018, Sanyal et al.[ applied CNN to distinguish papillary from the non-papillary carcinoma of the thyroid. They used the microphotographs of papillary carcinoma of the thyroid, and other non-papillary thyroid lesions. The cases of papillary carcinomas were identified with high sensitivity (90.48%) and moderate specificity (83.33%). Interestingly, the ANN in this study failed to identify those cases of nonpapillary lesions with vague papillary pattern.

GASTROINTESTINAL TRACT LESIONS

ANN was widely used in gastrointestinal tract lesions [Table 3].[ In 1996, Karakitsos et al.[ classified benign and malignant gastric epithelial cells with the help of a BP neural network. Both morphometric and textural features of nuclei were used to make the ANN model. The ANN model was not applied to identify the individual cases in this study. Subsequently, Koss et al.[ and Levine et al.[ used PAPNET system in esophageal and oral smears to screen carcinoma cells. Both these studies emphasized the utility of PAPNET system in the screening of malignant cells. Later on, in the year 2008, Lai et al.[ built an ANN model based on the clinical information, pathological findings, and genetic polymorphisms of candidate genes data to predict tumor staging. The ANN model had an accuracy of 81.82%.

Table 3:

Neural network of the gastrointestinal tract.

Author and year	Number of cases	Study design	Comments
Karakitsos et al.[27]1996	23 cases of cancer, 19 of gastritis and 58 of ulcer	Morphometric and textural features of nuclei were used to make an ANN model	ANN correctly classified 97.6% of benign cells and 95% of malignant cells
Koss et al.[28]1998	138 esophageal smears	PAPNET system was used	PAPNET identified many abnormal cells. It can be used as screening
Levine et al.[29]1998	62 oral smears	PAPNET system was used	PAPNET screening methods correctly diagnosed squamous cell carcinoma in 61% of patients
Lai et al.[30]2008	121 cases of gastric carcinomas	Clinical data and pathological findings were collected, and genetic polymorphisms of candidate genes data were used to build an ANN to predict tumor staging	ANN had an accuracy of 81.82%
Momeni-Boroujeni et al.[31]2017	75 cases of pancreatic lesions (20 malignant, 24 benign, and 31 atypical)	A multi-layered perceptron was made on the basis of morphometric input data	The model was 100% accurate in unequivocal benign and malignant cases. However, it was 77% accurate in atypical cases

ANN: Artificial neural network

EXFOLIATIVE CYTOLOGY

Urine cytology

Table 4 summarizes different ANN models on urinary cytology.[ Pantazopoulos et al.[ in the year 1997 built an ANN model from the images of the urinary epithelial cells of 465 cases. They made two types of neural network: BP type and LVQ mode. The nuclear sizes (five descriptors), shape (five descriptors), densitometric feature (one descriptor), and textural features (13 descriptors) were used as input nodes. About 97% cases of urothelial cell carcinoma (UCC) were diagnosed correctly classified by both the BP and LVQ.

Table 4:

Neural network of the urine cytology.

Author and year	Number of cases	Study design	Comments
Pantazopoulos et al.1997	UCC 255 case, 210 benign cases	Lower urinary tract. The data from the images of the cells were used to make an ANN	About 97% cases were diagnosed correctly
Vriesema et al.[33]2000	85 cases consisting of benign, low grade and high grade UCC	The digitized cell images of the slides of bladder wash cytology were used for neural network analysis	The ANN based technology was able to diagnose successfully the benign and malignant cases
Muralidaran et al.[6]2015	115 cases; 59 UCC and 56 benign cases	Back propagation ANN model was designed as 17-11-3 (17 input nodes, 11 hidden nodes and three output nodes). Nuclear area, diameter, perimeter, standard deviation of nuclear area, and integrated gray density along with detailed cytological features were used as input nodes	all the benign and malignant cases were diagnosed correctly. However, one of the low grade cases was diagnosed as high grade UCC by ANN model

UCC: Urothelial cell carcinoma, ANN: Artificial neural network

The ANN correctly diagnosed 97% of cases. Later on, Vriesema et al.[ applied PAPNET system to detect malignant urothelial cells from the bladder wash cytology. Muralidaran et al.,[ in the year 2015, made an ANN model to distinguish benign and malignant urothelial cells in urine cytology smear. They designed the BP ANN model as 17-11-3 (input nodes- hidden nodes -output nodes). The nuclear morphometric data, integrated grey density along with cytological findings were used as input nodes. The three output nodes were labeled as the “no malignancy,” low-grade and high-grade UCC. The supervised learning technique was used to train the ANN model. All the benign and malignant cases were diagnosed correctly by ANN.

Effusion cytology

Relatively little works were done on ANN of effusion cytology. Table 5 highlights the different ANN models on effusion cytology. Truong et al.[ in 1995 built an ANN model based on the densitometric and morphometric data of the cells of effusion cytology smear of 112 cases. The ANN model had total seven inputs: Five nuclear morphometry data and two densitometry data. There were ten hidden layers and one output neuron. The ANN had 95.3%, a sensitivity of 85.7%, specificity.

Table 5:

Neural network of the effusion cytology.

Author and year	Number of cases	Study design	Comments
Truong et al.[34]1995	112 cases	ANN model was made based on the densitometric and morphometric data of the cells	ANN showed 95.3%, sensitivity of 85.7%, specificity
Barwad et al.[35]2012	114 cases	ANN model was designed as 25-2-1 (25 input nodes-2 hidden nodes-1 output node). Cytological features, image morphometric data were used to make an input nodes	ANN identified correctly all the malignant cases

ANN: rtificial neural network

Barwad et al.[ in 2012 applied ANN on effusion cytology smears of 114 cases based on the cytological features, and image morphometric data. The ANN model was designed as 25-2-1 (input nodes- hidden nodes-output node). The BP type of supervised learning was used for the training of the network. ANN correctly identified all the benign and malignant cases in effusion cytology.

DISCUSSION

WSI is a significant development in digital pathology.[ The WSI preserves all the complexities of the image and color information. In addition, the digitized images are also available in several z-stack levels. Recently WSI has been approved by FDA for the primary diagnosis in histopathology sections,[ and it is a significant change in the pathology reporting system. In future, we expect that there will be the wide-scale applications of WSI in the cytology smear also. The digitized image of the entire section/smear in WSI may help to take the whole smear for analysis instead of a patchy selection of the field. It will help to build a more robust ANN. It is high time to apply ANN model in WSI images of the cytology cases.

CNN is a promising area in artificial intelligence in the cytology image. With the help of CNN, the entire image can be flattened to less than 100 pixels. The computer system can easily handle to make ANN model from the reduced number of pixels. Moreover, the CNN can extract data without bias, and the feature extraction does not need any expert pathologist. Hence, the cytologist without any knowledge of morphometry can handle CNN. Unfortunately, only a handful of published studies are available in this area.[ It is expected that CNN will bring revolutionary effects in cytology.

Computational pathology

In future, there will be artificial intelligence-based computational pathology that will incorporate the digitized WSI, the data from the laboratory information system including clinical features, hospital data incorporation containing biomarkers, and genomic data [Figure 5]. The ANN system will analyze these data and help in the diagnosis, classification, grading, predicting prognosis, and individual management of each patient.[ These integrated data will help the clinicians to understand the disease progression in different stages of the tumor. It will provide high-quality service to the patient. In future, the local laboratory data may be available in the global network system, and a large database can be created. With the help of ANN, the individual data of the patients will be interpreted. The global data-based approach will change the cytology and pathology reporting system.[

Figure 5:

Schematic diagram on future reporting system of cytology and artificial neural network.

Implementation of ANN in the laboratory: We need a team approach to implement ANN in the laboratory. The team will consist of the data scientist, engineer, and cytologist (pathologist). The data scientist will work on algorithm design and architecture, whereas the computer engineer will look over the construction of physical environments and maintaining hardware. The role of the cytologist in this team will be (1) the formulation of the medical questions and clinical applications, (2) evaluation of the system, and (3) further development of the algorithm.[

CONCLUSION

The implementation of ANN in the regular reporting is not the smooth way. There are several limitations of ANN. The hidden layer, also known as the black box, is responsible for the nonlinear function and to decide on complex problems. However, we do not understand how the nodes of the hidden layer work, nor do we have any control over these nodes. The black box of ANN is responsible for the nonlinear function that takes the decision in complex problems.

The speed of the computer and available disk space is still a challenge to us. A single WSI image in cytology smear usually takes 2–3-gigabyte space. Even if we use CNN, the disk space and speed of the computer may be problematic.

To make a successful and robust ANN model, we need a large amount of data set and images, which is difficult for a single institution. Hence, the interinstitutional collaboration maybe need in such case.

Another critical aspect of ANN is the medicolegal issue. In case of an error in the interpretation by ANN, who should be responsible? We cannot blame the machine in case of failure. Moreover, data safety is also important. Mainly the whole data and information should not go to the patient party or the commercial vendor.

In brief, ANN is the most promising area in recent times. The successful implementation of ANN in cytology may improve the diagnosis, classification, and grading of the diseases. In future, the incorporation of the clinical, biochemical, radiological, immunocytochemistry, and genetic data in the ANN system will help in the individual management of the patient. At present, it is high time to know basics and also the future prospects of ANN.