Diagnosing thyroid disorders: Comparison of logistic regression and neural network models.

BACKGROUND: The main goal of this study was to diagnose the two most common thyroid disorders, namely, hyperthyroidism and hypothyroidism, based on multinomial logistic regression and neural network models. In addition, the study evaluated the predictive ability of laboratory tests against the individual clinical symptoms score.
MATERIALS AND METHODS: In this study, the data from patients with thyroid dysfunction who referred to Imam Khomeini Clinic and Shahid Beheshti Hospital in Hamadan were collected. The data contained 310 subjects in one of three classes-euthyroid, hyperthyroidism, and hypothyroidism. Collected variables included demographics and symptoms of hypothyroidism and hyperthyroidism, as well as laboratory tests. To compare the predictive ability of the clinical signs and laboratory tests, different multinomial logistic regression and neural network models were fitted to the data. These models were compared in terms of the mean of the accuracy and area under the curve (AUC).
RESULTS: The results showed better performance of neural network model than multinomial logistic regression in all cases. The best predictive performance for logistic regression (with a mean accuracy of 91.4%) and neural network models (with a mean accuracy of 96.3%) was when all variables were included in the model. In addition, the predictive performance of two models based on symptomatic variables was superior to laboratory variables.
CONCLUSIONS: Both neural network and logistic regression models have a high predictive ability to diagnose thyroid disorder, although neural network performance is better than logistic regression. In addition, as achieving less error prediction model has always been a matter of concern for researchers in the field of disease diagnosis, predictive nonparametric techniques, such as neural networks, provide new opportunities to obtain more accurate predictions in the field of medical research. Copyright:

Introduction

The thyroid gland, as an important part of the human endocrine system, stabilizes the thyroid hormones to maintain the body's metabolism. The two most common thyroid disorders or thyroid diseases are hypothyroidism and hyperthyroidism.[12]

Specialists usually diagnose thyroid disorders thorough medical history; physical examination; and laboratory tests, including thyroid stimulating hormone (TSH), total thyroxine (TT4), and total triiodothyronine (TT3). However, TSH usually is the most definitive test done.[34] Diagnosis of thyroid disorders is difficult in groups that are taking other medications or patients with concomitant medical conditions.[5]

Thyroid disorders are relatively prevalent in the general population, but because of their milder symptoms and slow progression, they are usually overlooked.[6] Failure in diagnosis and correct management of these disorders may cause serious complications. For instance, hypothyroid patients are at an increased risk of cardiovascular diseases (CVDs) and hyperthyroidism is associated with osteoporosis.[7] At present, thyroid disorders are considered types of chronic diseases, which have influenced many people around the world. Therefore, timely and proper diagnosis and treatment of these patients and their future follow-ups play an important role in the prevention and reduction of their complications, and as a result, it can reduce morbidity and mortality related to these disorders.[78]

It is worth mentioning that the prevalence of thyroid disorders is increasing in Iran, as well as throughout the world, and considering the irreversible damages caused by these disorders to organs like heart, eyes, kidneys, etc., as well as the costs imposed on health care systems of governments and societies, timely diagnosis and treatment of these diseases is essential.[8]

Early diagnosis of many diseases is crucial in treating them. In this regard, determining the factors affecting the disease has an important role in preventing the disease. Using appropriate statistical models and accurate estimation methods along with clinical diagnosis can be effective in determining the correctness of these factors.

Prediction and estimation coupled with the medical diagnosis have a special and substantial role in statistical methods.[9] These estimation models, which are based on information gathered from evidence, are the targets of modeling and classification. These classification models are handy tools that can assist physicians in the proper diagnosis of thyroid disorders in a well-timed and more efficient manner. This is especially important for health care providers with scarce diagnostic resources. In recent years, outcome prediction models using artificial neural network and logistic regression are mainly used in areas like medical, dental, clinical epidemiology, and health services research for identifying related factor and classification of diseases.[910111213]

Moreover, many studies have only used laboratory variables for the diagnosis of thyroid disorder.[814] As far as we know, even in limited studies that use both sets of laboratory tests and some of the symptoms variables, the prediction performance of these two sets has not been evaluated.

The main goal of this study was to diagnose the two most common thyroid disorders— hyperthyroidism and hypothyroidism—and also evaluate the predictive ability of laboratory tests against individual clinical symptoms. For this purpose, the neural networks and multinomial logistic models have been used.

Method

Multinomial logistic regression

Multinomial logistic regression is used for a nominal dependent variable with more than two levels. In multinomial logistic regression, given the multinomial class variable (y) with j categories and p-dimensional predictor variables (x), forecasts whether a future data point y* observed at the predictor (x*) will belong to which class variable. The probability of belonging to the category j for a given person is expressed by using the following equation:

The unknown parameters of multinomial logistic regression are typically jointly estimated by maximum a posteriori (MAP) estimation, which is an extension of maximum likelihood.[15] This study was approved by the Ethics Committee of Hamadan University of Medical Sciences with IR.UMSHA.REC.1396.242 codes.

Neural networks

Artificial neural network, which originates from biological neural networks, is a machine learning technique widely used in the field of regression and classification problems. The learning process in the neural network method is learning through examples. A multilayer feed-forward network has three types of layers, namely, the input, output, and hidden, which is intermediate between the input and output layers. The number of hidden layers is usually determined with the cross-validation method. Each layer consists of neurons. The neurons in the two adjacent layers are fully connected with respective weights, while the neurons within the same layer are not connected. In the feed-forward neural network, information moves only one direction in the forward direction. In fact, information flows through the input node (neurons) and passes through the hidden layers (if any) to the output nodes. In neural networks, complex nonlinear mappings between input and output variables are learned through activation functions. The most common activation function for multiple class prediction is the softmax function.[16]

Model evaluation

To investigate the predictive performance of models mean accuracy and area under the receiver operating characteristic (ROC) curve (AUC) is used. AUC is a measure of model discrimination (i.e. how well the model separates subjects who did and did not experience an event). In the current study, to obtain comparable results, the same training and testing parts were used for different models. All analyses were implemented in R3.5.1 open-source statistical software.

Material

During a 6-month period, patients with thyroid dysfunctions, who were referred to Imam Khomeini Clinic and Shahid Beheshti Hospital in Hamadan were enrolled and examined. At the same time, a special questionnaire designed by two endocrinologists was filled for each subject after the interview with the respective physician or a trained research student. Written informed consent was obtained from all subjects participating in the study. The study took approval from the respective research ethics committee.

Final data included 310 subjects that were categorized into three respective groups—normal, hyperthyroidism, and hypothyroidism. Hundred patients were included in the normal, 55 in hyperthyroidism, and 155 in hypothyroidism groups.

Collected variables included demographic variables, variables associated with symptoms of hypothyroidism (fatigue, sleepiness, constipation, feeling cold, fluid retention, weight gain, reduce appetite, and menstrual irregularities [women]) and hyperthyroidism (increased appetite, diarrhea, fast heart rate, intolerance for heat, increased sweating, and menstrual irregularities [women]), as well as variables related to laboratory tests.

The hypo (hypothyroidism) and hyper (hyperthyroidism) score variables are defined as follows. If the person has the desired symptom, the value is 1, and if the symptom does not exist, the value is 0. Hypo and hyper score variables are defined as the sum of the symptoms for each individual. Therefore, when the value of the hypo score is 3 for an individual, it means that the person has three signs.

In addition, to compare the predictive ability of the clinical signs and laboratory tests, different neural networks and multinomial logistic regression models were fitted to the data. Model 1 is a model in which all variables, including demographic variables, such as age, sex, family history, and body mass index (BMI); variables related to clinical symptoms (hypo and hyper score); and laboratory tests (TSH and TT4) have been used. Model 2 is a model that includes demographic variables and clinical symptoms, and Model 3 includes demographic variables and laboratory tests.

It is noted that to evaluate the predictive ability of the three models as described above, the data set was randomly divided into training (7/10 of the data included 217 samples) and test (3/10 of the data included 93 samples) sets. Different models (Model 1, 2, and 3) were applied to the same training and test sets. To obtain comparable results, the same training and testing parts were used for the three models. These models were compared in terms of the mean of the accuracy and AUC in test sets for each model.

The hyperbolic tangent function was used as a hidden layer activation function and softmax function as an output layer activation function. Different models based on the cross-validation method were investigated to obtain the optimal number of hidden layer neurons in each situation. Then, various neural networks models (Model 1, 2, and 3) based on the optimal number of hidden layer neurons were trained. The number of neurons in the input layer for each of the Models 1, 2, and 3 was considered eight, six, and six neurons, respectively.

Results

Figure 1 presents the frequency distribution and comparison of variables, including the mean age, BMI, TSH, TT4, and hypo/hyper score. Females constituted 93.1% hypothyroidism and 78.2% of hyperthyroidism groups. In addition, 78.1% of subjects in the healthy group were females.

Figure 1

Bar plot for comparison of demographic, laboratory, and symptomatic variables in the three groups (Error bar: +/−2 SE)

A positive history of thyroid disease was found in 36% of the patients in the hypothyroidism group compared with 52.7% in hyperthyroidism patients. However, in the healthy (normal) group, positive family history was present in 21% of subjects.

While comparing the mean age, BMI, TSH, TT4, and hypo and hyper scores between the groups, there was a statistically significant difference between the mean values of these variables. The mean value for the hyper score in the hypothyroidism group was 3.55 ± 1.53, hyperthyroidism group was 0.45 ± 0.52, and normal group was 0.05 ± 0.25. The mean value for the hypo score was 3.69 ± 1.68 in the hypothyroidism group, 0.65 ± 0.72 in the hyperthyroidism group, and 0.06 ± 0.23 in the normal group.

The results of the fitting of various logistic regression models (Model 1, 2, and 3) based on the total data (n = 305) are presented in Table 1. The results related to Model 1, which include all variables, the history of thyroid disease in the family increases the odds of developing a person with hyperthyroidism 18.86 (1/0.053) times more than healthy people do. In addition, for every one-unit increase in the TSH, the risk of hyperthyroidism increases 1.405 times versus the healthy person. In addition, every one-unit increase in the hyper score, increases the risk of hyperthyroidism 47.65 times versus a healthy person. With a one-unit increase in the TSH, the risk of hypothyroidism increases 1.491 times versus a healthy person. The results related to Model 2, which includes laboratory (TSH and TT4) and demographic variables (age, sex, history, and BMI), indicate that the sex variable in the hypothyroid patients compared with healthy people is statistically significant. Besides, the history, TSH, and TT4 are significantly different in both groups of hyper and hypo in comparison with the healthy. The results related to Model 3 include symptomatic (hypo and hyper score) and demographic variables (age, sex, history, and BMI), indicating that the history and hyper score variables in the hyperthyroid patients compared with healthy people are statistically significant. Also, hypo score is significant in the hypothyroid group in comparison to the healthy.

Table 1

Results for different multinomial logistic regression models based on total data

Variable		MODEL 1 (All Variable)			MODEL 2 (Laboratory Variables)			MODEL 3 (Symptomatic Variables)
		Beta	Ex (B)	Sig	Beta	Ex (B)	Sig	Beta	Ex (B)	Sig
Age	Hyper	0.010	1.010	0.808	−0.028	0.972	0.085	−0.017	0.983	0.649
	Hypo	0.009	1.009	0.784	−.005	0.995	0.684	−0.040	0.961	0.132
Sex*	Hyper	1.488	4.426	0.150	0.085	1.089	0.875	1.316	3.728	0.165
	Hypo	−2.498	0.193	0.082	−1.725	0.178	0.011	−2.423	0.089	0.80
History**	Hyper	−2.938	0.053	0.010	−0.941	0.390	0.031	−2.210	0.110	0.026
	Hypo	−1.462	0.232	0.164	−0.922	0.398	0.016	−0.855	0.425	0.276
BMI	Hyper	0.152	1.164	0.220	0.015	1.015	0.774	0.189	1.208	0.096
	Hypo	−0.089	0.915	0.481	0.040	1.041	0.333	0.051	1.052	0.592
TSH	Hyper	0.340	1.405	0.049	−0.371	0.690	0.014
	Hypo	0.400	1.491	0.009	0.181	1.199	0.000
TT4	Hyper	0.022	1.022	0.890	0.273	1.314	0.000
	Hypo	−0.227	0.797	0.121	−0.303	0.719	0.000
Hyper Score	Hyper	3.860	47.650	0.000				3.810	24.074	0.000
	Hypo	0.719	2.052	0.542				−0.733	0.462	0.497
Hypo Score	Hyper	1.389	4.011	0.153				1.051	2.861	0.191
	Hypo	4.148	63.317	0.000				4.106	60.696	0.000
Intercept	Hyper	−8.294		0.032	−1.778		0.238	−6.965		0.015
	Hypo	0.063		0.958	2.032		0.110	1.870		0.444

*The reference category is: Male. **The reference category is: No History. ***The reference category for grouping variable is: Healthy. BMI=Body mass index, TSH=Thyroid stimulating hormone, TT4=Total thyroxine, Hype=Hyperthyroidism, Hypo=Hypothyroidism

The structure of the Neural Networks Model based on all variables for train data present in Figure 2. In addition, variable importance plot for different Neural Networks Models for train data provides in Figure 3. The results show that for neural networks, including all variables, TSH, and hypo and hyper score, variables had the most normalized importance in predicting thyroid disorder.

Figure 2

Structure of the neural networks model based on all variables for train data

Figure 3

Variable Importance for different neural network models for train data

The prediction accuracy along with overall ACU for a different model based on the testsets is provided in Table 2. The results showed that for logistic regression and neural networks, both Model 1 and 2 worked well in thyroid disorder prediction; however, Model 3 had a weaker performance than these two models. The best predictive performance for logistic regression (with a mean accuracy of 91.4%) and neural network models (with a mean accuracy of 96.3%) was when all variables were included in the model.

Table 2

Comparison of the predictive performance of different logistic regression and neural networks models based on the testset

	Model	ACC	Mean AUC
All Variable	Multinomial logistic model	0.914	0.952
(Model 1)	Neural Networks	0.963	0.970
Laboratory	Multinomial logistic model	0.831	0.841
Variables (Model 2)	Neural Networks	0.887	0.894
Symptomatic	Multinomial logistic model	0.914	0.942
Variables (Model 3)	Neural Networks	0.925	0.963

AUC=Area under the curve

Discussion

Thyroid diseases are widespread worldwide. Making public awareness about the symptoms and types of the disease and recognizing it is essential. Accurate and rapid diagnosis of thyroid diseases, provide better medicines for patients, minimizing the death risk.[8] Therefore, using a precise model for predicting thyroid disorder can be helpful for young doctors to use it as a supplementary model for predicting thyroid. In this study, the classification of thyroid disorder using multinomial logistic regression and neural network models are considered.

The purpose of this study was to compare the predictive ability of clinical symptoms with laboratory variables. Using a set of clinical symptoms to diagnose thyroid disorder can be very beneficial. As it does not cost much and the person can easily check for some of the symptoms. The results indicate a significant accuracy for the model with the symptom's variables based on the logistic regression and neural network models.

A few studies have been done using a multinomial logistic regression model to identify the predictors and diagnosis of thyroid disorder.[17] The study has made an effort to identify the predictors of thyroid disorder by developing a multinomial logistic regression model.

Usage of thyroid disorders data from the University of California Irvine (UCI) Machine Learning Repository's web site for thyroid studies and the aim of classification, is common in the literature.[518] The final purpose of this research works was the increase in accuracy of diagnosis, which would be based on different classification models. We used a real data set for the prediction of thyroid diseases. There have been different researches with different approaches for the thyroid classification all based on diverse data mining techniques.[814]

For example, Ozyilmaz et al. in 2002 predicted the thyroid disease using various neural network methods such as multilayer perception with back-propagation method (MLP), radial basis function (RBF), and adaptive conic section function neural network (CSFNN), the classification accuracies are 88.3%, 81.69%, and 85.92%, respectively.[19]

Polat et al. in 2006 conducted the artificial immune recognition system (AIRS) for the diagnosis of thyroid disease, and an accuracy of 81% was obtained. Also, they used a hybrid method that combines AIRS with a developed fuzzy-weighted preprocessing, and obtained a classification accuracy of 85%.[20]

Keles et al. in 2008 using an expert system based on Neuro Fuzzy classifier diagnosis thyroid disease, and obtained an accuracy of 95.33%.[21]

Temurtas in 2009 proposed a Multi-Layer Perception with the Levenberg- Marquardt (LM) algorithm (MLP with LM) for diagnosis of thyroid disease, and the accuracy was 93.19%.[22]

Dogantekin in 2011 conducted a Generalized Discriminant Analysis (GDA) and the Wavelet Support Vector Machine method for diagnosis of thyroid disease was presented, and 91.86% classification accuracy was achieved.[23]

Chen et al. in 2011 presented a particle swarm optimization optimized support vector machines with a Fisher score CAD system for thyroid disease diagnosis and obtained 97.49% classification accuracy.[24]

Kousarrizi et al. in 2012 proceed with an experimental comparative Study on Thyroid Disease Diagnosis based on feature subset selection and classification.[25] The proposed method has two stages including feature selections sequential forward selection (SFS), sequential backward selection (SBS) and Genetic Algorithm as a pre-processing step that used as feature selection methods. In the second stage, SVM is used to classify thyroid data.[26]

Pandey et al. in 2015 investigate the Thyroid Classification using Ensemble Model with Feature Selection. In this study, an ensemble of C4.5 and Random forest gives accuracy 99.47%.[27]

Dewangan et al. in 2016 proposed the CART-Info Gain and CART Gain Ratio method for classification of Thyroid disease that gives 99.47% and 99.20% accuracy with 25 and 3 feature respectively in UCL Thyroid data set.[28]

Sajadi et al. in 2018, used a fuzzy rule-based expert system for diagnosis of hypothyroidism. The results showed that the designed fuzzy rule-based system works well with about 97% accuracy. In addition, the fuzzy classifier has a better performance than the logistic regression model, especially for the subclinical hypothyroidism class.[29]

In general, neural networks can be seen as an extension of logistic regression models. The most important advantage of neural networks over logistic regression models lies in the hidden layers. In fact, neural networks are useful when there are implicit interactions and complex relationships in the data, while logistic regression models are a better choice if the statistical inference is needed.

Although the results of the present study show better performance of neural networks than logistic regression, nevertheless external validation of the designed models, using larger databases with different rates of outcomes is necessary to get an accurate measure of performance outside the development population.

Even though the results of this study indicate the high ability of prediction by clinical symptoms compared with laboratory tests, one of the best ways to diagnose whether these symptoms could be related to a thyroid disorder is to consider how long individuals have been experiencing them. It is noted that this application is only an initial diagnosis. People who found that they are in the thyroid disorder risk group should go to see a doctor and in particular endocrinologists for a formal diagnosis to prevent themselves from serious problems.

Conclusion

Both neural network and logistic regression models have a high predictive ability to diagnose thyroid disorder, although neural network performance is better than logistic regression. The use of predictive models such as neural networks with the ability to incorporate complex relationships provides new opportunities to obtain more accurate predictions in the field of medical research.

Declaration of patient consent

The authors certify that they have obtained all appropriate patient consent forms. In the form, the patient (s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.