Thermal Image-Based Temperament Classification by Genetic Algorithm and Adaboost Classifier.

Background: Temperament (Mizaj) determination is an important stage of diagnosis in Persian Medicine. This study aimed to evaluate thermal imaging as a reliable tool that can be used instead of subjective assessments.
Methods: The temperament of 34 participants was assessed by a PM specialist using standardized Mojahedi Mizaj Questionnaire (MMQ) and thermal images of the wrist in the supine position, the back of the hand, and their whole face under supervision of the physician were recorded. Thirteen thermal features were extracted and a classifying algorithm was designed based on the genetic algorithm and Adaboost classifier in reference to the temperament questionnaire.
Results: The results showed that the mean temperature and temperature variations in the thermal images were relatively consistent with the results of MMQ. Among the three body regions, the results related to the image from Malmas were most consistent with MMQ. By selecting six of the 13 features that had the most impact on the classification, the accuracy of 94.7 ± 13.0, sensitivity of 95.7 ± 11.3, and specificity of 98.2 ± 4.2 were obtained. Conclusions: The thermal imaging was relatively consistent with standardized MMQ and can be used as a reliable tool for evaluating warm/cold temperament. However, the results reveal that thermal imaging features may not be only main features for temperament classification and for more reliable classification, it needs to add some different features such as wrist pulse features and some subjective characteristics. Copyright:

Introduction

Temperament assessment

In traditional Persian medicine (PM), knowing a patient's basic temperament can help physicians to diagnose and treat the disease more accurately. PM employs several factors to determine temperament: wrist temperature when touched (named Malmas in PM), body structure, skeletal makeup, hair condition, skin color, quality of body excretions, sleep and wakefulness, mental states, and how actions are taken by the body.[1] A number of these factors, such as Malmas temperature, are strongly influenced by environmental factors (ambient temperature, the doctor's hand temperature during the examination, etc.), which can lead to errors in assessing the patient's temperament.[2] Depending on factors that vary from one examiner to another can adversely affect the accuracy of the diagnosis and result in errors in the physician's recommendation.[34] This dependency can also reduce the reliability of these methods. Therefore, there is a need for an objective tool or method that can be repeatable. It seems that some imaging modality tools that are less affected by external factors can provide reliable outcomes when used by different physicians, reduce diagnostic errors, and lessen the time for each visit.

Thermal imaging

Thermal imaging is a process that uses infrared radiation emitted from a surface to obtain an image of temperature distribution. From a medical and biological point of view, it is very valuable compared to other imaging modalities because it does not use ionizing radiation or contrast agents, and it is a noncontact and non-invasive procedure.

In fact, each anatomical region has a distinct thermal pattern that results from differences in the circulatory system and the distribution of heat in the tissues. Any change in the vascular system is effective in the temperature distribution in the body. Therefore, recording temperature distribution patterns in the tissue can lead to a physiological interpretation of it. For example, this imaging feature has been used for the early detection of breast tumors.[56]

Thermal imaging is often used as a powerful tool in studying the body's system in a variety of situations. Research has shown that thermal imaging can be used as a quantitative diagnostic tool for wounds and injuries such as arterial malformations, joint inflammation, bladder stones, gastrointestinal disorders, headaches and migraines, and other diseases that potentially affect the body's temperature patterns.[7] It can also be used to evaluate the body's normal heat regulation processes, the process by which the body exchanges excess heat through the skin's surface with the environment to keep its core temperature constant.[8] Tanda showed that thermal imaging could provide useful information about the characteristics of individuals in accordance with intense sports activities.[9] Indriðadóttir demonstrated that thermal imaging is a valid tool for assessing the quantitative description of the physiological process, such as skin response to thermal shocks.[10] This type of imaging has been used in several studies to compare the results of different treatment processes. In 2018, Boerner and Podbielska used thermal imaging to evaluate and compare treatment outcomes between cryotherapy and ultrasound therapy.[11] Thermal imaging has recently been used in conjunction with oxygen therapy to evaluate the healing process of burn wounds and better understand the dynamic changes in the healing process.[12]

It has shown that there is a correlation between warm/cold temperaments with the level of the basal metabolism, indicating that the basal metabolism in individuals with cold temperament is less than those of warm temperaments. Similar conditions in endocrine disorders such as hypothyroidism and hyperthyroidism also alter basal metabolic rate. So, the level of basal metabolism decreases in hypothyroidism, and symptoms such as coldness of the skin are seen; in hyperthyroidism, warmth of the skin occurs due to increased basal metabolic rate. As a result, warm or cold temperament can be reflected in skin temperature.[13]

Thus, thermal imaging is one of the tools that can help to assess temperature distribution throughout the body. However, no study has been conducted on the use of this tool to assess temperament in PM. The aim of this study was to investigate the possibility of using thermal imaging in determining warm/moderate/cold temperament in individuals as well as its compliance with standardized and accepted temperament questionnaires in the field of PM.

Methods

Participants

The present pilot study based on the medical ethics committee certification was conducted at the traditional healthcare center of Iran University of Medical Sciences, during the winter of 2020 with 34 volunteers with different temperament states (warm/moderate/cold). The inclusion criterion was lack of disease history. After obtaining written consent from the volunteers, a PM specialist assessed their temperament according to the standardized Mojahedi Mizaj Questionnaire (MMQ). This questionnaire has been standardized for PM by Mojahedi et al. in 2014.[14] Furthermore, the thermal imaging of their wrist, back of the hand, and the entire face were recorded under the supervision of the PM specialist. The exclusion criterion was the poor image quality.

Thermal images

To examine the reliability and validity of the assessment using thermal imaging, various factors must be controlled. Some of these factors that have been previously studied are environmental factors (temperature, humidity, and atmospheric pressure of the environment and radiation sources), personal factors (sex, age, anthropometry, skin radiation coefficient, physical activity), and technical factors (camera specification and data capture protocol).[15] Therefore, the imaging was performed in the same room under controlled ambient conditions (23°C ± 1°C, 25% RH). All candidates were asked to wash and dry the areas to be imaged. Also, all jewelry was removed. The individual was then asked to sit in a chair, and his/her hand was positioned on a white surface; then, after 5 min (for thermal equilibrium), the imaging was performed at a distance of about 50 cm. There was no heat source near the subject.

The thermal camera used in this research was the T2 camera (ULIRVISION Company) with a spectrum range of 7.5 to 14 μm. The following three images were captured from each participant:

(a) Wrist image: The imaging center was considered at two fingers above the wrist. (b) Image of the back of the hand: The center of the back of the hand was considered the center of the image. (c) Full face image: The distribution of blood flow and heat to the face was examined using a complete image of the face from the chin to the forehead. The center of the image was considered around the tip of the individual's nose.

Image analysis

The image analysis was done in the MATLAB toolbox (Math Works, R2017b). Figure 1 shows the flowchart of the activities of the study.

Figure 1

The flowchart of the activities

In each recorded image, there were undesirable extra regions. Therefore, the region of interest (ROI) needed to be identified. In medical imaging, Active Contour Models (ACMs) are used for ROI segmentation from different medical images such as brain computed tomography images, magnetic resonance imaging images of different organs, cardiac images, and different images of regions in the human body. The ACM uses the energy constraints and forces in the image for the separation of ROI.[16] The curve is attracted towards features as edges of a target object by the evaluation of internal and external forces. The classical implementation of ACM is prone to be trapped into local minima and it is also sensitive to initialization point. Therefore, they require being close to the target object. So, in this research, the operator was asked to identify the main central region of the site, then a modified ACM algorithm was executed to segment the ROI.[17] Our experiment showed that this method is reliable and not sensitive to the precision of initial region selection by the operator.

From the ROI, many features can be obtained regarding the magnitude and variation of the temperature in each area. For investigation on various characteristics of the images, we used some related features. In this study, 13 temperature-related features were extracted from three different body positions (wrist, back of the hand, and face) for each participant. The extracted features are described in Table 1. These features are related to the temperature value (features 1, 2, 7, 8, 13) and its variations in the desired area (features 3, 4, 5, 6, 9, 10, 11, 12).

Table 1

Feature description

No.	Features description/formula	Position
1	Average temperature	Back of the hand
2	Maximum temperature
3	Coefficient of Variation
4	Skewness of temperature fluctuations
5	Kurtosis of temperature fluctuations
6	Entropy of temperature fluctuations
7	Average temperature	Wrist (Malmas)
8	Maximum temperature
9	Coefficient of Variation
10	Skewness
11	Kurtosis
12	Entropy
13	Average the face temperature	Face

Skewness and kurtosis are coefficients that measure how a temperature distribution is different from a normal distribution. Skewness measures the symmetry of distribution while Kurtosis is a statistical measure that defines how heavily the tails of distribution differ from the tails of a normal distribution. Eq. (1) and (2) describe the definition of these parameters:

Where Yi is image quantity in i-th pixel and N is total number of pixel in ROI.

Entropy is a statistical measure of randomness of the temperature variation. Eq. (3) describes the definition of this parameter:

Where Pi is the image histogram in i-th bin.

Features normalization is used for the optimal performance of the classification method. In this study, Eq. (4) has been used to normalize the features.

Where F is the value of a feature before normalization, Fn is the value of the feature after normalization, and μF and σF are the mean and the standard deviation of each feature, respectively.

After extracting all features, it was necessary to select the features that were more effective in classifying people (based on warm/cold temperament). In 2018, Nafisi and Shahabi showed that a genetic algorithm (GA) in combination with the Adaboost classification method could determine the most effective combination of features, which resulted in the most accurate classification.[18] Figure 2 shows the flowchart of the feature selection algorithm. Initial condition is M = 2 and stop condition is M >13. Number 13 is the total number of the features in this study. It is worth mentioning that GA process starts from random initial population and the selected features may not remain fixed in each iteration. Hence, this process was repeated several times and finally, the most selected features were reported.

Figure 2

The flowchart of the selection of the best features

For compensation of low sample size and improving of validity and accuracy of the results, we used a kind of k-fold cross-validation (CV). There are various kinds of CV for low sample size data-set. These methods have been widely used in classification issues when these are no enough experimental samples.[1920] At first, the optimized features were selected by GA. Then, the selected features (for all participants) were randomly partitioned into k subsets which one subset is considered as the test data and the remaining k − 1 subsets are used as training data. The CV process is then repeated k times, with each of the k subsets used exactly once as the test data. These combinations of feature-selection and classification were repeated for 100 iterations. After that, the statistical measure (accuracy, sensitivity, and specificity) were averaged and its standard deviations were calculated. Figure 3 shows the flowchart of the method.

Figure 3

Performance calculation flowchart

Results

The demographic information of the 34 participants are listed in Table 2. For achieving to a 90% confidence interval in this pilot study, we should be examined about 65 participants,[21] but we encountered to COVID-19 epidemic in Iran so the final sample size was limited to 34 participants.

Table 2

Demographic information of participants

Range/Number	Parameter
37.11±7	Age (years)
Gender
Female	21
Male	13
Temperament state (based on MMQ)
Warm	15
Moderate	17
Cold	2

MMQ: Mojahedi Mizaj Questionnaire

According to MMQ, 15 subjects were warm, two subjects were cold, and 17 subjects had a moderate temperament. It should be noted that because the number of people with a cold temperament was very low in this study, the analysis and evaluation was limited to the two group of warm and moderate temperaments (32 samples).

At first, ROI needed to be identified. The operator was asked to identify the main central region of the site, then an automatic algorithm based on active contours was executed to recognize the ROI. A sample result is shown in Figure 4.

Figure 4

Region of interest extraction by the active contour algorithm (a and b) the back of the hand and (c and d) the wrist

The algorithm used in this research is based on a “training” step. The algorithm operates on a series of feature vectors labeled by the reference method (MMQ). Then, the mean and standard deviation of the accuracy, sensitivity, and specificity of the algorithm on the “test” data are calculated according to Eq. (6) to (8).

Where TP, TN, FP and FN are true positive, true negative, false positive, and false negative, respectively.

In this study, there were 32 vectors containing 13 features. We used the mentioned cross-validation for evaluating the proposed method. We used k = 7 and ∼85% of data were considered as the training set and ∼15% of data for the test. Table 3 includes the accuracy, sensitivity, and specificity.

Table 3

Classification performance for cross-validation fold=7

Parameter	Mean±standard deviation
Accuracy	94.7±13.0
Sensitivity	95.7±11.3
Specificity	98.2±4.2

We have attempted to answer the following questions. Which of the mentioned features has the greatest effect on temperament classification and with what accuracy? As mentioned in Section 2, more effective features were selected using the GA. We determined which features were the most effective after 100 times iteration and the results are shown in Figure 5. In this figure, the number of times each of the features shown in Table 1 was selected in the 100 repetitions is specified.

Figure 5

Number of times each feature was selected as an effective feature

As shown in Figure 5, features 2, 3, 4, 7, 12 were never selected and, therefore, had no effect on the classification, but the other features were considered effective.

Based on Figure 4, the top six features (1, 8, 9, 10, 11, and 13) in this study are listed below:

Average temperature of the back of the hand zone

Maximum temperature of Malmas

Coefficient of variation/Skewness/Kurtosis of temperature fluctuations in Malmas

Average temperature of the face.

Discussion

Because of the distribution of heat is affected by physiological activity, it can be expected to be related to warm/moderate/cold temperaments. The aim of this research was to determine the feasibility of thermal imaging to measure different temperaments in individuals and its compliance with a standardized temperament questionnaire in PM.

Our initial goal was the differentiation between warm/moderate/cold temperaments but as mentioned before the analysis and evaluation were limited to the two groups of warm and moderate temperaments. Therefore, our results showed that thermal imaging at least could be used as a tool to classify warm/moderate temperament. The features used in this study [Table 1] measured two factors: the quantity of temperature in each of these regions (features 1, 2, 7, 8, 13) and the temperature variations in the area (features 3, 4, 5, 6, 9, 10, 11, 12). After feature normalization, the combination of GA and Adaboost classifier were applied to select the optimal features to detect different temperaments.

GA is an optimization procedure and belongs to the evolutionary algorithms group. In some researches, this have been shown the effectiveness of GA for the selection of the best features for more accurate classification.[51822] Based on this method, it was found that among the 13 items, six features were more important. This result showed that both the mean/maximum value (features 1, 8, and 13) and the temperature fluctuations (features 9, 10, and 11) in each of the three regions could play an important role in the differentiation of warm/moderate temperaments. It should be noted that only one feature of the whole face image (temperature of face center) was used in this study because of technical reasons.

It is interesting that the temperature features of the wrist (Malmas) especially showed a greater impact for the temperament classification according to MMQ. It is worth mentioning this region is most interested for PM experts to examine. It seems that the temperature fluctuations in this region are better related to the body condition.

Since thermal imaging has not been used in PM to measure temperament, it has not been possible to compare the results of this study with similar researches. However, some studies confirmed the use of thermal imaging for diagnostic evaluation in traditional medicine which are in line with the findings of the present study on the use of thermal imaging in assessing the qualities of warm/cold in traditional medicine.

In 2016, Yoo et al. used a digital thermal camera to measure the temperature of the lower abdomen in infertile women. They showed that women with tubular or peritoneal causes have a lower abdominal temperature.[23] In 1990, Zhong et al. used thermal videos to measure the temperature of different areas of the tongue in patients with yin deficiency (an important disorder in Chinese medicine). The results showed that yin deficiency was directly related to the temperature of the tongue.[24] Infrared thermography is widely used in studies of meridians in acupuncture as well as in the diagnosis and treatment of diseases in Chinese medicine.[25262728] People with warm temperaments have more peripheral sympathetic nervous system activity and less parasympathetic nervous system activity than cold ones.[29] According to traditional PM, individuals with a cold temperament usually feel colder than others and tolerate heat better than cold; their fingers and limbs are usually cold. The opposite is true for persons with a warm temperament. They are expected to have a higher body temperature.[30]

Many studies have shown that thermal imaging can measure the distribution of heat in any region of the body. A study confirmed that there was a significant difference between the mean foot temperatures of diabetic patients with neuropathic and neurodegenerative problems as opposed to healthy people or diabetic patients without vascular problems, which could be used as a diagnostic criterion for differentiating between these patients.[31] One study found an association between temporal or fascial arterial blood flow and facial skin temperature. The results showed that the higher the volume of blood flow in the face, the higher the face temperature.[32] In 2004, Chan et al. studied the thermal images of the front and lateral sides of the face; they examined the relationship between the temperature of the thermal image and the core body temperature (inside of the ear or inside the mouth). Based on the results of this study, if the imaging position was chosen correctly, the thermal image had a high correlation with the central temperature.[33]

This research can be considered as the first step in the possibility of using thermal imaging in assessing temperament in traditional PM. Actually, it is a preliminary and feasibility study for thermal imaging application in PM. The best performance of the algorithm is shown in Table 3 for k = 7. As seen in the table, the standard deviations are approximately large due to low data-set size. Therefore, to create more reliable decision-making system based on this algorithm, future studies must include a greater number of participants. However, it can be realized that the performance of the algorithm is acceptable even at worst case (accuracy = 80%, sensitivity = 84%, specificity = 92%).

Also, we investigated on the impact of CV fold parameter at the performance of the algorithm. In fact, it is important that how much data to use for training and how much to use for test. If the percentage of training data is too high or too low, the prediction error of the algorithm could increases and we need to search for the best train-test datasets. Figure 6 shows the impact of CV fold in this study and on our dataset that low or high folds can reduce algorithm performance. Furthermore, we carried out the leave-one-out method which shows slightly lower performance than CV fold = 7 but, the standard deviation of the results (for accuracy and sensitivity) are very better. Usually, when the training set is increased, the performance should be increased. However, it seems that this trend is dataset-dependent. Vabalas et al. discussed on various factors influencing CV performance.[19] There are some variations in the performance when CV fold would vary and the performance could be higher for some folds than others. When the training set is increased, overfitting might be occurred and final performance is decreased. Figure 6 shows that fold 7 has the best performance based on the mean and standard deviation of accuracy, sensitivity, and specificity.

Figure 6

The performance of the algorithm for various cross-validation fold: Mean and error bar for (a) accuracy, (b) sensitivity, (c) specificity

Anyway, the relatively high standard deviations of the results reveal thermal imaging features may not be only main features for temperament classification and for more reliable classification, we need to add some different features such as wrist pulse features and some subjective characteristics. Furthermore, in this study, MMQ was used as “gold standard” for temperament labeling which might be an incomplete and slightly inconsistent standard for precise labeling. Our further research is focused on overcoming these problems.

One of the advantages of this research was the use of a relatively inexpensive camera with low accuracy. However, for more detailed studies, a higher-precision cameras may be needed. Lokaj et al. used a camera with an accuracy of 0.05°C to investigate the distribution of blood in laboratory mice.[34] However, the price of such camera is much higher and may not be feasible to use in a PM office. This study showed that using a relatively inexpensive camera can provide reliable results.

Finally, it is worth mentioning that thermal imaging has some advantages over using a common thermometer. Thermal imaging is the contactless method which can determine the temperature and its variations in the entire ROI (IR resolution: 80 × 60 and image resolution: 480 × 320), while a thermometer or a temperature sensor array can only determine the temperature at some points of contact. Furthermore, it has shown that some tissue characteristics (i.e., dry or wet conditions) can be evaluated by thermal imaging.[35]

Conclusion

Because of the distribution of heat is affected by physiological activity, it can be expected to be related to warm/moderate/cold temperaments. This research can be considered as the first project in the possibility of using thermal imaging evaluating of temperament in traditional PM. The aim of this research was to determine the feasibility of thermal imaging to measure different temperaments in individuals and its compliance with a standardized temperament questionnaire in PM. The results showed that some features extracted from thermal images could provide proper information for objective temperament assessment in PM. However, the results reveal that thermal imaging features may not be only main features for temperament classification and for more reliable classification, it needs to add some different features such as wrist pulse features and some subjective characteristics.

Financial support and sponsorship

None.

Conflicts of interest

There are no conflicts of interest.