A classification method of normal and overweight females based on facial features for automated medical applications.

Obesity and overweight have become serious public health problems worldwide. Obesity and abdominal obesity are associated with type 2 diabetes, cardiovascular diseases, and metabolic syndrome. In this paper, we first suggest a method of predicting normal and overweight females according to body mass index (BMI) based on facial features. A total of 688 subjects participated in this study. We obtained the area under the ROC curve (AUC) value of 0.861 and kappa value of 0.521 in Female: 21-40 (females aged 21-40 years) group, and AUC value of 0.76 and kappa value of 0.401 in Female: 41-60 (females aged 41-60 years) group. In two groups, we found many features showing statistical differences between normal and overweight subjects by using an independent two-sample t-test. We demonstrated that it is possible to predict BMI status using facial characteristics. Our results provide useful information for studies of obesity and facial characteristics, and may provide useful clues in the development of applications for alternative diagnosis of obesity in remote healthcare.

1. Introduction

Obesity and overweight have become major health issues, because the prevalence of obesity has rapidly risen worldwide. The causes of this phenomenon are excessive ingestion of food, lack of physical activity, and environmental and genetic factors [1, 2]. Obesity and abdominal obesity are potential risk factors for insulin resistance and type 2 diabetes, cardiovascular diseases, stroke, ischemic heart disease, and metabolic syndrome [3-6], and many studies have investigated the relationship between obesity, disease, and body mass index (BMI) [7-13]. In the medical field and public health, BMI is commonly used as an indicator of overall adiposity. So, BMI is essential medical information for the prognostic prediction of diseases and clinical therapy. The principal cutoff points for underweight (<18.50 kg/m2), normal range (18.50–24.99 kg/m2), overweight or preobese (25.00–29.99 kg/m2), and obese (≥30.00 kg/m2) have been set by the World Health Organization (WHO).

A large number of studies on human face have focused on facial morphology, face recognition, and medicine [14-23]. Facial characteristics provide clinical information on the present or future health conditions of patients. For example, the status of cheeks, neck circumference, and craniofacial morphology are associated with health complications, such as type 2 diabetes, hypertension, and sleep apnea [18]. Using computed tomographic (CT) scanning, Levine et al. [19] showed that the quantity of buccal fat is strongly related to visceral abdominal fat accumulation, based on the fact that patients with chubby facial cheeks tend to have upper-body obesity, and argued that plump cheeks of patients may be a high potential risk factor for metabolic complications related to obesity. Further, using facial measurements, Sadeghianrizi et al. [20] showed that craniofacial morphology is significantly different between normal and obese adolescents. They suggested that facial skeletal structures of obese adolescents tended to be relatively large, and that obesity was associated with bimaxillary prognathism.

The motivation for this study is conveyed by the following 2 questions: which features or facial characteristics are associated with overweight and normal BMI status? If we identify facial features that differ between normal and overweight, how accurately can we identify normal and overweight using these features? Contributions of this study are as follows. We first propose a method of classifying normal and overweight status using only facial characteristics. To date, no study has addressed a method that predicts BMI status using facial features. Furthermore, we introduce meaningful and discriminatory features that show a statistically significant difference between normal and overweight by statistical analysis, and identify compact and useful feature sets for BMI classification using facial features in female group. The results of this study will be useful in understanding the relationship between obesity-related diseases and facial characteristics.

2. Materials and Methods

2.1. Data Collection

A total of 688 subjects participated in this study. At the Korea Institute of Oriental Medicine, frontal and profile photographs of subjects' faces with a neutral expression were acquired using a digital camera with a ruler (Nikon D700 with an 85 mm lens) and the subjects' clinical information, such as name, age, gender, weight, height, blood pressure, and pulse were recorded. All images were captured at a resolution of 3184 × 2120 pixels in JPEG format. Height and weight of subjects were measured by a digital scale (GL-150; G Tech International Co., Ltd, Republic of Korea).

Based on identifiable feature points from the front and profile images of subjects, a total of 86 features were extracted. The extracted features included distance between points n1 and n2 in a frontal (or profile) image, vertical distance between n1 and n2 in a frontal (or profile) image, angles of 3 points n1, n2, and n3 in a frontal (or profile) image, area of the triangle formed by the 3 points n1, n2, and n3 in a profile image, and so forth. All points in a front and profile image are showed in Figure 1, and all the extracted features and brief descriptions are given in Table 1.

Figure 1

All points in a facial image for feature extraction ((a): points and areas in frontal image; (b): points in profile image; (c): points in right eye; (d): point in left eye). Distance, angle, and area measurements were done based on self-made tool using MATLAB on Window XP.

Table 1

All features used in this study and brief descriptions.

Feature	Brief description
FD n1_n2	Distance between points n1 and n2 in a frontal (or profile) image
FDH n1_n2	Horizontal distance between n1 and n2 in an image
FDV n1_n2	Vertical distance between n1 and n2 in an image
FA n1_n2_n3	Angle of three points n1, n2, and n3 in an image
FA n1_n2	Angle between the line through 2 points n1 and n2 and a horizontal line
FR02_psu	FD(17, 26)/FD(18, 25)
FR03_psu	(FD(18, 25) + FD(118, 125))/FDH(33, 133)
FR05_psu	FDH(33, 133)/FD(43, 143)
FR06_psu	FDH(33, 133)/FDV(52, 50)
FR08_psu	FD(43, 143)/FDV(52, 50)
FArea02	Area of the contour formed by the points 53, 153, 133, 194, 94, 33, and 53
FArea03	Area of the contour formed by the points 94, 194, 143, 43, and 94
Fh_Cur_Max_Distan	Distance between points 7 and 77 in a profile image
Fh_Angle_n1_n2	Angle between the line through 2 points n1 and n2 and a horizontal line
Nose_Angle_n1_n2	Angle between the line through 2 points n1 and n2 and a horizontal line
Nose_Angle_n1_n2 _ n3	Angle of 3 points n1, n2, and n3 in a frontal(or profile) image
SAn 1_n2	Angle between the line through 2 points n1 and n2 and a horizontal line
Fh_Cur_Max_R79_69	FD(77, 9)/FD(6, 9)
Nose_Area_n1_n2_n3	Area of the triangle formed by 3 points n1, n2, and n3 in a profile image
EUL_L_el1 ~ EUL_L_el7	Slope of the tangent at a point (el1~el7) in a frontal image
EUL_L_DH	FDH(el1, el7)
EUL_L_MAX	FDH(el1, elmax⁡)
EUL_L_RMAX	FDH(el1, elmax⁡)/FDH(el1, el7)
EUL_L_Sb	FDV(el7, el1)/FDH(el7, el1)
EUL_L_St	FDV(elmax⁡, el7)/FDH(elmax⁡, el7)
EUL_L_Sf	FDV(elmax⁡, el1)/FDH(elmax⁡, el1)
EUL_L_Khmean	Average curvature of the left (or right) upper eyelid contour
EUL_L_khmax	Maximum curvature of the left (or right) upper eyelid contour
EUL_R_er1~ EUL_R_er7	Slope of the tangent at a point (er1~er7) in a frontal image
EUL_R_DH	FDH(er1, er7)
EUL_R_MAX	FDH(er1, ermax⁡)
EUL_R_RMAX	FDH(er1, ermax⁡)/FDH(er1, er7)
EUL_R_Sb	FDV(er7, er1)/FDH(er7, er1)
EUL_R_St	FDV(ermax⁡, er7)/FDH(ermax⁡, er7)
EUL_R_Sf	FDV(ermax⁡, er1)/FDH(ermax⁡, er1)
EUL_R_Khmean	Average curvature of the left (or right) upper eyelid contour
EUL_R_khmax	Maximum curvature of the left (or right) upper eyelid contour
PDH44_53	Horizontal distance between n1 and n2 in a frontal (or profile) image

2.2. Normal and Overweight Cutoff Points

BMI was calculated as weight (kg) divided by the square of height (m) of the individual. Health consequences and BMI ranges of overweight and obesity are open to dispute [10, 24]. There is natural consequence. Physiological and environmental factors of race are associated with differences in BMI values and the assignment of BMI values for obesity and overweight depends on various factors, such as ethnic groups, national economic statuses, and rural/urban residence [8]. For instance, BMI values of a population in an Asian region tend to be lower than those of a population in a Western region; however, Asians have risk factors for cardiovascular disease and obesity-related diabetes at relatively low BMI values [11, 25]. In this study, we followed the suggestions of WHO to assign the cutoff point for each class in the Asia-Pacific region [25]. The proposed categories are as follows: normal, 18.5–22.9 kg/m2; overweight, ≥23 kg/m2.

Since the facial features and BMI are influenced by gender and age [26], participants were divided into 2 groups: female; 21–40 (females aged 21–40 years) and female: 41–60 (females aged 41–60 years). Detailed data and basic statistics of each group are presented in Table 2.

Table 2

Subject characteristics and basic statistics (data are presented as mean (standard deviation); N: number of subjects, BMI: body mass index).

Class		Female: 21–40	Female: 41–60
Normal	N	189	193
	Age	32.1 (5.64)	50.0 (5.42)
	BMI	22.2 (2.97)	23.6 (2.86)
Overweight	N	77	229
	Age	32.91 (5.29)	50.31 (5.44)
	BMI	26.0 (2.75)	25.6 (2.31)

For the selection of useful and discriminatory features, only features presenting P-values < 0.05 in each group by an independent two-sample t-test were used in this study. In other words, only features with a P value < 0.05 were included in classification experiments. Thus, features used in each group are different due to the difference of age. A detailed analysis of the statistical data and the selected features is presented in Section 3.2.

2.3. Preprocessing and Experiment Configurations

In the preprocessing step, the experiment was performed in 2 ways: (1) only the normalization method (scale 0~1 value) was applied to raw datasets, and (2) normalization and discretization were applied for better classification accuracy. We used the entropy-based multi-interval discretization (MDL) method introduced by Fayyad and Irani [27]. For classification performance evaluation, we used the area under the curve (AUC) and kappa as major evaluation criteria. Additionally, sensitivity, 1-specificity, precision, F-measure, and accuracy were used for detailed performance analysis. All the results were based on 10-fold cross-validation method for a statistical evaluation of learning algorithm. All experiments were conducted by Naive Bayes classifier in WEKA software [28], and statistical analyses were conducted by SPSS version 19 for Windows (SPSS Inc., Chicago, IL, USA).

3. Results and Discussion

3.1. Performance Evaluation

For brief summarization of performance evaluation, the AUC and kappa for the 2 groups with and without the use of MDL method (i.e., 2 ways of preprocessing) are depicted in Figure 2.

Figure 2

A comparison of performance evaluations using AUC and kappa in 2 female groups (AUC-MDL and Kappa-MDL: use of MDL, AUC and Kappa: without the use of MDL).

AUC values of the method using MDL in 2 female groups ranged from 0.760 to 0.861, whereas AUC of the method without the use of MDL ranged from 0.730 to 0.771. AUC and kappa values of the method using MDL showed improvements of 0.09 and 0.115, respectively, in the female 21–40 group, and 0.03 and 0.073, respectively, in female: 41–60.

Comparing AUC and kappa values, the classification performance of the method with MDL was higher than that of the method without MDL. These results showed that the BMI classification method of applying MDL was significantly better than that of not applying MDL.

The identification of normal and overweight in female: 41–60 group was more difficult than that of normal and overweight in female: 21–40 group. The exact reason behind this phenomenon is unknown, but obesity and menopause-related research studies offer some clues [29-31]. Menopause leads to changes in fat tissue distribution, body composition, waist-to-hip ratio (WHR), and waist-to-height (W/Ht) in females. For instance, Douchi et al. [29] demonstrated that the lean mass of the head of premenopausal and postmenopausal females were not different, while trunk and legs were altered following menopause. Detailed results of the performance evaluation of each class and group are described in Tables 3 and 4. We think that these results imply the possibility of predicting normal and overweight status using human face information.

Table 3

Detailed performance evaluation of experiments using the MDL method in 2 groups (Sen.: sensitivity, 1-spe.: 1-specificity, Pre.: precision, F-Me.: F-measure, and Acc.: accuracy).

Group	Class	Sen.	1-spe.	Pre.	F-Me.	Acc.
Female: 21–40	Normal	0.884	0.377	0.852	0.868	80.8%
	Overweight	0.623	0.116	0.686	0.653
Female: 41–60	Normal	0.653	0.253	0.685	0.668	70.4%
	Overweight	0.747	0.347	0.718	0.732

Table 4

Detailed performance evaluation of experiments without the use of MDL method (Sen.: sensitivity, 1-spe.: 1-specificity, Pre.: precision, F-Me.: F-measure, and Acc.: accuracy).

Group	Class	Sen.	1-spe.	Pre.	F-Me.	Acc.
Female: 21–40	Normal	0.788	0.364	0.842	0.814	74.4%
	Overweight	0.636	0.212	0.551	0.59
Female: 41–60	Normal	0.684	0.354	0.62	0.65	66.4%
	Overweight	0.646	0.316	0.708	0.676

3.2. Statistical Analysis of Facial Features

Statistical analysis of the comparison between normal and overweight classes was performed using an independent two-sample t-test, and a P-value < 0.05 was considered statistically significant. Features with a P-value < 0.05 in each group are described in Tables 5 and 6.

Table 5

Statistical analysis of female: 21–40 group by an independent two-sample t-test (Std.: standard deviation).

Feature	Class	Mean (Std.)	t	P-value
FD17_26	Normal	9.473 (1.317)	3.118	0.002
	Overweight	8.941 (1.115)
FD117_126	Normal	9.483 (1.303)	3.319	0.001
	Overweight	8.904 (1.257)
FDH25_125	Normal	96.53 (5.116)	−2.69	0.0076
	Overweight	98.52 (6.32)
FDH36_136	Normal	23.57 (2.469)	−2.75	0.0064
	Overweight	24.46 (2.191)
FD18_25	Normal	29.94 (2.675)	−2.036	0.0428
	Overweight	30.68 (2.753)
FD43_143	Normal	125.2 (7.101)	−8.625	0.0000
	Overweight	133.6 (7.384)
FD53_153	Normal	145.4 (5.941)	−5.991	0.0000
	Overweight	150.7 (7.642)
FD94_194	Normal	140.1 (6.022)	−8.875	0.0000
	Overweight	147.6 (6.934)
FDH33_133	Normal	147.2 (5.63)	−7.261	0.0000
	Overweight	153.1 (7.02)
FA18_17_25	Normal	126.2 (6.591)	−2.684	0.0077
	Overweight	128.6 (6.75)
FA118_117_125	Normal	125 (7.339)	−3.56	0.0004
	Overweight	128.3 (6.199)
FA18_25_43	Normal	95.38 (5.104)	−3.722	0.0002
	Overweight	97.91 (4.896)
FA118_125_143	Normal	96.16 (4.753)	−3.396	0.0008
	Overweight	98.39 (5.082)
FA18_17_43	Normal	76.97 (6.255)	−4.39	0.0000
	Overweight	80.66 (6.108)
FA118_117_143	Normal	76.82 (6.824)	−4.644	0.0000
	Overweight	80.9 (5.583)
FA117_125	Normal	21.24 (3.645)	3.983	0.0001
	Overweight	19.19 (4.142)
FA17_18	Normal	34.01 (5.091)	2.002	0.0463
	Overweight	32.61 (5.32)
FR02_psu	Normal	0.318 (0.044)	4.199	0.0000
	Overweight	0.293 (0.041)
FR05_psu	Normal	1.178 (0.055)	4.183	0.0000
	Overweight	1.148 (0.048)
FR06_psu	Normal	2.039 (0.117)	−5.334	0.0000
	Overweight	2.123 (0.115)
FR08_psu	Normal	1.736 (0.151)	−5.783	0.0000
	Overweight	1.854 (0.147)
FArea02	Normal	6470 (644.4)	−2.106	0.0362
	Overweight	6654 (652.2)
FArea03	Normal	3596 (364.9)	−5.637	0.0000
	Overweight	3873 (361.9)
Fh_Cur_Max_Distan	Normal	3.654 (1.564)	1.984	0.0483
	Overweight	3.233 (1.585)
FDH12_14	Normal	18.58 (2.713)	−3.006	0.0029
	Overweight	19.69 (2.817)
Nose_Angle_14_12	Normal	61.07 (4.611)	2.946	0.0035
	Overweight	59.29 (4.108)
Nose_Angle_12_14_21	Normal	106.7 (4.634)	2.397	0.0172
	Overweight	105.1 (5.237)
EUL_L_el2	Normal	−0.637 (0.095)	−3.135	0.0019
	Overweight	−0.597 (0.087)
EUL_L_ el3	Normal	−0.22 (0.118)	−3.206	0.0015
	Overweight	−0.17 (0.11)
EUL_L_ el6	Normal	0.483 (0.105)	3.473	0.0006
	Overweight	0.432 (0.113)
EUL_L_DH	Normal	3.178 (0.248)	−2.53	0.0120
	Overweight	3.268 (0.292)
EUL_L_Sf	Normal	0.408 (0.106)	2.442	0.0153
	Overweight	0.371 (0.132)
EUL_R_er2	Normal	−0.63 (0.087)	−3.957	0.0001
	Overweight	−0.582 (0.095)
EUL_R_ er3	Normal	−0.208 (0.112)	−2.822	0.0051
	Overweight	−0.167 (0.1)
EUL_R_ er6	Normal	0.466 (0.106)	2.492	0.0133
	Overweight	0.43 (0.111)
EUL_R_ er7	Normal	0.647 (0.235)	2.432	0.0165
	Overweight	0.556 (0.29)
EUL_R_DH	Normal	3.188 (0.226)	−4.292	0.0000
	Overweight	3.322 (0.241)
EUL_R_RMAX	Normal	0.443 (0.069)	2.061	0.0403
	Overweight	0.424 (0.066)
EUL_R_St	Normal	−0.633 (0.117)	−2.525	0.0122
	Overweight	−0.592 (0.123)
EUL_R_Sf	Normal	0.395 (0.106)	2.452	0.0149
	Overweight	0.36 (0.104)
EUL_R_Khmean	Normal	0.024 (0.007)	2.868	0.0045
	Overweight	0.022 (0.007)
PDH44_53	Normal	89.38 (6.081)	−3.017	0.0028
	Overweight	91.79 (5.527)

Table 6

Statistical analysis of female: 41–60 group by an independent two-sample t-test (Std.: standard deviation).

Feature	Class	Mean (Std.)	t	P-value
FDH25_125	Normal	94.63 (5.466)	−3.097	0.0021
	Overweight	96.29 (5.493)
FDH36_136	Normal	24.84 (2.283)	−2.055	0.0405
	Overweight	25.36 (2.805)
FD18_25	Normal	29.37 (3.287)	−2.199	0.0284
	Overweight	30.04 (2.923)
FD17_25	Normal	17.83 (2.717)	−2.076	0.0385
	Overweight	18.36 (2.471)
FD43_143	Normal	127.4 (6.471)	−8.184	0.0000
	Overweight	133.1 (7.721)
FD53_153	Normal	143.9 (6.343)	−4.848	0.0000
	Overweight	147.2 (7.141)
FD94_194	Normal	141.8 (6.01)	−8.385	0.0000
	Overweight	146.9 (6.485)
FDH33_133	Normal	146.8 (6.057)	−6.615	0.0000
	Overweight	150.9 (6.582)
FA18_25_43	Normal	99.88 (5.308)	−2.589	0.0100
	Overweight	101.2 (4.954)
FA118_125_143	Normal	99.74 (4.776)	−4.343	0.0000
	Overweight	101.9 (5.373)
FA117_125_143	Normal	124.7 (5.38)	−2.438	0.0152
	Overweight	126 (5.471)
FA18_17_43	Normal	81.11 (6.753)	−2.676	0.0077
	Overweight	82.85 (6.574)
FA118_117_143	Normal	80.69 (6.449)	−3.632	0.0003
	Overweight	83.16 (7.35)
FR02_psu	Normal	0.295 (0.044)	2.182	0.0297
	Overweight	0.285 (0.051)
FR05_psu	Normal	1.154 (0.046)	3.966	0.0001
	Overweight	1.135 (0.049)
FR06_psu	Normal	2.006 (0.104)	−5.688	0.0000
	Overweight	2.068 (0.121)
FR08_psu	Normal	1.743 (0.134)	−5.935	0.0000
	Overweight	1.827 (0.157)
FArea02	Normal	6358 (618.3)	−2.212	0.0275
	Overweight	6501 (696.7)
FArea03	Normal	3886 (397.6)	−4.245	0.0000
	Overweight	4052 (402.6)
FDV12_14	Normal	33.85 (3.313)	2.516	0.0123
	Overweight	33 (3.571)
FDH14_21	Normal	12.9 (1.633)	2.163	0.0311
	Overweight	12.53 (1.889)
Nose_Angle_14_21	Normal	45.73 (4.983)	−2.402	0.0168
	Overweight	46.98 (5.765)

In female: 21–40, 42 features were significantly different between normal and overweight classes (P < 0.05), and 11 of these features exhibited highly significant differences (P < 0.0000). Four features concerning distances between n1 and n2 points in a frontal image (FD43_143, FD53_153, FD94_194, and FDH33_133 related to the mandibular width or face width) exhibited particularly significant differences. The features FA18_17_43 and FA118_117_143 representing the angles between three points n1 (medial canthus), n2 (midpoint of the upper eyelid), and n3 (mandibular ramus) in a frontal image were highly significantly different. Comparing female: 21–40 and female: 41–60 groups, many features related to the eyelid were found in female: 21–40, but the features were not found in Female: 41–60. For instance, EUL_R_DH (horizontal distance from er1 to er7 in the eye image) was highly significantly different between the normal and overweight classes. The means of EUL_R_DH in normal and overweight status were 3.188 (0.226) and 3.322 (0.241) (t = −4.292, P = 0.0000). In female: 41–60, a total of 21 features were significantly different between the normal and overweight classes, and 8 of these features were highly significantly different (FD43_143, FD53_153, FD94_194, FDH33_133, FA118_125_143, FR06_psu, FR08_psu, and FArea03; P < 0.0000).

Many features that were significantly different between the normal and overweight classes in particular age group were identified. 25 features such as EUL_R_St, FD117_126, Fh_Cur_Max_Distan, FDH12_14, EUL_R_DH, and EUL_R_Khmean were found only in the female: 21–40 group, while the features FD17_25, FA117_125_143, FDV12_14, FDH14_21, and Nose_Angle_14_21 were only found in female: 41–60.

3.3. Medical Applications and Limitations

Patients or potential patients with obesity-related diseases must constantly check their own BMI based on their weight. Measurements using calibrated scales and ruler are ideal, but may not always be possible in the critically ill [32] and in telemedicine or emergency medical services in real time in remote locations. Our method was designed under the prerequisite that above method cannot be used in situations such as elderly trauma or intensive care in emergency medicine, remote healthcare, and so forth.

Several studies have been performed on patient BMI and weight estimation in emergency medical service and telemedicine [32-35]. These are important to enable accurate drug dosage, counter shock voltage calculation, or treatment, particularly in situations of serious illness, such as elderly trauma or intensive care [33, 34]. On the one hand, most patients are not aware of their body weight because the body weight of many individuals changes over time. For example, although patient self-estimates of weight are better than estimates by residents and nurses in emergency departments, 22% of patients do not estimate their own weight within 5 kg [34]. The method described herein can provide clues to the development of alternative methods for BMI estimation in the above situations or telemedicine, and the development of medical fields because facial characteristics provide substantial clinical information on the present or future health conditions of patients [18, 19].

4. Conclusions

The relationship between obesity, diseases, and face that are associated with health complications has been researched for a long time. Here, we have proposed and demonstrated the possibility of identifying normal and overweight status using only facial characteristics, and found statistically significant differences between the 2 classes in 2 female groups. Although there are still problems to be solved for the complete classification of BMI status, this method would provide basic information and benefits to studies in face recognition, obesity, facial morphology, medical science, telemedicine, and emergency medicine.