Evaluation of effectiveness of eye massage therapy via classification of periocular images.

This paper proposes a method to evaluate the effectiveness of the eye message therapy. The existing methods are via the diagnoses conducted by the medical professions based on the measurements acquired by the optical instruments. However, this approach is very expensive. To address this issue, this paper performs the classification between the periocular images taken before performing the eye massage therapy and those after performing the eye massage therapy to address the above difficulty. First, the median filtering is used to suppress the solitary point noise with preserving the edges of the image without causing the significant blurring. Then, the Canny operator is employed to accurately locate the edges. Next, the circle Hough transform (CHT) is used for performing the iris segmentation. Finally, various classifiers are used to perform the classification. The computer numerical simulation results show that our proposed method can achieve the high classification accuracies. This implies that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. In addition, the comparisons with the state of art Daugman method have been performed. It is found that the classification performance achieved by the CHT based method is better than those achieved by the Daugman method.

Introduction

Eye diseases such as the dry eye diseases, the myopia, the diabetic eye diseases and the glaucoma are the worldwide prevalent diseases. According to the report issued by the World Health Organization, nearly one billion of people in the worldwide have the above preventable eye diseases. The eye diseases can be due to many factors [2, 12, 18]. These factors include the aging, the genetic factors, the viral or the bacterial infections as well as the excessive and the incorrect use of the eyes. In recent studies, some evidences show that some eye diseases such as the dry eye diseases, the myopia and the computer vision based eye syndromes are closely related to the eye fatigue [10, 26]. Since doing the eye exercises to protect the eyes [17, 20] was promoted in the past few decades, many people take the eye massage therapy to soothe the eyes to relieve the eye strain and to protect the eyes. As a result, the eye massage therapy becomes popular.

The evaluation of the effectiveness of the eye massage therapy plays an important role in the treatment of the eye diseases. However, the works in this area are very limited. At present, the evaluations are mainly dependent on the diagnoses conducted by the medical professions based on the measurements acquired by the optical instruments [15]. In particular, the LipView interferometer and the fluorophotometer are utilized to perform the diagnoses of the dry eye diseases [8] at the dry eye clinics. Here, the interferometer provides two important measurements. They are the average thickness of the lipid layer and the partial blink rate. Besides, the fluorophotometer measures the penetration of the corneal fluorescein. However, conducting the above measurements is very expensive and it takes a very long processing time. On the other hand, an image based evaluation method was proposed [25]. The diagnoses of the dry eye diseases were performed using the images acquired from the vivo confocal microscopic and the optical coherence tomographic instruments. Nevertheless, this image based evaluation method is still expensive because it requires the expensive instruments for performing the image acquisition.

This paper proposes a periocular image classification based method for performing the evaluation of the effectiveness of the eye massage therapy. Here, the periocular images are taken by the portable cameras. First, the iris segmentation is performed. Different techniques were used for detecting the location of the eye via detecting the location of the centre of the pupil for performing the iris segmentation. The common methods include the CHT based method and the Daugman method [1, 7, 29]. In particular, the Daugman integro-differential operator [6] was used to segment the iris in the Daugman method. Also, many improvements were achieved based on this approach [23, 24]. On the other hand, the CHT based method was used to segment the iris [14, 19]. Then, the change of the size of the iris in the image is computed. However, these methods were sensitive to the noises. Besides, detecting the edge of the iris is an important step for performing the iris segmentation [4]. Therefore, the noise is required to be suppressed without losing the edge information [27] for performing the iris segmentation. Different methods [28] were used to suppress the noises. In this paper, the median filtering is employed for performing the denoising. This is because it can keep the edge information while does not blur the image. Besides, the Canny operator can detect the iris edge effectively. Hence, the CHT based median filtering with the Canny operator is employed in this paper to achieve the high accuracy of the iris segmentation.

Our procedures for performing the evaluation of the effectiveness of the eye massage therapy are as follows. First, the periocular images are taken before performing the eye massage therapy and after performing the eye massage therapy. Then, the iris in the images is segmented out using the CHT based method. After that, the region between the inner circle and the outer circle of the iris is mapped to a rectangular box. Next, the discrete cosine transform is used to normalize the size of the rectangular box to 32 × 64. By mapping the feature matrix to a feature vector, various classifiers including the random forest based classifier, the k nearest neighbors based classifier, the support vector machine based classifier and the artificial neural network based classifier are employed for performing the classification. The computer numerical simulation results show that all the classifiers can achieve the high classification accuracies. Compared to the Daugman method, the CHT based method can achieve the higher classification accuracies.

The contributions of this paper are as follows. First, a method for performing the evaluation of the effectiveness of the eye massage therapy is proposed. The computer numerical simulation results scientifically validate the effectiveness of the eye massage therapy. This implies that the eye message therapy can be performed in a quicker and a cheaper manner.

The outline of this paper is as follows. Section 2 reviews the CHT and the normalization method. Section 3 presents the method for performing the evaluation of the effectiveness of the eye massage therapy. The computer numerical simulation results are shown in Section 4. Finally, a conclusion is drawn in Section 5.

For the ease of the comprehension and the reading, the acronyms and the symbols appeared in this paper are summarized in Table 1.

Table 1

Acronyms and symbols

Long forms	Acronyms
CHT	CHT
support vector machine	SVM
discrete cosine transform	DCT
receive operating characteristic curve	ROC
area under ROC	AUC
k nearest neighbors	KNN
artificial neural network	ANN

Review on the CHT and the normalization method

CHT

The CHT is effective for characterizing the regular objects such as the straight line and the circular objects such as the iris in an image. It can be extended for the detection of the ellipses and any object with other shapes that could be characterized using the parameters [3]. Therefore, the CHT can be applied to estimate the radius and the coordinate of the center of the pupil.

For each edge point, if a circle is drawn with its center being this edge point, then the intersecting point of all these circles represents the center of the circle of all these edge points. Figure 1 illustrates this concept. Let r be the radius and (a,b) be the coordinate of the center of the circle. Let (x,y) be the coordinates of the edge points. Obviously, if x,y is on the circle with its radius r and its center coordinate (a,b), then it should satisfy [5, 11, 22]:

Fig. 1

Method for finding the center of the circle

DefineHere, votes if (x,y) is on the circle. DefineHere, H(a,b,r) counts the total number of the edge points on the circle. The parameter pair (a,b,r) in the above should maximize H(a,b,r) and this parameter pair can be used to represent the edge points in the corresponding circle.

Normalization

Normalization is a very important step that can be used for mapping the points between the inner circle and the outer circle of the iris defined by the CHT to a two dimensional rectangular box. Here, the rubber sheet model is employed for performing the normalization [16]. Since only the points in the iris are extracted, it can remove the irrelevant information outside the iris for performing the iris classification.

Let (r,θ) and (r,θ) be the coordinates of the points in the inner circle and the outer circle of the iris represented in the polar coordinate system, respectively. Let (r(θ),y(θ)) and (x(θ),y(θ)) be the coordinates of the points in the inner circle and the outer circle of the iris along the direction θ represented in the Cartesian coordinate system, respectively. Let (x,y) and (x,y) be the coordinates of the centers of the inner circle and the outer circle of the iris represented in the Cartesian coordinate system, respectively. Then, we haveand

Let x(r,θ) and y(r,θ) be the x co-ordinates and the y co-ordinates of the points between the inner circle and the outer circle of the iris represented in the Cartesian coordinate system, respectively. Let wϵ[0,1]. Then, the rubber sheet model maps the points between the inner circle and the outer circle of the iris to a rectangular box as follow [5, 22]:and

Figure 2 shows the rubber sheet model.

Fig. 2

The rubber sheet model

Evaluation of the eye massage therapy

The objective of this paper is to evaluate the effectiveness of an eye massage therapy via classifying the periocular images taken before performing the eye massage therapy and those after performing the eye massage therapy. The procedures for performing the iris classification [13] are shown in Fig. 3. Here, it includes (i) the image acquisition, (ii) the iris segmentation, (iii) the normalization, (iv) the feature extraction and (v) the iris classification.

Fig. 3

The procedures for performing the iris classification

Image acquisition

Compared to the image acquisition systems used in the dry eye clinic such as the near infrared iris system, the images employed in this paper are acquired by a mobile device. Obviously, the images can be acquired easier and more convenient. First, a video with its duration equal to 15 s is taken around the eye before performing the eye massage therapy. Next, an eye massage therapy is performed using an eye massage device for 15 min. Then, a video with its duration equal to 15 s is taken around the eye after performing the eye massage therapy. In order to minimize the effects of the devices, the same mobile device is used to capture all the videos and the same eye massage device is employed to perform the therapy for all the subjects. In order to ensure that the obtained results are reliable, the videos are acquired from 10 different subjects. Each subject has conducted 10 times of the data acquisitions. Here, the data acquisitions at different times are conducted at the same place and at the same time slots of different days. Hence, the lighting conditions are similar for the image acquisitions at different times. Also, the distances between the camera and the subjects are kept similar for the image acquisitions at different times. Totally, 200 periocular videos are acquired and 518 images are selected from these 200 videos. Two examples of the periocular images are shown in Fig. 4.

Fig. 4

Two examples of the periocular images

Iris segmentation

Various components of the eye are shown in Fig. 5. It can be seen that the eye mainly consists of three components [22]. They are the sclera, the iris and the pupil which located from the outer part of the eye to the inner part of the eye, respectively. In particular, the white outer area represents the sclera. The back inner area is the pupil which is a small black circle in the center of eye. The iris lies in the annular region between the sclera and pupil. It is worth noting that the lower part and the upper part of the iris region are sometimes occluded by the eyelids and eyelashes, respectively.

Fig. 5

Various components of the eye

In order to perform the iris segmentation, the CHT based median filtering with the Canny operator is proposed. The block diagram for performing the iris segmentation is shown in Fig. 6. First, the median filtering is applied to suppress the noise. Then, the Canny operator is employed to perform the edge detection. Next, the CHT is employed to characterize the iris.

Fig. 6

The block diagram for performing the iris segmentation

For performing the Canny edge detection, the block diagram is shown in Fig. 7. In particular, the gradients of the pixel values are computed. Then, a thresholding operator is applied to obtain the edge map.

Fig. 7

The block diagram for performing the Canny edge detection

The details of each step are discussed below.

Median filtering

Let (i,j) be the starting index of a block of an image. Let x be the grayscale pixel value at the index (p,q). Let median(∙) be the median operator. Let r × s be the size of a window. Let be the output of the median filter. Then,

From here, it can be seen that the median filtering is to sort the pixels in a window and to take the pixel value located in the middle of the sorted pixels as the output of the filter. Figure 8 shows an example of output after performing the median filtering. It is worth noting that the median filtering is a typical nonlinear operator used to suppress the impulsive noise. Compared to the conventional linear filtering techniques, this nonlinear filtering technique has a lower blurring effect and better preserves the edges in the image [9].

Fig. 8

The median filtering

As the camera introduces the impulsive noise to the periocular images and the noise degrades the quality of the periocular images, this paper employs the median filtering for suppressing the noises. First, each frame is extracted from each video. Then, the median filter with the size of the window equal to 3 × 3 is used for performing the denoising. The original image and the image denoised by performing the median filtering are shown in in Fig. 9a and Fig. 9b, respectively. It can be seen from Fig. 5 that the eyelashes near the iris are removed. Hence, it can facilitate the iris classification.

Fig. 9

The original image and the denoised image

Canny edge detection

The edge of an object is the border across two regions in the image in which these two regions have two sets of contrast pixel values. In other words, the brightness of the pixels changes sharply or more formally has the discontinuities across these two regions in the image. Performing the edge detection is to identify the pixels at the border. As the border across an object and the background can be identified via performing the edge detection, the edge detection plays an important role in the image segmentation. Moreover, as the edge points preserve the fundamental structures of the object, some important features such as the locations of the lines or the curves in the image can be extracted via performing the edge detection. In addition, only processing these edge points can reduce the required computational power for some image processing applications.

The Canny edge detection is the most common edge detection algorithm. It performs the following steps [21].

Computing the gradients: Let G and G be the horizontal gradient and the vertical gradient of an object in the image, respectively. Let G be the gradient of the object in the image. Then, we have

Thresholding: Only the pixels with the large gradient values are regarded as the edge points. Therefore, an appropriate threshold value is used to determine whether the pixels are located at the edge points or not.

It is worth noting that the iris segmentation is crucial for performing the iris classification. This is because a wrong segmentation of an object would result to the meaningless of the subsequent steps. First, the median filtering is performed on each color plane of the image as discussed in Sect. 3.2.1. Then, the Canny edge detection is applied to denoised image. Figure 10a and Fig. 10b show the inner edge points and the outer edge points of the iris in the image after performing by the Canny edge detection algorithm, respectively. It can be seen from Fig. 10 that the closed shape of the iris can be detected. Finally, the iris region is segmented from each color plane.

Fig. 10

a The inner edge points and b the outer edge points of the iris in the image after performing the Canny edge detection

Inner circle and outer circle

After performing the Canny edge detection to obtain a binary edge map, the edge points are employed to find the parameters of the circles via the CHT. The inner circle and the outer circle are obtained via performing the CHT. Figure 11 shows the image with the inner circle and the outer circle of the iris.

Fig. 11

a The inner circle and b the outer circle of the iris found using the CHT

Feature extraction and classification

After mapping the points between the inner circle and the outer circle of the iris to the two dimensional rectangular box, the sizes of the rectangular boxes are different for different images. In order to eliminate the size inconsistence issue, the DCT is applied to normalize the size of the rectangular box to 32 × 64. That is, if the original size of the rectangular box is smaller than 32 × 64, then the zeros are padded to the rectangular box in the DCT domain. On the other hand, if the original size of the rectangular box is larger than 32 × 64, then the high frequency components of the rectangular box are discarded in the DCT domain. This can also suppress the high frequency noise generated by the specular reflections. Next, the inverse DCT is applied to the processed rectangular box and the rectangular box with the unified sizes are obtained. Finally, the resized rectangular box is represented as a vector with its dimension equal to 2048 × 1. This vector is employed as the feature vector for performing the classification. Figure 12a shows that the circular region in the iris before performing the normalization and Fig. 12b shows the resized rectangular box.

Fig. 12

a The circular region in the iris before performing the normalization. b The resized rectangular box

Computer numerical simulation results

In this paper, the images are divided into two categories. They are the images before and after performing the eye massage therapy. Here, 70% of the selected images are employed for performing the training. On the other hand, the rest 30% of the selected images are employed for performing the testing. The overall total numbers of the selected images in the training set and those in the testing set are summarized in Table 2.

Table 2

The overall total numbers of the selected images in the training set and those in the testing set

Identity numbers of the subjects	1	2	3	4	5	6	7	8	9	10
Total number of the images selected from each subject	24	40	40	40	42	44	48	80	80	80
Total number of the selected images	518
Total number of the selected images in the training set	363
Total number of the selected images in the test set	155

Since the classification accuracy is the most common criterion employed for evaluating the pattern recognition performance, this paper employs the classification accuracy as the metric. In order to demonstrate the effectiveness of performing the eye massage therapy, several classifiers are employed. In particular, the random forest based classifier, the KNN based classifier, the SVM based classifier and the ANN based classifier are employed. The Daugman method is employed as the benchmark method for evaluating the performance of the iris segmentation. Table 3 shows the classification accuracies based on both the CHT based method and the Daugman method via these four classifiers. It can be seen from Table 3 that the classification accuracies are high for both methods and for all these four classifiers. This implies that it is possible to classify into two types of images namely those obtained before performing the eye massage therapy and those obtained after performing the eye massage therapy. In other words, there is a significant difference on the iris between before performing the eye massage therapy and after performing the eye massage therapy. Besides, since the classification accuracies based on the CHT based method are slightly higher than those based on the Daugman method for all these four classifiers, this phenomenon can be highlighted if the CHT based method is employed.

Table 3

The classification accuracies based on the test set achieved by various classifiers

Classifiers	RF	KNN	SVM	ANN
CHT based method	0.8526	0.8077	0.7115	0.7692
Daugman method	0.7308	0.7885	0.6795	0.7436

The confusion matrices are employed to further evaluate the classification performances achieved by various methods. In particular, the confusion matrices achieved by the CHT based method and the Daugman method via these four classifiers are shown in Fig. 13 and Fig. 14, respectively. It can be seen from Fig. 13 and Fig. 14 that the obtained results are consistent with those shown in Table 3. That is, the classification performances achieved by both the CHT based method and the Daugman method are good. Besides, the classification accuracies based on the CHT based method are slightly higher than those based on the Daugman method for all these four classifiers.

Fig. 13

The confusion matrix achieved by the CHT based method via a the RF based classifier, b the KNN based classifier, c the SVM based classifier and d the ANN based classifier

Fig. 14

The confusion matrix achieved by the Daugman method via a the RF based classifier, b the KNN based classifier, c the SVM based classifier and d the ANN based classifier

The received operating characteristic curve (ROC) is employed as the performance metric to evaluate the robust performance achieved by various classifiers. The ROC plots the true positive rate (TPR) against the false positive rate (FPR). If the FPR is small, then the classification accuracy will be high. Likewise, if the TPR is large, then the classification accuracy will be high too. The ROC can be divided into two parts. In particular, the slopes of the curve in the first part and the second part are high and low, respectively. Therefore, the ROC is also known as the sensitivity curve. Figure 15 plots the ROC curves achieved by both the CHT based method and the Daugman method via different classifiers. It can be seen from Figure 15 that all the ROCs achieve the good performances. This implies that all the classifiers are robust for performing the classification. This also demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy.

Fig. 15

a The ROC achieved by a the CHT based method and b the Daugman method

The area under the ROC (AUC) is used to evaluate the prediction accuracy. In general, the higher the AUC refers to the higher the prediction accuracy. Since the quicker response of the ROC corresponds to the smaller region with the small values of the FPR and the larger region with the large values of the TPR, the quicker response of the ROC corresponds to the larger value of the AUC and the higher prediction accuracy. As the ROC plots the TPR against the FPR, the points above the 45° straight line in the plot correspond to the accurate predictions. Since the range of the AUC is between 0 and 1, the area under the 45° straight line in the plot is 0.5. If the AUC is equal to 0.5, then the model likes performing a random guess such as flipping a coin. In this case, the prediction does not tell any information. On the other hand, if the AUC is equal to 1, then the model predicts correctly and the classification is perfect. However, there is not perfect classification in practice. Hence, the classification is effective if the AUC is between 0.5 and 1. Table 4 shows the AUCs achieved by both the CHT based method and the Daugman method via various classifiers. It can be seen from Table 4 that the AUCs achieved by both the CHT based method and the Daugman method are large for all these four classifiers. This demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. Besides, the CHT based method is superior than the Daugman method. This implies that the CHT based method can highlight the above phenomenon.

Table 4

The AUCs based on the test set achieved by various classifiers

Classifiers	RF	KNN	SVM	ANN
CHT based method	0.8971	0.8478	0.7114	0.7935
Daugman method	0.8048	0.7865	0.6554	0.7477

In order to evaluate the reliability performances achieved by various classifiers, the ten fold cross validations are performed. Table 5 shows the mean classification accuracies based on these ten fold cross validations achieved by both the CHT based method and the Daugman method via different classifiers. It can be seen from Table 5 that the mean classification accuracies based on these ten fold cross validations achieved by both the CHT based method and the Duangman method are high. This also demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. Besides, the mean classification accuracies based on these ten fold cross validations achieved by the CHT based method are slightly higher than those achieved by the Duangman method, this implies that the CHT based method can highlight the above phenomenon.

Table 5

The mean classification accuracies based on these ten fold cross validations achieved by the CHT based method and the Daugman method via different classifiers

Methods	CHT based method				Daugman method
Classifiers	RF	KNN	SVM	ANN	RF	KNN	SVM	ANN
Accuracy in the first validation	0.6154	0.6731	0.3462	0.5962	0.4615	0.3846	0.4808	0.4808
Accuracy in the second validation	0.5385	0.75	0.6538	0.6538	0.5769	0.5577	0.5	0.5769
Accuracy in the third validation	0.7308	0.7692	0.6538	0.6346	0.6154	0.5192	0.5192	0.5769
Accuracy in the fourth validation	0.8461	0.8269	0.5769	0.7115	0.6346	0.5769	0.4615	0.5385
Accuracy in the fifth validation	0.8077	0.8077	0.6923	0.7308	0.5385	0.4808	0.5962	0.5385
Accuracy in the sixth validation	0.8462	0.6923	0.6923	0.75	0.75	0.5769	0.5962	0.5962
Accuracy in the seventh validation	0.6731	0.8077	0.6154	0.6923	0.75	0.5769	0.6731	0.6731
Accuracy in the eighth validation	0.75	0.6885	0.7115	0.7308	0.6154	0.4902	0.5	0.5385
Accuracy in the ninth validation	0.8039	0.8824	0.6471	0.8039	0.5490	0.4706	0.4510	0.5882
Accuracy in the tenth validation	0.9608	0.8039	0.7059	0.8039	0.4314	0.5577	0.4510	0.5686
Mean accuracy across these ten validations	0.7572	0.7801	0.6295	0.7108	0.5923	0.5192	0.5229	0.5676

Conclusion

This paper evaluates the effectiveness of an eye massage therapy via performing the classification between the images taken before performing the eye massage therapy and those after performing the eye massage therapy. Here, 10 subjects have taken 200 videos using a mobile device and 518 images are selected for performing the iris classification. First, a median filtering is employed for suppressing the noises. Then, the Canny edge detection is employed for generating the edge maps. Next, the CHT is employed to characterize the inner circle and the outer circle of the iris in the images. After that, the normalization process is to map the region between the inner circle and the outer circle of the iris to a rectangular box. Then, the DCT is employed to generate the feature vectors. Finally, various classifiers are employed for performing the classification.

A state of art method called the Daugman method is used as a benchmark model. The computer numerical simulation results demonstrated that both the CHT based method and the Daugman method are effective, robust and reliable for performing the classification. This demonstrates that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. Besides, the CHT based method is slightly effective, robust and reliable than the Daugman method. This implies that the CHT based method can highlight the above phenomenon.

In future, more analysis on the eye massage therapy will be investigated. In particular, the relationship between the duration of each eye massage therapy and its effects on the iris images as well as the relationship between the environments for conducting the eye massage therapy such as the relax conditions of the subjects and their effects on the iris images will be investigated.