Image Retrieval Using Different Distance Methods and Color Difference Histogram Descriptor for Human Healthcare.

As multimedia technology is developing and growing these days, the use of an enormous number of images and its datasets is likewise expanding at a quick rate. Such datasets can be utilized for the purpose of image retrieval. This research focuses on extraction of similar images established on its different features for the image retrieval purpose from huge dataset of images. In this paper initially, the query image is searched within the available dataset and, then, the color difference histogram (CDH) descriptor is employed to retrieve the images from database. The basic characteristic of CDH is that it counts the color difference stuck among two distinct labels in the L ∗ a ∗ b ∗ color space. This method is experimented on random images used for various medical purposes. Various unlike features of an image are extracted via different distance methods. The precision rate, recall rate, and F-measure are all used to evaluate the system's performance. Comparative analysis in terms of F-measure is also made to check for the best distance method used for retrieval of images.

1. Introduction

In this society of modern era, the innovation to recover an image is widely used in a variety of fields, and it represents a viable solution for retrieving a comparative image from a series of images. With the advancement of the web, countless images would now be available in the fields of medication, training, sciences, business, and different fields. Depending upon the visualized characteristics, the images are categorized. A computer system for searching images from a huge database is known as an image retrieval system. To search any image, the query is provided by the user in terms of any phrase or a file/link, and another tag; the interface will provide images that are comparable to the query image [1]. Tags, image color distribution, region shape features, and other similarity-based search criteria could be employed.

The availability of imaging devices, such as digital cameras and scanners is increasing, and the size of a wide variety of digital photography is also increasing. These devices are supportive in the different fields of medication, teaching and learning, sciences and trade and commerce [2]. In the field of medicine and human healthcare, different images can be searched to facilitate the work of practitioners with the purpose of improving the accuracy in their designated work. In this manner, the image retrieval has evolved into a dynamic research topic in today's times. Mostly CBIR is concentrated on reduced level options of the image. Although it is accepted that differences in visual sensory activity involving two colors in color space are related to the distance being calculated, the presentation of a particular characteristic of an image is compelled to be more studied [3, 4]. The color difference histogram (CDH) is one of the CBIR descriptors discussed in this paper.

Feature extraction has become an important feature in image retrieval systems. Shaila and Vadivel [5] presented that according to human color vision, the histogram is designed to retrieve image-based content. For each pixel the gray scale images and their colors are estimated by using the significant weight function. Zhou and Huang [6] elaborated a comparison of various variations of the initial Hu moment invariants, which illustrates and retrieves two-dimensional (2D) objects with one cyclic contour to distinguish pathology from traditional brains. For feature extraction, Burger and Burge [7] applied ripple entropy (RE) and Hu moment invariants (HMI) and got the scanning done through magnetic resonance imaging (MRI). Gonzalez and Woods [8] represented that images were extracted using edge detection techniques; the various techniques used in the algorithm were signal processing and image compression. Bhute and Meshram [9] performed the experiment which was carried out on a wide range of image descriptors. The color histogram performance is compared with all other different descriptors. Deselaers et al. [10] presented a dominant color descriptor (DCD) to increase image retrieval correctness. Min and Cheng [11] described the descriptor totally based on color which represents the content of an image after combining both global and local characteristics. Fierro-Radilla et al. [12] hypothesized a novel semantic characteristic derived from dominant hues. Talib et al. [13] proposed the Spatial Dominant Color Descriptor (SDCD) as a top-down descriptor. Rejeb et al. [14] presented the approach of CDH which counts under various backdrops, the perceptually consistent color difference between two points in Lab color space. Liu and Yang [15] proposed the innovative approach to complete the retrieval process employed on color, texture, and shape. The “top hat transform” was used by Tajeripour et al. [16] to recognize and crop image components based on color and shape information. Yu et al. [17] presented an approach to improve the accuracy of diagnosis and to help endoscopists in order to identify various categories of lesions. The categorization task now includes an image retrieval element which provides a supplementary assurance to predict various types of esophageal lesions. Talouki et al. [18] presented an idea to apply neutrosophic space in image retrieval applications which helps in improving average recall and precision compared to conventional methods.

From the above literature, it has been concluded that different approaches have been used for image retrieval systems and to the top of our awareness nobody has offered the comparisons between different distances for the retrieval of images. In this paper, an attempt has been made to compare different distances to get back the images and the major contributions of our work in this article are as follows:

The color difference histogram is implemented as a descriptor for the image retrieval process

HSV histogram, color moments, autocorrelogram, and wavelet moments are among the color features retrieved from the dataset's and query image's images

To evaluate the difference in color, distances linking the query and the output and various distance methods like Euclidean distance, Manhattan distance, and Hamming distance are used

Different performance metrics like precision, recall rate, and F-measure are computed for distinct distance approaches

Comparative analysis using different performance metrics has been made for different features to check the effectiveness of the system

The remaining document is broken into four sections. The second section has brief descriptions of materials and processes to be used. In Section 3, the results are discussed. Section 4 depicts the culmination of the suggested task.

2. Materials and Methods

The below division contains a complete depiction of the methodology that has been proposed for retrieval of images and the dataset used for evaluation of the proposed model.

2.1. Datasets

There are a number of databases for which content-based retrieval has been conducted. Table 1 provides a description of different data sets, together with number of images, various types, and image size. In this paper, the simulation is performed on random images used for many healthcare purposes [19, 20].

Table 1

Datasets available for images retrieval.

Sr. no.	Name of the dataset	References used	Number of images	Number of categories	Size
(1)	Corel 1K	[21–23]	1000	10	256 × 384
(2)	Corel 5K	[21, 24]	5000	50	192 × 128
(3)	Corel 10K	[21, 24]	10,000	100	192 × 128
(4)	Corel-1500	[23]	1500	15	256 × 384
(5)	Salzburg texture (STex)	[21]	7616	476	1024 × 1024
(6)	MIT-VisTex	[21]	40	40	512 × 512
(7)	UFI (unconstrained facial images)	[25]	400	40	128 × 128
(8)	European 1M dataset	[22]	1081	—	—
(9)	LFW(Labeled faces in the wild)	[22]	13000		—
(10)	GHIM-10K	[24]	10000	20	300 × 400
(11)	Caltech-256	[23]	29,780	256	260 × 300

2.2. Proposed Methodology

The proposed methodology is basically focused onto retrieve images based on color difference histogram approach which is represented in the flowchart as shown in Figure 1. From enormous collection, the query image is posted after the images from the database have been loaded. After that, CDH is implemented for applications of feature extraction from images there in the record and on the uploaded image as well. Euclidean, Manhattan, and Hamming distances are used to calculate the nearest distance of similar images with the query image in the dataset. Appropriate images found are sorted as per the distance and compared with the threshold values. At the end, different performance metrics like precision, recall, and F-measure are computed through retrieving images using special distance methods. The detail of each stage of the proposed algorithm is explained in subsequent subsections.

Figure 1

Flow chart of proposed algorithm.

2.2.1. Loading of Images from Database

The random images to be used for various healthcare purposes are collected. Each image is 128 ∗ 192 pixels in size [21, 24]. This database is quite heterogeneous in nature which includes images of different body parts. For the experimental purpose, initially a dataset of 100 images is loaded which includes the four different categories of images like eye, nose, hand, and ear. Each category includes a set of 25 images and the size of each image computed at the time of simulation is 187 ∗ 126.

2.2.2. Resizing of Images

In digital image processing, image resizing has become an essential aspect and is applied in various fields. The resizing of all the images loaded from the database is done and each image is resized to a new scale of 384 ∗ 256. Hence the images are preprocessed at a fixed scale as shown in Table 2 and resized image for eye is represented by Figure 2(a).

Table 2

Description of images.

Classes of image	Loaded image	Size of original image	Size of resized image
Eye		[187 × 26]	[384 × 256]
Nose		[187 × 26]	[384 × 256]
Hand		[187 × 26]	[384 × 256]
Ear		[187 × 26]	[384 × 256]

Figure 2

(a) Resized image. (b) Lab color space. (c) Edge detected.

2.2.3. Extraction of Features of All Images of Dataset Using CDH

The CDH descriptor [22] evaluates the same color difference among two points beneath diverse domains in terms of direction of edge and color in Lab (CIELAB). It is preferred because the observed visual difference linking the colors in the Lab color is associated to distance measurement while components of RGB are extremely connected. As a result, the chromatic details are not straightforwardly related to the application. The CDH also takes into account the composition of the area without image fragmentation, learning processes, or clustering implementation [14]. The algorithm implemented for CDH is described by the flowchart as shown in Figure 3 and briefly explained in the following subsections.

Figure 3

Flow diagram for feature extraction of image.

(1) Image Conversion. In Lab color space, light is considered from black (0) to white (100) in L, from green (−) to red (+) in a, and from blue (−) to yellow (+) in b. RGB image is converted into Lab color space image as shown in Figure 2(b).

L , a, b can be evaluated by (1)–(3) respectively [7, 8, 14].

f (u) can be calculated as represented bywhere“where x, y and z are the values of x, y, and z for illuminant” as represented in (5) (reference white point) and  =  [7, 15].

(2) Edge Detection Using Sobel Operator. Most of the chromatic details will be lost if gradient size and form are calculated using a gray scale image. Therefore, edge shape recognition in Lab color space is used. Sobel operator is applied for detection purpose because it is a reduced amount of noise and has a lower computation load. The approximation of the gradients of the image intensity function is computed using this operator. The edges of an image are detected using gradient method and the maximum and minimum magnitudes are computed in the first derivative method. A grayscale or binary image is applied at input which returns the gradient magnitude Gmag and the gradient direction Gdir. Gmag and Gdir are the same size as of the input image and such edge detected image is represented by Figure 2(c). The various steps for edge detection are as follows:

1st Step: Apply the image

2nd Step: Mark Gmag, Gdir implied to the input image

3rd Step: Gradient as well as Sobel operator is applied

4th Step: Management of Gmag, Gdir discretely on the input image

5th Step: Combining the outputs for finding fixed gradient magnitude

Step 6: Magnitudes computed are called as output edges

(3) Quantization of L. Color quantization operation is performed so as to quantize the L channel into 10 bins and a and b channels into 3 bins. A color combination of 10 ∗ 3 ∗ 3 = 90 is obtained [15]. With the help of the quantization process, the amount of colors is reduced.

2.2.4. Extraction of Features of Image

The basic characteristic of an image is represented by different features present in it. In image processing, feature extraction is very significant. These features are divided into three categories: low, middle, and high level. Color and texture are low-level features, while shape is a middle-level feature and semantic gap is a high-level feature. The various kinds of features like HSV histogram, auto correlogram, color moments, and wavelet moments are extracted in the proposed work for all images of the dataset. The description of each feature is mentioned in coming subsections. The size of each feature extracted is shown by Table 3.

Table 3

Types of features extracted.

Sr. no.	Name of feature	Size
(1)	HSV histogram	1 ∗ 32
(2)	Auto correlogram	1 ∗ 64
(3)	Color moments	1 ∗ 6
(4)	Wavelet moments	1 ∗ 40

(1) HSV Histogram. Color is a key characteristic for describing the content of an image. The HSV histogram is a color representation method that represents the proportion of specific colors in an image. It provides HSV color space and RGB color space which shows how many pixels of each color are in there in the image. The database then stores the HSV histogram for each image. It is estimated showing the proportion of pixels of each color in between the image. Then database is stored with the HSV histogram for each image. During the search, the user has the option of specifying the required color proportions or submitting a reference image from which the histogram is calculated. The matching method then finds the images whose color histograms most closely match the query image. The size of HSV histogram is calculated as 1 ∗ 32.

(2) Auto Correlogram. Color information has an auto correlogram feature. The color correlogram has several advantages, including the ability to illustrate the worldwide circulation of correlation of colors and the ease with which it can be computed. The size of auto correlogram is calculated as 1 ∗ 64.

(3) Color Moments. To identify images, color moments are used based on their color characteristics. Similarity between images can be measured using these moments. For image retrieval, these similarity values can be compared to image indexing database values. Only the color information of each pixel in a picture is contained in the color histogram, color moments, and color set. Color moments are metrics that can be used to distinguish images based on their color characteristics. It provides a measurement for color similarity between images once they have been calculated. The size of color moments is calculated as 1 ∗ 6.

(4) Wavelet Moments. The wavelet moments convert the image into a multiscale representation with spatial and frequency properties. This enables efficient multiscale picture analysis at a lower cost of computation. The wavelet transform is a generally used method in computer vision and image processing. Many applications are already examined, including compression, detection, identification, and picture retrieval. Wavelet transforms are used to represent both shape and texture. The size of wavelet moments is calculated as 1 ∗ 40.

2.2.5. Upload Query Image

After extraction of features of all images of dataset, next step is to upload query image from the huge dataset. The images used for simulation purpose are from various classes for eye, nose, hand, and ear as shown in Table 2. Simulation is performed on all the four classes of images.

2.2.6. Extraction of Features of Query Image Using CDH

Image retrieval is done by matching various features of query and the retrieved image. Color histograms are analyzed for both the images. The matching of both images is performed and the distances between the feature vectors of the query and the database image are evaluated and used as a similarity dimension.

2.3. Computation of Distance between All Images of Dataset and Query Image

In image retrieval systems, distance measuring is extremely important. It is very important to identify how the query image is interrelated with the dataset of images and how they are similar to each other. Different below mentioned distances are used to present work which is briefly explained in the following.

(1) Euclidean Distance. Euclidean distance is defined as the most relevant method to find the distance between two points. Suppose we have (u, v) as the two points where u = (a1, b1) and v = (a2, b2), then Euclidean distance between these points is calculated as described by

But if the points have n number of dimensions instead of two, then Euclidean distance can be generalized by

(2) Hamming Distance. The Hamming distance can be thought of as the range of possibilities bits that need to be modified (corrupted) in order to show a single string into the other. It relates to the difference between two equal strings as shown by

(3) Manhattan Distance. Manhattan distance also named as the L1 distance and it calculates the absolute difference between two points. If u = (a1, b1) and v = (a2, b2) are the two points, then Manhattan distance between these points is calculated using

But if the points have n number of dimensions instead of two, then Manhattan distance can be generalized by

2.4. Sorting of Distances and Retrieval of Images Using Thresholding

After the computation of different distances, image retrieval operation is performed using an approach based on thresholding. This technique is used to retrieve most relevant images analogous to the query image from database. To achieve this, distances computed between the query image and retrieved image are sorted in ascending order and then only the images which have distances less than threshold values are considered. The threshold value employed in this work is determined by hit and trial approach. The value of threshold is chosen for the simulation purpose and is computed by taking 40% of the maximum distance obtained.

2.5. Evaluation of Performance Metrics

There are various methods to evaluate the performance of image retrieval systems. The precision is described by

The recall is defined as number of relevant images extracted to the total number of images in the dataset which is computed using

F-measure is the harmonic mean of precision and recall defined as

3. Results and Discussion

The implementation of the proposed technique is performed in MATLAB. After uploading of query image, similar images are retrieved which consist of all the true positive and false positive images. Results are evaluated on the basis of different distance methods. Precision and recall rate is calculated at a threshold value of 40% for different features extracted. The values of precision and recall using different distance methods applied for the extraction of various features of eye, nose, hand and ear image are shown in Tables 4–7 respectively.

Table 4

Performance parameter for eye image at different distances.

Sr. no.	Name of the features				Manhattan distance		Hamming distance		Euclidean distance
	HSV	Auto correlogram	Color_moments	Wavelet moments	Precision	Recall	Precision	Recall	Precision	Recall
1	√	X	X	X	1.00	0.40	1.00	0.60	0.95	0.80
2	X	√	X	X	1.00	0.04	1.00	0.04	0.17	0.24
3	X	X	√	X	0.92	0.88	0.48	0.96	0.53	1.00
4	X	X	X	√	1.00	0.12	0.55	0.68	0.53	0.84
5	√	√	X	X	1.00	0.04	1.00	0.04	0.50	0.20
6	√	X	√	X	0.92	0.88	0.48	0.96	0.53	1.00
7	√	√	√	X	0.92	0.88	0.48	0.96	0.53	1.00
8	√	√	√	√	0.95	0.80	0.52	0.96	0.54	1.00
9	X	X	√	√	0.95	0.80	0.52	0.96	0.54	1.00
10	√	X	√	√	0.95	0.80	0.52	0.96	0.54	1.00

Table 5

Performance parameter for nose image at different distances.

Sr. no.	Name of the features				Manhattan distance		Hamming distance		Euclidean distance
	HSV	Auto correlogram	Color_moments	Wavelet moments	Precision	Recall	Precision	Recall	Precision	Recall
1	√	X	X	X	1.00	0.08	1.00	0.20	0.70	0.28
2	X	√	X	X	1.00	0.04	1.00	0.04	0.33	0.32
3	X	X	√	X	0.77	0.40	0.52	0.60	0.40	0.76
4	X	X	X	√	0.73	0.32	0.41	0.48	0.34	0.56
5	√	√	X	X	1.00	0.04	1.00	0.04	0.75	0.12
6	√	X	√	X	0.77	0.40	0.52	0.60	0.40	0.76
7	√	√	√	X	0.77	0.40	0.52	0.60	0.40	0.76
8	√	√	√	√	0.82	0.36	0.59	0.52	0.44	0.76
9	X	X	√	√	0.82	0.36	0.59	0.52	0.44	0.76
10	√	X	√	√	0.82	0.36	0.59	0.52	0.44	0.76

Table 6

Performance parameter for hand image at different distances.

Sr. no.	Name of the features				Manhattan distance		Hamming distance		Euclidean distance
	HSV	Auto correlogram	Color_moments	Wavelet moments	Precision	Recall	Precision	Recall	Precision	Recall
1	√	X	X	X	1.00	0.40	1.00	0.56	0.65	0.68
2	X	√	X	X	1.00	0.04	0.50	0.12	0.20	0.56
3	X	X	√	X	1.00	0.24	0.81	0.52	0.42	0.56
4	X	X	X	√	0.75	0.12	0.58	0.28	0.40	0.32
5	√	√	X	X	1.00	0.04	1.00	0.16	0.60	0.48
6	√	X	√	X	1.00	0.24	0.81	0.52	0.42	0.56
7	√	√	√	X	1.00	0.24	0.81	0.52	0.42	0.56
8	√	√	√	√	1.00	0.24	0.86	0.48	0.48	0.56
9	X	X	√	√	1.00	0.24	0.86	0.48	0.48	0.56
10	√	X	√	√	1.00	0.24	0.86	0.48	0.48	0.56

Table 7

Performance parameter for ear image at different distances.

Sr. no.	Name of the features				Manhattan distance		Hamming distance		Euclidean distance
	HSV	Auto correlogram	Color_moments	Wavelet_moments	Precision	Recall	Precision	Recall	Precision	Recall
1	√	X	X	X	1.00	0.04	0.75	0.24	0.60	0.60
2	X	√	X	X	1.00	0.04	0.33	0.04	0.25	0.48
3	X	X	√	X	0.67	0.08	0.50	0.36	0.39	0.52
4	X	X	X	√	0.50	0.04	0.43	0.12	0.41	0.28
5	√	√	X	X	1.00	0.04	1.00	0.04	0.44	0.28
6	√	X	√	X	0.33	0.04	0.50	0.36	0.39	0.52
7	√	√	√	X	0.33	0.04	0.50	0.36	0.39	0.52
8	√	√	√	√	1.00	0.04	0.57	0.16	0.50	0.48
9	X	X	√	√	1.00	0.04	0.57	0.16	0.50	0.48
10	√	X	√	√	1.00	0.04	0.57	0.16	0.50	0.48

On the basis of values evaluated, each of the distance methods is represented in a graphical manner as shown in Figure 4. Figures 4(a)–4(d) represent the performance in terms of F-measure for Euclidean distance, Hamming distance, and Manhattan distance, respectively. It has been observed that if HSV and autocorrelogram features are extracted for eye image, F-measure is computed to 0.87 at 40% threshold value using Euclidean distance. It has also been observed for other images of nose, hand, and ear, if all the four features either applied individually or combined with different probabilities, that Euclidean distance gives a more accurate performance parameter of F-measure.

Figure 4

Performance chart for (a) eye, (b) nose, (c) hand, and (d) ear.

Comparative analysis for different distances is performed on the values of F-measure using the performance chart as represented in Figure 4. After computing this parameter, it shows that the Euclidean distance provided the best result as compared to all other distance methods because both the precision and recall rate are better for Euclidean distance as compared to all other distance methods.

4. Conclusion

In this paper, a color difference histogram descriptor is applied on the query image to retrieve the relevant similar images from the collection of images. Various types of features are extracted and different distance methods are used to retrieve the images. Performance of the system is represented using the precision rate, recall rate, and F-measure. In present work, simulation is performed on the small datasets using CDH. In future, large datasets having images related to different diseases and other different parts of the body can be used for retrieval of the images. Accuracy of the system can also be improved by using various types of descriptors based on texture and shape applied individually or in combination