Double-Granule Conditional-Entropies Based on Three-Level Granular Structures.

Rough set theory is an important approach for data mining, and it refers to Shannon's information measures for uncertainty measurements. The existing local conditional-entropies have both the second-order feature and application limitation. By improvements of hierarchical granulation, this paper establishes double-granule conditional-entropies based on three-level granular structures (i.e., micro-bottom, meso-middle, macro-top ), and then investigates the relevant properties. In terms of the decision table and its decision classification, double-granule conditional-entropies are proposed at micro-bottom by the dual condition-granule system. By virtue of successive granular summation integrations, they hierarchically evolve to meso-middle and macro-top, to respectively have part and complete condition-granulations. Then, the new measures acquire their number distribution, calculation algorithm, three bounds, and granulation non-monotonicity at three corresponding levels. Finally, the hierarchical constructions and achieved properties are effectively verified by decision table examples and data set experiments. Double-granule conditional-entropies carry the second-order characteristic and hierarchical granulation to deepen both the classical entropy system and local conditional-entropies, and thus they become novel uncertainty measures for information processing and knowledge reasoning.

1. Introduction

Rough set theory can effectively implement data mining for the imprecise, inconsistent, and incomplete information [1], and it has been extensively applied in artificial intelligence and machine learning [2,3,4,5,6,7,8]. In rough set theory, attribute reduction based on decision tables is a main topic for approximate reasoning and knowledge discovery, and there are three main construction strategies: from the positive region, information measure, and a discernibility matrix  [9,10,11,12,13,14,15]. By virtue of the discernibility matrix, Wei et al. [16] proposed an incremental reduction algorithm for dynamic data; Ma et al. [17] utilized the compressed binary discernibility matrix to construct an incremental reduction algorithm for group dynamic data; moreover, Nie and Zhou [18] proposed a new discernibility matrix defined by local conditional-entropies to compute the reduction core.

Information theory originated from Shannon’s entropy system [19], and it provides an effective method for uncertainty measurement, such as in attribute reduction. Currently, information theory has been introduced into rough set theory for uncertainty analyses and information processing [20,21,22,23,24,25]. As far as attribute reduction is concerned, Miao [26] offered the informational representation of knowledge reduction and decision reduction, where entropy and mutual-information are highlighted; Wang et al. [27] conducted a comparative study on attribute reduction from the algebra and information viewpoints, where the conditional-entropy acts as a main tool; Jiang et al. [28] presented the relative decision entropy to propose a feature selection algorithm; Slezak [29] used the conditional-entropy to define approximate reducts; moreover, Qian and Shu [30] provided the mutual information criterion to evaluate candidate features in incomplete data. In general, the entropy, conditional-entropy, and mutual-information together constitute the classical information system with integrality and comprehensiveness, and they can function on rough set applications (such as attribute reduction) but may exhibit different emphases in different application scenarios. In addition, information-theoretic measures have multiple variational forms [31,32,33,34,35]. As far as conditional-entropies are concerned, they are extensively applied in rough set theory from multiple pointcuts [26,27,29,31,34,36,37,38,39], while uncertainty measurement and reduction construction still serve as two basic issues. Aiming at probabilistic rough sets, Deng and Yao [40,41] used Shannon’s entropy and conditional-entropy to interpret and determine probabilistic thresholds by an information-theoretic approach, and Ma et al. [42] considered variants of conditional-entropies to construct heuristic reduction algorithms for the probabilistic model. In particular, local conditional-entropies are put forward by adopting double condition-granules and their union locality [18], and they can distinctively determine a new discernibility matrix for reduction core computation; moreover, the information measures exhibit a novel feature of second-order expressions, especially when compared to the traditional entropy system with only single-granule descriptions [19,26,27].

Granular computing is a structural methodology of hierarchical computing and information processing [43,44], and its technology of multi-granularity and multiple levels is useful for uncertainty analyses and knowledge acquisition regarding data. In rough set theory, the information granulation is of extensive concern [45,46,47,48,49], and the granulation monotonicity plays an important role in attribute reduction [12,50,51,52]. In particular, a decision table acts as a formal background of data mining [12,53,54,55], and it involves condition/decision granules and classifications from granular structures. According to granular computing, Zhang and Miao [56] introduced three-layer granular structures of decision tables, and they further hierarchically constructed three-way informational measures based on weighted-entropies; moreover, Wang et al. [57] utilized three-layer granular structures to research three-way weighted combination-entropies. These studies adhere to three-level analyses, and the latter are directly related to granular computing [43] and three-way decisions [58], as well as their interplay. Recently, Yao [59] discussed three-way granular computing by making use of two particular types of three granules and three levels, where thinking in three levels results in an important model. Additionally, three-level analyses were extensively utilized in the location allocation and programming/optimization modeling [60,61,62].

According to [18], the new discernibility matrix is used for reduction core calculations, and its creative implementation mainly depends on local conditional-entropies. Therefore, local conditional-entropies focus on the granule-union locality rather than their underlying double-granule interaction, and the latter more essentially adheres to the second-order characteristic; moreover, they lack the condition granulation to restrict their uncertainty measurement function and information procession prospect based on knowledge. Motivated by the two issues, this paper utilizes the two-granular essence and three-hierarchical evolution to propose double-granule conditional-entropies based on three-level granular structures. Regarding the contribution, this novel type of information measures improves local conditional-entropies from both the granular interaction and hierarchical/conditional granulation, and they will achieve multiple important properties (including the integration hierarchy, number distribution, calculation algorithm, three bounds, and granulation non-monotonicity) to offer both robust measurement functions and knowledge-application prospects. Moreover, three-level granular structures here (including micro-bottom, meso-middle, macro-top) adopt only the condition part of decision table, and thus they differ from and push forward the previous ones, which include both the condition and decision parts [56].

The remainder of this paper is organized as follows. Section 2 reviews the decision table and local conditional-entropies; Section 3 proposes and studies double-granule conditional-entropies from three-level granular structures; Section 4 provides a decision table example for mechanism illustration; Section 5 makes data experiments for effectiveness verification; finally, Section 6 concludes this paper.

2. Decision Table and Its Existing Entropy Measures

Rough set theory [1] focuses on the data that are represented in an information table U is the universe with finite objects, is the finite attribute set, is the value domain for , and is an information function to endow each object x with a value on attribute a. The decision table is a special type of information table with and , where C and D denote the sets of condition attribute and decision attribute, respectively, and it is simply denoted by in this paper. Furthermore, the granulation construction usually considers two parts.

The condition attribute subset induces an equivalence relation and the latter provides the condition granulation or partition , where represents the equivalence granule to exhibit number .

Similarly, the decision attribute set D induces the equivalence relation and further decision classification , which consists of decision classes.

The decision table and its granulation from and D constitute the basic background for information measure construction. The probability space establishes the usual probability framework, where and thus two usual probabilities are

([26,27,56]). The entropy on condition A, conditional-entropy on D given A, and mutual-information between A and D are respectively defined by where

([26,27,56]). The entropy, conditional-entropy, and mutual-information have granulation monotonicity. Concretely,

In terms of the decision table , the classical system of Shannon entropies has been introduced into rough set theory, as shown by Definition 1 and Theorem 1. As three basic information measures, the entropy, conditional-entropy, and mutual-information have uncertainty semantics and granulation monotonicity, so they are extensively used in attribute reduction and heuristic algorithms [26,27,42]. The granulation relation is equivalent to , that is, and it is usually induced by ; furthermore, relevant granulation monotonicity/non-monotonicity becomes an important index to assess and apply uncertainty measures.

According to the decision table and its formal structure, Zhang and Miao [56] recently introduced three-level granular structures, i.e., and further investigated weighted-entropy constructions. As a result, the previous entropy system (Equation (3)) is actually located at macro-top and has an equivalent construction from the weighted-entropy system; at meso-middle, Zhang et al. [10] established three-way informational class-specific reducts to be compared with the algebraic class-specific reducts [9].

In particular, Nie and Zhou [18] proposed a new discernibility matrix for computing the reduction core, and they tactfully utilized a kind of novel information of so-called local conditional-entropy. As our preliminary, the relevant entropy and matrix are reviewed as follows, where let and the cardinality form is mainly adopted.

([18]). The local conditional-entropy on decision table

([18]). The discernibility matrix where is determined to represent the conditional-entropy of local decision table when accompanied by new universe

3. Double-Granule Conditional-Entropies Based on Three-Level Granular Structures

The local conditional-entropy in Equation (5) implements effective uncertainty descriptions to guide the in-depth discernibility matrix and core calculation [18], thus exhibiting fundamental significance. However, this basic measure has three flawed aspects, and corresponding improvements for general applications.

According to Equation (5), the locality mainly refers to less range in universe U. More essentially, we can stand on the dual granules and to propose a novel notion of double-granule conditional-entropies, and it differs from the usual entropy system with only the single-granule representation which implies a kind of first-order style. Moreover, the measure properties are lacking in [18], and we will provide in-depth properties such as restriction bounds and granulation non-monotonicity.

Regarding granular structures, all decision classes () (or decision classification ) are considered, but condition granules involve only two factors and . A condition partition ) needs considering in practice to provide a system description of knowledge granulation, so we also focus on granulation to introduce three-level granular structures for hierarchical constructions of double-granule conditional-entropies.

Finally, the initial concept is limited to only C for expressing the discernibility matrix and reduction core, and a general subset has better theoretical and practical prospects, especially for the knowledge-based applications (such as attribute reduction or feature selection).

Along the above thoughts, this section mainly establishes double-granule conditional-entropies based on a universal attribute-subset and investigates relevant algorithms and properties, and we particularly use a kind of three-level granular structures.

From a viewpoint of only condition granulation, basic descriptions of three-level granular structures are provided in Table 1, and relevant concepts are usually intuitionistic and descriptive according to a supporting figure with granular structures: Figure 1. Micro-bottom focuses on only two granules, meso-middle consists of one granule and a partition, while macro-top considers the same partition with different construction origins. The three-level granular structures carry a kind of hierarchical integration (or decomposition) relationship, and they provide , n, and one parallel patterns, respectively; they will be presented in a table form with the mainbody data as well as the edge statistics. Moreover, they differ from the existing three-level granular structures for decision tables, which consider not only the condition granulation (with and ) but also decision granulation (with and ) [56].

Table 1

Three-level granular structures based on condition granulation of the decision table.

Structure Naming	Composition System	Granular Scale	Granular Level	Number of Parallel Patterns
Micro-Bottom	(A_p,A_q)	Micro	Bottom	n×n
Meso-Middle	(A_p,U/IND(A)) =(A_p,{A_q:q=1,⋯,n})	Meso	Middle	n
Macro-Top	(U/IND(A),U/IND(A)) =({A_p:p=1,⋯,n},{A_q:q=1,⋯,n})	Macro	Top	1

Figure 1

Schematic diagram of three-level granular structures.

3.1. Double-Granule Conditional-Entropy at Micro-Bottom

The local conditional-entropies are actually at only micro-bottom, i.e., regarding C. As a basis of hierarchical development, this subsection improves local conditional-entropies to construct double-granule conditional-entropies at micro-bottom (), which comes from an arbitrary condition-attribute subset . We first suppose weight coefficients where

At micro-bottom

The double-granule conditional-entropy based on

By using probabilistic and cardinal forms, Definition 4 proposes the double-granule conditional-entropy at micro-bottom. In contrast to the local conditional-entropy in [18], our measure generally adopts the same essence but a different viewpoint. In other words, Equation (9) with forms and is equivalent to Equation (5) with styles and when but the former becomes different and coherent when moreover, it more tends to the double-granule description rather than the granule-union locality. In Equation (9), conditional-information measures represent the uncertainty of decision classification regarding condition granules and , respectively, and they are integrated into by two complementary weight coefficients and . As a result, embodies a kind of information fusion of double-granule , to describe decision classification and its uncertainty, from the perspective of conditional information. Therefore, is naturally called the double-granule conditional-entropy, and it is actually located at micro-bottom . In particular, the double-granule measures utilize the double-granule fusion to capture a new feature of second-order, because main entropy systems (such as those in Equation (3)) utilize only the single-granule description which correspondingly refers to the so-called first-order information. Proposition 1 focuses on a specific case of , and the concrete result degenerates into a one-order measure regarding conditional-entropy.

At micro-bottom, double-granule conditional-entropies offer

Since both and have n granules based on and , offers number (Proposition 2) to correspond to micro-bottoms. The kinds of double-granule conditional-entropies are arranged in Table 2, and the mainbody refers to an square symmetric matrix where Based on Equation (9), Algorithm 1 resorts to a “for” loop to effectively offer a double-granule conditional-entropy for two arbitrary granules . Furthermore, we can achieve all entropies values by adding two “for” loops regarding and .

Table 2

Matrix distribution of double-granule conditional-entropies at micro-bottom.

U/IND(A)	A_1	⋯	A_q	⋯	A_n
A_1	H_(A_1,A_1)(D/A)	⋯	H_(A_1,A_q)(D/A)	⋯	H_(A_1,A_n)(D/A)
⋮	⋮	⋱	⋮	⋱	⋮
A_p	H_(A_p,A_1)(D/A)	⋯	H_(A_p,A_q)(D/A)	⋯	H_(A_p,A_n)(D/A)
⋮	⋮	⋱	⋮	⋱	⋮
A_n	H_(A_n,A_1)(D/A)	⋯	H_(A_n,A_q)(D/A)	⋯	H_(A_n,A_n)(D/A)

Compute to obtain two concrete granules , and determine .

Compute to obtain all decision classes ().

Let , .

fordo

,

.

end for

Obtain .

return.

At micro-bottom, the double-granule conditional-entropy has lower and upper bounds. Concretely, where

implies so . □

In Theorem 2, the double bounds of are acquired by the enlarging and reducing of weight coefficients. Regarding Equation (12), on the other hand, In other words, and have theoretical lower bounds and , respectively, but they usually have closer lower bounds and , respectively. Therefore, can theoretically achieve , such as in the case usually, it may be practically restricted by a better measure: which offers

We below provide another upper bound of , which may be better than in some cases.

At micro-bottom, the double-granule conditional-entropy has an upper bound. Concretely,

As shown in Figure 2, function () is convex, where . Thus, let and then the famous “Jensen’s inequality” in mathematics could induce where In other words, we can get □

Figure 2

Convex figure of information function .

In Theorem 3, the convex property of information function is utilized to provide a new upper bound of central measure . When comparing Equations (7) and (17), we can surprisingly discover that highly adheres to which naturally comes from (Equation (7)). In fact, when ; when where , there is a difference between two measures, and we obtain

Thus far, has one lower bound and two upper bounds , . An interesting question naturally emerges, i.e., can we necessarily determine the size relationship between and to provide an exact bound? Unfortunately, the answer is negative, and the later example and experiment will reveal the size uncertainty. We simply provide a mechanism analysis. Let and its numerator/denominator be the corresponding sum of numerators/denominators of and . According to [64], we can obtain but produces an uncertainty location between and . In view of the information function and its maximum point (Figure 2), never having the necessary size relationships, so also never have the necessary size relationships. In summary, and adopt different views to become irrelevant and interactive, and they together restrict . With the addition of lower bound of , there are in total three bounds to systematically emerge. Similar to and its distributional Table 2, they can also be arranged in a table with an square symmetric matrix, i.e., Table 3, and thus Table 3 correspondingly restricts Table 2.

Table 3

Three bounds of double-granule conditional-entropies at micro-bottom.

U/IND(A)	A_1	⋯	A_q	⋯	A_n
A_1	[H__(A_1,A_1)(D/A)(D/A),H¯_(A_1,A_1)(D/A)] H(A_1,A_1)*(D/A)	⋯	[H__(A_1,A_q)(D/A),H¯_(A_1,A_q)(D/A)] H(A_1,A_q)*(D/A)	⋯	[H__(A_1,A_n)(D/A),H¯_(A_1,A_n)(D/A)] H(A_1,A_n)*(D/A)
⋮	⋮	⋱	⋮	⋱	⋮
A_p	[H__(A_p,A_1)(D/A),H¯_(A_p,A_1)(D/A)] H(A_p,A_1)*(D/A)	⋯	[H__(A_p,A_q)(D/A),H¯_(A_p,A_q)(D/A)] H(A_p,A_q)*(D/A)	⋯	[H__(A_p,A_n)(D/A),H¯_(A_p,A_n)(D/A)] H(A_p,A_n)*(D/A)
⋮	⋮	⋱	⋮	⋱	⋮
A_n	[H__(A_n,A_1)(D/A),H¯_(A_n,A_1)(D/A)] H(A_n,A_1)*(D/A)	⋯	[H__(A_n,A_q)(D/A),H¯_(A_n,A_q)(D/A)] H(A_n,A_q)*(D/A)	⋯	[H__(A_n,A_n)(D/A),H¯_(A_n,A_n)(D/A)] H(A_n,A_n)*(D/A)

Finally, consider relevant granulation monotonicity/non-monotonicity. In fact, micro-bottom and its double-granule conditional-entropies focus on only two condition granules and thus never consider the condition granulation and further monotonicity/non-monotonicity. Moreover, implies the granulation refining and granule decomposition from A to B; thus and exhibit complex correspondence and uncertainty change, so we cannot mine fine relationships between and .

3.2. Double-Granule Conditional-Entropy at Meso-Middle

As analyzed above, double-granule conditional-entropies at micro-bottom never consider the condition granulation to lack robust functions of uncertainty descriptions. In terms of fixed decision granulation , at micro-bottom involves only two condition granules and their interactive uncertainty information. For the function promotion, the condition granulation with systematic granules is worth introducing based on double-granule conditional-entropy . Thus, we will gradually strengthen the knowledge granulation to establish better double-granule conditional-entropies, by virtue of three-level granular structures (Table 1). This subsection discusses double-granule conditional-entropies at meso-middle

At meso-middle

At meso-middle, the double-granule conditional-entropy has an analytic expression:

Double-granule conditional-entropies have a hierarchical integration from micro-bottom to meso-middle, i.e.,

By Definition 5 (Corollary 1) and Theorem 4, meso-middle’s measure (which can also be noted by ) hierarchically integrates double-granule conditional-entropies by condition-granular summation on . Thus, inherits the features of double-granule and conditional-entropy, it considers a granule and condition granulation to be at meso-middle , so it is called the double-granule conditional-entropy at meso-middle. As a transition, combines granule and partition to describe decision classification and its uncertainty, from the perspective of conditional information.

Similar to and based on previous discussions on (Section 3.1), we will provide corresponding properties of , including the number distribution, calculation algorithm, three bounds, and granulation monotonicity/non-monotonicity.

At meso-middle, double-granule conditional-entropies offer n values, i.e.,

In Proposition 3, double-granule conditional-entropies naturally exhibit number n to correspond to n meso-middles. The n values can be stored in an n-dimension vector to be related to the previous distributional Table 2. By enlarging Table 2, they are represented by the marginal vector of the bottom or right in Table 4, and they exactly correspond to the relevant row/column sum of micro-bottom’s information values. According to Equations (21) and (23), Algorithm 2 resorts to two “for” loops to effectively offer a double-granule conditional-entropy for an arbitrary granule . In fact, the inner loop invokes Algorithm 1 to calculate an arbitrary double-granule conditional-entropy at micro-bottom, while the outer loop integrates n related bottomed measures to produce . Furthermore, we can achieve all n middle entropies values by adding a “for” loop regarding .

Table 4

Marginal distribution of double-granule conditional-entropies at meso-middle and macro-top.

U/IND(A)	A_1	⋯	A_q	⋯	A_n	Meso-Middle
A_1	H_(A_1,A_1)(D/A)	⋯	H_(A_1,A_q)(D/A)	⋯	H_(A_1,A_n)(D/A)	H_(A_1)(D/A)
⋮	⋮	⋱	⋮	⋱	⋮	⋮
A_p	H_(A_p,A_1)(D/A)	⋯	H_(A_p,A_q)(D/A)	⋯	H_(A_p,A_n)(D/A)	H_(A_p)(D/A)
⋮	⋮	⋱	⋮	⋱	⋮	⋮
A_n	H_(A_n,A_1)(D/A)	⋯	H_(A_n,A_q)(D/A)	⋯	H_(A_n,A_n)(D/A)	H_(A_n)(D/A)
Meso-Middle	H_(A_1)(D/A)	⋯	H_(A_q)(D/A)	⋯	H_(A_n)(D/A)	Macro-Top: H(D/A)

Compute to obtain all condition classes () and a fixed granule .

Compute to obtain all decision classes ().

Let .

fordo

Compute .

Let , .

for do

,

.

end for

Obtain .

.

end for

return.

At meso-middle, the double-granule conditional-entropy has a lower bound and two upper bounds. Concretely, where

Theorem 5 naturally comes from Theorems 2–4. The three bounds in Equation (25) hierarchically integrate previous three bounds at micro-bottom (Equations (11) and (17)) to correspondingly restrict . They can be supplemented into distributional Table 4, and following Table 5 provides the relevant part.

Table 5

Three bounds of double-granule conditional-entropies at meso-middle and macro-top.

U/IND(A)	H_(A_p)(D/A)	H__(A_p)(D/A)	H¯_(A_p)(D/A)	H(A_p)*(D/A)
A_1	H_(A_1)(D/A)	H__(A_1)(D/A)	H¯_(A_1)(D/A)	H(A_1)*(D/A)
⋮	⋮	⋮	⋮	⋮
A_p	H_(A_p)(D/A)	H__(A_p)(D/A)	H¯_(A_p)(D/A)	H(A_p)*(D/A)
⋮	⋮	⋮	⋮	⋮
A_n	H_(A_n)(D/A)	H__(A_n)(D/A)	H¯_(A_n)(D/A)	H(A_n)*(D/A)
Macro-Top	H(D/A)	H_(D/A)	H¯(D/A)	H^*(D/A)

At meso-middle, introduces the condition granulation , but it still needs condition granule . Thus, we cannot make a positive assertion regarding granulation monotonicity/non-monotonicity. In fact, also implies chaos between and .

3.3. Double-Granule Conditional-Entropy at Macro-Top

As analyzed above, double-granule conditional-entropies at meso-middle consider the condition granulation, but in an insufficient way, and also depends on a single condition granule . For the thorough granulation and robust description, systematic measures () can be further integrated to generate double-granule conditional-entropies at macro-top. Based on the previous thought and result in Section 3.1 and Section 3.2, this subsection further discusses double-granule conditional-entropies at macro-top which is given in Table 1. We will directly provide the relevant integration definition, number distribution, calculation algorithm, three bounds, and we finally uncover an important conclusion of granulation non-monotonicity.

At macro-top

At macro-top, the double-granule conditional-entropy has an analytic expression:

Double-granule conditional-entropies have a hierarchical integration from micro-bottom and meso-middle to macro-top, i.e.,

By Definition 6 (Corollary 2) and Theorem 6, macro-top’s measure hierarchically integrates meso-middle’s entropies by a single summation on , and thus it further hierarchically integrates micro-bottom’s entropies by double summations on . As a result, inherits the features of double-granule and conditional-entropy. It considers only conditional granulation to be at macro-top , so it is called the double-granule conditional-entropy at macro-top. As an ultimate measure, completely utilizes the granulation information to effectively describe decision classification and its uncertainty, thus holding robust measurement functions for knowledge granulation. Moreover, can be noted by ).

At macro-top, the double-granule conditional-entropy offers only one value, i.e.,

In Proposition 4, the double-granule conditional-entropy naturally exhibits number 1 to correspond to the sole macro-top. In fact, the first top entropy comes from the fusion of either n middle entropies or bottom entropies; thus, three-level entropies accord with three-level granular structures (Table 1) from the quantitative and structural perspective, and they embody the hierarchical integration. In particular, the sole conditional-entropy is put into the lower-right corner of Table 4, thus corresponding to the summations of central micro values and marginal n meso values. According to Equations (26) and (28), Algorithm 3 resorts to three “for” loops to effectively offer the double-granule conditional-entropy . The two inner loops invoke Algorithm 2 to calculate an arbitrary double-granule conditional-entropy at meso-middle (where the central loop invokes Algorithm 1 to construct micro-bottom’s entropies), while the outer loop integrates n related meso-middle’s information values to produce . In other words, Algorithms 1–3 exhibit a kind of hierarchical evolution based on circulation development, and thus they constitute a novel kind of three-level algorithms.

Compute to obtain all condition classes ().

Compute to obtain all decision classes ().

Let .

fordo

Let .

for do

Compute .

Let , .

for do

,

.

end for

Obtain .

.

end for

.

end for

return.

At macro-top, the double-granule conditional-entropy has a lower bound and two upper bounds. Concretely, where

Theorem 7 naturally comes from Theorems 2–6. The three bounds in Equations (30)–(32) hierarchically integrate previous three bounds at meso-middle and micro-bottom, and thus they become three new uncertainty measures at macro-top to correspondingly restrict . They are supplemented into the bottom in the previous bound table: Table 5.

At macro-top, the double-granule conditional-entropy has granulation non-monotonicity. That is, and both cases can practically exist. In addition, the matched double bounds

At macro-top, the double-granule conditional-entropy completely breaks away from the condition granule dependence to establish the condition granulation description, so it becomes a powerful type of information measure for knowledge-based uncertainty representation. In terms of condition granulation, its non-monotonicity is finally revealed in Theorem 8, and the relevant evidence will be provided in the later example and experiment. Moreover, this fundamental non-monotonicity conclusion embodies information uncertainty, and it can be induced or explained by the previous complexity mechanism at micro-bottom and meso-middle. Based on macro-top and its granulation mechanism, the related three bounds (Equations (30)–(32)) and their monotonicity/non-monotonicity can be practically observed, and thus we also obtain the granulation non-monotonicity for and ; however, the case of upper bound becomes a remaining problem.

4. Decision Table Example

In this section, the above theoretical constructions and properties are illustrated by a decision table example. By extracting a part of VOTING data set (which comes from UCI database [65]), we provide a practical decision table in Table 6 with

Table 6

A decision table.

U	c_1	c_2	c_3	c_4	c_5	c_6	c_7	c_8	c_9	c_10	c_11	D
x_1	2	2	4	4	4	3	4	4	4	2	4	1
x_2	2	2	4	4	2	2	4	4	4	2	3	1
x_3	3	4	3	4	2	4	2	4	4	2	2	0
x_4	2	4	2	3	2	4	2	4	2	2	4	0
x_5	4	4	2	4	2	4	3	4	4	4	3	0
x_6	2	4	2	4	2	2	2	4	4	4	4	0
x_7	2	2	4	4	2	2	2	3	4	4	2	0
x_8	2	2	4	4	2	2	2	4	4	3	4	1

According to this decision table, provides . As an example, is chosen to generate condition granulation where . By virtue of three-level granular structures (Table 1), double-granule conditional-entropies and their three bounds are calculated by relevant algorithms and definitions, and they are compactly listed in Table 7 and Table 8, respectively. The measures at micro-bottom, meso-middle, macro-top have numbers 36, 6, 1, respectively, and they correspond to the central matrix, marginal 6-dimensional vector, lower-right-corner 1 digit, respectively. In part, we provide some processes of entropy calculation as follows.

Table 7

Information values of double-granule conditional-entropies in the example.

U	A_1	A_2	A_3	A_4	A_5	A_6	Meso-Middle
A_1	0	0.6887	0	0	0	0	0.6887
A_2	0.6887	0.9183	0.6887	0.6887	0.6887	0.6887	4.3619
A_3	0	0.6887	0	0	0	0	0.6887
A_4	0	0.6887	0	0	0	0	0.6887
A_5	0	0.6887	0	0	0	0	0.6887
A_6	0	0.6887	0	0	0	0	0.6887
Meso-Middle	0.6887	4.3619	0.6887	0.6887	0.6887	0.6887	Macro-Top: 7.8055

Table 8

Three bounds of double-granule conditional-entropies in the example.

U	A_1	A_2	A_3	A_4	A_5	A_6	Meso-Middle
A_1	[0,0]0	[0.1722,0.9183] 0.8113	[0,0]1	[0,0]1	[0,0]1	[0,0]1	[0.1722,0.9183] 4.8113
A_2	[0.1722,0.9183] 0.8113	[0.3444,1.8366] 0.9183	[0.1722,0.9183]1	[0.1722,0.9183]1	[0.1722,0.9183]1	[0.1722,0.9183]1	[1.2053,6.4281] 5.7296
A_3	[0,0]1	[0.1722,0.9183]1	[0,0]0	[0,0]0	[0,0]0	[0,0]0	[0.1722,0.9183] 2.0000
A_4	[0,0]1	[0.1722,0.9183]1	[0,0]0	[0,0]0	[0,0]0	[0,0]0	[0.1722,0.9183] 2.0000
A_5	[0,0]1	[0.1722,0.9183]1	[0,0]0	[0,0]0	[0,0]0	[0,0]0	[0.1722,0.9183] 2.0000
A_6	[0,0]1	[0.1722,0.9183]1	[0,0]0	[0,0]0	[0,0]0	[0,0]0	[0.1722,0.9183] 2.0000
Meso-Middle	[0.1722,0.9183] 4.8113	[1.2053,6.4281] 5.7296	[0.1722,0.9183] 2.0000	[0.1722,0.9183] 2.0000	[0.1722,0.9183] 2.0000	[0.1722,0.9183] 2.0000	Macro-Top:[2.0662,11.0196]18.5049

By Table 7 and Table 8, we can make relevant verification analyses. First, entropies and bounds naturally present hierarchical integration relationships from micro-bottom to meso-middle to macro-top. Indeed, conditional-entropies are correspondingly restricted by three bounds. Moreover, the two types of upper bounds exactly have no necessary size relationships, and a part but powerful proof is provided as follows:

Finally, the granulation non-monotonicity at macro-top (Theorem 8) is verified. For this, we chose a natural attribute-addition chain: () denotes the attribute subset in the chain, and its granulation is represented by In other words, corresponds to the kth chain element to represent the pth condition granule in partition . According to the subset chain, Table 9 provides double-granule conditional-entropies, including both part values at micro-bottom , meso-middle and all values (as well as the three bounds) at macro-top . As a supporting detail, previous Table 7 and Table 8 actually embrace the chain element and its partition , while double-granule conditional-entropies regarding attribute subset and corresponding condition granulation are supplemented in Table 10 for better observation and illustration.

Table 9

Double-granule conditional-entropies based on an attribute-enlargement chain in the example.

Level	Measure	CA_1	CA_2	CA_3	CA_4	CA_5	CA_6	CA_7	CA_8	CA_9	CA_10	CA_11
Micro-Bottom	H_(CA_k,1,CA_k,1)(D/CA_k,) H_(CA_k,1,CA_k,2)(D/CA_k) H_(CA_k,1,CA_k,3)(D/CA_k) H_(CA_k,2,CA_k,2)(D/CA_k) H_(CA_k,2,CA_k,3)(D/CA_k)	1.00000.85710.857100	0.81130.64900.540900	0.81130.64900.540900	0.81130.64900.649000	00.688700.91830.6887	00.688700.91830.6887	01000	01000	01000	01000	01000
Meso-Middle	H_(CA_k,1)(D/CA_k) H_(CA_k,2)(D/CA_k) H_(CA_k,3)(D/CA_k)	2.7143 0.8571 0.8571	2.6502 0.6490 0.5409	2.6502 0.6490 0.5409	3.4074 0.6490 0.6490	0.6887 4.3619 0.6887	0.6887 4.3619 0.6887	0.6667 0.6667 0.6667	000	000	000	000
Macro-Top	H(D/CA_k) H_(D/CA_k) H¯(D/CA_k) H^*(D/CA_k)	4.4286 2.2500 6.0000 4.9409	4.4891 1.6226 6.4902 6.6951	4.4891 1.6226 6.4902 6.6951	6.0035 2.0282 8.1128 8.5789	7.8055 2.0282 11.0196 18.5409	7.8055 2.0282 11.0196 18.5409	9.0000 1.7500 14.0000 28.0196	00030	00030	00030	00030

Table 10

Double-granule conditional-entropies regarding in the example.

U	CA_2,1	CA_2,2	CA_2,3	CA_2,4	Meso-Middle
CA_2,1	0.8113	0.6490	0.5409	0.6490	2.6502
CA_2,2	0.6490	0	0	0	0.6490
CA_2,3	0.5409	0	0	0	0.5409
CA_2,4	0.6490	0	0	0	0.6490
Meso-Middle	2.6502	0.6490	0.5409	0.6490	Macro-Top: 4.4891

Since different chain subsets may have different equivalence partitions and granule numbers, the measures at micro-bottom and meso-middle consider condition granules to have a distinctive number and difficult correspondence. Table 9 focuses on the small and the same granule number, but relevant granules have different connotations. For example, the granules of the first one — ()—may be different. Thus, we cannot acquire the so-called granulation non-monotonicity assertion because of granulation incompletion, although the values at micro-bottom and meso-middle actually exhibit a kind of non-monotonic change in Table 9.

In contrast, macro-top offers the complete condition granulation, so we can effectively focus on value monotonicity/non-monotonicity for both double-granule conditional-entropies and their three bounds. Observing the bottom part of Table 9 in the enlargement chain direction, we can discover that the three types of information measures are all non-monotonic, i.e.,

More vividly, the entropy and its three bounds regarding the chain are depicted in Figure 3, so the related granulation non-monotonicity becomes clearer. For example, the macro entropy value first increases and then decreases in the addition chain direction. Moreover, Table 9 and Figure 3 reflect the restriction properties of three bounds.

Figure 3

Macro-top’s double-granule conditional-entropies and their three bounds based on an attribute-enlargement chain in the example.

5. Data Experiments

In this section, the above theoretical results and their effectiveness are verified by data experiments. The new measures are mainly suitable for categorical (or nominal) data, which are usually used in the traditional rough set theory, and thus we adopt three relevant data sets from the UCI Machine Learning Repository [65], whose concrete descriptions on decision table are given in Table 11.

Table 11

Three UCI data sets.

Label	Name	|U|	|C|	|U/IND(C)|	|D|	|U/IND(D)|
(1)	VOTING	435	16	342	1	2
(2)	SPECT	187	22	169	1	2
(3)	Tic-Tac-Toe	958	9	958	1	2

Similar to the above example, we also adopt the attribute-addition chain and its relevant symbol such as Note that this attribute-subset sequence (Equation (34)) can deeply and typically probe the hierarchical knowledge-granulation within a framework of the complete lattice . As a representative manifestation, we provide two typical results regarding the first chain element and the last one .

Regarding VOTING, and C induce three and 342 granules, respectively, and relevant double-granule conditional-entropies and three bounds are provided in Table 12 and Table 13, respectively.

Table 12

Double-granule conditional-entropies in the VOTING data set.

U/IND(CA_1)	CA_1,1	CA_1,2	CA_1,3	Meso-Middle	⋯	U/IND(CA_16)	CA_16,1	⋯	CA_16,342	Meso-Middle
CA_1,1	0.9867	0.9782	0.8369	2.8018	⋯	CA_16,1	0	⋯	0	0
CA_1,2	0.9782	0.8113	0.6578	2.4473	⋯	⋮	⋮	⋱	⋮	⋮
CA_1,3	0.8369	0.6578	0.6479	2.1427	⋯	CA_16,342	0	⋯	0	0
Meso-Middle	2.8018	2.4473	2.1427	Macro-Top:7.3918	⋯	Meso-Middle	0	⋯	0	Macro-Top:0

Table 13

Three information bounds in the VOTING data set.

U/IND(CA_1)	CA_1,1	CA_1,2	CA_1,3	Meso-Middle	⋯	U/IND(CA_16)	CA_16,1	⋯	CA_16,342	Meso-Middle
CA_1,1	[1.0706, 1.9734] 0.9867	[0.5577, 1.7980] 0.9921	[0.8139, 1.6346] 0.9649	[2.4422, 5.4060] 2.9436	⋯	CA_16,1	[0,0]0	⋯	[0,0]0	[0,0] 221.3143
CA_1,2	[0.5577, 1.7980] 0.9921	[0.0448, 1.6226] 0.8113	[0.3009, 1.4592] 0.6596	[0.9034, 4.8798] 2.4630	⋮	⋮	⋮	⋱	⋮	⋮
CA_1,3	[0.8139, 1.6346] 0.9649	[0.3009, 1.4592] 0.6596	[0.5571, 1.2959] 0.6479	[1.6719, 4.3898] 2.2724	⋯	CA_16,342	[0,0]0	⋯	[0,0]0	[0,0] 221.3143
Meso-Middle	[2.4422, 5.4060] 2.9436	[0.9034, 4.8798] 2.4630	[1.6719, 4.3898] 2.2724	Macro-Top:[5.0174,14.6755]7.6790	⋯	Meso-Middle	[0,0] 221.3143	⋯	[0,0] 221.3143	Macro-Top:[0,0]50132

Regarding SPECT, and C produce two and 169 granules, respectively, and relevant three-level measures and three bounds are provided in Table 14 and Table 15, respectively.

Table 14

Double-granule conditional-entropies in the SPECT data set.

U/IND(CA_1)	CA_1,1	CA_1,2	Meso-Middle	⋯	U/IND(CA_22)	CA_22,1	⋯	CA_22,169	Meso-Middle
CA_1,1	0.2108	0.3815	0.5924	⋯	CA_22,1	0	⋯	0	1.5335
					⋮	⋮	⋱	⋮	⋮
CA_1,2	0.3815	0.5399	0.9215	⋯	CA_22,169	0	⋯	0	1.5335
Meso-Middle	0.5924	0.9215	Macro-Top:1.5139	⋯	Meso-Middle	1.5335	⋯	1.5335	Macro-Top:513.0879

Table 15

Three information bounds in the SPECT data set.

U/IND(CA_1)	CA_1,1	CA_1,2	Meso-Middle	⋯	U/IND(CA_22)	CA_22,1	⋯	CA_22,169	Meso-Middle
CA_1,2	[0.1015, 0.4217] 0.2109	[0.1908, 0.7508] 0.4030	[0.2922, 1.1725] 0.6138	⋯	CA_22,1	[0,0]0	⋯	[0,0]0	[0.0332, 1.9457] 8.8982
				⋮	⋮	⋮	⋱	⋮	⋮
CA_1,2	[0.1908, 0.7508] 0.4030	[0.2801, 1.0799] 0.5400	[0.4708, 1.8306] 0.9429	⋯	CA_22,169	[0,0]0	⋯	[0,0]0	[0.0332, 1.9457] 161.2140
Meso-Middle	[0.2922, 1.1725] 0.6138	[0.4708, 1.8306] 0.9429	Macro-Top:[0.7631,14.6755]1.5566	⋯	Meso-Middle	[0.0332, 1.9457] 8.8982	⋯	[0.0332, 1.9457] 161.2140	Macro-Top:[11.2085,657.6332]2867

Regarding Tic-Tac-Toe, and C determine three and 958 granules, respectively, and relevant entropies and bounds are provided in Table 16 and Table 17, respectively.

Table 16

Double-granule conditional-entropies in the Tic-Tac-Toe data set.

U/IND(CA_1)	CA_1,1	CA_1,2	CA_1,3	Meso-Middle	⋯	U/IND(CA_9)	CA_9,1	⋯	CA_9,958	Meso-Middle
CA_1,1	0.8742	0.9248	0.8794	2.6784	⋯	CA_9,1	0	⋯	0	0
CA_1,2	0.9248	0.9881	0.9509	2.8638	⋯	⋮	⋮	⋱	⋮	⋮
CA_1,3	0.8794	0.9509	0.8901	2.7203	⋯	CA_9,958	0	⋯	0	0
Meso-Middle	2.6784	2.8638	2.7203	Macro-Top:8.2625	⋯	Meso-Middle	0	⋯	0	Macro-Top:0

Table 17

Three information bounds in the Tic-Tac-Toe data set.

U/IND(CA_1)	CA_1,1	CA_1,2	CA_1,3	Meso-Middle	⋯	U/IND(CA_9)	CA_9,1	⋯	CA_9,958	Meso-Middle
CA_1,1	[0.3814, 1.7483] 0.8741	[0.3635, 1.8622] 0.9404	[0.2859, 1.7642] 0.8796	[1.0308, 5.3748] 2.6940	⋯	CA_9,1	[0,0]0	⋯	[0,0]0	[0,0]332
CA_1,2	[0.3635, 1.8622] 0.9404	[0.3455, 1.9762] 0.9881	[0.2680, 1.8781] 0.9628	[0.9770, 5.7165] 2.8913	⋮	⋮	⋮	⋱	⋮	⋮
CA_1,3	[0.2859, 1.7642] 0.8796	[0.2680, 1.8781] 0.9628	[0.1905, 1.7801] 0.8900	[0.7444, 5.4224] 2.7324	⋯	CA_9,958	[0,0]0	⋯	[0,0]0	[0,0]626
Meso-Middle	[1.0308, 5.3748] 2.6940	[0.9770, 5.7165] 2.8913	[0.7444, 5.4224] 2.7342	Macro-Top:[2.7522,16.5138]8.3178	⋯	Meso-Middle	[0,0]332	⋯	[0,0]626	Macro-Top:[0,0]415664

From the perspective of macro-top, double-granule conditional-entropies and their three information bounds based on the attribute-enlargement chain are finally summarized in Figure 4. These tables and figures can be utilized to effectively verify all previous conclusions, including the hierarchy, algorithm, restriction, and non-monotonicity. In particular, double-granule conditional-entropies are confined by three bounds, thus supporting the boundedness (Theorems 2, 3, 5 and 7); moreover, the entropies and their matched double-bounds fluctuate up and down, thus proving relevant granulation non-monotonicity (Theorem 8).

Figure 4

Macro-top’s double-granule conditional-entropies and their three information bounds based on an attribute-enlargement chain in data experiments.

6. Conclusions

The information measures implement fundamental uncertainty measurement in rough set theory and granular computing. The local conditional-entropies have the second-order feature, but they are limited to micro-bottom for describing discernibility matrix and reduction core [18]. In this paper, double-granule conditional-entropies achieve corresponding improvements of hierarchical/conditional granulation, and thus they become broader measures with uncertainty representation and information processing. They focus more on the double-granule interaction rather than granule-union locality, which is used in local conditional-entropies [18]. This strategy directly utilizes the second-order mechanism to implement more systematic and robust uncertainty measurements, especially when compared to the current mainstream of first-order information measures. In our studies, double-granule conditional-entropies and their hierarchies, granulation, algorithms, bounds, and non-monotonicity are acquired and verified at three-level granular structures (i.e., micro-bottom, meso-middle, macro-top), and these results underlie both the efficiency in information processing and effectiveness in knowledge-based data analyses. Furthermore, their future developments and in-depth applications can be explored as follows.

In contrast to the relevant technology in [56], the hierarchical granulation of three-level granular structures focuses on the conditional granulation and relevant number, and it can be generalized for granular computing.

The double-granule conditional-entropies and their three bounds become new types of information measures with the second-order feature. In contrast to the traditional first-order entropy system, their description power and application advantage need further practical verification.

The double-granule conditional-entropies have three-restrictive bounds and granulation non-monotonicity, which have been experimentally verified by a granulation-hierarchical sequence (i.e., Equation (34)). These results are worth deeply utilizing in uncertainty measurement and data mining.

The double-granule conditional-entropies originate from the local conditional-entropies to carry a potential and distinctive advantage of discernibility matrix representation, and they also have the complete conditional granulation to have application prospects in knowledge reasoning or acquisition. Both their relationships with the discernibility matrix and their functions on attribute reduction need be deeply researched by promoting the previous studies in [18].