A Consensus-Reaching Approach to the Evaluation of Product Design Alternatives with Multiple Preference Structures.

With interdisciplinarity being an important characteristic of contemporary product design, the evaluation of design alternatives also involves multiple disciplines, and the evaluator group usually consists of evaluators from different fields and with obvious heterogeneous characteristics. To effectively satisfy the heterogeneous needs of evaluators and improve the credibility of evaluation results, the paper introduces a consensus-reaching approach that incorporates multiple preferences to the evaluation of product design alternatives. First, in order to obtain individual preference information, each evaluator is asked to evaluate all the design alternatives using a preference structure that he/she is familiar with. Second, we use a transfer function to uniform the evaluation information obtained from various preference structures into a complementary judgment matrix. Then, we use the Hybrid Weighted Averaging (HWA) operator weight determination model to aggregate the preference information and obtain the group preference information. Then, we measure the consensus degree between individual evaluators and the group using a consensus measurement method. After that, we use the feedback mechanism to instruct individual evaluators to modify their preferences until a consensus is achieved. We explain the application steps and the feasibility of this approach through the evaluation of the design alternatives of multichannel fluorescence immunochromatography analyzers (MFIAs).

1. Introduction

In today's fast-developing global market, industrial design has become a holistic, interdisciplinary, and integrated design activity. As a result, in the evaluation of product design alternatives, multiple disciplines are concerned, and the heterogeneity among evaluators is prominent due to differences in ethnic culture, social experience, knowledge background, cognition, etc. [1-5]. To capture and visualize the cognitive process of participants and maximize the cognitive consistency between individuals, the customized individual semantics (CIS) based on heterogeneous subordinate positive information (SPI) was constructed [6]. Chen et al. customized individual semantics by means of the possibility distribution of attitude and modeling the heterogeneity of evaluators which reflected the individual differences of cognitive styles [7]. Meanwhile, due to the difference in educational backgrounds [8-11], knowledge [12, 13], experiences [8, 10–14], cultures [8, 10, 13, 14], cognitive degrees [8, 11], attributes [15], motivations [15], personalities [15], and expression habits [10, 16] of evaluators, they usually use heterogeneous preference representation structures to express their preference for alternatives [8-16]. The preference structure is supposed to meet the heterogeneous needs of evaluators, yet a single preference structure fails to meet the differential expression habits and needs of evaluators from different fields.

The evaluation of product design alternatives is usually conducted by a group of engineers, marketers, users, managers, etc. Due to the differences in subjective preferences, background knowledge, and experience, reaching a consensus is difficult. However, it is desirable that all or at least most of the participators are satisfied with the best alternative [17-21]. The consensus-reaching process refers to the practice of the evaluator group in the process of discussing and coordinating their opinions during the evaluation process until they come to an agreement. The consensus process involves the measurement of consensus and the coordination of nonconsensus [22]. Such process is helpful for reducing cognitive disagreements during the evaluation process and improving the credibility of the design alternative. Many consensus-reaching approaches with multiple preference structures are proposed by researchers [10]. At present, consensus measures are usually calculated by the opinions given by experts or the choice degrees of alternatives obtained from these opinions. The situation may not reflect the information about the true consistency because the same alternative ranking may have different choice degree vectors.

These are questions that we need to answer: How do we ensure that the heterogeneity needs of evaluators are satisfied in the evaluation? How do we consider the consistency degree of evaluators' opinions? How do we make sure that the evaluator group can reach a certain degree of consensus on the pros and cons of the design alternatives and decide on a design alternative that is acceptable to the group? How do we verify the approach?

To address the above questions, we propose an approach that incorporates multiple preference structures to the evaluation of product design alternatives. The purpose of this paper is to meet the needs of evaluators having heterogeneous characteristics during the evaluation of design alternatives, to optimize the evaluation process, and to make the evaluation results more credible. The rest of the paper is organized as follows: Section 2 reviews the literature that discusses the evaluation and selection of product design alternatives and the group consensus process; Section 3 proposes and elaborates the approach to reach a consensus in the evaluation of alternatives with multiple preference structures; Section 4 presents a numerical example to explain the detailed process of the proposed method; Section 5 summarizes the paper and discusses the significance and limitations of the proposed method.

2. Literature Review

2.1. The Evaluation and Selection of Product Design Alternatives

We review the evaluation and selection of product design alternatives from four aspects, i.e., indicator system, product design alternatives generation, decision-making methods being applied, and the real-life application scenarios for product design alternatives selection.

2.1.1. Indicator System

Researchers established different indicator systems according to the focus of the problems they solved. In the context of environmental sustainability, Shidpour et al. classified the most important factors in the key areas determined by life cycle assessment and obtained three qualitative criteria, that is, “safety,” “functional satisfaction,” and “aesthetic,” and three quantitative criteria, that is, “cost,” “reliability,” and “time to market.” The weight of each criterion was obtained by the fuzzy analytic hierarchy process [23]. To ensure the quality of products and meet the requirements of customers, Fan et al. determined the evaluation indicator system from the four dimensions of economy, functionality, innovation, and environment and applied the determined indicator system to yacht design evaluation [24]. From the point of product sustainability performance, Feng and Mai established a six-dimension sustainability evaluation model, which included environment (greenhouse emission, waste), economy (energy efficiency, material utilization, operation cost), society (health and wellness, operational safety), functionality (life, modularity, maintenance), manufacturability (processes technology, assembly, storage), and reusability (reuse, recycle, disposability), aiming to help designers and engineers evaluate and compare the relative sustainability performance of different products [25]. Shu and Zhong proposed evaluating the manufacturability of products from three aspects: economy, technology, and comprehensiveness to support the rapid response design task [26].

2.1.2. Product Design Alternatives Generation

Kang et al. combined fuzzy theory, similarity theory, and ant colony optimization to complete the generation and evaluation of design schemes [27]. Zhang et al. proposed a design model for multidisciplinary oriented complex product system and developed a vector-based mapping tool to support the rapid mapping to support conceptual design [28]. Gopsill et al. generated an automatic and evolving design structure matrix by monitoring the changes of the digital models that represent the products, which completed the identification and monitoring of product components, as well as supporting the change generation of existing product life cycle solutions [29]. Based on the Naïve Bayes cluster and rough set theory, Li et al. proposed a new product concept generation method driven by customer requirements, which helped the product development team obtain customer requirements and product attributes [30]. Hsiao et al. extracted the appearance characteristics of products based on the morphological analysis method and generated the alternative set of product appearance design through the combination of appearance components for subsequent evaluation [31]. Zamani et al. proposed a hybrid schedule generation scheme to the resource-constrained project scheduling problem, called the Polarized Adaptive Scheduling Scheme, which can operate in the spectrum between two poles, that is, to generate schemes through parallel and serial schedule [32]. In addition, there were innovative scheme generation methods based on the theory of innovative problem solving (TRIZ) [33], product design generation methods based on case-based decision theory [34], etc.

2.1.3. Decision-Making Methods

Guo and Ji took the metal handles of doors and windows as an example. The author used the analytic hierarchy process to calculate the weight of each goal and attribute from all design alternatives and obtained the pros and cons of each alternative through comprehensive calculation, thereby assisting decision-makers in making decisions [35]. Based on Analytical Network Process (ANP) and the modified technology for order preference by similarity to ideal solution (TOPSIS), Ayağ proposed a concept evaluation method in new product development. ANP method is used to determine the relative weight of a set of quantitative and qualitative evaluation criteria, and the improved ideal solution is used to sort the conceptual schemes according to the evaluation criteria to obtain the best scheme that meets customer expectations and company requirements [36]. Zhu et al. proposed an analytic hierarchy process that was based on the rough number to determine the weight of each evaluation criterion. They then used an improved comparison ranking method based on rough numbers to evaluate conceptual product designs. They verified the robustness of the method through the concept selection of a lithography tool [37]. Yumoto et al. proposed a decision support system for product selection based on the analytic hierarchy process, which used the decision rules of rough sets for qualitative evaluation. They verified the method through examples of used cars and sneakers [38]. Besharati et al. proposed a comprehensive design concept evaluation method based on fuzzy-technique for order preference by similarity to an ideal solution. A new “weighting criteria” was developed for the investigation process to quantify the evaluation criteria. It helps engineers to improve the effectiveness and objectivity of sustainable product development [39]. In addition, to improve the credibility of evaluation results, more hybrid methods have been developed [40]. To obtain an optimal design alternative, Huang et al. proposed several methods for the evaluation of conceptual designs under different conditions, which were based on computational intelligence, such as physical planning, genetic algorithms, neural networks, and fuzzy logic [41]. Tian et al. proposed a framework for the evaluation of design alternatives based on the analytic hierarchy process, gray correlation method, and ideal solution similarity ranking method and verified the effectiveness of the framework by applying it to the green performance evaluation of refrigerators [42].

2.1.4. The Real-Life Application Scenarios for Product Design Alternatives Selection

Rossi and Sihn described a comprehensive method of life cycle-oriented product alternatives evaluation and decision support, which promoted the identification effectiveness of product concepts that meet existing constraints within a given time period, and selected the most appropriate product scheme that meets the requirements set [43]. Josh and Gupta proposed an Advanced-Remanufacturing-To-Order-Disassembly-To-Order system to evaluate the design scheme of end-of-life (EOL) and the impact of product design on recycling [44]. Park and Seo discussed the approximate life cycle assessment of the product design scheme represented by solid models in the collaborative design environment and then developed a knowledge-based approximate life cycle assessment system to assess the environmental impact of product design schemes [45].

Since product design alternatives often involve multidisciplinary knowledge, when using the mathematical calculation method, the cognitive differences among evaluators from different knowledge backgrounds need to be considered. Only when a high degree of consensus is reached can the credibility and adaptability of the final design alternative be ensured. At present, there is still a lack of research on the measurement of consensus degree and its coordination method in the process of product design evaluation. In previous studies, researchers tended to consider the same form of preference information being presented by evaluators while neglecting the influence the heterogeneity of evaluators has on preference expression. Since there are differences in the evaluators' knowledge structure, judgment level, and personal preferences, the same form of preference representation cannot address the heterogeneous needs of the evaluators.

2.2. Consensus-Reaching Process in Group Evaluation

The consensus-reaching process is an effective decision-making tool to eliminate preference conflicts in group decision-making [46]. The consensus-reaching process refers to the process of evaluators discussing and coordinating their differing views, reaching a certain sense of agreement prior to the decision-making [47]. It mainly involves two steps: the measurement of group consensus and the correction of nonconsensus. Since the measurement of consensus directly determines the progress of the group decision-making procedure, it is always a hot issue to calculate group consensus effectively. It is found from existing researches that the calculation of group consensus usually includes two steps: the measurement and the aggregation of preference similarity [48]. Recently, in view of the characteristics of actual decision-making problems, some measurement methods have been proposed. For example, Meng et al. introduced a consensus indicator based on Manhattan Distance Measure to measure the agreement degrees of the decision-makers' opinions in group decisions and applied it to the model selection of Enterprise Resource Planning (ERP) software [49]. Zhang et al. measured the individual consensus levels and the group consensus levels by calculating the distance between each individual interval fuzzy preference relations (IFPRs) and the collective IFPR. A feedback mechanism considering experts' leadership and the bounded confidence levels of experts was proposed to guide experts to modify their opinions [50]. Zhao et al. used the distance between individual preferences to calculate the degree of similarity and proposed a feedback mechanism based on the degree of similarity in order to find inappropriate preferences and provide guidance to the modification and eventually reach a consensus [51]. Zhang et al. measured individual consensus degree and group consensus degree by calculating the distance between individual decision matrices and collective decision matrices and developed two optimization models. They also generated adjustment advice for decision-makers who must change their opinions in the process of reaching consensus and demonstrated the use of consensus-reaching algorithm through an example of ERP system supplier selection [52]. Zhang et al. proposed some distance measurement methods between intuitionistic multiplicative numbers/sets, including improved Hamming distance and improved Euclidean distance and its weighted forms, and further developed a new intuitionistic multiplicative preference relationship consensus measure to assist the decision-making process. The author then verified the proposed method through the selection of investment projects [53]. Mata et al. defined the similarity function based on the information center of the fuzzy set and pointed out the defect of using the traditional Euclidean distance to define the similarity function [54]. To measure the preference similarity level, Chen et al. proposed a similarity measure between intervals of linguistic two tuples and a weighted average method of interval linguistic two tuples [55]. According to existing research, the ordered weighted averaging (OWA) operator is commonly used to aggregate preference similarity [56, 57]. For example, Herrera et al. introduced a consensus and consistency-based induced ordered weighted averaging operator to aggregate the preferences of evaluators in the consensus-reaching process when the fuzzy preference relations are incomplete and then applied this operator to the consensus model as well as the selection process [58]. Palomares et al. extended the OWA aggregation operators and proposed the Attitude-OWA operator, the consensus model that incorporates the group's attitudes toward consensus into the consensus measurement. The authors then applied the method to the process of solving a decision problem with different attitude definitions [59]. Peláez et al. proposed an elective aggregated majority ordered weighted averaging operator (SAM-OWA); the SAM-OWA operator calculates the weight of each value on the satisfaction measurement table by counting the number of votes each value obtained [60].

However, the characteristic of the OWA operator is that the weight has nothing to do with the data. It simply associates weight with the position of the data, i.e., the order of the data. Considering the interdisciplinarity of product design, the heterogeneity of evaluators, and the complexity of evaluation indicators, it is inadequate to only consider the order of design proposals being evaluated. Thus, the paper adopts the HWA operator which not only considers the positions of each data but also considers the importance of the data itself. The operator rearranges the data in descending order and assigns a weight to each date according to the position and the importance of each data. It takes both the importance of position and the importance of data into account, with no ignorance of each individual factor. In this way, the HWA operator effectively avoids the situation where the order of the design alternatives has to be distorted when only taking into consideration the positions or importance of the alternatives. It can reduce the influence of some nonobjective factors on the aggregation results in the aggregation process, so as to make the aggregation result more robust and reasonable.

3. The Proposed Consensus-Reaching Approach

To meet the needs of evaluators with heterogeneous characteristics and improve the accuracy of evaluation results, the paper proposes an approach for reaching a consensus on product design alternatives with multiple preference information. The six important layers of the method (see Figure 1) are as follows: (1) determining the evaluation indicators, their weights, and the weight of the evaluator, (2) the evaluator choosing a preference representation form to express his/her initial or adjusted preferences, (3) uniforming different preference representation forms, (4) aggregating individual preferences into group preferences using aggregation operator, (5) using consensus measurement method to obtain the degree of consensus between individual evaluators and the evaluator group, and (6) using a feedback mechanism to guide evaluators to modify their preferences and then repeating steps (2) to (6) until the predefined consensus level is reached. Finally, we calculate the ranking of the alternatives.

Figure 1

Method flow.

To address the consensus-reaching problem on the evaluation of product design alternatives, we assume that the set of design alternatives to be evaluated is X = {x1, x2,…, x}(n ≥ 2), the evaluator group set is E = {e1, e2,…, e}(l ≥ 2), and the evaluation indicator set is I = {i1, i2,…, i}(m ≥ 2), where n, l, and m represent the number of programs, the number of evaluators, and the number of evaluation indicators, respectively. The evaluator weight set is W=(w1, w2,…,w), where w is the weight of the evaluator e, w ≥ 0, k = 1, 2, ···, l, and ∑w=1; the evaluation indicator weight set is Z=(z1, z2,…,z), where z is the weight of the evaluation indicator i, z ≥ 0, j = 1, 2,…, m, and ∑z=1. At the same time, the consensus level threshold is set γ. When the group consensus level reaches this threshold, the evaluator group reaches a consensus on the design alternative, and the evaluation result has a high degree of reliability. Otherwise, it is necessary to identify the evaluators who have disagreements and instruct them using the feedback mechanism to modify their preferences. Based on these assumptions, the proposed approach is established with seven steps as follows.

Step 1 .

Determining the evaluation indicators and rating their relative importance.

For the design alternatives to be evaluated, the evaluator group discusses and establishes an evaluation indicators system and uses the analytic hierarchy process (AHP) method to calculate both the weights of individual evaluators and the weights of indicators.

AHP is an indicator-system-based decision-making method that combines qualitative and quantitative decision-making methods [61]. The weights of the evaluation indicators calculated by this method are more accurate and more consistent [62]. Thus, this paper uses the AHP method to calculate the weights of the evaluation indicators.

For the indicator set of the design alternatives to be evaluated, the weights of every two design alternatives are compared by evaluators. The values 1, 3, 5, 7, and 9 are used to indicate the importance of indicator i relative to indicator j, and the meaning of the numerals is a follows: 1 means equally important, 3 means weakly important, 5 means essentially important, 7 means relatively important, and 9 means absolutely important. The obtained judgment matrix of the indicator set is

When i = j, a = 1; when i ≠ j, a=1/a. Then, the weight of indicator i is

To ensure the reliability of the weight calculation, the consistency index (CI) and the consistency ratio (CR) are used for judgment, where CI=(λmax − n)/(n − 1), CR=CI/RI, where RI is a random index whose value can be obtained by a table [63].λmax=∑((A · W)/(n · W)) is the maximum eigenvalue of the matrix. If CI ≤ 0.1 and CR ≤ 0.1 are satisfied, then, the judgment matrix has a good consistency and the weight calculation result is valid; otherwise, it needs to be judged again.

Step 2 .

Evaluation of design alternatives.

To fix a set of alternatives in a design alternatives evaluation problem, there are multiple preference representation forms that can be adopted by evaluators to present their preferences for certain sets of alternatives or the information form that he/she is more familiar with. Then, we assume the experts' preferences over the set of alternatives X against the set of the indicators I, and it may be represented in one of the following six ways.

Utility Function [64]. Suppose that the evaluator e evaluates the set of alternatives X under a certain indicator and gives a utility value set of the alternatives, U={u1, u2,…, u}, u ∈ [0,1], where u represents the utility value of the alternative x given by the evaluator e. And we assume that the larger the utility value u, the better the corresponding design alternative x

Preference Orderings [65]. Suppose that the evaluator e evaluates the alternative set X under a certain indicator and gives the order of the alternative as O={o1, o2,…, o} in which a ranking vector from the best to the worst is obtained

Interval Values [66]. Suppose that the evaluator e evaluates the set of alternatives X under a certain indicator and gives the evaluation vector of alternative x, vector d=[d, d], where d=[d, d] is the interval number, d and d are real numbers, and d < d

Linguistic Preference [67, 68]. Suppose that the evaluation given by the evaluator e to the set of alternatives X under a certain indicator is described by a matrix V. v can be understood as the degree to which alternative x is better than alternative x and the degree is an element selected from the predefined linguistic evaluation set L={l0, l1, l2,…, l, l(,…, l}, where there are g+1 elements in the set L. Its corresponding subscript i can be obtained by function I, I : L⟶N, I(l)=i,  l ∈ L. The matrix V=(v) meets v ∈ L, v=l. There exists a negation operator: v=neg(l)=l, v=l

Multiplicative Preference Relations [69]. Suppose that the evaluator e compares every two design alternatives in the set of alternatives X under a certain indicator and gives a reciprocal judgment matrix A=(a), where a represents the relative importance of the alternative x to the alternative x that evaluator e thinks, and the matrix A meets a > 0, a=1, a × a=1

Fuzzy Preference Relations [70]. Suppose that the evaluator e compares every two design alternatives in the set of alternatives X under a certain indicator and gives a complementary judgment matrix P=(p), where p indicates the degree to which the evaluator e thinks that the alternative x is better than the alternative x, and p ∈ [0,1], p+p=1, p=0.5

Step 3 .

Uniforming different preference representation forms.

With multiple preference forms being in presence, it is necessary to uniform different preference information so as to effectively aggregate group opinions and select the optimal design alternative. In this study, we consider uniforming the above-mentioned preference structures into a complementary judgment matrix.

We use the following function [71] to transform the utility value of x into a complementary judgment matrix.

We use the following function [72] to transform the preference order of x into a complementary judgment matrix.

We use the following function [73] to transform the evaluation value of the interval number of x into a complementary judgment matrix.

We use the following function [74] to transform the linguistic evaluation matrix of x into a complementary judgment matrix.

We use the following function [71] to transform the reciprocal judgment matrix of x into a complementary judgment matrix.

Step 4 .

Preference aggregation.

After all preference structures are uniformed into complementary judgment matrices, we aggregate the preference information of each evaluator into group preference information. This paper uses the HWA operator [75] for processing, which not only considers the importance of the position of each data but also reflects the importance of the data itself.

We use the HWA operator to aggregate the vector (p1, p2,…, p) and obtain the comprehensive evaluation value P of k evaluators on indicator i of design alternative x:where v is the element that ranks s in a decreasing order in a set of weighted data (λw1x1, λw2x2,…, λwx), where W=(w1, w2,…,w) is the evaluator weight vector, andω=(ω1, ω2,…,ω) is determined by the fuzzy linguistic quantifier. λ is the balance factor.The fuzzy linguistic quantifier θ is given by the following equation [76]:where a, b, r ∈ [0,1], under the principles of “at least half,” “most,” and “as many as possible,” the corresponding parameters (a, b) to the fuzzy linguistic quantifier Q(r) are (0, 0.5), (0.3, 0.8), and (0.5, 1) [77, 78].

Step 5 .

Consensus measurement.

The calculation of the consensus degree is based on the comparison between each individual evaluator's ranking of the alternatives and the evaluator group's ranking of the alternatives. By comparing the rankings of a given alternative, the approximate degree p(x) of the number x alternative of the individual evaluator e is calculated.where V=(V1, V2,…, V) is the ranking provided by the evaluator e, V is the ranking provided by the individual evaluator e on the alternative x under the indicator i, V=(V1, V2,…, V) is the ranking provided by the evaluator group, and V is the ranking provided by the evaluator group on the alternative x under the indicator i. Particularly, we take α=1/(n − 1). The parameter b controls the rigor of the consensus process. The closer the value of b is to 1, the lower the rigor is, and the fewer rounds of discussion among the evaluator group are needed. The closer the value of b is to 0, the higher the rigor is, and the more rounds of discussion among the evaluator group are needed.

1 − p(x) is the approximate value between the evaluator e's evaluation of alternative x and the group's evaluation. We calculate the consensus of all evaluators on the number x alternative:

We calculate the consensus degree C among all evaluators on the alternative set X. When there is only one optimal solution x in the solution set [79], we obtain the value of C by aggregating C(x):

Step 6 .

Adjust preferences based on the feedback information.

When the consensus degree C does not reach the predetermined level, the evaluators are asked to revise their preferences. In this case, there are three issues to be considered: (1) which evaluators need to modify their opinions, (2) which elements need to be modified, and (3) what is the direction of the modification.

We identify the evaluators that need to modify some of their preference information by calculating the approximate degree. The approximate degree P of the evaluator e is obtained through the aggregation of the approximation of each design alternative.

The evaluators are ranked based on the approximation value P, k = 1, 2,…,l. Every evaluator knows his/her position and approximation of each alternative. The evaluator with the smaller value of P or the evaluator whose rank is at the lower end needs to modify their preference. Now, we set a threshold p, p∈[0, 1] to determine how many evaluators we need to modify their preference. If P ＜ p, then e needs to modify his/her preference opinion. In this study, we set P=0.75 [80].

For evaluators who need to modify their preference, if V > V, then e needs to improve his/her appraisal of alternative x; if V < V, e needs to lower his/her appraisal for alternative x; if V=V, e's evaluation of alternative x remains unchanged.

Step 7 .

Calculate the ranking of the alternatives.

We use the HWA operator to aggregate the vectors (p, p,…, p) and obtain the comprehensive evaluation value of the alternative x by k evaluators which is P under m indicators:where v is the number s largest element in the set of weighted data (λz1x1, λz2x2,…, λzx), where Z=(z1, z2,…,z) is the indicator weight vector, and ω=(ω1, ω2,…,ω) is determined by the fuzzy linguistic quantifier.

4. A Numerical Example

In this section, the proposed model is applied in the evaluation of MFIAs. At first, the background of the evaluation of MFIAs is proposed. Then, the proposed method is used to evaluate the alternatives. Finally, the feasibility and rationality of the proposed method are demonstrated by sensitivity analysis and comparative analysis.

4.1. Background

Fluorescence immunochromatography is usually used for high sensitivity detection or rapid detection, which is promising and practical. And it is used to detect human serum, plasma, whole blood, and urine samples. The results are mostly applied as the biological basis of diagnosis for myocardial injury, heart failure, cardiovascular inflammation, acute and chronic nephropathy, and other diseases. Immunochromatography Assay is an analytical method combining immunoassay methods and chromatographic methods, with characteristics of strong peculiarity, simple operation, and rapid detecting.

At present, most of the fluorescence immune tomography analyzers are single-channel equipment, and it is difficult to realize the simultaneous detection of a single sample with multiple items. The efficiency is low. When the sample size is large, it is difficult to meet the detection requirements. For this reason, five MFIAs are designed. The whole mechanical part is composed of a reagent card placement structure, scanning detection structure, and guide rail structure. Six channels are designed, and a single-layer structure mode is adopted. The evaluation of the five MFIAs is conducted to verify the effectiveness of the methods proposed.

4.2. The Evaluation of MFIAs

We exemplify the approach through the evaluation of the design alternatives of MFIAs and elaborate the implementation results of the proposed approach as follows. After conducting the preliminary investigation, analysis, and design, 3 product designers proposed a total of 5 alternatives for evaluation (see Figure 2).

Figure 2

Design alternatives.

An evaluator group E = {e1, e2, e3, e4, e5, e6, e7, e8} with 2 senior product designers, 2 engineers, 1 project leader, 1 customer representative, and 2 product users participated in this project. After researching users' demands, integrating the results of group discussions, we finalized the indicators of the MFIAs (see Table 1).

Table 1

Evaluation indicators of the design alternatives.

First-Level indicators	Second-level indicators	Detailed description
Meeting technical requirements	i 1 easy to assemble and maintain	The assembling method makes it easy to replace the printing paper and it is easy to repair when a failure occurs.
	i 2 easy to be processed using plastics	Use plastics processing due to cost constraints.
Aesthetics of design	i 3 unification between shape and function	The design of functional components is consistent with the overall style.
	i 4 conforms to the rule of formal beauty	Well balanced in shape and proportions.
Human-machine interaction coordination	i 5 reasonable layout of functions	The layout of these functions is reasonable: a display screen, a port for test paper delivery and recycling, a port for test result printing, switch buttons, etc.
	i 6 easy to operate	The angle of the display makes viewing easy; the test paper is easy to deliver and recycle, etc.

Step 8 .

Use the AHP method to calculate the weights of evaluators and the weights of the indicators and the result is listed as follows: the weight vector of 8 evaluators is W = {0.17, 0.17, 0.18, 0.16, 0.06, 0.12, 0.07, 0.07}, and the weight vector of the 6 evaluation indicators is Z = {0.08, 0.10, 0.19, 0.15, 0.25, 0.23}. The consensus threshold is set to be 0.75.

Step 9 .

For indicator i1, 8 evaluators evaluate the 5 alternatives in the following way: e1 and e2 use reciprocal judgment matrix, e3 and e4 use complementary judgment matrix, e5 uses utility value, e6 uses interval number evaluation value, e7 uses linguistic evaluation value, and e8 uses preference ordering. The initial evaluation results collected are as follows:

Step 10 .

According to equations (3)–(7), all different preference structures are transformed into complementary judgment matrices:

Step 11 .

According to equations (9) and (10), we use the HWA operator and the fuzzy majority criterion based on the fuzzy quantization operator “as many as possible.” ω=(0,0,0,0,0.25, 0.25, 0.25, 0.25); the weight of the evaluator is W=(0.17, 0.18, 0.18, 0.15, 0.06, 0.12, 0.07, 0.07). According to equation (8), the group evaluation value obtained by using the HWA operator is

Step 12 .

Use the fuzzy majority criterion based on the fuzzy quantization operator “as many as possible” to calculate the ranking of the complementary judgment matrix, where ω=(0,0,0.2, 0.4, 0.4), and the evaluators' ranking of the 5 alternatives on indicator 1 is as follows:

Similarly, for indicator i1, the evaluator group thinks that the ranking of the 5 alternatives is x3 > x5 > x4 > x2 > x1. This ranking shows that the third alternative is the best under indicator i1.

Use equation (11) to calculate the individual evaluator's degree of approximation p(x) for each design alternative. First, we calculate V − V (see Table 2). For example, for the second element in the first row, the evaluator e1 ranks x2 and x4 in the 3rd place, while the group ranks x2 in the 4th place and x4 in the 3rd place; then, we have V2 − V21=−1 and V4 − V41=0. When taking a=1/(n − 1), b=0.7, p(x) is calculated according to equation (11) and the results are obtained (see Table 3).

Table 2

V − V calculation results of the 5 alternatives.

a	V 1 c 1 − V1k	V 2 c 1 − V2k	V 3 c 1 − V3k	V 4 c 1 − V4k	V 5 c 1 − V5k
e 1	0	−1	0	0	−1
e 2	0	−2	2	1	−1
e 3	0	−1	3	−1	−1
e 4	0	−1	3	−2	−1
e 5	−2	−2	0	1	3
e 6	−1	1	2	−2	−1
e 7	0	0	0	0	−1
e 8	−3	1	0	0	2

Table 3

Approximation of individual evaluators for each design alternative.

	P k (x1)	P k (x2)	P k (x3)	P k (x4)	P k (x5)
e 1	0.000	0.379	0.000	0.000	0.379
e 2	0.000	0.616	0.616	0.379	0.379
e 3	0.000	0.379	0.818	0.379	0.379
e 4	0.000	0.379	0.818	0.616	0.379
e 5	0.616	0.616	0.000	0.379	0.818
e 6	0.379	0.379	0.616	0.616	0.379
e 7	0.000	0.000	0.000	0.000	0.379
e 8	0.818	0.379	0.000	0.000	0.616

Then, we calculate the consensus degree of each alternative according to equation (12). C(x1) = 0.773, C(x2) = 0.609, C(x3) = 0.642, C(x4) = 0.704, and C(x5) = 0.537.

Then, we calculate the degree of consensus of all evaluators on the set of alternatives according to equation (13):C=(1 − β)((0.773+0.609+0.642+0.704+0.537)/5)+β × 0.642

When taking β=0.8, the consensus degree is C=0.644 < γ=0.75, and some evaluators need to modify their preference opinions.

Step 13 .

Evaluators modify their preferences according to the feedback mechanism.

By calculating the degree of approximation using equation (14), evaluators who need to modify some of their preferences are identified.

Suppose β=0.8; the degree of approximation of the evaluators is sorted from high to low: e7, e1, e8, e5, e2, e6, e3, and e4.

In this paper, we suppose p=0.75. Then, the two evaluators e3 and e with a low degree of approximation need to modify their evaluation. According to the proposed rules, their reevaluation results are

After obtaining the reevaluation results, we perform the second round of calculations. Similar to the calculation process of the first round, we obtain the consensus degree (0.787) from the second round, which now meets the requirements. Therefore, we take the evaluation results of the second round as the final decision-making basis.

In the same way, we proceed to Steps 2–6 to obtain the evaluation results of the other 5 indicators.

Step 14 .

We use the HWA operator and equation (15) to calculate, in which the indicator weight is Z=(0.08, 0.10, 0.19, 0.15, 0.25, 0.23). By leveraging the fuzzy majority criterion based on the principle of fuzzy quantization operator, i.e., considering as many indicators as possible, ω=(0,0,0, (1/3), (1/3), (1/3)), it is ensured that a design alternative performs well under the indicators as many as possible. Then, we get the overall evaluation and ranking of the 5 alternatives under all 6 indicators.

The alternatives are arranged in descending order: x5≻x4≻x3≻x2≻x1.

4.3. Sensitivity Analysis

Considering that the ranking of alternatives can be affected by the indicator weights, the sensitivity of the proposed method is analyzed and discussed in this section. Specifically, we increase each indicator by 30% and 60%, respectively, and then reduce it by 30% and 60%, respectively. When a particular indicator increases or decreases, other indicators also change proportionally, so that the sum of indicator weights is equal to 1. Twenty-four experiments are conducted, and the rankings of the 24 experiments are demonstrated (see Table 4).

Table 4

Rankings of 24 experiments.

Experiment No.	Ranking orders	Experiment No.	Ranking orders
1	x 5≻x4≻x3≻x2≻x1	13	x 4≻x5≻x3≻x2≻x1
2	x 5≻x4≻x3≻x2≻x1	14	x 5≻x4≻x3≻x2≻x1
3	x 5≻x4≻x3≻x2≻x1	15	x 5≻x4≻x3≻x2≻x1
4	x 4≻x5≻x3≻x2≻x1	16	x 5≻x4≻x3≻x2≻x1
5	x 5≻x4≻x3≻x2≻x1	17	x 5≻x4≻x3≻x2≻x1
6	x 4≻x5≻x3≻x2≻x1	18	x 5≻x4≻x3≻x2≻x1
7	x 5≻x4≻x3≻x2≻x1	19	x 5≻x4≻x3≻x2≻x1
8	x 5≻x4≻x3≻x2≻x1	20	x 5≻x4≻x3≻x2≻x1
9	x 5≻x4≻x3≻x2≻x1	21	x 5≻x3≻x4≻x2≻x1
10	x 4≻x5≻x3≻x2≻x1	22	x 5≻x3≻x4≻x2≻x1
11	x 5≻x4≻x3≻x2≻x1	23	x 5≻x4≻x3≻x2≻x1
12	x 4≻x5≻x3≻x2≻x1	24	x 4≻x5≻x3≻x2≻x1

According to the results of sensitivity analysis (see Figure 3), it is easy to find that, among the 18 experiments, x5 is better than the other four alternatives accounting for 75% of the total number of experiments, which shows that x5 is the best alternative of MFIAs. In 22 experiments, x4 is better than x3, accounting for 91.7% of the total experiments. x2 has been ranked fourth in all experiments. x1 ranks the lowest in all experiments, indicating that it is the worst option for MFIAs. Therefore, we can conclude that x5 can be recommended as the best alternative from a comprehensive point of view.

Figure 3

Result of sensitivity analysis.

Sensitivity analysis indicated that due to different indicator weights, the evaluation of MFIAs produced different results. Therefore, the indicator and weight determination should be more prudent when carrying out the evaluation research of MFIAs to ensure relatively fair and objective evaluation results.

4.4. Comparative Analysis

First, through literature review, it can be found that previous evaluation and selection of product design alternatives basically adopt a single preference structure [35, 37, 39], which cannot meet the different needs of evaluators. Apparently, these studies did not take the following factors into account, i.e., the difference in expression habits, educational backgrounds, knowledge, and experiences of different decision-makers. Evaluators usually adopt heterogeneous preference representation structures to express their preference for alternatives. Compared with previous research, this paper studies the evaluation of product design alternatives with multiple preference structures. Evaluators can use multiple preference structures to evaluate product design alternatives. It not only saves the time required for evaluators to adapt to unfamiliar preference structures but also allows evaluators with different expression habits and backgrounds to express their preferences in a more flexible way.

Secondly, TOPSIS [36, 39] and VIKOR [37] are often used to obtain the ranking results of the final alternatives in some existing researches on the evaluation of product design schemes or use the comprehensive coefficient method to get a total score of each scheme and then rank the schemes [35]. However, this method only focuses on the aggregation of individual opinions into group opinions, while ignoring the consensus degree among evaluators. Due to the influence of objective factors such as the uncertainty of things and subjective factors such as the knowledge structure and judgment level of evaluators, the opinions of evaluators often differ greatly. Therefore, it is necessary to consider the level of consensus among decision-makers. In view of this situation, based on the HWA operator and consensus measure, the consensus process is taken into account in the product design evaluation process. It provides effective support for the evaluation of product design alternatives, improves the credibility of the evaluation results, and avoids the consequences of the traditional forcible use of simple aggregate methods to achieve compromise results that are difficult to reflect the true preferences of evaluators.

Finally, multiple preference structures and consensus models were introduced into the evaluation process at the same time. Based on the advantages of multiple preference structures to meet the different needs and expression habits of evaluators, and the advantages of consensus models that can reduce differences of opinion, a comprehensive method combining multiple preference structures and consensus models is proposed. This method provides an effective method for the expression of individual evaluation in the evaluation of product design alternatives and reduces the influence of evaluation group divergence on the evaluation results. It is conducive to the smooth implementation of product design program evaluation and is conducive to building a more harmonious interpersonal relationship within the organization.

5. Conclusions

To solve the problem of the inability to express a certain form of information caused by the heterogeneity of the evaluators in the product design evaluation process, avoid the lack of information caused by using the same form of preference, ensure that all evaluators can naturally and accurately express their preferences, and ensure that a certain degree of consensus on the pros and cons of the program is reached in the case of heterogeneity among evaluators, this paper proposes an approach for reaching consensus with mixed preference information on the evaluation of product design alternatives. First, a random evaluator expresses his/her preference for each product design alternative by randomly using a preference representation from these: preference order, utility value, reciprocal judgment matrix, complementary judgment matrix, interval number evaluation value, and linguistic evaluation value, so as to retain the integrity and accuracy of his/her evaluation information. Second, we use the corresponding transfer function to uniform the different preference forms of the evaluators into the form of a complementary judgment matrix. Third, we use the aggregation operator to aggregate the opinions of the evaluators into group opinions and consider the weights of individual evaluators and evaluation indicators. Then, by comparing individual preferences and group preferences, we measure the group consensus and identify outlier opinions. Finally, we ask evaluators with the smaller value or the evaluator whose rank is at the lower end to modify their opinion according to the feedback mechanism until the group reaches a consensus. We then use a product design evaluation case to illustrate the feasibility and effectiveness of the proposed method. The results show that the approach is easy to operate, taking into account the heterogeneity needs among product design evaluators, and rationally uses the preference information provided by the evaluators. In this way, the evaluation results we obtain from the process are more accurate and robust. The approach has strong practical value and provides an important basis for the effective development of design activities.

This research has three main contributions. Firstly, we proposed an approach for reaching a consensus on product design alternative evaluation with mixed preference information reflected. In order to solve the problem of the evaluator group having obvious heterogeneity during the evaluation process, we have allowed for various preference representation forms. Research shows that this method can solve the problem of a single preference structure being unable to meet the requirements of heterogeneous evaluator groups. This approach also makes the evaluation results more reasonable and credible. Secondly, we introduced the HWA operator into the process of aggregating the evaluation results. The HWA operator considers both the importance of the ranking position of each alternative and the importance of the alternative itself, making the aggregation result more accurate. Furthermore, we take into consideration the consensus-reaching process in the evaluation of product design alternatives. This approach provides effective support for the multiobjective evaluation of product design alternatives, improves the credibility of the evaluation results, and avoids the situation where evaluation results are compromised and decision-makers' true preferences are not faithfully reflected when using traditional aggregation methods. At the same time, we make adjustments to the evaluation of the alternatives based on the feedback information, and only by modifying certain individual evaluators' preference opinion, individual decision-makers' heterogeneous characteristics as well as their influence within the group are reflected.

However, the approach proposed in this paper still possesses limitations. In the evaluation process of product design alternatives in real life, the six reference forms mentioned in this paper cannot comprehensively summarize all the forms preferred by evaluators with obvious heterogeneity. Meanwhile, complex linguistic expressions will be considered in future studies due to their increasingly widespread application, such as hesitant fuzzy linguistic information [81], distributed linguistic information [82], HFLTS probability distribution [83], and comparative linguistic expressions [84, 85] and other complex language expressions and more preference forms. Lastly, only 8 decision-makers participated in the research, which is another limitation of this paper. It is our hope that we can solve these problems in future research, focusing on meeting the heterogeneous needs of evaluators in the process of product design evaluation, and obtaining robust product evaluation results.