An integrated expert weight determination method for design concept evaluation.

Expert weight determination is a critical issue in the design concept evaluation process, especially for complex products. However, this phase is often ignored by most decision makers. For the evaluation of complex product design concepts, experts are selected by clusters with different backgrounds. This work proposes a novel integrated two-layer method to determine expert weight under these circumstances. In the first layer, a hybrid model integrated by the entropy weight model and the Multiplicative analytical hierarchy process method is presented. In the second layer, a minimized variance model is applied to reach a consensus. Then the final expert weight is determined by the results of both layers. A real-life example of cruise ship cabin design evaluation is implemented to demonstrate the proposed expert weight determination method. To analyze the feasibility of the proposed method, weight determination with and without using experts is compared. The result shows the expert weight determination method is an effective approach to improve the accuracy of design concept evaluation.

Introduction

Design concept evaluation is a critical phase in new product development (NPD). An ideal initial design concept can match customers’ requirements, and save time and cost for companies in the competitive global market[1]. The preliminary design concept often shows the novelty, feasibility and quality of the product. Design concept evaluation in the early stage is a phase to choose a suitable plan from the initial design concepts. It is essential in the early stage of NPD, especially for complex products[2], because once the design concept is fixed, it is not easy to modify it in later stages. The requirements and preferences of customers and the structure and material of the product are determined in this phase[3]. The cost and sustainability of the product are also estimated.

As the design concept evaluation is based on a variety of factors, it can be seen as a multiple attributes decision making (MADM) problem. In MADM problems, due to the uncertain environment, individual judgement may be imprecise and subjective, and thus group decision making (GDM) is an effective solution in design concept evaluation[4]. In GDM, the ranking of the alternatives is recommended by integrating experts’ judgements. Kabak et al.[5] reviewed related literature and proposed a generic conceptual MADM framework with three stages, as shown in Fig. 1. The first stage is the structuring stage. Alternatives, attributes and experts are defined in this stage, and the weights of experts are also determined here. After that, in the assessment stage, the weights of attributes are obtained. In the final selection phase, the alternatives are ranked based on an appropriate mathematical model.

Figure 1

Design concept evaluation process.

In the past, the alternative ranking approach attracted the most significant attention. Up to 2010, over 70 selection methods had been previously proposed in MADM problems. According to King[6], the analytical hierarchy process (AHP), utility theory, graphical tools, quality function deployment (QFD) and fuzzy logic[7,8] approaches are the most popular methods in design concept evaluation. In particular, the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), the VlseKriterijumska Optimizacija I KOmpromisno Resenje (VIKOR), the ELimination Et Choice Translating REality (ELECTRE) methods and their extended approaches have been developed to address MADM problems[9-12]. Scholars are willing to establish a proper evaluation model to deal with specific real-life projects. However, studies on expert weight determination are very limited[13,14]. Expert weight determination is only mentioned in 41% of the top-cited papers on GDM. Therefore, experts are treated as a homogeneous group by default, and the individual weight is set to average. Design concept evaluation is a specific application area of MADM methods[1], and the most frequently cited studies in this area are listed in Table 1. Although researchers have made efforts to optimize the method to obtain criteria weights, expert weight is rarely mentioned.

Table 1

Highest cited studies in design concept evaluation.

Reference (year)	Journal	Expert weight determination method	Criteria weight determination method	Example
Zhai (2009)[15]	Expert Systems With Application	N/A	Grey relation and rough set	Illustrative example
Zhu (2015)[16]	Advanced Engineering Informatics	N/A	Rough number based AHP	Lithography tool
Tiwari (2016)[17]	Advanced Engineering Informatics	N/A	Rough set and VIKOR	Testing ring machine
Geng (2010)[4]	Expert Systems With Application	N/A	Vague cross-entropy method	HDD machine
Song (2013)[1]	Journal of Engineering Design	N/A	Rough number-based AHP	Mini-fridge
Shidpour (2016)[18]	Expert Systems with Applications	N/A	Extendede method on fuzzy AHP	Mobile
Zhu (2020)[19]	Applied Soft Computing	N/A	Fuzzy rough-number based AHP	Heat exchanger

In Fig. 2, from the cognitive system of design concept evaluations, we can infer that an expert needs to make an individual judgement based on their cognition and preference. Thus, the judgement is subjective, depending on the expert’s knowledge, culture, experience and aesthetic. However, it is practically impossible to form a group of experts with a similar background. The drawback of an individual in the expert group may influence the accuracy of experts’ judgement.

Figure 2

Cognitive system of design concept evaluation[20].

Once the weight of each expert is distributed equally or ignored, the final decision may lead to an incorrect result. Thus, some researchers started to develop an appropriate method to determine the expert weight. In their investigations, not every selected expert is familiar with all the criteria requiring judgement, and some specialized criteria may be beyond the cognition of some experts. Thus, the contribution of experts may not be equal in the decision process. It is essential to determine the weight of experts in the decision process to eliminate any deviation caused by the experts’ imprecise cognition.

Generally, there are three ways to solve weight determination problems: the subjective approach, the objective approach and the integrated approach, as shown in Fig. 3. For the subjective methods, the weight of experts is calculated by integrating the evaluation of each other expert, depending on their age, attitude, experience etc.[21]. The objective methods are based on available evaluation data, and no extra information is required. Hence, the expert weight is computed based on an appropriate mathematical model[22]. Compared to the subjective methods, they are generally more objective, but they ignore the expert’s personal preference. In this part, the expert weight determination methods are reviewed.

Figure 3

Classification of weight determination methods.

In the early stage, subjective methods dominated in studies on expert weight determination. Expert weights were assigned by the supervisors or by comparison between individual expert groups. In the subjective expert weight determination, pairwise comparison matrices were established, then methods with a geometric mean were proposed to determine the expert weight. Multiplicative AHP[23] and Simple Multi-Attribute Rating Technique (SMART) are two key approaches in subjective expert weight determination[24].

Ramanathan and Ganesh[23] were the first scholars to develop the Multiplicative AHP framework in the expert weight determination area. They proposed an eigenvector based method by comparing the experts’ influence in pairs, as interpersonal comparison can help experts achieve a consensus without any interaction[23]. Barzilai et al.[24] improved the Multiplicative AHP method and explained the relationship between the Multiplicative AHP method and the SMART method. The geometric mean of the scaled gradation indices or values is applied in both methods[21]. As well as the methods mentioned above, Delphi is another competent subjective approach in expert weight determination. Azadfallah[25] applied the Delphi technique by comparing the attributes by experts in pairs, then computed the weights with the eigenvector method.

Subjective expert weight determination methods are based on relative judgements. To make precise judgement, the experts are expected to be familiar with every attribute. However, in real-life projects, it is not practical for every expert to meet all the requirements. Moreover, once the number of attributes, alternatives, or experts increases, pairwise comparative work would be a huge project. Due to the heavy workload, pairwise comparative judgements are not easy to implement in complex MADM problems. Objective weight determination methods have developed rapidly in recent years. There are three objective expert weight determination methods, as well as some special methods such as the Markov Chain’s theory[26].

In similarity-based methods, expert weights are determined by distinguishing the expert’s evaluation or measuring the expert’s distance to the aggregated decision[27,28]. The similarity-based study is very close to the TOPSIS method, in which the expert whose decision has minimum distance to the ideal solution has the highest weight. Yue[28,29] proposed a modified TOPSIS method, to determine the expert weights by measuring the distance between the expert judgement and the ideal decision. Yang proposed a rough group decision matrix to determine the expert weights[30]. Wan[31] constructed a bi-objective program which could maximize the minimum degree of acceptance and minimize the maximal degree of rejection, and also introduced three approaches to measure the distance between the individual preference and the group. Jiang[32] introduced a novel method to measure the distance between the expert preference and the ideal solution.

Index-based methods are divided into two groups: consensus-based methods and consistency-based methods. In consensus-based methods, experts are assigned to adjust their preferences, or vary their weights to make their judgements more similar[33]. Pang[34] developed a non-linear programming model, and determined the expert weight by maximizing the group consensus where the expert weight adjusts adaptively with the experts’ decision. Xu[35] also used a non-linear model, and proposed a genetic algorithm in expert weight determination. Dong[36] introduced a consensus reaching process in group decision making. The method is proposed for non-cooperative expert groups, and the expert weights are determined dynamically by a self-management mechanism. Consistency-based methods are superior in inaccurate judgements, using consistency indexes against the group decision to determine the weights of experts[37]. When some experts have higher reliabilities compared to others, the decision makers need to reach a consistent view through negotiations. Liu[38] introduced an expert weight adjusting method based on the consistency-based method. The expert weights are firstly computed by the AHP method and the entropy method, and then optimized by the consistency level of a black-start result. Chen[39] established a collective consistency matrix of all experts and determined the weight of experts by the consistency degree of each expert.

When the group is large, a clustering-based method may be appropriate in real-life problems. A multi-level weight determination method may optimize weight via different models according to the features of the current layer[40]. Sometimes experts are assembled in several specific groups, and the experts in the same group have a similar background, but the gaps between groups are large. As shown in Fig. 4 by Liu[40], a two-layer model is proposed based on the 2-tuple linguistic (2TL) model. In the first layer, decision makers are separated into clusters by certain criteria. The weight of the group depends on the importance of itself. The clusters are regarded as small systems with similar individual information, which is a well-organized system. Liu utilized an Entropy Weight Model to reach consistency. In each cluster, the experts’ status, occupations and experiences are close. Therefore, their judgements should be similar but not identical[41]. To reach a consensus, a Minimized Variance Model is implemented in this layer to seek a minimized deviation among all variables.

Figure 4

Two-layer expert weight determination method by Liu[40].

The literature proposed problem-solving methods through a specific mathematical model or operator to determine expert weights, through methods such as comparing the deviation between the expert preference and the ideal solution, or simply adjusting the expert weight to make the experts achieve consensus. However, there are still some drawbacks. First, as some of the mathematical models or operators may focus on the overall average, some information that deviates from the average level may be ignored. Second, existing studies usually obtain the expert weight by a specific method. Once the problem is complex, one single method cannot reflect a real-life problem accurately.

The integrated method relies on an integration of two or more methods to eliminate the drawbacks of a single approach or simplify a complex method. As integrated methods show their advantage in solving complex GDM problems which have become more pronounced in recent years, an increasing number of integrated expert weight determination methods are presented in decision making[22]. Qi[42] proposed models based on various conditions under interval-valued intuitive fuzzy decision environments and determined both criteria weight and expert weight. First, they introduced a method to measure the gap between decision matrices and the ideal decision matrix, and then they developed an approach to evaluate the similarity degree between individual decision matrices. Liu[43] proposed another expert weight determination method. The method integrated both subjective and objective expert weight determinations in decision making. First, a plant growth simulation algorithm is applied to get the generalized Fermat–Torricelli point of individual preferences with interval number decision matrices. Then a similarity-based expert weight determination method is used. Finally, the expert weights obtained from both methods are aggregated. Jabeur[44] also determined the expert weight based on subjective and objective components.

In product design concept evaluation, a group of experts must be selected in a proper way to ensure the correctness of the assessment. In complex product design concept evaluation projects, as shown in Fig. 5, experts are normally selected from experienced consumers and expert producers[45].

Figure 5

Development of design concept evaluation experts.

Normally, most expert producers are designers and manufacturers. Hence, we can categorize the decision makers into the designer cluster (DC), the manufacturer cluster (MC), and the consumer cluster (CC). The cluster category can perfectly reflect the clustering information of experts. In addition, the integration of subjective and objective weights can improve the accuracy of the design concept evaluation[46]. However, recent studies seldom consider the subjective expert weight due to the workload when the expert group is large.

In our work, we integrated the subjective and objective expert weights with a 2-layer cluster weight determination. The distribution of expert preferences in the clustering-based method is illustrated in Fig. 6. In the cluster layer (Layer 1), the Shannon entropy model can illustrate the organization of the condition of the clusters, but cannot well reflect the individual expert preferences. Hence, an aggregated method integrated by AHP and entropy weight model is proposed under this layer. In the subsection layer (Layer 2), the decision makers in the same cluster are the experts with similar knowledge and background, so their preferences should be highly consistent[40]. Under the circumstances, subjective pairwise comparison is omitted here, because on the one hand, it may largely increase workload, and on the other hand, the influence may not be obvious. Thus, an objective minimized variance model is used here.

Figure 6

Distribution of experts in clusters.

The rest of this paper is organized as follows. In “Preliminaries”, relevant expert weight determination methods are reviewed. In “Methodology”, the conceptual framework of expert weight determination is presented. In “Case study”, a real-life example is implemented and related analysis is presented. In “Comparison of methods with and without using weight determination”, a comparison with and without using expert weight determination is discussed. In “Conclusions”, the conclusion is provided.

Preliminaries

This study proposes a novel integrated expert weight determination method. Before presenting our method, some related expert weight determination methods are reviewed.

The multiplicative AHP method

Multiplicative AHP is a significant subjective expert weight determination method. The method is easy to implement in real-life cases[23]. Similar to other subjective methods, the Multiplicative AHP determines the expert weight through pairwise comparisons. Initially, each expert in the decision maker group is allocated to assess every group member. It may lead to a personal upward bias[21,26]. To eliminate the error caused by the expert’s assessment, in Honert’s study, each expert’s comparisons with the expert him/herself are no longer counted. For example, each expert in the decision maker group with experts only needs to make comparisons.

The expert weight determination method of Multiplicative AHP in Honert’s approach can be summarized in the following steps[21].

: Assume the expert group has members, DM is a member of the expert group. As is shown in Table 2, every expert is assigned to make a linguistic comparison between attributes by individual judgement using pre-provided words (Very strong preference / Strong preference / Definite preference / Weak preference / Indifferent). and represent the expert’s preference for the alternatives and , respectively. Hence, comparison between and can be converted into a numerical value by a geometric scale based on the content of Table 2. Then matrix can be obtained by the equation below:where represents a scale parameter, with a frequently-used value of . Next, approximate vector of stimulus values can be determined by the logarithmic least-squares method by Lootsma[47]. The vector minimizes

Table 2

Integer-valued index designating the gradations made by decision makers[24].

Comparative judgement	Gradation index δ_jkd
Very strong preference for S_k versus S_j	− 8
Strong preference for S_k versus S_j	− 6
Definite preference for S_k versus S_j	− 4
Weak preference for S_k versus S_j	− 2
Indifference between S_k versus S_j	0
Weak preference for S_j versus S_k	+ 2
Definite preference for S_j versus S_k	+ 4
Strong preference for S_j versus S_k	+ 6
Very strong preference S_j versus S_k	+ 8

: Substituting , and , the function transforms to

The associated set of normal equations can be transferred by towhere . The variate has the same value range in the following equations in this part. denotes the cardinality of . According to Lootsma, . comparisons are made for each expert, the maximum pairwise comparison is , and Eq. (4) can be rewritten as:

The equation can be simplified as:

: For any alternative and , from Table 2, we have , is empty and .

Assume each expert made all the comparisons, then , the equation can be written in the following form:

Thus the expert weight of decision makers can be computed by Eq. (8).

After normalization with the equation below, the subjective expert weight can be determined.

2-tuple linguistic and the model of the two-layer weight determination method

Definitions

As described in “Introduction”, the two-layer weight determination method is a critical cluster-based method. The 2-tuple linguistic (2TL) provides the environment of the two-layer method, and this part introduces the rationale to select 2TL. Due to the uncertain environment, fuzzy set[48], rough set[49], grey decision[50], and some other extended methods are applied in MADM problems[51]. 2TL is a model based on the linguistic fuzzy set proposed by Herrera and Martínez[52]. A symbolic translation value is introduced to the linguistic fuzzy set to describe the flexibility of words. Reviews and extended models of 2TL can be found in Martínez[53,54] and Malhotra[55]. Xu proposed non-linear aggregation operators in decision making with uncertain linguistic information[56,57]. Wang and Hao[58] introduced a new 2TL model based on ‘symbolic proportion’ to preserve the integrity of information, as the proportional 2-tuples can well illustrate the uncertainty of the linguistic judgement. As linguistic term sets distribute uniformly and unsymmetrically, Herrera[59] defined the term sets as unbalanced linguistic term sets to demonstrate the non-linear thinking of human beings, and proposed a method integrated by a representative algorithm and a computational approach. As an effective method in MADM, the 2TL model has produced crucial results in various areas such as quality assessment[60], web system design[61] and group decision making[62]. The 2TL model perfectly matches most subjective problems as the linguistic variables have their advantage in expressing ‘approximate information’. For example, we can use the 2TL model to describe ‘how young the person is’, using the linguistic words ‘very young, young or not young’.

In design concept evaluation, experts are assigned to evaluate plans on a large number of different attributes. Some of the attributes do not have an exact value, and thus the evaluation of the experts may be subjective and imprecise. Approaches such as the House of Quality method with multi-point scale measurement and intuitive fuzzy set are proposed to tackle the challenge. However, the 2TL is suitable for design concept evaluation problems for two reasons. First, design concept evaluation is complex multi-attribute decision making based on the aggregation of the decision makers’ judgements, which belongs to the MADM problem. Second, experts prefer to make judgements in natural language with some adverbs of degree such as ‘very’, ‘extremely’ etc.[55] In design concept evaluation, raw data and information may be uncertain, imprecise and vague, which well matches the category of 2TL model problem-solving, and the evaluation data can be used as raw data in some MADM methods such as TOPSIS and VIKOR.

The notions, terminology definitions, and related equations of 2TL are presented below.

Definition 1

Let set as a linguistic term set (LTS), let be a numeric result of the LTSs and be the LTS cardinality. We have two values and , where is called a symbolic translation, and . is the round operation in the definition.

Definition 2

[52] Let be the aggregation of LTSs from S, . The 2-tuple gives the same information of and can be described as:where in expression (10), is a function to obtain the 2-tuple linguistic information. With the help of value 0 as the “symbolic translation” in 2-tuple linguistic term demonstration, transfers to a 2-tuple . Herrera[52] also gives a comparison rule of 2-tuples. Assume and are both 2-tuples, then

If ;

If , then

If ;

If ;

If .

Definition 3

[52] Let . is a function restoring the 2-tuple to its numerical value , where

Definition 4

[52] Let . Then 2-tuples is denoted by to . The 2-tuple arithmetic mean (TAM) is given as:where in Eq. (14).

Definition 5

[63] The deviation between 2-tuples and can be described as:

Moreover, we can easily get the following results from Eq. (15):

;

.

Definition 6

[52] Let , 2-tuples is denoted by to , let as the weight vectors of S. The 2-tuple weight average (TWA) operator is given as:where in Eq. (16).

Definition 7

[40] Let . A 2-tuple matrix is expressed as , where ,. Let be the covariance between a and b, let 2-tuple , let be the deviation between and . Then

When , let be the variance deviation of , then have . The equation above can be converted to:where .

Computing process

The computing process of the two-layer determination method is described as:

Assume is an LTS. The criteria (attribute) group and the expert group are expressed as and respectively. represented as l alternatives. The experts (E) are assigned to evaluate the alternatives (A) according to the criteria (C) using the LTS (S). The experts are required to give their evaluation using linguistic terms. As for expert , the decision matrix is described as:where and represent the index of alterative (), criteria () and expert () respectively. In the two-layer method, m experts can be divided into f clusters, expressed as . There are experts in the cluster , where and .

Minimized variance model in layer 2.

The minimized variance model relies on minimizing the total variance preference in a cluster. The matrix can be transferred into 2-tuple as follows:where is a 2-tuple.

In the second layer, assume expert is the expert in the cluster , we denote by the decision matrix in . Hence, we can rewrite Eq. (20) as:where . To make the computing process simple, the matrix of an expert is converted to a vector:where . The decision matrix of cluster is

To get the optimized solution, it is required to make the sum of the variance of the attributes weight small.

According to Definition 7, we can get the weight evaluation of cluster using Eqs. (24) to (27):where is the arithmetic mean of , and indicates the weight vectors of experts in cluster , and . The definition of symbol * in this equation is defined as:

The summary of variances is:

The optimization model is shown below:

Then, the optimal weight value of variate can be determined as .

Entropy weight model in Layer 1.

In the first layer, the TAM operator is applied in the entropy weight model. As the basic equation of entropy, we have:where the constant k is calculated as . Let as the decision matrix of the clusters. The equation above can be converted to:

Let be the weight vector of clusters. We can compute the cluster weight by the equation below:

Expert weight calculation.

By now, the expert weight in cluster and the cluster weight are determined, and the i expert weight in cluster G, the expert weight w can be obtained by multiplying the two weights.

Thus, the expert weight can be determined by the two-layer weight determination method.

Methodology

In design concept evaluation, attributes are selected from multiple dimensions, such as shape, color, ergonomics, material and manufacturing technology, etc. The experts are categorized into clusters depending on their different backgrounds. Therefore, a two-layer expert weight determination is a good solution to design concept evaluation in NPD. However, there are still some drawbacks to this method. The complexity of different products varies, experts in the customer cluster may not be familiar with how the product works, and they may make judgements relying only on their experiences. Moreover, if the experts’ weights merely depend on the objective method, the subjective preferences of the experts are ignored. Thus, an integrated two-layer expert weight determination approach is proposed, as shown in Fig. 7. In the first layer, we used a mixed weight determination method based on the AHP and entropy weight model. In the second layer, a minimized variance model as mentioned above was applied. After that, the final expert weight is determined by combining the two layers’ results. The proposed method is illustrated below.

Figure 7

The framework of the proposed method.

In this step, a 9 LTS S is constructed according to the 2TL environment.

Then, the experts are assigned to give their preferences by the languages given in the LTS. The criteria (attribute) group and the expert group are expressed as and respectively. represents l alternatives. Then, m experts can be divided into f clusters, expressed as . There are experts in the cluster , where and .

The experts are assigned to give their evaluation using linguistic terms. As for expert , the decision matrix is described as:where and represent the index of alternative (), criteria () and expert (), respectively. The matrix can also be shown with 2-tuple:where is a 2-tuple. In the second layer, assume expert is the expert in the cluster , we denote by the decision matrix in . Hence, we can rewrite Eq. (33) as:where . To make the computing process simple, the matrix of an expert is converted to a vector:where . The decision matrix of the designer cluster is:

Thus, the decision matrix of layer 2 can be represented as .

The weight of experts in the corresponding cluster is calculated by the minimized variate model in the second layer.

Generally, experts in the same cluster have a similar background, which means the experts have a high possibility of having similar preferences for the alternatives. It is required to minimize the variance of the attributes of the clusters. To get the optimized solution, it is required to make the sum of the variance of the attributes weight small. Using Eqs. (37) to (40), we can get the weight evaluation of the designer cluster:where is the arithmetic mean of , and indicates the weight vectors of experts in cluster , and the summary of weight vectors is 1. The definition of symbol * in this equation is:

The summary of variances is:

The optimization model is shown below:

Then, the optimal weight value of variate can be determined as . Similarly, the optimal weight value of the rest of the clusters in the second layer can be determined in the same way.

Next, we determined the cluster weight in the first layer by a hybrid method. In this step, we use the entropy weight model to determine the objective weight of clusters.

In the first layer, the TAM operator is applied in the entropy weight model. For the cluster , as the basic equation of entropy, we have:where the constant k is calculated as . Let as the decision matrix of clusters. The equation above can be converted to:

Let be the weight vector of clusters. Then

The result of can be computed as the weight vector of clusters by the objective method.

Here in this step, a subjective method is also implemented in cluster weight determination. The experts are assigned to assess the importance of three groups by the AHP method. They give their comparison of preferences among the clusters.

First, the experts are assigned to compare the clusters, the comparison among . Every expert is assigned to make a linguistic comparison between clusters by individual judgement using specified words (Very strong importance / Strong importance / Definite importance / Weak importance / Indifferent). The comparisons made by experts in between cluster and are converted into a gradation index by geometric scale shown in Table 3. The comparison can be recorded in the comparative table in the form of Table 3. Where represents the comparison of made by the expert in cluster . After that, the arithmetic means of the comparison by cluster can be computed, which is also shown in Table 4. denotes the arithmetic mean of the comparison made by experts in cluster , then we have .

Table 3

Integer-valued cluster important judgement designating the gradations made by decision makers.

Comparative judgement	Gradation index
Very strong importance for G_α versus G_β	− 8
Strong importance for G_α versus G_β	− 6
Definite importance for G_α versus G_β	− 4
Weak importance for G_α versus G_β	− 2
Indifference between G_α versus G_β	0
Weak importance for G_β versus G_α	+ 2
Definite importance for G_β versus G_α	+ 4
Strong importance for G_β versus G_α	+ 6
Very strong importance G_β versus G_α	+ 8

Table 4

Form of the data recorded in the table based on pairwise comparative judgements.

Cluster G_1				Cluster G_2		⋯	Cluster G_f
	G_1/G_2	⋯	G_f-1/G_f		⋯	⋯		G_1/G_2	⋯	G_f-1/G_f
1	G11(G_1/G_2)	⋯	G11(G_f-1/G_f)	1	⋯	⋯	1	Gf1(G_1/G_2)	⋯	Gf1(G_f-1/G_f)
2	G12(G_1/G_2)		G12(G_f-1/G_f)	2			2	Gf2(G_1/G_2)		Gf2(G_f-1/G_f)
…	…		…	…			…	…		…
m1	G1m1(G_1/G_2)		G1m1(G_f-1/G_f)	m2			mf	Gfmf(G_1/G_2)		Gfmf(G_f-1/G_f)
AVE	G_1(G_1/G_2)	⋯	G_1(G_f-1/G_f)	AVE	⋯	⋯	AVE	G_f(G_1/G_2)	⋯	G_f(G_f-1/G_f)

The comparison matrix of clusters can be determined by the equation below:where represents a scale parameter, with a frequently-used value of . Next, the approximate vector of stimulus values can be determined by the logarithmic least-squares method by Lootsma[47]. The vector minimizeswhere and represent the relative power of and made by , respectively. Assume and , expression (45) can be described as:

The associated set of normal equations can be transferred by to:where , and the variate has the same value range in the following equations in this part. denotes the cardinality of . According to Lootsma, we have . comparison made for each expert, the maximum pairwise comparisons are , and Eq. (47) can be rewritten as:

The equation can be simplified as:

For any cluster and , from Table 2, we have , is empty and .

Assume each expert made all the comparisons, then , the equation can be written in the following form:

Thus the expert weight of decision makers can be computed by Eq. (48).

After normalization with the equation below, the subjective expert weight can be determined.

In Step 3 and Step 4 above, the objective cluster weight and the subjective cluster weights were determined. Here in this step, combined weights are determined using the equation below to get the combined weight of clusters.where is the adjusting coefficient, here represents the superiority of the subjective method over the objective method in the combination. When , the subjective determination of the DM group is superior, on the contrary, when , the objective method is superior, and when , both methods made the same contribution in the cluster weight determination.

Thus, the final weight of each expert can be computed by:

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

No informed consent was required, because the data are anonymized.

Case study

In our study, the new two-layer expert weight determination method is applied in optimization of the cabin design plan for a mid-sized cruise ship. Before the evaluation, three design schemes are proposed, and a decision needs to be made from the three alternatives. To make the decision, a 10 attribute criteria index shown in Table 5 is fixed to determine the alternatives.

Table 5

Attributes defined in the case study.

Attribute	Specification	Attribute	Specification
C1	Planning Compliance	C6	Style and trend
C2	User acceptance	C7	Reasonable placement of furniture
C3	Design humanized	C8	Innovation and competitiveness
C4	Design aesthetics	C9	Cost matches quality control
C5	Ergonomics	C10	Development feasibility

A 30-expert group is formed with 10 cruise ship interior designers, 10 cruise ship manufacturing specialists and 10 customers with over two cruise trip experiences. The raw data is shown in Appendix 1.

First, according to Eqs. (32) to (40), a minimized variance mode is established. Then the optimized lambdas(λ) are determined by the software Lingo. The corresponding weight of experts in each group are shown in Table 6.

Table 6

Weight of experts in the designer, manufacturer and customer clusters.

λ	Designer	Manufacturer	Customer
λ1	0.0994	0.0941	0.0884
λ2	0.1001	0.1042	0.0851
λ3	0.1002	0.1020	0.0904
λ4	0.1044	0.0992	0.0838
λ5	0.0982	0.1040	0.1073
λ6	0.1012	0.0984	0.1186
λ7	0.1060	0.0960	0.1107
λ8	0.0991	0.0982	0.1037
λ9	0.0945	0.1051	0.1048
λ10	0.0967	0.0987	0.1073

After that, according to the TAM operator from Definition 3, the decision matrix D is shown in Table 7.

Table 7

Decision matrix D.

	Alternative 1	Alternative 2	Alternative 3
Designers	(S6, − 0.29)	(S7, − 0.19)	(S7, − 0.17)
Manufacturers	(S6, − 0.21)	(S6, + 0.42)	(S6, + 0.42)
Customers	(S6, + 0.14)	(S7, − 0.43)	(S7, − 0.43)

We can calculate the objective weight of the clusters .

We get the pairwise judgement about the importance of the clusters from Appendix 2. The data in Table 8 is the arithmetic mean calculated from subjective pairwise judgements using the weighting method in each cluster. Here in the table, we leave the cell blank when the cluster is compared against itself. For other cells, the symbol “-” means that the corresponding row or column is not permitted to be compared with others. The normal equation can be expressed as:

Table 8

Hypothetical subjective pairwise judgements of clusters.

	DC			MC			CC
DC				–	–	–0.4	–	–1	–
MC	–	–	0.4				–1.8	–	–
CC	–	1	–	1.8	–	–

In the equation, , Eq. (55) can be simplified as:

For DC, , we have

For MC, , we have

For CC, , we have

Thus, we can compute as

The weight of the clusters can be calculated by Eqs. (50) to (52). In this case, the subjective weights of clusters are:

The integrated weight of the clusters can be computed by Eq. (53), and when , the weights of clusters are:

The final weights of each expert w are shown in Table 9.

Table 9

Determined expert weight by the proposed method.

Expert	Weight	Expert	Weight	Expert	Weight
D1	0.0422	M1	0.0351	C1	0.0179
D2	0.0425	M2	0.0389	C2	0.0173
D3	0.0425	M3	0.0380	C3	0.0183
D4	0.0443	M4	0.0370	C4	0.0170
D5	0.0417	M5	0.0388	C5	0.0218
D6	0.0429	M6	0.0367	C6	0.0241
D7	0.0450	M7	0.0358	C7	0.0225
D8	0.0420	M8	0.0366	C8	0.0210
D9	0.0401	M9	0.0392	C9	0.0213
D10	0.0410	M10	0.0368	C10	0.0218

Comparison of methods with and without using weight determination

Among MADM methods, TOPSIS is widely used in supply chain management, design concept evaluation, business and marketing management, and some other fields. To illustrate the influence of the expert weight, a comparative analysis is made using the TOPSIS method with and without expert weight determination.

The steps of the TOPSIS method are shown here.

:

For a decision matrix has alternatives and criteria, and , respectively. represents the normalized matrix.

: where represents the weight of criterion .

where and .

:

Let / be the distance between the alternative and the PIS/NIS.

The rank of the alternatives can be determined by comparing . The alternative with the highest is the best solution.

The criteria weights are determined by the subjective method, shown in Table 10.

Table 10

Criteria weight.

Criteria	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10
w_j	0.186	0.084	0.082	0.071	0.028	0.054	0.086	0.108	0.121	0.182

According to the TOPSIS method, the relative variates are calculated. ,, of alternatives with and without expert weights are shown in Table 11.

Table 11

Values of ,, with and without expert weight.

	Decision made without expert weight			Decision made with expert weight
	A_1	A_2	A_3	A_1	A_2	A_3
Si+	0.0271	0.0120	0.0095	0.0280	0.0114	0.0124
Si-	0.0033	0.0208	0.0241	0.0023	0.0234	0.0235
CI	0.1086	0.6336	0.7169	0.0752	0.6724	0.6541
Rank	3	2	1	3	1	2

From Table 11 and Fig. 8, it is clear that the ranks of alternatives are different with and without considering expert weight. In Fig. 8, both methods demonstrate alternative is the least ideal option, and far inferior to the other two solutions. However, when the expert weight is considered, the most feasible alternative changed from to . On the other hand, the variation of ,, with and without expert weight is obvious from Table 11.

Figure 8

CIs of alternatives with and without expert weight.

The difference is caused by the weight of experts. We can infer from the comparative analysis that if the gap between alternatives is huge, the expert weight determination may not influence the final decision. Otherwise, if a gap between alternatives exists but is not particularly large, the expert weight determination may be considered more in decision making. Ignoring the expert weight may cause us to miss the best alternative in real-life projects as verified in this section.

Conclusions

Expert weight determination in design concept evaluation is a critical part that is usually ignored by decision makers. A proper weight determination can make the decision making process more accurate. This paper presented an integrated two-layer expert weight determination method under a complex design concept evaluation process. In some complex problems, experts are divided into clusters by certain characteristics, and the weight of experts can be calculated by individuals (layer 1) and clusters (layer 2). In the first layer, the minimized variance model is presented to determine the individual weight in each group. In the second layer, a hybrid weight determination method is proposed by combining the entropy weight method and the AHP method. A case study in cruise ship cabin design was implemented using the proposed method. Comparison of the results showed that weight determination in a complex product design process is essential, and may sometimes cause different outcomes.

Supplementary Information.