Large-scale group decision-making (LSGDM) for performance measurement of healthcare construction projects: Ordinal Priority Approach.

People with various skill sets and backgrounds are usually found working on projects and thus, group decision-making (GDM) is one of the most important functions within any project. However, when projects concern healthcare or other critical services for proletariat or general public (especially during COVID19), the importance of GDM can hardly be overstated. Measuring the performance of healthcare construction projects is a critical activity and should be gauged based on the input from a large number of stakeholders. Such problems are usually recognized as large-scale group decision-making (LSGDM). In the current study, we aim to propose a decision support system for measuring the performance of healthcare construction projects against a large number of experts using ordinal data. The study identifies several key indicators from literature and recorded the observations of a large number of experts about these indicators. After that, the acceptable range of complexity is specified, the Silhouette plot is provided to find the optimal number of clusters, and the ordinal K-means method is employed to cluster the experts' opinions. Later, the confidence level is measured using a novel Weighted Kendall's W for the optimal number of the clusters, and the threshold is checked. Finally, the conventional problem is solved using the Group Weighted Ordinal Priority Approach (GWOPA) model in multiple attributes decision making (MADM), and the performance of the projects is determined. The validity of the proposed approach is confirmed through a comparative analysis. Also, a real-world case is solved, and the performance of some healthcare construction projects in China is gauged with a comprehensive sensitivity analysis.

Introduction

Amid the COVID-19 pandemic globally, healthcare construction projects have received special attention from the governments in various countries. In such circumstances, the speed of construction was recognized as one of the most critical factors because of the limited capacity of hospitals to admit patients [1]. However, various factors are essential in the performance of healthcare construction projects, and it is crucial to determine the performance of the healthcare construction projects to identify the current weaknesses and solve them for future projects. It is worth mentioning that performance measurement is not an individual activity; it is a teamwork activity to consider the stakeholders’ opinions, especially about the importance of the performance indicators. It can be challenging to determine the performance of healthcare construction projects based on a large number of experts and requires a powerful decision support system to handle such a problem. This question may arise how the performance of the healthcare construction projects can be calculated based on the opinions of a large group of stakeholders? How can this activity be simplified to decrease the computational time and cost? These research questions will be addressed in the current study carefully.

Chan et al. [2] proposed a conceptual framework to illustrate the critical performance factors of construction projects. They have considered five major groups, including project procedures, external environment, project-related factors, human-related factors, and project management actions, as important performance factors in construction projects. Jha and Iyer [3] identified 55 criteria associated with the quality performance of the construction projects. With the aid of statistical analysis, they have divided the specified criteria into two major groups, namely performance and failure factors. As a result, they found that top management support and project managers’ competence are two significant factors that can enhance the construction project’s quality performance. Rojas and Kell [4] incorporated the metrics such as change orders, cost growth, and unit cost to compare the cost performance of the construction projects. The main limitation of their study was focusing on the building projects of the Pacific Northwest school between 1987 and 2005. Bilbo et al. [5] employed two project delivery methods, namely Integrated Project Delivery (IPD) and Construction Manager at Risk (CMR), for healthcare facilities. They also addressed the advantages and challenges of implementing IPD and CMR methods. It was the first study that examined the performance of the IPD and CMR methods in healthcare projects and compared them together. Doulabi and Asnaashari [6] tried to identify the critical factors in the performance of the healthcare facility projects in Iran through interviews with experts with significant experience in this field of study. Finally, they identified eight groups of factors, including scope, time, cost, quality, human resource, risk, environmental, and external matters. It should be noted that 20 experts have identified these factors with at least 15 years of working experience. Hanna [7] evaluated the impact of IPD on the performance of building construction projects by utilizing a broad range of performance attributes. This study used quantitative analyses, which distinguished it from other existing research studies in the same field of study. Iskandar et al. [8] proposed a mathematical model to extract the weights of the performance attributes to determine the performance of the healthcare construction projects objectively. However, no study considers the performance measurement of the construction projects as a large-scale group decision-making (LSGDM) problem. There are several stakeholders in each project, and their opinions should be considered in the performance measurement process.

There are a few studies that utilized LSGDM in the healthcare management context. Liu et al. [9] proposed a new risk priority model utilizing prospect theory and cluster analysis, where there was a large number of experts. They used the similarity measure-based clustering method and considered conflict degree and majority principle, which could enhance the consistency of the experts’ opinions. Also, an entropy-based method was developed to measure the weight of risk factors that could determine the weights objectively. Liu et al. [10] proposed a large group dependence assessment method utilizing interval 2-tuple linguistic variables as well as cluster analysis to incorporate the dependence in human reliability analysis. They also improved the Muirhead mean operator so as to measure the dependence levels among operator actions. Li and Wei [11] presented an LSGDM framework for decision-making in healthcare management. In their study, the input data were hesitant fuzzy linguistic terms, and a novel clustering approach was proposed to categorize the decision-makers into clusters. Zhang and Meng [12] proposed an LSGDM method for selecting mobile health applications in hospitals. They considered both non-cooperative and cooperative behaviours of the experts in their study. To the best of our knowledge, no study used LSGDM for performance measurement in the healthcare management context. Moreover, this is the first study that extends the Ordinal Priority Approach (OPA) under LSGDM in healthcare management systems.

Based on the aforementioned studies, the contribution of the current study can be summarized as follows:

Pioneering a novel framework for determining the weight of attributes in LSGDM problems using the OPA. Moreover, proposing an equally-weighted OPA model and group-weighted OPA model, which can be used to decrease the computational complexity in special cases for solving group decision-making problems.

Proposing Weighted Kendall’s W to measure the confidence level in LSGDM problems, also proposing a formula to estimate computational complexity in the OPA model.

Proposing a framework to determine the performance of the healthcare construction projects using big data for the first time.

The rest of the study is organized as follows: after the introduction section and addressing the relevant studies in the LSGDM context, the preliminaries and methodology section are provided. Then, the proposed decision support system is explained. Afterwards, a comparative analysis is presented to show the feasibility of the proposed framework. Next, a case study that includes five healthcare construction projects is discussed, and the performance of the projects is determined using the proposed approach. Later, a comprehensive sensitivity analysis is performed on the case study to check the stability of the solutions. Finally, the conclusion and discussion are provided.

Large-scale group decision-making

This section aims to review the LSGDM context in the recent years. Liu et al. [13] proposed a decision-making framework for complex LSGDM problems in the intuitionistic fuzzy environment using partial least squares (PLS) path modeling. They addressed the application of the framework to the site selection of hydropower stations in China. Palomares et al. [14] presented a consensus model which was suitable for a large number of experts in decision-making problems using visual analysis tools. They believed that their model could identify non-cooperative behaviours in LSGDM problems using a fuzzy clustering-based approach. Liu et al. [15] proposed a decision-making framework to select the suitable contractor for mega construction projects using two-stage PLS path modeling. They also presented a case study in China and compared the typical PLS and two-stage PLS path modeling, which showed better reliability for two-stage PLS path modeling. Quesada et al. [16] presented a methodology based on uninorm aggregation operators to handle non-cooperative behaviours in LSGDM problems. They utilized computing with words and fuzzy theory approaches in the proposed weighting scheme to facilitate the usage of uninorm operators. Liu et al. [17] proposed a novel method for solving LSGDM problems using both subjective and objective viewpoints to calculate the groups’ weights. They believed that using percentage distribution can make the decision-making process more convenient and promote the ability of visualization. Xu et al. [18] developed a model for solving LSGDM problems under incomplete preference data. Their method was established based on subjective and objective trust degrees to convert the incomplete decision matrix to the complete decision matrix. Xiang [19] proposed a web-based approach to provide an automated platform for LSGDM under emergency conditions due to energy shortage while the time was also limited. Indeed, the experts could register on the platform and provide their opinions about the shortage of energy and the alternatives. These opinions could be used for the LSGDM to choose the best alternative in the emergency condition for the energy shortage. Zhang et al. [20] proposed a new computational model using linguistic information to solve the LSGDM problems. They addressed the application of the proposed model for the talent selection problem in universities. They extended various distance measurement and ranking methods to provide this approach. Rodríguez et al. [21] proposed a novel adaptive consensus reaching processes framework to solve LSGDM problems. This framework was established based on three important steps, including (i) clustering to determine the weight of the experts’ sub-groups, (ii) aggregating the opinions using the hesitant fuzzy approach to minimize information loss, (iii) providing a feedback mechanism, and evaluating the level of agreement. Song and Li [22] presented a model for solving LSGDM problems to attain the maximum level of information collected from the experts. They also employed extended Technique for Order Performance by Similarity to Ideal Solution (TOPSIS) for ranking the alternatives. Zhang [23] proposed a probabilistic method for solving LSGDM problems with probabilistic linguistic term sets. Moreover, a novel probabilistic averaging operator and distance measure formula were proposed to aggregate the experts’ opinions. Ding et al. [24] presented an intuitionistic fuzzy clustering method based on sparse representation to solve LSGDM problems. The main innovation of the model was investigating the intra-relations among the experts to find the leader of the group in the clustering process. Ma et al. [25] proposed a clustering method for solving LSGDM problems under hesitant fuzzy linguistic term sets. The proposed clustering method was advantageous in breaking large-scale problems into small problems. They also provided a new selection process to find the final solution to the problem. Wu et al. [26] stated that the experts’ opinions contained subjective uncertainty, to handle this uncertainty, they used interval type-2 fuzzy linguistic terms to solve LSGDM problems. They also proposed type-2 fuzzy k-means clustering to classify the experts’ opinions.

For more information, the applications of the LSGDM are summarized in Table 1 and were compared with the current study.

Table 1

A brief overview of selected studies involving LSGDM

Authors	Years	Problem description	Can it calculate the confidence level?	Input data type
Liu et al. [13]	2014	Site selection for hydropower station	No	Interval-valued intuitionistic fuzzy number
Palomares et al. [14]	2014	Decision-making about the recent discovery of fossil fuels	No	Fuzzy preference relations
Liu et al. [15]	2015	Biding evaluation for constructing hydropower station	No	7-point Likert Scale
Quesada et al. [16]	2015	Selecting a supportive action plan for the university	No	Fuzzy linguistic terms
Liu et al. [17]	2016	Selecting the teacher appointment system	No	Fuzzy preference relations
Xu et al. [18]	2016	Evaluating investment plans	No	Incomplete preference information
Xiang [19]	2017	Energy network dispatch optimization	No	5-point Likert Scale
Zhang et al. [20]	2017	Selection of dean for business school	No	Multigranular linguistic information
Rodríguez et al. [21]	2018	Selection of programming language for practicing	No	Hesitant fuzzy linguistic terms
Song and Li [22]	2018	Selecting the most sustainable supplier for raw materials for medical	No	Multigranular linguistic information
Zhang [23]	2018	Selecting the teacher appointment system	No	Probabilistic linguistic information
Ding et al.[24]	2019	Selecting path plan for highway construction project	No	Intuitionistic fuzzy information
Tang et al. [34]	2019	Selecting construction routes for adding a new light line to the public transportation system	No	Preference relations
Ma et al. [25]	2019	Selecting a location for constructing an advanced credit system	No	hesitant fuzzy linguistic terms
Wu et al. [26]	2019	Selecting the place for a dinner party	No	Interval type-2 fuzzy preference relation
Chu et al. [47]	2020	Analyzing social network community	No	Incomplete fuzzy preference relations
Gai et al. [48]	2020	Selecting enterprise expansion plan	No	Reciprocal preference relation
Xiao et al. [51]	2020	Selecting the most suitable long-term water management plan	No	Linguistic distribution preference relations
Chao et al. [46]	2021	Beneficiary selection for financial inclusion	No	Heterogeneous preferences
Li et al. [49]	2021	Selecting an emergency response plan	No	Crisp numbers, interval numbers, triangular fuzzy numbers
Lu et al. [50]	2021	Consensus reaching for determining the unit negotiation cost	No	Crisp numbers
The current study	2022	Performance measurement of healthcare construction projects	Yes	Preference relations

Although there are several research items on the LSGDM context, they have not yet employed LSGDM for performance measurement. Moreover, the OPA is a new multiple attributes decision-making (MADM) method, and this is the first extension of it in LSGDM. Indeed, the current study aims to propose a performance measurement framework based on LSGDM and with the aid of the OPA for the first time. Also, Weighted Kendall’s W is proposed for the first time to calculate the confidence level in LSGDM considering the weight of the clusters.

Preliminaries

In this section, we aim to review and propose the essential tools and techniques, which should be employed in the proposed decision support system. First, the OPA of MADM is discussed, and some extensions are provided. Afterward, the ordinal K-means is presented, and the silhouette coefficient is explained. Finally, Weighted Kendall’s W is proposed to develop a confidence level index for the LSGDM problems.

Ordinal priority approach (OPA)

The OPA is a new MADM method proposed by Ataei, Mahmoudi, Feylizadeh, and Li [27]. This method requires ordinal data as input data to determine the weights of experts, attributes, and alternatives. The current study aims to utilize the OPA for solving LSGDM problems to assess the performance of healthcare construction projects. The sets, indexes, variables, and parameters are provided as follows, which are essential for a better understanding of the succeeding models.

Sets

Set of clusters ∀i ∈ I

Set of experts ∀i' ∈ I'

Set of attributes ∀j ∈ J

Indexes

Index of the clusters (1, …, K)

Index of the experts (1, …, p)

Index of the attributes (1, …, n)

Variables

Objective function

Weight (importance) of jth attribute based on expert’s opinion i′ at rank r

Parameters

The rank of expert i′

The rank of cluster i

The rank of attribute j

The number of experts with the same opinions in cluster i

The steps of the original OPA are explained as follows:

Step 1: Specifying the critical indicators which are necessary for performance measurement in the related healthcare construction project.

Step 2: Collecting data regarding the attributes from the experts who are the stakeholder in the healthcare construction project.

Step 3: Preparing Model (1) by utilizing the collected data in Steps 1 and 2. Then, this model can be solved using LINGO software.

Step 4: By solving Model (1), the weights of the attributes and experts can be determined using Eqs. (2) and (3), respectively.

Theorem 1

When the weights of the experts are considered equally , the optimum weights of Model (4) are equivalent to the optimum weights of Model (1) where r = 1, ∀ i′. Also, the optimum objective function of Model (4) is equal to the optimum objective function of Model (1) divided by p. Model (4) is called the equally-weighted OPA model.

Proof of theorem 1

Let us multiply the constraints of Model (4) by p, where p is the number of experts. Hence, Model (5) is obtained.

If we assume that Z = pZ′ in Model (5), Model (6) is obtained.

Since p is always a positive number and there is only one variable in the objective function, solving Model (6) with or Z has no effect on the optimal value of the weights. Therefore, Model (6) can be rewritten as Model (7).

Hence, the optimal value of the weights in Model (4) is equal to Model (7). As a result, the optimum weights of Model (4) are equivalent to the optimum weights of Model (1).

However, it is not easy to solve a large-scale problem using the equally-weighted OPA model while there is a large number of stakeholders in the problem. Bazaraa et al. [28] addressed the complexity of the Simplex method for solving linear programming problems. They stated that for practical problems, in most cases, the iteration number is roughly between 3s/2 and 3s, where s implies the number of constraints. On the other hand, the number of constraints in the OPA method is (p × n × m) + 1, where p implies the number of experts, n implies the number of attributes, and m implies the number of alternatives. Hence, the OPA Computational Iteration (OPACI) can be estimated using Eq. (8).

Since the number of alternatives is zero (m = 0) in the current study, Eq. (8) can be rewritten as Eq. (9).

When the value of OPACI is high, the dimension of the problem should be decreased by considering the available software and hardware. Furthermore, Since the input data of the OPA are ordinal, the ordinal K-means can be employed for grouping the experts and decreasing the value of p in Eq. (9). Later, Model (10) can be used to solve the problem.

Theorem 2

Assume that the experts’ weights are equals in a problem, and some experts provided the same opinions for the rank of all attributes. These experts can be categorized into K groups () while the experts’ opinions in each group (p, ∀ i) are precisely similar. In this situation, the optimum weight of Model (10) with K group of experts is equivalent to the optimum solution of Model (4) with p experts. Also, the optimum objective function of Model (10) is equal to the optimum objective function of Model (4) divided by p. As a result, the optimum weights in Model (10) are equivalent to the optimum weights of Model (1). Model (10) is called the group-weighted OPA model.

Proof of theorem 2

Let us assume that there are p(∀i) experts in Model (4), who have the same opinion regarding the rank of the attributes, and we aim to categorize the experts with similar opinions in K group. Hence, the total weight of the experts with the same opinions is equal to . On the other hand, since we assumed that the weights of all experts are equal, we can declare that . Moreover, instead of in Model (4), we can use , which are equivalent. Considering the aforementioned information, Model (4) can be rewritten as Model (11).

If we multiply Model (11) by p, which is a positive number, Model (12) is obtained:

Let us assume that Z = pZ′ in Model (12). Therefore, Model (12) can be rewritten as Model (13).

Since p is always a positive number and there is one variable in the objective function, solving Model (13) with or Z has no effect on the optimal value of the weights. Therefore, Model (13) can be rewritten as Model (14).

As a result, the optimal values of the weights in Model (4) are equal to the optimal weights in Model (10).

An application of theorem 2

The proposed Model (10) can consider the priority of the clusters on the weights of the attributes using the number of members in each cluster. The application of Model (10) is not limited to this study and may be used to consider the years of work experience of the experts instead of ordinal preferences while calculating the weights of attributes. For this purpose, years of experience of expert i can be considered as p in Model (10).

Ordinal K-means

Clustering has a broad application in real-world problems. Especially when there is a large number of data, it is challenging to analyze them due to computational time and cost. In such a situation, clustering can contribute to the decision-maker to group the data based on one or more distinguishing factors. Cheng and Leu [29] addressed the application of clustering in construction management. In another study, Pham and Afify [30] discussed applying the clustering methods in engineering. Among clustering methods, K-means is one of the most well-known methods [31, 32]. For the first time, K-means was presented by Lloyd in 1982 [33]. K-means was used widely for solving LSGDM problems as well. Since the input of the current study are ordinal data, we aim to employ the ordinal K-means, which was proposed by Tang et al. [34]. However, this algorithm is adopted to be used in the OPA model. The steps of the ordinal K-means are presented as follows:

Step 1: by starting the algorithm, several initializations should be provided. Also, the iteration number should be placed one (I = 1).

Step 2: In this step, initial clustering points should be selected randomly.

Step 3: Using Euclidean distance, the distance from each point to each center should be computed. Eq. (15) can determine the distance from each point to each center.

It should be noted that when the value of n is even, the last sentence of (n − 1)2 + (n − 3)2 + … is 1. When the value of n is odd, the last sentence of (n − 1)2 + (n − 3)2 + … is 2. It should be noted that d ∈ [0, 1] because (n − 1)2 + (n − 3)2 + … is the biggest quadratic difference between two rank series.

Step 4: In this step, each point should be assigned to the closest cluster center based on the obtained value in Eq. (15).

Step 5: In this step, each cluster center should be updated. For calculating the cluster center, the preference vector ought to be used. Assume that is the preference vector regarding the attributes by expert i′. The preference vector should be calculated using Eq. (16)

After that, the clustering center for cluster C can be computed utilizing Eq. (17)

Where implies the preference vector of the i′th expert in the cluster C, and p implies the number of experts in cluster i. In this study, the weights of all experts are considered equally. In order to consider the priority of the clusters based on the number of experts, the value of in the OPA model can be calculated using Eq. (18), which is shown in Model (10) as well.

Where p implies the number of experts in cluster i.

Silhouette coefficient

Rousseeuw [35] proposed a graphical method to check the quality of the clustering, namely the Silhouette plot. The Silhouette plot was established based on the Silhouette coefficient value. The silhouette coefficient is a number between −1 and + 1 constantly. The value of +1 shows a strong association with a cluster as well as the distance from neighbouring clusters. The value of 0 implies no distinct association, and the value of −1 indicates the wrong assignment to a cluster [36]. Silhouette coefficient is employed by scholars widely to calculate the optimal number of clusters [37, 38]. Let us assume that A and C are two different clusters and i′ ∈ A. The value of the Silhouette coefficient can be calculated utilizing Eq. (19) or Eq. (20). or

Where

= Average dissimilarity of expert i′ to all other experts in cluster A.

= Minmum d(i′, C)

Therefore, a higher value of the Silhouette coefficient indicates better quality of clustering. It is essential to check the Silhouette plot during clustering analysis to decrease the error of false clustering. This issue is addressed in the current study carefully because low-quality clustering may change the solution of the problem entirely.

Confidence level measurement

The consensus among the experts plays a vital role in working with ordinal data. The degree of association among the experts’ opinions should be significant which shows the reliability of the input data. It should be noted that Kendall’s W is a non-parametric statistic test that can determine the concordance among the ranks which are provided by various experts [39, 40]. However, this method cannot consider the weight of the experts for calculating the degree of association among experts’ opinions. Hence, we aim to propose Weighted Kendall’s W (W) and employ it for calculating the consensus in each cluster to check the reliability of the input data. The MATLAB source file of the proposed approach is available on https://mathworks.com/matlabcentral/fileexchange/116775-weighted-kendall-s-w. The steps of the Weighted Kendall’s W are presented as follows:

Assume that i (1, 2, …, K) represents the index of the clusters, and j implies the index of attributes. r implies the rank of attribute j by cluster i. ϑ implies the weight of the cluster i (), which is in the current study. The total rank of attribute j (j = 1, 2, …, n) by all experts in cluster i can be calculated using Eq. (21).

After that, mean value of the attributes’ ranks should be calculated by utilizing Eq. (22)

Eq. (23) determines sum of squares of the observed deviations from the mean .

Using the obtained value in Eq. (23), the value of Weighted Kendall’s W can be calculated with the aid of Eq. (24)

Original Kendall’s W is a special case of Weighted Kendall’s W. When ϑ has a uniform distribution, the value of Kendall’s W and Weighted Kendall’s W are equal.

Theorem 3

Kendall’s W is a special case of Weighted Kendall’s W. The value of Kendall’s W is equal to Weighted Kendall’s W when the distribution of the weights is uniform.

Proof of theorem 3

Assume that there are K clusters (raters) in the problem. If we assume that the weights of the clusters are equal, the weight of each cluster is , where the summation of the weights is equal to 1. After replacing this value in Eq. (21), the revised formula of R is as follows:

Considering Eq. (22), the revised formula of is resulted as follows:

Later, the revised formula of S is obtained as follows using Eq. (23):

Based on Eq. (24), the revised formula of Weighted Kendall’s W is as follows:

As we can see, the above formula is equal to the original Kendall’s W. Therefore, Kendall’s W is equal to Weighted Kendall’s W when the distribution of the weights is uniform, and Theorem 3 is proved.

When there is a tie in the rank of the attributes, Eq. (24) cannot be used. Also, instead of tie ranks, the mean value of the positions should be used. In this situation, Eq. (25) should be utilized to determine the number of ties.

t represents the number of tie ranks in cluster i and attribute j. T implies the number of ties in cluster i. After that, the value of the corrected Weighted Kendall’s W should be determined using Eq. (26)

Theorem 4

Corrected Kendall’s W is a special case of corrected Weighted Kendall’s W. The value of corrected Kendall’s W is equal to corrected Weighted Kendall’s W when the distribution of the weights is uniform.

Proof of theorem 4

From Theorem 3, we are informed that,

Also, the weight of each cluster is . If we replace these values in Eq. (26), the revised formula of corrected Weighted Kendall’s W is as follows:

As we can see, the above formula is equal to the original corrected Kendall’s W. Therefore, corrected Kendall’s W is equal to corrected Weighted Kendall’s W when the distribution of the weights is uniform and Theorem 4 is proved.

Mahmoudi and Javed [41] proposed a confidence level index for the OPA model. However, they used the original Kendall’s W in their study based on the need. In this study, we extended their index by employing the proposed Weighted Kendall’s W to consider the weights of the cluster during confidence level measurement. For this purpose, the x statistic should be calculated by using Eq. (27) with the degrees of freedom v1 and v2.

Based on F-distribution, the value of the probability density function of x can be determined using Eq. (28) [42].

While gamma function (Γ) is defined in Eq. (29).

Finally, guided by the cumulative distribution function (CDF), the value of the confidence level should be determined by employing Eq. (30).

The value of the confidence level in 0.99, 0.95, 0.90 shows the significant test in levels 0.01, 0.05, 0.1 by F-distribution. Therefore, these points can be utilized as the threshold for the confidence level. Hence, the thresholds can be defined as in Fig. 1.

Fig. 1

The objective threshold for the confidence level [41]

The proposed framework

In this section, the steps of the proposed framework are explained. There are three main steps to calculate the performance of the healthcare construction projects, which are shown in Fig. 2. The first step focuses on the data collection and defining the problem, Step 2 tries to decrease the size of the problem and Step 3 measures the performance of the projects.

Fig. 2

The steps of the proposed approach

Step 1: First of all, the indicators for performance measurement must be defined. Selecting the indicators depends on several factors, including the availability of the data, feasibility, importance, etc. After that, we should collect the experts’ opinions associated with the indicators. The opinions should be in the form of ordinal data. On the other hand, the quantitative data related to the projects in the indicators should be collected, e.g., delay of the projects, cost of construction, etc.

Step 2: In this step, the complexity of the problem should be checked first. The complexity can be represented by the number of constraints or computational iteration formula in Eq. (9). The user ought to consider available resources (e.g., software, hardware, and effort of solving) before solving the problem. For example, if the user would like to use MS Excel software, he/she should be informed that the Excel solver cannot handle a large number of constraints. Hence, the acceptable range of complexity should be defined by the user. Next, a Silhouette plot should be provided using any software such as MATLAB within the acceptable range of complexity (See Eq. (20)). If there is any value higher than 0.5 for the Silhouette coefficient, we can select the number of clusters with the highest Silhouette coefficient value and run the ordinal K-means algorithm. Otherwise, a feedback mechanism should be applied, which includes various actions such as adding more experts, discussing with the experts to make their opinions more consistent, etc. Later, the value of the confidence level should be determined using Eqs. (21) to (30), and the obtained value must be compared with the thresholds based on the sensitivity level of the system. If it can meet the required threshold, we can go to the next step. Otherwise, the feedback mechanism must be applied, and Step 2 should be repeated.

Step 3: With making the final decision regarding the number of clusters, the conventional problem can be solved by employing Model (10). Next, the weights of the clusters and the weights of the indicators are obtained. Indeed, the LSGDM is employed for calculating the weight of the indicators for performance measurement. Finally, the relative project performance (RPP) can be determined using Eq. (31) for each project.

Where W is the weight of indicator j and is the normalized value of the quantitative data of project h in indicator j. When the indicator is benefit-type, Eq. (32) should be employed, and when the indicator is cost-type, Eq. (33) should be utilized to normalize the data in each indicator [43].

After determining the RPP of the healthcare construction projects, they can be compared, and the strengths and weaknesses of the projects in each indicator can be investigated.

Comparative analysis

Tang et al. [34] presented an illustrative example related to the public transportation system in China. They have provided the input data as Table 2, which are collected from 20 experts for 5 alternatives. In this section, the proposed approach is compared with the approach proposed by Tang et al. [34] to check the reliability and feasibility of both frameworks.

Table 2

Ordinal preference for the illustrative example

Ordinal preference	A1	A2	A3	A4	A5	Cluster No.
r(1)	2	3	4	5	1	3
r(2)	3	5	4	1	2	2
r(3)	2	3	4	5	1	3
r(4)	2	3	5	4	1	3
r(5)	1	3	5	2	4	2
r(6)	2	3	4	5	1	3
r(7)	3	4	2	5	1	3
r(8)	4	3	5	1	2	2
r(9)	1	3	4	5	2	3
r(10)	2	4	5	3	1	2
r(11)	4	2	3	5	1	1
r(12)	2	3	4	5	1	3
r(13)	3	2	4	5	1	3
r(14)	1	5	3	4	2	2
r(15)	5	1	3	4	2	1
r(16)	2	5	4	3	1	2
r(17)	4	3	2	5	1	1
r(18)	3	1	2	5	4	1
r(19)	1	5	3	2	4	2
r(20)	5	1	2	3	4	1

To illustrate the complexity of the problem, the value of the OPACI is determined using Eq. (9) and depicted in Fig. 3 for a various number of clusters. Here, we assume that considering the available software, hardware, and time, the acceptable range of complexity is between i = 2 to i = 6. It can be different for various users based on their situations.

Fig. 3

The complexity of the illustrative example

Therefore, we should draw the Silhouette plot for the aforementioned range of complexity. Using the MATLAB software, the Silhouette plot is illustrated in Fig. 4. As can be seen from Fig. 4, i = 3 reached the highest Silhouette coefficient value among others, with a value higher than 0.5. Hence, we select i = 3 and run the ordinal K-means.

Fig. 4

The Silhouette plot for the presented data in Table 2

After applying the ordinal K-means to the data in Table 2, the cluster numbers are shown in the last column of Table 2 and the cluster centers are depicted in Table 3.

Table 3

Cluster centers after applying ordinal K-means on the data in Table 2

Cluster No.	Cluster Centers
	A1	A2	A3	A4	A5
1	4	1	2	5	2
2	1	5	4	3	2
3	2	3	4	5	1

Since there is a tie rank in cluster number 1, we should replace the mean value instead of them (the mean value of the second and third positions is 2.5). Therefore, for cluster number 1, we have 4, 1, 2.5, 5, 2.5 respectively, to calculate the Weighted Kendall’s W. If we calculate the confidence level for the cluster centers in Table 3 using MATLAB software, its value is 0.7332, which is truly low and cannot meet the required threshold (CL = 0.90 equals a significant level of 0.1). Therefore, the feedback mechanism should be done to improve the consistency ratio of the input data. Also, Tang et al. [34] faced the same scenario, and they mentioned that the global consensus index value could not meet the threshold. After applying the feedback mechanism, Tang et al. [34] presented the data in Table 4.

Table 4

The input data after applying the feedback mechanism

Ordinal preference	A1	A2	A3	A4	A5	Cluster No.
r(1)	3	2	4	5	1	3
r(2)	3	4	5	1	2	2
r(3)	3	2	4	5	1	3
r(4)	3	2	5	4	1	3
r(5)	1	3	5	2	4	2
r(6)	3	2	4	5	1	3
r(7)	2	3	4	5	1	3
r(8)	3	4	5	2	1	2
r(9)	2	3	4	5	1	3
r(10)	2	3	5	4	1	3
r(11)	3	2	4	5	1	3
r(12)	3	2	4	5	1	3
r(13)	3	2	4	5	1	3
r(14)	2	3	4	5	1	3
r(15)	4	1	3	5	2	1
r(16)	3	2	4	5	1	3
r(17)	4	2	3	5	1	1
r(18)	3	2	1	5	4	1
r(19)	2	4	5	3	1	2
r(20)	4	2	3	5	2	1

Employing the MATLAB software, the Silhouette plot is provided in Fig. 5, which shows that the highest value of the Silhouette coefficient is associated with i = 3.

Fig. 5

The Silhouette plot for the presented data in Table 4

After applying the ordinal K-means to the data in Table 4, the cluster numbers are shown in the last column of Table 4 and the cluster centers are depicted in Table 5.

Table 5

Cluster centers after applying ordinal K-means on the data in Table 4

Cluster No.	Cluster Centers
	A1	A2	A3	A4	A5
1	4	1	3	5	2
2	3	4	5	1	1
3	3	2	4	5	1

If we calculate the confidence level for the cluster centers in Table 5 using MATLAB software, its value is 0.9182, which is suitable for sensitive problems and can meet the required threshold (CL = 0.90 equals the significant level of 0.1). It is worth mentioning that Tang et al. [34] calculated the consensus index in their study using their proposed index. They also confirmed that the data in Table 4 meets the threshold.

After that, we can solve the problem based on the data in Table 5 and Model (10) to calculate the weights of alternatives. The weights are resulted in Table 6 and compared with those which were obtained by Tang et al. [34].

Table 6

The comparative analysis

Alternatives	The current study		Tang et al. [34]
	Weight	Rank	Weight	Rank
A1	0.143	3	0.220	3
A2	0.263	2	0.255	2
A3	0.093	5	0.190	4
A4	0.103	4	0.085	5
A5	0.397	1	0.340	1

In order to compare the ranks in Table 6, the Spearman’s rank-order correlation is utilized, which is a non-parametric test, and its formula is presented in Eq. (34) [44, 45]. where d is the difference between two ranks, and n is the number of alternatives. Based on Table 6, the Spearman’s rank-order correlation is calculated using IBM SPSS software (version 26), and the results are presented in Table 7.

Table 7

The correlation between the results

			The current study	Tang et al. [34]
Spearman’s rho	The current study	Correlation Coefficient	1.000	.900*
		Sig. (2-tailed)	.	.037
		N	5	5
	Tang et al. [34]	Correlation Coefficient	.900*	1.000
		Sig. (2-tailed)	.037	.
		N	5	5

*. Correlation is significant at the 0.05 level (2-tailed)

As can be seen from Table 7, there is a strong correlation between the obtained ranks by the proposed approach and Tang et al. [34].

Case study

The policymakers in China look forward to finding a reasonable method for measuring the performance of the healthcare construction projects. It is very important to find out which projects are successful and identify the reasons to consider in future healthcare projects. As mentioned earlier, measuring the performance of healthcare projects is a challenging task, and it should be established based on all stakeholders’ opinions. Since there are a lot of stakeholders involved in each healthcare construction project, it is not easy to solve such large-scale problems. In the current study, five healthcare construction projects from China are selected for performance measurement and evaluation. The selected hospitals are located in Nanjing, Beijing, Guangzhou, Quzhou, and Dalian. The construction phase of these projects has been finished already.

The quantitative information associated with the projects in each indicator is presented in Table 8. Here, there are five indicators for project performance measurement, which can be more or less based on the need.

Table 8

The quantitative information regarding each project in each indicator for the case study

Projects	Location	Square footage (square meter)	Duration (day)	Construction cost at contract award (100 million yuan)	Number of beds	Outpatient capacity in each day
		A1	A2	A3	A4	A5
Hospital 1	Nanjing	224,800.00	1963	12.36	1600	10,000
Hospital 2	Beijing	216,000.00	1187	22.00	1200	7380
Hospital 3	Guangzhou	510,044.76	1038	38.00	1500	9000
Hospital 4	Quzhou	387,800.00	1098	28.97	2000	4000
Hospital 5	Dalian	203,100.00	730	20.00	3700	3447

In a survey, we collected the experts’ opinions related to performance indicators in Table 8. 43 experts participated in the survey. The experts’ opinions associated with the indicators are presented in Table 9.

Table 9

The experts’ opinions regarding the indicators for the case study

Expert No.	A1	A2	A3	A4	A5	i = 9
1	3	4	1	2	5	2
2	4	1	2	3	5	9
3	5	1	3	2	4	3
4	5	3	2	4	1	1
5	3	4	2	1	5	6
6	5	2	1	4	3	4
7	4	3	1	2	5	8
8	3	4	1	2	5	2
9	5	3	2	4	1	1
10	3	4	2	1	5	6
11	4	3	1	2	5	8
12	3	4	2	1	5	6
13	2	3	5	4	1	7
14	5	1	3	2	4	3
15	3	4	1	2	5	2
16	4	3	1	2	5	8
17	2	3	5	4	1	7
18	4	1	2	3	5	9
19	2	3	5	4	1	7
20	3	4	1	2	5	2
21	3	4	1	2	5	2
22	3	4	2	1	5	6
23	5	1	3	2	4	3
24	5	3	2	4	1	1
25	5	2	1	4	3	4
26	5	3	1	2	4	5
27	3	4	1	2	5	2
28	2	3	5	4	1	7
29	3	4	2	1	5	6
30	3	4	2	1	5	6
31	5	1	3	2	4	3
32	3	4	1	2	5	2
33	2	3	5	4	1	7
34	4	1	2	3	5	9
35	4	3	1	2	5	8
36	4	3	1	2	5	8
37	5	3	1	2	4	5
38	3	4	2	1	5	6
39	5	3	2	4	1	1
40	3	4	2	1	5	6
41	5	1	3	2	4	3
42	4	3	1	2	5	8
43	3	4	2	1	5	6

The value of OPACI for the various number of clusters is illustrated in Fig. 6. As shown in Fig. 6, the problem for i = 43 requires lots of iterations to be solved. Considering the available software and hardware, the acceptable range of complexity is between i = 2 to i = 9 for this case study.

Fig. 6

The complexity of the problem for various cluster numbers in the case study

Then, the Silhouette plot is provided for the acceptable complexity range which is illustrated in Fig. 7. As can be seen from Fig. 7, the highest value of the Silhouette coefficient is associated with i = 9. Hence, it can be selected as the optimal number of the clusters.

Fig. 7

The Silhouette plot for the case study

We applied the ordinal K-means to the data in Table 9, the cluster numbers are shown in the last column of Table 9. The centers of the clusters for i = 9 are illustrated in Table 10. Moreover, the number of experts in each cluster is provided, which is necessary to solve the problem.

Table 10

Cluster center for various cluster numbers

Number of clusters	Cluster	Number of experts in the cluster	Cluster Center
			A1	A2	A3	A4	A5
i = 9	1	4	5	3	2	4	1
	2	7	3	4	1	2	5
	3	5	5	1	3	2	4
	4	2	5	2	1	4	3
	5	2	5	3	1	2	4
	6	9	3	4	2	1	5
	7	5	2	3	5	4	1
	8	6	4	3	1	2	5
	9	3	4	1	2	3	5

Here, we calculated the confidence level for that data in Table 10 employing Eqs. (21) to (30). The confidence level is reported as 0.9523. Based on the thresholds in Fig. 1, the value of the confidence level is suitable for very sensitive problems. Hence, we do not need to apply a feedback mechanism and can go to the next step. Considering the data in Table 10 and Model (10), the weight of the attributes is obtained in Table 11.

Table 11

The weight and rank of the indicators through Model (10)

Indicators	Weight	Rank
A1	0.11907	5
A2	0.19233	3
A3	0.29892	1
A4	0.24892	2
A5	0.14078	4

Finally, using Tables 8 and 11, the relative performance of the hospital projects has resulted in Table 12.

Table 12

The relative performance of the hospital projects

Projects	Relative project performance	Rank
Hospital 1	0.48793	2
Hospital 2	0.39707	5
Hospital 3	0.41252	3
Hospital 4	0.40338	4
Hospital 5	0.65109	1

Based on Table 12, the project of Hospital 5 achieved the highest relative performance with a value of 0.65109. After that, Hospital 1 achieved the second position. These projects can be a perfect benchmark for implementing future projects, and the reason for their suitable performance can be extracted and considered as a lesson learned.

Sensitivity analysis

For more investigation, we have provided a comprehensive sensitivity analysis of the case study. First, the confidence level is calculated for a various number of clusters. As can be seen from Fig. 8, the confidence level is experiencing an increasing trend. There are two points which met the confidence level threshold, including i = 8 and i = 9. This graph shows that i = 9 is the best one to be selected as the optimal number of clusters with the highest confidence level value.

Fig. 8

The sensitivity of the confidence level for the case study

Also, we have calculated the weights of the attributes for various number of clusters to check the sensitivity of the weights. The results are shown in Table 13 in detail and compared in Fig. 9. As can be seen from Fig. 9, the error for i = 2, i = 3, and i = 4 is significant, which shows that they are not appropriate for selecting as the optimal number of clusters. On the other hand, the weights of the attributes for i = 9 and i = 43 are equivalent. The main reason is Silhouette coefficient value equals 1, which is shown in Fig. 7. It shows that the data are clustered perfectly, and the quality of clustering is significantly high. However, the error among i = 6, i = 7, i = 8, i = 9 is not significant, which shows that all of them are correlated.

Table 13

The sensitivity of the indicators’ weights for the case study

Indicators	Weights
	i = 2	i = 3	i = 4	i = 5	i = 6	i = 7	i = 8	i = 9	i = 43
A1	0.1040	0.1249	0.1060	0.1053	0.1158	0.1179	0.1156	0.1191	0.1191
A2	0.1776	0.2342	0.2038	0.2008	0.1903	0.1923	0.1923	0.1923	0.1923
A3	0.3799	0.3338	0.2716	0.2960	0.2960	0.2919	0.2919	0.2989	0.2989
A4	0.2113	0.2001	0.2798	0.2520	0.2520	0.2559	0.2559	0.2489	0.2489
A5	0.1272	0.1071	0.1388	0.1459	0.1459	0.1419	0.1443	0.1408	0.1408

Fig. 9

The sensitivity of the indicators’ weights for the case study

The relative performance of the hospitals is calculated for a various number of clusters which are shown in Table 14 and Fig. 10. As we expected, the relative performance of the hospitals is precisely the same for i = 9 and i = 43. Also, Hospital 5 is the best option in all conditions with a slight change for the various number of clusters.

Table 14

The relative performance of the hospital projects for various cluster numbers

Hospitals	Relative performance
	i = 2	i = 3	i = 4	i = 5	i = 6	i = 7	i = 8	i = 9	i = 43
Hospital 1	0.5483	0.4817	0.4626	0.4897	0.4904	0.4832	0.4853	0.4879	0.4879
Hospital 2	0.4296	0.4252	0.3855	0.4031	0.3969	0.3934	0.3947	0.3971	0.3971
Hospital 3	0.3703	0.4153	0.4101	0.4098	0.4124	0.4132	0.4128	0.4125	0.4125
Hospital 4	0.3993	0.4301	0.4037	0.4014	0.4004	0.4026	0.4013	0.4034	0.4034
Hospital 5	0.6556	0.6686	0.6743	0.6606	0.6501	0.6532	0.6532	0.6511	0.6511

Fig. 10

The relative performance of the hospital projects for various cluster numbers

Based on the aforementioned sensitivity analysis, the proposed framework works logically and can decrease the volume of the calculations efficiently. Indeed, we solved a small-scale problem (i = 9) instead of a large-scale problem (i = 43) while the results are entirely the same in both conditions. Thus, one can argue that the proposed approach can be used confidently for LSGDM problems while saving resources (efforts, time, etc.).

Discussion and conclusion

Performance measurement of healthcare construction projects plays a vital role in the success of future healthcare projects. The weaknesses and strengths should be identified and considered as a lesson learned for future projects. To determine the performance of the construction healthcare projects, there is a need to define several performance indicators. The degree of importance of the indicators should be specified by a large number of stakeholders, which can lead to more reasonable results, and finally, with the aid of the quantitative data regarding each project in each indicator, the performance of the projects can be determined. However, considering the opinions of a large number of stakeholders is not an easy job, and it requires a significant volume of calculations. To overcome this obstruction, a novel framework was proposed that works using preference data, and it was compared with the system proposed by Tang et al. [34].

Based on the comparative analysis, one can see several benefits in the proposed framework. The first one is related to the computational complexity during the selection of the optimal number of clusters. Indeed, one of the inputs for the illustration of the Silhouette plot is the range of the possible number of clusters. This study considers this matter before drawing the Silhouette plot, which was ignored in the study of Tang et al. [34]. The second benefit of the proposed framework is providing a confidence level index which considers various objective threshold levels based on the sensitivity of the problem. It will increase the flexibility of the proposed framework in the real-world situations, which was ignored by other scholars. For example, Tang et al. [34] offered one type of threshold for any problem. The third benefit of the proposed framework is decreasing the cost of calculation by running the feedback mechanism as earlier as possible. In other words, it is not essential to solve the problem till the end and obtain the final rank to understand whether there is a need for a feedback mechanism. There are various layers in the proposed framework to check the need for the feedback mechanism. It can decrease unnecessary computations and optimize the algorithm.

This study can be extended to solve LSGDM problems, including experts, attributes, and alternatives simultaneously using the OPA in the future. Indeed, the current framework can determine only the weights of the attributes in LSGDM problems. Moreover, metaheuristic algorithms such as particle swarm optimization (PSO) can be used to extract the optimal number of clusters that may lead to more accurate results. The current study can be extended to handle the uncertainty in preference data, such as incomplete input data, because the experts may not be sure about their opinions in real-world situations.