Norm-dist Monte-Carlo integrative method for the improvement of fuzzy analytic hierarchy process.

This paper presents the novel approach of the Norm-dist Monte-Carlo fuzzy analytic hierarchy process (NMCFAHP) to incorporate probabilistic and epistemic uncertainty due to human's judgment vagueness in multi-criteria decision analysis. Normal distribution is applied as the most appropriate distribution model to approximate the probability distribution function of the criteria and alternatives within Monte-Carlo simulation. To test the applicability of the proposed NMCFAHP, the case study of non-destructive test (NDT) technology selection is performed in the Petroleum Company in Borneo, Indonesia. When compared with the conventional triangular fuzzy-AHP, the proposed NMCFAHP method reduces the standard error of mean values by 90.4-99.8% at the final evaluation scores. This means that the proposed NMCFAHP significantly involves fewer errors when dealing with fuzzy uncertainty and stochastic randomness. The proposed NMCFAHP delivers reliable performance to overcome probabilistic uncertainty and epistemic vagueness in the group decision making process.

Introduction

Analytic hierarchy process is one of the most popular methodologies of multi-criteria decision making (MCDM) to evaluate both criteria and alternatives' degree of importance by interpreting experts' judgment. The research performed by various studies concludes that AHP applications are able to describe the weight factor of criteria and alternatives [1,3]. However, according to those researches, AHP has limitations in measuring vagueness and uncertainty that existed in the pairwise comparison. The potential problems often linked to AHP include experts' judgment may not always yield consistent result [3], and it is unable to incorporate fuzziness and uncertainty [4]. In order to overcome these limitations, adoption of integrative probabilistic method and fuzzy logic in multi-criteria decision making is capable to evaluate complex vagueness and uncertainty in the pairwise comparison of the AHP [5,6]. Implementations of fuzzy logic in multi-criteria decision making and evaluation of risk assessment have been widely used in many recent applications and studies [7,8,9]. The integrated analytic hierarchy process-fuzzy approach is the most popular methodology in the past ten years [10]. Fuzzy logic allows multiple stakeholders to participate in the process of decision-making for evaluating complex technological problems [8]. Fuzzy logic is capable of being integrated with other methodologies, such as scoring system [11], and other multi-criteria decision analysis, such as TOPSIS (technique for order of preference by similarity to ideal solution) and VIKOR (multi-criteria optimization and compromise solution) [12,13]. Jing et al., [14] implement a hybrid stochastic analytic hierarchy process for evaluating ballast water treatment technologies where environmental decision can be critical due to the inherent trade-off among social, ecological, and economic factors. Lavasani et al., [15] perform analysis of multi-attributes decision making in fuzzy environments for selecting the best barrier for offshore wells. The research employs a fuzzy decision matrix by considering evaluation attributes, and then calculates the weight factor of all risk control options. Eleye-Datubo et al., [16] present incorporative risk modeling of fuzzy and Bayesian network for evaluating marine and offshore safety assessment. The implementation of Fuzzy-Bayesian network methodology is proposed to determine human relative performance in maritime operation performance shaping factors.

In spite of several researches that demonstrated the use of integrative fuzzy logic in multi-criteria decision making, none of these researches actually presents analysis on a statistical approach, especially when the input data is gathered in a Normal distribution model. In addition, from the past years' researches, there has been no reference that specifically mentions the usage of a Normal distribution as an integrative method with fuzzy AHP to address both probabilistic uncertainty and epistemic uncertainty due to human's judgment vagueness. In fact, the pairwise comparison data forms as a Normal distribution which can be validated and evaluated using the Kolmogorov-Smirnov test.

This paper proposes the development of a Norm-dist Monte-Carlo fuzzy AHP (NMCFAHP) to incorporate probabilistic uncertainty which cannot be addressed by conventional fuzzy AHP. This paper integrates a Normal distribution fuzzy number (Norm-dist FN) to represent the epistemic uncertainty in a fuzzy logic system and Monte-Carlo simulation to quantify the probabilistic uncertainty by synthesizing random numbers in the pairwise comparison. The Normal distribution is applied as the most appropriate distribution model to approximate the probability distribution function of the alternatives' degree of importance. The Normal distribution involved in this paper is able to represent realistic human judgment over criteria and alternatives. The implementation of the proposed methodology is performed to evaluate the most optimum non-destructive test (NDT) technology for addressing cracks on piping and vessels. The result of this paper is then compared with conventional triangular fuzzy analytic hierarchy process to measure the performance and accuracy of the proposed NMCFAHP for evaluating the decision-making process.

Theoretical background

AHP is a multi-criteria decision approach which can evaluate both qualitative and quantitative criteria by providing mathematical reasoning behind the judgment [2]. For the last 20 years, AHP has been extensively studied due to its flexibility, wide applicability, and capability to be integrated with other MCDM approaches [10]. AHP was first developed by Saaty during his assignment in Wharton School (University of Pennsylvania 1971–1975) [17]. The remarkable study of AHP is performed by Saaty et al. [18], to investigate a structured scientific solution to the Israeli-Palestinian conflict. Based on the study, AHP works by: 1. decomposing a complex problem into a structured hierarchy; 2. use a measurement methodology to establish priorities among the elements for ranking the alternatives; 3. develop a series of pairwise comparison matrices based on the hierarchy structure. In particular cases, the decision-making process may possess criteria that are opposite in direction to other criteria, such as benefits versus costs and opportunities versus risks. To resolve this issue, Saaty and Ozdemir [19] present negative priorities number in the AHP evaluation. It is inferred that positive or negative priorities do not need to have a symmetric opposite value as the opposite criterion inexistence in practice. Saaty [20] also proves that AHP is capable of developing dynamic priorities in the pairwise comparison when the decision is more likely or more preferred over different time periods. This approach is well known as a theory for dynamic decision making.

Epistemic uncertainty in analytic hierarchy process

Epistemic uncertainty is defined as the uncertainty which comes from lack of knowledge, incomplete information, inadequate process understanding, or imprecise evaluation of the related characteristics [21]. Epistemic uncertainty may come from the process of underlying fundamental or total ignorance of influencing parameter. It may also arise from incompleteness data, simplification in modeling, or confusion in decision making [22]. Galvez et al. [23], state that the lack of knowledge in epistemic uncertainty is generated either due to the exact value of some criteria being unknown, the model is unable to appropriately represent the realistic judgment, or for both reasons. Several studies disclose the extent usage of fuzzy logic system to resolve the epistemic uncertainty. Rohmer and Baudrit [24] develop the scenario-based earthquake risk assessment by using the concept of fuzzy random variables based on the fuzzy logic methodology introduced by Zadeh [25]. Purba et al., [26] perform an investigation to quantify epistemic uncertainty by introducing fuzzy probability fault tree analysis. They use a fuzzy methodology principle instead of the Monte-Carlo simulation as it is more appropriate to quantify fuzzy probability based on a fuzzy logic system rather than the probabilistic Monte-Carlo method. The extent application of dual interval-and-fuzzy analysis method is introduced by Wang et al. [27], to investigate dual epistemic uncertainties in the thermal engineering process. Wang and Matthies [28] also investigate the safety assessment for engineering systems with hybrid epistemic uncertainties by integrating evidence variable and fuzzy variable system evaluation. In the field of multi-criteria decision making, epistemic uncertainty is marked with the extent usage of fuzzy logic system in the analytic hierarchy process. The most well-known fuzzy AHP approaches application is based on the methodology explained in chapter 2.1.1 until 2.1.4.

Laarhoven and Pedrycz's logarithmic least square

Fuzzy AHP methodology is first introduced by Laarhoven and Pedrycz [29] by utilizing a comparison of fuzzy ratio by triangular membership function. The detailed steps below explain how to conduct the evaluation.where are fuzzy pairwise comparison ratios based on decision maker judgment.where and are the lower and upper values of ln = - ln , the following equation then solved truly.

Develop pairwise comparison matrices based on triangular fuzzy number and obtain the n+ 1 fuzzy reciprocal matrix using Eq. (2.1).

Solve mathematical equation using Eqs. (2.2), (2.3), and (2.4) to disclose the value of ,

The solution for Eqs. (2.2), (2.3), and (2.4) is typically solved in Eq. (2.6),where p and p are chosen arbitrarily.where

Determine the fuzzy weight by generating logarithmic operations.

Eq. (2.7) is also known as performance score of .

repeat steps 1–3 so that all reciprocal matrices are solved. Therefore, we can obtain the fuzzy weight and performance score for alternative as written in Eq. (2.8).

Buckley's FAHP methodology

Buckley's FAHP methodology [30] is also known as fuzzy geometric mean FAHP. The name is derived due to the usage of a geometric mean to calculate the fuzzy weights for each fuzzy matrix. This methodology comes as a simple and efficient approach in FAHP evaluation. The following steps are written to explain how Buckley's FAHP conduct an evaluation.

Construct fuzzy pairwise comparison matrices based on trapezoidal fuzzy number whose elements are consisted of Eq. (2.9).

Calculate the fuzzy weight value by applying geometric mean operation for each row using Eq. (2.10).

The fuzzy weight is obtained by solving Eq. (2.11) as a fuzzy hierarchical sequencing operation.

Kahraman [50] states that the derivation of values are expressed as the left leg and right leg of . It is respectively defined in Eqs. (2.12) and Eq. (2.13),whereand

The computations for determining the values of b and b, c and c, and d and d is applied in Eq. (2.14).

Table 1 elaborates the value of membership function , and suppose x is a real number on a horizontal axis,where x or x , the value of x is defined in Eq. (2.15),and the value of are obtained in Eq. (2.16).

Table 1

Determination value of membership function

x	δ_wi(x)
≤(a_i/d)	0
≥(a_i/d)	0
[b_ic,c_ib]	1
[a_id,b_ic]	γ∈[0.1]
[c_ib,d_ia]	γ∈[0.1]

Repeat the calculations of Step 2 for all the fuzzy performance scores

Calculate the fuzzy weights and fuzzy final scores U by applying Eq. (2.17).

Cheng's entropy-based FAHP

Entropy-based FAHP is firstly developed by Cheng [32] to evaluate the naval tactical missile system based on the grade value of membership function. The following steps describe how the Entropy-based FAHP is conducted.

Develop pairwise comparison matrices based on the analytical hierarchy structure. The symmetric triangular membership function is used to demonstrate the relative strength of the fuzzy matrices' elements.

- Determine the fuzzy judgment matrices by implementing multiple operations of fuzzy subjective weight vector and the associating column of fuzzy judgment matrix using Eq. (2.18).

Eq. (2.19) is obtained by implementing interval arithmetic and values of α-cut.where

for and all i, j

Determine the value of performance of the judgment matrix , and estimate the index of optimism λ by solving Eq. (2.20).

The exact judgment matrix is obtained in Eq. (2.21).

The relative frequency is used to quantify the entropy of fuzzy pairwise comparison matrices in Eq. (2.22),whereAnd the entropy values are obtained by solving Eq. (2.23).

Therefore, the final entropy weight of the fuzzy AHP is formulated in Eq. (2.24).

Normal distribution fuzzy number

In this paper, the membership function of fuzzy set is represented by a bell-shaped curve, known either as a Normal distribution fuzzy number (Norm-dist FN) or a Gaussian fuzzy number. The Norm-dist FN is used to represent firmly epistemic uncertainty in fuzzy environment during decision making process. The membership function of Norm-dist fuzzy number is defined in Eq. (2.25).

The proposed methodology in this research compares the Norm-dist FN and triangular fuzzy number (TFN) with Eqs. (2.26), (2.27), and (2.28).

The description of value is explained in Figure 1.

Figure 1

Comparison of Norm-dist fuzzy number by triangular fuzzy number.

The membership function of Norm-dist FN μ will reach asymptote at the value of y=0. For the small value of α, it will be approximated for the Norm-dist membership function to the triangular function T(x: x, x). The definition of Norm-dist FN as conversion form TFN is explained by Eqs. (2.29) and Eq. (2.30) [33,34].

Suppose that T is the triangular fuzzy numbers, and G is the element of the preference matrix after performing a triangular approximation.where

To obtain a representative triangular approximation, the value of α is set as 0.01. This means that 99% of values are approximately represented by the Normal distribution function.whereand

Then, T can be converted into asymmetric Norm-dist fuzzy number as stated in Eqs. (2.31) and Eq. (2.32).where expressed the left deviation band of Norm-dist FN and illustrated the right deviation band of Norm-dist FN.

Here, T becomes the membership function of asymmetric Norm-dist FN as stated in Eq. (2.33).

Suppose that there are two Norm-dist FNs, i.e. and .

The intersection point between and is written as illustrated on Figure 2 and Eqs. (2.33), (2.34), and (2.35).

Figure 2

The intersection point between two Norm-dist functions (X, v).

Probabilistic uncertainty in analytic hierarchy process

Probabilistic or aleatory uncertainty is defined as the uncertainty which occurs due to random fluctuations of properties or condition leading to variability in outcomes [22]. Probabilistic uncertainty is often linked to the statistical process of complex variability. Probabilistic uncertainty refers to uncertainty caused by stochastic variation in a random event [21]. In the field of MCDM, probabilistic uncertainty is marked by several studies performed in various applications. Stam and Silva [35] present a stochastic approach in AHP methodology where the pairwise preference judgments are uncertain by developing multivariate statistical techniques. Wu et al. [36], administer the uncertainty in multi-attributes decision making by proposing an interval number with a probability distribution (INPD). This novel approach provides a uniform form for interval numbers and random numbers. Jalao et al. [37], propose a beta distribution to model the varying stochastic preference or judgment resulted from imprecise pairwise comparisons.

In several multi-criteria decision-making problems, both epistemic and probabilistic may occur due to complex fuzzy environments and statistical process. Researches to address both kinds of uncertainties have been demonstrated by adopting extent AHP approaches. Antucheviciene et al. [38], identify decision making method difficulties emerging from uncertainty quantification by means of fuzzy logic and probabilistic modeling. Wang et al. [39], integrates fuzzy logarithmic least square method (fuzzy LLSM) with fuzzy comprehensive evaluation (FCE), and employs the Monte-Carlo method to characterize random variables in judgment. Emec and Akkaya [40] proposed an integrative approach of stochastic MCDM by combining stochastic AHP and fuzzy VIKOR for warehouse location evaluation. Promentilla et al. [41], propose stochastic and fuzzy-based AHP approaches to address complexity and uncertainty resulted from conflicting multiple criteria in the clean technology selection. Monte-Carlo simulation is performed to model the uncertainty and probability distribution of the priority weights needed for ranking. Erdogan and Kaya [42] address two types of uncertainties in their study. Type-2 fuzzy AHP is used to determine the weights of the criteria in epistemic uncertainty and stochastic TOPSIS is applied to quantify probabilistic uncertainty for obtaining alternatives ranking. From the above researches, it is inferred that epistemic and probabilistic uncertainty shall be resolved by adopting the correct MCDM methodology.

Monte-Carlo simulation

The probabilistic theory has been used for many years to describe random variable and uncertain phenomenon. The Monte-Carlo simulation is used to address the probabilistic theory based on statistical information and is considered as the realistic form involves random sampling from a probability distribution (e.g., uniform, normal, beta, and lognormal), and it has been used to administer systems which are too complex to be solved analytically [43]. Principally, the Monte-Carlo simulation is where non-deterministic methods are employed to determine approximate solutions for complex systems which are beyond the resources of theoretical mathematics by experimenting with random numbers [44]. Sari [45] proposes a methodology to select an RFID solution provider by integrating a fuzzy multi-criteria decision model with a Monte-Carlo simulation based on a triangular fuzzy number. Negahban [46] implements a Monte-Carlo analytic hierarchy process to investigate the optimization of consistency improvement of positive reciprocal matrix by transforming reciprocal judgment matrices into near-consistent matrices, and further develops a sampling-optimizing-adjustment approach integrated into Monte-Carlo AHP framework. The aim of the research is to generate a distribution that more closely resembles a realistic probability distribution.

The Monte-Carlo simulation works by performing random sampling from the distribution of an uncertainty input [47]. The probability distribution function F(x) range from 0 to 1 to describe the probability P that the variable X will be less than or equal to x.

The inverse function of F(x) is namely G(x), where

The random samples can be generated by plotting inputs to the inverse function. Figure 3 explains the relationship between F(x) and G(F(x)). The random r is generated from the probability distribution function approximate to . A random sample for the probability distribution function, input r is entered with value between 0 and 1 to the distribution, firmly as

Figure 3

Graphical explanation relationship between F(x) and G(F(x)).

Research framework

This paper proposes the Norm-dist Monte-Carlo fuzzy AHP, a novel methodology which can compensate probabilistic uncertainty, human's thought of ambiguity during investigation and judgments, and risk of incomplete information or scattered data. The section of the research proceeds as follows. The first part, NMCFAHP methodology, is explained as research background. The second part elaborates the application steps for constructing the NMCFAHP methodology. Finally, the fourth part presents applications in the evaluation of technology and comparing with conventional fuzzy AHP. The structure of the NMCFAHP methodology is described in several phases (Figure 4.).

Figure 4

The research framework deployed in this study.

Norm-dist Monte-Carlo integrative method in FAHP

The proposed methodology employs two methods of validity test. Firstly, the Kolmogorov-Smirnov test is conducted to evaluate the judgments data normality and, secondly, a pairwise comparison inconsistency test is performed to measure evaluative matrices inconsistency. The detailed procedure to perform evaluation with Monte-Carlo fuzzy AHP is described as follow:where and s are the mean and standard deviation of the Normal distribution; (a), and (c) indicates the most probable, lower value, and upper value of the Normal distribution, respectively as stated in Eqs. (4.4), (4.5), and (4.6). These values are obtained based on confidence interval of mean 95%; n is the quantity of data used (e.g., 1,000). Figure 5 illustrates detail properties of Normal distribution curves with a confidence interval of 95% (1-α = 5%). The values of is determined based on a standard Normal distribution table, for (1-α = 5%), the value is 1.96.where CDF is the cumulative distribution function of Normal distribution. The random Monte-Carlo number can be generated by plotting inputs (X) varying from 0 to 1.

Figure 5

The Normal distribution curve and properties.

develop a hierarchical decision structure for the concerning problems. Complete the structure with criteria, sub-criteria, and alternatives. As specified in this paper, the goal is to determine the most important factor for the criteria and alternatives. The decision goal, criteria, sub-criteria, and alternatives' attributes shall be developed based on collaborative discussion.

collect the degree of importance for each alternative and criterion, respectively, in accordance with experts' judgment. This judgment can be developed through valid questionnaires, surveys, or direct observations. Develop initial judgment using Saaty's scale, similar to triangular AHP

list each criterion and alternative judgment and sort the data in a spreadsheet which can be evaluated by a Normal distribution. Determine the lower (a), most probable (b), and upper (c) values of the Normal distribution curve. Eq. (4.1) demonstrates the probability distribution function for the Normal distribution curve, and Eqs. (4.2) and (4.3) are used to determine independent Normal distribution properties.

generate a Norm-dist fuzzy number by applying the Monte-Carlo simulation of a Normal distribution by generating a random variable for , , and . This paper employs 1,000 random normal variables. The Monte-Carlo simulation is obtained by entering a spreadsheet formula, i.e. "". Eqs. (4.7), (4.8), (4.9), and (4.10) describe how random number can be generated.

Generate random variable for i = 1 to 1,000 times, and store the result as the random variable column. Then, determine the Monte-Carlo Normal distribution mean and standard deviation to state the value of random, random, and random using Eqs. (4.11), (4.12), and (4.13).where and σ are the mean and the standard deviation of the Monte-Carlo Normal distribution, and random , random , and random is the lower, most probable, and upper value. The values of reciprocal fuzzy sets are applicable.where is the superlative difference between CDF and EDF for the statistical data; is the cumulative distribution function (CDF), and is the empirical distribution function (EDF) of the statistical data formed as a Normal distribution. To simplify the calculation, this paper utilizes IBM SPSS software to perform the Kolmogorov-Smirnov test. Table 2 demonstrates how SPSS analyze the normality test.where

Table 2

Result of Kolmogorov-Smirnov test of normality. The ∗mark indicates the statistically significant Normal distribution (significant number > than 0.05.

	Kolmogorov-Smirnova			Saphiro-Wilk
	Statistic	df	Sig.	Statistic	df	Sig.
VAR00001	0.021	1000	0.200∗	0.999	1000	0.773

a. Lilliefors Significance Correction.

This is a lower bound of the true significance.

perform a normality test by using Kolmogorov-Smirnov (KS test) to ensure the normality of statistical data. Principally, the KS test measures the differences between cumulative distribution function (CDF) of the reference and empirical distribution function (EDF) of the statistical data. The KS test significantly indicates normality when the superlative differences of is smaller than the value in the Kolmogorov-Smirnov table. When this value becomes larger, then the data is not significantly Normal distributed. Mathematical equations of KS test are described in Eqs. (4.14), (4.15), and (4.16).

H0: P-value > 0.05, it means that the data forms as Normal distribution, and

H1: P-value 0.05 data does not form as Normal distribution.

From Table 2., we obtain the statistic value of Kolmogorov-Smirnov = 0.021 and the significance value, or P-value, = 0.200 > 0.05. By these terms, we conclude that the data are normally distributed. In addition, we can see the statistic value of Saphiro-Wilk = 0.999 and significance value = 0.773 > 0.05 which arrives to the same conclusion that the data forms as a Normal distribution.

construct fuzzy pairwise comparison (Eq. (4.17)) matrices for all criteria and alternatives in the hierarchical level. Perform computation based on the developed fuzzy judgment matrix based on the random lower value, most probable value, and upper value (random, random, random).

is named as non-diagonal fuzzy element. It indicates the pairwise comparison of one alternative or criterion to another. The index m indicates the size of pairwise comparison matrices which is equal to the number of criteria and alternatives. Let the random Normal distribution value for A1 and A2 be (a, b, c) and (a, b, c), respectively, then the value of and 1/are obtained in Eqs. (4.18) and (4.19).

perform normalization for each element by the sum of every matrix's column using Eq. (4.20).

Apply geometric mean using Eq. (4.21) to calculate the fuzzy weight for each pairwise comparison matrix for each criterion and alternative [48].where , , are the lower value, most probable value, and the upper value properties of the fuzzy pairwise comparison matrices ; the index m indicates the size of pairwise comparison matrices or the order of the pairwise comparison matrices; , , and are the geometric mean for each lower value, most probable value, and upper value of the fuzzy pairwise comparison matrices at the i-th row; are the fuzzy weight value of the i-th alternatives over the j-th criterion.where, , and are the lower value, most probable value, and upper value of the concerning fuzzy pairwise comparison matrices, respectively; , , are the sum of ; is the pairwise comparison scale (i.e., 1/9, 9, then pairwise comparison scale is 9); is the normality constant; and is the consistency index of the fuzzy pairwise comparison matrices. The pairwise comparison is considered as a consistent result when the value of relies amongst 0 to 0.1. The closer the value of to 0, the more consistent the result is.where are the fuzzy weight values of the j-th criteria to the i-th alternatives and are the fuzzy weight value for each criterion j-th irrespective to the goal.where a, b, c are the random Normal distribution value of lower, most probable, and upper of A. and are the randomized mean and standard deviation value, is typically the value of (1-α = 5%), 1.96, n is the number of data, i.e. 1000.where are the crisp value of the fuzzy weight value for the i-th alternative, and can be written as an integration of the Normal distribution PDF; are normalized crisp value; and it is considered as final weight value; are the probability density function of the and; are the lower and upper limit value of .

perform consistency analysis for each pairwise comparison matrices. As in triangular AHP consistency analysis should be performed to ensure that fuzzy pairwise comparison is valid for the evaluation. Once it is not consistent, then the concerning fuzzy pairwise comparisons need to be revised. As a fuzzy number is present in the calculation, then triangular AHP consistency analysis cannot be performed as it is considered ineffective to address uncertainty. Ramik and Korviny [49] propose a new methodology to measure the inconsistency of a pairwise comparison matrix with fuzzy elements by utilizing a new consistency index (KI). The calculation of consistency analysis is stated in Eqs. (4.26), (4.27), and (4.28).

Calculate the fuzzy final value by calculating hierarchical layer sequencing [31]. The overall fuzzy final values for each alternative can be calculated in Eq. (4.29).

Generate random variable by using the Normal distribution model following the values of and . All calculation results are then plotted as a probability distribution function. The values of and are obtained by concerning the values of random Normal distribution for Ai, namely (a, b, c). The values are stated in Eqs. (4.30) and (4.31).

Perform defuzzification by implementing the center of gravity method (COG), and rank all the alternatives based on the normalized crisp overall value . The center of gravity method is written as Eqs. (4.32) and Eq. (4.33).

Research application

The applicability of the proposed NMCFAHP is tested based on the case study in the selection of appropriate technology for addressing piping and vessels cracks using non-destructive testing (NDT). The concerning NDT technology are (1) Magnetic particle test; (2) Dye penetrant test; (3) Radiography; (4) Eddy-current test. This case study is performed to demonstrate the capability and reliability to evaluate probabilistic and fuzzy uncertainty in the decision making. An oil and gas processing facility, in Borneo, Indonesia, has been operating for more than four decades. Due to its aging facilities, a special inspection strategy must be applied to prolong the operational productivity. This case study demonstrates the most optimal technology in providing appropriate piping and vessels NDT for the Petroleum Company. The hierarchical structure of the decision making is illustrated in Figure 6.

Figure 6

Hierarchical structure of non-destructive test technology evaluation.

Data and judgments acquisition

The qualitative judgments are performed by developing questionnaires answered by the Petroleum Company employees and experts, with the process is carried out during November–December 2018. An explanation of recent NDT technology and working principles are delivered and prepared as the option for the panelists. Ten expert participants are selected in accordance with their métier and working scope, and mainly work in the Field Operation division within the Petroleum Company as demonstrated in Figure 7. A majority of engineer panelists are selected as they are the true front-liner to perform calculations on safety engineering factors. Some managerial positions are also selected, such as head department of production support and head of field operation safety and method services. This approach is taken as they are key personnel in the decision making process.

Figure 7

Working position of expert panelists involved in the judgments.

The questionnaires are formulated as open-ended that describe how important a criterion is compared to another criterion. The questionnaire data is obtained by providing a pairwise comparison of criteria for evaluating the NDT technology to address cracks in piping and vessels. For example, the questionnaire mentions: how important is the criterion "reliability and precision" as compared with the criterion "detection coverage area" for non-destructive test technology. The panelists can directly answer the questionnaire and list the answers for all criteria judgments. A summary of the questionnaire results are described in Table 3. The original result of criteria weighting judgment by the expert panelists is available in Appendix 1.

Table 3

The result of criteria judgments.

Criteria Evaluation	Mean (x)	Standard Deviation (s)
C1 vs C2	1.8000	1.0328
C1 vs C3	3.4000	0.8433
C1 vs C4	4.6000	0.8433
C1 vs C5	8.4000	0.9661
C2 vs C3	3.2000	0.6325
C2 vs C4	3.2000	0.6325
C2 vs C5	6.8000	0.6325
C3 vs C4	2.2000	1.0328
C3 vs C5	2.8000	0.6325
C4 vs C5	4.0000	1.0541

The results of the questionnaire explain that the criteria have been weighted according to their importance. For example, reliability and precision criteria (C1) brings significant importance when compared to capital and operational cost criteria (C2). Panelists respond in pairwise comparison involving C1 and C2 within the average value of 1.8 and standard deviation of 1.0328. These values are used to compare the NMCFAHP value. A similar approach is also applied for alternatives evaluations which is consisted as Magnetic particle test (A1), Dye penetrant test (A2), Radiography (A3), and Eddy-current test (A4). The summary result of alternatives weighting questionnaire is demonstrated in Table 4. The original result of alternatives weighting judgment by the expert panelists is available in Appendix 2.

Table 4

The result of alternatives judgments.

Alternatives Evaluation	Mean (x)	Standard Deviation (s)
Criteria 1. Reliability and Precision
A1 VS A2	1.7333	1.4555
A3 VS A1	2.9000	1.3703
A1 VS A4	3.2000	1.1353
A3 VS A2	2.7000	1.3375
A2 VS A4	2.6000	1.2649
A3 VS A4	5.4000	1.2649
Criteria 2. Capital and Operational Costs
A2 VS A1	7.2000	1.4757
A1 VS A3	1.5333	1.0328
A4 VS A1	7.4000	1.5776
A2 VS A3	6.0000	1.4142
A4 VS A2	2.4000	1.3499
A4 VS A3	7.8000	1.3984
Criteria 3. Cracks Detection Coverage
A1 VS A2	2.8000	1.4757
A3 VS A1	4.4000	1.6465
A1 VS A4	4.2000	1.3984
A3 VS A2	4.0000	1.6997
A2 VS A4	2.6000	1.2649
A3 VS A4	7.6667	1.4142
A1 VS A2	2.8000	1.4757
Criteria 4. Training and Development Costs
A2 VS A1	6.4000	1.8974
A3 VS A1	7.8000	1.6865
A4 VS A1	8.2000	1.3984
A3 VS A2	3.0000	1.6330
A4 VS A2	5.0000	1.8856
A3 VS A4	1.5000	0.8498
Criteria 5. Maturity of Technology and Market Availability
A2 VS A1	3.6000	1.3499
A3 VS A1	2.4000	1.3499
A1 VS A4	6.0000	2.1602
A2 VS A3	1.1200	0.7068
A2 VS A4	7.8000	1.9322
A3 VS A4	6.4000	1.6465

Development of NMCFAHP

The development of this process begins by applying random variables in accordance with a Normal distribution with mean and standard deviation of criteria as listed in Table 3. We develop a random function by applying a Normal distribution formula in the excel spreadsheet. From Table 3., we obtain the value of judgment's mean and standard deviation i.e. = 1.800; s = 1.0328. We then generate a Norm-dist fuzzy number by applying the Monte-Carlo simulation folowing the values of and s for each criteria evaluation. This paper employs 1,000 random normal variables. The Monte-Carlo simulation is obtained by entering a spreadsheet formula, i.e. "". The results of generated Monte-Carlo simulation for each criteria evaluation is available in Appendix 3. From the Monte-Carlo simiulation., we obtain the result of Monte-Carlo Normal distribution mean (μ) and standard deviation (σ).

Based on the random number generation for the criteria pairwise comparison, we determine the value of random, random, and random as the most probable, lower value, and upper value of the fuzzy pairwise comparison matrices For the evaluation of criterion 1. versus criterion 2., we determine the value of random, random, and random using Eqs. (4.11), (4.12), and (4.13). Similar calculation is also applied for the rest of criteria judgments, (C1 vs C3, C1 vs C4, C1 vs C5, C2 vs C3, C2 vs C4, C2 vs C5, C3 vs C4, C3 vs C5, and C4 vs C5). The value of random, random, and random are used for the fuzzy analytic hierarchy process calculation corresponding to the lower, most probable, and upper value of the pairwise comparison matrices. The results of evaluation is displayed in Table 5. The fuzzy pairwise comparison is developed for criteria and alternatives, respectively. Based on Eq. (4.28), the consistency index (KIf) of this Norm-dist fuzzy evaluation is quantified below 0.1 for every pairwise comparison matrices. Due to space limitation, a detailed calculation of KIf will not be displayed.

Table 5

Fuzzy pairwise comparison matrix for criteria evaluation. The fuzzification values are determined based on the value of random, random, and random.

Criteria	C1	C2	C3	C4	C5	Normalized Wi
C1	(1,1,1)	(1.7371, 1.8020, 1.8669)	(3.3433, 3.3947, 3.4461)	(8.3298, 8.3898, 8.4497)	(8.3298, 8.3898, 8.4497)	(0.4557, 0.4558, 0.4557)
C2	(0.5356, 0.5549, 0.5757)	(1,1,1)	(3.1444, 3.1842, 3.2240)	(3.1536, 3.1946, 3.2356)	(6.7617, 6.8009, 6.8400)	(0.2833, 0.2833, 0.2834)
C3	(0.2902, 0.2946, 0.2991)	(0.3102, 0.3141, 0.3180)	(1,1,1)	(2.1075, 2.1725, 2.2374)	(2.7711, 2.8109, 2.8507)	(0.1212, 0.1212, 0.1213)
C4	(0.1183, 0.1192, 0.1201)	(0.3091, 0.3130, 0.3171)	(0.4469, 0.4603, 0.4745)	(1,1,1)	(3.9721, 4.0390, 4.1059)	(0.0794, 0.0794, 0.0793)
C5	(0.1183, 0.1192, 0.1201)	(0.1462, 0.1470, 0.1479)	(0.3508, 0.3558, 0.3609)	(0.2436, 0.2476, 0.2518)	(1,1,1)	(0.0375, 0.0373, 0.0371)

KIf = 0.0991.

The pairwise comparison evaluations are then performed for all alternatives corresponding to the evaluative criteria. Similar mathematical processes are implemented for the evaluation of alternatives. The results of generated Monte-Carlo simulation for each alternatives evaluation are available in Appendix 4. We also determine the value of the most probable, lower value, and upper value of the fuzzy pairwise comparison matrices for all alternatives corresponding to the evaluative criteria. The results of alternatives pairwise comparison matrices are available in Tables 6,7,8,9,10.

Table 6

Alternatives evaluation matrix in terms of reliability and precision (C1).

Alternatives	A1	A2	A3	A4	Normalized Wi
A1	(1,1,1)	(1.6033, 1.6925, 1.7816)	(0.3348, 0.3447, 0.3551)	(3.1336, 3.2038, 3.2740)	(0.2403, 0.2415, 0.2426)
A2	(0.5613, 0.5909, 0.6237)	(1,1,1)	(0.3555, 0.3664, 0.3781)	(2.4375, 2.5185, 2.5995)	(0.1733, 0.1741, 0.1750)
A3	(2.8158, 2.9013, 2.9868)	(2.6449, 2.7291, 2.8132)	(1,1,1)	(5.2896, 5.3682, 5.4469)	(0.5077, 0.5062, 0.5046)
A4	(0.3054, 0.3121, 0.3191)	(0.3847, 0.3971, 0.4103)	(0.1836, 0.1863, 0.1891)	(1,1,1)	(0.0787, 0.0782, 0.0778)

KIf = 0.0208.

Table 7

Alternatives evaluation matrix in terms of capital and operational costs (C2).

Alternatives	A1	A2	A3	A4	Normalized Wi
A1	(1,1,1)	(0.1384, 0.1402, 0.1420)	(1.4763, 1.5402, 1.6042)	(0.1323, 0.1340, 0.1358)	(0.0680, 0.0684,0.0687)
A2	(7.0429, 7.1338, 7.2247)	(1,1,1)	(5.8526, 5.9369, 6.0212)	(0.4027, 0.4171, 0.4326)	(0.3305, 0.3303, 0.3302)
A3	(0.6234, 0.6492, 0.6774)	(0.1661, 0.1684, 0.1709)	(1,1,1)	(0.1279, 0.1293, 0.1307)	(0.0558,0.0557, 0.0557)
A4	(7.3642, 7.4621, 7.5599)	(2.3118, 2.3976, 2.4834)	(7.6491, 7.7338, 7.8184)	(1,1,1)	(0.5457, 0.5456, 0.5454)

KIf = 0.0003.

Table 8

Alternatives evaluation matrix in terms of detection coverage (C3).

Alternatives	A1	A2	A3	A4	Normalized Wi
A1	(1,1,1)	(2.6464, 2.7368, 2.8271)	(0.2219, 0.2271, 0.2326)	(4.0526, 4.1388, 4.2250)	(0.2342, 0.2347,0.2351)
A2	(0.3537, 0.3654, 0.3779)	(1,1,1)	(0.2457, 0.2524, 0.2595)	(2.5010, 2.5786, 2.6561)	(0.1289, 0.1294, 0.1300)
A3	(4.2989, 4.4027, 4.5066)	(3.8531, 3.9614, 4.0697)	(1,1,1)	(7.1709, 7.2938, 7.4166)	(0.5769, 0.5760, 0.5751)
A4	(0.2367, 0.2416, 0.2468)	(0.3765, 0.3878, 0.3998)	(0.1348, 0.1371, 0.1395)	(1,1,1)	(0.0600, 0.0599, 0.0598)

KIf = 0.0840.

Table 9

Alternatives evaluation matrix in terms of training and development costs (C4).

Alternatives	A1	A2	A3	A4	Normalized Wi
A1	(1,1,1)	(0.1537, 0.1564, 0.1593)	(0.1263, 0.1280, 0.1297)	(0.1202, 0.1215, 0.1228)	(0.0414, 0.0411, 0.0408)
A2	(6.2782, 6.3922, 6.5061)	(1.0000, 1.0000, 1.0000)	(0.3171, 0.3273, 0.3383)	(0.1953, 0.1999, 0.2046)	(0.1526, 0.1531, 0.1536)
A3	(7.7084, 7.8124, 7.9163)	(2.9561, 3.0550, 3.1539)	(1.0000, 1.0000, 1.0000)	(1.4467, 1.5013, 1.5558)	(0.4169, 0.4177, 0.4182)
A4	(8.1466, 8.2317, 8.3168)	(4.8872, 5.0034, 5.1196)	(0.6427, 0.6661, 0.6912)	(1,1,1)	(0.3891, 0.3882, 0.3874)

KIf = 0.0642.

Table 10

Alternatives evaluation matrix in terms of technology maturity and market availability (C5).

Alternatives	A1	A2	A3	A4	Normalized Wi
A1	(1,1,1)	(0.2713, 0.2775, 0.2840)	(0.4033, 0.4171, 0.4318)	(5.8002, 5.9330, 6.0658)	(0.1726, 0.1726, 0.1728)
A2	(3.5217, 3.6036, 3.6854)	(1,1,1)	(1.0887, 1.1315, 1.1743)	(7.6908, 7.8104, 7.9300)	(0.4333, 0.4328, 0.4322)
A3	(2.3157, 2.3977, 2.4797)	(0.8516, 0.8838, 0.9185)	(1.0000, 1.0000, 1.0000)	(6.2799, 6.3804, 6.4810)	(0.3476, 0.3483, 0.3492)
A4	(0.1649, 0.1685, 0.1724)	(0.1261, 0.1280, 0.1300)	(0.1543, 0.1567, 0.1592)	(1,1,1)	(0.0465, 0.0462, 0.0458)

KIf = 0.0208.

The process is continued by conducting the Kolmogorov-Smirnov test to evaluate random variables for every criterion pairwise comparison. The result of the Kolmogorov-Sminrov test is described in Table 11. The Kolmogorov-Smirnov test proves that all criteria pairwise comparison forms as a Normal distribution. In addition, we implement graphical analysis of Normal Q-Q plot and detrended Normal Q-Q plot using Minitab software illustrated in Figure 8. The graphics show that the criteria pairwise comparison evaluation forms as a Normal distribution. Figure 8 is obtained by plotting the results of Monte-Carlo random number generation for the criteria evaluation available in Appendix 3 into the Minitab software. This analysis shows that all significance values are stated >0.05. Based on this analysis, the pairwise comparison criteria are applicable for the proposed NMCFAHP evaluation.

Table 11

The result of Kolmogorov-Smirnov test for criteria pairwise comparison.

Tests of Normality
	Kolmogorov-Smirnova
	Statistic	df	Sig.	Mean(μ)	Std. Deviation(σ)	Saphiro-Wilk Sig.
C1vsC2	0.019	1000	.200∗	1.80198	1.047694	0.488
C1vsC3	0.023	1000	.200∗	3.39468	0.829563	0.027
C1vsC4	0.019	1000	.200∗	4.61654	0.809628	0.381
C1vsC5	0.017	1000	.200∗	8.38975	0.967064	0.789
C2vsC3	0.024	1000	.200∗	3.18418	0.642174	0.536
C2vsC4	0.014	1000	.200∗	3.19458	0.661144	0.200
C2vsC5	0.019	1000	.200∗	6.80087	0.632353	0.485
C3vsC4	0.012	1000	.200∗	2.17249	1.047965	0.845
C3vsC5	0.018	1000	.200∗	2.81092	0.642314	0.785
C4vsC5	0.020	1000	.200∗	4.03900	1.079334	0.420

This is a lower bound of the true significance.

Lilliefors Significance Correction.

Figure 8

(a) Normal Q-Q plot of pairwise comparison C1 versus C2; (b) Detrended Normal Q-Q plot of pairwise comparison C1 versus C2; (c) Normal Q-Q plot of pairwise comparison C1 versus C3; (d) Detrended Normal Q-Q plot of pairwise comparison C1 versus C3. These graphics show that the pairwise comparison forms as a Normal distribution.

Results and discussion

According to the criteria and alternatives pairwise comparison matrices, we calculate the geometric mean based on Eq. (4.21). This process is intended to calculate the fuzzy weight for each criterion and alternatives by implementing a hierarchical layer sequencing on Eq. (4.29). The results of this computation are shown as a triplet number which follows as a mathematical equation in Eqs. (7.1), (7.2), and (7.3).where , , and are the lower value, most probable value, and upper value of the calculated fuzzy pairwise comparison matrices; is the fuzzy standard deviation; and is the mean of fuzzy calculated values. Random variables for all alternatives are generated in accordance with the values of and . This process can be also performed by iterating several numbers. The results are then plotted as a probability distribution function, as demonstrated in Figure 9. The fuzzy final values are then calculated by applying a defuzzification using the center of gravity method (COG).

Figure 9

The result of probability density function for alternatives final values.

Alternative 1

By solving complex integration in Eq. (7.4), the fuzzy final values for alternative-1 are:

This process is applicable for other alternatives respectively.

Alternative 2

Alternative 3

Alternative 4

The final results of the fuzzy values have been obtained by implementing the proposed NMCFAHP methodology. As demonstrated in Figure 10., the probability distribution function of each alternative are elaborated with respect to the evaluation criteria. The histogram plots clearly state that alternative-3 (, ) as the most optimum solution for the evaluation of NDT technology. The result of alternative-3 is depicted without any overlap from other alternatives. This means that alternative-3 is confidently ranked as the first option in the evaluation. The other three alternatives are not considered preferable due to the numerous gaps from alternative-3 and the overlap condition among each other.

Figure 10

The result of probability density function for alternative final values.

The values of criteria weight are also plotted as a probability density function in Figure 9. It is exposed that reliability and precision (C1) and capital-operational costs (C2) are the most significant evaluation criteria. On the other hand, cracks detection coverage area (C3), training-development costs (C4), and technology maturity (C5) are the least influential criteria. Figure 9 also infers the small numbers of standard deviation. The calculation of standard deviations for C1, C2, and C4 are less than 0.5% of their means. Based on this figure, it is implied that the decision makers are confident in deciding the criteria weights of judgment. From the operational perspective, probability and precision criteria are projected as the first option as this criterion has the main characteristic needed by the organization. The reliability and precision function are considered as fundamental requirement for the NDT technology. Contrarily, technology maturity and market availability come as the least important criteria. In fact, the organization operating the Petroleum Company is currently operating worldwide in more than 20 countries. The aspect of technology maturity and market availability brings insignificant bother to the organization.

Comparison with triangular fuzzy analytic hierarchy process

This paper deploys a comparison between the conventional triangular fuzzy analytic hierarchy process (TFAHP) and the proposed NMCFAHP methodology. The comparison is intended to measure the performance and accuracy of the proposed NMCFAHP for evaluating the decision-making process. Conventional TFAHP is built based on a similar judgment for evaluating NDT technologies, as in the NMCFAHP. As the TFAHP cannot deal with the unbalanced weight scale, the modus values are chosen to replace the mean values for the initial pairwise comparison matrices. The lower, middle, and upper fuzzy values are developed based on triangular fuzzy sets. For the final evaluation, by using TFAHP, triangular distribution is applied to generate random numbers when compared in the Normal distribution for the proposed NMCFAHP. The result is compared by using a standard error of mean parameter and 95% confidence interval values. According to the fuzzy final values, both methodologies demonstrate the similar order of alternatives rank. These analyses bring A3 as the most preferred solution for the NDT technology, followed by A4, A2, and A1 (see Table 12).

Table 12

Comparison of statistical values between proposed NMCFAHP and triangular fuzzy AHP.

Alternatives	Proposed NMCFAHP				Triangular fuzzy AHP
	u_i^L	u_i^M	u_i^U	σ˜	lower	middle	upper	std.dev.
A1	0.15849	0.15943	0.16033	0.01454	0.14038	0.16141	0.16739	0.014537
A2	0.20576	0.20596	0.20632	0.00325	0.19202	0.20334	0.21891	0.003248
A3	0.35919	0.35928	0.35923	0.00082	0.33721	0.36965	0.37929	0.000823
A4	0.24048	0.23947	0.23847	0.01606	0.22130	0.23612	0.23818	0.016063

From Table 13 and Figure 11., it is inferred that the usage of Normal distribution in the Monte-Carlo simulation reduces the standard error of mean values by 90.4–99.8% of the final evaluation scores. This means that the proposed NMCFAHP significantly involves less error when dealing with uncertainty and stochastic randomness. The Normal distribution used in the evaluation tends to concentrate around the mean values and discloses similar values with modus and the defuzzification values. Therefore, it brings a relatively lower standard deviation error. In contrary, the triangular distribution in TFAHP cannot exactly represent the realistic standard deviation values bas the fuzzy sets approximation is based on a triangular fuzzy number instead of an initial standard deviation from the experts' judgement.

Table 13

Confidence interval (95%) values between proposed NMCFAHP and triangular fuzzy AHP.

Alternatives	Proposed NMCFAHP		Triangular fuzzy AHP
	Confidence interval (95%)	Std.dev. error	Confidence interval (95%)	Std.dev. error
A1	(0.15881, 0.16062)	0.00046	(0.12114, 0.19164)	0.01768
A2	(0.20686, 0.20732)	0.00010	(0.17121, 0.23830)	0.02713
A3	(0.35754, 0.35766)	0.00003	(0.30729, 0.41681)	0.04866
A4	(0.23911, 0.24108)	0.00051	(0.20899, 0.25475)	0.02829

Figure 11

Standard error of mean values of proposed NMCFAHP and triangular fuzzy AHP.

In addition, a Normal distribution Monte-Carlo in the proposed NMCFAHP presents a narrower 95% confidence interval compared to the conventional TFAHP. This analysis concludes that the NMCFAHP method can efficiently overcome uncertainty and figures realistic judgment scores in the decision-making process. It is inferred that the proposed NMCFAHP can provide a reliable decision support system.

Conclusion

In this paper, we proposed a novel methodology of the multi-criteria decision-making process, namely the Norm-dist Monte-Carlo fuzzy AHP (NMCFAHP). The development of NMCFAHP is divided into three phases. In the first phase, pairwise comparison data is gathered and analyzed for each criterion and alternative. This analysis is then inputted in to the evaluation of the proposed NMCFAHP. The second phase is concerned with the validity of judgments data which is simulated by extracting of random number based on the Normal distribution model. The Kolmogorov-Smirnov test is used to analyze the normality of the data used as input to the Monte-Carlo fuzzy AHP. In the third phase, we perform the evaluation of multi-criteria decision making by using the NMCFAHP approach. A case study is performed to evaluate the NDT technology for addressing piping and vessels cracks in Petroleum Company, Indonesia. The results of this paper depict that reliability and precision (C1) and capital-operational costs (C2) come as the most significant evaluation criteria, and the alternative technology-3 comes out as the most optimum solution for the NDT technology. In addition, the proposed NMCFAHP present less standard error of mean (by 90.4–99.8%) when compared with TFAHP. This means that NMCFAHP possesses better performance not only to address probabilistic and epistemic uncertainty but also when describing the realistic and confidence in ranking alternatives.

Declarations

Author contribution statement

F. D. Wicaksono: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Y. bin Arshad: Conceived and designed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data.

H. Sihombing: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

	Safety and Method Eng.I	Production/process method Eng.II	Head of Corrosion services	Safety and Method Eng.III	Head of Field Operation Safety and Method	Head of Inspection Department	Safety and Method Eng. I	Corrosion method Eng.	Corrosion method Eng.	Inspection- Instrument Eng.
Criteria Evaluation
C1 vs C2	3	1	1	1	1	3	1	3	3	1
C1 vs C3	3	3	3	3	3	3	5	5	3	3
C1 vs C4	5	5	5	3	3	5	5	5	5	5
C1 vs C5	7	9	9	9	9	7	7	9	9	9
C2 vs C3	3	3	3	3	3	3	3	3	3	5
C2 vs C4	3	3	3	3	3	3	3	3	3	5
C2 vs C5	7	7	7	7	7	7	5	7	7	7
C3 vs C4	1	1	3	3	1	1	3	3	3	3
C3 vs C5	3	3	3	3	3	3	3	3	3	1
C4 vs C5	3	3	5	5	3	5	5	5	3	3

	Safety and Method Eng.I	Production/process method Eng.II	Head of Corrosion services	Safety and Method Eng.III	Head of Field Operation Safety and Method	Head of Inspection Department	Safety and Method Eng. I	Corrosion method Eng.	Corrosion method Eng.	Inspection- Instrument Eng.
Criteria 1. Reliability and Precision
A1 VS A2	1	1	1	3	1	3	0.333333333	5	1	1
A3 VS A1	1	3	2	3	1	3	5	3	3	5
A1 VS A4	1	3	3	3	3	5	3	3	3	5
A3 VS A2	3	3	3	1	1	3	3	4	5	1
A2 VS A4	3	3	3	3	1	5	1	3	1	3
A3 VS A4	5	5	7	5	5	7	7	3	5	5
Criteria 2. Capital and Operational Costs
A1 VS A2	9	7	7	7	5	5	9	7	7	9
A3 VS A1	0.333333333	1	3	1	3	3	1	1	1	1
A1 VS A4	9	9	9	9	7	7	7	7	5	5
A3 VS A2	7	7	7	7	5	5	3	5	7	7
A2 VS A4	1	1	3	3	3	3	1	3	1	5
A3 VS A4	7	7	5	9	7	9	9	7	9	9
Criteria 3. Cracks Detection Coverage
A1 VS A2	1	3	1	1	5	3	3	3	5	3
A3 VS A1	5	5	5	7	3	5	5	5	3	1
A1 VS A4	5	5	3	3	5	5	1	5	5	5
A3 VS A2	5	5	1	1	5	5	5	3	5	5
A2 VS A4	3	3	1	1	5	3	1	3	3	3
A3 VS A4	9	9	7	7	7	9	5	7	3	9
Criteria 4. Training and Development Costs
A1 VS A2	7	7	5	9	7	7	3	5	5	9
A3 VS A1	9	9	7	9	9	9	5	7	9	5
A1 VS A4	9	9	9	5	7	9	9	9	7	9
A3 VS A2	1	1	3	5	3	3	5	5	1	3
A2 VS A4	7	7	5	3	1	5	7	5	5	5
A3 VS A4	1	1	1	3	1	1	1	3	1	2
Criteria 5. Maturity of Technology and Market Availability
A1 VS A2	1	3	3	5	5	5	5	3	3	3
A3 VS A1	3	3	3	1	1	1	1	3	3	5
A1 VS A4	7	7	7	5	1	5	9	7	7	5
A3 VS A2	1	1	3	1	1	1	1	0.2	1	1
A2 VS A4	9	9	9	9	5	9	5	9	9	5
A3 VS A4	7	7	5	5	3	9	7	7	7	7

	C1 vs C2	C1 vs C3	C1 vs C4	C1 vs C5	C2 vs C3	C2 vs C4	C2 vs C5	C3 vs C4	C3 vs C5	C4 vs C5
X¯	1.8000	3.4000	4.6000	8.4000	3.2000	3.2000	6.8000	2.2000	2.8000	4.0000
s	1.0328	0.8433	0.8433	0.9661	0.6325	0.6325	0.6325	1.0328	0.6325	1.0541
Number of iteration
1	2.235	2.666	4.969	7.049	4.683	3.502	6.547	5.350	3.506	4.875
2	0.939	4.262	5.418	8.988	3.070	2.489	7.713	3.143	1.613	2.636
3	1.663	3.520	4.123	7.586	3.536	2.704	7.024	3.109	3.006	5.485
4	1.552	2.951	4.948	7.399	2.719	3.437	6.947	3.369	2.675	1.739
5	1.434	2.316	3.825	9.128	3.379	4.093	7.102	2.241	2.790	2.216
6	2.628	3.382	4.605	5.209	3.799	3.457	8.145	2.291	2.743	4.011
7	1.567	2.804	4.938	8.434	2.897	4.424	6.686	1.504	2.754	5.610
8	2.347	4.243	3.940	7.199	2.112	2.489	6.961	2.800	2.707	2.696
9	1.898	4.857	3.544	7.924	3.150	2.801	7.480	0.756	1.476	4.755
10	-0.038	3.964	3.801	7.729	4.048	3.217	6.306	2.850	2.777	3.280
11	2.094	2.962	4.520	9.508	3.781	1.944	6.107	1.049	3.378	6.266
12	1.331	3.879	5.296	7.716	3.470	4.490	6.368	3.738	2.692	4.657
13	1.707	4.896	3.750	9.857	3.426	2.833	5.836	1.738	3.410	3.936
…	…	…	…	…	…	…	…	…	…	…
1000	0.295	3.411	4.737	7.861	2.903	4.357	6.151	2.533	3.180	3.483
μ	1.8020	3.3947	4.6165	8.3898	3.1842	3.1946	6.8009	2.1725	2.8109	4.0390
σ	1.0477	0.8296	0.8096	0.9671	0.6422	0.6611	0.6323	1.0480	0.6423	1.0793
Random a	1.7371	3.3433	4.5664	8.3298	3.1444	3.1536	6.7617	2.1075	2.7711	3.9721
Random b	1.8020	3.3947	4.6165	8.3898	3.1842	3.1946	6.8009	2.1725	2.8109	4.0390
Random c	1.8669	3.4461	4.6667	8.4497	3.2240	3.2356	6.8400	2.2374	2.8507	4.1059

Corresponding to Criteria 1. Reliability and Precision
	A1 VS A2	A3 VS A1	A1 VS A4	A3 VS A2	A2 VS A4	A3 VS A4
X¯	1.7333	2.9000	3.2000	2.7000	2.6000	5.4000
s	1.4555	1.3703	1.1353	1.3375	1.2649	1.2649
Number of iteration
1	-0.058	3.471	3.175	1.814	4.102	5.412
2	1.810	1.985	2.430	3.010	5.214	3.824
3	1.479	3.738	2.519	3.082	0.841	4.715
4	1.853	1.956	4.600	5.452	4.125	5.085
5	3.069	5.292	4.374	3.448	3.441	5.887
6	1.169	2.852	1.103	3.258	2.407	5.611
7	2.867	2.773	2.695	4.150	1.788	5.025
8	1.707	1.781	4.173	2.451	2.656	4.478
9	-1.477	0.327	6.178	2.715	2.106	7.839
10	0.935	3.340	5.103	2.760	1.889	2.617
11	2.130	2.930	3.309	1.709	0.636	5.284
12	5.036	3.825	1.796	3.405	3.103	6.925
…	…	…	…	…	…	…
1000	0.457	5.826	4.786	5.368	4.188	5.771
μ	1.6925	2.9013	3.2038	2.7291	2.5185	5.3682
σ	1.4390	1.3794	1.1330	1.3581	1.3069	1.2692
Random a	1.6033	2.8158	3.1336	2.6449	2.4375	5.2896
Random b	1.6925	2.9013	3.2038	2.7291	2.5185	5.3682
Random c	1.7816	2.9868	3.2740	2.8132	2.5995	5.4469