A comprehensive analysis of weighting and multicriteria methods in the context of sustainable energy.

This study presents a comprehensive and comparative analysis of weighting and multiple attribute decision-making (MADM) methods in the context of sustainable energy. As the selection problems of energy involve various conflicting attributes, MADM methods have been widely applied in addressing these issues. In this study, six weighting and seven MADM methods that constitute a total of 42 models are implemented to evaluate different weighting and multicriteria decision-making methods and determine the most efficient and sustainable energy option. To determine the weights of economic, environmental, socioeconomic, and technical attributes, two subjective methods-the analytic hierarchy process and best-worst method-and four objective methods-the criteria importance through intercriteria correlation, Shannon's entropy, standard deviation, and mean weight-are used. Thus, both expert evaluations and data-based assessments are considered. Using each attribute weight provided by the six methods, the ranking of electricity generation options for Turkey is obtained through seven MADM methods: the elimination and choice expressing the reality method, the weighted sum method, the weighted product method, the organization, rangement et synthese de donnes relationnelles (ORESTE) method, the technique for order performance by similarity to the ideal solution, the preference ranking organization method for the enrichment of evaluations, and the multiple criteria optimization compromise solution. Rankings obtained from all models are integrated through the Borda, Copeland, and grade average methods. The results indicate that hydro is the optimal electricity generation option, followed by onshore wind, solar PV, geothermal, natural gas, and coal. © Islamic Azad University (IAU) 2020.

Introduction

The major upheaval caused by the coronavirus crisis has shown how modern societies depend on electricity for healthcare, telecommunications, entertainment, shopping, work, and so on. In short, electricity is at the heart of modern economies and human life and provides an increasing share of energy services. The electrification of transportation and heat, the dependence on digitally connected devices, the growing services sector, and the increased use of technology have triggered increases in demand for electricity. Increasing electricity demand was a primary source of global CO2 emissions from the energy sector (impact of fossil fuels) that reached a record level in 2018 (IEA 2019). All these issues indicate that countries must develop long-term energy strategies that consider sustainability indicators.

Most countries tend to increase their use of sustainable energy to cope with environmental issues. In this regard, the transition from fossil fuels to renewable energy is an essential task for many countries. As a developing country, Turkey is mostly dependent on imports for electricity generation, in which natural gas has the highest share (30.34%), followed by hard coal (22.37%), hydro (19.66%), lignite (14.79%), wind (6.54%), solar PV (2.56%), geothermal (2.44%), others (1.19%), and liquid fuels (0.11%) (TEIAS 2020). In other words, a considerable share of total electricity in Turkey is generated by fossil fuels. In addition, strong economic and population growth and increasing income levels have triggered Turkey's greenhouse gas (GHG) emission growth to peak levels among the Organization for Economic Co-operation and Development (OECD) countries. However, Turkey set a mitigation target for 2030 under the UN Framework Convention on Climate Change. Nevertheless, energy is the primary source of GHG emissions in Turkey (OECD 2019).

Any decision regarding energy plays a vital role in future planning; thus, various aspects, such as economic, environmental, socioeconomic, and technical attributes, must be considered. Energy decision makers (DMs) need to take a hard, evidence-based view of current conditions and the results of their choices. Therefore, DMs need practical guiding tools to make the most appropriate decisions, particularly if many alternatives and conflicting criteria affect the choice of these alternatives. In this regard, multiple attribute decision making (MADM) is often adopted to examine specific alternatives and select the optimal choice based on multiple criteria (attributes). The evaluation of electricity generation options and the selection of the optimal option is a typical, critical, and highly influential MADM problem.

In MADM, the attribute weights play a vital role in the decision-making process. In other words, the result of the decision-making process is prominently affected by the attribute weights. There have been numerous approaches utilized for determining attribute weights. These methods can be grouped as subjective, objective, and integrated or hybrid based on the considered preferences and utilized data (Fu and Wang 2015; Yang et al. 2017). Subjective methods solely utilize the subjective evaluations of DMs to obtain the attribute weights. The most commonly used methods include the analytic hierarchy process (AHP) (Saaty 1986), the best–worst method (BWM) (Rezaei 2015), the simple multiattribute rating technique (SMART) (Edwards and Barron 1994), direct rating (Bottomley and Doyle 2001), point allocation (Doyle et al. 1997), linear programming techniques for multidimensional analysis of preferences (LINMAP) (Srinivasan and Shocker 1973), and the Delphi method (Hwang and Yoon 1981). Objective methods are utilized to prevent human-made instabilities and obtain more realistic results. The objective methods use mathematical models and data without considering the preferences of DMs. The most common objective approaches include the criteria importance through intercriteria correlation (CRITIC) (Diakoulaki et al. 1995), entropy (Xu 2004), standard deviation (SD) (Deng et al. 2000), mean weight (MW), and maximizing deviation method (Yingming 1997). Hybrid or integrated approaches combine the preferences of DMs with a decision matrix to obtain the criteria weights. Thus, they take advantage of both method types. Different forms of hybrid methods exist in the literature (Fan et al. 2002; Ma et al. 1999; Pei 2013; Rao et al. 2011; Wang and Parkan 2006).

In this study, the most common subjective approach (AHP) and a recently introduced and trending subjective approach (BWM) are considered as weighting approaches. In addition, four of the most common objective approaches (CRITIC, SD, MW, and entropy) are considered. These methods have been selected to reveal the differences among subjective methods, the differences among objective methods, and the differences among subjective and objective methods in general. Additionally, through evaluations of the weighting methods, the aim is to reveal the impacts of subjectivity and eliminate the uncertainty. Scholten et al. (2015) posited that uncertainty in attribute weights could result from inaccurate quantitative evaluations, personal prejudices, or the use of imprecise weights. In this context, uncertainties related to personal biases caused by subjective weighting methods (AHP and BWM) are minimized by including the objective weighting methods in the proposed methodology. Mean weights are also included in the study to reduce uncertainties that may be caused by attribute weights. In addition, uncertainties caused by inaccurate quantitative estimates are diminished with the inclusion of expert knowledge and experience in the proposed approach. Thus, potential uncertainties are minimized by considering various weighting sets. Notably, sensitivity analysis, which is defined as the analysis of the impact of uncertainty by Saltelli et al. (2000), is not conducted because various weight sets are already analyzed through several weighting methods.

The AHP is one of the most extensively implemented multiple criteria decision-making (MCDM) approaches because of its ease of application and success. The BWM is a new technique compared to the AHP. Nevertheless, it has attracted considerable interest thanks to its efficiency in reducing the number of pairwise comparisons and its success in preserving consistency between assessments. The benefits of the BWM over the AHP can be described as follows. First, the comparisons in the BWM take less time than the AHP because the AHP uses the whole matrix of comparisons. Second, the BWM eliminates redundant comparisons; thus, its performance in preserving the consistency of pairwise comparisons is better than that of the AHP. Last, the complexity of comparisons is reduced in the BWM because it utilizes a 1–9 scale rather than the 1/9–9 scale used in the AHP; this simplifies the evaluation process (Mi et al. 2019). The CRITIC method takes both the contrast intensity and contradictory character of the assessment criteria into account (Diakoulaki et al. 1995). The CRITIC, entropy, and SD methods consider the decision matrix to be the only source of information on the relative importance of indicators. In the MW approach, equal weights are given to the attributes. All these methods have been used for finding attribute weights either solely or as integrated with other MCDM (MADM) methods in various areas (Table 1).

Table 1

Literature review on hybrid weighting and MADM (including fuzzy) methods

Study	AHP	BWM	CRITIC	Entropy	MW	SD	ELECTRE	ORESTE	PROMETHEE	TOPSIS	VIKOR	WPM	WSM	Others
Ren et al. (2017)		✓								✓			✓	✓
Bonyani and Alimohammadlou (2018)		✓					✓		✓
Serrai et al. (2017)		✓								✓	✓		✓	✓
Tian et al. (2018)		✓		✓							✓
Mulliner et al. (2016)	✓									✓		✓	✓	✓
Villacreses et al. (2017)	✓									✓	✓			✓
Lee and Chang (2018)				✓			✓			✓	✓		✓
Beheshtinia Mohammad (2017)	✓									✓	✓			✓
Sivaraja and Sakthivel (2017)	✓						✓			✓	✓
Teraiya et al. (2018)	✓			✓						✓	✓			✓
Abdel-Basset and Mohamed (2020)			✓							✓
Akestoridis and Papapetrou (2019)			✓		✓	✓				✓		✓	✓	✓
Fazeli et al. (2017)			✓	✓	✓	✓				✓
Deng et al. (2000)			✓	✓	✓	✓				✓
Chang et al. (2010)			✓	✓	✓					✓
Moradian et al. (2019)	✓			✓						✓	✓			✓
Wu et al. (2019)	✓			✓							✓
Mian and Al-Ahmari (2019)	✓			✓			✓			✓			✓
Tian et al. (2018)		✓								✓	✓
Feizabadi et al. (2017)	✓			✓			✓			✓	✓			✓
Zanakis et al. (1998)	✓						✓			✓		✓	✓
Anojkumar et al. (2014)	✓						✓		✓	✓	✓
Asgharizadeh et al. (2019)							✓	✓	✓	✓	✓		✓	✓
Salminen et al. (1998)							✓		✓					✓
Opricovic and Tzeng (2007)							✓		✓	✓	✓
Gilliams et al. (2005)	✓						✓		✓
Yeh (2002)										✓		✓	✓
Chalgham et al. (2019)	✓						✓		✓	✓
Kokaraki et al. (2019)	✓						✓		✓	✓
Gao et al. (2018)	✓								✓	✓
Dey et al. (2017)	✓						✓		✓	✓	✓		✓	✓
Li et al. (2020)				✓							✓			✓

To rank the alternatives, seven of the MADM methods, namely the elimination and choice expressing the reality (ELECTRE) method, the weighted sum method (WSM), the weighted product method (WPM), the organization, rangement et synthese de donnes relationnelles (ORESTE) method, the technique for order performance by similarity to the ideal solution (TOPSIS), the preference ranking organization method for the enrichment of evaluations (PROMETHEE), and the multiple criteria optimization compromise solution (VIKOR), are utilized in this study. The reasons for selecting these methods can be explained as follows. First, these are among the most widely used MADM techniques (Table 1). Second, they belong to different MADM family groups and follow different procedures. Third, each method is likely to produce a different result. Therefore, it is more reasonable to use multiple techniques and evaluate the results compared to the use of one method. In this context, Ozernoy (1992) claimed that there was no perfect MADM method to produce intended results for all problems. Ishizaka and Siraj (2018) proposed the use of multiple MADM methods to improve the accuracy of the results for a specific problem. To minimize the uncertainties resulting from MADM methods (Haddad and Sanders 2018; Mosadeghi et al. 2013), multiple MADM methods are included in the proposed methodology.

One of the main objectives of this study is to assess and compare the results of subjective and objective weighting methods and MADM methods. It also aims to reveal the most effective integrated methods in the sustainable energy context. Another main objective of this study is to rank the six leading electricity generation technologies, namely natural gas, coal, hydro, onshore wind, geothermal, and solar photovoltaic (PV), for Turkey through numerous integrated MADM methods. In this context, four main attribute groups, namely economic, environmental, socioeconomic, and technical, are determined in the scope of the sustainable development concept. Twelve subattributes are set under the main criteria based on the literature and expert knowledge. Then, a unique and comprehensive methodology involving various weighting and MADM methods, Spearman's correlation coefficient analyses, and the Borda, Copeland, and grade average methods, is implemented to reveal the optimal electricity generation option for Turkey.

The main contributions of this study can be summarized as follows:

This study is the first to compare the BWM, AHP, entropy, SD, MW, and CRITIC methods.

To minimize the subjectivity of the AHP and BWM and the solely data-based evaluations of the objective weightings of entropy, SD, MW, and CRITIC, all of them are utilized to determine attribute weights.

Because using one MADM method may not guarantee an accurate selection, seven MADM methods are applied to rank the electricity generation options.

A comprehensive analysis is performed using six weighting and seven MADM methods, which constitute 42 models in total. Additionally, some hybrid models, such as entropy-based ORESTE, are used in the evaluation of electricity generation technologies for the first time.

The electricity generation options of Turkey are evaluated by considering sustainability indicators and using current data to the extent possible.

Unlike other studies, the rankings of integrated methods, involving various weighting methods and numerous MADM methods, are combined to determine the optimal electricity generation technology.

A unique and comprehensive methodology involving various weighting and MADM methods, Spearman's correlation coefficient analyses, and the Borda, Copeland, and grade average methods is presented.

The rest of this study is organized as follows. In Sect. 2, the energy profile of Turkey is presented. Section 3 presents a description of each method and the implementation details of the proposed methodology in the energy case. Section 4 includes the results and a discussion regarding each method and analysis. Section 5 concludes the study and provides insights into possible future studies.

Energy and electricity generation profile of Turkey

Due to population and economic growth, the demand for energy and natural resources in Turkey is increasing. With an annual growth rate of 5.5% in electricity demand since 2002, Turkey has set a record for the fastest growth among OECD members. Additionally, energy demand is expected to increase by 50% in the next ten years. Despite the growing energy demand, Turkey is dependent on imports. As preventive and precautious actions, it is aimed to improve energy efficiency, to add nuclear to the energy mix, to increase the share of domestic and renewable energy, and to diversify imported oil and gas supply sources and resources (MFA 2020).

The evaluation and prioritization of electricity generation technologies that consider sustainability indicators are vital. To reveal the current situation, the annual development of installed capacity by the primary energy resources (options) of Turkey is shown in Fig. 1. The general view varies by year. To compare options accurately, 2014 is taken as a reference year because data for solar PV are available starting in 2014. Based on year-by-year changes, it can be seen that solar PV use increased by 124.9% in 2018 compared to 2014. In addition, geothermal energy increased by 2.2%, followed by renewable sources (including waste) by 1.7%, onshore wind by 0.9%, hard coal by 0.5%, hydro by 0.2%, natural gas by 0.1%, and lignite by 0.1%. However, others decreased by 0.3% and liquid fuels by 0.4%. Notably, Turkey has an installed capacity of 90,400 MW as of July 2019 (MFA 2020).

Fig. 1

Annual development of Turkey's installed capacity by primary energy resources

The annual generation (GWh) of each electricity generation option and their shares are shown in Fig. 2. It can be seen that natural gas has the highest share, followed by hard coal, hydro, lignite, wind, solar PV, geothermal, others, and liquid fuels.

Fig. 2

Annual generation and share of each electricity generation option in Turkey in 2018

Turkey is among the largest growing renewable markets in the world. Turkey is the sixth-largest electricity market in Europe, and it has the highest market growth rate in Europe. Both demand and supply are expected to double by 2030. Turkey’s 2023 goal includes increasing the share of renewables to 30%, maximizing the use of hydropower to reach 34 GW, increasing wind and solar installed capacities to 20 GW and 10 GW, respectively, and decreasing the share of natural gas to below 30% of electricity generation. The targets for renewable installed capacity (MW) are 34,000, 20,000, 10,000, 1500, and 1000 for hydro, wind, solar, geothermal, and biomass, respectively (Presidency of The Republic of Turkey 2017). These facts demonstrate why Turkey was selected for the analyses in the study.

Materials and Methods

In this study, four methods for determining attribute weights are considered. The procedures for these methods are defined in the following subsections. The methods utilize the decision matrix given as follows:where denotes the value of alternative i concerning criterion j.

Weighting methods

Best–Worst method (BWM)

The BWM, which was introduced by Rezaei (2015), is a comparison-based approach for determining attribute weights. The outcome of the BWM depends heavily on the assessment of DMs. Like other such methods, it has benefits and drawbacks. The advantages of the BWM include less time required for comparisons and the delivery of more consistent and reliable outcomes compared to the AHP (Mi et al. 2019). Additionally, the BWM only uses integer numbers for pairwise comparisons. However, similar to other subjective methods, it relies on the subjective evaluations of DMs, which can be biased and lead to deceptive outcomes.

The BWM consists of the following steps (Rezaei 2015):

A set of decision criteria is formed. While reaching a decision, the DM identifies n criteria .

The best (most important) and the worst (least significant) criteria are determined. The DM categorizes the best and the worst criteria overall. There is no need for comparisons during this step.

The preference of the best criterion over all the other criteria is specified using a number from 1 to 9. The best-to-others vector is formed as follows:where represents the preference of the best criterion B over criterion j and .

Pairwise comparisons between the worst criterion and the other criteria are formed. The others-to-worst vector is formed as follows:where denotes the preference of criterion j over the worst criterion W and

The optimal weights are defined such that the maximum absolute differences and are minimized for all j. The following min–max model is formed accordingly:The model given above can be transformed to the following model:After solving this model, the optimal weights and are found.

The consistency of the comparison matrix is checked to ensure overall consistency. A consistency ratio is computed via the following function:where CR denotes the consistency ratio and CI represents the consistency index. The consistency index values are given in XX. The lower the CI, the more consistent the comparisons are.

Analytic hierarchy process (AHP)

The AHP, which was introduced by Saaty (1977), determines the weights of criteria through pairwise comparisons in Table 2. The AHP can also help the DM with relevant information to choose the optimal alternative by ranking a set of alternatives. If there are many alternatives, the AHP method may not be appropriate, as it requires a high number of pairwise comparisons. In such a case, the AHP is mostly utilized for weighting the attributes. The advantages and disadvantages of AHP are summarized in Table 3 (Ishizaka and Labib 2011).

Table 2

Consistency index (CI) values

aBW	1	2	3	4	5	6	7	8	9
Consistency index (max ξ)	0	0.44	1	1.63	2.3	3	3.73	4.47	5.23

Table 3

The advantages and disadvantages of AHP

Advantages

Disadvantages

1. It allows pairwise comparisons that improve the accuracy of judgments compared to simultaneously evaluating all the alternatives. It also permits consistency checking 2. The DM is not expected to provide a numerical judgment; instead, verbal judgments are adequate 3. It lets a hierarchical structure of the criteria that provides DMs with a better focus on specific criteria and subcriteria when assigning the weights 4. It uses a ratio scale, meaning that it does not require units in the comparison 5. It allows the assessment of both qualitative and quantitative criteria and alternatives on the same preference scale

1. The possibility of the inconsistency of the pairwise comparison matrix that may result in deceptive outcomes 2. In case the number of criteria or alternatives is high, the demanding pairwise comparisons may increase the complexity of the problem and decrease the consistency of pairwise comparisons 3. The number of indirect comparisons rises with the number of alternatives so that the calculation necessitates an extended processing time

The AHP decomposes a complex MADM problem into a hierarchy and evaluates the relative importance of decision attributes. The steps of the AHP can be explained as follows. First, the hierarchy is formed. The top level of the hierarchy represents the overall goal. The middle level(s) consists of decision attributes (and subcriteria, if any). The bottom level consists of decision alternatives. Second, pairwise comparisons of the criteria are formed based on the 1–9 scale given in Table 4. The relative importance of the attributes at each level is determined.

Table 4

The scale of numbers and definitions (Saaty 1987)

Intensity of importance	Explanation
1	Equal importance
3	Moderate importance
5	Strong importance
7	Demonstrated importance
9	Extreme importance
2,4,6,8	Intermediate values of preferences

To determine the attribute weights, the AHP method forms a pairwise comparison matrix A as follows: where the criteria set is . In the evaluation matrix A(n × n), each element a (i, j = 1, 2, …, n) denotes the relative preference of ith criterion to the jth criterion (Wang and Yang 2007). Then, mathematical operations take place to normalize and determine the relative weights for each matrix. The relative weights are denoted by the right eigenvector (w) matching to the largest eigenvalue (λmax) as follows:

If the pairwise comparisons are wholly consistent, matrix A has a rank of one and λ = n. In such a case, weights can be found by normalizing any of the columns or rows of A. Last, consistency of the judgments is checked to verify the results. The accuracy of the AHP results depends mainly on the consistency of the pairwise comparison evaluations. The relation between the data of A: a * a = a defines the consistency. The CI is obtained through Eq. 7:

The final CR is calculated by dividing CI by a random index (RI), as shown in Eq. 8: where 0.1 is the upper limit of acceptable CR. If the final CR exceeds this limit, the assessment procedure must be performed again to achieve an acceptable consistency value.

Entropy

The entropy concept, which is a measure of uncertainty in information expressed in terms of probability theory, was introduced by Shannon (1948). Shannon's entropy approach interprets the relative intensities of the criterion importance depending on the discrimination among data to evaluate the relative weights (Monghasemi et al. 2015). As an objective method, the entropy method provides reliable results in a case in which the results of subjective methods can be misleading due to prejudiced or inadequate decisions by DMs. This may be considered one of the principal advantages of this objective method. However, data dependence may be a disadvantage. The steps of the entropy method can be described as follows:

In step 1, the decision matrix is normalized for benefit and cost attributes:

In step 2, entropy values are computed using the following equation:

In step 3, the weights of each criterion are obtained using the following equation:

A low entropy value means that the degree of disorder in the system is low and the weight is high (Mohsen and Fereshteh 2017).

Criteria importance through intercriteria correlation (CRITIC)

In this method, the weights are determined based on the contrast intensity and conflict evaluation of the decision problem. Additionally, human intervention is not required for the evaluation process. The steps of the method can be described as follows:

The decision matrix is normalized using the following equation: where denotes the normalized value of the ith design on the jth response.

The following multiplicative aggregation formula is used to determine the amount of information contained in the jth response:where denotes the standard deviation of the jth response and represents the correlation coefficient between two different responses.

The objective weights (wj) are determined by using the following equation:

As a result, this method assigns a high value of weights to those responses with high standard deviation and low correlation with other responses.

Standard deviation (SD)

This approach determines the weights of attributes based on their standard deviations through the following equation:

Mean weight (MW)

MW is a straightforward weighting approach that considers all attributes equally important through the following function:

The MADM methods

Seven MADM approaches are utilized to rank the alternatives and reveal a comparative analysis. Each approach has its procedures, ranking and selecting capabilities, drawbacks, and advantages (Dessler 2006). The algorithms and features of these MADM approaches are summarized in Table 5 but are not detailed in this study, as they are well known and frequently used in the literature.

Table 5

The MADM techniques and their algorithms, advantages, drawbacks, and references

Technique	Algorithm	Feature	Reference
ELECTRE	1. The normalized decision matrix 2. The weighted normalized decision matrix 3. The dominant matrix 4. The dominated matrix 5. The concordance matrix 6. The discordance matrix 7. The dominant aggregate matrix	1. All ELECTRE approaches belong to the family of outranking methods 2. Independence of attributes is not required 3. The qualitative attributes are transformed into the quantitative attributes 4. It is particularly appropriate for decision problems that comprise a few attributes and many alternatives 5. The alternative which dominates all other alternatives is the optimal one	Alinezhad and Khalili (2019a), Benayoun et al. (1966b), Evangelos (2000) and Roy and Decision (1991)
ORESTE	1. The position matrix 2. The block distance 3. The block distance matrix	1. Attributes should be independent 2. It is not required to convert the qualitative attributes into the quantitative attributes 3. It comprises two main tasks that are computing the weak ranking and building the preference (P), indifference (I), and incomparability (R) structure	Alinezhad and Khalili (2019b) and Wu and Liao (2018)
PROMETHEE	1. The preference function 2. The preference index 3. The leaving and entering flows 4. The total flow	1. Transforming qualitative criteria into the quantitative criteria 2. The independence of attributes is not required 3. It is grounded on the pairwise comparison of alternatives along with each chosen attribute	Abedi et al. (2012) and Alinezhad and Khalili (2019c)
TOPSIS	1. The normalized decision matrix 2. The weighted normalized decision matrix 3. Calculation of positive and negative ideal solutions 4. Calculation of separation and relative closeness	1. It is grounded on the fact that the selected alternative should have the shortest distance from the PIS (positive ideal solution) and the farthest from the NIS (negative ideal solution) 2. The PIS is made up of all the best indices, whereas the NIS is composed of all the worst attainable indices 3. It provides a cardinal ranking of options 4. The independence of attributes is not required	Zhang et al. (2011) and Zyoud and Fuchs-Hanusch (2017)
VIKOR	1. The f^∗ and f^- indexes 2. The S and R indexes 3. The VIKOR index 4. The compromise solution	1. The attributes should be independent 2. The qualitative criteria should be transformed into the quantitative criteria 3. It can be considered as an updated version of TOPSIS 4. it concentrates on ranking and selecting from a list of alternatives where conflicting attributes exist, and on recommending a compromise solution	Alinezhad and Khalili (2019d), Kumar et al. (2017) and Opricovic and Tzeng (2007)
WPM	WPM=∏J=1na_ij^w_j w_j is the weight of the jth attribute, and a_ij is the corresponding value	1. It eliminates any units and thus allows dimensionless analysis 2. It can be applied to single- and multidimensional decision problems 3. Cost attributes are required to be transformed into benefit ones	Triantaphyllou and Mann (1989), Wang et al. (2010)
WSM	WSM=maxi∑j=1na_ijw_jfor i=1,2,3,…,m w_j is the weight of the jth attribute, and a_ij is the corresponding value	1. It is a utility-based method 2. It only deals with benefit criteria in general 3. Cost attributes are required to be transformed into benefit ones 4. Successful results for single dimensional decision problems	Mulliner et al. (2016) and Si et al. (2016)

1. The and indexes

ELECTRE, introduced by Benayoun et al. (1966a), consists of two main procedures. A multicriteria aggregation procedure that allows for the creation of one or more outranking relationship(s) aims at comprehensively comparing each pair of actions. An exploitation procedure leads to yield results according to the nature of the problem, including choosing, ranking, or sorting (Figueira et al. 2016). Among the ELECTRE family methods, ELECTRE III is preferred in this study, as it has been successfully implemented for numerous problems in various fields. The WSM, presented by Fishburn (1967), is a fundamental approach. The alternatives are ranked based on their WSM scores. The WSM score of each alternative is equal to the sum of the products. The weighted product method (WPM), introduced by Bridgman (1922), can be considered a different version of the WSM, as it was presented to overcome some of its drawbacks. ORESTE, introduced by Roubens (1982), is highly discriminatory in terms of conflicting alternatives and clearly shows incomparability (Pastijn and Leysen 1989). PROMETHEE, presented by Brans et al. (1986), is based on the pairwise comparison of alternatives along with each attribute and considers the internal relationships of each evaluation fact. Among all PROMETHEE versions, PROMETHEE II is used in this study due to its success in various problems. In the PROMETHEE II approach, the net flow is obtained as final values, and the full ranking of alternatives is provided. TOPSIS, introduced by Hwang and Yoon (1981), assumes that the utility of each attribute tends to increase or decrease monotonically (Evangelos 2000). Thus, defining the positive ideal and negative ideal solutions is possible. VIKOR, presented by Opricovic (1998), can solve MADM problems with contradictory criteria (perhaps in different units) grounded on assumptions that compromising is satisfactory for the resolution of conflict, the DM wants a solution that is the closest to the ideal, and the alternatives are evaluated based on all attributes.

Spearman's rank correlation coefficient

Spearman's rank correlation coefficient is utilized to determine the relationship between two sets of ranks produced by different methods. In this study, it is used to evaluate the results of weighting and MADM methods. A high coefficient value indicates a strong correlation between the two methods. The coefficient (rs) is calculated through the following function:where di is the difference between the ranks of the two methods and n is the number of options.

Application of the proposed method to the selection of electricity generation option

The implementation of the proposed methodology is summarized in Fig. 3.

Fig. 3

The framework of the proposed methodology

First, the decision problem is defined. The problem selected is to rank electricity generation options (natural gas, coal, hydro, onshore wind, geothermal, and solar PV) in Turkey based on sustainable indicators under the categories of economic, environmental, socioeconomic, and technical attributes (Table 6).

Table 6

Main attributes and regarding subattributes

Main attributes	Subattributes
Economic	Levelized cost of electricity (LCOE), economic support, domestic equipment support
Environmental	Land use, water use, GHG
Socioeconomic	Job creation, accident-related fatality
Technical	Electricity mix share, efficiency, capacity factor, lifetime

These attributes are determined through expert knowledge and a literature review, as summarized in Table 7. These attributes are selected because they are commonly considered in the evaluation of such problems in the literature, and they reflect the economic, environmental, socioeconomic, and technical aspects that provide a complete assessment in terms of sustainability. The descriptions and objectives of these attributes are presented in the table. The "min" and "max" objectives indicate that the attributes are cost and benefit, respectively.

Table 7

The selected attributes and descriptions

Attributes	Objective	Description	Reference
LCOE	Min	The average cost of electricity generation for a plant over its lifetime. It involves capital construction, fuel, operation and maintenance, carbon, and decommissioning and waste management costs	Khan (2020), Klein and Whalley (2015) and Yilan et al. (2020)
Economic support	Max	Feed-in-tariff provided for the generation option	Kahraman and Kaya (2010)
Domestic equipment support	Max	Additional maximum domestic equipment contribution
Land use	Min	Land part required for the generation technology	Kahraman and Kaya (2010), Khan (2020) and Troldborg et al. (2014)
Water use	Min	Water that is used and cannot be returned to the source	Evans et al. (2017), Khan (2020) and Klein and Whalley (2015)
GHG emission	Min	The lifetime GHG emissions from the option	Khan (2020), Klein and Whalley (2015), Streimikiene et al. (2012), and Troldborg et al. (2014)
Job creation	Max	Job years of full-time employment generated over the entire lifetime of the option	Goumas and Lygerou (2000), Klein and Whalley (2015) and Stein (2013)
Accident-related fatality	Min	Deaths stemmed from accidents in the entire lifetime of the option	Klein and Whalley (2015), and Yilan et al. (2020)
Electricity mix share	Max	The electricity generation share of the option	Yilan et al. (2020)
Efficiency	Max	The ratio of the output to the input energy	Chatzimouratidis and Pilavachi (2009), Khan (2020) and Stein (2013)
Capacity factor	Max	The ratio of the real output of the plant to the maximum possible output	Chatzimouratidis and Pilavachi (2009) and Stein (2013)
Lifetime	Max	Total lifespan of the electricity generation option	Klein and Whalley (2015) and Sharma et al. (2015)

The data for sustainability indicators in Turkey are collected from official reports, websites, articles, and periodical reports from reputable institutions. The units of attributes and the data sources for each indicator are given in Table 8. For some attributes, it is not possible to find Turkey-specific data. Therefore, average values are considered for such cases.

Table 8

Units of the attributes and data sources

Attributes	Unit	Source
LCOE (C1)	USD/MWh	Wittenstein and Rothwell (2015) and WWF-Turkey (2014)
Economic support (C2)	US Cent/kWh	Turkish Energy Foundation (2017)
Domestic equipment support (C3)	US Cent/kWh	Industrial Development Bank of Turkey(2019)
Land use (C4)	m2/kWh	Evans et al. (2017)
Water use (C5)	L/kWh	Evans et al. (2017)
GHG emission (C6)	gCO2-e/kWh	Khan (2020) and World Nuclear and WorldNuclear (2011)
Job creation (C7)	avg. job years/GWh	Bacon and Kojima (2011)
Accident-related fatality (C8)	Fatalities / GWeyr	Edenhofer et al. (2011)
Electricity mix share (C9)	%	TEIAS (2020)
Efficiency (C10)	%	Evans et al. (2017)
Capacity factor (C11)	%	Turkish Energy Foundation (2017) and Wittenstein and Rothwell (2015)
Lifetime (C12)	Year	Wittenstein and Rothwell (2015)

Then, the performance matrix is formed based on the collected data, as presented in Table 9. The range of data in each column is different. Additionally, the unit of each attribute is different (Table 8). To ensure consistency in the calculations, the data are normalized during the weighting process and MADM methods. Each method may require different normalization methods.

Table 9

Raw data of sustainability attributes for each electricity generation technology

	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12
Natural gas	156	0	0	0.0003	1.6	499	0.11	0.0721	30.34	49	85	30
Coal	92.5	0	0	0.0004	1.6	888	0.11	0.1200	37.2	38.5	85	40
Hydro	41.34	7.3	2.3	0.004	20	26	0.27	0.0027	19.7	90	35	80
Wind (onshore)	73.19	7.3	3.7	0.015	0.001	26	0.17	0.0019	6.54	34	33	25
Geothermal	116.33	10.5	2.7	0.05	156	170	0.25	0.0017	2.44	15	90	40
Solar PV	160	13.3	6.7	0.0003	0.01	85	0.87	0.0002	2.56	13	18	25

Then, each weighting method procedure is implemented. For subjective methods, the data are not required since they depend on subjective evaluations from the expert. In addition, the objective weighting methods process the performance matrix. Different attribute weight sets are obtained from each weighting method. The correlation between each method is evaluated through Spearman's correlation coefficient. Thus, the outcome of each method is analyzed, and their similarities/dissimilarities are revealed. By using the outputs of each weighting method as input, the MADM methods are implemented separately. Thus, the rankings of electricity generation options are obtained by 42 different models. Then, these results are analyzed through Spearman's rank correlation coefficients and variances. Finally, to reach an ultimate ranking, the Borda, Copeland, and grade average methods are implemented. In this way, an overall ranking is revealed, and the best option is recommended on a consensus basis. Overall, it can be stated that the proposed methodology is a unique hybrid decision-making approach.

Results and discussion

Based on the data in Table 9, each objective weighting method (CRITIC, entropy, SD, and MW) and subjective weighting approach (AHP and BWM) are implemented to obtain the relative importance (weight) of each attribute. The weights of all attributes provided by all methods are shown in Table 10. As seen in Table 10, the results of the AHP and BWM are similar, as they are based on the same expert knowledge. The most critical attribute is LCOE, followed by economic support and efficiency, according to the AHP and BWM. However, their results are not identical. The results of entropy indicate that water use is the most critical attribute, followed by accident-related fatalities and land use. For the SD and CRITIC method, the capacity factor is the most significant attribute, followed by the electricity mix share. All attributes are equally important in the MW method.

Table 10

Attribute weights obtained from subjective and objective methods

Method	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12
BWM	0.2660	0.1330	0.0665	0.0287	0.0503	0.0934	0.0517	0.0517	0.0268	0.1204	0.0669	0.0446
AHP	0.2614	0.1331	0.0558	0.0396	0.0495	0.0930	0.0523	0.0492	0.0444	0.1282	0.0544	0.0391
Entropy	0.0221	0.0780	0.0885	0.0999	0.2626	0.0893	0.0554	0.1602	0.0665	0.0350	0.0253	0.0173
SD	0.0821	0.0851	0.0782	0.0822	0.0831	0.0834	0.0791	0.0888	0.0897	0.0762	0.0936	0.0785
CRITIC	0.0961	0.1028	0.0932	0.0050	0.1024	0.0932	0.0955	0.0123	0.1306	0.0744	0.1332	0.0613
MW	0.0833	0.0833	0.0833	0.0833	0.0833	0.0833	0.0833	0.0833	0.0833	0.0833	0.0833	0.0833

The calculated Spearman's correlation coefficients are given in Table 11. Notably, MW is excluded because all weights are equal. The results indicate that there is a significant correlation between the AHP and BWM. Additionally, the correlation between the CRITIC and SD can be regarded as moderate. Otherwise, the remaining correlations are weak.

Table 11

Spearman's correlation coefficients for weighting methods

	AHP	BWM	Entropy	CRITIC	SD
AHP	1.0000	0.9530	− 0.1888	0.3468	− 0.1329
BWM		1.0000	− 0.2802	0.2140	− 0.1331
Entropy			1.0000	− 0.2207	0.2587
CRITIC				1.0000	0.5114
SD					1.0000

The MADM methods (ELECTRE, ORESTE, PROMETHEE, TOPSIS, VIKOR, WPM, and WSM) are applied based on the data (Table 9) and the result of each weighting method. The ranking results are given in Tables 12, 13, 14, 15, 16, 17.

Table 12

Ranking results of the AHP-based MADM methods

	ELECTRE III	ORESTE	PROMETHEE II	TOPSIS	VIKOR	WPM	WSM
Coal	4	5	5	5	6	6	5
Geothermal	2	3	3	4	3	4	4
Hydro	1	1	1	1	1	1	1
Natural gas	5	6	6	6	2	5	6
Solar PV	6	4	4	3	4	3	3
Wind (onshore)	3	2	2	2	5	2	2

Table 13

Ranking results of the BWM-based MADM methods

	ELECTRE III	ORESTE	PROMETHEE II	TOPSIS	VIKOR	WPM	WSM
Coal	4	6	5	5	6	6	5
Geothermal	2	3	3	4	3	4	4
Hydro	1	1	1	1	1	1	1
Natural gas	5	5	6	6	2	5	6
Solar PV	6	4	4	3	4	3	3
Wind (onshore)	3	2	2	2	5	2	2

Table 14

Ranking results of the entropy-based MADM methods

	ELECTRE III	ORESTE	PROMETHEE II	TOPSIS	VIKOR	WPM	WSM
Coal	6	6	6	5	4	6	5
Geothermal	4	5	5	6	5	4	6
Hydro	3	3	3	3	2	1	2
Natural gas	5	4	4	4	3	3	4
Solar PV	1	1	1	1	6	5	1
Wind (onshore)	2	2	2	2	1	2	3

Table 15

Ranking results of the SD-based MADM methods

	ELECTRE III	ORESTE	PROMETHEE II	TOPSIS	VIKOR	WPM	WSM
Coal	6	6	5	5	4	6	5
Geothermal	2	4	4	6	5	5	6
Hydro	3	1	1	1	1	1	1
Natural gas	5	5	6	4	2	3	4
Solar PV	1	3	2	2	6	4	2
Wind (onshore)	4	2	3	3	3	2	3

Table 16

Ranking results of the CRITIC-based MADM methods

	ELECTRE III	ORESTE	PROMETHEE II	TOPSIS	VIKOR	WPM	WSM
Coal	5	3	5	5	3	6	4
Geothermal	1	6	4	6	2	4	6
Hydro	3	1	1	1	1	1	1
Natural gas	6	4	6	4	4	3	5
Solar PV	4	5	3	2	6	5	2
Wind (onshore)	2	2	2	3	5	2	3

Table 17

Ranking results of the MW-based MADM methods

	ELECTRE III	ORESTE	PROMETHEE II	TOPSIS	VIKOR	WPM	WSM
Coal	6	5	5	5	4	6	5
Geothermal	4	4	4	6	5	5	6
Hydro	1	2	1	1	1	1	1
Natural gas	5	6	6	4	3	4	4
Solar PV	3	3	2	2	6	3	2
Wind (onshore)	2	1	3	3	2	2	3

As seen in Table 12, all AHP-based MADM methods suggest that hydro is the best alternative. The rankings of AHP-ORESTE and AHP-PROMETHEE are identical. Additionally, AHP-TOPSIS and AHP-WSM provide the same rankings.

Table 13 reveals the rankings provided by BWM-based MADM methods. All methods suggest hydro as the optimal alternative. The rankings for BWM-TOPSIS and BWM-WSM are the same.

The rankings provided by entropy-based MADM methods are given in Table 14. Solar PV is recommended as the optimal option in general. However, Entropy-WPM suggests hydro and Entropy-VIKOR recommends wind as the ideal alternatives.

As seen in Table 15, SD-based MADM methods suggest hydro as the best option except for SD-based ELECTRE, whose best option is solar PV.

CRITIC-based MADM methods also suggest hydro as the most reasonable alternative except for CRITIC-based ELECTRE, which recommends geothermal as the best option (Table 16).

For equally weighted attributes, MW-based MADM methods suggest hydro as the best option except for MW-ORESTE, which suggest wind as the optimal option, as given in Table 17.

Overall, hydro is suggested as the most reasonable electricity generation option by most of the weighting–MADM method pairs. However, other options can also be the best, as seen from the results. These results reveal that each MADM method provides different rankings for the same case (weights) and verify the importance of using multiple MADM methods for evaluations. As stated in the literature, using multiple MADM methods is more reasonable than using a single method. The final decision should be made after evaluating various MADM models.

The correlation between the methods is expressed through Spearman's correlation coefficients in Table 18. The significant correlations are marked. Spearman's rank correlation coefficient “1” indicates a perfect association of the models. In other words, their rankings are the same. For instance, there is a very strong correlation between the AHP-based TOPSIS and the WSM. The higher the correlation coefficient is, the stronger the correlation between the models is.

Table 18

Spearman's correlation coefficients for the MADM methods

		ELECTRE III	ORESTE	PROMETHEE II	TOPSIS	VIKOR	WPM	WSM
AHP	ELECTRE III	1.000	0.771	0.771	0.543	0.371	0.486	0.543
	ORESTE		1.000	1.000**	0.943**	0.257	0.886*	0.943**
	PROMETHEE II			1.000	0.943**	0.257	0.886*	0.943**
	TOPSIS				1.000	0.200	0.943**	1.000**
	VIKOR					1.000	0.429	0.200
	WPM						1.000	0.943**
	WSM							1.000
BWM	ELECTRE III	1.000	0.714	0.771	0.543	0.371	0.486	0.543
	ORESTE		1.000	0.943**	0.886*	0.486	0.943**	0.886*
	PROMETHEE II			1.000	0.943**	0.257	0.886*	0.943**
	TOPSIS				1.000	0.200	0.943**	1.000**
	VIKOR					1.000	0.429	0.200
	WPM						1.000	0.943**
	WSM							1.000
Entropy	ELECTRE III	1.000	0.943**	0.943**	0.829*	− 0.029	0.314	0.771
	ORESTE		1.000	1.000**	0.943**	0.086	0.371	0.886*
	PROMETHEE II			1.000	0.943**	0.086	0.371	0.886*
	TOPSIS				1.000	0.143	0.257	0.943**
	VIKOR					1.000	0.771	0.086
	WPM						1.000	0.314
	WSM							1.000
SD	ELECTRE III	1.000	0.543	0.657	0.314	− 0.486	0.143	0.314
	ORESTE		1.000	0.886*	0.771	0.314	0.829*	0.771
	PROMETHEE II			1.000	0.771	0.029	0.543	0.771
	TOPSIS				1.000	0.371	0.771	1.000**
	VIKOR					1.000	0.714	0.371
	WPM						1.000	0.771
	SM							1.000
CRITIC	ELECTRE III	1.000	− 0.086	0.600	− 0.086	0.257	0.314	-0.029
	ORESTE		1.000	0.543	0.600	0.257	0.600	0.657
	PROMETHEE II			1.000	0.714	0.143	0.600	0.771
	TOPSIS				1.000	− 0.143	0.543	0.943**
	VIKOR					1.000	0.314	− 0.086
	WPM						1.000	0.371
	WSM							1.000
MW	ELECTRE III	1.000	0.886*	0.886*	0.771	0.486	0.943**	0.771
	ORESTE		1.000	0.829*	0.600	0.371	0.771	0.600
	PROMETHEE II			1.000	0.771	0.200	0.771	0.771
	TOPSIS				1.000	0.429	0.886*	1.000**
	VIKOR					1.000	0.600	0.429
	WPM						1.000	0.886*
	WSM							1.000

**Correlation is significant at the 0.01 level (2-tailed)

*Correlation is significant at the 0.05 level (2-tailed)

However, there is no consistency in these correlations. For instance, although the correlation between ORESTE and PROMETHEE is strong for the AHP, BWM, entropy, SD, and MW weights, the correlation is moderate for CRITIC weights. The reasons for these discrepancies among rankings of different MADM methods can be generally explained as follows.

First, these methods process weights in different ways in their calculations. Second, algorithms vary in their approach to choosing the optimal option. Third, some procedures require additional parameters that affect the selection of the option. Last, many procedures try to scale the objectives that affect the already chosen weights (Zanakis et al. 1998). The possible reasons or explanations can also be explained explicitly at the method level. In addition to the features of the methods in Table 5, the following points may be explanatory in this regard:

TOPSIS identifies a solution with the shortest distance (Euclidean) from the PIS and the farthest distance from the NIS, whereas it does not take into account the relative importance of these distances (Hwang and Yoon 1981; Yoon 1987).

Even though both the VIKOR and the TOPSIS methods are distance-based, a compromise solution in VIKOR is determined based on mutual concessions (Opricovic and Tzeng 2007).

The WPM overvalues extremes, leading to misleading results, as it substantially favors/disfavors the final assessment of any alternative that is far from the average.

ELECTRE and VIKOR are based on similar principles in terms of consideration of a specific global measure (concordance and group utility), and the opposition of the other attribute is not too strong (nondiscordance) (Opricovic and Tzeng 2007).

In PROMETHEE, differences in attribute values are not considered entirely. The degree of exceedance preference threshold is ignored. Uncertainty is handled with constant thresholds.

In ELECTRE III, differences in attribute values are not considered wholly. Regardless of how significant the value is, the criterion is better than another criterion. Uncertainty is handled with constant or proportional thresholds.

The ranking of the options differs between PROMETHEE and ELECTRE III; the procedure of PROMETHEE is additive, whereas ELECTRE III uses distillation that is grounded on the qualification of each option (Salminen et al. 1998).

ORESTE stands out because of its conflict analysis test, which enables separate indifference, incomparability, and preference situations (van Huylenbroeck 1995).

Apart from the characteristics of the MADM methods that cause different rankings, the impact of the weighting method is also apparent. In Tables 12, 13, 14, 15, 16, 17, the impact of the weighting methods on the rankings can be observed evidently. The ranking provided by the same MADM method may vary due to the weighting method. To better observe and further analyze this situation, such as evaluating the robustness of the MADM methods, Figs. 4, 5, 6, 7, 8, 9 and 10 are provided.

Fig. 4

Ranking results of ELECTRE

Fig. 5

Ranking results of ORESTE

Fig. 6

Ranking results of PROMETHEE

Fig. 7

Ranking results of TOPSIS

Fig. 8

Ranking results of VIKOR

Fig. 9

Ranking results of WPM

Fig. 10

Ranking results of WSM

In Fig. 4, the ranking results of different weighting approach-based ELECTRE methods are illustrated. It can be inferred from the figure that the ranking of ELECTRE is susceptible to the weighting method—weights. Therefore, the right choice of the weighting method for ELECTRE is vital. For instance, hydro is ranked first by three of the methods, but it is ranked third by the remaining three methods. Similarly, the solar PV option is ranked first, third, fourth, and sixth based on different weights. These outcomes reveal the impact of the weights on the ELECTRE ranking.

In Fig. 5, the general overview of the results of ORESTE models is different from Fig. 4. For instance, hydro is ranked first in the four models, and the ranking average of hydro is lower compared to Fig. 4.

In Fig. 6, the rankings of PROMETHEE seem more stable than those of ELECTRE and ORESTE. The overall best option, hydro, is ranked first in five PROMETHEE models.

In Fig. 7, the impact of weights on the TOPSIS results can be observed. However, the impact may be regarded as limited compared to the previous models. For instance, the coal option is ranked fifth by all TOPSIS models regardless of the weighting approach.

In Fig. 8, the rankings of VIKOR also seem vulnerable to the method chosen for weighting. Nevertheless, hydro is ranked first by five VIKOR models out of six.

In Fig. 9, the rankings of the WPM can be observed as the most robust among all MADM methods used in this study. For instance, the rankings of hydro, wind, and coal options are the same for all WPM models, regardless of the weighting method.

In Fig. 10, the rankings of the WSM illustrated that hydro is ranked first by five of the WSM models.

The evaluations based on Figs. 4–10 are mostly grounded on observations. For further tangible analysis and evaluations, all outcomes are assessed based on variances. In this context, the values are calculated as given in Table 19. The variance measure mainly designates the variations from the mean and, consequently, the precision and effectiveness of the techniques. The minimum values of this measure are always desirable.

Table 19

Variance values of the models

	ELECTRE	ORESTE	PROMETHEE	TOPSIS	VIKOR	WPM	WSM
Coal	0.9667	1.3667	0.1667	0.0000	1.5000	0.0000	0.1667
Geothermal	1.5000	1.3667	0.5667	1.0667	1.7667	0.2667	1.0667
Hydro	1.2000	0.7000	0.6667	0.6667	0.1667	0.0000	0.1667
Natural gas	0.1667	0.8000	0.6667	1.0667	0.6667	0.9667	0.9667
Solar PV	5.1000	1.8667	1.4667	0.5667	1.0667	0.9667	0.5667
Wind (onshore)	0.6667	0.1667	0.2667	0.3000	3.1000	0.0000	0.2667
Average	1.6000	1.0444	0.6333	0.6111	1.3778	0.3667	0.5333

The results presented in Table 19 indicate that the WPM is the least susceptible to the weighting method. The WPM provides the most consistent rankings regardless of the weighting method compared to the other MADM methods for the case considered. However, the rankings of ELECTRE are vulnerable to the weighting method. Therefore, the selection of the weighting method for ELECTRE is crucial.

After evaluating all the weighting and MADM methods, further calculations and evaluations need to be made to determine an optimal option. In this context, the rankings of all 42 models are demonstrated together in Fig. 11. Since there are 42 different rankings, they should be integrated to attain a final ranking to reveal the optimal electricity generation option. Thus, three of the approaches are utilized.

Fig. 11

Rankings of the alternatives by 42 different models

Integrated ranking approaches

To determine a final ranking for various rankings produced by numerous models, three approaches are used.

Borda method

The Borda method was presented by Borda (1784). A pairwise comparison matrix is formed to calculate the Borda scores of the options. To score an option, for all other options that rank below the selected option, a value of one is assigned; otherwise, a zero value is assigned to the corresponding matrix value (Table 20). For this study, the results of 42 models are considered and compared. Once the pairwise comparison matrix is completed, the row sum for an option indicates the number of times that the option is preferred to the others. The Borda scores of each option are presented in Table 21.

Table 20

Calculation results for the Borda and Copeland methods for final ranking

	Coal	Geothermal	Hydro	Natural gas	Solar PV	Wind (onshore)	Row Sum	Difference
Coal	0	0	0	0	0	0	0	− 5
Geothermal	1	0	0	1	0	0	2	− 1
Hydro	1	1	0	1	1	1	5	5
Natural gas	1	0	0	0	0	0	1	− 3
Solar PV	1	1	0	1	0	0	3	1
Wind (onshore)	1	1	0	1	1	0	4	3
Column Sum	5	3	0	4	2	1

Table 21

Comparison of the results of different ranking strategies for integrated ranking

	Borda		Copeland		Grade average
	Value	Ranking	Value	Ranking	Value	Ranking
Coal	0	6	− 5	6	5.12	6
Geothermal	2	4	− 1	4	4.19	4
Hydro	5	1	5	1	1.36	1
Natural gas	1	5	− 3	5	4.55	5
Solar PV	3	3	1	3	3.29	3
Wind (onshore)	4	2	3	2	2.50	2

Copeland method

The Copeland method is an extended version of the Borda method (Dortaj et al. 2020). In addition to the calculations for the Borda method, the column sum for each alternative is calculated. The Copeland value of an option is equal to the difference between the row sum and column sum of that option (Tables 20, 21).

The grade average method

The arithmetic average of rankings obtained from different methods is calculated for each option, and the final ranking is determined accordingly. As expected, the lower-ranking average an option has, the better option it is. The calculated averages for each option are presented in Table 21.

Based on Borda, Copeland, and grade average values, it can be inferred that hydro is the best electricity generation option for Turkey, followed by wind (onshore), solar PV, geothermal, natural gas, and coal. As several weighting models provided different weights and numerous models that resulted in various rankings, a sensitivity analysis was not needed. The cases in which different weights are allocated to different attributes already exist.

To compare the results of this study with the previous results, several studies are considered. It may be reasonable to consider the studies conducted for Turkey initially. Yilan et al. (2020) examined the electricity generation options in Turkey based on sustainability indicators through two MADM methods, namely the WSM and multiattribute utility theory. As a result, hydro was found to be the best option, similar to this study. Atilgan and Azapagic (2016) evaluated the electricity generation technologies for Turkey in terms of sustainability through one MADM method: a multiattribute value theory. The results of the study indicated that hydro was the most sustainable option, followed by geothermal and wind. Özkale et al. (2017) assessed renewable energy options based on PROMETHEE and found that hydro was the best alternative. Büyüközkan and Güleryüz (2017) implemented MADM methods, namely TOPSIS, ANP (analytic network process), and DEMATEL (decision-making trial and evaluation laboratory), and the best option was found to be geothermal. In addition, it may be helpful to compare the results of studies conducted for other countries to reveal the impact of data on the outcomes of MADM methods. In a comparative analysis study of the MADM methods, Lee and Chang (2018) used Shannon's entropy, TOPSIS, WSM, ELECTRE, and VIKOR methods to evaluate renewable energy options in Taiwan. They found that hydro was the best option for Taiwan, followed by solar, wind, biomass and geothermal. Ali et al. (2020) used a subjective weighting method (BWM) and objective method (integrated determination of objective criteria weights (IDOCRIW)) and an evaluation based on distance from average solution (EDAS) to rank power generation technologies in Bangladesh. Additionally, a comparative analysis was conducted through three MADM methods, namely combinative distance-based assessment (CODAS), weighted aggregated sum product assessment (WASPAS), and multiobjective optimization based on simple ratio analysis (MOOSRA). The results indicated that gas was the best option, followed by oil, coal, solar, hydro, and wind.

It can be seen that the ranking of electricity generation technologies varies in these studies, although the best option can be the same. The reasons for this outcome can be explained as follows. First, the weighting methods significantly affect the ranking results of the MADM methods. The preference of subjective or objective methods changes the result to some extent. Additionally, the choice of different attributes affects ranking. In this context, the fact that the data forming the decision matrix are ​​different changes the ranking. Last, the selection of MADM methods is crucial because the procedures of MADM methods vary, which produces different outcomes. Therefore, the use of hybrid methods might increase the accuracy of the outcomes.

As seen in these studies, some used only subjective or objective methods as the weighting method. If two method groups were preferred, mostly one method from each of these groups was used. Additionally, the number of MADM methods used differed from study to study; however, the number was generally limited. It can be said that no study has used all the methods in this study in an integrated approach. Thus, the present study differs from previous studies in terms of integrating numerous approaches to determine the optimal electricity generation technology for the first time and presenting a unique and comprehensive methodology involving various weighting and MADM methods, Spearman's correlation coefficient analyses, and the Borda, Copeland, and grade average methods. In this way, the disadvantages of each method are minimized, and the advantages of each are utilized through aggregation; thus, the accuracy of the assessment is increased.

In summary, the following advantages of the proposed method might explain the reasons for choosing these methods and proposing the hybrid approach. First, the advantages of both subjective and objective weighting methods were integrated. Second, the MADM methods from different groups were used to utilize the advantages of each. Third, the results of the subjective and objective methods were compared and explicitly analyzed. Additionally, the results of each MADM method were revealed and compared. Thus, the results of 42 different models were presented and compared. Notably, the result of each model can be the main topic of a separate study. Fourth, the performance and behavior of each MADM method were analyzed. Fifth, the results of each hybrid model were integrated through different aggregation methods to allow for comparisons. Ultimately, a final consensus of the 42 models was proposed as the best option.

Overall, by utilizing various methods and combining their advantages, the proposed methodology provides a more effective and accurate evaluation and analysis from a broader perspective. However, there are some limitations to this study. First, the availability of the updated country (Turkey)-specific data might increase the usefulness and accuracy of the results. Second, the drawback of each weighting method group may be considered a limitation. Being entirely dependent on the subjective evaluations of experts and relying on the statistical evaluations of the data in the performance matrix, regardless of the knowledge and experience of the decision maker, can be considered the disadvantages of subjective and objective methods, respectively. In this context, the inclusion of “combinative weighting” methods could further expand the scope of the study. Additionally, the number of MADM methods could be expanded for a broader comparative analysis. Last, the fuzzy set theory might be useful to handle uncertainty.

Conclusion

In this study, six different weighting and seven MADM methods were utilized for raking electricity generation options in Turkey based on sustainability attributes. The output of each weighting method was used as the input for each MADM method. Thus, rankings of the options were obtained through 42 different models. To determine the optimal technology, the rankings were integrated through the Borda, Copeland, and grade average methods. According to the evaluations and results, some of the findings of this study can be summarized as follows:

Hydro is the best energy generation option, followed by onshore wind, solar PV, geothermal, natural gas, and coal, based on the evaluation of 42 different models. The rankings in previous studies regarding Turkey are different. However, hydro, solar, and wind are generally recommended as the best options.

As mentioned previously, it is more reasonable to use multiple MADM methods. If one of the MADM methods is used, based on the integrated results, it can be inferred that any of the hybrid AHP-WPM, BWM-WPM, Entropy-WPM, SD-ORESTE, CRITIC-WPM, and MW-ELECTRE III models can be implemented for the selection of electricity generation option problems.

The rankings of ELECTRE are susceptible to the weighting method. Therefore, the selection of the weighting method for ELECTRE is crucial.

Subjective methods, such as the AHP and BWM, provided similar weights. However, objective methods provided different weights. Therefore, choosing any of them is a prominent task.

Subjective and objective weighting methods provided different weights for attributes. If possible, using them together will prevent subjectivity and solely data-based evaluations.

The rankings of the WPM are more stable than the rankings of the other six MADM methods. In other words, the impact of the weighting method on the WPM outcome is less.

It can be concluded that the share of renewable energy options should be increased based on economic, environmental, socioeconomic, and technical attributes.

In addition to these highlighted results, the present study contributes to the literature by presenting a unique integrated approach that utilizes various methods, including subjective and objective weighting, MADM, and aggregation. Given that each weighting and MADM method has its advantages and disadvantages, the proposed integrated approach reduces the disadvantages and makes most of the advantages, thus providing more effective and accurate evaluations. For further research, in considering the significance of each case, more analyses and calculations considering more comprehensive attributes can be conducted. Additionally, the number of MADM methods can be expanded. Machine learning methods can also be integrated into the model for future predictions.