Ideals, practices, and future prospects of stakeholder involvement in sustainability science.

This paper evaluates current stakeholder involvement (SI) practices in science through a web-based survey among scholars and researchers engaged in sustainability or transition research. It substantiates previous conceptual work with evidence from practice by building on four ideal types of SI in science. The results give an interesting overview of the varied landscape of SI in sustainability science, ranging from the kinds of topics scientists work on with stakeholders, over scientific trade-offs that arise in the field, to improvements scientists wish for. Furthermore, the authors describe a discrepancy between scientists' ideals and practices when working with stakeholders. On the conceptual level, the data reflect that the democratic type of SI is the predominant one concerning questions on the understanding of science, the main goal, the stage of involvement in the research process, and the science-policy interface. The fact that respondents expressed agreement to several types shows they are guided by multiple and partly conflicting ideals when working with stakeholders. We thus conclude that more conceptual exchange between practitioners, as well as more qualitative research on the concepts behind practices, is needed to better understand the stakeholder-scientist nexus.

The global threats of climate change, rising inequalities, and unsustainable development pose major challenges for science. Emerging scientific fields, such as sustainability science (1–4) and transformative research (5–7), try to find innovative ways to cope with the “social embeddedness” (8), uncertainty (9), and complexity (10, 11) of these issues that affect the whole of society and thus touch upon a multitude of different interests (12–16). Especially in inter- and transdisciplinary (11, 17–22) as well as participatory research (23–25), scientists involve stakeholders to incorporate nonacademic actors’ views and knowledge (20, 26–29). [Stakeholders are here defined as “persons that, besides their expertise, also have an interest in shaping some aspect of reality because they (...) are a part of it. Stakeholders are e.g. representatives of associations, companies or non-governmental organizations” (30).]

Understanding Ideals and Practices of Stakeholder Involvement in Science

While stakeholder involvement (SI) is well reflected in the context of governance and public participation (31–33), its practices (34, 35) and underlying ideals (36) in scientific research processes that aim at improving knowledge and evidence (37)—rather than at collective decision- or policy-making—are being critically discussed and are yet to be stabilized. In this context, Brandt et al. (38) see a “lack of coherent framing” and “no clear set of tools required for different process phases or integration of different types of knowledge” when working with transdisciplinary approaches in sustainability science. In this paper, we want to address this research gap by substantiating existing analyses of conceptual foundations of SI in sustainability science (39), ranging from the codesign of research processes over the coproduction of knowledge, as well as questions on the science–policy interface, to evidence from current stakeholder practices. [Definitions of codesign differ. We follow that of Moser (40), referring to stakeholders and researchers designing the research process together.] A web-based survey among scholars and researchers engaged with sustainability or transition research was conducted internationally to shed light on the following research questions: (i) What kinds of scientists involve stakeholders and how? (ii) What kinds of ideals underlie scientists’ SI practice? (iii) Do those ideals match the practice? (iv) How do researchers’ ideals concerning SI relate to the types of SI identified in Mielke et al. (39)?

We collected data on scientific fields and researcher profiles as well as on ideals and practices of scientists concerning their understanding of science, the role they assign to stakeholders, their objectives when involving stakeholders, the kind of knowledge they want to gather, and how this knowledge is relevant in the political realm. By using ideal-typical [we refer to Max Weber’s (41) definition of ideal types] answer choices based on the technocratic, the functionalist, the neoliberal-rational, and the democratic type from Mielke et al. (39), we gathered information on how the typology reflects the ideals of practicing scholars. Moreover, we asked whether scientists see trade-offs between their scientific goals and SI. Finally, the survey addressed the question of necessary improvements that could allow scientists to integrate stakeholders better in the future. This paper proceeds as follows: the second section describes the theoretical framework the survey is based on and how it is made operational; in the third section, we present and analyze the responses to answer our four research questions; in the fourth section, we discuss results; the fifth section is dedicated to the methods for data collection and analysis and the sixth section concludes with a short summary and recommendations for practice.

A Framework for SI in Science

To systematize the various approaches of scientists regarding SI, Mielke et al. (39) developed a theoretical framework based on five criteria of differentiation: (i) the role of the scientist, including the stages of the research process where he or she involves stakeholders; (ii) the objectives of SI, including the main reason for involving stakeholders in different stages of the research process; (iii) the kind of knowledge obtained by SI, ranging from data over information and opinions to normative values; (iv) the understanding of science, referring to tools and methods perceived as appropriate by scientists, as well as to epistemic and ontological questions; and (v) the science–policy interface. According to different positions that scientists can take on these five criteria, Mielke et al. (39) derived four ideal types of SI: the technocratic, the neoliberal-rational, the functionalist, and the democratic type. These types will be elaborated in the next section.

SI Typology.

The technocratic type involves expert-stakeholders to receive a broader set of issue-specific, objective, and falsifiable information. The scientist solely defines the research process; its results are expected to inform policy makers, but are not actively promoted. In contrast, the neoliberal-rational type wants to actively promote his research by channeling his results into politics by means of SI. Stakeholders cooperate to influence the public or political arena with a “scientific seal of approval” (42). In this bargaining situation (43), experiential and value-based knowledge (24) can be obtained. The functionalist type perceives himself as a distant observer of “representative stakeholders” of different societal systems—as introduced by Luhmann (44, 45) and others (46–48)—aiming at triggering learning processes through irritation (49). For the democratic type, SI has the objective to integrate actors that are part of a societal transformation into research via dialogue processes that are moderated by the scientist, creating “socially robust knowledge” (50, 51) through codesign (52, 53) and the coproduction of knowledge (54–56). Thus, a “democratic element in the life of science” (57) is introduced.

These ideal types described above partly draw on more prominent classifications, such as those created by Renn (31, 32) and Habermas (58). Renn and Schweitzer classify “structuring processes that channel public input into public policy making” (31) into six prototypes. In contrast, Mielke et al. (39) classify scientist–stakeholder relationships solely in scientific research processes aimed at generating knowledge. Habermas (58) describes the interactions between the subsystems of politics and science with three models—a technocratic, a decisionistic, and a pragmatistic model—thus, defining the relationship of the subsystems in policy-making processes. While the typology used here by Mielke et al. (39) takes this relationship into account with its criterion of the science–policy interface, referring to the influence of science on political decision-making and vice-versa, it specifically concentrates on the sphere of science. [For example, the technocratic type in Mielke et al. (39) is close to the decisionistic model, since he conceives himself as producing knowledge that is relevant for policy makers, but would not—as in Habermas’ (58) technocratic model—take a prescriptive position in political decision-making.]

Making the Typology Operational.

To answer our research questions, we developed a web-based survey with the tool Survey Monkey (https://www.surveymonkey.com), posing 30 questions of varying types [for the advantages of online surveys, see Diekmann (59)]. The scale of measurement ranged from nominal (open and closed questions) to ordinal. To give our respondents the opportunity to bring in their own ideas, we also employed an open “other” category for most closed questions. The survey was comprised of five sets of questions. Table 1 summarizes these questions and relates them to our research questions.

Table 1.

Sets of questions in the survey

Topics	Characteristics queried	Research question
Demographics	Gender (Q1)	1
	Nationality (Q2)
	Level (Q4), type (Q5) and field of education (Q6)
	Place of work (Q7)
SI projects	How often are stakeholders (SH) involved (Q8)	1
	Nature (Q9), topics (Q11), and level (Q14) of projects
	Kind of SH involved (Q10)
	Kinds of funding (Q12)
	Methods (Q13)
SI ideals	Stages of research process in which SH should be involved (Q15)	2
	Role that scientist (S) and SH should play in SI projects (Q17)
	Kind of knowledge that should be produced (Q18)
	Reason for stage of involvement (Q16)
	Main goal of SI (Q19)
	Science–policy interface (Q20)
	Understanding of science (Q21)
SI practices	Role of S and SH (Q17)	3
	Kind of knowledge produced (Q18)
	Science–policy interface (Q20)
Looking ahead on SI	What is needed to improve SI in the future (Q22)	2
	Possible trade-offs between scientific goals and SI (Q24)
	Future involvement of SH (Q23)

The first set of questions covered demographics: for example, nationality and field of education. The second dealt with information on stakeholder projects that respondents carry out, addressing, for instance, topics, funding, and methods used. The third set of questions asked for ideals that scientists have in mind when involving stakeholders in their scientific projects. Here, the questions relate to the five criteria for SI described in this section, whereas the four possible answer items per question reflect the ideal types of SI described above (Table 2).

Table 2.

Making the ideal types operational: Association of answer items and questions

Questions on SI ideals	Answer items related to the ideal types
	Technocratic type	Neoliberal-rational type	Functionalist type	Democratic type
Stage (Q15)	Data collection	Data collection/planning phase/analysis of results/dissemination	Data collection	Data collection/planning phase/analysis of results/dissemination
Reason for stage (Q16)	To increase the extent and quality of data by consulting issue-specific experts	To find out about stakeholders’ interests and feed them into the research process	To test research findings against their perception and practicality in societal spheres	To allow stakeholders affected by the research to give feedback and join deliberative processes
Role (Q17)	Scientist leads the research process; stakeholders are considered issue-specific experts	Scientist is a stakeholder himself and bargains for his or her (scientific) interests in the research process	Scientist observes only from an external position to analyze the perspectives of stakeholders	Scientist facilitates and moderates a cooperative dialogue with affected stakeholders, trying to create trust
Kind of knowledge (Q18)	Objective data and information concerning technologies or scientific problems	Networks and interests of stakeholders	System-specific perspectives and languages	Needs and values of the stakeholders involved
Objective/goal (Q19)	Get better data by involving issue-specific experts	Increase relevance, ensure funding and impact of his/her research	Understand learning processes in science and society	Integrate the perspectives of all actors touched by societal transformations
Science–policy interface (Q20)	Science and policy-making should be two separate fields; policy makers can use the results of scientists	Through the integration of different interests, science can sketch out different paths or courses of action for policy makers	Scientific findings cannot directly be integrated into political decision-making processes but have to be translated by the scientist into information that is useful for policy makers	Science should address the gap between science and society, thus contributing to well-informed, democratically justifiable decisions
Understanding of science (Q21)	It should be autonomous, ethically neutral, and objective	It always depends on perceptions and constellations of the actors that carry it out	It is the societal sphere in which true statements are differentiated from false statements	It should address societal needs and thus support societal transformations

In questions (Q) 16, 19, 20, and 21, respondents could judge the given answer items according to a five-item Likert-scale (59–61), ranging from “strongly agree” (5 on the scale) to “strongly disagree” (1 on the scale). The two remaining questions (Q17 and Q18) only allowed selecting one of the four statements without grading it. Questions 17, 18, and 20 were then each accompanied by an open question concerning the respondents’ actual practice in their projects, comprising the fourth set of questions of the survey. In the fifth part of the survey, we wanted to look ahead on SI in science, asking for improvements of SI, for the future inclusion of stakeholders in projects and possible trade-offs between scientific goals and SI. With questions 26–30, we collected feedback on the questionnaire as well as contact information of participants. In the next section, we will present and analyze our results.

Results: Ideals, Practices and Future Prospects of SI in Science

First, we give an overview of the current landscape in SI as presented in our sample, through, for example, information on scientific fields, scholars, and institutions which carry out or finance research as well as on methods and tools applied. We thereby address the first research question. Second, we describe how respondents positioned themselves concerning ideals of SI in science, addressing the second research question. Third, we summarize the answers on scientists’ practice in their research projects, investigating whether they are in line with their ideals of SI and describing what they perceive necessary to improve SI. This refers to the third research question. Finally, we relate the respondents’ opinions to the typology of SI in science by Mielke et al. (39) to answer the fourth research question.

Current Landscape.

Our sample, which is methodically described in more detail in , consists of German and international sustainability scientists working mostly at universities (39%) (all percentages are rounded), as well as in leading research institutions (31%). The survey was conducted in English. While 64% of respondents were German, we overall reached scholars from 18 different countries (for example, Spain, France, China, Ghana, Iran, and Poland). The majority of respondents are researchers at the early stages of their career: almost 80% are 40 y or younger; 40% hold a Master’s degree, 35% hold a doctorate. The most common field of education is social sciences (57%). While only 37% of the respondents stated to have studied an explicitly interdisciplinary field, such as sustainability science, 64% described their education as “interdisciplinary.” Overall, 73% of the respondents involve stakeholders regularly, for the majority in a transdisciplinary (54%) or interdisciplinary (43%) manner. The stakeholders involved come from a broad spectrum (Fig. 1), with politics at the forefront (84%), followed by civil society (77%) as well as companies (73%). Citizens rank last with 57%. Some respondents specified the types of stakeholders they work with, such as artists, consultants, advocacy groups, faith groups, business associations, and international organizations.

Fig. 1.

Frequencies on Q10: “I work with stakeholders from: science, politics, companies, civil society, citizens, other.” Multiple answers allowed; total respondents: 82. Source: Survey Monkey.

The main research topics that our respondents deal with are energy (52%) and climate policy (42%). This was expected, since our sample contains mostly responses from sustainability researchers. Almost 50% named other topics: for example, coastal protection, agriculture, digitalization, finance and green business, urban development, or corporate sustainability. Respondents mostly receive funding from public institutions: 56% said their research was inter alia funded by national governments, 44% named European institutions, while 26% have foundations as one of their funding sources. Only 16% are financed by companies. Some named other (mainly public) funders, like universities, municipalities, or international entities like the United Nations. Respondents that said they receive funding from companies did not work on climate policy issues. The latter is, however, prominent among public funders: 41–47% of those respondents that receive some funding from public institutions also work on climate policy issues. With regard to methodology, workshops (78%) and interviews (72%) are used most frequently (Fig. 2). Cooperation, in the sense of actively collaborating with stakeholders in projects, ranges third, with 61%, followed by surveys and focus groups. Other methods named were, for example, participatory theater, informal personal exchange, and advisory/consultancy. Especially in research institutes, workshops are highly common (89%). Cooperation with stakeholders is most widespread in consultancies (73%) and in universities (69%). The level at which SI is used is primarily national or local (62% each), while the regional level ranks third with 46%. Supranational and international levels are less common (37% and 23%, respectively).

Fig. 2.

Frequencies on Q13: “I involve stakeholder mostly through...” Multiple answers allowed; total respondents: 81. Source: Survey Monkey.

Ideals.

To investigate the ideals that guide scientists when involving stakeholders, we asked respondents to pick or grade answer options for questions 15–21. In a first step, we looked at the mode and the median of all answers to identify a trend. In a second step, we analyzed “strong agreement” (grade 5) and “strong disagreement” (grade 1) to describe the respondents’ positions in detail (we assembled grades of 1 and 2 as “disagreement” and grades of 4 and 5 as “agreement”).

Stages of the research process.

When asked in which stages of the research process stakeholders should be involved (Q15; multiple answers were allowed for this question), data collection (90%) was the option most respondents chose, followed by the planning phase (87%) and dissemination (81%). Still, around 66% said they would also involve stakeholders in data analysis, which is the furthest-reaching option of involving stakeholders in the research process. Of all respondents, 44% aim to involve stakeholders in all stages of the research process. When asked why they want to involve stakeholders at a certain stage (Q16), the strongest motivation was “to find out about stakeholders’ interests and feed them into the research process,” to which 58% strongly agreed and no one strongly disagreed. The strongest disagreement could be found for the statement: “To allow stakeholders affected by the research to give feed-back and join deliberative processes” (4%).

Role of scientist and stakeholder.

Regarding the scientist’s main role (Q17), respondents had to select one of the four choices. The role of the scientist as a facilitator of dialogues (35%) was the answer chosen most often, followed by the idea of the scientist being a stakeholder himself, bargaining for his interest (27%). Of the respondents, 23% consider the scientist as the leader of the research process, while only 15% think the scientist should be an external observer. This shows a wide divergence of specific roles in SI practices. The different roles are illustrated in Fig. 3.

Fig. 3.

Role of scientist (S) and stakeholder (SH) in the research process. (1) S leads the research process, SH are considered issue-specific experts; (2) S is a SH himself and bargains for his or her (scientific) interests in the research process; (3) S observes only from an external position to analyze the perspectives of SH; (4) S facilitates and moderates a cooperative dialogue with affected SH, trying to create trust. Source: authors’ own illustration.

Kind of knowledge.

The kind of knowledge that is produced in SI processes is a highly contested issue. Nevertheless, the responses to Q18 (“According to your understanding of stakeholder involvement in your scientific field: What kind of knowledge should be mainly produced in stakeholder projects?”; total respondents: 70) were clearly leaning toward finding out about needs and values of stakeholders (43%), followed by system-specific perspectives and languages (30%). When looking at the respondents’ educational background, finding out about needs and values got the highest agreement among natural scientists, of which 60% chose this option. Only 7% of natural scientists seek “objective data and information” from stakeholders. Social scientists are just as interested in needs and values (38%), as in system-specific perspectives and languages, which 36% of them chose as their favorite option. Engineers favor needs and values (48%) and are the only ones who show strong interest in networks (27%). Scholars with an interdisciplinary background, like sustainability science, think that mainly knowledge on needs and values should be produced when working with stakeholders (46%).

Main goal of SI.

The highest agreement could be found for the position that a scientist mainly involves stakeholders to “integrate the perspectives of all actors touched by societal transformations” (55% strongly agreed). Of the respondents, 50% agreed that wanting to get “better data by involving issue-specific experts” is a main goal, while 44% strive to “understand learning processes in science and society.” Interestingly, there was barely strong disagreement with any of the statements, which is also reflected in the mode and median values.

Science–policy interface.

The perception that “through the integration of different interests, science can sketch out different paths or courses of action for policy makers” was the most agreed to answer concerning the science–policy interface. Of the respondents, 43% strongly agreed to this view, and none strongly disagreed; 42% also strongly agreed that science should “address the gap between science and society, thus, contributing to well-informed, democratically justifiable decisions.” The statement that science and policy-making should be two separate fields was the least popular position, with only 9% “strong agreement” and almost 50% “disagreement.” This answer also had low values for mode and median.

Understanding of science.

Of the respondents, 39% strongly agreed to the understanding that science “should address societal needs and thus support societal transformations.” None strongly disagreed with this position; 36% strongly agreed to the view that “science should be autonomous, ethically neutral, and objective.” Since we perceived these two positions to be mutually exclusive, we took a closer look at the individual responses. More than one-third of the respondents agreed to both of these positions (values for mode and median also reflected agreement for these positions). This inconsistency will be analyzed in , below. At the same time, one-fifth of the respondents answered as expected: agreeing to the two statements that lay close together—that science “should address societal needs and thus support societal transformations” and that science “always depends on perceptions and constellations of the actors that carry it out”—and rejecting or being neutral toward the positions that “science is the societal sphere in which true statements are differentiated from false statements” and that “science should be autonomous, ethically neutral, and objective.” The most contested statement was the one that “science is the societal sphere in which true statements are differentiated from false statements”: 34% of the respondents disagreed or strongly disagreed, while only 8% strongly agreed. This statement also had respective mode and median values.

Contrasting Ideals and Practices.

The following section compares scientists’ ideals with their practice. Additionally, we describe the trade-offs researchers see between scientific goals and SI.

Most scientists did not see a mismatch between their ideals concerning the relationship between scientists and stakeholders and their practice in past projects. However, some respondents pointed out that, depending on the project, the stages (“different phases need different relations”), intensities, formats, and research questions, the roles vary and, thus, the practice of SI is nothing static. One respondent perceived the roles “as a continuum.” Another scientist, who considered himself a stakeholder as well and agreed to be bargaining for his/her scientific interest (neoliberal-rational type answer item for Q17 on the roles), reported on the difficulties to accept these new roles for scientists: “The challenge for scientists is to accept the idea that they are not superior to the stakeholders.”

Twenty-one respondents stated that they gained other kinds of knowledge than expected. However, that was not always perceived negatively since some unexpected results were reported to be valuable input. One respondent commented: “System-specific perspectives and languages as well as needs and values were actually produce[d]—but it was important data, too.” The mismatch of expected and experienced kind of knowledge came in various combinations, so that some were hoping to get objective data but instead obtained needs and values, whereas others were looking for needs and values and got knowledge about networks and interests instead. Several respondents hinted at the fact that the kind of knowledge gathered depended on the specificities of the project concerned. Two respondents rejected the idea that knowledge can be obtained at all through SI, stating that “the best that can be achieved is mutual understanding” and that stakeholders provide “confusing perspectives.” However, the majority of respondents did not see a difference in the expected and the actually obtained kind of knowledge in their projects.

When asked about the gap between expectations and experiences in the science–policy interface, several respondents reported a mismatch. One scientist pointed to the learning process that researchers have to go through when using knowledge obtained by SI to consult policy: “It was a joint learning process. The idea that science can educate others unidirectionally is misleading.” In several statements, frustration about too little impact on political decision-making was expressed, as one respondent exemplified: “Though, in general I see other professions much more successful in policy-making than science is, thus, being more successful with b[e]ing heard.” Another saw “no measurable political impact at all,” while a third respondent criticized that “political will is averse to real data when this collides with votes gained or lost.” Several respondents mentioned the need for better communication between the two fields: “Scientific results need to be translated into useful information—also consultants, think tanks, NGO [nongovernmental organizations], journalists can have a role in this translation work.” This was sometimes also related to a lack of resources for better translation and dissemination activities. One respondent emphasized the need for “social scientists in order to carry out the essential qualitative research necessary to bridge science and policy-making.”

Trade-offs between scientific goals and SI.

Forty-three respondents acknowledged trade-offs between scientific goals and SI, while 29 explicitly stated they did not experience any trade-offs. Often (n = 9), scientists pointed to problems of timing that led to “less time for peer-reviewed publications,” saying that SI “reduces written academic output.” This was sometimes weighed against the increase of relevance that might come with successful stakeholder engagement: “[SI] increases—hopefully—the relevance and usefulness of that which is written (and thus also its academic quality).” Besides the time factor, several respondents (n = 11) saw trade-offs between scientific goals and the interests of stakeholders as “the questions relevant to stakeholders do not always match the questions and/or methods that are interesting from a purely academic position.” One respondent stated that when working with stakeholders, “objectivity might be more difficult,” while another pointed out that stakeholders try “to get the results they need instead of results that make sense.” Thus, these respondents think that the autonomy of science is in question when involving stakeholders. Finding the “right” stakeholders was mentioned as being difficult: “Not always desired stakeholders are available and eager to cooperate.” This was problematized especially with regards to hidden motives of stakeholders: “Stakeholder involvement relies on the commitment of stakeholders. If they are not reliable or doing it only for fame, money or other crazy motives, then participatory research is doomed to fail.”

Furthermore, the disciplinary perspective in the “traditional system/alignment of science in universities” was seen as a trade-off that could produce conflicts, which “emerge between different academic disciplines over the quality of data coming out of stakeholder involvement activities (qualitative vs. quantitative).” One respondent recommended making the trade-offs explicit by dealing with them “in a mixed co-operation between scientists and the stakeholders.” Another commented on the quality of SI that is decisive for the existence or absence of trade-offs by stating: “[W]hen it is done well, you can ask scientifically interesting questions that are also interesting and relevant to stakeholders, however, I think there is often a tendency to move towards being service providers to a certain extent and that type of research wi[t]h stakeholders needs to be avoided.” Some hints on how to avoid trade-offs were mentioned, such as reflection about the role of stakeholders (e.g., scientists as “experts for methods in need to collaborate with stakeholders as experts for relevance”), conflict resolution methods, and gender issues.

Whether a scholar sees trade-offs between scientific goals and SI also seems to depend on the kind of knowledge that he or she wants to gather: 73% of those respondents that see trade-offs were either looking for stakeholders’ “needs and values,” the democratic kind of knowledge (42%), or the functionalist kind of knowledge, namely “system specific perspectives and languages” (31%). Scholars looking for “objective data and information” (technocratic kind of knowledge) or “networks and interests” (neoliberal-rational kind of knowledge) were less worried; they only made up 27% of those concerned about trade-offs (15% technocratic and 12% neoliberal-rational). When looking at contingency tables, we found that more respondents from research institutes (61%) recognize trade-offs between their scientific goals and SI than respondents working for universities (57%). Furthermore, of those respondents who work in transdisciplinary projects (52%), the majority (60%) perceives such trade-offs.

Looking Ahead.

To find out what scientists consider helpful to improve their work with stakeholders (Q22), we offered six perspectives: more funding, more academic literature on SI, longer projects, a larger pool of stakeholders, fitting tools and methods for SI, and a network of practitioners. Fig. 4 gives an overview on the respondents’ assessment.

Fig. 4.

Frequencies on Q22: “What would you need to improve your work with stakeholders?” Multiple answers allowed; total respondents: 73. Source: Survey Monkey.

In general, one-third of all respondents want to work more frequently with stakeholders in the future, the majority wants to keep the level of involvement the same, and only one respondent would like to integrate stakeholders less frequently. Not surprisingly, most of the respondents (67%) think that more funding would improve the work with stakeholders, for example, through “funding schemes that support co-design, co-production and implementation of results additional[l]y to the research phase.” Some respondents think that funding for travel costs and other expenditures could increase the motivation of stakeholders to participate; 62% consider longer projects as an essential improvement, adding the importance to follow up on project work by “monitoring societal and sustainable effects after project ended” and getting feedback from stakeholders on research results. Some mentioned the role of time to establish trust and foster commitment on both sides as being crucial for good relationships. Furthermore, “the recognition that many time stakeholders might be interested but have other constraints that do not allow them to participate” was mentioned. This lack of understanding can also be located on the scientific side: “A better understand[ing] of the reasons why we do it—science on its own can’t change society!” As a solution, several respondents point to the development of methods, such as “toolboxes,” for SI that could be integrated into “curricula” or projects. Overall, 55% of respondents perceive the fitting of tools and methods as an important improvement, while only 14% of all respondents were seeking more academic literature on SI.

Conceptualization of Practices.

In our questionnaire, we asked researchers to position themselves regarding the five defining criteria in questions 16–21. For every question, we offered four answer items that each represent a view associated with one of our ideal types: item A for the technocratic, B for the neoliberal-rational, C for the functionalist, and D for the democratic type (Table 2). However, since these types were designed as a heuristic tool to conceptualize debates on SI rather than offering an empirical description of practices, we did not expect respondents to behave in an ideal-typical way. Rather, we wanted to derive common ground and critical points regarding the ideals that guide scientists to answer our fourth research question. We took three steps to analyze the relationship of the typology with the scientists’ answers [for this analysis, we only used complete datasets (n = 59)]. First, we looked at the level of agreement within a type. To do so, we calculated the relative frequency of grades given in all respondents’ type-related answers in questions 16, 19, 20, and 21, which provides the type score. For the technocratic type score, we counted the amount of grades that respondents gave for the technocratic items (grade 1 was chosen 16 times, grade 2 was chosen 29 times, grade 3 was chosen 53 times, grade 4 was chosen 57 times, and grade 5 was chosen 81 times) and divided these absolute frequencies by the amount of all grades given for the technocratic type (n = 59 and four items per type lead to 236). Fig. 5 shows that agreement within the democratic type (81%) was highest, followed by the neoliberal-rational type (76%). Expressed disagreement was highest within the technocratic type (19%).

Fig. 5.

Type scores show the level of agreement within a type across all respondents; agreement in green (grades 4 and 5), neutrality in yellow (grade 3), and disagreement in red (grades 1 and 2). Absolute frequencies (counting the amount of grades 5, 4, 3, 2, and 1 that respondents gave for each type’s items) were divided by the amount of all grades given per type (59 respondents graded 4 items per type, amounting to 236 grades). Source: authors’ own illustration.

Second, we examined the level of respondents’ agreement over all types. It was measured by the absolute frequency of a certain grade per type, divided by the amount of that grade over all types. For example, the distribution score for strong agreement was calculated as follows: over all types and questions, respondents chose grade 5 (strongly agree) 362 times. Of this total, 31% (112) were attributed to some democratic position, while 27% of strong agreement was expressed with regard to the neoliberal-rational (97), 22% (81) to the technocratic, and 20% (72) to the functionalist type. As Fig. 6 shows, overall disagreement was highest for the technocratic type, while agreement was highest for the democratic type.

Fig. 6.

Distribution scores show the overall level of agreement across all types. Agreement or disagreement was measured by the absolute frequency of a certain grade per type, divided by the amount of that grade over all types. Source: authors’ own illustration.

Due to the strong disagreement to the technocratic type, we took a closer look at the technocratic responses. The main reason for the strong disagreement with this type is the statement concerning the science–policy interface (75% of strong disagreement in the technocratic answers) that “science and policy-making should be two separate fields” (Fig. 7).

Fig. 7.

Distribution of strong disagreement with technocratic type. The technocratic main goal had 0% strong disagreement. Source: authors’ own illustration.

This also holds true for the functionalist, where the strong disagreement within the type mainly stems from the rejection of the answer concerning the understanding of science, being the “societal sphere in which true statements are differentiated from false statements” (67% of strong disagreement within functionalist answers).

In a third step, we took a closer look at correlations among the sum scores related to the four ideal types of SI in science to see whether and how strongly researchers’ positions were connected and to evaluate the discriminatory power of the types. Drawing from our typology, we assumed the types to represent certain, internally coherent positions on SI in science. Expressed in terms of the sum scores, by which we measured the overall agreement for a certain type, we would have expected the respondents to have a high sum score for one of the four types and lower sum scores for the others, thus implying negative correlations. This hypothesis was only partly supported by the data depicted in Table 3: while we found a significantly negative correlation between the technocratic and the democratic positions (−0.29), we found positive correlations between the technocratic and functionalist positions (0.28) as well as between the functionalist and the neoliberal-rational positions (0.24). The negative correlation between the democratic and technocratic sum scores shows that a person who takes a democratic position on SI in science tends to reject the technocratic view on SI and vice-versa. We discuss this observation in the following section.

Table 3.

Correlations among the respondents’ sum-scores across all types

	Technocrat	Neoliberal-rational	Functionalist	Democrat
Technocrat	1	0.06	0.28*	−0.29*
Neoliberal-rational	0.06	1	0.24†	0.15
Functionalist	0.28*	0.24†	1	0.2
Democrat	−0.29*	0.15	0.2	1

Significant at the 5% level.

Significant at the 10% level (P value of 0.080).

Discussion

The data collected in our survey give an overview of current practices and ideals in SI and show important trade-offs when involving stakeholders, ranging from time conflicts over the possible loss of the autonomy of science to quality conflicts concerning the research results. However, the picture of current practices of SI in sustainability science drawn from our data might be biased due to the socio-demographic structure of our sample: most of the 89 scientists are in earlier career stages (age between 20 and 40 y), and work with stakeholders in inter- or transdisciplinary projects (this might be due to our snowball sampling procedure). Even though the majority of respondents were of German nationality (64%), our results are transferable to and relevant for a broad international scientific audience. Roughly half of the respondents work on projects that are carried out at the supra- or international level, while about the same number receives funding from European Union institutions. Furthermore, the scientific standards discussed herein are shared by the global sustainability science community.

Concerning the links between researchers’ ideals of SI and our typology, a discussion of results is necessary. While we found the democratic and the neoliberal-rational perspectives to be most prominent among our respondents, the level of agreement was quite high across all types. This hints at scientists using hybrid forms of SI, which also becomes apparent in the fact that many respondents agreed to three or more of the four options offered in our typology questions, and thus to three or more types. Especially concerning the reason for the stages at which stakeholders are involved and the scientist’s main objective, the majority of scientists showed mixed conceptions. Table 4 summarizes this pattern.

Table 4.

Percentage of respondents who agreed to three or more options of questions 16, 19, 20, and 21 (of n = 59)

Question	Topic	Percent of respondents, %
16	Motivation to involve stakeholders at a certain stage of the research process	85
19	Main objective	71
20	Science–policy interface	51
21	Understanding of science	27

This result could be due to several reasons. The positions offered might have been perceived as being unclear. This became especially apparent in the question on the understanding of science (Q21), where more than one-third of the respondents agreed to both the democratic position that science “should address societal needs and thus support societal transformations” and the technocratic view that “science should be autonomous, ethically neutral, and objective” at the same time. Three respondents specifically referred to the answer choices in Q21 as being not mutually exclusive, ambiguous, and too similar.

The results also hint at a lack of conceptual clarity among practitioners, especially on the question concerning the understanding of science. Another reason for the high agreement to seemingly mutually exclusive positions might be that scientists work with stakeholders in different, sometimes even contradicting ways at the same time, taking diverse roles within different stages of the research process (addressed in Q15 and Q16; here divided into planning phase, data collection, analysis of results, and dissemination), or having varying understandings of science (Q21) in different projects. Both topics relate to discussions on codesign and coproduction in transdisciplinary research processes and are highly disputed in the literature (see e.g., refs. 11, 54, and 55 for a discussion of challenges). While authors like Cornell et al. (56) opt for a highly integrative stakeholder approach that includes “collective problem framing” and “societal agenda setting,” others like Lang et al. (17) take a more moderate position, by emphasizing “collaborative problem-framing” as well as “co-creation of solution oriented and transferable knowledge.” The difficulty of aligning these goals, roles, and understandings is reflected in the trade-offs presented the section, above.

Nevertheless, we were able to find interesting correlations. First, there was a significantly negative correlation between the technocratic and the democratic position (–0.29). Since the data showed a strong negative tendency for the technocratic answer to the science–policy interface question, we tested this item’s correlation with the democratic sum score and found an even stronger negative correlation of −0.39. This only partly corresponds to the typology used. Mielke et al. (39) acknowledge a distance between democrat and technocrat concerning the science–policy interface, but the neoliberal-rational type is designed to be furthest away from the democrat on this matter (see “bargaining vs. deliberation” in ref. 39).

Second, we found a positive correlation between the technocratic and functionalist position (0.28). This is reflected in the typology, as these positions were designed to be closest to each other concerning the science–policy interface (see “bargaining vs. deliberation” in ref. 39) and regarding the understanding of science (see “autonomy of science” in ref. 39).

Materials and Methods

To reach our respondents, we used a snowball sampling technique (62–64) as a first step. We accessed scientists that were already in contact with us, and then asked these scientists to pass on our survey within their networks. Additionally, we approached networks of sustainability scientists ourselves. The contacts included sustainability scientists working in leading research institutes for climate, environment, and economics (e.g., Potsdam Institute for Climate Impact Research, Helmholtz-Centre for Environmental Research, German Institute for Economic Research, Fraunhofer ISI, National Aeronautics and Space Research Centre of the Federal Republic of Germany), as well as in respective departments in universities (e.g., University of Bielefeld, Eberswalde University for Sustainable Development, Freie Universität Berlin, Technical University Berlin, Leuphana University). We also addressed different networks in which these scientists are associated [e.g., Förderschwerpunkt Sozialökologische Forschung (a group of scientists that is funded by the socio-ecologic program of the German Ministry of Education and Research; https://www.fona.de/en/society-social-ecological-research-soef-19711.html), Strommarkt-Verteiler (a German network of energy professionals in academia, policy-making, industry, and nonprofit organizations; www.strommarkttreffen.org/english)], as well as relevant foundations (e.g., Mercator and Böll-Foundation) and nongovernmental organizations (e.g., Germanwatch), which deal with sustainability transitions. The survey was online from July 7 to November 15, 2016 and was closed after 89 responses. Since the data were anonymized, informed consent procedures and approval by an ethics committee were not needed.

To make sure our items were constructed consistently within the types, we calculated item-total correlations (65) for each type. [Diekmann (59) uses the item-total correlation test to find out whether items show another dimension that leads away from the intended one, attributed to all items (“Fremddimension”).] The range for the technocratic type’s items was between 0.51 and 0.83, for the neoliberal-rational from 0.42 to 0.55, for the functionalist from 0.38 to 0.56, and for the democratic type from 0.61 to 0.75, showing internal coherence of the types constructed. To test reliability, we used a split-half reliability test (59), showing values between 0.61 and 0.72 for the technocratic, functionalist and democratic items, respectively. Only the scale for the neoliberal-rational type had a negative reliability measure (–0.08). Following Kim and Stoel (66), we consider measures above 0.5 as being reliable. The low value for the neoliberal-rational type is assumed to be due to the item “understanding of science.” To ensure content validity (59), we reviewed literature on sustainability science, interviewed experts in the field, and performed a pretest with practitioners. The survey and the full data sheet that the analysis is based on as well the tests developed are provided in and Dataset S1.

In our analysis of the current landscape and practices, we mostly used relative frequencies and qualitative interpretation for open questions. The percentages relate to the number of responses within each question (e.g., Q1 had 88 responses, Q3 had 87). For the Likert-scale questions, only respondents that replied to all four items of a question were counted to ensure comparability. Furthermore, we employed contingency analysis (67) as a multivariate statistical method to investigate interconnections among the scientists’ positions on different criteria and concepts of SI. For a deeper analysis of our ordinal data in the Likert-scale questions [there is an ongoing scientific debate whether Likert-scales can be interpreted as interval data; we have followed the interpretation of Diekmann (59)], we calculated: (i) type scores for the technocratic, neoliberal-rational, functionalist, and democratic items to check the agreement within a type; (ii) distribution scores to see how agreement and disagreement were distributed across the types; and (iii) correlation coefficients (correlations used were significant at the 1% or 5% level) (59) among the sum scores to investigate how the types were related. We calculated the sum scores for the each type by summing up a respondent’s answers for the respective type’s items in questions 16, 19, 20, and 21. Thus, they reflect the level of agreement to the respective type.

Conclusions

The findings presented in this paper offer an overview of current practices and ideals of scientists working with stakeholders. The survey shows that SI has become a common practice in inter- and transdisciplinary research projects and that there is common ground on how it should be carried out. While a broad array of nonacademic actors is involved, stakeholders from politics and civil society are at the forefront. Mostly, stakeholders are involved through workshops and interviews or in cooperative processes. Although energy is the most frequent thematic issue, the topics are broad, ranging from agriculture over resource efficiency to climate policy and mobility. Our respondents involve stakeholders at all stages of the research process, but find data collection and planning the most prominent points for involvement. When looking at ideals, the main role of the scientist is seen as being a facilitator of dialogue. The kind of knowledge that is supposed to be obtained consists of needs and values of the stakeholders involved, which corresponds with the most agreed-on main goal of integrating the perspectives of those actors that are affected by transformations. In relation to the policy world, scientists want to use SI to better sketch out paths for policymakers. The respondents most strongly agreed that science should address societal needs and support transformations. Nevertheless, divergences between scientists’ ideals and their practices concerning SI became apparent. Respondents wish for political impact, but many consider it being limited when looking back on their projects, partly due to different expectations of stakeholders and scientists, a lack of motivation on the stakeholder’s side, a lack of funding, or follow-up on results. Many also see a trade-off between their scientific goals and SI, saying that SI gives them less time to concentrate on their academic publications, admitting to not always be sure how the knowledge obtained through SI can be scientifically used, and fearing to be influenced by stakeholders to a point that threatens scientific autonomy. Respondents also indicated a need for improvement, mainly hoping for increased funding, more time, and better-fit methods. These trade-offs and improvement needs hint toward a lack of conceptualization in SI. The latter also became apparent when we related the survey results to the typology established in Mielke et al. (39). Although we found a preference toward the democratic and the neoliberal-rational type, there was high consent with all types, even among those that were designed to be mutually exclusive. This underlines the need for further qualitative research on SI as well as for conceptual tools for scientists that involve stakeholders. Heuristic conceptualizations like the typology can help to reflect on trade-offs before conducting research and, thus, may help to resolve some of the conflicts scientists named in our survey. Since respondents called for better-fit methods, there should be more training (4, 68) on how to perform SI in scientific research at universities and within projects. Moreover, the repercussions that SI might have on research questions, codes, language, tools, and methods should be better reflected, as scientific practices and concepts change over time.