Participatory research in times of COVID-19 and beyond: Adjusting your methodological toolkits.

Solving grand environmental societal challenges calls for transdisciplinary and participatory methods in social-ecological research. These methods enable co-designing the research, co-producing the results, and co-creating the impacts together with concerned stakeholders. COVID-19 has had serious impacts on the choice of research methods, but reflections on recent experiences of "moving online" are still rare. In this perspective, we focus on the challenge of adjusting different participatory methods to online formats used in five transdisciplinary social-ecological research projects. The key added value of our research is the lessons learned from a comparison of the pros and cons of adjusting a broader set of methods to online formats. We conclude that combining the adjusted online approaches with well-established face-to-face formats into more inclusive hybrid approaches can enrich and diversify the pool of available methods for postpandemic research. Furthermore, a more diverse group of participants can be engaged in the research process.

Introduction

The outbreak of the COVID-19 pandemic in early 2020 disrupted societies and economies across the globe. The pandemic has also had serious effects on the global scientific research community, as laboratories and offices have closed, academic events have been canceled, mobility for students and faculty has been restricted, and scientific careers have had to be put on hold. ,

Switching to online formats has been discussed as a solution to some of the pandemic-related challenges, but according to recent studies,4, 5, 6, 7, 8 researchers vary in their assessments of how successfully online formats can replace face-to-face formats for different research activities. According to a recent survey, most researchers believe that online formats are suited to handling a large share of the administrative work and to replacing project meetings, which will also apply after the pandemic. However, assessments regarding fieldwork, where personal interaction with stakeholders is essential, are much more pessimistic. Approximately one-third of the respondents believe that fieldwork cannot be replaced at all by online formats in the future, and another third state that only up to a quarter of all fieldwork might be performed online.

These concerns call for solutions to adapt and enrich our current methodological toolkits for fieldwork in order to mitigate negative effects of the COVID-19 pandemic. This is particularly relevant for transdisciplinary research, which requires close collaboration between researchers and stakeholders outside academia and the use of participatory methods for data collection and analysis. , Participation in this context means that researchers and stakeholders become essential partners in the co-design of the research, co-production of knowledge and results, and co-creation of impact. Although participation also comes with certain challenges (e.g., substantial time investments required on both sides, risk for stakeholder fatigue, difficulties in maintaining stakeholder engagement over longer time periods, biased representation of different stakeholder groups, and management of unrealistically high stakeholder expectations), , enhancing societal relevance and legitimacy through stakeholder engagement processes is considered a worthwhile and crucial aspect of transdisciplinary research. This goes beyond informing and consulting with stakeholders but requires regular involvement and genuine collaboration, with specific participatory methods to support stakeholder engagement.

Although there is already a surge in the literature of guidance as to how one can effectively replace physical face-to-face meetings with online formats, including first overviews of the literature, much of this advice remains generic and does not take into consideration how online formats influence the roles participants might take in the research process of specific tools and methods. In addition, little is known about potential changes in the empowering nature of participation, such as increased self-efficacy and direct uptake of results by stakeholders. ,

Thus, the objective of this perspective is to share and discuss our recent experiences with adjusting participatory methods used in transdisciplinary social-ecological research. We draw on rich experience from several ongoing projects (one in Costa Rica and four others in different parts of Europe), all of which explore innovative and more sustainable ways of promoting ecosystem services provision and biodiversity conservation in agricultural and forest ecosystems. On the basis of the pros and cons identified through several rounds of shared reflection, we elaborate on the lessons learned from adjusting our methods and discuss how this knowledge can enrich and diversify the pool of available methods after the pandemic. We conclude that combining the adjusted online approaches with well-established face-to-face formats into more inclusive hybrid approaches can even enrich and diversify the pool of available methods for the time after the pandemic.

Participatory methods in transdisciplinary research

Transdisciplinary research is a reflective approach that encompasses interdisciplinary collaboration among researchers of different scientific disciplines and stakeholders from practice concerned with a particular real-world problem. The aim is to integrate different bodies of knowledge in order to produce context-tailored solutions that are both scientifically sound and socially relevant. Stakeholders can be individuals, groups of individuals (e.g., citizens, farmers, policy makers, citizen movements), or organizations (e.g., government agencies, private enterprises, nongovernmental organizations [NGOs]), which either can affect or are affected by the identified real-world problem. In research on social-ecological systems, stakeholders typically include a multitude of actors with diverse interests from different societal spheres (civil, public, private), economic sectors (agriculture, forestry, conservation, tourism, industries, etc.), and governance levels (local to global). In transdisciplinary research projects, such heterogeneous actors work closely together to co-design the research agenda, implement it, and work toward practical solutions. Frequently, practice partners also become formal partners in the project. To allow meaningful involvement, participatory methods are needed throughout the research process (Figure 1 ).

Figure 1

The transdisciplinary research process

Sources: Authors’ own design, adapted and complemented after Lang et al., with icons from The Noun Project (www.thenounproject.com) shared under Free Creative Commons licenses, created by Larea (phase A) and Gregor Cresnar (phases B and C). The arrows on the left and right sides have slightly different designs, because in science, scientific discourse typically informs tangible results such as publications, which can directly be built upon for follow-up research, while in practice, outcomes might be less tangible, such as improved decision making or governance.

This is a significant departure from conventional research, as it questions the perceived superiority of scientific knowledge and actively invites non-academic stakeholders to contribute practical and administrative knowledge where researchers give up the role of sole knowledge providers and become co-learners. Recently, more and more funding programs (e.g., the European Union’s [EU] framework program Horizon 2020, BiodivERsA) explicitly design transdisciplinary research programs and score research proposals partly on the basis of the adequacy of proposed methods for stakeholder involvement in data collection and analysis throughout the entire research process. In this context, different participation levels , can play a role (Figure 2 ).

Figure 2

Participation levels and their relevance for transdisciplinary research

Source: Authors’ own design on the basis of four participation levels as defined by Durham et al.

For transdisciplinary research all levels have their roles, but particularly involvement and collaboration (levels 3 and 4) are aimed at, calling for methods that allow a genuine knowledge exchange and a two-way communication flow. Collaborative face-to-face formats generate new relationships, bridge multiple ways of knowing, and illuminate the role of values and principles, therefore creating the basis of an extended validity of research results. This has been the reason why mainly face-to-face formats are prioritized in transdisciplinary projects. Four main objectives of stakeholder involvement are considered: (1) “normative” objectives based on the principle of democracy to allow all concerned stakeholders to engage and represent their diverse value commitments; (2) “substantive” objectives to improve the quality, validity, and significance of the results for them (e.g., by considering all available knowledge types); (3) “social learning” objectives to build a shared problem understanding and mutual trust and to allow empowerment of all actors; and (4) “implementation” objectives to increase ownership, acceptance, and legitimacy of created solutions to maximize impact and trigger social-ecological change.

As a result of the pandemic, the pressure is high to find online alternatives for face-to-face formats that still allow pursuing these objectives. The loss of face-to-face interaction is likely to negatively affect all four objectives, as connecting to all stakeholders becomes harder and the quality of interaction might be less thorough, not offering the same possibility of building rapport, empathic listening, and generating a “thick” description of the problem. , , Stakeholders need good reasons to invest their time and know-how, and their withdrawal from the research process becomes more likely when they are frustrated with the inadequacy of methods in use.

Against this backdrop, we describe the adjustments we made because of COVID-19 to six selected methods, currently applied in five research projects, and reflect on how this has affected the interaction between researchers and research participants, on the basis of our perspective as researchers. To do so, we took a three-step methodological approach, described below.

Methodological approach

In the first step, we organized a group discussion to select suitable research projects. We opted for the five research projects listed in Table 1 , because they are all highly transdisciplinary in nature and pursue all four objectives of stakeholder involvement, as listed in the previous section. All require high participation levels for genuine collaboration between involved researchers and practitioners (up to levels 3 and 4; cf. Figure 2). Thus, using participatory methods is crucial for the interaction between researchers and research participants in order to achieve project objectives and address specific real-world problems.

Table 1

Overview of research projects and respective methods included in this perspective

Project short description, dependency on/selection of participatory methods, and project status when methods needed to be adjusted	Project further details: funding scheme, project run time, website, case studies, involved practice partners (and if they are formal project partners), and transdisciplinary approach
Contracts2.0 aims to co-design, test, and implement novel contracts to better incentivize farmers for the increased provision of environmental public goods in agricultural landscapes. Therefore the project implements a multi-actor approach and brings together stakeholders from practice and policy to innovate agri-environmental and climate schemes in response to real-world needs.Dependency on participatory methods: Very highSelected methods: Net-Map, Q-methodology, public goods games, living labsStatus when methods were adjusted: Year 1 (at this time, some stakeholder contacts were already established in person, so a minimum level of trust was already present when methods were adjusted to online formats)	Funding scheme: EU/Horizon 2020Run time: May/2019 to April 2023Website:www.project-contracts20.euCase studies: Belgium, Denmark, France, Germany, Hungary, Italy, Spain, the Netherlands, United KingdomPractice partners involved as formal project partners: Yes. Practice partners include NGOs and associations, regional municipalities, a national park administration, public authorities, public-private partnerships, and small and medium-sized enterprises.Transdisciplinary approach: In Contracts2.0 as an EU “research and innovation action” researchers and practitioners closely collaborate in so-called contract/policy innovation labs (so-called CILs and PILS) to co-design “dream contracts” for agri-environmental and climate measures. The testing and actual implementation of the co-created solutions within the project duration is aimed at. Initiated activities are to be continued by the practice partners after the project ends.
InnoForESt fosters the development and implementation of innovative policy, management, and business solutions to increase the delivery of forest ecosystem services. The project employs a multi-actor approach, closely collaborating with stakeholders in the forestry sector.Dependency on participatory methods: Very highSelected methods: Net-Map, living labsStatus when methods were adjusted: Year 3 (At this time, stakeholder contacts were already in place, adjustment of methods was mutually decided on and realized for the continuation and finalization of the last running tasks. These involved also the “road-mapping” for the continuation of initiated activities beyond the project end as well as final dissemination activities, e.g., through stakeholder workshops and a final conference, all held online.)	Funding scheme: EU/Horizon 2020Run time: November 2017 to December 2020Website:www.innoforest.euCase studies: Austria, Czech Republic, Finland, Germany, Italy, Slovakia, SwedenPractice partners involved as formal project partners: Yes. Practice partners include forest and environmental administrations, wood-processing and consultancy firms, hunting associations, forest managers, forest owners, and NGOs.Transdisciplinary approach: In InnoForESt as an EU “innovation action,” researchers and practitioners work together to “up-grade, up-scale and replicate” already existing innovations for the improved governance and valorization of forest ecosystem services. The majority of project resources are dedicated to implementation, while research accompanies this process. An important aspect is to enable co-learning across case studies.
PEATWISE aims at developing sustainable soil and water management technologies. It furthermore investigates options for incentivizing these management technologies in a long-term mitigation framework for greenhouse gas emissions.Dependency on participatory methods: Very high for one work package (but not for project as a whole)Selected methods: Net-MapStatus when methods were adjusted: Year 3 (At this point in time, stakeholder contacts were already established. Adjustment of the method was realized by research partners, tailored to stakeholder needs.)	Funding scheme: FACCE/ERA-GASRun time: November 2017 to April 2021Website:www.eragas.eu/en/eragas/research-projects/PEATWISE.htmCase studies: Denmark, Finland, Germany, the Netherlands, Norway, SwedenPractice partners involved as formal project partners: No. All formal project partners are research institutes/universities. The project works with farmers, water boards, NGOs, companies, and policy makers from the local, regional, and national levels.Transdisciplinary approach: PEATWISE implements a transdisciplinary approach to incentivize management options that reduce peatland emissions. Therefore it analyzes existing incentive-based policy instruments in three case-study countries, in order to develop coherent strategies which bundle governmental and private-sector initiatives.
PlanSmart explores innovative approaches for addressing waterborne challenges through planning and implementing nature-based solutions in river basins and urban regions.Dependency on participatory methods: Very high (funding format on social-ecological research)Selected methods: GeodesignStatus when methods were adjusted: Year 4 (At this stage, all stakeholder contacts were already established. Adjustment of the method was realized by research partners, tailored to stakeholder needs.)	Funding scheme: BMBF (SÖF/FONA³ junior research group)Run time: April 2016 to March 2021Website:www.plansmart.infoCase studies: Germany, Costa RicaPractice partners involved as formal project partners: Yes. Practice partners include water and environmental administration and ministries, river basin organizations, municipalities, and NGOs.Transdisciplinary approach: PlanSmart uses a transdisciplinary approach to work closely together with practitioners in the case-study regions of the Lahn River in Germany and the Grande de Tárcoles River in Costa Rica. This collaboration enables knowledge co-creation and co-development of scientifically sound and practice-relevant guidelines to boost nature-based solutions in river management.
SIMTWIST aims at simulating tourism water consumption with stakeholders through a participatory modeling approach.Dependency on participatory methods: Very high for one work package (but not for the project as a whole)Selected methods: Participatory modelingStatus when methods were adjusted: Year 1 (At this time, stakeholder contacts were partly established, with face-to-face interactions already realized. Adjustment of the method was done by the researchers on the basis of stakeholder needs.)	Funding scheme: Water JPIRuntime: June 2019 to May to 2022Website:www.simtwist.euCase studies: Italy, SpainPractice partners involved as formal project partners: No. All formal project partners are research institutes/universities. The project cooperates with stakeholders from the tourism sector.Transdisciplinary approach: SIMTWIST implements a transdisciplinary approach to support social learning through different scenario-building and modeling techniques. Stakeholder involvement is needed throughout the entire life cycle of the project to achieve meaningful results.

In the second step, we selected six methods applied in one or several of the chosen projects, namely, Net-Map, Q-methodology, participatory modeling, geodesign, public goods games, and living labs. We selected these six methods because from the pool of methods used in the projects, they represent the ones with the greatest potential for stakeholder participation (up to level 4), and adjusting them to online formats carries the highest risk for seriously affecting the quality of our research.

Table 1 provides a short description of each project, its dependency on participatory methods in general, the specific methods applied, which practice partners are involved and how relevant their involvement is given the transdisciplinary approach of each project, and the point when methods required adjustment to online formats because of COVID-19.

In the third step, to reflect on the pros and cons that resulted from the online adaptation of each method, we followed a sequence of sub-steps.

First, method descriptions were generated in a joint writing process by all co-authors involved in the application of the different methods, outlining the purpose of application, how the method was originally applied and how it was adjusted because of COVID-19, how practice partners participated in the process, and which pros and cons were linked to the adjustment. Second, these descriptions were reviewed by all authors in preparation for a second group discussion, where the authors reflected together to identify common patterns that formed the basis for synthesizing the pros and cons across methods. Third, method descriptions were refined and complemented, and the pros and cons section was drafted, again followed by a joint group discussion aimed at distilling generalizable aspects for the “lessons learned” section. Fourth, the “pros and cons” section was finalized and the “lessons learned” section further developed, followed by a fourth group discussion to discuss last amendments as well as the illustrations. Finally, the pre-final version of the perspective was reviewed again by all authors.

All group discussions were held online. The development of the perspective was done in Google Docs, which allowed us to work in parallel on the different sections.

Adjustments made to methods in methodological toolkit

Below, for each method, we first provide a general description of how it was typically used in the projects before the COVID-19 pandemic, then describe the adjustments made under COVID-19 restrictions (see also Figure S1), and, finally, detail the pros and cons that resulted from these adjustments.

Net-Map

Net-Map22, 23, 24 is an interview-based tool for social network analysis (SNA). It seeks to reveal the perceptions of different stakeholders on formal and, more important, informal relationships within a governance network. To do so, interviewee and interviewer work together to co-create maps of the actors' network, typically using pen and paper. The process consists of four steps: (1) identifying relevant actors, (2) visualizing the interactions between them (indicating, e.g., exchange of information and resources, trust, or conflict), (3) assigning actors' interests and motivations, and (4) measuring their influence and benefits. While drawing the network, interviewees engage to justify why each actor is crucial or elaborate on existing conflicts and how they are mitigated. Thus, depending on the preference of each research participant, he or she either engages actively in the drawing or takes a more passive stance and instructs the researcher for the drawing. All explanations are audio-recorded to support interpretation. For the analysis, collected network data are visualized using SNA software (e.g., UCINET/NetDraw), while audio recordings are transcribed and analyzed with qualitative content analysis software (e.g., MAXQDA). Results are fed back into group discussions in which stakeholders discuss their different perceptions. This helps participants be more aware of their own positions and function in the network in relation to others and pinpoint weaknesses in the current structure, such as bottlenecks in information flows. On the basis of this information, stakeholders can create strategies on how network collaboration can be improved.

Net-Map is used in the projects Contracts2.0, InnoForESt, and PEATWISE to explore governance networks for innovative agri-environmental management. As face-to-face interviews were not possible because of COVID-19, we had to find a substitute for the pen-and-paper visualization of the network, usually co-created during the interviews. This was realized by using MURAL, a digital collaboration platform that allows mutual visualization (www.mural.co) in combination with video-conferencing software. This combination allows real-time online collaboration, where both interviewer and interviewee can work on the network visualization simultaneously. Graphically, MURAL can closely mimic the pen-and-paper version (see Figure S1A for an example of Net-Map).

Although switching to MURAL allows us to continue using Net-Map in the projects, it also has some negative implications. First, in contrast to the pen-and-paper version, more preparation is needed to create the virtual canvas and provide guidance for interviewees on how to use the technology. Also, body language cannot be assessed in order to confirm that visualization is satisfactory. Furthermore, recruitment of interviewees tends to disadvantage participants who prefer face-to-face to online interaction. We noticed that the interviews are shorter on average, possibly because they are less engaging for interviewees. Correspondingly, interviewees tend to provide fewer explanations, while the technicalities of implementing the network visualization take up more time. This has the effect that the adjusted method yields less rich information for network interpretation. However, the technology offers new possibilities for the visualization, such as icons usable by drag and drop to express different motivations of mapped actors. Besides, it is easier to correct errors in MURAL than on paper, where multiple amendments easily make a network representation “messy” and harder to interpret later on.

Q-methodology

Q-methodology seeks to identify attitudinal profiles about a particular topic. , Understanding attitudes is critically linked to intentions and behavior under widely recognized theoretical frameworks such as the theory of planned behavior or other models aiming to predict people's choices. Q-methodology gains a participatory character by using semi-structured stakeholder interviews. Participants are asked to rank opinion statements, which are displayed on individual cards, into a Q-grid. In doing so, participants must provide reasoning for the placing of statements in the Q-grid and thus also defend their viewpoints. Through factor analysis, researchers compare individual sorting patterns and arrive at a small number of common factors, representing subjective viewpoints. In that respect, Q-methodology is a relevant tool for identifying discourses, enhancing knowledge and common visions among parties that could facilitate social participation and trust. As data from Q-methodology are easy to gather, the method is also considered a useful participatory exercise.

Q-methodology is used in Contracts2.0, PEATWISE, and PlanSmart to understand (1) stakeholder attitudes toward label-based approaches for the provision of ecosystem services, (2) farmers' perceptions toward peatland management, and (3) subjective viewpoints of political-administrative decision makers regarding the implementation of nature-based solutions. Typically, Q-interviews are conducted face to face, using printouts of statements and the Q-grid. Because of COVID-19, interviews were carried out using video-conferencing software or telephone, in combination with an online programmed version of the Q-sorting exercise, such as the freely accessible HtmlQ Q-sorting tool (note that there are other—potentially more user-friendly—commercial software alternatives, such as qmethodsoftware [https://qmethodsoftware.com]).

Implementing the method without personal interviews has several implications: first, online or telephone interviews save travel time and facilitate more interview appointments, and second, setting up the software online affects relative costs. Although in the short term, the initial cost of setting up the software (in terms of programming and becoming familiar with the platform) is high, it becomes easier to adapt the software for additional studies, resulting in comparatively lower long-term cost. This helps increase the number of interviews and include participants from more geographical areas, and it allows additional replications (i.e., covering an extra country). Access to technology and technological affinity with the online tools used remain major challenges for both the interviewer and the interviewee. Testing various web browsers and different video-conferencing software can be time consuming. Another limitation is that misunderstandings are more difficult to detect and resolve than in face-to-face interviews, as explicit body language is only partially visible.

Participatory modeling

Participatory modeling is a co-creation method in which selected stakeholders are actively involved by integrating local knowledge and making it available for processes related to decision making. In general, participatory approaches involve stakeholders in one or more stages of the modeling process, ranging from data collection, through interviews, to model development (referred to as group model building), testing, use, and actual application to inform decision making and policy design. , For instance, during the stage of group model building, selected stakeholders participate to construct system-dynamic models informed by negotiations and discussions among them. This enables team and social learning, which provides stakeholders with an increased sense of ownership of the research.

The participatory modeling research agenda of the SIMTWIST project includes the active participation of stakeholders through the entire life cycle of the project in (1) interviews to draw in situ conceptual maps, (2) workshops for group model building, and (3) workshops for feedback and validation in order to produce an agent-based model. The level of participation ranges between “involve” and “collaborate”, aimed at participatory and consented decision making at the end of the project. Because of COVID-19, almost all of these activities have been adjusted to online formats using video-conference platforms combined with MIRO, an online collaborative platform (www.miro.com), making it challenging to maintain the participatory core of this approach. Because the pandemic started at the end of the first year, only step 1 of one case study was carried out face to face.

The aim of the personal interviews (step 1) is to co-create concept maps on flip charts with sticky notes, drawing relationships between elements, identifying feedback loops, and producing an aggregated map agreed by all stakeholders. However, for some online interviews, screen sharing did not work technically, so interviews did not produce the interaction normally associated with this step. As the interviewee could not see the map the interviewer was drawing, more communication was necessary to describe the map to the interviewee and answer questions. Also, interviewees could neither make corrections nor draw the map themselves. Both issues diminished the intended level of involvement. Compared with personal interviews the process took longer, and stakeholders were not always available for such a long time.

Furthermore, the workshop planned for group model building (step 2) had to be adjusted into an online format. This meant reducing the interaction time by more than half to avoid overwhelming participants and substantially narrowing the originally planned activities for one full-day face-to-face workshop. The use of online platforms is another limitation, as many of the stakeholders are not familiar with them. Therefore, we developed a very detailed user manual for participants. During the online workshop, stakeholders presented their concept maps produced at the interviews, but it was not possible to carry out the group model building. As an alternative, we redesigned the methodology to select the most important common topics of all maps, which were then prioritized by stakeholders through voting, and we evaluated the consensus of the prioritization list. In this online setting, it is difficult to build trust with stakeholders who are meeting for the first time, and the interaction among them is limited. This also negatively affects opportunities for social learning, and motivation tends to drop fast. Although the team saved time and cost because of less travel, the main goal of the workshop was not achieved, and several extra activities needed to be planned to obtain what the workshops were initially planned to deliver.

Geodesign

Geodesign is a design and planning method that combines the creation of design proposals with impact simulations informed by geographic information, systems thinking, and online technology. In a participatory workshop, participants work on different tasks (e.g., locating landscape features on a spatial map and assessing related spatial effects on land use). The results are discussed and adapted in cooperation with the participants, and often a consensus map is created. The method serves as a decision support tool for environmental planning. ,

In the PlanSmart project, a Geodesign workshop in a Costa Rican river case study was planned for May 2020. To implement it, we designed a Geodesign tool on the basis of ArcGIS/CommunityViz, where participants in smaller groups use a touch-screen interface to spatially locate perceived problems and possible nature-based solutions within the river landscape and afterward discuss their choices. Because of COVID-19, we had to organize the workshop as an online event using the MAPTIONNAIRE online tool in combination with video-conferencing software.

At this stage of the project, a first field visit and explorative interviews with a number of the stakeholders had already been done, but the knowledge co-creation on spatial aspects in a second field visit and face-to-face workshops had to be adapted. In terms of transdisciplinary level, the stakeholders were highly involved, as they actively had to fulfill and discuss tasks and provide resources and data. For collaborative participation, more active decision making on future activities would have been necessary.

The online version distanced the researchers from the field as well as workshop participants from each other, despite the use of breakout groups. It took time for participants to understand and use the online tool, even though the participants had good information technology (IT) skills and the tool provided simpler spatial location functions than the original tool developed with ArcGIS/CommunityViz. IT skills and accessibility should ideally be confirmed beforehand, as people without access or with low IT skills will struggle to join the online interaction. Direct contact with and among stakeholders is essential to build trust and foster participation. European researchers can save travel costs and time, but because of the difference in time zones, they must work in the late evening.

Public goods games

Public goods games are a commonly applied methodology to study cooperation in experimental economics (for a meta-analysis of lab studies, see Zelmer). There is a quickly growing literature on applying and co-designing public goods games with field populations, also involving workshop formats. In a standard linear contribution mechanism of public goods game experiment, n players receive an endowment e, which they must allocate to a private and public account. Contributions to the public good are multiplied by a constant a (1 < a < n) and redistributed to players equally to reflect the positive externality occurring from the public good. As players only internalize a fraction a/n of their own contribution, free riding and not contributing to the public good constitute a unique Nash equilibrium that is at odds with the social optimum of contributing everything.

In the Contracts2.0 project, we used public goods games to study cooperation among farmers under different governance mechanisms to evaluate the scope of agri-environmental measures that build on group contracts rather than individual contracts. The goal was to organize a series of workshops to co-design public goods games with key decision makers and farmers. A list of treatments commonly applied in laboratory studies was used as a starting point. For instance, we wanted to discuss how heterogeneous endowments (some players have more than others), different group sizes, sanctions, or rewards would affect contributions to the public good.

Although the workshops to co-design the games could be moved online successfully, gathering a large number of farmers to pair them for real-time interactions to conduct the experiments was not an option. Popular events such as fairs, which we planned to include in our recruitment efforts, have been canceled. Instead of focusing on the dynamics of repeated interactions and real-time feedback, we shifted the focus to a large-stake, one-time interaction. In other words, multiple rounds of the game were replaced with a single round. Farmers receive payments and feedback on the basis of an ex post matching with other participants of an online survey. In addition, we apply the “strategy method” for sequential games (i.e., participants decide for various possible scenarios). Although we lose the opportunity to investigate dynamics of cooperation, moving the study online also helps us to reach a larger number of participants and to gather more data.

Living labs

Living labs are an innovative approach and phenomenon in which societal and ecological challenges are addressed from a multi-actor and participatory perspective. Living labs aim to develop an open innovation environment in real-life settings with the purpose of co-creation, validation, and testing new inputs for action over time. , Living labs are transdisciplinary spaces that combine different sources of knowledge to address a common goal. When establishing living labs, certain principles are followed, including continuity, openness, realism, empowerment of users, and spontaneity.

The living lab approach is used in the projects Contracts2.0 and (partially) InnoForESt to realize a collaborative multi-actor approach. In InnoForESt, living labs are used to establish innovation platforms to facilitate co-development and implementation of innovative governance approaches to secure forest ecosystem services. In Contracts2.0, labs involve the creation of contract innovation labs (CILs) and policy innovation labs (PILs) with the aim of offering an open and inclusive space for stakeholders to explore novel contract designs for agri-environment and climate measures among farmers. The CILs put farmers' and local stakeholders' real-life needs at the core of research, and stakeholder participation is used to co-create new contract designs. In parallel, the PILs bring together policy makers and experts to evaluate the broader socio-political context and to discuss institutional changes to better embed the novel contract designs.

Because of COVID-19, the originally planned face-to-face workshops in combination with field visits had to be adjusted to an online format with group meetings and bilateral discussions. However, the online format makes it challenging to maintain many of the key aspects of living labs, such as fostering participation, trust, creativity, and spontaneity. For instance, the schedule and duration of online meetings are fixed in advance, so there is less leeway to spontaneously extend conversations beyond the planned time. Also, the lack of informal spaces for dialogue tends to reduce bottom-up dynamics and limits the opportunity to take advantage of stakeholder initiatives. Altogether, it is harder to maintain the momentum of engagement and network building without face-to-face meetings. It will be impossible to reach stakeholders who value personal interaction above all, because they are hesitant to use online tools for sharing internal reflection usually expressed in a natural conversation. This is aggravated by the fact that participants with limited technological equipment and skills are excluded from participation. Nonetheless, online formats also have positive implications for the living labs, including that (1) introverted participants can use the chat feature instead of participating in public discussions, (2) participants learn new IT skills, and (3) contribution of experts from other geographic locations is possible, making the online event less costly and time consuming.

Synthesizing the pros and cons across methods

On the basis of the selected method examples presented above, we find that adjusting the original face-to-face to “online” formats in times of COVID-19 has some positive aspects in terms of time requirement and technology use. Negative aspects relate to lower data quality and impact generation. Our results mirror what others have found: “going online” entails important trade-offs. Thereby, different reference frameworks are used to make such assessments in the literature, for instance, focusing on the practicalities of conducting online research with specific methods in a certain field of research or pulling findings together to generalize and enable a broader comparison in a literature review. We did not notice an impact of the project's time stage on method adjustment. In all instances, at least some contacts were established before the pandemic, and none of the projects had to start from zero.

Below, we discuss both negative and positive implications of the online application of the concrete set of selected methods, which should also provide concrete guidance for researchers interested in these methods. Therefore, we distill our insights into four main topics: time requirement, technology use, data quality, and impact generation. These topics, although they emerged empirically, can be linked back to the four main objectives of transdisciplinary research: while available time and technology impacts link to the “normative” and “social learning” objectives by providing (or not) sufficient opportunities for the representation of and collaboration across diverse knowledge types, data quality is interlinked with the “substantive” objectives. Furthermore, impact generation is closely related to the “implementation” objectives. Moreover, these categories also allow us to make an aggregated assessment for a comparison of the selected methods regarding these four topics (Table 2 ).

Table 2

Assessment of negative (down arrows) and positive (up arrows) impacts across all methods

Methods	Assessment of impacts on
	Time requirements	Technology use	Data quality	Impact generation
Net-Map	↓↓↓ additional time needed for setup↑↑↑ ease in replication	↓ additional IT/software skills needed↑↑↑ improved outreach	↓↓ less interaction↑ better formalization	↓ missing personal exchange
Q-methodology	↓↓ additional time needed for setup↑↑↑ ease in replication	↓↓ additional IT/software skills needed↑↑↑ improved outreach	↓↓ less interaction↑ better formalization	↓↓ missing personal exchange
Participatory modeling	↓↓ additional time needed for setup↑↑ ease in replication	↓↓↓ additional IT/software skills needed↑↑↑ improved outreach	↓↓↓ less interaction↑↑↑ better formalization	↓ missing personal exchange
Geodesign	↓ additional time needed for setup↑ less travel required	↓↓↓ additional IT/software skills needed↑↑↑ improved outreach	↓↓ less interaction–	↓↓ missing personal exchange
Public goods game	↓↓↓ additional time needed for setup↑ ease in replication	↓ additional IT/software skills needed↑ improved outreach	↓↓ less interaction–	↓↑ (neutral)
Living labs	↓ additional time needed for setup↑↑ less travel required	↓↓↓ additional IT/software skills needed↑↑ improved outreach	↓↓ less interaction + inclusion of key partners harder–	↓↓ missing personal exchange

Three arrows, strong impact; two arrows, moderate impact; one arrow, low impact. The text next to the arrows refers to the aspect to which the main impact is attributed.

Time requirement

In general, we find that taking the methods online requires significantly more preparation time for the researcher when organizing a method for the first time, easily adding up to 4 to 8 weeks. This additional time is needed for researching and testing suitable online tools and platforms, and once a choice is made, the actual adjustment for implementation, generating new templates, and providing clear instructions for research participants (referred to as “setup” in Table 2). Also, the researcher needs extra time to become proficient in the application of the new method, and source information about the prospective participants' IT skills, access to hardware and software, and broadband internet. However, we see this effort as a one-time investment, which pays off in the long run, where learning by doing ensures optimization over time. Additional time savings occur because no travel is undertaken. For the research participants, we find that online application results in shorter interaction time on average, which is in contrast to other findings (e.g., Howlett reported that conversations lasted longer). This can point to higher efficiency for data collection (e.g., through replication in another context, once data collection tools have been designed) but also can imply that it is harder to uphold participants' engagement online over a longer period, which in turn limits the empowering nature of participatory methods. Additionally, less time could also mean a higher cognitive burden, as interviewees are asked to perform the same task in a shorter time.

Technology use

The application of most online methods depends on the use of a (video)-conferencing platform in combination with a specific method-related software. In order to fit the original methods with the selected technology, they need to be more structured and formalized and sometimes also simplified. To ensure successful participation, identifying participants' preferences, IT skills, access to technology, and adequate broadband connectivity, as well as a backup plan in case of IT problems, is crucial. Here, participants who have a clear preference for face-to-face interactions or lack the necessary skills and technology will be reluctant or unable to participate. , To mitigate this, researchers can act proactively by providing or helping download the technology, offering one-to-one short meetings beforehand for try-out, or starting meetings earlier to provide enough time to solve technical issues. If participants can be enabled to participate, it can open up learning fields for them to acquire new skills. There is also the possibility to meet more frequently for shorter sessions, as well as to arrange more meetings, allowing shorter intervals in favor of more reflection time between meetings. The use of online technology can also have the advantage that more participants from different locations can join, including participants with little likelihood of ever meeting one another in person. However, time differences between locations can be an issue when scheduling online interactions.

Data quality

As online interaction tends to be shorter, it may not take full advantage of the empowering nature of participatory methods, such as picking up new skills and increased self-efficacy. In addition, the higher degree of formalization needed to make the interview or workshop fit the capabilities of the selected technology can limit the depth of data collection. In consequence, both researchers and participants might be less satisfied with the interaction. In an online setting, the researcher is less able to build rapport, and participants' body language or gestures cannot fully be taken into account. Hence, obtained data might be less rich and carry fewer meanings for data interpretation and validation. Furthermore, the researcher cannot verify obtained information in situ through field visits, while participants might feel less ownership of the generated data, because the co-creation process may be limited. This suggests that results might be less robust if collected through online processes. To mitigate negative effects on the co-creation process, triangulation of different methods can be used, but at the expense of additional time and resource needs for the people involved. Another option is to build in additional feedback loops (e.g., by validating results in short one-to-one phone calls after the online event). A plus for participants might be the opportunity to “multi-task” while participating (e.g., a Dutch farmer who participated in a living lab meeting plowed his field at the same time). Nevertheless, technology use can also improve data quality, as immediate corrections are possible. Last, data quality may increase because of the higher degree of formalization, as data gained might be less prone to interviewer bias and more comparable.

Impact generation

In terms of lasting impacts generated for research participants, the most serious limitation relates to the importance of regular face-to-face interaction to build and maintain a high level of trust and social capital. This cannot be easily replaced by online interaction, so trust and social capital might be affected when no personal exchange takes place over longer periods, resulting in less buy-in and enthusiasm to stay engaged. This links back to the validity criteria discussed above. Participants also might feel less inclined to initiate something from the bottom up, as a higher degree of formalization and preparation from the side of the researcher leaves less leeway for doing so. This makes the interaction appear more top down, with limited participation options. This limitation might be easier to address for participants who already know one another and have personal relationships. Also, online meetings are generally more focused on content and achieving event objectives. This makes it harder to replace the casual personal exchanges that naturally take place during breaks in face-to-face meetings and that are essential to connect on a personal basis and to extend networks. To recreate such opportunities, online formats need careful planning. We also found that it is harder to express appreciation online, as one cannot bring presents or food, and creativity is needed to identify online substitutes (e.g., sending a music link via Spotify, providing a voucher). For instance, in the Dutch workshop for the planning of the public goods games, we sent chocolate and tea to participants, which was greatly appreciated by everyone and arguably helped increase the level of engagement. And in the Q-methodology study, printed material was sent to participants beforehand, which helped demonstrate researchers' engagement and to establish a trustful relationship. Nevertheless, the researcher has less control over this environment, and it is difficult to enforce a level playing field for all involved stakeholders in online interaction at all times. For example, some participants might be unwilling to interact with a camera or express their opinions in the chat. At the same time, participants might feel more in control and thus more comfortable in their home environments, as they decide where to put the camera. Also, online interaction creates numerous new opportunities for them, such as learning new skills by engaging with online tools and building networks that go beyond their usual reach. Furthermore, it helps extend the reach to geographical areas that researchers and participants would typically not travel to, because of time and budget constraints, at the same time saving greenhouse gas emissions.

Table 2 presents an aggregated assessment to visualize the differences across the methods. Linking back to the four objectives of stakeholder involvement, we can conclude that the “normative” and “social learning” objectives of transdisciplinary research can mostly be achieved in online settings if time and technology are used wisely, while the “substantive” and “implementation” objectives are at higher risk for failing to be met if a transdisciplinary research is fully converted to the online space.

Lessons learned

As discussed above for the selected methods, adjusting face-to-face to online formats has pros and cons. For the “time after COVID-19,” when it is no longer a “must” to use online formats, but a deliberate choice, we recommend intentionally taking more advantage of the pros when opting for online formats, while being reflective and trying to minimize the cons.

Developing online formats for existing methodologies that are traditionally based on face-to-face interaction considerably enlarges the pool of possible methods and creates more flexibility for their application. Online formats allow us to try out new and innovative technology and build new skills, but this might require extra efforts to provide clear instructions and manuals to support learning and knowledge building. Online formats help extend the scope for networking activities and connect stakeholders who are not able to travel because of time and budget constraints. When opting for online formats, careful planning is imperative, also accommodating time for informal exchanges. This is even more crucial if participants do not know each other personally.

As the dependency on technology increases, so does the risk that something might go wrong. Having a plan B (and C) in place is crucial (e.g., sharing your screen if participants are not able to join MURAL to draw on the canvas themselves, so they can at least follow while you draw) but requires more flexibility from participants. As some participants might still be excluded for different reasons (e.g., hardware or software, internet access, IT skills, time differences), this should be mitigated to reduce bias in the sample by foresighted project planning dedicating enough resources for IT training, providing suitable technology, or recruiting extra personnel with expertise in online formats. In order to allow a genuine level of participation in online formats, special attention should be given to facilitation. Here, working in tandem can be beneficial, where one facilitator takes care of the technicalities, while the other gives full attention to facilitating in-depth interaction between participants.

To prevent participants’ getting the notion that the researcher is not well prepared or that the participants' input is not properly acknowledged, because it is “just” another online event for the researcher, a clear effort is needed to show appreciation (e.g., by creating “online” presents). To ensure robustness of results, triangulation of methods can be used in combination with allocating more resources to validating results in short one-to-one phone calls as follow-up to the online events. Researchers should be aware that online formats have differentiated impacts for different methods: for instance, the Q-methodology interviews were not strongly affected, whereas the public goods games had to be redesigned from scratch.

Overall, transdisciplinary and participatory research have been widely affected by COVID-19, as we have experienced across the different projects aimed at co-developing more sustainable solutions for ecosystem services provision and biodiversity conservation in Europe and beyond. Adjusting our methodological toolkit to online formats can enrich and diversify the pool of available methods. Within the methodological toolkit, online formats are increasingly important because of COVID-19 and might become sustainable solutions in their own right after the pandemic. Online formats can also offer greater potential to better include “unusual” stakeholders less considered so far, such as younger generations (“digital natives”), who have a higher inclination to use online tools. Altogether, making more use of online formats calls for a shift in our perceptions of researcher-stakeholder interaction to ensure that our research activities are still carried out with the same ambition for stakeholder engagement and to take full advantage of the empowering nature of participatory research. As researchers, we should also train ourselves to be more alert for the unexpected ways in which participants might benefit from online formats, such as being more at ease in their home environment, having their family and/or pets around, or feeling a more symmetrical relation with us, as we also reveal something about our private lives. In online formats, stakeholders actually can also get a glimpse of our home environment, which they never get to see when we perform our field work the “traditional” way, as we as the researchers usually travel to their locations. Thus, face-to-face formats can certainly be replaced with online formats to a certain degree to save time, financial resources, and CO2 emissions resulting from travel. However, not all activities can be substituted, because they are still the essence of building trust between researchers and stakeholders. Hence, a balance needs to be found between both formats. Here, hybrid applications could be explored further in the future, combining both face-to-face and online formats. This could include a setting where both formats are used alternately over time or one where for some participants the researcher-stakeholder interaction is taking place online, whereas others are together in person locally. The latter would allow local in-person interaction and at the same time help reduce international travel and environmental impact.