Qualitative systems mapping for complex public health problems: A practical guide.

Systems mapping methods are increasingly used to study complex public health issues. Visualizing the causal relationships within a complex adaptive system allows for more than developing a holistic and multi-perspective overview of the situation. It is also a way of understanding the emergent, self-organizing dynamics of a system and how they can be influenced. This article describes a concrete approach for developing and analysing a systems map of a complex public health issue drawing on well-accepted methods from the field of social science while incorporating the principles of systems thinking and transdisciplinarity. Using our case study on HIV drug resistance in sub-Saharan Africa as an example, this article provides a practical guideline on how to map a public health problem as a complex adaptive system in order to uncover the drivers, feedback-loops and other dynamics behind the problem. Qualitative systems mapping can help researchers and policy makers to gain deeper insights in the root causes of the problem and identify complexity-informed intervention points.

Introduction

In recent years, systems thinking methodology is increasingly used to study health systems and complex public health problems [1, 2]. Researchers and policy makers around the globe are more and more aware of the need to shift away from reductionist cause-effect approaches towards a systemic understanding of public health issues [3]. Health systems may be conceptualised as complex adaptive systems (CASs), which entail a set of diverse, interrelated factors and which are characterized by dynamic behaviours such as emergence, self-organization and the formation of feedback-loops [4-6]. In a CAS, positive interventions in one part of a system, may have undesired effects in other parts of the system, depending on the paths that exist in the system and choices and events that happen along the way. This phenomenon is called path dependence.

Despite the rising interest in systems approaches, literature on the topic remains dispersed and a common jargon is yet to be developed [1]. Moreover, the available literature on complex systems approaches in the field of public health has remained largely theoretical. A commonly used method to visualise, understand and analyse a CAS is systems mapping. Systems mapping has been used to study diverse public health problems such as obesity, vaccine hesitancy and neglected tropical diseases [7-9]. The term systems mapping comprises a set of different methods for visualising and analysing complex adaptive systems. Depending on the exact nature of the research question, a different type, or combinations of types, of mapping can be used. One of the most used types of systems mapping is causal loop diagramming [1]. This is a qualitative approach in which the causal relationships between factors are represented. Connections between elements are directed and can be positive (both elements evolve in the same direction) or negative (both elements evolve in the opposite direction). An often-used example of a causal loop diagram is that of the heating and the thermostat (Fig 1). When the room temperature drops to a certain point, the thermostat will automatically increase the heating (negative causality). When the heating is on, the room temperature will increase (positive causality) up until the point in which the desired temperature is reached.

Fig 1

Causal loop diagram example.

Thermostat room temperature regulation as a (simplified) example of a causal loop diagram.

While in this manuscript we primarily focus on the mapping of causal loop diagrams, there are also other types of mapping such as stock and flow diagrams, which is a more quantitative approach to systems mapping and can be used to study the dynamic behaviour of a system over time, or social network analysis which aims at visualising and studying the relationships between social actors [10, 11]. Such visualisations of CASs help to gain deeper insights into the dynamics of complex problems and to develop a shared understanding between different stakeholders in order to come to a nuanced understanding of the complexity of the situation. Systems dynamics and types of modelling have been thoroughly described by Sterman (2000) [12]. Systems mapping aims to do more than integrating the perspectives of different stakeholders. It uncovers emerging dynamics which are built up of more than the sum of the elements involved and which would likely have remained uncovered if a linear approach were adopted. In some cases, the process of developing a systems map may be more valuable than the final product itself. In such cases, participatory practices such as group model building sessions may be used. Different stakeholders then come together to jointly develop insights and search for solutions while mapping the system [13-15]. However, group model building sessions are not always the most desirable or feasible option. For example, the topic under investigation might be highly stigmatised in the community, therefore not allowing participants to speak freely during a group model building session, the participants perspectives on the topic may be too diverse to organize a common discussion (e.g. technical experts vs family and friends), or participants might live in different parts of the world, making a physical meeting organizationally challenging [16, 17]. In some cases, individual interviews are also preferred when one aims to understand the individual mental models of stakeholders separately before generating an integrated overview of the combined viewpoints [18]. In this article we provide some practical guidelines and reflections on how to use systems mapping as a means to collect and investigate rich, complex data on public health challenges in order to integrate different perspectives and gain a deeper understanding of the complex systems dynamics. We use a case study concerning HIV drug resistance in sub-Saharan Africa as a practical example to clarify how complex data can systematically be collected, mapped and analysed while incorporating the principles of transdisciplinarity every step of the way (Fig 2) (Table 1).

Fig 2

Graphical abstract.

Overview of the described methodology, consisting of four iterative building blocks and continuously requiring the researchers to adopt a transdisciplinary approach and to be aware of their disciplinary biases.

Table 1

Overview of the guideline and the timing of each step throughout the process.

Step	Description	Timing
1	Data collection
1.1	Participant selection
1.2	Preparing and conducting semi-structured interviews	May lead back to 1.1
2	Data analysis and mapping
2.1	Interview analysis	May lead back to 1.1 and 1.2
2.2	Coding	May be in parallel with 2.1
2.3	From codes to systems map	After 2.2
2.4	Setting systems boundaries	After 2.3, though one can reflect about this throughout the data analysis
2.5	Determining the depth of the system	After 2.4, though one can reflect about this throughout the data analysis
2.6	Simplifying the system	After 2.5
3	Analysing the system	In parallel with 4
4	Exploring the system dynamics	In parallel with 3
5	Continuous transdisciplinary reflexivity	Throughout the whole process

Case study

The methodology discussed in this paper is illustrated by a study on the complex adaptive system of factors influencing HIV drug resistance (HIVDR) in sub-Saharan Africa [19, 20]. Although antiretroviral therapy (ART) is available, allowing people living with HIV to live a long and healthy life, the increase in HIVDR is threatening the success of the available therapies. HIVDR arises when the ART present in the body is insufficient to suppress the viral load, creating selective pressure which allows the virus to mutate in order to escape the effect of the therapy. This situation can be due to irregular adherence of PLHIV to their therapy which on its turn has many other possible causes. The aim of the study was to gain detailed insights in the underlying dynamics of factors influencing HIVDR and to identify suitable intervention points. To this purpose two systems maps were developed: one visualising the complex adaptive system of factors influencing HIVDR as understood by experts on an international level and one visualising the system at local level for a study site in Dar es Salaam in Tanzania. These systems maps were informed by interviews with 15 international experts, 12 PLHIV in Dar es Salaam and 10 local actors, who through their daily activities regularly come into contact with PLHIV in the study site. The findings of these studies are described in two separate publications [19, 20]. In the next sections we use examples from this case study to illustrate our guide.

Guidelines

1. Data collection

1.1. Choosing a data collection method and participant selection. The first consideration to make is which way of collecting data is most suitable for the topic under investigation. As already explained in the introduction, there are different reasons (both methodological and practical) to opt for either group modelling sessions or individual interviews. This guide focusses specifically on the mapping and analysing of complex data collected by semi-structured interviews. Participants should be recruited with the aim of obtaining a full picture of all aspects of the system. Next to interviewing patients and healthcare workers, one might therefore also consider interviewing people who are somewhat further removed from the core problem but are still in touch with certain parts of it. For example, architects designing certain hospital area’s relevant for the topic under investigation or religious leaders who provide spiritual support to patients could contribute unique insights into the topic. Next to the interviews themselves, other types of data such as participant observation and document analysis could also be used to triangulate the data and increase the validity of the results.

In our case study we opted for collecting the data for our systems map through individual semi-structured interviews. This had a dual reason. First, for our mapping at international level, we wanted to obtain broad insights in all possible factors influencing HIVDR. Individual interviews were chosen to give us the opportunity to collect deep insights in the specific expertise of the interviewee while verifying or building further on information obtained from previous interviews. For our local map, individual interviews were preferred for another reason. As people living with HIV still face strong stigmatisation, we wanted to create a safe environment for them to speak their mind, without other community members present. Moreover, a workshop would likely only have attracted PLHIV who felt comfortable with their HIV status and were facing less difficulties adhering to the therapy, while for the interviews we also managed to recruit PLHIV who had dropped out of care.

1.2. Preparing and conducting semi-structured interviews. A semi-structured interview guide should be developed based on the available scientific literature or already existing and validated guides on the topic, and adapted throughout the data collection process when new insights are developed. In order not to bias the data collection towards certain assumptions the researcher may have, it is advised to start the interview with a broad, open question about the complex problem, rather than asking a question about a single aspect of the problem. This way, the participants are inclined to start by expressing the aspect of the complex problem most important to them and the interviewer can explore the main believes and experiences the participant has to share about the topic. For example, an initial question like “how do you experience being HIV positive” may reveal to the interviewer that the patient’s whole perception of his or her HIV infection is based on the believe that it is a punishment of God. This information is important for the interpretation of the rest of the interview and may not have come up if the interview had started with a focus on a certain aspect of the system, such as the question “how do you perceive the healthcare service you receive?”. After this first question, the interviewer may continue covering a list of specific topics, retrieved from the literature or which came up in previous interviews. When a question is answered by “A happens because of B” the interviewer can ask for specific examples or experiences that support this claim and subsequently delve deeper into other possible underlying causes aiming to obtain the structure “A happens because of B, which is caused by C, D and E, etc.” This continues until a sufficient level of depth is reached or until the insights of the interviewee are exhausted, at which point the chain of causality may be built up further during interviews with other participants. Such chain of causality is built several times within one interview, each time starting from an open question. To further reduce bias, the interview circumstances should be well thought-through in order to create trust between the interviewer and interviewee. For example, interviews with PLHIV are best done in their native language and in a location that cannot be perceived as stigmatizing.

2. Data analysis and mapping

When developing a systems map based on the interview data, the first steps are largely similar to conventional interview analysis methods. The first steps of our data analysis method are inspired by the Qualitative Analysis Guide of Leuven (QUAGOL) which provides clear steps for capturing the rich insights in complex qualitative data [21, 22].

2.1. Interview analysis. Interviews should be recorded and transcribed verbatim. Throughout the interview process, the research team should have regular debriefing sessions to allow for modifications of the interview guide if needed. This can for example be the case when a new relevant topic comes up, which needs to be further investigated in the following interviews. Ideally, technical reports are written after each interview, describing the context of the interview, possible technical issues, characteristics of the participant and possible cultural clarifications important for the full comprehension of the data in their specific context. After thoroughly reading the transcripts, a series of meetings is organized between the research team in order to discuss the interpretation of the interviews and to make sure cultural elements are well-understood. It is advised to include researchers from different disciplinary backgrounds in the team, in order to prevent disciplinary bias in the analysis of the data.

2.2 Coding. Once an interview and its core messages are well understood, the coding process can start. Coding can be done with professional programmes such as NVivo or in an excel table. The researcher keeps a list of all elements that were mentioned as a direct or indirect cause of the problem under investigation and of each link between two of those elements. For further analysis purposes, other types of data can be stored behind each element or connection. In Table 2 we explain the types of data that can be stored behind one element, using the element “accessibility of healthcare centre” as an example.

Table 2

Coding examples.

Data type	Explanation	Example	Note
Element Name	The factor directly or indirectly influencing the problem under investigation.	Accessibility of healthcare centre
Definition	It is important to define the element and what is included or excluded in order to facilitate the interview coding.	Accessibility refers to road access, public transportation, road safety, transport costs, limited opening time, poor access due to other disease outbreaks or wars etc. Distance to healthcare centre is considered a separate element.
Number of Interviews	The number of interviews a certain element or connection was mentioned in.	7	This element has been discussed in 7 out of 22 interviews.
Interview Identification Number	The identification number of the interviews in which a certain element or connection was discussed.	For example: I01, I03, I04, I08, I09, I13, I15	(Fictive identification numbers are used due to confidentiality reasons).
Quote	The interview quotes in which the element or connection was described. Storing this information in the systems map will facilitate the analysis as all the quotes linked to a certain element can easily be revisited.	"Sometimes I don’t get a bus fare but I borrow somewhere because I must go for refill. When few drugs for two or three days remain, that is when I go to refill my drugs. I must go the same date written on my card by the health care providers so as I may not confuse them. If it is written fifth I must go to refill, so even if it is from my neighbour I borrow one thousand shillings so as I go to the facility to refill my drugs."	This is one quote given as an example. During data collection, all quotes relevant to this element would be collected here.
Tag	The opportunity to categorise elements.	Healthcare system related	This allows the researcher to easily filter out all elements related to a certain topic, in this case healthcare system related factors.
Other…	Several other types of data (for example: degree of importance) can be stored, depending on what may be useful during the analysis process.		For example, a degree of importance as judged by the interviewee could be given to elements based on how the elements or connections were described in the interviews. However, as this is a subjective indicator, it is advised to always use this parameter in combination with other ones when drawing conclusions.

Examples of types of data which could be retrieved during the coding process and stored behind elements and connections of the systems map. We illustrate with an example of our study on HIVDR.

2.3. From codes to systems map. While keeping the codebook updated after analyzing each interview, it can also be of interest to make a separate systems map of each transcript, visualizing the mental model of the interviewee in order to understand how he or she perceives the system (Fig 3). Mental models are graphical representations of how people internally understand causal relationships between elements to make sense of a complex problem [23, 24]. They often unconsciously affect our behavior or decision making and are useful for the researcher to gain a deeper understanding of the interviewees way of thinking about the problem [25].

Fig 3

Mental model example.

Example of a mental model of an interviewee, visualizing the elements and connections which came up during the interview and which are perceived to be true by the interviewee. The researcher tried to bring some first structure in the model by using a color code.

In a next phase these schemes can be merged manually or automatically by simply uploading the codebook in the used mapping tool. In our case study we used KUMU, a user-friendly online mapping tool which allows the storage of different types of data behind each element and connection and which has some built in analysis tools [26]. Once the codebook is imported, some immediate structure can be brought into the map by for example coloring or grouping the elements according to a common parameter or sizing the elements by number of occurrences. This structure will most likely be changed at a later stage in the analysis process when new insights are gained. Eventually, the researchers may opt to retain different visualizations of the model to highlight different structures. In the case of HIV drug resistance in sub-Saharan Africa, we developed one visualization showing how the elements relate to different societal layers, whereas the second visualization highlights the main dynamics of the system (Fig 4). In both visualizations each element is a factor influencing HIVDR as mentioned in the interviews and each connection indicates the relationship between those factors. An overview of all elements and connections is included in these maps as well as an interactive version of the systems maps where the reader can zoom in and click on elements and connections for more information is included in the supplementary files (S4 File) [27].

Fig 4

Different ways of visualizing a system.

The elements and connections in A and B are exactly the same. In A the system is organized according to the different layers ranging from biology on the micro level to the individual level, the social context, the healthcare system and overarching factors at the macro level. In B, the elements are divided in thematic clusters and the relationships between clusters are visualized. Figure adapted from Kiekens et al. [19] and for illustrative purposes only.

When a first basic structure is reached, we suggest to revise all elements and connections in order to avoid the same concept being visualized in different ways inside the map. For example, a pathway representing the difficulties PLHIV may face reaching the clinic due to their economic status may be represented as “economic status -> retention in care” or as “economic status -> ability to pay transportation -> retention in care”. Especially when the coding is done by more than one researcher, the codebook may contain such double pathways. To resolve this, the research team has to come to a common agreement on how to visualize such concepts.

2.4. Setting systems boundaries. Throughout this process the researcher can also start to set system boundaries, determine the level of depth the CAS will be represented in and simplify the system. In reality, the boundaries of CASs are often blurry as different CASs are interlinked and systems are constantly evolving [4]. For example, while our case study aimed at covering a public health issue (HIVDR), we realized throughout the study that our system is strongly interlinked with other complex systems such as poverty (e.g. having financial means to reach the healthcare center and food insecurity influencing adherence as medication needs to be taken with a meal). However, when visualizing a system, some choice in what to include in the system and what not, needs to be made. Though it might seem tempting to set boundaries at the beginning of the project, the authors recommend starting without boundaries in mind and representing the CAS as detailed as possible. While more time consuming, the advantage is that the possibility of excluding important factors due to preset limits is reduced and the researchers gain deep insights in all aspects of the system before analyzing it or reducing it to its essence if needed. Boundaries can be set in different ways depending on the information the map needs to transmit. For example, one can decide to consider the factors which are not part of a closed system as being exogenous, meaning they only have influence on the system but are not influenced by the system. For example, in Fig 4B, all endogenous elements are part of a closed feedback loop (they influence and are influenced by the system), whereas the exogenous factors (indicated in yellow), are exerting an influence on the system but are not influenced by the system (e.g. “gender inequality” influences “HIV status disclosure” and “HIV transmission” but is not influenced by any element in the system). Another way of determining the boundaries of the system could be to view all elements that form the core of a different CAS as exogenous factors (for example: gender inequality, poverty and war and disease outbreaks are all complex problems on their own, which are interlinked with our complex problem).

2.5. Determining the depth of the system. The depth of the system refers to the level of detail a system is represented in. Issues surrounding stigmatization of PLHIV could be separately represented as “stigmatization”, “self-stigmatization”, “gossip and discrimination” or as one common term such as “Stigma and discrimination” (Fig 5). Again, this depends on the research question and purpose of the systems map.

Fig 5

In-degree.

Illustration of mapping choices to be made by the researchers and the consequences for the in-degree metric.

2.6. Simplifying the system. Additionally, the systems map might need to be reduced or simplified to a smaller, more manageable system that is understandable for external stakeholders. In the rest of this paragraph we suggest some strategies for the simplification of systems maps. Other strategies (possibly topic dependent) could also be used. More important is to consequently apply the strategy to the whole systems map. When in doubt whether two elements should be merged or not, we suggest the researcher asks two questions: 1) are there significant differences in nuance between the content of both elements? And 2) do both elements have different connections to other elements? If the answer to both questions is “No”, the elements can be merged into one. Moreover, elements that have only one incoming and one outgoing connection (A->B->C) might be deleted and taken up into one connection from A to C (A->C), unless element B is crucial for the understanding of the system. When several loops are present, loops sharing a same broader theme can be summarized into one. This can be compared with a route on a roadmap [19]. When one wants to know the route from Paris to Brussels, there are several options. All the options pass by different towns but they all have one common theme: they represent ways to go from Paris to Brussels. Bundling these loops or pathways between two elements, may help to drastically simplify the map and to visualize only the core essence. Fig 6 is an example of a holistic, detailed system (A), summarized into its core feedback loops (B).

Fig 6

Summarizing a complex system.

A) Detailed system of factors influencing HIVDR. The main feedback loops or subsystems are highlighted with colored circles. B) The same system, condensed into the main feedback loops and with the main exogenous factors represented on the outside. Each cluster in panel A is represented as a single element in panel B, represented with the same color in the core of the element. All connections between two clusters in panel A are represented as one connection in panel B. This way, the main dynamics of the system are represented in a more condensed and comprehensible format. Figure adapted from Kiekens et al. [19] and for illustrative purposes only.

3. Analysing the system

In fact, the analysis of the CAS starts during the mapping process itself. Throughout the mapping process, the researcher will start to identify certain characteristics of the system. These could be for example reinforcing or balancing loops, time delays or clusters of elements or connections sharing the same characteristics. Though the analysis is foremost qualitative, involving a continuous (re)-reading of interview quotes or relevant literature, some quantitative elements may support the interpretation. McGlashan et al. propose some quantitative network analysis metrics and describe how to interpret them when applied to systems maps [28]. The in-degree describes the number of incoming connections (the number of elements influencing the element of interest). The higher the in-degree, the more the element is directly influenced by other elements of the system. In our case study, the element with the highest in-degree was “adherence”, which is not surprising because adherence is a well-studied factor with a known correlation to HIVDR and many influencing variables [29, 30]. The out-degree describes the number of outgoing connections (the number of elements thought to be influenced by the element of interest). In our case study, “understanding of HIV infection and treatment” had the highest out-degree, indicating that this element is perceived to exert the most influence on the rest of the system. Elements with a high out-degree but a low in-degree might be good candidates for leverage points in a system as they impact several parts of the system but are not influenced by many other elements. For example, social support is influenced by the status disclosure of the patient and the knowledge the family members have about HIV, while it has a direct impact on five different factors in the system, such as acceptance of HIV status and help with adherence. Another quantitative method by Finegood et al. can be used to quantitatively compare two systems maps visualizing the same system but from different points of view [31]. In the Finegood method, elements (based on the same thematic coding for both maps) are divided into clusters and inter- and intra-cluster relationships are compared. However, an important note has to be made concerning both methods. When interpreting these metrics, one needs to consider the coding choices made earlier in the process. Coming back to the example used before, psychological wellbeing can have an in-degree of four as it is influenced by stigmatisation, self-stigmatisation, discrimination and gossip, or it can have an in-degree of one if the researcher has decided to group all four elements in one. In both cases, the content of the map is the same, but the in-degree metric will be different (Fig 5). The authors therefore advise to be cautious when using such metrics as a supportive tool during analysis and always ground findings in qualitative evidence. Other methods to quantify causal loop diagrams and to select desirable future scenarios have been described in the literature [11, 32].

After analysis, it is advised to link back to the stakeholders and population in order to validate the findings. This is an important step in order to verify whether data was correctly interpreted and whether no major elements were overlooked. This can be done during participation in a conference if the target population are experts, or through a workshop, peer debriefings or member checks.

4. Exploring the system dynamics

While a systems map is a static representation of a CAS, in reality systems are constantly evolving and reorganizing when changes occur. Uncovering the potential for adaptiveness in a CAS requires an understanding of what is contributing to such dynamics. Once the system is mapped, one therefore needs to explore the characteristics that have the potential to lead to adaptiveness [5, 6]. These characteristics may be difficult to represent in a static systems map, which makes it all the more important to study them separately. In Table 3 we illustrate some of these characteristics with an example of our case study.

Table 3

Dynamic characteristics of CASs.

Characteristic	Explanation	Example
Emergence	Spontaneous behaviour which arises when individual actors or elements reorganize themselves into a bigger whole.	In order to prevent HIVDR, it is important that PLHIV take their medication on a daily basis. When there is a stock-out, healthcare workers organize themselves in WhatsApp groups in order to re-divide the stock and provide all patients with their doses.
Path dependence	Events that started in the same point, can lead to different outcomes, depending on the choices that are made during the process.	When a patient discloses their HIV status to family members it can lead to an increased social support and a better adherence, but also to stigmatisation, a decreased self-image or for example loss of employment opportunities.
Feedback loop	A series of elements that influence each other in a circular motion.	PLHIV need to take their medication with a meal in order to avoid side effects. When medication is taken daily, the patient will feel healthy and will be able to work and have access to daily meals as well as provide for their family. This reinforcing feedback loop is also used by healthcare workers to motivate PLHIV.
Tipping point	A point at which the system will rapidly change and eventually settle into a new balanced state.	Stigmatisation of PLHIV is for a large part caused by a lack of information and knowledge on the nature of the infection and the transmission modalities. When the point is reached where enough people have sufficient knowledge, and community stigmatisation decreases, it is possible that the system (which is now strongly influenced by stigmatisation), will rapidly adapt into a new state.
Culture	The shared values and believes which are intrinsically part of the system and which, as such, contribute to the system dynamics and information flows.	In the Tanzanian culture, religious leaders and traditional healers play a prominent role. PLHIV may believe they are punished by god when they first find out about their status, or believe they will get cured by praying. Religious leaders and traditional healers may therefore play an important role in the spread of correct information and the motivation to adhere to the medication.

Elements that contribute to the dynamics of a CAS, illustrated with an example of our case study.

5. Continuous transdisciplinary reflexivity

Though linearly described above, the process of data collection, mapping and analysis is actually an iterative process in which more data is collected based on newly gained insights and different mapping and analysis rounds are needed to explore different lines of thinking. Throughout all this, it is important that the researchers adopt a transdisciplinary approach, truly integrating the knowledge of different disciplines while transcending disciplinary boundaries. As our education system today is largely disciplinary, a quantitatively trained researcher will have to immerse him- or herself into the qualitative research paradigm and vice versa. Posner. et al. and McGregor describe this transition from mono- to transdisciplinarity as a conceptual shift in ones ideas about reality, logic and knowledge [33, 34]. Throughout the systems mapping and analysis process the researchers needs to be constantly aware of their potential disciplinary bias and need to search for active ways to avoid this, such as seeking continuous feedback from other disciplines or stakeholders, organizing group validation sessions etc. Moreover, the researcher should be aware that the systems map will never be truly finished as situations and conditions are continuously changing. Rather, the map should be seen as a dynamic tool that serves the research purposes, while staying open for changes. In short, we advise researchers to 1) immerse themselves into the literature and research paradigm of other relevant disciplines before starting the research, 2) aim for multi-disciplinarity within the research team, 3) continuously reflect on the possibility of disciplinary bias, and find ways to minimise it and 4) accept the dynamic and unfinished nature of systems maps.

Conclusion

Systems approaches are increasingly used to study complex health problems. The development of a systems map of the factors influencing the topic under investigation is not only useful as a process of transcending disciplinary boundaries and creating a holistic overview of the situation, but also as a means of gaining deep systems related insights in the underlying dynamics that drive this issue. In this article we have laid out a practical guideline for developing and analyzing a systems maps for complex public health issues. Such systems maps can be used to identify the root causes and intervention points in the system and to understand the dynamics that lead to the adaptiveness of a system. They may also potentially serve as a basis for further quantitative modelling.

Interview guide international experts.

Interview guide for the international expert interviews. This interview data is here used to illustrate our methodology and has been published elsewhere [19].

(PDF)

Click here for additional data file.

Interview guide PLHIV.

Interview guide for the interviews with PLHIV in Dar es Salaam, Tanzania. This interview data is here used to illustrate our methodology and has been published elsewhere [20].

(PDF)

Click here for additional data file.

Interview guide local actors.

Interview guide for the local actors in Dar es Salaam, Tanzania. This interview data is here used to illustrate our methodology and has been published elsewhere [20].

(PDF)

Click here for additional data file.

Codebook.

Codebook used for the systems maps represented in Figs 4 and 6. For privacy reasons quotes and references to interviews are not included. Tab 1 contains a weblink to an interactive version of the systems maps. Tab 2 represents the elements, their definition and a categorisation into layers (for Fig 4A), subsystems (for Fig 6B) and clusters (for Figs 4B and 6A). Tab 3 contains the connections, their polarity and their definitions. Tab 4 and 5 contain the Kumu code for both versions of the systems maps.

(XLSX)

Click here for additional data file.

1 Nov 2021

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Marco Cremonini, Ph.D.

University of Milan

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. When reporting the results of qualitative research, we suggest consulting the COREQ guidelines  or other relevant checklists listed by the Equator Network, such as the SRQR, to ensure complete reporting (http://journals.plos.org/plosone/s/submission-guidelines#loc-qualitative-research). Moreover, please provide the interview guide used as a Supplementary File." Do not ping with follow up, thanks!

3.PLOS ONE does not permit references to unpublished data; therefore, we request that you either include the referenced data or remove the instances of "data not shown," "unpublished results," or similar.

4. We note that Figures 3 and 5 in your submission contain copyrighted images. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

1. You may seek permission from the original copyright holder of Figures 3 and 5 to publish the content specifically under the CC BY 4.0 license.

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

2. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This was a tricky decision to make. On the one hand, the manuscript represents a potentially important contribution to the scientific literature on CAS. The writing style and line of argument was sound throughout, and I found the practical suggestions for mapping out a CAS to be helpful. This would be the case for any researcher working in a health-related capacity and studying processes of change, intervention effectiveness, etc. The specific example is pretty interesting too: Even the 'simplest' level here - the interaction of drugs, virus, and patient physiology, is clearly a CAS before we even add in the psychological, social, political and economic processes. It is for these reasons I'm suggesting a major revision, but I had several concerns, some quite serious about the manuscript which would need addressing before I could recommend publication:

1. The statement on ethics has been left blank, but the data underlying the manuscript, as far as I can tell, came from interviews with human participants about a sensitive topic. This should be addressed in full, including details of institutional approval etc.

2. This journal requires authors to make datasets publicly available, but I could not see any dataset attached to the submission. It looks like this might have been an oversight, but please be sure to include this with the re-submission.

I had a few substantive methodological points too:

1. The approach described is based on interview data, but the manuscript lacks detail on this - how many participants were recruited, were they all PWHIV or did you include other people working at other parts of the system (eg policy makers, health professionals, etc?)

2. On a related point, do you think interviews alone is enough to understand the dynamics of a system, or ideally should researchers include things like participant observation, documentary and policy analysis? I think at the very least you'd need interviews with a decent cross-section of the types of agent who interact within the system. This could be a useful part of any guide for developing CAS visualisations.

3. Although a lot of effort has clearly been put into coding, and explaining some of the process leading to drug resistance, some of the figures didn't make a great deal of sense to me. For example, what do the 'nodes' in fig 3 actually represent? This looks like some sort of social network map, but what are the individuals and how were they derived? What are the 'overarching factors' for instance? Figure 5A suffers from the same sort of problem: the nodes have general labels like 'psychosocial', but it's not clear exactly what psychosocial factors/ processes are at work in the system, nor how their relationships with the other nested levels of the CAS have been derived.

These visualisations also seem at odds with the qualitative approach taken, as they look like quantitative visualisations of social networks which I have seen in SNA articles, and which I've recently used myself with a colleague. Diagrams that show directions of causality between specific, more precisely defined actors and factors would work better in my view.

4. Where is Figure 1?

Be sure to review for typos, eg 'evens that started in the same point' in Table 2.

Reviewer #2: Abstract

The abstract describes the paper’s goals clearly and immediately sheds light on the adopted approach: proposing system maps to analyze a complex health issue, understanding the root causes, and intervening. The authors also specify that they based the paper on the insight from a previous study of theirs.

Data sources

The Authors specified in the Plos format that all relevant data are within the manuscript and its supporting information file (SI)

Introduction

The authors spend words to connect with the existent literature. They highlight that it is mainly theoretical, then they move to illustrate the concepts of system mapping. They provide adequate and accurate references to support their statements.

They also provide a concise introduction of system mapping through causal loop diagram notation. The authors could consider that it may confuse non-expert readers and evaluate whether distinct CLD by other methods (pictures? Schema? Table?). For readers’ learning, the Authors could also consider citing books about this method and more in general about CAS. (Only as examples, John Sterman’s contributions).

The introduction ends by clearly declaring the manuscript goals: Provide practical guidelines for using system mapping to investigate rich data on public health challenges. They specify that they demonstrate their guidelines based upon a case study.

The verb “demonstrate” has a strong meaning. Did the Authors use it on purpose?

Case Study

The case study description is clear, but I could not find Fig.1.

Data Collection

The Authors specify that they have collected data through individual, semi-structured interviews. The authors adduced suitable reasons for choosing this approach, but they should better specify the reference literature that supported their choice.

The Authors should also better specify:

• Why starting with an open question could/should/can reduce bias

• What they intend with “believes” and what is the literature they had referred to

The Authors should also discuss how they have mitigated the risk of hidden biases that can emerge in sequential questions caused by selective reinforcing loops. As for any ethnographic approach, bias existence should be taken for granted but explored carefully.

Mapping a complex adaptive system

The authors state that they adopted the QUAGOL method and correctly quoted the source. The following description of their application of quagol is clear and sounds correct. Table 1 aims to give an example, but it is unclear how the reader should use it. The Authors should consider clarifying how it relates to mapping complex adaptive systems in the main document, not in the table caption.

The Authors should also circumscribe the concept of mental models. While they are probably aware of the diverse interpretations, showing the connections between reasoning, making sense, deciding, and acting could be beneficial for the less expert readers – which seems to be their target, perhaps (works of Gerd Gigerenzer, Gary Klein, John Sterman, Senge et al, …).

That is of paramount importance as their method relies to some extent on the acceptance of these – here implicit – connections.

The Authors explore their case study in light of their approach thoroughly and accurately. Figures support their discourse, but they could consider adding either a schema or flow diagram to help the reader follow their examination.

The examination of the “depth of the system”, supported by figure 4, ends in an ambiguous description: are points 1) and 2) either rules or norms to follow? Or are they only suggestions the authors have made for increasing the robustness of this approach?

Could a less talkative and more schematic approach help? The whole section is informative, but the authors should carefully consider revising it, splitting the description of the guidelines from collateral enrichments and comments (beneficial, indeed!).

Transdisciplinary and system mapping

The section is informative, but the Authors should consider delivering sharper statements. While the considerations are undoubtedly correct and impactful, the less expert readers could not figure out how to use them.

Conclusions

The paper explores a relevant subject, and the Authors describe all the connections with previous literature at a sufficient level. They clearly express the paper's goals and the gaps it contributes to filling.

Reviewer's Syntesis

The paper explores a relevant subject, and the Authors describe all the connections with previous literature at a sufficient level. They clearly express the paper's goals and the gaps it contributes to filling.

The text is attractive and easy to read. Examples are clear, but sometimes they do not support the concept explanation sharply, while the readers can find themselves wondering about diverse interpretations.

The papers do not state the announced guidelines explicitly, while they are sparse and sound more like suggestions and reflection hints than normative guidelines.

Though the actual limits, this paper is grounded in peer reviews studies, and its goals, approach, and content are impactful and relevant. The way it transmits the subject is sometimes incoherent with its goals. While it promises guidelines, it often delivers informative and accurate reflections. Though they are helpful for those who want to adopt CAS perspectives and apply causal loop diagrams, the readers risk ending up in landscape plenty of theories without a clear map.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Tim Gomersall

Reviewer #2: Yes: Andrea Montefusco

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

29 Dec 2021

Dear reviewers,

Thank you for taking the time to carefully assess our submission. We have reviewed our manuscript accordingly and have addressed all your points raised.

Below you can find a detailed reply to each of your remarks. Please note that the indicated page and line numbers refer to the version of the manuscript containing track changes.

Reviewer #1: This was a tricky decision to make. On the one hand, the manuscript represents a potentially important contribution to the scientific literature on CAS. The writing style and line of argument was sound throughout, and I found the practical suggestions for mapping out a CAS to be helpful. This would be the case for any researcher working in a health-related capacity and studying processes of change, intervention effectiveness, etc. The specific example is pretty interesting too: Even the 'simplest' level here - the interaction of drugs, virus, and patient physiology, is clearly a CAS before we even add in the psychological, social, political and economic processes. It is for these reasons I'm suggesting a major revision, but I had several concerns, some quite serious about the manuscript which would need addressing before I could recommend publication:

Reply: Thank you for your appreciation of our manuscript and for recognizing the relevance of having a practical guideline for the mapping and analysing of CAS. We have addressed your concerns below and remain available for further clarifications if needed.

Reviewer: The statement on ethics has been left blank, but the data underlying the manuscript, as far as I can tell, came from interviews with human participants about a sensitive topic. This should be addressed in full, including details of institutional approval etc.

Reply: The interview data we refer to in this manuscript are used as a case study to illustrate our methodology and has been published elsewhere in two publications:

Kiekens A, Mosha IH, Zlatić L, Bwire GM, Mangara A, Dierckx de Casterlé B, et al. Factors Associated with HIV Drug Resistance in Dar es Salaam, Tanzania: Analysis of a Complex Adaptive System. Pathog 2021, Vol 10, Page 1535. 2021;10: 1535. doi:10.3390/PATHOGENS10121535

Kiekens A, Dierckx de Casterlé B, Pellizzer G, Mosha I, Mosha F, Rinke de Wit T, et al. Identifying mechanisms behind HIV drug resistance in Sub-Saharan Africa: a systems approach. BMC Public Health. : Pre-print under peer review. doi:10.21203/rs.3.rs-764469/v1

We provided the editor with all the necessary information on the ethical approval for those two publications, to be published at the editor’s discretion.

Reviewer: This journal requires authors to make datasets publicly available, but I could not see any dataset attached to the submission. It looks like this might have been an oversight, but please be sure to include this with the re-submission.

Reply: The datasets used to illustrate our methodological guideline has been published in a pre-print version of an article under review in BMC Public Health. We have included the dataset in the supplementary material (S4 File).

Reviewer: I had a few substantive methodological points too:

The approach described is based on interview data, but the manuscript lacks detail on this - how many participants were recruited, were they all PWHIV or did you include other people working at other parts of the system (eg policy makers, health professionals, etc?)

Reply: We have added some more information on the interviews conducted in our case study. We have also clarified that this case study has already been described in two separate publications and provided references. We feel it would be repetitive to provide all the details on the case study in this manuscript. We therefore specified that the case study is here used to illustrate our guidelines with examples. [page 5, line 110-115]

“These systems maps were informed by interviews with 15 international experts, 12 PLHIV in Dar es Salaam and 10 Local actors, who through their daily activities regularly come into contact with PLHIV in the study site. The findings of these studies are described in two separate publications [19,20]. In the next sections we use examples from this case study to illustrate our guide.”

Reviewer: On a related point, do you think interviews alone is enough to understand the dynamics of a system, or ideally should researchers include things like participant observation, documentary and policy analysis? I think at the very least you'd need interviews with a decent cross-section of the types of agent who interact within the system. This could be a useful part of any guide for developing CAS visualisations.

Reply: Indeed, we suggest to carefully recruit interviewees with the aim of covering all possible aspects of the system. Although we haven’t used additional methodology such as observations or document analysis, we do find this a good suggestion and have added it to the text. [page 6-7, line 130-138]

“Participants should be recruited with the aim of obtaining a full picture of all aspects of the system. Next to interviewing patients and healthcare workers, one might therefore also consider interviewing people who are somewhat further removed from the core problem but are still in touch with certain parts of it. For example, architects designing certain hospital area’s relevant for the topic under investigation or religious leaders who provide spiritual support to patients could provide unique insights into the topic. Next to the interviews themselves, other types of data such as participant observation and document analysis could also be used to triangulate the data and increase the validity of the results.”

Reviewer: Although a lot of effort has clearly been put into coding, and explaining some of the process leading to drug resistance, some of the figures didn't make a great deal of sense to me. For example, what do the 'nodes' in fig 3 actually represent? This looks like some sort of social network map, but what are the individuals and how were they derived? What are the 'overarching factors' for instance? Figure 5A suffers from the same sort of problem: the nodes have general labels like 'psychosocial', but it's not clear exactly what psychosocial factors/ processes are at work in the system, nor how their relationships with the other nested levels of the CAS have been derived.

These visualisations also seem at odds with the qualitative approach taken, as they look like quantitative visualisations of social networks which I have seen in SNA articles, and which I've recently used myself with a colleague. Diagrams that show directions of causality between specific, more precisely defined actors and factors would work better in my view.

Reply: Figure 3 (now figure 4) and figure 5a (now figure 6a) represent an overview of all elements influencing HIVDR as mentioned in the interviews and the connections between those elements. It does not represent interactions between people (as is the case in a social network map) but between factors. Moreover, both included systems are a qualitative representation of the system. We agree that it is difficult to understand the figures without further information. Therefore, we added a link to an online interactive version of both systems maps where the reader can browse through the systems maps, zoom in and click on elements and connections to gain a better understanding of how what the maps represent. We have added this to the text and we have included an overview of all the elements and connections present in this figure as supplementary file 4. The aim of figure 4 is to show that different representations of a same system are possible. The aim of figure 6 is to show how to summarize a system. For a detailed explanation and interpretation of the systems maps we refer the related publication:

Kiekens A, Dierckx de Casterlé B, Pellizzer G, Mosha I, Mosha F, Rinke de Wit T, et al. Identifying mechanisms behind HIV drug resistance in Sub-Saharan Africa: a systems approach. BMC Public Health. : Pre-print under peer review. doi:10.21203/rs.3.rs-764469/v1

“In both visualizations each element is a factor influencing HIVDR as mentioned in the interviews and each connection indicated the relationship between those factors. An overview of all elements and connections is included in these maps as well as an interactive version of the systems maps where the reader can zoom in and click on elements and connections for more information is included in the supplementary files (S4 File) [27].” [page 13-14 line 231-236]

Reviewer: Where is Figure 1?

Reply: We confirm that we did submit Figure 1 as we can see it in the file inventory of the editorial manager. We are not sure what went wrong but we will make sure to submit Figure 1 again with the resubmission. Note that after revision, this figure is now figure 2.

Reviewer: Be sure to review for typos, eg 'evens that started in the same point' in Table 2.

Reply: We reviewed the text for spelling and grammar errors.

Reviewer #2: Abstract

The abstract describes the paper’s goals clearly and immediately sheds light on the adopted approach: proposing system maps to analyze a complex health issue, understanding the root causes, and intervening. The authors also specify that they based the paper on the insight from a previous study of theirs.

Data sources

The Authors specified in the Plos format that all relevant data are within the manuscript and its supporting information file (SI)

Introduction

The authors spend words to connect with the existent literature. They highlight that it is mainly theoretical, then they move to illustrate the concepts of system mapping. They provide adequate and accurate references to support their statements.

They also provide a concise introduction of system mapping through causal loop diagram notation.

The authors could consider that it may confuse non-expert readers and evaluate whether distinct CLD by other methods (pictures? Schema? Table?). For readers’ learning, the Authors could also consider citing books about this method and more in general about CAS. (Only as examples, John Sterman’s contributions).

Reply: Thank you for your appreciation of our abstract and introduction. As suggested, we have added a visual example to understand causal loop diagrams (Fig. 1, page 3 and text page 3, line 58-62). We opted to only include this example in order not to distract the reader with too much detail on the different types of mapping, as this manuscript particularly focusses on causal loop diagrams. We have also added a reference to Sterman’s book to provide the reader with some background reading material. [page 4, line 74-75].

“An often-used example of a causal loop diagram is that of the heating and the thermostat (Fig 1). When the room temperature drops to a certain point, the thermostat will automatically increase the heating (negative causality). When the heating is on, the room temperature will increase (positive causality) up until the point in which the desired temperature is reached.

Figure 1: causal loop diagram example. Thermostat room temperature regulation as a (simplified) example of a causal loop diagram.

While in this manuscript we primarily focus on the mapping of causal loop diagrams, there are also other types of mapping such as stock and flow diagrams, which is a more quantitative approach to systems mapping and can be used to study the dynamic behaviour of a system over time, or social network analysis which aims at visualising and studying the relationships between social actors [10,11].”

“Systems dynamics and types of modelling have been thoroughly described by Sterman (2000) [12].”

Reviewer: The introduction ends by clearly declaring the manuscript goals: Provide practical guidelines for using system mapping to investigate rich data on public health challenges. They specify that they demonstrate their guidelines based upon a case study. The verb “demonstrate” has a strong meaning. Did the Authors use it on purpose?

Reply: We understand that the verb “demonstrate” can have a quantitative connotation. As we meant that we use a case study to provide examples for our guide, we rephrased the sentence. [page 4-5, lines 93 – 96]

“We use a case study concerning HIV drug resistance in Sub-Saharan Africa as a practical example to clarify how complex data can systematically be collected, mapped and analysed while incorporating the principles of transdisciplinarity every step of the way (Fig 2).”

Reviewer: Case Study

The case study description is clear, but I could not find Fig.1.

Reply: We confirm that we did submit Figure 1 as we can see it in the file inventory of the editorial manager. We are not sure what went wrong but we will make sure to submit Figure 1 again with the resubmission. Note that after revision, this figure is now figure 2.

Reviewer: Data Collection

The Authors specify that they have collected data through individual, semi-structured interviews. The authors adduced suitable reasons for choosing this approach, but they should better specify the reference literature that supported their choice.

Reply: We have added some references to literature about interviewing about sensitive topics and one reference which describes that individual interviews, in contrast to group sessions, allows the researcher to delve deep into personal and social matters. [page 4, line 83-90]

“ For example, the topic under investigation might be highly stigmatised in the community, therefore not allowing participants to speak freely during a group model building session, the participants perspectives on the topic may be too diverse to organize a common discussion (e.g. technical experts vs family and friends), or participants might live in different parts of the world, making a physical meeting organizationally challenging [15,16]. In some cases, individual interviews are also preferred when one aims to understand the individual mental models of stakeholders separately before generating an integrated overview of the combined viewpoints [17].”

Reviewer: The Authors should also better specify:

• Why starting with an open question could/should/can reduce bias

• What they intend with “believes” and what is the literature they had referred to

The Authors should also discuss how they have mitigated the risk of hidden biases that can emerge in sequential questions caused by selective reinforcing loops. As for any ethnographic approach, bias existence should be taken for granted but explored carefully.

Reply: Indeed, the interview analysis is an interpretative process and bias should be mitigated as much as possible. We started with a broad and open question about the complex problem in general, and not about only one aspect of it, because we wanted to allow the interviewee to share what is most important to them first. This way we avoided introducing bias ourselves by asking a question about a specific part of the system. With these open questions, we want to capture rich and nuanced data. To further avoid bias, especially for our case study example in which we interviewed PLHIV, we took some measures to create trust between the interviewer and the interviewee. For example, the interviews were done by a local researcher in the participant’s native language. There were no other people in the room and the interviews were held on a location which could not be associated with HIV, and therefore stigmatized. With regards to the potential bias with the sequential questions, we would also like to clarify that the chain of causality was built up several times throughout one interview, each time starting from a broad question. This way, some topics were covered from different angles, which also helped us to better understand how the participant experienced the situation and whether there may be bias. Furthermore, the questions were not cognitive questions but aimed to understand situations in the daily life of the interviewees which also lowers the chance of the interviewee feeling the need to give certain “correct” answers or hide their thoughts. We also added an example to explain that with “believes” we mean the underlying assumptions the participant has about the topic. All of this is added in the text on page 7-8, line 154-175.

“In order not to bias the data collection towards certain assumptions the researcher may have, it is advised to start the interview with a broad, open question about the complex problem, rather than asking a question about a single aspect of the problem. This way, the participants are inclined to start by expressing the aspect of the complex problem most important to them and the interviewer can explore the main believes and experiences the participant has to share about the topic. For example, an initial question like “how do you experience being HIV positive” may reveal to the interviewer that the patient’s whole perception of his or her HIV infection is based on the believe that it is a punishment of God. This information is important for the interpretation of the rest of the interview and may not have come up if the interview had started with a focus on a certain aspect of the system, such as the question “how do you perceive the healthcare service you receive?”. After this first question, the interviewer may continue covering a list of specific topics, retrieved from the literature or which came up in previous interviews. When a question is answered by “A happens because of B” the interviewer can ask for specific examples or experiences that support this claim and subsequently delve deeper into other possible underlying causes aiming to obtain the structure “A happens because of B, which is caused by C, D and E, etc.” This continues until a sufficient level of depth is reached or until the insights of the interviewee are exhausted, at which point the chain of causality may be built up further during interviews with other participants. Such chain of causality is built several times within one interview, each time starting from an open question. To further reduce bias, the interview circumstances should be well thought-through in order to create trust between the interviewer and interviewee. For example, for interviews with PLHIV are best done in their native language and in a location that cannot be perceived as stigmatizing.”

Reviewer: Mapping a complex adaptive system

The authors state that they adopted the QUAGOL method and correctly quoted the source. The following description of their application of quagol is clear and sounds correct. Table 1 aims to give an example, but it is unclear how the reader should use it. The Authors should consider clarifying how it relates to mapping complex adaptive systems in the main document, not in the table caption.

Reply: We have added an explanation on how to use the table in the text, as suggested. [Page 9, line 199-201]. Note that after revision table 1 is now table 2.

“In Table 2 we explain the types of data that can be stored behind one element, using the element “accessibility of healthcare centre” as an example.”

Reviewer: The Authors should also circumscribe the concept of mental models. While they are probably aware of the diverse interpretations, showing the connections between reasoning, making sense, deciding, and acting could be beneficial for the less expert readers – which seems to be their target, perhaps (works of Gerd Gigerenzer, Gary Klein, John Sterman, Senge et al, …).

That is of paramount importance as their method relies to some extent on the acceptance of these – here implicit – connections.

Reply: Thank you for pointing us to some reference literature. We have provided an explanation on what mental models are and why they are important for modelling a complex adaptive system. [page 13, line 211-215]

“Mental models are graphical representations of how people internally understand causal relationships between elements to make sense of a complex problem [18,19]. They often unconsciously affect our behavior or decision maker and are useful for us to gain a deeper understanding of the interviewees way of thinking about the problem [20].”

Reviewer: The Authors explore their case study in light of their approach thoroughly and accurately. Figures support their discourse, but they could consider adding either a schema or flow diagram to help the reader follow their examination.

Reply: Thank you for noticing this. In fact, Figure 2 contains diagram with an overview of the suggested method. We are aware that this figure did not reach you before and we hope this issue will be resolved during resubmission.

Reviewer: The examination of the “depth of the system”, supported by figure 4, ends in an ambiguous description: are points 1) and 2) either rules or norms to follow? Or are they only suggestions the authors have made for increasing the robustness of this approach?

Reply: Indeed, we suggest to apply these two questions to the whole systems map in order to have a common criterion to use for all elements and connections end, indeed, increase the robustness. However, we think other criteria (possibly topic-related) could also be used for this. This is why we do not claim this to be a rule that must be followed by anyone developing a systems map, but we rather include it as a suggestion. We have clarified this in the text. [page 16, line 287-297]

“In the rest of this paragraph we suggest some strategies for the simplification of systems maps. Other strategies (possibly topic dependent) could also be used. More important is to consequently apply the strategy to the whole systems map. When in doubt whether two elements should be merged or not, we suggest the researcher asks two questions: 1) are there significant differences in nuance between the content of both elements? And 2) do both elements have different connections to other elements? If the answer to both questions is “No”, the elements can be merged into one.”

Reviewer: Could a less talkative and more schematic approach help? The whole section is informative, but the authors should carefully consider revising it, splitting the description of the guidelines from collateral enrichments and comments (beneficial, indeed!).

Reply: We understand it is difficult for the reader to differentiate between the different steps of the guideline and our examples. We have tried to resolve this by giving titles to the sections, so that the article takes the shape of a step by step guide. There are four main sections which can also be found back in figure 2, and several subsections.

1. Data collection

1.1. Choosing a data collection method and participant selection

1.2. Preparing and conducting semi-structured interviews

2. Data analysis and mapping

2.1. Interview analysis

2.2. Coding

2.3. From codes to systems map

2.4. Setting systems boundaries

2.5. Determining the depth of the system

2.6. Simplifying the system

3. Analysing the system

4. Exploring the system dynamics

5. Continuous transdisciplinary reflexivity

Moreover, we have reformulated some sections to differentiate better between guidelines and examples from our case study. For example, the section on choosing a data collection method and participant selection on page 6 lines 126-138 and the section on preparing and conducting semi-structured interviews on page 7 line 151-153.

“The first consideration to make is which way of collecting data is most suitable for the topic under investigation. As already explained in the introduction, there are different reasons (both methodological and practical) to opt for either group modelling sessions or individual interviews. This guide focusses specifically on the mapping and analysing of complex data collected by semi-structured interviews. Participants should be recruited with the aim of obtaining a full picture of all aspects of the system. Next to interviewing patients and healthcare workers, one might therefore also consider interviewing people who are somewhat further removed from the core problem but are still in touch with certain parts of it. For example, architects designing certain hospital area’s relevant for the topic under investigation or religious leaders who provide spiritual support to patients could provide unique insights into the topic. Next to the interviews themselves, other types of data such as participant observation and document analysis could also be used to triangulate the data and increase the validity of the results.”

“A semi-structured interview guide should be developed based on the available scientific literature or already existing and validated guides on the topic, and adapted throughout the data collection process when new insights are developed.”

Reviewer: Transdisciplinary and system mapping

The section is informative, but the Authors should consider delivering sharper statements. While the considerations are undoubtedly correct and impactful, the less expert readers could not figure out how to use them.

Reply: We have added some concrete recommendations to this section. [page 22, line 379-383]

“In short, we advise researchers to 1) immerse themselves into the literature and research paradigm of other relevant disciplines before starting the research, 2) aim for multi-disciplinarity within research team, 3) continuously reflect on the possibility of disciplinary bias, and find ways to minimise it and 4) accept the dynamic and unfinished nature of systems maps.”

Reviewer: Conclusions

The paper explores a relevant subject, and the Authors describe all the connections with previous literature at a sufficient level. They clearly express the paper's goals and the gaps it contributes to filling.

Reviewer's Syntesis

The paper explores a relevant subject, and the Authors describe all the connections with previous literature at a sufficient level. They clearly express the paper's goals and the gaps it contributes to filling.

The text is attractive and easy to read. Examples are clear, but sometimes they do not support the concept explanation sharply, while the readers can find themselves wondering about diverse interpretations.

The papers do not state the announced guidelines explicitly, while they are sparse and sound more like suggestions and reflection hints than normative guidelines.

Though the actual limits, this paper is grounded in peer reviews studies, and its goals, approach, and content are impactful and relevant. The way it transmits the subject is sometimes incoherent with its goals. While it promises guidelines, it often delivers informative and accurate reflections. Though they are helpful for those who want to adopt CAS perspectives and apply causal loop diagrams, the readers risk ending up in landscape plenty of theories without a clear map.

Reply: Thank you for your appreciation of our subject and scientific intentions with this manuscript. We have aimed to provide a clearer step by step guide by adding section titles to the text and providing a table with an overview of the steps and their timing within the process [table 1, page 6]. We have also adapted some parts of the text to better differentiate between the guidelines and the examples meant to illustrate the guidelines. We hope that these changes and the ones described above, contribute to clearer formulation of our guidelines. While we have on some places also formulated our statements sharper as requested, we also want the reader to be aware that our guideline should always be considered within the research setting and may therefore be adapted to better suit the conditions.

We hope these changes have sufficiently improved our manuscript. We are happy to respond to any further questions and comments you may have.

Sincerely,

Anneleen Kiekens,

corresponding author

Submitted filename: Response to Reviewers.pdf

Click here for additional data file.

11 Feb 2022

Qualitative systems mapping for complex public health problems: a practical guide

PONE-D-21-29069R1

Dear Dr. Kiekens,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Marco Cremonini, Ph.D.

University of Milan

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Andrea Montefusco