Novel participatory methods for co-building an agent-based model of physical activity with youth.

Public health scholarship has increasingly called for the use of system science approaches to understand complex problems, including the use of participatory engagement to inform the modeling process. Some system science traditions, specifically system dynamics modeling, have an established participatory practice tradition. Yet, there remains limited guidance on engagement strategies using other modeling approaches like agent-based models. Our objective is to describe how we engaged adolescent youth in co-building an agent-based model about physical activity. Specifically, we aim to describe how we communicated technical aspects of agent-based models, the participatory activities we developed, and the resulting visual diagrams that were produced. We implemented six sessions with nine adolescent participants. To make technical aspects more accessible, we used an analogy that linked core components of agent-based models to elements of storytelling. We also implemented novel, facilitated activities that engaged youth in the development, annotation, and review of graphs over time, geographical maps, and state charts. The process was well-received by the participants and helped inform the basic structure of an agent-based model. The resulting visual diagrams created space for deeper discussion among participants about patterns of daily activity, important places for physical activity, and interactions between social and built environments. This work lays a foundation to develop and refine engagement strategies, especially for translating qualitative insights into quantitative model specifications such as 'decision rules'.

Introduction

Understanding how to improve behaviors like physical activity is complex. Many levels of influence, including individual motivations, interpersonal social networks, and built environments, interact to shape behavior over time. Scholars have called for the use of systems science and simulation modeling to help manage the complexity of this type of public health issue [1-4]. Systems science is an interdisciplinary field and approach to inquiry that focuses on understanding interrelated and interacting entities that form a unified whole. Simulation modeling, the process of creating and analyzing a digital prototype that emulates a real-life system, is a method often used in systems science. Furthermore, it is recommended that the simulation models are developed with stakeholders [1, 3, 5, 6], which has potential to produce better models, increase the social capital of communities, and improve the chance that a model successfully influences decision making [7].

To our knowledge, there has been limited work to develop and formalize methods that facilitate stakeholder engagement in building agent-based models for public health issues. Once an agent-based model is created, the simulated animations that result can be useful tools for engaging participants. For example, one description of engagement with an agent-based model related to primary care outcomes focused heavily on simulation results from the ultimate model produced [8]. Yet, there is less information about processes for initial structuring and development of agent-based models with stakeholders. One recent study that engaged stakeholders to build an agent-based model around the issue of food insecurity applied scripts from system dynamics modeling [9]. The authors concluded that the scripts were useful in creating conceptual models, but that future work was needed to establish scripts specific to agent-based models due to the vastly different scale of analysis between agent-based and system dynamics models.

In public health, there has been a relatively strong emergence of scholarly literature focused on participatory approaches to building system dynamics models [10-20]. The popularity of participatory system dynamics is likely due, in part, to the field’s historical participatory approach to model building [7], which has included the development, documentation, and sharing of best practices for facilitators [21] of group model building activities called “scripts” [22, 23]. While the detailed protocols are an important tool to support group model building in system dynamics, the choice and adaptation of a given script (or set of scripts) for use with certain communities or groups relies on managing the modeling process outputs as boundary objects. Boundary objects are the visual representations of the system dynamics model or model elements that result from the engagement process. In system dynamics, boundary objects such as behavior-over-time graphs, causal loop diagrams, and stock and flow diagrams have been considered critical to the process and valuable for their ability to be transformable by all and represent dependencies across stakeholders [24]. The close correspondence between the visual diagramming conventions of system dynamics models and their underlying mathematics has facilitated the management of boundary objects. In contrast, it is less clear how agent-based models can be managed as boundary objects because agent-based models are often defined in lines of software code that can be difficult to directly visualize.

We sought to develop and pilot test new scripted activities specifically for developing agent-based models. Our aim was to develop an approach that would aid participants’ understanding of elements of agent-based models that are difficult to visually represent and, where possible, develop new scripts that would result in boundary objects relevant for agent-based models. In this paper, we describe our process and outcomes from piloting a participatory agent-based model building approach alongside adolescent youth and with a focus on physical activity. We engaged youth as our stakeholders for their unique perspective as well as for the potential additional benefits including improvement in youth’s own leadership, personal agency, and collective empowerment.

Method

The study was reviewed and approved by the institutional review board at the University of North Carolina at Chapel Hill.

Setting

We conducted this project in one small town (population of 10,844) in eastern North Carolina. African Americans are 48.3%, Non-Hispanic Whites are 44.8%, and Hispanics are 5.1% of the population [25]. The town is situated in a county that has one of the highest unemployment rates in North Carolina and 24.6% of residents live in households with incomes below the federal poverty level [25].

Modeling team

In addition to three researchers experienced with participatory modeling, our team included community health leaders. One of the community health leaders and collaborator (DS) has been involved in public health research for over 20 years. Her work has included the formation of a county council for adolescent health and she had been involved in a prior system dynamics participatory modeling project [26]. Another community health leader has experience facilitating youth initiatives (DJ) and has designed and implemented youth curricula with an emphasis on mental and behavioral health. Our team also included public health students with a range of experience areas including the development and facilitation of asset mapping and socio-emotional learning curriculum for high school students. Finally, we worked with the youth participants to develop a list of roles and responsibilities that they could assume on a rotating basis (e.g., ice breaker expert–develops ice breakers for each session, feedback queen/king–communicates what is and is not working during the sessions). This was done to provide leadership opportunities for all youth involved as well as foster ownership and cohesiveness among the group.

Session development

We developed and implemented content for six sessions that were targeted to last between 1–2 hours each. During the sessions, we sought to elicit information from the students on: 1) important physical activity topics to consider, 2) where physical activity takes place, 3) how youths’ location and physical activity changes throughout the day, 4) with whom physical activity takes place, and 5) other factors that influence changes in locations and physical activity levels. Although, the majority of the session content was focused on activities to inform an agent-based model, we also introduced youth to concepts of community-engaged research, implemented team and leadership building activities, and initiated discussions about collecting data for a future quantified model.

When building the model structure, we recognized that agent-based models can be difficult to conceptualize [9]. To make the model structure more accessible, we used the concept of storytelling. We linked the core components of agent-based models; 1) the topic of study, 2) agents, 3) properties and rules, 4) environment, and 5) simulation results, to core components of storytelling: 1) the conflict, 2) characters, 3) character development, 4) setting, and 5) the plot. Table 1 highlights each of these linked concepts plus two examples of the storytelling links in application to agent-based models of physical activity. The first example applies the link to an agent-based model from the extant literature and the second illustrates the application from our pilot study with youth.

Table 1

Example modeling and storytelling linkages.

Modeling	Description	Storytelling Link	Extant Literature Example	Case Study Example
Topic of study / conflict	Research question, public health issue	Conflict	The impact of crime on African American women’s physical activity and obesity	The impact of social and built environment on rural adolescent’s physical activity
Agents	Simulated individuals in a model	Characters	African American women aged 18–65 living in Washington DC	Adolescents (ages 14–18) living in a largely rural and African American county in Southeastern US
Properties and Rules	Properties: Agent characteristics and goals	Character development	Properties: age, height, household location, income, probability of exercise	Properties: age, height and weight, household location, probability of exercise, mood/affective states
	Rules: Specifications for agent movement and decision making in a model		Rules: agents choose where to exercise, and at what intensity and duration to exercise
				Rules: agents choose to move to and from locations, interactions with peers and family members influence choices, perceptions of recreational spaces influence choices
Environment	Where agents can go in the model	Environment / setting	Washington DC Wards 5, 7, and 8	Tarboro, NC
Model simulation	How the model ‘plays out’ when run	Plot	As crime was reduced, leisure time physical activity increased, but this depends on how likely women are to exercise initially.	To be determined

Note: Modeling example derived from [27] Powell-Wiley, T. M., et al. (2017). "Simulating the Impact of Crime on African American Women's Physical Activity and Obesity." Obesity 25(12): 2149–2155.

The storytelling analogy was introduced in the first session and integrated into all following session materials. More specifically, we began our sessions with the concept of a storytelling ‘conflict’ and we elicited responses from the participants about what most ‘got in the way’ or ‘helped’ them to be physical active, which was used to refine the research questions. We also highlighted that agents in the model were like the characters of a story and used activities throughout the sessions to elicit more information about important qualities and factors that influenced (or were influenced by) their physical activity levels, akin to character development. The model environment was described as a story’s setting and participants were led through activities to identify and describe important locations in their lives and for physical activity. Finally, the model simulation was likened to a story’s plot. An existing simulation model was used to illustrate how simulation models produce outcomes over time and specifically highlight emergent dynamics of agent-based models.

Participant recruitment

Our community partner contacted the school district’s superintendent and held several meetings also including the high school guidance counselors and principal. These meetings focused on communicating the scope of project, outlining the resources needed to carry out the project (e.g., meeting space in the school), and how recruitment would take place. The high school distributed information about the project via an email to all students with a survey link they could respond to indicating their interest. In addition, our community partner recruited additional youth using convenience sampling strategies. Specifically, she contacted individuals who had been involved with her organizations’ other programs and received recommendations from the guidance counselor. To be eligible participants had to be in grades 9–11 and could not have significant cognitive impairments. After students expressed interest, we held a preliminary meeting to explain the scope of the project. The students completed assent forms and their parents/guardians completed and signed informed consent forms as well. Students were provided a $20 cash incentive for each session that they attended.

Session evaluation

After each session we asked students to respond to a written survey with 4–6 short-answer questions. The questions asked the students to recall what they learned during each session, points of confusion, as well as general feedback and feedback related to a specific scripted activities (e.g., “What, if anything, did you learn about by participating in the physical activity mapping activity?”). After the final session, we also implemented a low-risk, not time intensive debriefing technique called “keep, start, stop” [28, 29]. In sequence, we asked the participants: “what should we keep/start/stop doing?” for future sessions.

Overview of session scripts and boundary objects

Fig 1 provides a high-level overview of each session and evaluation. Table 2 provides a detailed summary of each session, description of how elements of storytelling were incorporated, and the specific scripts developed and used. Following, we describe four new scripts we created and implemented specifically for agent-based modeling.

Fig 1

Timeline of major model building activities by session.

Table 2

Detailed summary of the six agent-based model building sessions.

Session Schedule and Topic	Description of Activities	Link to Storytelling	Approximate Duration of Activity	Existing Sources Used or Adapted
Session #1 –Introductions and orientation to agent-based models
Setting group norms	Participants were provided with several sheets of paper of two different colors and instructed to write their “Hopes” for the project on one color and “Fears” on the other. Participants then shared their hopes and fears and the sheets were taped to the wall. A facilitator identified themes and then guided a conversation about what types of ‘ground rules’ or norms the group wished to establish in order to help the group achieve their hopes and reduce their fears. These were recorded and reviewed at the beginning of all subsequent sessions.	N/A	25 minutes	Hopes & Fears Script
				[30]
Orientation project and modeling	A simple simulation model was constructed to specifically highlight emergent dynamics of agent-based models. The model simulated students in a lunchroom and their selection of where to sit. Students chose where to sit based on the proportion of other students who were similar in their preference for Cardi B or Nicki Minaj music. Further, the model was used to illustrate the elements of storytelling.	N/A	20 minutes	Adapted from Concept Model Group Model Building Script
				[30]
Physical activity barriers/facilitators	The participants were asked to help identify important variables to consider for the model. The participants were asked to brainstorm responses to the prompt, “What are some things that help or make it harder to be physically active?”	Identifying the “Conflict”	20 minutes	Adaption of Variable Elicitation Script
				[30]
Session #2 –Understanding roles in research and daily trends in physical activity
Leadership in research	Participants were introduced to concepts of participatory action research and citizen science. They were oriented to example research projects that have been implemented in communities similar to theirs.	N/A	30 minutes	N/A
Identifying leadership types	Youth were guided through an activity to identify their leadership type and discuss how they can use their strengths to play a role in improving their community’s health.	N/A	20 minutes	Youth Engaged in Leadership and Learning
				Curricula [31]
Physical activity over time	Participants were shown how to draw an example graph that illustrates how a factor or variable changes over time. They were each provided with templates that had an x-axis of time (24-hour day) and y-axis with physical activity level. They were guided to complete and annotate the graph.	Identifying the “Characters”, i.e., who are the agents and what are they doing in time and space?	45 minutes	Graph over Time for ABMs Script (S1 Appendix)–Adapted from Graphs over Time
				[30]
Session #3 –Understanding the physical activity environment
Review Graphs over Time	An visualization of the graphs over time (aggregated across all participant responses) were presented and discussed at the beginning of the session.	N/A	15 minutes	N/A
Team-building exercise	Marshmallow structure challenge–the participants were divided into small groups of 3–4 and provided with a package of supplies (dried spaghetti, masking tape, marshmallows). The groups were then given 15 minutes to build the tallest tower possible. At the end, the group reflected on their process, how they worked together, how they used their leadership styles.	N/A	25 minutes	Leadership Challenge [32]
Mapping the physical activity environment	Participants were guided to identify their homes on a map of their city and then to identify places that encourage physical activity and make it more difficult. Stickers were placed on a large map and the group discussed the most important places.	Identifying the “Setting”, i.e., what is the environment that the agents ‘live’ in?	60 minutes	New Script—Mapping Important Locations (S1 Appendix)
Session #4 –Putting characters and environment together
Model Review	A simple model was created and presented at the beginning of the next session that showed the participants movement between their home and school.	N/A	20 minutes	N/A
State Charts	The concept of “state charts” was introduced to the participants. Draft state charts based on prior session activities were presented to the participants. They were asked to trace their potential pathways among the states. A facilitator led a discussion identifying edits and additions to refine the state charts.	Identifying/expanding on “Character Development”	20 minutes	New Script–State Chart Review (S1 Appendix)
Session #5 –Gaining a deeper understanding of physical activity
Recap of State Charts	Refined State Charts were reviewed and discussed.		15 minutes
Interviewing for decision rules	The participants were introduced to research interview best practices and practiced interviewing each other in order to gain a deeper understanding of their ‘decision rules’. Their ‘decision rules’ were what influences physical activity. After their interviews, participants shared what they learned and a facilitator helped the group identify themes and key ‘decision rules’.	Identifying/expanding on “Character Development”	45 minutes	New Script—Interviewing to Understand Decision Rules (S1 Appendix)
Session #6 –Putting our story together
Diagramming decision rules	A facilitator re-capped themes from the prior session’s interviews and then asked participants to further annotate the state charts with these new themes and insights in mind.	Identifying/expanding on “Character Development”	45 minutes	New Script—Interviewing to Understand Decision Rules (S1 Appendix)

Graphs Over Time script

The Graphs Over Time script is a well-established component of group model building in system dynamics that we adapted to understand the typical daily patterns of physical activity of youth [33]. The script engages participants in developing plots of one or more variables that capture how the issue of interest and other relevant factors change over time. This activity is typically done in the early stages of a model building effort in order to help better understand and frame the problem. The standard process for this script involves four steps: 1) a facilitator provides an example plot where the x-axis is labeled as “time” and the y-axis is labeled with variable name(s), 2) participants alone, or in small groups, develop their own plots, 3) the participants share their resulting plots with the group, and 4) the facilitator and participants identify themes that emerge from the graphs.

Our goal was to use a similar process to generate and understand plausible patterns of behavior for an agent-based model that could help guide early model calibrations. We followed these same steps with additional annotation exercises (S1 Appendix). A facilitator asked the participants to chart their physical activity level (y-axis) throughout a typical day (x-axis). However, because agent-based models have a focus on social interactions and the environment, we asked participants to further annotate their plots. Specifically, we asked participants to write who they were with (social interactions) and where they were (environment) on sticky notes and overlay these annotations on their plots (Fig 2A). This process created individual plots as boundary objects. Following the session, we created digital graphs that aggregated the groups’ plots. Photos of their individual graphs and the aggregated plots (Fig 2B) were presented and discussed at a subsequent session. Showcasing the aggregated results helped the group identify common themes such as sedentary and active times.

Fig 2

Resulting artifacts from the Graphs Over Time script.

Mapping Important Locations script

One of the major advantages of agent-based models is the ability to incorporate geographic information such as road networks, locations of specific exposures of interest, and attributes of spatial features. We developed a new script focused on identifying geographic model boundaries, important locations with potential to affect the outcome of interest, and anticipated movement of individuals between various locations (S1 Appendix). For this scripted activity, we obtained a poster-sized map (5x3.5 feet) of the general location of interest (i.e., where the participants lived, socialized, went to school/work) and smaller maps (8.5x11 inch) for each participant. Prior to the session, participants were asked to identify and take pictures of roughly five important places we should include in the model, including places where physical activity happens and places where it does not in their community. Working in small groups, participants were asked to use stickers to locate where their pictures were taken on the map, discuss the locations and their importance, and identify 3–6 locations that the small group felt were most important to their physical activity. Additionally, the participants were guided to annotate their individual maps with examples of how they typically moved from one location to another, highlighting specific routes and modalities of travel (e.g., walk, car). The facilitator also convened the whole group to locate and discuss their prioritized places, travel routes, and modes of travel on the larger map.

This process created individual maps and an annotated group map as boundary objects. Further, following the session, we used the information to create a simple agent-based model that showed the participants’ mobility patterns within their geography. The agent-based model was shown to participants and discussed at a subsequent session. This scripted activity helped identify major patterns of mobility among the population of interest and other important concepts that further refined the modeling problem. For example, from this exercise we identified car transportation as the primary mode of travel to and from school.

State Chart Review script

The facilitator team used the Graphs Over Time and Mapping Important Locations outputs to develop state charts to include in the model. States and transitions between them are abstractions used in agent-based models to model different conditions that individuals are in at different times (e.g., healthy, infected, dead) and the potential pathways between the states (i.e., an individual can move from healthy to dead or healthy to infected but not dead to healthy). State charts are visual depictions of these conditions. For this scripted activity, we created state charts using AnyLogic (Version 8.4.0 Personal Learning Addition, 2019) and created print-outs of the diagrams. We created states mostly based on the participants’ major locations that they identified as part of their typical daily routines (i.e., home, school, and after school/evening places); wherein each location could correspond to a general likelihood of engaging in different levels of sedentary or physical activity. During the session, we presented the state chart and provided print-outs to each participant. The facilitator provided an overview of the state charts and described how they were informed by the prior sessions. Participants were then asked to use highlighters to trace their potential pathways through the states. The group then discussed the comprehensiveness of the states and whether any states or transitions were missing. The state chart was a useful boundary object for discussion and led to the discovery of missing states (i.e., church, family/friend residences) and refinements to the state chart (Fig 3).

Fig 3

Resulting artifact from State Chart Review script.

Interviewing to Understand Decision Rules script

In an agent-based model, decisions and rules need to be formulated to guide each individual agent’s transitions between states (e.g., an agent’s number of friends who are exercising regularly could influence the probability that they will also exercise). Although the “State Chart Review” activity provided useful information about model states, deeper information surrounding what creates transitions between states was largely missing. Thus, we developed a script to help identify the factors that were perceived as important to creating shifts between states (S1 Appendix).

For this activity, we used a two-phased process implemented across two sessions. During the first session, we guided participants through an activity to interview each other with the goal to stimulate deeper thinking about the issue of interest, physical activity. We introduced them to the basics of interviewing for research (e.g., use of open-ended and probing questions, active listening, reflecting and summarizing answers) and two facilitators role-played a mock interview. Next, we divided the participants into pairs. We provided them with a guiding set of questions and approximately 30 minutes to interview each other. Afterwards, the group convened and each individual shared what they learned from their respective partner about influences on physical activity. A facilitator listened and summarized major themes that emerged from the discussion.

During the second session, the facilitators recapped themes from the prior session’s interviews and communicated the session’s goal of further annotating state charts. The facilitator projected simplified state charts that focused the participants on the transitions between specific states (i.e., Home not Active, to Home but Active; Home not Active, to Parks/Active Places). Participants were asked to think about what would make them stay versus make the various transitions and annotate their individual state charts. Next the facilitator asked each participant, in a round-robin style, to provide factors that influence their transition decisions. This process helped establish additional key relationships and model elements previously missing such as how having family members of similar age influenced activity. This activity resulted in annotated state charts as boundary objects (Fig 4)

Fig 4

Resulting example artifact from Interviewing to Understand Decision Rules script.

Results

The artifacts from each of the activities provided important insights for different aspects of the model building process. For example, Graphs over Time artifacts revealed insights with implications for the model time scale while Mapping Important Locations provided insights with implications for environmental boundaries and agent-environment interactions within the model. More specifically, the Graphs Over Time artifacts revealed time periods with more (after school) and less (during school) variation in daily activity and Mapping Important Locations artifacts indicated a few key locations (school, home) most relevant to sedentary and physical activity. Furthermore, all artifacts provided insights relevant to data collection and analysis for future model parameterization. For example, the Decision Rules artifacts indicated that social interactions were likely influential in physical activity choices and more information was needed to operationalize the pathways and processes by which friends and family impact activity decisions on a day-to-day basis.

Results from the post-session surveys indicated the participants found the storytelling format and scripted activities useful and rarely had areas of confusion. From the open-ended responses, we found four major themes about what participants learned, specifically participants described: 1) social connections: learning more about and from each other, 2) leadership growth: improving listening and critical thinking skills, 3) spatial knowledge: gaining new skills in reading and understanding maps, 4) physical activity: increasing awareness about their personal physical activity. Specifically, regarding social connections, participants reported learning more about their peers’ experiences outside of the study. For example, one participant noted learning that someone in the group had gone surfing and hiking and others learned that most people enjoyed physical activity with friends and family. In relation to leadership skills, beyond gaining tangible skills in active listening and thinking deeply about themselves, students were also exposed to youth leadership trainings in the area they could attend beyond the lifespan on the project.

From the ‘keep, stop, start’ activity, the participants indicated that they enjoyed the team-building and leadership activities and suggested they be kept in future sessions. Participants also desired more activities aimed at increasing group cohesion. Suggestions included several outdoor activities (i.e. a picnic and swimming) as well as a trip elsewhere (i.e. to an amusement park or other destination). These findings suggested that the group was continuing to learn more about one another even at the final session and that participants had a desire to form community outside of the traditional meeting space. Participants appreciated the value and insights gained from the model building activities and indicated they would not ‘stop’ any of them. Suggestions about new things to “start” focused primarily on group logistics such as the meeting room, food, and incentives.

In summary, beyond the tangible spatial and mapping skills as well as critical thinking and listening skills students gained, there was also an appreciation for fostering community both environmentally and on a personal level. What may appear as small details that were not obvious to the success to the study at onset, i.e. food suggestions, building in small celebrations outside of formal meeting spaces, and taking time to get to know one another on a personal level, were crucial elements in the eyes of participants.

Discussion

We developed and implemented several innovative strategies for partnering with community members to co-develop the basic structure of an agent-based model. Overall, our process was well-received by the participants. Our team found storytelling to be a helpful analogy in communicating the technical aspects of agent-based models. Additionally, the newly scripted activities resulted in visual representations of the model and boundary objects that the participants could view and discuss together.

We found that the storytelling analogy as well as allowing the individual participants to tell their own stories through the “Interviewing for Decision Rules” activity both improved the level of engagement. The use of imagery and narrative is a well-established method in participatory approaches as it relies less on the written word and provides a variety of media that leverages community expertise. A recent study that engaged participants in developing an agent-based model similarly described using aspects of storytelling [34]. The recent study presented clinical case histories, “life stories”, to help walk participants through the model structure and logic. In contrast to this method, our team drew on our participant’s first-hand knowledge and lived experiences to review state charts, identify pathways, and discuss factors that influence their transitions between places. Our strategy is likely more useful when engaging individuals who have directly experienced the issue at hand; whereas, case histories may be more useful when engaging stakeholders who have second hand experience from individuals they care for whom are affected (e.g., healthcare providers, teachers).

Our process resulted in artifacts that deepened discussion and insights surrounding social interactions and spatial locations, but we also uncovered areas where more work is necessary. Specifically, we found that developing quantifiable decision rules from our resulting artifacts and activities was a challenge. Others have noted that there is limited guidance on the process of translating the visual, qualitative models to quantified simulation models [34]. A recent case study described high-level elements of a participatory approach to develop a dynamic simulation model, but noted that modeling decisions were highly interactive and iterative. Further research is needed to develop more precise strategies for translating qualitative insights into formulaic and mathematical specifications for agent-based models. We recommend that public health draws from the environmental science discipline, which has a more substantial history of participatory engagement using agent-based models. For example, the field has developed and used “companion modeling” for agent-based modeling that emphasizes the use of role-playing sessions [35-37]. The role-playing sessions use information about potential factors that influence decisions to develop games where participants use spreadsheets or board games to ‘simulate’ decisions. The role playing sessions can help refine models by identifying irrelevant factors or clarifying relative weights placed on different factors in the decision making process [35, 36]. Alternatively, additional empirical, quantitative research may be necessary for such formulations. As such, one of our project's next steps is to employ primary data collection with the youth and their peers that will aid in model quantification.

In sum, while participatory engagement to develop system dynamics models has been emerging in the public health literature, there have been far fewer examples for agent-based models. Our work adds to the limited guidance for researchers interested in co-developing agent-based models alongside community stakeholders. We found our methods were promising and produced visual elements relevant for agent-based models that participants can equitably own and discuss in the process.

Agent-based model building scripts.

(DOCX)

Click here for additional data file.

(XLSX)

Click here for additional data file.

17 Aug 2020

PONE-D-20-12547

Novel participatory methods for co-building an agent-based model of physical activity with youth

PLOS ONE

Dear Dr. Frerichs,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The reviewers have indicated minor changes to the manuscript, as detailed below. The manuscript was an interesting read, and well written, and I look forward to reading the final amended version.

Please submit your revised manuscript by Oct 01 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Dr Emma Louise Giles

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.Thank you for stating the following in the Financial Disclosure section:

[This project was supported by the National Heart Lung and Blood Institute (Grant Number 5K01HL138159; PI Frerichs).  It was also supported in part by the National Institutes of Health under award numbers R01AG047869 and K24HL105493.  The funding bodies had no role in the design, collection, analysis, interpretation, nor writing of the study.  Its contents are the authors’ sole responsibility and do not necessarily represent official NIH views.  ].

We note that one or more of the authors are employed by a commercial company: System Stars, LLC and Independent Consultant

Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

2. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Overall, the manuscript was written well. I would give a suggestion to add a definition for all of the terms that are included in the introduction such as: agent-based models and systems dynamics. Not everyone who reviews will be familiar with these concepts especially understanding the link with CBPR and engagement.

Reviewer #2: This paper seeks to involve youth in the exploration and piloting of scripted activities intended to support the development of an agent-based model surrounding physical activity. The authors implemented a program consisting of six sessions, where students were asked what, when, where, with whom, and trends regarding physical activity via a variety of creative and facilitated activities. After each session, students were asked to indicate what they had learned in the session, as well as for their feedback on the sessions. The authors make a valid case towards developing a model surrounding physical activity that engages youth in the development. The research team was qualified to carry out the work – three researchers experienced with participatory modeling, two experienced community health leaders, and public health students, and youth.

A clear strength of the paper is the well-designed, described, and appended curriculum. Session content is clearly described in a way that others could implement similar activities in their own settings. Methods are clearly aligned with the agent-based model approach. The graphs and plots clearly described and well-selected.

One table that was unclear to me was “Table 1 – Example modeling and storytelling linkages”. Would it be possible to add more detail around how storytelling was incorporated? I did not find this table intuitive to understand after reading the text description. For example, does the content in the table represent hypothetical examples or samples from the project?

The results section is also very short. Is there an opportunity to include a few examples of quotes to support the themes you’re describing? Were there variations among participants in how they experienced the themes you report? Also, the results section only presents the post-session surveys. Consistent with your stated purpose of the paper of piloting new scripted activities for agent-based models, should results also present findings related to the new scripted activities you developed? Are there any insights that the authors/project team gleaned from the artifacts themselves that would be worth reporting on to meet the stated purpose of the paper?

Overall, the manuscript is technically sound and the data support the conclusions. All data is appended, and the authors also include figures and tables to illustrate examples of session artifacts. The authors did a thematic analysis of the summary evaluations, and no statistical analyses were completed. The manuscript is clear and easy to read. One suggestion to further improve clarity is to include definitions of ‘systems science’, ‘simulation modeling’, and ‘agent-based models’ when these terms are introduced early in the paper, to put the methods and findings in context. These terms may not be familiar to the reader.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Katherine Wisener

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

19 Sep 2020

We thank the reviewers for their thoughtful and positive reviews. We addressed each of the reviewers’ minor comments. Please see the responses below.

Reviewer #1: Overall, the manuscript was written well. I would give a suggestion to add a definition for all of the terms that are included in the introduction such as: agent-based models and systems dynamics. Not everyone who reviews will be familiar with these concepts especially understanding the link with CBPR and engagement.

RESPONSE: We thank the reviewer. We appreciate the suggestion, which was also raised by the other reviewer. We added brief definitions in the introduction to better orient the reader to these terms:

“Systems science is an interdisciplinary field and approach to inquiry that focuses on understanding interrelated and interacting entities that form a unified whole. Simulation modeling, the process of creating and analyzing a digital prototype to emulate a real-life system, is a method often used in systems science.”

Reviewer #2: This paper seeks to involve youth in the exploration and piloting of scripted activities intended to support the development of an agent-based model surrounding physical activity. The authors implemented a program consisting of six sessions, where students were asked what, when, where, with whom, and trends regarding physical activity via a variety of creative and facilitated activities. After each session, students were asked to indicate what they had learned in the session, as well as for their feedback on the sessions. The authors make a valid case towards developing a model surrounding physical activity that engages youth in the development. The research team was qualified to carry out the work – three researchers experienced with participatory modeling, two experienced community health leaders, and public health students, and youth. A clear strength of the paper is the well-designed, described, and appended curriculum. Session content is clearly described in a way that others could implement similar activities in their own settings. Methods are clearly aligned with the agent-based model approach. The graphs and plots clearly described and well-selected.

RESPONSE: We thank the reviewer for the thoughtful and positive summary of our research.

One table that was unclear to me was “Table 1 – Example modeling and storytelling linkages”. Would it be possible to add more detail around how storytelling was incorporated? I did not find this table intuitive to understand after reading the text description. For example, does the content in the table represent hypothetical examples or samples from the project?

RESPONSE: We added a stronger descriptive of how storytelling was incorporated, including a more detailed narrative that provides more context to Table 1:

“…Table 1 highlights each of these linked concepts plus two examples of the storytelling links in application to agent-based models of physical activity. The first example applies the link to an agent-based model from the extant literature and the second illustrates the application from our pilot study with youth.

The storytelling analogy was introduced in the first session and integrated into all following session materials. More specifically, we began our sessions with the concept of a storytelling ‘conflict’ and we elicited responses from the youth participants about what most ‘got in the way’ or ‘helped’ them to be physical active, which was used to refine the research questions. We also highlighted that agents in the model were like the characters of a story and used activities throughout the sessions to elicit more information from the participants about important qualities and factors that influenced (or were influenced by) their physical activity levels. The model environment was described as a story’s setting and participants were led through activities to identify and describe important locations. Finally, the model simulation was likened to a story’s plot. Finally, an existing simulation model was used to illustrate how simulation models result in outcomes over time and specifically highlight emergent dynamics of agent-based models.”

The results section is also very short. Is there an opportunity to include a few examples of quotes to support the themes you’re describing? Were there variations among participants in how they experienced the themes you report?

RESPONSE: Yes, we agree the section was very short. We added details to the section to support the themes as suggested. (p. 17-18)

Also, the results section only presents the post-session surveys. Consistent with your stated purpose of the paper of piloting new scripted activities for agent-based models, should results also present findings related to the new scripted activities you developed? Are there any insights that the authors/project team gleaned from the artifacts themselves that would be worth reporting on to meet the stated purpose of the paper?

RESPONSE: We thank the reviewer for this suggestion. We added a brief paragraph describing insights from the artifacts:

“The artifacts from each of the activities provided important insights for different aspects of the model building process. For example, Graphs over Time artifacts revealed insights with implications for the model time scale while Mapping Important Locations provided insights with implications for environmental boundaries and agent-environment interactions within the model. More specifically, the Graphs Over Time artifacts revealed time periods with more (after school) and less (during school) variation in daily activity and Mapping Important Locations artifacts indicated a few key locations (school, home) most relevant to sedentary and physical activity. Furthermore, all artifacts provided insights relevant to data collection and analysis for future model parameterization. For example, the Decision Rules artifacts indicated that social interactions were likely influential in physical activity choices and more information was needed to operationalize the pathways and processes by which friends and family impact activity decisions on a day-to-day basis.” (p. 16-17)

Overall, the manuscript is technically sound and the data support the conclusions. All data is appended, and the authors also include figures and tables to illustrate examples of session artifacts. The authors did a thematic analysis of the summary evaluations, and no statistical analyses were completed. The manuscript is clear and easy to read.

RESPONSE: Thank you for these positive comments.

One suggestion to further improve clarity is to include definitions of ‘systems science’, ‘simulation modeling’, and ‘agent-based models’ when these terms are introduced early in the paper, to put the methods and findings in context. These terms may not be familiar to the reader.

RESPONSE: We appreciate this suggestion, which was also raised by the other reviewer. We added brief definitions in the introduction to better orient the reader to these terms:

“Systems science is an interdisciplinary field and approach to inquiry that focuses on understanding interrelated and interacting entities that form a unified whole. Simulation modeling is the process of creating and analyzing a digital prototype to emulate a real-life system, which is a tool often used in systems science.”

Submitted filename: Response to Reviewers 9_19_20.docx

Click here for additional data file.

9 Oct 2020

Novel participatory methods for co-building an agent-based model of physical activity with youth

PONE-D-20-12547R1

Dear Dr. Frerichs,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

I thank you for revising your manuscript and for considering PLOS ONE.

Kind regards,

Emma Louise Giles

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

28 Oct 2020

PONE-D-20-12547R1

Novel participatory methods for co-building an agent-based model of physical activity with youth

Dear Dr. Frerichs:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Emma Louise Giles

Academic Editor

PLOS ONE