It is essential to connect: Evaluating a Science Communication Boot Camp.

Scientific knowledge has expanded dramatically in the 21st century. Yet, even in science where there is large consensus among the studies-evolution by natural selection, for example, or the human basis of accelerated climate change-the public and policymakers are not always in agreement with the science. To bridge this gap, scientists and educators need to connect and engage with diverse audiences with varying levels of science literacy. Communication scholars have identified several effective tactics to communicate effectively with non-specialist audiences. However, our sometimes-siloed thinking in science and higher education discourages sharing this knowledge across disciplinary lines. Furthermore, many training programs focus on educating about which communication strategies work, but they fail to provide participants with the opportunity to develop the skills required to listen effectively and respond in an engaging way. To that end, we created the Science Communication Boot Camp (SCBC) with support from an American Association for Anatomy innovations grant. The 3-day program engaged and immersed participants in training designed to develop audience-centered communication, distill scientific concepts into meaningful narratives, and connect effectively with the public, collaborators, and policymakers. Based on participant surveys at three timepoints (preworkshop, postworkshop, and 2-year follow-up), the SCBC was effective in helping participants to increase their communication skills and willingness to engage with the public and other non-specialist audiences.

INTRODUCTION

Science has a communication problem. As scientific fields have become more complex, the public has been left trying to understand an exponentially increasing pace of science, which leads to low scientific literacy, misinformation (unintentionally incorrect), disinformation (intentionally incorrect), and loss of confidence (Peretti‐Watel et al., 2015). A quick reflection on the real‐time communication challenges presented by the COVID 19 pandemic reinforces the point: what does the science conclude about masks, vaccines, testing protocols, lockdowns, and so on, and why does the science keep changing? Each of these opportunities for communicating science has suffered from communication breakdowns between scientists, policymakers, and the public (Jaiswal et al., 2020). Survey research from the Pew Center prior to the pandemic demonstrates that there is a wide gap between scientists' and the public's perception of how science is used to inform government policy (Funk et al., 2019; Funk & Rainie, 2015). Funk and Rainie (2015) showed that 84% of surveyed scientists said that limited public knowledge about science was a “major problem,” while 63% and 55% of nonscientists US adults, respectively, believe that the scientific method produces accurate results and that scientists' judgments are based solely on facts. Unsurprisingly given the current state of US politics, the nonscientist US adult opinions described above can be delineated by political party affiliation, with lower percentages in both categories among Republican‐leaning adults compared to Democrat‐leaning adults (Funk & Rainie, 2015). Scientists and the public need to work together to make sense of scientific information, but scientists have been slow to acknowledge the importance of clear communication with the public. To that end, the National Academy of Sciences, Engineering, and Medicine (NASEM) has published several special editions of their proceedings on “The Science of Science Communication” over the last several years, including a foundational report entitled “Communicating Science Effectively: A Research Agenda” (NASEM, 2012, 2014, 2017), all of which explicitly call for more research on the most effective means to communicate complex scientific information to a diversity of nonspecialist audiences. Yet, the communication gap between scientists, policymakers, and the public has seemingly widened with the pandemic.

Recognizing the importance of effective communication of science, universities have begun training scientists and learners using techniques of applied improvisational theater (AIT) to help them speak more spontaneously, responsively, and engagingly. Programs such as the Alan Alda Center for Communicating Science at Stony Brook University use innovative curricula based on improvisational theater games to help scientists transform their approaches to talking with the public about complex research. Likewise, programs such as those at Northwestern University (Watson & Fu, 2016) and Indiana University School of Medicine (Hoffmann‐Longtin et al., 2018) use AIT to teach health care providers to communicate more effectively within their teams and with greater empathy and clarity with their patients.

In 2017 and again in 2020, the Board of Directors of the American Association for Anatomy (AAA), the member organization for which The Anatomical Record is the flagship journal, committed to a strategic plan that places a high value on the importance of science communication with the public and policymakers, generally, while also committing to “equip AAA members to inspire scientific curiosity through public outreach and science communication” (AAA, 2020). With that commitment clearly articulated, the organization funded the establishment and delivery of the inaugural AAA Science Communication Boot Camp (SCBC) through the 2017 Innovations Grant program. SCBC is a 3‐day, hands‐on professional development workshop in effective science communication strategies. The inaugural workshop was convened in Indianapolis, IN, from July 10 to 12, 2019, with the following goals: (a) to equip a cadre of AAA members with a toolkit of methods proven to enhance the effectiveness of science communication: dynamic listening, message distilling, storytelling, and audience connection all derived from the principles of improvisational theater; and (b) to provide a collegial and interactive environment where SCBC participants can practice the communication skills they learn in real time, network with others, and train to communicate science with the media and with policymakers. The purpose of this study is to evaluate the extent to which the learning outcomes of the inaugural SCBC were achieved, using a mixed‐methods approach (Creswell & Clark, 2017) following the Kirkpatrick model (Kirkpatrick & Kirkpatrick, 2006). This study was granted exempt status from the Indiana University Institutional Review Board (Protocol #1906821907).

DESCRIPTION OF THE SCBC

The need for scientists to communicate effectively reaches far beyond the bench or the discipline. To that end, we developed a 3‐day workshop designed to train AAA members to communicate complex scientific information to those outside the discipline and bring this training back to colleagues at their campuses. Days 1 and 2 of SCBC focused on formal communication training from the faculty of the Alan Alda Center for Communicating Science at Stony Brook University. The Alda Center training borrows techniques from AIT to teach participants to effectively connect with their audience in real time, to distill their message into language that is free from jargon and easily understood by the public, and to use storytelling as a mechanism for these tasks. Day 3 of SCBC focused on communication with media, visual communication, and communication with policymakers. A morning panel of local and regional print, radio, and television reporters and media professionals spoke to participants about effective ways to communicate the importance of their work to general audiences. The media panel was followed by a session on the importance of visual communication. The afternoon included two sessions where SCBC participants were able to apply techniques learned over the previous 2 days: mock media interviews, and advocacy training for effective communication with policymakers led by staff from the Office of Public Affairs at the Federation of American Societies for Experimental Biology (FASEB).

Participants

Registration for SCBC was open to all AAA members in good standing and registration was free. The only costs incurred by participants were the costs of travel and lodging. All applicants were accepted and 32 participants attended the program, 29 of which were scientists and educators, and 3 were staff members of AAA or FASEB. Of the 29 scientists and educators, 25 (86%) identified as women. This trend in participation is in line with similar improvisation‐based science communication training programs (O'Connell et al., 2020). O'Connell et al. (2020) assert that experiences of gender bias in communicating science influence women's communication choices. Such experiences may have influenced the choice to participate in SCBC, for example. Participants were distributed across career stages, with 10 students (34%), 2 postdocs (7%), 1 lab director (4%), 1 senior lecturer (4%), 5 assistant professors (17%), 5 associate professors (17%), and 5 professors (17%).

Procedures

Educational outcomes of the SCBC were evaluated using the first three levels of Kirkpatrick's model (Kirkpatrick & Kirkpatrick, 2006). A widely‐used tool to evaluate training interventions, the Kirkpatrick model (also called Kirkpatrck's four levels of training evaluation) asks researchers to gather data in four subsequently more complex levels as an approach to evaluate a program's quality and effectiveness (Kirkpatrick & Kirkpatrick, 2006). We gathered both quantitative and qualitative data to allow us to explore the nuances of participants' experiences and ask follow‐up questions to gain additional depth and understanding.

Kirkpatrick's Level 1 (participants' reaction to the learning experience) was gauged during the interactive elements of the workshop. Each activity included a debriefing session using the “what, so what, now what” framework to gather participants' reflections in real time and adapt the content and strategies accordingly (Brown et al., 2015). Kirkpatrick's Level 2 (participants' learning and change in attitudes and skills) was evaluated using a pre‐ and postsurvey model, in which participants were asked about their level of preparedness and knowledge of effective science communication techniques. These surveys were administered at the time of registration for the workshop and 1 week after attending. In addition, the SCBC was evaluated with a qualitative follow‐up survey conducted 2 years after the event.

Change in behavior (Kirkpatrick's Level 3), defined as a willingness of learners to apply new knowledge and skills when they return from the training (Kirkpatrick & Kirkpatrick, 2006), was evaluated through a series of four, open‐ended questions included in the 2‐year follow‐up survey to better understand the ways in which participants may have incorporated the material from the SCBC into their work. This allowed us to add richness to the data and represent the participants' varied experiences with the program materials (Tracy, 2010).

We conducted an interpretive thematic analysis to make meaning across the four open‐ended questions. Two authors (K.H.L. and R.W.) read the qualitative responses closely to familiarize ourselves with the data. We then inductively coded the data using the constant comparative method (Glasser & Strauss, 1967; Tracy, 2020). After initial coding was complete, two authors (K.H.L. and R.W.) met to discuss the codes we each generated and developed a codebook. To finalize the themes, we engaged in a collaborative sensemaking process and then returned to the data to fine‐tune the themes and confirm they represented the data. A third author (J.M.O.) reviewed the data and analysis to triangulate the themes (Tracy, 2020). As Tracy (2020) indicates, we attended to the qualitative trustworthiness of the study by using “multiple types of data collection, at multiple points in time, with multiple co‐researchers, in order to construct a multi‐faceted, more complicated, and therefore more credible picture of the context” (p. 237). We then selected participant responses to represent each theme.

Statistical methods for quantitative analysis

Quantitative survey responses were collected for nine questions on a five‐point Likert scale (1 = strongly disagree, 5 = strongly agree). There is ongoing debate about the most appropriate statistical method to analyze Likert data (e.g., Bacchetti, 2002; Carifio & Perla, 2008). Because Likert data are ordinal, they typically violate assumptions of distribution normality required for parametric statistical tests. Norman (2010), however, demonstrated succinctly that “parametric statistics can be used with Likert data, with small sample sizes, with unequal variances, and with non‐normal distributions, with no fear of ‘coming to the wrong conclusion’. These findings are consistent with empirical literature dating back nearly 80 years. The controversy can cease (but likely won't)” (Norman, 2010, p. 631). Therefore, we used ANOVA to compare group mean Likert scores for each question across the three time points (preworkshop survey, postworkshop survey, and 2‐year follow‐up survey), with alpha set at 0.05.

A bigger concern than whether to use parametric or nonparametric analysis of variance is how to analyze equivalent data collected over multiple time points. A repeated measures statistical design, where changes in each individuals' responses to survey items at the three timepoints are assessed, is the most appropriate approach but was not available for this analysis because missing data at any timepoint excludes an individual's data from other time points. Yet, a repeated measures approach would account for the nonindependence of a given individual's data from timepoint to timepoint. To account for this, we applied a Bonferroni correction to the alpha value for the ANOVA, and interpreted statistical significance when p <  0.017 (alpha = 0.05/3, for three timepoints).

Statistically significant ANOVA analyses were followed by pairwise post hoc t‐tests to examine differences in Likert scores between (a) the pre‐ and postsurvey data, and between (b) the postsurvey and follow‐up data. Alpha values for pairwise t‐tests were adjusted with Bonferroni correction so that post hoc results were considered significant when p <  0.025, to account for multiple comparisons.

RESULTS

This study set out to evaluate the educational outcomes of the AAA SCBC according to Levels 2 and 3 of Kirkpatrick's model (Kirkpatrick & Kirkpatrick, 2006). Level 1 of the model, or the reaction of participants to the interventions, was evaluated in real time during the event. The SCBC organizers conducted informal needs analysis at the end of each day via a group reflection, to ensure that the content and format was meeting participants' needs. For example, during each activity debriefing, we solicited feedback from attendees using the “what, so what, now what” framework described above. Then, based this feedback from participants, we addressed length of particular topics covered and amount of interaction versus reflection time in the schedule. Level 2 of the model, which examines changes in attitudes and skills, is evaluated here by comparing preworkshop survey data with postworkshop survey data. Level 3 of the Kirkpatrick model examines changes in behavior and is evaluated through interpretive thematic analysis of the four open‐ended questions on the follow‐up survey. Our results follow for each of these levels below.

Quantitative survey data (Kirkpatrick Level 2)

Response rates for the preworkshop survey, postworkshop survey, and 2‐year follow‐up survey were 79%, 89%, and 45%, respectively. All survey questions exhibited a statistically significant positive change in mean Likert score between the preworkshop and postworkshop survey (Table 1). In their responses, participants' attitudes toward effective strategies for communicating science as well as their perceived skillsets to be effective communicators, were enhanced through participation in the SCBC program. Moreover, the 2‐year follow‐up mean Likert scores on the same questions remain statistically higher than the preworkshop survey means and are not statistically different from postworkshop survey results. These data clearly demonstrate that the SCBC interventions were effective at changing behavior even 2 years after the workshop, suggesting that the approaches to science communication taught have long‐lasting positive impacts on participants' self‐reported efforts to communicate their work to nonspecialist audiences.

TABLE 1

Five‐point Likert scores (1 = strongly disagree; 5 = strongly agree) for the pre‐ and postworkshop surveys and the follow‐up survey 2 years later

Survey question	Presurvey mean (SD) Likert score	Postsurvey mean (SD) Likert score	2‐Year follow‐up mean (SD) Likert score
I know how to have meaningful conversations with others	4.35 (0.65)	4.96 (0.20)	4.92 (0.28)
I use effective strategies to communicate my ideas	3.61 (0.84)	4.92 (0.27)	4.77 (0.44)
I am aware of the importance of listening to understand my audience's needs	4.39 (0.89)	4.96 (0.20)	4.92 (0.28)
I feel confident in my ability to listen to audience concerns	3.61 (1.03)	4.62 (0.57)	4.54 (0.52)
I am aware of opportunities to use rich descriptions and analogies to enhance empathy with my audience	3.61 (1.03)	4.88 (0.33)	4.62 (0.51)
I use examples and analogies to help the audience understand complicated information	4.09 (0.90)	4.62 (0.50)	4.69 (0.48)
I learned how stories help a speaker connect with an audience	4.26 (0.96)	4.92 (0.27)	4.85 (0.38)
I built confidence in my ability to gauge audience responses and modify my communication to better meet their needs	3.74 (0.86)	4.62 (0.50)	4.46 (0.66)
I used voice and body language to build trust with the audience	3.39 (0.94)	4.69 (0.55)	4.38 (0.65)

Note: Bolded scores are significantly different from presurvey mean scores (p < .025).

Thematic analysis of open‐ended survey questions (Kirkpatrick Level 3)

We identified eight themes within the survey responses: (a) defining effective communication, (b) storytelling, (c) moving from “knowing” to “knowing how”, (d) reaching a broader audience, (e) building a community of scientists, (f) feeling responsible and empowered to make science communication a part of their work, (g) new understanding of the power of communication, and (h) difficulty making time for science communication. Table 2 summarizes each theme, a description of the theme, and sample quotations from the participants to illustrate how they appeared in the data.

TABLE 2

Thematic analysis of open‐ended follow‐up survey questions

Theme	Example quote
Defining effective communication Several respondents explicitly mentioned “effective communication” when asked about the “take‐home” message from the boot camp program. Some participants defined effective communication as a necessary skill for connecting with others and as requiring effort and practice.	“Effectively communicating science is based on developing a connection with your audience and sense of collaboration.” “It takes practice and deliberate effort to communicate effectively.”
Storytelling The use of narrative to describe one's research, teach students, and discuss their work with others was reported to be employed more often by participants after attending the bootcamp program. Many were already aware of storytelling as a method of communication but either increased their use of the skill or used it in new ways in their professional work and personal lives.	“I've always been a fan of using evocative storytelling, but I learned more about how to use it in communicating my work. And I incorporated that into my poster presentations, lab meetings, and any other presentations or even casual conversations about my work.”
Moving from “knowing” to “knowing how” Through describing how they were incorporating SciComm into their work and reflecting on their memories from the bootcamp, participants demonstrated how the experience shifted their understanding of SciComm from conceptual to practical. Participants reported feeling capable of using and developing their own skills as science communicators. A general impression was noted that SciComm is available and valuable to all.	“Anyone can get better at communicating science, and there are many reasons to do so.” “Communication is something that benefits greatly from continued development. I'm sure there is a lot more work to do to develop this skill personally.”
Reaching/connecting with a broader audience Many participants claimed communicating with others, outside the scientific community, was important to them. Some emphasized that they had cultivated an increased awareness of the communities they exist within and that exist around them and recognized a need to tailor their message to make it accessible to those individuals.	“My perception of my work is not always what is communicated with those around me. I need to be mindful of how the vocabulary I am comfortable with is not always what my community understands or appreciates.” “It is essential to connect to the audience in order to build a trusting and empathetic relationship with them.”
Building a community of scientists Through their experience in the bootcamp program, several participants reported making connections with and learning from colleagues, who also participated, as an advantage of attending. Others expressed a desire to see more scientists seek out science communication opportunities to create a broader community of individuals sharing their work.	“This workshop was great at building community and the exercises were effective at helping people communicate better. I think this course is really valuable for both scientists who want to communicate their science and for educators as they find meaningful ways to explain things in class.” “I'd like to see more people do this, and while the refresher would not be as necessary for me, I'd probably do it as a show of solidarity and hope to learn something new”
Feeling responsible and empowered to make science communication a part of their work Participants reported feeling a sense of responsibility to incorporate science communication into their professional work in different ways. One participant stated the experience changed how their perceived their role as a scientist in society.	“It is awesome and has informed my whole professional and personal identity” “It is my responsibility to engage with an audience using storytelling and empathy.”
New understanding of the power of communication Participants reimagined the impact that communication can have on an audience as well as the impact they can make through effective communication of their work. Some participants made statements that focused less on the content or subject being communicated but on connection with the audience, nonverbal communication, and enthusiasm as tools.	“The SciComm Boot Camp gave me the confidence to share my research through social media. It reinforced my understanding of ways to connect with an audience and improve effective communication.”
Difficulty making time for science communication Concern over the time and effort necessary to use science communication techniques regularly was referenced by several participants, especially in the context of the COVID‐19 pandemic. Some lamented that due to time constraints in their personal and professional lives, science communication was not a priority. Although, a general sense of encouragement for those who wanted and had the time to pursue SciComm was expressed by these participants.	“I would not attend [a refresher course] as I think those who want to commit time to science communication should be prioritized.” “It has been difficult to focus on developing science communication, as 2020 was a very difficult year for obvious reasons.”

As illustrated by the qualitative data, many participants returned to their home institutions and implemented these communication methods in courses and programs, thus self‐reporting that they changed their behavior (Kirkpatrick Level 3) as a result of the workshop. Although this indicates broad interest and applicability, more research is needed to determine Kirkpatrick's Level 4, return on investment.

DISCUSSION

Skepticism about scientific integrity is complex and widespread among the American public (Funk et al., 2019). The AAA SCBC, convened inaugurally in July 2019, was designed to enhance participants' toolkits for effective communication and engagement with the public. The SCBC is built upon rhetorical education and communication theory, which provide essential complements to the approach of applied improvisation, the predominant training mode of the workshop. For example, communication theory teaches a sophisticated understanding of audience as cocreator of meaning in the communication process (Pearce, 1989). Effective communication practice responds spontaneously to the needs of an audience and takes advantage of the resources available in the interaction (Pearce, 1989). This understanding provides a more critical approach to communication practice. It treats communication as a translational process of meaning‐making, rather than solely information dissemination. Merely delivering a speech with more flair or better slides does not improve the communication gap with the public. Science experts need the ability to “perspective take” with their audiences. They need the skills of empathetic imagination, an ability to think like their audience, and responsiveness to an audience's needs and interests. This theoretical grounding guided the development of the AAA SCBC program.

Several expert science communicators serve as exemplars for the importance of public communication such as Neil deGrasse Tyson, Bill Nye, and Brian Greene. However, these experts can too easily make scientific communication seem like an innate skill reserved only for a few, naturally gifted celebrities. Communication education stresses that everyone requires these skills for communicating to broad public audiences and that anyone can develop these ways of thinking and speaking through practice, experience, and education (Tallapragada, 2018). This evaluation of the SCBC program underscores the idea that anyone can develop the skills necessary for effective science communication. Furthermore, and perhaps most important, the skills that SCBC participants learned at the program are still important and recalled after 2 years (or in pandemic life, what feels like a decade). Participants still remember and implement the most important skills learned at the bootcamp, which suggests an incredible return on investment from AAA. Moreover, the SCBC built a cohort of like‐minded AAA members who recognize the importance of engaging with the public around science in accessible ways. Programs built by major international member organizations, which seek to empower their members with skills to engage effectively with nonspecialists, can change the culture of the discipline and change public perception as well. The leadership of the AAA deserves acknowledgement for recognizing the importance of public engagement and for supporting its members in their pursuit to become more effective at it; the public and the scientific enterprise would all benefit if more professional associations followed AAA's example and adopted a similar approach to making science more accessible to the public.

It is clear, science communication needs expert and empathetic voices. Without them, misinformation, or worse disinformation, can quickly overtake public scientific discourse if scientists are unable to effectively communication their work. The internet and social media provide powerful tools to communicate and advocate for science, yet many scientists have avoided engaging the public online or even face‐to‐face. In part, this is because of an assumption that science and advocacy are fundamentally incompatible: science is concerned with objectively observing natural processes whereas advocacy is inextricably linked to subjectivity and a desire for the way the world ought to be (Nelson & Vucetich, 2009). Some scientists worry that engaging in advocacy will harm their credibility as objective scientists, even though data suggest otherwise (Kotcher et al., 2017). Without expert voices communicating science with the public and policymakers, fields like healthcare, climate science, and evolutionary biology have been overrun with conspiracy theories and misinformation. Because scientific fields continue to increase in complexity, the American public—whose tax dollars fund federal research grants—is left wondering what and who to believe about science. Programs such as the AAA SCBC will help make sure the public is not left behind.

AUTHOR CONTRIBUTIONS

Krista Longtin: Conceptualization (equal); data curation (equal); formal analysis (equal); investigation (equal); methodology (equal); project administration (equal); writing – original draft (equal); writing – review and editing (equal). Rebecca Wisner: Formal analysis (equal); writing – original draft (equal); writing – review and editing (equal). Jason M. Organ: Conceptualization (equal); data curation (equal); formal analysis (equal); funding acquisition (equal); investigation (equal); methodology (equal); project administration (equal); visualization (equal); writing – original draft (equal), writing – review and editing (equal).