Enhancing Graduate Students' Ability to Conduct and Communicate Research through an Interdisciplinary Lens.

This research is a part of a longitudinal study of the Computation and Mathematics for Biological Networks (COMBINE) program at the University of Maryland. The mission of COMBINE is to train doctoral students from a wide range of fields to pursue interdisciplinary research. Here, we focus on one component of COMBINE, a semester-long course titled Data Practicum at the Intersection of the Physical, Computer, and Life Sciences. The goal of this study was to explore the effectiveness of the teaching practices that were used in the Data Practicum. We investigated their impact on graduate students' confidence to conduct research through an interdisciplinary lens and to communicate their research to diverse audiences. We used validated pre- and post-course online surveys, in-class observations, collection of artifacts, and interviews. Interviewed students and instructors highlighted the course's iterative process, peer review system, and unique incorporation of outside research already being conducted by students as the most impactful aspects of the course. Based on students' reports and artifacts, the Data Practicum was successful in helping them to communicate their research visually, orally, and in text to a wide and varied audience, to critically review others' work, inside and outside their discipline, and to develop awareness of research in other disciplines. We observed that it is possible to enhance interdisciplinary communication skills through an iterative teaching approach that gives students a chance to incorporate feedback from multiple sources. This course could serve as a model for other graduate programs wishing to increase training in interdisciplinary skills.

INTRODUCTION

Researchers caution that traditional Ph.D. programs often fail to develop interdisciplinary knowledge, skills, and research approaches that are required in current workplaces (1, 2). Reasons for these shortcomings include insufficient mentoring, failure to acknowledge useful methodologies or perspectives from outside academic fields, and too much emphasis on narrow or siloed research (e.g., 2, 3, 4). Researchers (5) argue that interdisciplinary teams can often innovatively approach difficult tasks that no individual investigator could solve. Methodology and tools, particularly those found in computer science, may be borrowed between disciplines and applied to new problems that could use an interdisciplinary perspective (6). For example, cellular biologists can benefit from communicating what they know about actin-myosin filament networks to computer scientists, who can use computer simulations to model network interactions of actin-myosin filaments based on molecular composition.

An interdisciplinary research approach involves both specialized research techniques and communication (teamwork) between scientists (7, 8). Students should not only communicate material to one another, but also negotiate how to most effectively frame, synthesize, model, and present data (9). Klein (9) claims that the ability to navigate such complex negotiation should be a primary goal of graduate and even undergraduate research programs. However, creating researchers who are capable of working from an interdisciplinary perspective requires training them to integrate information, strategies, tools, conceptualizations, and theories from two or more disciplines in order to see or solve beyond a single lens (10).

Through an assessment of over 100 Integrative Graduate Education and Research Traineeship (IGERT) National Science Foundation (NSF)-funded proposals, Borrego and Cutler (11) established four desired student learning outcomes for Ph.D. students: “contributions to the technical area, broad perspective, teamwork, and interdisciplinary communication skills” (p. 355).

To this end, the Computation and Mathematics for Biological Networks (COMBINE) program, funded by the National Science Foundation (NSF) Research Traineeship Program (NRT), was created. The mission of COMBINE is to train doctoral students from a wide range of fields to pursue interdisciplinary research in the physical/mathematical, computer, and biological sciences. This study concentrates on one component of the COMBINE program, a semester-long course called Data Practicum at the Intersection of the Physical, Computer, and Life Sciences. Prior to the course, students design a well-defined short-term interdisciplinary research project with their advisors. This research project serves as a basis for their individual coursework and final paper. During the course, students repeatedly practice oral and written presentations given to their diverse classmates. Students also work in a small mixed-discipline team to present a seminal paper. Overall, the Data Practicum was tailored to enhance the following skills that are highlighted in the literature as crucial for the development of interdisciplinary research (11, 12): integrated, interdisciplinary problem solving, teamwork, and communication to diverse audiences.

Integrated, interdisciplinary problem solving

Klein (9) defines interdisciplinarity using the skill of problem solving. According to her definition, “Interdisciplinarity is a means of solving problems and answering questions that cannot be satisfactorily addressed using single methods or approaches” (p. 196). Similarly, Bruhn (13) defines interdisciplinarity as “two or more persons from different disciplines who agree to study a problem of mutual concern, and who design, implement, and bring to a consensus the results of a systematic investigation of that problem” (p. 59). Researchers have noted that the ability both to solve interdisciplinary problems and to understand methodological limitations enhances students’ creativity (14), independence, and critical thinking (12).

Rhoten and Pfirman (15) differentiate between four possible approaches to interdisciplinary problem solving: 1) A single person takes the responsibility to incorporate knowledge, data, and tools from different disciplines to solve a problem; 2) Several researchers from different disciplines build a network and each brings to the table knowledge, data, and tools from his/her own discipline; 3) A group of researchers create a “new field” in the intersection of one or more disciplines; and 4) A group of researchers from specific disciplines collaborate to respond to a specific problem, such as a social concern (e.g., climate change).

Teamwork

There is broad consensus in the literature that team-based collaboration is becoming a norm in science and engineering (11). Dunbar (14) stated that no longer is the “lone scientist under the light bulb the norm for science. Rather, groups comprised of members of different levels of experience and different scientific backgrounds form the basis of contemporary science” (p. 2). Nevertheless, teamwork requires social and intellectual collaboration that is not always easy for members to accept (16). Recent literature on science education strongly encourages undergraduate instructors to incorporate group work assignments into their classroom to allow students to build teamwork skills for their future careers (17). It is noteworthy that students in undergraduate programs often resist group work activities for a variety of reasons, such as competitiveness, difficulty relying on each other’s knowledge, and uneven division of responsibilities between team members (17). Interdisciplinary teams’ members will sometimes feel less threatened by competition and more willing to collaborate since every member brings his/her own disciplinary knowledge and is perceived as an expert in his/her own field (5).

Communication to diverse audiences

Researchers who collaborate on interdisciplinary research are required to communicate (orally and in writing) their own discipline’s ideas to other researchers from different disciplinary backgrounds (5). They therefore need to “acquire language” that allows them to “move comfortably across disciplinary boundaries” (5, p. 76). Language and terminology differences between disciplines have been cited as the most common challenge to interdisciplinarity (12). While earlier literature mostly discussed challenges in communicating with scientists from disciplines other than one’s own (5), recently the literature extends the discussion to include a broader range of stakeholders (diverse audience), including practitioners, politicians, and industrial personnel, suggesting that effective interdisciplinary training must include mechanisms of effective communication to nonscientific audiences in addition to scientific audiences outside a given area of expertise (18). Learning how to communicate knowledge in general, and especially to a diverse audience, is an iterative process that requires repeated feedback from peers, experts, and novices (5).

Using the above literature background, here we explored how the Data Practicum improves graduate students’ ability to 1) communicate their research visually, orally, and in text to a wide and varied audience, 2) critically review others’ work, inside and outside their discipline, and 3) develop awareness of research in other disciplines.

METHODS

University of Maryland is a public, research-intensive university with nearly 40,000 students enrolled in over 200 undergraduate and graduate programs. The COMBINE program was designed to attract students from four major graduate departments: mathematics, physics, biology, and computer science. The data practicum course was taught twice, in spring 2017 (N=13) and spring 2018 (N=11). Student demographics for the two sessions are presented in Table 1.

TABLE 1

Student demographics.

	Spring 2017 (N=13)	Spring 2018 (N=11)
How far into program
≤ 1 year	4	3
2–3 years	8	6
4 years	1	2
Field of study
Biological science	7	4
Computer/computational science or engineering	5	3
Physical/mathematical sciences	1	4
Gender (male/female)	9/4	7/4
Race
White	6	6
Asian-American/Pacific Islander	3	1
Latino	2	1
Other or did not answer	2	3

It should be noted that the authors/evaluators for the study are not the course instructors. This work has been approved by the University of Maryland Institutional Review Board (IRB protocol 927445-4).

The data practicum structure

The class was led by two instructors and met twice a week for 75 minutes for a total of 15 weeks. About half of the class time was devoted to introductory lectures for the topic of the week (e.g., peer-review process, preparing elevator speeches, explaining data visualization). Almost every lecture topic was followed by a homework assignment. From the second week until the final week, individual students gave presentations and turned in written assignments. The students were divided into two groups assigned to present during either the first or second meeting each week. This ensured that enough time could be allocated to each student during in-class presentations. Table 2 lists in-depth classroom activities. As culminating projects, all students submitted a final paper and a poster for the symposium held the winter after the course ended.

TABLE 2

In-class lectures and students’ presentations.

Week	Lecture Topic	Students’ Presentations
1	Course introduction and proposal development
2	The peer review process	Introductory presentations (Groups A+B)The purpose of these presentations was to introduce the student’s data set and research questions.
3	Instructions for writing an abstract	Assignment: one-page project proposal due Week 3
4	How to prepare a short elevator speech	“Surprise” elevator speech (Groups A+B)Students were given ten minutes in class to prepare a three-minute elevator speech about their research for a “general audience” with emphasis on context of research and potential application.
5	Data visualization, schematics, and simple plots	Planned elevator speech (Groups A+B)Students were given a week to prepare for their elevator speech.
6–7	How to give a scientific presentation	Research update presentations (Groups A+B)Short oral presentations were given on progress and results of individual research projects.Assignment: Abstract due Week 6
8	Characteristics of high-impact (seminal) research paper	Assignment: Draft of introduction and background due
9–10	Data visualization, complex information	Seminal paper group presentationsGroups of 3 or 4 students reviewed and orally presented their critiques of a previously selected seminal, data-driven, interdisciplinary research paper.Assignment: Outline of final paper due Week 10
11–12	How to prepare a scientific poster	Oral presentations for interdisciplinary audience (Groups A+B)A 10-minute oral presentation for an interdisciplinary science audience. Students covered motivation, background, key questions, methods, preliminary results, and future directions.
13–14	How to refine and edit your work	Edited oral presentations for interdisciplinary audience (Groups A+ B)Students revised their oral presentations based on instructors’ and peers’ in-class and discussion board comments.Assignment: Poster due Week 13
15	How to publish your research	Assignment: Final Paper due

Throughout the semester, students received instructor feedback, peer reviews, discussion board comments, and other forms of assessment of their work. For most of the written assignments, at least one of the two instructors gave feedback on the document. For some of the written assignments, students were assigned to list strengths, weaknesses, and general comments for two other students from a different primary discipline. These assigned reviewers for a student changed from one assignment to the next. Discussion board comments written by peers were left for each student on the class website after in-class presentations. These comments were public to the class until the discussion board period closed.

Research instruments and data analysis

To collect baseline data on students’ background and experiences, we used a validated pre-course survey that was developed by the Language Science NRT team at University of Maryland. The survey items were developed based on the extant literature on doctoral student development as well as the seven goals of the NRT program. The Language Science NRT team tested and validated the survey by gathering data from University of Maryland and three other universities (19). The pre-course survey included questions such as “At this time, how confident do you feel in your ability to collaborate with scientists outside of your field of expertise?” ranked on a Likert-type scale of 1 = Not at all, 2 = Not much, 3 = Somewhat, 4 = To a good extent, and 5 = To a great extent. The survey also included open-ended questions, such as “What skills/methods/expertise do you hope to gain (if anything) by connecting to two disciplines outside of your primary discipline?” (The survey can be obtained from the authors upon request.) The post-course survey (Appendix 1) was based on O’Meara and Hall (19) but was modified to include questions specific to our program (see example in Fig. 1). Face validity of the adapted survey was established using a science education faculty member, the two instructors of the course, and a public health graduate student. The surveys were administered during class time at the beginning and end of the semester through Qualtrics.

FIGURE 1

Students’ average responses (spring 2017, N=13; spring 2018, N=10) to the Likert-type question, “What skills did you gain or improve from taking the course?” (1=Not at all, 2=Not much, 3=Somewhat, 4=To a good extent, and 5=To a great extent). Error bars indicate standard deviations.

The pre and post surveys were analyzed quantitatively or qualitatively depending on the nature of the questions. For the Likert-type and multiple-choice questions, we calculated averages and standard deviations. We used Wilcoxon sign rank analysis to compare improvement from pre to post survey responses. The analysis was conducted in SPSS v23. Responses to the open-ended questions were analyzed qualitatively using an inductive approach (20, 21), in which we grouped related responses into subcategories that could be quantified. The authors categorized the responses separately and then discussed their categories until they came to agreement. Their inter-rater agreement was around 90%.

During weeks four and five, students were asked to present three-minute elevator speeches describing their research to the class. These presentations gave the students the opportunity to experience preparing their research for a diverse audience and receive feedback from the instructors and their peers, providing a baseline for improvement. The instructors scored the presentations with a rubric consisting of the following seven variables: 1) Describes the basic research problem and why it matters (total points – 2.5), 2) Describes the potential solutions (i.e., methods and/or approach (total points – 2.0), 3) Conveys the potential impact (total points – 2.0), 4) Accessible to a general audience (total points – 2.0), 5) Flows well (total points – 1.5), 6) Deduction for not meeting time constraints, and 7) Bonus for particularly impressive practice run or particularly striking improvement from practice run. On weeks 11 to 14, toward the end of the course, students were asked to give 10-minute oral presentations to the class describing their research. Again, the instructors scored the students, using a slightly different rubric from the one used for the elevator speech. The rubric for these presentations comprised the following variables: 1) Motivation (total points – 2.5), 2) Research questions (total points – 2.0), 3) Context (total points – 2.0), 4) Methods (total points – 2.5), 5) Results (total points – 2.0), 6) Future directions (total points – 2.0), 7) Targeting diverse audience (total points – 1.5), 8) Presentation style (total points – 1.5), 9) The use of visual aids (total points – 2.0), 10) Bonus, and 11) Time deduction. To measure students’ improvement along the course, we compared their scores on the elevator speech and the end-of-semester oral presentations in terms of the two variables Research Question and Presentation. Since the number of partial points was not the same in the two rubrics, we calculated each student score as a percentage of the possible highest score.

At the end of the semester the authors interviewed the instructors. Several months after the end of the semester, the second author interviewed three students, one from each of the three major disciplines. The instructor and student interviews were semi-structured (see interview protocol, Appendix 2). The authors reviewed the data for themes independently, then discussed findings until they reached agreement. During the semester, we collected and utilized discussion board threads, instructors’ feedback, video recordings of student presentations, and reflection notes on oral and written presentations.

RESULTS

Students’ prior experiences

We report here in aggregation on data collected from spring 2017 and 2018. When asked on the pre-course survey why they were taking the course, 13 out of the 22 students who responded to this question (59%) mentioned the desire to increase skills in analysis, modeling, or visualization of data. Seven students (32%) wrote that they were seeking background knowledge of another field; this included “a better mathematical foundation for biological networks,” and “life science knowledge that can inform my modeling.” Four students (18%) wrote they were hoping to gain interdisciplinary communication skills (Table 3).

TABLE 3

Student responses to the open-ended question (prior to the course), “Why did you take the course?”

	Number of Responses (Percentage)	Examples of Students’ Responses
Desire to increase skills in analysis, modeling, or visualization of data	13 (59%)	“Under the hood” knowledge of the computational basis behind network analysis I conduct for my graduate and future research (Biology student)
Seeking background knowledge of another field	7 (32%)	Broader biological knowledge, specifically neuroscience (Computer science student) Learning how to perform complex calculations using new computational tools (e.g., reservoir computing, CNNs) to learn about the physics of cells in many contexts (Biology student)
Gain interdisciplinary communication skills	4 (18%)	Primarily, the understanding of concepts and methods that could expand my research opportunities and enable communication and collaboration with experts in these respective areas (Biology student)

N=22. Not all students answered the question, and some students’ answers fit more than one category.

When asked how often they attended various professional development activities, students most frequently reported attendance at research talks given by faculty or students (Table 4). Additionally, most students (n=18 [78%]) reported that, in the 12 months prior to the course, they served as research assistants for one or more semesters. Despite their research experience, students reported receiving far less formal training for skills taught in the Data Practicum. Seventeen students (74%) reported that they had no formal written communication training, 16 students (70%) had no formal oral communication training, and 14 (61%) had no formal research skill training during the 12 months prior to the course.

TABLE 4

Students’ reported experiences with professional development activities in the last year prior to the course.

	None	One-Time Event	Multiple Times	Ongoing Throughout Semester or Academic Year
Formal written communication training	17	4	2	0
Formal oral communication training	16	4	3	0
Formal research skill training	14	2	4	3
Career advice workshop/seminar	11	5	7	0
Outreach activitiesa	9	6	6	1
Reading groups	7	4	7	5
Research talks given by students	3	2	10	8
Research talks given by faculty	0	1	9	13

Students (N=23) were asked to respond on a Likert-type scale to the prompt, “Please give your best estimate of how often you attended the following professional development activities over the last 12 months”.

One student did not respond to this question.

Student gains in skills through the course

In the post-course survey, we asked students to rate “What skills did you gain or improve from taking the course?” (from 1=Not at all to 5=To a great extent). Students reported high levels of improvement in various communication and presentation skills, in particular in their communication to those outside their field (~4.5) (Fig. 1).

The Likert scale responses were corroborated by the open-ended responses. We asked students to list the two most important things that they gained from this course (e.g., content knowledge, research experience, communication skills). The most common theme (Table 5) that emerged, mentioned by almost all participants (n=18 [82%]), was that the course improved their oral and written communication and presentation skills. One student explained that she gained “better presentation skills (including slides and time management).” Another student specifically mentioned improvement in writing skills: “Writing is a skill that truly takes practice to improve and by having such a writing-intensive course, I feel like it bettered my writing.”

TABLE 5

Student responses to the post-course survey item, “List the two most important things that you gained from this course.”

	Number of Responses (Percentage)	Examples of Students’ Responses
Oral and written communication and presentation skill	18 (82%)	Presentation experience—by having so many opportunities (requirements) to present, and by having detailed feedback from the instructors and my peers on each, I feel that my skills in this area have increased noticeably
Enhanced ability to communicate to diverse audiences	9 (41%)	The one most important thing was writing about my research in a way that was accessible to a broad audience
Research experience	8 (36%)	The most important things I learned were how to condense my research into an informative but concise presentation, and how to formulate research questions and place them within the context of the field.
Data visualization skills	5 (23%)	Graphic visualization—how to visually present data in a clear way for manuscript figures, posters, and oral talks.
Understanding the review process in science	5 (23%)	Peer review practice was also really useful …, both in terms of receiving and giving it.
Content knowledge in other discipline	3 (14%)	Was fun knowing other areas of science and how they are implementing network biology concepts to solve it.

N=22. Some answers fit more than two categories.

Another common theme (n=9 [41%]) was enhanced ability to communicate to diverse audiences. One student explained that the most important thing that she gained was “knowing what details of the methods can be omitted for a broad audience.” Another student explained, “I was really surprised by the difference clarity made to scientists in related, but slightly different, fields.” Eight students (36%) mentioned getting research experience as the most beneficial aspect of the course; one student specifically explained how the research component of the course enhanced her own research,

I really like that you could incorporate your research into the class. This class has a lot of assignments that take time. However, I never resented the amount of time the assignments took because it didn’t take time away from doing my research, only enhanced it.

Five students (23%) mentioned that they gained data visualization skills, such as “poster making,” and “graphic visualization—how to visually present data in a clear way for manuscript figures, posters, and oral talks.” Five students (23%) referred to the benefit of understanding the review process in science (i.e., receiving and providing critique). As one student pointed out, “[it was beneficial to] critique others’ interdisciplinary work. Even if it is outside of my research area, I received many opportunities to think critically about other work.”

Three students (14%) commented that they gained content knowledge in other disciplines. As one student, who declared himself as a computer/computational science and engineering student, stated, “[I have] a much better understanding of genetics after the course.”

Student gains in confidence through the course

Comparison between students’ reported confidence prior to and following the course showed that students gained confidence over time (Fig. 2). All confidence averages were higher on the post survey. A one-sided Wilcoxon sign test with an apriori 0.05 significance level showed that in all but two of eight skills, students significantly improved in their confidence. It is noteworthy that average confidence was above the “somewhat confident” level (> 3) on the Likert scale in the pre survey. This may have reduced the amount of growth measurable for these skills.

FIGURE 2

Student responses on the pre and post surveys to the Likert-type question, “At this time, how confident do you feel in your ability to…” (N=23) (1=Not at all, 2=Not much, 3=Somewhat, 4=To a good extent, and 5=To a great extent. *P<0.05; **P<0.01.

Student scores on elevator speeches and oral presentations

As stated in the methods section, we compared students’ performance in the beginning of the course (through elevator speeches) with their performance toward the end of the semester (through oral presentations). Results from spring 2017 (N=13) showed that students’ average percentage score and standard deviation for the variable Research Question was higher in the oral presentation (90±10.88) than in the elevator speech (86±11.73). Examples for the comments that the instructors provided regarding this variable following the elevator speeches included, “Gave a sense of the broad problem, but no sense of a research problem addressed,” and, “The question was stated quite late into the speech, losing impact. Intro to the problem could be more concise/straightforward.”

Students’ average percentage score and standard deviation for the variable Accessible to a general audience/Presentation” was also higher in the oral presentations (96±8.62) than in the elevator speeches (86±14.84). Examples for the comments that the instructors provided regarding this variable in the elevator speech included, “Used a few technical concepts like ‘knockout’” and “Could describe MRI in a more relatable way, for example by giving examples of its use, to engage the listener.”

It is noteworthy that the instructors were more lenient with the grading of the elevator speeches, and several students scored 100% already in the elevator speech on the Research question variable (n=3) and on the Accessible to a general audience variable (n=5), showing that they came to the course with a very articulated research question and a great ability to communicate their research to a diverse audience. Nevertheless, for the majority of students, there was a substantial improvement between the two presentations.

Suggestions for course improvement

Following the spring 2017 semester, when students were asked to suggest how to improve the course, four students (31%) reflected on the distribution of assignments throughout the semester. One student pointed out that “the schedule for the latter part of the course [was] a bit tight. Almost one assignment per week.” He suggested that the course could “somewhat adjust a bit to allow more time between the poster and the final paper, as both take much time to prepare.” Three students (23%) referred to the peer review group assignments, suggesting more permanent groups, which would “allow students to become more familiar with the details of the peer’s research and provide more useful feedback.” One of these students wished that the groups were assigned based on “similar research goals.” Two students (15%) wanted more instruction regarding network analysis. Other individual responses included a request for “more lectures from faculty members on complex visualization,” “more peer-review opportunities,” and “raise the grade.” One student wrote that there was “nothing to change.”

Interviewed students

To gain a more thorough understanding of how students benefited from the practicum, one-on-one interviews were held with three students. Artifacts from the class and responses from surveys were also pulled to contextualize the students’ responses. The names used are randomly chosen pseudonyms.

Jamie

Jamie is a Toxicology and Environmental Health student in the School of Public Health. At the beginning of the Data Practicum, she was in her second year and in the late coursework stage of her doctoral degree. Her main discipline is Biological Science, with a sub-discipline in bioinformatics. After graduating, she believes she will end up in industry. Like many others, Jamie reported that she had neither formal written or oral communication training, nor formal research skill training. She was seeking advice on proposal writing and “more experience in sophisticated computational visualization methods and mathematical models to build networks integrating sequence data with metadata variables.”

For Jaime, taking the Data Practicum course accomplished three things: peer review helped her establish better balance in her papers and improve her figures, the repetition of the class improved her confidence, and the incorporation of a student’s ongoing research project increased her motivation to succeed in the course.

At the beginning of the course, the most common feedback expressed by students and instructors was allocation of time and space for different parts of Jamie’s presentation or writing. For example, two students felt that her abstract had “too much detail when it came to describing the background of the problem,” which “may be a little too long to keep a reader waiting.” By the end of the semester, she received feedback such as, “the introduction does a good job providing the foundation and defining the terms” and “…the methods and results were discussed thoroughly… [she] had a clear writing style throughout the article which was much appreciated.”

Jamie also got many comments on the need to improve her figures so that they would be easier to understand, a concern that she mentioned in her post-course interview:

[Working with other students in the class] is good for helping prepare visualizations, to make it really easy for people … outside of the field to understand. I got a lot of tips on even small things, like color coordination, you don’t really get in general seminar.

When asked how she had improved through the course, Jamie believed she had become a better presenter. In her interview, she stated that before and during the Data Practicum, she disliked giving talks. When discussing her experience in the Data Practicum course, Jamie explained how the repetition in presentation made her more comfortable presenting:

I’m very nervous about presenting, and it’s the repetition that makes you better at it. So, I was able to actually give the talk I gave in class to a wide audience at a symposium hosted at the University… and I think I would have been a lot more nervous than I was if I’d not practiced this in front of a wide audience of the class. But it definitely feels like an improvement. [It is] the class with the most amount of time I’ve ever had [for] actually presenting.

While the Data Practicum course was writing-intensive, the real benefit of the course was how directly the writing assignments contributed to Jamie’s professional development:

Given you have to write for that class, it’s really useful because usually if you don’t have a deadline to write it’s hard to like, to stick to getting it done. But at the end I had a manuscript. It’s not just a class assignment where you’re like “After this class this is meaningless.” I can use this to actually graduate with.

In the post survey, Jamie scored herself as improving “to a good extent” for seven of the ten areas (Fig. 1), including “how to communicate across resolutions: from the broad/overview level to the level of details” and “knowledge of disciplines outside of your Ph.D. program.” At the time of the interview, Jamie had already presented the research from the Data Practicum at the annual COMBINE symposium, and in another symposium at the university. She also submitted her work to a journal for publication.

Salina

Salina is a physics student who was in the first year but in the late coursework stage of her Ph.D. program. Her subdisciplines include network science and machine learning. Salina’s goal is to become a professor of network science, where she can keep doing interdisciplinary research. Prior to the Data Practicum, she reported that she had no formal written communication training or formal research skill training but had received formal oral communication training “multiple times.” In the pre-course survey, Salina expressed a desire to learn techniques in computational biophysics and have more qualitative help on the actual specifics of her project.

For Salina, taking the Data Practicum course accomplished two major things: peer review helped her understand the use of analogy or concrete examples to explain her research, and the mixed discipline makeup of the class improved her understanding of interdisciplinary science communication.

The area Salina struggled with the most was explaining her methodology to her fellow classmates. As the only individual coming from a primarily physics discipline, she had the most distinct project from the rest of the students. Often in peer reviews of Salina’s presentations, students asked basic questions, such as, “How do you estimate the accuracy of the reservoir computing method compared to other machine learning/neural nets?” and “[Does the refraction curve shown] mean there’s a threshold to the size of the training set where any more won’t improve it?” A few students recommended that she use analogy or concrete examples throughout her presentation, since she “jumps into technical jargon about machine learning and [analogy or examples] would help to ground it conceptually.” After the class, Salina felt she had become a better presenter to interdisciplinary audiences:

I think the thing that I appreciate the course for really trying to instill in everyone is to be able to present to an audience that’s not your typical audience, and I think most of it was, to me it was surprising that some of the “genetics” people were sort of lost and maybe would ask questions that seem very obvious but clearly wasn’t for someone outside the field.

At the time of the interview, Salina had already presented the research from the Data Practicum at the annual COMBINE symposium.

Noah

Noah is a computer science student, with subdisciplines in machine learning and bioinformatics. At the beginning of the Data Practicum, he was two years into his graduate studies and at the dissertation stage. He hoped that, through the COMBINE program, he could gain life science knowledge to inform his modeling. His initial goal was to enter either the private sector doing data journalism or academia.

For Noah, taking the Data Practicum course accomplished three major things: peer review allowed him to find the best way to communicate his research to a lay audience, instructor feedback helped him edit his own work more effectively, and course lectures gave Noah practical tools for writing and publishing his work.

Noah’s primary challenge in the course was being able to explain the methods and context in a simple enough way to lay or interdisciplinary audiences. On his early proposal, one student commented, “This proposal may not be easy to understand to a person outside this area.” The same student also commented on Noah’s research update presentation, “I do not fully understand this work, due to my limited knowledge of biology and genetics.” Another student commented on the final paper, “The statistical methods part seemed nice, but I would have appreciated more attention to explaining what you’re doing so that a biologist could understand it without having to go digging through the problem statement and methods section.”

In the interview, in response to the question, “How did you change from the beginning to the end of the course in terms of being a presenter or researcher,” Noah responded that receiving feedback from peers and instructors helped him to edit his own work:

There were a lot of things that I wrote that seemed clear to me, and I had spent four months working on this research and had come back and the instructors had said “this section doesn’t make sense,” and when I read it, not from my perspective but from the perspective of someone just encountering this, I was like, “Oh yeah this doesn’t make any sense at all.”

Noah stated that the skills taught in the Data Practicum (e.g., writing a paper or being a journal’s referee) were both very useful and practical:

One thing I thought was really useful was the whole course seemed to be focused around turning your research into something useful. … I’d never taken a course that sat you down, sat me down, like, “if you want to get a paper published, this is the mechanics of that, if you want to end up being a referee for a journal, that’s the way to go about it.”

At the time of the interview, Noah had already presented the research from the Data Practicum at the annual COMBINE symposium.

Feedback from the course instructors

In their interviews, the course instructors said that they were very satisfied with the course. They mentioned that, at the beginning of the course, students often failed to explain complicated material thoroughly or at an accessible level to diverse audiences. However, most students tremendously improved in this area by the end of the course. One instructor pointed out the benefits of the iterative process of instruction, explaining that often students better understood the instructions only following their own completion of an assignment:

After their first round of oral presentations, I reiterated my earlier advice to them. I think it had a much bigger impact the second time because they could connect it to their own presentations at that stage. In the second round, the presentations were greatly improved.

When asked which class components they thought improved student performance the most, one instructor referred to the feedback on assignments:

I think detailed feedback. They really did pretty well responding to feedback… From everybody! So even our own comments to their peers. You would see them sort of pick up some of the techniques that their peers were recommended to use.

The instructors after their interviews reviewed the post-course survey results, including the proposed changes to the course. Regarding the students’ suggestion to pace the assignments, the instructors decided in the next iteration to introduce several assignments (e.g., the paper outline) earlier in the semester and to reduce the overall number of different presentations but have more iterations of each type of presentation. Regarding the need for more instruction on complex visualization, they commented that the Data Practicum was meant to be taken after a course on network visualization; students in future versions of the class would likely not have the same criticism because they would already have plenty of instruction on the topic within the COMBINE program.

DISCUSSION

Recent reports (10, 22) state that future generations in the “science and engineering workforce will need to collaborate across national boundaries and cultural backgrounds, as well as across disciplines” (22, p. 6). Nevertheless, most training programs for graduate students in science and engineering are traditionally performed as disciplinary silos, and their members find it difficult to communicate with members of other disciplines (6). In the introduction, we provided a classification for approaches to performing interdisciplinary research (15). The COMBINE program aspires to develop interdisciplinary researchers through all these mechanisms. Students from each discipline are required to take courses from other disciplines, broadening their interdisciplinary knowledge, but they are not expected to become experts in the other disciplines. Instead, they are expected to collaborate with other students from those disciplines and learn how to communicate to a diverse audience in interdisciplinary teams.

The Data Practicum, specifically, was designed to teach future scholars and researchers in physical/mathematical, computer, and life sciences how to communicate and present their research to a diverse audience. We learned that students’ motivation to take the course was to increase their general knowledge of other scientific fields and become more effective communicators. A noteworthy finding from this study was that even students who are able to present effectively within their disciplines often struggle to structure presentations that can be understood by diverse audiences. As stated in the introduction, each discipline has its own language and terminology, and these differences are the biggest barrier to interdisciplinary communication (12). In this study, we observed that it is possible in a single semester to enhance interdisciplinary communication skills through a teaching approach that gives students a chance to incorporate feedback from a diverse audience. For example, Salina learned from the class that, in order to clearly explain her study to a diverse audience, she needs to consistently use the same analogy. By the end of the semester, most students firmly believed they had improved their skills and confidence in science communication, a belief that was supported by the increase in average scores on the variable Presentation from the elevator speeches at the beginning of the semester to the end-of-semester oral presentations.

The interviews with three students highlighted the course’s iterative process, peer review system, and unique incorporation of outside research already being conducted by students. All three students believed they received many chances to implement and reflect on their work. Reflection included reviewing their own work and getting peer reviews. In the interview with Jamie, the “repetition” of reviewing presentation performance and giving the presentation a second time was what she believed “makes you better.” Additionally, the benefits of peer review were noted in all student and instructor interviews. The benefits of peer review ranged from improved general writing skills to increased ability to communicate research to diverse, interdisciplinary audiences. The unique design of the course that uses and enhances previously established research both facilitated participation in the class and provided students with finished papers and posters.

Another mission of the data practicum, which is aligned with the literature on interdisciplinary research (14), was to encourage interdisciplinary team work. During the course, students were divided to interdisciplinary groups of three or four students to review and orally present interdisciplinary research papers. The feedback that we received from students in spring 2017 was that they prefer to be in permanent groups, since they could develop a level of comfort with each other and learn more about each other’s research. Based on this feedback, the instructors decided in the next semester to keep permanent groups.

Overall, students’ reports and artifacts showed the Data Practicum was successful in helping them to communicate their research visually, orally, and in text to a wide and varied audience, to critically review others’ work, inside and outside their discipline, and to develop awareness of research in other disciplines. As mentioned, researchers argue that the ability to communicate interdisciplinary research should be a primary goal of graduate and even undergraduate research programs (3, 9). As one student explained it, the value of such a course is evident to many because “it doesn’t matter how great, fundamental, or world-changing your research is if you can’t frame those qualities inherently to lay people.” This Data Practicum could serve as a model for other educational programs wishing to increase training in interdisciplinary skills among their students.

Click here for additional data file.