Literature DB >> 35324983

Design thinking teaching and learning in higher education: Experiences across four universities.

Jacqueline E McLaughlin1, Elizabeth Chen2,3, Danielle Lake4, Wen Guo5, Emily Rose Skywark3, Aria Chernik6, Tsailu Liu7.   

Abstract

A growing body of literature highlights the increasing demand on college graduates to possess the problem finding, problem framing, and problem-solving skills necessary to address complex real-world challenges. Design thinking (DT) is an iterative, human-centered approach to problem solving that synthesizes what is desirable, equitable, technologically feasible, and sustainable. As universities expand efforts to train students with DT mindsets and skills, we must assess faculty and student DT practices and outcomes to better understand DT course experiences. Understanding how DT is taught and experienced within higher education can help schools promote student learning and align their training programs with professional, personal, and civic needs. In this study, surveys were completed by 19 faculty and 196 students from 23 courses at four universities. DT teaching and learning was characterized by three DT practices and five outcomes. Statistically significant differences were found by discipline of study and student type (i.e., graduate vs undergraduate), but not by gender or race/ethnicity. These results can be used to inform the development of classroom-based DT teaching and learning strategies across higher education institutions and disciplines.

Entities:  

Mesh:

Year:  2022        PMID: 35324983      PMCID: PMC8947127          DOI: 10.1371/journal.pone.0265902

Source DB:  PubMed          Journal:  PLoS One        ISSN: 1932-6203            Impact factor:   3.240


Introduction

Universities have faced considerable scrutiny in recent years for their apparent failure to adequately equip students with the complex reasoning and problem solving skills thought to be at the core of higher education [1-3]. A growing body of literature highlights the demand on college graduates to simultaneously master the disciplinary knowledge and mindsets necessary to address complex real-world problems [4-7]. These demands, coupled with ongoing concerns about the quality of higher education, have drawn attention to the need to rethink our focus within higher education [8-11]. Design thinking (DT) is an iterative, creative approach to problem finding, problem framing, and problem solving that synthesizes what is desirable to real stakeholders, equitable, technologically feasible and sustainable [12, 13]. Most models of design thinking move through (1) inspiration, empathy and problem definition, (2) ideation, (3) prototyping and testing and (4) implementation stages [14]. By beginning with the goals and needs of stakeholders and engaging in short iteration cycles, DT supports collaborative solutions that roll out with lasting impact [12, 15]. Professional design consultancies often use this method to design innovative products or services. Thus, DT is a tool frequently taught in business, engineering and design schools. A growing number of disciplines are utilizing and teaching DT to solve complex problems, including public health, healthcare, and the liberal arts [12, 16–18]. The success of innovations developed with the DT process has led to the uptake of DT to solve various challenges, including customer experience and strategic planning, and to support various sectors, including government agencies, non-profits, educational institutions, and community organizations [15]. Power dynamics can be challenged at the intersection of design for social innovation; this form of participatory design can help students imagine new ways of thinking to solve complex problems [19]. The human-centered, real-world solutions generated from the DT process have the potential to provide more systemic solutions to difficult social problems, like climate change, poverty, housing instability, and health promotion. As DT is adopted by a broader audience, there is an onus on educators to equip students across university disciplines with tools and mindsets valuable for addressing these complex, real-world problems. This includes, but is not limited to “situatedness”, self-reflection, empathetic listening, critical observation and creative collaboration [20-23]. DT pedagogy should (1) frame both the situation and the student’s place within the situation; 2) allow for iterative exploration across space and time alongside diverse stakeholders 3) require the generation of divergent possibilities 4) the prototyping and actionable testing of these possibilities and (5) develop sustainable commitments to cultivated change [21]. Within higher education, we see new centers, programs, and courses being established outside common DT fields (i.e., business, engineering, design) with a focus on teaching DT mindsets and skills, such as Tulane University’s Phyllis M. Taylor Center for Social Innovation, University of Illinois’ Seibel Center for Design, and Design Thinking and Elon University’s Center for Design Thinking. As universities expand efforts to train students with DT mindsets and skills, we must assess faculty and student DT practices and outcomes to better understand DT course experiences. In one single institution study, for example, researchers found that DT requires time and trust which can be constrained by the imposed deadlines of semester-based projects [21]. In a single course study, students indicated that their “whirlwind” course promoted almost “exponential” growth [22]. While survey instruments that measure DT practices and outcomes have been validated across diverse workplace settings [24], the assessment of DT practices and outcomes across higher education is still new. DT pedagogy in higher education is incompletely understood, particularly in fields beyond traditional design disciplines (i.e., de-disciplined design). We expanded a single institution study of faculty by Lake and Colleagues [21] in an effort to assess DT teaching and learning (DT-TL) experiences of undergraduate students, graduate students, and faculty at four universities within a southeastern state of the United States. Overall, we set out to answer the following research question: how do faculty and students experience design thinking within higher education courses? To answer this question, we explored the following subquestions: What kinds of DT practices are experienced within higher education courses? In what ways do higher education DT practices align with and differ from other industries? What kinds of outcomes are experienced within higher education courses? In what ways do outcomes of DT in higher education align with and differ from other industries? Is DT a valid construct within higher education teaching and learning? What differences exist between groups, such as discipline (business, design & engineering, other), and student type (undergraduate, graduate)?

Methods

To answer these research questions, we utilized a mixed methods design that combined faculty and student surveys with semi-structured interviews. This manuscript focuses on the survey data only. Additional qualitative and mixed methods findings are detailed elsewhere [25].

Survey development

The survey used in this study was adapted from Liedtka and Bahr, who studied DT practices and outcomes among employees of for-profit, non-profit, and government sectors [24]. Liedtka and Bahr survey items were adapted to align with the context of higher education and additional items were included based on research team experiences. The final survey included 11 items about DT practices, 42 items about outcomes from DT, 11 demographic items (e.g., What is your gender identity?), and 11 course descriptor items (e.g., Was the term "design thinking" explicitly referenced in the course?). All DT practices and outcomes items were prompted by the following stem: Please note how often, as a direct result of this specific course, you observed the following [practices/outcomes] and measured on a scale from 1-Never to 5-Almost Always. The research team reviewed the survey for face validity prior to data collection.

Data collection

Students and faculty at four universities in the southeastern United States were recruited for the study during the 2020–2021 academic year. Purposive sampling was used to identify and recruit participants based on their experience with the research focus [26]. In August 2020, research team members recruited faculty at their home institutions who they knew taught design thinking courses. The email recruitment invited faculty to fill out an initial interest survey about their course, including: course number and title; number of students enrolled in their course; whether their course was open to undergraduate students, graduate students, or both; and number of credit hours associated with the course. The survey also asked faculty for informed consent, contact information, and whether they were interested in participating in an optional semi-structured interview at the end of the semester. Faculty who agreed to participate were also expected to recruit students from their DT course to the study. Toward the end of the Fall 2020 semester, research team members emailed the faculty survey to consented faculty from their home institutions. Faculty participants were also asked to forward study invitations to students enrolled in their DT courses. The email invitations drafted by our research team included information about the study and a link to the student survey. Students interested in participating in the study provided informed consent in the online survey and were also asked whether they wanted to schedule an optional follow-on interview. Faculty participants received three email reminders regarding study invitations to their students before the end of the Fall 2020 semester.

Data analysis

Survey data were first analyzed using descriptive statistics, with continuous variables reported as mean ± standard deviation (SD) and categorical variables reported as frequency (percent). An exploratory factor analysis was conducted to identify DT teaching and learning (DT-TL) constructs using principal components analysis with varimax rotation and Kaiser rule (ie, eigenvalues < 1.0) for student survey items also included in the Liedka and Bahr study [24]. Bivariate correlations were calculated using Pearson rho (rp) and reliabilities were calculated using Cronbach α. Group comparisons were examined using independent t-tests and one-way ANOVA with Bonferroni post hoc analysis. Parametric statistics were considered appropriate due to normality of data and sufficient sample size. Statistical significance was established at α<0.05. All data analysis was performed in SPSS for Windows, v26 (IBM, Armonk, NY).

Ethical considerations, consent

This project was submitted to the UNC Institutional Review Board (#20–2316), Elon University Institutional Review Board (#21–031), Duke Campus Institutional Review Board (#2021–0168), and North Carolina State University (#23502). The submission was approved or determined to be exempt from further review by each review board according to 45 CFR 46.104. Written consent was obtained electronically from all participants at the start of the survey.

Results

Surveys were completed by 19 faculty and 196 students from 23 courses at four universities. The response rate for faculty was 84.2%. Based on the number of students enrolled in each course, our estimated response rate of students was 20.8%. As seen in Table 1, student participants were predominately white (n = 132, 63.4%), female (n = 126, 64.2%), and majoring in Interdisciplinary Humanities/Social Sciences (n = 105, 53.6%). Eighty-seven percent (n = 170) of students were undergraduate students. Similarly, faculty participants were predominately white (n = 14, 73.7%), female (n = 11, 57.9%), and from Interdisciplinary Humanities/Social Sciences (n = 11, 57.9%).
Table 1

Characteristics of survey participants and courses.

Participant CharacteristicsStudents (n = 196)Faculty (n = 19)
Race: White132 (63.4%)14 (73.7%)
Black/African American 14 (7.1%)0 (0%)
Asian 21 (10.7%)4 (28.6%)
Hawaiian/Pacific Islander 5 (2.6%)0 (0%)
American Indian/Alaska 2 (1.0%)0 (0%)
Ethnicity: Hispanic10 (5.6%)0 (0%)
Gender: Female126 (64.2%)11 (57.9%)
Discipline: Business32 (16.3%)4 (21.1%)
Design & Engineering 45 (23.0%)4 (21.1%)
Interdisciplinary Humanities/Social Sciences 105 (53.6%)11 (57.9%)
Undergraduate Student: Yes170 (86.7%)N/A
DT Expertise Prior to Course: None51 (26.0%)0 (0%)
Limited 95 (48.4%)7 (36.8%)
Moderate 26 (13.3%)7 (36.8%)
Extensive 5 (2.6%)4 (21.1%)
DT Expertise After the Course: None0 (0%)0 (0%)
Limited 25 (12.8%)2 (10.5%)
Moderate 118 (60.2%)12 (63.2%)
Extensive 34 (17.3%)4 (21.1%)
Course Characteristics
DT Explicitly Mentioned: Yes149 (76.0%)14 (73.7%)
Had Resources Needed for DT: Agree143 (72.9%)9 (47.7%)
Space Was Conducive to DT: Agree100 (51.0%)9 (47.7%)
Real World Project used for DT: Yes160 (81.6%)13 (68.4%)
Percent of Course Time Spent in Teams (mean ± SD)69.69±21.65%66.89±26.14%

DT = Design Thinking; SD = Standard Deviation; N/A = Not Applicable

NOTE: some variables have missing data; percentages may sum to less than 100%

DT = Design Thinking; SD = Standard Deviation; N/A = Not Applicable NOTE: some variables have missing data; percentages may sum to less than 100% When asked about the course, most students (n = 149, 76.0%) and faculty (n = 14, 73.7%) indicated that DT was explicitly taught in the course and that they utilized a real-world project (n = 160 (81.6%) students and 13 (68.4%) faculty). On average, students reported spending 69.69% ± 21.65% of their time working in teams. Most students indicated having none or limited DT expertise prior to the course (n = 146, 74.4%) and moderate or extensive expertise after the course (n = 152, 77.5%). Similarly, the number of faculty reporting none (n = 0, 0%) or limited DT expertise (n = 7, 36.8%) prior to the course dropped to 2 (10.5%) after the course. Table 2 provides item-level responses for DT practices in the current study and two related DT studies using the same survey items. On a five-point scale from 1-Never to 5-Almost Always, student and faculty participants reported that they used all 11 of the DT practices with moderate to high frequency in the course. Faculty indicated that they most commonly followed a structured process (4.16 ± 0.69), created prototypes of ideas (4.11 ± 0.66), and emphasized active listening among team to find shared meaning (4.11 ± 0.88). Similarly, students most commonly emphasized active listening among team to find shared meanting (4.38 ± 0.77), followed a structured process (4.16 ± 0.75), and generated a diverse set of ideas (4.16 ± 0.85). Faculty and students executed real world experiments least frequently (3.16 ± 1.26 and 2.96 ± 1.34, respectively).
Table 2

Survey responses for DT practices in current study (faculty, students) and in Lake et al. [21] and Liedtka and Bahr [24].

Design Thinking PracticeFaculty (n = 19)Students (n = 196)Lake, et al. [21]Liedtka & Bahr [24] (n = 416)
Please note how often, as a direct result of this specific course, you observed the following practices:
(n = 35)
Mean (SD)Mean (SD)
Mean (SD)Mean (SD)
1. Followed a structured process4.16 (0.69)4.16 (0.75)---3.80 (0.86)
2. Formed a diverse team3.74 (0.99)4.09 (0.89)4.09 s(---)3.93 (0.87)
3. Emphasized active listening among team to find shared meaning4.11 (0.88)4.38 (0.77)3.80 (---)4.01 (0.95)
4.Done user research using ethnographic tools3.63 (1.21)3.89 (1.13)2.69(---)3.91 (1.07)
5. Focused your problem definition on user’s perspective rather than the organization’s4.32 (0.82)4.12 (0.78)3.00(---)4.09 (0.88)
6. Created a set of design criteria that described an ideal solution based on user research3.34 (1.30)3.99 (0.89)3.11(---)3.60 (1.03)
7. Generated a diverse set of ideas based on your user research3.79 (0.98)4.16 (0.85)---3.90 (0.95)
8. Created prototypes of your ideas4.11 (0.66)3.86 (1.10)3.27(---)3.80 (1.05)
9. Moved multiples ideas into prototyping and testing3.26 (0.93)3.22 (1.19)2.74(---)3.39 (1.02)
10. Got feedback from users and other stakeholders on the prototype3.68 (1.25)3.73 (1.24)2.51(---)3.85 (1.05)
11. Executed real world experiments to test your ideas3.16 (1.26)2.96 (1.34)2.71(---)3.43 (1.07)
Cronbach Alpha 0.890.81------

SD = Standard Deviation; All items measured on a scale from 1-Never to 5-Almost Always

“—" indicates item was not included on survey and/or value was not reported.

SD = Standard Deviation; All items measured on a scale from 1-Never to 5-Almost Always “—" indicates item was not included on survey and/or value was not reported. As seen in Table 3, factor analyses indicated that DT-TL practices can be broadly characterized by three constructs accounting for 59.03% of the variance: Discovery and Ideation (4.04 ± 0.70, 21.67% of variance); Team Formation and Functioning (4.21 ±0.53, 16.02% of variance); and Prototyping and Experimentation (3.44 ±0.94, 21.34% of variance). Survey items loaded into the same three factors as Liedtka and Bahr [24], with the exception of “Followed a Structured Process,” which loaded to Discovery and Ideation in Liedkta and Bahr (2019) instead of on Team Formation and Functioning in our study. Students who reported that DT was explicitly taught in their course more frequently engaged in Discovery and Ideation than those who indicated DT was not explicitly taught (4.16±0.65 vs 3.42±0.72, p<0.001). Students from Business and Design & Engineering disciplines also more frequently engaged in Discovery and Ideation than those from Interdisciplinary Humanities/Social Sciences (4.21±0.53 vs 3.85±0.69, p = .04; 4.26±0.74 vs 3.85±0.69, p = 0.004 respectively). Undergraduate students reported Prototyping and Experimentation more frequently than graduate students (3.49±0.88 vs 3.07±1.24, p = 0.04). There were no differences in DT practices found by race/ethnicity, or gender.
Table 3

Factor loadings and group differences for DT practices experienced in higher education courses (n = 196).

DT Practice Factor
Discovery & IdeationTeam Formation & FunctioningPrototyping & Experimentation
Followed a structured process0.46
Formed a diverse team0.75
Emphasized active listening among team to find shared meaning0.70
Done user research using ethnographic tools0.67
Focused your problem definition on user’s perspective rather than the organization’s0.75
Created a set of design criteria that described an ideal solution based on user research0.73
Generated a diverse set of ideas based on your user research0.69
Created prototypes of your ideas0.69
Moved multiples ideas into prototyping and testing0.75
Got feedback from users and other stakeholders on the prototype0.81
Executed real world experiments to test your ideas0.70
Factor Mean (SD) 4.04 (0.70) 4.21 (0.58) 3.44 (0.94)
Race/Ethnicity: Underrepresented*4.06 ± 0.744.06 ± 0.753.30 ± 0.87
Other 4.04 ± 0.714.25 ± 0.553.51 ± 0.94
Gender: Female4.09 ± 0.704.22 ± 0.593.45 ± 0.96
Male 3.95 ± 0.734.24 ± 0.583.56 ± 0.80
Discipline: Business4.21 ± 0.53**4.29 ± 0.603.65 ± 1.08
Design & Engineering 4.26 ± 0.74**4.19 ± 0.563.40 ± 0.87
Interdisciplinary Humanities/Social Sciences 3.85 ± 0.69**4.15 ± 0.593.43 ± 0.92
Student Type: Undergraduate Students4.03 ± 0.704.22 ± 0.583.49 ± 0.88**
Graduate Students 4.06 ± 0.704.19 ± 0.583.07 ± 1.24**

SD = Standard Deviation; DT = Design Thinking

Principal Component Analysis with Varimax rotation using Kaiser Normalization converged in 5 iterations and accounted for 59.03% of variance.

*Underrepresented race/ethnicity includes Black/African American, Hawaiian/Pacific Islander, American Indian/Alaskan, and Hispanic

**p<0.05 for Discovery and Ideation (Business and Design & Engineering disciplines more frequently engaged than Interdisciplinary Humanities/Social Sciences) and Prototyping and Experimentation (Undergraduate students more frequently engaged than graduate students).

SD = Standard Deviation; DT = Design Thinking Principal Component Analysis with Varimax rotation using Kaiser Normalization converged in 5 iterations and accounted for 59.03% of variance. *Underrepresented race/ethnicity includes Black/African American, Hawaiian/Pacific Islander, American Indian/Alaskan, and Hispanic **p<0.05 for Discovery and Ideation (Business and Design & Engineering disciplines more frequently engaged than Interdisciplinary Humanities/Social Sciences) and Prototyping and Experimentation (Undergraduate students more frequently engaged than graduate students). Table 4 provides factors loadings for outcomes of DT in higher education courses, which are broadly characterized by five constructs accounting for 63.00% of the variance: Implementation Support (4.00 ± 0.70, 18.51%), Psychological Benefits and Motivation (4.15 ± 0.64, 14.13%), Relationships and Trust (3.76 ±0.82, 12.74%), Quality of Solutions Generated (4.21 ± 1.75, 8.98%), and Individual Adaptation and Flexibility (4.13 ± 0.66, 8.64%). Students who reported that DT was explicitly taught more frequently experienced Psychological Benefits and Motivation than those who indicated DT was not explicitly taught in their course (4.21±0.62 vs 3.85±0.87, p = 0.008). There were no differences in outcomes of DT found by race/ethnicity, gender, or student type (ie, undergraduate vs graduate).
Table 4

Factor loadings and group differences for outcomes of DT in higher education courses (n = 196).

OutcomeFactor
Implementation SupportPsychological Benefits & MotivationRelationships &TrustQuality of Solutions GeneratedIndividual Adaptation & Flexibility
Helped me see the problems0.76
Enhanced my ability to pivot when initial solutions didn’t work0.76
Built new relationships locally that continued after the initial project was completed0.71
Expanded access to new resources for individuals and teams0.63
Helped pool resources for greater impact
Enhanced other stakeholders willingness to collaborate on new solutions0.72
Built trust among team members0.48
Built trust between problem-solving teams and other stakeholders0.70
Allowed new and better solutions, not visible at the beginning of the process, to emerge during it0.62
Fostered the inclusion of user input0.79
Helped people involved to examine their own biases and preconceptions0.52
Created a sense of safety to try new things0.58
Gave people more confidence in their own creative abilities0.44
Improved the likelihood of the implementation of new solutions0.47
Made it easier to discard solutions that didn’t work as planned0.44
Helped people interested in trying new things to connect and support each other0.70
Encouraged people’s open-mindedness to try new things0.60
Encouraged shifts in organizational culture that made it more customer-focused0.66
Encouraged changes in organizational culture that made risk-taking more acceptable0.75
Kept people motivated to work on a project to achieve impact0.58
Broadened organization’s definition of what innovation is0.45
Increased a sense of ownership and acceptance of a solution0.72
Increased appreciation for use of data to help drive decisions0.62
Increased engagement of teammates involved in the design thinking process0.77
Factor Mean (SD) 4.00 (0.70) 4.15 (0.64) 3.76 (0.82) 4.21 (0.75) 4.13 (0.66)
Race/Ethnicity: Underrepresented*3.89 ± 0.764.14 ± 0.513.59 ± 0.724.28 ± 0.664.05 ± 0.73
Other 4.03 ± 0.704.15 ± 0.663.79 ± 0.834.19 ± 0.784.17 ± 0.64
Gender: Female4.03 ± 0.724.19 ± 0.633.77 ± 0.864.26 ± 0.764.14 ± 0.68
Male 3.96 ± 0.674.09 ± 0.643.73 ± 0.694.08 ± 0.724.17 ± 0.60
Discipline: Business3.98 ± 0.874.14 ± 0.743.57 ± 0.994.23 ± 0.644.22 ± 0.65
Design & Engineering 3.89 ± 0.614.15 ± 0.543.52 ± 0.794.21 ± 0.704.13 ± 0.48
Interdisciplinary Humanities/Social Sciences 4.04 ± 0.744.11 ± 0.633.88 ± 0.774.13 ± 0.804.10 ± 0.74
Student Type: Undergraduate Students4.01 ± 0.694.14 ± 0.653.77 ± 0.824.23 ± 0.744.14 ± 0.64
Graduate Students 3.95 ± 0.804.20 ± 0.573.62 ± 0.794.07 ± 0.844.05 ± 0.81

SD = Standard Deviation; DT = Design Thinking

Principal Component Analysis with Varimax rotation using Kaiser Normalization converged in 9 iterations and accounted for 63.01% of variance; no significant differences found between groups.

*Underrepresented race/ethnicity includes Black/African American, Hawaiian/Pacific Islander, American Indian/Alaskan, and Hispanic

SD = Standard Deviation; DT = Design Thinking Principal Component Analysis with Varimax rotation using Kaiser Normalization converged in 9 iterations and accounted for 63.01% of variance; no significant differences found between groups. *Underrepresented race/ethnicity includes Black/African American, Hawaiian/Pacific Islander, American Indian/Alaskan, and Hispanic As seen in Table 5, four of the five outcomes factors contained similar items and items that differed. Factor 3, for example, emphasized Relationships in both studies and included the survey item Built new relationships locally that continued after the initial project was completed. However, in the current study, other items in that factor addressed Trust (e.g., Built trust among team members) while other items in the Liedtka and Bahr study addressed Resources (e.g., Expanded access to new resources for individuals and teams.)
Table 5

Side-by-side comparison of DT outcome factors in higher education and DT outcome factors in for-profit, non-profit, and Gov’t settings.

Current Study: Higher EducationLiedka and Bahr: For-Profit, Non-Profit, and Gov’t [24]
Factor 1: Implementation Support Factor 1: Improved Implementation and Adaptation
Improved the likelihood of the implementation of new solutionsEncouraged shifts in organizational culture that made it more customer-focusedEncouraged changes in organizational culture that made risk-taking more acceptableBroadened organization’s definition of what innovation is• Expanded access to new resources for individuals and teams• Created a sense of safety to try new things• Helped people interested in trying new things to connect and support each other• Encouraged people’s open-mindedness to try new thingsImproved the likelihood of the implementation of new solutionsEncouraged shifts in organizational cultural that made it more customer-focusedEncouraged changes in organizational culture that made risk-taking more acceptableBroadened organization’s definition of what innovation is• Enhanced your ability to pivot when initial solution didn’t work• Made it easier to discard solutions that didn’t work as planned• Kept people motivated to work on a project to achieve impact• Increase a sense of ownership and acceptance of a solution• Increased appreciation for use of data to help drive decisions
Factor 2: Psychological Benefits and Motivation Factor 2: Individual Psychological Benefits
Gave people more confidence in their own creative abilities• Kept people motivated to work on a project to achieve impact• Increase a sense of ownership and acceptance of a solution• Increased appreciation for use of data to help drive decisions• Increased engagement of teammates involved in the design thinking processGave employees more confidence in their own creative abilities• Created a sense of safety to try new things• Helped people interested in trying new things to connect and support each other• Encouraged people’s open-mindedness to try new things
Factor 3: Relationships and Trust Factor 3: Expanded Network Relationships and Resources
Built new relationships locally that continued after the initial project was completedEnhanced other stakeholders willingness to collaborate on new solutions• Built trust among team members• Built trust between problem-solving teams and other stakeholders• Helped people involved to examine their own biases and preconceptionsBuilt new relationships locally that continued after the initial project was completedEnhanced other stakeholders’ willingness to collaborate on new solutions• Expanded access to new resources for individuals and teams• Helped pool resources for greater impact
Factor 4: Quality of Solutions Generated Factor 4: Quality of Solutions Generated
Allowed new and better solutions, not visible at the beginning of the process, to emerge during itFostered the inclusion of user inputAllowed new and better solutions, not visible at the beginning of the process, to emerge during itFostered the inclusion of user input• Helped teams see the problems in new ways, resulting in solving more promising problem• Increased engagement of employees involved in the Design thinking process• Helped people involved to examine their own biases and preconceptions
Factor 5: Individual Adaptation and Flexibility Factor 5: Trust Building
• Helped me see the problems in new ways, resulting in solving more promising problems• Enhanced my ability to pivot when initial solutions didn’t work• Made it easier to discard solutions that didn’t work as planned• Built trust among team members• Built trust between problem-solving teams and other stakeholders

DT = Design Thinking; Gov’t = Government; Italicized items indicate the same items were within the same factor for both studies.

DT = Design Thinking; Gov’t = Government; Italicized items indicate the same items were within the same factor for both studies. All correlations between the three DT practice and five outcome constructs were statistically significant, ranging from positive, moderate relationships (rp = 0.33) to positive, very strong relationships (rp = 0.76) (Table 6). Cronbach alpha exceeded .6 for seven of the eight DT constructs (0.53 ≤ α ≤ 0.89), suggesting that the items used to create each construct demonstrated acceptable internal consistency.
Table 6

Bivariate correlations (rp) and reliabilities (α, in parentheses) for DT constructs.

12345678
1. Discovery & Ideation(0.76)
2. Team Formation & Functioning0.51(0.53)
3. Prototyping & Experimentation0.390.36(0.77)
4. Implementation Support0.460.600.45(0.89)
5. Psychological Benefits & Motivation0.410.520.330.76(0.84)
6. Relationships & Trust0.330.510.440.730.65(0.82)
7. Quality of Solutions Generated0.520.520.420.580.530.53(0.68)
8. Individual Adaptation & Flexibility0.470.460.340.660.570.540.47(0.74)

p < .05 for all correlations

p < .05 for all correlations

Discussion

This study explored faculty and student experiences with DT in courses from various disciplines within four universities. Given the increasing uptake of DT in higher education and across professional fields, this study is timely and critical for understanding the types of DT practices and outcomes experienced, and ways in which DT in these settings might vary among different industries and stakeholders. This work extends research conducted in workplace settings [24] and compliments studies exploring conceptual frameworks and uses of DT in education [27, 28]. Luka, for example, integrated multiple DT models to design an international English course that engaged learners in a four-phased learning cycle of experiencing, reflecting, thinking and acting [27] while Wrigley and Straker described the Educational Design Ladder, which illustrates the organization of a multidisciplinary DT program [28].

DT-TL constructs

The results of this study suggest that DT-TL is a valid construct in its own right within the context of higher education. Specifically, the factor analysis revealed 8 distinct factors with high factor loads and the majority of variance accounted for by the analysis, providing support for content specificity. While the 3 DT-TL practice constructs and 5 outcomes constructs are aligned with those described in other contexts [24], they also embody items differently, which may be attributable to the different processes and contexts associated with student learning in higher education environments. As noted below, the experiences of students and faculty in DT-TL–as indicated by item ratings and construct scores–are likely influenced by interactions with others, constraints of higher education systems (e.g., semester timelines), and disciplinary differences in DT-TL approaches (e.g., business vs humanities). Additional research is needed to understand how and why DT is experienced differently by various stakeholders, and which organizational aspects of these contexts might mediate or support relevant outcomes.

DT-TL practices

Although participants indicated frequently utilizing all DT practices, some were used more than others. Namely, participants indicated engaging in Team Formation and Functioning most frequently, which aligns with previous studies [21, 24] and likely reflects DT’s commitment to collaborative project-based problem-solving. In contrast, participants engaged in Protyping and Experimentation least frequently, which also aligns with previous studies in higher education [21]. We speculate that a lack of prototyping and experimentation could be a byproduct of dominant approaches to classroom learning that deemphasize the need for experiential and experimental practices and the constraints of a semester-long course (i.e., limited time available to iterate through the full DT process). While there are many student benefits to experiential learning [29], applied learning [30], process-based learning [31], and service-learning [32], they are time-consuming and resource intensive to execute and come with their own limitations and challenges [32]. We recommend faculty consider generating lower-stakes, quicker-paced student learning opportunities to prototype and test in addition to offering consecutive semester-long courses to ensure students are provided opportunities to develop these skills. Compared to research in workplace settings [24], our faculty and students reported a greater frequency of the following front-end practices: followed a structured process, emphasized active listening among team to find shared meaning, and focused problem definition on the user’s perspective rather than the organization’s. On the other hand, some practices associated with the later phase of the DT process were reported at lower frequency by faculty and students in our study compared to participants in the Liedtka and Bahr [24] study, including: moved multiple ideas into prototyping, got feedback form users and other stakeholders on prototype, and executed real world experiments.

Outcomes of DT-TL

Recent research suggests that DT can empower students to design desirable, feasible, transdisciplinary solutions that promote practical and sustainable outcomes [22, 33]. Our results align with Lake and colleagues [22], who also found that the Quality of Solutions Generated was the most frequently experienced outcome, highlighting the process of engaging students in the DT process as a means for solving complex problems. Not surprisingly, Quality as an outcome had a moderately strong, positive relationship with the practice of Team Formation and Functioning, highlighting the potential benefits of well-designed, high-structured teamwork within courses. In addition, literature highlights potential cognitive and behavioral benefits of DT, showing positive cognitive and behavioral changes for learning and decision-making [28, 34]. Our results align with this literature, suggesting that participants frequently experienced Psychological Benefits and Motivation (e.g., kept people motivated to work on a project to achieve impact). These findings are crucial for educators seeking evidence that DT teaching practices provide students with skills and mindsets for more inclusively and resiliently addressing complex, real world challenges. Both benefits highlight the importance of establishing relevance between student learning and real-world situations through DT pedagogies. We posit that our outcomes of DT-TL were similar and different from those of Liedtka and Bahr [24] due to our different study populations and contexts. We adapted the survey instrument for faculty and students in higher education institutions involved in semester-long classes whereas Liedtka and Bahr [24] designed it for employees of for-profit, non-profit, and government entities. In our sample, the items on trust (built trust among team members, built trust between problem-solving teams and other stakeholders) loaded alongside other relationship items whereas, in Liedtka and Bahr [24], those items loaded onto its own factor. In academic settings, we often talk about relationship-building and trust together, especially as we discuss community engagement, so this makes sense. In community-based participatory research in particular, relationship-building and trust-building go hand-in-hand [35, 36]. We also found that survey items kept people motivated to work on a project to achieve impact, increased a sense of ownership and acceptance of a solution, and increased appreciation for use of data to help drive decisions loaded with the factor on psychological benefits rather than the improved implementation and adaptation factor from Liedtka and Bahr [24]. These three items mark shifts in individual mindsets so this grouping makes sense. Relatedly, the new factor we proposed is termed “Individual Adaptation and Flexibility” and centers around growth mindset and a willingness to learn and change.

DT-TL group differences

Our findings generally indicate that DT-TL practices and valued outcomes are prevalent across disciplines, providing early insights into the potential merits of DT-TL as an interdisciplinary process that is valued across institutions. Several group comparisons warrant discussion and further research, such as undergraduate students experiencing Prototyping and Experimentation more frequently than graduate students. Although graduate degree programs typically immerse graduate students in research, we posit that the structure and processes within graduate programs–which tend to include close oversight from an expert faculty member in a focused area of study–may limit the creativity and brainstorming opportunities afforded to undergraduates. Alternatively, if graduate students experience Prototyping and Experimentation frequently in their degree program, they may perceive this practice as less frequent in DT coursework compared to undergraduate students who are not engaged in similar research programs. As it relates to race and ethnicity, lack of differences should be interpreted with caution since data were collected from predominantly white institutions and the sample sizes for subgroups were relatively small. However, this finding should be explored further given legitimate concerns that DT practices can further privilege those with privileged identities [37, 38].

Future directions

This study marks an important step in understanding DT across institutions and disciplines within higher education and across educational and professional divides. Our findings, coupled with those from K-12 education–where DT is lauded as an integral cognitive process involving creation, collaborative sense-making, reasoning with evidence, experimentation, and evaluation [39-41]–pose several intriguing opportunities for next steps. Taken together, K-12, higher education, postgraduate, and professional studies contribute to the growing body of research that demonstrates clear DT benefits for students throughout their educational careers and beyond. As education faces ongoing scrutiny about its relevance and value, educators must adopt strategies that enable students to address increasingly complex real-world challenges. In Doctors as Makers, for example, Baruch implores medical curricula to foster creative and critical thinkers that can work through not-knowing, seek compassionate solutions, and explore questions differently within an environment supportive of iterative development [42]. Understanding the ways in which DT compliments traditional problem-solving frameworks, such as the scientific method and clinical decision making, could be an important step toward optimizing how DT-TL is integrated into our curricula [16]. With its emphasis on situated and relational problem-solving frameworks, DT-TL also aligns with “problem-posing” pedagogical theory and praxis that is core to various academic disciplines, such as innovation and entrepreneurship. In Pedagogy of the Oppressed, Freire described problem-posing education as a shift from a content-centric, hierarchical educational model to a contextually responsive, co-creative one [43]. Specifically, DT-TL maps seamlessly onto many of the problem-posing processes taught in innovation and entrepreneurship education, particularly to the discovery process (e.g., problem identification through understanding stakeholder needs) and the development process (e.g., iterative testing and evaluation of prototypes). Similarly, other pedagogical models, such as community-based, project-based, and problem-based-learning include components that align closely with DT. Since institutions who make DT a central strategic focus are more likely to bolster and sustain a competitive advantage [44], more research is needed in higher education to understand the potential prevalence and relationship of DT-TL to pedagogical praxis and other pedagogical models (e.g., problem-based learning) across various disciplines. Moving forward, we encourage DT-TL educators to extend our work by collecting similar data, developing improved quantitative measures, and implementing longitudinal studies with faculty, students, and project partners. Larger, more representative samples will promote generalizability and enable analyses and sub-analyses examining additional institution- (e.g., university size, Carnegie Classification, institutional control) and faculty-level (e.g., number of years teaching DT, discipline of home department or school, rank or track) factors. Moreover, it would be helpful to know how often students experience DT prior to taking these courses, shortly after taking these courses, and several months or years after taking the courses. This could allow us to better establish causality and attribute changes to the specific courses. Additional research should also be conducted to understand DT-TL outcomes for co-curricular opportunities, such as student organizations and internships to determine what other opportunities in higher education settings can yield similar or better outcomes.

Limitations

With regards to study limitations, we encountered standard challenges with response rates, missing data, and non-response. We recruited faculty and students from four predominantly White higher education institutions in the same state and had a small sample size of faculty at each institution. Data also reflect student and faculty perception at the end of a course–not the long-term outcomes/value of these practices for them or for their project collaborators. That being said, our results are not meant to be generalizable to all DT courses in across higher education settings. Rather, they provide an important first glimpse into DT-TL practices across multiple universities and disciplines.

Conclusion

College students must be equipped with the critical, creative, and collaborative skills needed to address society’s increasingly complex and difficult challenges. As this study further verified, DT helps to generate trust across collaborators, fosters the motivation needed to sustain problem-solving efforts, and increases the quality of solutions generated. This study demonstrated the validity of DT across disciplines and universities, the ways in which DT practices and outcomes are experienced in higher education, and a number of important differences between DT practices within higher education and other sectors. Extending this work to include additional universities and research methodologies is critical for providing insight into enhancing the value of DT pedagogies.

Outcomes of design thinking coursework.

(DOCX) Click here for additional data file. 19 Jan 2022
PONE-D-21-32075
Design Thinking Teaching and Learning in Higher Education: Student and Faculty Experiences Across Four Universities
PLOS ONE Dear Dr. McLaughlin, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Mar 05 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Oathokwa Nkomazana, MD MSC PhD Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. 1. Please ensure that you include a title page within your main document. We do appreciate that you have a title page document uploaded as a separate file, however, as per our author guidelines (http://journals.plos.org/plosone/s/submission-guidelines#loc-title-page) we do require this to be part of the manuscript file itself and not uploaded separately. Could you therefore please include the title page into the beginning of your manuscript file itself, listing all authors and affiliations. 2. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables (should remain/ be uploaded) as separate "supporting information" files. 3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know Reviewer #2: Yes ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This is a very important manuscript regarding DT-TL. As authors have stated, this is a subject that taught mainly in certain disciplines and not in other. The authors therefore have shared pioneering work in regard to DT in Humanities/ Social Sciences. The introduction is clearly written. There are a number of queries that I have noted that may improve the quality of the manuscript: 1. The authors could also include and infer about the impact of DT in various curricula in their introduction or discussion since they mention that DT is already taught in certain disciplines. The authors could also highlight the type of curricula for the four universities and/or courses of the participants. Certain curricula have some components of the DT such as Problem Based Learning curriculum. 2. They adapted a survey from Liedtka and Bahr who studied DT practices among the workforces. The authors were not clear how after adaptation they validated the tool before they started their research. 3. In their research questions, they could have grammatic review and edit the questions to make them more clearer and highlight the sub-questions. 4. There are sentences where a semi-colon is more appropriate than a comma or a hyphen (line 96) 5. With regard to ethical consideration, the authors obtained 3 IRB approvals, but 4 universities participated. They remain silent on the fourth university IRB approval. 6. Within the results section, the authors mention that the majority of participants majored in Humanities/ Social Sciences and that they were mainly white and undergraduates. Since the authors discusses these variables in the results/discussion, it remains succinctly prudent for them to mention all the disciplines, non-white population participating in the study and, the number of graduate students involved. Is discussing these variables in place in terms of population size of each? 7. Should the standard deviations be written without a zero (.68) or, in full starting with a zero (0.68)? If so, it should be consistently added to all the standard deviations. 8. In the discussion, they could briefly share their thoughts of some faculty who teach DT without the DT expertise. 9. In the discussion, the authors mention how their research complements conceptual frameworks of DT in education: they could mention two or three of them. 10. Write org in full (line 243) 11. Line 287: the authors could share their thoughts on the reason why undergraduate students reported Prototyping and Experimentation more frequently than graduate students as the reverse is expected as the norm with graduate. 12. The authors decided to publish the qualitative component of the research separately, some of the critical perspectives about DT-TL may be missing in this manuscript especially that it seems the authors have not fully answered some of the research questions such as the No.3 research question (Is DT a valid construct within teaching and learning?) 13. The section under Future Directions seems to be too long and at times reads more like the general discussion. 14. They should be consistent with their referencing Reviewer #2: Design Thinking Teaching and Learning in Higher Education: Student and Faculty Experiences Across Four Universities Summary observations Abstract is clear although the results component is in places inconsistent with what is indicated in the main results section of the manuscript. In the introduction, the authors indicate that ‘DT practices and outcomes across higher education is still new’. Therefore, they did not discuss or reference any available published literature in the ‘introduction’, to give a sense of the overall context. This creates the impression that nothing has been published in this area. The specific research questions are well-articulated. The researchers should have stated the main research question before listing the specific research questions. Appropriate references are quoted, institutional review boards’ approval were obtained and informed consent obtained. Tables are clear and readable but Table 1 needs to be revised to reflect ‘missing data’. The data were collected, analysed and interpreted by and large correctly. However, the results that reflect the ‘main findings’ of the study as stated in the abstract should have been presented in a table to enable the reviewers to interrogate the data analysis and the conclusion thereof. This was not possible. The results are adequately discussed but there are instances where the key findings are stated with little or no discussion. The authors discuss the limitation of the study. However, some of the limitations mentioned are not limitation per se. Language editing is strongly recommended. Specific Comments Abstract L15/16 contradicts/inconsistent with L287. ‘Differences found based on discipline and students’ level’. Pay attention to statement on ‘student level’ Introduction L26-29 Long sentence L48 – ‘wicked’ not sure it is appropriate for formal writing L66 – “outcomes across higher education is still new’. Though ‘new’ what does the limited/available published data show? and give references. L73-81. Are specific research questions but what is the main/overall research question? State it first before giving specific research questions? Data Collection L101-104 has to be reconciled with L152/153 (Results section). Purposive sampling was used and faculty who teach design thinking courses were recruited (L101-L104). If this is true, how come faculty with ‘none’ DT expertise (L152) were recruited? How does this help answer the research questions? Ethical considerations, consent L132-134. Study involved four (4) universities (L100) but only three (3) review boards are mentioned. Results L159/160. It is ‘shared meeting’ or ‘shared meaning’? L170-175. Reading the abstract it appears L170-175 represent the ‘main findings’ of the study. How come then the analysis is not presented in a table to enable the reviewers to interrogate the data and the analysis. Suggest that the results be tabulated. Discussion L209. ‘First and foremost’ unnecessary words L213. ‘8 outcomes constructs’ shouldn’t it be ‘5 outcome constructs’ L236-242. It is simply reporting the results and there is no discussion; should be moved to either the results section or inserted after line 226 L243. ‘Org.’ should be written in full. L266-267. Please revise to connect the two sentences better L283. ‘centers around growth mindset and a willingness to learn and change’. Revise this sentence, not easy to understand. Should it be ‘centers around (on) growth mindset (growth of mindset) and a willingness to learn and change ‘ L290/291. Needs elaborating and clarifying. Can this conclusion be drawn?, given that the study was conducted in predominantly white institutions and the numbers for other races (individually) are low; much underrepresented. Future Directions L310. Is it ‘institution who…’? L316-328. Not sure about the relevance of these lines to the manuscript. Looks like it is referring to the ‘qualitative component of the overall study’ which is not the subject of this manuscript. Limitations L358. Reconcile this with L85-87. How can ‘lack of qualitative data’ be a limitation when this was a mixed methods study and the researchers ‘intentionally’ decided not to consider the qualitative and quantitative data together? Conclusion L368-370. Please revise. It is unclear and incomplete. For instance, what is “promote mindsets…’? Table 1 • The students’ numbers do not add up to 196 and percentages do not up to 100% for the participants’ characteristics. Please include in the table ‘missing/unknowns/no responses’ in your table. • Why is it that only one ethnicity is recorded? It is also not clear what its relevance is. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Enoch Sepako, PhD, MHPE [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.
18 Feb 2022 The following responses are also included as a separate file in this resubmission: Academic Editor Comments: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf Manuscript and file names revised to meet style requirements 2. 1. Please ensure that you include a title page within your main document. We do appreciate that you have a title page document uploaded as a separate file, however, as per our author guidelines (http://journals.plos.org/plosone/s/submission-guidelines#loc-title-page) we do require this to be part of the manuscript file itself and not uploaded separately. Could you therefore please include the title page into the beginning of your manuscript file itself, listing all authors and affiliations. Title page added to the manuscript. 2. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables (should remain/ be uploaded) as separate "supporting information" files Tables added to the main manuscript directly after the paragraph in which each is first cited. 3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide. As noted in the cover letter, we have made our data available in ICPSR at https://doi.org/10.3886/E151681V1 (ie the DOI for our data is 10.3886/E151681V1) Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. References updated per style guidelines. No references were removed or added. Reviewer Comments to the Author Reviewer #1 This is a very important manuscript regarding DT-TL. As authors have stated, this is a subject that taught mainly in certain disciplines and not in other. The authors therefore have shared pioneering work in regard to DT in Humanities/ Social Sciences. The introduction is clearly written. Thank you for this feedback. We have addressed your recommendations below and believe this has strengthened our work. There are a number of queries that I have noted that may improve the quality of the manuscript: 1. The authors could also include and infer about the impact of DT in various curricula in their introduction or discussion since they mention that DT is already taught in certain disciplines. The authors could also highlight the type of curricula for the four universities and/or courses of the participants. Certain curricula have some components of the DT such as Problem Based Learning curriculum. Thanks for this comment. As noted in Table 1, we collected some data about course characteristics (e.g., Real World Project Used for DT, Percent of Course Time Spent in Teams). However, we did not collect any other details or data about the curricula/pedagogy for this study (such as PBL). In the discussion, we have revised our paragraph about innovation and entrepreneurship training in higher education to address the need to explore the impact DT more closely as it relates to the various disciplines and pedagogies (e.g., PBL). 2. They adapted a survey from Liedtka and Bahr who studied DT practices among the workforces. The authors were not clear how after adaptation they validated the tool before they started their research. This paper provides validity evidence for the revised tool. Prior to administering the survey, we used face validity to adapt the tool, as now noted in the methods. After administering the survey, we used factor analysis and correlation analysis to generate validity evidence. 3. In their research questions, they could have grammatic review and edit the questions to make them more clearer and highlight the sub-questions. Research questions and sub-questions revised for clarity. 4. There are sentences where a semi-colon is more appropriate than a comma or a hyphen (line 96) Punctuation revisions made where considered appropriate by the authorship team. 5. With regard to ethical consideration, the authors obtained 3 IRB approvals, but 4 universities participated. They remain silent on the fourth university IRB approval. Revised to include the fourth review board. 6. Within the results section, the authors mention that the majority of participants majored in Humanities/ Social Sciences and that they were mainly white and undergraduates. Since the authors discusses these variables in the results/discussion, it remains succinctly prudent for them to mention all the disciplines, non-white population participating in the study and, the number of graduate students involved. Is discussing these variables in place in terms of population size of each? Statistics for all demographic variables collected in this study are included in Table 1. We have chosen not to write all of these results in the text, since it would be duplicative of the Table. 7. Should the standard deviations be written without a zero (.68) or, in full starting with a zero (0.68)? If so, it should be consistently added to all the standard deviations. Leading zeros have been added to all data, including standard deviations. 8. In the discussion, they could briefly share their thoughts of some faculty who teach DT without the DT expertise. All faculty who teach DT indicated having some DT expertise, as noted in Table 1. We have clarified this point in the results. 9. In the discussion, the authors mention how their research complements conceptual frameworks of DT in education: they could mention two or three of them. Two examples added. 10. Write org in full (line 243) Revised. 11. Line 287: the authors could share their thoughts on the reason why undergraduate students reported Prototyping and Experimentation more frequently than graduate students as the reverse is expected as the norm with graduate. This has been addressed in the discussion section, which posits that graduate degree programs typically immerse graduate students in research– typically with close oversight from an expert faculty member in a focused area of study – that may limit the creativity and brainstorming opportunities afforded to undergraduates. Alternatively, if graduate students experience Prototyping and Experimentation frequently in their degree program, they may perceive this practice as less frequent in DT coursework compared to undergraduate students who are not engaged in similar research programs. 12. The authors decided to publish the qualitative component of the research separately, some of the critical perspectives about DT-TL may be missing in this manuscript especially that it seems the authors have not fully answered some of the research questions such as the No.3 research question (Is DT a valid construct within teaching and learning?) The factor analysis conducted in this study provides strong construct validity evidence for DT-TL. As such, we argue that DT-TL is a valid construct based on this evidence. This is described in detail in the Discussion: DT-TL Constructs section. 13. The section under Future Directions seems to be too long and at times reads more like the general discussion. This section has been shortened and revised so that each paragraph concludes with a next step/future direction. 14. They should be consistent with their referencing. References were revised for according to journal style requirements for consistency. Reviewer #2 Summary observations Abstract is clear although the results component is in places inconsistent with what is indicated in the main results section of the manuscript. In the introduction, the authors indicate that ‘DT practices and outcomes across higher education is still new’. Therefore, they did not discuss or reference any available published literature in the ‘introduction’, to give a sense of the overall context. This creates the impression that nothing has been published in this area. The specific research questions are well-articulated. The researchers should have stated the main research question before listing the specific research questions. Appropriate references are quoted, institutional review boards’ approval were obtained and informed consent obtained. Tables are clear and readable but Table 1 needs to be revised to reflect ‘missing data’. The data were collected, analysed and interpreted by and large correctly. However, the results that reflect the ‘main findings’ of the study as stated in the abstract should have been presented in a table to enable the reviewers to interrogate the data analysis and the conclusion thereof. This was not possible. The results are adequately discussed but there are instances where the key findings are stated with little or no discussion. The authors discuss the limitation of the study. However, some of the limitations mentioned are not limitation per se. Language editing is strongly recommended. Thank you for this feedback. We have addressed your observations below in point-by-point responses and believe these revisions have strengthened our work. Specific Comments Abstract L15/16 contradicts/inconsistent with L287. ‘Differences found based on discipline and students’ level’. Pay attention to statement on ‘student level’ Corrected in abstract. “Student level” revised to “student type” throughout. Introduction L26-29 Long sentence Sentence shortened. L48 – ‘wicked’ not sure it is appropriate for formal writing Word removed. L66 – “outcomes across higher education is still new’. Though ‘new’ what does the limited/available published data show? and give references. To our knowledge, ours is the first study to collect and validate data across multiple higher education institutions. We have expanded this paragraph to note what others have found and that their studies have been implemented at a single institution or single course. L73-81. Are specific research questions but what is the main/overall research question? State it first before giving specific research questions? The overarching research question has been added (how do faculty and students experience design thinking within higher education courses?), prior to giving the specific research questions. Data Collection L101-104 has to be reconciled with L152/153 (Results section). Purposive sampling was used and faculty who teach design thinking courses were recruited (L101-L104). If this is true, how come faculty with ‘none’ DT expertise (L152) were recruited? How does this help answer the research questions? As seen in Table 1, all faculty reported have at least limited expertise (none of them indicated having “none”). This has been clarified in the Results section. Ethical considerations, consent L132-134. Study involved four (4) universities (L100) but only three (3) review boards are mentioned. Revised to include the fourth review board. Results L159/160. It is ‘shared meeting’ or ‘shared meaning’? Corrected to “shared meaning” L170-175. Reading the abstract it appears L170-175 represent the ‘main findings’ of the study. How come then the analysis is not presented in a table to enable the reviewers to interrogate the data and the analysis. Suggest that the results be tabulated. Demographic results added to Tables 3 & 4 Discussion L209. ‘First and foremost’ unnecessary words Removed. L213. ‘8 outcomes constructs’ shouldn’t it be ‘5 outcome constructs’ Revised. L236-242. It is simply reporting the results and there is no discussion; should be moved to either the results section or inserted after line 226 Section removed. L243. ‘Org.’ should be written in full. Revised. L266-267. Please revise to connect the two sentences better Revised. L283. ‘centers around growth mindset and a willingness to learn and change’. Revise this sentence, not easy to understand. Should it be ‘centers around (on) growth mindset (growth of mindset) and a willingness to learn and change ‘ Removed “growth mindset” and revised to “since it embodies…” L290/291. Needs elaborating and clarifying. Can this conclusion be drawn?, given that the study was conducted in predominantly white institutions and the numbers for other races (individually) are low; much underrepresented. Revised to acknowledge low sample size and predominantly white institutions. Future Directions L310. Is it ‘institution who…’? Revised. L316-328. Not sure about the relevance of these lines to the manuscript. Looks like it is referring to the ‘qualitative component of the overall study’ which is not the subject of this manuscript. This section has been revised to align more clearly with the current study. Limitations L358. Reconcile this with L85-87. How can ‘lack of qualitative data’ be a limitation when this was a mixed methods study and the researchers ‘intentionally’ decided not to consider the qualitative and quantitative data together? Limitation removed. Conclusion L368-370. Please revise. It is unclear and incomplete. For instance, what is “promote mindsets…’? Removed. Table 1 • The students’ numbers do not add up to 196 and percentages do not up to 100% for the participants’ characteristics. Please include in the table ‘missing/unknowns/no responses’ in your table. A note has been added to Table 1 to clarity that some variables have missing data. • Why is it that only one ethnicity is recorded? It is also not clear what its relevance is. The use of this ethnicity is standard practice for demographic data collection in the United States. Per the US Census Bureau “Though many respondents expect to see a Hispanic, Latino, or Spanish category on the race question, this question is asked separately because people of Hispanic origin may be of any race(s).” ________________________________________ Submitted filename: Response to Reviewers.docx Click here for additional data file. 10 Mar 2022 Design Thinking Teaching and Learning in Higher Education: Experiences Across Four Universities PONE-D-21-32075R1 Dear Dr. McLaughlin, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Alessandro Margherita Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know Reviewer #2: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors have sufficiently responded to reviewers comments. The authors mentioned publishing the qualitative component of the research separately. This part of work should be discussed only if it adds-value to this manuscript discussions. Reviewer #2: L103 -L106: 'In one single institution study, for example, researchers found that DT requires time and trust which can be constrained by the imposed deadlines of semester-based projects [21]. In a single course study, students indicated that their “whirlwind” course promoted almost “exponential” growth [22]' should be inserted after L108. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: Yes: Enoch Sepako, PhD 15 Mar 2022 PONE-D-21-32075R1 Design thinking teaching and learning in higher education: Experiences across four universities Dear Dr. McLaughlin: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Alessandro Margherita Academic Editor PLOS ONE
  5 in total

Review 1.  Enhancing Community-Based Participatory Research Through Human-Centered Design Strategies.

Authors:  Elizabeth Chen; Cristina Leos; Sarah D Kowitt; Kathryn E Moracco
Journal:  Health Promot Pract       Date:  2019-05-25

2.  Design Thinking as a Tool for Interdisciplinary Education in Health Care.

Authors:  Tim C van de Grift; Renske Kroeze
Journal:  Acad Med       Date:  2016-09       Impact factor: 6.893

3.  Doctors as Makers.

Authors:  Jay M Baruch
Journal:  Acad Med       Date:  2017-01       Impact factor: 6.893

4.  Computational Thinking Is More about Thinking than Computing.

Authors:  Yeping Li; Alan H Schoenfeld; Andrea A diSessa; Arthur C Graesser; Lisa C Benson; Lyn D English; Richard A Duschl
Journal:  J STEM Educ Res       Date:  2020-05-18

Review 5.  A qualitative review of the design thinking framework in health professions education.

Authors:  Jacqueline E McLaughlin; Michael D Wolcott; Devin Hubbard; Kelly Umstead; Traci R Rider
Journal:  BMC Med Educ       Date:  2019-04-04       Impact factor: 2.463

  5 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.