Innovative Assessment That Combines Collaborative and Self-Directed Learning With Integration and Application of Knowledge: Teamwork Group Projects.

INTRODUCTION: There is a worldwide focus on the early development of collaborative skills in medical students as reflected in the design of the medicine program at the University of New South Wales, Australia. Integral to the success of student-centered curricula, is early development of students' self-directed and collaborative learning skills. The purpose of this innovative assessment is to develop and assess students' skills in self-directed and collaborative learning while they concurrently engage with stage-appropriate content knowledge.
METHODS: The educational design of the group projects allows junior medical students to work collaboratively to develop a deep understanding of the concepts and principles of a clinical scenario. Students are required to integrate and apply knowledge from different disciplines and share their learning with a wider peer group through appropriate peer teaching strategies. Two variants of these group projects are described in this resource, and generic versions of student and assessor instructions are included as Appendices A and B.
RESULTS: Feedback on the projects collected over the last 7 years has been positive. Students begin to see the relevance and benefits of learning together and appreciate the impact this has on the quality of their learning. They also begin to appreciate the relevance of collaborative skills to their future practice as doctors. DISCUSSION: The group projects are based on well-established educational principles, and the templates provided in the appendices can be adapted by other medical educators.

Educational Objectives

Through clinical scenario group project activities, students are expected to:

Develop skills in identifying and using learning strategies that are most relevant to the concepts and principles that arise from the clinical scenarios.

Develop skills in integrating knowledge from various disciplines.

Develop, use, and evaluate appropriate peer teaching strategies.

Develop and use skills in self-directed learning and collaborative learning (teamwork).

Apply their learning to a similar but different scenario that they develop themselves.

Introduction

The importance of teamwork is well recognized throughout medical education and medical practice.[1] Medical schools recognize the importance of nurturing collaborative learning and teamwork skills in students from an early stage of the medicine program. This has led to a worldwide revolution in medical education in general, with an emphasis on collaborative learning. Group work is now an important educational tool in medicine.[2]

The need to develop skills in self-directed learning, critical evaluation, and teamwork is recognized by a range of medical education regulatory bodies.[3-6] The medicine curriculum at the University of New South Wales (UNSW), Australia, places a high emphasis on developing these skills in medical students. As such, teamwork and self-directed learning/critical evaluation are two of the many graduate capabilities of the program.[7] In every course across the medicine program, medical students work within small groups. In the first 2 years of the program (Phase 1), these groups are named scenario groups, in which the content and instructional methods are based around two to three clinical/health scenarios per course. Each scenario group is divided into three subgroups for the group projects. The group projects constitute one of three summative assessments in each course.

The teamwork group projects reported here combine collaborative and self-directed learning with the integration and application of content knowledge. Although self-directed learning and collaborative learning are sometimes perceived as opposing strategies, the innovative group project design supports simultaneous development of skills in self-directed and collaborative learning. In self-directed learning, learners diagnose their learning needs and formulate learning goals. This process is enhanced by giving and soliciting peer feedback on these endeavors. This is a critical ability that should be developed in a self-directed learner.[8] A type of collaborative learning that supports such self-directed learning in particular is cooperative learning.[9] The concepts of positive interdependence and individual accountability in these group projects promote self-direction, while the concepts of face-to-face promotive interaction, social skills development, and analysis of group process promote feedback and reflection. The concurrent development of these aspects was a key aim of the teamwork group project design.

In teamwork group projects, every student in the group has the responsibility to effectively contribute to learning and discussion within the group. Contributions range from formulating learning goals to making decisions on the best learning and teaching strategies. Meanwhile, in individual self-study time, each student is expected to prepare and learn the relevant material for the project. While collaboratively working in the group, students learn from each other and exchange feedback that further develops their learning. This collegial interaction also helps in developing group and personal goals in learning and achievement. Students receive feedback on their peer teaching and provide formal feedback to each other through an online peer feedback system.

The Context Within Which These Assessments Were Developed

In 2004, the UNSW medicine program underwent reform informed by evidence-based innovations in medical education.[7] One of the significant features of the medicine program at UNSW is its focus on collaborative learning. In each course during the first 2 years of the program, medical students form teams of four to six students and work on a group project. During the second half of each year, students from Years 1 and 2 are combined for learning and teaching activities and are expected to work together in all activities, including group projects.[7] This adds a novel dimension to the development of collaborative skills. We refer to these combined-year groups as mixed-year groups.

The medicine program is continuously monitored and evaluated. Despite an initial positive response and pleasing evaluation results, an important finding was that merely changing to a more student-centered curriculum and providing opportunities for collaborative and self-directed learning could polarize the students. This manifested as unexpected patterns of change in students’ approaches to learning.[10,11] These findings indicated that innovative strategies were needed to develop students’ learning skills in order to maximize the benefits of the curricular innovation.[10-12] A further gap was identified in junior medical students’ understanding of the relevance and importance of teamwork skills.[13] Student feedback suggested that attempts to teach teamwork skills were of limited value, but students continued to demonstrate a high level of interest in activities related to assessment. This led to a decision to design and trial a series of assessment tasks to drive the development of teamwork skills, based on the principle of assessment for learning.[14] Student feedback also suggested a limited understanding of the scenario-based learning approach utilized in the first 2 years, as well as highlighting the need to support students to develop the required skills in self-directed learning. The projects described here were developed in response to these needs. While it may seem somewhat ambitious to develop a project that could address all these needs concurrently, the design and implementation of these projects over the last 7 years have suggested that this is indeed feasible. In fact, student feedback suggests that the close linking of the self-directed and collaborative skills to better understanding of content knowledge has had a very positive impact on the effectiveness of the project.

Methods

Educational Approach and Rationale

The skills of self-directed learning and collaborative learning cannot be developed in isolation. Therefore, the project activities are designed for students to engage in self-directed and collaborative learning while working on a relevant body of content.

The design of the projects was informed by a combination of educational theory and experience and follows the standard format used for group projects in Phase 1 of the UNSW medicine program. A comprehensive literature review was conducted, and a range of experienced educators were consulted. The assessment design team also drew on a range of scholarly work that they had previously undertaken on the use of assessment tasks to develop teamwork skills in medical students.[15,16] The group projects draw on a number of important principles and include a range of innovative approaches, as summarized below.

A holistic approach to combining content knowledge learning with development of collaborative and self-directed learning: The skills of collaborative learning and self-directed learning cannot be taught or learned in isolation. Appropriate tasks need to be designed to develop these skills in students. The design of the teamwork group project allows students to develop these skills alongside the integration and application of stage-specific content.

Combined focus on teamwork and self-directed learning: Student feedback indicated that students tend to view self-directed learning and teamwork as opposing approaches that are not compatible with each other.[12] The teamwork group projects require students to work on activities such as identifying the key concepts and principles (termed learning issues), developing strategies for learning and peer teaching, designing concept maps, and applying their learning to alternate health scenarios. These activities help students work with each other to identify their learning goals and develop appropriate strategies to address them. The collaborative element adds richness to the self-directed approach at both individual and group levels. This positive interdependence, a basic element of cooperative learning, is thus directed towards self-directed learning. The student groups need to take individual and collective responsibility for directing themselves towards achieving their learning goals: This is another basic element of cooperative learning. These projects thus illustrate how self-directed learning can be undertaken by a team in a collaborative manner. This complementary nature of collaborative and self-directed learning is described in the literature. For example, Love, Dietrich, Fitzgerald, and Gordon described how the use of collaborative learning strategies and projects in learning communities led to the communities becoming more self-directed and independent in their learning, in addition to developing a range of other skills.[17]

The use of concept maps (during the first-year group project): Concept maps are known to optimize collaborative and meaningful learning within a group.[18] The group project introduces students to concept maps that help to integrate their learning from different disciplines.

Applying the relevant concepts to an alternate scenario (during the mixed-year group project) and an early introduction to clinical concepts: For mixed-year student groups, this activity is an extension of developing concept maps; students apply the concepts to an alternate clinical scenario that they design themselves. This is an opportunity for students to start understanding the relevance of their learning to future clinical practice. It should be noted that early introduction to clinical aspects of medicine is highly encouraged by regulatory bodies,[19] and many medical schools strive to incorporate clinical skills teaching and learning from the start of the medical degree.[20]

Evaluation of team skills: Students are required to evaluate their performance and that of their colleagues. This aspect applies across all group projects in the first 2 years. This is supported by a suite of instruments that the students are recommended to use in order to evaluate their general teamwork skills within the project team. These include the Knowles group behavior outline and rating chart, Belbin Self-Perception Inventory, sociogram recording activity, and group communication observer sheet.[21] The teamwork evaluation and reflection support students in understanding the challenges they have faced as a team, formulating solutions to overcome the problems, and developing a plan of action for future work as a team. Such analysis of group process is another important element of cooperative learning.

Peer teaching and evaluation: Students are required to design peer teaching activities to assist the wider scenario group to gain a deeper understanding of the key learning concepts. These peer teaching activities are evaluated by participating peer learners based on their usefulness and outcomes achieved. The project requires students to reflect on the peer feedback and iteratively improve their peer teaching. According to a recent systematic review and meta-analysis of peer teaching, such early experience in teaching is useful for both the learner students and the tutor students.[22] However, it is important to ensure the quality of peer teaching methods[22]; this aspect has been addressed in these group projects through evaluation and feedback.

Educational Principles

As described above, these teamwork group projects were designed to develop a range of competencies within an integrated medical program. The group projects can be adapted to other curricula that strive to achieve integration and application of knowledge, as well as the development of collaborative and self-directed learning skills.

Type of Assessment

This is an assessment that combines integration and application of knowledge with the development of self-directed and collaborative learning skills. Two variants of the assessment (the first-year project and the mixed-year project) have been developed to cater to the relevant stages of the students. The overall assessment and the features of each variant are presented in the Table below. We have also created a Figure to map the group project tasks to a revised version of Bloom's hierarchy of cognitive behaviors.[23]

Table:

Overall Features of the Group Project and Its Two Variants

Assessment Characteristic	First-Year Project: Understanding the Course Scenario	Mixed-Year Project: Integrating Learning Through Scenarios
Target population	Year 1 medical students	Mixed-year groups of Year 1 and Year 2 students
Project tasks	Identify key concepts and principles from the relevant scenario.Develop strategies to address the identified key concepts and principles.Conduct peer teaching around these key concepts and principles.Create a concept map of the key concepts and principles.	Identify key concepts and principles from the relevant scenario.Develop strategies to address the identified key concepts and principles.Conduct peer teaching around these key concepts and principles.Develop an alternate scenario that highlights the key concepts and principles.
Project aims and outcomes	To develop students’ skills in both self-directed and collaborative learning.To develop students’ skills in integrating concepts that arise from a health scenario.	To develop students’ skills in both self-directed and collaborative learning.To develop students’ skills in integrating concepts and applying them to similar but different (alternate) scenarios.
Formative evaluation	Peer evaluation and feedback on teaching and learning activities.Facilitator feedback on peer teaching activities within scenario session.Group evaluation and reflection of teamwork skills within the group.Group presentation of the project within the scenario group with evaluation and feedback.	Peer evaluation and feedback on teaching and learning activities.Facilitator feedback on peer teaching activities within scenario session.Group evaluation and reflection of teamwork skills within the group.Group presentation of the project within the scenario group with evaluation and feedback.
Summative evaluation and project grading	Examination of written report (2,500 words). The report is assessed against specific assessment criteria that are provided in advance. This assessment results in a grade along a 4-point scale (F, P−, P, P+) and comments on students’ performance in relation to the assessment criteria.	Examination of written report (2,500 words). The report is assessed against specific assessment criteria that are provided in advance. This assessment results in a grade along a 4-point scale (F, P−, P, P+) and comments on students’ performance in relation to the assessment criteria.

Figure.

Project tasks mapped to a revised version of Bloom's hierarchy of cognitive behaviors.[23] (Image by Viran Weerasekera, used with permission.)

Implementation

The first-year project is offered to commencing medical students in the second course of the medicine program, which is their first summatively assessed course. The mixed-year project is offered to mixed-year (Years 1 & 2) students who are more advanced in Phase 1 of the medicine program. Generic versions of student and assessment instructions/guidelines as well as a sample marking guides are located in Appendices A and B.

In both projects, students work collaboratively over a period of 8 weeks. A standard scenario group consists of 12–14 students, which is too high a number for inclusion in a single group project team. Thus, students are requested to form three teams of four to six students each, and each team has choice in selecting a particular group project from a range of offerings focusing on different graduate capabilities. Students are free to self-select the group they prefer to work in, but balanced representation from both Year 1 and Year 2 students for the mixed-year group project is required. By the end of the first 2 years of the medicine program (i.e., within eight courses), each student is expected to complete at least one project that focuses on teamwork, and completion of this project is one way to meet that requirement.

A total of four courses are completed in Year 1 of the medicine program and four more in Year 2. Each course is approximately 7 to 8 weeks in duration. A clinical scenario relevant to the course theme is presented to the students on the first day of the course. There are many different formats of the clinical scenario presentation ranging from video scenarios to patient interviews to expert-led seminars. For example, the kidney disease scenario focuses on a 62-year-old male with autosomal dominant polycystic kidney disease. Based on a recorded interview with the patient, the scenario illustrates his overall circumstances as well as his symptoms of vomiting, fatigue, lethargy, back pain, and related complications. This scenario thus stimulates students’ intellectual curiosity and leads them to explore chronic kidney disease (CKD) and its clinical presentations. The second part of the scenario describes the patient's home hemodialysis routine and explores the principles of CKD management. An example of a similar scenario is presented in Appendix C, along with a description of the process used to develop scenarios.

Students may submit a written report of their group project via the electronic learning management platform of the UNSW medicine program known as eMed or through alternative methods. The written project report is assessed according to predefined assessment criteria. The assessment criteria specify the standard of performance expected for a satisfactory (pass) grade in each of the relevant graduate capabilities. In addition, the student is also assigned an overall score. These assessment criteria are shared in advance with the students; therefore, students have a clear understanding of what is expected from them. The project reports are graded using a standard grading system of F (fail), P− (borderline pass), P (pass), and P+ (excellent pass). The group project team receives a grade and written feedback for each capability that is assessed in the project. The feedback explains why the grade was assigned and the extent to which the assessment criteria were addressed and also includes guidance for future improvement.

The assessment criteria and a marking guide were developed by the educational design team, which included experts in medical education and faculty members who designed the clinical scenario (Appendices A & B). The marking guide clarifies the standard expected for each criterion. The educational design team is available to provide further information and guidance on marking if required.

Consistency of assessment standards across courses is ensured by linking the assessment criteria to predetermined graduate capability statements that guide the entire program. Graduate capability statements are descriptors of the standard of performance expected at each phase, across each of the eight graduate capabilities of the UNSW medicine program.

Results

The first iteration of this project was introduced in 2008 in a single course in Year 2 of the medicine program. Following its success, the two variants of this assessment were developed to suit junior students and mixed-year students. The projects have been adopted by all other first- and second-year courses, and currently, a teamwork project is offered across all Phase 1 courses (with one exception where students complete a set project focusing on research). Evidence of the effectiveness of these group projects is drawn from student reflections on the teamwork capability in the project reports and from tutor feedback on the group project over the last 7 years.

Student Reflections

Student reflections suggest that the project has achieved many of its aims. These comments, spanning 7 years since 2008, are thematically presented below. Please note, a common them that emerges in students’ reflections is on understanding the relevance of teamwork and teamwork skills for clinical practice.

“It may have taken some of us over a year, but this project, particularly the section on teamwork, made us realise the importance of the methods used in the UNSW Medicine Course. Some of us had been cynical in the past, thinking that SGs and tutorials were a waste of time, but we now realise that out of all of our learning, these small, group discussion-based classes are probably the most effective tools for learning if utilised effectively. Wow! At least we worked this out before we graduated!” (Group reflection, Project 2: Beginnings, Growth, and Development course A 2008)

“This assignment encouraged development of skills in both self-directed and collaborative learning. In our first meeting we all contributed to a concept map … ensuring we all had a good understanding of how all our concepts linked together…. This assignment has made us appreciate the utility of concept maps in bringing lots of differing and contrasting information together and drawing out inter-relationships. In peer teaching within our group we found that the concept map allowed us to identify the key learning issues.” (Group reflection, Project 2: Health Maintenance course A 2014)

“This project also taught us the importance of understanding the relationships between the learning issues. This enabled the application of these issues into alternative scenarios, which helped to reinforce our understanding of the various topics. We started off our teaching sessions with a broad overview of the scenario and learning issues before giving a microscopic view of each topic. This was done with the aim to ensure that our peers truly understood the big picture and content instead of practicing rote learning.” (Group reflection, Project 2: Health Maintenance course A 2014)

“It is apparent that effective teamwork is crucial in providing quality healthcare to patients, as health professionals work as a team to provide a proper diagnosis and treatment. Because health professionals share a common vision and common goals to promote patient health, communication and collaboration are essential skills to develop, both personally and professionally. This group project provided a medium for teamwork skills to cultivate and ensure each member was accountable for his or her contributions. Each of the scenarios provided a basic framework and illuminated the need for teamwork among health professionals. The teaching opportunities also allowed for structured learning and the development of knowledge for both the project and scenario group members.” (Group reflection, Project 2: Health Maintenance course A 2014)

“In previous group projects, we had taken for granted the complexities of team dynamics and made assumptions that the way in which the group works is entirely secondary to the content of the project itself. Undertaking a project focused on teamwork itself made us aware of what constitutes an effective team and illuminated that the process of the project may prove to be just as effective an educational tool as the content.” (Group reflection, Project 1: Beginnings, Growth, and Development course B 2009)

“Despite having done numerous other group projects prior to this, this project proved to be a challenging one. This was due to the exceptional need for collaboration among members since this project required a significant amount of input (in terms of hours spent in practice sessions and meetings) and output (numerous peer teaching sessions)…. This enhanced our efficiency in completing our assigned tasks. Honing our teamwork and effective communication skills through this project will prove to be beneficial in the future since these are essential skills when working in multidisciplinary healthcare teams.” (Group reflection, Project 2: Health Maintenance course A 2014)

“This realization of progression in terms of group dynamics, learning issues and difficulties encountered was evident in all group members’ reflections, and has increased our ability to pursue knowledge through working with others not only in the current medical course, but also in our future medical careers.” (Group reflection, Project 1: Beginnings, Growth, and Development course B 2009)

Content Validity

The assessment focuses on two important student capabilities: (1) self-directed learning and critical evaluation and (2) teamwork. Specifically, to address the self-directed learning and critical evaluation capacity, the project identifies questions, key concepts, and principles arising from scenario sessions and other teaching activities and engages in appropriate activities to address identified needs. To address the teamwork capacity, the project develops appropriate methods of peer teaching and discusses the effectiveness of these methods, as well as identifying different purposes of group work and analyzing how well groups work. Teamwork is also addressed through the project's discussion of differences in contribution styles and identification of contributions in terms of task-focused behavior, group-support behavior, and nonproductive behavior. The tasks in the project were designed under these two domains, and the assessment criteria were aligned accordingly.

Response Process

The project guidelines, as outlined in Appendices A and B, provide all the relevant information for the students, such as aims, task description, report requirements, and assessment criteria. An online discussion board is managed by the project design team; there, students post their queries and discuss the project. Students develop a draft of their report, make a presentation to the wider scenario group including their facilitator, and receive feedback. They incorporate this feedback into the final version of their group report, which is submitted electronically for examination.

Consequence

This assessment provides evidence of development in capabilities, which collectively form the basis for progression within the program. In the UNSW medicine program, this longitudinal evidence is accumulated through a portfolio, but there is scope to adapt this to other types of curricula. An important principle is that all capabilities are considered equally important and that poor performance in any capability can have an impact on progress. For example, poor performance in teamwork cannot be compensated for by good performance in other capabilities. If students receive poor grades in this assessment, they have the opportunity to repeat the assessment in future courses. They are supported to remediate any shortcomings through detailed feedback, and they also have the opportunity to work with different peers in future courses. This enables them to develop a range of perspectives on self-directed and collaborative learning. Examiner feedback on the student reports typically includes constructive comments to improve future work and highlights the strength of the approach taken by the student. Some examples are provided below.

Many tutors commented on the strengths of the approaches taken by students:

“You have made a meaningful effort to evaluate your teamwork. You have objectively reflected on strengths and weaknesses of the team. The ability to identify specific practical issues (such as the imbalance in workload consequent to the VARK approach) is commended. There is evidence of collaborative learning efforts both within the project group and the wider scenario group. Please continue to build on this great start.” (Tutor feedback, Project 2: Health Maintenance course A 2014)

“A wide range of peer-teaching methods have been used. There is evidence of consulting with SG members to ensure that the selected methods are suitable for the ‘audience’. There is very good discussion of the factors that were considered when selecting appropriate peer-teaching methods. Feedback has been sought and reflected upon. A range of methods have been used to evaluate the effectiveness of the peer-teaching methods. The attempt to differentiate between learners’ perceptions of ‘usefulness’ and ‘enjoyability’ is highly commended.” (Tutor feedback, Project 2: Health Maintenance course A 2014)

Some tutors commented on average performances and provided constructive developmental feedback:

“There is a fair attempt to evaluate the effectiveness of the collaborative process. The roles of each project group member have been analysed using an appropriate theoretical framework. The identification of strengths and areas for improvement is acceptable. However, this discussion lacks depth. You need to demonstrate that you actually learnt some strategies that could be applied to your future learning and practice.” (Tutor feedback, Project 1: Beginnings, Growth, and Development course B 2009)

Some tutors offered feedback on self-directed learning and critical evaluation and included comments on effective approaches taken by students:

“Your reflections and discussion of integrated nature of the learning issues and how they fit over more than one capability are very well thought through. It is good to also see that you understand the benefits and practical limitations of such an approach. This suggests a deep level engagement with this capability.” (Tutor feedback, Project 1: Beginnings, Growth, and Development course B 2009)

Tutors also picked up on areas requiring further attention and provided constructive suggestions to address these:

“The reference list does not include any references that relate to the content of the learning issues. Given that self-directed learning of specific content was a focus of this assignment, you should have given this much more attention. Please note that when you include scientific content in a report, you must cite the relevant source.” (Tutor feedback, Project 2: Health Maintenance course A 2014)

“Once an issue has been raised, please try to clarify what the question or gap in knowledge is. This is an important skill to develop and use during your future scenario debrief activities. Rather than record the issue verbatim, try to clarify what the question or issue actually is.” (Tutor feedback, Project 1: Beginnings, Growth, and Development course B 2009)

The quality of the marking process, including the quality of the feedback provided, is crucial to the success of the projects. Assessors need to be trained and require a good understanding of skills relating to teamwork, self-directed learning, and critical evaluation. To further strengthen this, clear assessment criteria are provided to the students, and a clear marking guide is provided to the assessors. A sample marking guide is included at the end of each project description in Appendices A and B.

A limitation of this type of assessment is the time required for peer teaching activities in the wider scenario group sessions. Students need to schedule these activities themselves, and some find this challenging. This limitation could be overcome by scheduling time slots within the scenario group tutorials, but this would be at the cost of other scheduled activities. While allowing students to be responsible for scheduling can be challenging, this can also be seen as an important skill in self-direction.

Discussion

The group projects in the medicine program provide an opportunity for students to develop self-directed and collaborative learning skills. The learning of these skills is embedded in learning tasks within realistic health scenarios: Students work together to learn and apply the relevant scientific and medical concepts. This is a unique feature of these teamwork group projects because students simultaneously develop these important learning skills while integrating and applying relevant content knowledge. The project design incorporates some of the best evidence in medical education to promote integration and application of content, using strategies such as designing concept maps and creating alternate scenarios.

The teamwork group projects could be used to complement learning activities across health care professions in undergraduate curricula and postgraduate settings. Educators are encouraged to use the projects in their current form or to adapt the projects to suit their own context.

A. First-Year Group Project - Student & Instructor Guidelines.docx

B. Mixed-Year Group Project - Student & Instructor Guidelines.docx

C. Sample Case Scenario.docx

All appendices are peer reviewed as integral parts of the Original Publication.