Medical Student Portfolios: A Systematic Scoping Review.

Phenomenon: Medical Student Portfolios (MSP)s allow medical students to reflect and better appreciate their clinical, research and academic experiences which promotes their individual personal and professional development. However, differences in adoption rate, content design and practice setting create significant variability in their employ. With MSPs increasingly used to evaluate professional competencies and the student's professional identity formation (PIF), this has become an area of concern. Approach: We adopt Krishna's Systematic Evidence-Based Approach to carry out a Systematic Scoping Review (SSR in SEBA) on MSPs. The structured search process of six databases, concurrent use of thematic and content analysis in the Split Approach and comparisons of the themes and categories with the tabulated summaries of included articles in the Jigsaw Perspective and Funnelling Process offers enhanced transparency and reproducibility to this review. Findings: The research team retrieved 14501 abstracts, reviewed 779 full-text articles and included 96 articles. Similarities between the themes, categories and tabulated summaries allowed the identification of the following funnelled domains: Purpose of MSPs, Content and structure of MSPs, Strengths and limitations of MSPs, Methods to improve MSPs, and Use of E-portfolios. Insights: Variability in the employ of MSPs arise as a result of a failure to recognise its different roles and uses. Here we propose additional roles of MSPs, in particular, building on a consistent set of content materials and assessments of milestones called micro-competencies. Whislt generalised micro-competencies assess achievement of general milestones expected of all medical students, personalised micro-competencies record attainment of particular skills, knowledge and attitudes balanced against the medical student's abilities, context and needs. This combination of micro-competencies in a consistent framework promises a holistic, authentic and longitudinal perspective of the medical student's development and maturing PIF.

Introduction

At a time when medical education is embracing a more personalised approach to knowledge attainment, skills training and development of professional behaviours, portfolios promise a means for medical students to better understand, reflect upon and actively shape their learning and development . Complementing traditional assessment methods with wider longitudinal appraisals of an individual’s growth, portfolios add a personalised dimension to logbooks[4,5], by serving as a repository for written examinations, tutor-rating reports and bedside assessments as well as individual reflections and analyses.

Indeed, portfolios offer medical students “a self-regulated, cyclical process in which [they may] mentally revisit their actions, analyse them, cogitate alternatives, [and] try out alternatives in practice” . It is this platform to showcase individual educational, research, ethical, personal and professional development[1,8], and guide specific, holistic and timely feedback and remediation throughout the individual’s medical education that underscores growing interest in portfolio use among medical students (henceforth medical student portfolios or MSPs)[4,12]. However, despite their growing traction , MSPs show significant variability in their structure and content. With local, practical, sociocultural, educational and healthcare considerations prioritising different types of data, the role of MSPs remains limited.

Need for the Review

With MSPs representing a sustainable and effective educational undertaking that provides insight into the medical student’s development, needs, values and beliefs that may guide their professional identity formation (PIF), better understanding of the principles behind their use, the key elements within them and a framework for consistent utilisation is required.

Methods

To determine what is known about MSPs, a systematic scoping review (SSR) is proposed to study current literature to enhance understanding of their roles and structure. These insights will also help guide the design of a consistent framework for MSPs to be used across different settings, purposes and specialities given their ability to evaluate data from “various methodological and epistemological traditions” .

To overcome SSR’s variable methodological steps, guidance and standards, this review adopts the Systematic Evidence Based Approach (SEBA) . A SEBA guided SSR (henceforth SSR in SEBA) facilitates the synthesis of an evidence-based, accountable, transparent, and reproducible analysis and discussion.

Steering this process and boosting accountability, oversight, and transparency, this SSR in SEBA sees an expert team involved in all stages of this review. The expert team comprised of medical librarians, local educational experts, and clinicians.

SSRs in SEBA are built on a constructivist perspective acknowledging the personalised, reflective, and experiential aspect of medical education and recognising the influence of particular clinical, academic, personal, research, professional, ethical, psychosocial, emotional, legal and educational factors upon the medical student’s learning journey, professional development and personal growth .

To operationalise the SSR in SEBA, the research team adopted the principles of interpretivist analysis to enhance reflexivity and discussions[18,32] in the six stages outlined in .

(Insert Figure 1. The SEBA Process)

Figure 1.

The SEBA process.

Stage 1 of SEBA: Systematic Approach

1. Determining the title and background of the review

The expert and research teams determined the overall goals of the SSR and the population, context and concept to be evaluated.

2. Identifying the research question

Guided by the PCC (population, concept and context), the expert and research teams agreed upon the research questions. The primary research question was “what is known about medical student portfolios?”. The secondary questions were “what are the components of MSPs?”, “how are MSPs implemented?” and “what are the strengths and weaknesses of MSPs?”.

3. Inclusion criteria

All peer reviewed articles, reviews and grey literature published from first January 2000 to 31st June 2021 were included in the PCC and a PICOS format was adopted to guide the research processes[35,36]. The PICOS format is found in .

4. Searching

A search on six bibliographic databases (PubMed, Embase, PsycINFO, ERIC, Google Scholar and Scopus) was carried out between first to 10th September 2021. Limiting the inclusion criteria was in keeping with Pham et al’s (2014) approach to ensuring a sustainable research process . The search process adopted was structured along the processes set out by systematic reviews.

5. Extracting and charting

Using an abstract screening tool, members of the research team independently reviewed the titles and abstracts identified by each database to identify the final list of articles to be reviewed. Sambunjak et al’s (2010) approach to ‘negotiated consensual validation’ was used to achieve consensus on the final list of articles to be included . The six members of the research team independently reviewed all the articles on the final list, used the Medical Education Research Study Quality Instrument (MERSQI) and the Consolidated Criteria for Reporting Qualitative Studies (COREQ) , discussed them online and were in consensus that none should be excluded (Supplementary File 1).

Stage 2 of SEBA: Split Approach

Three teams of researchers simultaneously and independently reviewed the included full-text articles. Here, the combination of independent reviews by the various members of the research teams using two different methods of analysis provided triangulation , while detailing the analytical process improved audits and enhanced the authenticity of the research .

The first team summarised and tabulated the included full-text articles in keeping with recommendations drawn from Wong et al’s (2013) “RAMESES publication standards: meta-narrative reviews” and Popay et al’s (2006) “Guidance on the conduct of narrative synthesis in systematic reviews” . The tabulated summaries served to ensure that key aspects of the included articles were not lost (Supplementary File 1).

Concurrently, the second team of three trained reviewers analysed the included articles using Braun & Clarke’s (2006) approach to thematic analysis . In phase one, the research team carried out independent reviews, actively reading the included articles to find meaning and patterns in the data. In phase two, ‘codes’ were constructed from the ‘surface’ meaning and collated into a code book to code and analyse the rest of the articles using an iterative step-by-step process. As new codes emerged, these were associated with previous codes and concepts. In phase three, the categories were organised into themes that best depict the data. An inductive approach allowed themes to be “defined from the raw data without any predetermined classification”. In phase four, the themes were refined to best represent the whole data set. In phase five, the research team discussed the results of their independent analysis online and at reviewer meetings. ‘Negotiated consensual validation’ was used to determine a final list of themes.

A third team of three trained researchers employed Hsieh & Shannon’s approach to directed content analysis and independently analysed the included articles . This analysis using involved “identifying and operationalising a priori coding categories”. The first stage saw the research team draw categories from Davis et al.’s (2001) “AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment” to guide the coding of the articles. Data not captured by these codes were assigned a new code in keeping with deductive category application. Categories were reviewed and revised as required. In the third stage, they discussed their findings online to achieve consensus on the final codes. These final codes were compared and discussed with the final author.

Stage 3 of SEBA: Jigsaw Perspective

As part of the reiterative process, the themes and categories identified were discussed with the expert team. Here, the themes and categories were viewed as pieces of a jigsaw puzzle and areas of overlap allowed these pieces to be combined to create a wider/holistic view of the overlying data. The combined themes and categories are referred to as themes/categories.

Creating themes/categories relied on use of Phases 4 to 6 of France et al.’s (2016) adaptation of Noblit and Hare's (1998) seven phases of meta-ethnography . To begin, the themes and categories were contextualised by reviewing them against the primary codes and subcategories and/or subthemes they were drawn from. Reciprocal translation was used to determine if the themes and categories could be used interchangeably.

Stage 4 of SEBA: Funnelling Process

To provide structure to the Funnelling Process, we employed Phases 3 to 5 of the adaptation. We described the nature, main findings, and conclusions of the articles. These descriptions were compared with the tabulated summaries. Adapting Phase 5, reciprocal translation was used to juxtapose the themes/categories identified in the Jigsaw Perspective with the key messages identified in the summaries. These verified themes/categories then form the line of argument in the discussion synthesis.

Results

A total of 14501 abstracts were reviewed, 779 full text articles were evaluated, and 96 articles were included (see .). The funnelled domains identified were: Purpose of MSPs, Content and structure of MSPs, Strengths and limitations of MSPs, Methods to improve MSPs, and Use of E-portfolios.

PRISMA flow chart.

Funnelled Domain 1: Purpose of MSPs

The purpose behind the employ of MSPs are often poorly explained and have been summarised in for ease of review.

Links practical experience with pre-existing medical knowledge

Collection and selection of work samples to allow for reflection and analysis of learning

Provides a platform to express inner dialogue

Individualise learning goals and plans

Encouraged by clinical components of portfolio

A combination of formative and summative assessment[1,47,53,106] establishes portfolios as a “very powerful assessment tool”

Students assess their own learning[5,7,126], strengths and weaknesses and performance[47,72,111]

Funnelled Domain 2: Content and structure of MSPs

1. Content in MSPs

Similarly, discussions on the contents of MSPs are limited and have been summarised in Table 3. The content can be broadly categorised into content provided by the institution, medical students, and feedback/assessments by other stakeholders.

Table 3.

Content in MSPs.

CONTENT	ELABORATION AND/OR EXAMPLES
Contributed by institution
Learning objectives	Institutions list out clear learning objectives that students can refer to as a guide for their learning[5,7,47,54,57,67,76,79,81,102,108,111,121,124,140] Some institutions refer to professional accreditation guidelines[5,7,67,76,79,82,102,108,111] For example, several institutions have made use of the Canadian CanMEDS framework[79,82]. Other institutions utilised descriptions of professional roles to substitute learning objectives[54,57] as these are easier to comprehend 54 . For example, the university of Maastricht requested for its students to include evidence within their portfolio regarding their role as a ‘researcher’, ‘healthcare worker’, ‘medical expert’ and ‘person’. 54
Educational Resources	Web links 72 Graphics and streamed videos 72 Checklists to highlight OSCE steps 72 Training package on specific topics 72 Reflection writing framework[68,129] E-Learning cases 82
Reflective prompts	Questions to stimulate student reflection[3,7,54,64,65,68,69,86,103,108,115,127,144].
Contributed by medical student
Evidence of Activities	Curriculum Vitae[54,138] Research projects[47,55,67,80,145] Elective reports[1,91] Presentations[67,80,112] Personal achievements[63,68,129] Membership in professional societies 145 Extracurricular activities 80 Evidence of learning activities Learning diaries[53,65,72,144] Case summaries, reports, discussions[11,47,67,128,131,136,146] Logbooks[55,66,70,80,83,91,115] Essays to document progress in meeting competency standards[47,70,105] Group Learning Assignments[130,132] Graded assignments Workplace Based Assessments Mini CEX[59,67,75,131] Direct observations[59,75,85] Multi-source feedback (MSF) assessments[59,75,99,131] Case based discussions[59,75,85] Patient write-ups 67 Summative assignment and assessment grades[59,67,80,85,91] Critical appraisals of a topic 131 Standardised patient assessments 67 Evidence based medicine project 67 Posting learning outcome grades 1 Progress test results 59 Anatomy lab 134 Small group assessments showcasing student’s teamwork skills 134 Longitudinal clinical preceptorships 134
Evidence of reflection	Written reflections from students[1,3,6,7,11,53,54,57,59,62,63,65,67,69,72,75,86,87,91,103,106,108,111,115,116,131,135,147] Topics: ▪ Professional development/skills acquisition[6,54,57] ▪ Plans for future self-development/improvement[54,57,62,131] ▪ Personal learning goals[1,53,57,62,72,116] Content: ▪ Patient encounters[1,86,131] ▪ Short summaries of patients seen by the student and reflections on what they had learned in the process 1 ▪ Learning activities[53,108] ▪ Activities may be those conducted internally or extra-curricular activities 108
Evidence of self-assessment	Performance in competencies[59,70,72,80,111,115,123] and roles 7 Personal strengths and weaknesses 111 Personal learning[61,115] and growth 47 Professionalism 47
Contributed by other stakeholders (eg assessors, peers)
Assessments	Assessors Tutors[53,55,59,60,85,111,116,135,139,146] Faculty[70,122,138] Peer assessors[72,111,131,134] Patients 72 Examiners from courses taken in other faculties 63 Domains Clinical skills/competencies[1,3,6,47,59,63,108,112,115,128,129,131,133,139] Communication skills[111,115,117,139] Behavioural competencies 142 Authentic learning, referring to the learning of practical knowledge[54,128,148] Personal and professional development[1,47,55,67,70,91,131]

Other institutions utilised descriptions of professional roles to substitute learning objectives[54,57] as these are easier to comprehend . For example, the university of Maastricht requested for its students to include evidence within their portfolio regarding their role as a ‘researcher’, ‘healthcare worker’, ‘medical expert’ and ‘person’.

Web links

Graphics and streamed videos

Checklists to highlight OSCE steps

Training package on specific topics

E-Learning cases

Membership in professional societies

Extracurricular activities

Patient write-ups

Critical appraisals of a topic

Standardised patient assessments

Evidence based medicine project

Posting learning outcome grades

Progress test results

Anatomy lab

Small group assessments showcasing student’s teamwork skills

Longitudinal clinical preceptorships

Short summaries of patients seen by the student and reflections on what they had learned in the process

Activities may be those conducted internally or extra-curricular activities

Performance in competencies[59,70,72,80,111,115,123] and roles

Personal strengths and weaknesses

Personal learning[61,115] and growth

Professionalism

Patients

Examiners from courses taken in other faculties

Behavioural competencies

2. Structure of MSPs

Standardisation within and across portfolios may be achieved through the use of a clear template or set of guidelines . MSPs with clear delineation of contents required were found to boost student receptivity[55,56] and enhanced reliability and validity during portfolio assessment[47,55,57].

However, a flexible approach allowing medical students to personalise their MSPs and express themselves more freely facilitates portfolio student-centricity[60,61] and ownership . By encouraging students to incorporate their own content, such as reflective diary entries , reflective essays , video recordings , audio recordings , poetry or art , improvements may be seen in the quantity and quality of their reflections .

Funnelled Domain 3: Strengths and Limitations of MSPs

Given the lack of elaboration, much of the data for this domain is summarised in tables to aid easy review.

1. Strengths

Strengths of MSPs are highlighted in .

“Act of logging ‘learning moments’ helped facilitate memorisation”

May improve performance in other knowledge-based assessments

Problem solving

Critical thinking

Portfolio assessment can cater to a range of learning styles because it can be easily personalised based on the student .

Unique evidence may be selected to express their capabilities to examiners .

“Combines information from both subjective and objective assessment procedures ‘to see the whole picture’”

Students will be spurred on to improve themselves should they receive negative feedback

Better demonstrates achievement in competencies such as professionalism, teamwork, and communication skills

Encourages students to discuss their private thoughts

Easily transferable when needed in the future to facilitate job applications[103,104] or acquisition of letters of recommendation for future training

Helps to ease transition to postgraduate educational practice as portfolios and portfolio assessment are often utilised at postgraduate level

Better understand students’ thinking and attitudes

Identifies gaps in the curriculum[56,101] such as through providing opportunities for students to evaluate teaching activities

Helping students to develop better rapport with others including patients[62,118,122], clinical teams and other students

2. Limitations

The limitations of MSPs are highlighted in .

Limited use for theoretical knowledge

Overly prescriptive structure of reflective prompts may hinder reflective process

Provide only vignettes of a student’s journey , and students may hide evidence of their weaknesses[54,59,63,70,104,126], fail to express their authentic views or even fabricate reflections

Students may create their portfolios differently based on their own interpretation of the purpose of the portfolio

Student’s portfolios may unknowingly be judged on irrelevant aspects such as layout and format

This may be amplified if student identity is not anonymised to examiners evaluating the portfolios

There may be insufficient time for comprehensive assessments in the clinical setting as taking time to assess students must be balanced with providing quality patient care

Poorly standardised assessment procedure leads to poor consensus among assessors

Lack of training for assessors limits the use of work-based assessments within portfolios for assessing student competence

Students prioritise coursework that contributes towards their final examination marks

Interference with other studies , including clinical learning and time that should be spent with patients or studying for exams

Unaware of how portfolios can be integrated into their education

Stressful and difficult to fill out[61,78]

Worried about the negative comments they could receive from their mentors

Felt the time given to complete their portfolios was too short, leading to reduced value

Poor time management

Did not engage in reflection personally

Difficulty finding methods to help students

Poor relationship with student

Funnelled Domain 4: Methods to Improve MSPs

The potential methods to improve MSPs are highlighted in .

Train mentors[66,78,87,123] and utilise verified teaching methods that foster reflection and ensure mentors are able to stretch their students in their reflective practice

Willing to engage students

Understands reflection and their responsibility to teach students how to utilise reflections purposefully

Able to build trust and rapport with students

Some institutions encourage frequent weekly meetings with mentees , while others believe that mentorship can occur as infrequently as two to three times a year[4,57,64]

Students with a better understanding of portfolio usage had more positive attitudes towards portfolios

Embed portfolio into the curriculum[54,64,72,104] and encourage faculty and department staff to reference it in daily practice

Organise the portfolio based on its purpose .

For a portfolio focused on enhancing learning, the portfolio should include more self-reflection[54,56] and reasoned tasks that demonstrate student learning .

For a portfolio meant for assessment, content should mainly compose of evidence that competencies have been achieved and prompts should be minimal as the student's choice of reflection is also important in assessment

Increase emphasis on writing reflections rather than describing activities

Refrain from limiting word count

Utilise innovative tools such as the visual analogue scale or audio recordings

Portfolios should also be organised to facilitate effective teaching by faculty

Focus assessment on promoting student development through providing useful feedback[121,124]

Ensure assessment does not compromise reflection

Assess students based on the authenticity of their reflections

Institute a central committee to review assessments and ensure ample learning experiences and assessment evidence exist to guide student learning

Prepare students adequately for the assessment[91,105,116,131] by providing guidelines on the purpose and format of the assessment , clarifying expectations , providing guidance from trained portfolio advisors[105,131].

Ensure assessment occurs immediately after a clinical experience

Increase number of assessment points such as by adopting more work-based assessments within the portfolio

Create and validate clear rubrics to assist assessors in their grading of students

Encourage students to include evidence to support their self-assessments to reduce inaccurate self-assessments

Allows for evaluation and alignment of portfolio with teaching, learning and assessment data

Funnelled Domain 5: E-Portfolio

The electronic portfolio (e-portfolio) is a form of MSP that is hosted on electronic platforms[5,6,9,47,53,56,58,61,63], and may be created using unique software[47,63,65,76,86]. Compared to hardcopy portfolios, they are more durable , user friendly[63,75,77], accessible[6,53,58,61,80] collaborative[5,67,73,76,81] and superior for assessment in certain areas . Furthermore, they are able to include a wider variety of evidence including videos or website links[5,63,75,78,79], provide increased privacy and confidentiality for users including students and coaches[67,73,86] and allow for instant comparison between students . These factors enhance their receptivity among medical students[53,61,63].

However, accessibility may be limited by poor interface design[64,67,73,74,77,87,88], limited administrative support[67,73,88], poor technology[66,67,73,79], and a lack of time or finances to upgrade and support e-portfolio technology . Similarly, the lack of immediate access to computers in a clinical setting[58,66,73], poor data security[58,65,66], issues with communicating with mentors online or mentors not being tech-savvy also limit their applicability.

Stage 5 of SEBA: Analysis of Evidence-Based and Non-Data Driven Literature

Evidence-based data from bibliographic databases were separated from grey literature such as opinion pieces, perspectives, editorial, letters and non-data based articles drawn from bibliographic databases and both groups were thematically analysed separately. The themes from both groups were compared to determine if there were additional themes in the non-data driven sources that could influence the narrative. In this review, the themes from the two data sources overlap, suggesting no undue influence upon the findings of this review.

Stage 6 of SEBA: Synthesis of SSR in SEBA

The narrative produced from consolidation of the funnelled domains was guided by the Best Evidence Medical Education (BEME) Collaboration guide and the STORIES (Structured approach to the Reporting In healthcare education of Evidence Synthesis) statement .

Discussion

In answering its primary and secondary research questions, this SSR in SEBA reveals that MSPs have expanded beyond merely repositories of assessments and are now seen as a means of triangulating and contextualising assessments and their impact upon individual medical students. MSPs also allow students, faculty, and institutions to better understand the medical student’s needs, abilities, expectations, and aspirations, aiding the provision of personalised mentoring and remediation. However, to meet these wider roles, manageable and “authentic” portfolios that improve levels of engagement are key. Here, authenticity refers to the “extent to which the outcomes measured represent appropriate, meaningful, significant and worthwhile forms of human accomplishments” and serves to enhance the trustworthiness of what is largely qualitative data, and the validity of longitudinal assessments that help to map the development of their clinical competency and professional identity formation[4,12,92].

However, current MSPs lack a consistent structure. While broad commonalities including learning objectives and professional expectations and roles to be met, and reflections, learning activities, self-assessments, achievements, and other evidence of competencies, MSPs vary significantly in their focus and content. Yet, these variations and particularities are unsurprising given the different practice settings, structure and program goals established by the host institution. These differences underpin the presence of different types, “depth” and nature of content prioritised. Inherent variability brought about by personalisation of longitudinal data, “choice of materials by the student” and “individualised selection of evidence” , ultimately limits the use of portfolios beyond the confines of a specific institution. This lack of consistency raises concerns about the efficacy of MSPs in providing a holistic perspective of the medical student’s personal, academic, clinical, and professional development.

We believe that these concerns may be bridged in part by harnessing the ability of current MSPs to capture education and assessment in specific areas of practice. Our findings suggest that current MSPs encapsulate several entrustable professional activities (EPA)s . Each EPA however shares common aspects of other EPAs that may not be directly contained within a particular MSP. We believe that it is possible to harness these overlapping aspects to make MSPs more widely applicable. Here, we build upon the notion that micro-credentialling that incorporates “circumscribed assessments” of a specific EPA, such as “interpreting and communicating results of common diagnostic and screening tests”, may be extrapolated to other EPAs such as “[communicating] in difficult situations” in a different practice setting .

Hong et al’s (2021) and Zhou et al’s (2021) adaptations[98,99] of Norcini’s (2020) concept of micro-credentialling and micro-certification in medical education which forward the concepts of generalised and personalised micro-competencies provide a viable bridge between prevailing MSP content without compromising the rich mix of structure and customisation within MSPs. Based on the certification of micro-competencies within an EPA, Zhou et al. (2021) suggest that generalised micro-competencies are the standards and expectations applicable to all medical students. They are small, professional learning milestones that all students need to attain before proceeding to the next competency-based stage. These are requisite knowledge, skills and attitudes all soon-to-be clinicians must have. Personalised micro-competencies, in turn, are determined by the individual’s particular goals, training, abilities, skills and experiences. They are determined by the medical student and tutors and must be consistent with institutional codes of conduct and expectations. They underscore the importance of assessing the student's individual needs and circumstances which influence which in turn shape the kind of training and support proffered. With expectations differing across practice settings and levels of training, both generalised andpersonalised micro-competencies must be clearly conveyed to the medical student and tutors in a timelyand structured manner. To encapture their learning and attainment, MSPs must forward clear learning plans to align expectations with evidence of diverse learning activities, reflective prompts and diaries, multisource formative and summative evaluations via standardised assessment tools and constructive feedback. These standardised baseline guidelines will lend clarity to portfolio developers and users. This may boost the latter’s trust and receptivity towards regular portfolio use[55,56].

We believe that structured and consistent micro-certification of micro-competencies could be extrapolated beyond the initial goals of the MSPs and could provide a longitudinal perspective of the medical student’s development. This is especially useful when considering competencies such as interpersonal, communication skills and systems-based practices. Perhaps here, too, the silver lining to changes in medical education practices due to the COVID-19 pandemic can be harnessed.

With many institutions incorporating online learning, e-portfolios should be institutionally sanctioned with a dedicated team of portfolio developers and invested faculty members onboarding and overseeing their implementation. These considerations foreground the need for orientation sessions[10,62,64,67,104] to educate students and faculty on the identified EPAs as well as the use of generalised and personalised micro-competencies to ensure learning and assessment congruity and objectivity[91,105,106]. Embedding the portfolios into the formal curricula, assigning students mentors trained in reflective engagement, and establishing protected time for regular portfolio reviews would help to facilitate their consistent usage. Concurrently, portfolio use must be part of a continuous quality improvement process, building on feedback and lessons learnt to promote further improvement to MSPs and portfolio assessment[10,11,47,62,78]. Indeed, both forms of micro-competencies underline the need for effective recording and oversight. This is especially important when micro-competencies provide a holistic appraisal of the medical student’s progress and achievements, needs and abilities and provides insights into their professional identity formation. Capturing this data in a comprehensive, longitudinal manner replete with the medical student’s reflections reveals a new dimension to portfolio use.

Limitations

Firstly, the review is limited by the omission of articles not published in English. This creates the risk of missing key papers. Furthermore, the focus on papers published in English led to focus on studies in North America and Europe.

Secondly, while the articles comment on the sentiment of users including medical students on the effectiveness of portfolios for learning and assessment, there are a limited number of articles highlighting the perspectives of doctors who previously undertook the task of undergraduate portfolios. Hence, the review is limited by its inability to assess the long-term effectiveness and acceptability of portfolio usage after medical students enter the workforce as practicing medical professionals.

Conclusion

This SSR in SEBA reveals that if portfolios are to remain relevant and maintain their user-friendliness and accessibility, the future of MSPs must lie in improving assessments and in enhancing the manner in which they are designed.

While it is clear that assessments tools need to be enhanced to meet new perspectives of education and training, it is perhaps timely that this SSR in SEBA suggests key changes to portfolio use. In adopting e-portfolios for its accessible and expansive potential, it is clear that a robust and well-supported platform is critical. This platform ought to accommodate all manner of data and assessment results and remain a comprehensive repository of data. Categorised into different, sometimes overlapping, domains, data from this repository may be drawn to populate different designs of MSPs. Changing from one goal to another should therefore be simple. Such flexibility will still allow medical students to personalise their e-portfolios in a manner that they feel best represents their development without compromising faculty evaluation. A flexible yet robust e-portfolio such as this will also enable collaborations and facilitate input of corroborative data from third parties where required.

Moving forward, further research may be undertaken to identify the long-term effects of portfolio usage, the manner that portfolios are evaluated, and the impact it has on professional identity formation throughout and beyond medical school.

Table 1.

PICOS, inclusion and exclusion criteria.

PICOS	INCLUSION CRITERIA	EXCLUSION CRITERIA
Population	Undergraduate and postgraduate medical students	Allied health specialties such as Pharmacy, Dietetics, Chiropractic, Midwifery, Podiatry, Speech Therapy, Occupational and Physiotherapy Non-medical specialties such as Clinical and Translational Science, Alternative and Traditional Medicine, Veterinary, Dentistry
Intervention	The use of portfolios for medical students
Comparison	Comparison of the various use of portfolios (approaches, modalities, processes, objectives, motivations, challenges, facilitating characteristics/resources)
Outcome	Approaches, modalities, processes, objectives, motivations, challenges, facilitating characteristics/resources in the current and potential uses of portfolios Impact of the use of portfolios on medical students
Study design	Articles in English or translated to English Grey literature, case reports and series, ideas, editorials and commentaries Electronic and print information not controlled by commercial publishing All study designs including: Mixed methods research, meta-analyses, systematic reviews, randomised controlled trials, cohort studies, case-control studies, cross-sectional studies, descriptive papers Date of Publication: Jan 2000 – June 2021

Table 2.

Purpose of MSPs.

CONTENT	ELABORATION AND/OR EXAMPLES
Learning	Reflective learning:[7,11,47,53,54,56,57,61,62,64,66,68,69,72,74,75,79,80,86,87,91,101,108] Links practical experience with pre-existing medical knowledge 108 Collection and selection of work samples to allow for reflection and analysis of learning 125 Provides a platform to express inner dialogue 7 Self-directed learning[7,11,53,54,62,64,65,67,71,73,91,101,102,105,109,115,117,126] Identify personal learning needs[11,115] Individualise learning goals and plans 110 Workplace-based learning[66,115,131] Encouraged by clinical components of portfolio 66 Group learning[130,132]
Assessment	Formative Assessment[1,3,5,7,47,60,61,68,75,76,78,84,85,101,106,116,117,124,133] Platform to receive constructive feedback[60,68,116,134] Summative Assessment[1,5,13,47,54,59,68,70,71,78,80,82,102,105,106,116,121,123,124,130,135,136,139] Ensure students have met curriculum learning objectives by quantifying their performance through grades or numerical marks[5,13,47,68] Results are utilised to inform decisions on promotion, graduation and licensing[47,54,62,68,71,106,139] A combination of formative and summative assessment[1,47,53,106] establishes portfolios as a “very powerful assessment tool” 47 Self-Assessment[1,5,7,47,54,62,66,67,71,72,91,110,111,115,126,131,135] Students assess their own learning[5,7,126], strengths and weaknesses 54 and performance[47,72,111] Encourages positive changes in behaviour[7,62,127] Modalities include interviews to discuss portfolio content between students and assessors[1,4,11,57,68,108,123,131,136] or portfolio review by assessors[1,4,8,11,54,59,70,75,91,105,111,112,116,117,123,131,134,140]

Table 4.

Strengths of MSPs.

STRENGTHS	ELABORATION AND/OR EXAMPLES
Learning	Highlights important skills and competencies Allows medical educators to reshape and redefine core concepts of medical practice through the development of portfolio criteria Streamlines learning and teaching focused on important competencies[4,11,53,72,80,115,123,124,133] Stimulates learning[5,11,74,77,102,109,118,130,132,135] Feedback provided highlights potential areas for improvement[5,6,9,66] “Act of logging ‘learning moments’ helped facilitate memorisation” 109 May improve performance in other knowledge-based assessments 132 Promotes development of important skills Problem solving 132 Communication[56,63,105,111,115,121,128,131,142] Ethical and legal responsibility[7,53,87,149] Professional development[5,8,11,47,53,56,63,74,78,86,103,105,111,116,123,131,135,141,142] Teamwork[63,87,111,130,132,135,142] Critical thinking 121
Assessment	Examiners and faculty generally accept portfolios[6,60,65,74,114,116,150] and their assessment methods[1,6,116,135,140] as they are: Individualised[47,55,63,117] Portfolio assessment can cater to a range of learning styles 117 because it can be easily personalised based on the student 55 . Unique evidence may be selected to express their capabilities to examiners 63 . Comprehensive[1,54,61,70,83,117,123,126,135,137,140] “Combines information from both subjective and objective assessment procedures ‘to see the whole picture’” 140 Able to evaluate competencies that are otherwise not easily assessed[1,54,83] such as professionalism[123,137] Longitudinal[1,47,67,74,80,99,117,133,141] Portfolios are assembled over a period of time and hence can be used to monitor student’s progress over the period of compilation Educational Use in assessment has helped stimulate learning[1,66,73,74] Guides tailored teaching by faculty members[54,91,126,133,134] Guides remediation plans for underperforming students[1,62,91,105,111,116,135,140,142] Specific to summative portfolio assessment: Ensures that students take the portfolio exercise seriously[57,114] Students will be spurred on to improve themselves should they receive negative feedback 75 Better demonstrates achievement in competencies such as professionalism, teamwork, and communication skills 111 Specific to formative portfolio assessment: Enables constant improvement through feedback and reflection[6,7,60,71,75,105,116,127,133,140] Fosters self-motivation[5,69] and intrinsic motivation to reflect[91,115].
Others	Encourages students to discuss their private thoughts 103 Prepares students for postgraduate work Easily transferable when needed in the future 80 to facilitate job applications[103,104] or acquisition of letters of recommendation for future training 80 Helps to ease transition to postgraduate educational practice 74 as portfolios and portfolio assessment are often utilised at postgraduate level 55 Improves teaching within undergraduate programs Improves faculty’s understanding of students ▪ Better understand students’ thinking and attitudes 65 ▪ Directs discussion during meetings with advisees[65,74] Identifies gaps in the curriculum[56,101] such as through providing opportunities for students to evaluate teaching activities 56 Helping students to develop better rapport with others including patients[62,118,122], clinical teams 62 and other students 132

Table 5.

Limitations of MSPs.

LIMITATIONS	ELABORATION AND/OR EXAMPLES
Learning	Limited use for theoretical knowledge 121 Limited use for reflective learning Does not guarantee that reflection will take place[7,54,56,64,78,87,103] Students are sceptical about the reflective process[53,67,68,87,110] Challenging for individuals who are not intuitively reflective[64,72] Overly prescriptive structure of reflective prompts may hinder reflective process 64
Assessment	Limited reliability and validity[4,54,55,59,62,63,71,72,91,108,111,112,117,135,137] Inauthentic ▪ Provide only vignettes of a student’s journey 59 , and students may hide evidence of their weaknesses[54,59,63,70,104,126], fail to express their authentic views 63 or even fabricate reflections 78 ▪ They may also perform poorly under stress during assessments included in their portfolios such as directly observed work-based assessments[59,137] ▪ Students tend to have a poor self-assessment capacity[72,111,151] ▪ Perceived quality of portfolio relies heavily on the individual’s reflective ability[55,105,121] which is unfavourable for students with poor reflective skills Subjective ▪ Students may create their portfolios differently based on their own interpretation of the purpose of the portfolio 59 ▪ Student’s portfolios may unknowingly be judged on irrelevant aspects such as layout and format 4 ▪ This may be amplified if student identity is not anonymised to examiners evaluating the portfolios 119 Overly structured[47,53,57,59,62,64,119] ▪ Highly structured portfolios with a rigid format can lead to students including less of their personal observations and reflections, which diminishes the portfolio’s capacity for authentic assessment of the student and their development Problematic assessment process Poor student understanding[11,53,62,63,73,104,116] Time consuming ▪ There may be insufficient time for comprehensive assessments in the clinical setting as taking time to assess students must be balanced with providing quality patient care 59 ▪ Time consuming for assessors[1,5,11,13,53,55,60,63,65,68,74,104,112,116,140] ▪ Human resource intensive[6,112,137,140] ▪ Excessive paperwork[1,55,74,106] Lack of standardisation among examiners ▪ Poorly standardised assessment procedure leads to poor consensus among assessors 117 Lack of training for assessors limits the use of work-based assessments within portfolios for assessing student competence 137
Portfolio Implementation	Negative student sentiments Resistance[5,11,53,59,61,63,66,67,74,102,104,106,126] ▪ Perceived to be redundant[61,102] and incompatible with studying format[61,77,78] Non-priority ▪ Students prioritise coursework that contributes towards their final examination marks 146 ▪ Interference with other studies 123 , including clinical learning 91 and time that should be spent with patients 1 or studying for exams 78 Poor understanding and engagement[1,4,54,61,66,74,78,108,150] ▪ Unaware of how portfolios can be integrated into their education 110 ▪ Stressful 78 and difficult to fill out[61,78] Burdensome ▪ Time consuming[11,66,79,108,115,116] ▪ Excessive paperwork[1,55,77,102,106,108] Worried about the negative comments they could receive from their mentors 61 Felt the time given to complete their portfolios was too short, leading to reduced value 123 Lack of support from mentors[64,66,110] Not all mentors provided feedback and engaged the students[64,78,103,118] Factors leading to faculty’s lack of support ▪ Poor time management 64 ▪ Failure to understand role as portfolio mentors[64,110] ▪ Did not engage in reflection personally 64 ▪ Difficulty finding methods to help students 78 ▪ Poor impression of portfolios and their role in education[66,78] ▪ Poor relationship with student 103

Table 6.

Methods to improve MSPs.

METHODS	ELABORATION AND/OR EXAMPLES
Increase MentorshipMentorship refers to a system where students are assigned to faculty throughout their training and portfolio creation to coach them[54,57,101], engage them in supportive dialogue[63,64,108,118,148], provide feedback[1,61,63,64,133] and encourage them to fully engage with their portfolios[74,78,103,131,146].
Benefits of Mentorship	Crucial to portfolio success[4,7,63,64,78,79,87,104,131] because it helps guide the students’ reflective process[57,65,131,146], enhances learning[1,57,74,135] and increases student receptivity towards their use [7,64,103]
Improving quality of mentorship	Train mentors[66,78,87,123] and utilise verified teaching methods that foster reflection 152 and ensure mentors are able to stretch their students in their reflective practice 78 Recruit good mentors Willing to engage students 108 Understands reflection 129 and their responsibility to teach students how to utilise reflections purposefully 79 Able to build trust and rapport with students 64
Having a structured mentoring programme to guide portfolio use	Some institutions encourage frequent weekly meetings with mentees 108 , while others believe that mentorship can occur as infrequently as two to three times a year[4,57,64] Keep the student to mentor ratio small such as having one-to-one interactions[6,70,79]
Encourage portfolio uptake
Improve understanding	Students with a better understanding of portfolio usage had more positive attitudes towards portfolios 108 Introduce and orientate students to the portfolio[6,54,57,61,63,73,104,108] Educate students on purpose and objectives of portfolio[62,64,67,70,101,104,123] Provide clear instructions and portfolio guidelines[7,61,63,70,73,102,104,108,114,116,118,123,143] Structure portfolios clearly[4,7,53,54,56,57,60,64,65,70,91,102,114,121,123]
Increase Exposure	Students who had been exposed to them for some time[6,91] had more positive attitudes towards portfolios. Embed portfolio into the curriculum[54,64,72,104] and encourage faculty and department staff to reference it in daily practice 77 Early portfolio introduction[54,129]
Structure portfolio appropriately
Organise portfolio based on its purpose	Organise the portfolio based on its purpose 125 . For a portfolio focused on enhancing learning, the portfolio should include more self-reflection[54,56] and reasoned tasks that demonstrate student learning 56 . For a portfolio meant for assessment, content should mainly compose of evidence that competencies have been achieved 5 and prompts should be minimal as the student's choice of reflection is also important in assessment 143 If the portfolio is meant to promote reflection, design the portfolio to ensure it is conducive for reflection Provide reflective prompts[3,7,54,64,65,68,86,108,119,127,143,144] Increase emphasis on writing reflections rather than describing activities 108 Refrain from limiting word count 62 Utilise innovative tools such as the visual analogue scale 151 or audio recordings 59 Portfolios should also be organised to facilitate effective teaching by faculty 56
Improving portfolio assessment process
Enhance learning through assessment process	Focus assessment on promoting student development 88 through providing useful feedback[121,124] Enhance reflective learning Ensure assessment does not compromise reflection 54 Assess students based on the authenticity of their reflections 53 Institute a central committee to review assessments and ensure ample learning experiences and assessment evidence exist to guide student learning 70
Standardisation	Standardisation improves the reliability of the assessment process[8,72,116,131] The following may be standardised Portfolio content[8,72,116,131] Standardising assessment criteria[1,8,47,55,72,112,116,124,131,135] including standardising portfolio interview questions[1,13]
Improve assessment procedure	Prepare students adequately for the assessment[91,105,116,131] by providing guidelines on the purpose and format of the assessment 116 , clarifying expectations 91 , providing guidance from trained portfolio advisors[105,131]. Ensure assessment occurs immediately after a clinical experience 129 Increase number of assessment points such as by adopting more work-based assessments within the portfolio 137 Reduce subjectivity of assessment Create and validate clear rubrics to assist assessors in their grading of students 121 Increase number of assessors to achieve better inter-rater reliability[62,72,112,121] Provide training to assessors[4,53,62,64,67,68,74,85,87,104,111,121,124,135] Providing opportunities for discussion or feedback between assessors[4,8,63,72,105,111,116,117,124] Introduce portfolio interviews where students can discuss and elaborate upon their portfolios personally[4,8,53,72,105,116,140] or even assess their own portfolios[5,55]
Improve self-assessment process	Encourage students to include evidence to support their self-assessments to reduce inaccurate self-assessments 111
Evaluate Feedback
Importance	Student empowerment and feedback have all been valuable tools in successful portfolios[47,53]: Allows for evaluation and alignment of portfolio with teaching, learning and assessment data 113 Help to ensure the portfolio is being used appropriately[11,68,74] Helps to introduce positive changes[11,47,62,78]