Involving medical students in service improvement: evaluation of a student-led, extracurricular, multidisciplinary quality improvement initiative.

BACKGROUND: Quality improvement (QI) is considered a duty of every doctor and, as such, it is fundamental that medical schools nurture QI skills of medical students. At a London medical school, a novel initiative was designed to involve medical students in QI. Such novel aspects include its student leadership, multidisciplinary approach and extra-curricular nature. The aim of this study is to evaluate the effectiveness of the initiative, and thus add to the experiences of existing medical student QI programs, as well as provide guidance to other institutions wishing to involve medical students in QI.
METHODS: The key features of the initiative's design is described. Its effectiveness was evaluated by the collection of retrospective data on the quality of the initiative's QI projects (QIPs), including the proportion which: 1) reached completion; 2) resulted in a significant improvement in their primary outcome; 3) had sustained results at follow-up; 4) achieved publication; and 5) contributed towards a prize or conference presentation.
RESULTS: There were 109 students involved throughout 10 projects from 14 different undergraduate and postgraduate courses from 2015-2019. 50% of the initiative's projects achieved a significant improvement in their primary outcome, and the proportion of projects which sustained these improvements at follow-up was 100%. Furthermore, 20% of projects were published, and 60% contributed towards a prize or conference presentation.
CONCLUSION: The results of this study show that the initiative was effective at involving medical students in QI. As such, other groups establishing medical student QI programs may benefit from replicating positive elements of its design and operation.

Introduction

The NHS faces an aging population with complex, long-term health conditions, in conjunction with insufficient capacity, stretched resources and financial constraint.1 Despite these challenges, it is essential that quality of services are maintained and improved.2,3 As such, service improvement is considered an essential duty of every doctor.4

Alongside doctors, other groups involved in the delivery of healthcare can influence service improvement. In particular, medical students possess attributes and are exposed to opportunities which place them in a strong position to improve the quality of care. For instance, they: regularly rotate between different institutions and specialties; have frequent contact with patients, families and front-line staff; are not yet burdened with clinical commitments; and are unbiased and fresh observers of health-care teams. As a result, they are more likely to identify outdated practice and produce innovative solutions to quality issues.5,6 Indeed, evidence suggests they are capable of completing successful quality improvement projects (QIPs),2 in addition to advancing the quality of care during clinical placements.6,7 It is essential, therefore, to nurture the quality improvement (QI) skills of medical students.

In recognition of this, a number of medical schools have introduced QI training programs into their curricula. However, this still constitutes a minority, offering limited guidance to other institutions.8,9 In contrast, at University College London Medical School (UCLMS), students have engaged in QI through an extracurricular student-led initiative; providing a unique environment to explore how they can be involved in service improvement. This paper describes and qualitatively evaluates the initiative, including its organization and outputs. In doing so, it aims to add to the experiences gained by other groups, and offer guidance for establishing medical student QI programs.

Methods

The initiative’s design

Leadership

The initiative comprized of a core leadership team of four medical students, named the initiative’s leaders, ranging from their second to their fifth year of study. One out of the four students acted as the president, and retained ultimate control over the initiative.

Student recruitment

Involvement in the initiative was advertised to students via social media, a website, campus posters, email bulletins and the medical school student magazine. The applicants were required to provide a curriculum vitae (CV), and successful candidates selected to participate by the initiative’s leaders. Students from Medicine, Pharmacy, Medicine with Data Science, Computer Science and Engineering, and Data Science were invited to apply. Furthermore, students from all year groups were invited to apply.

Student education

Prior to project commencement, students were educated on QI methodology through a half day seminar or lecture organized by the initiative’s leaders, depending on when they joined the initiative.

Project conceptualization

The initiative’s leadership sourced QIPs through attending the University College London Hospital (UCLH) board meetings, as well as keeping regular contact with the medical school faculty, local research hospital leaders, Health Foundation fellows and third parties. QIPs also originated from students’ own observations from clinical placements, as well as those gained during participation in the Trust’s Improving Care Walk Rounds (Figure 1). These Walk Rounds are led by the Trust’s Quality and Safety Committee and Department of Informatics, and involve staff auditing the standard of care on a regular basis.

Figure 1

Illustrative example of the initiative’s structure and operation. Firstly, the initiative’s leaders seek to conceptualize projects. This is done in two ways: route (1) the initiative’s leaders regularly attend UCLH (University College London Hospital) board and research meetings in order to find suitable service improvement projects; or route (2) the initiative’s leaders recruit a small number of students who conceptualize projects from their observations of the clinical environment, which are gained by attending the UCLH Improving Care Walk Rounds or from their medical school clinical placements. Secondly, the supervision of projects is decided. Those projects conceptualized in route (1) can be supervised by senior clinical staff/researchers, or the initiative’s leaders. Alternatively, projects conceptualized in route (2) remain supervised by the initiative’s leaders. After then, students from a range of university disciplines are recruited to join the projects and form MDTs (multidisciplinary teams). Depending on the presence of a senior academic supervisor, teams take the form of either Project Structure A or Project Structure B. In both project structures, a recruited student is selected to act as the leader for that project.

Project supervision

Throughout project work, students were mainly supervised by the initiative’s leaders and student project leaders, or more senior academic supervisors if present. Progress was monitored through regular meetings, and communication was maintained via Slack (a messaging platform with file sharing capabilities and team communication tools) (Figure 1).

Timeframe for project completion

Involvement in the initiative was extracurricular, and not constrained by the academic calendar.

Participant incentives

Following project completion, students were encouraged by the initiative’s leaders to write up their project and consequently present and publish it.

Study design and data collection

In order to measure the effectiveness of the initiative, retrospective data was collected on the proportion of the initiative’s projects which: 1) reached completion (for example, projects which resulted in the implementation of a change and where post change data was collected and analyzed); 2) resulted in a significant improvement in their primary outcome; 3) had sustained results at follow-up (conducted 3 years post project completion for the earliest projects, and only on projects which resulted in the implementation of a change (regardless of whether they achieved full completion)); 4) achieved publication; and 5) contributed towards a prize or conference presentation. The data collection period was from the initiative’s launch in 2015, until 2019.

In addition to the aforementioned points, retrospective data was collected on how external parties evaluated the effectiveness of the initiative, as evidenced through prizes awarded to the initiative’s concept and output as a whole. Furthermore, how the initiative has influenced practice locally, such as the medical school and local hospital, has been described.

Ethical approval

Where required, each individual project produced by the initiative sought ethical approval. Considering this study evaluates these individual projects in the aggregate, we feel that ethical approval is not required.

Results

The initiative’s leadership, student education, project timeframe and participant incentives occurred as described in the methods section.

There were 109 students involved in the initiative from its launch in 2015 until 2019. Students came from a range of university courses including 19 Pharmacy (17.5%), 75 Medicine (68.8%), 6 Data Science (5.5%), and 10 Computer Science and Engineering (9.2%) students (Table 1).

Table 1

Detailed team information of the student teams working on initiative’s projects and the prizes, posters and publications of each project

Project name	Team demographic	Generation and supervision of projects		Primary outcome		Project methodology	Project prizes, publications or conference presentations
University discipline	Student generated?	Academic student supervisor	Statistically significant achievement?	Change sustained at follow up?
Improving Maternal Confidence in Neonatal Care Through a Checklist Intervention	7 Medicine 1 Pharmacy	Yes	Yes	Yes	Yes	PDSA	Publication in BMJ Open Quality Presentation at Be the Change NHS Quality Improvement Championships 2016 Institute of Women’s Health Annual conference Best Oral Presentation International Health Congress 2016 1st Prize
Waste Not, Want Not: Improving the Usage of Patient’s Own Drugs	8 Medicine 3 Pharmacy	Yes	Yes	Yes	Yes	PDSA	Presentation at Be the Change NHS Quality Improvement Championships 2016 Paper currently under review in the Pharmacy journal. Presentation at the NHS Change Day 2016
Tackling Frailty in Elderly Inpatients by Improving Food Intake	4 Medicine 1 Pharmacy	Yes	Yes	No	N/A	PDSA	Presentation at Be the Change NHS Quality Improvement Championships 2016
Safeguarding Patient Property in a Hospital Setting	3 Medicine	Yes	Yes	N/A	N/A	PDSA
Reducing Medicinal Waste and Overprescription in Cancer Patients	4 Medicine 3 Pharmacy	Yes	Yes	N/A	N/A	PDSA
Green Bags	4 Medicine 3 Pharmacy	No	Yes	Results unobtained (see Table 1)	Yes	PDSA	Presentation at the UCL School of Pharmacy Science Day
Improving the Flow of Patients Through University College London Hospital	27 Medical	No	Yes	N/A	N/A	PDSA	In progress, write up planned
Artificial Intelligence Project	8 Medicine with Data Science 10 Computer Science and Engineering 6 Data Science 2 Pharmacy	No	No	N/A	N/A	Research	In progress, write up planned
A pilot study to assess the utility of a freely downloadable mobile application simulator for undergraduate clinical skills training: a single-blinded, randomised controlled trial	7 Medicine 5 Pharmacy	Yes	Yes	No	Yes	Research	Publication in BMC Medical Education Presentation at Be the Change NHS Quality Improvement Championships 2016 2nd prize at The Dragon’s Den Inspire Medicine Conference at UCL in January 2016
Improving CPD points for Medical Students	3 Medicine	Yes	Yes	N/A	Yes	Research	1st prize (£300) at The Dragon’s Den Inspire Medicine Conference at UCL in January 2016 One of the finalist projects at the Rosalind Franklin Appathon in March 2016 1st place People’s Choice Award at the London Business School Health Tech Challenge organized by Deloitte in May 2016

The recruitment process occurred as described in the methods section, however, it attracted a greater number of students than places available. Subsequently, applicants were screened and selected based on their CVs, and the process became competitive.

Seven out of 10 projects had their initial concepts generated by students.

Nine out of 10 projects were academically supervised by students, with the presence of more senior medical and faculty staff only acting as clinical facilitators.

Effectiveness of the initiative

During the time period, the initiative generated 10 projects, two of which are still currently ongoing. A brief description of these projects can be found in Table 2. Two types of methodology were used, seven based on “Plan-Do-Study-Act” (PDSA) cycles, and two using a conventional research format. Four projects reached completion, half of which achieved a significant improvement in their primary outcome (two out of four). At follow-up, the changes implemented were still in place in all applicable projects (five out of five applicable projects). Four projects did not reach completion, however, two of these projects (half of uncompleted projects) still resulted in the implementation of a change; only failing to generate post-change data. Two out of 10 projects (20%) have been published in journals, and six have been presented at at least one conference in addition to directly receiving, or contributing, towards a prize (60%) (Table 1).

Table 2

Brief descriptions of individual projects organized by BTC UCL with achieved outcomes

Project title	Project abstract or summaries
Improving Maternal Confidence in Neonatal Care Through a Checklist Intervention	Background: Previous qualitative studies suggest a lack of maternal confidence in care of their newborn child upon discharge into the community. This observation was supported by discussion with healthcare professionals and mothers at University College London Hospital (UCLH), highlighting specific areas of concern, in particular identifying and managing common neonatal presentations. The aim of this study was to design and introduce a checklist, addressing concerns, to increase maternal confidence in care of their newborn child.Methods: Based on market research, an 8-question checklist was designed, assessing maternal confidence in: feeding, jaundice, nappy care, rashes and dry skin, umbilical cord care, choking, bowel movements, and vomiting. Mothers were assessed as per the checklist, and received a score representative of their confidence in neonatal care. Mothers were followed up with a telephone call, and were assessed after a 7-day-period. Checklist scores before as compared to after the follow-up period were analyzed. This process was repeated for three study cycles, with the placement of information posters on the ward prior to the second study cycle, and the stapling of the checklist to the mother’s personal child health record (PCHR) prior to the third study cycle.Results: A total of 99 mothers on the Maternity Care Unit at UCLH were enrolled in the study, and 92 were contactable after a 7-day period. During all study cycles, a significant increase in median checklist score was observed after, as compared to before, the 7-day follow up period (p<0.001). The median difference in checklist score from baseline was greatest for the third cycle.Conclusion: These results suggest that introduction of a simple checklist can be successfully utilized to improve confidence of mothers in being able to care for their newborn child. Further investigation is indicated, but this intervention has the potential for routine application in postnatal care.
Waste Not, Want Not: Improving the Usage of Patient’s Own drugs	Background: Patient’s own drugs (PODs) represent an under-utilized resource within the British National Health Service (NHS) and significant benefits can be achieved by improving the use of these medicines. This study aimed to improve the usage of PODs through the introduction of bedside lockers.Methods: This prospective observational study used quality improvement tools to investigate and improve the usage of PODs on cancer wards at a tertiary academic hospital. Data was initially collected to establish the baseline usage. Ishikawa diagrams were constructed to identify factors contributing to low POD usage. From this information we decided to trial bedside lockers within a Plan-Do-Study-Act cycle to investigate whether they could make a sustainable improvement. A run-chart was plotted outlining the median weekly POD usage across all the wards to analyze the impact of this intervention. A time-and-motion study was used to identify whether there was an impact on nursing drug rounds. The financial cost associated with not reusing PODs was evaluated using the NHS Electronic Drug Tariff.Results: Baseline median weekly POD usage for patients was 20% of patients (range 0–50%). Introduction of the lockers resulted in an increase of median weekly POD usage to 55% (range 36–82%) (p<0.05, run chart). A trend towards reduction in the mean time taken to administer medication to a patient was noted, from 14 mins 35 seconds (±10m23s) to 11 mins 36 seconds (±9m44s) (p=0.17, t-test).Conclusions: Introduction of bedside lockers led to a marked improvement in the usage of PODs across our wards. A trend towards reduction in nurse medicine administration time was evident. The implementation of bedside lockers represents an effective strategy for increasing PODs usage with potential for financial savings.
Tackling Frailty in Elderly Inpatients by Improving Food Intake	Background: Malnutrition and poor food intake are independently associated with inpatient mortality, as well as readmission rates and greater length of stay in hospital. The aim of this project was to increase the food intake of elderly patients on the Care of the Elderly Ward at University College London Hospital.Methods: Clinical observation, discussions with staff and previous literature indicated reduced manual dexterity and availability of staff could be affecting food intake. As such, our intervention was the implementation of medical student feeding teams. Our data collection period lasted three months. Meal trays were collected and photographed, with the primary outcome measure being the proportion of patients eating at least half of their meal. At the end of the data collection, these assessments were verified by an independent assessor who saw the photos in a blinded, randomized order. The secondary (balance) outcome measure was whether there was a net time saving to the staff. This was determined by asking the attending nurses and healthcare assistants how much time had been saved as well as how much time had been taken up helping the volunteers.Results: There was no significant difference in the primary outcome measure (ie the proportion of meals at least half eaten) before vs after the intervention (41% vs 38%, p=0.53). The total staff time saved was 55 mins per day on average.The overall attendance was 100% for the core members and 93% for the additional volunteers.Conclusion: Our intervention did not create any tangible increase in our primary outcome measure. This may be due to a number of factors: the intervention lasted for an insufficient time to enable feeding teams to maximize their skills in feeding, or it could be that insufficient manpower was actually not the limiting factor in the volume of food consumed. Other possible limiting factors and future interventions include the portion size, timing of meals, and the quality of food.
Safeguarding Patient Property in a Hospital Setting	This project addresses safeguarding patient property during their hospital stay. In a busy hospital setting, it is often for patients’ belongings to go missing or be misplaced during transfers between different wards. This project aims to reduce this and the stress it causes for the patients. Areas of particular interest are the transfers from A&E to Ambulatory Medical Unit, and to other wards such as the stroke ward.The students captured one incident of lost property during a four-week data collection period. The incident was reported and thoroughly investigated. A joint training session with the nurses in A&E was performed afterwards and new measure put in place to ensure patient property is not lost during hospital admission. As such, this project did not generate an adequate amount of data for an entire QI project.
Reducing Medicinal Waste and Overprescription in Cancer Patients	Patients are prescribed drugs in hospital that they already have at home. It can take quite a long time for these prescriptions to be sorted out in hospital which will ultimately increase the amount of time the patient spends in hospital. Furthermore, a lot of these medications end up not being used which contributes to waste. In reducing this waste, this project aims to minimize the cost to the NHS.This project was not fully completed due to the required patient leaflets not arriving in time.
Green Bags	Patients are advised to bring in their own drugs when they come in for elective surgeries. This is to ensure a full understanding of the drugs they regularly take and that doses of life sustaining medications are not missed. However, they often leave them at home, or do not bring all of their medicines with them. Furthermore, they often are not aware that creams, inhalers and eye/ear drops are also medicines. This creates several problems including: over prescribing of their regular medications, increased medication waste, reduced working efficiency and increased cost to the NHS. As such, the aim of the project was to increase the proportion of patients bringing in their own medications for surgery.Green bags with instructions for patients bringing in their own medications serve as an easily recognizable container for patients to store their drugs in during surgery. Process mapping of the patient surgical pathway at University College London Hospital was undertaken to identify opportunities for their introduction, and then used during the surgical pathway. Unfortunately results were not obtained regarding their effectiveness as the involved students could no longer contribute to this phase of the project.
Improving the Flow of Patients Through University College London Hospital	Background: The four hour national standard performance is the percentage of patients assessed, treated and either discharged or admitted to a bed within four hours. The success of fulfilling this target gives an indication of quality as well as patient safety. Currently, University College London Hospital is under-performing. Previous investigations exploring the reasons behind this highlight a number of factors, namely admission and discharge pathways. The aim of this project is, therefore, to observe patient flow throughout the hospital and identify areas for improvement.Methods: Medical students focused on three areas: A&E majors admissions, surgical admissions and discharge. Within each area, they tracked patient journeys, observed from the control tower, shadowed staff members and handed out questionnaires.Results: Ongoing - after data analysis from this observation period, solutions will be implemented and post-change data collected.
Artificial Intelligence Project	Due to the ongoing nature of this project, details of the abstract cannot be disclosed. However, students are working under the supervision of clinical researchers looking at adopting artificial intelligence to improve care for patients.
A pilot study to assess the utility of a freely downloadable mobile application simulator for undergraduate clinical skills training: a single-blinded, randomised controlled trial	Background: Medical simulators offer an invaluable educational resource for medical trainees. However, owing to cost and portability restrictions, they have traditionally been limited to simulation centers. With the advent of sophisticated mobile technology, simulators have become cheaper and more accessible.Touch Surgery is one such freely downloadable mobile application simulator (MAS) used by over one million healthcare professionals worldwide. Nevertheless, to date, it has never been formally validated as an adjunct in undergraduate medical education.Methods: Medical students in the final 3 years of their program were recruited and randomized to one of three revision interventions: 1) no formal revision resources, 2) traditional revision resources, or 3) MAS. Students completed pre-test questionnaires and were then assessed on their ability to complete an undisclosed male urinary catheterization scenario. Following a one-hour quarantined revision period, all students repeated the scenario. Both attempts were scored by allocation-blinded examiners against an objective 46-point mark scheme.Results: A total of 27 medical students were randomized (n=9 per group). Mean scores improved between baseline and post-revision attempts by 8.7% (p=0.003), 19.8% (p=0.0001), and 15.9% (p=0.001) for no resources, traditional resources, and MAS, respectively. However, when comparing mean score improvements between groups there were no significant differences.Conclusions: Mobile simulators offer an unconventional, yet potentially useful adjunct to enhance undergraduate clinical skills education. Our results indicate that MAS’s perform comparably to current gold-standard revision resources; however, they may confer significant advantages in terms of cost-effectiveness and practice flexibility.
Improving CPD points for Medical Students	Background: By interviewing more than 100 medical students from universities across the UK, we found that: 52% are still undecided on their career path 60% are not able to easily arrange research projects or shadowing opportunities Earning CPD points has also been shown to be time-consuming and labour-intensive for doctors, especially doctors with portfolio careers. Furthermore, private companies charge the NHS or the independent doctors to help manage their CPD points.Solution: The research team implemented a web-based platform that allows students and doctors to arrange clinical/surgical shadowing schemes, along with research collaborations. It also enables doctors to earn CPD points while allowing students to find mentors and choose their career paths. Students register with their university email, and doctors with their GMC number. Students will be able to choose the specialties that they are interested in and will then be able to find and connect to available doctors based on the geolocation. In addition to providing shadowing opportunities, the platform provides both students and young professionals with opportunities in academic work; where students mainly lack the vision or credibility, and professionals lack the time to produce the data.Outcome: Following roll-out of the platform, results have not yet been obtained regarding its effectiveness.

The initiative has in itself resulted in five awards, as well as three presentations, and catalyzed change at UCLMS (Table 3). After the initiative was successful at the NHS Quality Improvement Championships in 2016, the students were invited to hold the first ever student-led Grand Round to talk about their success. Key faculty representatives were present, which sparked discussion surrounding involvement of QI education into the curriculum. The conclusions of these discussions are currently still in development. It also led to the Change-Makers project which set about surveying the baseline QI knowledge of UCLMS students, and educating them on the key principles. Most importantly, it led to the establishment of an official collaboration with the Chief Executive’s Office of the Local Hospital and the Deans of Medical and Pharmacy schools. Students from the initiative are now involved in the Research Hospital initiative at UCLH and play a fundamental role in clinical audit and patient safety. The student representative now sits on the Quality and Safety Committee and in the Research and Innovation Committee of the Trust.

Table 3

Selected initiative’s achievements 2016–2019

Initiative-wide output
Winner of Be the Change NHS Quality Improvement Championship (£1000) 2016 awarded by the Health Foundation
UCL Change-Makers Prize (£1000) 2016
UCL Grand Round 2016
Three Faculty of Medical Leadership and Management Abstract Submissions 2016
Faculty of Medical Leadership and Management Travel Award 2016
Rosalind Franklin Appathon Finalist 2016
London Business School 1st prize People’s Choice Award 2016
Lead In National Conference Presentation 2016
Royal Society of Medicine Dean’s Commendation 2016

Discussion

The majority of student recruitment was done through social media, with an active website, Facebook page, and Twitter page providing multiple media through which the initiative gained exposure. Additionally, interest was generated via an active blog, with content redistributed through social media pages which showcased the achievements of the initiative. Since recruitment was carried out by a student team rather than senior supervisors, there was greater ease of access to the student network. One of the main advantages of this type of digital communication is that it provides a cheap and efficient vehicle through which many university students can be contacted.

It was also acknowledged that, although social media is ubiquitous, it is important to employ multiple avenues of advertising as a sole reliance on social media has the potential to exclude students and key non-student stakeholders not part of this social media network. The initiative was, thereby, also advertised in the medical school’s weekly newsletter, the medical school student magazine, and public spaces.

As demonstrated by the oversubscription of places on all projects, this strategy overcame issues experienced by other extracurricular research initiatives, which describe difficulty surrounding student awareness of available opportunities, and an inability to attract enough interest from students.10

As previously established, our initiative utilized and observed the unique attributes of medical students as agents of change. In particular, this includes greater flexibility, availability and ease of adoption of new technologies and methods of inter-team communication. We suspect that the extracurricular and competitive recruitment nature of the initiative meant that students were more likely to be self-motivated, ambitious and possess prior leadership, research or QI experience;10 and we believe this may have benefited the QIP outcomes. As such, we feel it is a factor worth formally testing in future research.

The multidisciplinary approach including students from a variety of university disciplines is an entirely novel concept.2 Students from different courses possess different skills, capabilities and knowledge and, subsequently, we speculate this will have widened the potential focus, solutions and impact of the projects. Furthermore, working in a team composed of people from different academic and professional backgrounds more closely resembles real working environments, which may have helped to equip the initiative’s participants with valuable professional skills.

The initiative educated medical students on the basics of QI methodology using either a half day seminar or lecture, organized by the initiative’s leaders. This provided considerably less QI education compared to curriculum based programs, which involved more detailed and regular QI training delivered by teachers with extensive QI experience.11 Such extensive QI training will have, undoubtedly, enhanced students’ ability to carry out effective QIPs, and motivated them to participate above and beyond what was provided by the initiative described here.12

Despite this, we assume that students will still have gained a range of skills and knowledge through involvement in the initiative, even if projects were not completed. We believe this, not only because of the success of the initiative, but due to the fact students were at the forefront of the majority of the various projects’ conception and organization; thus, allowing them to develop and apply knowledge on the methodology and importance of QI. However, the gain of such knowledge was not formally tested in this study. In the same regard, considering many of the projects required front-line interaction, we assume that students’ will have gained a better understanding of the clinical environment and healthcare system, as well as the chance to relate to patients on a more patient-centered level. This is opposed to the purely medical perspective that students are exposed to during medical school, and such benefits are also reflected in previous studies.7,13,14 Collectively, this is likely to improve the students’ future medical practice and generate doctors who practice evidence-based medicine.10

Project Conceptualization

Project concepts can be generated either by students or by senior clinical supervisors; while the study was not designed to directly compare the two methods, based on previous literature, we speculate that both have the potential to influence QIP outcomes. For instance, we suspect that student-generated projects were advantageous in that the involved students likely engaged more, experienced greater ownership, and gained a deeper understanding of the clinical problem. This is in accordance with Morrison & Sullivan (1997),13 who found that students learn less when joining preconstructed projects. Additionally, the conceptualization of projects may have been aided by students’ unbiased and unprejudiced view of healthcare environments. This is supported by Ibrahim et al (2013),5 who showed that medical students were more likely than experienced clinicians to notice potential areas of improvement. On the other hand, students’ limited clinical experience may have risked student QIPs possessing lower clinical significance. Similar difficulties have been described by Jackson et al (2018),12 and Morrison & Sullivan (1997).13

Students were also able to conceptualize projects through their participation in key hospital quality improvement initiatives. For example, students were included in UCLH’s Improving Care Walk Rounds, led by the Quality and Safety Committee and Department of Informatics. These initiatives, despite being lightly resourced, generate essential reviews on a regular basis which ensure the quality of care. Consequently, this provides these initiatives with additional resources in the form of individuals who are interested and impartial.

The majority of the initiative’s projects were supervised by the students themselves, with senior clinicians only acting as clinical facilitators. Projects which received supervision or facilitation by senior clinicians may have benefited from their experience, connections within the clinical environment, access to resources and continued presence beyond project completion which could have enhanced sustainability. However, we suspect a number of ways through which senior supervision could have hindered the progress of projects; first of which is their lack of availability. For instance, when projects were directed by senior clinicians, students lacked the autonomy to proceed with project development in their absence. This had the tendency to delay the progression or commencement of projects depending on the period of their unavailability. Secondly, is senior clinicians’ occasional negative perception surrounding medical student involvement, as they play a pivotal role in encouraging the adoption of changes within a healthcare environment. If projects are to be supervised by senior clinicians, Jackson et al (2018),12 Nikkar-Esfahani et al (2012)10 and Morrison & Sullivan (1997)13 suggest these clinicians should be both adequately trained in QIP leadership, and appreciative that medical student projects can improve care.

Time restriction, as mentioned by previous curriculum-based program, is one of the most influential factors affecting completion and potential scope of medical student QIPs. Furthermore, timeframe restriction has been reported to discourage students pursuing QIPs altogether.10,12,13 Due to the extracurricular nature of the initiative described here, students were not constrained within a curriculum-set timeframe and had the flexibility to choose how much time to dedicate. As supported by previous literature, we believe this may have allowed a higher rate of project completion, sustainability and primary outcome success.10,12,13

Equally, due to the initiative’s strong connections with executives at UCLH, students were able to be transiently involved with larger scale, long-term QIPs. This allowed students unable to commit to the entirety of an individual QIP the opportunity to gain QI skills and experience, as well as providing valuable input towards project progress.

At the initiative’s conception, we hypothesized that student engagement would decline throughout the year without adequate incentives. This was based on the leaders’ previous experience that students’ interest in extracurricular activities tends to wane throughout the year due to growing coursework and clinical responsibilities. Therefore, the initiative incentivized students with the opportunity to present their work at a conference or aim for publication, and to plan this from the outset. Whilst this study did not formally assess the effectiveness of this incentivization, considering the high output of projects (10) within the study period (2015–2019), it is unlikely that student engagement was difficult to maintain.

Additionally, we believe incentivizing students to publish and present their work will aid the initiative in the future, by disseminating its success and thus attracting future participants, stakeholders and supervisors.

We believe the initiative was effective at involving medical students in QI because of the high number of projects generated by the initiative (10 projects) during the timeframe (2015 to 2019), as well as the proportion of completed projects which achieved a significant improvement in their primary outcome (50%). This is supported by the proportion of projects which resulted in the implementation of a change which was sustained at follow-up (100%); a measure that has been difficult to obtain when measuring the effectiveness of previous medical student QIPs.2 Furthermore, 20% of projects were published in journals, and 60% were presented at atleast one conference, in addition to directly receiving, or contributing, towards a prize. This further validates the quality of the QIPs, as judged by the respective journals and judgement panels. Collectively, this suggests that the quality of projects were such that they correctly identified practice that could be improved, and implemented strategies which resulted in change.

The initiative’s quality is further affirmed by the number of awards achieved, especially considering the leadership and QI expertise possessed by the judging panels. Furthermore, the local impact of the initiative is important because it gained the attention of key local stakeholders, such as the medical school faculty, executive hospital board and QI team; which will provide ongoing support to the initiative, and facilitate its sustainability. Moreover, the fact that it was able to gain the support of such stakeholders, reflects the strength of the initiative as judged by them.

Study strengths and limitations

The strength of this study comes from the initiative’s novel design. In particular, this includes the use of students from a range of university subjects and its extracurricular nature, as well as considering elements such as participant incentives, and innovative ways of both recruiting students and utilizing technology. It also builds upon knowledge surrounding the effectiveness of other QI student program characteristics, such as project conceptualization and student supervision. Unfortunately, this study does not test each of these characteristics in turn, and therefore the results cannot be directly attributed to any single characteristic. As such, we suggest that further research should look at testing these factors in turn.

Another weakness of this study is due to the unvalidated method used to evaluate the quality of QIPs. Due to the retrospective nature of the study, crucial data required for such methods, for some of the projects, were not available. Future studies should incorporate validated methods, such as SQUIRE,15 to assess the quality of QIPs when testing the characteristics of their programs.

Moreover, the study did not formally test the students’ gain of skills and knowledge, and the initiative’s design incorporated far less QI training and support as offered by other programs. Therefore, we plan to adjust the initiative’s education process in the future and collect data to support how students gained skills through participation.

The final limitation is that our work, although conducted over several years and in various medical fields, comes from a single university, and different local regulations may be in place at other institutions. Therefore, the model described in the paper may need to be adjusted based on the local needs.

Conclusion

The results of this study suggest that the unique and innovative design of this quality improvement initiative has the ability to generate quality QIPs, and successfully involve medical students in QI in the context of a London Medical School and its local teaching hospital. Such features of our initiative’s design include its: student leadership; multidisciplinary approach; extracurricular nature; utilization of technology and social media to recruit students and aid team communication; use of student incentives such as publication and presentation; and formation of connections with key stakeholders such as the medical school faculty and local hospital executives and QI team. Furthermore, it builds upon existing knowledge provided by the analysis of other medical student QI programs, indicating that both student- and supervisor-generated or supervised projects can be equally successful. Moreover, due to the involvement of students in all stages of the initiative, it is highly likely that the initiative provided students with QI skills, knowledge and experience.

As such, other institutions building their own QI programs or involving medical students in QI, may wish to use some, or all, of the aspects of our program to enhance their own; depending on the local requirements and restrictions.

Future research should be aimed at further elucidating which individual factors of our program has the greatest effect on QIP quality, as well as quantifying and validating the skills, knowledge and experiences gained by students, and the effect this has on their future careers.