Usability testing of an e-learning resource designed to improve medical students' physical activity prescription skills: a qualitative think-aloud study.

OBJECTIVE: An e-learning resource (MEdic GAming, MEGA) was developed based on the contents of the Faculty of Sports and Exercise Medicine exercise prescription booklet. This study aimed to (i) explore medical students' perspectives of physical activity promotion and e-learning and (ii) investigate medical students' response to the design, content and usability of the MEGA e-learning resource.
DESIGN: Qualitative think-aloud interview study.
SETTING: A London medical school. PARTICIPANTS: 19 undergraduate medical students were interviewed using the think-aloud method while using the e-learning resource concurrently.
RESULTS: In general, medical students felt current education on physical activity is inadequate and held a strong desire for more teaching on exercise medicine. Students believed the MEGA e-learning resource addressed a gap in their knowledge on physical activity but noted e-learning should not replace face-to-face teaching and suggested physical activity education would be best delivered through a blended learning approach. Students felt such an approach would allow better opportunity to practice physical activity counselling skills with patients while on clinical placement. Students' motivation to engage with the MEGA e-learning resource was positively impacted by aesthetically appealing design and interactive gamification elements such as self-assessment quizzes and visual progress tracking.
CONCLUSION: Medical students value the role of physical activity in health but are disappointed by the lack of teaching within the current medical curriculum. E-learning resources, such as MEGA, which contain interactive features are a viable means to integrate physical activity into the undergraduate curriculum but should be supplemented by the opportunity to practice physical activity counselling in-person. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The think-aloud methodology allowed candid responses to the MEdic GAming e-learning resource allowing insight on whether such an approach to integrating physical activity to the undergraduate curriculum would be well received by students.

This study expands on previous questionnaire studies and provides insight on medical students’ desire for opportunities to practice physical activity counselling while on clinical placement.

The study sample comprised students from one London medical school limiting the generalisability of the results to other medical schools with different curriculum structures.

Introduction

Despite physical activity playing a key role in the prevention and management of chronic disease1 2 many doctors do not feel confident discussing physical activity with their patients.3 4 Doctors cite lack of knowledge and limited training on exercise prescription as two of the main barriers to physical activity promotion.4 5 As tomorrow’s doctors, medical students need to be competent in physical activity prescription. However, most final year medical students are unaware of the Chief Medical Officer physical activity recommendations and many are unfamiliar with patient counselling techniques used to encourage lifestyle change.6–8 The shortfall in medical students’ knowledge has led to several calls for education on physical activity promotion to be implemented into the undergraduate medical curriculum.9 Existing efforts to integrate physical activity education into the undergraduate curriculum have taken the form of short lecture series or didactic teaching sessions.10 However, implementing additional contact-hours on physical activity is problematic as there is minimal time available in the undergraduate medical curriculum for additional face-to-face teaching. Self-directed co-curricular learning material delivered via e-learning platforms may be a potential time and resource efficient solution to integrate physical activity material into the undergraduate curriculum.11 Asynchronous physical activity education delivered via e-learning would be especially beneficial to students in the clinical years of medical school who, due to geographical and temporal differences between placements, require flexible access to study materials, the ability to ask questions and opportunity to revise at a time that suits them most.

MEdic GAming (MEGA) is a computer-based, e-learning resource designed using Talent LMS software (Epignosis UK, London, UK). It was developed within the Centre for Sports and Exercise Medicine (SEM) at Queen Mary, University of London. MEGA is designed to educate undergraduate medical students on physical activity prescription through self-guided workbooks, interactive multiple-choice questions (MCQs) and case-based quizzes. It is based on the 155-page ‘Medical Student Exercise Prescription Booklet’ published by the Faculty of Sport and Exercise Medicine (FSEM).12 While a stand-alone e-learning resource to improve qualified doctors’ physical activity prescription skills exist,13 to the best of the authors’ knowledge MEGA is the first such independent e-learning resource for medical students. Previous educational interventions have included e-learning but only in a blended learning format to supplement face-to-face delivery.10

Usability testing and iterative refinements are key steps in e-learning resource development.14 Despite several studies investigating medical students’ awareness of physical activity guidelines, there is limited understanding of medical students’ viewpoints on how physical activity education could be integrated into the curriculum. Therefore, this study aimed to (i) explore medical students’ perspectives of physical activity promotion and e-learning and (ii) investigate medical students’ response to the design, content and usability of the MEGA e-learning resource.

Methodology

To ensure reporting transparency, the COnsolidated criteria for REporting Qualitative research (COREQ) checklist (online supplemental file A) was followed.15

Study design

Think-aloud methodology, whereby participants freely verbalise thoughts while engaging in a task, was used to conduct the interviews.16 Prior to the interview, the researcher set-up the prototype e-learning resource on a desktop computer. Participants were asked to complete a short questionnaire on their age, gender, year of study and self-reported weekly exercise time adapted from the Godin Leisure-Time Exercise Questionnaire (online supplemental file B).17

Each interview followed the same structure outlined within the interview guide (online supplemental file C). At the beginning of the interview participants were asked to describe their views on exercise medicine and their existing thoughts towards e-learning by answering set semi-structured interview questions. Participants were then asked to complete the think-aloud task. During this task participants were asked to verbalise their thoughts aloud while using the prototype e-learning resource. Participants were encouraged to voice both positive and negative comments and identify potential improvements that could be made to the e-learning resource. If a participant stopped talking, the interviewer used open questions such as ‘You are doing well—what are you thinking now?’. At the end of the interview after the think-aloud task had concluded, participants had the opportunity to make any additional statements and ask questions. All interviews were audio-recorded and there was no defined time limit. Only the participant and interviewer were present during the interview. No participant had engaged with the e-learning resource beforehand. Field notes were made during and after the interviews to aid analysis.

Patient and public involvement

Stakeholders (undergraduate medical students and members of the FSEM) were involved in the conceptual development and design of the MEGA e-learning resource and provided feedback on the MCQs and case-based quizzes. An undergraduate medical student (HC-R) was also involved in the design of the qualitative think-aloud interview guide, provided feedback on the overall study concept and assisted in data collection and analysis.

Participant eligibility and recruitment strategy

The eligibility criteria for study participants are outlined in table 1. A convenience sample of medical students from a London medical school were invited to take part in the study. Advertisements on campus, in the weekly e-bulletin, and on student social media groups were used for recruitment. The study aimed to enrol 15–25 participants. This sample size was based on existing think-aloud studies, the likelihood of dropout, and the expected number required to meet data saturation and congruency.18–20

Table 1

Eligibility criteria

Inclusion criteria	Exclusion criteria
An undergraduate medical student (MBBS)	Qualified doctors
Any year of study within a medicine degree	Postgraduate SEM MSc students
Participants that have an intercalated BSc in SEM or are studying medicine after graduating with Sports Science or Public Health degrees are eligible	Students or qualified allied healthcare professionals, for example, physiotherapists

SEM, Sports and Exercise Medicine.

Participant consent

All interviews were conducted by a female iBSc student (HC-R) in March 2020 at Mile End Hospital, London. HC-R received think-aloud interview training before conducting the research and had no personal relationship with any of the participants. Study participants received a £15 online shopping voucher to compensate them for their time.

Data analysis

The interview recordings were transcribed full verbatim. The qualitative data were checked for accuracy and the six-phase process of thematic analysis was followed.21 Line-by-line coding was carried out by HC-R using NVivo, a qualitative data analysis software tool, V.12 (QSR International Pty, Melbourne, Australia) to generate an initial coding framework (online supplemental file D). Another member of the research team (GP) independently coded the transcripts using this framework. Results were then compared with ensure inter-rater reliability. A final list of themes was agreed by the wider research team (HC-R, GP and RA) through an iterative discussion process. No repeat interviews were carried out, the transcripts were not returned to participants for comments or correction, and the themes generated were not discussed with participants. No new themes were emerging by the final few interviews; therefore, data saturation was reached.

Results

The advertising for study participants generated 27 participant responses. Of these, eight participants were excluded from taking part due to ceasing correspondence with the research team and inability to travel. In total 19 participants were interviewed. On average each interview lasted 32 min (range, 20–45 min).

Study participant characteristics

Table 2 outlines the population demographics of participants. The mean age of participants was 21.3 years (range, 19–24 years) and the majority of participants were men (57.9%, n=11). All study participants were in years 1–4 of medical school. The students who participated were highly active, reporting an average of 599.5 min (range, 180–1020) of exercise per week. Students’ average time spent completing low-intensity exercise (eg, walking) per week was 94.2 min (range, 0–100) and for high-intensity exercise (eg, running and cycling) was 274.7 min (range, 90–720).

Table 2

Population demographics of study participants

Participant characteristic	Total (n=19)
Age in years, mean (SD); range	21.3 (1.2); 19–24
Gender, % (n)
Male	57.9 (11)
Female	42.1 (8)
Year of MBBS study, % (n)
1	21.1 (4)
2	26.3 (5)
3	26.3 (5)
4	26.3 (5)
5	0 (0)
Average time participants spend in minutes, mean (SD); range
Exercising per week	599.5 (261.9); 180–1020
Low-intensity exercise per week	94.2 (125.0); 0–300
Moderate intensity exercise per week	230.5 (216.4); 0–840
High-intensity exercise per week	274.7 (151.4); 90–720

Core emergent themes

Four core themes emerged from the interviews: (i) value of physical activity prescription and promotion recognised, but underutilised; (ii) the importance of a blended learning approach; (iii) the need for stage appropriate information on physical activity promotion and (iv) the importance of engaging design and interactive features. Figure 1 provides a thematic map derived from the data.

Figure 1

Thematic map. PA, Physical activity.

Value of physical activity prescription and promotion recognised but underused

Students recognised the value of education on physical activity prescription and promotion to their subsequent clinical practice post-medical school. However, many students reported receiving limited teaching on physical activity. Students often described the teaching they had received as inadequate noting physical activity was only mentioned briefly under lifestyle recommendations or as an afterthought in lectures.

thinking of our own medical curriculum and trying to think of how much teaching we have directly had on physical activity, apart from it just being indirectly mentioned in some lectures […] we have never had actual dedicated learning to it (P7, year 2)

Students reported disappointment that they had not observed physical activity prescription on clinical placement, especially in secondary care. However, some students did report that they had received physical activity teaching from general practitioners with a special interest in SEM but that it was often not clear what the process was for promoting or prescribing physical activity. When prompted to reflect on why physical activity was not observed in clinical placement students cited barriers for qualified doctors including: knowledge, time pressures, avoidance of difficult conversations about physical activity for weight-loss and patients’ expectations to leave appointments with a paper prescription.

Although I have been on practicing GP [placement], I don’t really see it used. I have never seen it actually initiated. And whether that is because GPs don’t have enough time to do it properly and effectively or whether they don’t agree with it, I don’t know (P19, year 3)

Students recognised that physical activity prescription and promotion was a practical skill that is currently underused. Without dedicated teaching students felt they would not have the relevant skills to promote physical activity to patients.

We do learn about the importance of it but we’re not really taught about how to actually give these recommendations to patients and that’s probably why they don’t give this advice, because they [GPs and doctors] don’t know how to do it. So I think it’s important to learn a bit more about that at university, or in medical school. (P10, year 3)

it is surprising that I’ve not had any formal teaching on it. You would think that since exercise is such a massive thing in terms of health, and they promote it so much, why aren’t we being taught about it? Because if we’re going to promote it to our patients, then we should know more about it (P2, year 2)

Importance of a blended learning approach

Participants reported mixed experiences with using e-learning resources in the past. Students highlighted that e-learning often felt tokenistic, not well thought through and an easy way to teach the masses. However, students noted that e-learning provided a means to learn independently and at a time convenient to them.

Some of the ones that I’ve had access to in the past haven’t been the easiest to use, which kind of puts you off using them. But the ones that are made very well, are then good because you can go through them often at your own pace, and really understand it (P2, year 2)

I do prefer having online resources because it’s something you can do in your own time when you’ve got a chance and you can go over it many times. It’s not one of those things that … If you go to our anatomy sessions, you're in there once and then you take away a booklet—you don’t have the rest of the information there. Whereas histology practicals, because they’re all on an online thing, you can redo them as many times as you want. I feel that’s better for reinforcing the knowledge (P9, year 1)

Students were aware that there was a general shift towards e-learning in medical education. However, many students noted that as medicine is a vocational degree, e-learning can complement face-to-face teaching in the form of blended learning but it cannot replicate or replace real-life patient interaction.

I think it gets to a point in medical school where [clinical skills and patient interaction] are your fundamental principles. I know there are things like virtual patients coming in […] but for the most part you need person-to-person interaction in order to develop those things. I would say I don’t think it [e-learning] can replace those more practical means of learning (P13, year 4)

Need for stage appropriate information on physical activity promotion

Students were asked to voice their opinions while using the MEGA e-learning resource. Students highlighted that the resource was concise, informative and addressed existing gaps in their knowledge on how physical activity can be prescribed for the management of chronic diseases. Students liked that the content was progressive and started with basic explanations before becoming more detailed.

I like that it’s got this FITT principle as well, because I’ve never heard of it, and so having that and explaining it is really nice, and it’s explained clearly […] I’d never have known about these isometric exercises that shouldn’t be completed and stuff. It really gets to extra areas that you wouldn’t think of yourself, which is good, because it’s really teaching you something as well […] I’ve learnt a lot that I don’t think I would have ever had the … given the opportunity to learn about (P2, year 2)

I really like this bit on the steps of how to prescribe exercise. It’s got the four different colours and then it goes into more details, with the matching colours below. I think it’s really good that in the first bit it’s more concise with the steps and then it goes into more detail, but uses bullet points. It’s really easy to understand at a glance I think. And then there’s the contraindications but in red so I think that’s good, with the stop sign that shows that it’s really important. And I like that it’s got this recap diagram at the end, which basically covers everything that’s on this page in a nice picture, so I like that (P11, year 3)

Medical students felt the e-learning resource was relevant in both preclinical and clinical years. This was because physical activity promotion is overarching to different stages of the curriculum through its influence on different body systems and pathologies. However, several students noted that the e-learning resource could benefit from being less generic and instead tailored towards the specific learning outcomes of a particular year group. For example, the case-based scenario questions could be included only as an extension activity for preclinical years (where the focus is less on differential diagnoses and management of conditions). Some students felt the level of medical jargon and acronyms was too high which could be challenging for preclinical students.

I think certain things, like prescribing exercise, it would be pretty similar over the five, six years […] I think it’s pitched at an appropriate level for all medical students […] I think it’s good that the two levels of testing can be more appropriate for certain types of students, with the single best answer questions potentially being more aimed at first, second, third years, but being useful for all years. And then this section, Section 3 [case-based scenario], being for students in their clinical training […] it should be able for [pre-clinical students] to have a look at, but I think it should be emphasised to them that this is aimed at a higher level (P1, year 4)

Importance of engaging design and interactive features

As outlined within table 3 the design of the e-learning resource was well received by participants. Colours and font type used were perceived as aesthetically pleasing. Images that captured attention and summarised key points of the accompanying text were particularly valued. Students also thought the images helped retention of the written information. Bullet points and the presentation of information in short segments was important. Participants thought that emphasis of keywords by bolding or underlining would further enhance user experience.

Table 3

Participant quotations on the design elements

Subtheme	Quotation and participant information
Aesthetics	‘I really like the fact that each section was a different design and colour and I think red really goes well with hypertension’ (P10, year 3)
	‘aesthetically, I liked the use of the pictures and font is good, a good size, easily readable’ (P17, year 4)
	‘when I’m thinking back to something I quite often picture it, so having the visual (image) next to it helps’ (P15, year 4)
	‘the information is a lot of bullet points so it’s quite simple to read, and whilst it’s longer you still haven’t got huge chunks of text so it’s still relatively simple for everyone to follow’ (P9, year 1)
	‘I definitely think it is good to highlight the key words, because if you are going to go back over it and you want to look over something quickly, you don’t want to be reading through the whole thing’ (P5, year 1)
Ease of navigation	‘it’s very user-friendly […] it’s got the flowchart style again, it makes it easy to take in the information in these little chunks […] I can go back and forth with the information at my own pace […] I like having it all in one page […] I think sometimes with a click through you feel like the information is endless. You never know if you’re near the end or you’ve still got loads to go’ (P11, year 3)
Interaction and its impact on students’ motivation	‘I like the fact that you are able to track your progress […] it’s something to just have that little bit more motivation […] it brings that competitive aspect to it, so that will probably entice people to actually put more effort in’ (P7, year 2)
	‘maybe if you had a way of clicking on each panel and then it will just flick round and then you can click it, it will go back […] Maybe just make it a bit more interactive like that […] Sort of like flashcards’ (P17, year 4)
	‘I do agree with having a quiz at the end to try and reinforce the knowledge and make sure you have actually learned something from it’ (P19, year 3)
	‘I don’t really like this one, the filling in the blank, because I find it’s hard to get the exact word […] or if you spell it wrong, for example I'm dyslexic, so if I got one letter wrong then it would say the answer is wrong […] I'm not really a fan of the free text answer’ (P11, year 3)

The flowchart design enabled easy navigation of the e-learning resource and the vertical scrolling layout was favoured by the majority of participants. The sidebar could visually track progress and reviewing previous sections was easier than with the more traditional horizontal click through often found in other e-learning resources.

The blended approach of traditional text-based content with interactive learning elements was popular with participants. The progress bars (figure 2) and interactive elements in the e-learning resource increased participant engagement levels. Students reported that these led to increased levels of motivation. Some participants desired even more interaction from the resource, for example embedded flashcards and expandable boxes for definitions.

Figure 2

Example of the tracking of progress on the e-learning resources’ homepage.

Participants stated that the self-assessment quizzes helped consolidate knowledge. The MCQ style (figure 3) was the most popular with students. They acknowledged that this format of question can be easily completed, is good to recap learning, and is the format found in most medical school examinations. Explanatory sentences as to why participants’ answers were correct/incorrect were also favourably received. Concluding a topic with a case-based scenario after the MCQs was well-received. Students acknowledged its free-text answer format tested understanding in a different way however disliked the additional time required to complete and had negative past experiences with computer programme marking free-text inaccurately. Consequently, participants would prefer the case-based scenarios to have multiple-choice instead of free-text answers.

Figure 3

Example of a multiple-choice question within the resource.

Discussion

This study explored medical students’ views on the usability of an e-learning resource (MEGA) related to physical activity promotion and prescription. Results suggest that although current medical students receive minimal teaching on physical activity prescription, they recognise and value the role of exercise in the prevention and management of chronic conditions. The study findings suggest that medical students found the MEGA e-learning resource informative and a potential solution to integrate physical activity into the undergraduate medical curriculum. However, it was evident that students thought that e-learning on physical activity prescription should be supplemented by the opportunity to practice patient counselling face-to-face in either dedicated teaching sessions or on clinical placements.

Previous cross-sectional studies have highlighted the gap in graduating medical students’ knowledge of physical activity prescription.7 8 22 This qualitative study highlights that despite being aware of the importance of physical activity in the prevention and management of disease medical students were often unsure of how to prescribe physical activity to a patient. It is unsurprising that students report that physical activity prescription was not observed on clinical placements given existing evidence that most doctors do not discuss physical activity with their patients.4 Akin to existing survey studies6 8 23 medical students within this study desired more teaching on the subject of physical activity prescription. This provides further stimulus for physical activity to be embedded within the undergraduate medical curriculum. Results also suggested that medical students value e-learning and believe medical education is evolving to incorporate it more frequently. This is reflective of the growing popularity in e-learning over traditional learning strategies and its increasing use in the medical curriculum.24 COVID-19 will also further accelerate technology-enhanced distance learning and e-learning resources within medical education.25 26

In general students believed the MEGA e-learning resource was a feasible approach to teach physical activity promotion. However, students clearly stated that e-learning cannot completely replace face-to-face teaching especially as it does not allow the opportunity to practice patient counselling. A 2018 Cochrane review, examining educational methods among healthcare professionals, concluded that while e-learning makes little empirical difference to knowledge (when compared with traditional learning), blended learning provides healthcare professionals the opportunity for learning practical skills in person.27 Blended learning is defined as applying multiple teaching methods within education, for example, lecture-based and e-learning. In direct comparison to traditional lecture-based teaching blended learning has also been found to result in higher student satisfaction and engagement.24 This study suggests that although e-learning may be beneficial in medical education the vocational elements (clinical placements and opportunity to practice patient counselling) cannot be replaced by digital resources.

Study findings suggest that medical students found the content MEGA e-learning resource informative. Existing quasi-experimental studies28 highlight that e-learning improves students health behaviour counselling skills in the short term. Similarly, the feasibility testing of the FSEM e-booklet demonstrated that knowledge of physical activity guidelines and confidence in advising patients about PA significantly improved after simply reading the through the contents.8 It remains unknown if addressing medical students’ deficit of physical activity knowledge is beneficial in the long-term to their future clinical skills. Future research should investigate the feasibility and effectiveness of learning resources delivered during medical school on the physical activity promotion skills of graduate doctors in clinical practice. Future research could also be in the form of focus groups to facilitate debate among student participants about the e-learning resource, enabling a deeper insight into their opinions.

The present study shows that design elements impacted medical students’ motivation to engage with the e-learning resource. This finding is in line with previous usability studies of digital-based interventions.19 In particular the interactive elements in the e-learning resource, such as the visual tracking of progress, were valued by participants. Existing literature demonstrates that gamified design elements within e-learning tools, including progress tracking, can increase user motivation and ultimately lead to greater learning success. The self-assessment exercises within the resource (MCQs and case-based scenarios) were appreciated by participants. Evaluation of a cardiovascular e-learning tool demonstrated that medical students value the ability to assess their own knowledge and the majority of students recommended the inclusion of quizzes in future resources. The value medical students place on design and formative knowledge testing are important considerations when developing e-learning resources.

Limitations

Although this research provides useful data for the development of e-learning resources, as only one resource was evaluated, the results may not be generalised to all types of e-learning. Second, recruitment was from one London based medical school only, so results may not be fully generalisable to other medical institutions where teaching methods may differ. A further limitation of the results is that students in the fifth year of medical school were unable to be recruited due to the scheduling of final examinations. However, since results were consistent across years 1–4, it would be unlikely for year 5 data to differ substantially. Analysis of the baseline questionnaire data indicated that students who are frequent exercisers appeared to be more inclined to participate. The average time participants spent exercising was around 600 min/week, nearly four times the UK physical activity guidelines. This indicates that volunteer bias could be present and participants might have a particular interest in SEM or lifestyle medicine. During data collection some participants tried to engage with the researcher despite being briefed on the interview think-aloud protocol meaning the participants’ cognitive processes could have been possibly influenced. It is important to highlight that qualitative research is limited by the subjective nature of theme extraction. Within this study this was minimised by having two investigators concurrently extracting themes and the whole research team being involved in theme review to ensure inter-rater reliability. Finally, while the sample size is small (n=19), it was sufficient to reach data saturation, congruency, and is large compared with existing think-aloud studies.18 19

Conclusion

This think-aloud study improves current understanding of medical students’ views on e-learning to teach physical activity promotion. It appears that e-learning resource is a suitable medium to deliver educational content on physical activity however it is evident e-learning should be supplemented with the opportunity to observe or practice patient counselling.