Using low-cost Android tablets and instructional videos to teach clinical skills to medical students in Kenya: a prospective study.

OBJECTIVES: To assess the feasibility and impact of using a low-cost Android tablet to deliver clinical skills training to third-year medical students in Kenya.
DESIGN: A prospective study using a low cost tablet called 'connecTAB', which was designed and manufactured specifically for areas with low bandwidth. Instructional video tutorials demonstrating techniques of cardiovascular and abdominal clinical examinations were pre-loaded onto the tablet.
SETTING: Maseno University School of Medicine, Western Kenya. PARTICIPANTS: Fifty-one third-year medical students from Maseno University School of Medicine were subjects in the study. Twenty-five students were assigned to the intervention group and 26 to the control group. MAIN OUTCOME MEASURES: At the start of the study, students from both groups completed an Observed Structured Clinical Examination (OSCE) of the cardiovascular and abdominal evaluations. Students who were allocated to the intervention group then received the connecTAB, whereas students in the control group did not. After a period of three weeks, students from both groups completed a post-study OSCE for both the cardiovascular and abdominal evaluations.
RESULTS: There were significantly higher improvements in the scores for both cardiovascular and abdominal examinations (p < 0.001) within the group who received the e-tablets as compared to the control group.
CONCLUSION: The study suggests that access to connecTAB improves clinical education and efficacy and holds promise for international training in both medical and allied healthcare professional spheres in resource-limited settings.

RCT Entities:   Population Interventions Outcomes

Introduction

The World Health Organization (WHO) reports a global healthcare workforce shortage of 7.5 million doctors, pharmacists, nurses, community health workers and other basic healthcare professionals.[1] Estimates indicate that this deficit will grow to 12.9 million in the coming decades. While sustainable, high-quality medical education is essential to fulfill the healthcare workforce shortage in low and middle-income countries (LMICs), medical education faces unique challenges in these settings. Government funding has become increasingly limited, forcing universities to require more students to pay larger fees to ensure their program’s financial solvency.[2] The decrease in government funding has led to larger class sizes and rising student-to-faculty ratios, with less one-on-one teaching as a result.[3] Medical education is further hindered by the migration of highly skilled healthcare professionals to more affluent countries – a phenomenon commonly referred to as the ‘brain drain’.[2,4] Consequently, there are fewer experienced faculty available to teach students in LMICs.[5] Together, large class sizes and inexperienced faculty have led to rushed and unprepared clinical teaching.[3]

Recent advances in educational technology offer a potential solution to the on-going challenges facing medical education in LMICs. In the last decade, a largely non-profit industry has spearheaded educational innovations. For example, the Khan Academy and EdX have, and continue to offer, online courses that are utilised by tens of millions of students around the globe.[6,7] These video-based courses are high-quality, free to the user, and accessible anywhere via an internet-connected computer, tablet or smart phone. The efficacy of online learning media (known as E-Learning) when integrated into medical curricula has been validated. Studies have concluded that E-learning provides greater educational opportunities and resources for students while also improving faculty efficiency and efficacy.[8]

The exponential growth of E-learning has led public health and medical education experts to consider it as a potential tool to aid in closing the health-workforce gap in resource-poor countries.[9] Mobile broadband subscriptions in LMICs have increased more than 26-fold between the years of 2005 and 2014, and access to computers and smart phones is becoming increasingly common.[10] These developments make it possible to extend supplemental educational materials to under-equipped medical educators and their students via tablets and smartphones.[11]

In resource-poor settings, E-learning may prove to be particularly beneficial in teaching the clinical examination.[12,13] Pre-loaded or streamed videos can teach clinical examination skills in a manner similar to the lessons taught by the Khan Academy, and they can be accessed at any time for additional review. Furthermore pre-loaded videos mean that students do not have to rely on broadband internet connection, which is often difficult to find in many LMICs. The effective teaching of examination skills is critical in these regions primarily because many laboratory tests and imaging modalities are unavailable, rendering the clinical examination a healthcare practitioner’s most important diagnostic.

The purpose of this study was to better understand the potential of the connecTAB, a low-cost Android tablet, as an educational aid for the learning of clinical examination skills amongst medical students in Western Kenya. Specifically, the aim of our study was to assess the efficacy of using low-cost Android tablets for video-based clinical-skills training and medical education in resource-limited settings, by comparing Observed Structured Clinical Examination (OSCE) scores between a cohort of students who had connecTABs and a cohort of students who did not.

Methods

Materials

The connecTAB is a low-cost, Android computer tablet that was developed for educational purposes and for this study. It is equipped with Jellybean 4.2 software, a forward and backward facing video camera, microphone, internal speakers, and 1 GB of internal memory with the option to expand it to 32 GB through an external memory card slot. Each connecTAB has the ability to tether to the local 2G and 3G cellular networks and, therefore, works in areas of low bandwidth. It also has the ability to work via a Bluetooth network, allowing local file sharing even in areas with slow internet speeds. For more details of the exact technical specifications of the connecTAB please refer to Figure 1 in the Appendix. Each connecTAB cost $50 USD to produce and for the purposes of this study came preloaded with instructional video tutorials of how to perform basic clinical examinations. This enabled students to access high-resolution instructional videos independent of the internet. In addition, a pre-loaded PDF document with relevant supplementary material was also included.

The videos were developed by Geeky Medics, an organisation specialising in open-access clinical video tutorials.[14] The cardiovascular and abdominal video tutorials that were used in this study were each reviewed by two independent medical doctors for accuracy. Both doctors had extensive training in clinical examination assessment and hold Master’s Degrees in Clinical Education.

Ethical approval

The Partners HealthCare Institutional Review Board (Boston, MA, USA) reviewed the study and determined that it did not meet the definition of human subjects research, thus the project was exempted from further review.

Recruitment of medical students

Kenya was selected as the pilot country due to its location; East Africa has one of the lowest density of doctors and medical schools anywhere in the world.[15,16] Third-year medical students from Maseno University School of Medicine volunteered to participate in the program in response to an email solicitation from the school’s administration sent to the entire student body. Students were informed that participation was voluntary and that they would be required to complete a pre- and post-study questionnaire, as well as a pre- and post-study clinical assessment. Students were also notified that they would be randomly allocated to either the intervention group or the control group. At the conclusion of the study, students in both the control and intervention group received a connecTAB.

Data collection using OSCE

At the beginning of the program, all students were asked to participate in pre-study OSCEs in which they performed cardiovascular and abdominal physical examinations on a fellow student. These particular examinations were chosen because they are widely regarded as the basic core examinations that should be mastered in the early stages of clinical training.

Each OSCE used a structured 25-point mark scheme, allowing an examiner to objectively score students on their clinical examination performance. A score of 16/25 (64.0%) was deemed a ‘Satisfactory Pass’, and a score of 20/25 (80.0%) or more was deemed ‘Excellent’ and worthy of academic distinction at the medical school. The full mark scheme for both cardiovascular and abdominal examinations can be found in Figure 2 in the Appendix. Students were given 7 min to complete each physical examination. The examinations were observed and assessed by an independent certified doctor. The mark scheme had been previously assessed for accuracy by two independent medically qualified doctors and was also approved and personally reviewed by the Dean of Maseno University School of Medicine.

Students were randomly assigned to either an intervention group (n = 25) or a control group (n = 26). Students in both groups received their usual clinical teaching for the duration of the study. Two students withdrew from the control group during the pre-assessment period and were, therefore, ineligible to continue in the remainder of the study. Each member of the intervention group (n = 25) received the connecTAB with the preloaded clinical examination videos and a PDF document with theoretical information relating to the examinations. The students in the control group (n = 24) did not receive connecTABs during the study but did receive them upon completion. All students involved in the study were reminded regularly that students in the control group should not have access to tablets or videos.

The students in the intervention group were instructed to use their connecTAB to access the instructional videos and to download any other relevant material or medical applications onto the tablet to assist them with their learning. After three weeks, students from both intervention and control groups were asked to return for post-study OSCE assessments in which they were reassessed on the cardiovascular and abdominal examinations.

An independent board certified doctor, who was blinded to whether the students were in the control or intervention group, assessed the OSCE scores for both the cardiovascular and abdominal evaluations.

Statistical analysis

Statistical analysis was conducted using SPSS Version 21 (SPSS Chicago).[17] An independent t-test and a paired two-sample t-test were used to determine significance of pre- and post OSCE scores for the control and intervention groups. A p-value of <0.05 was considered statistically significant.

Reliability analysis

Inter-rater reliability was assessed by a second independent medically qualified professional who viewed the same 10 videos as the original assessor and assigned scores based on the standardised 25-point OSCE mark schemes. These scores were then compared with the scores assigned by the original assessor and an alpha coefficient was calculated.

Results

Pre-study OSCE scores

An independent sample t-test revealed no significant differences between the control and intervention groups’ pre-study OSCE scores in either the cardiovascular or abdominal assessment.

Post-study OSCE scores

All 100% (n = 25) of students in the intervention group and 71% (n = 17) of those in the control group returned for the post-study OSCE assessment. An independent t-test revealed statistically significant differences in post-study examination scores between intervention and control groups in both the cardiovascular (p < 0.0001) and abdominal (p < 0.0001) domains, with the intervention group scoring significantly higher than the control group in the post-study assessment (Tables 1 and 2).

Table 1.

Cardiovascular OSCE assessment (total points possible: 25).

Cardiovascular OSCE	Intervention group: mean score (range, SD)	Control group: mean score (range, SD)	p Value
Pre-study assessment	5.92 (11, 2.67)	4.3 (11, 3.07)	p = 0.201
Post-study assessment	20.48 (11, 2.76)	9.88 (14, 3.24)	p < 0.001

Table 2.

Abdominal OSCE assessment (total points possible: 25).

Abdominal OSCE	Intervention group: mean score (range, SD)	Control group: mean score (range, SD)	p Value
Pre-study assessment	5.96 (10, 2.89)	6.21 (11, 2.95)	p = 0.107
Post-study assessment	20.63 (12, 3.09)	9.11 (14, 2.87)	p < 0.001

A paired two-sample t-test was also conducted to determine whether the change in scores between pre- and post-study for each of the four ‘domains’ (intervention cardiovascular, control cardiovascular, intervention abdominal, control abdominal) were statistically significant. All four trials had statistically significant increases; additionally, the change in score was an average of 8–10 points higher for both intervention groups as compared to both control groups (Table 3).

Table 3.

p Values for change in score from pre- to post-test in all four domains.

Domain	Change in score	p Value
Control cardiovascular	5.17	p < 0.001
Control abdominal	2.34	p = 0.005
Intervention cardiovascular	14.66	p < 0.001
Intervention abdominal	14.51	p < 0.001

Reliability

The intra-class correlation coefficient (ICC) for inter-rater reliability was 0.960.

Discussion

This study suggests that the introduction of the low-cost Android tablet, connecTAB, with preloaded demonstrational videos, to a cohort of third-year medical students in Kenya was associated with marked improvements in clinical skill examination outcomes; medical students with the connecTAB performed significantly better on two different clinical skills examinations than medical students without the tablet (all p values <0.05).

In addition to supporting existing literature that highlights the potential of E-learning as a useful pedagogic adjunct for clinical training,[12] our findings also suggest that having the videos pre-loaded on an inexpensive mobile device can allow for quality clinical learning in areas where financial constraints and poor infrastructure limit dependable internet access and access to technological devices. More than half of the participants did not own an internet-enabled device of any kind prior to the study, and while the convergence of Internet and cell phones has become pervasive in countries like the United States and United Kingdom, the majority of mobile phones sold in Africa are not internet-enabled smartphones. In Kenya, where 82% of the population owns a cell-phone, only 19% of those are smartphones.[18] Although these trends are predicted to change over time, smartphones remain a luxury available primarily to high-GDP nations.[18] Our innovation of a pre-loaded, low-cost tablet that works in areas of low-bandwidth proved to be beneficial in Maseno, Kenya, and would likely be advantageous for LMICs in general, because of the low-cost of the device.

From a design standpoint, the study revealed a number of interesting concepts. Pre-loading a video onto a tablet or smart-phone device allows viewers to fast forward without having to wait for videos to buffer or load – something that regularly occurs in areas with poor internet. Secondly, pre-loading ensures that all instructional videos demonstrating clinical examination are accurate and standardised. Furthermore, the ability to watch a video anytime, rewind, and view the source multiple times may also give pre-loaded videos a distinct advantage in teaching clinical skills.

In terms of study limitations, the OSCE is by no means a perfect tool for assessing clinical ability. It has been recognised that the OSCE tests the student’s knowledge and skills in compartments, rather than their ability to look at the patient as a whole.[19] Despite this disadvantage, the OSCE is still regarded as a valid and reliable method to objectively grade and score clinical skills of junior medical students, and has long been recognised as the gold standard for the assessment of students’ clinical competence.[20,21] The nature of an OSCE means that the variables of the examiner and the patient are to a large extent removed.[19] The use of a checklist marking criteria results in a more objective examination and means that crucial skills must be demonstrated in order for the student to fulfill the criteria to obtain a pass mark.

A further limitation is that it is difficult to determine the degree to which changes in examination scores can be attributed to the integration of connecTAB in exam preparation as opposed to other outside resources. Anecdotally, we know that there were students who accessed additional materials on the tablets, including e-textbooks, and PowerPoint presentations. The acquisition of these external materials could have led to confounding if the external resources played a greater role in improving the scores than the videos themselves. However, unless the students were all utilising the same outside resources, it seems unlikely that one would observe such a dramatic increase in mean scores. Looking at the browsing history of the tablet could provide a unique opportunity to understand what information the students perceived that they needed that the videos lacked. To utilise these data, we are in the process of developing an analytics program (similar to YouTube analytics) that will allow us to assess factors such as the number of times a video was accessed and the duration of viewing by an individual student. If any of these variables correlate with performance, this knowledge can then be used to inform content development.

Another limitation to the study is the possibility that students in the control group might have accessed the tablets and the videos from the students in the intervention group, despite explicit instructions against doing so. Additionally, because two students dropped out of the study before it began and seven students in the control group failed to return for the post-assessment, there is the potential for skewed data as a result of uneven group sizes. Finally, the duration of the study was only one month and, thus, it is impossible to comment on the long-term retention of the skills acquired.

Integrating smart phones or tablets with pre-loaded instructional video materials into existing teaching methods, especially in resource-limited settings, may have positive implications for learning outcomes, which could be magnified if key personnel, such as course leaders and facilitators, support the program. This buy-in from local stakeholders would both increase ownership of the technology and the results that it yields, which would allow students to receive feedback from in-country or institution senior members of staff.

In terms of future research, the present study was limited to cardiovascular and abdominal examinations. It is possible that these examinations are particularly responsive to video demonstrations. It may be unwise, therefore, to generalise these findings to all clinical skills examinations. We suggest expanding to other, more varied clinical examinations in future studies to test this claim. Furthermore, the potential of the connecTAB program, utilising affordable technology equipped with open-access videos should be explored further in different contexts like the post-graduate training of doctors and with populations such as nurses and community health workers. If connecTAB proves effective in these other environments, then it may well be worth scaling up the project to other LMICs.

Conclusions

The results of this small, experimental study suggest that the use of low-cost tablets or smart phone devices with preloaded instructional videos could offer an innovative and cost-effective pedagogical approach to enhance medical students’ clinical skill competencies and self-confidence levels. To be sure, demonstration videos are not a panacea and need to supplement traditional teaching methods rather than replace them.[22,23]

Display	7 inch touchscreen 1024 × 600 resolution at 171 ppi, SD video playback
Size	191 × 115 × 10.6 mm (7.5" × 4.5" × 0.4”)
Storage	8GB (with a microSD card slot for up to 128GB of storage)
Battery life	Up to 10 hours
Connectivity	Single-band Wi-Fi. Supports public and private Wi-Fi networks or hotspots that use the 802.11 b, 802.11 g or 802.11 n standard with support for WEP, WPA and WPA2 security using password authentication. Also supports Bluetooth connectivity for peer-to-peer networks.
Ports	USB 2.0 (micro-B connector) to connect to a PC/Macintosh computer; microSD slot for external storage
Software	Android Jelly Bean 4.2.2