The Use of Mobile Devices to Enhance Engagement and Integration with Curricular Content.

This perspective describes the processes, pilot designs, and deployment strategies that the Yale School of Medicine (YSM) employed when integrating iPads into the undergraduate pre-clinical medical curriculum. We also explore the multiplier effect this technology inspired in expanding the program into our clerkship curriculum with the iPad Mini and how integrating mobile technology into our existing E-systems afforded us opportunities to enter the e-book and augmented reality technology spaces. Our hope is that this perspective provides a framework that may guide other institutions researching their own technological innovations.

Introduction

An important challenge for medical education is the exponentially growing body of medical knowledge that our trainees must be able to access, master, evaluate, and be able to apply. For example, Densen [1] estimated that in 1950, it would have taken 50 years for the known body of medical knowledge to double in quantity. This doubling rate was reduced to 3.5 years in 2010, and is predicted to be 0.2 years by the end of 2020. As the quantity of skills and knowledge that healthcare learners need to master continues to grow it is important that the mechanisms for engaging in curricular content be transparent, intuitive, and integrated. The more complex and diverse our curriculum engagement systems, the more they compete with the time that our learners need to acquire the expanding array of clinical skills and knowledge.

Designing, implementing, and deploying a curriculum is an expensive and resource intensive process [2,3]. Prior to 2010 our institution delivered curricular materials, including syllabi and note packets as files that were individually downloaded from the learning management system for use on personal computers and other devices. We also distributed hard copies of these materials, syllabi, and notes at the start of each course. This method of delivery posed a number of challenges for students, including that once distributed, they could not be updated to reflect faculty modifications of their materials after the start of the course, and the economics of hard copy distribution limited the volume of materials provided to students. For example, PowerPoint or Keynote presentations were almost never provided since they tended to require many pages of limited information and were often modified by faculty before a lecture or workshop.

Managing the growing volume of material in medical education is a daunting task [4]. As part of our educational strategic planning process for curriculum renewal in 2010 we began exploring ways to simplify both our curriculum management and delivery systems. We moved to a new Learning Management System (LMS) for both the pre-clerkship and clerkship periods which allowed us to centralize management and curriculum delivery for our administrative staff. We also wanted an affordable, practical, and engaging mobile technology to assist our students in navigating and annotating the substantial volume of information they were receiving. It was also important that we provide our students with a unified platform from which they could seamlessly and securely engage with our multiple E-Systems. The ideal device would have a small form factor, offer cellular and Wi-Fi networking, have a high-resolution display, offer extensive storage capacity, and permit easy content organization, searchability and annotation. It needed to be secure for generating and transmitting electronic protected health information (ePHI), be Health Insurance Portability and Accountability (HIPAA) [5] compliant and have a large ecosystem of existing scientific and medically focused applications (apps). We also wanted students to be able to rely on this device to access and participate in all curricular sessions and assessments and to be capable of imposing an organizational file and folder structure that would be common across all courses. Ideally, it would serve as a multipurpose device that would integrate into a student’s daily life (email, scheduling, study, entertainment) so that the student would naturally keep it with them and would not have to “remember” to bring it to class. Finally, we considered it important to limit the options to a single vendor and device to reduce individual and institutional support costs and to minimize software variability across applications.

The introduction of the Apple iPad in 2010 was timely for us in that this tablet met many of our requirements for a student-centered one-to-one device. It had a high-resolution color display, adequate internal storage, included both Wi-Fi and cellular networking and had nearly 140,000 apps available on launch [6]. By the fall of 2011, YSM started their iPad initiative purchasing over 500 tablets at a cost of approximately $500,000 and gifting them to the entire undergraduate medical student population. This perspectives piece will explore the benefits and challenges of integrating the iPad into the curriculum, the multiplier effect it had for inspiring innovative pedagogies, and how its success influenced our aspirational goals for the curriculum over the past 10 years.

Topics

iPad Pilot

In the medical environment all devices that may be used to store, generate or access patient information must be secure and HIPAA compliant [7]. In the education environment a student’s device should be portable, reliable, fast, and easy to use. At the time of the iPad release in 2010 a number of tablet devices were also available that used the Android open operating system (OS). An important consideration in choosing the iPad for our program was the simple design, ease of use, and managed, or closed, OS. We considered Android tablets, but the open OS meant that there was little standardization across devices. Each Android vendor created their own “flavor” of the OS and in 2010 this made enforcing HIPAA compliance difficult since each Android device’s OS could have different vulnerabilities to malware and cyberattack. The iPad met all of our initial requirements. It had built-in hardware encryption turned on by simply adding a locking passcode to the device. It could be remotely wiped in cases of a lost or stolen device using an easily configured internal app or via our Microsoft Exchange email server. HIPAA compliance was configured by installing a password-protected policy on the device. Finally settling on the iPad meant our support services had only one hardware device and OS to support.

Although we had a process to secure student personal laptops and make them HIPAA compliant, it was fraught with complaints and meant central administration of their personal devices. The diversity of devices and OSs also meant that IT support services could not simply deploy a single solution for the myriad of devices that students brought with them. Thus, this was a time consuming, complicated process, and often resulted in degraded device performance and reduced stability. We decided to pilot the iPads in our curriculum which, if successful, would have at minimum a two-fold benefit: providing the students with a secure, portable device for engaging with curricular material and freeing the students from the time, performance, and stability issues related to securing their personal laptops.

We started the project with a small, agile, administrative team to keep it on track and responsive to stakeholder needs. A large group with multiple participants assuming disparate roles could too easily get bogged down in scheduling conflicts and communication issues due to various levels of understanding of the problem domain. We decided on a team that included three roles:

• Educator, expert in pedagogy

• Administrator

• Technologist

These roles were populated by the Dean of the Curriculum, Curriculum Manager, and Associate Director for Technology, respectively.

In addition to the administrative team our pilot group included 10 student volunteers: five who self-identified as tech-savvy, and five who self-identified as tech-naïve. The inclusion of both tech-savvy and naïve students was an important consideration in the design of the pilot. Despite all one hears about students being “digital natives” our experience has been that every incoming class displays a wide range of proficiencies and levels of comfort with technology consistent with the data of Helsper and Enyon [8]. Students in the pilot were provided with an iPad, an Apple external keyboard, an annotation app (iAnnotate®) and $125 to use for purchasing any curriculum appropriate apps or hardware they thought might be useful or that would enhance the use of the tablet (e.g. case, Bluetooth keyboard, stylus). The curricular e-documents, including lecture PowerPoints, for two undergraduate medical courses were deployed to Dropbox on a weekly schedule so that current versions would be available to the students allowing annotation of the files that they were seeing in class. Students were given read-only access to these cloud-based materials and the annotation app was configured to synchronize to Dropbox at the touch of a virtual button. To address the issue of a student having annotated a document that was later centrally updated in the cloud, the annotation App was configured to prompt the student to either save a new version of the material, or to replace their annotated copy with the new content.

We scheduled regular weekly meetings over pizza to gather feedback, and the pilot ran for the 2010 spring semester. Important points we learned for the full implementation in the fall of 2011 included that the:

• portability, mobility, and “ease of use” of the iPad exceeded expectations

• iPad did not act as a distraction in class since students were actively annotating their files and not just using it as a reader

• students preferred third-party Bluetooth keyboard/cases over Apple Bluetooth keyboards or the virtual keyboard

• curriculum documents needed a compact naming convention so that they could be easily scanned as a list on the devices limited screen size

• course materials should share the same folder structure so students did not need to learn where materials were found from course to course

• deployment of cloud-based course materials to folders on the mobile device needed to be automatic, not requiring the student to identify the destination folders

• presentations used in the classroom must be up to date and exactly match those available on the iPads to encourage students to annotate on them during the class session

iPad Program

The full implementation of an iPad-facilitated curriculum was begun in the fall of 2011. All medical students were provided with iPads with cellular capability and configured with 64 gigabytes of memory, the maximum available. We also purchased the PDF annotation app GoodReader® that was the student preferred annotation app from our pilot. We did not activate the cellular capability of the devices, but including it allowed students to selectively activate it on their own. This was particularly useful for students if they were at clinical sites or during breaks where Wi-Fi access was not available or reliable. In addition to the student iPads, we purchased a number of additional iPads that could be loaned to faculty so they could engage with the curriculum as a student would and gain insight into how their content would appear on the new platform. Simultaneous with the distribution of iPads to all students we also deployed a new Learning Management System (LMS). Built into the LMS was a feature that provided for the automatic generation of a PDF for every document uploaded to the system. Working with the LMS vendor, DaVinci Education [9], we created a document distribution system that would move these PDFs to a WebDAV [10] server on the hour, every hour. We selected a WebDAV server for deployment as we already had our own web server and adding WebDAV functionality was trivial. The WebDAV server allowed us to easily restrict the students to read-only access while providing curriculum administrators read/write access.

Our document distribution system ensured that any new content uploaded to our LMS would be available to the student’s iPads within an hour. Students could get the most up to date content on their iPads at the touch of a single button in the annotation application. To handle the case where a faculty member came to class with new or updated material we created a drag-and-drop applet that would convert and transfer the file to a WebDAV server for instant student access. The current workflow for these instances has been that an audio-visual assistant drops the new or updated material onto the applet icon on the podium computer, the applet converts the file into a PDF and sends it to a WebDAV server. Conversion of the file and transport to the WebDAV server typically takes less than 3 minutes. Students in the classroom then synchronize these files to their iPads ensuring that they are annotating presentations that are identical to what they are seeing in class. The applet also sends an email to our administrative staff to alert them that there is new content that should be uploaded to our LMS.

An essential element of the iPad program is the orientation session that all students must attend. Our pilot program and experience with technology rollouts made it clear that although incoming students have extensive experience using mobile devices and e-materials, if we want them to use these materials most effectively and in a way that facilitates their learning it is important to offer appropriate training and support. During orientation all students participate in a one-hour session devoted to the use of iPads for classroom and curricular activities and how they are used for generating and communicating ePHI in a HIPAA compliant manner. Students are then assigned to one of three smaller groups to assist them in setting up the security profile on their device, email, multi-factor authentication, and all curricular apps. The HIPAA profile prevents backup to the cloud, encrypts local iTunes backups, enforces an 8-character alphanumeric locking passcode, and locks down the device after five minutes of inactivity. Before they graduate, students must attest to removing all ePHI from their devices allowing computer support staff to remove the password-protected HIPAA security profile. During the small group session students are also given a $75 Amazon gift card, which they typically use to purchase a Bluetooth cover/keyboard or Apple pencil. Before each student leaves the session, we answer any questions, introduce them to the support staff, and perform their first curriculum synchronization to ensure everything is working properly.

We had concerns that the introduction of so many devices to the medical campus might create the need to substantially increase our technical support staff. To our surprise, it turned out that visits to our computer support center actually declined. In a poll of students 6 months into the program 68% of them reported that they had never visited the support center.

While cost was not the driving factor in our iPad project, it turned out that the cost of giving the device to every student versus the cost of printing materials was about the same: approximately $1,000/student. However, when taking into account the administrative activities of our staff required to pursue faculty for updated materials to print at the start of each course, send these materials to the printer, bind them, and then distribute them to students, there were substantial cost savings beyond the cost of the device. While this did not lead to a reduction in staff, thus making it difficult to put a precise dollar figure on the savings for staff effort, it has allowed us to provide greater support to educational programs at the start of each semester. Over the past 10 years the iPad has increased student engagement with the curriculum, required no increase in support staff or central management, and continues to be an important, reliable device that is integral to our curriculum. Since the students are taking notes directly on the device, annotating the material being presented in class, it is a much more active learning environment which promotes better knowledge retention [11]. Student comments below from a survey 6 months into the program remain true today:

• “It’s amazing to be able to bring around everything I need to study medicine in such a small package. I really feel like I’m able to use it in almost everything I do.”

• “I have found it much more convenient to use and carry around than a laptop. I also really like that Yale cares enough about the environment to implement a paperless curriculum.”

• “I can take all of my notes with me everywhere!”

iPad Mini

In the initial years of the iPad initiative it became apparent that students in clinical rotations were not using the devices with any consistency. Surveys of the students suggested that they felt they were too cumbersome to carry in the clinics merely as a research tool and that since we had not deployed the electronic health record (EHR), Epic, on them they were not useful for patient care. In October of 2012, Apple introduced the iPad mini [12] and in June 2013 YSM decided to see if the reduced footprint of the device would increase student use of the iPad in the clinical setting. We put together a pilot group to evaluate the device that included a small number of clerkship students, residents, and attending faculty. Each was given an iPad mini setup with a security profile similar to the full-size iPads and configured to access Epic. Again, we met weekly to gather feedback on how they used the devices and to discuss any barriers and advantages they encountered. Important points we learned included that the:

• iPad mini could easily fit in a white coat pocket

• integration with the EHR benefited the pilot group by reducing the time spent waiting to use the in-service computers to access patient records

• ability to access imaging in the EHR allowed sharing of information with patients in a more intimate and effective manner

• distribution of the minis at the start of the clerkship year allowed a hardware refresh/update for the student’s two-year old full-size iPad devices in a timely manner and to facilitate their use in clerkships

An unexpected benefit of student use of the iPad minis in the clinic was that it resulted in a greater role and integration of the students into the rounding teams. With the mobile access to the EHR students often reported that they became the source of patient information for the team without the need for the mobile computers in the clinic. As iPads and other mobile devices have become more common place in the clinic this benefit has been reduced, but the advantage of easier and more timely access to patient information remains important to our students.

Rather than “gifting” the iPad mini to the students we decided to subsidize their purchase of the device. This gave the students some “skin-in-the-game” and made it more likely that they would use the devices in their clinical training. Each year the majority of our students (approximately 95%) participate in the program. In 2019, the total cost of an iPad Mini with 256 Gigabyte capacity, lacking cellular capability, with AppleCare was $588. The model with cellular capability and AppleCare cost $688. Students paid $159 for an iPad Mini without cellular capability and $218 for an iPad Mini with cellular capability. Table 1 shows the usage across all of our clerkships after the first year of the program.

Table 1

How Often Did You Use the iPad Mini in Each Clerkship?

Course	Average %	SD	Responses
Medical Approach to the Patient - Internal Medicine	68	29	39
Medical Approach to the Patient - Neurology	64	28	38
Primary Care & Psychiatry - Primary Care	52	31	38
Primary Care & Psychiatry - Psychiatry	53	29	34
Surgical Approach to Patient - Emergency Medicine	56	31	35
Surgical Approach to Patient - Surgery	59	31	34
Women & Children’s Health - OB/Gyn	64	26	36
Women & Children’s Health - Pediatrics	65	29	37

iBooks

In a student survey we did 6 months into our first year with the iPads 92% of our students reported that they found the iPad “Good” to “Excellent” for reading eBooks. We had explored the delivery of curriculum content using e-books for supplemental curricular material and had met with various publishers and attended many demonstrations over the years, but most of the existing platforms had licensing as well as content issues that made us reluctant to make an investment.

In January 2012, Apple released the free app iBooks Author 2.0 for creating e-books [13]. The iBooks Author platform offered us a way to create our own e-books with material specific to our curriculum, at low cost, and with minimal training for faculty content creators. An iBook is easy to deploy and update and the technical expertise required for creation and design is on the level of a PowerPoint or Keynote presentation. It can present text, high-resolution color images, animations, audio, video, self-assessments, and engaging user interface elements via a library of user interface widgets. The iBooks Author app creates a table of contents, auto-generates indexes, and can include a glossary as well as links to external content in a mobile friendly format that is available on all Apple platforms. Also, Apple is well known for their engaging user interface design and the ease of use of all their products, which were important for e-book adoption by our faculty and students [14,15]. Since every medical student already had an iPad and most also had iPad minis, the fact that iBooks were an Apple only resource was not a deterrent.

In 2013, we decided that our iPad program was sufficiently stable and integrated into our curriculum that it made sense to pilot iBooks Author for creating e-books for curriculum delivery. Our pilot iBook was a migration of the web-based Neurobiology Lab Manual. We worked with a medical student who collaborated with the authors in the iBook’s design and interactive features. The iBook was distributed to students via a web link in our LMS. Students could engage with the material using either the existing website or the iBook. At the end of the semester we polled the students and they reported they found the iBook easier to navigate and more enjoyable to use than the website. The faculty author found the iBook much easier to update and enhance than the website as well. The following year we decommissioned the website and provided the material solely as an iBook. The success of the Neurobiology iBook and pressure on other courses by students led us to create training workshops for faculty interested in converting course content to an iBook. In collaboration with local Apple representatives these individualized sessions allowed faculty to bring current or reworked curricular materials with them and then were assisted in importing them into an iBook. Faculty were helped with content organization, adding images, configuring assessment widgets, and user interface design. Over the past 7 years, our library of faculty developed iBooks has grown to over 25 volumes across multiple disciplines including Neuroscience, Clinical Skills, and Human Anatomy. We deploy the iBooks out of our Amazon Web Services (AWS) Simple Storage Service (S3) via uniform resource locators (URLs) in our LMS. The majority of our iBooks are only available internally, although we are exploring the idea of publishing Yale branded iBooks in the App Store.

Innovative Pedagogies

One of our students’ common struggles is to understand the complicated spatial relationships between organs and systems in anatomy in 3D [16]. Typically, these relationships are taught with 2D imagery that the student must translate in their mind to 3D. Some can grasp this easily; others struggle. One approach that we have explored to facilitate this learning has been with the use of Augmented Reality (AR). AR allows virtual objects to be observed in the “real world” using a device with a camera and some specialty software. Just as one can walk around a real object to understand the spatial relationships between elements on its different surfaces, AR allows the observer to virtually inspect different element surfaces by moving the mobile device relative to the virtual image.

The authors collaborated with an undergraduate computer science student to design and implement an AR app for exploring a high-fidelity 3D artist’s rendering of the human brain and spinal cord and its component structures. The rendering was purchased from an online site [17]. The app presents a 3D brain model as if hovering in the room in front of the student and has opacity adjustments for the right and left cerebral hemispheres to reveal internal structures. Additionally, entire neural sub-systems can be hidden completely to isolate and study specific structures. The app runs on an iPad or iPhone and is integrated with our Neuroanatomy iBook via “Universal Links” (UL). A UL is essentially a URL but rather than point to a website it will open up an application installed on a mobile device. An instructor can use the app to configure the brain to display only a specific structure or a collection of structures. The app provides the faculty member with a UL as well as a snapshot for that neural configuration to insert into an iBook. For example, a section in the iBook discussing the spinal system would include a UL that opens the AR app and displays the isolated 3D spinal cord for the student to study. Tapping a link in the AR app returns the student to the iBook page where they started. (Figures 1 and 2).

Figure 1

iBook with Universal Link to AR Application.

Figure 2

AR App with Faculty Configured Component.

The app has been designed as a modular shell that can be used to view any anatomical structure we provide. Currently, students have access to artists renderings of 3D structures, however, steps are in progress to provide a mechanism for integrating 3D renderings of “real” human organ structures derived from MRI and CT imaging. Our aspirational goals include having a cloud-accessible catalog of structures which the students can study. We would also like to take advantage of Apple’s multi-user AR capabilities so that a group of students could interact in the augmented reality space in a faculty led session.

Conclusions and Outlook

We have learned over the past decade that when introducing technology into the curriculum it is important to include student, faculty, and administrative stakeholders in every step of the process. Considerations for a successful program should start with the expected enhancements in teaching and learning, student engagement, include the estimated administrative support load, be cognizant of any training requirements, and recognize the importance of integration with existing electronic systems [18]. The integration of mobile devices in both the pre-clerkship and clerkship periods of the curriculum also helps our students develop the necessary digital skills and facility with technology they will need to be effective clinicians as the body of medical knowledge continues to grow and as E-Systems based on mobile technologies become the standard for patient records and care [19-23].

Our aspirational goals are to continue seeking opportunities to enhance learner engagement with innovative uses of mobile technology. We are particularly intrigued with the potential transformative pedagogy available to us with the iPads in the burgeoning field of augmented reality.