Utilization of an online module bank for a research training curriculum: development, implementation, evolution, evaluation, and lessons learned.

BACKGROUND: Students enter Oakland University William Beaumont School of Medicine's required research program, Embark, with variable levels of experience. Recognizing this, Embark allows for progression through the individual research project with flexibility. Since 2014, student self-directed curriculum personalization is promoted through a menu of online modules.
OBJECTIVE: This evaluation sought to understand student usage of the modules, identified strengths of the modules and preferred attributes of the modules. Gaining this evidence will provide information on how to best meet students' needs in a just-in-time format.
METHODS: A retrospective mixed methods analysis of the module library was conducted. The library was constructed using best practices as an educational tool. The retrospective evaluation included analysis of students' viewing patterns and answers to required course evaluations during the fall semesters of 2014 to 2017. Students' preference for modules was determined by viewing records and conjoint analysis.
RESULTS: Students' milestone preparation was not negatively impacted by relocation of curricular content from lecture to modules. Changes in module implementation within the course (2016) resulted in an increase of students viewing modules beyond only the minimum course requirements (71% (2016) from 10% (2014)). Data from both quantitative and qualitative evaluation questions show an increase in students' identifying the modules as a strength to individualize the course. The identified module strengths include content individualization, just-in-time access, while identified needs included a desire for additional modules. Students preferred modules that were animated, shorter in duration and curated from an external source.
CONCLUSIONS: Online modules provide students with a rich set of resources allowing for individualized learning. Lessons learned in the implementation of the online modules may be transferable to many educational topics. When implementing similar technology projects, usage rates, learner feedback, and effect on appreciation of the content are important to frequently monitor.

Introduction

Instruction for the conduct of a required research project in undergraduate medical training can be difficult to implement due to the concentrated nature of the general curriculum, the limitations of course time and the need for white space. This is especially true when the curricular construct for the conduct of the research project allows for a self-regulated pace of execution. The solution for Oakland University William Beaumont School of Medicine (OUWB) was to create a large menu of online, cafeteria-styled modules that could be used to promote student self-regulated personalization of the curriculum.

Students at OUWB participate in a required scholarly concentration program (Embark). Through the Embark program medical students complete 10 research training courses that span the pre-clinical and clinical-years including a clerkship year dedicated research rotation, and a faculty-mentored individual research project [1]. The courses build a strong research foundation for all students by focusing in the first year on developing relationships with mentors, generating research questions, designing and implementing a research plan, and navigating compliance requirements. Subsequent years emphasize data collection and analysis with a focus on developing effective methods of communicating research results. Students are also encouraged to focus on the operationalization and implementation of the specific requirements of their project throughout the four years, while in the process developing professional practice and transitional skills such as time management and research team communication. During the first year in the Embark program, students are expected to progress through several research project milestones (research mentor identification, research question development, approval of a project proposal, and regulatory body approval if appropriate).

Medical students enter into the Embark program with variable levels of prior research experience. Recognizing this, the Embark program was designed so that each student may progress through the training courses and research project with an independent and flexible pathway. A significant challenge to the program’s individualized pace was that there was insufficient course time available to provide all the course content at a time when students felt it was most appropriate to them. To address this concern, the modules were introduced in 2014 and were designed to support the completion of the Embark program research project milestones. The collection of modules is comprised of short videos and text under the threads of mentor selection, research question development, project proposal, Institutional Review Boards (IRB) and compliance and communicating science. With the introduction of the modules, significant blocks of curriculum were moved from didactic lectures to online components. Specifically, content that was deemed both by students and program leadership to be more relevant to subsets of project designs (e.g., survey design) was offered in module format. Content intended to build upon and support in-class presentations, and content intended to encourage self-directed seeking of further knowledge were included in the module menu.

E-learning technology and modalities are used in many aspects of medical education [2-5]. E-learning and digital resource implementations range from accessory course materials placed online to entire course modules being offered in an asynchronous online setting [6-11]. Lessons learned from previous e-learning curricular usage include 1) accounting for the student expectations for and interest in technology, 2) availability and reliability of tech resources, 3) context in which the e-learning is used, 4) and educational approach through which the online curriculum is implemented [2,4,12,13]. The Embark program modules were constructed or curated using these four best practices in educational technology design. Embark leadership, being cognizant of time required to experience the content, developed short modules containing animations and videos with links to additional resources [10,14,15]. Moreover, the intent of the online modules was that students utilize them as part of a blended learning opportunity.

The Embark online modules were designed to address the cognitive, affective and psychomotor domains of learning in a self-directed manner. Modules were organized into concise, research practice related themes according to information type and milestone outcome goals [17]. Students selected the modules based on content descriptions to fill their individual knowledge gaps that were self-identified from participation in classroom active learning and self-reflection exercises. The module content was structured to address student information needs for topics that would assist them with their project submission milestones. The media style variability was offered to influence the affective dimension of student interaction with the content. The psychomotor aspect came from the utilization of the information to support the creation and submission of the milestone product.

Students selected specific online modules from within the aforementioned threads (mentor selection, research question development, project proposal, IRB and compliance and communicating science) to watch independently. Ideally, students viewed a module prior to attending a large class setting lecture/activity building from the online module content, or as a supplement to support the learning of information previously presented in class. Additional modules were reviewed and/or accessed in a just-in-time fashion as students’ independent projects progress. Since the implementation of the Embark program online modules in 2014, several key lessons were learned and subsequent improvements were made. In this article, the strategies used to develop, implement, and evaluate the Embark program modules will be discussed towards understanding a) student identified usage, b) student identified strengths, and c) student preferred attributes of the modules.

Materials and methods

The generation of the module library was an educational response to offer a just-in-time option that would support best practices in the dimensions of learning (cognitive, affective, psychomotor). A retrospective mixed methods analysis of the module library was conducted from data collected during normal educational practices.

Online module content

The short modules cover important concepts in traditional research design curriculum, as well as specific topics requested by previous Embark program students. The module design includes multiple information delivery modalities (including website links, articles, and short videos.) Program faculty supplied several components of module content, including edited portions of several previous lecture recordings. Additionally, links to research education content created for training purposes from well-respected external research organizations were provided (e.g., NIH Office of Intramural Training & Education, U.S. Department of Health and Human Services Office for Human Research Protections (OHRP) and the Office of Research Integrity). Many modules contain additional links to referenced external resources. Modules were built and exist exclusively within an e-learning creation software (SoftChalk: eLearning Authoring Tool https://www.softchalkcloud.com/). Several modules contain internally made cartoon videos designed with animation software and aimed at capturing students’ attention while providing Embark program-specific processes (PowToon, https://www.powtoon.com/home/?). In multiple instances, content previously presented in a traditional PowerPoint fashion was transferred to an animated video. The complete menu of modules is housed within a password protected internal curriculum management software platform (Moodle https://moodle.org/). Table 1 describes the threads of online module content, modalities included in each module, as well as sample sources of information included in modules. Screen capture videos of example modules are provided in Supplemental Digital Content A and B.

Table 1.

Threads of Embark program online modules and content titles.

Required Modules
1. Embark Mentor & Student Roles and Responsibilities – OUWB Internal – PowToon animated cartoon
2. Introduction to SmartSheet – OUWB Internal – PowToon animated cartoon
Mentor Selection Modules
3. Authorship – External video and website links
4. Beaumont Mentors & Culture – OUWB Internal – lecture recording
5. Finding a Mentor – External video and website links
6. Questions to Ask your Mentor – OUWB Internal – PowToon animated cartoon
7. Team Dynamics – External video and website links
Research Question Development Modules
8. Community Based Participatory Research – External video and website links
9. Epidemiology – External video and website links
10. Evaluating Resources – OUWB Internal website; External video and website links
11. Hypothesis Testing – External video and website links
12. Introduction to Community Based Research – OUWB Internal – lecture recording
13. Introduction to Faith-Based Community Research – OUWB Internal – lecture recording
14. Introduction to Clinical Research – OUWB Internal – lecture recording
15. Introduction to Global Health Research – OUWB Internal – lecture recording
16. Introduction to Medical Education Research – OUWB Internal – lecture recording
17. Introduction to Quality and Safety Research – OUWB Internal – lecture recording
18. Introduction to Systematic Review Research – OUWB Internal – lecture recording
19. Is Research Wrong? – External video and website links; Journal article
20. Reading a Scientific Paper – External video and website links
21. Research Questions – OUWB Internal – PowToon animated cartoon
22. Research Types and Innovation – OUWB Internal – PowToon animated cartoon; External video and links
23. Study Designs – External video and website links
24. Qualitative or Quantitative Research – External video and website links
Project Proposal Modules
25. Bias and Confounding – External video and website links
26. Choosing a Statistical Test – External video and website links
27. Grounded Theory – External video and website links
28. How to Conduct a Focus Group – External video and website links
29. Locating Existing Surveys – OUWB Internal video
30. Qualitative Research Design – OUWB Internal video
31. Sample Size – External video and website links
32. Social Media in Healthcare – OUWB Internal video
33. Specific Aims Development – External video and website links
34. Statistics Primer – External video and website links
35. Survey Design – OUWB Internal video
36. Using Public Databases – External video and website links
37. Using Secondary Datasets – External video and website links
38. Using Storyboards in Research – External video and website links
39. Writing the Research Proposal – External video and website links
Institutional Review Boards (IRB) and Compliance, Communicating Science Modules
40. Animal Testing Ethics – OU Internal resources; External website links
41. Budget & Travel – OUWB Internal – PowToon animated cartoon
42. Communicating Science – External video and website links; Journal article
43. Conflict of Interest – External video and website links
44. Data Management- External video and website links
45. De-identifying Data – OUWB Internal – PowToon animated cartoon; External website links
46. Giving an Oral Presentation – External video and website links; Journal articles
47. Informed Consent – External video and website links
48. Preparing a Manuscript – External video and website links; Journal articles
49. Privacy & Confidentiality – External video and website links
50. Protecting Human Subjects – External video and website links
51. Scientific Misconduct – External video and website links
52. Tips for a Poster Presentation – External video and website links; Journal articles
Example External Sources (all cited and linked to from within modules that are housed within our password protected internal curriculum management software platform)
A. Hendrix, M. J.C. and Campbell, P. W. (2001), Communicating science: From the laboratory bench to the breakfast table. Anat. Rec., 265: 165–167.
B. Neill US. How to Write a Scientific Masterpiece. The Journal of Clinical Investigation. 2007;117(12):3599–3602.
C. Bourne PE (2007) Ten simple rules for making good oral presentations. PLoS Comput Biol. 3(4): e77.
D. https://www.nih.gov/institutes-nih/nih-office-director/office-communications-public-liaison/clear-communication/plain-language/plain-language-getting-started-or-brushing
E. https://ori.hhs.gov/TheResearchClinic
F. https://www.hhs.gov/ohrp/regulations-and-policy/guidance/exculpatory-language-in-informed-consent-documents/index.html

Online module bank implementation

Digital resource implementation can be optimized when the following five aspects are addressed: 1) awareness of the resources must be raised, 2) activities to get learners started in using the resources must be offered, 3) the resources must support learning of the content, 4) a mechanism for ensuring the completion of the resources should be offered, and 5) the collection of feedback for the formative evolution of the resource [16]. The modules were first designed in 2014 in recognition of these aspects. All module offerings were created to provide content support for competency in project milestone submissions. Over time the menu of modules grew from a total of 38 modules to the current list of 52 modules. Student feedback directed removing repetitive information wherever possible, and additions of requested topics (e.g., questions to ask a potential mentor, research types and innovation, and expansion of statistics topics). Modules were also revised to be as concise as possible, and include multiple modalities of information (e.g., documents, videos, websites). After the 2014 first module bank iteration, a completion attestation statement was added. The objective of this statement along with the monitoring of the time spent of module viewing was to ensure completion of the modules.

During the first two years of module bank usage, each student was required to complete a total of 10 modules elected from the fall theme inventories (mentor selection, research question development, and project proposal) during the fall semester (August – December). Each student was required to complete at least one module from each of the three themes, and the 10 total modules needed to be completed by the end of the semester. During the winter semester of the first two years of module bank usage (January–May), each student was required to complete three modules from the winter theme inventory (IRB and compliance and communicating science) by the end of the semester.

In the fall of 2016, a redesign of the course to further support the just-in-time scheduling of topics and activities resulted in a decrease to seven total modules required in the fall semester. The viewing deadlines for modules within themes were also moved from the end of the semester and aligned with the in-class presentation and active learning exercises relevant to each theme topic. The objectives of this move were to get the learners familiar with the modules and to get them started with integration of the information into the overall course design. The revised requirements are: one module from the mentor selection theme by mid-late August, two modules from the research question development theme by early October, and two modules from the project proposal theme by late November. Each of the module deadlines now immediately preceded an in-class group activity/active learning session building on the respective topic (i.e., Mentor Networking events, Building Specific Aims and Research Question development activity, and Example Project Proposal critique). In the winter of 2017, all winter theme modules were made optional. Students still completed a required CITI Program training component of the course during the winter semester.

Student usage and impressions

To assess the affective component of the module design, student feedback on the online module bank implementation process was collected anonymously through standardized questions administered by the OUWB Office of Medical Education as well as module-specific questions on end-of-course evaluations. Open-ended course evaluation questions requesting students to discuss strengths and weaknesses of the course were also qualitatively analyzed by a single coder for the frequency of references to the online modules.

Students’ perceived value of modules as determined by selection choice was determined by component part-worth. Part-worth was categorized by the factors of length of the module (duration), type of media presentation, and content source. Each factor had three levels of attribution (Table 2). Existing modules were categorized by these factors and levels and the number of times the modules were viewed to was quantified. Evaluation of module worth by students was conducted by conjoint analysis using SPSS software. The Oakland University Institutional Review Board for the Protection of Human Subjects determined this work did not meet the definition of research under their purview according to federal regulations (OU IRB Reference #984,476–1).

Table 2.

Factors and levels for module attributes.

Factors	Levels	Percentage
Duration	Long	26%
	Medium	28%
	Short	45%
Media Type	Didactic.	35%
	Animation	43%
	Mixed Type	22%
Source	OUWB	47%
	External	42%
	Mixed Source	11%

Percentages calculated as all modules that contain the factor at that level. N = 549

Results

Usage of modules was tracked (including date of usage, first viewing and repeat viewings). These data can be aligned with dates of individual student completion of Embark program milestones. In the first iteration of online module deadlines 90% (93) and 71% (89) of the classes (fall 2014; fall 2015 respectively) viewed only the minimum modules required to meet the course assignment by the end of semester deadline. These data informed the recalibration of the module thread deadlines to align with course content and milestone deadlines (fall 2016; fall 2017), so that the students viewing the modules only to meet the course assignments dropped to 29% (35) and 37% (47) respectively (Figure 1). This corresponds to 71% (86) and 63% (79) of the students using the modules above and beyond the course requirements for each of the last two years. Above and beyond is defined as re-watching previously viewed modules and viewing additional modules after the assignment date in a ‘just-in-time’ manner.

Figure 1.

Percent of students viewing modules during the fall semester of their M1 year. Module viewing was tracked. Total percent of students viewing the minimum number of modules for the course assignment (blue) and percent of students viewing more modules than required for the course assignment (beyond the course assignment, red) are shown here. Students viewing modules beyond the course assignment increased after the recalibration of module thread deadlines in the Fall of 2016 (Fall 2014, 10%; Fall 2015, 29%; Fall 2016, 71%; Fall 2017, 63%).

The top three module topics students were most likely to view during their M1 fall semester are listed in Table 3. Even as the modules were revised and the number of module offerings increased, the desire for getting information about finding a mentor and writing the research proposal remained high across the four academic years. These data illustrate the primary concerns of students in the research program during their first year of medical school.

Table 3.

Most preferred modules in the fall semester.

Program Year	Module Title	Number of Times Accessed
2014–2015	Finding a Mentor	124
	Proposal Development	123
	Developing a Research Question	122
2015–2016	Writing a Research Proposal	139
	Finding a Mentor	113
	Developing a Research Question	112
2016–2017	Questions to Ask a Mentor	94
	Writing a Research Proposal	82
	Study Designs	67
2017–2018	Questions to Ask a Mentor	98
	Writing a Research Proposal	68
	Finding a Mentor	63

Throughout the usage of the online module bank, program evaluation feedback was collected through both anonymous in-class survey questions, as well as targeted and open-ended course evaluation questions at the conclusion of each semester. An increase in students Strongly Agreeing or Agreeing to the statement ‘I felt that the online modules format helped me individualize the course material to my capstone project needs’ is observed from the fall of 2014 to the current class of students (Figure 2; 2014–2015, 22 (52%); 2015–2016, 66 (54%); 2016–2017, 84 (71%); 2017–2018, 87 (69%)).

Figure 2.

Student responses regarding online modules and course individualization. Students responded via a Likert scale to the prompt ‘I felt that the online modules format helped me individualize the course material to my capstone project needs.’ The responses were collected from in-class anonymous survey questions (2014–2015, n = 42) and course evaluations (2015–2016, n = 123; 2016–2017, n = 118; 2017–2018, n = 126).

Course evaluations also included two open-ended items. These were: ‘Please list the strengths of this course.’ and ‘Please list ways this course might be improved.’ Comments were qualitatively analyzed for statements related to the online module bank. The introduction of the modules (in fall 2014), as well as the most recent fall semester analyzed (fall 2017) had the highest percentage of open-ended responses describing the modules as a strength of the course (Figure 3, 34 (33%) and 15 (29%) respectively). Identified strengths of the modules include 1) flexibility in choice of content and timing of completion, 2) making information pertinent & efficient, 3) the variety of topics and resources provided, 4) the ability to refer back to information outside of class and in a just-in-time manner.

Figure 3.

Qualitative analysis of student feedback regarding strengths and weaknesses of the course in open-ended course evaluation questions. Total student comments related to modules are shown from ‘Please list the strengths of this course’ and ‘Please list ways this course might be improved’ course evaluation questions. In the ‘could be improved’ category, a larger portion of comments related to modules focused on the request for more content to be moved to modules (gray bars).

The fall of 2014 also had the highest rate of students mentioning the modules in the ‘could be improved’ category (Figure 3, 30 (29%)). Of those students, 26 (27%) of the ‘could be improved’ comments focused on the technology aspects of the modules. There were multiple technology hurdles with the initial release of the modules that needed to be addressed (including browser updates, flash and java requirements, and software updates). For the first iteration of the end of semester online module deadlines, many of the student evaluations offered ‘could be improved’ comments regarding the required module viewing being prior to when the student would be using the content for his/her project. Since the winter 2016 semester, a large portion of the ‘could be improved’ comments have focused on moving more content to modules (winter 2016, 10 (45%); fall 2016, 17 (85%); winter 2017 NA (no ‘could be improved’ comments focused on modules); fall 2017, 4 (50%)).

Modules were created with the intention of providing just-in-time information in a valuable and meaningful way. To determine what attributes of modules students valued most, an analysis was conducted on modules selected to fulfill the milestone of research question generation for the fall 2016 and 2017 courses. This time frame was selected as it was after the reformulation of course requirements. This specific milestone was selected as it was decided that students at the beginning of the semester may have felt overwhelmed by the start of their medical school education. Modules for this thread were categorized based on duration, media type, and source as shown in Table 2. Student choice selections for this thread were then tallied based on the factor and level categorization. The percentage of modules viewed based on the attribute is shown in Table 2.

Next, an analysis of preferred attributes was conducted by conjoint analysis. Conjoint analysis is a statistical method used in market research to determine which attributes of a product consumers value [18]. It was hypothesized that students would only select short modules to fulfill their course requirement. While shorter modules were preferred, the module duration was not the most significant factor in student module selection (Figure 4). The analysis was limited by the fact that the module library was created for educational purposes, not for the purpose of conducting a research analysis, and thus was not distributed equally among the attributes resulting in the possible 27 permutations (3 factors x 3 levels). Seventeen modules corresponding to the research question generation thread were available to students (Table 1). Of those, nine were actually selected by students resulting in a factoral, non-orthogonal post-hoc analysis. The prediction model calculated by the conjoint analysis correlated well with the data observed (Pearson’s R 0.821, p = 0.003; Kendall’s tau 0.778, p = 0.002). According to the analysis, averaged importance values credited to each factor identified duration as being of least importance (12.149), with media type (37.893) and source (49.958) being substantially more important to students. As dependence of the factors media type and source could not be discounted, it can only be said that the length of the modules was of lesser importance to students’ choice selection (Figure 4). The module combination that students most preferred was short animation style modules curated from an external source. These results were supported by the attribute preferences reported as percentages in Table 2.

Figure 4.

The average importance values of student preferred module factor attributes as determined by conjoint analysis using SPSS software. Analysis was conducted on nine modules from the research question development thread available to students during the fall semesters of 2016 and 2017.

The impact of the removal of content from the lecture sessions to the module library did not adversely impact students’ ability to complete the program milestones as end-of-year submission numbers stayed relatively unchanged over time. While students are encouraged to submit a proposal prior to entering the M1/M2 summer break, students are able to submit their project proposals in a self-regulated manner. Prior to the revision of the modules in 2016, an average of 57% of students submitted their project proposals by the end of the M1 academic year. After the revision of the course and increase in module library, an average of 66% of students’ proposals were submitted by the end of the academic year.

Discussion

The perception by both faculty and students that in-class lecture and active learning time was not optimally scheduled in earlier iterations of the research design courses prompted the development of the Embark program online module bank. The program is designed to allow students to pursue independent research projects in many different fields, and at different rates based on individual project specifics [1]. Although there are course topics clearly important to all research fields (e.g., developing a research question, literature searches, and writing a project proposal), there are many others that are likely to be of interest to only select project types (e.g., focus groups, locating existing surveys, and grounded theory). Further, there is limited class time within the busy medical student curriculum calendar, so topics relevant to most projects are only presented at one time during the course. This presents a challenge when trying to deliver a large amount of content in a limited time. Additionally, the timing of topics often does not align with many student’s individual project timelines for when they would need the information. The approach of moving much of the course content to the online module bank provides students with a rich set of resources allowing for individualized and just-in-time learning.

It was found that after realignment of the modules with the in-class presentations and milestone due dates, the students did use the Embark modules as a just-in-time resource. This availability is described by Ruiz, Mintzner & Issenberg as one of the advantages for learners along with adaptability to individual learning styles and ability to promote comfort [19]. They coined these digital objects as Learning Objects (LO). The criteria to be considered a learning object is that the resource should be able to stand alone as a single unit aligned with an outcome as well as be able to be aggregated with additional units. The LO should be usable to diverse users and be interactive. The LO should be able to be used with multiple digital platforms and accessible to users. It is believed that the Embark modules fit this description of a digital learning object.

As with many new educational components, the online module bank implementation was met by initial roadblocks. The initial release of modules included many modules that were longer in length, and faced many technological hurdles. Although students appreciated the initial module bank, as seen in Figures 2 and 3, many students offered feedback that they did not find the modules useful and struggled with the technology issues. The modules were introduced at the start of the course, and discussed throughout the semester’s in-class sessions. However, in the first two years of implementation, it was observed that many students were waiting until the end of the semester to begin viewing the modules, even when they may have individually passed the research milestones for which the modules were designed to support. This disconnect in viewing the modules as simply a course requirement rather than a just-in-time resource prompted the realignment of module thread deadlines in the fall of 2016, as well as significant module redesign.

In the second iteration, modules were shortened whenever possible in order to influence the affective response to usage. Redundant content was removed and the total number of required modules was decreased. Most importantly the deadlines for the module threads were separated and now occur in line with in-class topic presentation. Redistributing the deadlines throughout the semester prevents the previous issue of module viewing only in the final weeks of the semester. It was also found that requiring the students to have viewed their elected modules from a particular thread prior to attending a large group in-class lecture/active learning session on that topic led to increased student appreciation of the topics presented (course evaluations, data not shown). Raising awareness of the digital resources is important when offering a microlearning library such as this. Most interestingly, with the implementation of the second module iteration, there was observed a large increase of students using the modules above and beyond the course requirement, including re-watching the modules (Figure 1). It is believed that introducing the students to the just-in-time content available through the online modules by requiring of a small amount of the content to be viewed in-line with the course calendar has led to a greater appreciation of the resources available and encouraged more students to use the resources in their intended manner.

The analysis of student evaluation comments demonstrates that students want more of the lecture topics to be offered via modules. It is not believed that this is simply due to preference for online instruction versus lecture attendance, as live lectures are recorded and currently available to students in a lecture capture format. Simply offering recorded lectures is not necessarily meeting students’ needs in a similar manner. Modules offer greater detail, although in smaller discreet pieces than can be available in lecture. The benefits of the modules are similar to that of lecture capture in that they offer students active control over their learning [20]. Students can piece together in a self-directed way the information from several modules to formulate a comprehensive unit that specifically serves their needs at a time when they feel it is of use. The challenge with independent instruction is that students have a default view of the information being an assignment as opposed to a resource for further learning as was demonstrated by the data from the 2014 and 2015 cohorts (Figure 1). It is felt that this was solved by aligning the resources to activities conducted in in-class sessions, thereby making the resources more relevant. Additionally, this alignment has provided the psychomotor dimension to the desired learning as students have the opportunity to apply the information in classroom activities as well as in the construct of their research milestone submissions. The idea that course content can be broken down into pieces and delivered via a learning commons modality that can be assembled in various ways seems to be a preferred method of today’s millennial learners [21,22]. In this regard, a similar construct to what was conducted in this research course could be relevant to other courses that would allow students to move through a curriculum at a self-regulated pace. Indeed, Khogali et al. also found module resources to be of benefit to a blended learning paradigm for cardiovascular instruction. In their analysis, they too found that students valued modules that provided video animations from expert sources [4].

There is great benefit in moving much of the research training content from lectures to the online module bank. By doing so, face-to-face time with students can be focused and maximally beneficial to the class as a whole (regardless of their individual project type). Concurrently, protected time was able to be created for OUWB students to pursue their projects in individualized manners, and allowed for the availability of important content to students when they were ready for the topics (either before or after the general course session schedule). The lessons learned in the implementation of the online module bank for research training may be transferable to many educational topics. Moving content from in-class traditional sessions to asynchronously available modules relieves pressure of scheduling course-time, enables individualization of content in a self-directed and self-regulated manner by students, and meets learners where modern society finds them – in a fast-moving, technology-heavy, on-demand lifestyle. When implementing similar technology projects, usage preferences and rates, learner feedback, and effect on appreciation of and ability to use the content are important to frequently monitor.