Exploring the effectiveness of technology-based learning on the educational outcomes of undergraduate healthcare students: an overview of systematic reviews protocol.

INTRODUCTION: Rapid technology development due to the introduction of Industrial Revolution 4.0 and Internet of Things has created a demand and gradual transition from traditional teaching and learning to technology-based learning in higher education, including healthcare education. The COVID-19 pandemic has accelerated this process, with educators now required to quickly adapt to and adopt such changes. The abundance of available systematic reviews has made the effectiveness of such approaches ambiguous especially in healthcare education. Therefore, a protocol of the overview of systematic reviews (OoSR) is planned to extrapolate the effectiveness of technology-based learning in undergraduate healthcare education. METHODS AND ANALYSIS: Scopus, CINAHL, Academic Search Complete, Cochrane Library, MEDLINE and Psychology and Behavioral Sciences Collection databases were selected. Screening was conducted independently by at least two authors and the decision for inclusion was done through discussion or involvement of an arbiter against a predetermined criteria. Included articles will be evaluated for quality using A MeaSurement Tool to Assess systematic Reviews and Risk of Bias in Systematic Review tools, while primary systematic review articles will be cross-checked and reported for any overlapping using the 'corrected covered area' method. Only narrative synthesis will be employed according to the predefined themes into two major dimensions-theory and knowledge generation (focusing on cognitive taxonomy due to its ability to be generalised across disciplines), and clinical-based competence (focusing on psychomotor and affective taxonomies due to discipline-specific influence). The type of technology used will be identified and extracted. ETHICS AND DISSEMINATION: The OoSR involves analysis of secondary data from published literature, thus ethical approval is not required. The findings will provide a valuable insight for policymakers, stakeholders, and researchers in terms of technology-based learning implementation and gaps identification. The findings will be published in several reports due to the extensiveness of the topic and will be disseminated through peer-reviewed publications and conferences. PROSPERO REGISTRATION NUMBER: CRD4202017974. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

In light of the global COVID-19 pandemic, it is important to assess the effectiveness of technology-based learning in comparison with traditional teaching among undergraduate healthcare students.

To the best of our knowledge, this is the first overview of systematic reviews that elucidate the effectiveness of technology-based learning across a wide range of healthcare programs.

The overview of systematic reviews protocol describes a robust and rigorous methodology that could become a good point of reference for other researchers to adopt.

One of the limitations of the overview of systematic reviews is the lack of granularity of information provided by the authors.

Heterogeneity is expected from this overview of systematic reviews; therefore, grand meta-analysis is not being considered.

Introduction

Technology has evolved since the modern day of human civilisation from mechanical advancements to the electrical age, to the digital age and now, internet networking. Currently, Industrial Revolution 4.0 (IR 4.0) and Internet of Things (IoT) emphasise on internet and automation technology becoming a norm in daily human life.1–3 Many traditional and manual technological approaches have become obsolete and archaic and are no longer relevant. As technology has become an integral part of human life, its significance is even more so apparent in the education sector. The traditional approach of teaching, such as manual face-to-face lectures, rote learning, class attending, and manual practice has transitioned towards more dynamic and interactive learning with the use of technology to facilitate learning activities.4–7 Similarly, technology is now being incorporated into medical and nursing education for knowledge and/or clinical teaching.8 Taking the learning dynamics and the preferences of net-generation learners (digital native learners) into account, technology can indeed be used in teaching and learning to benefit the students and educators.4 9–11

Although technology application in the education sector of developing and underdeveloped countries is still a luxury and is mainly dominated by traditional teaching approaches,12–14 its requirement and relevance has never been more apparent than during this current COVID-19 pandemic. COVID-19 has become a global pandemic. It has interrupted daily human activities at an unimaginable scale; causing changes in behaviour, social aspects and overall lifestyle.15 16 With no medication and/or vaccine for COVID-19, the best approach is to rely on preventive measures such as by practising social distancing and good hygiene.17–20 On a larger scale, some countries have implemented movement restrictions on their citizens in an effort to break the chain of infection.20–22 Some professional representative bodies have suggested using technology-based learning (TBL) to observe social distancing to minimise such contact, especially among students.23 24 This measure has directly impacted the higher education teaching and learning ecosystem in universities and colleges. In turn, a new trend is created where technology-based globalised, borderless, and seamless teaching and learning may become ‘the new normal’.25–33

TBL can provide a safe and secure environment for learning, besides reducing the burden on educators, and enabling the catering to mass groups irrespective of time zone or geographical location. However, it is important to note that the acceptance and effectiveness of TBL is still contented in comparison with traditional didactic lectures. For example, this method cannot provide a real-life context for student survival after graduation. It also requires the educators to put in more effort for material preparation. Besides, the technology might be unaffordable. Other drawbacks include incompetent educators, unstable, unreliable technology and data providers, and the digital divide.13 14 34–38 Additionally, medical and healthcare practitioners are now giving precedence to practice based on the best evidence, including teaching and learning aspects, known as BEME.39 However, there is a dearth in studies evaluating the implementation of TBL and whether it is effective, feasible, acceptable, or potentially reasonable to complement, replace or substitute traditional teaching.

To the best of our knowledge, no overview of systematic reviews (OoSR) has been conducted or registered in the International Prospective Register of Systematic Reviews (PROSPERO) similar to this project. Two overviews were identified related to TBL,40 41 however there are gaps that were not addressed. First, both overviews solely focused on nursing education with findings that may be less applicable to other healthcare professional education, as nursing education emphasises on discipline-specific skill acquisition. Second, an overview by Rouleau et al 40 focused on post-professional nursing education but pedagogy and andragogy concepts for pre-professional and post-professional education are different.42 Mature students are more independent, self-reliant, and have more initiative compared with younger students who require class attendance and require frequent contacts with peers and teachers. Third, the overview by Cant and Cooper41 focused only on clinical-based outcomes and not technology-based education. Fourth, the overviews were specific to certain types of technology; thus, these studies are unable to provide a comprehensive picture or cover a wide range of educational technology applications. Additionally, a scoping systematic review was found in the literature but it is only limited to the use of handheld devices among healthcare professionals.43 Therefore, another overview of systematic reviews is required to comprehensively synthesise the best available evidence to determine whether or not technology-based teaching and learning is effective. That is, whether it can substitute or complement traditional teaching and learning approaches in healthcare education from a variety of aspects, such as, but not limited to, improving knowledge, critical thinking, engagement, clinical skills, and affective and mindful learning.

Methods and analysis

The OoSR is also known as an umbrella review or a systematic review of systematic reviews. It is a methodology that can be used to assist readers to work through the outcomes of related systematic reviews by collecting, disseminating, synthesising and harmonising its conclusions to guide practice.44 One of the purpose of OoSR is to catalogue available research syntheses pertaining to the topic of technology-based education in healthcare and harmonising the evidence outcome for recommendation of practice and future research. Therefore, an OoSR protocol is required to cater to upcoming reports.

To meet the above objective, we followed the existing framework given by Cooper and Koenka.44 The framework outlines seven steps: (1) formulating the problem, (2) searching the literature, (3) gathering information from syntheses, (4) evaluating the quality of evidence, (5) analysing and integrating the outcomes of synthesis, (6) interpreting the evidence and (7) presenting the result.

Formulating the problem

Technology-based teaching and learning is vast and complex to define. There are no standard criteria to capture the overall interpretation of technology-based approaches. TBL is defined as learning with the means of electronic technology that encompasses the internet, the intranet, satellite broadcasts, audio and video conferencing, bulletin boards, chat rooms, webcasts and compact disc read-only memory (CD-ROM).5 45–47 Other related terms associated with TBL are e-learning, online learning, digital learning, and web-based learning, among others.

There are numerous TBL approaches.8 48 TBL in healthcare education covers simple to sophisticated technology to intangible and tangible technology. For instance, simple TBL uses CD-ROM or PowerPoint presentations, while examples of sophisticated technology include virtual synchronous online classes and 3D holography for anatomy training. Furthermore, TBL can also take the form of intangible technology, such as augmented or virtual reality, while tangible technology could be presented as a simulated high-fidelity manikin. The availability of these technologies has been beneficial in advancing medical education and enhanced learners’ understanding due to its interactive nature, while at the same time presenting new challenges to educators as there is an abundance of available technologies that make it difficult to choose the best possible technology to implement in teaching and learning.49 50

Therefore, it is paramount that educators are well informed on the available technologies and the proper application of specific technology in targeting specific skills in learning. Referring to a systematic review is one of the best approaches because a systematic review pools similar technologies and investigates its effectiveness. However, systematic reviews have limitations, such as being homogeneous and focusing on a specific objective. Therefore, educators must search other systematic reviews to supplement remaining objectives.

A preliminary search for this OoSR project using the keywords “systematic review”, “e-learning” and “technology” yielded thousands of articles. This result indicates that there are presumably heaps of systematic reviews available that are related to TBL. Searching and reading separate and multiple literature works would be time consuming and require a lot of effort. Educators are also busy with other responsibilities, so with the many literature works available, all the studies may not be read comprehensively (ie, teachers may only be able to read the abstract).51 This may lead to a lack of comprehensive understanding on the efficacy of the technology for education purposes. In most case scenarios, educators tend to apply certain technology based on prejudices, hunches, opinions and guesses, which assume that the technology is beneficial for teaching and learning.52 Such an assumption may be detrimental because the technology might be costly, effort or time consuming, not fully accepted in practice, and could have a high chance of being abandoned.53–55 Hence, it is important to apply the concept of BEME even in TBL.52 Therefore, conducting an OoSR for TBL may benefit educators, as it can serve as a one-stop point of reference.

To ensure that this OoSR project comprehensively encapsulates all possible angles and aspects of TBL, a research objective was developed in accordance to the Population, Intervention/Exposure, Comparison and Outcome (PICO/PECO) concept.56 Table 1 details out the criteria of each PICO. Developing a detailed OoSR objective is important to ensure the methodology is properly developed and explicit, and the overview is focused, manageable and systematic.57 Therefore, the objective of this OoSR is ‘to explore the effectiveness of TBL in improving the educational outcomes of undergraduate healthcare students’.

Table 1

Research objective developed using PICO

Population	Undergraduate healthcare students
Intervention	Technology-based learning
Comparison	Non-technology-based or conventional/traditional technology
Outcome	Educational outcome—effectiveness in cognitive, psychomotor or affective domains

Searching the literature

This OoSR was carried out by conducting a search on several electronic databases subscribed by Universiti Putra Malaysia. The electronic databases were selected by discussing among the research team and finding a commonality in the electronic databases used in the topic. First, several published systematic reviews on the topic of TBL in healthcare education were explored. Then, the research team generated some keywords. The relevant keywords were initially identified from the researcher’s knowledge, selected published systematic reviews and by searching the synonyms of the terminologies using Google. Second, the pooled keywords were scrutinised and selected from the discussion among the research team. Third, the selected keywords were tested in the search engines and further customised, expanded or removed until an optimal, manageable number of articles were retrieved. Boolean operators, wildcards, exact, truncation and other commands were used whenever appropriate. Relevant Medical Subject Headings terms identified during the search were included. Table 2 lists the final keywords and search strategy used in this OoSR. If manual search was required, the reference list of the included articles would first be reviewed and any relevant citation to be extracted and screened is then identified, including any relevant article in the possession of, known, or encountered by the researchers during the project duration. The initial electronic databases searching was conducted on 20 March 2020 on six electronic databases. A contingency effort will be taken to regularly update the searching until the acceptance of our OoSR.

Table 2

Search strategy keywords, modified as needed for the electronic databases

(Search engine) Database	Keywords
(Scopus) Scopus	(TITLE-ABS-KEY (“medic*” OR “health” OR “health science*” OR “nurs*” OR “biomedic*” OR “pharma*” OR “nutrition” OR “dietetic*” OR “dental” OR “dentist*” OR “allied health” OR “occupational health” OR “environmental health” OR “occupational therap*” OR “physiotherap*” OR “phsyical therap*” OR “speech therap*” OR “speech language phatolog*” OR “occupational safety” OR psycholog* OR “audiolog*” OR forensic* OR “radiotherap*”) AND TITLE-ABS-KEY (student* OR “undergrad*” OR “universit*” OR “college” “higher education*” OR “tertiary education*”) AND TITLE-ABS-KEY (“e-Learning” OR “e-learning” OR “online” OR “web based” OR “blended” OR “internet” OR “computer*” OR “mobile” OR “simulation*” OR game* OR gamification* OR “MOOC*” OR “e-content” OR “digital” OR “virtual” OR “electronic” OR “computer-based learning” OR “online” OR “technology enhanced learning” OR “augment*” OR technolog* OR “hybrid learning” OR multimedia OR “massive open online course” OR “m-learning” OR moodle OR “flexible” OR “e-pedagogy” OR “social media” OR iot OR “Internet of Things”) AND TITLE-ABS-KEY (“systematic review”))
(EBSCOHost)CINAHLAcademic Search CompleteCochrane Database of Systematic ReviewsMEDLINEPsychology and Behavioral Sciences Collection	AB (“medic*” OR “health” OR “health science*” OR “nurs*” OR “biomedic*” OR “pharma*” OR “nutrition” OR “dietetic*” OR “dental” OR “dentist*” OR “allied health” OR “occupational health” OR “environmental health” OR “occupational therap*” OR “physiotherap*” OR “phsyical therap*” OR “speech therap*” OR “speech language phatolog*” OR “occupational safety” OR psycholog* OR “audiolog*” OR forensic* OR “radiotherap*”) AND AB (student* OR “undergrad*” OR “universit*” OR “college” “higher education*” OR “tertiary education*”) AND TI (“e-Learning” OR “e-learning” OR “online” OR “web based” OR “blended” OR “internet” OR “computer*” OR “mobile” OR “simulation*” OR game* OR gamification* OR “MOOC*” OR “e-content” OR “digital” OR “virtual” OR “electronic” OR “computer-based learning” OR “online” OR “technology enhanced learning” OR “augment*” OR technolog* OR “hybrid learning” OR multimedia OR “massive open online course” OR “m-learning” OR Moodle OR “flexible” OR “e-pedagogy” OR “social media” OR IoT OR “Internet of Things”) AND AB (education* OR learn* OR teach* OR study* OR train*) AND “systematic review”

Gathering information from syntheses

The third step involves a screening process of the retrieved articles from the systematic searching. The screening process of the articles will be based on predefined inclusion and exclusion criteria. The criteria developed will be categorised into two: standard and tailored. The standard criteria will be developed by consulting previous reviews, while the tailored criteria will be purposely defined for this project. All the criteria developed will be agreed on by the research team. The predefined criteria must be aligned with the OoSR objective to minimise the mismatch scope of the included systematic review with this overview.58

The following section lists the operational definition of the inclusion criteria in more detail:

Undergraduate healthcare students: only field of study under the mainstream medical and health sciences education will be considered, such as medicine, biomedical sciences, pharmacy, nursing, dentistry, nutrition and dietetics, occupational and environmental health, and forensic, etc. Other non-mainstream allied health field of study, such as complementary and traditional medicine (ie, chiropractic, acupuncture, homeopathy) will not be considered. In addition, undergraduates in this OoSR are referred to as entry-level or pre-registration education programme, which abide with the minimum requirement for practice. This group includes Bachelor’s degree and in some countries entry-level Master’s or Doctoral programmes (eg, Doctor of Occupational Therapy, Doctor of Physiotherapy), as the differences in skill and knowledge are minimal.59 60 Any postgraduate education programme (eg, Master of Surgery) or post-professional programme (eg, post-basic training, continuous medical education (CME)) will not be considered. The reason for postgraduate or post-professional training not being included as this group could have prior fundamental knowledge pertaining to the subject that may make it easier for them to grasp new knowledge,61 62 while experience equips the participants with better clinical reasoning skills.63 Hence, such a situation may pose an advantage to post-professionals when it comes to acquiring understanding as compared with undergraduates who have minimal information on the topic. Therefore, biases could result where the outcome may be compromised, and the improvements to the educational outcomes may not be solely due to intervention. Therefore, this group will be considered only when at least more than half (≥50 %) of the primary studies included in the investigated systematic reviews involve undergraduate or pre-professional training.

TBL: technology in this OoSR should abide with the concept of IR 4.0 or IoT.64–67 Therefore, archaic, low or common technology, such as PowerPoint presentation-only lectures, manuals or mechanical equipment, such as low-fidelity manikins and gamification concept teaching not involving sophisticated technology, such as social and cooperative games, will not be considered eligible. Simulation-based studies that do not use technology, such as real patients, standardised patients or healthy individuals, will not be considered. Also, the use of technology as a medium for assessment and evaluation only, such as for objective structured clinical examination and final examination, will not be considered eligible.

Effectiveness: effectiveness in this OoSR refers to the teaching effectiveness using technology-based approaches. Therefore, teaching effectiveness is defined as the capability to produce gains in student achievement, taking account of a baseline measure of student prior attainment and other characteristics of student intake. The achievement can take the form of improvement, among others, but is not limited to assessment score, knowledge, understanding, critical thinking skills, personal attributes, performances, teamwork and collaboration, or communication skills.68 69 Effectiveness will also take into account the cognitive, psychomotor and affective taxonomies in terms of knowledge generation and clinical-based competencies.

Systematic review: the usage of the term ‘systematic review’ either in the title or the full text is insufficient to be classified as a systematic review.70 A systematic review is defined as a type of review that conducts systematic searching and applies a screening process of the literature reviewed against predefined criteria, where each included study (either in tabular or narrative format), and the quality analysis of the included studies are conducted and reported.

The exclusion criteria use standard criteria found in review studies, such as no full text available after the research team has exhausted all possible options to retrieve the full text, such as downloading from the library, searching in online deposits (ie, ResearchGate, Academia), contacting the author and the inability to purchase the article; available in grey literature format (ie, theses, books, conference abstract, reports) as such articles have not undergone rigorous peer-review process; other types of reviews (ie, scoping review, umbrella review, literature review), as these reviews do not abide with the systematic review methodology; and non-English articles, as the research team has limited capacity to understand languages other than English and have limited resources to subsidise translation services. The decision to not include non-English articles will have minimal impact on the outcome of the OoSR.71

The screening process will be recorded using Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA).72 Example of the PRISMA flowchart is shown in figure 1. Screening will be conducted independently by at least two researchers in each stage of the title and abstract, and full-text screening. The screening process will be recorded using a Microsoft Excel document according to the predefined inclusion and exclusion criteria. EndNote V.X8 software (Thompson Reuters) will be used to manage the citations.

Figure 1

PRISMA flowchart for screening process. PRISMA, preferred reporting items for systematic review and meta-analysis.

Pre-consensus agreement will be obtained based on two points; (1) after the abstract screening and before proceeding to full-text screening, and (2) after full-text screening and before synthesising the included articles. The type of agreement analysis depends on the availability of screening reviewers at each stage. Fleiss kappa will be used if more than two reviewers screened the same articles, while Cohen kappa will be used if the screening only involves two reviewers.73 The analysis will be conducted on the whole list of articles; not only on the accepted pre-consensus. The interpretation of the kappa agreement is based on Landis and Koch,74 where a kappa value of less than 0 is considered poor, whereas a slight agreement is indicated by a value between 0.01 and 0.20, followed by fair is (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80) and almost perfect (0.81–1.00). Percentage agreement will be used to supplement the agreement value as the kappa value is known to fluctuate and is sensitive to the characteristics and sequences of the ratings.75 76 Percentage agreement is calculated based on pre-consensus agreed and accepted articles divided by the total articles reviewed during a particular stage. Discussion will be conducted on disagreed articles between the involved reviewers and any unresolved disputes will be resolved by a third reviewer acting as an arbiter.

A matrix table using the Garrad’s Matrix Method77 will be implemented to extract the data from the included articles. Commonly, the pieces of information extracted are citation (ie, authors and year), objective of study and review conclusion. Other pieces of information will be extracted included characteristics of the participants, databases used in the review, number of included studies and quality assessment used. At least one researcher that is not involved during the screening process will be responsible for extracting the information. The extracted information will be cross-checked and verified by researchers involved in the screening process. Subsequently, the researchers will meet and discuss the comprehensiveness and explicitness of the extracted table. Primary studies in the included reviews will be extracted and compared to determine any overlapping evidence, as suggested by Pieper et al 78. Considering overlapping studies is important to meticulously develop the conclusion and not to overestimate the overview.

Evaluating the quality of evidence

A MeaSurement Tool to Assess systematic Reviews (AMSTAR) will be used in this OoSR project. The AMSTAR tool is considered the ‘gold standard’ instrument for assessing the quality of systematic reviews and is among the most widely used assessment tools.79 AMSTAR has 11 items scored dichotomously as ‘Yes’, ‘No’, ‘Can’t Answer’ or ‘Not Applicable’. The total score is calculated by adding the YES responses. The total score for AMSTAR ranges from 0 to 11, where a higher score indicates a better quality review. An article is considered of good quality if the score is between 9 and 11, followed by moderate quality (5–8) and low quality (0–4).80 AMSTAR has established adequate validity and reliability.81 82

Risk of Bias in Systematic Review (ROBIS) aims to evaluate the level of bias present within a systematic review.83 ROBIS consists of four domains, namely (1) study eligibility criteria, (2) identification and selection of studies, (3) data collection and study appraisal, and (4) synthesis and finding. ROBIS is rated based on the judgement of the assessor either as low, high or unclear on each domain based on the signalling questions provided. The final judgement on the overall level of risk of bias will be made considering the results of the four domains. ROBIS has an acceptable level of validity and reliability.83

Prior to the AMSTAR and ROBIS assessment, a pretest study will be carried out to evaluate the validity and reliability of the assessment tools. An inter-rater reliability is planned; where one researcher with experience in publishing overview, experience in using AMSTAR and experience providing training in conducting an overview study is considered as a gold-rater, while at least two inexperienced researchers in overview study will be recruited as comparison-raters. Articles that are accepted during the screening process will be sampled and randomly selected using a computer-generated application (https://www.randomizer.org/). At least 10% of the total included articles or 10 articles, whichever higher, will be selected for inter-rater reliability testing. The sample articles will be independently rated on AMSTAR by the gold-rater and the comparison-raters. The rating will then be compared and calculated for agreement in the percentage format and based on Cohen’s kappa analysis. It is hypothesised that the rating should achieve an overall average of 80%. If the cut-off value is achieved, then, the rating of the remaining articles can be done independently by one rater. If not achieved, then, a discussion will be held between the raters to resolve any confusion and to reach a consensus, and additional training would also be provided. At that juncture, the remaining articles will be rated independently by one rater and verified thereafter. A concurrent validity will be conducted between these two tools (ie, AMSTAR and ROBIS) as none of the research team members are experienced in using ROBIS. All included articles will be selected to be tested for the concurrent validity. These two tools were previously used in unison and compared for reliability and construct validity.83

Analysing and integrating the outcomes of synthesis

According to Cooper and Koenka,44 there are several methods available to assist in integrating the outcomes of synthesis. The methods include from the most robust to the least robust are as follows: (1) grand meta-analysis, (2) pool and synthesise primary studies, (3) accept latest publication conclusion, (4) accept the highest quality review conclusion, (5) count the evidence, and (6) simply ignore and report the findings. It is also possible to combine several methods for synthesis purposes. Grand meta-analysis or pool and synthesise primary studies are not suitable for this OoSR as the topic investigated is expected to have a high level of heterogeneity because it involves various types of technology available. Besides, various outcomes will be observed and innumerable amounts of studies are anticipated. Accepting the latest publication is less suitable because it may not be representative for every technology investigated. Therefore, this OoSR will consider to employ the methods of selecting reviews with high quality (with moderate and high quality with AMSTAR and ROBIS) and calculating the available evidences in supporting or refutation of the use of technology.

Interpreting the evidence

Themes will be generated to focus on organising the synthesis into a cohesive narrative and meaningful summary. The themes will be based on the outcomes and grouped per Bloom’s learning taxonomy model: (1) cognitive, (2) psychomotor and (3) affective.84 The cognitive domain defines knowledge classification and understanding of theories and information. Thus, the cognitive domain is liberal, and its application transcends a wide spectrum of population. Therefore, the findings on cognitive domains can be generalised across disciplines. The psychomotor domain focuses on physical skills or tasks development, so it is more discipline-specific and its exclusive application in one discipline may not be applicable to others. Meanwhile, the affective domain is based on perspective, attitude and values, which are much more subjective in measurement and tailored to individual personality; also, it is difficult to be generalised across disciplines. Although the three domains are interconnected,85 its separation during the analysis for reporting is required, as the approach for each domain will differ. Therefore, the synthesis of the summary will be divided into two major dimensions: theory and knowledge generation, and clinical-based competencies.84 86 Moreover, the TBL technologies used will be identified and extracted from the included systematic reviews. The technology will be grouped and listed into two categories, such as (1) internet or application-based, and (2) hardware technology that are either commercially available or purposely developed technology.

As this OoSR will provide a holistic and comprehensive scenario on the effectiveness of TBL in healthcare education, it will also provide the best platform to find overarching gap(s), that is, an absence of evidence from the current available literature. This is in lieu with the suggestion by Paton et al 87 which requested researchers to dig deeper and find a unique gap available to conduct further distinctive research and contribute towards richer knowledge development. Therefore, this OoSR will strive to search for that originality and comprehensiveness.

Presenting the result

There is currently no gold standard guideline available in reporting an OoSR; such a guideline is still under development.58 88 Therefore, the research team will abide with Cooper and Koenka’s44 methodology as a foundation for conducting the OoSR and will be enriched with other literature deemed valuable for producing the highest reporting quality. The abstract of the OoSR will be written according to the Preferred Reporting Items for Overview of Systematic Reviews abstracts outlined by Bougioukas et al 89 Ensuring a robust yet explicit abstract is critical, as the abstract is an important source of information that practitioners and researchers will initially read and rely on.51 The full text will be presented as outlined in this protocol. The outcome of this OoSR project is aimed for several journal publications and conference presentations.

Discussion

An inestimable number of systematic reviews are available; over 40 000 systematic reviews are registered in the PROSPERO alone since its inception and more than a dozen systematic reviews are published daily.90–92 This situation results in overwhelming numbers of available systematic reviews. Hence, OoSR can collect and synthesise the evidence as a whole to create concrete findings.91 93 A similar situation is also observed with TBL. The rapid development of technology has contributed towards abundant technology development for education purposes and in turn has resulted in many systematic reviews produced pertaining to this topic. However, systematic reviews have limitations where some studies are too homogeneous or focused on a single aspect of either one type of population or a specific technology or outcome. Another drawback is that a certain level of skill is required to retrieve suitable and appropriate articles. Therefore, OoSR could facilitate educators to be quickly informed and to become on par with the current available evidences on a particular topic.

Nonetheless, OoSR is not without limitations. OoSR is still considered a new and emerging methodology93; therefore, the available guideline is yet to mature. Integrating several systematic reviews may result in the combination of too many primary studies that are not related. Therefore, the findings may become too generic and diluted.93 Another huge challenge is the reporting of overlapping studies; where the same studies could be included in multiple systematic reviews and thus overestimating its effect.58 78 91 However, the overestimation in this OoSR will have little impact and harm due to the nature of the topic and because it is controlled by calculating the overlapping value.78 Systematic review alone is a secondary research and requires between 6 months and 2 years to be published; therefore, the evidence may become outdated.91 Then again, OoSR may also suffer greater deficiency in recentness, as it pools systematic reviews.58 However, our protocol will implement regular updates on the searching to overcome this issue. The latest and most recent systematic reviews found will be included in this OoSR, thus ensuring that the abreast gap is kept to a minimal. Additionally, we will also continuously keep track of the list of original publications pertaining to the topic. Any critical and important original study will be synthesised in the body of introduction and discussion sections in our OoSR.

Expected significance of the study

This OoSR is expected to provide valuable information regarding the status of TBL either as a main education medium or maintain its use as a complementary approach alongside conventional/traditional educational teaching and learning among healthcare students. The comprehensive and immediate information will benefit stakeholders, such as future undergraduates, educators in higher education, higher education industries and policymakers. Undergraduate students currently in higher education consist mainly of Generation Z and in a couple of years, Generation Alpha will make up the bulk of students. These groups of learners are considered as technology natives and technology savvy.94–96 They heavily rely on the use of technology in daily activities. Generation Z, although still requiring face-to-face lectures or meetings with the lecturer physically for guidance, however, prefer more TBL to be incorporated, to have greater ownership and flexibility in learning.97 Generation Alpha, on the other hand, are expected to be totally connected digitally and are very comfortable with the highly innovative and free lifestyle. They tend to challenge conventional views in every aspect and as such consider the current model of education as archaic and impractical.98 99 Therefore, the higher education sector should prepare for a revolution in teaching beyond the border of institution locality and brick-and-mortar setting. Educators will need to consider the implications of the needs of these generations as they develop curricula and plan for meaningful lesson experiences.

The findings of this OoSR would prove to afford valuable insights that will inform policymakers and the higher education sector of the need of a paradigm shift to cater to such changes and to look at the effectiveness of TBL in future education. This initiative is important because the higher education sector will not only compete among higher education providers to attract pupils (e.g., prospective students), but also between industries such as employment sectors, and the self-employed that rely on the online business model, working without restrictions in terms of working hours and office space.100 For educators, this OoSR will equip them with available TBL approaches that help reduce the generational gap to make the teaching and learning activities more meaningful.94 96 In addition, this OoSR may provide a comprehensive view on TBL and identifying gap(s) that will inform future research in this field.

The goal of ensuring best evidence practice in medical and health sciences education is to promote learning, enhancing clinical practice and improving client outcome.39 This action will indirectly benefit the clients (ie, patients) as the implementation of best education approaches will produce highly competent practitioners. Learning through technology facilitates towards a more dynamic and supportive learning environment that helps enhance understanding and enculturates continuous learning on the current theories and best practices.8 In addition, the application of technology in healthcare education may ensure both client and practitioner safety, as students can practice in a safer and controlled environment with reduced contact, and the students can repeat any action without worry or risking the patients’ life.8 39 Therefore, this OoSR may contribute towards the above objective.

Ethics and dissemination

This research does not require ethical approval, as the study is a systematic review of published literature. Any changes to the current protocol will be considered protocol amendment, and this will be communicated to the journal, along with a motivation and justification for the protocol amendment. We aim to submit the results of this OoSR to several peer-reviewed journals.