Could a massive open online course be part of the solution to sport-related concussion? Participation and impact among 8368 registrants.

OBJECTIVES: A massive open online course (MOOC) has the potential to help address the public health burden of concussion across all levels of sport and leisure activities. The main objectives of this study were to document the volume of participation and to estimate the impact of a MOOC on concussion protocol implementation.
METHODS: Between April 2016 and October 2018, four editions of a French-language MOOC on concussion were presented. Each of the six modules contains a section presenting the main learning content and a section proposing a reflective process to support the implementation of a concussion protocol in the participant's environment. The proportion of registrants who achieved successful completion of the course was the main outcome. Surveys were also used to document the types of participants and their intent to implement or update a protocol.
RESULTS: Thirty per cent of 8368 registrants successfully completed the course. Of the 3061 participants who completed a survey about their background, 58.8% were healthcare professionals, 16.3% were sport or school stakeholders, and 10.1% were parents or persons who sustained a concussion. Of the 1471 participants who completed a survey about their intent to implement or update a concussion protocol in their environment, 39.4% answered positively.
CONCLUSION: This study describes the first use of a MOOC to address the issue of concussion. The experience of a French-language MOOC shows promising results supporting the use of this innovative educational strategy as part of the solution to the public health issue of concussion. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The completion rate documented suggests that a massive open online course (MOOC), as a knowledge transfer modality, can help to address the public health burden of concussion.

The high satisfaction rate suggests that the educational strategies used within this MOOC were relevant to the participants.

A MOOC on concussion can virtually bring together stakeholders from a broad range of perspectives.

By increasing public awareness and supporting sport and school settings in the implementation of proper concussion prevention and management, a MOOC in concussion can reduce the healthcare burden of concussion and facilitate the proper implementation of the clinical counselling provided following this injury.

Introduction

Concussion as a public health issue

Media attention regarding the catastrophic complications of sport-related concussion (SRC), such as the injury suffered by Zackery Lystedt in 2006,1 and recurrent concussion in high-profile athletes contributed to the identification of SRC as a public health issue. A recent analysis of ambulatory consultations between 2003 and 2013 documented a fourfold increase of reported diagnosis of concussion.2 Increased awareness and competencies within the healthcare professions are among the factors that might explain this increase in reporting. Yet, this epidemiological data most likely still represent an underestimation of the true incidence of concussion. Accordingly, further efforts are warranted to develop awareness around concussion for healthcare professionals, in sport and school environments, and in the general population.

One of the major initiatives to address the issue of SRC is the ongoing process of integrating knowledge into consensus statements and recommendations such as the Berlin Consensus on Concussion in Sport.3 The implementation of such recommendations has been beneficial at the highly resourced and structured elite levels of sport participation. However, their implementation across levels of participation, from grassroots to elite, with proper consideration for the broad spectrum of sport and leisure activities, environments (eg, school-based, non–school-based), available resources and access to qualified health professionals likely remains a challenge. This situation is illustrated by the fact that, following the Berlin consensus, the Canadian national strategy on concussion has focused most of its dissemination efforts at the elite level of national sport organisations.4 This allowed the development of sport-specific concussion management strategies that will hopefully facilitate further efforts at other levels of participation.

One remaining challenge is to achieve the implementation of best practices at every level of sport and leisure participation. This will likely rely on multifaceted strategies. Part of the solution might include legislation, such as the Lystedt laws adopted in the 51 US jurisdictions.5 Data on the implementation of these laws indicate that, although they contributed to increase awareness, measures of enforcement remain largely non-existent.1 6 In line with these findings, amendments have already been adopted in several States.7 In Ontario (Canada), a policy that outlined formal requirements about concussion in high schools was implemented in 2015.8 However, very limited information regarding the impact of this policy is available.9 More recently, the first legislative requirement related to SRC in Canada was adopted by the Ontario legislature.10

Challenge of global implementation

A key characteristic of additional implementation strategies should be to support a reflective process that prompts consideration of the challenges of knowledge transfer in a broad range of environments.11 Reflective learning, or the acquisition of knowledge with reflection about its context of application, is acknowledged as an important component of education on health-related issues.12 Relevant considerations for knowledge implementation related to SRC include the characteristics of the environment and sport, identification of barriers and tailoring of the implementation strategy.13

One key consideration was captured in a recent joint statement from the national family medicine and sport medicine organisations in Canada: “… key aspects of concussion prevention, detection and management occur prior to as well as after the initial medical intervention. … Physicians with expertise in sport and exercise medicine can contribute to these strategies by working in conjunction with families, schools, sports organisations, employers and governments to educate, support and empower the implementation of proper concussion prevention, detection and management protocols”.14 “Empower” is a key word in this statement because the availability of qualified healthcare professionals to directly manage the day-to-day aspects of concussion management in the sport environment, at home and through the return-to-school process will always be limited to all but the most resource-rich organisations. Outside of such environments, many aspects of concussion prevention, detection and management will depend on stakeholders in the sport, family and school environments. Multidisciplinary collaborations between healthcare professionals that are defined by the terms of medically supervised protocols have been proposed as a potential solution.15 For example, if timely access to medical experts on concussion is limited, a protocol could define how a school personnel, nurse or team therapist can play a role in the management of concussion, including timely referral for medical evaluations.

A potential strategy to improve access to concussion education for every possible stakeholder is the use of a massive open online course (MOOC). The term MOOC was used for the first time in 2008 and, over the last decade, with the availability of the World Wide Web, MOOCs have become a knowledge dissemination strategy used by many major universities.16 MOOCs are defined by several key characteristics: (1) a topic of broad public interest; (2) free and open registration; (3) online, self-paced delivery; (4) interactive educational support; (5) formal assessment.16 This strategy has been used in the field of sport and exercise medicine to promote the benefits of physically active lifestyles17 but, to our knowledge, had not been used to address the issue of concussion. By providing free access to education outside of formal educational programmes, MOOCs have the potential to increase the health literacy of the public and provide an innovative teaching model for learning focused on patient-centred and family-centred care.18 19

Accordingly, the assumption driving the development of a MOOC on concussion by Laval University (Quebec, Canada) was that it would successfully create a learning environment that brings together participants from every stakeholder group concerned with concussion regardless of the context of sport and leisure participation. The initial four iterations of the MOOC on concussion have been presented between April 2016 and October 2018. The current study had two main objectives: (1) to document the volume of successful participation and (2) estimate the impact of the course on protocol implementation. The analysis also provided insights regarding how to optimise the integration of an evaluation strategy in future adaptations of the MOOC.

Methodology

Structure of the MOOC

The MOOC was developed in French language on the ‘MonPortail’ learning and content management platform developed and owned by Laval University (Quebec, Canada).20 Table 1 summarises the reference framework used to develop the educational structure of the MOOC. This framework is based on the recommendations of the Canadian Concussion Collaborative (CCC) regarding the objectives of a concussion management protocol (CMP).15 A scientific committee of eight persons representing relevant patient and public stakeholder groups reviewed and approved the objectives of the course (for details, see online supplementary additional material). Each of six modules includes two main sections: (1) the primary learning content and (2) an optional reflective process to support the adaptation of a CMP to each participant’s context.

Table 1

Structure of the MOOC on concussion

Module 1	Introduction	Concussion and the spectrum of traumatic brain injuries
Module 2	Prevention	An environment that will minimise the incidence of concussions
Module 3	Detection	An environment that will optimise the early identification of possible concussions
Module 4	Management	Optimised early management of concussed athletes in all spheres of their life
Module 5	Access to care	Timely access to proper expertise and tools that can guide the gradual return to normal activities and minimise the chances of persistent consequences of concussions
Module 6	Review and dissemination	A periodic process for protocol review and a communication/education strategy that will keep all stakeholders involved and informed

For more details on the topics addressed by each module, see the detailed course description in the online supplementary file 2.

MOOC, massive open online course.

This course was developed as part of the Laval University ongoing MOOC programme21 in areas of expertise, such as concussion, where a MOOC was felt to have potential for success. The course was offered four times between April 2016 and the fall of 2018, and relevant new developments (eg, Berlin consensus3 and Canadian guideline on concussion in sport4) were integrated prior to each edition. Returning participants can therefore use this course as a resource for updates on concussion.

The reflective process proposed for developing a CMP was adapted from the ‘Road map’ published by the CCC as a support for adapting and implementing such protocols.22 The participant’s role (eg, parent, coach, teacher, health professional), the specificities of the sport or activity, the characteristics of the environments (eg, school-based, non–school-based) and the available resources (eg, coach, healthcare provider) are examples of the factors that participants were invited to consider as part of the reflective process. Discussion forums were available in each module to allow participants to share experiences, comment on the contents of the different modules, and ask questions or request clarification. Forum participants were free to use their true identity or a pseudonym. The course instructor (PF) monitored the forum daily and provided feedback, additional information and answers as needed. The process for the evaluation of participants’ knowledge following each module and for the final evaluation was a set of 10 multiple choice questions randomly generated from a pool of 3 options for each question.

Patient and public involvement

Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research.

Participants and data source

The only requirement to participate was to provide a name and a valid email address. The terms of use of the MOOC included a specific mention that de-identified information would be collected and could be used to evaluate the utilisation of the site. The tracking functions of the ‘MonPortail’ platform provided de-identified information including number of registrations, number of accesses to each module and number of distinct participants accessing each module. Statistics on participation and success rates for each component of the evaluation were also available. A ‘Registrant’ was defined as a person who proceeded to register for the course regardless of subsequent participation. A ‘Participant’ was defined as a registrant that logged into the course at least once over the duration of the course. ‘Successful completion’ was defined as achieving an overall 60% success mark for the evaluations (success for each individual module was not required). Individuals who achieved a successful completion were eligible for a certificate of achievement. Optional survey questions were also used through the modules to document the collective profile of the participants (eg, sport, education, healthcare) and their intent to update or implement a concussion protocol. The forums also provided quantitative and qualitative data on the volume of interaction with and between participants. Finally, following completion of the course, participants were asked to complete an optional satisfaction survey. The Laval University ethical review committee considered that the data presented in the study meet the criteria for “research that relies exclusively on secondary use of anonymous information” and, therefore, did not require ethical review (Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans; Canada, 2018).

Results

The data regarding registration, participation and successful completion are presented in table 2. Among the 8368 registrants, 43%, 26%, 16% and 15% were participants of the first, second, third and fourth iterations, respectively. The proportion of registrants achieving ‘successful completion’ was consistent over the four iterations (range, 26%–32%), with 30% of all participants being eligible to obtain the attestation of success. Forty-three per cent of the “participants that accessed the course at least once” achieved successful completion.

Table 2

Registration, participation and successful completion

	April 2016		January 2017		October 2017		October 2018		Total
Registrants	3566	%	2168	%	1343	%	1291	%	8368	%
Participants (accessed at least once)	2711	76	1686	78	754	56	783	61	5934	71
Participants who accessed the first four modules*	1341	38	890	41	490	37	512	40	2721	39
Participants who accessed all modules	1192	33	749	35	430	32	452	35	2823	34
Successful completion	1088	30	694	32	350	26	406	31	2538	30

*The topics of the first four modules are introduction, prevention, detection and early management recommendations.

An optional survey completed by 36.6% of the registrants was used to estimate the proportion of participant from different stakeholder groups (table 3). Over the four iterations of the course, “healthcare professional other than MD” consistently represented over half (54.6%) of the participants, whereas physicians represented only 4.2%. Coach and sport administrator was the next most prevalent group, with 10.1% of the participants. Education stakeholder, parents and persons affected by a concussion each represented from 3% to 7% of the participants. The answer “Other (not specified)” was selected by 14.8% of the respondents, but the functionalities of the course platform did not allow to collect free-text specifications with the answers.

Table 3

Answers to optional survey question: “what category defines best your perspective or interest as a participant in this course?”

	April 2016	January 2017	October 2017	October 2018	Total
No of survey participants	861 (%)	979 (%)	614 (%)	607 (%)	3061(%)
Parent of young athlete	70 (8.1)	64 (6.5)	35 (5.7)	43 (7.1)	212 (6.9)
Person who suffered a concussion	N/A	46 (4.7)	29 (4.7)	23 (3.8)	98 (3.2)
Teacher or school administrator	39 (4.5)	81 (8.3)	39 (6.4)	31 (5.1)	190 (6.2)
Coach or sport administrator	82 (9.5)	109 (11.1)	57 (9.3)	61 (10.1)	309 (10.1)
Physician (MD)	25 (2.9)	46 (4.7)	24 (3.9)	34 (5.6)	129 (4.2)
Healthcare professional other than MD	415 (48.2)	534 (54.5)	362 (59.0)	359 (59.1)	1670 (54.6)
Other (not specified)	230 (26.7)	99 (10)	68 (11.1)	56 (9.2)	453 (14.8)

N/A: choice not offered in April 2016.

Another optional survey, completed by 1471 responders (17.6% of registrants or 24.8% of participants), was used to assess the intent of the participants to implement or update a CMP (table 4). The results indicate that 579 individuals (6.9% of registrants; 9.8% of participants; 39.4% of survey responders) across the four iterations of this course intended to update or implement a CMP in their sport or leisure environment.

Table 4

Answers to an optional survey question about the participant’s intent to update or implement a concussion protocol

	April 2016	January 2017	October 2017	October 2018	Total
No of respondents	565 (%)	417 (%)	256 (%)	233 (%)	1471 (%)
Yes, the participant intends to update or implement a protocol	308 (55)	135 (33)	71 (28)	65 (28)	579 (39)
No, the participant does not have that intention	257 (45)	282 (67)	185 (72)	168 (72)	892 (61)

The volume of activity in the discussion forums is summarised in table 5 and demonstrates that about 15%–18% of participants used the forums. Key information from the optional satisfaction survey that followed each iteration is summarised in box 1.

Table 5

Volume of discussion activity and access by participants in the forums*

	April 2016	January 2017	October 2017	October 2018
Topics†	54	36	25	20
Messages	197	132	73	62
Visits	10 987	5608	2137	2662
Distinct participants(% of registrants)	580(16)	319(15)	201(15)	237(18)
Messages by teacher	71	47	31	28

*Excluding any activity not related to concussion topic (eg, questions regarding technical aspects or evaluations, suggestions for improvement, etc).

†Topic: any new question or point discussion point initiated by one of the participants.

87% declared spending 1 to 3 hours per week (over 6 weeks).

Over 90% declared that the workload and duration of the course are adequate.

Global satisfaction for each module was rated as ‘highly appreciated’ or ‘appreciated’ by over 90% of participants.

A minority (23%) wished there were more opportunities for interaction between participants.

58% were satisfied or highly satisfied by the interaction with the educational support team (39% “did not know” or answered “does not apply”).

Over 90% felt the access to the course was easy, navigation was intuitive and answering the survey questions was easy.

96% of participants who used the forum answered that it was easy to use.

Discussion

Participation and completion

Registration rates for MOOCs are often impressive, but data related to participation and completion are more relevant from an educational or knowledge transfer perspective. Although criteria used to define eligibility for a ‘Statement of accomplishment’ vary across studies and are sometime not reported, the data allowing to establish the proportion of registrants that meet these criteria are generally reported and allow to compare participation between MOOCs.22 23 In this study, the criteria for successful completion was consistently met by about 30% of the 8368 persons who registered for the course and 43% of the participants who accessed the course at least once. By comparison, an analysis of nine MOOCs offered by the Johns Hopkins School of Public Health showed that registration ranged from 11 000 to 101 000 participants while the proportion that reached the criteria for a ‘Statement of accomplishment’ ranged from 4.2% to 14.9%.22 Based on a review of information available about registration in 91 MOOCs and completion for 42 MOOCs on a broad range of topics (not limited to health), Jordan24 found an average MOOC registration of around 43 000 students (range, 4500–226 000), with an average completion rate of 6.5% (range, 0.9%–36%). Recently, a completion rate of 38% for a MOOC on dementia was reported.18 We are not aware of published data from MOOCs on sport and exercise medicine topics. Nevertheless, in terms of successful completion rate, the French-language MOOC on concussion is in the upper range by comparison with other published MOOC experiences.

A gradual decrease in the total number registrants was observed over the first three iterations of the MOOC with a subsequent stabilisation at about 36% of first offering. This may indicate a gradual saturation of the reach for potential new participants. Among the considerations that might help define the optimal frequency of delivery of a MOOC on concussion is the 4-year cycle for the update of the international concussion in sport group recommendations.3

Over the four iterations, although only 15%–18% of registrants accessed forum activities, 23% of responders to the satisfaction survey reported that they wished there were more opportunities for interaction between participants. This suggests that additional strategies to stimulate interaction between participants could be considered in the future.

Factors contributing to completion

Beyond the public awareness about concussion and the associated motivation to gain knowledge on this topic, the success of the MOOC in terms of successful completion is likely explained by multiple factors that have been described in the growing literature on MOOCs. Karsenti25 has identified three strategies that likely contribute to completion: (1) mobile learning, (2) ‘gamification’ of learning and (3) adaptive learning. While the ‘gamification’ of learning was not exploited, mobile learning was present in the form of flexible formats (eg, video or podcast format), short learning units (±3–10 min), limited workload (1–3 hours per module), short overall duration (6 weeks) and discussion forums that allowed for interaction regarding a broad range of concussion-related issues. Together, combined with the flexibility allowed by the online nature of the course and current technological options to access the web, these mobile learning strategies likely contributed to the satisfaction and completion rates reported.

Adaptive learning is also a strategy found in the MOOC in the form of a basic learning profile, combined with an optional reflective process around the adaptation and implementation of a CMP in the specific setting of the participant. Considering that over 70% of the respondents (see table 3) reported being healthcare, school or sport stakeholders, this optional learning pathway might have been a factor that motivated sustained participation for many registrants. On the other hand, as illustrated by 61% of the survey respondents declaring that updating or implementing a protocol was not an objective (table 4), participants such as parents, victim of a concussion or medical doctors might have chosen not to participate if the entire focus of the course had been on protocol implementation.

Potential reach of an English MOOC on concussion

Of the 8368 persons who registered to the 4 iterations of the French-language MOOC, 70.0% were from Canada, 6.8% from Haiti, 6.1% from France, 11.2% from countries of the African continent and 5.8% from other countries. Data from the 2016 census estimated the Canadian population who declared ‘knowledge’ of English, French or both official languages as 23.8, 4.1 and 6.2 million, respectively.26 Assuming a proportional representation of potential participants, this indicates that an English adaptation of a MOOC on concussion has the potential to reach over three times more participants in Canada. The estimation of the potential reach of an English-language MOOC on concussion should also take into consideration the potential for registrations by international participants with a global English-speaking population that is more than five times larger than French-speaking.27 Based on the experience of the French-language MOOC on concussion, the University of Calgary and Laval University have developed an English adaptation of a MOOC in concussion with the assumption that it could reach three to four times more registrants.28

Limitations and potential for improvement

The integration of an evaluation strategy should be part of the development of MOOCs.19 29 Proposals for the design of MOOCs include multiple evaluation components: (1) pretesting as a comparative reference for subsequent evaluations throughout the course, (2) evaluation of both conceptual (measures of ‘what’) and procedural (measures of ‘how to’) learning, and (3) validated outcome measures to enable comparison across MOOCs.19 In the French-language MOOC, the evaluation of each individual module had a predominant focus on conceptual knowledge, while the final evaluation had a predominant focus on procedural knowledge based on clinical vignettes. However, the absence of comparative data from a pre-test evaluation did not allow us to assess knowledge increases.

Another limitation of the French-language MOOC experience is the limited capacity to directly evaluate its impact in terms of knowledge transfer and implementation. Beyond the fact that 579 participants intended to implement or update CMPs in their setting, and the finding that 2538 participants met the criteria for successful completion, no direct measure of knowledge improvement or subsequent implementation of CMPs was included in the evaluation strategy. To document such outcomes, a future MOOC would need to obtain consent from participants to take part in subsequent data collection processes.

Conclusion

In summary, the experience of four iterations of a French-language MOOC shows promising results supporting the use of MOOCs as an innovative educational strategy that may be part of the solution to the public health issue of concussion. The development of future MOOCs should include a comprehensive evaluation strategy to document the impact on knowledge and, if applicable, measures of knowledge implementation.