Wikipedia: A Medical Student Educational Project to Edit Wikipedia in Preparation for Practicing Evidence-Based Pain Medicine.

OBJECTIVE: Wikipedia is commonly used to acquire information about various medical conditions such as chronic pain. Ideally, better online pain management content could reduce the burden of opioid use disorders. Our goal was to improve the quality of the content available on Wikipedia to make it more accurate and applicable to medical students and the general public while training medical students to practice evidence-based medicine and critically assess their sources of information.
METHODS: An elective class in Neuroscience, Pain, and Opioids composed of 10 medical students met biweekly to discuss landmark and practice-changing research articles in the fields of acute pain, chronic pain, and opioid management. The professor chose Wikipedia articles relevant to this course. Three independent viewers analyzed the quality of citations, anecdotal medical content, and content value for both patients and medical professionals. As part of their coursework, students then edited the Wikipedia articles.
RESULTS: Although some of the Wikipedia pain topic content (6.7% ± 2.0) was anecdotal, financially biased, or inconsistent with Western Medical Practice content, overall articles included primarily high-quality citations (85.6% ± 3.1). On a 0-5 Likert scale, students felt content would be moderately helpful for both medical students/professionals (3.4 ± 0.2) and laypersons (3.5 ± 0.2). Editing and adding citations was feasible, but novel material was often reverted.
CONCLUSION: A significant amount of pain medicine content was relevant and amenable to student editing. Therefore, future use of this tactic could provide a unique opportunity to integrate evidence-based medicine into the medical curriculum and have a direct impact on the widely available medical information. Future refinement in the editorial process may also further improve online information.

Introduction

With 2 billion visits for medical information in 2019, Wikipedia is potentially the most used source for medical information in the world.[1] Indeed, multiple reviews and studies have argued that Wikipedia’s popularity and content make it a worthwhile destination for writing assignments[2,3] and academics to address public health concerns.[4,5] While frequently utilized, there are questions regarding the accuracy of the information available, given that it is an open resource that can be edited by anyone. A survey of medical students found that 76% had used Wikipedia to seek out medical knowledge, with 97% reporting finding incorrect information at some point.[6] In this survey, 65% of the students did not know how to correct the online information. Thus, educational experience on how to fix Wikipedia articles could lead to further improvements in online content.

While medical professionals and students are encouraged to contribute to information sources like textbooks and journals, there is less discussion about the role they can play in openly-editable, collaborative resources like Wikipedia. A 2017 study enrolled medical students in a course to edit Wikipedia pages.[7] After completing the course, physicians reviewed the pages students edited and found consistent improvement.[7] Also, students found the class useful for understanding their role as a physician in an increasingly networked world.[7] Course designs such as this can help medical students learn how to critically appraise information, communicate health concepts to a broader audience, and improve the content of information available on Wikipedia. A comprehensive review of health information on Wikipedia suggested that it would significantly benefit from even further research as an educational tool.[5]

Although providing courses for Wikipedia editing may be a useful idea, many students may not elect to take such a class. Therefore, we evaluated whether it would be worthwhile to embed Wikipedia evaluation and editing within a medical student elective course on pain. Notably, pain is an important topic known to need online improvements.[8,9] Specifically, for medical students, improved training could lead to better prescribing behaviors that could help mitigate the opioid crisis.[10] Simultaneously, enhanced patient education about the risks associated with opioid misuse is also crucial in developing public awareness of this problem and addressing it. Thus, a two-prong strategy improving pain education for both medical students and patients could be valuable in revamping prescribing behavior, chronic opioid dependence, and pain management.

In the present study, students enrolled in a formal medical education course elective on Neuroscience, Pain, and Opioids were tasked with reviewing and editing select Wikipedia pages. Articles reviewed were selected by the course instructor and covered several subtopics related to acute and chronic pain, as well as therapy and policy surrounding standard and alternative treatment measures. Students were asked to independently analyze the quality of the citations, anecdotal medical content, and content value of each assigned page. Afterward, they made direct edits to these selected pages. This approach is recommended as part of the flipped classroom strategy which allows for online resources and readings before a discussion in class.[11]

Methods

This elective course was composed of 10 students in their fourth year of medical school. The students met biweekly for 90 minutes to discuss landmark and practice-changing research articles on acute pain, chronic pain, and opioid management. The course director selected recent articles reflecting the current state of the field. The chosen literature contained original basic research articles, clinical trials, or meta-analyses from high impact journals. The class would discuss 2 articles each 90-minute session, evaluate the study’s quality, compare it to other evidence on the topic, and discuss the clinical impact of the study.

Simultaneously students were charged with reading and evaluating 29 different Wikipedia articles on pain, treatments for pain, and opioids selected by the course director (KH). The list of 29 articles (see Table 1) was posted with weblinks on a google drive sheet with space for entering evaluations. Each article was reviewed independently by 3 authors for the following parameters: estimated percent of high quality and evidence-based citations, anecdotal content, financially motivated content, and inconsistent with Western medical practice content. Eight of the study authors were part of the course and edited the Wikipedia pages. A Likert 5-point scale (1 point if “strongly disagree” and 5 points if “strongly agree”) was used to evaluate the overall value of the article’s content for a layperson or patient (1-5), and overall value of the article’s content for a medical student or healthcare provider (1-5). The final score for each parameter for each article was the mean of the 3 reviewer’s scores. The raw data is available upon request from the corresponding author.

Table 1.

Rated quality of Wikipedia content.

Topic	High quality citations (%)	Anecdotal, financially based, or inconsistent with western medical practice content (%)	Content value for lay person/Patient (likert scale)	Content value for medical students/Profes-sionals (likert scale)
1. Acute to chronic pain transition
Pain	93.3	2.2	4	4
Chronic_pain	92.5	2.8	3	3
2. Neural mechanisms of pain
Allodynia	93.3	1	2.3	3
Hyperalgesia	96	2.7	2.7	3
Nociception	95	3.7	1.7	3.3
Diffuse_noxious_inhibitory_control	96	2.3	1.3	2.7
Nucleus_raphe_magnus	93.3	4.3	2.7	2.7
3. Biopsychosocial model
Biopsychosocial_model	88.3	3.3	3.8	3.3
Sleep	75	5	4.3	3.7
Depression_(mood)	86.7	6.3	3.3	3.3
Anxiety	95	4.3	4.7	4.3
Somatization	86.7	8.3	4	2.3
4. Typical analgesic medications
Opioid	95	3.8	5	4.3
Tricyclic_antidepressant	92.5	1.5	4	4
Serotonin_norepinephrine_reuptake_inhibitor	92.5	5	4	5
Anticonvulsant	97	4	3	3.7
Nonsteroidal_anti-inflammatory_drug	91.7	4.7	3	3.7
5. Typical surgical interventions
Neurectomy	16.7	58.3	1.7	1
Spinal_cord_stimulator	88.3	11.7	3.7	2.3
Deep_brain_stimulation	93.3	5	3.2	3
6. Other treatments
Physical_therapy	76.7	4.2	4	3.3
Transcutaneous_electrical_nerve_stimulation	85	11	2.3	2.3
Acupuncture	86.7	5.3	4.8	4.8
Aromatherapy	88.3	4	4	3.3
Chiropractic	90	3.5	3.8	4
7. Community/Government policy
Opioid_epidemic_in_the_United_States	78.3	7.3	4.7	4.3
Naloxone	76.7	5	4.3	4
Opioid_use_disorder	95	0	4.5	4.7
Drug_liberalization	46.7	12.5	2.8	2
Average	86 ± 3	6.7 ± 2.0	3.5 ± 0.2	3.4 ± 0.2

Percent of high-quality citations; percent of anecdotal, financially based, or inconsistent with Western Medical Practice content; content value for a lay person/patient and the content value for medical students/professionals when reviewing all 29 Wikipedia articles.

Finally, each student edited articles with incorrect information or articles with incomplete information. Students were instructed on how to make Wikipedia accounts and provided edits to their selected articles with citations from high-quality medical literature.

Results

During the student’s participation in the course, all students (10) who enrolled (January 2020) reviewed selected articles from the 29 Wikipedia pages relating to pain and opioids (Table 1). On average, it was estimated that 85.6 ± 3.1% of the citations were of high quality in the selected articles. In contrast, only 6.7 ± 2.0% of the citations appeared anecdotal, financially biased, or inconsistent with western medical practice. It is worth noting one outlier, the article on surgical interventions, had over 50% anecdotal support. On a 5-point Likert scale, the overall value of the articles’ content for the layperson or patient was 3.5 ± 0.2, and for medical students or professionals was 3.4 ± 0.2.

After reviewing the pages, students identified content for improvement, focusing on topics on which they had an interest or prior knowledge. Three students were able to edit Wikipedia articles and add citations without in-person instruction. Six students were able to edit Wikipedia articles and add citations after a brief 10-minute demonstration within 3 days. One student conducting residency interviews completed his edits at a later date. Some of the student’s comments are listed in Table 2. Unfortunately, over 50% of the edits were reverted by either editors or automatic processes. The Wikipedia editors often preferred review articles and textbooks than citations from the primary clinical trial literature. The comments during the reverting process allowed students to pinpoint this reason.

Table 2.

Student’s comments about editing Wikipedia.

It was difficult to decide on information that would be a valuable addition to the online content.

I was concerned about deleting content for fear of jeopardizing prior work.

It was a lot of fun.

I was happy that my edits were not reverted.

I felt proud of my changes and the contributions they were able to make in improving online medical knowledge.

It felt useful to use our education to update articles containing unreliable content without much scientific backing.

I felt empowered when I added a new section to an existing brief Wikipedia page. The fact that my edits are still on that page makes me feel like I made a difference and potentially taught someone something new.

Discussion

As rated by a group of medical students, the information about pain on Wikipedia mostly had high-quality citations. Nevertheless, given the minority amount of biased content, it would benefit from medical student editorship. Although students enjoyed and found the process easy, improvements in the current paradigm and widespread adoption at other medical programs would be needed to make substantial advancement in the vast health content on Wikipedia.

Other studies have also used Wikipedia within a medical educational context. For example, the University of California, San Francisco (USCF), pioneered a medical education course that pushed students to improve Wikipedia pages in their fourth year of medical school.[7] A similar approach at Queens college resulted in over 1000 edits.[12] Although the UCSF program was successful, it demanded specific resources and precious class time. Our smaller group of students was able to edit articles with content specific to their course in Neuroscience, Pain, and Opioids. Refining these Wikipedia pages complemented the class as it allowed students to participate in evidence-based learning and to apply critical thinking while contributing to the value of this medical resource. Although this was not the principal goal of the class, we show that it is possible and useful to incorporate online editing as part of the medical school curriculum.

This project was successful in incrementally increasing the quality of health content available on Wikipedia and suggests avenues for improvement. Strengths include the student-driven initiative for edit selection leading to content creation that is likely more applicable to students or laypeople than seasoned providers.

Interestingly, while our group of medical students rated the articles to be of mostly high quality, historically others have questioned the reliability of Wikipedia’s content.[13] However, articles analyzing the quality of Wikipedia citations suggest the content is valuable. Large scale analyses have shown that Wikipedia medical articles preferentially cite the medical literature rather than non-medical sources.[14] Also, more recent, higher impact, and authoritative publications were preferentially cited in Wikipedia.[15] Intriguingly, a survey of 913 faculty and 2 different large public universities showed that faculty had positive views of Wikipedia article quality and used it, but were reluctant to use it as a teaching tool.[3] Thus, other reasons may also explain academics reluctance to use Wikipedia, such as plagiarism[16] and career implications. It is challenging to demonstrate career productivity with Wikipedia as a platform,[13] though benchmarks could address this limitation.[17] Intrinsic biases against Wikipedia must also be addressed. Indeed age, gender, political and comfortability with technology have been shown to create significant biases within editing and usage of Wikipedia.[18-21] Increasing diversity by encouraging student and faculty contributors can address these prejudices.

The project engaged medical students applying to a wide variety of specialties for residency, which allowed them to bring perspectives and explanations covering a full breadth of medicine. The main limitations of this study are the scale and longevity of the changes made. Some article edits received pushback from the Wikipedia community, especially when cited evidence did not come from a secondary source such as a systematic review or medical society guidelines. In preparing for similar courses in the future, students should avoid citing primary sources as recommended by the Wiki Education Dashboard. Additionally, planned focused collaboration with Wikipedia editors could result in more efficient improvements. For example, articles within WikiProject Medicine have received special editorial attention and resulted in significant improvements in the quality of content.[22] Additionally, courses should consider using available training (https://dashboard.wikiedu.org/training/students/evaluating-articles) and specific advice for editing health and psychology topics (https://dashboard.wikiedu.org/training/students/editing-medical-topics). These modules take less than 30 minutes to complete and teach critical thinking skills while helping to ascertain references will be inserted correctly.

Our elective was primarily accessible to fourth-year medical students who had covered a strong foundation of material to be better equipped to analyze the online literature. However, this could be a valuable exercise for students in earlier phases of education to help them become more comfortable critiquing scientific content. Future directions of this semester-long project would be to incorporate similar assignments into other medical or graduate school classes. This provides a unique opportunity for medical students worldwide to improve the quality of information online collaboratively. Promoting group discussions among students to evaluate web content such as Wikipedia will enhance their knowledge of the most current literature. An ability to think critically and to assess widely accessible literature sources is an imperative part of becoming a healthcare professional. Early in their professional careers, acquiring such skills will better prepare students to practice evidence-based medicine as future clinicians.[23,24] It is also worth noting that Wikipedia can contribute more memorably. In a study of 116 medical students randomized to get 1 of 3 sources (Wikipedia, UpToDate, or a digital textbook), students using Wikipedia had significantly better short-term recall of educational content than students using a digital textbook.[25]

The past few decades have seen an unprecedented increase in the accessibility of medical information. While the positive impact of this advance is enormous, it has become just as easy for misinformation to spread through sources that seem credible to the layperson. In the past medical students were discouraged from utilizing Wikipedia for day-to-day medical facts.[26] As medical information becomes more accessible online, students within different medical fields are trying to address the potential misinterpretation of medical facts online by improving the quality of Wikipedia content.[7,12,27] Regardless of the widespread concerns for accuracy on medical health websites, they continue to be used as a starting point for many looking to learn. The project taken on by the students of this class was an attempt to enhance the information available for those that find themselves seeking guidance on a variety of topics related to the neuroscience of pain and various therapies. Even if moderators reverse a portion of the edits, those that remain can improve the knowledge of millions of people. From a medical perspective, the more laypersons can understand disease processes and therapy, the more they can be involved and engaged in their care. Students will benefit from accessing trustworthy information to quickly familiarize themselves with topics before delving into primary literature that is often dense and difficult to digest rapidly.