Medi-wiki worries?

The idea to publish scoping and systematic reviews1,2 in wiki format is exciting and the authors and the journals editors are to be congratulated in being at the frontiers of journal publishing. However, I cannot help but raise questions about the pitfalls in such a process. In teaching history (of sport, health and physical education) in a kinesiology faculty, I have been forced to engage with wikis since my students love them and frequently turn to the granddaddy, Wikipedia, as their first source of information. There are clear issues that should be considered here.

Obviously the editors of Open Medicine have thought much of this through. Opening an article to contributors beyond verified experts does, as they suggest, have the potential to bring in more participants and different and useful forms of knowledge. However, a large community of contributors seems to be required for wikis to maintain a level of information quality3, and we have examples where those involved in wikis have become disenchanted with the process – Wikipedia co-founder Larry Sanger left to establish other web-based encyclopedias (Nupedia, Citizendium) in which expert opinion could have more prominence.4

Vandalism and innocent mistakes are frequently raised as wiki-related problems.5,6Open Medicine’s editors have instituted a process of requiring contributors to state their name, affiliation and competing interests to ensure responsibility for edits. It seems though, that despite this well-intentioned policy, there is little to stop someone from declaring themselves as “Dr. Joe Schmo of the Podunk Institute” and making whatever edits they want.

In the context of medical reviews as wikis, I am less worried about vandalism (which is easily detectable) than innocent mistakes. Research on wikis shows that vandalism is typically reverted quite quickly, but that plausibly-sounding mistakes last much longer, in some cases for months.7 Furthermore, as we are all well aware, even at its best, medicine can be sometimes quite inexact. Despite all of the science and evidence-based practice, we often cannot explain why one case in 100 is spontaneously cured, or why what works 99 times does not for patient #100. Wikis, by their very nature, offer the potential for extreme circumstances and people to be given the same gravity as the more normal case.

In addition, there is my main problem with wikis as a source-obsessed historian, which is also relevant to medicine. On open wikis, even if the information is usually very good, there is no guarantee about any particular version that a person is consulting at any given time. This, and all of the issues mentioned above, seem to me to be particularly dangerous if it is medical knowledge that is open access and open edit.

In the case of most good wikis, mechanisms evolve to try to ensure quality and reliability. The journal’s editors have instituted a tracking process with the authors as a first step, which is very sound. I am excited to see how this project will evolve, and wish the journal the best of luck in wiki-ing forward.