Disaster scholarship.

In this editorial, the Editor considers four domains of disaster in health professions education; the scholarship of disasters, disasters in the conduct of scholarly inquiry, disasters in the presentation of the results of scholarly inquiry, and disasters flowing from the misinterpretation or misappropriation of scholarly activity.

Often in the last year, I found my mind turning to the subject of disasters in health professional education scholarship. With extreme weather, social unrest, conflict, and pandemic filling the news, sometimes it was (and still can be) hard to think of anything else. Trying to find order in the apparent chaos of the world we live in is a fundamental part of being a scientist and how we might apply that lens to disasters is what I explore in this edition’s editorial. To that end, I reflect on different kinds of disasters that play out in our idiosyncratic field by focusing on four broad themes of disaster in the scholarship of health professional education.

My first theme focuses on the scholarship of disasters; research and other scholarly activity that explores the nature and impact of unpredictable, unavoidable, and destructive life-changing events. These events may impact whole populations or systems, or they may impact more discrete groups or even individuals. Disasters may be caused by conflict, pandemic, weather, earthquakes and volcanoes, economic and social unrest or collapse, criminality, or some other disruptive event or series of events. In the field of health professions education (HPE), we might consider how such events or circumstances impact the training of health professionals and how they might be faced, overcome, or accommodated. This can include how HPE systems, programs, or individuals and groups prepare for and/or respond to natural disasters (such as floods and storms) (Krane et al., 2007), man-made disasters (such as war and social unrest) (Challoner & Forget, 2011), or medical disasters such as pandemics and iatrogenic murder (Clark, 2003; Stewart, 1999). Most of this literature is retrospective, forensic, and case-study based (what happened, why it happened, and what we can learn from it) (Balalian et al., 2014; Powell et al., 2012) although there are also prospective studies recommending better preparedness for the next such event. (John et al., 2017; Smith et al., 2012)

One of the signature characteristics of a disaster is that it is not ‘business as usual.’ An emergency room in a city plagued by gang violence may seem like a disaster zone to someone unfamiliar with such things but it would not locally be considered to be a disaster, however, awful or disastrous it was to those involved. Scholarly writing about unexpected and highly disruptive events can be challenging. If the author had firsthand experience of the event, then their biases and potential trauma might make it difficult for them to be objective and critical. Writing during a disaster will perhaps convey a better sense of what is like in the moment (although writing and coping with the situation will itself be taxing), while writing after the event (with the benefits and drawbacks of hindsight and historical interpretation) may be more considered and generalizable but perhaps less authentic. Scholarly analysis as opposed to ‘we did this’ or ‘this happened’ writing is critical here. The slew of purely descriptive and speculative papers we saw submitted to Advances early in the COVID-19 pandemic last year led to a previous editorial (Ellaway et al., 2020) where we asked for analytical and theory-informed approaches to considering the impacts and implications of COVID-19 on health professions education. After all, has COVID-19, or rather have our individual and systemic responses to the pandemic truly changed HPE? Arguably, the learning context has changed, but learning is still learning; the assessment context has changed, but assessment is still assessment, and so on. These changes and the ways in which teachers and learners accommodated them may be temporary or they may reflect or precipitate deeper and further reaching changes that we have yet to fully appreciate or understand. Considering such issues requires triple-loop learning; ‘are we even asking the right questions?’ (Tosey et al., 2012)

It is notable that much of the literature on disasters in HPE has focused on describing logistical challenges, perhaps because they are so often written from the perspective of leaders and teachers. Theory has played little part in this work, which might suggest that theory plays little role in abnormal situations. There has also been rather less published, it would seem, on the changing affordances of disasters; how different learning environments are perceived and experienced, and what they can and cannot provide in different circumstances. For instance, there has been little written about the axiological changes (changes in the values and value systems) brought about by disasters. As an example of this, in my own country of Canada, the reactions to COVID-19 and the opioid crisis have been very different; the logics of how much resource and commitment is made belie assumptions the healthcare system can save everyone and serve everyone no matter what the cost. (Simmons & Scott, 1998) As much as there is talk about the ‘new normal,’ thus far it seems very similar to the old one. Fifty years after the release of ‘Who’s Next?’ and the anthemic song ‘Won’t Get Fooled Again’ it would seem that the new boss indeed seems eerily like the old one.

My second theme is scholarship that has had to contend with disasters during its execution. This might include research that was impacted by unavoidable (or at least unavoided) events such as loss of data, loss of access to data, loss of personnel or expertise, academic misconduct or impropriety within the team, or being undermined by changing regulations or regimes. As much as research on disasters makes up a small part of the literature, research into disasters in research is even less common. And yet there is surely much to learn in what not to do, how to react when things go wrong, what kinds of mitigation and recovery strategies might help, and the craft and challenges of conducting research in general? I know in my own career I have had to contend with conflicts with people in positions of authority, the death and major illness of investigators and participants, data loss, equipment failures, problematic relationships with funders, and intra-team conflict. Add to this the exigencies of conflicting professional commitments, the challenges of negotiating the day-to-day business of conducting research, and broader societal challenges such as the disaster situations I noted earlier, then sometimes it might seem that getting any study to a successful conclusion is a remarkable feat in itself.

And yet, like any hidden curriculum, we tend not to talk about such things in our official communications. There are of course rhetorical conventions for discussing reflexivity, bias, and study limitations, but these are usually argued so as to support the credibility of the work rather than being something of interest in their own right. Moreover, we tend not to publish failed studies. Studies that are simply poorly designed, conducted, or reported would not meet Glassick et al.’s basic criteria for scholarly work, (1997) let alone those of a journal such as Advances. On the other hand, studies that are robust scientifically but have as their focus disasters and failures in HPE research and generalizable learnings from them would likely be of interest and publishable in a journal such as ours. However, as the current Editor, I have yet to see a submission that proposes a robust study of research failure or disaster. Maybe there are those of you reading this editorial who would like to try to turn that particular corner.

My third theme picks up on a topic from the previous one; papers and other scholarly products that are so flawed that they are unpublishable, or rather, since there are ways in which anything can be published in some way or other, work that should not pass the scrutiny of peer review and our other academic checks and balances. There may be many contributing factors to such disasters; bad questions, bad study designs, bad execution, incomprehensible or jumbled writing, insignificant or confusing findings, and/or indefensible logical or ideological arguments. Disasters may also reflect academic misdemeanors, such as plagiarism, autoplagiarism, and salami slicing, concerns we considered in another Advances editorial (Tolsgaard et al., 2019) I do not wish to be condescending or to trivialize the challenges of scientific writing, particularly for those new to the field, those crossing idioms or paradigms, of for those writing in a second language. One of the jobs of an editor is to distinguish between work that is sound but communicated poorly (and remediable) and work that is unsound but obfuscated by poor presentation. Nor will I identify specific papers or studies that are (or that I would consider are) writing disasters, although we do see them submitted to Advances from time to time. Given that poorly written or presented work will usually not be published then it would be hard to systematically analyze this null literature. Nevertheless, the potential for scientifically robust critiques of how science is presented in HPE would be of interest as Varpio et al. recently argued. (2021)

My fourth theme is scholarship that leads to or contributes to disasters. In this framing, I interpret disasters as HPE research being misused, appropriated, or misinterpreted. These consequences may seem rather less extreme and mortal than the disasters I considered in my opening theme. Or are they? We know that poorly trained health professionals can lead to patient harm and health system inefficiency. (Leape, 1997) We have also noted that evidence can be liberally reinterpreted to serve other ends. (Horsley et al., 2020) However, if disasters in executing research and in reporting research have a small footprint in the literature, research that contributes to or precipitates disasters is even less visible. To some extent, this is a knowledge translation or implementation science issue (see Thomas & Bussières, 2021), but there are more troubling issues if research is misrepresented or reinterpreted, for example (and I speak from personal experience) where evidence and findings are used to argue for a course of action that is the opposite to or way beyond that recommended by the researchers who conducted the study. This is also reflected in the problems with research that has focused on one demographic but that is generalized to a much wider population with unfortunate results. (Criado Perez, 2019) Debates over interpretation of findings do not mean that the researcher is always right of course, and publishing your work means engagement in public discourse that you cannot possibly control. The reinterpretation of research may even happen at the review and editorial stage; I am sure that both reviewers and writers will have encountered the ‘why didn’t you do the study I would have done?’ phenomenon.

So, what do all these themes have in common, other than that I have used the term ‘disaster’ in all of them? Well, I think there is much to learn from unfortunate events, disasters even. Not just in terms of forensic analyses of what happened and why it happened, not even to document the vagaries and challenges that we face as health professional educators. There is much untapped potential here to explore the nature of research through our understanding of disasters and misadventures in ways that can lift the field and help those both novice and experience researchers to improve their craft. To be clear, I am not proposing a ‘Disasters Corner’ as a new section in Advances, only to encourage reflection on the scholarly opportunities to engage with disasters in new and creative ways. When disasters next strike, you might wait for the Thunderbirds, or Buffy, or The Avengers to save you (or whoever your generation-appropriate fictional rescuers might be), or you might just get writing.