Confessions of a chagrined trialist.

As a clinical trialist, I had thought that the methods I employed were far more challenging than those I had thought were needed for quality improvement. However, some personal experiences and participation in the Cliveden conference led me to a new appreciation of the methodological and conceptual challenges faced by those trying to improve medical systems.

Sometimes, one is ambushed by insight. I had long been ensconced in a cocoon of what one might call methodological snobbery. I had spent my professional life designing and analysing studies of biomedical interventions, and found their technical, ethical and inferential challenges endlessly fascinating. I knew about quality improvement (QI); one of my close medical school friends had become a leader in paediatric QI, and a colleague at Johns Hopkins was making celebrated contributions to the field through his ‘checklist’ approach. But while I viewed their goals as laudable and their work essential, I must confess that I viewed the scientific challenges they faced, to be polite, as somewhat less daunting than those that regularly commanded my skills and attention. My mental model for QI was this; we know what should be done, and QI is about getting people to do it. The evaluative part was seeing if they did it or not. Simple, right?

Well, anyone reading this volume knows better, and now so do I. I have a new model, and new insights. My new model is that QI is about changing complex healthcare systems to produce collectively better health. These systems can be as or more complex than the biological systems affected by the drugs and devices, and the system changes, like biological interventions, can have unexpected adverse and beneficial effects. My insight was that far from being beneath me, the skills and tools necessary for QI included an array of concepts and tools far broader than I had been using for evidence-based medicine. The methods I had spent a lifetime teaching and using were inadequate for this task. My methodological snobbery? Simple hubris.

It is hard to say exactly when the light started to dawn, but I will start with a personal story. Approximately 8 months before I was to attend the Cliveden conference on QI, my mother was diagnosed as having terminal cancer. She was cared for in one of New York City's major academic hospitals. What I and my family observed over the course of her stay would not surprise anyone in QI, nor indeed anyone who has seen hospital care for a complex illness up close; an unremitting litany of lapses and errors that, if not hastening her death, certainly made her last month of life more miserable.

I will recount just one of the many episodes we witnessed. Late in her course, she needed a new regimen of sedation. It was difficult to get the changes communicated and implemented through the various care givers, so in frustration, the head of her palliative care team took charge and personally made sure the right cocktail of drugs was ordered and hung. Many people were involved in the blizzard of activity provoked by this medication change. But by morning, it was apparent that the medications were not working. Why? Someone had not turned on the intravenous pump.

As an advocate for the patient in such situations, one is tempted to use rather blunt interventions, such as complaining and yelling loudly at every care giver walking into the room, or even some passing by. Other family members did take this tack, and their complaints were indeed loud enough to be heard among hospital leadership. These administrators started a virtual daily parade of penance to the bedside, albeit marching to the unchanging bolero beat of continued lapses.

Hearing me recount this, a colleague pointed me towards Don Berwick's classic call to arms, ‘The Escape Fire,’ which tells an eerily similar tale of being an eyewitness to a loved one's victimisation by a healthcare system seemingly designed for error, and of the inability of even a Harvard professor and QI expert at the bedside to change it. This led me to speak during this period about how ‘the system was broken,’ with problems complicated enough to defy simple solutions. But somehow, there was a disconnect vis-à-vis QI.

Then, came Cliveden. Although I was not sure exactly why I was invited, I was attracted by the audacity of putting ‘epistemology’ into the title of a conference addressing practical change. And I was, in fact, interested and informed about the epistemology of clinical research and felt that one cannot get evaluation strategies right if the epistemology is not understood. But an epistemology for QI? Oxymoronic, I thought. Doubtful but intrigued, I crossed the ocean and found myself surrounded by thoughtful, smart and dedicated scientists, some in disciplines such as mine, and most far afield from my own, who had spent careers trying to fix the broken system I railed against.

Suddenly, I saw what had been before my eyes all along. QI was not about implementing a change whose outcome was predictable. Perhaps I had been fooled by the name, or by analogy to QI efforts in other fields or just by sheer inattention. The challenge of QI—both intellectual and technical—was that even if one knows what needs to happen at the bedside, one does not know, at a system level, how to achieve that in a safe, efficient and sustainable way. And having systems as the unit of intervention, and perhaps analysis, poses immense challenges for both implementation and evaluation.

This new awareness also led me to look at clinical trials—my domain of expertise—through a different lens. Clinical trial protocols are often designed to structure the delivery of care, altering the context in which an intervention is delivered to minimise the variability in ancillary care characteristics. This structure is often derided for limiting the generalisability of clinical trial findings, leading to a call for ‘pragmatic’ or ‘practical’ clinical trials where the effect of an intervention is assessed ‘au naturel,’ as it will be applied in the community, with fewer restrictions, which often results in poorer outcomes. But rather than surrender to real-world variation, if the clinical trial protocol represents a system change that improves outcomes, could we use clinical trials not just to evaluate the intervention, but also to study which system changes (ie, protocol elements) are critical for the intervention's effectiveness? Could we look to clinical trials not as artificial care systems, but instead as model care systems that can provide clues for what needs to change in our current models of care delivery? If the clinical trial can study outcome changes due to a biological intervention, they can simultaneously be laboratories for examining the effects of system changes by including arms where such system modifications are made.

I will leave it to the other pieces in this volume to flesh out these issues. But I must confess chagrin at having dedicated my life to improving medical outcomes by evaluating interventions without fully appreciating the complexity of understanding how they are to be implemented within care systems. However, I still have some life and career in me, and thanks to the Cliveden conference I look forward to dedicating some fraction thereof, with new colleagues, to the challenging, fascinating and potentially greatly rewarding field of QI.