Understanding unwarranted variation in clinical practice: a focus on network effects, reflective medicine and learning health systems.

In the past decades, extensive research has been performed on the phenomenon of unwarranted clinical variation in clinical practice. Many studies have been performed on signaling, describing and visualizing clinical variation. We argue that it is time for next steps in practice variation research. In addition to describing and signaling variation patterns, we argue that a better understanding of causes of variation should be gained. Moreover, target points for improving and decreasing clinical variation should be created. Key elements in this new focus should be research on the complex interaction of networks, reflective medicine, patient beliefs and objective criteria for treatment choices. By combining these different concepts, alternative research objectives and new targets for improving and reducing unwarranted variation may be defined. In this perspective, we reflect on these concepts and propose target points for future research.

Warranted and unwarranted clinical variation

For almost five decades, scholars have examined the phenomenon of unwarranted clinical practice variation. To quote Wennberg ‘there is variation in the utilization of health services that cannot be explained by variation in patient illness or patient preferences’ [1]. Typically, the variation is large, omnipresent, persistent and difficult to grasp. Does the explanation to practice variation rest on cultural or professional norms, and if so, is that acceptable? Practice variation was first documented by Glover, and has been confirmed across many healthcare settings [2-12]. However, the challenge remains of how to best explain why variation exists, and how the issues that seem inherent in such differences might be addressed.

Some propose the hypothesis that individual doctors or local groups of doctors are uncertain about what is the right thing to do, a so-called ‘professional uncertainty’ [3, 4]. Variation, it is argued, occurs when different treatment options are available and when considerable uncertainty exists about the effectiveness of these alternative approaches. In addition, these preferences and beliefs are influenced by environmental circumstances and local standards. Physicians who work together in the same hospital may show similarities in their medical decisions. In other words, the variation within a hospital will be smaller than the variation between hospitals [8]. Perhaps, the extent to which patient preferences and shared decision-making are taken into account in medical practice may be an important factor that explains variation [4].

Another hypothesis is that variation is not primarily explained by professional uncertainty but by different behavioral styles or approaches. In a very recent publication by Cutler and colleagues, the authors classified physicians as either ‘cowboys’ (preference for aggressive medical interventions) or ‘comforters’ (preference for more conservative actions) based on their judgment of different clinical scenarios. From the results, they conclude that variation was primarily explained by different types of physicians, and to a much lesser extent by patient preferences. Moreover, high-utilization areas appear to rely on physician’s beliefs that are not necessarily supported by clinical evidence [13].

Recently, Sutherland et al. published a framework with six domains to explain residual unwarranted variation. The domains pertain to three main categories: capacity (allocative decisions, organizational design and lack of acumen), evidence (lack of adherence to guidelines, unjustified deviation of evidence base) and agency (providers’ needs and preferences, lack of engagement) [14].

Variation is not bad or unwarranted per se. To some extent, variation should always exist, because patients are unique and different. Care could be called appropriate when decisions reflect such differences, especially differences in informed patient preference [1]. Variation may be unwarranted when it cannot be explained by sensitivity to patient characteristics or well-informed preferences.

In this perspective, we propose alternative hypotheses for mechanisms underlying unwarranted variation in healthcare and propose new target points for research to better understand, reduce and improve unwarranted variation in care quality in daily medical practice.

Target points for research to explain and improve unwarranted variation

Joint decision-making at a professional level

Professional autonomy is highly valued by clinical doctors and is often held as a principle necessary to deliver higher quality of care to the targeted needs of individual patients [15, 16]. Self-evidently this may be true in many cases. Nevertheless, we suggest that professional autonomy may be a potential barrier for doctors to collaborate with peers. Professional autonomy emphasizes individual practice over team practice, limiting the extent to which doctors might undertake systematic comparisons of how they arrive at decisions. As a consequence, individual practice may not always become transparent. This is especially true for routine and mild cases and in elective and preference-sensitive care, where questions like ‘How do you approach a certain case?’ or ‘What can we learn from each other?’ do not fit into a culture that prioritizes individual clinical autonomy.

Besides professional autonomy, medical environments with high workloads and time constraints may also limit the opportunities for feedback or time necessary to discuss routine care with colleagues. In addition, financial incentives and targets based on volume as well as organizational factors may drive choices for certain types of care, even if not supported by evidence. This adheres to the framework of Sutherland [14]. We argue that across the domains in this framework professional autonomy and lack of feedback are important mechanisms. Together, these factors may accumulate and exacerbate variation.

We hypothesize that in elective, less acute preference-sensitive care, clinical variation decreases in an environment where reflection on standard care and joint decision-making between doctors is incentivized and becomes more commonplace. Joint decision-making makes a similar call to doctors to collaborate as shared decision-making does to doctors and patients to collaborate. Feedback loops and inter-professional contacts should not only be applied to complicated cases but should also be incorporated in reflection meetings where more routinely provided care in less complicated patients is discussed (‘is it still appropriate to do what we always do?’).

Future research should focus on investigating the influence of feedback systems, teaming and joint decision-making between doctors on the occurrence and reduction of unwarranted clinical variation.

Collaboration, feedback and reflective practice within networks

As described, professional autonomy and individualism may be important drivers of variation. Intensive collaboration between doctors and feedback loops are imperative to achieve a healthcare system that is less prone to unwarranted variation. In addition, consensus about clear clinical criteria should exist to consider which variation can be considered appropriate. We argue that feedback and reflective practice approaches within networks may enhance more collaboration between doctors and to decrease variation in medical practice decisions. Previous research showed that multi-source feedback employing medical colleagues, coworkers and patients is reliable, valid and feasible to assess physicians’ performance [17]. Accordingly, Sargeant described the following four-step process in reflective approaches: (i) assessment of performance (understanding context and the performance measures used), (ii) provision of feedback (content of feedback, process of providing feedback), (iii) reflection and clinical decision-making (factors influencing feedback acceptance and use), (iv) use feedback for learning and change (use feedback for practice improvement) [18]. Furthermore, it has been reported that factors that influence the effectiveness of multi-source feedback include the format of the feedback, e.g. whether it was facilitated and narrative comments were included. In addition, it also proved to be important whether the feedback came from competent credible sources [19, 20]. Collaboration may attract individuals that prefer sharing and team performance over individual performance. It will make individual behavior visible and create learning and feedback mechanisms in order to increase beneficial outcomes and results. Uncertainty due to autonomy and lack of mutual exchange of decisions between physicians in clinical practice can change when feedback and reflection are organized in the work environment and workflow in a safe way. In the end, multi-source feedback may be helpful in mutual learning, openness in clinical decision-making and collaboration between doctors and inter-collegial relationships [21].

An important approach to incorporate collaboration, joint decision-making between doctors and feedback loops, is the organization of networks around patients. We hypothesize that strong networks may further enhance mutual learning and feedback mechanisms. As a result, this may diminish unwarranted clinical variation in healthcare. Network analysis is a relatively new development in clinical variation research. Few studies have been performed and new network methodologies are emerging [22-25]. However, studies about the association between the strengths of networks and healthcare utilization and clinical variation are still rare. In 2018, a first analysis on the association between physician networks and health care spending, utilization and quality of care was published [26]. The study showed that total spending, number of hospital days of admission increased with strengthening of networks. To our knowledge, the first study on clinical variation and the influence of patient-sharing networks was published in 2018 [27]. The study investigated the variation of Implantable Cardioverter Defibrillator Therapy Guideline Adherence in relation to patient-sharing networks. Counter to their hypothesis, the researchers found a reduced guideline adherence when networks were stronger (i.e. ‘referral hospitals having more connections to other hospitals’). The authors speculated that more connections may lead to more dissimilar information flows and enhance the visibility of clinicians willing to work outside clinical guidelines. This adheres to the line of reasoning proposed in the study by Cutler, in which physician’s beliefs seem to relate to expectations of physicians with whom they interact and in which their treatment policies are adapted to local norms [13].

Future research should aim at the value of networks to enhance collaboration, feedback and teamwork within and between hospitals. Furthermore, it should be investigated whether care that is embedded in strong networks show less unwarranted variation and better outcomes.

Objective clinical criteria and learning health systems (LHS)

Another important driver for practice variation is the persistence of low-value care and diffuse clinical criteria for certain medical procedures [28, 29]. Unclear decision processes may lead to stronger physician preferences and beliefs, unnecessary burden to the patient, variation between care providers and ultimately high costs. In order to diminish unwarranted practice variation, it is, therefore, crucial to better define appropriate care and to develop objective clinical standards with clear and objective criteria to guide treatment options. Together with carefully weighing patient preferences, high quality and patient-centered decisions can be reached. In this way, the problem of over- and under-use and subsequent practice variation will be addressed beforehand, at the time the patients present themselves, instead of assessing afterwards that the care was inappropriate and practice variation was observed. Improving clinical criteria can be achieved by a combined focus of consensus groups and mutual learning, and constructing algorithms. In textbox (shown below), we describe a study from our own group, in collaboration with Quin, Amsterdam, The Netherlands, in which we defined clinical criteria for hip surgery, by combining expert meetings with orthopedic surgeons and quantitative modeling techniques.

The hip surgery algorithm.

With respect to total hip and knee replacement procedures, widely accepted criteria are limited and based on low-quality evidence. The goal of the study was to develop an algorithm, derived from a prediction model, to guide the decision whether or not to perform a hip replacement surgery in osteoarthritis patients. We used patient characteristics, clinical parameters, patient reported outcomes and radiologic parameters in the model. We requested orthopedic surgeons to blindly revise about 400 medical files of patients with hip osteoarthritis who underwent hip replacement surgery or received conservative treatment. The purpose was to reach consensus about the optimal treatment considering all clinical information of the patient in two consecutive rounds. The outcome of this consensus procedure was used to fit a prediction model and to derive the algorithm. Based on patient characteristics, patient reported outcomes and clinical parameters, we managed to construct an algorithm with high predictive power. The addition of radiologic parameters further increased the predictive performance. The advantage of the model without radiologic parameters lies in the applicability and feasibility of clinical setting without a radiologic infrastructure. In conclusion, the algorithms have potential to help the orthopedic surgeon to decide whether surgery should be performed or not, based on the clinical situation of the patient. It is important to note that the algorithms are meant as tools to guide the treatment decision from a clinical point of view and as input in the conversation with the patient about treatment options. In addition to the algorithms, other considerations like patient’s preferences and social context should be weighed and taken into account as well. Atsma F, Molenkamp O, et al. Paper submitted

The approach of using clinical data for the construction of algorithms and application in the clinical decision process adheres to the concept of LHS, which was for the first time defined by the Institute of Medicine [30]. According to their Roundtable’s vision ‘a learning health system describes a health infrastructure characterized by evidence-based care that ensures proper decision-making for each patient and provider, and generates scientific evidence as a natural byproduct of the care process’. Progress in computational science, information technology and biomedical and health research methods have made it possible to develop LHS that enable knowledge generation by using information from daily practice in order to improve quality of healthcare targeted to the individual patient. LHS can be described by four key elements [31, 32]: an organizational infrastructure that facilitates the formulation of communities of patients, families, clinicians, researchers and LHS leaders to produce big data; large electronic health and healthcare data sets; quality improvement for each patient at the point of care brought about by the integration of relevant new knowledge generated through research; and observational research and clinical trials done in routine clinical care settings.

Recently, a systematic review of the state of the art of LHS was published. The authors concluded that although the idea of LHS has widely been supported by many researchers and literature on this topic is already available, the next step of implementation in clinical practice and evaluation of the impact on healthcare delivery and patient outcomes is often lacking [33].

A promising example of an LHS that succeeded and was adopted is the implementation of ImproveCareNow for children with inflammatory bowel disease. This LHS was scaled up within a national network (PEDSnet) that support research and quality improvement and inter-professional collaboration. The aim of PEDSnet was to create a network-based platform to enhance quality improvement and research across various pediatric specialties and geographical regions by, amongst other things, extensive engagement of stakeholders and linkage of different and relevant data sources [31].

Future research should focus on using algorithms and LHS in objectifying clinical criteria to support physicians and patients in treatment decisions, in which additional information about the social context and preferences of the patient are also taken into account. Research should aim at whether these clinical objectivations lead to less unwarranted clinical variation.

Conclusion and research perspectives

In conclusion, we argue that it is time for next steps in practice variation research. In addition to describing and signaling variation patterns the focus should be on a thorough understanding of unwarranted and warranted variation in medical practice and on defining strategies to improve the quality of healthcare. A key element in this new focus in research should be on the complex cohesion of network effects, reflective medicine, patient beliefs and objective criteria for treatment choices.

The first target point for future research pertains to the influence of collaboration within networks. In the field of practice variation research, networks research is still very young and future research should be conducted aiming at (i) methodologies to properly identify and quantify medical networks, and (ii) to understand in detail the effect of networks, physician’s belief, collaboration and feedback mechanisms on healthcare utilization and practice variation patterns. A second target point for research relates to generating objective criteria for medical decisions and its improvement potential on unwarranted variation. LHS and accompanying algorithms may be promising research objectives to obtain more uniform clinical criteria and may form target points to further understand and decrease variation in clinical practice. In this context, challenges in future research consist of defining relevant indicators, outcomes and levels of measurement to ensure that research will yield valuable and ready to use deliverables.

When our knowledge about the causes and mechanisms of clinical variation grows, effective improvement strategies can be defined. A one-size-fits-all approach will, however, not be realistic nor the ultimate goal. Improvement strategies should always take into account differences between specialties, cultures and should target local situations while also weighing costs and benefits in each specific situation. Stakeholders, especially doctors, should be aware of the problem of unwarranted variation and should be involved in the development of improvement strategies. It only works if doctors themselves acknowledge the presence of unwarranted variation and are ready to initiate change.