Effect of a Best Practice Alert on Birth-Cohort Screening for Hepatitis C Virus.

INTRODUCTION: We assessed the influence of a best practice alert (BPA) embedded within the electronic medical record on improving hepatitis C virus (HCV) birth-cohort screening by primary care physicians (PCPs).
METHODS: Screening by 155 PCPs was monitored during 2 consecutive 9-month periods before and after implementation of the BPA. All tests were reviewed to differentiate true screening from other testing indications.
RESULTS: Of 155 PCPs, 131 placed screening orders before and after BPA. Twenty-two PCPs started testing after BPA (P = 0.02). The number of tests placed and screening rates per PCP increased from 16 to 84 and from 3.3% to 13.2%, respectively (P < 0.0001). Before BPA, most PCPs rarely ordered screening HCV tests, whereas a small group of physicians generated most tests, indicative of an underlying power-law distribution. After the BPA, a new group of high-performing PCPs emerged, whose screening patterns were again characterized by a power-law distribution. However, pre-BPA test rates of individual PCPs were not predictive of their post-BPA rates. Overall, the introduction of the BPA narrowed the gap between low- and high-performing testers, indicating that modest increases in testing by a large number of low-performing PCPs could drive substantial improvement in program implementation. DISCUSSION: HCV birth-cohort screening by PCPs was shaped by an underlying power-law distribution. This distribution was preserved after the implementation of a BPA, although pre-BPA test rates were not predictive of post-BPA rates. Increases in test rates by high- and low-performing PCPs both contributed to the overall success of the BPA.

INTRODUCTION

Chronic hepatitis C virus (HCV) infection represents a major cause of cirrhosis, hepatocellular carcinoma, and death from liver disease, as well as a leading indication for liver transplantation in the United States and worldwide (1,2). The advent of direct-acting antiviral agents has shifted the paradigm of HCV, with most patients now being amenable to cure. An estimated 3.5 million people in the United States are currently infected by HCV (3). A large proportion of these patients are unaware of their infection (4). In 1998, the Centers for Disease Control and Prevention recommended HCV testing in high-risk populations; however, risk-based screening failed to identify more than half of all infected patients (5,6). Subsequent studies revealed that persons born between 1945 and 1965 accounted for three-quarters of all HCV infections in the United States (7). Consequently, the Centers for Disease Control and Prevention updated its HCV screening guidelines in 2012, recommending one time HCV antibody testing in this birth cohort, regardless of risk profile (5). This recommendation was endorsed by the US Preventive Services Task Force (8).

Despite these recommendations, HCV birth-cohort screening rates in the United States have remained low (9). A variety of interventions have been proposed to increase screening and uptake rates across the United States, with mixed results. We recently reported an improvement of HCV screening rates from less than 1% to over 10% at our institution within a few months after implementation of a systemwide, electronic medical record (EMR)-based best practice alert (BPA) (10). The aim of this study was to evaluate the longer-term impact of the HCV BPA on HCV birth-cohort screening rates and to study the test implementation rates of individual primary care physicians (PCPs).

METHODS

Establishment of the BPA

An EMR-based BPA to prompt HCV birth-cohort screening was established at NorthShore University Health System in July 2017. Test-naive patients born between 1945 and 1965 were automatically identified by the EMR system at the time of a clinic visit with their PCP. A screening alert was generated and displayed prominently within the EMR at the point of care. The alert was linked to an order for the HCV antibody test, allowing the PCP to place the order with a single keystroke, or to forgo testing at the PCP's or patient's discretion. In addition, PCPs had the choice of opting out of the BPA without entering a response. The BPA design was chosen to minimize the additional physician workload and to avoid alert fatigue.

Data collection and monitoring

The number of HCV BPA alerts, the number of HCV antibody tests ordered in response, and the number of times the BPA was rejected or ignored were tabulated for each PCP on a monthly basis. All data were stored in a secure, password-protected file within the institution's research drive. Monthly performance trends were provided for the investigators through an electronic dashboard.

Data analysis

HCV birth-cohort testing was analyzed for the two 9-month intervals before and after the implementation of the BPA, respectively. For the pre-BPA period, all outpatient primary care visits of patients born between 1945 and 1965 were extracted from the electronic data warehouse, which in turn was derived from the healthcare system's EMR. Patients who had not previously undergone HCV testing were identified. All encounters resulting in a test order underwent manual chart reviews to determine the indication for testing (true birth-cohort screening vs risk factor–based testing vs evaluation of liver disease). The numbers and percentages of patients eligible for birth-cohort screening who underwent testing were tabulated for each PCP.

For the post-BPA period, the number of BPAs triggered and the PCP's responses were automatically captured by analysis of the electronic warehouse data. A successful response was defined as the placement of an HCV antibody test order in response to a BPA. Conversely, a failed response was defined as the PCP's decision to decline placing the order or not to respond to the BPA at all.

Statistical analysis

The PCPs' HCV screening activities before and after implementation of the BPA were compared using χ2 analysis. For each PCP, the number and percentage of HCV tests ordered before and after establishment of the BPA were analyzed using χ2 tests with continuity correction, paired t tests, and least-square linear regression analyses. The relationship between HCV screening orders per physician and physician counts per orders placed was analyzed using least-square linear curve fitting after logarithmic transformation of both variables. All analyses were performed using Microsoft Excel statistical tools. A P value of less than 0.05 was considered significant.

RESULTS

A total of 155 PCPs were evaluated during the pre-BPA and post-BPA observation periods. Of these, 131 PCPs ordered HCV tests both pre-BPA and post-BPA implementation. Twenty-two PCPs ordered tests only after implementation of the BPA, whereas 2 PCPs never placed orders. The decision against screening was due to the physician's choice in most cases. The post-BPA increase in the number of PCPs participating in screening was statistically significant (P = 0.0192) (Table 1).

Table 1.

Pre-BPA and post-BPA participation by PCP in HCV screening

	Before BPA
	Success	Failure	Sum
After BPA
Success	131	22	153
Failure	0	2	2
Sum	131	24	155

The increase in the number of PCPs, who performed screening after BPA, was statistically significant (χ2 = 5.48, P = 0.0110).

BPA, best practice alert; HCV, hepatitis C virus; PCP, primary care physician.

During the 9-month pre-BPA period, patient visits generated 73,539 prompts for HCV testing, and test orders were placed in 2,424 cases (3.3%). In comparison, 13,012 of 98,835 (13.2%) post-BPA prompts resulted in test orders, an approximately 4-fold increase (P < 0.0001). Similarly, the number of tests ordered per PCP increased significantly after BPA (P < 0.0001) (Table 2).

Table 2.

Number and rate of successful orders before and after best practice alert (BPA)

	Before BPA, mean (SD)	After BPA, mean (SD)	t	P
No. of successful orders	16 (26)	84 (89)	9.90	<0.001
Success rate (%)	3.3 (4.7)	13.2 (13.9)	8.72	<0.001

Many PCPs rarely if ever participated in HCV screening, whereas a small number of PCPs generated a disproportionately large number of test orders. There was an inverse relationship between the number of physicians placing test orders and the number of orders placed per physician. The test distribution was markedly skewed, with approximately 100 PCPs ordering less than 10 screening tests, followed by a long shoulder representing a minority of high-performing PCPs ordering 50 or more tests (n = 17) (Figure 1a). Dual logarithmic transformation of the data revealed an essentially straight regression line with a highly significant R2 value of 0.803, consistent with an underlying power-law distribution (Figure 1b). A similar power-law distribution was found after BPA (Figure 1c,d), with a markedly increased range of patient counts.

Figure 1.

HCV testing follows a power-law distribution. The x axes display the number of patients for whom HCV test orders were placed by each individual PCP, in bins of 5 (a) or 10 (b) patients. The y axis indicates the number of PCPs placing orders for each bin. a/b and c/d represent pre-BPA and post-BPA analyses, respectively. Dual logarithmic transformation of the data in a and c revealed significant linear correlations, consistent with a power-law distribution (b and d). BPA, best practice alert; HCV, hepatitis C virus; PCP, primary care physician.

Interestingly, we found no significant overall correlation between pre-BPA and post-BPA screening performance by individual physicians, regardless of whether the absolute number of patients screened per PCP (Figure 2, top), or the percentage of test-eligible patients per PCP (Figure 2, bottom) were compared. This finding suggested that the screening performance of individual physicians relative to their peers was fluid.

Figure 2.

Pre-BPA screening performance does not predict post-BPA performance. The x axes display the absolute number (upper panel) or the percentage (lower panel) of screening-eligible patients, for whom test orders were placed by individual PCPs before the BPA. The y axes display the corresponding post-BPA number or percentage. Linear regression analyses showed no significant correlations. BPA, best practice alert; PCP, primary care physician.

To identify possible mechanisms underlying this apparent fluidity, we compared screening rates between the top 20% (n = 31) and bottom 80% (n = 125) performers for the pre-BPA and post-BPA intervals. We noticed that 21 of the 31 (68%) top-performing PCPs before BPA dropped to the bottom 80% group after BPA, despite the fact that 9 of them increased their personal screening rates. This marked change in rankings was due to the emergence of a subgroup of low-performing pre-BPA PCPs with robust increases in post-BPA rates.

A second, more subtle BPA effect became apparent when we analyzed the relative contributions of the top 20% and the bottom 80% performers to the overall number of patients being screened. Before BPA, the top 20% generated 1,734 of 2,424 screening orders, corresponding to 72% of the total. This percentage decreased to 54% after BPA, despite a marked increase in absolute numbers (7,067/13,012). Conversely, the contribution of the bottom 80% increased from 690/2,424 (29%) to 5,945/13,012 (46%) (Figure 3). Taken together, these data indicate that the overall increase in screening rates was the net effect of improvements by “low” as well as “high” screeners, and that the gap between high and low performers narrowed subsequent to the BPA.

Figure 3.

The relative contributions of high- and low-performing PCPs to the overall screening effort before BPA and after BPA. Before BPA, the top 20% and bottom 80% of PCPs generated 72% and 28% of all test orders, respectively. The corresponding results after BPA were 54% and 46%, respectively. BPA, best practice alert; PCP, primary care physician.

In a subgroup of 132 physicians, follow-up data were available 2–3 years after the initial BPA implementation. The annual numbers of screening tests submitted by individual physicians during consecutive years were significantly correlated, with correlation coefficients ranging from 0.35 to 0.56. Over time, the response to the BPA further improved and eventually leveled off at 23.2%. The cumulative effect of improving screening rates led to a 50% decline in the overall pool size of patient available for screening.

DISCUSSION

One-time HCV screening of all US baby boomers was first endorsed in 2012. Although conceptually simple and predicted to be cost-effective, the implementation of this mandate has been suboptimal, with reported screening rates ranging from 5% to 45% in commercial health systems. As a result, novel strategies have been proposed to improve adherence to the screening guidelines and to ultimately screen the entire at-risk population. Reports of successful potential strategies include house staff and faculty education (11,12), emergency room testing (13), concurrent testing at the time of screening colonoscopy (14), and a variety of automated reminders generated within the EMR system (4,11). A recent comparison of EMR-based alerts, repeated patient mass mailings, and direct patient solicitation suggested that the former were more effective and less expensive than the alternatives (15).

Our results with an EMR-driven BPA confirm previous published reports by us (10) and others (4,9,11,12,16). We observed an immediate and significant increase in the overall screening rates (Tables 1 and 2). In addition, our study revealed a previously unrecognized complexity of HCV screening patterns within our PCPs. Before BPA implementation, most PCPs rarely if ever ordered the HCV antibody test for their eligible patients. The bulk of the overall screening load was shouldered by a relatively small number of highly diligent physicians. This suggested an underlying power-law distribution, which was confirmed by our statistical analysis, as shown in Figure 1a,c.

Power-law distributions have been identified in a wide range of unrelated phenomena and scientific fields, including population dynamics, geologic and weather events, sociology, physics, and linguistics. Commonly referred to as Pareto's principle or the 80/20 rule, they refer to the universal phenomenon that for many outcomes, most consequences can be attributed to a minority of causes (17). In the field of medicine, they underlie the uneven utilization of medical diagnoses, medication usage, laboratory tests (18), and healthcare resources (19,20). To the best of our knowledge, they have not previously been documented in test implementation in hepatology or other medical specialties.

The successful introduction of the BPA provided us with an opportunity to further analyze its effect on physician behavior. As shown in Figure 1b,d, the power-law distribution was maintained after BPA. We initially hypothesized that the individual physician's propensity to participate in screening would remain invariant, and that the same group of high-performing physicians would carry the bulk of the workload after the BPA implementation. Unexpectedly, this was not the case. As shown in Figure 2, pre-BPA and post-BPA screening rates of individual physicians were only loosely correlated, indicating that pre-BPA screening habits of individual PCPs were not predictive of their post-BPA performances, despite the fact that a power-law distribution within the entire physician group was still preserved after the BPA. Our observation of a maintained power-law distribution but changing physician characteristics within this distribution is suggestive of a self-organizing behavior that has been documented in many complex systems (21).

To better understand this surprising result, we arbitrarily divided the PCPs into 2 groups, representing the “top 20% and bottom 80% of screeners, and tracked the movement of physicians between the 2 groups. Strikingly, approximately two-thirds of the initial top 20 dropped to the bottom 80 group after BPA, although approximately one-half of them managed to improve on their initial performance. This was due to the emergence of a new group of initially low-performing physicians who markedly increased their implementation rates.

Although our study highlights the contribution of a small subgroup of high-performing PCPs, it also shows the importance of modest improvements by the low-performing physicians. As shown in Figure 3, the relative contribution of PCPs in the bottom 80% increased from 29% (before BPA) to 46% (after BPA). Successful closing of the performance gap between the 2 groups underscores the potential of a simple EMR-driven intervention to affect behavioral changes in most PCPs. Ultimately, targeting specific subgroups of providers to improve overall screening may be ineffective because the complex system of primary care providers may adjust to restore the power-law distribution (21).

With regard to the efficacy of the screening alert, the post-BPA rate of 13.2% may seem unimpressive. However, seen in the broader context of changing practice habits in primary care medicine, modest improvements are the norm rather than the exception (22). Moreover, we did not observe any fatigue in the response rate to continued BPAs. In a subgroup of physicians followed for 2–3 more years, the screening rates improved even further before leveling off around 23%. Although the BPA did not change the underlying power-law distribution, over time even a relatively modest increase in the baseline screening rate resulted overall in a substantial cumulative effect with respect the total number of screened patients. HCV birth-cohort screening is unique in that it only requires 1 test per patient. Assuming 1 annual patient visit to the PCP, and an average test rate of 20%, a 5-year program would be predicted to accomplish screening in almost 80% of the at-risk population, an outcome that would not be achievable with the pre-BPA screening rate of 3.3%. Assuming generalizability and stability of our findings, the BPA would result in comparable screening rates to those of the US Veterans Affairs program—which actively enforces HCV screening, including the use of penalties for noncompliance (23).

In conclusion, we have demonstrated that a BPA embedded within the EMR markedly improved HCV age cohort testing rates by PCPs. Furthermore, we discovered that HCV birth-cohort screening was shaped by an underlying power-law distribution. Our study reveals the complexity of physicians' screening behavior and highlights the opportunity to improve screening performance by using the EMR system.

WHAT IS KNOWN

✓ The Centers for Disease Control and Prevention and the US Preventive Services Task Force recommend HCV screening in patients born between 1945 and 1965.

✓ Adherence to these recommendations has been suboptimal (5%–45%) in commercial health systems.

✓ EMR-based alerts have been evaluated as effective, low-cost interventions to improve screening.

WHAT IS NEW HERE

✓ HCV birth-cohort screening patterns by PCPs are shaped by an underlying power-law distribution.

✓ EMR-based alerts improve screening by high- and low-performing PCPs while leaving the underlying power-law distribution unaffected.

TRANSLATIONAL IMPACT

✓ EMR-based identification of unique, power-law–based screening patterns can pave the way for targeted, effective electronic interventions that inevitably improve HCV screening by PCPs.

CONFLICTS OF INTEREST

Guarantor of the article: Claus J. Fimmel, MD.

Specific author contributions: C.J.F.: study design. M.Q.K., Y.B., A.G., I.G., M.I.B., P.I., and A.S.: Data collection and data analysis. C.J.F., M.Q.K., and A.S.: writing of manuscript. All authors were involved in the proof-reading and revision of the final manuscript.

Financial support: C.J.F. was supported by an investigator-initiated grant (IN-US-334-1585) from Gilead Biosciences.

Potential competing interests: None to report.