An Automated Telephone Call System Improves the Reach and Cost-effectiveness of Panel Management Outreach for Cancer Screening.

This implementation and comparative effectiveness study compared an automated call system (ACS)-assisted method to enhance staff efficiency in panel management cancer screening outreach compared with standard outreach using manual calls. One panel manager assisted by the ACS at the intervention primary care practice completed outreach workflows for 43% more patients than 2 unassisted panel managers at comparison practices, with 78% more patients in the ACS-assisted panel management program ultimately having a preventive screening gap closed. Outreach cost per completion of 1 or more cancer screenings was $45.39 under standard procedures and $15.85 using the ACS-assisted method.

PANEL MANAGEMENT is a key task of primary care, involving proactively identifying patients with preventive and chronic care gaps, such as overdue cancer screening tests, and making efforts to close those gaps (Bodenheimer et al., 2014; Rogers et al., 2015). Panel management consists of both inreach (identifying and closing care gaps at the time of a clinic visit) and outreach (contacting patients outside a visit to notify them about a needed service and helping to arrange that service). Passive forms of outreach, such as mailing notices to patients, have limited efficacy (DeFrank et al., 2009). As a result, many practices and health delivery organizations deploy more active methods of panel management outreach, including having staff call patients overdue for services. Because this approach to panel management outreach is labor intensive, there is considerable interest in understanding whether automating some or all of the process may yield efficiencies.

One promising approach is automated call systems (ACSs), which use computer-generated calls with recorded messages and, in more advanced configurations, interactive voice or keyboard responses. Evidence suggests that the ACS may improve appointment attendance, adherence to medications, and self-management behavior (Posadzki et al., 2016). Although prior research has demonstrated that ACS outreach compared with no active panel management outreach can increase cancer screening rates (Baker et al., 2014; Cohen-Cline et al., 2014; DeFrank et al., 2019; Mosen et al., 2010; Phillips et al., 2015; Posadzki et al., 2016), no study has evaluated the effect of combining the ACS with personalized staff outreach on clinical outreach efficacy and screening rates compared with personalized staff calls alone.

We conducted an implementation and comparative effectiveness study to investigate the reach, effectiveness, and cost-efficiency of an ACS-assisted method designed to enhance staff efficiency in panel management cancer screening outreach compared with standard outreach using manual calls.

METHODS

Study setting

The study was conducted in 2017-2018 in 3 primary care practices (1 family medicine, 1 general medicine, and 1 women's health staffed by family physicians and internists) operated by UCSF Health, an academic health system in San Francisco, California. The 3 practices have a mix of faculty physicians, nurse practitioners, and resident clinicians collectively caring for a racially and ethnically diverse population of about 52 000 active patients, approximately equally covered by private and public (Medicare and Medicaid) insurance. UCSF Health receives pay-for-performance incentives from health plans and the state Medicaid waiver program for achieving preventive care and other performance targets.

Study population

All patients with at least 1 in-person visit or patient-initiated electronic message in the prior 36 months were considered actively empaneled at the given practice. U.S. Preventive Services Task Force guidelines were used to determine whether empaneled patients were up-to-date on cancer screening (U.S. Preventive Services Task Force, 2017). Eligible patients for panel manager outreach were women ages 50 to 74 years overdue for mammography (no mammogram in the past 2 years), women ages 21 to 64 years overdue for cervical cancer screening (no cervical cytology in the past 3 years [ages 21-64 years] or cytology-human papillomavirus co-testing in the past 5 years [ages 30-64 years]), and men or women ages 50 to 75 years overdue for colorectal cancer screening (no fecal occult blood test in the past 12 months or sigmoidoscopy or barium enema in the past 5 years or colonoscopy in the past 10 years). UCSF Health uses cancer screening registries extracted from the electronic medical record (EMR) database to generate monthly reports for each practice indicating patients who are overdue for cancer screening and eligible for panel manager outreach. At the time of the study, the registry indicated that 13 128 patients at these practices were overdue for at least 1 cancer screening.

Intervention

We compared a 2-week period of standard outreach at the family medicine and women's health practices with a 2-week period of ACS-supported outreach at the general medicine practice. Each of the 3 panel managers worked full time at their respective practices during the 2-week study periods.

Manual panel manager outreach intervention process

Prior to the study, each practice had 1 full-time panel manager on staff who was responsible for outreach to patients to close preventive and chronic care gaps. These 3 medical assistants received the same training in panel management from the UCSF Center for Excellence in Primary Care (https://cepc.ucsf.edu/panel-management-0) and had been working for several months prior to the study period using the existing UCSF Health primary care outreach workflows.

The standard workflow consisted of the panel managers accessing an electronic copy of the cancer screening registry on an excel spreadsheet that was updated on a monthly basis. The registry included the patient's name, phone number, demographic characteristics, and overdue cancer screening tests. Panel managers manually called each patient. If the patient answered, the panel manager ascertained whether the patient was still receiving care at UCSF Health and, if so, explained that the medical record indicated that the patient was overdue for a cancer screening. The patient was asked whether he or she had the screening done elsewhere. If the patient confirmed that the screening was done elsewhere, the panel manager requested documentation and updated the medical record accordingly. If the screening had not been done, the panel manager counseled the patient about the screening procedure and offered to help schedule the needed screening test or, in the case of fecal occult blood testing, mail a kit to the patient. If the patient did not answer the call, the panel manager left a message asking the patient to call back. Panel managers would attempt to reach a patient up to 3 times. Panel managers attempted to reach as many of the patients on the registry as they could, but invariably had many more patients on the registry overdue for screening than they could call. Panel managers at the standard workflow practices used an Excel spreadsheet modeled from the ACS electronic dashboard (described later) to track and document their outreach efforts and completion of workflows.

ACS-enabled panel manager outreach intervention process

One of the 3 practices (general medicine) piloted a novel ACS-enabled outreach method during the study period. The ACS was developed by CipherHealth in partnership with the UCSF Health Office of Population Health and Accountable Care. The ACS included an electronic workflow dashboard and automated call service (Figure 1). Cancer screening registry data were uploaded into the ACS dashboard, and during the study period the system was programmed to generate an automated call to 50 to 60 patients daily who were overdue for cancer screening. After confirming the patient's identity, the automated call delivered a message, recorded by one of the UCSF panel managers, explaining that it was addressing routine cancer screening. The patient was then presented with a series of questions to indicate whether the patient was still enrolled at the practice, was interested in staff helping to schedule a screening, and if so, would like the panel manager to call back in the morning or afternoon sometime within the next week. The responses to the automated call fed into the ACS electronic dashboard. The panel manager used the dashboard to identify patients who were contacted and requested a callback. The workflow from this point on was the same as for the standard workflow, with the panel manager manually calling the patient to ascertain more information and assist with scheduling or fecal occult blood kit mailing. The dashboard provided an organized electronic tool for the panel manager to document all relevant workflow actions taken.

Figure 1.

Automated call system-assisted panel manager workflow.

Two patient exclusion criteria were added for the ACS process due to current technical barriers: (1) patients who did not have English listed as their primary language in the EMR (the piloted ACS model only was available in English), and (2) patients who did not have a practice encounter within 18 months rather than the standard 36-month empanelment definition. The latter exclusion criterion was due to a California law stipulating that businesses can only send prerecorded telephone messages to clients who have an established business relationship based on a financial transaction within the prior 18 months.

Study measures

Using an implementation science framework (Glasgow et al., 1999), the study was designed to assess reach, effectiveness, and cost-effectiveness. Reach was assessed as the number of patients successfully contacted by phone. For the standard method, this was defined as a patient answering a call from a panel manager. For the ACS-assisted method, this was defined as a patient answering the ACS call and entering a response (either affirming or declining the offer for a panel manager to call to help schedule a screening, or indicating that they were no longer enrolled in the practice). Effectiveness was measured in 2 ways: (1) a process measure, defined as a panel manager completion of a care gap closure workflow (eg, scheduling a mammogram, entering into the EMR results of a colonoscopy that was done elsewhere), and (2) a clinical measure, defined as at least one of the overdue screening items being completed within 7 months of the initial outreach effort. Seven months was selected for follow-up to allow a reasonable amount of time for patients to schedule and complete needed tests. Reach and workflow closure were measured from information entered into the ACS dashboard and Excel flow sheet by panel managers. Completion of the actual screening test was measured from the EMR-based registry.

Analysis

We described the characteristics of the patients in the call-attempted groups in the intervention and control practices, and tested for differences in these characteristics using χ2 or t tests. Using an implementation science framework, our analysis emphasized absolute counts to assess reach and effectiveness (eg, number of patients called and number of patients responding to a call). We also computed rates for some measures using the denominator of the patients who had calls attempted (eg, percentage of patients called who were reached). For these measures, we computed 95% confidence intervals (CIs) for rates and used those to impute 95% CIs for the absolute numbers. For the effectiveness outcome of completing a screening test, among patients successfully reached by a call we computed the odds of having a test completed for the ACS-assisted group relative to the standard process group. We used logistic regression analysis to adjust the odds ratio for significant differences in underlying patient characteristics (age, gender, and insurance) in the intervention and control groups.

We also measured panel management outreach costs, consisting of the personnel costs for the panel managers and the fee charged by CipherHealth for a 1-year service agreement. Costs did not include the time spent by UCSF Health and CipherHealth personnel in designing the ACS.

RESULTS

At the 2 standard method practices, 7540 patients were overdue for at least 1 cancer screening, and 5588 were overdue at the ACS-assisted practice. During the 2-week study periods, the 2 panel managers at the standard method practices made phone calls to 420 patients, and the ACS-assisted system generated calls to 588 patients (Figure 2). Reflecting the different populations served by the family medicine, women's health, and general medicine practices, the patients called in the outreach ACS-assisted (general medicine) practice were significantly older and more likely to be male than those called at the standard workflow practices (Table 1).

Figure 2.

Comparative reach and effectiveness of panel management outreach by 1 panel manager assisted by an automated call system and 2 panel managers using standard manual call method.

Table 1.

Characteristics of Patients in the Standard Outreach Method Group and Automated Call System-Assisted Outreach Group

Patient Characteristic	Standard Outreach Method n = 420	Automated Call System-Assisted Method n = 588	P Value
Age, mean (standard deviation)	48.0 (12.4)	62.3 (7.0)	<.0001
Gender, n (%)
Female	408 (97.1)	432 (73.5)	<.001
Race, n (%)
White or Caucasian	216 (51.4)	293 (49.8)	.47
Black or African American	59 (14.1)	107 (18.2)
Asian	65 (15.5)	88 (15.0)
Native American or API	13 (3.1)	19 (3.2)
Other/unknown/declined	67 (16.0)	81 (13.8)
Ethnicity, n (%)
Hispanic or Latino	33 (7.9)	46 (7.8)	.68
Not Hispanic or Latino	369 (87.9)	523 (88.9)
Unknown/declined	18 (4.3)	19 (3.2)
Insurance, n (%)
Commercial	313 (74.5)	253 (43.0)	<.001
Medicaid	64 (15.2)	104 (17.7)
Medicare	30 (7.1)	221 (37.6)
Other	10 (2.4)	7 (1.2)
Missing	3 (0.7)	3 (0.5)

The 2 panel managers in the standard method practices reached 165 (95% CI, 156-174) of the 420 patients called, representing 39% (95% CI 37%-41%) of the patients called (Figure 2). Workflows were completed for 200 of these patients (95% CI 188-208), or 47% (95% CI 45%-49%) of the patients 420 called. At the end of the 7-month follow-up period, 138 of the patients (95% CI 132-144) had completed 1 or more of the overdue cancer screenings, representing 33% (95% CI 32%-34%) of the patients called. At the practice using the ACS-assisted method, 1 panel manager reached 326 (95% CI 310-342) of the 588 patients called, representing 55% (95% CI 53%-57%) of the patients called. The panel manager completed workflows for 285 (95% CI 271-299) of the 588 patients called (48%, 95% CI 46%-50%) and 246 (95% CI 237-255) of the patients completed 1 or more cancer screenings by the end of the 7-month follow-up, representing 42% (95% CI 40%-44%) of the 588 patients called. When examining only the denominator of patients successfully reached by a panel manager, the relative efficacies of the standard and ACS-assisted methods were similar after adjusting for differences in underlying patient characteristics. That is, among patients successfully reached, the likelihood of a patient completing a screening test was similar among the ACS and standard method groups (adjusted odds ratio 1.12, 95% CI 0.79-1.57).

The personnel cost including benefits for 2 weeks of panel manager time was $3132 per panel manager (Table 2). The annual fee for the ACS system was $20 000, or $769 prorated to a 2-week period. The overall costs for 2 weeks of panel management outreach were thus $6264 for the 2 panel managers at the standard workflow practices and $3901 for 1 panel manager and the ACS technology at the ACS workflow practice. Cost per completed workflow was $31.32 using standard procedures and $13.68 using the ACS-assisted method. Cost per completion of 1 or more cancer screenings was $45.39 under standard procedures and $15.85 using the ACS-assisted method.

Table 2.

Cost-effectiveness of Automated Call System-Assisted Versus Standard Method

Input Costs	Cost (Annual)	Employee Benefits (Annual)
1 FTE panel manager (salary)	$62 640	$18 792
ACS technology (fee)	$20 000	N/A
Operating Costs (Labor and Technology)	Per 2 wk
ACS-assisted method with 1 panel manager	$3 901.23
2 panel managers using standard manual system	$6 264.00
Cost-effectiveness	Completed Workflows	Completed Screenings	Cost per Completed Workflow	Cost per Completed Screening
Standard manual outreach by 2 panel managers	200	138	$31.32	$45.39
ACS-assisted method with 1 panel manager	285	246	$13.68	$15.85
Cost savings using ACS-assisted method	N/A	N/A	$17.64	$29.54

Abbreviations: ACS, automated call system; FTE, full-time equivalent; N/A, not available.

DISCUSSION

Adding an ACS to personalized staff outreach dramatically increased the “reach of the outreach.” Automating a portion of the outreach work was associated with almost twice as many completed cancer screenings with half the amount of labor. In a 2-week study period, 1 panel manager assisted by the ACS was able to generate calls to 40% more patients and contact twice as many patients as 2 panel managers working without the assistance of the ACS. The panel manager assisted by the ACS was able to complete workflows for 43% more patients than the 2 unassisted panel managers, with about 75% more patients in the ACS-assisted panel management program ultimately having a preventive screening gap closed over the ensuing 7 months. The effectiveness of the ACS-assisted program was largely accounted for by the improved reach rather than by improved efficacy. That is, the modality used to initially contact the patient (automated or live phone call) did not appear to significantly change the likelihood of that contact ultimately translating into a completed workflow and cancer screening among those answering a call. Adding the ACS markedly improved the cost-effectiveness of panel manager outreach, reducing the cost per closed care gap by nearly two-thirds, from $45.39 to $15.85. The cost-effectiveness was similar to a study conducted in 2012 of ACS-assisted panel management, which reported a cost of $13.14 per colon cancer screening gap closed (Phillips et al., 2015).

A key principle of using automation to gain efficiency is identifying components of a workflow that do not require human skills to be successfully performed. Under the standard workflow, panel managers spent many hours fruitlessly trying to contact patients by phone and leaving messages when patients did not answer calls. Our focus of the ACS-assisted approach was to address the inefficient first step in the workflow—generating calls to patients to make initial contact and explain the reason for the outreach. The ACS-assisted process allowed the panel manager to focus her time on patients who responded to the automated calls with an interest in being contacted to address a screening gap, resulting in engaging many more patients than a panel manager unassisted by the ACS. The electronic dashboard integrated into the ACS may also have improved the efficiency of documenting panel management activities and increased the panel manager's sense of accountability for outreach tasks. The ACS added a predetermined number of patients to the call list every day, removing discretion on the part of the panel manager in this aspect of outreach. Future integration of the dashboard into the EMR might add even greater efficiency to the ACS-assisted workflow.

Our cost-effectiveness analysis was framed from the perspective of a health care organization committed to performing active panel management outreach and interested in understanding how to achieve efficiencies in outreach. Even with an ACS-assisted method, there is still a cost to an organization to perform active outreach. UCSF Health operates in a payer environment with incentive payments contingent on closing preventive care gaps. An improvement of a few percentage points in cancer screening rates can have major financial implications. For organizations with similar incentive programs, there are sound financial reasons as well as a quality of care rationale for efficient and efficacious panel management outreach.

Our study has several important limitations due to its pragmatic design. We did not randomize patients or practices. The evaluation was not resourced to be able to examine more than 2-week periods at each of the practices. The patient population at the control practices was younger and more likely to be female than the population at the intervention practice. Although we considered that these demographics might affect patients' responsiveness once contacted, we believe that these differences are unlikely to explain the large difference in reach we observed since the major factor driving reach is the number of calls being generated by the panel management process. When we adjusted for these demographic differences, there was no significant difference in the odds of a contacted patient closing a screening care gap. However, even if we apply the percentage of contacted patients in the control group closing a care gap (33%) to the number of patients contacted in the intervention group, the absolute number of patients with care gaps closed in the intervention group would still be much greater than that in the control group due to the increased number of patients contacted using the ACS-assisted method. As noted, patients with a preferred language other than English were not included in the ACS group but were included in the manual call control group. However, only 4 patients in the manual group had a preferred language other than English, and any potential language barriers impeding successful panel management outreach for these few patients would not meaningfully change the overall results.

Future research is needed to better understand which types of patients are most responsive to automated calls to guide targeting of this mode of outreach. This line of inquiry would also clarify whether there is a “digital divide” in responsiveness to ACS calls based on patients' race-ethnicity, socioeconomic status, English fluency, and related factors, which might require special attention to ensure that an ACS approach to panel management outreach does not widen disparities. Research is also needed to understand whether the ACS approach may complement other emerging modes of panel outreach, such as bulk messaging of screening gap notifications to patients through an EMR patient portal.

In summary, our pragmatic implementation study suggests that an ACS-assisted method improves the efficiency of panel management outreach, allowing more patients to be reached and increasing the productivity of panel manager staff. Based on this evidence, UCSF Health is in the process of adopting the ACS-assisted method for all its primary care practices. As part of this expansion effort, we have translated and recorded the ACS into 4 additional languages (Spanish, Cantonese, Mandarin, and Russian). We have also pursued advanced technical integrations between the CipherHealth system and the EMR, which has allowed UCSF to develop additional panel management-driven workflows using the ACS, such as hypertension management, depression care management, and specific outreach designed to improve the accuracy of primary care provider panel size.