Self-Scheduling Process Efficiency and Utilization of Online Self-Scheduling of Lab Tests: A Retrospective Analysis of Self-Scheduled Appointments for COVID Testing.

Introduction: The COVID 19 pandemic increased the need for rapid and accurate diagnostic testing for COVID. When testing became available, a systems response was needed to efficiently accommodate the high-volume flow of patients who needed testing. Self-scheduling of COVID testing was developed to help patients safely and efficiently schedule their COVID testing online or with a mobile app.
Methods: We captured the counts of COVID test appointments, time patients spent in scheduling COVID test appointments, appointment lead times, and no-shows for COVID test appointments. For 17 months of self-scheduling, we retrospectively compared self-scheduling with the concurrent staff scheduling of COVID tests.
Results: From November 2020 through March 2022 there were 619 104 scheduled appointments for COVID testing with 22% (136 252) being self-scheduled. For asymptomatic self-scheduled COVID tests, accounting for 10.3% (63 605/619 104) of total COVID tests scheduled, median time to self-schedule was 3.1 min, interquartile range (IQR) [2.4,4.7]. For symptomatic self-schedulers accounting for 11.7% (72 647/619 104) of total COVID tests scheduled, the median time to self-triage and self-schedule was 5.8 min, IQR[4.3,8.9]. Self-scheduled COVID appointments increased to 44% (42 387/97 086) of the total COVID appointments during the peak month of January 2022. Median appointment lead time for symptomatic self-scheduled COVID test appointments was 6.6 h compared to 2.9 h (P < .0001) for symptomatic staff scheduled appointments. However, adjusting for the 24% (32 194/135 252) that self-scheduled during hours when testing was unavailable, the median appointment lead time for symptomatic self-scheduled patients dropped to 3.6 h. No-shows were 2.5% for self-scheduled appointments compared to 3.0% no-shows that were staff scheduled (odds ratio 0.83, P < .0001).
Conclusion: COVID testing was self-scheduled for a large percent of scheduled COVID tests, taking patients only a few minutes to complete. Self-scheduling use increased over time, associated with a decreasing use of staff scheduled appointments and lower no-shows.

Introduction

During the COVID-19 pandemic there was a large surge in the need for COVID-19 testing. Both patients and workplaces had reasons for COVID testing. Patients wanted to know if COVID was causing their symptoms and workplaces required symptomatic and asymptomatic patients to quarantine based on a positive test. COVID testing was also required for travel and onsite event attendance.

Processes for COVID testing have varied to meet the demand while protecting testing personnel and using physical distancing for patients to avoid contact with others. Drive-through testing was implemented to help isolate and distance patients from one another and protect testing personnel by outdoor testing.[1-3] For symptomatic patients, the testing process was combined with a triage process to direct severely symptomatic patients to acute care settings such as an emergency department, with less symptomatic patients able to get outpatient testing without further evaluation.

Like other institutions, Mayo Clinic had to develop and change testing processes during the course of the pandemic. To help patients efficiently schedule their COVID test, a process was developed that allowed patients to self-schedule a COVID test either online or by mobile. Prior to COVID, Mayo Clinic had other self-scheduling opportunities such as for well-child visits and screening mammograms.[5,6] However, prior to COVID test self-scheduling, patients were not able to request and schedule a lab test in a seamless single online encounter as they could with the COVID testing we describe in this study. We examined measures of efficiency and utilization of the COVID test self-schedule process over 17 months of use.

Methods

Setting

The study took place at Mayo Clinic Rochester (MCR) and the Mayo Clinic Health System (MCHS). Mayo Clinic is a multispecialty healthcare institution located in multiple locations in the United States and internationally. We examined all scheduled COVID test appointments in MCR and MCHS from November 2020 through March 2022. Scheduled COVID tests were available to registered Mayo Clinic patients.

COVID Testing Process Background

For the duration of the study, Mayo Clinic had processes for both symptomatic and asymptomatic patient COVID testing. COVID testing at Mayo Clinic evolved over the course of the pandemic. This study exclusively examines the scheduled COVID testing process. During the initial stages of the pandemic, COVID testing was generally an unscheduled process which consisted of 1) a telephone triage process, 2) a testing order creation, and 3) directions to a drive-through testing site. Outdoor drive-through testing in the cold Minnesota winter was not a good option for patients or staff so testing moved indoors later in 2020.

Indoor testing required a scheduled process to manage surges of demand and keep potentially infectious patients from congregating indoors while waiting for a test. The COVID scheduling process worked in parallel with measures to increase physical distance between patients. With scheduled COVID tests, increased demands for testing could be matched with the capacity of the testing personnel. The matching of demand and capacity using the scheduling process helped to avoid excessive crowding of patients indoors where person to person transmission was more likely to occur. Physical distancing was also maintained by having patients wait in their vehicles until contacted by phone or text when staff was ready to do the test. After being called to have their test, patients exited their vehicles and were met at the testing facility by greeters who directed them in and out of the isolated testing rooms in a flow pattern to maintain physical distancing.

Both symptomatic and asymptomatic patients were able to get a COVID test scheduled by telephone or via online or mobile within the patient portal (Mayo Clinic Patient Online Services). The flow for this is shown in Figure 1.

Figure 1.

Process flows for scheduling COVID tests. Self-scheduling and staff scheduling both had separate process flows for symptomatic and asymptomatic patients for a total of 4 different appointment processes. Process flows are numbered 1 through 4, and the process step sequence is added as a letter after the process flow number. The same letter indicates a similar process step for comparison between process flows. Red outlined process steps had completion times available to analyze.

COVID Testing Process Flows

Figure 1 shows the process flows for patients to get a scheduled COVID test. The letter attached to the numbered process flow indicates a similar step in the process. For example, the letter B is the pre-schedule step (triage step in process flows 1 and 3, pre-schedule questions in flow 2). The C step is the scheduling step. The red colored process boxes and arrows identify sections of the process map where time data was available.

As shown in Figure 1, there were 4 processes available for patients to schedule a COVID test, two process flows for online/mobile self-schedulers and two to schedule by telephone. Patients wanting to self-schedule online chose between a symptomatic or asymptomatic scheduling process. Online symptomatic patients initially went through a self-triage questionnaire process step (1B, Figure 1) that determined whether a COVID test should be offered. Those with emergency symptoms identified from the self-triage questions were excluded from self-scheduling and recommended to seek emergency care. If no exclusions were identified in self-triage, they were presented an option to self-schedule a COVID test (1C, Figure 1). Multiple online uses of the self-triage self-scheduling (process 1, Figure 1) were allowed over the course of the study. However, only one use of the self-triage self-scheduling process was allowed in the same calendar day. Those online or mobile without declared symptoms could elect to “direct self-schedule” (process flow 2, Figure 1), bypassing the more rigorous self-triage question set. Those in the asymptomatic self-schedule process were blocked from further scheduling if they already had an upcoming scheduled COVID test appointment.

For patients asking about a COVID test appointment by telephone, two flows were available depending on whether they had symptoms or not. Callers would start with a telephone screener who would direct those with symptoms to the nurse COVID triage. After the nurse triaged for emergencies and other exclusions (3B, Figure 1), callers would be transferred to a staff scheduler to schedule the COVID testing appointment (3C, Figure 1). Asymptomatic patients could directly schedule with the telephone screener who assumed the role of staff scheduler with no telephone handoff (4C, Figure 1). For both symptomatic and asymptomatic telephonic patients, staff schedulers used a streamlined process for COVID test scheduling (3C and 4C, Figure 1). A “one-click” scheduling process was developed specifically for COVID tests so that staff schedulers could bypass the usual review and updating of demographic and insurance information. The “one-click” process required staff scheduler input of a free text field that was used to document whether the patient was symptomatic or asymptomatic.

All available COVID appointment slots 7 days into the future were visible and selectable for both self and staff schedulers. There were no hidden or reserved appointments within a 7-day appointment window for any of the appointment processes, whether staff or self-scheduled. However, staff schedulers had the option to schedule beyond the 7-day window that was not available to self-schedulers.

Self-triage and self-scheduling can be illustrated by some screen shots of what the user saw when going through the self-triage and progressing to self-scheduling. Supplemental file image 1 shows the COVID start page. Supplemental file image 2 shows a patient self-triage question page from the COVID self-triage algorithm. Supplemental file image 3 shows the result page of the triage. Supplemental file image 4 shows an appointment selection page.

Staff scheduling (both for asymptomatic and symptomatic patients) occurred during the hours of 7 AM to 7 PM, 7 days a week for much of the COVID scheduling process. Self-scheduling was available 24/7 with only limited interruptions for software issues. The COVID nurse line had variable hours during the pandemic, up to 24/7 during the first year of the pandemic but was 6 AM to 9 PM during the omicron variant surge in January 2022. The self-triage and self-scheduling process was completely software driven except for a software generated COVID test order that needed to be bulk signed by a provider delegated to sign off on the COVID test.

Data Collection

We collected all scheduled COVID test appointments occurring from November 1, 2020, through March 31, 2022. For self-scheduled test appointments, we had the dates and times (to the minute) of initiation and completion of self-triage and direct (asymptomatic) self-scheduling (1B and 2B, Figure 1). For staff scheduled COVID test appointments we had the time (in seconds) that the staff scheduler spent in the medical record in making the appointment (3C and 4C, Figure 1). For all the appointments (self or staff scheduled) we had dates and times when the appointment was made (booked) and the dates and times of when the patient was scheduled to be tested.

A sample of 2110 nurse phone calls for COVID were used to compare median call times with median online times. We had only aggregated call length data to work with. Call times from nurse telephone data were aggregated into 48 time segments (each ½ hour) throughout the day. Total call time and total calls were available for each 30-min segment of time for the 17 months of the study. Fortunately, there were 2110 of the 24 761 30 min time segments during the 17 months that had only one call. We used these 2110 individual call times for a Kaplan-Meier curve to compare with the Kaplan-Meier curves for individual online time for the online users. We also collected the free text field described above used by staff schedulers to categorize the scheduled COVID testing appointments as symptomatic or asymptomatic.

Timing of the appointment schedulers time was limited to the time spent in the medical record to schedule and did not include telephone transfer times from nurses or telephone screeners. As noted above, the staff scheduled COVID testing appointments were streamlined; questions about possible changes to demographics and insurance were waived.

No show appointments were measured by count of scheduled patients who did not arrive for the previously scheduled appointment.

Our focus was on successfully completed self-scheduled COVID test appointments. Comparison of self-scheduling process failures to staff scheduling process failures was not in the scope of this study. The counts in Figure 1 are the total numbers of successfully scheduled COVID test appointments for each of the 4 separate processes. Counts of dropouts from each process step were not collected for this study.

Outcomes Measures

Primary measures were utilization of self-scheduling, time for individuals to self-schedule, and appointment outcomes of lead times and no-shows. Briefly, utilization was measured as monthly counts of self-scheduled COVID lab appointments and proportion of self-scheduled COVID lab appointments to total scheduled COVID lab appointments. Individual self-scheduling time was the minutes from the initiation of the self-scheduling process (starting from the self-triage or pre-schedule questions) to the time the COVID test was booked. Appointment lead time was the time from completion of the appointment scheduling (when it was booked) until the scheduled COVID swab collection time. No-show counts were those scheduled for a COVID test who did not arrive for their COVID swab.

Table 1 completes the description of the outcome measures by defining secondary measures and associating all measures with process flows and steps in Figure 1.

Table 1.

Primary and Secondary Outcomes Measures. Process Flow Numbers and Letters Refer to Figure 1.

Outcome measure	Description (process flows and steps numbered in Figure 1)	Process flows used (see Figure 1)
Primary measure: utilization over time
Longitudinal change in utilization of self-scheduling and staff scheduling for COVID test visits using self-scheduling and staff scheduling	Counts per month of appointments self-scheduled (sum of scheduled appointments in self-scheduled flows 1 and 2). Counts per month of appointments staff scheduled (sum of scheduled appointments in staff scheduled flows 3 and 4).	Self-scheduling and staff scheduling
Primary measure: individual time to complete self-scheduling a lab test
Symptomatic self-scheduled start to end completion time from start of self-triage questions to appointment scheduled	Time from start of self-triage questions to time COVID test was booked online/mobile (time from start of 1B to completion of 1C)	Process flow 1
Asymptomatic self-scheduled start to end completion time from start of pre-schedule questions to appointment scheduled	Time from start of pre-schedule questions to time COVID test was booked online/mobile (time from start of 2B to completion of 2C)	Process flow 2
Primary measure: outcome of scheduled appointment
Appointment lead time	Time in hours from when the appointment was booked to the scheduled appointment time (time when COVID swab was to be collected).	Process flows 1,2,3, and 4
No-show percent	Percent of scheduled appointments that patient did not arrive for COVID swab	Self-scheduling and staff scheduling
Secondary measure: 24/7 utilization of self-scheduling
Patient convenience of self-scheduling after normal business hours	Percent of time that self-scheduled booking time was after normal business hours (between 7PM and 7AM all days of the week)	Self-scheduled process flow
Secondary measure: individual times to self-triage, nurse triage, or preschedule triage component
Self-triage time (symptomatic)	Time from start of self-triage questions to finish of self-triage questions (time in 1B)	Process flow 1
Nurse triage time (symptomatic) in the staff scheduling process	Time on telephone with patients in dedicated COVID nurse triage calls (time in 3B)	Process flow 3
Time for prescheduling questions in the asymptomatic self-schedule process	Time from start of pre-schedule questions to finish of pre-schedule questions (time in 2B)	Process flow 2
Secondary measure: individual times for self-schedule components
Self-schedule (symptomatic) completion time component	Time from end of self-triage questions to time COVID test was booked online/mobile (time from end of 1B to completion of 1C)	Process flow 1
Staff scheduler time (symptomatic)	Time in staff scheduling (3C)	Process flow 3
Self-schedule (asymptomatic) completion time component	Time from end of pre-schedule questions to time COVID test appointment booked online/mobile (time from end of 2B to completion of 2C)	Process flow 2
Staff scheduler time (asymptomatic)	Time in staff scheduling (4C)	Process flow 4

Appointment Lead Time Adjustment

Appointment lead times can be misleading when comparing different scheduling processes. In the case of COVID test scheduling, self-scheduling was 24/7, but staff scheduling and the COVID test swab appointments were not available during much of the evening and early morning hours. Consider the following example. A patient self-scheduling at 8 PM wanting the first appointment possible could book an appointment at earliest when the testing site reopened at 8AM. The 8 PM booking time to 8 AM scheduled visit time would be an appointment lead time of 12 h. On the other hand, the same patient at 8 PM wanting to schedule an appointment by telephone would be told to call back during scheduler hours at 7 AM. For the telephone patient who waited overnight to call back for an 8 AM appointment, the lead time could be 1 h or less compared to 12 h for the self-scheduler. We adjusted for this by subtracting out the lead time minutes that occurred when testing sites were not open.

Statistics and Ethics

We used Stata 17.0 (College Station, Texas) for statistical analysis. Chi square and odds ratios were used to compare the demographic categories of those completely self-scheduling to those who had staff scheduling of COVID testing appointments. Time to event analysis using the log rank test was used to compare times to completion for self-triage and nurse triage as well as appointment lead times. The median test was used to compare median times involved in scheduling. This study met the institutional review board criteria for exemption (IRB-2020-006809).

Results

Over the 17-month study period there were a total of 619 104 appointments successfully self and staff scheduled for the COVID testing sites across the Mayo Clinic Health System and in Rochester, Minnesota. Of these scheduled appointments, 22% (136 252/619 104) were completely self-scheduled; 78% (482 852/619 104) were scheduled by staff. Of the self-scheduled, 53% (72 647/136 252) went through the self-triage flow and 47% (63 605/136 252) elected to go the asymptomatic direct self-schedule flow and bypassed the self-triage algorithm. There were 320 566 calls to the nurse COVID line with a total of 39 762 h on the telephone with patients and an additional 13 415 h devoted by nurses to work in after call documentation.

Demographic Comparison

Patient demographics for self-scheduled COVID test appointments were compared with telephonically staff scheduled appointments (Table 2). Scheduled test appointments showed statistically significant demographic differences between those self-scheduled (combined symptomatic and asymptomatic) and telephonically staff scheduled appointments (combined symptomatic and asymptomatic).

Table 2.

Patient Demographics Associated with Scheduled COVID Test Appointments. Exclusively Self-Scheduled Appointments are Compared to Those Using Staff Resources (Telephone Nurse Triage and Staff Appointment Schedulers).

Demographic	All scheduled COVID Tests, N= 619 104 n(%)	Completely Self-Scheduled, N = 136 252 n(%)	Staff scheduled including nurse telephone triage if indicated, N = 482 852 n(%)	P value*	Odds Ratio, self-scheduled to staff scheduled [CI 95%]
Age (years)
0-17	174 320 (28.16)	40 418 (29.66)	133 902 (27.73)	<.0001	1.09 [1.08 - 1.11]
18-34	163 888 (26.47)	38 575 (28.31)	125 313 (25.95)	<.0001	1.13 [1.11 - 1.14]
35-49	127 553 (20.60)	32 232 (23.66)	95 321 (19.74)	<.0001	1.26 [1.24 - 1.28]
50-64	95 306 (15.39)	17 131 (12.57)	78 175 (16.19)	<.0001	.74 [.73 - .76]
65-79	49 079 (7.93)	7216 (5.3)	41 863 (8.67)	<.0001	.59 [.57 - .60]
80 +	8956 (1.44)	680 (.50)	8276 (1.71)	<.0001	.29 [.27 - .31]
Mean age (SD)	34.0 (21.22)	31.3 (19.7)	34.8 (21.6)	<.001
Sex
Female	350 271 (56.58)	82 765 (60.74)	267 506 (55.4)	<.0001	1.25 [1.23 - 1.26]
Race
White	539 052 (87.07)	123 501 (90.64)	415 551 (86.06)	<.0001	1.57 [1.54 - 1.6]
Black	22 456 (3.62)	2377 (1.74)	20 079 (4.16)	<.0001	.41 [.39 - .43]
Asian	19 906 (3.22)	5092 (3.74)	14 814 (3.07)	<.0001	1.23 [1.19 - 1.27]
Other/Not disclosed	37 690 (6.09)	5282 (3.88)	32 408 (6.71)	<.0001	.56 [.54 - .58]
Ethnicity
Hispanic	39 011 (6.30)	4947 (3.63)	34 064 (7.05)	<.0001	.50 [.48 - .51]
Not Hispanic	561 679 (90.72)	128 459 (94.28)	433 220 (89.72)	<.0001	1.89 [1.84 - 1.94]
Undisclosed/Unknown	18 414 (2.97)	2846 (2.09)	15 568 (3.22)	<.0001	.64 [.61 - .67]

*Ho: proportion self-scheduled equals staff scheduled.

Patient Utilization and Convenience of Self-Scheduling

Self-scheduling of COVID tests showed a major increase in utilization over the course of the pandemic. Figure 2 shows the variability of scheduled COVID test volumes over the 17 months from November 2020 through March 2022 along with the increasing utilization of self-scheduled COVID test appointments during that time. Self-scheduled COVID appointments rose from 1489 in December 2020 to 42 387 in January 2022.

Figure 2.

Counts of COVID test scheduling activity and nurse line COVID calls by month.

Patients took advantage of the 24/7 convenience to self-schedule. Over 30 000 self-scheduled COVID test appointments were scheduled between 7 PM to 7 AM (weekdays and weekends), accounting for 23.6% (32 194/136 252) of the total self-scheduled appointments.

Figure 2 shows the counts over time of self-scheduled COVID test appointments compared to staff scheduled. Nurse COVID call counts are also shown.

Figure 3 shows that the percent of self-scheduled COVID test appointments was 4.3% (1489/34 855) in December 2020 and increased to 44% (42 387/97 086) at the peak of nearly 100 000 appointments scheduled in January 2022.

Figure 3.

Self-scheduled COVID tests, staff scheduled COVID tests, and nurse COVID triage calls as a percent of total scheduled COVID test appointments over time.

Time Involved in Self-Scheduling and Staff Scheduling

The median time of the self-scheduled self-triage process flow from start of symptomatic self-triage to appointment scheduled was 5.8 min, IQR [4.3,8.9] (time from Figure 1 process 1B start to 1C completion). Median time for an asymptomatic self-scheduled appointment from pre-schedule questions through appointment completion was 3.1 min, IQR [2.4, 4.7] (time from Figure 1 process 2B start to 2C completion). The 5.8 min median time to completely self-schedule a COVID test for a symptomatic patient was significantly greater than the median asymptomatic patient self-scheduling time of 3.1 min (P < .0001). We did not have telephone hand off times and telephone screener times, so an “apples to apples” staff scheduling comparison to self-scheduling was not available (see Figure 1). However, the overall self-scheduling times given above (both asymptomatic and symptomatic) were both significantly less (P < .0001) than the median nurse COVID triage call time of 7.11 min IQR [3.9,10.68] (time in 3B, Figure 1).

Table 3 shows a comparison of the available component times involved in self-scheduled and staff scheduled appointments (definitions in Table 1). Staff schedulers appeared to efficiently schedule COVID tests with median staff scheduling time under 1 min whether or not preceded by nurse triage.

Table 3.

Times in Components of Self-Scheduled and Staff Scheduled Appointments. Appointment Lead Times for Self-Scheduling and Staff. Adjustment for Appointment Lead Times Explained in Text.

	Symptomatic Patients			Asymptomatic Patients
Measures of elapsed time in triage and scheduling	Patient Online self-triage then self-schedule. Figure 1, Flow 1; n = 72 647	Telephonic staff scheduled (nurse triage may be used). Figure 1, Flow 3; n = 356 121	P-value*	Patient Online direct self-scheduled, with pre-schedule questionnaire. Figure 1, Flow 2; n = 63 605	Telephonic staff scheduled. Figure 1, Flow 4; n = 126 707	P-value*
Median triage time (symptomatic) or pre-schedule time (asymptomatic) in minutes. Corresponds to process map step B, Figure 1. [interquartile range]	3 [2,4]	7.11 [3.9, 10.68]	<.0001	1 [1,2]	Not Available	NA
Median schedule time in minutes. Corresponds to process map step C, Figure 1. [interquartile range]	2.32 [1.48, 4.37]	0.90 [0.58, 1.45]	<.0001	1.82 [1.2,3.2]	0.90 [0.57, 1.52]	<.0001
Median appointment lead time, in hours [interquartile range]	6.57 [2.56, 19.04]	2.92 [1.27, 17.59]	<.0001	18.6 [4.76, 43.13]	19.74 [2.60, 50.66]	<.0001
Median adjusted appointment lead time in hours [interquartile range]	3.57 [1.82, 18.14]	2.89 [1.26, 17.57]	<.0001	16.79 [2.97, 42.15]	19.75 [2.56, 50.77]	<.0001

*Ho: Medians are equal.

Figure 4 shows the Kaplan-Meier curves for the times to completion of asymptomatic self-preschedule questions (2B, Figure 1), symptomatic self-triage questions (1B, Figure 1), and nurse telephone triage (3B, Figure 1). These were all pairwise significantly different (P < .0001 by the log rank test).

Figure 4.

Kaplan-Meier time to event. Cumulative percent of patients whose assessment for COVID test eligibility (triage or pre-schedule questions) were less than the minutes on the horizontal axis. Corresponds to time in process flow steps 1B, 2B, and 3B in Figure 1.

Appointment Outcome Comparison of Self-Scheduled Versus Staff Scheduled

Appointment lead times (defined in Table 1) were significantly different between the self and staff scheduled COVID tests (P < .0001, Figure 5).

Figure 5.

Kaplan-Meier time to event. Cumulative percent of COVID tests scheduled where appointment lead times were less than or equal the hours on the horizontal axis.

Much of the differences in these Kaplan-Meier time to event curves can be explained by self-schedulers booking appointments during the evening and night. When adjusted for differences between the 24/7 availability of self-scheduling and testing appointments starting at 8 AM, both the asymptomatic and symptomatic flows track together well but are still statistically different (Figure 6, P < .0001 for both pairs of flows).

Figure 6.

Kaplan-Meier time to event, adjusted. Cumulative percent of COVID tests where appointment lead times were less than or equal the hours on the horizontal axis. Appointment lead times were adjusted for self-scheduling during hours when COVID testing was not available (see Methods: Appointment Lead Time Adjustment).

No show percent was significantly lower in the self-scheduled group at 2.5% (3372/136 252) compared to 3.0% (14 260/482 852) in the staff scheduled group (P < .0001, OR 0.83, CI 95; 0.80, 0.87).

Discussion

Principal Findings

For 17 months, patients self-scheduled COVID testing appointments 22% (136 252/619 104) of the time. At the peak COVID omicron surge in January 2022, 44% (42 387/97 086) of the COVID test appointments were self-scheduled. Median self-schedule time for asymptomatic patients was 3.1 min, stretching only to 5.8 min for symptomatic patients who answered self-triage questions prior to scheduling.

Patient Implications

The 44% of appointments self-scheduled during the January 2022 COVID omicron surge is evidence for excellent patient utilization of the self-scheduling process. Evidence for patient convenience is the 23.6% self-scheduling occurring between 7 PM and 7 AM. Speed of self-scheduling was demonstrated by 75% of asymptomatic patients booking a COVID test appointment in under 5 min; 75% of symptomatic patients booked an appointment in under 9 min. Using adjusted appointment lead times as a measure, patients appeared to have similar scheduling outcomes whether self-scheduled or not.

Concern about symptoms and achieving a “peace of mind” have been motivators for patients to seek COVID testing.[8,9] Our results showed that patients could rapidly and conveniently self-triage and self-schedule COVID tests 24/7. It is possible that the ready availability of self-scheduling COVID tests had an impact on the peace of mind that patients sought.

Practice Implications

We were fortunate to have the self-triage and self-schedule process in place during the unanticipated surge of COVID variant omicron in the winter of 2021-2022. As shown in Figure 2 there was a major surge in volume of COVID tests scheduled. Much of that COVID omicron surge was absorbed by the increase in patient self-scheduling. As shown in Figure 2 and Figure 3, the nurse COVID call volumes and staff scheduler volumes were held in check as self-scheduling increased to fill the demand.

Although not reported in results, there was an expected strong linear relationship of staff scheduled appointments to the nurse COVID line call volume. The r2 of this relationship was 0.92 with a linear coefficient of 0.57 [CI95; 0.48, 0.65]. This meant that 92% of the nurse call volume could be accounted for by a linear relationship with staff scheduled appointment volume, with each incremental 100 staff scheduled appointments associated with about 57 nurse triage calls. With this strong linear relationship, it appears reasonable to make some estimates about potential nurse COVID calls avoided by self-scheduling. From a managerial epidemiology viewpoint, we can estimate that the 136 252 self-scheduled appointments saved 0.57 × 136 252 nurse COVID calls (77 520) in addition to 136 252 telephonic scheduler interactions. Based on the average call time for the COVID nurse triage (7.44 min) and the average nurse after call work time associated with each call (2.51 min), the estimated nurse time saved by self-scheduling was 12 822 h with a 95% confidence interval of 10 875 to 14 774 nurse hours saved, based on the linear regression confidence interval above. We have the actual derived times of staff scheduling activity while in Epic. Based on this, 136 252 self-scheduled appointments supplanting an equal number of staff scheduled appointments would be a saving of 136 252 × 1.28 min, or 2913 scheduler hours.

The appointment lead time analysis showed some opportunities for future consideration. The differences in appointment lead times in Figure 5 that were almost eliminated by adjusting for after-hours scheduling were striking (Figure 6). Would a practice decision to COVID test 24/7 result in patients actually scheduling a midnight test when they were online scheduling at 11 PM? If so, there could have been a major impact on the test result turnaround (assuming lab hours and test transport to the lab were also 24/7). We found that 253 189 h among 32 194 scheduled appointments were spent waiting from self-scheduling an appointment until the testing site opened for test visits. Thus, there was a potential 7.9 h average faster time to COVID diagnosis if those after-hours self-schedulers had a comprehensive 24/7 COVID testing process.

Practice Promotion of Self-Scheduling

The increase in self-scheduling utilization in July 2021 may be associated with changes in our patient online services. During the summer of 2021, we placed COVID resources more prominently in the patient portal so that patients knew that they had the option of self-scheduling. The high-profile placement for COVID self-scheduling created a rapid ability to self-schedule without page searching. There were also more general communication efforts to highlight the ability to self-schedule a COVID test. Previous work also showed that patients were increasingly getting familiar with self-scheduling though use of self-scheduling for well-child visits and screening mammograms.[5,6] Online symptom self-triage linked with self-scheduling was also being implemented in 2020 and throughout 2021.

Comparison with Other Studies

Patient utilization of COVID self-scheduling was much greater than has been reported by other self-scheduling processes. A more general online appointment process was used by 11% of patients in a primary healthcare practice in Australia. A Canadian study of six medical practices found that 12% of active patients had signed up for online appointment access, for an average use of 1.6 online appointments from date of enrollment. Mayo Clinic experience with self-scheduling appointments showed that 5% of well-child appointments and 21% of screening mammograms were self-scheduled.[5,6]

No-show rates have been examined in other studies of online appointment scheduling and in general depend on a number of factors.[12,13] Since no-shows are often associated with significant expense incurred by idle staff or equipment, the no show rate is an important metric for management. Our no-show rate for COVID testing self-scheduled appointments of 2.5% was significantly lower than the 3.0% no-shows with COVID testing staff scheduled appointments.

Limitations

There are several limitations to our study. In particular, the process timing was incomplete both for self-scheduling and staff scheduling appointments. In the case of online and mobile self-scheduling, there is the time it takes to sign on and to navigate to COVID test scheduling. In the case of staff scheduling, there were other process timings which we could not capture. In particular, screening telephone times and telephone handoff times shown in the Figure 1 process flow were not available. Also, the staff scheduling time is likely an underestimate. The staff scheduling times were from Epic data. Epic uses sophisticated algorithms that track keystrokes which may not reflect time on a telephone call while not actively using keystrokes.

In this retrospective study, we had significant differences in demographics between self-scheduled appointments and those staff scheduled. There are potentially many other unmeasured differences that are likely to affect results. For example, those with more severe symptoms may have wanted to talk to a nurse, influencing the time difference between the self-scheduling process and the nurse triage process. As noted in Methods, sampling of nurse triage calls for the Kaplan Meier comparison curves with self-scheduling times was not random. However, it should be noted that the total time of nurses on the telephone was captured completely. The estimate of nursing time saved was based on actual total nursing time involved in the 17 months of calls, not a sample of calls.

We did not examine repeat users of the staff scheduling or self-scheduling process. Users that switched from one process to another could have potentially given more information about the usability of the process but that was outside the scope of the current study. Differences in individual use counts could also bias the individual times in the scheduling process as more experience could decrease the time spent self-scheduling.

Conclusions

Patient utilization of self-scheduling COVID tests climbed to 44% during the peak of the 2022 omicron variant surge, with patients successfully self-scheduling tests throughout the day and night. Seventy-five percent of symptomatic patients successfully self-triaged and self-scheduled in less than 9 min; 75% of asymptomatic patients went through prescheduling questions and self-scheduled in less than 5 min. It was estimated that about 17 500 combined nursing and appointment scheduler hours would have been needed to schedule the 136 252 COVID test appointments that patients successfully scheduled themselves.

Click here for additional data file.

Supplemental material, sj-jpg-1-hme-10.1177_23333928221125034 for Self-Scheduling Process Efficiency and Utilization of Online Self-Scheduling of Lab Tests: A Retrospective Analysis of Self-Scheduled Appointments for COVID Testing by Frederick North, Elissa M. Nelson, Rebecca J. Majerus, Matthew C. Thompson, Aric J. Knutson and Brian A. Crum in Health Services Research and Managerial Epidemiology

Click here for additional data file.

Supplemental material, sj-jpg-2-hme-10.1177_23333928221125034 for Self-Scheduling Process Efficiency and Utilization of Online Self-Scheduling of Lab Tests: A Retrospective Analysis of Self-Scheduled Appointments for COVID Testing by Frederick North, Elissa M. Nelson, Rebecca J. Majerus, Matthew C. Thompson, Aric J. Knutson and Brian A. Crum in Health Services Research and Managerial Epidemiology

Click here for additional data file.

Supplemental material, sj-jpg-3-hme-10.1177_23333928221125034 for Self-Scheduling Process Efficiency and Utilization of Online Self-Scheduling of Lab Tests: A Retrospective Analysis of Self-Scheduled Appointments for COVID Testing by Frederick North, Elissa M. Nelson, Rebecca J. Majerus, Matthew C. Thompson, Aric J. Knutson and Brian A. Crum in Health Services Research and Managerial Epidemiology

Click here for additional data file.

Supplemental material, sj-jpg-4-hme-10.1177_23333928221125034 for Self-Scheduling Process Efficiency and Utilization of Online Self-Scheduling of Lab Tests: A Retrospective Analysis of Self-Scheduled Appointments for COVID Testing by Frederick North, Elissa M. Nelson, Rebecca J. Majerus, Matthew C. Thompson, Aric J. Knutson and Brian A. Crum in Health Services Research and Managerial Epidemiology