Getting patients in the door: medical appointment reminder preferences.

PURPOSE: Between 23% and 34% of outpatient appointments are missed annually. Patients who frequently miss medical appointments have poorer health outcomes and are less likely to use preventive health care services. Missed appointments result in unnecessary costs and organizational inefficiencies. Appointment reminders may help reduce missed appointments; particular types may be more effective than other types. We used a survey with a discrete choice experiment (DCE) to learn why individuals miss appointments and to assess appointment reminder preferences.
METHODS: We enrolled a national sample of adults from an online survey panel to complete demographic and appointment habit questions as well as a 16-task DCE designed in Sawtooth Software's Discover tool. We assessed preferences for four reminder attributes - initial reminder type, arrival of initial reminder, reminder content, and number of reminders. We derived utilities and importance scores.
RESULTS: We surveyed 251 adults nationally, with a mean age of 43 (range 18-83) years: 51% female, 84% White, and 8% African American. Twenty-three percent of individuals missed one or more appointments in the past 12 months. Two primary reasons given for missing an appointment include transportation problems (28%) and forgetfulness (26%). Participants indicated the initial reminder type (21%) was the most important attribute, followed by the number of reminders (10%). Overall, individuals indicated a preference for a single reminder, arriving via email, phone call, or text message, delivered less than 2 weeks prior to an appointment. Preferences for reminder content were less clear.
CONCLUSION: The number of missed appointments and reasons for missing appointments are consistent with prior research. Patient-centered appointment reminders may improve appointment attendance by addressing some of the reasons individuals report missing appointments and by meeting patients' needs. Future research is necessary to determine if preferred reminders used in practice will result in improved appointment attendance in clinical settings.

Introduction

As the US health care system continues to focus on population-based, value-driven care, it is essential for primary care providers and health care organizations (HCOs) to get patients in the clinic doors. It is estimated between 23% and 34% of outpatient medical appointments in the US are missed annually.1–3 Patients who miss appointments do not receive necessary health care services, and prevent or delay other patients from being able to schedule appointments for treatment, follow-up, or preventive care. Furthermore, individuals who frequently miss medical appointments have poorer health outcomes4–9 and are less likely to utilize preventive health care services than individuals who keep medical appointments.10 In addition to clinical consequences for patients, missed appointments result in an underutilization of health care providers who have excess capacity to see patients, as well as underutilization of equipment, space, and staff. As a result, HCOs and providers experience a loss in revenue, which may be offset by charging patients’ fees or increasing overall charges to patients and payers.11 Moreover, provider and organizational compensation, as well as health system reimbursement models are often contingent upon patient populations meeting quality metrics.10 As such, missed appointments are also missed opportunities for providers and HCOs to improve health care quality by increasing the number of patients receiving preventive care and medical treatments.

Patients commonly give several reasons for missing appointments that include forgetfulness, confusion, or miscommunication over appointment information, feeling better, transportation issues, and difficulty leaving work or school.12 HCOs and providers have used different strategies to increase appointment attendance and to mitigate the impact of missed appointments.13 Strategies using appointment reminders have been generally well received by patients.3,14–16 Traditionally, HCOs have adopted a one-size-fits-all approach to appointment reminders, where one type of reminder is sent to all patients.17 However, research suggests that peoples’ preferences and attitudes impact their behavior.18–20 One study suggests patients who receive preferred reminders may be more likely to attend an appointment.17 Appointment reminder types that have been shown to be effective include mail, phone calls, email, and text messaging.11,21–25 Our objectives are to explore why patients miss appointments, to evaluate patient preferences for reminders, and to consider the use of patient-centered appointment reminders to reduce missed appointments. We hypothesize customizing appointment reminders to meet patients’ needs and preferences may improve appointment attendance.

One way of understanding the diversity of patients’ preferences toward appointment reminders is to elicit their preferences using a discrete choice experiment (DCE). DCEs are widely used in marketing to assess preferences and have gained prominence in the health care realm in recent years.26 DCEs use features of specific products or services to investigate the relative value individuals associate with a product or service. By understanding how individuals value features, products, or services, reminders can be designed to better meet the needs of patients. DCE methodology presumes that a product or service, such as a medical appointment reminder, can be described by attributes. These attributes can be organized into comparative choice tasks from which individuals choose their most preferred configuration.27 HCOs may be able to use reminder preferences to strategically design reminder systems to increase patient attendance, improve operational efficiency, augment revenue, and advance health care quality.

Methods

Online survey

We designed and fielded a survey based on the literature and pre-testing feedback.12,13,17 Participants were asked to respond to demographic questions and appointment reminder habit questions, and to complete a DCE. Participants provided informed consent online before beginning the survey. We used a survey panel company, Survey Sampling International (SSI), to obtain the participant sample. This research was approved by the Institutional Review Board at the University of North Carolina at Chapel Hill.

DCEs

In a DCE, participants make choices between two or more hypothetical alternatives that are described by a set of attributes and levels. The levels of each attribute systematically change in a series of choice tasks where participants select the option they prefer the most. Responses to the choices are analyzed to determine the relative value participants attach to the different levels of the attributes as well as to the attributes being considered.27

Selection of attributes and levels

We used the literature to select four salient appointment reminder attributes: initial reminder type, arrival of initial reminder, reminder content, and number of reminders. We developed plausible levels for each attribute. We presented the initial attributes and levels we selected to 12 members of a patient advisory council at a medical clinic for feedback. The group discussed each attribute and its associated level as well as alternative attributes and levels. Based on the group’s feedback, we refined the attributes and levels we planned to use in the questionnaire (Table 1).

Table 1

Attributes and levels

	Attributes
	Initial reminder type	Arrival of initial reminders	Reminder content	Number of reminders
Levels	Postal mail	1–6 days prior to appointment	Clinic location information (clinic address, directions and map)	1 reminder
	Phone call	1–2 weeks prior to appointment	Rescheduling information (phone number or email)	2 or 3 reminders–same type
	Text message	3–4 weeks prior to appointment	Reason for visit	2 or 3 reminders–different types
	Email	>1 month prior to appointment	Clinic location information (clinic address, directions and map) and rescheduling information (phone number or email)
	Social media		Clinic location information (clinic address, directions and map) and reason for visit
	Electronic calendar (Outlook, Gmail, etc.)		Rescheduling information (phone number or email) and reason for visit Clinic location information (clinic address, directions and map), rescheduling information (phone number or email) and reason for visit

We framed the initial reminder type attribute to include two broadly adopted reminder types (postal mail, phone), two commonly used but not universally adopted types (text message, email), and two emerging reminder types (social media, electronic calendar). Furthermore, we presented three general time frames in four levels for the arrival of initial reminder attribute: immediately preceding the appointment (1–6 days prior), advance notification (1–2 weeks prior, 3–4 weeks prior), and distant notice (more than 1 month prior). We did not include time frames over 1 month because the evidence suggests longer lead times can increase missed appointment rates.28 Moreover, we framed the reminder content attribute to include reminders with varying amounts of information: limited (clinic location only, rescheduling information only, reason for visit only) multicomponent (clinic location and rescheduling information, clinic location and reason for visit, rescheduling information and reason for visit), and comprehensive (clinic location, rescheduling information, and reason for visit). Finally, we represented the number of reminder attributes as being 1 reminder or 2–3 reminders with the option of having the same type of reminder each time or having different reminders. We considered offering more than three reminders, but the patient advisory council expressed concerns that more than two reminders would not be appreciated by most patients.

Development of the DCE

We used Sawtooth Software’s online tool, Discover, to create an efficient set of 16 choice tasks using the selected attributes and levels. Discover is an online software as a service tool that assists users in developing and fielding an efficient choice-based conjoint (CBC) questionnaire.29 The software is designed to be user-friendly for individuals who do not have significant knowledge of or experience with DCE development. As such, the software guides the user by making recommendations for an appropriate number of tasks and concepts per task to create an efficient design. The user is warned if the number of tasks is too few to achieve high-quality utility estimates.29 The recommended DCE design is based on prior parameters which are derived from several rating questions participants complete before beginning the DCE. The rating questions allow respondents to differentiate between levels using four preference categories: very desirable, desirable, extremely desirable, and no opinion.29 The data obtained from the rating questions serve as utility constraints to allow for robust individual utility estimation using logit with data augmentation employing empirical Bayes (EB). Additionally, the rating data provide individual-level preference information to avoid dominated concepts as the software creates on-the-fly experimental designs during data collection.29 All designs recommended by the software are near orthogonal, have a high relative D-efficiency, and are statistically efficient.29

Based on our attribute list and decision to have three active appointment reminder options, Discover recommended using 16 choice tasks. We have used 16 choice tasks in the past, and we found the number to be feasible for participants to complete.30,31 We decided not to include an opt-out choice, indicating a preference for not receiving a reminder. We wanted to force participants to make a choice because in practice, it is uncommon for patients to be offered an opportunity to opt out of appointment reminders. Figure 1 is a sample choice task. We followed the International Society for Pharmacoeconomics and Outcomes Research Guidelines for DCE design.32

Figure 1

Example discrete choice experiment task.

We used Johnson’s formula to determine that a sample size of 73 participants or more would yield reasonably precise estimates of utility levels, given the use of 16 choice tasks, three active alternatives, and a maximum number of levels within a single attribute of seven.27 Based on our pretest results and previous experience with online surveys, we thought it was feasible to enroll at least 200 participants.

Pretesting

The online survey along with seven embedded usability questions was pretested in a random sample of 200 adults from September 9, 2015 to September 13, 2015. Participants were recruited using a mass informational email that was sent to subscribers of a listserv at the University of North Carolina at Chapel Hill. We aimed to recruit up to 250 participants, the maximum number of completed surveys our Discover license allows (in some instances, the software will allow more completed surveys if participants complete surveys simultaneously). Participants were screened online; exclusionary criteria consisted of being younger than 18 years or residing outside of the US. Individuals who completed the survey were eligible for one of four $25 gift certificates. Based on participant feedback from an embedded usability questionnaire and the participants’ responses, we made three changes. First, for the arrival of initial reminder attribute, we changed two levels indicating time in terms of number of days to number of weeks. Then, we modified the levels for the number of reminder attributes which used the phrase “multiple reminders” to 2–3 reminders because individuals stated they wanted to know a definitive number. We also changed the language stating “One reminder” level to “1 reminder”. Results from the pretest can be found in Tables S1 and S2.

Data collection and analysis

From September 25, 2015 to September 29, 2015, a stratified sample of survey panelists was obtained from an online survey panel company (SSI). To assure sufficient representation of each gender, we requested half of the sample to be female; exclusionary criteria included being younger than 18 years or residing outside of the US. The primary outcomes of interest were reasons for missing medical appointments and mean utilities of four attributes and overall attribute importance.

We used Sawtooth Software’s online Discover tool to perform descriptive analyses with mean values and proportions as well as to analyze the DCE data. Discover is a streamlined tool for individuals who want high-level, rapid results soon after the data have been collected. Discover uses EB methods to obtain individual-level utilities.29 EB methodology closely approximates the posterior mean values for individual-level utilities. The EB method involves computing an aggregate logit solution across all respondents using all of the data gleaned from the DCE. Then, each respondents’ answer to each CBC task is augmented by average population preferences to improve the quality of individual-level estimates so that results closely approximate those obtained using hierarchical Bayesian methods, which are considered the gold standard.29

Utilities are zero-centered numerical values that represent the relative desirability of the levels within each attribute. The higher the number, the more desirable the characteristic is to participants. Within Discover, utilities are used to calculate individual-level attribute importance scores, which are computed for each respondent by percentaging ranges for each attribute to determine mean importance scores.29 Attribute importance scores represent the relative importance or impact the four attributes have on the choice, given the range of levels used in the experiment.27 The attribute with the highest importance score is considered to be the most important attribute.

Results

We enrolled 251 adults nationally with a mean age of 43 (range 18–83) years, and 51% were female. Most were White (84%) and had a bachelor’s degree or higher (50%). Approximately half were employed full-time, and 38% had annual household incomes less than $45,000. About one-quarter (23%) reported missing one or more appointments in the past 12 months. Participants provided the following main reasons for missing appointments – transportation problems (28%), forgetfulness (26%), and confusion over time, date, or location (14%). Additional reasons given include problems leaving school/work (12%), feeling better (2%), and other (10%). Further participant characteristics are summarized in Table 2.

Table 2

Discrete choice experiment results: medical appointment reminder preferences

Participant characteristics	n=251
Mean age (SD)	43 (15)
Female	51.2%
Race
White	84.1%
African American/Black	7.6%
Asian	5.2%
Native American	0.4%
Other	2.8%
Latino/Hispanic	7.2%
Education
Less than high school	1.6%
High school graduate	17.6%
Some college/trade school	30.0%
Bachelor’s degree or higher	50.8%
Household income
<$45,000	37.8%
$45,000–$89,999	36.0%
$90,000+	25.9%
Employment
Full-time	49.2%
Part-time	13.4%
Unemployed	19.5%
Retired	13.4%
Other	4.5%
Appointments in 12 months
Had 5+	35.9%
Had 2–4	41.4%
Had 1	11.6%
I do not recall how many I had	0.0%
No appointments	11.2%
Missed 1+	22.4%
Reasons for missed appointments
I forgot	26.0%
I had problems leaving work/school	12.0%
I was confused about time, date, or location	14.0%
I had transportation problems	28.0%
I felt better	8.0%
I was too sick	2.0%
Other	10.0%

Abbreviation: SD, standard deviation.

Mean utility levels and importance scores are shown in Table 3. Attribute importance scores indicate the initial reminder type attribute was the most important attribute (44%). Among the other attributes, the reminder content attribute (24%) and arrival of initial reminder attribute (21%) were also relatively important. The number of reminder attributes (10.3%) was least important. Of the initial reminder type attribute levels, email and phone calls were the most preferred, followed by text message. Electronic calendar reminders and postal mail reminders were not preferred. The negative utility assigned to social media reminders suggests social media reminders are strongly disfavored. Preferences for the different levels of the reminder content attribute were not as distinct as preferences for other attributes’ levels. Some participants preferred reminders that only included clinic information, while others preferred reminders that contained all three types of information (location, rescheduling, and reason for visit) the most. Reminders containing only rescheduling information were not generally favored, and reason for visit was the least preferred level. Overall, for the arrival of the initial reminder attribute, participants preferred receiving reminders within 2 weeks of an appointment with reminders received between 1 and 6 days prior to appointment being most preferred. Appointment reminders received prior to 2 weeks before an appointment were least preferred. Additionally, participants preferred receiving one reminder over receiving two or three reminders even if multiple reminders were of different reminder types.

Table 3

Medical appointment reminder attributes and levels with corresponding utility scores, n=251

Attribute	Levels	Mean utilities	Lower 95% CI	Upper 95% CI	Mean attribute importance scores (CI)
Initial reminder type	Postal mail	−7.54	−14.4	−0.67	44% (42%–46%)
	Phone call (personal or automated)	32.88	26.26	39.5
	Text message	21.24	14.86	27.62
	Email (from provider’s office or EHR)	36.9	31.48	42.32
	Social media (Facebook, Twitter, etc.)	−78.91	−87.15	−70.68
	Electronic calendar (Outlook, Gmail, etc.)	−4.57	−11.01	1.88
Arrival of initial reminder	1–6 days prior to appointment	31.68	26.97	36.4	21% (19%–23%)
	1–2 weeks prior to appointment	11.53	8.42	14.65
	3–4 weeks prior to appointment	−14.75	−18.73	−10.78
	>1 month prior to appointment	−28.46	−32.52	−24.4
Reminder content	Clinic location information (clinic address, directions and map)	5.43	1.22	9.64	24% (23%–26%)
	Rescheduling information (phone number or email)	−0.78	−5.04	3.48
	Reason for visit	−4.19	−9.18	0.81
	Clinic location information (clinic address, directions and map) and rescheduling information (phone number or email)	0.37	−3.63	4.37
	Clinic location information (clinic address, directions and map) and reason for visit	−3.11	−7.42	1.19
	Rescheduling information (phone number or email) and reason for visit	−0.62	−5.15	3.91
	Clinic location information (clinic address, directions and map), rescheduling information (phone number or email) and reason for visit	2.9	−1.71	7.51
Number of reminders	1 reminder	9.1	5.69	12.52	10% (9%–11%)
	2 or 3 reminders–same type	−5.34	−8.01	−2.68
	2 or 3 reminders–different types	−3.76	−6.36	−1.16

Abbreviations: CI, confidence interval; EHR, electronic health record.

Discussion

Consistent with the literature, we found 23% of individuals reported missing one or more appointments in the past 12 months.1–3 Among individuals who reported missing an appointment(s), two of the top three reasons given for missing appointments (forgetfulness and confusion over appointment time, date, or location) can be addressed through the use of appointment reminders. This finding suggests appointment reminders have the potential to increase appointment attendance. Consequently, improved patient appointment attendance may result in improved health outcomes for patients, improved clinic efficiency, as well as increased revenue. Additionally, using appointment reminders may increase opportunities for providers and HCOs to meet or exceed quality benchmarks for their patient populations.

Previous research evaluating patient preferences for appointment reminder strategies is relatively limited17,33 and may not reveal complex or novel reminder preferences. To our knowledge, this work is the first to use DCE to examine patient preferences for medical appointment reminders. Overall, we found participants preferred a single reminder that comes by email, phone, or text message, arriving 2 weeks or less prior to a scheduled appointment. Some individuals seem to prefer reminders that contain information about only the clinic location. In contrast, other individuals prefer reminders that provide all three pieces of information we evaluated in the DEC – clinic location, rescheduling information, and reason for visit.

Given initial reminder type was the most important attribute, sending patients their most preferred type of reminder may have the most impact on appointment attendance. HCOs should consider using one or more of the most preferred reminder types such as email, phone calls, or text messages. One recent study found three-quarters of individuals were somewhat or very willing to exchange or receive medical appointment reminders via a mobile device.34 Our results confirm this finding and may suggest patients prefer “mobile reminders” that can be received wherever an individual happens to be in the world to “static reminders”, which must be retrieved from a fixed location. Furthermore, HCOs should evaluate their use of postal mail, which we found to be relatively unpreferred, and consider using alternative reminder methods, which may be more effective. Moreover, electronic calendar reminders were somewhat unpreferred by participants, and social media reminders were strongly unpreferred. These results seem to contradict the popularity of electronic calendars and social media tools such as Facebook and Twitter. While we are not able to explain these findings within scope of this work, future work should explore whether these findings are due to unfamiliarity with using electronic calendar appointments or social media in this context, privacy or security concerns, or other possible worries about the delivery modality.

We found preferences for levels within the reminder content attribute were not as clear as preferences for levels within other attributes. This may be due to the use of overlapping levels within the attribute that could have confounded the results. Moreover, relatively little is known about how appointment reminder content influences appointment attendance.35 Consequently, with little context, our results are challenging to interpret and glean information to improve reminders for patients. However, as the literature base grows, researchers and HCOs should consider exploring and testing emerging information regarding reminder content. Of recent interest, preliminary research conducted in the UK National Health Service revealed that including appointment costs in text message reminders proved more effective at increasing appointment attendance than reminders without cost information.35

Limitations

Our results should be considered in light of several limitations. Foremost, we were only able to test a limited number of attributes and levels in the DCE while maintaining a valid experimental design. If we had used other attributes or levels, we may have gleaned different results. We believe that we selected relevant attributes and levels based on participant feedback, pretest results, and the existing literature. However, the use of overlapping levels for the attribute “reminder content” likely yielded results that may confound the result as participants may not have been able to differentiate between levels enough to express meaningful preferences. Moreover, there are various ways to determine the relative attribute importance.36 We used the standard method used by Sawtooth Software, but there may be other methods that could better represent the relative impact of each of the attributes. Additionally, our participant sample was drawn from an online pool of individuals who are highly educated and mostly White. As such, results may not be generalizable to other populations. For example, the national sample yielded results that were somewhat different from the pretest results.

Furthermore, our results show individuals’ stated preferences for reminders. It is unclear if an individual’s actual appointment attendance will change due to receiving a “preferred reminder” as the psychology research suggests. Finally, we were unable to assess individuals’ health conditions and thus were not able to evaluate how the type or severity of illness impacts reminder preferences.

Conclusion

Ultimately, as communication patterns and mechanisms change over time, it is important for HCOs to continuously evaluate the effectiveness of appointment reminders to encourage attendance. Our findings may be used by HCOs seeking to improve appointment attendance to achieve a variety of goals such as improving patient outcomes or operational efficiency. Ultimately, future research and informal experimentation is necessary to determine whether patient-centered reminders will result in improved appointment attendance.

Table S1

Pretest participants

Participant characteristics	n=200
Mean age (SD)	41 (15.5)
Female	85.0%
Race
White	80.0%
African American/Black	10.5%
Asian	6.0%
Native American	0.5%
Other	3.0%
Latino/Hispanic	4.0%
Education
Less than high school	0.0%
High school graduate	9.0%
Some college/trade school	19.5%
Bachelor’s degree or higher	72.0%
Household income
<$45,000	24.7%
$45,000–$89,999	33.8%
$90,000+	41.1%
Employment
Full-time	73.0%
Part-time	15.5%
Unemployed	5.0%
Retired	1.5%
Other	5.0%
Appointments in 12 months
Had 5+	35.9%
Had 2–4	41.4%
Had 1	11.6%
I do not recall how many I had	0.0%
No appointments	11.2%
Missed 1+	22.4%
Reasons for missed appointments
I forgot	47.8%
I had problems leaving work/school	17.4%
I was confused about time, date, or location	15.2%
I had transportation problems	4.3%
I felt better	4.3%
I was too sick	0.0%
Other	10.9%

Abbreviation: SD, standard deviation.

Table S2

Pretest discrete choice experiment results (n=200)

Attribute	Levels	Mean utilities	Lower 95% CI	Upper 95% CI	Mean attribute importance scores (CI)
Initial reminder type	Postal mail	−17.64	−24.87	−10.41	53% (51%–55%)
	Phone call (personal or automated)	19.46	14.25	24.66
	Text message	39.06	32.78	45.34
	Email (from provider’s office or EHR)	61.32	56.36	66.28
	Social media (Facebook, Twitter, etc.)	−121.15	−128.38	−113.92
	Electronic calendar (Outlook, Gmail, etc.)	18.95	12.82	25.08
Arrival of initial reminder	1–5 days prior to appointment	19.88	16.13	23.63	16% (14%–17%)
	6–14 days prior to appointment	11.23	8.61	13.84
	15–30 days prior to appointment	−12.56	−15.54	−9.59
	>1 month prior	−18.55	−22.28	−14.81
Reminder content	Clinic location information (clinic address, directions and map)	5.62	1.07	10.17	23% (21%–24%)
	Rescheduling information (phone number or email)	−4.89	−9.71	−0.08
	Reason for visit	−17.09	−21.7	−12.48
	Clinic location information (clinic address, directions and map) and rescheduling information (phone number or email)	13.57	9.14	17.99
	Clinic location information (clinic address, directions and map) and reason for visit	4.44	0.03	8.86
	Rescheduling information (phone number or email) and reason for visit	−5.79	−9.68	−1.9
	Clinic location information (clinic address, directions and map), rescheduling information (phone number or email) and reason for visit	4.15	0.11	8.18
Number of reminders	1 reminder	6.53	3.13	9.93	9% (8%–10%)
	Multiple reminders–same type	−5.9	−8.07	−3.73
	Multiple reminders–different types	−0.63	−3.33	2.06

Abbreviations: CI, confidence interval; EHR, electronic health record.