Appointment reminder systems are effective but not optimal: results of a systematic review and evidence synthesis employing realist principles.

Missed appointments are an avoidable cost and resource inefficiency which impact upon the health of the patient and treatment outcomes. Health care services are increasingly utilizing reminder systems to manage these negative effects. This study explores the effectiveness of reminder systems for promoting attendance, cancellations, and rescheduling of appointments across all health care settings and for particular patient groups and the contextual factors which indicate that reminders are being employed sub-optimally. We used three inter-related reviews of quantitative and qualitative evidence. Firstly, using pre-existing models and theories, we developed a conceptual framework to inform our understanding of the contexts and mechanisms which influence reminder effectiveness. Secondly, we performed a review following Centre for Reviews and Dissemination guidelines to investigate the effectiveness of different methods of reminding patients to attend health service appointments. Finally, to supplement the effectiveness information, we completed a review informed by realist principles to identify factors likely to influence non-attendance behaviors and the effectiveness of reminders. We found consistent evidence that all types of reminder systems are effective at improving appointment attendance across a range of health care settings and patient populations. Reminder systems may also increase cancellation and rescheduling of unwanted appointments. "Reminder plus", which provides additional information beyond the reminder function may be more effective than simple reminders (ie, date, time, place) at reducing non-attendance at appointments in particular circumstances. We identified six areas of inefficiency which indicate that reminder systems are being used sub-optimally. Unless otherwise indicated, all patients should receive a reminder to facilitate attendance at their health care appointment. The choice of reminder system should be tailored to the individual service. To optimize appointment and reminder systems, health care services need supportive administrative processes to enhance attendance, cancellation, rescheduling, and re-allocation of appointments to other patients.

Introduction

Missed health care appointments are a major source of avoidable inefficiency that impacts on patient health and treatment outcomes. Data on non-attendance vary, however studies from around the world consistently report non-attendance rates of between 15% and 30% in outpatient health clinics.1–4 In England, more than 12 million appointments at consultant led clinics,5 and a similar number of general practice appointments are missed each year.6 The cost of missed appointments to the UK National Health Service (NHS) has tripled since 1999.7 In 2009, non-attendance was estimated to cost over £600 million (around US$970 million).8

The consequences of non-attendance include increased appointment waiting times,9 increased costs of care delivery,10,11 underutilization of equipment and personnel,10 reduced appointment availability,6,11 reduced patient satisfaction,12,13 and negative relationships between patients and staff.6,9 Missed appointments may delay presentation at health services, resulting in a lack of follow-up of chronic conditions which may ultimately lead to complications, unnecessary suffering, and costly hospital admission.10,14 Pressures from referring agents to manage waiting lists, can potentially increase staff stress, anxiety, and fatigue levels.15 Reducing the number of missed appointments may be a relatively inexpensive way to increase health care efficiency, effectiveness, and quality.

Numerous reviews have demonstrated the effectiveness of existing reminder systems in varied service settings. However, research to-date focusses on the use of reminder systems in particular service contexts or technologies,16–18 rather than synthesizing knowledge across different contexts and patient groups. This study explores the effectiveness of reminder systems for promoting attendance, cancellations, and rescheduling of appointments across all health care settings and for particular patient groups and the contextual factors which indicate that reminders are being employed sub-optimally.

Material and methods

Our project incorporated three components: the development of a conceptual framework to provide an understanding of the contexts and mechanisms which influence reminder effectiveness (review 1); a systematic review (SR) of the reminder effectiveness literature (review 2), and an evidence synthesis informed by realist principles to explain the contexts and mechanisms which influence reminder effectiveness (review 3). We used realist inquiry because it clarifies the context–mechanism–outcome relationships in an attempt to understand better what works, for whom, under what circumstances.19 Further detail on the methodology employed is available in the TURNUP project report.20

Searches were conducted on 13 databases with date limits of January 1, 2000 to February 15, 2012: AMED, CINAHL Plus with Full Text, Cochrane Library, Embase, HMIC, IEEE Xplore, Kings Fund Library Catalogue, Maternity and Infant Care, MEDLINE, PEDro, PsycINFO, SportDiscus, and Web of Science. The strategy used the concept of (reminders/prompts/alerts) in proximity to (appointments) (Figure 1). Where supported, appropriate database headings/thesaurus terms were used. The reference lists of included randomized controlled trials (RCTs) and SRs were screened for additional relevant studies and citation (forward-) searches were performed in respect of the included RCTs. English-language studies of various quantitative and qualitative designs were included if they investigated the effectiveness of outpatient appointment reminders, appointment attendance behavior, or explicated theories/models/frameworks relating to reminder systems or appointment attendance. Studies were excluded if they investigated reminders sent to a patient inviting them to schedule an appointment. All members of the project team were involved in screening and selection of studies and data extraction from included studies.

Figure 1

Example search strategy.

Note: Example search strategy: CINAHL Plus with Fulltext, MEDLINE, SportDiscus (via EBSCO) 2000 to January 11, 2012.

Review 1

We could find few pre-existing conceptual models or frameworks that directly explain the mechanisms by which reminder systems support appointment attendance. We therefore drew on a variety of models that have been developed to understand behavior in relation to medical adherence. Included models related to use of reminders to promote clinical outcomes;21 health care utilization theory;22 the theory of planned behavior;23 the trans-theoretical model;24 self-determination theory;25 protection motivation theory;26 rationale choice theory;27 and complexity theory.28 Our conceptual framework was developed through an iterative process involving examination of the various theories and discussions about context, mechanisms, and outcomes that were important to explain how reminder systems work to promote attendance, for whom, and in what circumstances. The framework consisted of six broad factors that could potentially influence the effectiveness of the reminder or whether patients would attend, cancel or reschedule their appointment, namely: the reminder-patient interaction, reminder accessibility, health care settings, wider social factors, cancellation and rebooking systems, and patient attributes. This framework was then used to support data extraction.

Review 2

Our SR of effectiveness investigated the impact of reminder systems on improvements in attendance, cancellations, and rescheduling of appointments. The questions addressed in this were: 1) how effective are reminder systems at reducing non-attendance at appointments and increasing cancellation/rescheduling of appointments? and 2) which types of reminder systems are most effective in improving the uptake of health service appointments? We used standardized methods to select, quality assess, extract and synthesize the findings of SRs and RCTs.29 The Critical Appraisal Skills Program appraisal tool for RCTs was used to quality assess those RCTs not already assessed in pre-existing SRs.30 The quality of the included SRs was assessed against the criteria used by the Centre for Reviews and Dissemination when evaluating reviews for inclusion in the Database of Abstracts of Reviews of Effects.31 We used these quality assessments to moderate our interpretation of the review findings, not to exclude the papers.32

Review 3

Our evidence synthesis aimed to explore the differential effectiveness of reminder systems for particular population sub-groups; to identify contexts and mechanisms which influence the effectiveness of different reminder systems for particular population sub-groups; and identify any disadvantages which should be considered when introducing reminder systems for specific populations. The data extraction framework used the six elements of the conceptual framework. In accordance with realist principles, not all potentially relevant papers identified from the screening contributed to the synthesis.33 All RCTs investigating reminder systems and all reviews (systematic and otherwise) about reminder systems and appointment systems were prioritized for full extraction of contextual and explanatory variables. Whereas RCTs were required to meet minimum quality standards in order to be included in the effectiveness SR, the studies excluded from this SR still had the potential to contribute to the evidence synthesis informed by realist principles. In many cases findings from such studies contributed to the evidence base regarding the mechanisms and contexts that shape the operation of reminder systems in real world settings. Examination of the trial evidence was followed by exploration of qualitative, mixed-methods and non-RCT quantitative studies about reminders and appointments for Europe, America, Canada, Australia, and New Zealand. Thematic analysis was used to examine the evidence available for each section of the framework. Subsequently a narrative synthesis was developed that sought to explain the context and mechanisms influencing how reminders support attendance, cancellation, and rebooking.

Results

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart (Figure 2) shows the numbers of included papers for review 2 and 3. Preliminary database searches yielded 638 unique papers; a further 139 were identified from subsequent searches. Following the screening stages, 466 potentially relevant papers were identified. Eleven SRs met the inclusion criteria for review 2 (Table 1).16–18,34–41 These SRs either examined a single technology, eg, an SR of short message service (SMS) reminder systems,17 or explored the role of information technologies along a patient care pathway, one of which might be appointment reminder systems.41 The quality of included reviews was variable (Table 2). The five Cochrane reviews had been scrutinized against the highest quality standards.16,18,34,36,40 Four reviews passed the Centre for Reviews and Dissemination SR quality threshold.17,35,39,41 Two reviews did not pass the minimum standard for SRs.37,38

Figure 2

PRISMA flowchart for review 2 and 3.

Abbreviations: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; SRs, systematic reviews; RCTs, randomized controlled trials; TI, titles; AB, abstracts.

Table 1

Reminder technologies covered by each review

Study	Letter	Manual telephone	Automated telephone	Mobile/SMS	Voice messaging	Email	Other
Atherton et al18						✓
Car et al34				✓
Free et al35				✓	✓
Glynn et al36	✓	✓		✓
Guy et al17				✓
Hasvold and Wootton37		✓	✓	✓
Henderson38	✓	✓	✓
Jacobson Vann and Szilagyi16	✓	✓
Krishna et al39				✓
Reda and Makhoul40	✓	✓	✓	✓		✓	Personal visit
Stubbs et al41	✓	✓	✓	✓		✓	Open access scheduling

Abbreviation: SMS, short message service.

Table 2

Reviews included in this review with an assessment of their quality

Study	Review question/aims	Overall review quality	Implications for technologies	Implications for specific populations
Atherton et al18	To assess effects of using email for coordination of health care appointments and attendance reminders, compared to other forms of coordinating appointments and reminders, on outcomes for health professionals, patients and carers, and health services, including harms.	Cochrane Empty Review with no eligible studies. A limitation of this review is the date of the search. The search was conducted in January 2010. Length of time between search date and publication of review means it is possible that relevant studies have been published in interim period.	Not applicable	Not applicable
Car et al34	To assess effects of mobile phone messaging reminders for attendance at health care appointments. Secondary objectives include assessment of patient and health care provider evaluations of intervention; costs; and possible risks and harms associated with intervention.	Cochrane Review. Included studies were of varying methodological quality; most provided insufficient information to accurately assess risk of bias. Sequence generation for randomization considered adequate (although randomization method unclear in one study) but in two studies it was not clear whether, and how, allocation was concealed. Lack of blinding in all studies can be partly explained by interactive nature of text message interventions, which does not permit blinding of participants or health care providers. Potential bias from apparent lack of blinding of outcome assessors.	Included 4 RCTs involving 3,547 participants. Three studies with moderate quality evidence showed that mobile text message reminders improved rate of attendance compared to no reminders (RR 1.10 [95% CI] 1.03 to 1.17).One low quality study reported that mobile text message reminders with postal reminders, compared to postal reminders, improved rate of attendance at health care appointments (RR 1.10 [95% CI 1.02 to 1.19]). However, two studies of moderate quality showed that mobile phone text message reminders and phone call reminders had a similar impact on health care attendance (RR 0.99 [95% CI] 0.95 to 1.03). Costs/attendance of SMS lower compared to phone reminders. No studies reported harms or adverse effects of the intervention, nor health outcomes or user perception of safety related to the intervention.	Further research should focus on older patients, given that this population has, on average, more health care appointments and uses mobile phones less frequently than the younger population.
Free et al35	To quantify effectiveness of mobile technology based interventions delivered to health care providers or to support health care services, on any health or health care service outcome.	Identified as meeting CRD criteria. No full evaluation yet available.	Pooled effect on appointment attendance using text message (SMS) reminders vs no reminder increased, with RR of 1.06 (95% CI 1.05–1.07, I2=6%). Pooled effects on number of cancelled appointments was not significantly increased RR 1.08 (95% CI 0.89–1.30). No difference in attendance using SMS reminders vs other reminders (RR 0.98, 95% CI 0.94–1.02, respectively). SMS reminders no more effective than postal or phone call reminders, and texting reminders to patients who persistently missed appointments did not significantly change number of cancelled appointments. SMS appointment reminders have modest benefits and may be appropriate for implementation.
Glynn et al36	To evaluate the effectiveness of reminders on improving the follow-up of patients with hypertension.	Cochrane Review. Included RCTs with a contemporaneous control group. The methodological quality of included studies was generally poor to moderate with 40% of included articles describing their randomization processes and only 19% describing adequate concealed allocation processes.	Included 8 RCTs investigating appointment reminder systems. All but one of the RCTs were associated with improved outcomes. The pooled results favored appointment reminder systems for follow-up of patients (odds ratio of being lost to follow-up 0.4, 95% CI 0.3 to 0.5).
Guy et al17	To assess the effectiveness of SMS reminders at increasing the uptake of appointments in health care settings.	Although gray literature was searched, it was still possible that some evaluations were not identified, particularly those with a negative outcome. Unable to assess possibility of effect according to clinical reasons for attending, as few papers presented this information. Clinical presentation could affect priority placed by patients on the need for keeping an appointment.To maximize value of future evaluations, studies should collect and report information on clinical reason for attendance as well as the visit status (new, follow-up).	Summary effect from RCTs was 1.48 (95% CI: 1.23–1.72). No significant sub-group differences by clinic type (primary care clinics, hospital outpatient clinics) or message timing (24, 48, and 72+ hours before scheduled appointment). SMS reminders substantially increase likelihood of attending clinic appointments. SMS reminders appear to be simple and efficient option for health services to improve service delivery, as well as bringing health benefits for patients who receive the reminders.	No significant sub-group differences by target age group (pediatric, adult, older). Age classification based on median age of patient receiving SMS reminders, or specification of clinic type as pediatric. RCTs demonstrated SMS reminders effective in wide age range from pediatric to older. Mobile usage data demonstrate that over 90% of population in many countries own mobile phones, but uptake is higher in younger people.86 As younger patients have higher non-attendance rates at clinical services,100 SMS reminders may be more beneficial in this group. However, older patients have considerably more health appointments each year, often at outpatient clinics where non-attendance costs UK NHS estimated £790 million per year.101
Hasvold and Wootton37	1) What is the best estimate of effect of sending reminders on non-attendance rates?2) Are there any differences in non-attendance when using reminders sent manually (ie, from phones operated by a human) or automatically (ie, by SMS text messages or by automated voice recordings)?3) Does time at which the reminder is sent influence the effect on non-attendance rates?4) What are costs and benefits of using reminders?	Not classed as systematic review by DARE (CRD). PubMed only searched.	Weighted mean relative change in non-attendance was 34% of baseline non-attendance rate. Automated reminders less effective than manual phone calls (29% vs 39% of baseline value). No difference in non-attendance rate, whether reminder sent day before or week before. Cost and savings not measured formally, but almost half included cost estimates. Average cost of using either SMS, automated phone calls or phone calls was 0.41 Euros per reminder.	All studies except one102 showed positive effect from using reminders. (Patients themselves chose in advance whether they wished to receive reminder or not – potential bias in intervention group.) Overall no-show rate (outpatients in vascular laboratory) was 12% (average 7.6 missed appointments/week: gross annual revenue loss of US$89,107 based on ultrasound costs). Of 8,766 patients offered automated reminders, only 53% agreed to receive calls. No-show rate significantly greater for patients choosing automated reminders (8.9% vs 5.9%, P<0.0001).
Henderson38	To assess the effectiveness of appointment reminders as a means of increasing attendance and reducing DNA rates at new outpatient appointments.	Not classed as systematic review by DARE (CRD). Undertaken by single reviewer, with possibility of reviewer bias. Hand-searching not performed. Non-English language papers not included. Relevant material may have been missed. Methodological quality of evidence base generally poor. Several trials failed to describe randomization. In many studies blinding was poorly addressed, study participants were inadequately described and only a small number of participants were recruited.	Telephone reminders, if received, can have a positive impact on attendance and DNA rates. Postal reminders found to be effective. Although limited, literature suggests that the impact of “standard” reminders is similar to that of telephone reminders. Suggests that “Reminder plus” is more effective than “standard” reminders.	Impact of implementing initiatives on inequalities in access to services not considered by any studies included in review.Majority of included studies conducted in psychiatric settings outside the UK, raising issues of generalizability.
Jacobson Vann and Szilagyi16	To assess overall effectiveness of patient reminder or recall systems, or both, in improving immunization rates; compare effectiveness of different types of reminder or recall interventions (eg, postcard, letter, telephone), or combination of both reminder and recall.	Cochrane Review.	Reminding people over telephone, sending a letter or postcard, or speaking to them in person increased vaccinations. Providing numerous reminders was more effective than single reminders. Reminding people over telephone more effective than postcard or letter reminders. Reminders over telephone may be expensive compared with alternative approaches. Reminders worked whether from private doctor’s office, medical center, or public health department clinic. Studies all from developed countries.	Reminding people to have vaccinations increased the number of people vaccinated, whether the people were due or overdue for vaccinations. Increases were observed in both children and adults for all types of vaccines, but not among urban adolescents in one study.
Krishna et al39	To investigate role of cell phones and text messaging interventions in improving health outcomes and processes of care.	Research question supported by inclusion criteria for study design, intervention and outcomes. Authors did not report searches of unpublished data. Only studies published in English/English-language abstracts eligible for inclusion. Publication/language bias could not be ruled out. Authors did not report review process, so not known whether steps taken to reduce possible error and bias (such as performing processes in duplicate). Study quality not assessed, so unknown whether results of included studies were reliable. Many studies had small sample sizes. Narrative synthesis was appropriate given diversity of included studies. Due to possibility of bias and error in review process and unknown quality of included studies, authors’ conclusions may not be reliable.	Text messaging associated with fewer days to diagnosis (one study). Failure-to-attend rates significantly improved in two studies, but did not differ significantly between intervention/control groups in two other studies. Mobile phone reminder, disease monitoring and management, and education can improve health outcomes and care processes.	Text messaging associated with improved communication in participants with disabilities (one study).
Reda and Makhoul40	To estimate the effects of simple prompting by professional carers to encourage attendance at clinics for those with suspected serious mental illness.	Cochrane Review.	No clear difference between those prompted by telephone 1 or 2 days before appointment vs those given standard appointment management system (2 RCTs, n=457, RR missed appointment 0.84 95% CI 0.7–1.1). Text-based prompts such as a letter, a few days before the appointment day, may increase clinic attendance vs no prompt (3 RCTs, n=326, RR missed appointment 0.76 95% CI 0.43–1.32). One small study (n=61) combined telephone/text-based prompts vs no prompt, no real difference between groups (RR missed appointment 0.7 95% CI 0.4–1.2). Telephone prompts vs text-based prompts (1 RCT, n=75), the latter, as an “orientation statement” may be more effective than telephone prompt (RR missed appointment 1.9 95% CI 0.98–3.8). One study (n=120) compared standard letter prompt vs a letter orientation statement. Overall, results tended to favor orientation statement vs simple letter but not statistically significant (RR missed appointment 1.6 95% CI 0.9–2.9). For prompts regardless of type, results of greater significance suggest increased attendance (RR missed appointment 0.80 95% CI 0.65–0.98).	Indication that for serious mental illness it may be important to provide textual reminder plus orientation statement (a short paragraph, taking about 30 seconds to read, explaining the program of care, the fee system, and providing gentle encouragement).65 Contrasts with general trend in favor of telephone reminders.
Stubbs et al41	To compare telephone, mail, text/SMS, electronic mail and open-access scheduling to determine which is best at reducing outpatient non-attendance and providing net financial benefit.	Review addressed broad research question to evaluate impact of all methods for reducing outpatient non-attendance. Methods used to identify and select studies for inclusion generally clear, but publication bias cannot be ruled out. No attempts to assess study quality or minimize errors and bias in review process mentioned. Included studies extremely diverse in populations, settings, and research methods. Analysis based on simple weighted average for each approach (telephone, text, post, or open access). Influence of other relevant factors on non-attendance not explored. Authors acknowledge that only more recent studies (electronic rather than paper reminders) likely to be relevant now. Potential publication bias notwithstanding, authors conclude that most included interventions modestly improved attendance. Appears reliable, but did not investigate factors that might influence effectiveness of these interventions in different populations and settings. Different reminders not compared with each other. Conclusion that telephone reminders were better than text and post reminders may not be reliable. No formal evaluation of cost-effectiveness, so conclusion on relative cost-effectiveness of interventions also may not be reliable.	Telephone, mail, and text/SMS interventions all improved attendance modestly but at varying costs. Text messaging most cost-effective of the three, but its applicability may be limited. Few data available regarding electronic mail reminders, whereas open-access scheduling is area of active research.

Note: We acknowledge the assistance of the Centre for Reviews and Dissemination in providing quality assessments for included reviews, including the Stubbs et al41 review which was provided on demand.

Abbreviations: RCT, randomized controlled trial; RR, relative risk; CI, confidence interval; CRD, Centre for Reviews and Dissemination; SMS, short message service; DARE, Database of Abstracts of Reviews of Effects; DNA, did not attend; NHS, UK National Health Service.

Of the 31 RCTs that met our inclusion criteria for review2,4,42–72 only ten were uniquely identified by our review. The included RCTs related to the use of systems to remind patients to attend a health-related appointment that had already been scheduled (Table 3). The majority of the included RCTs examined either automated telephone reminders (15/31) or SMS texting services (12/31). Seven RCTs examined personalized telephone calls and 9/31 studies examined postal (letter/postcard) reminders. In most studies the comparator was no intervention. The principal functions of the various reminder systems were reminder only, reminder requiring confirmation, reminder plus orientation or reminder plus supporting clinical information. A variety of attendance related outcomes were measured. These included attendance, cancellation, rescheduling, and patient satisfaction. A judgement of the quality of the uniquely identified RCTs is shown in Table 4.

Table 3

Reminder technologies assessed and outcomes reported by each RCT

Study	Study characteristics	Letter	Personalized telephone call	Automated telephone	Mobile/SMS	Voice messaging	Email	Other	Comparator	Attendance outcomes	Overall effect
Bos et al42	the Netherlands, orthodontic clinic, (N=301)	✓		✓	✓				No reminder	Standardized failure rate; respondents’ attitudes to receiving reminder; respondents’ reminder preferences	Non-attendance rate reduced by 4.5%
Can et al43	UK, orthodontic clinic, (N=231)	✓i							No reminder	Attendance rates	Non-attendance rate reduced by 4.2%
Chen et al44	People’s Republic of China, health promotion center, (N=1,859)			✓	✓				No reminder	Attendance rates; cost per attendance of intervention	Non-attendance rate reduced by 7%
Chiu45	Hong Kong, radiology outpatients, (N=311)		✓		✓				No reminder	Attendance rates	Non-attendance rate reduced by 9.4%
Cho et al46	Korea, hospital-based family practice outpatients, (N=918)	✓			✓				No reminder	Attendance rates; cost per attendance	Non-attendance rate reduced by 3.4% (SMS) and by 1.1% (telephone call)
Christensen et al47	USA, children’s dental clinic, (N=313)			✓ii					No reminder	Punctuality for appointment (15 minutes); rate of missed appointments	Non-attendance rate reduced by 21% (48 hours) and by 26% (24 hours)
Comfort et al48	USA, substance abuse clinic, (N=102)		✓						No engagement	Engagement with services	No statistically significant differences
Costa et al,49Costa et al50	Portugal, outpatients clinics, (N=3,362)				✓				No reminder	Non-attendance rate	Non-attendance rate reduced by 3.5%
Fairhurst and Sheikh51	UK, inner-city general practice, (N=418)				✓				No reminder	Non-attendance rates	Non-attendance rate reduced by 5.3%
Goldenberg et al52	USA, teaching clinic, (N=723)		✓iii						No reminder	Attendance (show) rates	Non-attendance rate reduced by 10%
Griffin et al53	USA, colposcopy clinic, (N=1,876)		✓	✓iv					No reminder	Appointment non-attendance; patient perceptions about the call	38%, 42%, and 41% did not attend in IVR7, IVR3, and NDC arms, respectively; 33% (FS) and 38% (colonoscopy) non-attendance at baseline
Hashim et al54	USA, urban family practice, (N=930)			✓					No reminder	Outcome of call (confirmed, unable to leave message, appointment cancelled by patient/family, appointment re-scheduled by patient/family, or no active telephone number); cost of reminders	Non-attendance rate reduced by 6.9% (95% CI, 1.5%–12%)
Irigoyen et al55	USA, pediatric vaccination clinic, (N=1,273)	✓v	✓						No reminder	Appointment rates; vaccination coverage; cost of reminders	Non-attendance rate reduced by 6.7%
Kitcheman et al56	UK, inner-city outpatients, (N=764)	✓							No reminder	Attendance at first appointment; continuing attendance; hospitalization, transfer of care, discharge, presentation at accident and emergency and death by 1 year	Non-attendance rate reduced by 6.5%
Koury and Faris57	UK, ear, nose and throat clinics; (N=291)	✓			✓				No reminder	Non-attendance rate; willingness to receive SMS	Non-attendance rate reduced by 8%
Kwon et al58	USA, electrodiagnostic laboratory; (N=404)		✓						No reminder	Non-attendance without prior notification	Non-attendance reduced by 2.6% but not significantly. For appointments of particular test eg, electromyography, non-attendance rate reduced by 21.7%
Leong et al59	Malaysia, primary care clinics, (N=993)			✓	✓				No reminder	Attendance rates; costs of interventions	Non-attendance rate reduced by 10.9% (SMS); non-attendance rate reduced by 11.5% (mobile); cost of SMS reminder lower than mobile phone reminder
Liew et al60	Malaysia, primary care clinics, (N=931)			✓	✓				No reminder	Non-attendance rates	Non-attendance rate reduced by 9.3% (telephone call); non-attendance rate reduced by 7.4% (SMS)
Maxwell et al61	USA, inner-city clinics, (N=2,304)	✓		✓					No reminder	Appointment adherence rates	Non-attendance rate reduced by 3.2% (mailer). Non-attendance rate reduced by 2.1% (telephone call)
Nelson et al62	USA, pediatric dental clinics, (N=318)				✓ SMS				Mobile	Attendance rates	8.97% improvement in voice over text
Oladipo et al63	UK, colposcopy clinic, (N=189)			✓					No reminder	Attendance rate	Non-attendance rate reduced by 22%
Parikh et al64	USA, academic outpatient clinics, (N=9,835)		✓	✓					No reminder	Non-attendance rate; cancellation rate; patient satisfaction	Non-attendance rate reduced by 9.5% (personalized); non-attendance rate reduced by 5.8% (automated)
Perron et al65	Switzerland, HIV/primary care clinics, (N=2,123)	✓		✓	✓				No reminder	Rate of missed appointments, cost of intervention, and profile of patients missing their appointments	Non-attendance rate reduced by 3.6%
Prasad and Anand66	India, dental preventive care, (N=206)				✓				No reminder	Attendance rate	Non-attendance rate reduced by 43.7%
Reti67	New Zealand, hospital outpatients department, (N=109)			✓vi					No reminder	Non-attendance rates	Non-attendance rate reduced by 22%
Ritchie et al68	Australia, hospital outpatients department, (N=400)			✓					No reminder	Making the recommended appointment; attendance at scheduled appointment; and reasons for non-attendance at scheduled appointment	Non-attendance rate reduced by 16.2%
Roberts and Partridge69	UK, respiratory clinics, (N=504)			✓					Usual care	Attendance rate; cost of intervention	Non-attendance rate reduced by 15% compared with control (71%, n=258) and with patients who could not be contacted (68%, n=142) (P=0.007; P=0.004)
Rutland et al70	UK, genitourinary medicine clinic, (N=252)				✓				SMS plus health promotion message and no reminder	Re-attendance rates	Non-re-attendance rate reduced by 3.7% for text reminder only. Non-re-attendance rate reduced by 10.7% when reminder accompanied by health promotional message
Sawyer et al71	Australia, adolescent clinics (N=53)	✓		✓					No reminder	Clinic non-attendance, reason for non-attendance, and satisfaction with the booking system	Non-attendance rate reduced by 12%
Taylor et al4	Australia, physical therapy clinic, (N=679)				✓				No reminder	Rate of non-attendance without cancellation; cancellation and attendance rates; factors associated with non-attendance	Non-attendance rate reduced by 5%
Tomlinson et al72	UK, colposcopy clinic, (N=500)	✓vii							Standard information – no reminder	Attendance and default rates	Non-attendance rate reduced by 17%

Notes: Plus reply slip;

telephone call at 24 hours and 48 hours prior to appointment;

telephone call (doctor) and telephone call (secretary);

interactive voice response at 3 and 7 days prior to appointment,

postcard and postcard/telephone call;

telephone call from hospital and telephone call from doctor;

postal reminder and leaflet.

Abbreviations: N, number of participants in study; RCT, randomized controlled trial; SMS, short message service; CI, confidence interval; IVR, interactive voice response; FS, flexible sigmoidoscopy; NDC, Nurse-delivered calls.

Table 4

Judgement on quality of included trials (not already covered in included reviews)

Study	Review question/aims	Overall review quality	Implications for technologies	Implications for specific populations
Comfort et al48	To examine the effect of the provision of “tangible” engagement services during the intake period to women’s outpatient substance abuse treatment on rates of admission, retention, and service utilization. The engagement group received additional services such as transport and child care to help with the uptake and sustainability of service use.	Poor quality of reporting of the study design makes it difficult to judge quality. No reporting of blinding, allocation to groups, follow-up of all participants. The sample size was very small, which may have led to the lack of significant result.	Nil	This was a study about women receiving outpatient substance abuse programs, but did not produce any significant findings.
Costa et al,49Costa et al50	Reduce rates of FTA by sending SMS 2 days before appointment. Sent by IT department. Personalized to extent they included: name of institution, patient name, type of appointment, date and time. When necessary, included advice to arrive earlier.	Lack of reporting on allocation to intervention groups, blinding of participants, and lack of reporting on follow-up of all participants. Sample size calculations undertaken. Seems reasonably robust study, but not well reported.	SMS improves attendance rates compared with no reminder.	Rates of non-attendance highest in younger age groups (under 15, and decreases with age); in males; people attending for the first time; people attending for tests; highest on Wednesdays; higher in the morning; varies by mobile phone network; and is higher for medical than surgical specialties, but even higher in “others” (and specialty is the most significant difference). All P<0.5.
Goldenberg et al52	Improve adherence to new patient appointments; assess effectiveness of telephone reminders on compliance; to identify other factors affecting patient adherence to appointments.	Conference abstract only, so lacks detail, RCT, sample size 393, intention to treat used. Unable to comment on blinding or process of randomization.	Direct personal contact with patient increases likelihood of attendance.	Privately insured patients more likely to attend than Medicaid patients (61% vs 46%). More deprived communities less likely to attend.
Prasad and Anand66	To evaluate the efficacy of appointment reminders, sent as SMS text messages to patients’ mobile telephones, in comparison with no reminders given to patients, at outpatient clinics at the ITS Centre for Dental Studies and Research, Muradnagar, Ghaziabad, Uttar Pradesh, India.	Randomized at departmental, rather than patient level, so differences may be due to different treatment type/patient groups. No blinding, no intent to treat analysis provided.	SMS may improve attendance rate in comparison with no reminder (although evidence weak).	When appointment reminders were being scheduled, 22 patients stated that they did not understand the English language. Reminders for these subjects were sent in the local language on the mobile phone. In addition, a picture message of the institution was sent to seven patients.
Taylor et al4	To determine whether SMS reduces rates of non-attendance in physiotherapy outpatient appointments. Secondary aims were to evaluate effect of SMS reminders on cancellation and attendance rates and explore factors associated with non-attendance.	Good quality, well powered, well conducted RCT.	Strong evidence to show that people who were not sent an SMS were 1.77 times more likely not to attend their appointment (includes adjusting for other factors).	Other statistically significant contributors to the model were health condition/diagnosis of neck and trunk musculoskeletal disorder (OR, 2.86; 95% CI, 1.53–5.32), neuromuscular disorder (OR, 3.27; 95% CI, 1.17–9.17), and age (OR, 0.98; 95% CI, 0.97–0.995).
Griffin et al53	To test whether an interactive voice response (IVR) system phone call was equally effective as nurse-delivered phone call at educating and preparing patients for flexible sigmoidoscopy (FS) and colonoscopy examinations. Outcomes were appointment non-attendance and preparation non-adherence. Non-attendance defined as cancelling appointment or not attending appointment. Appointments cancelled by clinic not considered as non-attendance. Preparation non-adherence assessed whether patients had adequately prepared to complete procedure.	Well-constructed study with sample size calculation and intent to treat, randomization not specified.	Three arm RCT; nurse phone call 7 days before procedure, IVR system call 7 days before procedure, and IVR system call 3 days before procedure. All calls included an appointment reminder, information about preparation for examination, and encouragement to prepare for and attend the examination. IVR system was effective at reminding patients of their appointments. IVR system can effectively deliver complex information, eg, preparation information; equally effectively as phone calls from clinic nurses at delivering information; patients receiving IVR messages reported more “neutral” perceptions about phone calls; patients receiving calls from nurses reported more “very positive” perceptions about phone calls.	Nil
Rutland et al70	To determine whether SMS follow-up of patients who DNA booked GUM appointments improves subsequent re-attendance rates and to assess the impact of inclusion of a health promotional message on re-attendance rates.	Conference abstract only, so lacks detail, RCT, sample size 252, unable to comment on blinding, process of randomization or intention to treat analysis.	SMS message to clinic defaulters improves re-attendance rates compared with no reminder. An SMS reminder with the addition of a health promotional message SMS follow-up of clinic defaulters improves subsequent re-attendance rates compared with a reminder alone. The addition of a health promotional message to current routine clinic reminder texts may reduce DNA rates and warrants further study (although evidence weak).	Patients with a GUM health problem did not re-attend a clinical appointment, unless a reminder was sent. Reminders and reminders with an additional health promotional message may increase the likelihood of patients with a GUM health problem re-attending a clinical appointment.
Kwon et al58	To measure the effect of telephone reminders on electrodiagnostic laboratory attendance. Electrodiagnostic laboratory bookings of patients were randomly assigned to either a telephone reminder 1 day prior to their appointment, or a routine booking (no reminder). Non-attendance was the primary outcome measure, defined as non-attendance without prior notification.	Conference abstract only, so lacks detail, RCT, sample size 404, unable to comment on blinding, process of randomization or intention to treat analysis.	Telephone reminders reduced non-attendance at all appointments by 2.6% but not significantly. For appointments of particular test eg, electromyography, non-attendance rate reduced by 21.7%. This may indicate that specific types of technical appointments may be more effectively targeted (although evidence weak).	Patients who may be concerned that they have a genuine health problem may be more effectively targeted by telephone reminders.
Chiu45	To investigate the effectiveness of telephone reminders on attendance at CT scan appointments. The primary outcome measure was non-attendance at CT scan appointments.	Good quality, well powered, well conducted RCT.	Telephone reminders were effective at reducing non-attendance rate for radiological appointments. Patients who were successfully contacted were significantly more likely to attend than those patients who were not successfully contacted.	Nil
Koury and Faris57	To investigate the effectiveness of using an SMS reminder compared with usual procedures in NHS ENT outpatient departments. Primary outcome was attendance rates in each of the groups.	Lack of reporting on randomization procedures, allocation to intervention groups, blinding of participants, and lack of reporting on follow-up of all participants. No information about sample size calculations undertaken. Seems reasonably robust study, but likely that editorial constraints may have led to poor reporting.	In comparison with no reminder, SMS reminders were effective at reducing non-attendance rates at ENT appointments.	Nil

Abbreviations: FTA, failed to attend; SMS, short message service; IT, information technology; RCT, randomized controlled trial; OR, odds ratio; CI, confidence interval; IVR, interactive voice response; FS, flexible sigmoidoscopy; DNA, did not attend; GUM, genitourinary medicine; CT, computerized tomography; NHS, UK National Health Service; ENT, ear, nose and throat.

Reminders increase attendance at appointments

There was consistent evidence that reminder systems improve appointment attendance across a range of health care settings and patient population sub-groups. Only one of the 31 RCTs did not show a significant reduction in non-attendance.48 “Simple reminders”, which provide details of date, time, and location of appointments, were most frequently investigated. “Reminder plus”, which provides additional information (eg, orientation information, health information, etc) over and above date, time, and location of the appointment, was less commonly investigated. Both were effective at reducing non-attendance.

There was consistent, strong evidence from SRs16,34,35,37,41 and RCTs32,65 that simple reminders are effective at increasing attendance at appointments compared with no reminder. In SRs, the pooled effects of simple reminders on appointment attendance vs no reminder indicated significant increases in attendance, with relative risks ranging between 1.06–1.10.34,35 One SR reported a weighted mean relative change of 34% from the baseline non-attendance rate.37 In RCTs the difference in attendance between subjects who received reminders and those who did not ranged from 5% in an Australian physiotherapy clinic to 44% in an Indian dental preventive care clinic.4,66 There was strong evidence from SRs16,34,35 and RCTs42,65 that there is no differential effectiveness between different reminder technologies, eg, SMS reminders, phone call reminders or other reminders.

There was weak, but consistent evidence from five studies that “Reminder plus” is more effective than simple reminders at reducing non-attendance. Examples of “Reminder plus” interventions include SMS notification of appointment with a health promotional message or postal reminders with additional information about medical procedures and the importance of follow-up.70,72 A Cochrane Review40 of the effects of reminders on clinic attendance for those with suspected serious mental illness, identified one small study favoring a letter with an orientation statement (ie, a short paragraph, taking about 30 seconds to read, explaining the program of care, the fee system, and providing gentle encouragement) over a simple letter prompting attendance.73 A second SR to assess the effect of reminder systems on non-attendance rates at new outpatient appointments, found limited evidence in three studies, that “Reminder plus” was more effective than simple reminders.38 In these studies, the reminders threatened sanctions for non-attendance, offered rewards for attendance or provided orientation information about the clinic.

Reminders promote cancellation/reallocation of appointments

There is evidence from three RCTs that personal phone reminders significantly increase patient cancellation and rescheduling rates.54,64,67 Patients who received a telephone reminder were more likely to cancel or reschedule their appointment (17%–26%) compared with a control group who had received no reminder (8%–12%).54,67,74 Clinics were then able to re-allocate between 27% to 40% of the cancelled appointment slots.54,65,74 Telephone reminders carry the inherent advantage that patients who are unable to attend can cancel and/or reschedule their appointment at the time of their contact with staff.67 We also found strong evidence that SMS reminders do not increase appointment cancellation or rescheduling,75–77 however this may be because SMS reminders are not conventionally deployed with this in mind.

Reminder systems are not optimally employed

Our review found sufficient strong consistent evidence to indicate that the performance of reminders, and therefore appointment systems, is suboptimal. Six key areas which lead to sub-optimal reminder effectiveness were identified.

Accuracy of patient records

Patient contact details are frequently incorrect or out-of-date.78,79 The likelihood of inaccurate patient records corresponds with populations at greater risk of non-attendance,80 including less geographically stable communities such as students, young adults or socio-economically deprived groups who may frequently change address or telephone numbers.81

Reminders may not be received

Successful contact rates for telephone reminders are low ranging from 30% to 60% in most health care settings. Reasons for non-receipt of telephone reminders are that landline calls are often made during business hours (9 am–5 pm), during the working week (Monday to Friday), when it is likely that patients will be out.82 In addition, non-receipt may occur because patients do not have a telephone, they do not answer the telephone or the contact number was incorrect.74,83 Most telephone reminder systems do not leave messages for reasons of confidentiality.

SMS reminders are reported to have successful contact rates of 97%–99%.79,84 Successful contact is assumed when the mobile phone indicates “message sent” being received by the sender.44,59,66 However, many patients may either not receive their SMS reminder or may receive and ignore a reminder that was not intended for them due to incorrect data entry on hospital systems.77,79 Some clients may not receive their text message until after their scheduled appointment because of delays in delivery of the text or because their phones were switched off, out of battery or out of credit.85 One disadvantage of using SMS reminders alone is the different levels of access to a mobile telephone. Mobile phone ownership declines sharply with increasing age,86,87 although the total numbers of older people with mobile phone are increasing annually.

Understanding the reminder

Cognitive ability, literacy level, and language determine patient comprehension of reminders, irrespective of format. These are important considerations for health services serving older populations, travelling communities, inner-city deprived populations, and multilingual communities. The studies included in our review did not explore these factors. Two RCTs explicitly excluded patients who did not speak the official language (English) fluently, those with dementia, or with significant cognitive impairment.68,88 Only one RCT used multilingual research assistants to make the reminder phone call.65 Reminder systems can cater for different languages.41,66

Timing of reminders

We found strong evidence that the timing of reminders, between 1 and 7 days prior to the scheduled appointment, has no adverse effect on patient attendance behavior.37,89 SMS or telephone reminders are typically sent either the day before or on the day of the health care appointment.49,57,63 Sending the reminder close to the appointment means that the patient may either not have time to act on it or they may receive the reminder after the allotted appointment time.65 Sending reminders early allows patients to re-arrange commitments, which may increase the likelihood of a patient attending, cancelling or rescheduling.54,64,67

Patient does not cancel or re-schedule the appointment

There are numerous reasons why patients fail to either cancel or reschedule their appointment. Simple reminders rarely ask patients to cancel appointments, particularly SMS reminders where space for text is limited.46,62 Some SMS reminders ask patients to call a telephone number rather than replying to the text.4,57 Patients frequently encounter problems accessing health care systems which can thwart their intention to cancel and rebook.86,90 Problems include difficulties accessing central booking lines, including the phone being engaged, having to wait a long time to speak to someone or the call being disconnected with no option to wait or leave a message.91,92 In some cases, patients were warned by others of the difficulties of accessing a central booking line, which deterred them from making contact.86 In two studies, patients who failed to attend stated that they had already phoned or written to cancel their appointment, indicating difficulties with cancellation systems or internal hospital communication prevented cancellations being passed on to the relevant clinic.81,86

Lack of tailoring to high risk groups

There was weak evidence that patient age has no impact on reminder effectiveness, suggesting reminder systems can be employed across all age groups.18 However, few studies have investigated the differential impact of reminder systems between population sub-groups. There was weak but consistent evidence that deprivation, minority ethnicity, substance abuse, mental health problems, and comorbidities/illness are associated with non-attendance at appointments.93,94 There was little evidence of tailoring of reminder systems to meet the needs of these groups of patients.

Discussion

This review found consistent, strong evidence that all reminder systems are effective at reducing non-attendance at appointments across diverse service contexts and patient populations. There is no clear indication of differential effectiveness between different simple reminder systems. However, there is some evidence that “Reminder plus” interventions can be more effective than simple reminders. Our review of the available evidence suggests that “Reminder plus” may result in higher attendance than simple reminders for first appointments and screening appointments and that for subsequent follow-up appointments simple reminders and “Reminder plus” may produce comparable increases in attendance for most patients most of the time. However, further research employing appropriate comparative designs is needed before firm conclusions can be drawn.

There is also strong consistent evidence that reminders can increase patient cancellation/rebooking rates, however the success may depend to some extent upon the nature and the timing of the reminder. We found only three studies investigating this area of effectiveness,54,64,67 therefore further research exploring the effectiveness of reminder systems to promote cancellation/rebooking and rescheduling of appointments is warranted.

Based on the findings presented in this review, the small amount of evidence that some patients find reminders intrusive or confusing is outweighed by the benefits.95 The use of reminders appears to be both acceptable and feasible across a range of health care settings,42,65 and we therefore propose that all patients should receive a reminder and that all health care services operating outpatient appointment systems should employ reminder systems.

Whilst reminder systems can increase attendance, cancellation, rescheduling and reallocation of appointments, this review identified six key factors which limit the efficiency of both reminder and appointment systems. Reminder systems are often employed with the objective of increasing attendance rates, with limited attention given to cancellation and/or rescheduling of appointments. Full attendance at appointments is unlikely to be achievable; therefore appointment cancellation and rescheduling should be seen as desirable outcomes. Appointment systems can be optimized if patients cancel and reschedule unwanted appointments, allowing health care services to re-allocate the cancelled appointment to a different patient. If appointment and reminder systems are to realize their full potential this will require a whole systems approach to looking at the characteristics of current systems for attendance, cancellation, rescheduling and re-allocation of appointments to other patients. A summary of proposed strategies is outlined in Figure 3 and discussed in greater detail to follow.

Figure 3

Summary of strategies to optimize reminder systems.

Abbreviation: SMS, short message service.

Optimization strategies

Health services, particularly those serving geographically less stable communities, should have robust procedures for maintaining and updating patient records.81

In many health services it will be relevant to consider the use of both simple reminders and “Reminder plus”. Depending on the nature of the information provided, “Reminder plus” may help patients to feel more confident about attending their appointment, particularly for first appointments and screening appointments.56,96 Thereafter, the use of simple reminders may be sufficient for increasing attendance at follow-up appointments in most health care settings.

Since the timing of appointment reminders appears to have no appreciable impact on attendance behavior when delivered up to 7 days before an appointment, we propose that reminders should be delivered early enough to allow patients to re-arrange commitments so that they can attend the appointment and receive the care that they need. Alternatively, if unable to attend, patients will have sufficient time to cancel and reschedule their appointments and allow health services to re-allocate and rebook appointments.54,64,67 To support and enhance rescheduling it is appropriate to frame reminders to ask patients to cancel and rebook inconvenient appointments. In addition, robust structures, which are easy to navigate and which require minimal effort from the patient, are required to support cancellation. Automated methods of cancellation, eg, SMS messages or email, are perceived by many patients as easier than methods which require direct contact since they offer flexibility to cancel at a time convenient to the patient and reduce the need to provide explanations for cancellation.18,34 Following cancellation of appointments, rescheduling of the appointment, if it has not occurred synchronously, also needs to be easy for the patient. For example it may be sensible, in some health care settings, to have central booking lines which are open 24 hours a day.

There is little evidence of tailoring of reminder systems to meet the needs of vulnerable groups of patients who are at high risk of non-attendance; this includes deprived and ethnic groups, substance abusers, and populations with co-morbidity and illness.93,94 Given the likely coincidence of higher levels of non-attendance and health need, it is in the interests of health services to monitor whether specific groups of patients are being disadvantaged by the chosen reminder systems. Simple reminders and automated reminders may be ignored, overlooked or misunderstood, particularly if patients are experiencing an increase of their health problem. We therefore hypothesize that reminders with direct personal contact might be appropriate in these groups, since the flexibility of information, advice or support which can be offered may help to overcome barriers to attendance or to cancel unwanted appointments. To facilitate attendance in these groups more intensive reminder systems are advocated. Examples of this include sequential reminders which were effective at improving attendance in a Swiss AIDS clinic.65 This consisted of: first, a phone call to either landline or mobile; second, an SMS if participants do not answer the phone after three attempts and have a mobile phone; and finally a postal reminder if participants do not answer the phone, have no mobile phone or landline at all. Intensive approaches, such as “stepped reminders” and patient navigators have also been effective at increasing attendance at screening and immunization programs in disadvantaged and vulnerable populations and might also be effective at reengaging similar groups of patients who have dropped out of treatment.36,97–99 Such designs, though labor intensive would reach the maximum number of participants and may increase attendance rates and simultaneously have a cost benefit.

An effective reminder and cancellation system will increase the already heavy workload of outpatient clinics.52 Clinicians frequently fill missed appointments with alternative activities such as completing dictation, making telephone calls or consulting with colleagues.56 If building in processes to optimize cancellation and rescheduling, then health services will need also to consider the impact on staff that frequently utilize non-attendance at appointments as an opportunity to catch up on other health care related activities.

Strengths and limitations

Our approach to this review, which combined an SR with an evidence synthesis informed by realist principles, has numerous strengths, including a structured search protocol requiring thorough searches of electronic databases, reference lists, and citations. As a consequence we believe that we have assembled the widest possible body of relevant knowledge which has relevance across all health care services which use appointment systems. In addition, our review informed by realist principles includes the strong embedding of our findings in the extracted data. This stems from the practical orientation of our review and facilitates the production of implications for practice. There are also limitations to our review. Generally speaking the SR method seeks to provide a precise answer to a tightly focused question. Such reviews are most useful where there is a high degree of homogeneity around the five PICOS elements, namely the Population, Intervention, Comparison, Outcomes, and Study types. A wide range of population types, intervention, comparison, and outcomes is included within the RCTs we identified. However, use of this wider approach offers greater analytical capability in terms of understanding contextual and mechanistic factors that would not have been evident in a more narrowly focused review and increases confidence that the findings have relevance in a wide range of service settings.

Research implications

We recommend future research activities in three main areas. Firstly, more studies should routinely consider the potential for differential effects of reminder systems between patient groups in order to identify any inequalities and remedies. Secondly, “Reminder plus” systems appear promising but there is a need for further research to understand how they influence attendance behavior. Finally, further research is required to identify strategies to “optimize” reminder systems and compare performance against current approaches.

Conclusion

In the absence of clear contraindications all health services should use simple reminders or “Reminder plus” for all patients. More intensive reminder alternatives may be relevant for key groups of patients: deprived, ethnic, substance abusers, and those with comorbidities and illness.

There is evidence that reminders are used sub-optimally. To optimize appointment and reminder systems, health services should tailor reminder systems and adopt supportive administrative processes to enhance attendance, cancellation, rescheduling, and reallocation of appointments to other patients.