Measuring access to primary care appointments: a review of methods.

BACKGROUND: Patient access to primary care appointments is not routinely measured despite the increasing interest in this aspect of practice activity. The generation of standardised data (or benchmarks) for access could inform developments within primary care organisations and act as a quality marker for clinical governance. Logically the setting of targets should be based on a sound system of measurement. The practicalities of developing appropriate measures need debate. Therefore we aimed to search for and compare methods that have been published or are being developed to measure patient access to primary care appointments, with particular focus on finding methods using appointment system data.
METHOD: A search and review was made of the primary care literature from 1990 to 2001, which included an assessment of online resources (websites) and communication with recognised experts. The identified methods were assessed.
RESULTS: The published literature in this specific area was not extensive but revealed emerging interest in the late 1990s. Two broad approaches to the measurement of waiting times to GP appointments were identified. Firstly, appointment systems in primary care organisations were analysed in differing ways to provide numerical data and, secondly, patient perceptions (reports) of access were evaluated using survey techniques. Six different methods were found which were based on appointment systems data.
CONCLUSION: The two approaches of either using patient questionnaires or appointment system data are methods that represent entirely different aims. The latter method when used to represent patient waiting times for 'routine' elective appointments seems to hold promise as a useful tool and this avoids the definitional problems that surround 'urgent' appointments. The purpose for which the data is being collected needs to be borne in mind and will determine the chosen methods of data retrieval and representation.

Background

Primary care is under scrutiny along with other public services to improve access to its users. Access in primary care is typically conceptualised as the achievable access to appointments with clinical professionals, although it is not routinely measured in most practices. It is widely recognised that it represents an important dimension in determining the quality of care [1,2]. Nevertheless, it is also known that waiting times can vary widely in differing localities and countries, from patients being seen the same day to a wait of several weeks. Most primary care organisations have no more than a perception of variable demand and no method of comparing fluctuating levels of access to appointments within or between practices. Measuring patient access could generate useful information for patients, clinicians and practice managers. Demand management initiatives and ways to optimise access could then be audited bearing in mind the impact of such initiatives on opportunity costs [3-5].

There are potential difficulties, recognised by a recent discussion document [6], that have been accentuated by policy and political influences. The NHS Plan in the UK [7] suggested that patients should have access to primary care services within 48 hours, but the concept lacked sufficient definition. The proposed new GMS contract includes an optional 48 hour target for access to GP appointments. But what exactly should we measure? Access can be measured at many different interfaces, from the wait for traditional services such as appointments with a clinician to the alternative solutions of nurse triage, nurse led clinics, telephone advice (including NHS Direct) or electronic mail responses. Differing interpretation of terms can also cause confusion. How soon for example should a problem that is defined by a patient as urgent be seen in general practice? The perception of urgent differs between patients themselves [8] and between doctors and patients. The concept of routine appointments is easier to define and quantify. A proposed measure is the waiting time for the next available routine appointment but this provokes debate about whether this should be practice-based or specified for individual clinicians. If the latter, factors such as part-time working, practitioner popularity, the creation of multiple review appointments are likely to rapidly diminish a clinician's accessibility. How transparent would a practice want to be about such data and how useful or acceptable would it be to publish information at the clinician level? [9]

Recognising the complex nature of this issue, we set out to review the literature. Our main aim was to search the international primary care literature for methods that had been, or were being developed to measure access to GP appointments, focusing on measures using appointment system data. Once identified, existing methods (tools, scales or other instruments) would be compared, with specific attention given to the type and levels of access they aimed to assess.

Method

Preliminary searches indicated that this area did not have an extensive or long-standing research literature; most articles had been published after 1998. A broad but systematic search process was designed to allow for a poorly indexed publication pool. Medline, PubMed, Clin Psyc and ASSIA were searched for relevant publications between 1990 and 2001. The following MeSH terms were used: family practice, health service accessibility (organisation and administration, statistics and numerical data, standards, trends, methods, manpower), appointments and schedules (waiting lists), research design, health service needs and demand, weights and measures, quality of healthcare, management audit, patient satisfaction, health service needs and demand. In addition the following keywords were used: general practice, access, appointments (same day, urgent, routine) appointment systems, measurement, measures, tools, scales, demand, availability, audit and waiting times. Terms were used both singly and in combination. Title searches were used to increase the sensitivity. All citations and abstracts were appraised for relevance and full articles selected for examination by two researchers independently (GE and WJ). Key authors were contacted directly [1,12,16,18,21,25,27] and searches conducted on their previous work. Departments of General Practice in Universities in the United Kingdom were also asked to send details of any relevant research. Conference literature [21] and non-peer reviewed literature obtained from websites was also appraised. Relevant websites were identified using and the search terms health service delivery, access to primary care and general practice. The following sites were reviewed: the Royal College of General Practitioners [6], the National Primary Care Research and Development Centre [1], The National Primary Care Development Team [10] and the Centre for Innovation in Primary Care [11].

Studies or articles were included in the review if they described tools, scales, questionnaires or other methods of measuring actual patient access to appointments. We also included descriptions of methods that were currently being developed in this field, provided they had undertaken pilot studies and had completed one data collection exercise. Articles were excluded if they were purely editorial.

Results

A total of 1763 citations were initially identified and 38 articles retrieved for detailed assessment from the Pubmed and the Medline searches. Clin Psych and ASSIA searches provided some overlap but no new relevant material. The most helpful pointers to relevant publications were obtained from the website searches and personal communications rather than the traditional search engines. Two broad approaches to the measurement of patient access were identified. Firstly, appointment systems in organisations were analysed in differing ways to provide numerical data and, secondly, patient perceptions (reports) of access were evaluated using survey techniques.

Methods using appointment system data

Table 1 summarises the six identified methods that were based on appointment system data.

Table 1

Comparison of methods based on appointments systems to measure access to primary care

	Third appointment [21]	NEMAS [15]	Ledlow [14]	Access Response Index AROS [22]	Campbell [12]	Kendrick [13]
Measurement	The 3rd available routine appointment at 12 midday, one day per week, for every clinician	Date of patient call and appointment provided. GP requested and GP allocated.	Appointments demanded but not available in US style primary care clinics compared to community clinics. Demand versus availability gap coded into 4 categories.	Number of days until next available routine appointment (with any clinician) at 4 pm, every working day	Number of appointments provided at the beginning of the day and the number still available. Total number of patients seen during the day, noting the number of 'extras'	Number of appointments available at the start of the day and the number of patients seen as 'extras' at the end of the day
Frequency of data collection	Once a week	Continuous	Daily	Once a day	Twice a day	Twice a day
Weighted for part time staff	Yes	No	No	No	No	No
Named clinician access measured	Yes	Yes	No	No	No	No
Data analysis	Weekly median score and monthly average	Computerised	Demand versus availability gap	Computer to work out 5 day moving average	Data related to practice list size, with rates given per 1000 patients	Daily tally
Results	Weekly snapshot of patient access profile	Complete computerised analysis of practice appointment system	Feedback reports generated to clinic staff	Trends across weekly schedules.	Bar charts represent number of appointments offered versus number of patients seen. Start of day appointment availability categorised as low, medium and high	Graphical display of extras versus number of free appointments during the day
Extent of and reason for use	Primary Care Collaborative in England. To inform implementation of advanced access	145 teaching practices Audit	US Military Clinic Study	10 practices To inform improvement	19 practices Research Study	1 practice Research Study
Co-ordination	National Primary Care Development Team	Department of General Practice, University of Glasgow.	Healthcare Programs Central Michigan University.	University department of General Practice	University Department of General Practice, Edinburgh.	Department of Primary Health Care, University of Southampton.

Three methods determine appointment availability and/or the satisfaction of demand on a daily basis but do not measure the days wait for appointments. Campbell measured clinician availability by recording the number of provided appointments at the beginning of each day plus the number of these un-booked at this time [12]. At the end of the day numbers seen and numbers of extras were counted, and adjusted for practice list size. Kendrick and Kerry recorded the number of available appointments at the beginning of the day and the number of extras seen at the end [13]. Ledlow also suggested a categoriations system for recording differing levels of unmet daily demand in a military medical service [14]

Three methods measure access as days waited by representing appointment system data. A computerised package, NEMAS [15,16] enables practices to audit four areas: practice service provision (including 'appointment availability' and patient satisfaction), chronic disease management, drug monitoring and significant event analysis. The method calculates the mean time waited in days plus the minimum and maximum patient waits. Data can be presented for the whole practice or for individual GPs. Data entry involves using electronic forms to record the date of appointment request, the date the patient was seen, whether it was an elective, forced (i.e. next available clinician) or urgent appointment, the clinician requested and the clinician actually consulted. The costs of collecting data has varied considerably depending on which staff member is employed for the task [17]. Transfer of data from the practice system can be automatic with compatible systems (personal communication).

The National Primary Care Development Team is measuring access as part of the adoption of the system of 'Advanced Access', a system developed in the USA [18,19], as a response to patient waiting times of 4 – 6 weeks for routine appointments in primary care [20]. This method collects data on one day per week (which can vary) at 12 noon. The number of days to the third available routine appointment for each clinician is recorded and a median figure calculated to represent an access score for the specified week. Over a month, the average of four median values is taken to represent a monthly access score. The third appointment is chosen, rather than the first, in order to negate the effect of sudden cancellations which otherwise could give false impressions of availability if at the moment of measuring there is one sudden cancellation in a schedule that is otherwise booked for several days ahead. The third appointment has been found by trial and improvement to best represent the actual waits involved (personal communication). Embargoed appointments are not included. A system of weighting has been developed [21] for part time workers to enable their scores to be incorporated.

The Access Response Index (AROS) was developed as a rapidly calculable measure of organisational access [22]. This index is derived by counting the number of days until the next available routine appointment, with any clinician, once during every normal working day. The data is recorded at four pm – a time chosen to avoid the influence of embargoed appointments that many organisations use to maintain urgent same day availability. The results are plotted on a graph, the daily fluctuation represented by the raw data is smoothed to a demonstrate trends by calculating a 5-day moving average (a data point which is the mean of every successive 5-day group). An example of the data produced by this measure is shown in Figure 1.

Figure 1

AROS scores for routine appointment availability (data from 11 practices)

Methods using patient questionnaires

Four patient experience questionnaires were found that contained access assessments and the relevant items are outlined in Table 2. The General Practice Assessment Survey (GPAS) [23] was developed by the National Primary Care Research and Development Centre, by adapting the Family Practice Assessment Survey (FPAS)[24]. A second survey named Europep [25,26] has been validated in 10 European countries. Thirdly Baker describes the use of a validated surgery satisfaction questionnaire (SSQ) that he developed [27,28] and finally Grogan's patient Satisfaction Questionnaire contains two sections that ask about access and appointment availability[29].

Table 2

Patient survey instruments: items used to determine access perceptions

	Survey items		Response ratings
GPAS [23]	6.	Thinking of times when you want to see a particular doctor:	a) 5 point scale, 1 = same day, 5 = more than 5 days
		a) How quickly do you get an appointment?
		b) How do you rate this?	b) Range from 1 = very poor, 6 = excellent
	7.	Thinking of times when you are willing to see any doctor:
		a) How quickly do you get an appointment?	8. Yes / No / Not applicable / Don't know
		b). How do you rate this?
	8.	If you need an urgent appointment to see your GP can you normally get one on the same day?
EUROPEP [25,26]	What is your opinion of the general practitioner and/or the practice over the past 12 months with respect to:		5 point scale (poor to excellent)
	19)	Getting an appointment to suit you?
	23)	Providing quick services for urgent health problems?
Baker [28]	10)	It can sometimes be difficult to get an appointment with my doctor at this surgery.	5 point agreement scale
	14)	It can be hard to get an appointment for medical care right away.
Grogan [29]	33)	Getting an appointment at a convenient time is easy.	5 point agreement scale
	34)	Appointments are easy to make whenever I need them.
	35)	It is often difficult to get an appointment with a doctor.
	36)	It is easy to see a doctor of my choice.

Discussion

Principal findings

This review of access measurement reveals the heterogeneous nature of the methods and the lack of any widely accepted conceptualisation of patient access. Identified measures are either practice centred using appointment data or patient orientated via surgery satisfaction questionnaires. It is clear that these two methods represent entirely different aims. It is not possible for episodic patient surveys to provide data that has enough currency or accuracy to inform organisational responses to patient demand.

Also demonstrated are inherent problems over definition of terms – defining what is to be measured and setting of targets. There appear to be three appointment categories over and above emergencies (which by definition cannot be given 'appointments'). Appointments for healthcare could be categorised into urgent, soon, and elective. Urgent appointments are typically seen as requests for same-day consultations. The soon category would fit problems that should be seen within two or three days to prevent escalation or symptom prolongation. Finally, routine or elective appointments suit individuals who value an agreed time window over other factors. The methods that represent patient waiting times for 'routine', i.e. elective appointments seem to hold promise as they avoid the definitional problems that surround 'urgent' appointments, and the different views that patients, clinicians and others have about 'urgency'. The task of deciding whether to represent access profiles for organisations or for individual clinicians also needs careful consideration. It may be more feasible (and less threatening) to routinely measure organisational access, especially if the data is to be used for benchmarking purposes.

To attempt to measure access means to obtain meaningful data from a dynamic system that is not always in equilibrium. The access experienced by an individual varies determined by demand, adequate appointment provision, sudden cancellations and block release of held appointments. We have distinguished two approaches to overcome this, either for a full statistical analysis using compatible systems or data retrieval software, or to use a simple snapshot method, deciding whether data-smoothing methods such as daily moving averages or aggregated weekly median scores are the best portrayal of overall access patterns.

Strengths and weaknesses of the study

Multiple search methods were used to ensure that the breadth of literature and online resources were examined as systematically as possible. The searches proved difficult and reflect the emergent status and the diversity of terms used in this area. We may have overlooked methods developed in other healthcare systems.

Implications of the findings to healthcare services and research

The lack of a widely agreed measurement method to represent patient access to primary care services will make it impossible for practices to compare their response to patient demand with any degree of certainty. The 'third appointment' system is the most widely used method and is currently supported by the National Primary Care Development Team in England (but has no equivalent support in Scotland, Wales or Northern Ireland). It is however a relatively complex manipulation of appointment system data, and incorporates individual clinician availability. It seems from first principles that the important requirements of a tool designed to measure a dynamic concept such as patient access is simplicity and ease of regular data collection, so that longitudinal data patterns capable of indicating trends in organisations can be generated rather than data on individual clinician availability.

A recent survey compared mechanisms used to manage requests for same day appointments[30]. Murray's proposal of doing today's work today (Advanced Access) eliminates appointment categories and the work involved in negotiating urgency by dealing with virtually all demand on the day it arises[31]. Access is an important determinant of healthcare quality but what are the implications of this approach to the balance of overall quality? Continuity of care, whilst not important for some problems is desirable for others [32,33]. Too drastic a shift in favour of access is likely to be at the cost of reduced continuity and a diminution of other services, such as screening and chronic disease management. Measures of organisational quality need to be aware of the dangers of focusing too much on one dimension, and should work towards the creation of measures that balance scores across internal and external requirements [34].

An effective consultation with a well trained clinician who knows the patient and who has access to a well structured longitudinal record will probably remain a gold standard and the issue of immediate 'access' should not be elevated above all the other components in this equation. Nevertheless, it seems prudent to develop measures that provide a better understanding of patient access to organisations with similar resources. Measuring the interval to the next available routine appointment, whilst measuring the measurable [35] will, if a consistent standard can be agreed, provide benchmark data, and mark an important step towards a compendium of methods to assess quality in primary care.

Conclusion

The two approaches of either using patient questionnaires or appointment system data to measure access are methods that represent entirely different aims. The latter method when used to represent patient waiting times for 'routine' elective appointments seems to hold promise as a useful tool and this avoids the definitional problems that surround 'urgent' appointments. The purpose for which the data is being collected needs to be borne in mind and will determine the chosen methods of data retrieval and representation

Competing Interests

The authors of this article have piloted the AROS Index.

Authors' Contributions

Peter Edwards, Melody Emmerson and Glyn Elwyn were involved in the development of the Aros index. Wendy Jones and Glyn Elwyn conducted the Aros pilot study and the literature review with contributions from Adrian Edwards and Richard Hibbs.

Pre-publication history

The pre-publication history for this paper can be accessed here: