Determinants of out-of-hours service users' potentially inappropriate referral or non-referral to an emergency department: a retrospective cohort study in a local health authority, Veneto Region, Italy.

BACKGROUND: A growing presence of inappropriate patients has been recognised as one of the main factors influencing emergency department (ED) overcrowding, which is a very widespread problem all over the world. On the other hand, out-of-hours (OOH) physicians must avoid delaying the diagnostic and therapeutic course of patients with urgent medical conditions. The aim of this study was to analyse the appropriateness of patient management by OOH services, in terms of their potentially inappropriate referral or non-referral of non-emergency cases to the ED.
METHODS: This was an observational retrospective cohort study based on data collected in 2011 by the local health authority No. 4 in the Veneto Region (Italy). After distinguishing between patients contacting the OOH service who were or were not referred to the ED, and checking for patients actually presenting to the ED within 24 hours thereafter, these patients' medical management was judged as potentially appropriate or inappropriate.
RESULTS: The analysis considered 22 662 OOH service contacts recorded in 2011. The cases of potentially inappropriate non-referral to the ED were 392 (1.7% of all contacts), as opposed to 1207 potentially inappropriate referrals (5.3% of all contacts). Age, nationality, type of disease and type of intervention by the OOH service were the main variables associated with the appropriateness of patient management.
CONCLUSIONS: These findings may be useful for pinpointing the factors associated with a potentially inappropriate patient management by OOH services and thus contribute to improving the deployment of healthcare and the quality of care delivered by OOH services. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

This is the first study to investigate out-of-hours (OOH) referrals to emergency departments (EDs), considering the appropriateness of non-referrals and referrals by OOH physicians, and delineating the sociodemographic, clinical, environmental and logistic determinants of any inappropriate behaviour.

The strength of the study lies in the analysis of all 22 662 OOH service contacts made by a population served by a local health authority (LHA 4 in the Veneto Region), meaning that the findings cannot be distorted by any research hypothesis.

A limitation of the study stems from our arbitrary classification of potentially inappropriate referrals to the ED (based on a nationally adopted definition of appropriateness), which clearly influences the reported prevalence of potentially inappropriate patient management in our sample.

Another limitation is in that our analysis was conducted only on the LHA 4 records, so we were unable to follow-up patients admitted to an ED outside the territory covered by LHA 4.

Introduction

Primary care serves as the cornerstone for building a strong healthcare system that ensures positive health outcomes and health equity.1 Out-of-hours (OOH) services are a fundamental part of primary healthcare, providing continuity of care for patients with urgent clinical conditions who cannot wait until the next working day to see a doctor. OOH physicians consequently act as gatekeepers for the provision of secondary care for patients with problems that are not life-threatening and who do not need immediate high-level care. It has also been demonstrated that when general practitioners (GPs) manage patients with primary care needs, there is a reduction in the associated costs with no apparent detrimental effect on outcome.2 3

OOH physicians should only refer cases to an emergency department (ED) if they have clinical conditions requiring urgent treatment or higher level diagnostic services without delay.4 For patients who are genuinely urgent cases, non-referral or delayed referral to the ED can pose a serious problem: prehospital times are often a matter of life and death, and delaying hospital admission even by just a few hours raises the risk of death associated with several diseases.5–8 OOH services and EDs are two types of service that differ in their organisation, policy and structure; they are not interchangeable but complementary, and both should operate appropriately to ensure the best possible functioning of the health service as a whole.

In recent times, there has been a significant worldwide increase in ED attendance, relating mainly to higher numbers of non-urgent cases. In Italy, for example, the Italian Society of Emergency Medicine (SIMEU) reported in 2010 that ED visits had grown by 5–6% a year over the previous 5 years, and this was partly as a consequence of inappropriate referrals by primary care physicians.9 Similarly, visits to the ED in the USA rose from about 92.6 million in 1993 to 133.6 million in 2013, reaching 423 visits per 1000 population/year.10 The Australian Institute of Health and Welfare (AIHW) recently reported that 2 176 612 ED attendances in 2012–2013 were potentially referrals by GPs.11 The situation is much the same in Europe: France, Germany and the UK have seen a significant increase in the number of cases seen at the ED. Non-urgent patients have been recognised as a potentially avoidable contributor to the problem of ED overcrowding: research found a prevalence of inappropriate ED use that varied from 10% to 90%, depending on the criteria used to judge appropriateness, and in nearly half of the studies it ranged from 24% to 40%.12 There is no generally accepted and practical definition of what constitutes an ‘appropriate’ case for referral to the ED, and what constitutes an ‘emergency’. It is therefore hardly surprising that we find an enormous diversity in the reported proportions of visits judged to be inappropriate.12

Numerous studies have reported that an excessive number of patients with non-urgent clinical conditions are seen at the ED, but few have focused on the degree to which physicians' inappropriate referrals contribute to this problem. Some studies in Europe assessed the influence of OOH primary care services on ED attendance, and found that improving the OOH services reduced the population's recourse to the ED.13 14

Only a very few studies also considered patients who were erroneously not referred to the ED, and most of these cases were due to inaction by GPs and other primary care providers.15 In the case of curative services, the relationship between timing and efficacy is crucial, making it important to ensure that patients go to the right place at the right time for the right intervention.16 Safe, good quality, consistent and effective in-hours and OOH primary care services are crucial for providing care as close to a patient's home as possible.

The aim of this study was to investigate how often OOH physicians' patient referrals or non-referrals to the ED are potentially inappropriate. After identifying the dimension of the problem, a second step involves seeking the sociodemographic, process/context and clinical variables associated with a higher risk of potentially inappropriate patient referral or non-referral to the ED.

Materials and methods

Context

The Italian National Health System (NHS) was established in 1978 and modelled along the lines of the British NHS. It is mainly a public system financed by general taxation. From an organisational viewpoint, the Italian territory is divided into 140 local health authorities (LHAs), each responsible for providing health services to its local population.

All Italian citizens or foreigners residing in Italy are registered with a GP and they are supposed to consult their GP for health conditions. During the day, patients can visit the general practice where they are registered; after hours, they can consult one of the local OOH services, which have infrastructure and resources allocated by the LHA. Despite regional differences being introduced when responsibilities for the NHS were decentralised, OOH services are still regulated by a nationally shared agreement that defines the tasks, activities and salaries of OOH physicians. Dedicated personnel for OOH services are recruited from waiting lists drawn up by the LHA, and preference is given to qualified GPs. OOH services in Italy are currently provided by about 12 057 physicians working under an agreement with the LHA at 2893 OOH service delivery points. They are either regular postholders, or temporary substitutes, and about one in three of them are qualified GPs.17 Italian OOH services operate every day of the week from 20:00 to 8:00, at weekends from 10:00 on Saturdays to 8:00 on Mondays, plus bank holidays, and also from 8:00 to 20:00 on days when GPs attend continuing education courses. The OOH physicians receive telephone calls from patients and deliver services that may involve: providing advice over the phone; visiting a patient at home (or in a rest home); examining them at the walk-in clinic (at premises provided by the LHA). However, it is not necessary to book visits to the walk-in clinic, and patients may attend without any previous phone contact. If a patient's condition is judged to be unmanageable in the primary care setting, the patient may be advised to go directly to the ED in their own car. If a patient's condition is judged to be life-threatening, an ambulance may be called instead.

People can access the ED at their own discretion, or they may be referred by a physician. At the ED, they are received by a triage nurse who assigns them a colour code, depending on the severity and urgency of their case. After patients have been examined by a physician, their triage code may be changed.

Setting

The study was conducted at the LHA ‘ULSS 4—Alto Vicentino’, which occupies an area in the north-western part of the Veneto Region and serves a population of about 190 000 with a mean density of 111 inhabitants per square mile (290/km2). In 2011, foreign residents accounted for ∼10.3% of the total population (about 2% more than the national average). This LHA has three OOH service points with a total of 25 physicians working on a rota system for 24 hours a week. During service periods, patients have to phone a single call centre that records their personal details using an electronic call management system before they can talk to a doctor. In 2006, the LHA ‘ULSS 4’ implemented an information technology system that enables OOH doctors to consult a patient's personal health records held by their GPs or the local hospital.

Participants and materials

This study was based on data recorded from 1 January to 31 December 2011 in the LHA 4 electronic database. A contact to request OOH services (as a statistical unit) was defined as any walk-in patient visit, home visit, rest home visit or telephone consultation followed by no further contact in the 12 hours thereafter (so OOH contacts were recorded as telephone consultations only if the physician provided advice over the phone and did not see the patient afterwards either at the walk-in clinic or at the patient's home or rest home). The computer database of OOH contacts acquires patients' demographic details (sex, age, nationality and place of residence) from the LHA's administrative archives, so Google Maps could be used to calculate the distance of their home from the nearest OOH service point. The database also captures further information from the LHA archives concerning the primary care services available for a given patient's condition, for example, non-cancer integrated home care, cancer-related integrated home care, palliative home care and nursing home care. The electronic database also records logistic aspects such as: date of contact, classified for analytical purposes as working or non-working days (the latter including Saturdays and days before public holidays); time of contact, classified as daytime (8:00 to 20:00) or night-time (20:00 to 8:00). Patients' diagnoses were recorded by means of major diagnostic categories,[i] some of which were collapsed for the purposes of the present study, and the recording system also created another six new categories (state of health certification, death certification, renewal of medical prescriptions, information on drugs, fever and ‘others not specified’). For the purposes of this study, OOH contacts concerning death certification, medical prescriptions or state of health certificates were disregarded because these conditions could not be associated with a potential patient referral to an ED. All contacts made by non-residents were ignored too. The database also records the possible OOH outputs for a given contact, classified as patients returning home or remaining at home (if the contact involved the patient being seen at the walk-in clinic, at home or at a rest home), referral to an ED or telephone counselling alone. The database is also linked with some sociodemographic details of the OOH physicians handling the contacts (such as years since they graduated, gender, type of employment contract (temporary substitute or permanent postholder)).

Another database at the ED records all cases seen at the ED after contacting the OOH services. This ED database was record-linked with the OOH database using a unique identifier code for each patient. The linkage concerned only ED attendances within 24 hours after contacting the OOH service. In addition to demographic and logistic variables, the ED database also records other important information: the incoming triage code assigned by nurses and the outgoing triage code assigned by physicians (white=non-critical patients who should receive primary care; green=not life-threatening conditions; yellow=critical patients at risk of their clinical condition deteriorating; red=very critical patient needing immediate treatment); any activation of a short-stay observation period; and discharge from the ED (classified as patient hospitalised, arrived dead, sent home, sent to another institute, died at the ED, refused hospitalisation, left the ED before being examined or referred for outpatient care).

Outcome definition

A case seen at the ED was considered urgent and potentially appropriate if it met at least one of the following conditions (as recently defined by the Italian Agency for Regional Health Services, AGENAS):18

Red or yellow outgoing triage code (if no outgoing code was assigned, red or yellow incoming triage code);

Patient hospitalised or refused recommended hospitalisation;

Patients assessed for trauma;

Short-stay observation was activated;

Arrived dead or died at the ED.

Patients who contacted the OOH services were divided into two groups according to whether or not they were referred to the ED. Both groups included patients who actually went to the ED within 24 hours after contacting the OOH and others who did not.

Among those not referred to the ED (first group), patients were considered as cases of ‘potentially inappropriate non-referral’ if they went spontaneously to the ED within 24 hours after contacting the OOH services and were found to meet at least one of the criteria for appropriate access to the ED.

Among those referred to the ED (second group), patients were considered as cases of ‘potentially inappropriate referral’ if they did not go to the ED within 24 hours, or if they did go to the ED, but it was found that they did not meet at least one of the criteria for appropriate access to the ED.

Statistical methods

Statistical analyses were performed using the STATA V.12 software.

The data were summarised as numbers (percentages) of patients for categorical variables. Bivariate analyses were run to assess differences in counts for categorical variables using the χ2 test (or Fisher's test when <5 cases were expected).

Two logistic regression models were applied, one for each dependent variable:

Potentially inappropriate non-referrals to the ED by the OOH service (for this regression, Firth's penalised likelihood approach was applied to correct for rare events);

Potentially inappropriate referrals to the ED by the OOH service.

The two regressions tested the sociodemographic, process/context and clinical variables associated with the dependent variables.

A p value of <0.01 was considered significant, to take multiple comparisons into account.

Ethical considerations

The data analysis was conducted on anonymised aggregated data with no chance of individuals being identifiable. The study complied with the Declaration of Helsinki and with Italian Law n. 196/2003 on the protection of personal data. The recent resolution n. 85/2012 of the Italian Guarantor for the Protection of Personal Data also confirmed the allowability of processing personal data for medical, biomedical and epidemiological research, and that data concerning health status may be used in aggregated form in scientific studies. Permission to use non-identifiable, individual data extracted from administrative databases was granted by the ULSS 4 Veneto Region, which is responsible for any use of the data concerning the population it serves.

Results

From 1 January to 31 December 2011, the LHA 4 OOH service was contacted by 23 980 people (see figure 1).

Figure 1

Distribution of patients contacting the OOH service. ED, emergency department; OOH,out-of-hours.

Around 9% of all patients contacting the OOH service were referred to the ED. Table 1 shows the sample's characteristics by referral group. The percentage of patients referred to the ED increased with patient’s age, and was higher at night (13.4% from 20:00 to 8:00) than during the day (8.4% from 8:00 to 20:00). Patients requiring nursing care, physiotherapy or GP home care were more likely to be referred to the ED after contacting the OOH service (the proportions were 20.66%, 19.12% and 19.83%, respectively). Cardiovascular disease (28.3%) and trauma (23.4%) were the diagnostic categories of patients most likely to be referred to the ED.

Table 1

Characteristics of the sample contacting the OOH service by patients' referral or non-referral to the ED

		Not referredn=20 596 (%)	Referredn=2066 (%)
Patients' demographic variables
Sex
Male	10 450	90.6	9.4
Female	12 212	91.1	8.9
Age (years)
0–1	1254	96.0	4.0
2–14	4984	95.0	4.0
15–18	624	94.6	5.4
19–44	6245	93.6	6.4
45–64	3927	90.1	9.9
65–74	1867	86.5	13.5
75–84	2067	81.7	18.3
>84	1694	79.6	20.4
Paediatric cases (0–14 years)
No	16 424	89.0	10.0
Yes	6238	95.8	4.2
Nationality
Italian	20 099	90.5	9.5
Foreign	2560	94.2	5.8
Patients' process and context variables
Distance from nearest OOH
0–5 km	11 214	90.2	9.8
>5 km	10 322	91.6	8.4
Time of day
8–20	19 366	91.6	8.4
20–8	3296	86.6	13.4
Type of day
Working day	7368	89.9	10.1
Holiday	8285	91.4	8.6
Day before a holiday	6799	91.2	8.8
Type of OOH intervention
Phone advice	9970	92.5	7.5
Home visit	2134	76.5	23.5
Walk-in clinic visit	10 558	92.3	7.7
OOH physician
Temporary substitute	17 169	90.5	9.5
Postholder	5493	92.0	8.0
Gender of OOH physician
Male	14 489	89.8	10.2
Female	8173	92.9	7.1
Patients' clinical variables
Death within a year
No	20 846	92.0	8.0
Yes	1816	78.6	21.4
Home physiotherapy
No	22 594	90.9	9.1
Yes	68	80.9	19.1
Home nursing services
No	20 973	91.8	8.2
Yes	1689	79.3	20.7
No	21 976	91.2	8.8
Assisted at home by GP
Yes	686	80.2	19.8
Disease
Dermatological	1501	97.3	2.7
Musculoskeletal	1453	91.4	8.6
Ophthalmological	532	85.0	15.0
Odontostomatological	634	97.8	2.2
Oncological and haematological	143	79.7	20.3
Ear-nose-throat	2744	96.6	3.4
Pregnancy and reproductive system	194	84.0	16.0
Psychiatric	381	89.0	11.0
Respiratory	1734	86.5	13.6
Trauma	752	76.6	23.4
Nervous system	853	77.5	22.5
Infectious	389	97.2	2.8
Genitourinary system	1092	89.5	10.5
Gastroenterological	3148	89.4	10.6
Fever	2835	97.9	2.1
Endocrinological	127	85.8	14.2
Advice about ongoing therapy	1969	99.3	0.7
Cardiovascular	1348	71.7	28.3
Not specified	833	91.2	8.8

ED, emergency department; GP, general practitioner; OOH, out-of-hours.

The total number of cases potentially inappropriately managed by the OOH service, in terms of referrals and non-referrals to the ED, amounted to 1599 (about 7% of the sample): 392 were cases of potentially inappropriate non-referral to the ED (1.7%); and 1207 were cases of potentially inappropriate referral to the ED (5.3%); this latter group includes 422 patients who did not go to the ED despite being referred there (20.4% of all those referred) and 785 who did go to the ED but who did not meet any of the criteria for potentially appropriate access (3.8% of all those referred to the ED).

Table 2 shows the demographic, process/context and clinical characteristics of the patients contacting the OOH services by referral to the ED and its appropriateness. For example, the percentage of patients inappropriately referred to the ED was higher for infants aged up to 1 year (reaching 87.1%) and decreased with age (35.1% in patients aged >84 years); conversely, the percentage of inappropriate non-referrals increased with age, reaching 5.3% for patients aged >84 years.

Table 2

Bivariate analysis of characteristics of patients and their potentially inappropriate management by referral and non-referral to the ED

	Not referredn. 20 596			Referredn. 2066
	Appropriate n=20 204	Inappropriate n=392	p Value	Appropriate n=859	Inappropriate n=1207	p Value
	%	%		%	%
Patients’ demographic variables
Sex
Male	98.0	2.0	0.268	40.8	59.2	0.468
Female	98.2	1.8		42.3	57.7
Age (years)
0–1	98.6	1.4	<0.001	12.9	87.1	<0.001
2–14	98.9	1.1		25.9	74.1
15–18	98.8	1.2		26.5	73.5
19–44	99.2	0.8		25.1	74.9
45–64	98.3	1.7		34.6	65.4
65–74	96.4	3.6		51.4	48.6
75–84	95.4	4.6		52.9	47.1
>84	94.7	5.3		64.9	35.1
Paediatric cases (0–14 years)
No	97.8	2.2	<0.001	44.3	55.7	<0.001
Yes	98.8	1.2		22.8	77.2
Nationality
Italian	98.0	2.0	0.003	42.9	57.1	<0.001
Foreign	98.9	1.1		24.1	75.9
Patients' process and context variables
Distance from nearest OOH
0–5 km	98.3	1.7	0.149	41.1	58.9	0.635
>5 km	98.0	2.0		42.1	57.9
Time of day
8–20	98.5	1.5	<0.001	41.3	58.7	0.613
20–8	95.8	4.2		42.6	57.4
Type of day
Working day	98.0	2.0	0.823	40.9	59.1	0.431
Holiday	98.1	1.9		43.4	56.6
Day before a holiday	98.1	1.9		40.0	60.0
Type of OOH intervention
Phone advice	97.6	2.4	<0.001	41.4	58.6	<0.001
Home visit	94.5	5.5		67.3	32.7
Walk-in clinic visit	99.2	0.8		25.9	74.1
OOH physician
Temporary substitute	98.2	1.8	0.198	40.0	60.0	0.004
Postholder	97.9	2.1		47.5	52.5
Gender of OOH physician
Male	98.2	1.8	0.425	41.9	58.1	0.592
Female	98.0	2		40.7	59.3
Patients’ clinical variables
Death within a year
No	98.5	1.5	<0.001	35.2	64.8	<0.001
Yes	93.3	6.7		69.2	30.8
Home physiotherapy
No	98.1	1.9	0.301	41.6	58.4	0.819
Yes	100	0		38.5	61.5
Home nursing services
No	98.4	1.6	<0.001	37.2	62.8	<0.001
Yes	94.0	6.0		63.0	40.0
Assisted at home by GP
No	98.2	1.8	<0.001	40.0	60.0	<0.001
Yes	94.0	6		64.7	35.3
Disease
Dermatological	99.9	0.1	<0.001	7.5	92.5	<0.001
Musculoskeletal	98.5	1.5		22.4	77.6
Ophthalmological	99.8	0.2		1.2	98.8
Odontostomatological	100	0		7.1	92.9
Oncological and haematological	93.9	6.1		65.5	34.5
Ear-nose-throat	99.6	0.4		1.1	98.9
Pregnancy and reproductive system	96.9	3.1		32.3	67.7
Psychiatric	96.5	3.5		38.1	61.9
Respiratory	97.6	2.4		52.3	47.7
Trauma	98.6	1.4		100	0
Nervous system	96.4	3.6		43.2	56.8
Infectious	100	0		18.2	81.8
Genitourinary system	97.4	2.6		32.2	67.8
Gastroenterological	96.2	3.8		36.7	63.3
Fever	98.3	1.7		26.2	73.8
Endocrinological	96.3	3.7		44.4	55.6
Advice about ongoing therapy	98.8	1.2		28.6	71.4
Cardiovascular	96.5	3.5		47.8	52.2
Not specified	96.6	3.4		35.6	64.4

ED, emergency department; GP, general practitioner; OOH, out-of-hours.

Table 3A, B shows the results of two logistic regression models. In particular, table 3A shows the measure of association between the demographic, process/context and clinical variables with potentially inappropriate non-referrals to the ED. The regression confirmed that potentially inappropriate non-referrals increased with age, revealing a twofold higher OR for those aged 65–74 and 75–84 years than for the newborn. Females appeared to have significantly lower odds of being inappropriately not referred to the ED. Foreign people were at higher risk of being inappropriately not referred to the ED than Italians (OR 1.62), though the difference did not reach statistical significance (p=0.03). Contacting the OOH service at night also coincided with a twofold higher odds of potentially inappropriate referral than for daytime contacts. A number of diagnostic categories were associated with higher odds of potentially inappropriate non-referral than for the diagnostic category taken for reference (dermatological). Contacts involving patients attending the walk-in clinic were only half as likely as patients managed by means of telephone consultations alone to be cases of potentially inappropriate non-referral to the ED.

Table 3

Results of multivariate logistic regression for associations between patients’ characteristics and their potentially inappropriate management—for cases not referred to the ED

OR	95% CI		p Value
(A) Potentially inappropriate non-referral to the ED
Sex (ref. male)
Female	0.75	0.61 to 0.92	0.007
Age (ref. 0–1 years)
2–14	0.99	0.57 to 1.69	0.939
15–18	1.17	0.49 to 2.79	0.731
19–44	0.68	0.39 to 1.20	0.187
45–64	1.26	0.72 to 2.20	0.421
65–74	2.21	1.25 to 3.92	0.007
75–84	2.14	1.21 to 3.80	0.009
>84	2.03	1.11 to 3.69	0.021
Nationality (ref. Italian)
Foreign	1.62	1.04 to 2.47	0.029
Distance from nearest OOH (ref. 0–5 km)
>5 km	1.19	0.97 to 1.47	0.103
Time of day (ref. 8–20)
20–8	2.30	1.80 to 2.93	<0.001
Type of day (ref. working day)
Holiday	1.27	0.98 to 1.64	0.073
Day before a holiday	1.42	1.08 to 1.86	0.012
Home nursing services (ref. no)
Yes	1.21	0.86 to 1.71	0.269
Assisted at home by GP (ref. no)
Yes	1.05	0.68 to 1.62	0.834
Death within a year (ref. no)
Yes	1.65	1.19 to 2.30	0.003
OOH physician (ref. temporary substitute)
Postholder	0.93	0.66 to 1.30	0.651
Gender of OOH physician (ref. male)
Female	1.27	1.02 to 1.58	0.030
Disease (ref. dermatological)
Musculoskeletal	5.99	1.59 to 22.41	0.008
Ophthalmological	1.79	0.26 to 13.65	0.573
Oncological and haematological	8.87	2.02 to 38.59	0.004
Ear-nose-throat	2.60	0.65 to 10.35	0.176
Pregnancy and reproductive system	24.63	5.38 to 112.7	<0.001
Psychiatric	8.26	2.06 to 33.05	0.003
Respiratory	7.63	2.09 to 27.86	0.002
Trauma	7.14	1.73 to 29.46	0.007
Nervous system	11.02	2.96 to 41.05	<0.001
Genitourinary system	10.10	2.73 to 37.35	0.001
Gastroenterological	13.76	3.90 to 48.60	<0.001
Fever	6.82	1.90 to 24.56	0.003
Endocrinological	7.96	1.64 to 38.50	0.010
Advice about ongoing therapy	3.49	0.94 to 13.03	0.063
Cardiovascular	8.70	2.38 to 31.83	0.001
Not specified	10.90	2.95 to 40.38	<0.001
Type of OOH intervention (ref. phone)
Home visit	1.10	0.82 to 1.47	0.543
Walk-in clinic visit	0.49	0.36 to 0.66	<0.001
(B) Potentially inappropriate referral to the ED
Sex (ref. male)
Female	1.23	0.98 to 1.54	0.076
Age (ref. 0–1 years)
2–14	0.60	0.21 to 1.75	0.349
15–18	0.21	0.06 to 0.77	0.019
19–44	0.27	0.10 to 0.74	0.010
45–64	0.17	0.06 to 0.45	<0.001
65–74	0.12	0.04 to 0.33	<0.001
75–84	0.15	0.05 to 0.39	<0.001
>84	0.15	0.05 to 0.41	<0.001
Nationality (ref. Italian)
Foreign	0.88	0.52 to 1.48	0.633
Distance from nearest OOH (ref. 0–5 km)
>5 km	0.86	0.69 to 1.08	0.202
Time of day (ref. 8–20)
20–8	1.00	0.76 to 1.33	0.966
Type of day (ref. working day)
Holiday	1.11	0.85 to 1.46	0.431
Day before a holiday	1.27	1.95 to 1.70	0.114
Home physiotherapy (ref. no)
Yes	3.05	0.88 to 10.55	0.079
Home nursing services (ref. no)
Yes	1.04	0.74 to 1.45	0.824
Assisted at home by GP (ref. no)
Yes	0.95	0.60 to 1.49	0.812
Death within a year (ref. no)
Yes	0.46	0.33 to 0.64	<0.001
OOH physician (ref. temporary substitute)
Postholder	0.64	0.47 to 0.89	0.007
Gender of OOH physician (ref. male)
Female	1.12	0.87 to 1.44	0.392
Disease (ref. dermatological)
Musculoskeletal	0.48	0.13 to 1.72	0.258
Ophthalmological	7.16	0.71 to 72.26	0.095
Odontostomatological	1.08	0.10 to 11.90	0.949
Oncological and haematological	0.16	0.04 to 0.72	0.017
Ear-nose-throat	8.21	0.81 to 82.92	0.074
Pregnancy and reproductive system	0.17	0.04 to 0.74	0.018
Psychiatric	0.41	0.10 to 1.66	0.211
Respiratory	0.23	0.07 to 0.80	0.021
Nervous system	0.26	0.08 to 0.92	0.036
Infectious	0.56	0.07 to 4.38	0.583
Genitourinary system	0.26	0.07 to 0.92	0.037
Gastroenterological	0.22	0.06 to 0.76	0.016
Fever	0.27	0.07 to 1.06	0.061
Endocrinological	0.26	0.05 to 1.25	0.093
Advice about ongoing therapy	0.40	0.07 to 2.31	0.305
Cardiovascular	0.22	0.06 to 0.75	0.016
Not specified	0.38	0.10 to 1.42	0.148
Type of intervention (ref. phone)
Home visit	0.43	0.32 to 0.59	<0.001
Walk-in clinic visit	1.33	1.00 to 1.79	0.052

ED, emergency department; GP, general practitioner; OOH, out-of-hours.

Table 3B shows the measure of the association between the variables and inappropriate referrals to the ED. Unlike the first regression, the odds of potentially inappropriate referral to the ED dropped with increasing age (as compared with the newborn). There were no significant differences with regard to contacts' nationality or gender. Patients who died within a year of contacting the OOH service had lower odds of being referred inappropriately to the ED. Also, the contacts managed by a postholder with respect to those managed by a temporary holder had lower odds of being referred inappropriately to the ED. None of the diagnostic categories changed the likelihood of potentially inappropriate referral vis-à-vis the reference category. Contacts handled by means of a home visit were only half as likely as those managed by means of a telephone consultation alone to be cases of potentially inappropriate referral to the ED.

Discussion

This study addressed the phenomenon of potentially inappropriate patient referral and non-referral to the ED by OOH physicians. The study revealed that some determinants of potentially inappropriate non-referral to the ED mirror those of potentially inappropriate referral, for example, older age reduces the probability of inappropriate referral and increases the risk of inappropriate non-referral.

Frequency of potentially inappropriate handling of OOH contacts

Only one in eight patients went to the ED after contacting the OOH service, and <1 in 20 who contacted the OOH subsequently went to the ED at their own discretion and failed to meet the criteria for urgent medical care. This confirms the gatekeeping role of OOH physicians and underlines the importance of this service in ensuring a continuity of primary healthcare and preventing patients from going to the ED instead of a primary care service. The unwarranted use of secondary healthcare services gives rise to a lack of continuity of care and generates an overload that adversely affects ED activities.19 A previous systematic review, however, found that action taken to increase OOH primary care services did not generate any reduction in ED attendance (though the studies considered in this review received low global quality ratings and various different models of OOH primary medical care service had been considered).20 Another previous study21 also showed that decisions to refer patients to ED were associated with the patient's clinical and process/context characteristics, as well as with the particular physician involved: there were statically significant differences in the adjusted odds of referral to EDs between physicians working at the same OOH services, meaning that each doctor's attitude had an important influence on their referral rates.

This study found that <10% of cases (7%) were handled by the OOH services in a potentially inappropriate manner. As regards potentially inappropriate ED referrals, we found that <1 in 20 of all patients contacting the OOH services (5.3%) were referred to the ED inappropriately. In theory, OOH physicians are contacted when patients have borderline health conditions that are not urgent enough for them to go to the ED, but too severe for them to wait until the next working day. This delicate borderline situation should be borne in mind when we consider the above percentages. According to an English study, the percentage of cases inappropriately managed by means of telephone triage services delivered by the NHS-24 service is around 35%, judging from GPs' subsequent reassessment.22 An Australian study instead reported that about 10% of patient referrals to the ED by GPs were inappropriate.23 It is rather difficult to draw comparisons between studies conducted on different healthcare systems and using different methods to assess ‘appropriateness’. These figures are in any case considerably lower than the percentage of inappropriateness identified for self-referred patients, which reaches as high as 80%,24 confirming once again the important gatekeeping role of the OOH services in relation to low-acuity patients.

Potentially inappropriate non-referral to the ED

Of course, inappropriate non-referral to the ED is a more serious issue because a diagnostic delay in a genuinely urgent patient may even be fatal. Our study found this situation fairly infrequent (1.7%). To the best of our knowledge, no other published studies have investigated the frequency of potentially inappropriate non-referrals to the ED with which we might draw a comparison. We might nonetheless argue that the low rate of inappropriate non-referral to the ED identified in this study could partly be due thanks to the IT system that enables OOH physicians to consult patients' electronic medical records and check their medical history as soon as they make contact. Generally speaking, communication breakdown is a major contributor to diagnostic errors and an increasingly recognised preventable factor in medical mishaps. Using new technologies to enhance communication between health providers and health systems could therefore facilitate the consultation of patients' medical records,25 though a literature review found that improper use of such technologies can give rise to errors in the electronic hospital records, which can in turn lead to errors that endanger patients' safety or negatively affect the quality of their care.26

With a view to enhancing the quality of OOH services, it would be wise to address the determinants of OOH physicians' potentially inappropriate management of the patients who contact them. The odds of inappropriate ED non-referral were found to increase with patients' age, atypical disease presentations, polypharmacy and multiple comorbidities, which may complicate patient management.27 Cognitive impairment, which is more common among the elderly, could also negatively influence the likelihood of appropriate patient management. Cognitively impaired older patients can have trouble remembering things, become confused, and have attention deficits or difficulty expressing themselves.28 An analysis of error reports submitted to the Applied Strategies for Improving Patient Safety (ASIPS) indicated that communication problems represented the most common error process in the ambulatory care setting.29 Failure to ensure complete communication between healthcare providers and patients was also associated with a higher risk of clinical harm. The apparently higher likelihood of OOH services inappropriately not referring elderly people to the ED (especially those patients who then go to the ED anyway) may relate, however, to ED physicians having a greater propensity to admit older patients to hospital as a prudential choice.30

Our study revealed that sociodemographic and clinical conditions, as well as logistic and organisational variables, may be determinants of potentially inappropriate non-referral to the ED. In particular, we found that patients attending the OOH walk-in clinic were less likely to be managed inappropriately than those only making telephone contact. Another study reported that telephone consultations were shorter and less detailed than face-to-face visits for similar clinical problems, increasing the risk of diagnostic or management errors.31 The phenomenon of inappropriate non-referral to the ED for patients managed by the OOH services over the phone could be contained by adopting well-structured procedures based on checklists and protocols, or more advanced solutions such as decision-supporting software packages, to help doctors consulted over the phone provide comprehensive advice.32 33 An approach applying root cause analysis should be adopted to identify proactive ways to address patient safety incidents with a view to reducing the risk of error, rather than more passive methods that encourage staff to be more vigilant in their working practice,34 or to take more care when prescribing medication, for example.

Potentially inappropriate referral to ED

Older children and adults had lower odds of inappropriate referral to the ED than infants. A cross-sectional examination of ED attendance in England found age to be strongly related to inappropriate referral.35 The odds were highest for the very young (peaking for the 1 and 2 years old), and were also high between the mid-teens and mid-20s, followed by a steady drop with ageing thereafter. The sizeable presence of young people inappropriately attending EDs around the world has been recognised in various studies, as emphasised in the review conducted by Carret et al.12 These findings suggest that action to prevent inappropriate management should target early childhood and young adolescence. The phenomenon probably reflects the pressures of parenthood,36 37 or a lack of confidence with children's diseases. Given the difficulty of managing paediatric patients, and especially infants, paediatric OOH services need to be expanded, or training courses could be organised to improve OOH physicians' understanding of paediatric emergencies.

Our analysis showed that, among the OOH physicians, regular postholders were less likely to inappropriately refer patients to the ED than their temporary substitutes. This may be an indication of the former's greater work experience, but could also relate partly to the fact that some people contacting the OOH services are frequent attenders, and probably well known to the permanent staff, who consequently find it easier to manage their disorders.38 Using the available data on OOH physicians (also taking each OOH physician's personal identification code into account) might be useful for the purpose of giving physicians regular feedback on their performance, which could be compared with a benchmark, for instance. It has been demonstrated in the literature that this could have a positive impact on their performance.39

Limits and strengths

Our study suffers from several limitations. One stems from our arbitrary classification of potentially inappropriate referrals to the ED (based on a national definition of appropriateness) that clearly influences the reported prevalence of potentially inappropriate patient management in our sample. There is no general consensus on how to define appropriateness of ED use in the literature. In addition, contacts at OOH services and at the ED happened at a different time, by personnel equipped with different diagnostic features. Another limitation is in that our analysis was conducted only on the LHA 4 records, so we were unable to follow-up patients admitted to an ED outside the territory it serves. The scientific literature suggests, however, that patients usually go to their nearest ED, in their own district.40 The main strength of our research, on the other hand, is that this was a population-based study conducted not on a limited or selected sample of patients, but using a register of routinely collected data, so our findings could not be biased by any research hypothesis. It goes to show that administrative data can be used to monitor this phenomenon and its determinants. Another recent study described a new database infrastructure (iCAREdata) linking data from general practice cooperatives (GPCs), EDs and pharmacies during OOH care. These data, with the same vision of this study, could be used for feedback reports for individual GPCs or EDs, benchmarking and giving the opportunity to optimise the quality, safety and the organisation of OOH care Moreover, it could be used to define a programme of continuing professional education for OOH doctors more focused on their real needs.41

Conclusion

This work paints a comprehensive picture of the predictors of potentially inappropriate behaviour on the part of OOH physicians in terms of their referral or non-referral of patients to the ED. The novelty of our work in fact will require that further studies addressing the factors associated with potentially inappropriate referral and non-referral of patients to the ED are needed to confirm these results.