Towards an optimal composition of general practitioners and nurse practitioners in out-of-hours primary care teams: a quasi-experimental study.

OBJECTIVES: To gain insights into the ability of general practitioners (GPs) and nurse practitioners (NPs) to meet patient demands in out-of-hours primary care by comparing the outcomes of teams with different ratios of practitioners.
DESIGN: Quasi-experimental study.
SETTING: A GP cooperative (GPC) in the Netherlands. INTERVENTION: Team 2 (1 NP, 3 GPs) and team 3 (2 NPs, 2 GPs) were compared with team 1 (4 GPs). Each team covered 35 weekend days. PARTICIPANTS: All 9503 patients who were scheduled for a consultation at the GPC through a nurse triage system. OUTCOME MEASURES: The primary outcome was the total number of consultations per provider for weekend cover between 10:00 and 18:00 hours. Secondary outcomes concerned the numbers of patients outside the NPs' scope of practice, patient safety, resource use, direct healthcare costs and GPs' performance.
RESULTS: The mean number of consultations per shift was lower in teams with NPs (team 1: 93.9, team 3: 87.1; p<0.001). The mean proportion of patients outside NPs' scope of practice per hour was 9.0% (SD 6.7), and the highest value in any hour was 40%. The proportion of patients who did not receive treatment within the targeted time period was higher in teams with NPs (team 2, 5.2%; team 3, 8.3%) compared with GPs only (team 1 3.5%) (p<0.01). Team 3 referred more patients to the emergency department (14.7%) compared with team 1 (12.0%; p=0.028). In teams with NPs, GPs more often treated urgent patients (team 1: 13.2%, team 2: 16.3%, team 3: 21.4%; p<0.01) and patients with digestive complaints (team 1: 11.1%, team 2: 11.8%, team 3: 16.7%; p<0.01).
CONCLUSIONS: Primary healthcare teams with a ratio of up to two GPs and two NPs provided sufficient capacity to provide care to all patients during weekend cover. Areas of concern are the number of consultations, delay in patient care and referrals to the emergency department. TRIAL REGISTRATION: NCT02407847. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This is the first comparative evaluation of teams with nurse practitioners and general practitioners during out-of-hours primary care practice.

The study has a large representative patient sample and a long follow-up period, although undertaken in only one centre. Health outcomes were not measured.

The use of a cost-minimisation analysis provides limited insight into the costs.

No change in patient allocation gives an accurate representation of daily practice and peak hours.

Introduction

The quality of out-of-hours primary care influences the functioning of the whole healthcare system. Internationally, different organisational models are used to deliver urgent care during out-of-hours practice. Patients in the UK have access to services such as walk-in centres, urgent care centres, out-of-hours centres, telephone consultations and emergency departments (EDs), which often operate side by side.1 However, these services show varying results in terms of patient outcomes and efficiency.2–4 In the Netherlands, out-of-hours care is organised in general practitioner cooperatives (GPCs).5 Although these large GP-based models show positive results,3 current and expected problems, such as population ageing, the increased prevalence of chronic conditions and the shifting of tasks from hospitals to the community, put pressure on (out-of-hours) primary care.6 7 The challenge for policymakers is to find a model that ensures accessibility, quality and efficiency in out-of-hours care.1 8

As many complaints during out-of-hours care do not necessarily require the knowledge and skills of a GP, there is increasing interest in care delivery models that include nurse practitioners (NPs) in primary care teams.9–11 Systematic reviews of published research have shown that NPs in daytime primary care provide good-quality and safe care to patients, but not necessarily more efficient care compared with GPs.12–14 There are models in which care is provided by teams with only NPs, but such services are not able to provide high-quality care to some patients due to a lack of capacity, resources or skill levels. In light of the above, team-based care involving both GPs and NPs is an alternative model for delivering out-of-hours care.

Current evidence does not provide insights into the optimal ratio of GPs and NPs in out-of-hours teams. The results for NPs in daytime primary care cannot simply be translated to out-of-hours care. Organisations differ in size, the incidence of life-threatening conditions is higher in out-of-hours settings and care outside office hours is unpredictable in terms of patient flow. The acute nature of complaints limits the potential for forward scheduling and the main complaint after triage does not always correspond to the main complaint evaluated during consultation.15 16 Second, while overall patient care is determined by the sum of its parts, most studies compare care between healthcare providers rather than comparing teams.17 To the best of our knowledge, this is the first randomised comparative study to provide insights into the optimal composition of GPs and NPs in primary care teams during out-of-hours provision.

Aim

The aim of the study was to compare teams with different ratios of GPs and NPs in terms of the number of consultations, patient care and GPs’ performance and provide insights into the number of patients outside the NPs’ scope of practice in out-of-hours primary care.

Methods

Design

A quasi-experimental study was conducted to measure the total number of patients and the distribution of patients outside NPs’ scope of practice in out-of-hours primary care over the weekend (Saturday and Sunday) between 10:00 and 18:00 hours. Two types of teams with NPs were compared with a team comprising only GPs, as follows:

Team 1: care provided by a team of four GPs (care as usual);

Team 2: care provided by a team of three GPs and one NP;

Team 3: care provided by a team of two GPs and two NPs.

Study setting

The study was conducted at a GPC situated within a hospital next to the ED in the south-east of the Netherlands. In this GPC, GPs work in shifts from 17:00 to 08:00 hours on weekdays and over the entire weekend, taking care of a population of approximately 3 04 000 people. All patients in need of acute care outside regular office hours contact the GPC using a single, regional telephone number. Triage nurses then allocate patients to an appropriate care pathway based on risk stratification. Patients who are eligible for a consultation at the GPC are scheduled on a common presentation list, depending on the urgency of the complaints based on the Netherlands Triage Standard (NTS; see table 1). A maximum of five patients are scheduled every hour per healthcare provider. GPs and NPs select attending patients from this presentation list.18

Table 1

The Netherlands Triage Standard urgency levels42

Urgency level	Description	Time period for consultation
U1	Life threatening: Immediate action required, the vital functions are threatened or delaying treatment will cause serious and irreparable damage to the patient’s health	Within 15 min
U2	Emergent: Vital functions are not (yet) in danger, but there is a fair change that the patient’s condition will soon deteriorate or delaying treatment will cause serious and irreparable damage to the patient’s health. Take action as soon as possible	Within 1 hour
U3	Urgent: Do not postpone too long. Treat within a few hours because of medical or humane reasons	Within 3 hours
U4	Non-urgent: There is no pressure resulting from medical or other grounds. Time and place of treatment should be discussed with the patient	No time pressure
U5	Advice: A physical examination can wait until the next day	No time pressure

Study population

General practitioners

All GPs who delivered patient care during the study period were included. This included both practice owners (n=162) and GPs employed by another GP. Their mean age was 47.5 years (SD 9.7) and 50.3% were male. Those employed by another GP have often recently graduated.

Nurse practitioners

A sample of 10 NPs participated in the study. Their mean age was 45.2 years (SD 9.4) and one was male. On average, they had been qualified as an NP for 1.8 years (SD 1.2) and had worked at the GPC for 1.6 years (SD 1.1). All NPs had completed a 2-year Master’s programme on ‘Advanced Nursing Practice’ (NLQF/EQF level 7). This programme included an academic course on treating common complaints in primary care and an internship in general practice.19 20 During office hours, they took care of patients with minor ailments in general practices and undertook elderly care or care for disabled people. To ensure their competency to work in out-of-hours care, they received three half days of additional training concerning complaints commonly presented during out-of-hours care: eye disorders; musculoskeletal disorders, such as fractures, bruises and sprains; wound care (eg, suturing). NPs in the Netherlands have the authority independently to indicate and perform reserved procedures (including prescribing) in their area of expertise, using the same practice guidelines as GPs.21 22 The numbers of support staff at the GPC (one receptionist and one medical assistant per shift) were equal for the different teams.

Patients

All patients who had a consultation at the GPC during the period of data collection were included in the study. Due to the exploratory nature of the study, no calculation of statistical power could reliably be made. To attain reasonably accurate estimates, a 35-week follow-up period per team was selected to obtain a sufficiently large sample. NPs decided which patients from the common presentation list would be called in for consultation; other patients received a consultation with a GP. In the case that the patient’s complaint during triage was different from that during the consultation, NPs were allowed to decide autonomously whether they felt competent or not to complete the consultation themselves. If not, they could consult a GP about the patient or refer the patient to a GP at the GPC.

Allocation to study arms

The teams were rotated systematically between Saturday and Sunday. The rotation scheme was determined in advance. GPs were randomly assigned to the days over the weekends and they did not know whether they would work with an NP at the time of scheduling.

The scheduling of the patients was done by triage nurses at a call centre, which is in charge of scheduling patients for several GPCs. They were blind to the composition of GPs and NPs in the team, only knowing the total number of team members. As a consequence, patients were not informed of the presence of NPs in the teams when they contacted the call centre.

Measures and data collection

The primary outcome was the number of consultations per team and per healthcare provider. This was measured as the mean number of patients per team per day and per healthcare provider per hour.

In terms of secondary outcomes, we first focused on the number of patients outside NPs’ scope of practice. In the Netherlands, NPs are allowed to enter independently into a treatment relationship in their area of expertise and take independent decisions about the interventions to be executed.22 In addition to this national authority, the GPC has formulated a scope of practice for NPs based on their professional training. All the patients meeting the following criteria were defined by the GPC as being outside NPs’ scope of practice: patients younger than 1 year, or suffering psychiatric complaints, abdominal pain, chest pain, a neck ailment, headache or dizziness (see table 2). All other patients were within NPs’ scope of practice.18 We looked at all patients presenting at the GPC on Saturdays and Sundays between 10:00 and 18:00 hours and measured the percentage and distribution of those patients who were outside NPs’ scope of practice.

Table 2

Patients outside the predefined scope of NP care

Patient characteristics and complaints expressed during triage defined by the GPC as being outside NPs’ scope of practice	Patient characteristics and diagnoses defined as outside NPs’ scope of practice during data analysis
–Patients younger than 1 year	–Age <1 year
–Patients suffering from psychiatric complaints	–ICPC group P Psychological
–Patients suffering abdominal pain	–ICPC group D Digestive o (except ICPC codes: D04 (Rectal/anal pain), D05 (Perianal itching), D19 (Teeth/gum symptom/complaint), D20 (Mouth/tongue/lip symptom/complaint)
–Patients suffering chest pain	–ICPC group K Cardiovascular o (except ICPC codes: K06 (Prominent veins), K07 (Swollen ankles/oedema), K95 (Varicose veins of leg), K96 (Haemorrhoids)
–Patients suffering neck ailment	–ICPC code L01 Neck symptom/complain
–Patients suffering headache or dizziness.	–ICPC group N Neurological o (except ICPC code: N72 (Tetanus)

ICPC, International Classification of Primary Care.

Next, we measured the effect of different team compositions comprising NPs and GPs on: 1) patient care and 2) aspects of GPs’ performance. Patient care included four measures, based on which the different teams were compared: patient safety, resource use following a consultation at the GPC and direct healthcare costs. Patient safety was examined using two measures, the first of which included the number of (near) incidents. In the Netherlands, GPCs are required by law to report (near) incidents to an internal committee for the reporting of patient care incidents. Both patients and providers are able to report (near) incidents. Second, the number of patients who did not receive care within the targeted time period was calculated. At the call centre, triage nurses classify all patients into urgency levels. The NTS defines the time period in which a patient needs treatment (see table 1). Resource use included X-rays, drug prescriptions and referrals to the ED. Imaging tests or laboratory samples other than X-rays could not be ordered by the providers. If such diagnostic tests were necessary, patients were referred to the ED or to their own GP the next day. Next, direct healthcare costs were calculated based on personnel costs (based on the number of consultations per hour and salary) and combining volumes of resource use by unit prices that constitute costs.

The impact on aspects of GPs’ performance was measured by comparing GPs’ patient characteristics and resource use. The characteristics of GPs’ patients included age, urgency level and the International Classification of Primary Care (ICPC) code. In addition to these characteristics, the number of patients outside NPs’ scope of practice treated by GPs in different teams was compared. Finally, the percentage of consultations in which NPs asked for consultation with a GP was measured.

All data were extracted from the electronic medical patient records at the GPC and coded by the providers as part of their routines during the consultations. Data were collected from May 2014 to November 2015.

Statistical analysis

Baseline characteristics

Baseline characteristics were presented as proportions (%) and included potential confounders for the comparison: age (in four categories), urgency (in five categories), gender and type of complaint (indicated as an ICPC code). Differences between team 1 and teams 2 and 3 were tested using a χ² test.

Primary outcome

First, the total number of patients per team was calculated. An independent sample t-test was used to test differences in the number of consultations per shift between team 2 and teams 3 and 1. The mean number of consultations per professional per hour was calculated by dividing the total number of patients per team by the exact number of hours and the number of healthcare providers per team. In addition, we calculated the number of consultations per hour for the GPs and NPs separately.

Percentage and distribution of patients outside NPs’ scope of practice

First, we took the ICPC codes from all patients presenting themselves at the GPC over weekends and identified those patients whose diagnosis fitted the complaints excluded from NP care (see table 2). Descriptive analysis (mean; SD) was used to indicate the percentage of patients outside NPs’ scope of practice.

To gain an insight into the distribution of patients over a day, the total number of patients outside NPs’ scope of practice per hour was divided by the number of patients who could be scheduled per hour (maximum of 5 patients per healthcare provider per hour=20 patients per team per hour).

Comparison of patient care between teams

Patient safety was evaluated through descriptive analysis, used to determine the number of (near) incidents. Differences between teams in terms of the number of patients receiving treatment within the targeted time period were tested using logistic regression analysis for dichotomous outcomes. Estimates were adjusted for ICPC group, age and the proportion of patients with a U2 urgency level per day.

Resource use (ie, X-rays, drug prescriptions and referrals to the ED) was evaluated by analysing differences in volumes between teams. Logistic regression analysis for dichotomous outcomes, corrected for age, gender, urgency level and ICPC group was conducted to compare teams 2 and 3 with team 1.

Direct healthcare costs were examined through an economic evaluation designed as a cost-minimisation analysis, considering only the direct healthcare costs of the consultation.23 Direct costs were calculated for each consultation separately including costs for personnel, X-rays, drug prescriptions and referral to the ED.

Costs for personnel per consultation were calculated by dividing the tariff per hour by the mean number of patients per hour. The tariff per hour for NPs was set at €61.32 based on their salary from the GPC, including social security contributions (approximately 40%) and premium pay (50%). The tariff for GPs was set at €74.66 based on the payment agreements with health insurance companies. GPs in the Netherlands receive a tariff per patient for providing 24/7 care. Based on these tariffs, the total tariffs per hour per team were €298.64 for team 1, €285.30 for team 2 and €271.96 for team 3. To provide a better comparison between GPs’ and NPs’ tariffs, we also calculated a tariff based on the salary for GPs employed by another GP (specified in collective labour agreements). This tariff included social security contributions and premium pay, similar to NPs, and was set at €93.56 per hour. The inclusion of this tariff resulted in total tariffs per hour per team of €374.24 for team 1, €342.00 for team 2 and €309.76 for team 3.

Next, following the guidelines of the Dutch manual for costing, the cost of each referral to the ED was set at €261 and for an X-ray at €52.79.24 As a result of the differences between the minimum and maximum prices for medicine, two separate costs were calculated per drug prescription. All costs were valid for the year 2015.

To provide insights into the cost differences between teams 2 and 3 and team 1, a linear regression model was used, corrected for case mix (ie, age, gender, urgency level, ICPC group). In the primary analysis, the minimum price per medicine and the personnel costs valid for the GPC were used. Deterministic uncertainty was explored through: (i) one-way sensitivity analysis for the costs of drug prescriptions, including the maximum prize per medicine; (ii) one-way sensitivity analysis of personnel costs, including the tariff for GPs employed by another GP. Finally, we applied a bootstrapping procedure (with 1000 replications) to manage the highly skewed costs across patients.

Comparison of aspects of GPs’ performance between teams

To obtain the patient characteristics for those seen by GPs in the different teams, descriptive statistics were used for patients’ age, gender, urgency level and type of complaint (ICPC). Differences between GPs in teams 2 and 3 and team 1 were tested using the χ² test for categorical data. Descriptive analysis was used for the number of patients outside NPs’ scope of practice treated by GPs in different teams.

Resource use (ie, X-rays, drug prescriptions and referrals to the ED) was evaluated by analysing differences in volumes between GPs in different teams. Logistic regression analysis for dichotomous outcomes, corrected for age, gender, urgency level and ICPC group, was conducted to compare resource use by GPs in teams 2 and 3 and that of GPs in team 1.

Consultations between NPs and GPs were examined using the codes reported by NPs in patients’ medical records when they consulted a GP concerning a patient. There were three codes in the case that NPs consulted a GP but completed the patient consultation themselves: (i) consultation with a GP by telephone; (ii) consultation with a GP outside the surgery room; (iii) consultation with a GP in the surgery room. A fourth code was reported when the patient was referred to a GP to complete the patient consultation. Descriptive analysis was used to indicate the percentage of patients for whom NPs requested consultation with a GP.

The outcomes of two-tailed tests were considered statistically significant at an alpha level p<0.05. The statistical analyses, including bootstrapping, were carried out using SPSS software V.22 (SPSS, Chicago, Illinois, USA).

Deviation from the original study protocol

The study protocol (ClinicalTrials.gov ID NCT02407847) described an extra study arm comprising a team with one GP and three NPs. Ethical approval for this study was obtained based on the arms in the study being part of GPCs’ normal routines (CMO-no. 2014–1409). This meant that the teams followed on from each other in consecutive phases and each phase was followed by an evaluation. The final decision to continue with the last phase, incorporating the team with more NPs, was in the hands of the GPC management. Because the GPC decided not to continue, data on the team with one GP and three NPs as described in the protocol could not be compared with data from the other teams and are therefore not part of this paper.

Results

All patients presenting during the study period were included in the analyses (see figure 1). There were no significant differences in terms of age, gender or ICPC group between the teams (see table 3). In all teams, the top four of ICPC codes covered more than two-thirds of all patients and included skin (21%), musculoskeletal (21%), respiratory (14%) and digestive (11%) complaints. In comparison to team 1, team 2 treated slightly more patients with an urgency level of U2 (14.3% vs 13.2%) and fewer patients with an urgency level of U3 (47.5% vs 51.8%) (p=0.01).

Figure 1

Flow diagram of the study. GP, general practitioner; ICPC, International Classification of Primary Care; NP, nurse practitioner.

Table 3

Baseline patient characteristics

	Team 1 (control; 4 GPs)	Team 2 (3 GPs and 1 NP)	Team 3 (2 GPs and 2 NPs)	GPs team 1	GPs team 2	GPs team 3
Age in categories (%)						‡
0–1 year	4.1	3.5	4.6	4.1	3.9	7
2–17 years	22.6	24.3	25.8	22.6	22.4	22.4
18–64 years	57.5	56.2	54.3	57.5	56.7	54.2
65 years and older	15.8	16	15.3	15.8	16.9	16.5
Gender (% male)	46.7	46.9	47.9	46.7	46.2	46.6
Urgency (%)		*			†	‡
U1	0.1	0.1	0.1	0.1	0.1	0.1
U2	13.2	14.3	15.3	13.2	16.3	21.4
U3	51.8	47.5	50.5	51.8	46.9	47.8
U4	31.0	33.8	30.6	31	32.4	27.4
U5	3.9	4.3	3.4	3.9	4.3	3.3
Complaints top 10 (%)					†	‡
Skin	22.0	21.8	19.3	22	18.7	13.6
Musculoskeletal	20.6	21.7	22.0	20.6	19	18.5
Respiratory	14.3	13.0	15.8	14.3	12.6	16
Digestive	11.1	9.6	11.0	11.1	11.8	16.7
General and unspecified	7.3	7.4	7.4	7.3	8.1	7.9
Eye	5.9	6.0	6.1	5.9	6.4	5.3
Urological	5.6	7.1	5.2	5.6	7.9	5.6
Ear	4.7	4.3	4.7	4.7	4.2	3.7
Neurological	2.9	2.8	2.6	2.9	3.6	3.9
Cardiovascular	1.9	1.9	1.8	1.9	2.5	2.7
Other	3.8	4.3	4.1	3.8	5.1	5.9

Tested using a χ² test.

*Significant difference with team 1 p<0.05.

†Significant difference with GPs in team 1 p<0.01.

‡Significant difference with GPs in team 1 p<0.001.

Primary outcome: total number of consultations

In total, 9503 patients had a consultation during the study period. Team 1 had contact with 3287 patients, team 2 with 3166 patients and team 3 with 3048 patients. The mean number of consultations per shift by the teams was 93.9 (SD 9.0) in team 1 vs 90.5 (SD 7.2) in team 2 (not significant) and 87.1 (SD 6.2) in team 3 (p<0.001). The mean number of consultations per hour per healthcare provider was 3.1 consultations in team 1, 3.0 consultations in team 2 (GP 3.2, NP 2.6) and 2.9 consultations in team 3 (GP 3.3, NP 2.5).

Secondary outcomes

Overall, the number of patients outside NPs’ scope of practice, expressed as proportion of the total number of patients per day, was 19.1% (SD 50.4). The range of patients per day outside NPs’ scope of practice was 6%–33% (see figure 2). There was no difference between Saturdays (18%) and Sundays (20%), or between team 1 (19.9%), team 2 (18.0%) and team 3 (19.4%).

Figure 2

Patients outside nurse practitioners (NPs') scope of practice per day over weekends (expressed as the proportion of the total number of patients per day).

The absolute number of patients outside NPs’ scope of practice was a minimum of 0 and a maximum of 8 per hour. Expressed as proportion of the total number of patients who could be scheduled (=20 per hour), the maximum proportion of patients outside NPs’ scope of practice per hour was 40% (mean 9.0%, SD 6.7) (see figure 3).

Figure 3

Patients outside nurse practitioners (NPs') scope of practice per hour over weekends (expressed as the proportion of the total number consultations that can be scheduled per hour).

No (near) incidents were reported during the study. The proportion of patients who did not receive a consultation within the targeted time period according to the NTS was 3.5% in team 1, 5.2% in team 2% and 8.3% in team 3. After adjusting for confounders, the proportion of patients who did not receive a consultation within the targeted time period was significantly higher in team 2 (p=0.001) and team 3 (p<0.001) compared with team 1 (see table 4).

Table 4

Comparison of teams in terms of resource use and patient safety

	Team 2 (3 GPs and 1 NP) vs team 1 (control; 4 GPs)				Team 3 (2 GPs and 2 NPs) vs team 1 (control; 4 GPs)
		95% CI for exp b				95% CI for exp b
	B (SE)	Lower	Exp b	Upper	B (SE)	Lower	Exp b	Upper
X-ray*	−0.09 (0.16)	0.67	0.91	1.24	−0.17 (0.16)	0.62	0.84	1.15
Drug prescription*	0.13 (0.06)*	1.01	1.14	1.28	−0.07 (0.06)	0.83	0.93	1.05
Referral ED*	0.10 (0.09)	0.92	1.11	1.33	0.20 (0.09)*	1.02	1.22	1.45
Consultation not within targeted time period†	0.30 (0.14)**	1.02	1.35	1.77	0.67 (0.13)***	1.51	1.95	2.52

*Tested within a logistic regression model adjusted for age, gender, urgency level and ICPC group.

†Tested within a logistic regression model adjusted for age, ICPC group and proportion of patients with urgency level U2 per day.

*p<0.05.

**p< 0.01.

***p<0.001.

GP, general practitioner; ICPC, International Classification of Primary Care; NP, nurse practitioner.

Across the overall sample, adjusted volumes of resource use did not change significantly for X-rays between teams 1, 2 and 3. Compared with team 1, after correction for casemix, team 2 more often prescribed drugs (41.3% vs 44.2%, respectively; p=0.033). In contrast, team 3 did not prescribe more drugs (39.5%; not significant). The number of patients referred to the ED was 12% in team 1, 13.2% in team 2 and 14.7% in team 3. After adjusting for casemix, the difference between team 3 and team 1 was significant (p=0.028) (see table 4).

The costs for personnel per consultation were €23.85 in team 1, €23.65 in team 2 and €23.41 in team 3. The inclusion of costs of other resources (X-rays, medication, referrals to the ED) led to total mean costs per consultation in the primary analysis of €59.22 (SD 86.63) in team 1, €62.23 (SD 90.49) in team 2 and €65.68 (SD 94.11) in team 3. After adjusting for age, gender, urgency and ICPC group, the costs per consultation in team 3 were significantly higher compared with those in team 1 (p=0.04). In the sensitivity analysis, which used the tariff of a GP employed by another GP, the costs for personnel per consultation were €29.89 for team 1, €28.36 for team 2 and €26.66 for team 3. There were no significant differences between teams in the sensitivity analyses using the tariff for a GP employed by another GP or in the sensitivity analyses using the maximum price for medications (see table 5).

Table 5

Comparison of teams regarding direct healthcare costs

	Team 1 (control; 4 GPs) vs team 2 (3 GPs and 1 NP)		Team 1 (control; 4 GPs) vs team 3 (2 GPs and 2 NPs)
	Corrected mean difference	95% CI	Corrected mean difference	95% CI
Primary analysis (personnel cost valid GPC, X-ray, minimum price drug prescriptions, referral ED)	€−3.01	€−7.33 to €1.48	€−4.55*	€−8.94 to €−0.09
Sensitivity analysis 1 (personnel cost valid GPC, X-ray, maximum price drug prescriptions, referral ED)	€−3.07	€−7.65 to €1.09	€−4.45	€−8.83 to €0.05
Sensitivity analysis 2 (personnel cost tariff GP employed by another GP, X-ray, minimum price drug prescriptions, referral ED)	€−1.68	€−6.00 to €2.81	€−1.76	€−6.15 to €2.70

Tested within a linear regression model with bootstrapping (1000 replications) adjusted for age, gender, urgency and ICPC group.

*p<0.05.

ED, emergency department; GP, general practitioner; GPC, GP cooperative; ICPC, International Classification of Primary Care; NP, nurse practitioner.

Compared with GPs in team 1, GPs in team 3 treated patients with different age categories (p<0.001), especially more patients aged <1 year (see table 3). Moreover, there were significant differences in urgency level between GPs in team 2 (p=0.001) and team 3 (p<0.001) compared with team 1; specifically, GPs treated more patients at urgency level U2 and fewer patients at U3. Finally, there were significant differences in the types of complaints for patients treated by GPs in team 1 compared with GPs in team 2 (p<0.01) and team 3 (p<0.001). Major differences included more digestive complaints and fewer skin problems. Moreover, GPs treated a greater proportion of patients outside NPs’ scope of practice with increasing numbers of NPs in the team. In team 1, 19.9% (SD 5.1) of GPs’ patients were outside the scope of NP practice, in team 2 22.5% (SD 6.4) and in team 3 30.8% (SD 9.1). Based on the number of consultations per shift, the absolute number of patients outside NPs’ scope of practice treated per GP per shift was on average 4.7 patients in team 1, 5.3 in team 2 and 7.6 in team 3.

Across the overall sample, adjusted volumes of resource use did not change significantly for X-rays between GPs in teams 1, 2 or 3. Compared with GPs in team 1, GPs in team 2 more often prescribed drugs (41.3% vs 45.4%, respectively; p=0.002). There was no difference between GPs in team 1 and those in team 3 (40.8%). In addition, GPs in team 3 more often referred patients to the ED (18.5%) compared with GPs in team 1 (12.0%) (p=0.003) (see table 6).

Table 6

Comparison between GPs in terms of resource use

	GPs team 2 (3 GPs and 1 NP) vs GPs team 1 (control; 4 GPs)				GPs team 3 (2 GPs and 2 NPs) vs GPs team 1 (control; 4 GPs)
		95% CI for exp b				95% CI for exp b
	B (SE)	Lower	Exp b	Upper	B (SE)	Lower	Exp b	Upper
X-ray	−0.26 (0.18)	0.54	0.77	1.11	−0.12 (0.07)	0.77	0.89	1.02
Drug prescription	0.21 (0.07)*	1.08	1.23	1.40	0.02 (0.03)	0.97	1.02	1.07
Referral ED	0.14 (0.10)	0.95	1.16	1.40	0.10 (0.03)*	1.03	1.11	1.18

Tested within a logistic regression model adjusted for age, gender, urgency level and ICPC group.

*p<0.01.

ED, emergency department; GP, general practitioner; ICPC, International Classification of Primary Care; NP, nurse practitioner.

Finally, in team 2, NPs completed 93.4% of their consultations autonomously, without consulting a GP. In team 3, they completed 97.5% of the consultations without recourse to a GP. Across the overall sample, consultations among NPs and GPs were within the surgery room in 1.9% of cases, outside the surgery room in 1.3% of cases and on the telephone in 0.6% of cases. There were no cases reported in which the patient was referred to the GP in order for the GP to complete the patient consultation.

Discussion

Statement of principal findings

As a consequence of NPs treating fewer patients per hour than GPs, the total number of consultations per team per shift decreased by approximately 3.7% when the NP-GP ratio increased by one NP (team 1: 93.9, team 2: 90.5, team 3: 87.1). Of the total number of patients who can be scheduled per hour, the mean observed proportion of patients outside NPs’ scope of practice was 9.0% and the highest value in any hour was 40%. This increase lead to higher healthcare costs, although this was not sustained in the sensitivity analysis.

Teams with more NPs were associated with an increased number of patients who did not receive care within the targeted time period. Although there were no adverse events reported in any of the teams, this might have a negative impact on patient safety. Moreover, there were more ED referrals by the teams with more NPs. This increase lead to higher healthcare cost, although this did not sustain in the sensitivity analysis.

As a consequence of increasing the number of NPs per team, GPs treated a larger proportion of patients outside NPs’ scope of practice. These included patients those younger than 1 year, patients with urgent complaints and patients with digestive problems. After adjusting for the case mix, GPs working in teams with more NPs referred more patients to the ED. In the overall sample, NPs asked advice from a GP in 3.8% of cases. This means each GP was asked for advice once in every two shifts.

Strengths and weaknesses of the study

As far as we are aware, this is the first study to provide a rigorous comparison between teams providing out-of-hours care and to examine the impact on patients and GPs. The strengths of the study include the comparative evaluation design and large patient sample. The study duration of a year and a half ensured all seasons (with presumably different patient complaints) were included. The limitations of the study are that it was conducted in a single centre only and limitations in the data available, in particularly the relatively large number of missing ICPC codes. The missing ICPC codes were caused by a few GPs who repeatedly did not report ICPC codes (>50% of the missing codes were caused by 7% of the GPs), indicating that bias is at the level of the GP and not diagnosis.

A potential limitation includes the method of identifying patients outside NPs’ scope of practice based on the diagnosis after consultation. The initial exclusion was based on the complaint presented during triage, which can differ from the actual complaint presented during consultation.15 16 However, because there were no reports of consultations initiated by an NP but completed by a GP, it appeared to be uncommon for patients who seemed to be within NPs’ scope of practice after triage to turn out not to be during the consultation.

Finally, the economic evaluation was limited to a focus on costs considered relevant from the GPCs’ viewpoint, so we cannot draw conclusions on efficiency from a societal viewpoint.

Comparison with other studies

Although evidence is limited, in line with this study, previous studies have suggested that NPs are able to provide 67%–93% of all primary care services.25 However, this is the first comparative study to show how teams comprising NPs and GPs may respond to peak loads among patients who do not fit the scope of NPs’ practice. As the ICPC codes used in this study are comparable to those of other out-of-hours services in Western countries, the results are readily generalisable to other models of out-of-hours primary care delivery.26 The generalisability of findings has to be considered with respect to NPs’ education, legislation and scope of practice between and within countries and healthcare systems.27 28

Reviews of previous studies indicate that patient safety is not negatively influenced by the inclusion of NPs in teams based on the quality of care provided by NPs.12 As far as we know, this is the first study to measure patient safety in terms of the number of patients who were not treated within a targeted time period based on urgency level. However, this measure only indicates one aspect of increased patient risk and therefore has its limitations. To draw firm conclusions on patient safety, we need more insight into patients’ health outcomes after a longer follow-up period. Moreover, it may be queried whether patients who were indicated as being urgent by the call centre were actually urgent cases when they presented themselves at the GPC. A recent study in the Netherlands showed that more than half of the patients who were indicated as being urgent (U2) by the triage nurse were found by the GP at the GPC to be non-urgent (U3 or lower).29 However, there should be no reason why patients in teams with more NPs would not get treatment in time as only a maximum of 40% of the patients who can be scheduled per hour are outside NPs’ scope of practice. Delay in care for patients who are outside NPs’ scope of practice seems more likely when teams do not collaborate effectively, for example, when GPs do not focus on the patients who cannot be treated by NPs.30 NPs should treat the full range of patients that fit their scope of practice. However, working in mixed teams is an innovation and GPs express different views concerning team collaboration.31 32 Critical factors for successful implementation of the NP role, such as the involvement of all GPs in the implementation process, acceptance of the NP role and understanding of the intentions of role implementations, are especially difficult in large-scale organisations like GPCs.30 31

In line with reviews of previous studies, the introduction of NPs does not necessarily result in greater efficiency.12 14 Fewer consultations among NPs can be the result of greater use of protocols, better provision of information or less experience compared with GPs and might have further influenced delays in patient treatment.33–35 As the evidence shows that the diagnostic accuracy and use of resources of NPs are comparable to those of physicians, we did not expect an increase in ED referrals when the NP–GP ratio increased.33 36 37 We cannot determine whether this increase relates to overuse by one team or underuse by the other because there is no capacity to examine how outcomes would differ if care were provided by another team. Moreover, it remains difficult to draw firm conclusions on healthcare costs due to mixed results from the primary and sensitivity analyses.13 38 Consistent with previous studies, care delivered by teams with NPs does not necessarily seem to be associated with lower healthcare costs compared with that delivered by the GP-only team in this study.

In accordance with the literature, this study shows a slight increase in the complexity of GPs’ caseload.32 39 More qualitative insight is needed into how this is experienced by GPs. It might be considered an advantage for GPs to practice more to the full scope of their training.40 Supervision of NPs barely had an effect on GPs’ workloads and the need for supervision decreased even further during the study as NPs gained more experience.

Implications for policymakers and future research

Following the UK’s National Health Service (NHS) report ‘General Practice Forward View,’9 41 this study provides an evidence base for expanding the primary healthcare workforce through the deployment of nurses. In 99% of hours over the weekend, the proportion of patients outside NPs’ scope of practice was <25% of all those who could be scheduled. This indicates that teams with both NPs and GPs are well suited to providing all care during out-of-hours practice. The assumption is clearly that NPs provide good-quality care to those patients who are within their scope of practice. Reviews of previous studies show that the quality of care delivered by NPs is comparable to that of GPs.14 41

Our results show that incorporating NPs with GPs in out-of-hours primary care teams is a feasible option for reducing GPs’ workloads in terms of the number of shifts and increasing service capacity. However, it is still uncertain whether this is a cost-effective solution. Therefore, the optimal ratio of GPs and NPs should not be defined by the impact on efficiency of care itself, but by a long-term vision regarding (expected) demands for care, workforce needs and professional roles. More research is needed on the impact of NPs in out-of-hours care on patient safety in a larger sample of GPCs, developing a more in-depth understanding of team collaboration during out-of-hours provision.

Conclusion

A model in which out-of-hours primary care is provided by teams with a ratio of up to two GPs and two NPs offers sufficient capacity to provide care for all patients during out-of-hours practice. Teams with two GPs and two NPs were associated with a decrease in the number of patients per shift and a small increase in referrals to the ED by the team. Patient safety needs extra attention, as the number of patients who did not receive care within the targeted time period in both teams increased. There was a minimal difference in aspects of GPs’ performance.