Feasibility, acceptability and effectiveness of an online alternative to face-to-face consultation in general practice: a mixed-methods study of webGP in six Devon practices.

OBJECTIVES: To evaluate the feasibility, acceptability and effectiveness of webGP as piloted by six general practices.
METHODS: Mixed-methods evaluation, including data extraction from practice databases, general practitioner (GP) completion of case reports, patient questionnaires and staff interviews.
SETTING: General practices in NHS Northern, Eastern and Western Devon Clinical Commissioning Group's area approximately 6 months after implementing webGP (February-July 2016). PARTICIPANTS: Six practices provided consultations data; 20 GPs completed case reports (regarding 61 e-consults); 81 patients completed questionnaires; 5 GPs and 5 administrators were interviewed. OUTCOME MEASURES: Attitudes and experiences of practice staff and patients regarding webGP.
RESULTS: WebGP uptake during the evaluation was small, showing no discernible impact on practice workload. The completeness of cross-sectional data on consultation workload varied between practices.GPs judged 41/61 (72%) of webGP requests to require a face-to-face or telephone consultation. Introducing webGP appeared to be associated with shifts in responsibility and workload between practice staff and between practices and patients.81/231 patients completed a postal survey (35.1% response rate). E-Consulters were somewhat younger and more likely to be employed than face-to-face respondents. WebGP appeared broadly acceptable to patients regarding timeliness and quality/experience of care provided. Similar problems were presented by all respondents. Both groups appeared equally familiar with other practice online services; e-consulters were somewhat more likely to have used them.From semistructured staff interviews, it appeared that, while largely acceptable within practice, introducing e-consults had potential for adverse interactions with pre-existing practice systems.
CONCLUSIONS: There is potential to assess the impact of new systems on consultation patterns by extracting routine data from practice databases. Staff and patients noticed subtle changes to responsibilities associated with online options. Greater uptake requires good communication between practice and patients, and organisation of systems to avoid conflicts and misuse. Further research is required to evaluate the full potential of webGP in managing practice workload. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The evaluation has benefited from employing a range of data collection methods, resulting in a detailed picture of webGP implementation in practices with a range of list sizes and deprivation levels.

Our methods successfully captured webGP’s impact on consultation patterns, on general practitioners’ management of consultations and staff experiences of this approach to patient–practice communication.

We have established ‘proof of principle’ with regard to appropriate methods for assessing the impact of webGP on the volume of consultations within participating general practices.

The evaluation was limited to a small group of practices participating in the webGP pilot implementation in one Clinical Commissioning Group area. The ethnic mix within the sampled practices and patients reflects the general population in this area.

Although important data were gathered about patient experiences of online options, there is scope for an in-depth exploration via one-to-one interviews or focus groups to achieve greater understanding of the patient perspective.

Background

General practices in the UK are under pressure to adopt new ways of working, particularly in the light of the increasing demand for general practitioner (GP) appointments1 2 and there are calls for primary care to do yet more to relieve the burden of emergency and out-of-hours care.3 4 At the same time, a high proportion of GPs are considering retiring early and fewer medical students are choosing general practice as a career.1 5 A recent report from NHS England6 highlighted the potential of new technologies, already used routinely in other areas of life, to address new and future demands on primary care.

Although a recent UK study revealed a range of ways in which some general practices provide alternatives to face-to-face GP consultations, including consultations via telephone, email, internet video and secure online portals for patients to submit messages to their GP, many GPs in the UK seem reluctant to implement alternative approaches.7 Elsewhere, recent research from the Netherlands has found that, although GPs report that they offer a range of online services and generally have a positive attitude towards eHealth, patients may be unaware of these services.8 A recent national survey in Denmark, where patients have been able to email their GP as part of routine service provision for >10 years, found that 65% of respondents had communicated with their GP online,9 representing an increase in email communications for such purposes from 0.8 million in 2007 to 4 million in 2013. A recent Australian study10 highlighted the potential advantages of electronic assessments of lifestyle risk factors prior to a GP consultation for preventive care, resulting in saving face-to-face time. Several US-based studies have evaluated and reported on the use of online care services, and some have concluded that younger patients and female patients most often use services such as structured online consultation forms (‘eVisits’) or secure instant messages via online patient portals.11–16

In England, senior practising GPs within one medical group (http://hurleygroup.co.uk/) developed ‘webGP’, a GP e-consultation and self-help web service accessed by adult patients (≥18 years) via their practice’s website. They introduced and piloted the system in 20 London general practices over the course of 6 months from December 2013 to May 2014. Their pilot report17 included information about the core features of webGP and reported results from the pilot implementation. During the pilot, 18% of the registered practice population across all 20 practices was estimated to have used webGP, with large proportions of patients and GPs reporting high levels of satisfaction with the webGP consulting experience. Although the report provided a useful overview of webGP and its potential, there is still a need for an independent evaluation of the feasibility, acceptability and effectiveness of webGP as a viable alternative for patients wishing to consult their GP across a range of general practice settings.

WebGP consists of a suite of five services which are available from participating general practices’ websites:

symptom checker which gives brief information about a range of conditions

self-help guidance provides more detailed information about symptoms/conditions

signposting to other services, such as pharmacy

information about the 111 telephone service

e-Consult, whereby the patient completes an online form which is emailed to the practice.

Aims and objectives

In 2015, the University of Exeter Medical School, Department of Primary Care was invited by the NHS Northern, Eastern and Western Devon Clinical Commissioning Group (NEW Devon CCG) to undertake an evaluation of webGP, whose implementation was being supported and piloted in six Devon practices. The research aims were to:

establish ‘proof of principle’ regarding methods for examining the impact of webGP on volume of consultations;

explore management decisions made by GPs in response to e-consult requests;

investigate patient experience of using webGP;

explore staff experiences of using webGP, with particular regard to GPs’ workload and stress.

The specific objectives were to:

describe the mode and volume of GP and nurse consultations for each of the participating practices, comparing consultation workload before and after the introduction of webGP;

describe the actions taken by GPs following receipt of e-consults from information recorded on case report forms (CRFs) during a 4-week data collection period;

investigate patient experience by surveying patients who had used webGP and comparing their experience with controls (patients who had received a face-to-face consultation during the same time period) matched for age and gender;

examine the experiences of clinical and administrative practice staff regarding the introduction and use of webGP.

Methods

Practice recruitment

All six practices participating in the NEW Devon CCG webGP pilot were approached by email. Two of the practices were the main surgery and student health centre of the same overall practice, and shared staff (practices 4(i) and 4(ii), respectively). A researcher followed up the initial email by visiting each practice to explain the evaluation and to elicit basic practice information (list size, IT system, online services, major staff/system changes within the last three years and ways in which the practice had promoted webGP). Written consent was obtained from the practice to participate in the evaluation, and a specific 4-week period for data collection was agreed. All of the practices had implemented the use of webGP prior to November 2015.

Data collection

Extraction of data from practice databases

To examine the impact of webGP on volume of consultations, practices were asked to extract retrospective electronic data: the number of face-to-face, telephone and home visit consultations undertaken by a practice GP or nurse (separately) and taking place over a specified 3-month period (January, February and March) for each of 2013, 2014 and 2015, and the corresponding data for 2016. The latter included the number of webGP e-consults. WebGP had been in use at each practice for at least 1 month prior to collecting these data. We also collected data on the total number of patients registered with the practice in April of each year from 2013 to 2016.

In addition, we obtained information on webGP (e-consult) usage in the six practices from monthly reports provided directly from webGP developers to the CCG (this did not include the demographic characteristics of webGP users). This information formed an overall estimate of webGP usage by patients, considered in relation to practice list size within the period of interest.

Completion of CRFs

To explore GPs’ management decisions in response to receipt of e-consult requests, GPs completed a short CRF for each e-consult they dealt with during a 4-week data collection period. Data collection weeks were consecutive and undertaken sometime between February and May 2016. GPs were asked to complete a CRF after reading the e-consult request and before taking any further action. The CRF recorded the date and time of receiving and responding to the e-consult request, brief details about intended GP actions planned in response to initial consideration of the e-consult request and general questions about handling the request.

Patient survey

To investigate patient experience of using webGP, we developed two surveys, both incorporating items based on the most recent General Practice Patient Survey (GPPS),18 the largest routine survey of users’ experience of primary care in the world (total current annual sample approximately 1.2 million patients registered with English general practices per year). GPPS includes items addressing issues such as the accessibility of care and overall patient satisfaction. Patients who had submitted an e-consult request during the 4-week data collection period were invited to complete Survey 1, comprising 35 items. Relevant items from the GPPS were modified to focus on patients’ recent use of webGP. The e-consulting patients were matched with the next two consulting patients of the same age ±5 years (those aged 18–24 years were matched by patients in the same age range) and the same gender, who had had a face-to-face GP or nurse consultation during the same period. (Practice staff advised the researcher with regard to any patients with severe mental illness, in receipt of terminal care, with moderate to severe cognitive impairment, or unable to complete a questionnaire in English, and these individuals were excluded from the mailing.) The matched patients were invited to complete Survey 2, comprising 26 items. Relevant items from the GPPS were modified for direct comparison with e-consulters’ responses. The wording of 16 items in both surveys was exactly as for the GPPS.

Both surveys included questions about patient experience of care (25 items in Survey 1 and 16 items in Survey 2), health status (3 items in each survey) and sociodemographic details (7 items in each survey). Both surveys included free-text boxes for expansion/clarification of responses.

Patients were posted the appropriate questionnaire and supporting information from the practice 7–14 days following their consultation. In order that the elapsed time between appointment and receipt of questionnaire did not exceed 3 weeks for any patient, questionnaires were sent in two mailings, one following week 2 and a second following week 4 of data collection. The questionnaire pack included an invitation letter and information leaflet describing the evaluation and providing contact details for the research team, and a reply-paid envelope. One reminder was sent to each patient who had not returned a completed questionnaire 14 days after the initial mailing.

Staff Interviews

To explore staff experience of using webGP, each practice was asked to identify potential interviewees (one GP and one administrator/receptionist) who had been involved with implementing the webGP service. Potential interviewees were sent a written invitation from the research team, including an information sheet. The interview followed a semistructured topic guide (introduction and operation of webGP within the practice, acceptability of webGP to staff and patients, problems encountered with webGP, general perceptions of webGP and other online alternatives). Interviews were conducted either face-to-face or by telephone, and all were audio-recorded and transcribed verbatim.

Data analysis

Practices extracted data on the total number of consultations for each type of consultation for each calendar month during the data collection periods. For each type of consultation, a rate per 1000 registered patients was calculated and presented across each year of interest. Quantitative response data were described in respect of (i) CRF responses, (ii) responses to analogous items in both surveys and (iii) responses to items included in the e-consult survey only. Outcomes involving durations of time were analysed using Stata V.14; all other analyses were performed using SPSS V.23. Applying information provided by practices 2 and 4, list sizes for student populations only (ie, practices 2 and 4(ii)) were calculated as 40% of the overall practice list size. The list size for practice 4(i) (main surgery) was accordingly calculated as 60% of the overall list size for practice 4.

CRF responses were compared with e-consult survey responses for two items: (i) ‘GP decision’ regarding patient disposition as recorded on CRF and ‘health professional spoken to’ recorded by patient; and (ii) ‘date/time of receipt of and response to e-consult’ as recorded on CRF and patient response to ‘After submitting your e-consult via webGP, when did you receive a response from the practice/GP?’. Responses to selected survey items that were analogous to GPPS items were compared with findings reported from the most recent GPPS results for participating practices (July 2016). Results for individual practices and CCGs are publicly available from the GPPS website.19

Transcribed qualitative data from staff interviews and free-text comments provided by patients were analysed thematically. Transcripts were read and re-read, and significant content within the data was noted and highlighted. From these notes, one researcher (MC) developed a coding framework. This was discussed with a second researcher (AS) and subsequently refined. Both researchers coded a subset of the interview transcripts according to the coding framework, and further refinements were made until agreement was reached. QSR NVivo V.11, a specialist software program, was used to facilitate systematic organisation and coding of the qualitative data. Principal themes were identified and checked to ensure a balanced and accurate interpretation of the data.

Results

All six practices participating in the webGP pilot agreed to take part in the evaluation. WebGP had been in use at five of the practices for at least 3 months, and for the remaining practice for just over a month. All practices had promoted webGP via a large poster and/or television screen in the waiting room and the practice website. One practice had also added information about webGP to the bottom of prescriptions and to the telephone answerphone message. List sizes, deprivation scores20 and date of webGP introduction for participating practices are shown in table 1.

Table 1

Participating practices’ list sizes (2013–2016), deprivation score/decile (IMD 2015) and date of webGP introduction

Practice	List size April 2013	List size April 2014	List size April 2015	List size April 2016	Deprivation score* (decile†)	Date webGP introduced
1	9327	9415	9356	9232	15.3 (8)	26 November 2015
2	5808	5925	5598	5868	21.5 (6)	10 September 2015
3	11 762	11 731	11 713	11 786	14.9 (8)	1 September 2015
4(i)	7900	8197	7551	7739	28.9 (4)	8 October 2015
4(ii)	5266	5465	5034	5160		8 October 2015
6	10 116	10 134	10 032	10 327	34.5 (2)	17 September 2015

Practice 4 divided into 4(i) main surgery and 4(ii) student health centre. List sizes and IMD 2015 deprivation scores (higher numbers indicate more deprivation) and deprivation deciles (lower numbers indicate more deprivation) from Public Health England: general practice profiles.20

*Higher number more deprived.

†Lower number more deprived.

Consultation data

The completeness of consultation data downloaded from practice systems varied considerably (table 2) due to changes taking place in some practice IT systems during the period under study.

Table 2

Available consultation data by participating practice

Practice	2013						2014						2015						2016
	GP				Nurses		GP				Nurses		GP				Nurses		GP					Nurses
	FtF	Tel	Home	A&E	Nurses FtF	Nurses Tel	FtF	Tel	Home	A&E	Nurses FtF	Nurses Tel	FtF	Tel	Home	A&E	Nurses FtF	Nurses Tel	FtF	Tel	Home	A&E	webGP	Nurses FtF	Nurses Tel
1	X	X	X	X	X	X	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
2	X	X	X	X	X	X	X	X	X	X	X	X	✓	✓	X	✓	✓	NA	✓	✓	X	✓	✓	✓	NA
3*	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
4(i)	X	X	X	X	X	X	X	X	X	X	X	X	✓†		✓	X	✓	N	✓†		✓	X	✓	✓	NA
4(ii)	X	X	X	X	X	X	X	X	X	X	X	X	✓†		NA	X	✓	N	✓†		NA	X	✓	✓	NA
6*	✓	✓	✓	X	✓	✓	✓	✓	✓	X	✓	✓	✓	✓	✓	X	✓	✓	✓	✓	✓	X	✓	✓	✓

*Also provided data for nurse practitioners.

†Data for FtF and telephone consultations together.

A&E, Accident & Emergency; FtF, face-to-face; GP, general practitioner; NA, not applicable; Tel, telephone; ✓, data available; X, data not available.

Table 3 shows the average monthly consultation rates for each of the 4 years (2013–2016). For the four practices (1, 2, 3 and 6) reporting data for face-to-face and telephone consultations separately, the rate of face-to-face consultations with a GP within the data extraction period decreased slightly but consistently over the 4-year period. The greatest decrease in consultation rate (face-to-face with a GP) was in practice 1 (251 in 2014 to 196 in 2016). Consultation rates for combined GP face-to-face and telephone consultations increased in two practices (3 and 4(i)), but decreased in the other four (1, 2, 4(ii) and 6). The greatest decrease was in practice 1, based on 3 years of data (306 in 2014 to 250 in 2016).

Table 3

Consultation rate by type (3-month period January–March, 2013–2016, rate per 1000 registered patients)

Practice	Year	List size	GP consultations N, rate*						WebGP† consultations N, rate*		Nurse consultations N, rate*
			FtF		Tel		Combined				FtF		Tel		Combined
1	2014	9415	2359	251	522	55	2881	306	–		1182	126	18	2	1200	127
	2015	9356	2149	230	555	59	2704	289	–		1210	129	32	3	1242	133
	2016	9232	1805	196	500	69	2305	250	13	1	1344	146	101	11	1446	157
2	2015	5598	1258	225	23	4	1281	229	–		1448	259	0	0	1448	259
	2016	5868	1119	191	50	9	1169	199	57	10	1321	225	0	0	1321	225
3	2013	11 762	2994	255	1062	90	4056	345	–		2460	209	26	2	2487	211
	2014	11 731	2917	249	1299	111	4216	359	–		2323	198	39	3	2362	201
	2015	11 713	2867	245	1038	89	3905	333	–		2283	195	59	5	2341	200
	2016	11 786	2739	232	1471	125	4210	357	47	4	1973	167	64	5	2037	173
4(i)	2015	7551	No data				2215	293	–		1031	136	0	0	1031	136
	2016	7739	No data				2569	332	67	9	1183	153	0	0	1183	153
4(ii)	2015	5034	No data				662	132	–		185	37	0	0	185	37
	2016	5160	No data				632	122	30	6	542	105	0	0	542	105
6	2013	10 116	1918	190	586	58	2503	247	–		340	34	7	1	347	34
	2014	10 134	1723	170	606	60	2329	230	–		306	30	8	1	314	31
	2015	10 032	1755	175	613	61	2368	236	–		407	41	9	1	416	41
	2016	10 327	1679	163	510	49	2189	212	116	11	297	29	12	1	309	30

*Rate; N per 1000 registered population.

†Based on data from January to March 2016.

FtF, face-to-face; GP, general practitioner; n, number of consultations; Tel, telephone; webGP, e-consult.

All six practices offered nurse face-to-face consultations. The rate of such consultations within the data extraction period increased in three practices (1, 4(i) and 4(ii)) and decreased in the remaining three (2, 3 and 6). The largest increase was in practice 4(ii), based on 2 years of data (37 in 2015 to 105 in 2016). Where it was possible to calculate combined consultation rates for face-to-face and telephone consultations with nurses, rates decreased in two practices (3 and 6) and increased in one (1).

Using data provided by the webGP developers to the CCG, we calculated the consultation rate for practice patients having e-consults during January, February and March 2016. The consultation rate for documented webGP consultations averaged 7 (range across practices 1–11).

CRFs completed by GPs

Of the 77 e-consult requests submitted by patients across all practices during the data collection period, 61 CRFs were completed by GPs (79%). CRFs were completed for more female patients (n=45) than for males (n=16), and the largest number was for patients in the age group of 25–34 years (n=18). The smallest number was for patients in the age groups of 65–74 years (n=1) and ≥85 years (n=1). The number of GPs in each practice who completed CRFs ranged from 2 (who completed a total of 5 CRFs) to 5 (who completed a total of 35 CRFs). The number of CRFs completed during a 4-week period ranged from 4 to 35 per practice.

In 44 e-consults (72%), the GP suggested that the patient needed a subsequent face-to-face or telephone consultation with a GP or nurse. In 36 cases (59%), the GP indicated that an administrator would contact the patient to convey their decision; in 24 cases (39%), the GP indicated that a GP would make contact with the patient. In 38 cases (72%), the GP anticipated referring to the patient’s notes to handle the e-consult. For the majority of cases (37/61; 63%), the GP reported being ‘not at all familiar with the patient’; only three (5%) reported being ‘very familiar’. In virtually all cases (58/61; 97%), GPs reported feeling either ‘very confident’ or ‘confident’ about managing the e-consult request.

Using data from 22 CRFs which included the date/time of the e-consult request and that of the GP’s response, and which we could match with the appropriate completed patient survey, we investigated the timings of an e-consult request and the delay between its receipt and the GP’s response. The majority (14/22, 64%) were managed by a GP on the same calendar date. A further 4/22 e-consults (18%) were managed the following day, and 4/22 (18%) were managed at a subsequent date. In 16 cases, there were discrepancies in the recorded delay between the time of the e-consult request and the time of the GP’s response (collected by GPs on the CRF), and the patient’s reported delay in receiving a response (collected within the patient survey). We did not collect data from the practice regarding the time/date that the patients were contacted regarding the outcome of their e-consult, which may explain this discrepancy in some cases.

Patient survey (e-consulters and face-to-face consulters)

We received 29 completed questionnaires from 77 e-consulters (38% response rate) and 52 from age-matched and sex-matched patients who had had a recent face-to-face consultation with a GP (52/154; 34% response rate). Since the survey was sent out directly from the practices to patients, we were unable to characterise the demographic characteristics of the survey sample. Of the e-consulter respondents, 23 (23/29, 79%) had found out about webGP services from a poster/leaflet or from ‘other’ means (often identified as the practice website), and most reported that it was ‘very easy’ to find on the practice website (20/29; 69%). Principal reasons for using webGP were to obtain a rapid response or to save on time or cost of travel (both approximately 19% of respondents). Nearly half of the respondents (13/29; 45%) reported receiving a response from the practice or GP ‘on the next day’. Most (23/29, 79%) reported their waiting time to receive a response as being ‘about right’. Most responders reported being in employment (18/29; 62%). Demographic details of all survey respondents are presented in table 4.

Table 4

Demographic details of survey responders

	Face-to-face responders	e-Consulters
Gender
Female	40	20
Male	11	9
Prefer not to say	1	0
Age (years)
18–24	10	5
25–34	10	6
35–44	8	8
45–54	12	4
55–64	6	4
65–74	4	1
75–84	0	0
≥85	2	1
Ethnic group
White	49	29
Asian/Asian British	1	0
Mixed or multiple	1	0
Prefer not to say	1	0
Occupation
Full-time paid	18	12
Part-time paid	4	6
Full-time education	6	4
Unemployed	3	1
Permanently sick/disabled	6	1
Fully retired	7	3
Looking after the home	3	1
Doing something else	2	1
Prefer not to say	2	0
Multiple response	1	0
Caring responsibility
Yes	14	11
No	37	18
Missing response	1	0

A large majority of respondents reported (i) being satisfied with their overall experience of using webGP (24/28; 86%), (ii) being likely to use webGP in the future (25/29; 86%) and (iii) would recommend webGP to their family/friends (22/28; 79%).

Substantial differences (>10% between groups) existed between e-consulters and face-to-face consulters in respect of reported problem resolution (55% vs 33% reported that their problem had been ‘completely resolved’) and regarding seeing or speaking to a GP following the consultation request (52% vs 93.8%). Other smaller differences were observed between e-consulters and face-to-face consulters: the former reporting less favourable impressions of their opportunity to provide all relevant information (69% vs 79%) and in respect of overall satisfaction with the practice’s response following the consultation request (78% vs 86% being satisfied).

Fewer e-consulters than face-to-face consulters reported their experience of making an appointment as ‘very good’ or ‘good’ (44% vs 59%). A much larger proportion of e-consulters than face-to-face consulters reported having used their practice’s online services within the previous six months (89% vs 27%). More e-consulters than face-to-face consulters reported having a preferred GP (67% vs 57%). E -Consulters also reported being less satisfied with their ability to consult their preferred GP (44% vs 57%).

Analysis of seven items that were analogous to questions in the GPPS (table 5) showed that e-consulters’ responses differed from the face-to-face responders to our survey and to the GPPS results. Fewer e-consulters reported their practice receptionists as being ‘helpful’ or ‘very helpful’ (76% vs 86% for face-to-face consulters and 92% from GPPS data), fewer e-consulters reported being able to see their preferred GP ‘always’ or ‘a lot of the time’ (44% vs 57% for face-to-face consulters and 64% from GPPS data) and fewer e-consulters rated their overall experience of making an appointment as ‘good’, ‘fairly good’ or ‘very good’ (67% vs 77% for face-to-face consulters and 78% from GPPS data). Overall, webGP users tended to report less favourable experiences of accessing and using GP services compared with face-to-face consulters and with individuals from the practice completing the national GPPS.

Table 5

Comparison of General Practice Patient Survey (GPPS)/e-consult/face-to-face (FtF) survey results

Survey item	Survey group responses
	GPPS*	e-Consult	FtF	GPPS*	e-Consult	FtF	GPPS*	e-Consult	FtF
Ease of getting through to GP surgery on the phone	Very easy/fairly easy n=705	Very easy/easy n=16	Very easy/easy n=29	Not very/not at all easy n=106	Not very/not at all easy n=10	Not very/not at all easy n=20
	80%	59%	57%	13%	37%	39%
Helpfulness of receptionists at GP surgery	Very helpful/fairly helpful n=807	Very helpful/helpful n=22	Very helpful/helpful n=44	Not very/not at all helpful n=20	Not very/not at all helpful n=5	Not very/not at all helpful n=6
	92%	76%	86%	2%	18%	12%
Have a preferred GP	Yes n=471	Yes n=18	Yes n=29	No n=394	No n=9	No n=22
	56%	62%	56%	43%	31%	42%
See preferred GP	Always/a lot of the time n=303	Always/a lot of the time n=8	Always/a lot of the time n=16	Never/almost never n=132	Never/almost never n=3	Never/almost never n=2
	64%	44%	57%	3%	15%	6%
Overall experience of making appointment	Very/fairly good n=677	Very good/good/fairly good n=18	Very good/good/fairly good n=40	Fairly/very poor n=38	Fairly poor/poor/very poor n=6	Fairly poor/poor/very poor n=5
	77%	62%	77%	4%	7%	6%
Overall experience of surgery	Very/fairly good n=790	Very good/good/ fairly good n=23	Very good/good/fairly good n=44	Fairly/very poor n=0	Fairly poor/poor/very poor n=1	Fairly poor/poor/very poor n=2
	90%	79%	85%	0%	3%	4%
Confidence in managing own health	Very confident n=403	Very confident n=9	Very confident n=21	Fairly/not very confident n=445	Fairly/not very confident n=20	Fairly/not very confident n=28	Not at all confident n=0	Not at all confident n=0	Not at all confident n=3
	46%	31%	40%	51%	69%	54%	0%	0%	6%

*GPPS indicates averages from GPPS results (2015) of five practices (student medical centres not separated from main practice).

Qualitative results

Ten practice staff interviews were conducted (five GPs, five administrators/receptionists; three in person, seven by telephone), with a mean interview length of 14 min (SD 6, range 5–27 min). Patients provided comments in free-text boxes as part of their questionnaire survey.

Feasibility of webGP

Many interviewees voiced concerns about webGP and how it was being used by patients. Some felt that patients were using webGP to circumvent existing systems within the practice, such as appointments and telephone consultations:

Because I think sometimes they just try to bypass the telephone system by, you know, saying ‘I want a phone call with my doctor’ … Do you see what I mean, they don’t use it as sort of a symptom checker as such, they just use it to get a message to the doctor. (P3_01/GP)

Some interviewees were also critical about the types of problems reported by patients via webGP and unnecessary delays in care which may be incurred:

Sometimes just by reading something they’ve requested, like ‘I keep fainting’ … Really I think … sounds like she needs to be seen. I think she’s delaying herself access to a GP in person by doing this email overnight. (P2_01/receptionist)

Through free-text comments, some patients confirmed that they were using webGP as an alternative to existing, less efficient systems:

Normally difficult to get an appointment by phone within a reasonable time frame. WebGP is brilliant. I have used this service many times. Every time I have been satisfied with the quick response. (60201_webGP)

Conversely, the efficiency of other systems in the practice was cited by one staff interviewee as a possible reason for the low rate of use:

I wonder if they (patients) get a pretty good service from our telephone consulting … That works pretty quickly for them and I wonder if there’s therefore a lack of incentive to pursue other or more novel approaches. (P4_01/GP)

Acceptability of webGP

Staff interviewees perceived benefits for particular groups of patients, such as the flexibility of using webGP for working people:

For people who work it’s sometimes difficult to sit on a phone and make a phone call, whereas you can sit and type an email quite quickly. (P1_02/administrator)

Some also mentioned particular conditions and situations for which webGP may be advantageous, such as problems with mental health:

There’s one really good example … It was a girl with anxiety … She got embarrassed easily and stumbled over her words and that was her barrier to actually coming and discussing it in the first place. (P2_02/GP)

Patients concurred with this view through free-text comments:

I find the online consultation option very convenient as I can give all the information I want to give and it saves me anxiety and having to travel or make a phone call. My partner is physically disabled so this will be useful for him too. (40291_webGP)

Effectiveness of webGP

WebGP is promoted by its developers as a means of reducing GP workload. Interviewees had a range of views about this and reported shifts in responsibility in more than one direction. Some suggested that webGP had resulted in an increased workload for the administrative team:

So it’s giving us more work, cos we’re ringing them, to make them an appointment, rather than them ringing us to make an appointment. (P1_02/administrator)

Some staff perceived a shift in workload from the GP to the patient:

… you’re shifting time and effort to the patient’s … responsibility, who can afford to spend an extra couple of minutes here and there at the expense of saving time for the GPs … (P2_02/GP)

Others were concerned that the anticipated reduction in GP workload had not materialised and that webGP had in fact increased the GP’s work:

I think it might give extra work to the GP, because they’ve then got to access the document that we’ve attached, read it, phone the patient. (P2_01/receptionist)

Several interviewees suggested improvements to the webGP programme, such as a way of alerting patients to expect a call from their practice following submission of an e-consult request:

I think quite often you need to make a very quick phone call to the patient to clarify a detail or to explain a plan and that is often a problem. They don’t answer the phone, it’s not at a convenient time for them, they’re not expecting it … I wonder whether a more ready way of being able to reply by email. (P4_01/GP)

Patients too were concerned about some aspects of webGP in its current form:

Although I agree with the concept of webGP, it is not yet an alternative. I gave confidential information via a web request, only to be then contacted by a female receptionist. What confidentiality does that offer! (60331_webGP)

Discussion

Principal findings

This study examined the feasibility, acceptability and effectiveness of an online GP e-consultation and self-help web service, webGP and its potential for contributing to managing increasing levels of demand for primary care services. A mixed-methods approach was adopted to capture webGP’s impact on consultation patterns, on GPs’ management of consultations and patient and staff experiences of this approach to patient–practice communication. The overall uptake and workload was less than was anticipated by some practice staff, and better communication with and marketing to patients, as exemplified by the practice with the largest volume of completed e-consults, may result in increased use. Practice staff noticed, however, some modest shifts in responsibility between different members of the team and from the practice to its patients. WebGP did not easily integrate with some existing practice IT systems, and more consideration may be needed in respect of the organisation of appointments, and for providing guidance to patients as to when to consult online. Our exploration of GPs’ management decisions in response to e-consult requests suggests that GPs feel confident in dealing with this new way of consulting, but frequently decide that the patient needs to see or speak to a GP or a nurse. The responsibility for conveying this decision to the patient is often given to an administrator, shifting such workload towards non-clinical staff. In contrast to findings from the Alt-Con study,7 the GPs participating in the webGP evaluation did not appear to be reluctant to use this new way of consulting.

The investigation of patient experience suggests that online alternatives for consultation are predominantly used by patients of working age, who describe themselves as working full-time or part-time. These patients had often used other online services provided by their practice during the 6 months prior to completion of the survey. Although patient accounts of practice response time following their e-consult request were different from (and generally less favourable than) GP records, the majority appeared satisfied with their experience and would use it again. The motivation for using webGP mostly concerns time—the desire for a quick response or to save travel time or cost. This may be influenced by work pressures for e-consulters, who report more difficulty in taking time away from work to visit the GP than face-to-face consulters. Another driver may be previous experience of other aspects of interacting with their practice, such as booking an appointment, seeing a preferred GP and obtaining help from receptionists, about which e-consulters are less positive than face-to-face consulters. It thus appears that webGP may offer improved access to GP care for some groups of the population.

The exploration of staff experience concurred with CRF findings, revealing a small shift of responsibilities within the practice and in relation to the patient. These shifts in responsibility were linked with perceived fit or clash with existing practice systems. WebGP was sometimes seen as an additional option for patients who struggle to obtain or attend a ‘traditional’ appointment, but many staff believed that webGP was used by some patients to circumvent existing systems.

This small study has demonstrated that, by applying a range of complementary methods, it is possible to learn valuable lessons about the impact of innovative approaches to managing workload in general practice.

Strengths and weaknesses of the study

This evaluation has benefited from employing a range of data collection methods, resulting in a detailed picture of the webGP implementation in practices with a range of list sizes and deprivation. We used aggregated consultations data from the same 3-month period in all practices across four consecutive years to ensure that seasonality did not influence the results; such data provided useful contextual information, and we were able to electronically extract data according to type of consultation and health professional. Completion of a short CRF during a specified 4-week period was an acceptable burden for most GPs and provided useful information about clinical decisions made in response to e-consult requests. The two patient surveys provided important information about patient views and enabled comparisons between e-consulters and their associated CRF reports, between e-consulters and face-to-face consulters, and between both e-consulters and face-to-face consulters and practices’ national GPPS results. Free-text response boxes gave patients an opportunity to make further comments about webGP and their practice, adding richness of data to the qualitative evaluation. Interviews with staff in each practice provided a range of perspectives about webGP from those likely to be involved with its implementation.

The evaluation was limited, however, to a small group of practices participating in the webGP pilot implementation in just one CCG area, with a predominantly white British population. Uptake of webGP was varied, with very few patients in most practices making use of the new system. In general, the nature of the study and the small numbers involved did not merit the use of inferential testing of data. Future research might recruit practices in which webGP has been used more frequently and for a longer period of time. Changes to the practice IT system in several practices during the period under examination also affected our evaluation. Although the CRFs provided valuable information on clinical decisions, not all were completed. A more robust system for the collection of completed forms from participating GPs would be advisable in any future study. A means of collecting data on the outcomes of face-to-face consultations booked by patients on the advice of their GP (following submission of an e-consult) would inform a longitudinal perspective to the assessment of changes to workload.

The response rates (38% for the e-consulters’ survey and 34% for the face-to-face consulters’ survey) are similar to response rates achieved by the national GPPS (35% in 2012/2013, 34% in 2013/2014 and 33% in 2014/2015). Time and budgetary constraints precluded the provision of an online alternative for completing the patient surveys, which may have been particularly appropriate for e-consulters and may have achieved a higher response rate. There was insufficient capacity for follow-up patient interviews, which would have added depth to our understanding of their views about webGP and other online systems. In addition, we acknowledge that the small number of interviews does not allow us to assume that saturation of emergent themes21 has been achieved.

Implications for clinicians and policymakers

Staff are sometimes protective of their existing practice systems and may be wary about patients using webGP to gain an unfair advantage over other patients. Promotion of a shared message among practice staff and with patients, as found by previous research about the introduction of new systems for telephone triage and real-time feedback in general practice,22 23 may result in greater uptake and more effective use of online approaches such as webGP.

Unanswered questions and future research

Although important data were gathered about patient experiences of online options, there is clearly scope for an in-depth exploration via one-to-one interviews or focus groups to achieve greater understanding of the patient perspective. Further exploration of the discrepancies between clinician and patient reports of timeliness of service delivery with regard to e-consults is also warranted. Based on the GP reports we collected, a large majority of e-consults were managed within the timeframe recommended by webGP guidelines (the patient should receive a response to their e-consult request by the following working day), but we did not collect data on the date/time that the practice actually contacted the patient with the result of their e-consult request.

Any future assessment of online approaches should include an examination of the longer-term outcomes of e-consults in terms of primary care consulting and Accident & Emergency attendance and a comparison with face-to-face consulters for the same outcomes. Future research should also include an assessment of the impact of online consultation options for different presenting conditions.

Conclusions

There is a growing need to extend feasible options for patients beyond the traditional consultation model. Online consulting may provide an effective and convenient alternative for some groups of patients (such as those in employment) with non-urgent problems, who do not require face-to-face contact with the practice. It may also offer an acceptable means by which patients can consult their clinicians about sensitive topics, which may otherwise remain unarticulated.