Patients' use and experiences with e-consultation and other digital health services with their general practitioner in Norway: results from an online survey.

OBJECTIVES: To explore patients' use and experiences with four digital health services implemented in Norway to enable electronic communication between patients and their general practitioner (GP): (1) electronic booking of appointments; (2) electronic prescription renewal; (3) electronic contact with the GP's office for non-clinical inquiries; and (4) e-consultation for clinical inquiries.
DESIGN: An online survey consisting of quantitative data supplemented by qualitative information was conducted to explore: (1) characteristics of the users; (2) use; (3) experiences, perceived benefits and satisfaction; and (4) time spent using the digital health services.
SETTING: Primary care. PARTICIPANTS: 2043 users of the digital health services answering the survey.
RESULTS: There was a higher proportion of women, younger adults and digitally active citizens with high education. Electronic booking of appointments was the most used service (66.4%), followed by electronic prescription renewal (54.3%). Most users (80%) could more easily and efficiently book an appointment electronically than by phone. Over 90% of the respondents thought that it was easier to renew a prescription electronically, 76% obtained a better overview of their medications and 46% reported higher compliance. For non-clinical inquiries, most respondents (60%) thought that it was easier to write electronic messages than communicate by phone. For clinical enquiries, many patients agreed that e-consultation could lead to a better followup (72%) and improved quality of treatment (58%). Users were highly satisfied with the services and recommended their use to others. Time saving was the most evident benefit for patients. This was confirmed by the differences in time spent using the digital health services compared with conventional approaches, all found to be statistically significant.
CONCLUSION: Citizens using e-consultation and other digital health services with their GP in Norway are satisfied and consider them as useful and efficient alternatives to conventional approaches. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

One of the first examples of nationwide implementation of digital health services for citizens in primary care.

The quantitative survey was supplemented by qualitative feedbacks with interesting insights and useful suggestions for improvement.

High number of respondents considered to be representative of those who used the services.

Respondents were early adopters who could have a more positive attitude.

Findings are relevant to citizens, general practitioners and policymakers.

Background

The increasing demand in primary care within a limited capacity and the need to improve patients’ access has prompted the consideration of alternatives to face-to-face contacts with the general practitioner (GP), including telephone contacts as well as electronic communication, such as email, electronic messaging via patient portals or Internet video.1 2 Electronic communication between patients and physicians has the potential to empower patients3 and improve healthcare services.4 There is also growing recognition that many patient encounters do not require face-to-face contact, and the increasing use of the Internet creates the opportunity for remote consultations, where the interaction between the physician and the patient is completely virtual.5

In the early 2000s, a number of studies on e-mail communication between patients and GP were conducted. Patients found e-mail communication easy to use6 7 and preferred it over phone calls for the communication of non-urgent problems,6 8 9 including updates to the GP, prescription renewals, health questions, questions about test results or referrals, appointments and requests for non-health-related information.8–10 However, the empirical evidence for the use of email for clinical communication between patients and healthcare professionals11 and for the provision of information on disease prevention and health promotion12 remains limited. Despite email being commonly offered by GP offices for making appointments and renewing prescriptions, its use for direct contact with the GP is not commonplace.13 In a retrospective observational study, even though email consultation was adopted by half of the general practices in the Netherlands in 2014, its actual use was extremely low.14 Moreover, the security level for regular email is considered insufficient4 and non-compliant to current requirements for authentication and encryption.15

Today, the electronic communication between patients and GP is generally done in an asynchronous manner via a secure electronic patient portal.5 16 17 Web messaging systems address issues around security and liability associated with conventional email communication since they offer encryption capability and access controls.6 13 The GP Online is a national programme introduced in the UK in 2016 to offer online services to primary care patients in addition to the traditional telephone and face-to-face means of interacting with a GP practice.18 Patients can access online services through patient portals, enabling them to book and manage appointments, order repeat prescriptions and view their GP medical records.19 In 2019, the National Health Service (NHS) application was introduced to contrast the slow progress in widescale adoption of these services and provide a better solution to patients.20 Through the NHS application, citizens can now identify themselves through the national Citizen Identity tool and use additional facilities, including a symptom checker and access to donor information.20 Some private providers are also offering access to GP consultations via mobile or online triage platforms, including askmyGP, eConsult and Egton Online Triage.20 21 Such forms of electronic communication have been considered by patients as equivalent or better than face-to-face contacts22 and represent an appropriate and potentially cost-saving addition to in-person delivery of primary care.22 23 Patients also experienced easier access to their GP for minor health problems and received quick responses to their requests.4 A recent evaluation of an online consultation system tested in 36 general practices in South West England found that the use of online consultations was very low (two per 1000 patients per month), more common among women and working-age adults, and mostly suitable for administrative requests, such as repeat prescriptions and test results.24 According to the recent NHS long-term plan, all patients in England are expected to have access to online GP consultations by 2023/2024.25

Electronic communication between patients and GP can also be conducted through real-time video linkage.5 Benefits of video communication in primary care compared with in-person encounters include convenience, efficiency, communication, privacy and comfort.26 In 2014, the UK government made a commitment to spend £3.6 million on the introduction of Skype video calling consultations in general practice.27 Evidence of the use and effects of video communication in primary care is still scarce. Preliminary studies aimed at exploring the attitudes of GPs towards video consultations in Australia and UK concluded that the majority of the GPs recognised potential benefits but also expressed concerns.27 28

Digital dialogue with the GP in Norway

The «Digital dialogue with the general practitioner» implemented in Norway is a suite of four e-health services which enable secure communication between patients and their GP over the Internet. The four digital services include: (1) an electronic booking service to make appointments with the GP; (2) an electronic prescription service to request renewal of maintenance drugs, with direct integration with the electronic prescription system of pharmacies; (3) a service for electronic contact with the GP office for text-based non-clinical inquiries (eg, opening hours and results from diagnostic tests) as an alternative to phone calling; and (4) a service for electronic consultation (e-consultation) with the GP for clinical inquiries. The distinction between e-consultation and electronic contact is that an e-consultation must include a medical evaluation equivalent to a face-to-face consultation.5

The four services were introduced simultaneously in September 2014 and tested by selected GP offices. Following a pilot stage, the «Digital dialogue with the general practitioner» was implemented nationwide in 2016. Its use by GPs is not mandatory. By December 2019, these services were offered via the national health portal helsenorge.no by a total 386 GP offices (out of 1542 offices) which volunteered as early adopters (figure 1). These services are accessible to residents in Norway aged 16 years or older from the private section of the national portal helsenorge.no available after login. The national health portal was established in 2011 to provide health information and accommodate digital health services gathered in one place.29 Secure access is obtained through a unique identification and authentication procedure via a national ID portal with Security Level 4 (the same authentication procedure used for Internet-banking). All the four services are integrated with the GP’s electronic patient record (EPR) system. GPs who adhere to this national initiative and offer these services to their patients are able to receive and send sensitive information and communicate digitally via their own EPR system, and the information exchange is documented in the journal. The services can be activated and deactivated by each GP. This makes it possible to use only some of the four services. The services are supported by the Norwegian Health Network and the EPR suppliers.

Figure 1

GP offices adopting the Digital dialogue with the general practitioner. GP, general practitioner.

The presented suite of e-health services between patients and GPs has the potential to improve the accessibility, quality and efficiency of primary healthcare. According to the Norwegian Directorate of Health, e-consultation and other digital health services with the GP might result in a number of possible benefits for patients including: digital communication with their GP which ensures confidentiality, information security and privacy; time savings (work, travel and waiting); faster detection or treatment of serious diagnoses; possibility to contact their GP at any time, even when on holiday; and easier formulation of a request for health assistance or practical inquiry by written message than by oral communication.30 Despite these expected benefits, there is limited research-based knowledge of the effects of the implementation of the «Digital dialogue with the general practitioner» in Norway. Governments and vendors have been criticised for being overly optimistic about the expected favourable outcomes from employing digital health services,31 and the realisation of these benefits has often been slower than anticipated.32 There is a strong need for those undertaking the implementation of e-health to understand factors that affect implementation.32 However, to date, only few studies have been performed on large-scale implementation of digital health services for citizens in primary care.

Study aim

The aim of this study was to explore patients’ use and experiences with e-consultation and other digital health services with their GP implemented in Norway. A survey consisting of quantitative data supplemented by qualitative information was conducted to explore: (1) the characteristics of the users; (2) the use of e-consultation and other digital health services with the GP; (3) experiences, perceived benefits and satisfaction with e-consultation and other digital health services with the GP; and (4) time spent using e-consultation and other digital health services with the GP compared with conventional methods (eg, phone and office visit).

Methods

Description of the digital health services

The «Digital dialogue with the general practitioner» is a suite of four e-health services which enable secure communication between patients and their GPs over the Internet.

Electronic booking of appointments

The service includes two possible options for electronic booking of appointments. The GP can make time slots available for electronic booking via helsenorge.no. The patient can choose among the time slots available in their GP’s calendar and book an appointment directly through the service. Alternatively, the patient can to send an electronic inquiry for an appointment with a text-based message via the system without selecting a specific time slot. In this case, the GP office will find a free time slot. This can be useful to book an appointment for children or other relatives who are not digitally active. Once the patient has been assigned an appointment, the system sends out a confirmation via short message service (SMS). The text message does not contain sensitive information. Each GP can decide which times slots are made available for electronic booking. The GP can also decide which of the two alternatives to offer to their patients. Normally, appointments can be made available 4 to 5 weeks in advance.

Electronic prescription renewal

Patients can send a message to their GP and ask for a prescription renewal of maintenance medications or medical equipment. The prescription service available via helsenorge.no provides an overview of all prescriptions and which of them are active. The GP office receives a message from the patient in the EPR system containing which prescriptions the patient wishes to renew. It is possible to ask for renewal of multiple prescriptions within one request. GP offices have established routines for which types of medications can be renewed without a face-to-face consultation (eg, by phone), and this also applies to the electronic prescription renewal service. The doctor who approves an electronic prescription renewal must make an individual medical assessment of the validity of the prescription. The most common routine today is to allow prescription renewals without a face-to-face consultation when maintenance medications must be renewed before the patient is due for control (with the exception of certain addictive prescription medications). The inquiry for an electronic prescription renewal contains a free text field with the description of which medications the patient wishes to renew (name, dosage form and dosage) and an additional comment field. The GP can accept or reject the request. In both cases, a confirmation is sent to the patient.

Electronic contact with the GP’s office

This service provides patients with a secure communication channel with the GP’s office. The service can be used for text-based non-clinical inquiries (eg, opening hours and results from diagnostic tests) as an alternative to a phone call to the reception at the GP’s office. The service can also be used to send mail digitally to the patient and can include attachments (PDF, JPG or PNG). The questions are normally answered by the reception at the GP’s office. The service is not intended for providing health assistance. Examples where the electronic contact with the GP’s office can be used include: booking and confirmation (time and location) of patient transport otherwise done by phone; practical short questions (eg, holidays); and practical questions before an appointment or clinical examination. Health personnel can also initiate an electronic dialogue with the patient if the patient is digitally active (marked in the EPR system). The service is free of charge for the patient. In case a patient should improperly ask for health assistance via this service, the GP’s office will ask the patient to initiate an e-consultation instead.

E-consultation

The e-consultation service evaluated in the current study is an online text-based clinical consultation with the GP conducted as alternative to a face-to-face appointment. Patients pay the same out-of-pocket fee as for office visits (NOK 155, approximately € 15, in 2018). To date, this service only applies to known health conditions, and only when physical attendance is not required. Patients need to book a regular appointment if new symptoms or health problems occurs. The service cannot be used for immediate assistance or emergency situations. Information on what e-consultation is suitable for is provided on the national health portal. Examples of situations where e-consultation can be used safely include: follow-up of patients with chronic health conditions (eg, worsening during treatment); follow-up questions about use of medications (eg, compliance, side effects and lack of effect); follow-up of mild mental disorders (eg, events that aggravate anxiety); requests for certificates and statements which do not require re-examination; and use of the written documentation from the e-consultation for referral to a specialist. Examples where use of e-consultation is not recommended include: provision of sick leave certificates; assessment of acute exacerbations; occurrence of new health problems which require a new examination (eg, severe side effects after starting with a new medication); assessment of exacerbations requiring clinical examination (eg, bothersome rashes and wounds, and psychiatric issues which require a dialogue); complex issues which require extensive measures (eg, worsening of chronic illness with need to review medications in case of multimorbidity); and issues unsuccessfully solved in previous e-consultations.

The use of e-consultation does not change ordinary treatment liabilities for the GP. Patients should use the service according to the information provided on the national health portal. The GP must independently assess whether the information provided by the patient is sufficient to be able to deliver proper healthcare. The GP must ask the patient to book an ordinary appointment if in doubt about whether a request can be resolved through an e-consultation. A specific tariff for e-consultation had been introduced since 2016. The e-consultation must include a medical assessment of the patient’s request and is considered completed when the doctor has processed the inquiry and given the patient an answer. The GP is obliged to answer the patient’s inquiry within 5 working days.

Study design

We conducted an online survey of citizens who had activated their personal account at the national health portal helsenorge.no and accessed at least one of the digital health services with the GP by November 2017. The survey was available after secure login on the national health portal helsenorge.no from 14 November 2017 to 28 November 2017. All active users received an invitation through a pop-up window with a brief description of the study and a link to the survey.

The online survey included a total of 29 questions distributed over six pages. All mandatory questions had to be completed before moving to the next page. Questions regarding demographic characteristics of the users and use of the services were presented in two different pages. Moreover, for each of the four e-health services with the GP, respondents were asked about their experiences with the service and the time spent using the service compared with conventional methods (eg, phone and office visit). Demographic characteristics of the users included information on gender, age, education level, health-related background, computer literacy and work status. Use of the services was explored through questions related to which of the four e-health services had been accessed by the respondents and, for electronic booking of appointments only, which of the two possible options was used. Patients’ experiences with the services were evaluated through a number of questions concerning perceived benefits (quality of care, communication with the GP office, formulation of an inquiry and efficiency) and satisfaction (with technology, security, information and educational material, overall satisfaction and future use).

Questions on demographic characteristics and use of the services were multiple choice with a number of alternatives ranging from two to eight depending on the questions. Questions concerning users’ experiences were scored on a 5-point Likert scale (1=strongly agree, 2=agree, 3=neutral, 4=disagree and 5=strongly disagree). Respondents were also able to refrain from providing an answer by selecting ‘no opinion’. Four non-mandatory open-ended questions were also included so that respondents could provide additional feedbacks on their experience with each of the four e-health services.

The online survey was developed by the Norwegian Centre for E-health Research in collaboration with the Centre for Quality Improvement in Medical Practice (SKIL) and the Norwegian Directorate of eHealth. The survey was published on the national health portal helsenorge.no by the Norwegian Directorate of eHealth. The link to the survey was available for a period of 2 weeks. All information collected through the survey was anonymised and not personally identifiable. The Checklist for Reporting Results of Internet E-Surveys (CHERRIES) was used to develop the survey and report its results.33 The online survey was developed with the online data collection solution Questback Essentials (Oslo, Norway) and its technical functionality tested before being published.

Data analysis

Data analysis was performed by the Norwegian Centre for E-health Research from January 2017 to April 2018. All questionnaires received and analysed were completed. Respondents were analysed by gender and age according to the following groups: 16 to 24 years, 25 to 34 years, 35 to 44 years, 45 to 54 years, 55 to 64 years, and over 65 years. Population data for the year 2017 were retrieved from Statistics Norway and used to compare the demographic characteristics of the users of digital health services in primary care with patients attending their GP face-to-face and the general population. Participation and completion rates were not reported as data on unique visitors were not available. The selection of respondents to this survey was assumed to be representative of those who actually used the services. Data on patients’ use and experiences with the service were summarised by descriptive statistics as well as by diagrams. In the analysis of the questions concerning users’ satisfaction with the service, results were summarised by the proportion of respondents who agreed with a certain aspect (scores 1 and 2) and those who disagreed (scores 4 and 5). Differences between time spent using e-consultation and other digital health services with the GP and time spent via conventional methods (eg, phone and office visit) were analysed with the Wilcoxon signed-rank test. A p value <0.05 was considered significant. Data were extracted in Excel and further analysed in IBM SPSS Statistics (V25.0, Armonk, New York: IBM Corp).

Qualitative data provided in the open text fields for each of the four e-health services were used to support and supplement the quantitative data. The content of these answers was analysed and categorised into ‘positive’ (eg, perceived benefits and good user experiences), ‘neutral’ or ‘negative’ (perceived disadvantages, poor user experiences and suggestions for service improvement). Answers categorised as ‘negative’ were further analysed in detail as these were found to be more significant and diverse than those categorised as ‘positive’ (often described by short statements such as ‘working fine for me’). Answers were subject to a content analysis34 and summarised into two levels: (1) common opinions reported by several respondents, and (2) individual opinions containing strong anecdotal experiences.

Patient and public involvement

The four e-health services evaluated in this study were part of a national initiative led by the Norwegian Directorate of eHealth. The services were implemented on a large scale and accessible to residents in Norway from the national health portal helsenorge.no. Patient involvement in the design and conduct of our research was beyond the scope of the study. The results of this study, however, are intended to be disseminated to the public as well as to health authorities to support the further development of these services and their features.

Results

Characteristics of the users

In total, 2043 users answered the survey (table 1). There was a higher proportion of women among users of digital health services in primary care (64.9%) compared with citizens attending their GP face-to-face (59.4%) and the general population (49.8%). Users in all age groups accessed the services. There was a higher proportion of younger users (online supplementary file 1) compared with citizens attending their GP face-to-face (online supplementary file 2) and the general population (online supplementary file 3). Moreover, there were more women among younger users, while there were more men among older users.

Table 1

Demographic characteristics of the users

Users (n)	2043
Gender, n (%)
Male	717 (35.1%)
Female	1326 (64.9%)
Age, n (%)
16 to 24	153 (7.5%)
25 to 34	394 (19.3%)
35 to 44	350 (17.1%)
45 to 54	440 (21.5%)
55 to 64	387 (18.9%)
over 65	319 (15.6%)
Education, n (%)
Primary school / lower secondary school	119 (5.8%)
High school (general)	407 (19.9%)
High school (vocational)	304 (14.9%)
University (3 years)	644 (31.5%)
University (more than 3 years)	569 (27.9 %)
Health-related background, n (%)
Yes	509 (24.9%)
No	1534 (75.1%)
Data literacy, n (%)
Far below average	24 (1.2%)
Below average	86 (4.2%)
Average	984 (48.2%)
Above average	738 (36.1%)
Far above average	211 (10.3%)
Work status, n (%)
Working	1159 (56.7%)
Homemaker	15 (0.7%)
Retired	266 (13.0%)
Unemployed	44 (2.2%)
Student	132 (6.5%)
Sick leave	187 (9.2%)
Disability pension	186 (9.1%)
Other	54 (2.6%)

Over half of the users (59%) had an education at university level or higher. Only 5.8% of the respondents had an education at primary or secondary school level. About one-fourth of the respondents had a background as health professionals. The vast majority of the respondents described their data literacy as average or above average, while only 5.4% of the users had a low data literacy. Over half of the respondents were working at the time they answered to this survey.

Patients’ use of the services

Electronic booking of appointments was the most used of the four digital health services with the GP (66.4%), followed by electronic prescription renewal (54.3%) (table 2). The two other services which implied a dialogue the GP office were used to a lesser degree. Almost half of the respondents accessed only one of the four digital health services, while only 7.6% of the respondents used all the services. Electronic booking via the GP’s calendar was the most commonly used option to book appointments, while sending an electronic inquiry for an appointment with a text-based message was reported as less used. About 20% of the respondents used both solutions.

Table 2

Patients’ use of e-consultation and other digital health services with the GP

Access to services, n (%)
Electronic booking of appointments	1356 (66.4%)
Electronic prescription renewal	1109 (54.3%)
Electronic contact with the GP’s office	528 (25.8%)
E-consultation	762 (37.3%)
Number of services accessed by respondents, n (%)
1	982 (48.1%)
2	566 (27.7%)
3	339 (16.6%)
4	156 (7.6%)
Use of electronic booking of appointments, n (%)
GP’s calendar	793 (58.5%)
Electronic inquiry	294 (21.7%)
Both	269 (19.8%)

GP, general practitioner.

Over 80% of the respondents considered it easier to book an appointment through the electronic service compared with booking via phone or SMS (figure 2). Over half of the users agreed that they could book an appointment at a more appropriate time and within shorter time. Most users agreed that the technology worked well (90%), that the service was safe (93%) and that the information provided on how to use the service was sufficient (81%). Overall, the vast majority of the users were satisfied with the service (90%) and would recommend its use to others (85%).

Figure 2

Patients’ experiences, perceived benefits and satisfaction with the four e-health services. GP, general practitioner.

Over 80% of the respondents agreed that they saved time by booking an appointment electronically (figure 2). Data showed that, while patients used on average 13.5 min (median 10 min) to book an appointment by phone, it took only 4.4 min (median 4 min) to book an appointment electronically via helsenorge.no, meaning a time saving of 9.1 min (−67.4%). The difference was statistically significant (p<0.001).

Over 90% of the respondents thought that it was easier to renew a prescription electronically than by phone (figure 2). Most users (76%) agreed that they obtained a better overview of their medications after using the service, and about half of the respondents (46%) reported that it was easier to follow their doctors’ advice on use of medications. Most users agreed that the technology worked well (93%), that the service was safe (95%) and that the information provided on how to use the service was sufficient (81%). Overall, respondents were highly satisfied with the service (93%) and would recommend its use to others (88%).

Over 90% of the users agreed that they saved time by renewing a prescription electronically (figure 2). Patients saved, on average, 10.5 min (−70.9%) each time they renewed a prescription electronically compared with a renewal by phone (p<0.001). While it took 14.8 min (median 12.5 min) to request to renew a prescription by phone, an electronic prescription renewal via helsenorge.no took only 4.3 min (median 3.5 min).

Over 80% of the respondents agreed that the service allowed for an easier communication with the GP’s office (figure 2). In particular, 60% of the users thought that it was easier to submit an inquiry electronically than by phone. The vast majority of the users agreed that the technology worked well (87%) and that the service was safe (91%). While the majority of the users (69%) was satisfied with the information provided on how to use the service, a higher percentage (11%) compared with the other services thought that the information was not sufficient. Overall, most users were satisfied with the service (82%) and would recommend its use to others (77%).

Three-fourths (76%) of the respondents agreed that they saved time by sending an electronic enquiry to the GP office rather than taking contact by phone (figure 2). Data showed that, while patients used on average 15.2 min (median 12.5) to the GP office by phone, it took only 5.7 min (median 5 min) to send an electronic inquiry via helsenorge.no, with a consequent time saving of 9.5 min (−62.5%). The difference was statistically significant (p<0.001).

Results showed that 72% of the respondents experienced a better follow-up by their GP as a consequence of using e-consultation, and an additional 58% reported improved quality of their treatment (figure 2). While 41% of the respondents agreed that it was easier to explain a clinical problem by written message than by oral communication, 24% expressed their preference towards a traditional face-to-face appointment. The out-of-pocket fee was deemed to be acceptable by 64% of the respondents. Most users agreed that the technology worked well (92%) and that the service was safe (92%). As for electronic contact with the GP’s office, 11% of the users thought that the information provided on how to use the service was not sufficient. Overall, respondents were very satisfied with the service (85%) and would recommend its use to others (81%).

Almost 90% of the users agreed that they saved time by sending a clinical inquiry via the service compared with attending a face-to-face visit. (figure 2). Patients saved, on average, more than 1 hour (72.3 min; −88.5%) each time they used an e-consultation instead of a face-to-face appointment (p<0.001). While it took, on average, 81.7 min (median 60 min) for a face-to-face appointment (including travel time, waiting time and visit time), an online text-based clinical consultation with the GP took only 9.4 min (median 7.5 min).

Qualitative feedback on the services

A total of 656 comments were provided in the open text fields. Most of the comments concerned the electronic booking of appointments, while e-consultation had the lowest number of answers. About half of the comments were categorised as negative feedbacks describing perceived disadvantages, poor user experiences as well as suggestions for service improvement, such as new functionalities. The most commonly occurring responses and some individual opinions were selected for each service (table 3). Moreover, four common themes across services were identified.

Table 3

Most commonly occurring responses and selected anecdotal statements the for the four e-health services

Electronic booking of appointments
Common responses	Selected anecdotal responses
Want to be able to book urgent appointments	The availability of bookable slots is poor
Want the possibility to book appointment for own children	Want the possibility to cancel appointments
Want to be able to attach a comment to the appointment enquiry	Want a mobile application with push notifications
Electronic prescription renewal
Common responses	Selected anecdotal responses
Want a receipt that the enquiry is sent and estimated time to answer	Hard to spell the medical names correctly
Want a list of expired prescriptions with a ‘renew’ button	Risk to mix up prescriptions (eg, melting tablets and debot tablets)
Want the possibility to manage own children’s prescriptions	Poor readability when lists are long and complicated
Electronic contact with the general practitioner’s office
Common responses	Selected anecdotal responses
Want a receipt that the message is read	Easier to call by phone than to write
Service can be confused with e-consultation	Uncertain about who reads the message
Service not easy to navigate	Want to send a message on behalf of own children
E-consultation
Common responses	Selected anecdotal responses
Unreasonable that out-of-pocket payment is charged	Want a receipt that the message is read and estimated time to answer
The present limit of 1000 characters for messages is too short	Want the possibility to write on behalf of own children
Want an autosave function so that the text is not lost while writing	Written communication is not suited for clinical contact

Safety and security

Respondents pointed out that they were uncertain about who read the information they submitted (eg, receptionist or GP). Moreover, they indicated the need for a confirmation that the request was sent and received by the GP’s office, and information on when they could expect an answer.

User friendliness

Several respondents pointed out that the interface was generally slow and little intuitive, and that it worked poorly on certain web browsers, operating systems and devices.

Time utilisation

Some respondents indicated that the time elapsed from when they contacted the GP’s office until they received an answer was perceived as more important than the time spent in using the service, depending on the nature of the problem, occupational status and personal characteristics. GP offices tend to keep a few slots daily available for acute visits, which can only be booked by phone. On the one hand, if patients wanted to visit their GP as soon as possible, they might prefer to book an appointment by phone. On the other hand, full-time workers who do not have urgent issues would value the possibility of asynchronous communication and spend the least amount of time.

Functionality for parents

Many of the respondents pointed out the lack of a functionality to manage their children’s medical contact through the services.

Discussion

The «Digital dialogue with the general practitioner» introduced in Norway since 2016 is one of the first examples of nationwide implementation of digital health services for citizens in primary care. The current study provides evidence on patients’ use and experiences with e-consultation and other digital health services with their GP. Overall, the services have been used by early adopters (approximately 25% of all GP offices) and the trend over the first years of implementation shows a steadily growing nationwide adoption (figure 1). According to the technology adoption curve,35 the majority of users is expected to adopt the services within the next few years. From January 2018 to October 2018, the use of e-consultations in Norway grew from 0.8% to 2.2% of the total number of consultations with the GP,36 thus exceeding levels reported by other studies. In a retrospective observational study, even though email consultation was adopted by 52.8% of the general practices in the Netherlands in 2014, only 0.7% of the GP consultations were conducted by email.14 Future use of e-consultations in Norway is estimated to account to 30% of all consultations with the GP,36 and up to 40% if supported by apps and wearables.37 Similarly, regions in Denmark have recently set a political ambition that one-third of all consultations with the GP will be digital.38 While electronic booking of appointments and electronic prescription renewal seem to be widely used by patients in Norway, the e-consultation service for medical inquiries with the GP office is still used at a lower degree, as indicated in a recent qualitative study.39 Our findings are in line with other studies where the use of e-consultation was low compared with use of electronic communication for administrative requests, such as repeat prescriptions and test results.7 19 24 In the UK, the low uptake of alternatives to face-to-face consultations might be explained by the fact that users are still early adopters.2

Time saving (work, travel and waiting) represents the most evident benefit for patients. In a study on use of e-mail communication with the GP, 95% of the users perceived it as more efficient than the phone.8 In the current survey, the majority of patients (85%, 92%, 76% and 89% for electronic booking of appointments, electronic prescription renewal, electronic contact with the GP office and e-consultation, respectively) agreed that the services were time saving. This was confirmed by the differences in time spent using the digital health services compared with conventional approaches, all found to be statistically significant. The highest efficiency (−88.5%) was estimated for e-consultations compared with face-to-face appointments. However, some users indicated that the time elapsed from when they contacted the GP office until they received an answer was more important than the time spent in using the service. Response time is recognised as an important factor in the delivery of digital health services in primary care. Findings from previous studies reported that the majority of patients received a response within 2 days6 24 and that a slow response was the main reason for dissatisfaction.6 Despite e-consultations implemented in Norway are used for non-urgent health issues only, and the GP is obliged to answer the patient’s inquiry within 5 working days, quicker response times might further improve patient satisfaction.

Most users seemed to acknowledge the practical utility of digital services with their GP. Patients could easily and efficiently book an appointment electronically, at a time more appropriate to them and with a shorter waiting time compared with booking via phone. Electronic prescription renewals were also preferred to renewals made at the GP office. Patients also recognised that they obtained a better overview of their medications and even a higher compliance. For non-clinical inquiries, most respondents thought that it was easier to write electronic messages to the GP’s office than communicate by phone. For clinical enquiries, many patients agreed that use of e-consultation could lead to a better follow-up and even to improved quality of treatment, as suggested by other studies.8 These were, however, perceived benefits. More systematic research is needed to measure objectively clinical and other outcomes of interest, including cost-effectiveness and health service resource use.13 Users seemed to be generally satisfied with their ability to explain a problem via e-consultation. Compared with oral communication, written communication has been considered more intimate,40 as patients can feel more emboldened to ask questions electronically.8 Moreover, electronic messages can support patients aiding recall and providing evidence of the exchange.8 9 However, it is important that messages are concise, formal and medically relevant.7

The overall satisfaction expressed by the respondents of this survey with e-consultation and digital health services with their GP was very high. Such result is not surprising and confirms that patients have a positive attitude towards e-health services in primary care.19 Electronic communication with the GP’s office has been considered convenient,2 10 26 40 41 appropriate,7 22 accessible4 6 40 and easy to use4 6 by patients. Moreover, despite providers’ reluctance1 and concerns about patients’ inappropriate and inefficient use of the technology,7 patients find the electronic communication with their GPs efficient compared with phone or face-to-face contacts.2 6 8 The results from our survey indicated that the information provided to patients on how to use the services was generally satisfactory, but could be improved. GPs previously reported that the service designed for administrative electronic communication with the GP’s office was sometimes confused by patients with the medical e-consultation service.39 Such confusion was also expressed by a number of respondents to this survey. Improved patient education is needed to avoid improper use and inefficiency.8 Other suggestions were provided in the open text fields. Some users expressed their wish to use the services on behalf of their children. Furthermore, the need for a receipt (eg, that the request has been successfully delivered, read by the receiver or being handled) was in demand. As these services were new, the frustration caused by technical issues could make some people who try them for the first time to go back to the traditional alternatives. It is therefore important to make the services functional on all platforms and easily accessible to all users, as well as provide adequate training.42

The results from this survey confirm that users of digital health services in primary care are more likely to be women21 24 36 43 and younger adults.2 5 8 17 21 36 43 These services seem to be more attractive to digitally active users with a higher education, as also reported by other studies.8 Over half of the users (59%) had an education at university level, which is high compared with the general population and those accessing their GP face-to-face. Data from Statistics Norway show that, in 2017, 33.4% of the total population aged over 16 in Norway had an education at university level. Moreover, the vast majority of the respondents (94.6%) described their data literacy as average or above average. Consequently, elderly and people with low computer literacy might still need traditional alternatives.38 Despite these digital health services currently catering to competent health users, less competent users would still benefit indirectly if such services succeed in freeing up resources in primary care. Reduced phone load, increased efficiency, released time for medical assessments and less crowded waiting rooms are, for example, advantages for GP offices which have been acknowledged in a recent study conducted on GP’s perceptions towards the use of «Digital dialogue with the General Practitioner».39 In addition to the day-to-day efficiency gains, it is also possible to envision that a robust and well-established suite of services for electronic communication can be a useful tool for managing situations where in-person attendance at the GP’s office is less desired, or when capacity needs to be reserved for critical cases, such as during pandemic outbreaks or large-scale emergencies.

Study strengths and limitations

GPs and citizens using digital health services in Norway are early adopters who might have a more positive attitude towards innovation than the general population and thus be more enthusiastic and inclined to use the services. Moreover, respondents might overall have a better level of satisfaction than non-respondents. The information regarding the number of unique users of the services was not available due to privacy issues related to the national platform. As a consequence, the potential number of patients who were eligible for this survey was unknown and it was not possible to calculate a response rate. Despite this, the high number of respondents in this survey is considered to be representative of those who used the services.

The comments provided in the open text fields were optional, and thus the qualitative results only reflect the opinion of a minority of users who were sufficiently motivated to provide a feedback. It is also commonly observed that responses in open text fields can have a negativity bias compared with structured surveys.44 However, the open text responses in this study provided interesting insights and useful suggestions for improvement.

Conclusion

Citizens using e-consultation and other digital health services with their GP in Norway are highly satisfied and consider them as useful and efficient alternatives to conventional approaches. These digital health services are currently catering to competent health users, mostly women, younger adults and digitally active citizens with high education. It is important to make the services functional on all platforms and provide adequate information and training so that they become easily accessible to all users, including citizens who are not digitally active.