A retrospective study on how primary care providers manage specialists' recommendations after an e-consultation.

INTRODUCTION: E-consultations are asynchronous text-based consultations between providers which can facilitate patient access to timely specialty care. In contrast to traditional face-to-face consults, conveying and completing recommendations of the specialist is the responsibility of the referring provider. This presents a new workflow for primary care providers who have multiple options (face-to-face, telephone, letter, secure message) to communicate the e-consultation recommendations. This study examines how primary care providers are managing this new workflow.
METHODS: We performed a retrospective random sampling of e-consultations with individual medical record review and classified e-consultations by type of recommendation, how recommendations were communicated to patients, and whether recommendations were carried out.
RESULTS: We randomly selected 220 e-consultations in 13 different specialties for review. In all, 85% of e-consultations contained recommendations for referring providers. Recommendations on medication(s) were most common (35%) followed by recommendations on ordering laboratory tests (29%). In all, 25% of the time e-consultants gave multiple possible courses of action for referring providers to choose from. Patient notification of recommendations was found for 192 (87%) of e-consultations with providers performing the notification 63% of the time and nursing staff performing the notification 37% of the time. The communication back to the patients included communication via nurse telephone calls (37%), provider telephone calls (23%), secure messages (24%), face-to-face visits (11%), and by written correspondence (5%). DISCUSSION: Managing recommendations from e-consultations results in a new workflow for primary care providers. Healthcare institutions that utilize e-consults should be aware of this new workflow. Further study is needed to determine best practices for this task that is now increasing in primary care.

Introduction

E-consultations are asynchronous, text-based consults that are performed via an electronic health record (EHR) or web-based platform. E-consultations have been found to have many benefits including improving access to care in areas with limited specialty access[1-3] and decreasing wait time to obtain input from a specialist.[4,5] Referring providers, in general, report satisfaction with e-consultations and feel they benefit patient care.[1,6,7]

In many cases, e-consultations can obviate the need for a face-to-face (f2f) visit with a specialist;[4,7] less than 10%[1] to 20%[8] of e-consultations subsequently have an f2f visit with a specialist in the same specialty. As a result, e-consultations are being increasingly used in some institutions, notably with Mayo Clinic tripling e-consultations from 6 per 1000 primary care patients to 18 per 1000 in just 3 years.[9]

E-consultations can be used internally within an institution to refer patients from a generalist to a specialist or between different specialists in the same institution (i.e. an internal e-consultation).[10] Internal e-consultations eliminate some of the barriers associated with e-consultations. For example, transfer of medical records from referring physician to specialist is not needed when they share the same EHR system. At Mayo Clinic, a variety of specialists are available by internal e-consultation. These internal e-consultations are requested by a referring provider entering an order through the same ordering process that an f2f visit would be ordered. The specialist then reviews the referring provider’s clinical question along with any pertinent information in the shared EHR and renders a written e-consultation viewable in the EHR. It is then the responsibility of the referring provider to discuss the recommendations with the patient, implement any recommendations, order any recommended tests, and follow up on results of those tests. Because care coordination remains the responsibility of the referring provider, e-consultations may contribute to continuity of care.[11] However, with the referring provider reviewing, explaining, and implementing the specialist’s recommendations, this also shifts some of the work from the specialist provider (in the traditional f2f specialty visit) to the referring provider. As a result, e-consultations can put more responsibility on the referring provider.

Early internal e-consultation studies by Angstman et al.[12] found that 17% of surveyed family medicine providers thought that e-consultations caused more work, 28% were neutral, and 55% did not think e-consultations created more work for the referring provider. Additionally, a study by Chaudry et al.[13] found that 51% of primary care providers (PCPs) surveyed felt e-consultations required more work for them and 47% felt that they needed dedicated time to convey the results and recommendations of the e-consultation to the patient.

To evaluate the workflow shifted to the primary care team as a result of e-consultation, we examined e-consultations requested by PCPs for type of recommendation made, method of communicating recommendations to patients, and the extent to which PCPs completed specialist recommendations.

Methods

Setting

Mayo Clinic is a large multispecialty group practice in the United States. Our study took place at the Mayo Clinic, Rochester, Minnesota campus.

All Mayo Clinic physicians on the Rochester campus are salaried. Specialists have no additional financial incentives to perform e-consultations. Each specialty determines how their e-consultants incorporate e-consultations into their workflow. Referring providers within Mayo Clinic also have no additional financial or other incentive to initiate an e-consultation referral. Follow-up work after the specialist completes the e-consultation is likewise not compensated.

E-consultation sample and review

A list of all internal e-consultations that were ordered from 1 January 2012 through 30 June 2013 by PCPs in the specialties of family medicine, primary care internal medicine, and general pediatrics was obtained. From this list, we selected the 13 specialties that comprised the highest number of e-consults. For these 13 specialties, we selected a subset of e-consultations for manual record review by randomly selecting 20 e-consultations from each specialty that comprised 5% or more of total e-consultations (psychiatry, spine clinic, gastroenterology, endocrinology, hematology, cardiology, neurology, nephrology, and pediatric subspecialties) and 10 e-consultations from specialties that comprised 2% to 4.9% of the total (obstetrics-gynecology, pulmonology, rheumatology, and infectious disease). Specialties that comprised less than 2% of total e-consultations were not reviewed. The patients whose EHRs were manually reviewed had given prior research authorization.

E-consultations were reviewed for the type of recommendations made by the e-consulting specialist. E-consultations could code for more than one type of recommendation. Specialist recommendations were characterized into the following categories:

Laboratory recommendations.

Imaging recommendations.

Procedure recommendations.

Medication recommendations (i.e. recommendations on dose adjustment or discontinuation of current medication(s) and/or implementation of new medications).

Physical therapy recommendations.

Recommendations for a patient to be referred to a specialist for an f2f visit. Coded as:

No recommendations for a specialty f2f visit;

Explicit recommendations for specialty f2f visit (i.e. e-consultant recommends patient should be referred for specialty f2f visit);

Conditional recommendations for an f2f visit (i.e. e-consultant recommends a specific test but recommends a specialty f2f visit only if the test result is abnormal).

Surveillance recommendations for ongoing care (i.e. when to proceed with next surveillance colonoscopy, imaging, or lab work).

Multiple possible options for proceeding in the same category (e.g.: “you could start medication 1, medication 2 or medication 3 would be reasonable also”). This category was included as e-consultants giving multiple medication or test options may add to the workflow of the referring provider by requiring providers to decide which option to recommend to the patient. Alternatively discussing the pros and cons of different options of care could increase the amount of work on the referring provider.

“Other” recommendations (e.g. integrative practices, counseling, lifestyle changes, nutrition referral).

No recommendations (reassurance that no further evaluation, treatment, or follow-up was needed).

Patient records were reviewed for the following outcomes:

Patient notification within 3 months of e-consultation recommendations. Coded as present or absent; if present, then both the method of notification (e.g. telephone, secure message) and notifying provider type were categorized and recorded.

Completion of recommendations by the referring provider within 3 months: complete, partial, none, not applicable (i.e. no recommendations were made by the e-consultant). We arbitrarily chose a 3-month follow-up interval for the first and second outcomes to allow adequate time for patients to have an f2f visit with their PCP at which time e-consultation recommendations could be discussed.

f2f visit with the e-consulting specialty within 6 months of e-consultation (to determine whether referring providers were ordering f2f visits when recommended by specialists).

The first author (J.L.P.) reviewed all 220 records for all data, F.N. reviewed 100 records for patient notification and whether recommendations were followed, and J.M.F. reviewed 100 records for type of recommendations made. Two reviewers examined each record. Agreement was adjudicated by J.L.P.

Statistical analysis

The significance of categorical differences in notification method and notifying team member was determined by the chi-square test. We used Cohen’s Kappa to assess interrater agreement between reviewers. JMP version 10.0 (SAS Institute Inc, Cary, NC) was used for statistical analysis.

Results

Between 1 January 2012 and 30 June 2013, 1041 e-consultations from 23 different specialties were completed by Mayo Clinic specialists in response to requests from local referring PCPs. Table 1 shows the distribution of e-consultations that comprised 2% or more of e-consultations. The distribution of referring PCPs was family medicine 142 (64%), primary care internal medicine 59 (27%), and general pediatrics 19 (9%). Median time for e-consultation completion was 1 day (interquartile range (IQR), 1–4 days).

Table 1.

Distribution of e-consultations by responding specialty (specialties comprising <2% not shown).

E-consultation specialty (N = 1041)	Number of e-consultations (% of total)
Spine Center	147 (14)
Gastroenterology & Hepatology	145 (14)
Endocrinology	120 (11)
Cardiovascular Diseases	115 (11)
Neurology	75 (7)
Hematology	68 (6)
Pediatric and Adolescent Medicine	66 (6)
Nephrology & Hypertension	61 (6)
Psychiatry & Psychology	50 (5)
Obstetrics and Gynecology	38 (4)
Pulmonary & Critical Care Medicine	38 (4)
Rheumatology	29 (3)
Infectious Diseases	26 (3)
Other specialties	63 (6)

Categories of e-consultant recommendations are listed in Table 2. Recommendations regarding medications were most common, followed closely by laboratory recommendations. E-consultations giving multiple possible courses of action were common, occurring in approximately one-fourth of the e-consultations reviewed. An f2f specialty visit was explicitly recommended in 38 (17%) and conditionally recommended in 48 (22%) of e-consultations. In all, 53 (24%) patients had a specialty f2f visit in the same specialty as their e-consultation within 6 months of their e-consultation. Out of these 53 patients, 12 had no recommendations for an f2f visit at the e-consultation, 31 had explicit recommendations for an f2f visit, and 10 were conditionally recommended to have an f2f visit by the e-consultant.

Table 2.

Recommendation categories of 220 e-consultations.

E-consultation recommendations (N = 220)	n (%)
Lab	64 (29)
Imaging	27 (12)
Procedure	29 (13)
Medication	78 (35)
Physical therapy	12 (5)
F2F explicitly recommended	38 (17)
F2F conditionally recommended	48 (22)
Test surveillance type/interval	39 (18)
More than one option of proceeding	56 (25)
Other	31 (14)
No recommendations/reassurance	33 (15)

F2F: face-to-face.

Percentages may add up to more than 100% as recommendations could be coded into more than one category.

Documentation of patient notification of e-consultation recommendations was identified in the medical record for 192 (87%) e-consultations. Median time from e-consultation completion to notification of the patient was 3 days (IQR, 1–6 days). Of the 192 e-consultations where notification was documented, providers notified the patient of the e-consultation recommendations 63% of the time with nursing staff performing the notification 37% of the time. Notification methods and frequencies are shown in Table 3. Contact by telephone, followed by secure messaging on the patient portal, were the most common methods of notification; however, the patient portal was used primarily by providers as a method of notification. There was only one instance of nursing staff using the portal to notify a patient of recommendations. Notification method and notifying team member by category of e-consultation recommendation are shown in Tables 4 and 5, respectively. There were no statistically significant differences in relative proportions of how the patient was notified (e.g. phone, portal, f2f visit, letter) based on the type of e-consultation recommendation. There was one statistically significant difference in who notified the patient (Table 5). For the recommendation of an f2f specialist visit, there was a higher percentage of patients that had no documented notification. When e-consultations with no documented patient notification are excluded, there are no statistically significant differences between different recommendation types and relative proportions of who notified the patient.

Table 3.

Method of communication used to notify patient of e-consultation results and recommendations; total and by notifier.

Method used for notification (N = 192)	All, n (%)	Provider notified, n (%)	Nurse/assistant notified, n (%)
Telephone	115 (59)	44 (23)	71 (37)
Patient portal (secure message)	46 (24)	45 (23)	1 (0.5)
Face-to-face visit with referring provider	22 (11)	22 (11)	None
Letter (by mail)	9 (5)	9 (5)	None

Table 4.

Method of notification by category of e-consultation recommendation.

E-consultation recommendations (N = 220)	Notified phone, n (%)	Notified by portal, n (%)	Notified at F2F visit, n (%)	Notified by letter, n (%)	No documented notification, n (%)	p value
Lab	33 (52)	14 (22)	7 (11)	1 (2)	9 (14)	0.79
Imaging	19 (70)	2 (8)	1 (4)	1 (4)	4 (12)	0.20*
Procedure	15 (52)	7 (24)	2 (7)	2 (7)	3 (10)	0.86*
Medication	44 (56)	13 (17)	8 (10)	4 (5)	9 (11)	0.75
Physical therapy	8 (67)	1 (8)	1 (8)	1 (8)	1 (8)	0.69*
F2F explicitly recommended	17 (45)	7 (18)	4 (11)	0	10 (26)	0.05*
F2F conditionally recommended	30 (62)	8 (17)	3 (6)	0	7 (15)	0.25*
Test surveillance type/interval	17 (44)	13 (33)	1 (3)	3 (8)	5 (13)	0.08*
More than one option of proceeding	30 (54)	10 (18)	7 (12)	3 (5)	6 (11)	0.85
Other	17 (55)	5 (16)	4 (13)	2 (6)	3 (10)	0.83*
No recommendations/reassurance	17 (51)	19 (30)	1 (3)	2 (6)	3 (9)	0.37*

F2F: face-to-face.

20% of cells have expected count less than 5, chi-square suspect.

Table 5.

Notifying team member by category of e-consultation recommendation.

E-consultation recommendations (N = 220)	Notified by provider, n (%)	Notified by nursing staff, n (%)	No documented notification, n (%)	p value
Lab	41 (64)	15 (23)	8 (12)	0.15
Imaging	17 (63)	7 (26)	3 (11)	0.66
Procedure	18 (62)	8 (28)	3 (10)	0.71
Medication	41 (53)	28 (35)	9 (11)	0.76
Physical therapy	5 (42)	6 (50)	1 (8)	0.43*
F2F explicitly recommended	19 (50)	9 (24)	10 (26)	0.01
F2F conditionally recommended	25 (52)	16 (33)	7 (15)	0.83
Test surveillance type/interval	26 (67)	9 (23)	4 (10)	0.26
More than one option of proceeding	32 (57)	19 (34)	5 (9)	0.68
Other	16 (52)	12 (39)	3 (9)	0.72
No recommendations/reassurance	17 (51)	13 (39)	3 (11)	0.63

F2F: face-to-face.

20% of cells have expected count less than 5, chi-square suspect.

Table 6 summarizes what the referring provider did with the e-consultation recommendations. In 78% there was documentation of completion of all the recommendations and in 10% there was documentation of partial completion of the recommendations. There were 28 e-consultations where no indication of patient notification was evident in the EHR. Out of these 28 e-consultations, 3 (11%) did not have any recommendations made by the e-consulting specialist. Out of the 25 e-consultations where the e-consultant made recommendations, 14 (56%) had all of the suggested recommendations completed (i.e. all tests ordered), 9 (36%) had none of the recommendations followed, and 2 (8%) had some of the recommendations followed despite lack of documentation that notification occurred.

Table 6.

Recommendation completion status of e-consultations where recommendations were made.

E-consultation recommendations completed (N = 187)	n (%)
All	146 (78)
None	19 (10)
Partial	11 (6)
Partial: patient declined some	8 (4)
None: patient declined all	3 (2)

There was no difference in completion of all recommendations based on who notified the patient of the e-consultation recommendations with 82% of those notified by the provider having all recommendations completed versus 77% of those notified by nursing staff (p = 0.2).

Kappa agreement statistics ranged from 0.65 to 1.0 with the exception of the “other” category for recommendations where the kappa agreement statistic was 0.45.

Discussion

Our study shows that the majority of e-consultations contained recommendations for further evaluation and/or treatment resulting in a qualitative and quantitative change in workflow for PCPs. From a quantitative perspective, PCPs made medication recommendations and new laboratory and imaging recommendations based on e-consultations. This represented work that specialists would have done had the PCPs ordered an f2f consultation. Qualitatively, our study showed that PCPs utilized several different workflows for communicating e-consultation recommendations. Over half of the notifications performed were done by the provider themselves by phone, via the patient portal, an f2f visit, or a letter. In our review, providers chose to do the notification primarily using the phone or the patient portal. However, in 10% of records reviewed, providers gave notification of the recommendations at an f2f visit with the patient. Our study design did not allow us to delineate what percentage of these f2f visits with the referring provider was for the sole purpose of discussing the e-consultation recommendations (i.e. f2f visits which may not otherwise have occurred). A previous study at our institution revealed that patients who had an e-consultation with a specialist were more likely to follow up with the PCP within 2 weeks when compared to patients who had an f2f visit with a specialist, suggesting that some of these f2f visits may have been directly related to the clinical issue covered in the e-consultation.[14]

Recommendations regarding medications (starting, stopping, or changing dose) were the most common recommendations seen and were present in one-third of e-consultations we reviewed. Although patient counseling for discontinuing a medication often may be accomplished fairly quickly, medication prescribing, because it involves patient counseling and monitoring,[15] has the potential to involve significant time for the provider and/or their care team. Additionally, in 25% of the e-consultations we reviewed, the specialist gave multiple possible options for different courses of action to choose among. The multiple options likely reflect the decision-making process that a specialist would go through if seeing the patient for an f2f visit. However, for the referring provider, sorting through and selecting options could require changes in workflow as the provider would need to evaluate each option in the context of the specific patient and then discuss the pros and cons of the options with the patient.

We found that 6% of the time the patient declined some or all of the recommendations. An additional 16% of the time recommendations were only partially followed or not followed at all. As there was no documentation in these cases of why the recommendations were not followed, we do not know if this was based on patient or provider (or both) decision. For those explicitly recommended an f2f specialty visit, in 18% we could not find the recommended visit in the EHR within 6 months after the e-consultation. These findings need further examination to determine the conditions in which patients and providers decline the specialists’ recommendations.

Limitations of our study include the retrospective nature allowing only data that were in the EHR for review. It is possible, and in fact likely, that some notification of patients occurred that was not recorded in the EHR. Supporting this contention is that some patients without any evidence for notification of e-consultation recommendations still had all recommendations completed. Although the goal of our study was to explore potential workflow changes of the referring provider, our study did not review how many of the recommended laboratory results or tests that were performed at the recommendation of the e-consultant were subsequently abnormal and thus may have required additional ongoing work for the referring provider. Our retrospective study design did not allow us to measure the time spent by referring providers and care team members to complete notifications and recommendations. Murphy et al.[16] reviewed categories of EHR alerts received by primary care physicians in a day and found that test result alerts comprised over half of daily alerts with each alert taking from just under half a minute to almost 2 min to process. In a simulation model reviewing e-consultation “cycling time” (the time between initiation of an e-consultation and the completion of treatment recommendations), Zoll et al.[17] found that the cycling time was increased when PCPs had to respond to more alerts. Thus, additional time would be expected both for reviewing and notifying e-consultation recommendations as well as for responding to the results of recommended tests. Finally, this study was confined to a single medical system with a shared EHR where all providers are salaried, so our results may not be generalizable to other systems.

Our study has potentially important implications for health care systems and payers who support e-consultations. It provides evidence that there is a change in workflow for the referring provider when a patient is referred for an e-consultation. To accommodate this change in workflow, referring providers may require incentives (either in time allotted for addressing e-consultation recommendations or additional pay for coordinating care).

Future studies will need to examine the overall referring provider time required in e-consultations as we have yet to fully understand all the time issues involved in e-consultations. While this study advances our understanding of the types of work needed from the referring provider, we still need to quantify this work and examine all of the requesting provider and responding specialist factors involved in completing an e-consultation. To fully understand the potential benefits of e-consultations, we need to know not only how long it takes the specialist to do the e-consultation but also the actual time it takes the referring provider to formulate the question, digest the recommendations, contact the patient with the recommendation, explain it, and order additional tests and follow up as required. In addition, our study showed that a number of different kinds of workflows are being used to communicate e-consultation recommendations. Future work should investigate this variation.