Evaluating the primary care clinical pharmacist visit transition to telehealth during the COVID-19 pandemic by comparing medication related problems from telehealth visits and in-person visits.

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic forced healthcare systems to rethink healthcare delivery, and forced primary care pharmacists in our healthcare system to switch all visits that were previously face to face (FTF) to telehealth.
METHODS: We conducted a retrospective observational cohort study to examine the association between medication related problems (MRPs) resolved in telehealth vs FTF primary care clinical pharmacist visits. The telehealth visits took place in the context of the COVID-19 pandemic, which forced health care systems to rethink care delivery. Data was collected for patient visits for 2 weeks in January before the pandemic and 2 weeks in June during the pandemic.
RESULTS: There was significantly more average MRPs resolved per patient encounter in FTF visits compared with telehealth visits, particularly in patient encounters that were previously seen by the pharmacist, who were under 65 years old, identified as Black/African American, had chronic kidney disease but not on dialysis, diabetes with end organ damage, and had uncontrolled blood pressure and uncontrolled A1c.
CONCLUSION: These results provide a start to establish criteria for which patients should be seen by a clinical pharmacist in person vs over the phone.

INTRODUCTION

The coronavirus disease 2019 (COVID‐19) pandemic has forced health care systems to rethink care delivery. The Centers for Disease Control and Prevention and Health Resources and Services Administration have established guidance and a list of toolkits for telehealth in order to protect the safety of health care workers and patients. Telehealth has been shown to satisfy the quadruple aim, with evidence in patient satisfaction, outcomes,, , cost,, and provider experience. Potential problems with telehealth include limited ability to perform physical assessments or vitals, potential privacy issues, reimbursement, and failures in technical equipment.,

Many primary care clinical pharmacy practices are inherently set up using telehealth, such as Kaiser, the Veterans Health Administration, and rural health outreach. These systems have provided providers and patients access to adequate technology to conduct video visits. The benefit of pharmacists in telehealth primary care management of chronic disease has been documented in improving hypertension,, diabetes,, , asthma, anticoagulation, human immunodeficiency virus (HIV), and hepatitis C., ,

An American College of Clinical Pharmacy White Paper published in 2018 outlines guidance on implementing comprehensive medication management (CMM) virtually. Telehealth is defined as using technology to deliver health care, health information, or health education at a distance. Telehealth visit formats must take into account maintaining patient safety and privacy, protecting the patient‐pharmacist relationship, and enhancing communication and coordination. Additionally, the patient and the pharmacists have several technology requirements. There are no studies comparing the number of medication related problems (MRPs) resolved by pharmacist services between telehealth and in‐person visits within the same practice, when a pandemic created the need to switch all visits that were previously face to face (FTF) to telehealth. MRPs are well‐established measurements for pharmacists' unique impact on patient care within a team‐based model. This study evaluates the impact of clinical pharmacist care via in‐person and telehealth by comparing the average MRPs resolved during the visits.

METHODS

San Francisco Health Network (SFHN) is the city's only integrated safety net delivery system (federally qualified health center) operating under the city's department of public health (DPH). All DPH staff are considered disaster service workers that can be deployed to essential services in emergency situations such as pandemics and other natural disasters. SFHN consists of ambulatory care clinics, mental health facilities, and two hospitals. The 12 adult primary care clinics treat 90 000 publicly insured or uninsured individuals. Fourteen primary care pharmacists practicing in all of these clinics provide medication management under a collaborative practice agreement that allows them to prescribe medications, order labs, and make referrals. Prior to the pandemic, over 4500 patients were seen each year in FTF visits by the primary care clinical pharmacists. As with other health care professionals, the COVID‐19 pandemic has required the clinical pharmacists with SFHN to provide care to patients largely virtually.

CMM services are the professional activities needed to meet the standard of care. Patients are seen by the primary care clinical pharmacist, MRPs are identified and resolved, and then scheduled for follow up with the pharmacist for on‐going monitoring and medication titration, with the objective of meeting chronic disease patient‐centered goals. All patients seen by primary care pharmacists are over the age of 18 and are referred to primary care pharmacist visits generally because they have one or more uncontrolled chronic conditions, are on treatments that require close monitoring, have complicated medication regimens, or have difficulty understanding or taking their medications.

Overview of design

This is a retrospective observational cohort study designed to examine the association between MRPs resolved in telehealth vs FTF primary care clinical pharmacist visits. Patient encounters included in our study were those seen by primary care pharmacists within 9 primary care clinics in the San Francisco Health Network over the course of 2 weeks in January 2020 and 2 weeks in June 2020. Only 9 out of the 12 clinics were included in this evaluation because they have the most consistent CMM primary care practices, whereas other clinics may have more specialty or disease‐specific practices (anticoagulation, infectious disease, etc.). January 2020 was chosen because it was a pre‐pandemic month with fully staffed clinical coverage. June 2020 was selected because it was the first full month that telehealth was fully rolled out network‐wide. Visits in January 2020 were all FTF encounters, and visits in June 2020 included both telehealth and FTF encounters.

The primary predictor was telehealth or FTF encounter, and the outcome was the average number of MRPs per encounter. MRPs were collected by a researcher outside of our practice through a retrospective manual chart review using the data collection form (see key in Supporting Information). Patient characteristics were collected as well as number and types of MRPs resolved. All telehealth encounters were done as scheduled telephone visits, as access to video technology was not available. Although the determination criteria for telehealth vs in‐person visits was not clearly defined because of the abrupt switch to telehealth visits, the general guiding principle was patients would be seen via telehealth if no physical assessment or vitals or labs was absolutely needed. All components of CMM were done virtually including gathering home health data such as home blood sugar and blood pressure (BP) readings, but physical exams and vitals were not able to be done. MRPs were categorized by type (needs additional drug therapy, unnecessary drug therapy, different drug needed, dose too low, adverse drug reaction, drug interaction, dose too high, non‐adherence, lab monitoring needed). In order to identify the unique role that clinical pharmacists play in the primary care team, education MRPs were not included. MRPs were all identified by retrospective review of pharmacist chart notes, and only MRPs that were completed in that encounter were entered into the data collection form. For example, a lab ordered in the encounter was included but a consideration for a lab in 3 months was not included. MRPs were collected by encounter as opposed to aggregated per patient because a patient may have been seen FTF or telehealth between the two time periods. This analysis aims to consider the specific encounter. Of note, MRPs are not routinely tracked in current practice, but discussions are in place to implement measures to do so.

Additional data points collected include clinic location, follow up vs initial visit, age, gender, ethnicity/race, history of hospitalizations or emergency department (ED) visits in the past year, homelessness, number of medications, presence of high risk medications, last BP within the past year, last A1c within the past year, and components of the Charleson Comorbidity Index. See Supporting Information for the data collection form key, which has additional clarifications on how ambiguous data was collected.

To understand the overarching pharmacist visit patient encounter landscape before and after the pandemic, an analysis of show rates between January vs June, show rates between telehealth vs FTF encounters in June, and initial vs follow‐up encounters was done using chi‐square tests. A bivariate analysis was done using student t tests to compare the number of MRPs in telehealth encounters vs FTF encounters for the primary outcome. A subgroup analysis was done to compare if there were differences in encounter‐specific patient characteristics in MRPs for telehealth vs FTF encounters for the following variables: initial vs follow up visit, age (< or ≥65 years old), gender, race, number of hospitalizations or ED visits in the past year, housing status, number of medications (< or ≥7), taking high risk medications, and Charleson comorbidity index components. Continuous variables were separated into categorical variables, and student t tests were used.

RESULTS

Data was collected for patient visits between January 6‐17, 2020 and June 15‐26, 2020. Baseline characteristics of patients by encounter are listed in Table 1. The patient population in this evaluation are older, diverse, and medically complex, with an average age of 62.52 (±12.45) years old, mostly non‐white (87.4%), and taking an average of 9.14 (±4.9) daily medications.

TABLE 1

Baseline characteristics by encounter

Category	Characteristic	Total encounters N (%) or mean (SD)
Age, mean		62.52 (±12.45) years
Age	<65 years old	302 (56.2%)
	≥65 years old	235 (43.8%)
Gender	Female	273 (50.8%)
	Male	260 (48.4%)
	MTF	2 (0.4%)
	Other	2 (0.4%)
Race (self‐identified)	White	68 (12.6%)
	Black/African American	87 (16.2%)
	Hispanic	129 (24.2%)
	Asian	233 (43.3%)
	American Indian/Alaska Native	7 (1.3%)
	Native Hawaiian/Pacific Islander	6 (1.1%)
	Other	7 (1.3%)
Hospitalizations or emergency department visits in the past year	0	384 (71.5%)
	1	97 (18.1%)
	2	26 (4.8%)
	3	13 (2.4%)
	4	7 (1.3%)
	≥5	10 (1.9%)
Homeless	Yes	14 (2.6%)
	No	523 (97.4%)
Daily medications mean		9.14 (±4.9) medications
High risk medications	Anticoagulants	14 (2.6%)
	Opiates	21 (3.9%)
	Insulin	181 (33.7%)
	Chemotherapy	3 (0.6%)
	Antipsychotics	14 (2.6%)
	Antibiotics	21 (3.9%)
Systolic blood pressure mean		130.17 (±0) mmHg
Diastolic blood pressure mean		72.12 (±19.9) mmHg
Blood pressure goals control	<140/90	383 (71.3%)
	≥140/90	142 (26.5%)
	<130/80	238 (44.3%)
	≥130/80	387 (53.5%)
	N/A (no reading within the past year)	12 (2.2%)
A1c mean		8.33%
A1c goals control	<8%	264 (49.2%)
	≥8%	230 (42.8%)
	<7%	163 (30.4%)
	≥7%	331 (61.6%)
	N/A (no reading within the past year)	43 (8.0%)
Diabetes	No	121 (22.5%)
	Uncomplicated	211 (39.3%)
	End‐organ damage	205 (38.2%)
Liver disease	No	432 (80.4%)
	Mild	97 (18.1%)
	Moderate to severe	8 (1.5%)
Solid tumor	None	492 (91.6%)
	Localized	42 (7.8%)
	Metastatic	3 (0.6%)
Chronic kidney disease	None	311 (57.9%)
	Stage 1–3	197 (36.7%)
	Stage 4–5	20 (3.7%)
	Dialysis	9 (1.7%)
Other Charleson comorbidity index conditions	Acquired immunodeficiency syndrome	0 (0%)
	Congestive heart failure	34 (6.3%)
	Myocardial infarction	43 (8.0%)
	Chronic obstructive pulmonary disease	32 (6.0%)
	Peripheral vascular disease	30 (5.6%)
	Cerebrovascular accident/transient ischemic attack	46 (8.6%)
	Dementia	8 (1.5%)
	Hemiplegia	10 (1.9%)
	Connective tissue disorder	102 (19.0%)
	Leukemia	0 (0%)
	Malignant lymphoma	0 (0%)
	Peptic ulcer disease	0 (0%)

Abbreviation: N/A, not available.

Show rates

A total of 537 encounters were evaluated between January and June (173 as telehealth and 364 as FTF). All of the visits in January were FTF. In June, 172 visits were telehealth and 79 visits were FTF. Table 2 describes the show rates comparing January and June and comparing June telehealth and FTF encounters. There was no difference in the show rates between January and June (75.5% in January vs 80.4% in June, P = .13), or between telehealth and FTF visits in June (80.8% telehealth vs 79.8% FTF, P = .84).

TABLE 2

Show rates

	January scheduled	January seen	Show rate	June scheduled	June seen	Show rate
Telehealth	0	0	N/A	213	172	80.8%
FTF†	378	286	75.5%	99	79	79.8%

Note: P = .13 comparing January show rates (75.5%) vs June show rates (80.4%). P = .84 comparing telehealth (80.8%) show rates vs FTF† (79.8%) show rates in June.

Abbreviation: FTF, face‐to‐face.

MRP results

There was significantly more average MRPs per encounter resolved in patient encounters seen in FTF visits compared with telehealth visits (1.70 [±1.56] for FTF vs 1.07 [±1.20] for telehealth, P < .01). There were 26 individual patients that were seen in both January and June, and 4 individual patients that were seen in June twice. Most of these patients were seen once as telehealth and once as FTF.

The subgroup analysis found that a majority of the factors analyzed had more average MRPs for FTF visits. Please see Table 3 for details. Follow‐up visits (encounters that a patient had previously been seen by a pharmacist) had significantly more average MRPs resolved in FTF visits compared with telehealth visits (1.58 [±1.53] for FTF vs 1.04 [±1.20] for telehealth, P < .01). However, this was not the case for visits that a patient was being seen by the pharmacist for the first time.

TABLE 3

Mean MRP differences in telehealth vs FTF encounters

Category	Subcategory	Total telehealth encounters N (%)		Total FTF encounters N (%)	Telehealth MRPs mean per encounter (SD)	FTF MRPs mean per encounter (SD)	P value
Average MRPs per encounter		173 (32.2%)		364 (67.8%)	1.07 (±1.20)	1.70 (±1.56)	<.01
Encounter type	Initial	14 (11.1%)		112 (88.9%)	1.36 (±1.34)	1.57 (±1.64)	.32
	Follow‐up	159 (38.7%)		252 (61.3%)	1.04 (±1.20)	1.58 (±1.53)	<.01
Age	<65 years old	104 (34.4%)		198 (65.6%)	0.97 (±1.18)	1.81 (±1.58)	<.01
	≥65 years old	69 (29.4%)		166 (70.6%)	1.22 (±1.24)	1.46 (±1.52)	.24
Gender	Female	94 (34.4%)		179 (65.6%)	0.98 (±1.05)	1.69 (±1.62)	<.01
	Male	76 (29.2%)		184 (70.8%)	1.13 (±1.27)	1.63 (±1.50)	.01
	MTF	1 (50.0%)		1 (50.0%)	0 (± N/A)	0 (±N/A)	N/A
	Other	2 (100%)		0 (0%)	3.50 (±3.54)	N/A	N/A
Race (self‐identified)	White	12 (17.6%)		56 (82.4%)	1.25 (±1.76)	1.38 (±1.70)	.82
	Black/African American	25 (28.7%)		62 (71.3%)	0.72 (±0.89)	1.96 (±1.54)	<.01
	Hispanic	63 (48.8%)		66 (51.2%)	1.11 (±1.15)	1.69 (±1.61)	.02
	Asian	67 (28.8%)		166 (71.2%)	1.11 (±1.29)	1.58 (±0.50)	.03
	American Indian/Alaska Native	2 (28.6%)		5 (71.4%)	0.5 (±0.71)	1.6 (±1.52)	.39
	Native Hawaiian/Pacific Islander	3 (50.0%)		3 (50.0%)	1.67 (±0.58)	2.67 (±2.07)	.47
	Other	1 (14.3%)		6 (85.7%)	1 (±N/A)	2.00 (±1.55)	N/A
Hospitalizations or emergency department visits in the past year	0	123 (32.0%)		261 (68.0%)	1.10 (±1.22)	1.66 (±1.51)	<.01
	1	32 (33.0%)		65 (67.0%)	0.91 (±1.17)	2.02 (±1.91)	<.01
	2	8 (30.8%)		18 (69.2%)	0.76 (±0.94)	0.94 (±0.94)	.88
	3	4 (30.8%)		9 (69.2%)	2.38 (±1.22)	1.22 (±1.30)	.79
	4	3 (42.9%)		4 (57.1%)	1.33 (±1.53)	1.25 (±0.96)	.76
	≥5	5 (50.0%)		5 (50.0%)	0.80 (±0.45)	0.60 (±0.89)	.67
Homeless	Yes	3 (21.4%)		11 (78.6%)	0.66 (±0.58)	0.64 (±2.01)	.44
	No	170 (32.5%)		353 (67.5%)	1.08 (±1.22)	1.65 (±1.55)	<.01
Daily medications	<7	49 (28.8%)		121 (71.2%)	1.02 (±1.23)	1.68 (±1.61)	.01
	≥7	124 (33.8%)		243 (66.2%)	1.09 (±1.20)	1.64 (±1.54)	<.01
High risk medications	Yes	81 (35.5%)		147 (64.5%)	1.25 (±1.19)	1.78 (±1.58)	<.01
	No	92 (29.8%)		217 (70.2%)	0.91 (±1.57)	1.57 (±1.55)	<.01
Blood pressure goals controla	<140/90	115 (30.1%)		268 (69.9%)	1.17(±1.31)	1.51 (±1.47)	.03
	≥140/90	51 (35.9%)		91 (64.1%)	0.86 (±0.94)	2.10 (±1.78)	<.01
	<130/80	71 (29.8%)		167 (70.2%)	1.15 (±1.35)	1.60 (±1.47)	.03
	≥130/80	95 (33.1%)		192 (66.9%)	1.01 (±1.10)	1.71 (±1.65)	<.01
A1c goals controlb	<8%	62 (23.5%)		202 (76.5%)	0.90 (±1.39)	1.41 (±1.49)	.02
	≥8%	91 (39.6%)		139 (60.4%)	1.29 (±1.11)	2.10 (±1.61)	<.01
	<7%	26 (16.0%)		137 (84.0%)	0.62 (±1.24)	1.18 (±1.34)	.37
	≥7%	127 (38.4%)		204 (61.6%)	1.24 (±1.22)	2.03 (±1.62)	<.01
Diabetes	No	21 (17.4%)		100 (82.6%)	0.67 (±1.28)	1.07 (±1.24)	.92
	Uncomplicated	81 (38.4%)		130 (61.6%)	1.14 (±1.33)	1.65 (±1.60)	.02
	End‐organ damage	71 (34.6%)		134 (65.4%)	1.11 (±1.02)	2.09 (±1.61)	<.01
Liver disease	No	138 (31.9%)		294 (68.1%)	1.05 (±1.22)	1.70 (±1.58)	<.01
	Mild	33 (34.0%)		64 (66.0%)	1.18 (±1.21)	1.48 (±1.55)	.33
	Moderate to severe	2 (25.0%)		6 (75.0%)	0.50 (±0.71)	1.67 (±0.75)	.32
Solid tumor	None	151 (30.7%)		341 (69.3%)	1.06 (±1.21)	1.65 (±1.56)	<.01
	Localized	20 (47.6%)		22 (52.4%)	1.20 (±1.28)	1.59 (±1.76)	.42
	Metastatic	2 (66.7%)		1 (33.3%)	0.50 (±0.71)	2.00 (±N/A)	N/A
Chronic kidney disease	None	108 (34.7%)		203 (65.3%)	1.09 (±1.33)	1.49 (±1.50)	.02
	Stage 1–3	53 (26.9%)		144 (73.1%)	1.11 (±1.01)	1.80 (±1.63)	<.01
	Stage 4–5	8 (40.0%)		12 (60.0%)	0.75 (±0.71)	2.92 (±1.38)	<.01
	Dialysis	4 (44.4%)		5 (55.6%)	0.50 (±0.58)	0.80 (±0.84)	.56
Other Charleson comorbidity index conditions	Acquired immunodeficiency syndrome	Y	0 (0%)	0 (0%)	0 (±N/A)	0 (±N/A)	N/A
		N	173 (32.2%)	364 (67.8%)	1.07 (±1.20)	1.70 (±1.56)	<.01
	Congestive heart failure	Y	5 (14.7%)	29 (85.3%)	0.80 (±1.24)	1.24 (±1.46)	.52
		N	168 (33.4%)	335 (66.6%)	1.08 (±1.22)	1.69 (±1.57)	<.01
	Myocardial infarction	Y	9 (20.9%)	34 (79.1%)	0.56 (±0.73)	2.09 (±1.98)	.03
		N	164 (33.2%)	330 (66.8%)	1.10 (±1.22)	1.61 (±1.51)	<.01
	Chronic obstructive pulmonary disease	Y	5 (15.6%)	27 (84.4%)	1.20 (±1.64)	1.33 (±1.98)	.89
		N	168 (33.3%)	337 (66.7%)	1.07 (±1.20)	1.68 (±1.53)	<.01
	Peripheral vascular disease	Y	13 (37.1%)	22 (62.9%)	1.77 (±1.69)	1.41 (±1.44)	.24
		N	160 (31.9%)	342 (68.1%)	1.09 (±1.16)	1.67 (±1.57)	<.01
	Cerebrovascular accident/transient ischemic attack	Y	10 (21.7%)	36 (78.3%)	0.50 (±0.71)	1.75 (±1.57)	.02
		N	163 (33.2%)	328 (66.8%)	1.10 (±1.23)	1.64 (±1.57)	<.01
	Dementia	Y	2 (14.3%)	12 (85.7%)	0.50 (±0.71)	1.33 (±1.50)	.47
		N	171 (32.7%)	352 (67.3%)	1.08 (±1.21)	1.66 (±1.57)	<.01
	Hemiplegia	Y	1 (10.0%)	9 (90.0%)	3.00 (±N/A)	2.22 (±1.92)	N/A
		N	172 (32.6%)	355 (67.4%)	1.06 (±1.20)	1.64 (±1.55)	<.01
	Connective tissue disorder	Y	32 (31.4%)	70 (68.6%)	1.03 (±0.93)	1.45 (±1.61)	.17
		N	141 (32.4%)	294 (67.6%)	1.08 (±1.27)	1.70 (±1.55)	<.01
	Leukemia	Y	0 (0%)	0 (0%)	0 (±N/A)	0 (±N/A)	N/A
		N	173 (32.2%)	364 (67.8%)	1.07 (±1.20)	1.70 (±1.56)	<.01
	Malignant lymphoma	Y	2 (66.7%)	1 (33.3%)	0.50 (±0.50)	6.0 (±N/A)	N/A
		N	171 (32.0%)	363 (68.0%)	1.08 (±1.21)	1.64 (±1.55)	<.01
	Peptic ulcer disease	Y	2 (50.0%)	2 (50.0%)	0 (±0)	4.50 (±2.12)	.10
		N	171 (32.1%)	362 (67.9%)	1.08 (±1.21)	1.64 (±1.55)	<.01

Abbreviations: FTF, face to face; MRP, medication related problems; MTF, male to female transgender individual; N, no; N/A, not available; Y, yes.

12 (2.2%) of patient encounters did not have any BP reading in the past year.

Data was out of the 416 patient encounters that had diabetes. 43 (8.0%) of patients had no diabetes or no A1c in the past year; 9 (2.2%) patients with diabetes had no A1c in the past year.

FTF visits had significantly more average MRPs compared with telehealth visits for the following factors (P < .01): age under 65 years old, patients who identify as Black/African American, having diabetes with end‐organ damage, and having CKD stages 1‐5 but not dialysis. The patient encounters with uncontrolled BP using both goals of <140/90 mmHg (2.10 [±1.78] for FTF vs 0.86 [±0.94] for telehealth, P < .01) and <130/80 mmHg (1.71 [±1.65] for FTF vs 1.01 [±1.10] for telehealth, P < .01), and those with diabetes with an uncontrolled A1c using both goals of <8% (2.10 [±1.61] for FTF vs 1.29 [±1.11] for telehealth, P < .01) and < 7% (2.03 [±1.62] for FTF vs 1.24 [±1.22] for telehealth, P < .01) have significantly more average MRPs in FTF encounters compared with telehealth encounters.

Certain factors showed significantly more average MRPs in FTF encounters for both comparator categories: gender (male and female), taking more than or less than 7 medications, and taking or not taking high risk medications. Additionally, other factors particularly within the Charleson comorbidity index showed a more significant impact for FTF encounters, but the total number of patients in the comparator group was low in the following categories: 0 or 1 hospitalizations in the past year, not being homelessness, not having liver disease, solid tumor, acquired immunodeficiency syndrome (AIDS), congestive heart failure (CHF), myocardial infarction (MI), chronic obstructive pulmonary disease (COPD), peripheral vascular disease (PVD), cerebrovascular accident/transient ischemic attack (CVA/TIA), dementia, hemiplegia, connective tissue disorder, leukemia, malignant lymphoma, and peptic ulcer disease (PUD).

There were no differences in average MRPs resolved in telehealth vs FTF visits for the different types of MRPs. Results for the MRP type analysis can be seen in Table 4.

TABLE 4

Type of mean MRPs per encounter

Type of MRP	Telehealth MRPs mean per encounter (SD)	FTF MRPs mean per encounter (SD)	P value
Needs additional drug therapy	0.12 (±0.40)	0.37 (±0.70)	.527
Unnecessary drug therapy	0.07 (±0.25)	0.07 (±0.27)	.307
Different drug needed	0.06 (±0.23)	0.09 (±0.31)	.423
Dose too low	0.27 (±0.53)	0.29 (±0.52)	.596
Adverse drug event	0.12 (±0.34)	0.05 (±0.45)	.495
Drug interaction	0.01 (±0.08)	0.00 (±0)	1.000
Dose too high	0.02 (±0.15)	0.09 (±0.29)	.721
Nonadherence	0.01 (±0.08)	0.01 (±0.14)	1.000
Lab monitoring needed	0.33 (±0.95)	0.60 (±1.12)	.123
Home monitoring needed	0.08 (±0.29)	0.07 (±0.27)	.640

Abbreviations: FTF, face to face; MRP, medication related problems.

DISCUSSION

Worldwide health care systems have had to profoundly change the health care delivery model in response to the COVID‐19 pandemic. As institutions emerge from the initial phases of the pandemic, it is imperative and invaluable to examine what can be learned from how the preliminary changes impacted the ambulatory care health care delivery model. At the start of the pandemic, SFHN's 12 primary care clinics had to quickly shift from all FTF visits to a mixture of FTF and telehealth (by phone, no video capabilities). This retrospective observational cohort study aimed to examine and evaluate the impact of FTF visits as compared with telehealth visits by comparing a two‐week time period from before the pandemic, and a later two‐week time period months after the pandemic started and telehealth was implemented. The outcomes of this assessment serve a crucial role in determining network‐wide criteria for telehealth as the future of telehealth services will very likely remain a permanent part of our network.

There was a higher clinical impact observed in FTF visits as compared with telehealth visits. A difference of 0.7 average MRPs for telehealth visits vs 1.7 average MRPs for FTF visits is both statistically and clinically meaningful. Another way to view this difference would be that if a visit was done FTF instead of telehealth, a pharmacist would likely identify 1 additional MRP. This was specifically true with patients who had uncontrolled BP and A1c, likely because of disadvantages of telehealth: (1) inability to check vitals or random glucose leads to fewer MRPs, and (2) the national shortage of Omron BP cuffs reduced the ability to obtain self‐monitoring BP for evaluation.

Other factors that showed greater impact in FTF visits include follow‐up visits, age under 65 years old, Black/African American patients, having diabetes with end‐organ damage, and having CKD stages 1‐5 but not dialysis. Reasons for why age, race, and visit type factors demonstrated greater impact in FTF visits is currently unknown. More complicated conditions such as diabetes with end organ damage and CKD seem to have more impact in FTF visits compared with telehealth. The following factors had both factors demonstrate significance which is not as meaningful to discuss: male and female patients, 0 or 1 hospitalizations in the past year, taking more than or less than 7 daily medications, and taking or not taking high risk medications. Furthermore, other factors that showed significantly more average MRPs for FTF visits compared with telehealth visits may not be as accurate given the low number of patients with these factors: not being homelessness, not having liver disease, solid tumor, CHF, MI, COPD, PVD, CVA/TIA, dementia, hemiplegia, connective tissue disorder, leukemia, malignant lymphoma, and PUD. While the average MRPs between telehealth and FTF differed, the types of MRPs resolved did not. This could indicate that the same MRPs can be resolved by telehealth, but the time and resources may have limited the ability to resolve more.

The execution of telemedicine within SFHN was rapid and heavily driven by need to reduce in‐person clinic visits in response to the surge of COVID cases. With an extremely short turnaround time, it is understandable that the switch to mainly telehealth visits did not follow a standardized process. Clinical decisions such as which patients to convert to telehealth, when to obtain labs or schedule next vital checks, when to reschedule surgeries or specialty visits, and delaying educational classes or group visits varied from provider to provider and department to department. As telehealth visits remove travel time to clinics, we had believed that there would be a higher show rate in telehealth visits, but there was no difference shown in our study. Institution infrastructure and staffing also seriously impacted workflow and access. Numerous staff were deployed for extended periods of time to shelter sites, a COVID Command Center, and contact tracing. Delays of preventative primary care can lead to even more worrisome downstream effects such as more emergency department or urgent care visits when the health system itself is at increased risk of being stretched by the pandemic. The pandemic also profoundly altered the personal lives of personnel including clinical pharmacists. Even though there were less clinical pharmacy full time employees (FTE) available due to child care or sick leave, patients were still cared for. This is reflected in the 68 encounters per FTE in January vs 78 encounters per FTE in June.

Other notable factors influencing the implementation of telehealth may include language, health literacy, access to resources, and equity challenges. An evaluation through Kaiser looked at patient factors that influenced who chose FTF visits vs telehealth and found that younger patients with better access to technology who may have had barriers to FTF visits (such as paid parking at their clinic) chose telehealth visits. Additionally, patients who would usually bring in their written home BP or glucose values had a harder time communicating those data points to the provider. This is even harder when the provider and patient are unable to communicate with hand gestures or basic written visuals. It was difficult to measure the impact of health literacy on quality of telehealth in this study. However, the majority of our clinical pharmacy group expressed the immense trial they experienced caring for patients who were not able to read medication names or identify where to locate medication dose on home papers or prescription bottles.

In general, our patient population has extremely limited access to technology, many of whom struggle with even maintaining a reliable phone number. Video telehealth could have addressed many of the above barriers, but patients did not have the resources, access, or training. The MyChart feature of our EHR is well appreciated by caregivers and patients who have e‐mail addresses as training videos on device teaching can be sent to them. Nevertheless, many patients and families do not have or use e‐mail. Furthermore, equity is a San Francisco Health Network fundamental goal, and remains a focus of our quality improvement and quality assurance initiatives. In an analysis of SFHN telemedicine implementation comparing February 17‐28 to March 23‐April 3, 2021, the proportion of visits with populations at risk for limited digital literacy/access decreased significantly. As telemedicine moves forward, it is vital to apply systems wide measures to ensure health care access and delivery is equitable.

It is important to acknowledge the limitations of this study. Telehealth was and still is new for the health system and providers in June. If the time period was chosen for September instead of June, perhaps more MRPs would be resolved. Another ideal time period comparison would have been June 2019 and June 2020. However, our health system was in the middle of a large electronic medical record transition for most of 2019, which would have affected the results. Clinic flow may have been lower in June due to the drive to push patient care further out in the hopes that the COVID surge would pass. There also could have been seasonal differences in numbers of MRPs. Moreover, telehealth became more difficult the further it was from the last FTF visit because more time had lapsed since last vitals and lab measurements. There was also more time for medication errors to develop on changing medication lists from multiple providers. It should be acknowledged that pre‐pandemic dates were all FTF vs post‐pandemic dates that had a mixture of FTF and telehealth. There may have been differences in clinical decisions made between pre‐pandemic and post‐pandemic FTF visits because of lesser availability of labs being obtained and delay of specialty consults. However, there was no difference in the “lab monitoring needed” MRP type between telehealth and FTF, likely because needing self‐monitoring was also considered a “lab monitoring needed” MRP for both telehealth and FTF visits. Telehealth itself leads to selection bias as it more appealing for patients who have dependable phone numbers, access to adequate telehealth resources, and good health literacy. Retrospective chart review study is highly dependent on documentation. We encountered ambiguous documentation related to the problem list (may not be fully comprehensive), medication list, and homelessness (sometimes there is an address listed even though a patient may be marginally housed).

This timely study validates the essential role of clinical pharmacists actively participating in the system‐wide plans for telehealth rollout to help improve access and timeliness for patient care. Telehealth by clinical pharmacists served to provide vital primary care services when in person visits were drastically reduced due to the pandemic and when other providers were deployed as disaster service workers to numerous shelter sites, contact tracing, occupational health, and the city's COVID command center. Our study found that telephone follow up has its benefits within our primary care pharmaceutical care practice; however, it did not replace the need for FTF visits. The COVID pandemic has fundamentally changed the health care delivery model, and the model of mixed FTF and telehealth visits is here to stay. Opportunities for improvement include exploring video visits, offering and standardizing training for providers and patients, and a more guided system wide roll out based on clear criteria and consideration of individual patient factors. Moreover, continued careful evaluation of telehealth implementation is necessary to address barriers related to language, health literacy, and equity. The results from this study show that in a health system that had visits primarily FTF prior to the pandemic, an immediate transition to telehealth proved difficult. This could be why telehealth visits did not yield as much impact as FTF visits. Further exploration of solidifying telehealth patient care processes is necessary.

CONCLUSION

For health care institutions who are re‐examining their current health care delivery model, this study serves to set the stage for discussions surrounding criteria for determining which patients would be best served by a primary care clinical pharmacist with FTF or telehealth encounter, benefits and drawbacks of telehealth, and the pivotal role of clinical pharmacists in chronic disease state management when many public health clinicians have been deployed to other essential pandemic efforts.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting Information Telehealth Form Key

Click here for additional data file.