Quality Assurance in Telehealth: Adherence to Evidence-Based Indicators.

Background: Value enhancing telehealth (TH) lacks a robust body of formal clinically focused quality assessment studies. Innovations such as telehealth must always demonstrate that it preserves or hopefully advances quality. Introduction: We sought to determine whether adherence to the evidence-based Choosing Wisely (CW) recommendations (antibiotic stewardship) for acute sinusitis differs for encounters through direct-to-consumer (DTC) telemedicine verses "in-person" care in an emergency department (ED) or an urgent care (UC) center. Materials and
Methods: Study design was a retrospective review. Patients with a symptom complex consistent with acute sinusitis treated through DTC were matched with ED and UC patients, based upon time of visit. Charts were reviewed to determine patient characteristics, chief complaint, final diagnosis, presence or absence of criteria within the CW guidelines, and whether or not antibiotics were prescribed. The main outcome was adherence to the CW campaign recommendations.
Results: A total of 570 visits were studied: 190 DTC, 190 ED, and 190 UC visits. The predominant chief complaints were upper respiratory infection (36%), sore throat (25%), and sinusitis (18%). Overall, there was a 67% (95% CI 62.3-71.7) adherence rate with the CW guidelines for sinusitis: DTC visits (71%), ED visits (68%), and UC visits (61%). There was a nonsignificant difference (p = 0.29) in adherence to CW guidelines based upon type of visit (DTC, UC, and ED). Discussion: The challenge is to demonstrate whether or not DTC TH compromises quality.
Conclusion: In this study, DTC visits were associated with at least as good an adherence to the CW campaign recommendations as emergency medicine (EM) and UC in-person visits.

Introduction

Telehealth and telemedicine have become an expected healthcare communication tool linking providers, consultants, and patients in interactions that are secure, inexpensive, and well documented.[1] The Rock Health 2016 consumer survey (n = 4,015) found that “digital health reached a tipping point in 2016, as consumers adopted digital health tools at a record rate. Forty-six percent of consumers are now considered active digital health adopters … up from 19% in 2015. Only 12% of Americans are non-adopters, down from 20% in 2015.”[2] Despite the rapid growth, there is a scarcity of data evaluating the quality of care after evaluation of patients with telemedicine, particularly for urgent care (UC) conditions.

Several recent studies have questioned the quality of telemedicine compared with in-person visits.[3,4] Specifically, one of the major concerns for this type of care is the prescription of unnecessary antibiotics for acute upper respiratory infections.[4] One of the major campaigns for antibiotic stewardship is Choosing Wisely (CW), a campaign endorsed by >70 medical associations.[5,6] The goal of this campaign is to educate both consumers and providers as to best practices in healthcare resource management. With regard to antibiotic utilization in sinusitis, the key elements of the CW guidelines are the clinical constellation of the presentation as opposed to the diagnosis (see Limitations). Using CW as a guide, we compared adherence to antibiotic prescribing guidelines for sinusitis in the context of telemedicine videoconferencing direct-to-consumer (DTC) visits to care received through “in-person” visits in UC and the emergency department (ED). Our hypothesis was that the rates of adherence to CW guidelines would be the same for all three types of visits.

Materials and Methods

Study Design

This is a retrospective chart review study of patients with symptoms of sinusitis who were evaluated through telemedicine, in UC or in the ED. As this was a quality improvement project in antibiotic stewardship, this study was exempt from review by the Institutional Review Board (IRB).

Setting and Chart Selection

The study was conducted in a large urban academic medical center where emergency physicians staff the ED (annual volume: c/y 2016: 63,127, c/y 2017: 65,585), UC center (annual volume c/y 2016: 14,475, c/y 2017: 15,171), and the DTC service (first year [partial 2015] volume: 822, c/y 2016: 1018). Patients presenting to the DTC and UC services were almost always self-referrals.

In our institution, the enterprise-wide telehealth (TH) program offers on-demand video visits 24 h per day, 7 days per week, 365 days per year with an emergency physician through web or mobile application. By downloading the app or going to the website, any patient can register, enter medical information including chief complaint, medical history, medications, and allergies. They can then initiate a visit with the emergency physician for a cost of $49.

Patients with a chief complaint of cough, sinusitis, upper respiratory infection, cold, sore throat, and congestion were identified through a search of our telemedicine records from September 2015 through August 2016. The chief complaint elements were determined by using the definition provided by the American Academy of Allergy, Asthma, and Immunology sinusitis symptoms, which consist in some or all of the following: “persistent symptoms of an upper respiratory tract infection, purulent rhinorrhea, postnasal drainage, anosmia, nasal congestion, facial pain, headache, fever, and cough.”[8] In order not to potentially miss cases, we used the broad search terms mentioned. Each telemedicine chart was matched to corresponding charts filtered for similar chief complaints from our UC and ED services based upon visit date. If more than one matched potential UC or ED chart for the date was identified, the candidate charts were numbered (1 to N). If Y charts were needed, then the number N was randomized (Random.com) for Y random numbers. Those numbers then indicated the charts to be reviewed. ED charts for admissions, codes, major trauma, and unstable medical conditions were excluded. Providers at the three sites were not coached on the CW recommendations or antibiotic stewardship specifically for this study; therefore, the term “adherence” is used retrospectively, implying that the providers prescribing was consistent with the CW recommendations. All providers were credentialed under emergency medicine (EM). A cohort of providers self-selected to practice in the UC center or on-demand service, whereas others were randomly assigned between all three services. There were 55 pediatric patients (12 years and lower) who were evaluated under the pediatric CW recommendation ().

Table 1.

Choosing Wisely Recommendation, Source Organization, and Rationale[a]

RECOMMENDATION	ORGANIZATION	RATIONALE
Avoid prescribing antibiotics in the ED for uncomplicated sinusitis.	American College of Emergency Physicians	Sinusitis is a common reason for patients to visit the ED. Most patients with acute sinusitis do not require antibiotic treatment, because ∼98% of acute sinusitis cases are caused by a viral infection and resolve in 10–14 days without treatment. For some patients with sinusitis, antibiotics might be appropriate, such as those patients taking drugs that reduce the effectiveness of the immune system, those with prolonged severe symptoms, or those with worsening symptoms. Antibiotics can cause many side effects and have potentially severe complications, and these risks usually outweigh the benefits of their use for sinusitis. In addition, inappropriate antibiotic use for sinusitis can contribute to the development of antibiotic-resistant infections and contributes to avoidable healthcare costs.
Antibiotics should not be used for apparent viral respiratory illnesses (sinusitis, pharyngitis, bronchitis, and bronchiolitis).	American Academy of Pediatrics	Although overall antibiotic prescription rates for children have fallen, they still remain alarmingly high. Unnecessary medication use for viral respiratory illnesses can lead to antibiotic resistance and contributes to higher healthcare costs and the risks of adverse events.

See Ref.[6]

ED, emergency department.

Data Analysis

Using the methodology as outlined by Kaji et al.[7] for chart review studies, we developed a structured close ended data abstraction tool. Data extraction was accomplished by two trained abstractors (D.H.-R. and A.S.) using the structured closed question instrument. The two abstractors (D.H.-R. and A.S.) reviewed 25 charts with an inter-rater reliability of 96%. The same two abstractors then independently completed the rest of the charts. If there were any questions or disagreements on the charts, a third reviewer (A.M.C.) would review the case. Based on CW guidelines (), sinusitis was defined as “uncomplicated” (no evidence to suggest that antibiotic therapy would either shorten and/or reduce the severity of the illness) or “complicated” (evidence demonstrates that antibiotic intervention has clinical usefulness). Complicated sinusitis includes persistent and not improving symptoms >10 days, worsening or “double sickening,” or immunocompromised status (HIV, immunosuppressants, and steroids).[6,8,9] Data were entered using the Redcap data collection system. Data were then analyzed using Excel (Microsoft, Redmond, Washington) and Principles of Biostatistics (2nd edition, Duxbury 2000), using standard mean, proportions, confidence interval, and chi-square calculations.

Results

In the 570 cases reviewed, the predominant chief complaints were upper respiratory infection (36%), sore throat (25%), and sinusitis (18%). Patients' average age was 35.6 (95% CI = 35.6 ± 0.58) years old and 59% of the sample was female (95% CI = 59% ± 12%). Patients' comorbid conditions included diabetes (4.6%) (95% CI = 4.67% ± 1.4%), asthma or COPD (9.8%) (95% CI = 10% ± 3.7%), and hypertension (9.5%) (95% CI = 9.3% ± 3%). The majority of DTC patients were female, whereas ED patients were sicker, with more comorbidities, but were treated with fewer antibiotic prescriptions ().

Table 2.

Patient Demographics

	DTC	UC	EM	TOTAL	95% CONFIDENCE INTERVAL
Patients (n)	190	190	190	570	—
Age (average)	36.3	35.1	35.4	35.6	35.6 ± 0.58
Female (n, %)	127 (67%)	98 (52%)	110 (58%)	335 (59%)	59% ± 12%
Hypertension	14 (7%)	15 (8%)	25 (13%)	54 (9%)	9.3% ± 3.0%
COPD/asthma	11 (6%)	19 (10%)	26 (14%)	56 (10%)	10% ± 3.7%
Diabetes	5 (3%)	9 (5%)	12 (6%)	22 (5%)	4.67% ± 1.4%
Antibiotics prescribed at encounter	83 (44%)	108 (57%)	57 (30%)	248 (44%)	43.67% ± 12%

COPD, chronic obstructive pulmonary disease; DTC, direct-to-consumer; EM, emergency medicine; UC, urgent care.

Overall, 248 patients (43.5%) were prescribed antibiotics at the end of their encounter () Of all encounters, 266 cases (47%) were diagnosed by review with something other than sinusitis; 33% (187 cases) had uncomplicated sinusitis, and 21% (117) had complicated sinusitis ().

Table 3.

Diagnosis by Chart Review and Adherence to Choosing Wisely Guidelinesa,b

		ANTIBIOTICS GIVEN	NO ANTIBIOTICS	CONSISTENT WITH CW (%)
ED (n = 190)
Uncomplicated	55	12	43	73
Complicated	27	13	14	59
Not sinusitis	108
UC (n = 190)
Uncomplicated	62	31	31	52
Complicated	39	31	8	69
Not sinusitis	89
DTC (n = 190)
Uncomplicated	70	20	50	71
Complicated	51	36	15	76
Not sinusitis	69

See Ref.[6]

See Table 1.

CW, Choosing Wisely; DTC, direct-to-consumer; ED, emergency department; UC, urgent care.

Recalling that adherence to the CW recommendations is the sum of appropriately providing antibiotics for complicated sinusitis cases and not providing for uncomplicated cases, we found antibiotic prescribing to be 67% (95% CI 62.3–71.7) consistent with the CW guidelines overall. There was a nonsignificant difference (p = 0.29) in adherence to CW guidelines based upon location of visit (DTC, UC, and ED).

Discussion

The study was not designed to determine whether patients actually received treatment or had different outcomes. It was designed to determine whether prescribing patterns differed depending upon whether it was an in-person or video visit. As such, we do not have outcome data for patients or know whether they obtained prescriptions elsewhere or whether they developed antibiotic-related complications. There were a significant number of patients who met inclusion criteria by our review but did not receive a diagnosis of sinusitis by the attending clinician. However, we felt that relying on discharge diagnosis would have biased the prescription rate, as the decision to prescribe antibiotics might impact the assigned final diagnosis.

Providers practiced in either DTC telemedicine or UC areas or in all three sites but were all EM boarded and experienced. Ideally, all three sites would be staffed by a group of providers randomly assigned to the three sites. Owing to the unpredictable staffing needs so much a part of random-access healthcare, we had to work with a lesser staffing pattern.

We found no significant differences in CW guideline adherence for antibiotic prescription for acute sinusitis between a telemedicine video visit service compared with in-person visit in UC or ED settings. To our knowledge, this is the first demonstration comparing quality of care between an on-demand telemedicine program, UC, and emergency medicine services.

Our three lines of service are all staffed by the same group of board-certified EM physicians, thus increasing the strength of the study by eliminating a portion of physician variation. It could be argued that socioeconomic factors could be a play with certain groups of patients being either more or less demanding to receive antibiotic therapy. Informal questioning of providers practicing in all three locations revealed that they did not have a sense that these differences existed.

There is significant interest in telehealth services, with almost half of healthcare consumers now considered to be digital health adopters.[2] There is concern that these services would provide a variable quality of care. Uscher-Pines et al. recently evaluated the use of Teladoc, a DTC telemedicine service, and found less diagnostic testing and overuse of antibiotics compared with physician office visits.[4,10] Our overall antibiotic prescription rate was 44% in all sinusitis cases, with the highest rate from UC. In the case of complicated sinusitis, the antibiotic prescription rate was 68%, similar to what was been reported in the literature.[10,11] By classifying cases as complicated versus uncomplicated sinusitis by two independent clinicians, we were able to reduce the bias from a physician final diagnosis.

Of the total 570 cases chosen, 69 (36%) from the telemedicine group, 89 (47%) from the UC group, and 108 (57%) from the EM group were identified as not having sinusitis (mean 46.7% [95% CI 37.0%-56.4%]). This is not surprising because we designed the study to identify patients based upon chief complaint rather than final diagnosis.

Appropriate antibiotic stewardship is increasingly at the front and center of quality improvement and safety efforts.[9,12] In using the CW recommendations, it is as much a failure to prescribe for a case of uncomplicated sinusitis as it is to not prescribe for a case of complicated sinusitis.

This study carries significant clinical impact. Our objective evaluation of the increasingly used TH modality is needed to establish TH's clinical reliability and efficacy. This study is an early entry in that database. Owing to cost efficiencies and patient preference and given that TH usage provides the same or better quality, TH offers incremental value in healthcare delivery. Demonstrating uses of TH that will deliver incremental healthcare value is a high priority clinical task. In this context, our study speaks of the validation of similar clinical practice in the three venues evaluated. Under the circumstances of our study, clinical behavior in the virtual setting is at least as good as that in two separate “in-person” settings. Given the significantly decreased cost of the virtual visit and the similarity of an indicator of quality, the virtual clinical tool brings incremental value.

Using the CW recommendations as a quality measure, on-demand telemedicine provided at least as high a quality with regard to antibiotic therapy for sinusitis as did “in-person” care by UC and emergency medicine provider.