A quality improvement pathway to rapidly increase telemedicine services in a gynecologic oncology clinic during the COVID-19 pandemic with patient satisfaction scores and environmental impact.

The primary goal was to convert 50% of all outpatient Gynecologic Oncology (GynOnc) encounters during the COVID-19 pandemic to telemedicine within one week. The secondary goal was to reach 100% documentation of telemedicine consent. The tertiary goal was to analyze patient satisfaction scores. An additional goal was to estimate CO2 emissions prevented from being produced. The period from 3/16/2020-4/15/2020 was targeted. The initial intervention involved transitioning surveillance visits. A second intervention, with nursing and advanced-practice-provider support, included transitioning additional visit types, and distributing a note template. The Telehealth Satisfaction Survey (TeSS) was administered to patients. Descriptive statistics and run charts were used to analyze and depict results. Within four weeks, there were 408 encounters; 217 were telemedicine (53.2%). Following the second intervention, 13 of 15 days (86.7%) reached the 50% telemedicine target and consent was documented in 96.6% of the telemedicine encounters. The TeSS had a 74.8% response-rate. Patients rated the following aspects of the telemedicine encounter as good or excellent: call quality (96.5%), personal comfort (92.9%), length-of-visit (94.7%), treatment explanation (93.8%), overall experience (88.5%). Moreover, 82.3% of patients would use telemedicine again. Additionally, 6.25 metric tons of CO2 emissions from travel were prevented from being produced. A GynOnc clinic can rapidly implement telemedicine systems. With multidisciplinary team planning and standardized note templates, transitioning 50% of encounters to telemedicine and achieving high rates of consent documentation were accomplished in four weeks. This increase in telemedicine represented a measurable decrease in the amount of CO2 emissions. Additionally, patients were overwhelmingly satisfied.

Introduction

While various forms of telemedicine have been used for decades (Kvedar et al., 2014), the COVID-19 pandemic has accelerated the use of telemedicine in oncology. Due to the COVID-19 pandemic, the National Comprehensive Cancer Network (NCCN) recommended converting outpatient oncology visits to telemedicine whenever possible (Cinar, et al., 2020). While studies have demonstrated that telemedicine is safe and effective for postoperative follow-up in the field of Gynecologic Oncology (GynOnc), data regarding other types of outpatient encounters are lacking (Graetz et al., Aug. 2018); (Andikyan, 2012).

Early in the pandemic, cancer patients were considered to be a high-risk population for SARS-coV-2 infection and COVID-19 sequelae. Data from Wuhan, China suggested that the infection rate for cancer patients was nearly twice that of the general population (Yu et al., 2020). An early Italian study published in JAMA identified 20.3% of the deaths were accounted for by people with active cancer (Onder et al., 2020). In addition to the direct health risks posed by SARS-coV-2, concerns regarding indirect risks to cancer patients have been raised. Specifically, the implications of patients missing appointments or not seeking medical care due to factors surrounding the COVID-19 pandemic may not be understood for some time (Wang and Zhang, 2020).

Teleoncology is a concept that has existed for many years and has been particularly vital due to the fact that access to a gynecologic oncologist is limited in the U.S. with 14.8% of women living more than 50 miles from their nearest provider (Satcher, 2014). In the current climate, it is also important to touch on the ecological impact of teleoncology. Due to the fact that gynecologic oncologists exist at great distance from many patients, ecological impact of patient travel is an important consideration. The United States (US) is the second largest contibutor to CO2 emissions in the world. In 2018, 5.4 billion metric tons of CO2 were produced in the US with the worldwide total being approximately 36.6 billion metric tons (CO2 Emissions). In the US, transportation is the greatest contributor to greenhouse gas emissions, with light-duty vehicles, including passenger cars, making up more than half of transportation contributions (United States Environmental Protection Agency, 1990). According to the Environmental Protection Agency (EPA) the average passenger vehicle produces 4.03 x10-4 metric tons of CO2 emissions per mile driven (Greenhouse Gases Equivalencies Calculator). Carbon dioxide is the largest contibuor to greenhouse gas emissions and therefore has the largest role to play in climate change.

Given a confluence of factors including institutional policy change, recommendations from the NCCN, as well as the need to continue critical cancer treatment and surveillance, the GynOnc Carbone Cancer Center clinic at University of Wisconsin School of Medicine and Public Health made the decision to rapidly convert in-person encounters to telemedicine. During this adjustment, we sought to collect data regarding both the institutional implementation as well as patient perspectives. For this quality improvement project, there were two aims. The primary aim was to convert at least fifty percent of all outpatient clinical encounters to telemedicine within one week of the initial intervention. The secondary aim was to have one hundred percent documentation of telemedicine consent in providers’ notes within one week of the initial intervention.

Our tertiary aim was to elicit patient feedback regarding this new type of encounter and determine the potential interpersonal impact on patient care. An additional goal consisted of calculating the amount of CO2 emissions prevented from being produced by patient travel to demonstrate an important secondary effect of transitioning to telemedicine.

Materials and methods

This quality improvement project was conducted from March 16, 2020 through April 15, 2020 at the GynOnc Carbone Cancer Center clinic at University of Wisconsin School of Medicine and Public Health. The project was deemed exempt by the Health Sciences Institutional Review Board under 45 CFR 46.102(d). SQUIRE 2.0 guidelines were used in the implementation of the project and here in its reporting (SQUIRE). Prior to this date, telemedicine was not used for GynOnc outpatient encounters at this institution. During the course of this quality improvement project, telemedicine visits were conducted by phone.

Context of the project/aims: In order to respond to the need to decrease exposures during the COVID-19 pandemic and follow guidance from our institution, the GynOnc division established three goals. Beginning March 16, 2020, the primary goal for this quality improvement project was to convert fifty percent of daily encounters to a telemedicine encounter within one week of the project’s initiation. The secondary goal was to achieve 100% documentation of patient consent to participate in telemedicine encounters within the same time frame. The tertiary goal was to assess patient satisfaction regarding the transition to telemedicine.

Interventions: Using plan-do-study-act (PDSA) cycles, the initial intervention was agreed upon at a meeting of GynOnc physicians. The meeting topics included: 1) the institutional telemedicine platform, location of instructions and available support; 2) visit coding in the electronic health record (EHR), time documentation, consent documentation; 3) the workflow for transitioning encounters to telemedicine through visit schedulers. The initial intervention required all of the GynOnc providers to identify and convert appropriate surveillance visits to a telemedicine format through the clinic scheduler. One week after the initial intervention, data was analyzed and fell short of our primary and secondary aims, therefore a second PDSA cycle was implemented with an additional intervention. The secondary intervention was enacted over a two-day period and included: 1) advanced-practice providers (APPs) and nursing staff leading the conversion to telemedicine visits with physician approval; 2) implementation of a standardized telemedicine note template; 3) additional review of coding in EHR, time documentation, and consent documentation.

Study of the interventions/measures: To determine the percentage of visits conducted via telemedicine, the clinic schedules of all twelve GynOnc providers (physicians and APPs) for the allotted study period were queried to determine the percent of visits each day conducted using telemedicine compared to total encounters. New patient visits were excluded from our study population due to desire to maintain these appointments as in-person visits by the GynOnc providers. Our EHR clearly designates which visits are conducted by telemedicine on the schedule face page. To determine the rates of documentation of patient consent for telemedicine visits, study staff queried the telemedicine encounter notes for each telemedicine visit in the EHR for the allotted study period.

For the tertiary goal of assessing patient satisfaction with telemedicine encounters, the Telemedicine Satisfaction Survey (TeSS) was administered to patients by phone within one to four weeks of completing their GynOnc telemedicine encounter (Telehealth Satisfaction Questionnaire); (Nations, 2001) . Up to three contact attempts by phone were made for each patient. The exclusion criteria, as seen in Table 1, for administration of the TeSS included: telemedicine encounters with other medical specialties within the study period (n = 35), if the patient was under the care of Hospice or transitioned to Hospice care during the project period (n = 3), need for interpreter services (n = 2), or an activated health care power of attorney (n = 1). Before the administration of the TeSS, the specific encounter date and provider were referenced to orient the patient. This 10-item version of the TeSS was adapted from the original 12-item instrument taken from the National First Nations Telehealth Research Project in coordination with the Agency for Healthcare Research and Quality (AHRQ) (Telehealth Satisfaction Questionnaire). The TeSS has been successfully adapted with appropriate psychometric properties in other patient populations (Morgan).

Table 1

Eligibility Criteria for Telehealth Satisfaction Survey (TeSS) Administration. Describes the inclusion and exclusion criteria for administration of the TeSS. Patients who participated in multiple telemedicine visits for different medical specialties, required an interpreter, were involved in hospice care, or had an activated power of attorney were ineligible. Of the patients who were eligible for administration of the TeSS, some patients were unable to be reached after three contact attempts or declined participation.

Total encounters (n = 192)	Criteria	n (%)
Eligible encounters		151 (78.7)
	Respondents	113 (74.8)
	Not reached	34 (22.5)
	Declined	3 (2.0)
	Other	1 (0.7)
Ineligible encounters		41 (21.3)
	Multiple visits	35 (85.4)
	Hospice	3 (7.3)
	Interpreter	2 (4.9)
	Power of Attorney	1 (2.4)

Analysis: Out of a total of 217 telemedicine encounters completed during this study period, we identified 192 unique patients. Demographics with descriptive statistics were generated for these 192 patients (Table 2). Annotated run charts were created for the primary and secondary goals of the study (Fig. 1, Fig. 2, respectively). For the patient satisfaction portion, individual responses were tabulated and reported in Table 3.

Table 2

Demographic Information (n = 192). Describes demographic information for the entire cohort of unique patients reviewed. The mean age of our patients was 63.4 years old. The site of primary cancer and stage were identified in these patients. The type of visit being conducted was recorded. Additionally the distance from a patients’ home to the GynOnc clinic was calculated.

Age (years)
	Mean	63.42
	Standard Deviation	13.26
Distance travelled one-way (mi)
	Mean	40.40
	Standard Deviation	42.20
Primary Cancer Site		n (%)
	Uterus	80 (41.7)
	Ovary	62 (32.3)
	Fallopian tube	13 (6.8)
	Cervix	11 (5.7)
	Vulva/Vagina	5 (2.6)
	Peritoneum	4 (2.1)
	GTN	2 (1.0)
Stage	I	72 (37.5)
	II	17 (8.9)
	III	47 (24.5)
	IV	27 (14.1)
Visit type
	Surveillance	104 (54.2)
	Postoperative	31 (16.2)
	Chemotherapy	44 (22.9)
	Results	10 (5.2)

Fig. 1

Telemedicine Visits. This run chart depicts the entire 30-day project period and illustrates the percentage of telemedicine visits completed daily in the context of the initial and second interventions.

Fig. 2

Consent Documentation. This run chart depicts the entire 30-day project period and illustrates the percentage of providers’ notes that documented consent for telemedicine encounters for 192 unique patients in the context of the initial and second interventions.

Table 3

Telehealth Satisfaction Survey (TeSS) (n = 113). Details patient responses to the 10-item TeSS.

How would you rate …	n (%)
	Excellent	Good	Fair	Poor	No response
Voice quality of the equipment?	85 (75.2)	24 (21.2)	2 (1.7)	0 (0)	2 (1.7)
Your comfort in using the telehealth system?	69 (61.1)	36 (31.9)	6 (5.3)	0 (0)	2 (1.7)
The length of time with the provider?	86 (76.1)	21 (18.6)	3 (2.7)	0 (0)	3 (2.7)
The explanation of your treatment by the telehealth staff?	77 (68.1)	29 (25.7)	4 (3.5)	0 (0)	3 (2.7)
The thoroughness, carefulness and skillfulness of the telehealth staff?	85 (75.2)	19 (16.8)	3 (2.7)	1 (0.9)	5 (4.4)
The courtesy, respect, sensitivity and friendliness of the telehealth staff?	104 (92.0)	8 (7.1)	0 (0)	0 (0)	1 (0.9)
How well the telehealth staff respected your privacy?	95 (84.0)	10 (8.8)	0 (0)	0 (0)	8 (7.1)
Your overall treatment experience at using telehealth?	72 (63.7)	28 (24.8)	11 (9.7)	0 (0)	2 (1.7)
	Yes	No	No response
Would you use telehealth again?	93 (82.3)	17 (15.0)	3 (2.7)
Would you recommend telehealth to another person?	97 (85.8)	12 (10.6)	4 (3.5)

Ethical considerations: The study was conducted following standards of quality improvement processes at the University of Wisconsin School of Medicine and Public Health.

Results

There were 408 total outpatient encounters during the 30-day period. This represents 82% of the mean monthly encounter volume compared to the prior six months in the same clinic. There were 192 unique patients that were involved in 217 telemedicine encounters (53% of total encounters). Of the 217 telemedicine encounters, 54% were for surveillance, 23% for chemotherapy, 16% postoperative, and 7% other types (Table 2).

Fig. 1 depicts the entire 30-day project period and depicts the percentage of telemedicine visits completed daily in the context of both the initial and second interventions. In the first week following the initial intervention, only 33 of the 95 encounters were conducted by telemedicine (34.7% of encounters). One of the five clinic days (20% of days) met the 50% telemedicine target threshold. In the first week following the second intervention, 41 of the 73 encounters (56.2% of encounters) were conducted by telemedicine and four of the five clinic days (80% of days) met the 50% telemedicine target threshold. In the total three weeks following the secondary intervention, 163 of the 256 encounters were conducted by telemedicine (63.7% of encounters). Thirteen of the fifteen clinic days (86.7% of days) met the 50% telemedicine target threshold.

Regarding the secondary aim, in the first week following the initial intervention, only 3.6% of providers’ notes documented patient consent to participate in telemedicine encounters. Following the secondary intervention, 96.6% of notes documented consent (Fig. 2).

Of the 192 unique patients, 41 were ineligible for the TeSS as detailed above (Table 1). For the 151 patients eligible for the TeSS, there were 113 respondents for a response rate of 74.8% (Table 1). As detailed in Table 3, there was overwhelming satisfaction with voice quality of the encounter as 75.2% (n = 85) responded excellent and 21.2% (n = 24) responded good. In response to comfort in using the telemedicine system, 61.1% (n = 69) selected excellent and 31.2% (n = 36) selected good. Regarding length of time with the provider during the telemedicine encounter, 76.1% (n = 86) responded excellent and 18.6% (n = 21) responded good. A total of 68.1% (n = 77) of patients felt that the explanation of treatment by the telemedicine staff was excellent and an additional 25.7% (n = 29) said the explanation was good. The thoroughness, carefulness and skillfulness were rated as excellent by 75.2% of patients (n = 85) and good by 16.8% (n = 19). The courtesy, respect, sensitivity and friendliness of staff during the telemedicine encounter was rated as excellent by 92.0% of patients (n = 104) and good by 7.1% (n = 8). All participants who responded either rated that their privacy was respected as either excellent (84.0%, n = 95) or good (8.8%, n = 10). The overall telemedicine treatment experience was rated as excellent by 63.7% of patients (n = 72), good by an additional 24.8% (n = 28), and fair by only 9.7% (n = 11). Overall, 82.3% of patients endorsed using telemedicine again and 85.8% would recommend using telemedicine to another person.

The distance from a patient’s home address to the Carbone Cancer Clinic was calculated and multiplied by two to represent a round trip estimate of the number of miles saved by conducting a telemedicine visit. This number came to 15,511.8 miles for our sample of 192 patients. Using the EPA’s estimate of 4.03 x10-4 metric tons of CO2 emissions per mile driven in an average passenger vehicle, we calculated that 6.25 metric tons of CO2 emissions were prevented from being produced by our patients during the four-week study period.

Conclusions

This quality improvement process demonstrated that a rapid escalation of telemedicine in a GynOnc outpatient practice is feasible and practical with a high volume of patient encounters. Appropriate coding and documentation requirements were also achieved. After two separate interventions, a successful implementation bundle included: 1) provider education of the institutional telemedicine platform; 2) provider education of coding requirements; 3) a multidisciplinary team approach with clearly delineated responsibilities among physicians, APPs, nursing staff, and scheduling coordinators; 4) implementing standardized note templates.

Implementation and ongoing monitoring of telehealth platforms are critical for providing appropriate cancer care throughout the pandemic and beyond. Early in the pandemic (March 17,2020), the Centers for Medicare and Medicaid Services (CMS) rapidly expanded the opportunity for telehealth services, including the ability to receive those services from any location and more than doubling the allowable services. On December 1, 2020, CMS announced some permanent changes to the telehealth expansion (Newsroom, 2020).

Usefulness of the work: This project confirms the feasibility of the use of telemedicine for GynOnc outpatient clinics. Moreover, an outpatient GynOnc center can rapidly escalate its cancer services in times of duress. This is vital as the number of infections of SARS-coV-2 and deaths from COVID-19 have been rising steadily throughout the summer and fall months (Maps & Trends - Johns Hopkins Coronavirus Resource Center). In various regions of the country, it may be advisable, or even mandated, to curtail certain patient care services throughout the coming fall and winter seasons. To that end, this project identifies a simple pathway to successfully scale up telemedicine services that comply not only with institution policy and billing, but also yield high patient satisfaction. This model has the ability to function in a long-term context as well, which is relevant to the uncertain trajectory of the COVID-19 pandemic. Although the composition of a GynOnc clinical team will be different throughout institutions, the general concepts of the intervention bundle used at the University of Wisconsin could be easily and rapidly adapted across the range of practice types and locations.

Sustainability: For six weeks following the project period reported, our GynOnc clinic was able to maintain appropriate target metrics for telemedicine encounters and documentation of consent. In the second week of May 2020, our institutional pandemic incident command unit allowed for a phased return to routine clinical operations, although telemedicine encounters continued to be offered above the baseline rate from prior to the COVID-19 pandemic. With concerns about future spikes in infections and hospital resource utilization, our intervention pathway is poised to be enacted again as needed to decrease exposure to outpatient GynOnc patients. On June 25, 2020 and again on July 3, 2020, the public health commission over our hospital system revised guidelines to be more restrictive to gatherings and step back from “re-opening” due to significantly increasing rates of SARS-coV-2 infections (Public Health Madison & Dane County). Regarding environmental sustainability, we also documented the decrease in carbon footprint by transitioning to telemedicine services during this project period. The impact of health care decisions on carbon emissions will be an emerging area of research in the coming years with the first study regarding the environmental impact of surgery published in 2017 (MacNeill et al., 2017).

Potential for other contexts/implications for practice and further study: GynOnc continues to be in a unique medical niche with both medical and surgical patients. There is great potential for the use of telemedicine platforms to be a useful alternative as we head into what is predicted to be a season of resurgence for SARS-coV-2 in the Fall/Winter 2020–2021. Surgical practices can adapt by using pictures through EHR patient portals and video visit platforms. For example, surgical wounds and incisions can be assessed and evaluated by video as well. Various surgical services have made this successful transition (Morgan). Having a multi-disciplinary approach to surgical and medical care for cancer patients is of utmost importance as cancer diagnoses continue to come into GynOnc offices. A combined, multi-disciplinary approach offers the best opportunity to continue providing essential in-person services (e.g. some new patient consultations, non-emergent surgery, chemotherapy, and acute, non-emergent visits that require in-person assessment). Cancer care will continue to be a critical public health issue in the background of the COVID-19 pandemic (Maps & Trends - Johns Hopkins Coronavirus Resource Center); (Public Health Madison & Dane County).

Suggested next steps: Looking beyond the COVID-19 pandemic, there is a high likelihood that outpatient medical practices will have to repeatedly implement and escalate telemedicine platforms. Therefore the long-term impact of telemedicine on patient satisfaction and clinically relevant outcomes is of interest. Comparing patient satisfaction and preference for telemedicine versus in-person visits would be an important next step. An additional area of interest specific to GynOnc involves determining whether complication rates differ between patients who had telemedicine visits instead of in-person visits for indications such as postoperative and surveillance visits. Complications could theoretically consist of delayed diagnosis of recurrence or unrecognized postoperative complications. Additionally, policy and practice implications related to long-term feasibility for systems and reimbursement are critical but unable to be assessed at this time.

There has been research into using telemedicine for the diagnosis and initiation of treatment in the field of oncology using a multidisciplinary approach (Shalowitz and Moore, 2020); (Shalowitz). With telemedicine platforms now functioning in our outpatient GynOnc practice, expanding this infrastructure to include other specialties such as pathology, radiology, and medical oncology in a coordinated manner is of interest.

Examining how GynOnc physicians, APPs, and nursing staff viewed this different method of patient care would be quite interesting. Understanding how provider perception of the quality of care as well as how perception of interpersonal relationships with patients changes based on the type of encounter would be valuable and an area for future study.

More broadly, the environmental impact of daily living is becoming an increasingly urgent issue. It is hard to ignore the potentially positive effect less personal travel could have on our planet. Therefore, determining the environmental impact of traveling long distances to seek specialty medical care and subsequently decreasing that impact through telemedicine platforms is an interesting and potentially positive consequence to this necessary shift in practice.

Strengths/weaknesses/summary: This quality improvement study describes an important learning process that occurs when an organization attempts to pivot from an established workflow to a completely new way of practice. In this situation, the challenges of this process were compounded by the speed at which we challenged ourselves to establish a new infrastructure and workflow to support telemedicine outpatient encounters. Following the initial intervention, both the telemedicine conversion rates and documented telemedicine consent did not meet our goals. After additional interventions were made, we were able to meet our goals with regards to telemedicine encounter rates as well as much improve rates of patient consent for telemedicine encounter documentation. We learned that a multidisciplinary approach is what ultimately led to our ability to quickly pivot a solely in-person outpatient practice to one that offered over fifty percent telemedicine visits during the height of the COVID-19 pandemic. Additionally, we have now established a workflow that is not only easily redeployable and scalable within our practice, but could also be easily adapted to other specialties and institutions.

We did not examine the impact of telemedicine on billing and reimbursement of our GynOnc practice. Though discussion regarding billing codes and documentation requirements were included in the intervention bundle, we did not look into the resulting financial implications of this shift in practice.

It also important to note that we examined patient satisfaction of telemedicine in the context of the COVID-19 pandemic. We cannot presume this data represent patient perspectives on telemedicine outside of a pandemic.

By implementing telemedicine quickly in our clinic, we were able to continue to care for complex medical patients whose health often depends on timely access to a subspecialist. Whether telemedicine visits prove to be equal to in-person visits with respect to diagnosis or recurrence or postoperative complications is yet to be seen. However, we can conclude within the duration of this study period, our patients received appropriate and timely care with positive patient response.

Authors contributions

Dr. Rachel Mojdehbakhsh: formal analysis, investigation, writing – original draft, writing – review and editing, visualization Dr. Stephen Rose: supervision, resources Dr. Megan Peterson: methodology, investigation Dr. Laurel Rice: writing – review and editing, supervision, resources Dr. Ryan Spencer: conceptualization, methodology, formal analysis, investigation, writing – original draft, writing – review and editing, project administration

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.