RadTranslate: An Artificial Intelligence-Powered Intervention for Urgent Imaging to Enhance Care Equity for Patients With Limited English Proficiency During the COVID-19 Pandemic.

PURPOSE: Disproportionally high rates of coronavirus disease 2019 (COVID-19) have been noted among communities with limited English proficiency, resulting in an unmet need for improved multilingual care and interpreter services. To enhance multilingual care, the authors created a freely available web application, RadTranslate, that provides multilingual radiology examination instructions. The purpose of this study was to evaluate the implementation of this intervention in radiology.
METHODS: The device-agnostic web application leverages artificial intelligence text-to-speech technology to provide standardized, human-like spoken examination instructions in the patient's preferred language. Standardized phrases were collected from a consensus group consisting of technologists, radiologists, and ancillary staff members. RadTranslate was piloted in Spanish for chest radiography performed at a COVID-19 triage outpatient center that served a predominantly Spanish-speaking Latino community. Implementation included a tablet displaying the application in the chest radiography room. Imaging appointment duration was measured and compared between pre- and postimplementation groups.
RESULTS: In the 63-day test period after launch, there were 1,267 application uses, with technologists voluntarily switching exclusively to RadTranslate for Spanish-speaking patients. The most used phrases were a general explanation of the examination (30% of total), followed by instructions to disrobe and remove any jewelry (12%). There was no significant difference in imaging appointment duration (11 ± 7 and 12 ± 3 min for standard of care versus RadTranslate, respectively), but variability was significantly lower when RadTranslate was used (P = .003).
CONCLUSIONS: Artificial intelligence-aided multilingual audio instructions were successfully integrated into imaging workflows, reducing strain on medical interpreters and variance in throughput and resulting in more reliable average examination length.

Introduction

Controlling for other factors, patients with limited English proficiency (LEP), individuals for whom English is not their primary language who have difficulty communicating effectively in English, are at increased risk for medical complications and poor outcomes [1]. As such, the role of health care provider–patient language concordance is critical and continues to be a topic of interest in medicine [2]. Language concordance is most commonly achieved through real-time direct translation services using certified medical interpreters, including in-person and remote (telephonic or video) interpretation [3].

Although the benefits of language concordant medical care have been well documented, it has typically been studied in medical office settings or within emergency departments. Many LEP interventions, including the use of remote interpreters, may not be operationally feasible for urgent and emergent radiology settings in which there is expected rapid throughput of patients and both interpreters (in person or remote) and imaging equipment are limited resources. Although LEP has been shown to affect image quality during CT and MRI [4,5], interpreters may not be routinely used during standard radiography in which examination times are substantially shorter, with an average duration on the order of 4 min [6].

The coronavirus disease 2019 (COVID-19) pandemic has increased demand for chest radiography (CXR) as well as interpreter services. Communities of color and immigrant populations have been disproportionally affected by COVID-19, with nonwhite individuals more likely to test positive and have more severe disease at time of presentation [7]. To assist in the surge of patients with suspected COVID-19, reduce crowding in the emergency department, and provide continued community health center support, our institution created respiratory infection clinics (RICs), ambulatory triage clinics for patients with suspected COVID-19. These clinics included COVID-19 testing and standardized care algorithms, implemented to aid in triage, with an efficient referral process for same-day immediate imaging [8]. As a result, there was a precipitous increase in demand for CXR as well as interpreter services in some communities. Institutional experience demonstrated that the need for interpreter services in the radiography suite was driven by three factors: ensuring that the patient can safely undergo the examination, minimizing close contact to ensure patient and staff safety, and ensuring high-quality image acquisition, because COVID radiographic findings can be subtle.

To enhance multilingual patient care in this rapidly evolving setting and to ensure high-quality imaging for all patients during COVID-19, including those with LEP, a web application was developed and piloted that, using novel artificial intelligence–created, natural-sounding audio clips, provides examination instructions to patients in their preferred language. Its utility was evaluated though an implementation science framework using technologist surveys and tracking key operational indicators.

Methods

Context

This project was developed through diversityxMESH, a collaboration between the diversity, equity, and inclusion (DE&I) committee of a large academic medical center’s radiology department and the MESH Incubator, a technology and entrepreneurship center [9,10]. The mission of diversityxMESH is to improve health equity though the development, implementation, and dissemination of practical technologies, relying heavily on user-centered design strategies piloted by MESH (Fig. 1 ) as well as implementation science.

Fig 1

MESH implementation and design philosophy. Adapted with permission from Succi et al [10].

This intervention was guided by the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework, a methodology to comprehensively plan and evaluate public health interventions [11]. Although other metrics focus solely on efficacy, RE-AIM focuses on robustness, translatability, and impact. The RE-AIM framework is applied by evaluating how a project satisfies the following five domains:

Reach—How do I reach the targeted population?

Effectiveness—How do I know my intervention is effective?

Adoption—How do I develop organizational support to deliver my intervention?

Implementation—How do I ensure the intervention is delivered properly?

Maintenance—How do I incorporate the intervention so it is delivered over the long term?

Using the standard lexicon for implementation science put forth by Proctor et al [12], defined in Table 1 , these five domains can be further characterized as eight outcomes: acceptability, adoption, appropriateness, feasibility, fidelity, cost, penetration, and sustainability. These outcomes were considered throughout the design of the application, during its implementation and release, iterative refinement, and data collection, except for fidelity and sustainability, which were outside the scope of this pilot study.

Table 1

Standard lexicon for implementation outcomes put forth by Proctor et al [12] with relevant questions as they pertain to the development and implementation of RadTranslate

Implementation Outcome	Definition	Question
Acceptability	Perception among stakeholders that an intervention is agreeable, palatable, or satisfactory.	Are there rules/perceptions preventing tools that provide translated instructions from being used in outpatient clinics?
Adoption	Individual or organizational decision to try or use an intervention.	How can we create a user-friendly product that people want to use?
Appropriateness	How compatible/relevant is an intervention? For example, if an intervention is effective but requires considerable preparation, it may not be appropriate for a high-throughput clinical setting.	Can we create a simple tool that quickly and simply provides translated commands?
Feasibility	Similar to appropriateness but focused on whether or not there are sufficient resources for the intervention to be successfully used.	Can we create a tool that is device agnostic?
Fidelity	How does the adopted use of an intervention differ from its intended use?	How else will staff members try to use this tool?
Implementation cost	How much does it cost to implement the intervention?	Can we use readily available tools or tools that are relatively low cost?
Penetration	Integration of a practice within a service setting. Mathematically, it is the number of people who routinely use the intervention divided by the people who were trained in the intervention.	Can we ensure the maximum staff draw utility from this product?
Sustainability	Maintenance or continued use or institutionalization of a tool.	Can we create something that is dynamic and continues to meet the needs of end users?

Intervention Development

During the COVID-19 pandemic, imaging end users (nurses, doctors, clinical operations managers, and technologists) of RICs were interviewed and queried about clinical workflow, obstacles to productivity, and acceptability of technology-based solutions. Using the principles of rapid prototyping, RadTranslate™ (www.radtranslate.com), a web application, was developed to provide standardized audible imaging examination instructions to patients in their preferred language using a simple user interface. The application is device agnostic, conforming to any device (desktop, laptop, phone, tablet) and readily accessible, because it requires no login, purchase, or application download; users simply navigate to or bookmark www.radtranslate.com. To maximize adoption and penetration, an iterative design process was used with continual refinement of the interface by end-user feedback, including navigation tree, button size and style, and consensus phrases.

Standardized examination instructions were created by a consensus group of frontline technologists, radiologists, and clinical operations managers. These standardized scripts were translated into Spanish by certified bilingual staff members who were native speakers of the languages they were translating. Translators were instructed to use colloquial language in their translations (eg, “picture” instead of “radiograph”) whenever possible to facilitate comprehension by patients with LEP.

Once written translations were completed, audio instructions were synthesized using a neural text-to-speech program, capable of creating high-quality human-like audio files using synthetic voices [13]. The audio files were integrated with the secure web application and cached to data centers regionally and globally, allowing audio clips to load instantly. Quality control was performed on the audio files by certified bilingual staff members to ensure accuracy and clarity. To solicit continued input from the end users, an integrated feedback system was included, which allows users to suggest new phrases, languages, and any additional updates suggestions.

Study Population

This quality improvement study was approved with exemption by our institutional review board. The study population included all patients over a 2-month period after launch of the application who required Spanish translation during RIC CXR evaluation for which RadTranslate was used. The control group consisted of all patients who required Spanish translation over the 29-day preimplementation period and were conducted by a technologist who subsequently used RadTranslate at least once after its implementation.

Pilot Site Implementation

This pilot study was restricted to Spanish language instructions for CXR. During the study period, beta versions of Mandarin and Portuguese language instructions were implemented, as were audio versions of an institutional COVID-19 screen, falls risk screen, and mammography instructions. RadTranslate was piloted at a predominantly Latino, Spanish-speaking community health center affiliated with a 999-bed quaternary academic medical center. The health center was part of an RIC and included a single radiography suite, with a standardized immediate on-site CXR referral workflow after provider evaluation. Pilot implementation was restricted to instructions for CXR. A dedicated tablet (Apple, Cupertino, California) was affixed to a mobile stand in the RIC’s single radiograph suite and connected to the RIC’s wireless network. The tablet was also connected to a Bluetooth speaker near the patient to increase audio clarity. RadTranslate was bookmarked as the only icon on the tablet desktop and was always open, and all other noncore applications were removed.

The aim of this pilot study was to integrate RadTranslate into the clinical workflow of technologists with secondary outcomes of evaluating if RadTranslate decreased imaging appointment duration (IAD) or the variability in appointment time. As such, use statistics for the application were tracked, including page visits and use of each translated audio phrase. At the pilot site, eight technologists staffed the site, two of whom were fluent in Spanish. Technologists were asked to track their use of the application for CXR by manually recording the radiograph accession number. Using the electronic medical record system (EPIC; Epic Systems Corporation, Verona, Wisconsin), the IAD was calculated as the difference between the time the patient was marked as arrived by the front desk of the imaging clinic to the time the technologist marked the examination as complete.

Two weeks after the implementation of RadTranslate, technologists were given an anonymous, optional 16-question survey that investigated their experience of human interpreters and RadTranslate. A suggestion page was also implemented on www.radtranslate.com, which allowed users to submit phrases, request languages, and provide general feedback.

Analysis

Analysis of IAD was performed using MATLAB (The MathWorks, Natick, Massachusetts). Descriptive statistics were calculated, and IAD was visualized using a box-and-whisker plot with median, upper and lower quartiles, and interquartile range, with outliers considered values greater than 1.5 times the interquartile range away from the respective quartile. The Shapiro–Wilk test was used to test for normality of the different variables. IADs were then compared using Student t test for normally distributed variables; otherwise a nonparametric Mann–Whitney U test was used. Additionally, a power analysis was performed. To evaluate variance in examination times, a two-sample F test for equal variances was performed if the data were well modeled by a normal distribution; otherwise a nonparametric Ansari–Bradley test was used. The collected technologist survey data were qualitatively evaluated.

Results

As the importance of language concordance was understood by staff members, the issue became minimizing the barriers to use of RadTranslate. Although web-based tools offering real-time translation, and in some cases text to speech, are available, staff members were concerned that there was no way to ensure the accuracy and quality of the translation (acceptability). To address this, RadTranslate relies on standardized scripts, agreed upon by an interdisciplinary consensus conference, translated by certified medical translators. Using the accelerated prototyping model used in MESH, translations of CXR instructions were completed within 24 hours of the first standardized language consensus conference, and a beta version of the web application was built within 24 hours after interviews with end-user staff members. The piloted design of the web application can be found in Figure 2 .

Fig 2

Left to right: RadTranslate landing page as accessed on an iPhone, examination selection, chest radiography user interface with Spanish selected as the language, and more chest radiography phrases. Using a device-agnostic infrastructure, the tool will appropriately display on a desktop, tablet, phone, and so on. The distributed database design allows the site and audio clips to load instantaneously.

To ensure that the tool was appropriate for a high-turnover secure environment, navigation was made simple by requiring no user login, securing the web application via industry standard secure sockets layer protocols, and not requiring application store access to download (application store access on institutional devices is prohibited by many hospital networks). At the pilot site, there was 100% penetration among non-Spanish-speaking technologists (six of the eight technologists at the outpatient site did not speak Spanish). The device-agnostic nature of the tool ensured that it was feasible to implement in a variety of environments with varying levels of technology, including older or legacy devices in developing environments. Additionally, the cost to the user is none, because RadTranslate is free, and it can be used on any device with a speaker. Alternatively, the design used here (tablet, mobile stand, Bluetooth speaker) can be replicated using equivalent equipment for $238.

RadTranslate was launched on April 30, 2020. The pilot study period lasted from April 30, 2020, to June 24, 2020, and the preimplementation (control) period was from April 1, 2020, to April 29, 2020. During the pilot period there were 1,267 uses of the web application. Of the specific CXR instructions, the most commonly used Spanish phrases were a general explanation of CXR (30% of total), “Remove everything from the waist up including any necklaces and put the gown on with the opening in the back” (12%), and “Take a deep breath and hold it” (9%). A full list of phrases can be found in Table A1 (see the supplementary online Appendix).

Following implementation at the RIC, all non-Spanish-speaking technologists voluntarily switched exclusively to using RadTranslate for Spanish-speaking patients. A total of 117 patients were included in the analysis (42 men, 75 women; mean age, 47 years; 95% confidence interval [CI], 45-50 years); 18 patients were given instructions using RadTranslate (5 men, 13 women; mean age, 45 years; 95% CI, 37-54 years), and the control group was composed of 99 patients who met inclusion criteria but underwent imaging before implementation of RadTranslate (37 men, 62 women; mean age, 47 years; 95% CI, 45-50 years). The distribution of control cases and RadTranslate cases by performing technologist are presented in Figure 3 .

Fig 3

Distribution of RadTranslate and control patients by performing technologist.

Descriptive statistics including median, upper and lower quartiles, limits, and outliers are detailed in Figure 4 . The control group had six outliers with IADs ranging from 24 to 39 min. There were no outliers in the RadTranslate group. The mean IAD was 11 ± 7 min for the control group and 12 ± 3 min for RadTranslate. The IADs of patients who received RadTranslate were well modeled by a normal distribution. The control group was not well modeled by a normal distribution (P = 6 × 10−8), and statistical evaluation with nonparametric tests was subsequently used. Differences in IADs were not significant between the control and RadTranslate groups using a Mann–Whitney U test (P = .11); however, there was a statistically significant difference in the examination time variance as measured using the Ansari–Bradley test (P = .003). A power analysis for differences in IAD was 64.9%, assuming a two-tailed α level of 0.05.

Fig 4

Imaging appointment duration for the intervention (RadTranslate) versus control (in-person or telephone interpreter).

A user survey was sent to eight technologists, five of whom (33%) noted that they had Spanish-speaking patients for which they used RadTranslate. Of note, 80% reported that traditional interpreter services hindered their workflow, and 60% agreed that interpretation services need to be better integrated into their workflow. Technologist opinions of RadTranslate can be found in Figure 5 .

Fig 5

Survey of technologists at pilot site on their experience with RadTranslate.

Discussion

This pilot study presents a minimal viable product developed using the RE-AIM framework and implementation science to improve quality and health equity in radiographic imaging workflow for patients with LEP. The web-based intervention was implemented during COVID-19 at an interdisciplinary RIC with a relatively high proportion of patients with LEP. The intervention was well adopted by technologists, all of whom agreed that it was an effective addition to their workflow and made their jobs easier. Tracking use and appointment metrics demonstrated no significant difference in IAD between standard of care and RadTranslate, but there was a statistically significant difference in the variability in examination length, with patients receiving instructions using RadTranslate having less variability in IAD. Reduced variability allows better scheduling prediction models to minimize unused room time and reduce patient wait times. Similarly, there were multiple outliers in the standard-of-care group, with appointment times lasting up to 39 min, likely reflecting substantial language barriers and inefficiencies in the current human interpretation, while there were no such outliers in the RadTranslate group.

Consideration of implementation outcomes while designing RadTranslate was important to its success. MESH engineers and developers were deeply engaged in discussions with the interdisciplinary team of doctors, technologists, nurses, and operations staff who served as the expanded user-centered development team. As a result of its user-centered design, RadTranslate had 100% penetration with non-Spanish-speaking technologists. Additionally, although characterization of the fidelity requires follow-up interviews or monitoring its use, the inclusion of diverse role groups throughout the design and fabrication process helped identify the end-user needs and ensure that RadTranslate met those needs. Furthermore, the user feedback option can ensure that the tool adapts to the needs of its end users. At the time of publication, submitted feedback predominantly requested an expansion of languages in addition to expansion of examinations covered, including mammography, ultrasound, and MRI. Similarly, a true measurement of sustainability requires longitudinal evaluation, but preliminary results have shown formal demand from our institution’s emergency medicine and anesthesiology departments for expanded use in their clinical contexts.

As a small-scale proof-of-principle study, there are limitations to the generalizability of the results, mainly due to the small sample size and relative variability in the environment, which in turn affect its ability to measure efficacy of the intervention. The sample size was based on convenience, derived from the available number of patients who underwent the intervention. With a calculated power of 64.9%, this study was underpowered to measure changes in IAD between groups. Additionally, because conserving patient throughput was placed above creating an ideal measurement environment, no requirements were made on operating procedure. Unlike the study of Hartmann et al [6], in which technologists used stopwatches to record the duration of examinations, technologists were only asked to record the examination accession when they used RadTranslate, and data were mined from the electronic medical record. Because of workflow differences among technologists, this limited the time variable to IAD, which included the time after the patient had arrived to the imaging waiting room but before he or she was received by the technologist. The accuracy of some of these measurements is unclear. For instance, an IAD in the control group measured 2 min, meaning that within 2 min after the patient had been checked in by the front desk, the patient’s identity had been confirmed, he or she had changed into a hospital gown, and chest radiographs had been acquired. Also, the use of an interpreter in the imaging suite could not be definitively verified, and thus the control population may contain a mixture of in-person interpreters, telephone interpreters, and cases in which no interpreter was used, although we believe that this approximates real-world experience. To evaluate the impact RadTranslate has on workflow, future work could randomize patients to RadTranslate versus standard of care.

This work underscores the potential for technology-driven interventions to address practical challenges in clinical workflow through an equity lens. The impact of this intervention is significant, because its free access allows it to be easily scaled and distributed to regions where interpreter services may be limited, it can reduce strain on a limited resource of medical interpreters, and it can improve operations efficiency, which may reduce unused equipment times, patient wait times, and potential overcrowding of waiting rooms. Furthermore, because RadTranslate was designed for technologists and with technologist input, it addresses a common problem technologists face and therefore has broad interest.

More broadly, this work demonstrates the utility of collaboration between innovation centers and DE&I initiatives. Using the previously published MESH Incubator health care design-thinking framework, soliciting end users for problem identification, rapid iteration, continued end-user feedback, and ultimately production deployment led to the development of a user-friendly tool to improve the patient experience and operational efficacy. From an institutional DE&I standpoint, recruitment and engagement of groups not traditionally included in innovation or DE&I projects can result in creative solutions, a workforce in which commitment to diversity becomes normalized, and improved patient experience and workflow.

Take-Home Points

The free RadTranslate web application (www.radtranslate.com) was created and successfully pilot at a local community health site during COVID-19 in response to a need for language-concordant provider-patient interactions.

This artificial intelligence–based tool showed a statistically significant reduction in the variability of IAD for patients with LEP.

Recruitment and engagement of groups not traditionally included in DE&I projects can result in novel and creative solutions, a workforce in which commitment to diversity becomes normalized, and improved patient experience and workflow.