Trainee and Attending Perspectives on Remote Radiology Readouts in the Era of the COVID-19 Pandemic.

RATIONALE AND
OBJECTIVES: Social distancing mandates due to COVID-19 have necessitated adaptations to radiology trainee workflow and educational practices, including the radiology "readout." We describe how a large academic radiology department achieved socially distant "remote readouts," provide trainee and attending perspectives on this early experience, and propose ways by which "remote readouts" can be used effectively by training programs beyond COVID-19.
MATERIALS AND METHODS: Beginning March 2020, radiologists were relocated to workspaces outside of conventional reading rooms. Information technologies were employed to allow for "remote readouts" between trainees and attendings. An optional anonymous open-ended survey regarding remote readouts was administered to radiology trainees and attendings as a quality improvement initiative. From the responses, response themes were abstracted using thematic analysis. Descriptive statistics of the qualitative data were calculated.
RESULTS: Radiologist workstations from 14 traditional reading rooms were relocated to 36 workspaces across the hospital system. Two models of remote readouts, synchronous and asynchronous, were developed, facilitated by commercially available information technologies. Thirty-nine of 105 (37%) trainees and 42 of 90 (47%) attendings responded to the survey. Main response themes included: social distancing, technology, autonomy/competency, efficiency, education/feedback and atmosphere/professional relationship. One hundred and forty-eight positive versus 97 negative comments were reported. Social distancing, technology, and autonomy/competency were most positively rated. Trainees and attending perspectives differed regarding the efficiency of remote readouts.
CONCLUSION: "Remote readouts," compliant with social distancing measures, are feasible in academic radiology practice settings. Perspectives from our initial experience provide insight into how this can be accomplished, opportunities for improvement and future application, beyond the COVID-19 pandemic.

INTRODUCTION

The COVID-19 pandemic has led to drastic operational restructuring of radiology departments (1, 2, 3). While effective pharmacologic treatment for COVID-19 is being investigated, traditional public health measures such as isolation, quarantine, and social distancing have been enacted to reduce disease transmission (4). Social distancing refers to efforts to reduce interactions between infected (symptomatic or asymptomatic) and non-infected individuals (4,5). As the novel coronavirus is thought to be transmitted primarily by respiratory droplets, public health experts have recommended people remain physically distant by at least 6-feet (4).

Social distancing mandates have necessitated adaptations of established radiology trainee workflow and educational practices (2), including the radiology “readout,” the traditional practice of side-by-side attending-trainee review of diagnostic imaging exams and case-based teaching.

The purpose of this manuscript is to describe how our large tertiary-care academic radiology department has adapted to social distancing mandates by instituting “remote readouts,” provide trainee and attending perspectives from our early experience, and propose ways by which “remote readouts” can be used effectively in radiology training programs after COVID-19.

MATERIALS AND METHODS

Decentralization of Reading Rooms to Permit Social Distancing

As part of a broader hospital-wide response to the COVID-19 pandemic, on March 6, 2020, our radiology department began to identify offices and conference rooms throughout the hospital system where radiology Picture Archive and Communication System (PACS) workstations could be redistributed, to reduce the number of radiologists within each reading room and to allow for 6-feet distancing between individuals (1).

Institution of Virtual Remote Readouts

Prior to the COVID-19 pandemic, trainees at our institution reviewed radiology examinations and drafted a report to be reviewed with an attending. Draft reports were assigned into the reporting queue of the supervising attending in the dictation system (PowerScribe 360, Nuance, Burlington, MA). After the trainee accumulated several exams, a traditional side-by-side review with the attending ensued; the report was then edited by the trainee and forwarded to the attending for final review, attestation and signature.

To comply with social distancing measures, we utilized commercially-available information technology tools to allow for new means of remote communication between trainee and attending. These included videoconferencing and screensharing technology (Microsoft Teams, Redmond, WA) and chat technology (Microsoft Teams or Primordial messenger, Nuance, Burlington, MA). These were selected as our institution had pre-existing business agreements providing access to both tools and both are Health Insurance Portability and Accountability Act-compliant. Only those with institutional logins can participate in Microsoft Teams videoconferencing/screensharing and only radiologists with institutional logins can participate in Primordial messenger, which is built-in to our PACS system. Our in-house electronic peer learning platform, a closed-loop online feedback reporting tool, was also used to communicate between attending and trainee (6). This tool automatically links to the loaded patient exam, with study information prepopulated. The sender provides information regarding the reason for the feedback and a secure email notifies the receiver. When used for remote readouts, this is usually sent to a trainee for asynchronous feedback on a missed finding or interpretation, or for making a “great call.”

Surveying Trainee and Attending Perspectives on Remote Reads

After 3 weeks in this new workflow, we sought trainee and attending perspectives on remote readouts, to determine opportunities for improvement. We surveyed radiology trainees (41 residents and 64 clinical fellows) and attendings (90 clinical attendings participating in trainee readouts) with an optional, anonymous, open-ended, two-question online questionnaire, asking them to list the “pros” and “cons” of remote readouts.

An attending radiologist examined the responses using thematic analysis to determine thematic categories. Thematic analysis is a method to identify, analyze, organize, describe, and report themes found within a qualitative data set (7). Thematic analysis identified the following themes: (1) social distancing, (2) technology, (3) autonomy/competency, (4) efficiency, (5) patient care, (6) practice variation, (7) education/feedback, and (8) atmosphere/professional relationship. Responses were organized as positive or negative as entered by the respondent and/or based upon the language used in the response. Each open-ended response was subdivided into comments and fit to the above themes. The frequency of each theme addressed by comments were summarized using descriptive statistics. Within each abstracted theme, the net rating of the theme was calculated from the number of positive comments minus the number of negative comments. If a comment applied to more than one theme, it was counted for both of those themes. Responses from trainees and attendings were separately tabulated. This project was undertaken as a Quality Improvement Initiative and as such, was not formally supervised by the Institutional Review Board per their policies.

RESULTS

To enact social distancing measures, between March 16 and 20, 2020, workstations from 14 existing reading rooms were relocated to 36 reading “pods,” each accommodating 1–2 radiologists (trainees and/or attendings). This was done by removing every other PACS station from the pre-existing reading rooms (to allow for 6-feet distancing between workstations) and relocating them to the new pods. These pods were located throughout the hospital system in physically distant rooms, floors, and some off-campus locations. Finally, to comply with social distancing guidelines, side-by-side readouts were suspended.

Using information technology tools, we developed two models of “remote readouts”: synchronous and asynchronous (Table 1 ). Synchronous remote readouts allow for trainees and attendings to readout remotely, but in real-time, utilizing videoconferencing/screensharing technology. “Asynchronous” remote readouts allow attendings to review the images and the report that was drafted by the trainee. The attending could make edits, and sign the report, all without real-time review or discussion with the trainee. Feedback may then be provided at the discretion of the attending, via technology tools including phone calls, chat technology or our in-house peer learning system, a closed-loop online feedback reporting tool (6). Asynchronous readouts were rarely used at our institution during day-time rotations prior to the COVID-19 pandemic.

Table 1

Summary of Synchronous and Asynchronous Remote Readouts

Model	Definition	Ideal Situations for Use	Technology Tools*
Synchronous	Attending and trainee participate in real-time review of images and discussion of preliminary report utilizing videoconferencing/ screensharing technology	• Junior residents • Complex indication/exam • Trainee preliminary interpretation is inaccurate or needs revision	• Videoconference/screensharing -Microsoft Teamsa• Phone call while simultaneously looking at images
Asynchronous	Attending reviews images and preliminary report separately from trainee, edits and finalizes report and provides trainee feedback at a later time	• Senior residents and fellows • Common or uncomplicated indication/exam • Trainee preliminary interpretation is accurate	• Videoconference/screensharing -Microsoft Teamsa• Chat -Microsoft Teams-Primordialb messenger-Text message• Phone call • In-house peer learning tool (6) • Report comparison tool in PowerScribe 360 systemc

Need to ensure Health Insurance Portability and Accountability Act (HIPAA) compliance before use.

Microsoft Teams (Microsoft, Redmond, WA).

Primordial (Nuance, Burlington, MA).

PowerScribe 360 (Nuance, Burlington, MA); comparison tool allows for comparison of final report to prior drafts.

To facilitate communication and denote exams which are ready for attending review, some divisions implemented template reporting language (reporting macros) for trainees to insert as the first line of their draft report impression: “ready” or “review.” “Ready” indicates to the attending via their reporting queue that the report is ready for attending revision usually in an asynchronous manner, while “review” indicates a need for a dedicated trainee-attending synchronous readout.

Survey Responses: Trainee and Attending Perspectives on Remote Reads

A total of 39 of 105 (37%) trainees and 42 of 90 (47%) attendings responded to the survey. Responses represented a total of 245 thematic comments, 148 (60%) positive and 97 (40%) negative. Trainee responses represented a total of 110 comments: 72 positive (65%, mean 1.85 per trainee, range number of comments 1–4), and 38 negative (35%, mean 0.97 per trainee, range number of comments 0–2). Among trainees, the difference between the number of positive and negative comments results in a net rating of +34. Attendings provided a total of 135 comments: 76 positive (56%, mean 1.81 per attending, range number of comments 1–5), and 59 negative (44%, mean 1.4 per attending, range number of comments 1–3). Among attendings, the resultant net rating was +17. Seventeen comments applied to more than one theme. Descriptive data are reported in Table 2 . Representative comments are included in Tables 3a and 3b .

Table 2

Themes Abstracted From Trainee and Attending Perspectives of Remote Readouts*

	Positive Comments		Negative Comments		Net Rating⁎⁎
	Trainee	Attending	Trainee	Attending	Trainee	Attending
Total	72	76	38	59	+34	+17
Social distancing	14 (19%)	12 (16%)	1 (3%)	1 (2%)	+13	+11
Autonomy/Competency	6 (8%)	14 (18%)	-	-	+6	+14
Technology	17 (23%)	18 (24%)	11 (29%)	5 (8%)	+6	+13
Efficiency	20 (27%)	18 (24%)	4 (10%)	19 (32%)	+16	−1
Patient Care	1 (1%)	1 (1%)	-	-	+1	+1
Practice Variation	-	-	-	1 (2%)	-	−1
Education/Feedback	10 (14%)	12 (16%)	12 (32%)	17 (29%)	−2	−5
Atmosphere/ professional relationship	4 (6%)	1 (1%)	10 (26%)	16 (27%)	−6	−15

Counts of comments (percentage of column total).

Net rating calculated from net of positive minus negative comments.

Table 3a

Representative Positive Comments by Trainees and Attendings*

Trainees	Attendings
Atmosphere/personal	Atmosphere/personal
The atmosphere is more relaxed.	Great ability to stay in touch with trainees during this difficult time.⁎⁎
I feel more comfortable throughout the day as I'm assigned to one attending.
Autonomy/competency	Autonomy/competency
I have learned to become more definitive in my reporting and obtain more sense of independence.	This has encouraged residents to take more ownership of reports
More independence. Good transition to practice.	Can see fully formed (or unformed) thoughts of trainees; better assesses competency as they have to commit to an interpretation
Education/feedback	Education/feedback
Feedback focuses on missed incidentals and recommendations that would change management.	You interact with fewer trainees on a daily basis, so you get to know their work and skills better.
I'm still getting the benefit of reading cases actively and forming differentials.	Can easily have multiple trainees observe/participate in readouts.
Efficiency	Efficiency
Faster readouts and able to read more cases if volume allows.	More efficiency; fewer interruptions of trainees.
It's quicker when the attending just texts over Primordial if they agree or minor adjustments—prefer this for simpler cases.	Can review on own schedule.
Patient Care	Patient Care
There are two blind reads on each case.	Double, independent reads on each scan.
Social distancing	Social distancing
Keeps social distancing possible.	Hopefully, flattening the curve.
No risk of getting COVID from attending.	Confirms to social distancing guidelines in the era of COVID.
Technology	Technology
Primordial messenger, phone calls and in-house peer learning tool (6) are great tools used by different attendings. Comparison tool in PowerScribe 360 is handy to review changes made by attending in each report!	I like the ability to take control of the screen in both Zoom and Microsoft TeamsGreat ability to stay in touch with the trainees during this difficult time.⁎⁎
Can share screens. Can request control. Can see each other's cursors.

Comments edited only for readability, not for content.

Example of doubly counted comment.

Table 3b

Representative Negative Comments by Trainees and Attendings*

Trainees	Attendings
Atmosphere/ personal	Atmosphere/ personal
Getting to know the other fellows/residents/attendings is a little more challenging.	Loss of important collegial interactions.
Miss in-person interaction with attendings.	Inability to actively engage and be sure of understanding through facial expressions and non-verbal cues.
Education/feedback	Education/feedback
Would have been a problem if it happened early in the fellowship year where there was more discussion at readouts regarding foundational concepts.	Decreases feedback detail/discussion.
Loss of finer teaching points/granularity, less discussion about cases, more difficult to understand “normal” and/or what's not important for junior trainees.	Limited feedback to trainees, only summary of important discordant findings.
Efficiency	Efficiency
Have to call/message if I have questions.	Much slower and less efficient, particularly for complex cases.
Traditional side-by-side readout is more efficient for difficult cases with complex history.	More time consuming to review asynchronously/edit reports (which usually come nearly completely edited after in-person read outs).
	Many typos in reports—trainees are not proofreading before sending them to draft status.
Practice Variation	Practice Variation
	Everyone doing it differently; some still reviewing in real time [using technology]—others only reviewing if they disagree; no real direction given as to what is preferred (if any particular way).
Social distancing	Social distancing
When attending is in the same room, some staff forget that they have to maintain social distancing, so reading out in separate rooms and utilizing phones or communication applications is better.	If the resident is in the reading room and attending at home, it causes discomfort/unease because the attending is at a safer distance.
Technology	Technology
When not screensharing, exact finding/location of finding may be unclear.	Variability in hardware and internet connection can create disparities in qualities of readouts.
Audio doesn't work all the time (audio comes through the Dictaphone and also makes echoes) and have to talk over the phone instead.	Slow internet connection can blur the images or cause lag in quality.

Comments edited only for readability, not for content.

Positively Rated Themes and Comments

Four themes resulted in positive net ratings by both trainees and attendings: social distancing (+13 trainees, +11 attendings), technology (+6 trainees, +13 attendings), autonomy/competency (+6 trainees, +14 attendings), and patient care (+1 trainees and attendings). Efficiency was positively rated by trainees (+16), but negatively rated by attendings (−1).

In aggregate, the most net positively-rated theme was that of social distancing among trainees (+13) and attendings (+11). Positive comments from both groups endorsed remote readouts for their intended benefits of social distancing and reducing the risk of contracting the novel coronavirus while at work (26 of 148 positive comments, 18%).

Among positive comments, the most prevalent theme described by both trainees and attendings was efficiency (total 38 of 148, 26%; 20 of 72 trainee comments and 18 of 76 attending comments), although as mentioned above, there was a slight overall net negative rating by attendings (−1, representing 19 of 59 attending comments). Both groups described faster readouts due to less time spent reviewing straightforward exams by utilizing asynchronous readout. Attendings also reported asynchronous readouts leading to increased flexibility in the scheduling/timing of reviewing cases.

The second-most prevalent theme was the application of information technology as it relates to remote readouts (35/148, 24%). Trainees and attendings described ease of use of most of the tools and the beneficial ability to use different tools for different needs (whether synchronous or asynchronous readouts). Both groups also reported using these technology tools to engage more than one trainee during the readout (including trainees at home) or record readout/teaching (in a Health Insurance Portability and Accountability Act-compliant manner) for educational purposes.

Trainee and attending comments reflected only positives and no negatives in regards to trainee autonomy/competency (30/148, 20%), primarily related to use of asynchronous remote readouts: trainees reported feeling more independence, prompting them to be more definitive in their interpretations, while attendings reported an enhanced ability to assess trainee competency by being able to review their independently drafted reports before the readout.

One trainee and attending both noted a potential benefit to patient care (2/148, 1%), as when utilizing the asynchronous model, each exam was now being interpreted by two, unbiased, independent readers.

Negatively Rate Themes and Comments

Negative comments represented 40% of thematic comments (97/245). Two themes resulted in net negative ratings by both trainees and attendings: education/feedback (−2 trainees, −5 attendings) and atmosphere/professional relationship (−6 trainees, −15 attendings).

Among the negative comments, the most prevalent theme described by both trainees and attendings was worsening of atmosphere and professional relationships (26 of 97 negative comments, 27%). This theme was also the most negatively-rated by attendings (−15). Comments from both groups described feelings of isolation and loss of camaraderie as an indirect effect of social distancing guidelines in the workplace.

Trainees and attendings negatively rated the theme of education/feedback (−7, 29/97, 30%). Trainees and attendings described asynchronous readouts leading to decreased direct feedback and discussion about the exams, especially about less clinically important findings. Some attendings also noted loss of non-verbal cues to sense trainee engagement and understanding during readouts and feedback.

As described above, while the theme of efficiency was positively-rated by trainees (+16, 20 of 72 positive comments, 27%), it was the most prevalent among the negative comments by attendings and was overall negatively rated (−1; 19 of 59 negative comments, 32%). Most comments cited increased time to review cases asynchronously, particularly complex exams, and due to the additional time required edit trainees’ reports, which is usually done by the trainees during and after side-by-side readouts.

A single attending shared a negative comment in the theme of practice variation, reflecting a lack of specific instruction from the department on how and when to perform synchronous versus asynchronous readouts, leading to variation between attendings.

DISCUSSION

Within a 2-week period, our academic radiology department decentralized reading rooms into one-to-two person reading “pods” and developed an infrastructure for remote attending-trainee exam readouts utilizing communication technology tools. Early perceptions of remote readouts by radiology trainees and attendings was generally positive, specifically regarding the themes of social distancing, technology, and autonomy/competency, with negatively rated themes of education/feedback and atmosphere/professional relationships. As social distancing is unlikely to be needed (or deemed positively) in the postpandemic future, even when excluding the results of this theme, remote readouts are still perceived to be more positive than negative by both trainees and attendings (net +21 by trainees and +6 by attendings).

Effective Social Distancing and Impact on Atmosphere and Professional Relationships

While remote readouts were most positively rated in the theme of social distancing, they were most negatively rated in the theme of atmosphere/personal relationships, to a greater extent by attendings (net rating −15) than trainees (−6). Social distancing measures may be associated with perceptions of increased social isolation (2), and these findings further underscore the importance of department-wide efforts to address the social isolation experienced by radiology trainees and attendings (1,2,8). At our institution, the radiology resident wellness committee has scheduled weekly videoconference events for the trainees (Gaviola et al, unpublished report, 2020) and a department-wide virtual community was piloted via a Microsoft Teams to promote and share wellness resources.

Enhanced Trainee Autonomy and Attending Assessment of Trainee Competency and Effect on Education/Feedback

Trainee autonomy/competency was the only theme to be solely positively rated by both trainees and attendings. Trainees reported a greater sense of independence when using the asynchronous readout model, requiring them to become more definitive in their interpretations, while attendings identified asynchronous readouts as a means to review trainees’ “unedited” exam interpretations.

This suggests potential roles for asynchronous readouts in the postpandemic future. This model may facilitate graduated responsibility for senior trainees, to review asynchronously a larger number or more complex exams. Trainees still benefit from attending feedback, in the context of greater trainee autonomy and a supervised experience of post-training clinical practice. Use of this model has previously been reported for readout of residents on call rotations (9, 10, 11).

Asynchronous readouts can be used by attendings as a competency assessment tool. For example, they can be employed in the last days of a trainee's rotation as a way to review their unedited exam interpretations for knowledge base and clinical reasoning.

While there may be benefits to asynchronous remote readouts, the results suggest a possible negative effect on trainee education and feedback. To maximize education benefit, attendings should be encouraged to provide teaching and feedback for every exam, even those which are straightforward or with clinically insignificant misses by the trainee. Attendings should also be encouraged to use synchronous readouts for complex exams or in which the trainee's draft needs revision.

Efficiency

While there were a large number of positive comments by both trainees and attendings regarding the efficiency of remote readouts, the overall net rating differed between both groups (net rating −1 for attendings and +16 for trainees). Negative perception by attendings may be related to the use of asynchronous readouts for complex exams, with junior trainees, or when faced with poorly-composed report drafts, while for trainees, the asynchronous model may be perceived as more efficient as relatively less time is spent reviewing the case with the attending.

To improve efficiency of asynchronous readouts for attendings, it should be emphasized to trainees to thoroughly edit report drafts, to decrease the amount of time the attending spends editing typographical errors. Moreover, while the benefits of asynchronous readouts are synergistic with imaging exams and indications of relatively lesser complexity, attendings may benefit from preferentially employing synchronous readouts for more complex exams or those read with more junior trainees.

Overall, asynchronous readouts may result in efficiency benefits in certain circumstances postpandemic, such as when confronted with high exam volumes (9), particularly those of lesser complexity or read by senior trainees who may not require as much direct discussion and feedback.

Technology

Remote readout technology was positively rated by both attendings and trainees for ease of use and diversity allowing for different functions; however, negative comments regarding issues with hardware and internet connections demonstrate room for improvement. As this technology continues to be used, the department can work on addressing these issues by providing the needed hardware (such as microphones) and faster internet connection.

Increased familiarity with these tools has also provided opportunities for use after resolution of the pandemic. Traditionally at our institution, attendings at satellite locations read studies independently, without trainees, given inability to participate in side-by-side readouts. By using the technological tools discovered during the pandemic, these attendings could be assigned trainees (as above, ideally more senior residents and/or fellows) and partake in both synchronous and asynchronous remote readouts.

Limitations

This report provides insight on the impact of social distancing measures on diagnostic radiologist workflows at an academic medical center and identifies opportunities for future process improvement and investigation. Its limitations include the fact that our results are based on early experience with remote readouts and we may encounter changing perceptions by trainees and attendings as the pandemic continues to unfold and our workflow evolves.

Demographic information, including trainee postgraduate year and attending years in practice was not obtained to maintain survey anonymity. However, several of the comments indicate potential differences in opinions regarding synchronous versus asynchronous reads may be dependent upon trainee level. It is also possible that attending years in practice may impact comfort level in reading out trainees remotely and also in use of new technologies. Additionally, as a single attending comment on practice variation mentions, aside from sharing technology capabilities, no guidelines were given to trainees or attendings on how conduct remote readouts, which has likely led to variable experiences in both groups. Although the responses suggest some benefits of remote readouts, especially asynchronous readouts, these reported positives may not persist in a nonpandemic environment when side-by-side readouts can be resumed. We also did not objectively assess whether asynchronous readouts are in fact more or less efficient than traditional readouts. Finally, although one trainee and attending each felt there were positive patient care benefits, we did not directly evaluate whether there is a difference in quality or accuracy of reports generated during remote readouts compared to traditional side-by-side readouts.

CONCLUSION

Socially distant “remote readouts” can be successfully implemented in a large academic radiology department as a response to the COVID-19 pandemic. The early perspectives provided by initial trainee and attending experience with remote readouts demonstrates both how this can be accomplished and opportunities for improvement. As we look forward to when we can return to the reading room side-by-side, it may be beneficial to retain elements of remote readouts for integration in the post-COVID future.