Structured Reporting versus Free-Text Reporting for Appendiceal Computed Tomography in Adolescents and Young Adults: Preference Survey of 594 Referring Physicians, Surgeons, and Radiologists from 20 Hospitals.

OBJECTIVE: To survey care providers' preference between structured reporting (SR) and free-text reporting (FTR) for appendiceal computed tomography (CT) in adolescents and young adults.
MATERIALS AND METHODS: An ethical committee approved this prospective study. The requirement for participant consent was waived. We distributed the Likert scale-based SR form delivering the likelihood of appendicitis across 20 hospitals through a large clinical trial. In the final phase of the trial, we invited 706 potential care providers to participate in an online survey. The survey questions included usefulness in patient management, communicating the likelihood of appendicitis, convenience, style and format, and overall preference. Logistic regression analysis was performed for the overall preference. Three months after the completion of the trial, we checked if the use of the SR was sustained.
RESULTS: Responses were analyzed from 594 participants (175 attendings and 419 trainees; 225 radiologists, 207 emergency physicians, and 162 surgeons). For each question, 47.3-64.8% of the participants preferred SR, 13.1-32.7% preferred FTR, and the remaining had no preference. The overall preference varied considerably across the hospitals, but slightly across the departments or job positions. The overall preference for SR over FTR was significantly associated with attendings, SR experience for appendiceal CT, hospitals with small appendectomy volume, and hospitals enrolling more patients in the trial. Five hospitals continued using the SR in usual care after the trial.
CONCLUSION: Overall, the care providers preferred SR to FTR. Further investigation into the sustained use of the SR is needed.

INTRODUCTION

Acute appendicitis is the most common indication for emergent abdominal surgery. Computed tomography (CT) plays a vital role in the diagnosis of appendicitis owing to its availability and excellent diagnostic performance. Care providers are often dependent on CT results in determining patient management, ranging from hospital discharge to appendectomy; because clinical and laboratory findings are known to be insufficient for accurate diagnosis (1). Preoperative CT utilization rates in patients undergoing appendectomy now reaches 90% in the United States (2) and South Korea (3).

CT results regarding the likelihood of appendicitis need to be delivered accurately and efficiently by radiologists to referring physicians or surgeons. This is particularly important when CT findings are not judged as “definitely positive” or “definitely negative” by radiologists. Equivocal results are reported in approximately 5–13% of appendiceal CT examinations and 31–52% of the equivocal cases are subsequently confirmed as appendicitis (45). In such uncertain cases, accurate diagnosis and appropriate patient management are determined through further in-hospital clinical observations (1) and/or additional imaging tests (67). The failure to communicate a diagnostic uncertainty (or certainty) may lead to unnecessary additional observations or imaging tests, potentially resulting in inappropriate or delayed patient management. Miscommunication may be compounded by involvement of many care providers with varying experience levels including rotating trainee doctors (89), who use different or ambiguous terms to deliver diagnostic certainty. Further confusion may result when non-English speaking care providers attempt to communicate the diagnostic certainty in English, which is not a rare situation throughout the globe.

The diagnostic certainty needs to be clearly communicated, ideally in a highly-standardized manner. A Likert scale-based structured reporting (SR) form may be an effective measure for successful standardization. The need for SR has long been recognized in the field of radiology (101112). However, recent surveys (1314) showed that the SR is rarely used in practice except for breast (15) or cardiac imaging (16). It has therefore been suggested that SR should be developed for clinically important but less complex examinations, where communication issues have been observed (17). In this context, several researchers have advocated the use of SR for appendiceal CT (1819) or ultrasonography (202122). To date, the proposed SR forms have not gained much traction.

We recently had the opportunity to distribute SR form for appendiceal CT across 20 hospitals through the course of a large clinical trial (Low-dOse CT for Appendicitis Trial, LOCAT; http://www.locat.org) (723). The trial aimed to compare low-dose (2 mSv) and standard-dose (8 mSv or less) CT in the diagnosis of appendicitis, among patients aged 15–44 years. We anticipated that standardized communication using the purpose-designed SR form may lead to better diagnostic and clinical outcomes. An investigation to demonstrate the advantages of SR over free-text reporting (FTR) in association with patient outcomes would require an unrealistically large sample. This is because regardless of the reporting style, the diagnostic performance of CT for appendicitis is already excellent, which leaves only a small margin of improvement by using SR over FTR. It is more practical to first survey the willingness of care providers to adopt the SR. This study aimed to survey care providers' preference for the use of SR vs. FTR for appendiceal CT, in adolescents and young adults.

MATERIALS AND METHODS

This prospective study was approved by an ethical committee. A waiver of consent was granted. In October 2012, the LOCAT lead investigator invited 121 hospitals to participate in LOCAT through the Korean Society of Abdominal Radiology network. Twenty teaching hospitals with a median annual appendectomy caseload of 269 (interquartile range, 158–428) participated in LOCAT (723). LOCAT finally enrolled 3074 patients aged 15–44 years and more than 500 care providers participated in LOCAT from December 2013 to February 2017. We disseminated the SR form for appendiceal CT across the 20 hospitals as a part of the LOCAT protocol. The SR form has been used in research studies (2425) and also as part of usual care in the LOCAT lead center. The form aimed to determine the likelihood of appendicitis on a 5-point Likert scale (Supplementary Table 1 in the online-only Data Supplement). During the study period, the 20 LOCAT sites used the SR form for all trial patients and also as a part of usual care.

Our survey was conducted from February 2016 through November 2016, during the last phase of LOCAT. Three radiologists jointly created the online survey after reviewing the literatures for key features of a high-quality radiology report (1011171826). The survey included five preference questions (question items Q3–Q7) including usefulness in patient management, communicating the likelihood of appendicitis, convenience, style and format, and overall preference (Table 1). Responses were recorded using a 5-point Likert scale ranging from “definitely SR” to “definitely FTR.” Each participant was also questioned regarding his/her prior clinical experience with using an SR for appendiceal CT (Q1), and for any other CT applications (Q2), as well as his or her commitment in answering all the questions (Q8). An invitation with a link to the survey was emailed to 706 care providers, who were reported by the site principal investigators to be involved in the care of trial patients in 2016. Invitees included 231 radiologists, 254 emergency physicians, and 221 surgeons. The median number of the invitees per site was 31 (interquartile range, 23–40) (Fig. 1, Table 2). The initial rollout involved three hospitals, and then gradually expanded to other hospitals. Weekly reminder emails were sent to non-responding invitees to solicit their participation, until the response rate for each hospital reached 80%.

Table 1

Survey Questionnaire

Question Items	Questions
Prior experiences in SR
Q1. For appendiceal CT	(For radiologists) How many cases of SR have you made (, assisted in making, or revised)?
	(For referring physicians and surgeons) How many cases of SR have you reviewed?
Q2. For other CT application(s)	Have you ever experienced SR in any CT application(s) other than appendiceal CT?
Preference between SR and FTR*
Q3. Usefulness in patient management	Which is more useful in patient care?
Q4. Communicating likelihood of appendicitis	Which is clearer in communicating likelihood of appendicitis between care providers?
Q5. Convenience	Which usually requires less time or effort for you in making (or reviewing) report?
Q6. Style and format	Which do you prefer in terms of style and format?
Q7. Overall preference	Which do you overall prefer to use in your future practice?
Q8. Commitment in response†	How would you describe your commitment in answering above questions?

Participants were informed that all questions but Q2 and Q8 were pertinent to appendiceal CT in patients aged 15–44 years. *Responses were obtained using 5-point Likert scale: definitely SR, probably SR, indeterminate, probably FTR, definitely FTR, †Responses were obtained using 5-point Likert scale: never committed, hardly committed, unsure, partly committed, fully committed. CT = computed tomography, FTR = free-text reporting, SR = structured reporting

Fig. 1

Flow diagram for selection of study participants.

Table 2

Number of Participants

Care Providers	Invitees	Participants	Included in Analysis
Total	706, 31 (23–40)	608 (86.1%), 25 (20–35)	594 (84.1%), 25 (20–35)
Radiologists	231, 11 (7–14)	229 (99.1%), 10 (7–14)	225 (97.4%), 10 (7–14)
Attendings	62, 3 (2–4)	61 (98%), 3 (2–4)	61 (98%), 3 (2–4)
Trainees	169, 8 (6–11)	168 (99%), 8 (6–11)	164 (97%), 8 (6–11)
Emergency physicians	254, 11 (8–14)	210 (82.7%), 9 (7–12)	207 (81.5%), 9 (7–12)
Attendings	78, 4 (2–5)	64 (82%), 3 (2–5)	64 (82%), 3 (2–5)
Trainees	176, 8 (4–13)	146 (83%), 7 (3–10)	143 (81%), 7 (3–9)
Surgeons	221, 9 (6–13)	169 (76.5%), 7 (5–13)	162 (73.3%), 7 (4–12)
Attendings	74, 4 (3–6)	53 (72%), 3 (2–5)	50 (68%), 3 (2–4)
Trainees	147, 7 (4–15)	116 (79%), 6 (3–12)	112 (76%), 6 (2–10)

Data are numbers of care providers (and percentages out of invitees), median numbers of care providers per hospital (and interquartile ranges).

In May 2017, three months after the completion of LOCAT, we conducted another online interview with 20 radiologists who respectively led the LOCAT study at each of the 20 hospitals. We asked each of the radiologists which of SR or FTR the hospital was using for appendiceal CT in their usual care of patients aged 15–44 years. For hospitals not consistently using SR, the reason/s for not using SR were sought. We also asked if the hospital was shifting to SR in the near future.

We analyzed the data after excluding participants with insufficient self-reported commitment (defined as a response of “never committed,” “hardly committed,” or “unsure” to Q8) in the response. Descriptive statistics, including percentages and medians, were used to summarize the results. Stacked bar graphs were generated to illustrate the results for subgroups, categorized by specialty department and job position, and for individual hospitals.

For the overall preference (Q7), a multivariate logistic regression analysis was performed after binary collapse of the 5-point scale responses, such that “definitely SR” or “probably SR” were included as preference for SR. The tested covariates included participant characteristics (department, job position, and prior experience in SR) and hospital characteristics (annual number of appendectomies, and the number of enrolled patients in LOCAT). A p value of < 0.05 was considered to indicate statistical significance. Stata 14.0 (StataCorp LLC, College Station, TX, USA) was used for all statistical analyses.

RESULTS

Of the 706 invitees, 608 (86.1%) participated in the survey. The participation rate was 99.1% (229/231), 82.7% (210/254), and 76.5% (169/221), for the radiologists, emergency physicians, and surgeons, respectively (Table 2). Four radiologists, three emergency physicians, and seven surgeons were excluded from the analysis due to self-reported insufficient commitment in the response (Supplementary Table 2 in the online-only Data Supplement). Final analyses included a total of 594 participants (median 25 [interquartile range, 20–35] per hospital; including 225 radiologists, 207 emergency physicians, and 162 surgeons). Participants had variable prior experience in SR for appendiceal CT: 143 (24.1%), 155 (26.1%), 158 (26.6%), and 138 (23.2%) of them had experience of 10 or less, 11–30, 31–100, and more than 100 cases, respectively. For other CT applications, 320 (53.9%) participants had prior experience in SR (Table 3).

Table 3

Participant Characteristics

Characteristics	Radiologists (n = 225)	Emergency Physicians (n = 207)	Surgeons (n = 162)	Total (n = 594)
Overall clinical experience
Attendings (%)	61 (27.1)	64 (30.9)	50 (31)	175 (29.5)
Clinical experience (years)*	9 (5–16)	9 (6–12)	10 (8–16)	9 (6–15)
Trainees (%)	164 (72.9)	143 (69.1)	112 (69)	419 (70.5)
Prior experiences in SR
For appendiceal CT (cases) (%)
0–10	88 (39.1)	33 (15.9)	22 (14)	143 (24.1)
11–30	55 (24.4)	57 (27.5)	43 (27)	155 (26.1)
31–100	43 (19.1)	69 (33.3)	46 (28)	158 (26.6)
101–500	33 (14.7)	38 (18.4)	42 (26)	113 (19.0)
> 500	6 (2.7)	10 (4.8)	9 (6)	25 (4.2)
For other CT application(s) (%)
Experience	132 (58.7)	99 (47.8)	89 (55)	320 (53.9)
No experience	93 (41.3)	108 (52.2)	73 (45)	274 (46.1)

*Data are medians (and interquartile ranges). Otherwise, data are numbers of participants (and percentages). Percentages may not add up to 100% because of rounding.

The numbers of participants who responded “definitely SR” or “probably SR” to Q3–Q7 were 319 (53.7% [95% confidence interval, 46.9–57.8%]), 385 (64.8% [60.8–68.7%]), 291 (49.0% [44.9–53.1%]), 281 (47.3% [43.2–51.4%]), and 306 (51.5% [47.4–55.6%]), respectively. In contrast, the numbers of participants who responded “definitely FTR” or “probably FTR” to Q3–Q7 were 88 (14.8% [95% confidence interval, 12.1–17.9%]), 78 (13.1% [10.5–16.1%]), 159 (26.8% [23.2–30.5%]), 194 (32.7% [28.9–36.6%]), and 163 (27.4% [23.9–31.2%]), respectively. The remaining 187, 131, 144, 119, and 125 participants responded with “indeterminate” to Q3, 4, 5, 6, and 7, respectively. For the overall preference (Q7), 87 (14.6%), 219 (36.9%), 125 (21.0%), 132 (22.2%), and 31 (5.2%) participants responded with “definitely SR,” “probably SR,” “indeterminate,” “definitely FTR,” and “probably FTR,” respectively. In terms of communicating the likelihood of appendicitis (Q4), 121 (20.4%), 264 (44.4%), 131 (22.1%), 60 (10.1%), and 18 (3.0%) participants responded with “definitely SR,” “probably SR,” “indeterminate,” “definitely FTR,” and “probably FTR,” respectively.

For each of Q3–Q7, there were only minor variations in the response pattern across the participants' departments or job positions (Fig. 2). The overall preference (Q7) for SR over FTR was slightly higher among surgeons and emergency physicians than among radiologists, and among attendings than among trainees. In terms of communicating the likelihood of appendicitis (Q4), the preference for SR was slightly higher among attendings than among trainees.

Fig. 2

Stacked bar graph showing survey responses for questions Q3–Q7 regarding preference between SR and FTR according to participants' departments and job positions.

Length of each stack indicates percentage of participants in that category with respect to entire study group. Each number is number of participants included in stack. FTR = free-text reporting, SR = structured reporting

There were considerable inter-hospital variations in the response pattern (Fig. 3). The per-hospital percentage of the participants who responded with “definitely SR” or “probably SR” to Q7 ranged from 30% to 74% across the 20 hospitals. The percentage was above 50% in 11 hospitals, and below 50% in nine. On the contrary, the per-hospital percentage of participants who responded with “definitely FTR” or “probably FTR” was below 30% in 12 hospitals, 30–50% in seven, and 60% in one.

Fig. 3

Stacked bar graph showing per-hospital survey responses for overall preference (Q7) between SR and FTR.

Length of each stack indicates percentage of participants in particular hospital with respect to entire study population. Each number is number of participants included in stack. As of May 2017, five hospitals (denoted by *) were completely using SR, eight hospitals (denoted by †) were partially using SR, and seven hospitals (denoted by ‡) were rarely using SR in usual care.

The logistic regression analysis showed that the overall preference for SR over FTR for future practice (i.e., responses of “definitely SR” or “probably SR” to Q7) was significantly associated with attendings (p = 0.007), prior experience with using SR for appendiceal CT in more than 100 cases (p = 0.003), hospitals performing less than 400 appendectomies annually (p = 0.048 or less), and hospitals with 170 or more patients enrolled in LOCAT (p = 0.030). More surgeons preferred SR to FTR than radiologists, although the difference did not reach statistical significance (Table 4).

Table 4

Multivariable Logistic Regression Analysis for Overall Preference (Q7)

Variables	Prefer SR to FTR* (%)	Odds Ratio (95% CI)	P
Participant characteristics
Department
Radiologists	103/225 (45.8)	Reference
Emergency physicians	110/207 (53.1)	1.14 (0.75–1.72)	0.54
Surgeons	93/162 (57)	1.39 (0.89–2.15)	0.144
Job position
Trainees	200/419 (47.7)	Reference
Attendings	106/175 (61)	1.67 (1.15–2.43)	0.007
Prior experiences in SR
For appendiceal CT
0–10	61/143 (43)	Reference
11–30	72/155 (46)	1.06 (0.65–1.71)	0.82
31–100	77/158 (49)	1.09 (0.67–1.79)	0.72
≥ 101†	96/138 (70)	2.27 (1.33–3.89)	0.003
For other CT application(s)
No experience	132/274 (48.2)	Reference
Experience	174/320 (54.4)	1.16 (0.82–1.65)	0.39
Hospital characteristics
Annual number of appendectomies
< 200	90/195 (46)	1.72 (1.01–2.95)	0.048
200–399	118/201 (58.7)	2.05 (1.29–3.24)	0.002
≥ 400	98/198 (49)	Reference
Number of enrolled patients in trial
	74/157 (47)	Reference
60–169	95/214 (44.4)	0.78 (0.50–1.21)	0.267
≥ 170	137/223 (61.4)	1.84 (1.06–3.19)	0.030

*Data are numbers of participants preferring SR to FTR out of all participants in category (and percentages), †Categories had to be merged because of small number of participants in extreme category. CI = confidence interval

As of May 2017, five hospitals were using the SR form in nearly all relevant patients. Eight hospitals were using the SR form in some cases. Seven hospitals rarely used the SR in usual care. The reasons for not using SR in the 15 partially-using or rarely-using hospitals included: care providers' unfamiliarity to SR, and the lack of SR education or promotion to the care providers. Two of the 15 hospitals were planning to completely shift to SR in the near future, ten hospitals chose to keep using FTR partially, and the remaining three hospitals decided to abandon SR (Supplementary Table 3 in the online-only Data Supplement).

DISCUSSION

FTR for radiologic studies has been criticized due to non-standardized, ambiguous descriptors for diagnostic certainty (10). Considerable discrepancies between radiologists' intended nuances in their reports and the referring physicians' understanding of the reports exist (272829). SR is often advocated for improving communication (1011172730). However, it is well known that SR does not always increase referring physicians' satisfaction, or reporting accuracy (31). SR has also been criticized for the inefficiency in making reports due to frequent disruptions of established routines in image interpretation, and the rigidity of the form, which limits descriptions for complex cases or unexpected finding (31323334). Due to these limitations, current usages of SR have not met expectations (131426).

However, appendiceal CT in adolescents and young adults is uniquely suitable for SR. First, the diagnostic task is simple (i.e., the likelihood of appendicitis), and the report conclusion can be efficiently summarized in SR using a Likert scale. Second, unexpectedly complex cases are rare, since most alternative diagnoses are limited to a small number of non-serious diseases in the target patient population (523). Third, standardized communication is essential because interdisciplinary collaborative decisions need to be made urgently and often involve less experienced care providers including rotating trainee doctors, who may use different or ambiguous diagnostic descriptors.

Our survey involved 594 care providers from 20 hospitals. In comparison, recent studies advocating for the use of SR in appendiceal ultrasonography (202122) or CT (1819) involved only smaller numbers of participants, from one or two hospitals. Our result showed the participants' overall preference for SR over FTR, although not unanimously. Responses to the preference questions (Q3–Q7 regarding the usefulness in patient management, communicating the likelihood of appendicitis, convenience, style and format, and overall preference) indicated that 47.3–64.8% of the participants preferred SR (responding with either “definitely” or “probably”), while 13.1–32.7% preferred FTR (responding with either “definitely” or “probably”). In total, 51.5% of the participants responded as preferring SR, while 27.4% responded as preferring FTR in the overall preference (Q7), which we considered as the most comprehensive and important question item.

Importantly, the overall preference between SR and FTR varied considerably across the hospitals, while there were only minor variations across the participants' departments or job positions. This reflects that SR implementation is a matter of hospital system rather than of an individual department or job position. Therefore, a successful SR implementation requires hospital-wide systematic collaborative efforts to make small but substantial changes in the workflow. In our experience, a motivated radiologist is key to successful SR implementation in a hospital. The lead radiologist must guide the implementation through persistent and collaborative efforts, e.g. incorporation of the SR form into the radiology information system, education of colleagues and care providers, regular compliance audit for radiologists, and constructive feedback to reluctant care providers.

Our subgroup analyses revealed several care-provider or hospital characteristics that were associated with the overall preference for SR over FTR (Q7). These results conversely indicate challenges in the SR implementation. For successful SR implementation, reluctant care providers and hospitals must be the focus of further education and encouragement. First, the preference for SR over FTR was more pronounced for a participant with greater experience with the use of SR for appendiceal CT (more than 100 cases), and for a hospital that enrolled more patients (170 or more) in LOCAT, in which the same SR form was mandatorily used as a trial procedure. These results suggest that there are learning curves to overcome until the care providers become proficient in, and thereby comfortable, with SR. Second, the preference for SR was more pronounced among attending care providers than among trainees, and among surgeons than among radiologists, although this difference did not reach statistical significance. These results are interesting because the attendings and surgeons, rather than the trainees and radiologists, were typically responsible for the final clinical decision regarding patient management, particularly whether to operate or not, and therefore, they would see the benefit of clearer communication of the CT results. Third, the preference for SR was associated with a hospital with smaller appendectomy volume (less than 400 annually). The higher-volume hospitals had less need to shift to SR, probably because they were able to effectively communicate through conventional FTR.

In our follow-up survey three months after the completion of LOCAT, only five hospitals (25%) kept using SR for nearly all the relevant patients, while the remaining 15 hospitals partially or rarely used SR in usual care. These disappointing results clearly highlighted the difficulty in the sustained use of SR, despite our efforts in embedding SR into the rigid framework of a clinical trial. Further study is needed to develop a measure for the sustained use of SR.

Our study had limitations. First, despite the large scale of the study, it is unclear if our survey results can be generalized. All our 20 hospitals were teaching hospitals participating in LOCAT, and therefore, they may have been more motivated or better resourced toward the report standardization. Second, our survey relied on the voluntary participation of care providers involved in LOCAT. The survey participants may have answered more positively than non-participants (35). Third, we did not measure the diagnostic or clinical advantages of SR over FTR. As we stated earlier, such a comparative investigation would require a huge sample. Despite the considerable inter-site variations in the preference for SR over FTR, LOCAT data showed little difference in clinical outcome, such as negative appendectomy rates and appendiceal perforation rates, across the 20 trial sites (23). We are not aware of any prior study directly comparing SR and FTR for appendiceal CT with respect to care providers' preference, diagnostic performance, or clinical outcomes. In terms of appendiceal ultrasonography, recent studies (21363738) of a before-and-after design advocated that the use of SR instead of FTR can lead to lowering second-line CT utilization and marginal diagnostic improvement.

In conclusion, we disseminated the Likert-based SR form for the likelihood of appendicitis across 20 hospitals through the course of a large clinical trial. Our survey including 594 care providers from the 20 hospitals showed their overall preference for SR over FTR. The subgroup analysis results suggest that successful SR implementation requires hospital-wide systematic efforts and involves some learning curves. However, further study is needed to investigate the sustained use of SR.