Barriers and facilitators to answering clinical questions in the Americas: a cross-sectional study of surgical trauma care providers.

BACKGROUND: We aimed to understand how surgical trauma providers in the Americas acquire answers to clinical questions and what barriers and facilitators they face in efforts to practice according to recommendations for common surgical cases. We hypothesized that increased English proficiency and country income improved providers' acquisition and application of clinical knowledge.
METHODS: A 23-question survey evaluated reported confidence in interpretation of evidence, perceived language fluency, and access to and application of recommendations on sepsis and appendicitis. Electronic surveys were distributed across the Americas to Pan American Trauma Society members.
RESULTS: 108 participants from 21 countries completed this survey. 59% had ≥21 years of provider experience. 38% reported their English reading comprehension as less than or equal to "limited working proficiency." 44% endorsed using Google Translate; 35% reported they did not need translation tools to evaluate medical literature. 59% felt uncertainty regarding clinical care at least weekly. 65% reported inability to answer their clinical questions at least once per month. 86% felt confident in their ability to interpret and apply evidence for their practice. To answer clinical questions, participants listed guidelines (76%), full-text peer-reviewed journal articles (61%), and meta-analyses (49%) as their most used resources. 25% answered all five clinical questions correctly, whereas 43% answered three or fewer correctly. 79% felt they had adequate access to resources to answer the five clinical questions. When controlling for individual demographic characteristics, decreased age (p<0.01) and increased country income level (p=0.03) positively impacted correct answers to questions. DISCUSSION: Uncertainties in clinical care are unavoidable. Language, age, and country income level impacted provider acquisition and application of knowledge relevant to select clinical scenarios. These findings highlight disparities in access and training and add urgency to the movement for improved dissemination and implementation approaches for evidence-based practice in surgery. LEVEL OF EVIDENCE: IV. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Nearly one-third of all deaths worldwide occur due to conditions potentially treated with surgical care.1 Many surgical systems across resource settings lack the necessary tools, including those required to advance human resources through continuous education, to achieve the best outcomes.2 The Lancet Commission on Global Surgery prioritizes high-quality, safe care2 as a crucial part of surgical systems, yet one USA-based analysis measuring quality of care in various healthcare settings found that only 50% to 60% of surgical patients receive quality, evidence-based care.3

Clinical practice that is based on evidence may be described to follow the sequence “ask, acquire, appraise, apply, assess.”4 Barriers arise from the very start of this process. The proven method to frame vague questions into researchable phrases using the mnemonic “population, intervention, comparison, outcome” (PICO) is not widespread in provider groups globally.5 Providers across resource settings also feel stuck at the “acquire” stage in evidence-based practice.6 To improve practical acquisition of evidence-based answers to clinical questions, the Lancet Commission on Global Surgery promoted standardized clinical protocols using the most updated evidence.2 However, cited reasons why providers do not use guidelines include lack of adequate dissemination,7 awareness, familiarity, or agreement on guidelines.8 Some practitioners feel that guidelines slow their practice or do not apply to low-resource settings.8 Institutions pay millions of dollars per year for access to research behind paywalls, and individuals pay on average $30 per paper.9 Even for publicly funded research, paywalls exist for the first year after publication, slowing the already sluggish pace of scientific progress.10 To push for faster access to scientific results, the US government drafted policies requiring immediate public access to papers funded by taxpayers’ dollars, but academic publishers effectively lobbied against the proposed policies.11 In 2016, only 19% of published articles were immediately available on publication.12 A new plan in 11 European countries, known as “Plan S,” aims to increase the number of articles immediately available on publication by requiring that all research with grants from European research councils or funding bodies publish directly to a public-access forum.12 13

Even as the scientific community pushes for public access, such as that provided through “Plan S,” many providers receive no training in the “appraise” portion of the evidence-based framework. The “appraise” stage of evidence-based practice requires financial resources, expertise, and time. In a 2010 study, only 33% of surgeons used primary literature to inform their practice; lack of education in appraising primary literature was described as a primary barrier.14 Most initiatives for education on evidence-based practice require hours of training,15 16 further preventing already time-strapped surgeons from acquiring the skills required to appraise literature.17 As one attempt to overcome these barriers, evidence-based search engines may provide predigested answers, providing an effective “short-cut” when primary literature is inaccessible or too dense to quickly appraise. The American College of Surgeons promotes the global use of these search engines,18 yet many surgeons still lack access to all papers referenced by the search engines and instead use abstracts to guide their practice.19 Aside from one study reporting that cost and language were barriers to practice of evidence-based medicine (EBM),6 very few data exist related to sociodemographic factors related to EBM or using these search engines.

Without adequate support for “acquiring” and “appraising,” the flow of evidence-based practice ends before “application” and “assessment,” never reaching the patient. This study aimed to better understand influencing factors on surgical trauma providers in the Americas as they acquire answers to clinical questions and to assess their level of confidence in appraising that information. We hypothesized that increased English proficiency and country income level would improve providers’ acquisition and application of clinical knowledge.

Patients and methods

For this cross-sectional study we partnered with the Pan American Trauma Society (PTS), an international professional society of trauma providers across the full spectrum of education and clinical care, who collaborate to improve surgical trauma systems throughout the Americas.20 We used their listserv and Facebook account to electronically invite all 619 current members to anonymously participate in this survey (online supplemental appendix A). We partnered with a surgeon (FV) with a history of leadership within the PTS to design the approach and promote participation. The survey was distributed to participants to choose between English, Spanish, and Portuguese. The 23 multiple-choice question survey asked about inperson, paper, and electronic resources most often used when answering clinical questions (online supplemental file 1). It queried participant perception of language barriers including English reading comprehension and quality of translation resources (online supplemental file 2). Finally, the survey included five knowledge questions designed to assess participants’ ability to acquire information on common and life-saving clinical scenarios found in international guidelines for sepsis and appendicitis, relevant across resource settings and updated in the past 4 years (table 1, online supplemental appendix A).21 The survey link was available through REDCap (Research Electronic Data Capture) until over 100 participants completed the survey.

Table 1

Knowledge questions included in the survey

Question	Answer choices	Resources cited
1. What is your target mean arterial blood pressure in a patient with septic shock requiring vasopressors?	□ 55 mm Hg□ 60 mm Hg□ 65 mm Hg□ 70 mm Hg	A mean arterial pressure of at least 65 mm Hg is required to maintain perfusion to vital organ systems.28
2. What would you use as your first-choice vasopressor for a patient in septic shock?	□ Dopamine□ Dobutamine□ Vasopressin□ Norepinephrine□ Epinephrine	Dopamine and vasopressin may also be indicated, but as options after norepinephrine has not improved a patient’s condition.21
3. Based on which hemoglobin concentration should you begin red blood cell transfusion?	□ <7.0 g/dL□ >7–9 g/dL□ 10–12 g/dL□ 12–14 g/dL	Transfusing at 7 g/dL hemoglobin or 9 g/dL have similar 60-day or 90-day mortality, recorded ischemic events, and use of life support.21 With no difference in these groups, introducing fewer infusions improves patient safety by decreasing the risks associated.
4. For uncomplicated appendicitis, how long should you continue antibiotics after an appendectomy?	□ 0–24 hours□ 4 days□ 7 days□ There is not enough evidence to decide.	The duration of antibiotic use between one, three, and five doses did not change postoperative infections or hospital length of stay.29 Prescribing the lowest dose that provides the same impact for patients prevents wasted resources and risks associated with antibiotic treatments.
5. Should you provide a postoperative drain in appendicitis patients?	□ Yes, it lowers the incidence of intra-abdominal abscess.□ No, it causes more complications and increases length of stay and recovery time.□ There is not enough evidence to decide.	Postoperative drain for patients after appendectomy lengthens hospital stay and increases the cost of the operation.30

The correct answers to the questions are in bold.

Each participant’s knowledge score was the sum of their correct answers to knowledge questions. STATA, version 7 (StataCorp, 2015, College Station, TX) was used for data analysis. All variables of interest were ordinal categorical variables. For simplicity in reporting and interpreting our results, we used linear regression with robust SE to test for association between variables.22 In a post-hoc analysis, ordinal logistic regression resulted in the same statistical inference. In univariate analyses, a single independent variable was included in the regression model. In the multivariate analysis, the dependent variable was knowledge score, and the independent variables were selected a priori: age, country income level, English reading comprehension, translation quality, frequency of unanswerable clinical questions, frequency of uncertainty regarding clinical practice, and access to effective resources. For each variable of interest, we report a crude (univariate) and adjusted (multivariate) regression coefficient and 95% CI. For analyses related to knowledge score, the regression coefficient can be interpreted as the average change in knowledge score (ie, 1–5) associated with a one-unit increase in the independent variable. Statistical significance was set at p<0.05.

Results

Demographics

One hundred and eight participants from 21 countries finished the survey (online supplemental file 1). Most participants were general surgeons over 45 years old (67%) in upper-middle-income countries (U-MICs) (74%)—as compared with 44% of countries in Latin America and the Caribbean listed as U-MICs (World Bank)—working at urban public hospitals (78%) with fewer than 300 beds (69%)—as a reference, an urban tertiary public hospital in Lima has 452 beds (table 2).

Table 2

Demographic information

Country income level	n (%)	Age (years)	n (%)
				HIC	21 (19)	18–30	8 (7)
U-MIC	80 (74)	31–45	28 (26)
L-MIC	7 (6)	46–60	47 (44)
Hospital zone		>60	25 (23)
Rural	85 (79)	Years practiced*
Suburban	22 (20)	Student	1 (1)
Urban	1 (1)	1–10	15 (14)
Hospital size (number of beds)		11–20	28 (26)
<50	15 (14)	21–30	37 (34)
50–149	28 (26)	>30	27 (25)
150–300	31 (29)	Service
>300	32 (30)	General surgery	62 (57)
I don’t know	2 (2)	Emergency/trauma care	26 (24)
		Intensive care unit	7 (6)
		Other	13 (12)

*Years practiced include all years providing care after graduating from medical school, including residency.

HIC, high-income country; L-MIC, low-middle-income country; U-MIC, upper-middle-income country.

Language

Of the participants, 38% reported their English reading comprehension as less than or equal to “limited working proficiency,” 51% reported “professional proficiency,” and 11% were native English speakers. Lower English reading comprehension was associated with a decreased self-reported confidence in interpreting and applying medical literature to clinical practice (p<0.01). In selecting multiple ways to translate clinical resources, 44% of the participants used Google Translate, whereas 31% reported they did not need translation services and 35% reported understanding the language of the published research. Other participants (8%) used colleagues or other translation services for help. Of the participants, 19% reported language competency as a barrier to evidence-based clinical practice.

Resources

Of the participants, 91% self-reported having uncertainty at least monthly about any aspect of clinical care they provide, with the remainder denying ever having clinical uncertainty (table 3).

Table 3

Comparison of demographics and knowledge scores for participants with and without clinical uncertainty

		Clinical uncertainty at least monthly (n=98), %	No clinical uncertainty ever (n=10), %
		91% of participants	9% of participants
Age (years)	18–45	33	30
	>45	66	70
Experience (years)	1–10	14	10
	11–20	26	30
	21–30	35	30
	>30	24	30
Knowledge score (out of 5)	1–4	93	60
	5	7	40

Most participants (66%) reported they were unable to answer a clinical question at least monthly using all available resources. When asked to select their top three resources for addressing uncertainty in clinical care, 76% chose guidelines, 61% chose peer-reviewed journal articles, 49% chose meta-analyses, and 42% chose point-of-care information summaries.

For the clinical questions tested within the survey instrument, 79% felt they had adequate resources to answer correctly. When participants selected multiple options, the top three resources used to answer the clinical questions imbedded in the survey instrument were similar for sepsis and appendicitis: guidelines (45%, 44%), peer-reviewed journal articles (37%, 35%), and clinical point-of-care summaries (26%, 28%).

Confidence and knowledge

Although most participants (86%) felt adequately confident in applying literature to their clinical practice, most (75%) answered at least one out of five clinical questions incorrectly on the survey (online supplemental appendix A). Participants reported their average weekly patient load, and all those who answered at least one survey knowledge question incorrectly may be estimated to collectively care for anywhere from 4795 to >7672 patients per week (table 4).

Table 4

Knowledge questions and percent of participants who answered each individually correctly

	Question 1	Question 2	Question 3	Question 4	Question 5
Topic	Target mean arterial blood pressure	Vasopressor use in shock	Hemoglobin	Antibiotics in appendicitis	Postoperative drain
Participants who answered correctly, n (%)	55 (51)	80 (74)	89 (82)	86 (80)	83 (77)

In univariate regression analysis, higher country income level and higher reported English reading comprehension were significantly associated with higher knowledge scores (p=0.002 and p=0.003, respectively). A higher self-reported ability to interpret and apply evidence to clinical practice was significantly associated with a decreased knowledge score (p=0.014). Other variables of interest were not statistically significant (table 5).

Table 5

Univariate analysis of variables of interest in association with knowledge score

Variable	Coefficient	95% CI	P value
Age	−0.22	−0.45 to −0.01	0.065
Country income level	0.53	0.20 to 0.86	0.002*
English reading comprehension	0.35	0.12 to 0.58	0.003*
Translation quality	−0.19	−0.39 to 0.00	0.047
Frequency of unanswerable clinical questions	−0.02	−0.23 to 0.19	0.85
Frequency of uncertainty regarding clinical practice	−0.08	−0.28 to −0.12	0.42
Access to effective resources	−0.09	−0.35 to 0.17	0.49
Number of years in practice	−0.18	−0.35 to 0.00	0.051
Zone (urban, suburban, rural)	0.09	−0.21 to 0.38	0.56
Hospital type (public, private)	−0.51	−0.91 to −0.11	0.013
Hospital size	0.16	−0.040 to 0.35	0.12
Number of patients seen weekly	0.05	−0.13 to 0.22	0.61
Reported ability to interpret and apply evidence to clinical practice	−0.32	−0.58 to −0.07	0.014*

*Meets statistical significance.

After adjusting for the variables listed in table 6, age and country income level were significantly associated with lower and higher knowledge score, respectively. As age increases between predefined categories (online supplemental appendix A), knowledge scores decreased by 0.31 of the total 5 points (p<0.006). Practicing in a high-income country as compared with a low-middle-income country was associated with knowledge scores that were higher by 0.45 (p=0.026) on average (table 6). Feelings of adequate access to resources, English reading comprehension, quality of translation services, and confidence in interpreting and applying evidence to clinical practice were not significantly associated with changes in knowledge score in multivariate analysis.

Table 6

Multivariate analysis of variables of interest in association with knowledge score

Variable	Coefficient	95% CI	P value
Age	−0.31	−0.52 to −0.10	0.006*
Country income level	0.45	0.09 to 0.81	0.026*
English reading comprehension	0.25	−0.07 to 0.57	0.13
Translation quality	−0.07	−0.33 to 0.19	0.58
Frequency of unanswerable clinical questions	0.02	−0.24 to 0.26	0.93
Frequency of uncertainty regarding clinical practice	−0.06	−0.24 to 0.12	0.49
Access to effective resources	−0.10	−0.38 to 0.17	0.43

*Meets statistical significance.

Future learning

Many participants reported interest in learning to conduct research to answer clinical questions (67%) and in learning to appraise and apply existing evidence to their clinical practice (73%).

Discussion

This study set out to understand how surgical trauma providers in the Americas acquire and appraise information to properly inform their clinical practice. Overall, these providers felt they had adequate resources to address their clinical uncertainties and were confident in their ability to interpret and apply those resources to their clinical practice. However, most reported difficulty resolving clinical questions even when using all resources available to them and answered at least one survey knowledge question incorrectly. Although there will always be room for growth in clarification of clinical issues, and the available literature may not provide direction in complex individual patient scenarios, we also captured a significant degree of uncertainty regarding more discrete issues which do have relevant, publicly available, recommendations based on a strong evidence base.

Interestingly, higher reported confidence in interpreting and applying literature to clinical practice correlated to lower knowledge score in univariate but not multivariate analysis. Previous studies parallel our findings, underlining the Dunning-Kruger effect describing the cognitive bias in which people overestimate their medical knowledge and ability.23 24

Participants perceived their translation services, when required, to be sufficient for clinical practice. This mirrors prior research on Google Translate working well in medical settings.25 However, participants with lower reported English reading comprehension also reported lower confidence in interpreting and applying literature to their clinical practice, showing that even with the use of translation services, language persists as a barrier to answering life-saving clinical questions. One in five providers felt that language prevented them from practicing medicine using evidence. Language competency has been previously reported to be an explicit barrier to evidence-based care6 and is here seen to undermine provider confidence in evidence interpretation.

In this analysis, when holding other variables constant, increased age correlated with decreased score on the included knowledge assessment. Although experience likely improves various other skills relevant to the provider, previous research also connects more years in practice with decreased adherence to current standards of practice and therapies and decreased knowledge.26 More senior providers are further from medical education and may not be exposed to practice updates outside of the academic setting. In some reports, age correlates with lower rates of technological literacy and thus potentially difficulty accessing guidelines related to our survey knowledge questions.27 The authors recognize that endorsement of and adherence to EBM is only one part of clinical care. These data do not capture other important components of clinical decision-making, including judgment, cognitive biases, collaborative decision-making, and other features of clinical care that may have different trends throughout providers’ careers. Providers may depend on their early formal clinical training years to acquire clinical knowledge, and then have limited and compromised options for new knowledge acquisition throughout their career. Further research is indicated on customized interventions to address barriers to EBM unique to various groups.

Country income level also significantly influenced knowledge scores, when holding other variables constant, yet all answers to the survey knowledge questions were available in well-established and public-access online guidelines. This confirms the complexity of the socioeconomic barriers to evidence-based practice. These data do not capture reasons for higher knowledge scores in high-income countries, and future research could focus on underlying factors. Most participants reported interest in learning more about research and evidence-based practice, indicating that providers recognize the value of, and would appreciate more focus on, these topics.

Limitations

This voluntary study may over-represent surgical trauma providers from a specific demographic engaged in an academic society who are interested in participating in electronic surveys and the topic of evidence-based practice. Also, our data may not fully represent Latin America and the Caribbean, as more participants reported working in U-MICs as compared with the World Bank distribution of Latin American and Caribbean country income level. Our survey was not previously validated, as the authors did not find existing validated tools to measure our outcomes. Participants were not asked about their level of experience in clinical scenarios for each knowledge question; we thus cannot comment on the relationship between experience with a particular clinical phenomenon and knowledge scores. Although instructed, some participants may not have used all resources available to them. Finally, and likely most importantly, although we wrote survey knowledge questions for applicability across resource settings and included input from local coauthors, our results may reflect uncaptured distinctions in acceptable practice pattern that varies by country income level. This approach mirrors and highlights the well-described inequity not only in research consumption, but in production. Most guidelines are based on data gathered from populations that do not represent global diversity, and then those data are further collated and reviewed by practitioners who similarly do not represent the global population of providers.

Conclusions

Most surveyed surgical trauma providers felt confident in their evidence-based practice yet do not answer all clinical questions correctly even in an “open book” assessment. Lower country income level and higher age of provider corresponded to fewer clinical questions answered in congruence with international guidelines. Almost one out of every five participants felt their English competency was a barrier to evidence-based practice. These data highlight the broken, and even sometimes absent, dissemination and implementation schemes for new, effective interventions—many of these interventions only available in English.

Our participants were motivated to expand their knowledge of EBM and research, indicating a disconnect between opportunity and desire for practicing using the latest recommendations. Equity in healthcare necessitates overcoming linguistic barriers so all providers can use evidence-based research written in any language to drive their clinical practice. Furthermore, the logistical, resource, and behavioral barriers that contribute to an age and resource gap in clinical knowledge scores merit further research and ultimately customized intervention.

Every single patient depends on their provider to adequately acquire and appraise new data and practice EBM. In surgical trauma settings, lives are often at stake. These data underscore the urgency to holistically improve global trauma systems by investing in continuous education and building accurate, rapid language translation to improve dissemination and implementation of evidence-based practice.