Information-seeking behavior during residency is associated with quality of theoretical learning, academic career achievements, and evidence-based medical practice: a strobe-compliant article.

UNLABELLED: Data regarding knowledge acquisition during residency training are sparse. Predictors of theoretical learning quality, academic career achievements and evidence-based medical practice during residency are unknown. We performed a cross-sectional study on residents and attending physicians across several residency programs in 2 French faculties of medicine. We comprehensively evaluated the information-seeking behavior (I-SB) during residency using a standardized questionnaire and looked for independent predictors of theoretical learning quality, academic career achievements, and evidence-based medical practice among I-SB components using multivariate logistic regression analysis. Between February 2013 and May 2013, 338 fellows and attending physicians were included in the study. Textbooks and international medical journals were reported to be used on a regular basis by 24% and 57% of the respondents, respectively. Among the respondents, 47% refer systematically (4.4%) or frequently (42.6%) to published guidelines from scientific societies upon their publication. The median self-reported theoretical learning quality score was 5/10 (interquartile range, 3-6; range, 1-10). A high theoretical learning quality score (upper quartile) was independently and strongly associated with the following I-SB components: systematic reading of clinical guidelines upon their publication (odds ratio [OR], 5.55; 95% confidence interval [CI], 1.77-17.44); having access to a library that offers the leading textbooks of the specialty in the medical department (OR, 2.45, 95% CI, 1.33-4.52); knowledge of the specialty leading textbooks (OR, 2.12; 95% CI, 1.09-4.10); and PubMed search skill score ≥5/10 (OR, 1.94; 95% CI, 1.01-3.73). Research Master (M2) and/or PhD thesis enrolment were independently and strongly associated with the following predictors: PubMed search skill score ≥5/10 (OR, 4.10; 95% CI, 1.46-11.53); knowledge of the leading medical journals of the specialty (OR, 3.33; 95% CI, 1.32-8.38); attending national and international academic conferences and meetings (OR, 2.43; 95% CI, 1.09-5.43); and using academic theoretical learning supports several times a week (OR, 2.23; 95% CI, 1.11- 4.49). This study showed weaknesses in the theoretical learning framework during residency. I-SB was independently associated with quality of academic theoretical learning, academic career achievements, and the use of evidence-based medicine in everyday clinical practice. STUDY REGISTRATION: CNIL No.1797639.

INTRODUCTION

Resident physicians learn their specialty while providing clinical care—the optimal allocation of time to education, training, patient safety, and the provision of care has yet to be established.[1,2] Competence in medicine has been defined as the habitual and judicious use of knowledge, communication, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individuals and communities being served.[3] A retrospective cohort study pointed out that poor performance on behavioral and cognitive measures during residency are associated with greater risk for actions of the state licensing board against practicing physicians at every point on a performance continuum.[4]

In France, outside academic training seminars, the theoretical academic learning of medical students in the third cycle of medical studies (fellows, resident physicians), are based on various teaching sources including: reference textbooks, scientific journals, and recommendations from scientific societies and organizations. Resident physicians will use their expertise to find the information necessary to meet their research and theoretical knowledge in these various teaching academic materials, which fits within the overall framework of information-seeking behavior (I-SB). The concept of I-SB is relatively new in the medical literature, the first article that reported this concept dating back to 1983.[5] I-SB can be defined as the conscious effort to acquire information in response to a need or a gap in knowledge.[6] A widely accepted definition of I-SB from an academic point of view it that I-SB begins when a person realizes that there is a need for information and ends when the need is felt to be satisfied.[7]

In a cross-sectional survey reported in 1991 on second-year residents listed in the American Medical Association, residents rated other residents as contributing most to their learning (score of 2.3), with special patients ranked second (2.1), using a scale of 0 to 3.[8] In this survey, satisfaction with the residency experience was associated with the presence of factors that enhanced resident physician learning.[8] According to the American Board of Internal Medicine, a “problem resident” is defined as a trainee who demonstrates a significant enough problem that requires intervention by someone of authority.[9] A survey based on US internal medicine residency program directors reported that the mean point prevalence of “problem residents” was 6.9% and that 94% of internal medicine residency programs had problem residents.[9] The most frequently reported difficulties of problem residents were insufficient medical knowledge (48%).[9]

Adequate patient management is based on the integrated approach of 2 key elements: information collected from the patient and the physician's working medical knowledge.[10] In the era of evidence-based medicine (EBM) and the systematization of the medical approach by scientific organization guidelines, the question of theoretical learning of fellows takes on a new dimension. However, at present, we do not have clear evidence that could report objectively on resident physicians’ behavior regarding their access and use of educational resources as part of their theoretical learning. Furthermore, limited evidence is available on the effect of residency training on later practice.[11] We therefore performed a survey to comprehensively evaluate the I-SB components during residency training and to look for independent predictors of theoretical learning quality as perceived by fellows, evidence-based medical practice, and academic career achievements among I-SB components.

METHODS

Design Overview and Questionnaire Development

We performed a Web-based cross-sectional survey using a standardized questionnaire. After an exhaustive review of the literature, the questionnaire entitled: “The Nancy-Nice information-seeking behavior study” was developed in the Faculty of Medicine of Nancy (University of Lorraine) to assess respondent demographics, training program characteristics, and learning preferences. Content validity was established by two 2 in medical education (MB and JPF). The final version of the questionnaire encompassed 4 domains, including 13 items and 26 questions (multiple-choice and open-ended questions). First domain: respondent demographics and training-program, and academic path (13 items). Second domain: assessment of the knowledge of the resident physician of the academic environment and the use of the various theoretical learning materials (17 questions). Third domain: assessment of the ease of access to the theoretical learning materials (2 questions). Fourth domain: scientific production (7 questions) (see Supplemental Digital Content). The questionnaire was developed into an online survey using the Google docs platform (https://docs.google.com/). The surveys took <10 minutes to complete. All responses were kept confidential. The study was approved by the Institutional Review Board of the University Hospital of Nancy and registered at French National Commission for Data Protection and Liberties (CNIL N°1797639).

Setting and Participants

In February 2013, a cover letter including a web link, which led to the anonymous, online survey, was made available to the Informatics and Communication departments of the medical faculties of Nancy (University of Lorraine) and Nice (University of Nice Sophia-Antipolis). The online survey was conducted through a systematic mailing to all resident and attending physicians. A second mailing was conducted in April 2013 using the same procedure.

Statistical Analysis

All quantitative variables are described as medians and percentiles (interquartile range [IQR], 25–75th percentile). All proportions are expressed as percentages with 95% confidence intervals (95% CI). Univariate analysis was performed using the Fischer exact test. When considering the continuous variables for dichotomous analysis, optimal cutoff values were determined using receiver-operating characteristic (ROC) analysis, as described by DeLong et al.[12] Sensitivity, specificity, positive and negative likelihood ratios, optimal threshold, and area under the ROC curve (AUROC) with the associated P value were calculated. A bias-corrected and accelerated (BCa) bootstrap interval after 10,000 iterations for the Youden index and its associated criterion value was performed.[13] To look for independent predictors of theoretical learning quality, academic career achievements and evidence-based medical practice among I-SB components, all significant items obtained in univariate analyses, were integrated into stepwise multivariate logistic regression analysis to identify independent predictors and to estimate their relative predictive weights (coefficients). All variables with P < 0.1 were initially included in the model and variables with P < 0.05 were retained in the model. Results were shown as odds ratios (ORs) and 95% CIs. We assessed the model discrimination using ROC analysis and assessed model calibration using the Hosmer and Lemeshow goodness-of-fit test. All statistical analyses were conducted with MedCalc for Windows, version 14.8 (MedCalc Software, Ostend, Belgium), on the basis of a 2-sided type I error with an alpha level of 0.05.

RESULTS

Characteristics of the Respondent Population

From February 2013 to May 2013, 338 fellows and attending physicians completed the survey among the 582 who received the mailing (University of Lorraine, n = 433; University of Nice, n = 149), which corresponds to a response rate of 58%. Table 1 shows respondent demographics and characteristics. Fellows represented 86% of the respondents. The median age of participants was 27 years. The main specialty observed among survey participants was “general medicine” 36.4% (123/338), followed by anesthesia (8.3%, 28/338), surgery (8.0%, 27/338), psychiatry (6.5%, 22/338), and pediatrics (5.6%, 19/338) (see Supplemental Digital Content: Table 1, http://links.lww.com/MD/A204 for the exhaustive list of residency programs represented in the study).

TABLE 1

Characteristics of Fellows and Attendings Who Participated to the Nancy-Nice Information-seeking Behavior Study

I-SB During Residency Training

Textbooks and international medical journals were reported to be used on a regular basis by 24% and 57% of the participants, respectively. Among the respondents, 47% refer systematically (4.4%) or frequently (42.6%) to published guidelines from scientific societies and organizations. There was no clear preference for the format of theoretical supports, with 52% of the respondents preferring electronic versions of textbooks (e-books) and journals instead of papers versions (see Supplemental Digital Content: Table 2, http://links.lww.com/MD/A204). Within the main specialty department in the university hospital, 46% of the respondents have access to a library offering the leading textbooks of their specialty and 56% are aware about the method for accessing online medical journals through the computers of their university hospital. The PubMed alert system is known by 39% of participants and only 12% of the respondents use it regularly. For the resolution of challenging clinical cases in routine practice, 28% of participants use the MEDLINE database through the PubMed search engine, 63% refer to textbooks, and 74% of them refer to MEDLINE and/or textbooks (see Supplemental Digital Content: Table 3, http://links.lww.com/MD/A204).

Only 48% of the respondents stated that they know their leading specialty textbooks and report to use them at a frequency of several times a week (31%), once a week (20%), several times a month (29%), or rarely during the year (9%). Eleven percent of participants use leading specialty textbooks only for the purpose of preparing their medical doctoral thesis, university diploma, or other dissertation (see Supplemental Digital Content: Table 4, http://links.lww.com/MD/A204).

Guidelines published by scientific societies and organizations are accessed systematically upon publication by 4% of the participants and frequently by 43% of them. The main sources of information concerning the publication of new guidelines was, in their decreasing order of frequency: discussion with faculty and colleagues (41%), regular checking of scientific societies websites (20%), attending national and international conferences and meetings (15%), and medical case discussion within the department (13%) (see Supplemental Digital Content: Table 5, http://links.lww.com/MD/A204).

Scientific Production and Academic Achievements During Residency

During their residency training, 47% of fellows and attendings have participated in a scientific project outside of their medical thesis and 65% had participated in the writing of at least 1 manuscript from the various projects in which they have been involved. Among respondents, 31% had at least 1 manuscript published in a peer-reviewed, Medline-indexed journal. The median number of scientific project per participant was 2 (IQR, 1–3) (see Supplemental Digital Content: Table 6, http://links.lww.com/MD/A204). At the time of survey completion, 14% of the respondents defended their medical doctoral thesis and 15% of them were enrolled in Master 2 and/or PhD thesis curricula (see Supplemental Digital Content: Table 7, http://links.lww.com/MD/A204).

Theoretical Learning Quality Score and PubMed Searching Skill

When responders were asked to report the quality of their theoretical learning, as they perceived it, using a score ranging from 0 (very bad) to 10 (very good), the median score was 5 (IQR, 3–6; range, 1–10). The same strategy was used to ask responders to quantify their EBM literature searching skills using the PubMed search engine, the median score was 5 (IQR, 3–6; range, 1–10).

Independent Predictors of a High-quality Theoretical Learning Score, Master 2 or PhD Thesis Enrolment, and EBM use in Routine Clinical Practice in Multivariate Analysis

High-quality Theoretical Learning Score

In multivariate logistic regression analysis, a high-quality theoretical learning score (≥7; IQR 75th) was independently and positively associated with the following predictors: systematic reading of guidelines published by specialty scientific societies and organizations upon their publication (OR, 5.55; 95% CI, 1.77–17.44; P = 0.003); having access to a library offering the leading textbooks of the specialty in the medical department (OR, 2.45; 95% CI, 1.33–4.52; P = 0.004); good knowledge of the leading textbooks of the specialty (OR, 2.12; 95% CI, 1.09–4.10; P = 0.03); and a PubMed search skill score ≥5/10 (OR, 1.94; 95% CI, 1.01–3.73), P = 0.04) (Table 2)[12] (The optimal ‘PubMed search skill score’ threshold was calculated using ROC analysis; data not shown).

TABLE 2

Independent Predictors of a High-quality Theoretical Learning Score (≥7; IQR 75th) During Residency as Stated by the Respondent Fellow or Attending

Academic Career Achievements

The proportion of fellows or attendings who enrolled in Master 2 curriculum, PhD thesis, or Master 2 curriculum and/or PhD thesis was 13.9% (47/338), 4.1% (14/338), and 14.5% (49/338), respectively (see Supplemental Digital Content: Table 7, http://links.lww.com/MD/A204). In multivariate logistic regression analysis, Master 2 and/or PhD thesis enrolment was independently and positively associated with the following predictors: PubMed search skill score ≥5/10 (OR, 4.10; 95% CI, 1.46–11.53; P = 0.007); good knowledge of the leading international medical journals of the specialty (OR, 3.33; 95% CI, 1.32–8.38; P = 0.01); attending national and international academic conferences and meetings (OR, 2.43; 95% CI, 1.09–5.43; P = 0.03); and using academic theoretical learning supports (textbooks, medical journals, etc) several times a week (OR, 2.23; 95% CI, 1.11–4.49; P = 0.02) (Table 3).[12]

TABLE 3

Independent Predictors of Master 2 or PhD Thesis Achievement Among Fellows and Attendings

EBM use in Routine Clinical Practice

In multivariate logistic regression analysis, practicing EBM through the use of PubMed and textbooks for the resolution of challenging clinical cases in routine clinical practice was independently and positively associated with the following predictors: attending status (vs fellow) (OR, 13.54; 95% CI, 1.79–102.43; P = 0.01); using academic theoretical learning supports (textbooks, medical journals, etc) several times a week (OR, 3.19; 95% CI, 1.59–6.39; P = 0.001); and good knowledge of the leading textbooks of the specialty (OR, 2.80; 95% CI, 1.62–4.84; P = 0.0002). Interestingly, the use of nonacademic websites (eg, Wikipedia, Google)—as opposed to websites of scientific societies and organizations— for the purpose of academic learning was independently and negatively associated with an EBM practice (OR, 0.29; 95% CI, 0.12–0.73;, P = 0.009) (Table 4).[12]

TABLE 4

Independent Predictors of the Use of Evidence-based Medicine Use in Routine Clinical Practice

DISCUSSION

To our knowledge, ours is the first study that includes fellows from various residency programs and that comprehensively evaluates I-SB components during residency training in Europe. We have demonstrated that I-SB components were independently associated with the self-assessed quality of academic theoretical learning, academic career achievements, and the use of EBM in everyday clinical practice. In our study, more than half of fellows and attendings do not have access to a library which offers the leading specialty textbooks within their medical department and half of them consulted their educational learning supports at least once a week. A study reported in 2002 investigated the impact of electronic journals on research processes, such as information seeking, and found that fellows, students, and residents preferred electronic journals, whereas faculty preferred print journals.[14] In a multicenter study conducted in 2010 in 5 North American centers, residents’ reading habits and preferred educational resources were assessed.[15] The majority (77.7%) of residents reported reading <7 hours a week and most of them (81.4%) read in the context of patient care.[15] A national online survey assessing preference in knowledge acquisition was sent via e-mail to all directors of adult gastroenterology programs in the United States.[16] Among the 176 responders (85 fellows and 91 attendings), only about one-fifth stated that a designated textbook had been required during fellowship and 91% of these respondents indicated the same specialty textbook as required reading.[16] Asked how they acquire knowledge best, 45 fellows and 67 attendings responded; 42% of attendings favored journal articles, and 40% of fellows favored conferences.[16] In this study, the main sources of theoretical learning considered by fellows and attendings as potentially effective in the specialty knowledge acquisition were: local conferences (92%), attending national conferences (79%), and writing a manuscript (75 %).[16] Although, this study reported objective evidence regarding the landscape of information acquisition among fellows, it only evaluated one specialty. Moreover, in this study, objective endpoints such as quality of learning self-assessment, academic career achievement, and EBM practice were not assessed in relation to I-SB components. To our knowledge, our study is the first to demonstrate that I-SB influences the perceived quality of theoretical learning during the residency.

In our study, a high self-assessed quality theoretical learning score during residency was independently associated with the systematic reading of clinical guidelines, the good knowledge of the leading specialty textbooks and their availability in the medical department, and a high PubMed search skill score. Self-assessment is increasingly being incorporated into competency evaluation in residency training.[17] Indeed, resident physicians commonly identify general learning objectives focusing on medical knowledge regardless of the structure of the self-assessment form.[17] It is now admitted that the development of self-assessment and self-directed learning skills is essential to lifelong learning and becoming an effective physician.[18] For instance, pediatric residents in the United States are now required to use Individualized Learning Plans to document self-assessment and self-directed learning,[18] a complex process that requires collecting and interpreting data from various sources.[19] Therefore, we can consider that the I-SB and self-assessment are interdependent processes that are integrated into a comprehensive learning framework during the residency training.

Our study showed that I-SB components were associated with scientific production and academic career achievements. In our study, <15% of fellows or attendings were enrolled in Master 2 or PhD thesis curricula. In France, the equivalent of master's degrees is the combination of 2 individual diplomas the Master 1 (M1) and Master 2 (M2), following the Bologna Process. In medical studies, the Master 2 is also called “Research Master,” which often requires 2 years after the Master 1 to conduct research and write a Master 2 dissertation. In a study reported in 2009 and on the scientific productivity of 95 graduating chief residents in surgery at the University of California over a 16-year period (1990–2005), the average number of publications by a resident was 2.[20] A study reported in 2011 by Duke University demonstrated the strong positive impact of the quality of theoretical research training during fellowship on the evolution of academic career.[21] Fellows achieving academic careers were statistically more likely to have committed ≥2 years to a protected research experience during training and a first job at an academic institution upon completion of training.[21]

We demonstrated that I-SB components were independently associated with the use of EBM in routine clinical practice. Interest in EBM has grown exponentially, and professional organizations and training programs have shifted their agenda from whether to teach EBM to how to teach it.[22] A practice survey conducted in Ireland in 2009 on hospital-based pediatricians established the I-SB of pediatricians in answering everyday clinical queries.[23] Among the 156 respondent pediatricians, 85% believed that Web-based resources have improved medical practice, with 88% reporting that Web-based resources are essential for medical practice.[23] Information technology resources play a key role in helping physicians to deliver, in a time-efficient manner, solutions to clinical queries at the point of care. A systematic review demonstrated that learning EBM in a clinical practice setting achieves improvements in substantial outcomes such as knowledge, critical appraisal skills, attitudes, and behavior.[24] In our study, it is noteworthy that the use of nonacademic websites (eg, Wikipedia) for the purpose of academic learning was negatively and independently associated with the use of EBM in everyday clinical practice. According to Sackett,[25] EBM can be defined as the integration of best research evidence with clinical expertise and patient values. In our study, the main EBM resources used by residents and attendings in clinical routine practice were the MEDLINE database through the PubMed search engine and clinical textbooks. Although various EBM resources are available (eg, TRIP database, Cochrane Library), a survey on UK doctors showed that Medline/PubMed was the most established and well-known EBM resource in routine clinical practice.[26] The UpToDate tool is a popular source for medical information.[27-29] In a crossover randomized controlled trial, the UpToDate tool, compared with PubMed Clinical Queries, led to both higher proportion of relevant answer retrieval within a shorter time and a higher users’ satisfaction.[29] However, the use of the UpToDate tool has not been assessed in our study since it was not available in both universities (Lorraine and Nice Sophia-Antipolis Universities).

Among the strong points of our study, we might consider the following: the participation of 2 medical schools; the participation of a large number of fellows and attendings, which yielded robust results on descriptive and inferential plans; the systematic and comprehensive evaluation of several I-SB components and their relationship with major fellowship tasks (learning, academic career, and daily clinical practice); the anonymous design of the survey, which yielded results closer to reality given the declarative nature of the investigation; and the lack of recall bias because the majority of participants were still undergoing fellowship and report their everyday experience. This study had several limitations. First, the study was nonexhaustive with regard to the whole population of fellows and attendings of the 2 medical schools of Nancy and Nice. Second, the study was designed as a statement-based survey. Nevertheless, most of the responders were still on their residency training and report their real-life experience, hence reducing the risk of recall bias. Third, our results should not be extrapolated to old faculty because the study was designed to assess the impact of I-SB during residency, and the median age of the responding population was 27 years.

CONCLUSION

The data from the “Nancy-Nice information-seeking behavior” survey can help us to have an objective overview of resident physicians’ academic learning landscape in France. These results can be used as a starting point for the improvement of the academic teaching and training systems during fellowship to attract the best and brightest into academic careers while maximizing opportunities to pursue research and have an academic career post-membership. In this way, we will be able to reach many of the unmet needs in medical education during the residency training.