The utilization of educational resources published by the Thoracic Surgery Residents Association.

Objective: The Thoracic Surgery Residents Association (TSRA) is a trainee-led cardiothoracic surgery organization in North America that has published a multitude of educational resources. However, the utilization of these resources remains unknown.
Methods: Surveys were constructed, pilot-tested, and emailed to 527 current cardiothoracic trainees (12 questions) and 780 former trainees who graduated between 2012 and 2019 (16 questions). The surveys assessed the utilization of TSRA educational resources in preparing for clinical practice as well as in-training and American Board of Thoracic Surgery (ABTS) certification examinations.
Results: A total of 143 (27%) current trainees and 180 (23%) recent graduates responded. A higher proportion of recent graduates compared with current trainees identified as male (84% vs 66%; P = .001) and graduated from 2- or 3-year traditional training programs (81% vs 41%; P < .001), compared with integrated 6-year (8% vs 49%; P < .001) or 4 + 3 (11% vs 10%; P = .82) pathways. Current trainees most commonly used TSRA resources to prepare for the in-training exam (75%) and operations (73%). Recent graduates most commonly used them to prepare for Oral and/or Written Board Exams (92%) and the in-training exam (89%). Among recent graduates who passed the ABTS Oral Board Exam on the first attempt, 82% (97/118) used TSRA resources to prepare, versus only 48% (25/52) of recent graduates who passed after multiple attempts, failed, have not taken the exam, or preferred not to answer (P < .001). Conclusions: Current cardiothoracic trainees and recent graduates have utilized TSRA educational resources extensively, including to prepare for in-training and ABTS Board examinations.

Utilization of educational resources among recent graduates for Board Exam preparation.

The TSRA should continue to develop print and digital formats of educational content, and trainees should use TSRA resources in combination with other study tools during and after training.

The TSRA is a trainee-led cardiothoracic surgery organization in North America that has published numerous educational resources, but the utilization of these resources has not been described. This cross-sectional survey study showed that current and former trainees have used TSRA resources extensively in clinical practice and test preparation over the past decade.

The Thoracic Surgery Residents Association (TSRA) is a trainee-led organization established in 1997 under the guidance of the Thoracic Surgery Directors Association (TSDA)., The mission of the TSRA is to provide peer-based resources and support for cardiothoracic surgery residents to succeed during training and beyond through endeavors in 5 primary domains: (1) education, (2) outreach, (3) diversity, equity, and inclusion, (4) wellness, and (5) community. To fulfill its educational goals over the past decade, the TSRA has published a series of review books, reference guides, and textbooks, as well as an online multiple choice question bank, podcast series, webinar series, and many additional educational resources. The creation of these resources has been previously described in detail and typically includes review by Board-certified cardiac and thoracic faculty, as well as multiple rounds of peer review by a section editor, primary editor(s), and copy editor before publication.

Despite the publication of numerous TSRA educational resources, their utilization and effectiveness remain unknown. Another function of the TSRA has been to facilitate multiple survey-based studies, which have covered topics relating to all aspects of trainee development and transition into practice as cardiothoracic surgeons.3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 Although these survey studies have provided important insights into various aspects of cardiothoracic training and developing a career in cardiothoracic surgery, a survey has not been administered to evaluate the use of TSRA educational resources. The benefit of collecting these data would be mutually informative to cardiothoracic trainees and the leadership in discovering which, if any, TSRA resources were found to be beneficial to current and past trainees, as well as if any additional resources are needed. Because of the combination of high variability in cost and a relative paucity of available educational resources within the cardiothoracic surgery specialty, determining the utility and effectiveness of these resources is important to optimize cardiothoracic resident education. Furthermore, the preferences of cardiothoracic trainees toward content format and delivery might have broader implications for surgical education and the dissemination of educational content.

In this survey-based study, we aimed to characterize learning practices and preferences among cardiothoracic trainees in preparing for in-training and Board examinations. To do so, we sought to understand the utilization patterns of TSRA educational resources published over the past decade.

Methods

This cross-sectional study was deemed exempt by the University of Michigan institutional review board on May 6, 2020 (HUM00181620). The need for informed written consent was waived.

Target Audience

Separate surveys for current cardiothoracic trainees (12 questions) and recent cardiothoracic surgery graduates (16 questions) were constructed by the authors. The survey was initially constructed on the basis of in-depth literature review for content and question construction. The current trainee survey was reviewed and modified by cardiothoracic trainees on the 13-person TSRA Executive Committee and the recent graduate survey by 3 recent graduate volunteers, whereas both surveys were also reviewed and edited by 2 experts in educational research in May 2020. Individual feedback was reconciled with the literature review and the survey was pilot-tested before dissemination. The final surveys (Figures E1 and E2) were disseminated on TSRA social media accounts and through 3 email messages between June and October 2020 to 527 current cardiothoracic trainees and 780 former trainees who graduated between 2012 and 2019.

Figure E1

Current trainee survey (12 questions).

Figure E2

Recent graduate survey (16 questions).

Electronic Surveys and Outcomes

The surveys (Figures E1 and E2) included 9 common questions regarding demographic characteristics, exposure to and perceived benefit of individual TSRA resources in preparing trainees for clinical practice as well as in-training and Board certification examinations, and format preferences for learning cardiothoracic educational content. Two questions for current trainees and 6 questions for recent graduates relate to American Board of Thoracic Surgery (ABTS) Written and Oral Board Exams, whereas the final question asks for open-ended feedback for improvement. All surveys were constructed using Qualtrics software.

Individual TSRA educational resources referenced in the survey were all published between 2011 and 2020 and include the Review of Cardiothoracic Surgery (first edition: 2011; second edition: 2016),, Clinical Scenarios in Cardiothoracic Surgery (first edition: 2013; second edition: 2020),, Primer of Cardiothoracic Surgery (2013), Operative Dictations in Cardiothoracic Surgery (2014), Multiple Choice Review of Cardiothoracic Surgery (2015), Intern Survival Guide (2016), Pocket Mentor (2017), Podcast Series (ongoing), Online Question Bank (ongoing), and Decision Algorithms in Cardiothoracic Surgery (2019).,

Statistical Analysis

Responses to common questions for the 2 surveys were compared. Bivariate comparisons for categorical variables used either χ2 testing or Fisher exact test (if frequency ≤5). Missing data were designated with notation of a different denominator for each analysis, as necessary. Data were analyzed as provided and no imputation was performed. All analyses were performed using Stata 16.0 (StataCorp LLC).

Results

Demographic Characteristics

A total of 143 (27%) current trainees and 180 (23%) recent graduates provided responses. Among respondents, a higher proportion of current trainees identified as female (34% [48/143] vs 16% [29/180]; P = .001). A higher proportion of current trainees compared with recent graduates were a part of an integrated 6-year program (49% [70/143] vs 8% [14/180]; P < .001) as opposed to traditional 2- or 3-year programs (41% [59/143] vs 81% [145/180]; P < .001). The proportion in 4 + 3 pathways did not differ among groups (10% [14/143] vs 11% [19/180]; P = .82; Table 1). The most common level of current trainee was postgraduate year 8 or greater (n = 28; 20%) and the most common year of graduation among recent graduates was 2018 (n = 32; 18%; Figure 1).

Table 1

Demographic characteristics of current trainee (n = 143) and recent graduate (n = 180) respondents

	Current trainees (n = 143), n (%)	Recent graduates (n = 180), n (%)	P value
Gender
Male	95 (66)	149 (83)	.001
Female	48 (34)	29 (16)
Non-binary	0	1 (1)
Training program type
Integrated 6-year	70 (49)	14 (8)	<.001
Combined 4 + 3	14 (10)	19 (11)	.82
Traditional 2-year or 3-year	59 (41)	145 (81)	<.001
Training track or practice structure
Adult cardiac	69 (48)	81 (45)	.56
General thoracic	42 (29)	65 (36)	.20
Congenital cardiothoracic	22 (15)	24 (13)	.60
Mixed practice	5 (4)	7 (4)	1.00
Undecided/other	5 (4)	3 (2)	.47
Type of practice (planned or current)
University	80 (56)	98 (54)	.79
University-affiliated	23 (16)	26 (14)	.68
Community	13 (9)	48 (27)	<.001
Military	1 (1)	3 (2)	.63
Undecided (current trainees only)	26 (18)	–
Super fellowship (recent graduates only)	–	2 (1)

Recent graduates graduated from their primary cardiothoracic surgery training between 2012 and 2019.

Figure 1

Distribution of respondent postgraduate year (PGY) among cardiothoracic surgery current trainees (left) and year of graduation among recent graduates (right) among those responding to the Thoracic Surgery Residents Association educational resources survey.

The most common training or career track for current trainees (n = 69; 48%) and recent graduates (n = 81; 45%) was adult cardiac surgery. The distribution of cardiothoracic tracks did not statistically differ among groups (Table 1). A large percentage (56%; n = 80) of current trainee respondents indicated they were most likely to join a university practice, whereas 18% (n = 26) were undecided and another 16% (n = 23) planned to join a university-affiliated practice. Similarly, 54% (n = 98) of recent graduate respondents are in a university practice and 14% (n = 26) in a university-affiliated practice. However, 27% (n = 48) of recent graduates are in a community practice, whereas only 9% (n = 13) of current trainees planned to join a community practice (P < .001; Table 1).

Preferred Educational Content Format

Most current trainee (78%; 95/122) and recent graduate (81%; 136/168) respondents indicated the print format to be either “extremely useful” or “very useful” (P = .52). Digital content on a computer was indicated to be extremely or very useful by a higher proportion of current trainees (81%; 93/116) compared with recent graduates (70%; 117/168; P = .047), whereas a lower proportion indicated digital content on a tablet to be extremely or very useful (44% [51/117] vs 59% [98/167]; P = .012). The proportion of respondents who indicated digital content on their phone to be extremely or very useful did not differ among current trainee (66%; 77/117) and recent graduate (61%; 102/167) respondents (P = .42).

The least commonly used format was digital content on a tablet for both groups (current trainees: 33% [39/117] vs recent graduate 20% [34/167]; P = .014). In contrast, the most commonly used format was print, with only 1% (1/122) of current trainee and 4% (7/168) of recent graduate respondents who indicated that they do not use print format at all (Figure 2).

Figure 2

Six-point Likert scale responses to survey questions about cardiothoracic educational content format. Respondents chose one of “Extremely Useful,” “Very Useful,” “Moderately Useful,” “Slightly Useful,” “Not At All Useful,” or “Do Not Use” for each of print, digital—computer, digital—phone, and digital—tablet educational content formats. Stacked bar sizes correspond to the proportion of respondents selecting each choice. Only 1 choice could be selected for each type of content. “Extremely Useful” and “Very Useful” were combined into 1 bar, as were “Moderately Useful” and “Slightly Useful.”

TSRA Educational Resource Utility

The most frequent way current trainee (50%; 56/111) and recent graduate (52%; 87/168) respondents first heard about TSRA resources was through a colleague, whereas the second most common method was through a mentor for current trainees (19%; 21/111) and via email among recent graduates (21%; 36/168). Current trainees most commonly used TSRA resources to prepare for the in-training exam (74%; 105/142), operative preparation (73%; 104/142), and nonoperative clinical care (68%; 96/142). Recent graduates most commonly used them to prepare for the ABTS Oral and/or Written Board Exams (93%; 165/178) and the in-training exam (85%; 151/178). Only 9% (13/142) of current trainees and 1% (2/178) of recent graduates reported having never used a TSRA resource (Figure 3).

Figure 3

Reported scenarios for which the Thoracic Surgery Residents Association (TSRA) educational resources were used by current trainees (blue) and recent graduates (red). All comparisons between groups had differences of P < .05. Respondents were able to choose more than 1 scenario for which they used TSRA resources, unless they selected “Never used a TSRA text.” ABTS, American Board of Thoracic Surgery.

Among respondents, 73% (105/143) of current trainees and 89% (161/180) of recent graduates provided answers on utilization and benefit of specific TSRA educational resources. The most commonly used TSRA resource was Clinical Scenarios in Cardiothoracic Surgery, used by 86% (90/105) of current trainees and 89% (143/161) of recent graduates (P = .45). The second most commonly used resource was the Review of Cardiothoracic Surgery, used by 84% (88/105) of current trainees and 85% (137/161) of recent graduates (P = .78). Among additional TSRA resources, a higher proportion of current trainees used the Multiple Choice Review of Cardiothoracic Surgery, Intern Survival Guide, Pocket Mentor, Podcast Series, Online Question Bank, and Decision Algorithms in Cardiothoracic Surgery, whereas a higher proportion of recent graduates used Operative Dictations in Cardiothoracic Surgery (Figure 4). When only considering the most recent graduates from 2018 and 2019, preferences were similar among groups, except for a higher proportion of current trainees used the Online Question Bank, whereas a higher proportion of 2018 to 2019 graduates used Operative Dictations in Cardiothoracic Surgery (Figure E3).

Figure 4

Reported utilization of individual Thoracic Surgery Residents Association educational resources among current trainee (blue) and recent graduate (red) respondents. Respondents were able to choose multiple Thoracic Surgery Residents Association resources to indicate those they had ever used. Comparisons with P < .05 are indicated with an asterisk. CT, Cardiothoracic.

Figure E3

Reported utilization of individual Thoracic Surgery Residents Association (TSRA) educational resources among current trainee (blue) and recent graduate (red) respondents including only recent graduates from 2018 and 2019. Respondents were able to choose multiple TSRA resources to indicate those they had ever used. CT, Cardiothoracic. *P < .05.

The Review of Cardiothoracic Surgery was ranked as the most beneficial TSRA resource by 40% (42/105) of current trainee respondents, followed by Clinical Scenarios in Cardiothoracic Surgery by 39% (41/105), and the Podcast Series by 7% (7/105). Clinical Scenarios in Cardiothoracic Surgery was ranked as the most beneficial TSRA resource by 50% (80/161) of recent graduates, followed by the Review of Cardiothoracic Surgery by 37% (60/161), and Operative Dictations in Cardiothoracic Surgery by 4% (6/161).

Finally, feedback on how to improve TSRA resources was provided by 76 current trainees, among whom 33% (25/76) suggested more content, 20% (15/76) a different format, 11% (8/76) different content, and 7% (5/76) a different level of complexity, whereas another 38% (29/76) indicated no improvements were needed. Among 119 recent graduates who provided at least 1 answer on how to improve TSRA resources, 27% (32/119) suggested adding more content, 12% (14/119) a different format, 7% (8/119) different content, and 6% (7/119) a different level of complexity, whereas another 50% (59/119) answered that no improvements were needed. Specific free-response recommendations for improvements are included in Table E1.

Table E1

Open-ended answers by current trainees and recent graduates to the final survey question: “How would you improve TSRA resources?”

Chosen answer	Free text responses
	Current trainees	Recent graduates
More content—please explain:	“Would be great to expand content to cover more newer and upcoming technologies and techniques.”“More anatomical pictures to describe operations rather than just writing.”“Curriculum format”“Expand on podcast series”“More question banks”“Access to textbooks and chapters.”“The TSRA clinical scenarios is a great book but I think it needs more content. It is quite basic and not enough for oral board prep.”“I have found the podcasts to be extremely useful. Please make more.”“Journal club, main articles and papers that shape our evidence based practices”“Online question bank”“Question banks”“Access to more e-books”“Would be great to expand content to cover more newer and upcoming technologies and techniques.”“Audiobook TSRA Review Book”“More questions online”“Pictures in review of CT surgery. Make it outline based, shorter. Aim to make it like Fishers, but for CT surgery.”“Since we intend to learn how to be a good doctor as well as a good surgeon, I would like to see content about basic technical aspects about the appropriate way of instrument handling, details to how to perform a procedure (needle angles, size of needles…), how to trouble shoot common intraoperative complications such as inadequate venous drainage, elevated pressure in cardioplegia or arterial line, and distended LV. Lastly, more knowledge about intensive care management (rhythms, hemodynamic deterioration, vent management, AKI…)”“TSRA review book could be organized better and perhaps even have links to or directly placed graphics”	“Especially about congenital heart surgery”“More scenarios and questions”“The print materials don't go into incredible depth and often gloss over major topics”“Current content is not in depth enough for the boards. It serves mostly as a review.”“Content is great, some probably needs updating by now”“Frequent updates”“More scenarios for oral board prep”“Updated content—some of this is outdated”“For written boards, more and similar question bank to SESATS”“More questions in the question bank app and more format like the clinical case reviews”“More board type question at the Q bank”“Some of the content has been papers which are somewhat outdated”“More questions weren't available to me when I was preparing for my boards”“Phone app resources”“Update all the chapters”“I found the podcast very helpful, particularly for oral boards. Covering more topics would be great and beneficial.”“A great question bank”“More board prep content”“On line Brain was the best tool. I really liked that”“Books should be translated to online content that can be updated over time (similar to UpToDate) with links to anatomy and illustrations as they are missing from current books”“Expand question bank similar to SESATS”“Update the current content”“Online content usable on phone is most readily accessible in the busy days we all have. I found apps the best for board prep by far.”“A separate book for congenital would be great”
Different content—please explain:	“Would benefit from up-to-date evidence-based reviews”“Question bank!”“The questions need to all be evaluated for being relevant. There are a lot that are minimally relevant now. Also the questions are poorly sourced for answers and sometimes the answers on the web brain contradict given answer that is marked as correct.”“Videos of mock oral board scenarios to help practice and prepare for oral boards”“More up to date articles than what is offered in the STS website”“More images in the review books”“Update prior texts w/current info and revise the question bank”“Pictures in review of CT surgery. Make it outline based, shorter. Aim to make it like Fishers, but for CT surgery.”	“Updated and designed specifically for board prep”“Would like to see more audiovisual and graphical content”“Needs to be updated to reflect AJCC 8th edition staging”“Subspecialty specific resources in depth for TEVAR/aorta & TAVR. Also consider mini mitral, mech support/transplant”“If the content can be explained from day to day requirements as well as target written and oral boards in mind; if some ideas can be o gained from people who formulate board questions or expect certain”“More practice questions and scenarios”“More qbank”“More information on Echo, cath, CT and MR”
Different format—please explain:	“More integrated between resources”“Again, more images rather than just tables and text”“Post via Twitter would be great to create more awareness”“Provide everything on one topic in one folder instead of fishing for it everywhere.”“Integrated (Review, Scenarios, Questions, Supplements)”“Include video content”“It would be great to have a section on sentinel papers in cardiothoracic surgery; also would be nice to include relevant new papers”“The podcasts are good but sometimes feel very dry.”“The podcasts are very helpful. It would be nice if we had access to an outline of what was said for quick reference. Instead of listening to the whole session each time to review.”“The varied sources can be very confusing. It would almost be better to choose one book for each discipline and have everything come out of that book with the other stuff to supplement. Plus, outlines or slides for each topic would be helpful for the faculty lecturers who don't necessarily have time to prepare.”“As much in an online, Q&A based learning format as possible”“PDF versions of clinical scenarios available”“More clinical based approach”“The cases on learnctsurgery.sts.org are outdated and the videos (cath/CT/echo) do not function in the new system”“The TSRA Lit Review holds great potential but is currently underdeveloped and poorly organized. For a specialty as data-driven as CTS, this should be a way for trainees to rapidly familiarize themselves with seminal work in the field as a pathway to engaging with more current papers. The formatting/editing of the new edition of Clinical Scenarios leaves a lot to be desired. The off-white pages, strange grey-scale in the lettering, and inconsistent font usage are all very distracting to the reader. The body text is also not centered on the pages, which is also very distracting.”	“More digital”“Video and photo would be good”“Optimize for phone”“The content is basically pulled from textbooks or journals with little thought into the integration of each topic”“More question banks—SESATS”“Could use more/better multiple choice “board exam-type” questions”“I didn't think the review of cardiothoracic surgery was super helpful in its format without any pictures.”“I like the current format where you have many topics in all sun specialized fields”“Available on tablet and smartphone”“Having electronic versions on the phone would be great.”“Fill in blank and would have direct reference”“More formats available for different types of learners”“Tablet and podcast based as a focus”“More content easily accessible with iPhone. More operative videos of fundamental operations.”
Different level of complexity—please explain:	“More range of cases, for example, there are many ways to skin a cat. There are some approaches described in the books that are not applicable or not used at my institution, so would be beneficial to have an approach to the various ways of doing things.”“More operative technique focused rather than just general overview of pathology and procedural options.”“Review of cardiothoracic surgery needs more detailed info that is covered on the boards”	“Would love a teaching pearls sort of book, to help residents with teaching juniors”“I would have the written board exam prep and oral exam prep with different levels of complexity/depth for each—more detail for written boards, less detail more conceptual for orals”“Current TSRA review material is too basic”“Current books are too superficial”“Get into more details and more scenarios”
Other—please explain:	“Updates to Clinical Scenarios. Updates to material in Review posted online in-between new editions (eg, changes in staging, standards of care, etc)”“Need more pictures and figures. Check out the two best gen surg books for board review “clinical scenarios in surgery” by Justin Dimmik and Cameron's “current surgical therapy””“A better pdf version and a video presentation”“More pictures”“More organized and better developed content and review question banks for board review. Concise separate review books thoracic, cardiac, and congenital and online question banks that are geared directly for board prep would be the most useful resources. ““There is a significant need to adjust the curriculum to fit the needs of junior learners. Additionally the old “Moodle” units are outdated and not updated and it would be great to collate the TSRA resources to be available by topic (aortic disease units/chapters/articles together for example) instead of going through the various TSRA and other extensive resources to put together materials for weekly conferences.”“Make it more easily acceptable”“Inclusion of images/diagrams in Review of Cardiothoracic Surgery would be helpful and make for a more engaging read.”	“The earliest version of the clinical scenarios book had some errors. I believe they have been corrected now”“Library of operative videos”“Update content at least every 2-3 years”“When I went through board exam prep the decision algorithm resource was not available. As I prepared, that was something I had wished existed. I have not reviewed the decision algorithm resource, but I suspect it's very helpful and would recommend making sure it's as comprehensive as possible.”“Coordinate with learnctsurgery for single high quality resource”“Took the AOA CTS Written and Oral Exams. Passed both on first attempts.”“The Osler course is not good but everyone pays a lot of money for it because it's the only thing out there.”“Better question formats and better way to create online exams”“The question bank is poorly written and needs to be drastically improved.”“Didn't know it has expanded so much since I graduated”“For oral–combo of clinical scenarios and podcasts worked well, though scenarios book needs updating”“Integration because determining how to use each is a bit overwhelming. But, this is minor. They are really well done esp the clinical scenarios. Many of these didn't exist when I graduated.”“I didn't know all these different options resources exist”“More exposure to resources available when in fellowship and early career.”“More frequent updates”“Procedural videos”

TSRA, Thoracic Surgery Residents Association; SESATS, Self-Education, Self-Assessment in Thoracic Surgery; CT, cardiothoracic; STS, Society of Thoracic Surgeons.

ABTS Written Board Exam Preparation

When asked which resources current trainees anticipated using to prepare for the ABTS Written Board Exam, 80% (97/122) who provided an answer chose Self-Education, Self-Assessment in Thoracic Surgery (SESATS) practice questions, 77% (94/122) TSRA educational resources, 66% (81/122) the Society of Thoracic Surgeons (STS) online curriculum, 61% (74/122) standard textbooks, and 58% (71/122) board course textbooks (eg, Doty). Among 171 recent graduate respondents who provided an answer about the ABTS Written Board Exam, 89% reported passing the exam on the first attempt, 5% on the second attempt, 5% had not yet attempted the exam, 1% failed the exam once, and 1% preferred not to say. The most commonly used resources used to study for the Written Board Exam were SESATS questions used by 90% of respondents, TSRA educational resources by 83%, standard textbooks (eg, Cohn, Shields, Sabiston, Spencer, etc) by 68%, and board course textbooks (eg, Doty) by 51% (Figure 5, A). When asked which single resource was the most beneficial to study for the ABTS Written Board Exam, 52% (88/168) answered SESATS, 20% (34/168) board course textbooks (eg, Doty), and 14% (23/168) TSRA educational resources.

Figure 5

Utilization of individual educational resources among recent graduates for the purpose of preparing for the ABTS (A) written Board Exam and (B) oral Board Exam. ABTS, American Board of Thoracic Surgery; SESATS, Self-Education, Self-Assessment in Thoracic Surgery; TSRA, Thoracic Surgery Residents Association; STS, Society of Thoracic Surgeons.

ABTS Oral Board Exam Preparation

When asked which resources current trainees anticipated using to prepare for the ABTS Oral Board Exam, 70% (85/121) chose practice with faculty, 66% (80/121) TSRA educational resources, 65% (79/121) practice with peers, and 64% (77/121) an in-person Oral Board Exam prep course (eg, Osler). Among 170 recent graduates who provided an answer about the ABTS Oral Board Exam, 69% reported passing on the first attempt, 6% on the second attempt, 2% on the third attempt, 18% had not yet attempted the exam, 1% had failed the exam once, 1% had failed twice, and 2% preferred not to say. Among the 153 respondents who provided further information, 80% used TSRA educational resources to study for the Oral Board Exam, 70% used an in-person oral board prep course (eg, Osler), and 63% practiced with peers (Figure 5, B). When asked to identify the single most beneficial resource for studying, the 3 most common answers were in-person Oral Board Exam prep course (eg, Osler) for 29% (44/153), TSRA educational resources for 25% (38/153), and practice with peers for 20% (30/153).

Among recent graduates who passed the ABTS Oral Board Exam on the first attempt, 82% (97/118) report using TSRA resources to study, versus only 48% (25/52) of recent graduates who passed on the second or third attempt, failed once or twice, had not taken the exam, or preferred not to answer (P < .001). Similarly, 72% (85/118) of those who passed on the first attempt reported using an in-person Oral Board Exam prep course (eg, Osler) to prepare, versus only 42% (22/52) of those who passed on the second or third attempt, failed once or twice, had not taken the exam, or preferred not to answer (P < .001). No such relationship was present when the reported use of SESATS, practice with peers, practice with faculty, standard textbooks, or the STS online curriculum were evaluated as tools to prepare (all P > .05).

Discussion

In the present survey study of educational content for current and recent cardiothoracic surgery trainees, we demonstrate that TSRA educational resources are widely utilized by current trainees to prepare for the in-training exam and the operating room, and by recent graduates to prepare for the in-training and ABTS Written and Oral Board Exams (Figure 6). Additionally, Clinical Scenarios in Cardiothoracic Surgery and the Review of Cardiothoracic Surgery were the most utilized and considered the most beneficial TSRA resources. Furthermore, cardiothoracic educational content in print and on a computer were the most useful formats for both groups of respondents. Finally, the most commonly used resource to prepare for the ABTS Written Board Exam was the SESATS question bank, whereas TSRA educational resources were most commonly used for ABTS Oral Board Exam preparation. Collectively, these findings indicate TSRA educational resources have been used by and are beneficial for current and former cardiothoracic trainees.

Figure 6

Study methods and results. TSRA, Thoracic Surgery Residents Association.

Over the past decade, the TSRA has published multiple review textbooks, reference guides, and textbooks, while also developing an online multiple choice question bank, extensive podcast series, the TSRA Newsletter, social media presence, and multiple webinar series. The standard process for each of these resources has been described in detail previously and typically includes each chapter or section to be written by a mentee (trainee) and mentor (faculty surgeon) tandem, followed by peer-review by the TSRA through a section editor, primary editor(s), and copy editor for each resource. Despite this robust presence in cardiothoracic trainee education, the utilization of these resources had not previously been assessed. Previous studies in cardiothoracic surgery have focused on implementing and evaluating simulation curricula,28, 29, 30, 31, 32 case-based didactic programs, and debate-style journal clubs as targeted methods to improve cardiothoracic surgical education. Additionally, in a previous analysis on the previous national Moodle-based online thoracic surgery curriculum the curriculum was reported to be most beneficial for individuals with midrange scores on the in-training exam before using the curriculum, with greater improvement in score with heavier use of the curriculum compared with low curriculum use (+17% vs +7%; P = .09). Another study assessed cramming by recording the number of logins to the national online curriculum before the in-training exam and showed cramming not to be effective in affecting in-training exam percentile, but potentially beneficial to a subset of initially low performers to significantly improve their score. Although these studies focused on establishing curricula and determining the importance of various curricula and study methods for the in-training exam, they do not assess trainee perceptions of educational resources and do not associate utilization of specific resources with ABTS Board preparation and test-taking. The data from this study add perspective not only from current trainees but recent graduates, many of whom have already gained ABTS certification, which might provide additional insight beyond education for in-training exams and residency training. Future analyses could evaluate the same array of educational resources and associate utilization with in-training and ABTS Examination scores.

The findings in this study indicate that most trainees use multiple resources to study for Boards. An overwhelming 93% of recent graduate respondents used TSRA educational resources to study for either the Written and/or Oral Boards. Additionally, more than 4 of 5 recent graduate respondents who passed the ABTS Oral Board Exam on the first attempt used TSRA resources to study, versus only less than half of all others (P < .001). However, the highest proportion of respondents rated the SESATS question bank the most beneficial resource to prepare for the ABTS Written Board Exam, whereas an in-person prep course (eg, Osler) was most commonly ranked the most beneficial for ABTS Oral Board Exam preparation. Using TSRA resources was ranked by the third and second highest proportion of respondents as most beneficial to prepare for the Written and Oral Board Exams, respectively. These findings suggest that although trainees preparing for and taking the ABTS Boards might not find TSRA resources to be the most important component of their preparation, nearly all candidates who self-reported success on Board Exams incorporated TSRA educational resources into their studying.

The near universal utilization of TSRA resources might be explained by the value of these resources, considering the relatively inexpensive cost of each. The 2 most popular TSRA resources were Clinical Scenarios in Cardiothoracic Surgery and the Review of Cardiothoracic Surgery, which can be purchased for $60 and $25, respectively. In contrast, standard textbooks often cost hundreds of dollars, whereas the CORE Review/Doty course provided by Intermountain Healthcare costs $650-$700 plus travel, and the Osler Institute course for thoracic surgery costs over $1000. The widely popular SESATS question bank is available for $300, whereas previous editions of the question bank can be accessed at no charge. Interestingly, the STS online curriculum was used by only 23% of respondents for Written Board Exam prep and 12% for Oral Board Exam prep, despite extensive topic coverage and being available for at no charge to all trainees. However, a more substantial 66% of current trainees indicated that they anticipated using the STS online curriculum to prepare for Written Board Exam prep and 42% for Oral Board Exam prep, which might reflect the availability of the online curriculum for a higher proportion of current versus former trainees. In addition, in this study we did not assess the extent to which residents used non-TSRA resources such as the STS online curriculum throughout training, such as while studying for the in-training exam, which might also explain the relatively small proportion of recent graduates that used the STS online curriculum. Furthermore, the STS recently revamped their education system and launched the STS Cardiothoracic Surgery E-Book, with its utilization and effectiveness yet to be assessed. Future analyses could include a broader assessment of all cardiothoracic educational content and include an assessment of value between different options.

These data also have important implications for the future development of educational content. Although newer educational content has increasingly become digital such as the e-book versions of TSRA educational resources and the exclusively digital STS Cardiothoracic Surgery E-Book, most respondents in both groups in this study viewed the print format of cardiothoracic content highly favorably compared with computer, phone, and tablet formats of digital content. This insight is important for any organization creating surgical educational content. However, a competing interest against the print format is the desire and expectation for continuously updated content, which requires a digital format for ease of integration and updating. Among the 3 main forms of digital content, computer content was viewed most favorably by respondents, which reiterates the primary platform that has been used for the former TSDA and current STS national online curriculum, as well as many current TSRA resources. The least useful form of digital content for both groups was tablet-based, and this view was most prominent among current trainees. Additionally, current trainees were more likely to use newly-diversified and potentially less conventional educational resources to obtain content, such as the TSRA Podcast Series, iterative online question bank, Intern Survival Guide, Pocket Mentor, and Decision-Making Algorithms in Cardiothoracic Surgery. This pattern towards using a wider variety of types of resources is reinforced by one-third of current trainees suggesting “more content” as feedback for future TSRA resource development. In aggregate, these data indicate that areas of importance for future TSRA educational content include developing a diversified, integrated, and up-to-date digital resource, while also maintaining a print option for cardiothoracic trainees.

Limitation

This study has several limitations. First, the generalizability of these findings is limited by the survey response rates of 27% and 23% for current trainees and recent graduates, respectively. However, these rates are comparable or higher than in similar recently published surveys of cardiothoracic trainees and practicing cardiothoracic surgeons., Second, respondents who chose to respond to survey invitations via email and on social media might be more familiar and view the TSRA more favorably than those who chose not to respond, which might affect generalizability of these responses to all cardiothoracic trainees. In addition, although surveys were not sent directly to non–US-based trainees, it is possible that international trainees responded to the survey through social media links.

Considering the response rates of 27% and 23% for current and recent graduates, respectively, the data from this study are at risk of response bias from individuals familiar with the TSRA. This response bias might particularly affect our analyses associating board pass rate with specific educational resources, because the 23% of recent graduates who did respond are more likely to be familiar with the TSRA. We attempted to mitigate this bias in part by forming an exhaustive list of all current trainees and all recent graduates from 2012 to 2019, and not only ones who had participated in TSRA activities. Because of the nature of all TSRA educational materials being authored and edited by TSRA membership (eg, North American trainees), it is not possible to exclude each person who helped create these resources from taking part in these surveys, which might introduce bias in favor of TSRA resources.

In addition, the respondent-reported first-time pass rate of 89% for the Written Board Exam might not reflect the overall population of graduates who took the test. However, pass rates published by the ABTS for the period 2014 to 2018 was an aggregate 87.1%, sequentially increasing annually from 81%, to 72%, 86%, 89%, and 95%. For the Oral Board Exam, only 69% of respondents passed on the first attempt, compared with ABTS pass rates from 2015 to 2019 of 72%, 78%, 84%, 77%, and 74%, respectively (aggregate: 76.9%). However, another 18% of respondents had not yet attempted the exam, which suggested the overall pass rate might well be similar to national rates.

Conclusions

Current and former cardiothoracic trainees have used TSRA educational resources extensively in clinical practice and test preparation over the past decade. Although some trainees used Board prep courses, the most commonly used educational resource to study for the ABTS Written and Oral Board Exams were the SESATS question bank and TSRA educational resources, especially among those who self-reported successfully passing. The TSRA and other educational organizations should continue to use print and digital formats to disseminate education resources, whereas trainees should consider using TSRA educational resources in combination with other study tools during and after cardiothoracic surgical training.

Webcast

You can watch a Webcast of this AATS meeting presentation by going to: https://aats.blob.core.windows.net/media/Publications/Brescia.mp4.

Disclosure

TSRA resources are provided as a basic guideline for the study of cardiothoracic surgery and should be used in conjunction with a variety of other educational references and resources. TSRA resources should not be construed as definitive study guides for either the TSDA In-Training Exam or the ABTS Part I (Written) and Part II (Oral) Certification Exams. The TSRA makes no claims regarding the value of these educational resources in preparing for, or their contribution toward performance on, either the TSDA In-Training Exam or the ABTS Certification Exams. TSRA resources are educational tools only and any medical decisions should be made only after discussions with appropriate health care providers.

Conflict of Interest Statement

The authors reported no conflicts of interest.

The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest.