Geovanne Pedro Mauro1,2, Gabriel Faria Najas1, Heloisa de Andrade Carvalho1,3, Rosangela Correa Villar1,4. 1. Department of Radiology and Oncology - Faculdade de Medicina FMUSP, Universidade de Sao Paulo, São Paulo, Brazil. 2. School of Medicine, Universidade Nove de Julho (UNINOVE), São Paulo, Brazil. 3. Hospital Sirio - Libanês, São Paulo, Brazil. 4. Radiotherapy Department of Boldrini Childrens Center, Sao Paulo, Brazil.
Abstract
OBJECTIVES: To develop a tool that could assess residents' knowledge beyond simple information gathering and evaluate its reliability. METHODS: An assessment tool of 40 objective questions and at least one essay-based question was developed to assess residents' comprehension of general radiation oncology accordingly to validated training curricula beyond level 2 in the Bloom scale. With randomized content, questions were developed such as to be classified as at least 2 in the Bloom scale, so that reasoning and not only information gathering could be assessed. Criteria validation was made using the Classical Test Theory to describe difficulty and discrimination of each item. Reliability was tested by internal consistency using the Cronbach alpha test. RESULTS: Between 2016 and 2019, 24 residents were assessed. Six different versions of the test were made with a total of 240 objective questions and 18 essay-based questions. Five of the six versions were deemed valid and reliable. Comparisons between 1st (PGY-1) and 3rd (PGY-3) year residents were made. Consistently, PGY-3 residents had scores 150% higher than PGY-1 residents. Only two different PGY-3 reached the most complex level of answers in the essay-based questions. The results demonstrated that the major part of the acquired knowledge and retention occurs in the first six months of training rather than in all the following period. CONCLUSION: The instrument can be considered valid. This developed instrument also raised the hypothesis that residents may not assess and analyze their acquired knowledge beyond the application level.
OBJECTIVES: To develop a tool that could assess residents' knowledge beyond simple information gathering and evaluate its reliability. METHODS: An assessment tool of 40 objective questions and at least one essay-based question was developed to assess residents' comprehension of general radiation oncology accordingly to validated training curricula beyond level 2 in the Bloom scale. With randomized content, questions were developed such as to be classified as at least 2 in the Bloom scale, so that reasoning and not only information gathering could be assessed. Criteria validation was made using the Classical Test Theory to describe difficulty and discrimination of each item. Reliability was tested by internal consistency using the Cronbach alpha test. RESULTS: Between 2016 and 2019, 24 residents were assessed. Six different versions of the test were made with a total of 240 objective questions and 18 essay-based questions. Five of the six versions were deemed valid and reliable. Comparisons between 1st (PGY-1) and 3rd (PGY-3) year residents were made. Consistently, PGY-3 residents had scores 150% higher than PGY-1 residents. Only two different PGY-3 reached the most complex level of answers in the essay-based questions. The results demonstrated that the major part of the acquired knowledge and retention occurs in the first six months of training rather than in all the following period. CONCLUSION: The instrument can be considered valid. This developed instrument also raised the hypothesis that residents may not assess and analyze their acquired knowledge beyond the application level.
Authors: Thomas Philip Shakespeare; Michael Frederick Back; Jiade Jay Lu; Christopher John Wynne; Leah Bloomfield Journal: Int J Radiat Oncol Biol Phys Date: 2004-07-15 Impact factor: 7.038
Authors: P van Hagen; M C C M Hulshof; J J B van Lanschot; E W Steyerberg; M I van Berge Henegouwen; B P L Wijnhoven; D J Richel; G A P Nieuwenhuijzen; G A P Hospers; J J Bonenkamp; M A Cuesta; R J B Blaisse; O R C Busch; F J W ten Kate; G-J Creemers; C J A Punt; J T M Plukker; H M W Verheul; E J Spillenaar Bilgen; H van Dekken; M J C van der Sangen; T Rozema; K Biermann; J C Beukema; A H M Piet; C M van Rij; J G Reinders; H W Tilanus; A van der Gaast Journal: N Engl J Med Date: 2012-05-31 Impact factor: 91.245
Authors: Jane McHarg; Paul Bradley; Suzanne Chamberlain; Chris Ricketts; Judy Searle; John C McLachlan Journal: Med Educ Date: 2005-02 Impact factor: 6.251
Authors: Robert Englander; Jason R Frank; Carol Carraccio; Jonathan Sherbino; Shelley Ross; Linda Snell Journal: Med Teach Date: 2017-06 Impact factor: 3.650
Authors: Jay Burmeister; Zhe Chen; Indrin J Chetty; Sonja Dieterich; Anthony Doemer; Michael M Dominello; Rebecca M Howell; Patrick McDermott; Adrian Nalichowski; Joann Prisciandaro; Tim Ritter; Chadd Smith; Eric Schreiber; Timothy Shafman; Steven Sutlief; Ying Xiao Journal: Int J Radiat Oncol Biol Phys Date: 2016-03-19 Impact factor: 7.038
Authors: Kim Benstead; Pedro C Lara; Demetrius Andreopoulos; Jean-Emmanuel Bibault; Anthony Dix; Yannick G Eller; Pierfrancesco Franco; Meredith Guiliani; Jana Jaal; Antonio Juretic; Elvisa Kozma; Graeme Lumsden; Marta Maddalo; Stefano Magrini; Ingvil Mjaaland; Raphael Pfeffer; Olga M T de Sousa de Sa Pinto; Mateusz Spalek; Marie-Catherine Vozenin; Christine Verfaillie; Viviane Van Egten; Jesper G Eriksen Journal: Radiother Oncol Date: 2019-09-05 Impact factor: 6.280
Authors: Sarah C Darby; David J Cutter; Marjan Boerma; Louis S Constine; Luis F Fajardo; Kazunori Kodama; Kiyohiko Mabuchi; Lawrence B Marks; Fred A Mettler; Lori J Pierce; Klaus R Trott; Edward T H Yeh; Roy E Shore Journal: Int J Radiat Oncol Biol Phys Date: 2010-03-01 Impact factor: 7.038