BACKGROUND: The use of Workplace-Based Assessment (WBA) has increased in recent years. AIMS: To create a modified version of Direct Observation of Procedural Skills (DOPS) adapted to the Chilean context and establish its psychometric properties. METHODS: The content validity of DOPS was established through interviews and consensus survey. To be included in the new version, the respective domain should have been considered by over 80% of interviewers and it should have a Content Validity Index (CVI and kappa statistic over 0.78 and 0.6, respectively. For four months, the new DOPS was used to assess the intubation skills of the anesthesia trainees. Generalizability theory was used to establish reliability and internal consistency. RESULTS: The interview suggested a DOPS with twelve domains. All were included in the final version as the CVI and kappa values were above 0.9 and 0.8, respectively. 585 procedures were assessed. The G coefficient was 0.90. The intubation needs to be assessed with DOPS at least six times to obtain a G coefficient of 0.80. CONCLUSIONS: The modified DOPS was a valid, reliable and practical tool for assessing the intubation procedure. Effort needs to be made to improve the staff's feedback skills.
BACKGROUND: The use of Workplace-Based Assessment (WBA) has increased in recent years. AIMS: To create a modified version of Direct Observation of Procedural Skills (DOPS) adapted to the Chilean context and establish its psychometric properties. METHODS: The content validity of DOPS was established through interviews and consensus survey. To be included in the new version, the respective domain should have been considered by over 80% of interviewers and it should have a Content Validity Index (CVI and kappa statistic over 0.78 and 0.6, respectively. For four months, the new DOPS was used to assess the intubation skills of the anesthesia trainees. Generalizability theory was used to establish reliability and internal consistency. RESULTS: The interview suggested a DOPS with twelve domains. All were included in the final version as the CVI and kappa values were above 0.9 and 0.8, respectively. 585 procedures were assessed. The G coefficient was 0.90. The intubation needs to be assessed with DOPS at least six times to obtain a G coefficient of 0.80. CONCLUSIONS: The modified DOPS was a valid, reliable and practical tool for assessing the intubation procedure. Effort needs to be made to improve the staff's feedback skills.