The Physicians' Competence in Substance Abuse Test (P-CSAT): a multidimensional educational measurement tool for substance abuse training programs.

BACKGROUND: Efforts to enhance physician substance abuse (SA) management skills lack tools to assess skills and training effectiveness. We sought to develop an easily administered survey to assess SA fact-based skills and clinical decision-making.
METHODS: We prepared 60 fact-based items dealing with SA knowledge, attitudes, and behaviors (KAB) and 53 script concordance test (SCT) items assessing SA decision-making. We used expert review and standard psychometric criteria to eliminate discordant or non-contributory survey items. We tested 92 draft items in 117 physicians, including 13 with additional SA training (trained), and 17 recognized SA experts. We assessed final survey internal consistency with Cronbach's alpha and differences in scores between experts, trained physicians, and physicians without SA training (novices) with the Kruskal-Wallis test.
RESULTS: Following refinement, the draft survey was reduced to 30 KAB and 33 SCT items. Alpha was 0.901 for the final 63-item survey and 0.887 and 0.797 for the KAB and SCT subscales, respectively. Novices, trained physicians, and experts scored means of 196, 213, and 261 respectively out of 315 possible points on the final survey. The KAB and SCT subscale results showed similar patterns. Score differences for the overall survey and its subscales were highly significant (p<0.001).
CONCLUSIONS: This survey, which we have named the Physicians' Competence in Substance Abuse Test (P-CSAT) and placed in the public domain, meets baseline criteria for reliability and validity. Future studies should determine the extent to which the P-CSAT provides consistent results in other practitioner populations and responds to SA educational efforts.