OBJECTIVE:Standardized patients (SPs) have been developed to measure practitioner performance in actual practice settings, but results have not been fully validated for psychiatric disorders. This study describes the process of creating reliable and valid SPs for unannounced assessment of general-practitioners' management of depression disorders in Iran. METHOD: Ten psychology and nursing students (potential SPs) took part in a five-session course involving training in dialogue and body language. Five scenarios, along with corresponding checklists representing common presentations of mood disorders in primary-care settings, were developed by an expert group. The SPs' role-play performance of their respective scenario was videotaped and scored independently by three psychiatrists according to an observational rating scale to assess validity. The role-play was repeated after 1 week with the same scenario and the same doctor, to assess test-retest reliability. The reliability of each checklist to be used by the SPs was assessed by testing interrater reliability between groups of SPs. RESULTS: The cutoff score for the SPs' portrayal validity was 90% or above for all SPs. Mean interrater reliability for the checklists was acceptable for the SPs watching the same videos and filling in the checklists, while the mean kappa for assessing concurrent validity in filling in the checklists was lower. The test-retest performance for assessing reliability resulted in a mean kappa of 0.72. All SPs except one, who was not recruited, performed acceptably well. CONCLUSION: The authors have demonstrated a thorough validation of the technique of using standardized patients in the portrayal of depressive disorders in primary-care settings in Iran, which creates confidence in employing this technique to evaluate doctors' performance, for example, after an educational intervention. Similar methods of validation can be used for SPs' portrayal of other psychiatric disorders.
RCT Entities:
OBJECTIVE: Standardized patients (SPs) have been developed to measure practitioner performance in actual practice settings, but results have not been fully validated for psychiatric disorders. This study describes the process of creating reliable and valid SPs for unannounced assessment of general-practitioners' management of depression disorders in Iran. METHOD: Ten psychology and nursing students (potential SPs) took part in a five-session course involving training in dialogue and body language. Five scenarios, along with corresponding checklists representing common presentations of mood disorders in primary-care settings, were developed by an expert group. The SPs' role-play performance of their respective scenario was videotaped and scored independently by three psychiatrists according to an observational rating scale to assess validity. The role-play was repeated after 1 week with the same scenario and the same doctor, to assess test-retest reliability. The reliability of each checklist to be used by the SPs was assessed by testing interrater reliability between groups of SPs. RESULTS: The cutoff score for the SPs' portrayal validity was 90% or above for all SPs. Mean interrater reliability for the checklists was acceptable for the SPs watching the same videos and filling in the checklists, while the mean kappa for assessing concurrent validity in filling in the checklists was lower. The test-retest performance for assessing reliability resulted in a mean kappa of 0.72. All SPs except one, who was not recruited, performed acceptably well. CONCLUSION: The authors have demonstrated a thorough validation of the technique of using standardized patients in the portrayal of depressive disorders in primary-care settings in Iran, which creates confidence in employing this technique to evaluate doctors' performance, for example, after an educational intervention. Similar methods of validation can be used for SPs' portrayal of other psychiatric disorders.
Authors: Carolyn Baylor; Michael I Burns; Jennie Struijk; Lindsay Herron; Helen Mach; Kathryn Yorkston Journal: Am J Speech Lang Pathol Date: 2017-08-15 Impact factor: 2.408
Authors: James A Cartreine; Trina E Chang; Janette L Seville; Luis Sandoval; John B Moore; Shuai Xu; Mark T Hegel Journal: Depress Res Treat Date: 2012-11-14