Validation of the Family Meeting Behavioral Skills Checklist. An Instrument to Assess Fellows' Communication Skills.

RATIONALE: Fellows in pulmonary and critical care medicine are required to show competency in facilitating family meetings for critically ill patients. There are many assessment measures available for evaluating physician-patient communication (e.g., the SEGUE Framework [Set the stage, Elicit information, Give information, Understand the patient's perspective, End the encounter]) and some designed for family meetings. However, no validated measure exists that is specifically designed to assess communication skills during family meetings with surrogate decision makers in intensive care settings.
OBJECTIVES: We developed the Family Meeting Behavioral Skills Checklist (FMBSC) to measure advanced communication skills of fellows in family meetings of critically ill patients based on a literature review and consensus of an interdisciplinary group of communications experts. We evaluated the psychometric properties of the FMBSC.
METHODS: We digitally recorded 16 pulmonary/critical care fellows performing a simulated family meeting for a critically ill patient at the end of 1 year of fellowship training. Two clinical health psychologists evaluated each recording independently using the FMBSC Rating Scale and the SEGUE Framework. Judges recorded the number of skills performed using the checklist and employed a summary rating scale to judge the level of performance for each of nine subsets of skills. Each instrument was scored and converted to percentage scores. The FMBSC and SEGUE Framework items were summed and converted to percentage scores for each category and as a total for each instrument. The rating scale items on the FMBSC were also summed and converted to a percentage score. Four primary analyses were conducted to evaluate interjudge reliability, internal consistency, and concurrent validity.
MEASUREMENTS AND MAIN RESULTS: Interrater reliability was higher for the FMBSC (intraclass correlation [ICC2,2] = 0.57) than for the SEGUE instrument (ICC2,2 = 0.32) or the FMBSC Rating Scale (ICC2,2 = 0.23). The FMBSC demonstrated evidence of concurrent validity through high positive correlations with both the FMBSC Rating Scale and the SEGUE instrument (r = +0.83, P ≤ 0.01; r = +0.65, P ≤ 0.01 respectively). All but one of the nine subscales on the FMBSC showed adequate internal consistency (reliabilities ranged from 0.18 to 0.68). Interjudge reliability was higher for the FMBSC (ICC2,2 = 0.57) than for the SEGUE instrument (ICC2,2 = 0.32) or the FMBSC Rating Scale (ICC2,2 = 0.23).
CONCLUSIONS: The FMBSC demonstrated internal consistency and structural validity in assessing advanced communication skills of fellows in facilitating family meetings of critically ill patients in the ICU. Interjudge reliability was better for the FMBS Checklist than it was for the other measures.