OBJECTIVES: To evaluate the psychometric properties of HEARTSMAP, an emergency psychosocial assessment and management tool, and its impact on patient care and flow measures. METHODS: We conducted the study in two phases: first validating the tool using extracted information from a retrospective cohort, then evaluating implementation on a prospective cohort of youth presenting with mental health complaints to a tertiary Pediatric Emergency Department (PED). In phase 1, six PED clinicians applied HEARTSMAP to extracted narratives and we calculated inter-rater agreement for referral recommendations using Cohen’s Kappa and the sensitivity and specificity for identifying youth requiring psychiatric consultation and hospitalization. In phase 2, PED clinicians prospectively used HEARTSMAP and we assessed the impact of the tool’s implementation on patient-related outcomes and Emergency department (ED) flow measures. RESULTS: We found substantial agreement (κ=0.7) for cases requiring emergent psychiatric consultation and moderate agreement for cases requiring community urgent and non-urgent follow-up (κ=0.4 each). The sensitivity was 76% (95%CI: 63%, 90%) and specificity was 65% (95%CI: 55%, 71%) using retrospective cases. During pilot implementation, 62 patients received HEARTSMAP assessments: 46 (74%) of HEARTSMAP assessments triggered a recommendation for ED psychiatry assessment, 39 (63%) were evaluated by psychiatry and 13 (21%) were admitted. At follow-up, all patients with HEARTSMAP’s triggered recommendations had accessed community resources. For those hospitalized for further psychiatric care at their index or return visit within 30 days, 100% were initially identified by HEARTSMAP at the index visit as requiring ED psychiatric consultation. CONCLUSIONS: HEARTSMAP has strong reliability, and when applied prospectively is a safe and effective management tool.
OBJECTIVES: To evaluate the psychometric properties of HEARTSMAP, an emergency psychosocial assessment and management tool, and its impact on patient care and flow measures. METHODS: We conducted the study in two phases: first validating the tool using extracted information from a retrospective cohort, then evaluating implementation on a prospective cohort of youth presenting with mental health complaints to a tertiary Pediatric Emergency Department (PED). In phase 1, six PED clinicians applied HEARTSMAP to extracted narratives and we calculated inter-rater agreement for referral recommendations using Cohen’s Kappa and the sensitivity and specificity for identifying youth requiring psychiatric consultation and hospitalization. In phase 2, PED clinicians prospectively used HEARTSMAP and we assessed the impact of the tool’s implementation on patient-related outcomes and Emergency department (ED) flow measures. RESULTS: We found substantial agreement (κ=0.7) for cases requiring emergent psychiatric consultation and moderate agreement for cases requiring community urgent and non-urgent follow-up (κ=0.4 each). The sensitivity was 76% (95%CI: 63%, 90%) and specificity was 65% (95%CI: 55%, 71%) using retrospective cases. During pilot implementation, 62 patients received HEARTSMAP assessments: 46 (74%) of HEARTSMAP assessments triggered a recommendation for ED psychiatry assessment, 39 (63%) were evaluated by psychiatry and 13 (21%) were admitted. At follow-up, all patients with HEARTSMAP’s triggered recommendations had accessed community resources. For those hospitalized for further psychiatric care at their index or return visit within 30 days, 100% were initially identified by HEARTSMAP at the index visit as requiring ED psychiatric consultation. CONCLUSIONS: HEARTSMAP has strong reliability, and when applied prospectively is a safe and effective management tool.