Do panels vary when assessing intrapartum adverse events? The reproducibility of assessments by hospital risk management groups.

INTRODUCTION: A national audit project, Scotland-wide Learning from Intrapartum Critical Events (SLICE), included local assessment of quality of care in cases of perinatal death and neonatal encephalopathy due to intrapartum events. Concerns had been raised about interobserver variation in case assessment by different panels. We therefore studied the extent of agreement and disagreement between assessment panels, and examined the areas in which agreement and disagreement tended to occur.
METHODS: 8 cases were randomly selected from all 42 cases identified during a 6-month period (1 January-1 July 2005). Each case was independently reviewed by three panels: the local hospital clinical risk-management group and two specially convened external panels. Panels assessed quality of care in three areas: admission assessment, recognition of incident, and method and timing of delivery. Predefined standards of care were provided for these three areas. Panels were also asked to assess the overall quality of care.
RESULTS: For each area of care, agreement between the two external panels was lowest. The lowest levels of agreement between panels were seen in assessment of overall care (50% crude agreement between external panel 1 and the hospital (kappa = 0.24, AC(1) = 0.36); 29% crude agreement between external panels 1 and 2 (kappa = -0.11, AC(1) = 0.1); 47% crude agreement between external panel 2 and the hospital (kappa = 0.36, AC(1) = 0.46). The lowest level of agreement among all three panels was also in the assessment of overall care (crude agreement 48%; kappa = 0.16, AC(1) = 0.34).
CONCLUSION: Moderate to substantial agreement among the three panels was achieved for the three areas in which explicit standards were provided. Therefore, a systematic approach to analysis of adverse events in perinatal care improves reproducibility.