Improving Physical Abuse Documentation and Photography through a Remote Peer Review Intervention.

Child Abuse Pediatrics is a small and geographically dispersed specialty. This article reports on an intervention to improve written and photodocumentation quality and uniformity in suspected child physical abuse cases, using a remote, de-identified case review system.
METHODS: In each cycle, participants submitted de-identified medical reports and photographs for review by a child abuse pediatrics expert. Experts evaluated 3 cycles of 5 cases using a novel rubric and assigned quality interventions for the participants based on their scores.
RESULTS: 15 of 16 participants improved scores between cycles 1 and 3 (78% versus 89%, P < 0.001). All participants rated the program as helpful and would recommend it to a colleague.
CONCLUSION: A quality improvement project administered via the internet improves the quality and uniformity of written and photographic documentation in child physical abuse evaluations.

INTRODUCTION

Problem Statement

Child physical abuse is an important cause of pediatric morbidity and mortality associated with significant physical and mental health problems extending into adulthood. In 2018, US authorities received 4.3 million suspected child abuse or neglect reports.[1] Of the estimated 675,000 children found upon investigation to be victims, 15%–20% experienced physical abuse. An estimated 1770 children died of abuse and neglect—a rate of 2.39 per 100,000 children in the United States.

Recognizing and reporting physical abuse is essential to quality pediatric healthcare.[2] However, pediatricians have historically felt uncomfortable diagnosing physical abuse and have expressed that they feel inadequate to care for these victims optimally.[3]

Documentation of the medical evaluation of child physical abuse is a cornerstone of professional effort to diagnose and treat suspected child abuse victims accurately. In contrast to purely clinical medicine, a forensic note by a child abuse pediatrician may directly benefit the patient by facilitating legal interventions for the child’s well-being. Careful documentation of visible injuries by the written description and digital photographs facilitates peer review, investigation, and court procedures.[4]

Available Knowledge

Having participated in previous learning collaboratives utilizing the IHI Breakthrough Series collaborative model,[5] the authors are aware of the impact of interweaving learning opportunities with action periods in which participants implement new skills in their clinical practice. Improved knowledge and practice have been demonstrated for child abuse pediatricians (CAPs) and others in child sexual abuse through the use of expert case review.[6] A prior ABP-approved Maintenance of Certification Part 4 project addressed quality of sexual abuse evaluation.”[7] Lastly, evidenced-based recommendations[2,8] for physical abuse evaluation provide best practices to target for the present intervention.

Rationale

Unlike other forms of medical documentation, the written evaluation of an injured child for suspected abuse is a tool to communicate with a multidisciplinary audience outside of medicine. This suspect abuse report requires written and photographic documentation of the highest quality and clarity. Medical records that reflect specific concerns, alternative diagnostic possibilities, and additional testing results are essential for later review and to assist child protective services or police investigations.[4] The quality of written documentation and photographs can directly influence outcomes in child abuse cases.[9]

Based on their experience as child abuse practitioners and educators, the authors know common inadequacies in this written and photographic documentation. These include failure to collect and document sufficient history, lack of consideration of alternative diagnoses, and poor photographic methods.[10,11] Many medical records do not adequately reflect the medical provider’s thoughts about how the injuries could have been sustained, including the mechanism, force, and timing.[12]

Specific Aims

This project intended to (1) improve the completeness of physical abuse history and examination documentation, (2) increase the quality of physical abuse photography as judged by expert evaluators, and (3) increase compliance with evidence-based best practices for the radiologic and laboratory evaluation of physical abuse.

The project aimed to achieve these improvements using an internet-based, remote intervention that is feasible for a small specialty with widely dispersed practitioners.

METHODS

Context

In 2006, the American Board of Pediatrics approved a new subspecialty named child abuse pediatrics (CAP) to provide medical expertise to care for maltreated children.[13] The field has grown slowly since that time. CAPs now practice in most US states, but their distribution is uneven, averaging 0.6 CAPs per 100,000 child population. Some states have no certified CAPs.[14] Most CAP practices are small, employing 3 or fewer CAPs.

The Midwest Regional Children’s Advocacy Center (MRCAC) is a federally funded Department of Justice Juvenile Delinquency and Prevention project. Its mission is to improve the community response to child abuse through strategic leadership, collaboration, and capacity-building, with the vision that all children and all families will have access to a strong team of highly qualified professionals for the response to and healing from child abuse. The center provides training and technical assistance to members of the multidisciplinary team, including medical professionals. The MRCAC provided the web platform and administrative oversight for the project.

The authors expected low initial scores for photographic quality, mainly based on prior experience with a similar program for sexual abuse documentation but expected all score components to improve as participants progressed through the program.

Intervention

To assess and improve the quality of medical written and photographic image documentation for child physical abuse by pediatricians, we undertook a distributed QI project modeled after a prior project involving sexual abuse documentation.[7] The overall design included a QI project for physician participants and an internal QI project for the expert case reviewers. The project was reviewed and approved by a single Institutional Review Board at the American Academy of Pediatrics.

Direct emails, online notices,[15] and a platform presentation at a national meeting of child abuse medical providers announced the availability of Part 4 Maintenance of Certification credit for physician participants in this project. This cost of participation was $250, which is similar to other Maintenance of Certification part 4 projects.

Each participant completed 3 PDSA cycles consisting of:

Participants submitted records from 5 actual physical abuse cases evaluated by the participant.

Expert evaluators scored the documentation and photographs using a rubric described below. A single evaluator, assigned sequentially on entry, evaluated each participant throughout that participant’s participation in the project.

Participants reviewed the scores and comments from the evaluators. A minimum of 30 days elapsed between the end of one cycle and initiation of the next.

After cycle 1 and cycle 2, participants completed educational interventions assigned by the reviewer based on their scores.

In each cycle, participants submitted 5 consecutive physical abuse cases. Each case consisted of the physician’s de-identified clinical written documentation and 2-10 digital photographs. Participants submitted consecutive cases instead of choosing the 5 “best” cases to assure that the cases submitted accurately represented the physician’s standard work product.

Following cycles 1 and 2, participants received feedback from their evaluator. Feedback included both scores on the rubric described below and narrative feedback regarding each case. Evaluators assigned interventions based on learning opportunities derived from the assigned scores. Interventions included watching an online video presentation, reading selected journal articles, or exercises such as taking several pictures using different camera settings. A complete list of available interventions is included in Supplemental Digital Content 1. (See appendix, Supplemental Digital Content 1, which shows interventions available for assignment to participants. http://links.lww.com/PQ9/A316.) The available interventions were created or selected by the authors based on their experience.

Expert evaluators for this project are also the physician authors of this report. Each of the expert evaluators is an actively practicing child abuse pediatrician. The expert evaluators have published studies regarding physical abuse evaluation, lectured on the topic of physical abuse diagnosis, and participated as experts in another quality improvement project.

Measures

The authors identified key drivers of quality written and photographic documentation based on their general experience as CAP educators and prior quality improvement projects. A key driver diagram is displayed in Figure 1.

Fig. 1.

Key driver diagram for the project.

The authors devised a novel rubric to evaluate the written consultation note and photographs (Table 1). Seventeen yes/no items represent aspects of the key drivers previously identified. Despite efforts to be objective, many items (especially photo-quality items) are inherently subjective. Participants received the rubric before their participation.

Table 1.

Scoring Rubric

Item	Points
History
Narrative history of event from caregiver, or caregiver is noted to be unavailable.	1
Mechanism of injury is either documented, or documented to be unknown.	1
Time of injury or time last known to be healthy is documented.	1
Caregiver’s explicit denial of additional recent trauma is documented.	1
Examination
Anthropometrics and age percentiles are documented.	1
All significant injuries documented both in writing and photographs.	1
All injuries mentioned in the history are discussed in the examination.	1
Evaluation
Evaluator agrees with decision to obtain / not obtain skeletal survey.	1
Evaluator agrees with decision to obtain / not obtain neuroimaging.	1
Evaluator agrees with decision to obtain / not obtain labs to screen for abdominal trauma.	1
Evaluator agrees with decision of obtain / not obtain coagulation labs.	1
Assessment
An assessment of abuse likelihood is clearly documented in lay language. (Point not dependent on the evaluator agreeing with the assessment.)	1
Image Quality
Images demonstrate sufficient sharpness and focus.	1
Images demonstrate appropriate colors.	1
Images demonstrate sufficient brightness and contrast.	1
Images demonstrate appropriate composition.*	1
Sufficient images presented to document the described findings, or the maximum of 10 images were submitted.	1
Total	17

*Composition refers to a photo that clearly shows the finding of interest, with appropriate zoom, and does not include unwanted elements.

The project did not attempt to validate the rubric formally. A single case was selected and discussed among the evaluators at each quarterly evaluation meeting. The evaluators informally agreed on interpretations of the rubric. We minimized interrater reliability problems by having each participant evaluated by a single evaluator.

Analysis

Descriptive statistics characterize participant scores for each PDSA cycle. A paired t-test assessed improvement in scores between cycle 1 and cycle 3.

Ethical Considerations

The project was reviewed and approved by the institutional review board at the American Academy of Pediatrics. Participants de-identified all materials before submission. The project coordinator promptly returned any identifiable submission to the participant before review.

Because all submissions were strictly de-identified, expert evaluators did not incur mandated reporting responsibilities due to their participation in the program. While the program provided timely feedback to the participating physician, evaluation and implementation of any suggested change in management was left exclusively to the participating physician.

RESULTS

Eighteen physicians participated in the program in the first 3 years. Of these, 14 were board-certified CAPs, 7 had practiced for 10 or more years, and 10 practiced for 5–9 years. Most (10) saw more than 100 cases of physical abuse per year. Sixteen of the group (74%) completed all 3 case sets.

Figure 2 is a run chart of mean scores for the 16 completers for each of the 15 submitted cases. Initial scores were above expectation (78%) and steadily increased in set 2 (83%) and set 3 (89%). A paired t-test revealed a significantly better performance on cycle 3 than cycle 1 (78% versus 89%, t = 6.44, P < 0.0001). Individual participant’s changes in scores throughout the program ranged from −5.9% to +21.2%, with a mean of +10.8%. A single participant had a decreased score, but that participant’s initial score was high (88%).

Fig. 2.

Run chart of mean scores for 16 participants in the project.

Various components of the total score showed important differences in the changes in scores from cycle 1 to cycle 3. Figure 3 shows run charts for various subsets of the entire metric. Photographic quality showed the most significant improvement (18%), followed by examination (13%) and history (11%) documentation. Improvement was less impressive for workup (3%) and assessment (4%) documentation. Significant improvements in examination and photographic quality correspond to lower baseline scores (65% and 70%, respectively) relative to workup and assessment (88% and 94%, respectively).

Fig. 3.

Run chart of various components of the score. A, History. B, Physical examination. C, Photography. D, Workup and assessment.

Of the 16 participants who completed all case sets, 11 (69%) returned a program evaluation survey. Ten participants (91%) rated the project as excellent in every category requested. One participant gave the program lower ratings regarding the helpfulness of the feedback received and the time required to complete the project. In narrative comments, participants noted improvements in their photo-documentation technique, explicitly mentioning that they had learned more about lighting and the importance of reviewing their images before patient discharge. Others felt that their diagnostic skills had improved. Still, others felt that the feedback helped reaffirm what they knew and that the feedback helped them with what they might want to change/improve based upon the points mentioned.

DISCUSSION

Our results demonstrate that a remote intervention, administered over the internet, consistently improves the quality of photographic and written documentation of child physical abuse. In addition to improving overall quality assessments, participants demonstrated improvement in each area identified in the specific aims. The intervention requires modest resources and demonstrated high acceptability with the participants.

This project is not a faithful implementation of the IHI Breakthrough Series model. Most notably, the learning sessions were individual and by computer rather than in person, and participants were individuals rather than teams. Our project does use multiple implementation periods interspersed with an evaluation of the participant’s actual performance and learning opportunities. The project achieved significant improvements in observed performance despite these deviations from the model.

Interpretation

This model of quality improvement fits well with the demographics of our specialty. Child abuse pediatrics (CAP) is a young and small specialty with 375 board-certified CAPs.[16] Many CAP practices include only 1 or 2 CAPs. This intervention’s distributed-model allows CAPs in small programs to participate in quality improvement specific to their subspecialty and relevant to their daily work.

Another opportunity provided by this project is the ability to improve consistency among CAP practices. Consistency is particularly desirable in CAP because of the legal implications of child abuse diagnoses. In purely clinical medicine, it may be acceptable for different centers to adopt differing protocols, none of which are demonstrably superior to others. However, we would be justifiably concerned if various child abuse providers’ idiosyncratic beliefs led to different criminal or child protection outcomes in jurisdictions served by different child abuse practices.

This quality improvement project explicitly notified providers that they would be evaluated on documentation, photography, and compliance with guidelines[8,17,18] for workup, but not for the final case diagnosis or opinion expressed regarding the cause of the injury. The legal environment in which CAPs practice and the project organizers’ goals contributed to the decision to not evaluate the final opinion. Indeed, the final opinion in a child abuse evaluation will often rely on other multidisciplinary team partners’ information and not medical factors alone. Future projects may engage diagnostic consistency more directly.

LIMITATIONS

This report has several limitations. First, the 18 participants in our program is a relatively small number, even though this number represents 5% of the board-certified CAPs. Our results may not generalize to larger programs and reflect only those aiming to improve their practice via a quality improvement initiative. We did not measure any post-intervention assessments with which to judge the sustainability of the improvements noted above.

Another limitation is the evaluation rubric, which has not been validated. Despite our efforts to be objective and reduce inter-rater differences among the reviewers, many items contain significant subjective judgments. While the expert evaluators reviewed one case quarterly as a group and discussed differences in opinion, we did not collect enough data from these meetings to formally evaluate interrater reliability.

Additionally, participants may have learned to include the specific items listed in the rubric, especially historical and physical examination items, without a corresponding increase in the quality of their notes. This possibility is supported by the observation that the majority of the improvement in the history and examination scores happened between the first and second PDSA cycles. Photographic quality, which requires the participants to develop new skills, improved steadily over the second and third PDSA cycles.

Reviewers found the photo assessment portion of the rubric challenging to use. Reviewers awarded a point if the photos overall had an appropriate composition, focus and sharpness, color rendition, and whether the participant provided an adequate number of photos to document the reported injuries. In retrospect, attempting to rate the photos as a group was problematic, and simple yes-no queries provided insufficient precision. This imprecise metric somewhat limits the evaluation of improvement in photographic abilities.

CONCLUSIONS

Peer review with interventions customized to address quality issues resulted in improved documentation and photographic quality in child physical abuse evaluations. This remote intervention is feasible and acceptable to CAPs in a variety of practices. This model of quality improvement may apply to other small subspecialties.

In CAP, the quality of written and photo documentation is uniquely crucial because accurate and thorough physician documentation directly contributes to the legal findings necessary to impose protective measures. We anticipate that demonstrated improvements in written and photographic documentation will directly impact patient outcomes. Future work in this field may include addressing diagnostic agreement in addition to documentation quality.

DISCLOSURE

Drs. Palusci, Moles, and Melville have each provided expert testimony in cases alleging child maltreatment. They or their hospitals may have been compensated for some of this testimony. Ms Martinez has no financial interest in relation to the content of this manuscript.

ACKNOWLEDGMENTS

The authors thank Dr. Elizabeth Mack, MD for helpful advice and her review of a draft of the article. This study was supported by Midwest Regional Child Advocacy Center (MRCAC) and the American Academy of Pediatrics. MRCAC’s participation was supported by Grant 2019-CI-FX-K004 awarded by the Office of Juvenile Justice and Delinquency Prevention, Office of Justice Programs, U.S. Department of Justice.