Implementation and evaluation of a curriculum on the assessment and treatment of disruptive behaviour disorders.

Disruptive behaviour disorders (DBDs)-which can include or be comorbid with disorders such as attention-deficit hyperactivity disorder, oppositional defiant disorder, conduct disorder and disruptive mood dysregulation disorder-are commonly seen in paediatric practice. Given increases in the prescribing of atypical antipsychotics for children and youth, it is imperative that paediatric trainees in Canada receive adequate education on the optimal treatment of DBDs. We describe the development, dissemination, and evaluation of a novel paediatric resident curriculum for the assessment and treatment of DBDs in children and adolescents. Pre-post-evaluation of the curriculum showed improved knowledge in participants.

Disruptive behaviour disorders (DBDs) may be broadly considered to include attention-deficit hyperactivity disorder, oppositional defiant disorder, conduct disorder and disruptive mood dysregulation disorder. DBDs, when persisting to adulthood, can interfere with both professional and personal life, and can negatively influence overall quality of life (1). Thus, providing effective and safe treatments for aggression and disruptive behaviour is of extreme importance.

Pharmacoepidemiologic studies (2–5) have found a significant increase in the prescribing of antipsychotics for children and youth. A major concern regarding the use of antipsychotics is their propensity to cause metabolic, hormonal, and extrapyramidal side effects (6).

Strong evidence supports the efficacy of psychosocial interventions and behavioural modification programs for the management of aggression in youth, and published clinical practice guidelines (7) emphasize a decreased role for antipsychotics.

The main study objective was to increase paediatric residents’ knowledge in the assessment and treatment of childhood DBDs, with an emphasis on nonpharmacologic interventions, in order to improve quality of care.

METHODS

The curriculum development has been described previously (8). Briefly, we organized a meeting of researchers, practitioners, and knowledge users from across Canada, to develop educational goals for the curriculum based on Royal College of Physicians and Surgeons of Canada objectives of training. A needs assessment confirmed the knowledge needs among physicians surveyed in the assessment of DBDs, psychosocial therapy recommendations, choice of pharmacological interventions, and appropriate drug safety monitoring. This meeting resulted in the creation of guidelines (9) that, with the T-MAY guidelines (7), formed the curriculum foundation.

The curriculum includes slide sets, videos, small group exercises, evaluation materials, and key references. Before dissemination, it was piloted with paediatric residents at a single site, and revisions—such as video demonstrations of parent- and child-based group therapy and interactive small group exercises—were added.

Dissemination

Paediatricians and child and adolescent psychiatrists from each Canadian medical school were recruited to attend a ‘Train-the-Trainer’ event, with the end goal of participants implementing the curriculum at their home institutions. We also created an open access website (www.readycanada.org), where the curriculum materials were available for attendees—as well as other interested educators—for use in their training programs.

Evaluation was conducted via pre- and post-tests and an Observed Structured Clinical Examination (OSCE). As the pre- and post-tests and OSCE were held on different days, not all trainees participated in both evaluations. A priori sample sizes were not calculated, as this was a pilot study.

Pre- and post-test

A 20-question multiple choice test was developed based on the curriculum materials and was administered to participants before and after delivery of the curriculum. Paired t-tests were used for pre- and post-test comparisons.

OSCE

An OSCE station and scoring sheet was developed based on curriculum materials, with the scoring sheet comprising a checklist and global assessment. The OSCE station and scoring sheet were piloted at a single site on five recently graduated paediatricians and revised based on their feedback.

In Canada, all paediatrics programs provide a practice OSCE, in which residents across the country are evaluated using the same standardized OSCE stations. We submitted our station for inclusion in the national OSCE. It was reviewed by the Canadian program directors OSCE Vetting Committee and delivered in the Fall of 2018.

To measure the impact of the curriculum, we performed a nonblinded cluster randomized trial (ClinicalTrials.gov Identifier: NCT03151187), where we randomized (via a random number generator) some programs to deliver the curriculum to their trainees in the 6 months before the OSCE, and others programs to deliver it in the 6 months after the OSCE. Sites were randomized depending on the size of their residency program (Figure 1). Independent samples t-tests were used to compare these groups.

Figure 1.

Eligibility, randomization, and analysis.

Informed consent was given by all participants before inclusion in the study. The study was approved by the research ethics boards at all participating sites.

RESULTS

Six residency programs participated in the pre–post-test evaluation (44 residents). There was a significant difference between the scores (t=−13.67, P<0.001), where the post-test scores (mean=16.81, SD=1.60) were significantly higher than the pre-test scores (mean=12.36, SD=1.66).

Please see Figure 1 for the flow diagram of participation in the OSCE component. Forty-four residents (representing two programs) received the curriculum before the OSCE. This ‘early’ group consisted of the following trainees: 14 Postgraduate year (PGY) 1, 18 PGY 2, 2 PGY 3, and 10 PGY4. Fifty-one residents (representing four programs), received the curriculum after the OSCE. This ‘late’ group consisted of the following trainees: 11 PGY 1, 17 PGY 2, 16 PGY 3, and 7 PGY4. There were no significant differences in OSCE scores (t=0.120, P=0.905) when comparing the early curriculum group (mean=49.28, SD=5.46) to the late curriculum, group (mean=49.11, SD=8.35). There were also no significant differences between the ‘early’ and ‘late’ groups when comparing OSCE scores amongst postgraduate years.

DISCUSSION

We describe a novel curriculum that provides training to paediatric residents in the approach to care for youth with DBDs, with a focus on nonpharmacologic interventions. Findings showed improved knowledge in participants after the curriculum was delivered.

Although knowledge improved following completion of the training, performance on the OSCE did not. One explanation for the latter finding is that the analysis may not have been powered sufficiently to detect a difference in OSCE scores. In addition, it is possible that trainees who took the OSCE before the curriculum might have informally learned part of the curriculum in other areas of their overall paediatrics training, thus contributing to the lack of difference in OSCE scores. Arguing against this is the larger standard deviation in the ‘late’ curriculum group (post-OCSE) as compared to the early curriculum group. This may indicate that the ‘early’ curriculum group were relatively ‘naive’ to the components of the DBD curriculum, and that the late group may have been more heterogenous, including trainees who have incorporated the knowledge from the curriculum as well as those that had not. The reasons for the heterogeneity in the ‘late’ group are unclear, but it could be argued that our curriculum should include more ‘hands-on learning’ to help translate the knowledge gained into behavioural change.

Strengths of our study include involving pertinent stakeholders in the development of the curriculum, and performing a needs assessment before initiation. We also demonstrated the feasibility of implementing the curriculum in different paediatrics residency programs across Canada. Limitations include the small number of trainees who participated in the study and the low proportion of Canadian residents who consented to be in the study. It is possible that the ones who consented are strongest in the knowledge of DBDs, resulting in no significant difference in OSCE results.

Moving forward, we would endeavour to implement the curriculum in every paediatrics residency program across Canada and include more experiential learning in the curriculum. One possibility would be to consider longitudinal delivery of the content with patients in the community setting to test the application of the knowledge acquired in the curriculum. If this were to occur, higher level outcomes could also be evaluated, such as whether implementation of the curriculum results in decreased prescribing of atypical antipsychotics.

Funding: This research was supported by grants from the Canadian Institutes of Health Research, the SickKids Foundation, and the Royal Bank of Canada Knowledge Translation Fund. The sponsors were not involved in the study design; the collection, analysis or interpretation of data; the writing of the report; or the decision to submit the article for publication.

Potential Conflicts of Interest: The authors disclose that funding for this research was provided by the Canadian Institutes of Health Research, the SickKids Foundation, and the Royal Bank of Canada Knowledge Translation Fund. There are no other disclosures. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.