A Review of Methods for Monitoring Adverse Events in Pediatric Psychopharmacology Clinical Trials.

Pediatric psychotropic prescription rates are rising, emphasizing the need for careful monitoring of drug safety in this population. Currently, no standardized assessments are used in clinical trials for adverse event (AE) elicitation focused on long-term drug treatment in pediatric patients. Despite a lack of standardized AE elicitation methods in psychiatric clinical trials, it is clear that psychiatric medications have developmentally dependent AEs that differ from those observed in adults. In this review, we discuss the use of general inquiry elicitation, drug-specific checklists, and systematic elicitation scales for AE reporting in pediatric psychopharmacology trials. The checklists evaluated include the Barkley Side Effect Rating Scales (SERS), the Pittsburg side effect rating scale, and the Systematic Monitoring of Adverse events Related to TreatmentS (SMARTS) checklist. The systematic assessment scales discussed include the Systematic Assessment for Treatment of Emergent Events (SAFTEE) and the Safety Monitoring Uniform Report Form (SMURF). We review the advantages and disadvantages of each method and discuss the need for optimal assessment of AEs. AE instruments that are created and utilized for pediatric psychiatric trials must begin to incorporate symptoms that are relevant to this population and account for the nature of the disorders to better characterize treatment-emergent AEs and monitor long-term safety.

Key Points

Introduction

As childhood psychiatric disorders are increasingly recognized and treated, the need for rigorous study of psychotropic medications with an emphasis on drug safety increases. A challenge in such trials is the lack of standardized systematic methods to evaluate safety by effectively assessing adverse events (AEs). Despite a lack of standardized AE elicitation methods in psychiatric trials, it is clear that psychiatric medications have developmentally dependent AEs that differ from those observed in adults. Studies have demonstrated that children and adolescents experience higher rates of nausea and activation when prescribed antidepressants [1]. Antipsychotics are associated with higher rates of sedation, weight gain, prolactin elevation, and withdrawal dyskinesia in children than in adults. Mood stabilizers have been associated with greater weight gain in children [2], and children receiving lamotrigine experience serious dermatologic side effects, including Stevens–Johnson syndrome, at higher rates than adults [3]. Perhaps one of the most serious and problematic examples of this was noted in 2003, when GlaxoSmithKline issued a letter to physicians warning them of a link between paroxetine, a selective serotonin reuptake inhibitor (SSRI), and increased risk of suicide in children. This link was established by the US FDA using post-marketing pooled data from several unpublished trials [4]. Had there been a more systematic method of AE elicitation, perhaps this AE would have been better captured.

Another example is that of the stimulant medication, pemoline, used to treat attention-deficit/hyperactivity disorder (ADHD) and narcolepsy in children. After two decades of spontaneous safety reporting to the FDA, it was discovered that pemoline caused liver toxicity in some children, which then prompted the FDA to require a label change to warn of this rare but serious side effect [5, 6]. Rare and serious AEs need to be captured but are often missed during pre-marketing clinical trials. The sample size of most clinical trials is too small to allow detection of rare events but could be improved if studies used more standardized systematic AE-elicitation methods [7]. This would also facilitate the combination of safety information from numerous studies of the same drug, and rare events would be less likely to be missed. Pediatric-specific phenomena such as those just discussed indicate that it is especially important to closely monitor AEs in pediatric populations, and systematic AE-elicitation methods would allow for better comparisons between studies.

Optimal assessment of AEs includes careful and thorough evaluation of symptom emergence, frequency, severity, associated concerns, and any impact on functioning and life satisfaction. Pediatric psychiatric populations present unique challenges to AE elicitation [8], as children may not be reliable reporters because of various factors, including age, verbal abilities, self-awareness, and ability to recall events chronologically. Parents or caregivers may naturally focus on AEs that are most easily recognized (e.g., vomiting, increased aggression) but struggle to recognize those that may be hidden (e.g., worsening anxiety or sadness). Further, the psychiatric condition being treated can itself impede accurate AE reporting. For example, children with anxiety may experience prominent somatic symptoms (e.g., stomachache, headache, muscle tension) during periods of high anxiety and struggle to differentiate these experiences from a side effect of a medication. A child with autism spectrum disorder may struggle to recognize and communicate their feelings and bodily sensations to others, impeding a parent’s ability to accurately report an AE. Further pediatric psychiatry populations may experience AEs that are very similar phenotypically but may have different underlying mechanisms and treatments (e.g., motor restlessness from akathisia vs. increased motor activity due to hyperactivity).

These challenges are an important backdrop to discussion of the three major methods currently used to elicit AEs in pediatric psychopharmacology trials: (1) a general inquiry prompt (e.g., “Has anything changed since the last visit?”), (2) a checklist based on side effects previously reported in other drugs of the same class, and (3) a systematic method of inquiry that covers all body systems. This article reviews the most commonly utilized AE scales in psychiatric clinical trials, highlighting the advantages and disadvantages of each method or scale. Table 1 summarizes the key features of each scale.

Table 1

Common adverse event-elicitation methods for psychotropic medications in pediatric clinical trials

Instrument/method	Informant	Grading/scoring scale	Advantages	Drawbacks
General inquiry	Patient	Spontaneous reporting by patients. Usually elicited by clinician prompt, “Has anything changed since last visit?”	Brief. Decreased risk of clinician bias on symptom reporting	Relies on patient recall. Risk of misclassification by clinician, specifically behavioral changes. Patient ability to share developmental/personal symptoms (i.e., sexual dysfunction, puberty). Not pediatric specific
Drug-specific checklist (SMARTS)	Patient	11-question checklist. Patients provide their responses to AE questions by circling relevant side effects	Brief. Utilizes established AEs from drugs in a similar “class” of medication	Not used independently—physical exam, blood tests, and probing for additional details recommended. Not a comprehensive AE elicitation—limited to known side effects for a particular drug class. Not specific to pediatric population. Not longitudinal
Drug-specific checklist. Barkley’s SERS	Patient (child), caregiver	17-item checklist for stimulant treatment. 9-point Likert scale. Requests feedback on behavioral AEs, including drowsiness, anxiety, irritability, sadness, social engagement	Brief. Pediatric specific. Addresses both behavioral and physical AEs. Encourages caregivers and children to complete together for accurate reporting	Not a comprehensive AE elicitation—limited to known side effects for a particular drug class. Not comprehensive. Not longitudinal
Systematic elicitation scale (SAFTEE-SI)	Patient, physician	Contains ~ 25 detailed questions that systematically address 29 body systems. Rating on five levels of severity. Takes 30–45 minutes to complete. Can be modified to fit specific trials appropriately. Suggested probe questions provided to elicit details	Covers all body systems. Developed for psychiatric trials. Elicits information about onset, duration, pattern, judgment of attribution. Preferred event terms—limit variance between physical symptoms. Standardized formats increase consistency of AE data	Time and cost burden make it difficult to implement. All potential AEs captured, makes treatment-emergent AEs difficult to extrapolate. Preferred event terms—not clear for behavioral symptoms, overlap between symptoms. Not longitudinal, but does address time intervals. Not pediatric specific
Systematic elicitation scale (SMURF)	Patient (child), physician	Adapted from the SAFTEE-SI. Composed of general inquiry questions and, if answered in the affirmative, then body system review questions and an AE report form are completed	Covers full range of body systems. Pediatric and psychiatric specific. Elicits information about onset, duration, pattern, status, severity, possible contributing factors and action taken	Time and cost burden make it difficult to implement. Additional forms are required for follow-up. Not longitudinal

AE adverse event, SAFTEE-SI Systematic Assessment for Treatment of Emergent Events-Specific Inquiry, SERS Side Effect Rating Scale, SMARTS Systematic Monitoring of Adverse events Related to TreatmentS, SMURF Safety Monitoring Uniform Report Form

General Inquiry

Historically, most pediatric psychopharmacology studies have used general inquiry prompts for AE elicitation. This method allows for symptom reporting without suggestive leads by a clinician that may influence what is reported. Many of the developmental differences in psychotropic AEs discussed in Sect. 1 were identified via a general inquiry prompt, illustrating its potential utility in identifying AEs. While this method is thought to be the most time efficient, it relies exclusively on the subject or caregiver’s ability to accurately recall a physical symptom or behavior change and on the clinician to accurately capture and classify the symptom as an AE. In theory, this should be relatively easy for most physical symptoms (e.g., a headache), but one could also envision a scenario in which several symptoms (e.g., headache, runny nose, sneezing) would be categorized as seasonal allergies by one clinician, a viral respiratory infection by another, and as individual AEs by a third. Accurate and consistent categorization of behavioral changes is even more challenging given the subjective element of interpretation and potential limitations in a child’s ability to accurately self-report internal states. Further, some potentially sensitive subjects, such as AEs related to puberty or sexual dysfunction, may be difficult for a subject or caregiver to spontaneously discuss.

Switching from a general inquiry prompt to a systematic method of elicitation has been shown to increase the number of AEs reported, as illustrated during assessment of suicidal AEs in the TORDIA (Treatment of SSRI Resistant Depression in Adolescents) study. Initially, 184 participants were assessed for suicidal events via spontaneous reporting; midway through the study, suicidality was elicited via systematic assessment in another 153 participants. Results demonstrated that fewer suicidal (8.8 vs. 20.9%) and non-suicidal self-injury (2.2 vs. 17.6%) AEs were detected via general inquiry than via systematic monitoring. There was no indication that treatments were different or that a cohort effect existed between the two groups. This example highlights the degree to which general inquiry can underestimate AEs compared with a more systematic approach [9]. Given this concern, the FDA now mandates use of the systematic assessment of the Columbia-Classification Algorithm for Suicide Assessment (C-CASA) in all antidepressant and anticonvulsant trials as well as other central nervous system (CNS) agents versus general inquiry to elucidate suicidal ideation and behaviors [10].

Drug-Specific Checklists

Drug-specific checklists are completed either by the clinician or directly by the subject/caregiver and/or teachers. These checklists provide a time-efficient but targeted method of AE elicitation. These checklists are designed based on previously reported AEs of other drugs in the same class.

The Barkley Side Effect Rating Scale (SERS) is an example of a caregiver-completed rating directed to caregivers or teachers of children receiving psychostimulants [11, 12]. It requests feedback on 17 items on a 9-point Likert scale, including typical physical AEs (e.g., decreased appetite, insomnia, stomachaches, and headaches) and behavioral symptoms (e.g., irritability, anxiety) [11]. Some investigators have encouraged caregivers and children to complete the rating scale together, which may provide more accurate reporting of AEs, particularly of internalizing symptoms [13]. The SERS has been used to compare AEs associated with different psychostimulants. For example, comparison of AEs between dextroamphetamine and methylphenidate in children with ADHD demonstrated that, on a group level, the rates of AEs were similar for both medications [13]. Clinical studies using psychostimulants in pediatric patients have also used the Pittsburg Side Effect Rating Scale [14]. This is a 13-item checklist that is completed by a caregiver/teacher and reviewed by a physician. AEs in this checklist are characterized as not present, mild, moderate, or severe. Importantly, this psychostimulant checklist was used to compare teacher- and parent-reported AEs in a large clinical trial for methylphenidate [15]. This trial found that teachers detected only one AE—appetite suppression—related to increasing dose. Teachers reported AEs most often when children were receiving placebo. Conversely, parental AE ratings increased as stimulant treatment dose increased and they noticed AEs at times that drug effects were peaking. These data suggest that caregivers may be the best source for AE ratings [16].

A clinician-administered drug-class-specific checklist known as the Systematic Monitoring of Adverse events Related to TreatmentS (SMARTS) has recently been developed. It is a short 12-item checklist of general inquiry questions that targets general physical AEs typically seen with antipsychotic use [17]. The questions are open-ended, allowing clinicians to ask for more details if desired. The SMARTS checklist captures the most common and important antipsychotic side effects but is brief enough to be efficiently implemented. The developers of the SMARTS emphasized that the checklist is not a comprehensive method of AE elicitation and should be combined with a physical exam and blood test analysis. The authors highlight that it is not a quantitative research rating scale, but rather an aid to stimulate discussion between patients and clinicians and ensure a more systematic approach to screening for side effects as against an unguided interview. Of note, this scale is not specific to pediatric populations and is not designed to capture AEs not typically seen with antipsychotics. To date, no studies have been published on the utilization of this checklist to monitor AEs in a clinical trial.

While drug-specific checklists are more precise than general inquiry, they remain limited to known AEs for a specific drug class. New and as yet unknown AEs for a drug could be missed by clinicians and overlooked by patients because they are not specifically emphasized. A systematic global elicitation method could provide AE information not captured in a drug-class-specific checklist and could allow for better comparisons between studies.

Systematic Elicitation Scales

Several scales that utilize systematic elicitation techniques have also been developed. These scales are longer and more time consuming than the methods discussed in previous sections but allow the reporting of any medical event that may occur over the full range of body systems. Systematic elicitation reduces the likelihood that a clinician will miss an AE, which is particularly important in a pediatric psychiatry population that is inherently difficult to assess. Inquiry about each body system at each visit increases awareness by the patient/caregiver and provides the clinician with important information needed to capture all possible AEs. For example, inquiry into sleep disturbance allows for discussion of sleep quality, potentially shedding light on an AE (e.g., nightmares) that a parent may not have directly observed. The specific measures discussed include the Systematic Assessment for Treatment of Emergent Events (SAFTEE) and the Safety Monitoring Uniform Report Form (SMURF).

Systematic Assessment for Treatment of Emergent Events

The SAFTEE is one of the first comprehensive AE-elicitation instruments developed specifically for use in psychiatric clinical trials.  A modified version of the assessment, the SAFTEE-specific inquiry (SAFTEE-SI), is a clinician-administered scale developed by Levine and Schooler [18] in the mid-1980s as part of a National Institute of Mental Health (NIMH)-sponsored workshop. The SAFTEE-SI is not a pediatric scale but deserves discussion because it inspired an AE scale for pediatric populations described in Sect. 4.2. It contains 25 detailed questions, typically takes 30–45 min to complete, and systematically addresses 29 body systems. The SAFTEE-SI allows ratings of five levels of severity and collects information about the onset, duration, pattern, judgement of attribution of cause, and action taken by the clinician. Suggested probe questions are also provided, which the clinician can use to elicit detailed information about the AE. Furthermore, the SAFTEE requires the clinician to determine a time interval of inquiry to be used in the trial (e.g., “Have you experienced any nausea in the last [week/4 weeks/6 weeks, etc.]?”). All AEs are reported, regardless of whether they are suspected of being related to the study drug; the authors state this removes a layer of subjectivity and speculation from the instrument. All versions of the SAFTEE incorporate a list of preferred-event terms to be used for reporting to eliminate some variability between perceived symptoms. For example, the same symptom can be described differently by each patient and then transcribed differently by each clinician. Using preferred-event terms can be helpful for physical symptoms such as when someone is reporting that they are “frequently tired” because in this case they would be mapped to the preferred term “fatigue.” When the symptom is behavioral, such as an “angry outburst,” the connections between terms are not clear. This behavior may be coded with numerous preferred terms, including “affect lability,” “aggression,” “disturbance in social behavior,” “irritability,” and “mood swings.” The SAFTEE is a standardized method, which increases consistency of AE data, both within and across clinical trials. Furthermore, the broad range of systematic information collected allows for thorough analysis of AEs [7, 19, 20].

Since its development, the SAFTEE or a modified version of the instrument has been used in at least 22 clinical trials for a variety of psychiatric conditions, including bipolar depression [21], major depressive disorder [22], alcohol dependence [23], and schizophrenia [24-26]. The SAFTEE has also been used in clinical trials for treatment of other medical conditions, such as HIV [27], epilepsy [28], and arthritis [29]. While comprehensive in the elicitation of AEs, this monitoring system would increase the time and costs associated with a visit and therefore may be difficult to widely implement in research or clinical practice. All AEs are captured, which may make it difficult to disentangle what may be treatment emergent.

Safety Monitoring Uniform Report Form

The SMURF was adapted from the SAFTEE to develop an AE-elicitation tool specifically aimed at pediatric populations by the NIMH-funded Research Units on Pediatric Psychopharmacology [30].

The SMURF contains a general inquiry and body system review (BSR). The body system inquiry section asks specifically whether the patient has experienced AEs specific to each of 26 body systems. If the answer is “Yes” to any of these questions, the clinician then asks for additional information using specified probes and completes an AE report form that collects information on onset, duration, pattern, status, severity, factors possibly contributing to the event, and action(s) taken because of the AE.

In the pilot study for the SMURF, clinicians administered the measure to a group of 59 caregivers and their children, who were being treated with stimulants, SSRIs, both, or other drugs. The SMURF in this study included a general inquiry, a drug-specific inquiry, and a comprehensive BSR. The BSR section of the SMURF took approximately 15 minutes longer to complete. Most clinicians felt the SMURF was too detailed and/or too long; however, most parents rated the detail of the measure favorably. Greenhill et al. [30] compared the systematic section (BSR) with the general inquiry section and found that approximately three times as many AEs were identified with systemic inquiry. In addition, overall, the BSR section elicited additional clinically significant AEs that had been missed with the less detailed methods. Along with disadvantages noted for the SAFTEE, including time/cost burden, the SMURF was not developed as a longitudinal assessment measure. Although it captures information about the past pattern of a specific AE, there is no set interval at which the SMURF is repeated to follow-up about the progression of the AE. Studies could develop a protocol for how often they repeat the SMURF questionnaire to determine treatment-emergent events, but this functionality is not part of the SMURF design.

Published clinical trials utilizing the SMURF to monitor AEs include several studies of medications to treat core symptoms of autism spectrum disorder [31-33] as well as the safety and efficacy of metformin to reduce weight gain in youth with autism spectrum disorder receiving an antipsychotic [34]. The SMURF is also currently being administered in ongoing behavioral therapy trials for Tourette’s, chronic tic disorder, and obsessive compulsive disorder [28, 29]. Further, since 2010, Spanish clinicians have been using an online registry to track antipsychotic AEs in youth aged > 18 years using several tools, including the SMURF, highlighting its potential use outside of a clinical research trial [35].

Conclusion

This article reviews several methods for eliciting and assessing AEs in psychopharmacological clinical trials. These methods vary in terms of informant, assessor, degree of detail, and time burden. Development of instruments for AE elicitation in pediatric psychopharmacology trials has proceeded slowly. There is also no accepted gold standard for AE elicitation and therefore no standard against which to test new methods. Methods for assessing AEs continue to improve, but several challenges remain. These include working with a pediatric population, the inherent difficulty in defining psychiatric terms, monitoring treatment-emergent AEs, and the trade-off between more detailed instruments, which elicit more AEs, and the increased time/cost burden of administering these instruments.

AE instruments that are created for pediatric psychiatric trials need to incorporate symptoms relevant to this population and consider the nature of the disorders to better characterize treatment-emergent AEs. Baseline AE measurements that can be followed longitudinally are critical for establishing treatment-emergent events. A longitudinal AE instrument would allow for symptoms experienced before drug treatment to be captured as well as serially over time, which would facilitate differentiation of truly treatment-emergent events versus symptoms of the disorder. A new instrument would also have to better define psychiatric terms so that the AE is recorded similarly by different clinicians. Standardized behavioral AE definitions, which continue to use dictionaries, such as the Medical Dictionary for Regulatory Activities (MedDRA®), may improve behavioral AE categorization. Incorporating the impact of the behavioral AE on the child’s daily functioning will help to better establish and standardize the level of severity. Improving and defining characteristics such as these can ensure an AE-elicitation instrument has a shared basis across studies. The development of methodologies for eliciting and recording drug safety has not been on the forefront of clinical studies in pediatric psychopharmacology to date, but the implications are particularly salient in this population. With the use of psychotropic medication in these populations increasing, it is crucial that we begin to monitor drug safety more rigorously to be able to provide an informed risk–benefit analysis to this population.

Adverse event (AE) elicitation in pediatric psychiatry populations is a complex task of significant importance.

Existing measures used to characterize AEs are reviewed and critiqued, with particular emphasis on the need for systematic methods used longitudinally to provide more accurate characterization of AEs.