Real-World Evidence for Assessing Treatment Effectiveness and Safety in Pediatric Populations.

The race for effective treatments against coronavirus disease 2019 has reminded pediatricians of historical delays that children have faced in drug development and clinical trials. In response to this, legislation and regulations including the Best Pharmaceuticals for Children Act (US, 2002), Pediatric Research Equity Act (US, 2003), and Pediatric Regulation (European Union, 2007) spurred the conduct of pediatric trials through a combination of incentives and requirements for pharmaceutical companies to increase pediatric drug approvals and evidence supporting treatments for children. Although randomized controlled trials (RCTs) are considered the most rigorous design to demonstrate a drug's efficacy, design alone does not determine whether evidence from the trial is sufficient to establish substantial evidence of effectiveness, as defined in the Kefauver Harris Amendment to the Federal Food, Drug, and Cosmetic Act (US, 1962). New indications for treatments are sometimes approved by the US Food and Drug Administration (FDA) and European Medicines Agency without RCT results, though the reasons for not requiring evidence from RCTs are not always evident. RCTs have many well-known limitations, including high costs, small sample sizes, short follow-up, and questionable generalizability to broader populations, and they cannot answer all the important questions, such as rare but serious harms or long-term safety. Pediatric trials have additional feasibility concerns, relating to smaller potential pools of participants with longer enrollment process; diverse physiology, pathophysiology, and treatment responses across different ages; and practical challenges with recruitment and informed consent. Furthermore, reporting from some mandated pediatric studies has been limited, and pediatric trials and approvals may lag for years despite the legal requirements in place. Despite increases in pediatric labeling, outpatient off-label drug prescribing to children has risen, particularly for conditions without FDA approval at any age.

Certain collaborative research networks, such as the Children's Oncology Group, have sufficient size, resources, and capacity to conduct large RCTs with long-term follow-up that address some of the aforementioned limitations of many pediatric trials.4, 5, 6 Nonetheless, for many conditions, drugs, and outcomes, the challenges are not solved uniformly through more high-quality pediatric RCTs, if feasible. Even in the field of pediatric oncology, certain outcomes (eg, severe, late-onset cardiomyopathy) are too rare for even large trials to detect and study with any precision. Rigorous research using alternative approaches is also vital to inform pediatric prescribers, caregivers, and patients. Such approaches can help us more fully understand the effects—favorable and unfavorable—that treatments have in children often more efficiently, cheaply, and with greater generalizability than through RCTs.

The 21st Century Cures Act (US, 2016) places additional focus on uses of data collected outside traditional RCTs (real-world data [RWD]) to generate clinical evidence (real-world evidence [RWE]) to support regulatory decision-making. RWD encompasses sources such as electronic health records (EHRs), insurance claims, product/disease registries, patient-/caregiver-reported outcomes, and wearable/other devices. The framework for FDA's RWE program suggests that RWE may fill evidentiary gaps when traditional RCTs are not feasible, including for populations under-represented in RCTs. RWE may also help assess outcomes/endpoints more valued by patients, families, and payers. Nonetheless, the successful application of RWE to pediatrics requires understanding of both the opportunities and challenges that exist (Table ).

Table

Opportunities and challenges in using real-world evidence for pediatric populations

Issues	Role of RWD/RWE	Example(s)
Opportunities
Pediatric RCT is not feasible or ethical	Characterize the natural history of a disease and provide the supplementary contextual data necessary to interpret results from single-arm trials	Approval of medications for rare pediatric diseases following single-arm trials9 (eg, approval of cerliponase alfa as a treatment for a form of Batten disease), following a single-arm study which used a natural history RWD external control10, 11
Treatment is approved for adults and available for use in children before official pediatric approval	Provide evidence on safety and effectiveness of treatments in pediatric populations	Safety and effectiveness of treatments for multiple sclerosis before pediatric RCTs are completed or regulatory approval is granted12
Clinical outcomes used and validated in adult studies may not be appropriate or adequate in pediatric patients	Validation of pediatric outcomes or surrogate measures for clinically important outcomes in pediatric populations	Observational cohort studies using RWD to elucidate the associations among childhood hypertension and surrogate or subclinical measures of cardiovascular disease13
Treatment is tested in and approved for children, but RCTs are limited to narrow populations, short-term exposures, and limited sets of outcomes	Address questions about more diverse populations, chronic exposures, and unexamined outcomes (eg, delayed, rare, untested)	Effectiveness and safety of ADHD medication in underserved or nonadherent children, in children with autism, or when taken at unapproved doses or with antidepressants; impact of treatment on future scholastic performance or risks of substance abuse or suicide
Challenges
Bias from confounding in observational research on effects of treatment in children	Use designs that address confounding by indication or disease severity	Comparison of treatments given for similar patients and indications (active-comparator design)14, 15
	Statistical adjustment for measured confounders	Multivariable modeling, propensity scores, disease risk scores, or other approaches16
	Statistical adjustment for proxies of unmeasured confounders	Adjustment for health utilization metrics (eg, hospitalization, number of office visits)17
	Address missing variables in individual data sources	Linkage between complementary data sources (eg, administrative claims with dispensing data and EHR data with metrics related to disease severity or growth)18
		Following individuals as their own controls over time to see whether the timing of treatment corresponds to the timing of outcomes, inherently controlling for time-invariant confounders (self-controlled study design)19
		Comparison of siblings to determine whether differences in treatments correspond to differences in outcomes, controlling for shared genetic and environment factors (sibling-controlled design)20
		Use of proxies of treatment selection, otherwise unrelated to the outcome (eg, variable prescribing practices independent of disease severity), for unbiased estimates of treatment effects (instrumental variable design)21
		RCT using broad inclusion criteria (eg, all children with persistent asthma in a health care system) and RWD collection (eg, EHR data) to produce RWE on treatment effectiveness or safety (pragmatic clinical trials)22
Understanding the effects of dose on pediatric outcomes	Use data source with weight data (eg, EHR data) or impute weight based on applicable growth charts	Study of dose-effects of glucocorticoids by imputing weight-based-dose using median weight for age and sex (for population-level, not individual-level, estimates)23
Challenge of studying impact of treatment on growth and impact of growth on treatment response	Use data source with weight and height data (eg, EHR data)	Study of how antipsychotic dose differentially affects weight of obese and non-obese children
Limited access to patient populations or outcomes of interest	Use RWD to standardize and validate condition or outcome of interest	Validation of algorithm for ventricular arrhythmia and cardiac arrest using combination of diagnostic codes and treatments24
	Link to data sources with available outcome data	Linkage between electronic health care data and educational outcome data, for example, to study the relation between antidepressant use and educational performance or attainment25
Limited statistical power because of rarity of pediatric diseases, exposures, and outcomes, as well as considerations of age subgroups	Use large administrative or clinical database or combination of databases (eg, global multidatabase study) with a sufficiently large pediatric population	Use of linkable regional or national databases to study pediatric mortality as a study endpoint26
Limited statistical power or selection bias in study of long-term pediatric outcomes because of loss to follow-up (eg, change in health plans, loss of insurance)	Use data sources from settings with universal health care and comprehensive follow-up or use additional data sources to gather the missing information	Use of Scandinavian registry data to study long-term outcomes of prenatal or early childhood exposure27

ADHD, attention deficit/hyperactivity disorder.

Opportunities

Before treatments are approved for use (initially usually for adults), substantial evidence of efficacy of medical products is required from adequate, well-controlled studies, usually RCTs. However, in certain circumstances, randomization to placebo or alternative treatment is not feasible or ethical, necessitating the use of single-arm trial designs. Real-world benchmark data from real-world external (historical) controls can provide the supplementary contextual data necessary to interpret results from single-arm trials. Multiple medications have received regulatory approvals with supportive RWE derived from real-world external controls, particularly for oncologic and rare disease indications. For example, cerliponase alfa was approved as a treatment for a rare pediatric lysosomal disorder (neuronal ceroid lipofuscinosis type 2 disease) following a single-arm study, which used a natural-history RWD external control. , As another example of RWE supporting pediatric drug approvals, blinatumomab was approved for precursor B-cell acute lymphoblastic leukemia in children based on efficacy data from an open-label phase I/II trial, leveraging historical outcomes data , and safety data from a single-arm, open-label (observational) expanded access study. ,

After treatments are approved initially for adults, although all use in children is by definition off-label, RWD can be used to produce much-needed evidence on whether these treatments are safe and effective in children. Where the course and outcomes of disease are sufficiently similar in adults and children, regulatory agencies may conclude that pediatric effectiveness can be extrapolated from well-conducted studies in adults. In such cases, regulatory agencies may combine efficacy data from adults with pediatric pharmacokinetic and safety data for pediatric approval. For example, mepolizumab was approved down to age 6 years as add-on maintenance treatment of patients with severe asthma and an eosinophilic phenotype, based on pharmacokinetic and safety data in 6- to 11-year-olds and extrapolated efficacy data from RCTs in adolescents and adults. In this case, there was overlap in the clinical presentation of both adult and pediatric severe eosinophilic asthma, consistency in the therapeutic approach, consistency of the mepolizumab mechanism of action, and relevance of the clinical endpoints. When uncertainty remains regarding the extent to which adult data can reasonably apply to children, RWD represent a valuable vehicle for supplying critical missing evidence about treatment effectiveness and safety in routine pediatric care.

Even when pediatric RCTs are feasible and ultimately performed, these trials are often limited to narrow populations, short-term exposures (eg, to identify suitable pediatric doses based on pharmacokinetics), and a limited set of outcomes. Such trials leave many unanswered questions about more diverse populations, chronic exposures, and unexamined outcomes, whether delayed, rare, or simply not addressed. Unique safety concerns for children, such as growth and neurodevelopment, may not be addressed fully or sufficiently by RCTs. Careful analyses of RWD and judicious appraisal of RWE provide opportunities to fill in the often large evidentiary gaps in knowledge and examine the uses and effects of treatments (including long-term benefits and risks) in much larger, heterogeneous, and generalizable populations. For instance, a medication approved to treat children with attention deficit/hyperactivity disorder may be shown to significantly improve children's attention and behavior in an RCT setting, but how well does this medication work in underserved, less heavily supervised, or less adherent children? Does the medication affect future scholastic performance? Is it effective and safe when combined with antidepressants, antipsychotics, or hypnotics, or used by children with autism, or taken at doses not tested in the trials? Countless questions that RCTs are either underpowered or not designed to answer can be addressed by the sound analysis of RWD and thoughtful appraisal of RWE.

Challenges

Although RWD holds great promise, the elephant in the room is its validity: can we really trust pediatric RWD to generate valid answers/evidence to our questions? The answer is, yes, sometimes, and it depends. Just as RCTs may be flawed and biased (eg, due to differential dropout across treatment groups), pediatric observational research faces a considerable set of unique potential biases and other limitations that must be appropriately recognized and reckoned with if RWE is to be effectively leveraged for regulatory and clinical purposes. Publications using RWE, including from pediatric populations, vary substantially in quality, and their findings must be viewed critically based on their methods. Fortunately, many common limitations are addressable (Table).

Without the powerful benefits of randomization, overcoming bias from confounding represents a major hurdle for observational pediatric research. Adjustment for measured confounders such as age and comorbidities is essential. Nonetheless, unmeasured confounding may arise when key analytic variables are missing, such as gestational age, date of birth (birth year is suitable for studies of adults but not neonates/young children), weight and height measurements, family history, or measures of disease activity/severity. EHRs may more reliably include these data than claims databases, but many EHRs generally do not originate from closed health care systems and may not capture all treatments and outcomes of interest. Furthermore, EHRs typically record prescribing, which is a giant step farther from a child's mouth than the claims-based dispensing data. These omissions in EHR data could be important sources of selection or ascertainment bias. When possible and done properly, linking claims and EHR data allows pediatric researchers to bridge the gaps and produce more valid results. Linkage between children's and their parents' records can facilitate ascertainment of useful covariates, such as prenatal exposures, perinatal events, and family history. Metrics of health care utilization (eg, hospitalization, number of outpatient visits) can also help reduce bias from unmeasured confounders.

Doctors treat patients for a reason, and children who receive certain treatments may be fundamentally different from children who receive other treatments or none at all. To overcome confounding by indication, disease severity, and other factors in observational research, various designs may help reduce bias, including active-comparator designs, self-controlled designs, sibling-controlled designs, or instrumental variable designs (Table). In a pinch, a little randomization can go a long way: with pragmatic clinical trials, broad inclusion criteria and fit-for-purpose RWD collection can produce valid RWE on treatment effectiveness or safety. As noted in the Framework for FDA's RWE Program, the agency will explore pragmatic approaches and strategies for trials that generate RWE in some capacity (eg, pragmatic randomized trials integrated into health care systems).

Pediatric research using RWE presents additional unique challenges relating to sample size requirements, evaluation of dose-effects, and lack of available, standardized, or validated outcomes (Table). For example, because of the fragmented nature of US health care delivery, including frequent changes in insurers and settings of care and pediatric-to-adult transitions of care, pediatric RWD may be limited in duration, greatly restricting one's ability to study long-term outcomes. Underinsured or uninsured children at particularly high risk for adverse outcomes may not show up in insurance-based datasets and may be overlooked in RWE. Population-representative pediatric data (eg, from Europe) may, nonetheless, have too small populations to study rare outcomes or lack data on populations of interest (eg, hospitalized neonates).

Ultimately, successful use of RWE to improve outcomes and to support regulatory decision-making in pediatrics requires skilled understanding to ask appropriate research questions, access to fit-for-purpose data, infrastructure for managing and analyzing the data, and resources (including funding) to conduct and disseminate reproducible and transparent research.38, 39, 40 Increasingly, such successes rely on collaborative, multidisciplinary teams with relevant expertise and on robust engagement of broad stakeholders, including regulators and patients/caregivers. Integration of patients and caregivers into research teams not only provides a valuable source of RWD but can also enhance the relevance, quality, and impact of research.

Rises in pediatric medication use, overall and off-label, as well as increasing availability of RWD, provide valuable and novel opportunities to harness RWE and improve children's health. Given the various challenges, RWE is no substitute for RCTs, and vice versa: they are both essential, complementary, sometimes intertwined approaches. Our children deserve nothing less.