Using Decision Analysis to Support Newborn Screening Policy Decisions: A Case Study for Pompe Disease.

Background: Newborn screening is a public health program to identify conditions associated with significant morbidity or mortality that benefit from early intervention. Policy decisions about which conditions to include in newborn screening are complex because data regarding epidemiology and outcomes of early identification are often incomplete.
Objectives: To describe expected outcomes of Pompe disease newborn screening and how a decision analysis informed recommendations by a federal advisory committee.
Methods: We developed a decision tree to compare Pompe disease newborn screening with clinical identification of Pompe disease in the absence of screening. Cases of Pompe disease were classified into three types: classic infantile-onset disease with cardiomyopathy, nonclassic infantile-onset disease, and late-onset disease. Screening results and 36-month health outcomes were projected for classic and nonclassic infantile-onset cases. Input parameters were based on published and unpublished data supplemented by expert opinion.
Results: We estimated that screening 4 million babies born each year in the United States would detect 40 cases (range: 13-56) of infantile-onset Pompe disease compared with 36 cases (range: 13-56) detected clinically without screening. Newborn screening would also identify 94 cases of late-onset Pompe disease that might not become symptomatic for decades. By 36 months, newborn screening would avert 13 deaths (range: 8-19) and decrease the number of individuals requiring mechanical ventilation by 26 (range: 20-28). Conclusions: Pompe disease is a rare condition, but early identification can improve health outcomes. Decision analytic modeling provided a quantitative data synthesis that informed the recommendation of Pompe disease newborn screening.

Newborn screening is a public health program for the identification of serious but usually rare conditions that benefit from early detection followed by interventions that can reduce morbidity and mortality.[1] Although individual states choose which conditions are included in newborn screening, the Secretary of the US Department of Health and Human Services recommends a panel of conditions for which there is recognized benefit for screening. Conditions are added to the panel, known as the Recommended Uniform Screening Panel (RUSP), based primarily on recommendations to the Secretary from the Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC).[2,3] The ACHDNC evaluates the benefits and harms of screening for a candidate condition compared with usual clinical care by evaluating a systematic evidence review conducted by an external condition review workgroup (CRW), which is supported by a contract from the Health Resources and Services Administration.[4] Given the rare nature of conditions considered for newborn screening, the evidence base is often sparse.[5,6]

Since 2011, decision analysis has been incorporated into the evidence review process used by the CRW. Decision analytic modeling is a systematic approach to quantify potential outcomes for decisions with uncertainty.[7] Through the estimation of expected outcomes for alternate options, it allows the decision maker to identify which alternative is expected to yield the most health benefit. It also allows researchers to characterize the uncertainty associated with projections of outcomes and highlight evidence gaps, thereby enhancing the overall decision making process.[8] This article describes the decision analytic modeling used to assess Pompe disease newborn screening for the ACHDNC, illustrating how this approach can be used to synthesize data and inform policy decisions.

Pompe Disease Overview

Pompe disease is a rare genetic disorder caused by mutations in the glucosidase alpha acid (GAA) gene, leading to low levels of a specific enzyme, acid alpha-glucosidase (GAA).[9] GAA is one of many enzymes that degrades cellular glycogen within lysosomes. Accumulation of glycogen can lead to irreversible damage to the heart, skeletal muscle, and the lungs. There are more than 200 known mutations of the GAA gene, which vary in the amount of GAA produced and its catalytic capacity. The course of the disease (i.e., phenotype) can vary widely based on the amount and activity of the GAA enzyme that is produced.[10,11]

Patients with a mutation on both GAA alleles that prevent the production of functioning GAA have a uniformly severe phenotype referred to as the classic infantile form, which in the absence of targeted therapy leads to progressive weakness and cardiomyopathy with death in early infancy. Aggressive supportive care, including mechanical ventilation, does not significantly extend survival among those with classic infantile Pompe disease.[12] In contrast, some patients have a variant of the GAA gene that allows for low levels of functioning GAA activity, leading to the nonclassic infantile form, which without targeted therapy is characterized by progressive muscle weakness and respiratory failure without cardiomyopathy with death later in childhood. In addition to infantile-onset forms, there is also a late-onset form of Pompe disease. Some individuals may not develop significant weakness for decades, but there is a wide spectrum of disease severity, timing of onset, and which organ systems are affected.[9,13] Although most present with symptoms in adulthood, some become symptomatic in early childhood.

In 2006, the Food and Drug Administration approved a targeted enzyme-replacement therapy (ERT), alglucosidase alfa, for Pompe disease, which is available in two preparations (Myozyme and Lumizyme, Genzyme Corp). ERT does not cure the underlying disorder or reverse the damage caused by accumulated glycogen. However, it can provide sufficient enzyme to cells to degrade glycogen accumulated in cellular lysosomes.[14] ERT is provided by infusion, typically given weekly or biweekly over the life of the affected individual. One challenge to the use of ERT is that some individuals develop neutralizing antibodies. The immune system of individuals who make no endogenous enzyme, referred to as being cross-reactive immunologic material (CRIM)-negative, can recognize the ERT as being foreign and mount an antibody response that cancels out the benefit of the therapy and can also lead to a serious allergic response. Immunologic modulation therapy can overcome this problem, but it also complicates therapy.[9]

Individuals with Pompe disease are asymptomatic at birth, but can be identified based on low GAA enzyme activity in the dried-blood spots used for newborn screening. Although the phenotype cannot be directly predicted based on the enzyme activity level, sequencing the gene can help predict the course because certain mutations are associated with specific forms.[9,13] Those with the infantile form will have findings in the newborn period that can be determined through diagnostic testing after the low enzyme activity level is confirmed (e.g., cardiomyopathy identified by echocardiogram, muscle damage identified by biopsy).[15] ERT is recommended to be started as soon as the infantile form, either with or without cardiomyopathy, is confirmed. For those predicted to have the late-onset form, ERT is recommended once signs or symptoms develops. Studies are underway to further define the optimal time to begin ERT in those with late-onset disease.

In 2008, the ACHDNC evaluated Pompe disease newborn screening. At that time, there were significant evidence gaps related to the accuracy of screening and to the benefit of presymptomatic diagnosis, which precluded its recommendation for the RUSP.[16] In 2013, the ACHDNC reconsidered Pompe disease after it was nominated again. Based in part on new information presented to the ACHDNC by the CRW, Pompe disease newborn screening was recommended to the Secretary for addition to the RUSP[17] and was added in March 2015.[18]

Although Pompe disease is rare, sufficient data became available before the ACHDNC reconsideration in 2013 to model the impact of Pompe disease newborn screening, including the projected number of averted deaths and cases of ventilator dependence compared with usual clinical case detection. The application of this model provides insight into how policy recommendations can be made for newborn screening in the absence of randomized trials that would provide higher quality evidence but are not feasible to conduct.

Methods

Development of the Decision Analytic Model

Based on the systematic evidence review, the CRW identified a list of key outcomes specific to the progression of Pompe disease and developed an initial decision analytic model to project outcomes for screening compared with clinical identification. An expert panel was convened that included clinical and scientific experts in Pompe disease who were identified through the systematic evidence review process. Expert panel members were asked to provide input on the structure of the decision analysis model, including the identification of key health outcomes to be included in the analysis.[17]

All meetings were conducted via webinar. Expert panel participants received a discussion guide prior to each meeting. The discussion guide was developed using established methods for obtaining estimates from experts.[19] The discussion guide included background information, a schematic of the draft model, key questions for the expert panel, proposed data inputs, and data sources for review. After each expert panel meeting, the model structure and inputs were revised based on expert feedback. This process utilized a modified Delphi or “Decision Delphi” model for expert panel input.[20]

A series of three expert panel meetings were conducted to identify key health outcomes and data sources and derive probabilities for each outcome in the model; to provide feedback on the structure of the initial and revised decision analytic model, including the relevant timeframe for key health outcomes; and to develop assumptions where little or no data were available. The same set of six experts was invited to attend all three panel meetings. If any expert was not able to participate in a panel, they were invited to contribute via email or separate telephone meeting. During the second and third expert panel meetings, the proposed set of parameter inputs for the decision model was reviewed. These parameter estimates were revised following each expert panel meeting based on new data sources identified during the previous expert panel and ranges identified to reflect expert opinion in cases where no data were available. Ranges for parameter estimates were approved by the expert panel. The timeline for the decision analysis is shown in Table 1.

Table 1

Timeline for Decision Analysis.

Date	Decision Analysis Milestones
2012	Pompe disease nominated for addition to uniform newborn screening panel; referred to external condition review group
Fall 2012	Initial development of decision analytic model to evaluate newborn screening for Pompe disease
December 2012	Technical Expert Panel 3: Review Model Structure
January 2013	Technical Expert Panel 4: Review Revised Model Structure & Assumptions
April 2013	Technical Expert Panel 5: Review Model Inputs
May 2013	Final Pompe disease evidence review report and decision analysis findings presented to Advisory Committee

Model Structure

The strategies compared in the model were diagnosis through newborn screening versus through clinical identification. It was assumed that diagnosis is followed by treatment initiation for early infantile Pompe disease, regardless of diagnosis method. The final version of the simulation model had two submodels, one for each strategy (Figure 1). Two key health outcomes were modeled: ventilator dependence and death due to Pompe disease. Pompe disease was classified into one of three forms: classic infantile-onset (<12 months), nonclassic infantile-onset (<12 months), or late-onset (≥12 months).[9] The model also tracked screening outcomes: positive screens, confirmed diagnoses, false positives, true negatives, and false negatives. The target population was the US newborn cohort of 4 million newborns annually, not otherwise at high risk for Pompe disease based on family history.

Figure 1

Simplified schematic for Pompe disease model.

aNo known increased risk for Pompe disease.

bLow/“absent” GAA enzyme.

cVia DNA sequencing, or referral to specialist.

dRepeat screen on a new blood spot (Screen 2).

eAssumed that some proportion of Pompe disease cases would not be detected under clinical identification.

fAssumed to late-onset cases only.

gSurvival outcomes further categorized as either “ventilator free” or “ventilator dependent.”

Parameter Inputs and Data Sources

The identification of data sources and the development of a decision analytic model is typically an iterative process. In the first expert panel, ventilator dependence was added as a health outcome. Although the first expert panel suggested that CRIM status be considered, subsequent expert panels simplified the need to include CRIM status with the assumption that CRIM-negative patients would receive immunomodulation therapy so that their major health outcomes would be the same as those who are CRIM-positive.

The final parameter inputs and associated ranges are shown in Tables 2 and 3. Initial parameter estimates were derived using published and unpublished data, and these estimates were subsequently revised based on expert input, as described above. New data were obtained from the Pompe disease registry following the third expert panel immediately prior to the ACHDNC meeting, and these data were also incorporated into the parameter ranges reported below. Each parameter was defined using a most likely value and a range for sensitivity analyses. Table 3 also presents the risk of mortality and the likelihood of ventilator-free survival for clinically identified cases not treated with ERT. These estimates apply to cases that would be missed by screening or clinical identification.

Table 2

Probability Inputs, Pompe Disease Prevalence, and Forms[a]

Pompe	Universal Newborn Screening			Clinical Identification
	Most Likely	Range (Min-Max)	Sources	Most Likely	Range (Min-Max)	Sources
Pompe disease (all forms)	1/27,800	0.3–2.7/27,800	Scott and others (2013)[23]	1/27,800[b]	0.3/27,800–2.7/27,800	Scott and others (2013)[23]; Expert opinion
Infantile (<12 months)	0.278	0.132–0.424	Chiang and others (2012)[22]; Mechtler and others (2012)[34]; Scott and others (2013)[23]	0.250	0.109–0.391	Chiang and others (2012)[22]; Mechtler and others (2012)[34]; Scott and others (2013)[23]; Expert opinion[c]
Classic infantile-onset	0.236 (0.85)[d]	0.195–0.250 (0.70–0.90)	Expert opinion	0.235 (0.94)	0.2–0.250 (0.80–1.00)	Expert opinion; Kishnani and others (2006)[12]
Nonclassic infantile-onset	0.042 (0.15)	0.028–0.083 (0.10–0.30)	Expert opinion	0.015 (0.06)	0–0.05 (0.0–0.20)	Expert opinion; Kishnani and others (2006)[12]
Late-onset (≥12 months)	0.722	0.576–0.868	Chiang and others (2012)[22]; Mechtler and others (2012)[34]; Scott and others (2013)[23]	0.7502	0.609–0.891	Chiang and others(2012)[22]; Mechtler and others (2012)[34]; Scott and others (2013)[23]; Expert opinion
Sensitivity[e]	0.9322	0.9315–0.9329	Chiang and others (2012)[22]; Expert opinion
Specificity[e]	0.99997	0.9993–1.0000	Chiang and others (2012)[22]; Expert opinion

95% confidence interval derived using a binomial distribution.

This number does not represent “clinical prevalence” for infantile onset Pompe disease. It is assumed that under clinical detection that some proportion of the late-onset cases identified under newborn screening might never be diagnosed. There is very scant data on this parameter. The assumption is that clinical prevalence may fall somewhere between 1/40,000 and 1/100,000, which would imply that as many as 40% to 70% of cases would be detected later during the lifespan (i.e., later than the first year of life) or may go undiagnosed.

Adjusted to assume that only two out of six cases of classic infantile-onset would be detected under clinical identification compared with newborn screening based on expert opinion and the proportion of cases identified with and without cardiomyopathy from a retrospective cohort study.

Numbers in parentheses are conditional probabilities given that a newborn has the infantile form of the condition.

Sensitivity is included in the model through two parameters: probability of a positive screen 0.0000655 (range: 0.000034–0.0000734) and probability of confirmed Pompe following a positive screen (0.512). Specificity is included as the probability of Pompe following a negative screen: 0.00000244.

Table 3

Probabilities of Health Outcomes for Three Populations, Infantile-Onset Cases Only (<12 Months): Screened and Treated, Clinically Diagnosed and Treated, Clinically Diagnosed and Untreateda

(a) Mortality.
Pompe	Mortality 24 Months			Mortality 36 Months
	Most Likely	Range (Min-Max)	Sources	Most Likely	Range (Min-Max)	Sources
Screened, treated immediately
Infantile, with cardiomyopathy	0	0–0.029	Chen and others (2009)[35,b]	0	0–0.029	Chen and others (2009)[35,b]; Personal communication[d]
Infantile, without cardiomyopathy	0	0–0.029	Assumption[c]	0	0–0.029	Assumption[c]
Clinically diagnosed, treated
Infantile, with cardiomyopathy	0.258	0.182–0.334	Primary data[f]	0.351	0.267–0.439	Primary data[f]
Infantile, without cardiomyopathy	0.012	0.001–0.111	Derived[g]	0.080	0.020–0.208	Derived[g]
Clinically diagnosed, untreated
Infantile, with cardiomyopathy	0.928	0.882–0.960	Chen and others (2009)[35,b]; Kishnani and others (2006)[12]	0.979	0.959–0.999	Chen and others (2009)[35,b]; Kishnani and others (2006)[12]
Infantile, without cardiomyopathy	0.203	0.093–0.364	Hopkins and others (2005)[27]	0.289	0.161–0.468	Hopkins and others (2005)[27]
(b) Ventilator-Free Survival.
Pompe	Ventilator-Free Survival, 24 Months			Ventilator-Free Survival, 36 Months
	Most Likely	Range (Min-Max)	Sources	Most Likely	Range (Min-Max)	Sources
Screened, treated immediately
Infantile, with cardiomyopathy	1	0.971–1	Chen and others (2009)[35,b]	1	0.971–1	Chen and others (2009)[35,b]; Personal communication[d]
Infantile, without cardiomyopathy	1	0.971–1	Assumption[c]	1	0.971–1	Assumption[c]
Clinically diagnosed, treated[e]
Infantile, with cardiomyopathy	0.686	0.476–0.694	Primary data[f]	0.590	0.467–0.694	Primary data[f]
Infantile, without cardiomyopathy	0.875	0.710–0.965	Derived[g]	0.843	0.672–0.947	Derived[g]
Clinically diagnosed, untreated
Infantile, with cardiomyopathy	0.046	0–0.159	Chen and others (2009)[35,b]; Kishnani and others (2006)[12]	0.010	0–0.024	Chen and others (2009)[35,b]; Kishnani and others (2006)[12]
Infantile, without cardiomyopathy	0.667	0.499–0.814	Derived[h]	0.524	0.347–0.681	Derived[h]

Minimum and maximum values derived from 95% confidence intervals assuming a binomial distribution

The article describing treatment effectiveness does not specify whether cases of infantile-onset include cardiomyopathy; however, a separate study[10] published contemporaneously delineates which infants had confirmed hypertrophic cardiomyopathy. These are the infants whose results are reported as the NBS subgroup from which we derive effects of treatment in the Screened/Treated arm. Of the clinically identified patients, it is unclear which had cardiomyopathy. Our assumption is that all or most of these patients had cardiomyopathy.

Assumes similar effects of treatment for infantile-onset cases with cardiomyopathy as derived from Chen and others (2009).[35]

Data on 36-month outcomes for patients reported in the Chen and others (2009)[35] study were communicated to the condition review working group via telephone interview with researchers from Taiwan.

Clinically diagnosed cases are identified and treated several months after birth (on average between 4 and 5 months of age).

Effectiveness of treatment in the clinically diagnosed/treated population was derived from the Genzyme Pompe Registry and provided to the CRW via personal communication from Joan Keutzer (30 April 2013). This panel excludes patients from Taiwan, some of whom would have been detected by newborn screening and would have higher rates of survival and ventilation-free survival.

Effectiveness of treatment for the subgroup of infantile-onset without cardiomyopathy is based on results from Kishnani and others (2006)[12] and assuming that efficacy is similar to that observed for individuals with infantile-onset with cardiomyopathy.

Assumes same proportion between alive and vent-free conditional on being alive for infants with and without cardiomyopathy.

Modeling Assumptions

The analysis assumed that identified cases of infantile-onset Pompe disease would be treated with ERT whether diagnosed through newborn screening or through clinical identification.

Additional assumptions were made in consultation with the expert panel participants. It was assumed that all cases of classic infantile-onset Pompe disease would be detected by newborn screening or clinical identification within the first 12 months of life. However, clinical identification would lag identification by newborn screening. Since newborns identified in both submodels receive ERT, the difference in outcomes related to the difference in the timing of the initiation of treatment, at approximately 22 days of life with newborn screening, compared with 4 to 5 months of life with clinical identification.[21] In this study, earlier initiation of ERT was assumed to be associated with the elimination of mortality before 36 months and also with many fewer patients requiring ventilator assistance.

We assumed the prevalence of Pompe disease to be 1 in 27,800 births and that 23.6% of affected individuals would have the classical infantile onset, 4.2% would have the nonclassical infantile-onset form, and 72.2% would have late-onset disease. Without newborn screening, all infantile-onset cases would be detected after the presentation of catastrophic symptoms; however, without newborn screening the majority of the late-onset cases would never be diagnosed. The modeling results represent the benefits of earlier identification, diagnosis, and initiation of treatment for classic infantile-onset Pompe disease associated with newborn screening.

We assumed that nonclassic infantile-onset Pompe disease is less severe than the classic form (Table 3). Some cases of nonclassic infantile-onset Pompe disease would be missed by clinical identification within the first 12 months of life. For this form, the modeling results reflect the health benefits of identifying and treating nonclassic infantile-onset cases that would be identified at or close to birth by newborn screening compared with later by clinical identification. The timing of non–infantile-onset cases for newborn screening compared with clinical identification is unknown. Non–infantile-onset cases are not included in health outcomes predicted by the decision analytic model, but the size of this cohort is quantified.

Test characteristics for screening were based on data from Taiwan and an analysis of anonymized blood spots from Washington State.[22,23] All false negative screens were assumed to be late-onset cases.

Analysis

The time horizon for the analysis was 36 months. Identical hypothetical cohorts of newborns were simulated through both arms of the model to compare outcomes. Using the most likely values and the range for each parameter, we projected most likely values and ranges for all the screening and health outcomes. Ranges were derived by conducting one-way sensitivity analyses on all parameters in Table 1.

Results

Using a decision analytic model, newborn screening for Pompe disease was projected to provide health benefits as measured by averted deaths and averted cases of ventilator-dependence when compared with clinical identification followed by treatment.

Screening Outcomes

For a US newborn cohort of 4 million newborns, projected screening outcomes are shown in Table 4. The analysis predicted 262 positive screens including 134 true positives and 128 false positive screens. Ten false negatives were also predicted. This reflects a baseline prevalence of 1/27,80017 and includes both infantile and late-onset cases (Table 4).

Table 4

Projected Screening Algorithm Outcomes for Newborn Screening for Pompe Disease for a Cohort of 4 Million Children (US Population).

	Newborn Screening (n)[a]	Range[b]
Total positive screens	262	134–2,934
True positives[c]	134	[d]
False positives[e]	128	0–2,800
Total negative screens	3,999,738	3,997,066–3,999,866
True negatives	3,999,728	3,997,056–3,999,856
False negatives	10	[d]
Repeat screens[f]	147	75–1,646[g]

FP, false positive; GAA, acid alpha-glucosidase; NAG, neutral alpha-glucosidase.

Base case test characteristic values for sensitivity (0.9322) and specificity (0.99997) were derived from Chiang and others (2012),[22] and applied to the US population prevalence of Pompe disease.

Ranges for sensitivity (0.9315–0.9329) and specificity (0.9993–1.0000) were derived from Chiang and others (2012).[22]

Includes all subtypes.

Varying test characteristics resulted in very small changes for true positives and false negative cases, but not reported here due to rounding.

False positive rates were calculated based on definition (1) of Table 3 in the Results section of this report (i.e., FP rate of Inconclusive [NAG/GAA ≥ 60] or Abnormal [NAG/GAA ≥ 100] first dried-blood spot screen).

Repeat screens are defined as an inconclusive first dried-blood spot screen (NAG/GAA ≥ 60), as described in Table 3 of this report.

This range assumes the same proportion of Inconclusive to Abnormal initial screens as the base case value.

Projected Cases of Infantile-Onset Pompe Disease

We projected the annual number of infantile-onset cases that would be identified with newborn screening compared with clinical identification (Table 5). The total annual number of all types of Pompe projected was 134 cases, including 40 infantile-onset cases (range: 19–61) by newborn screening and 36 infantile-onset cases (range: 16–56) by clinical identification. Within the group of infantile-onset cases, 34 were projected to be the classic form for both newborn screening and clinical identification. Six nonclassic infantile-onset cases would be detected by newborn screening, but 4 of these cases (67%) would be detected after the first year of life under clinical identification. Sensitivity analyses showed that the results could range as high as 61 or as low as 19 for the total number of infantile-onset cases identified under newborn screening, compared with 56 to 16 with clinical identification.

Table 5

Projected Cases for Newborn Screening for Pompe Disease Compared With Clinical Identification for a Cohort of 4 Million Children[a] (US Population), Infantile-Onset Only[b]

	Newborn Screening	Clinical Detection
Infantile onset (<12 months), number of cases	40 (19–61)	36 (16–56)
With cardiomyopathy	34 (28–36)	34 (28–36)
Without cardiomyopathy	6 (4–12)	2 (0–8)

Not at higher risk for Pompe disease.

Ranges represent one-way sensitivity analysis on each parameter.

Averted Deaths and Ventilator-Dependent Cases Among Infantile-Onset Cases

By 36 months of life, we projected that identifying 40 infantile-onset cases of Pompe disease through newborn screening would avert 13 deaths (range: 9–19) and 26 cases of ventilator dependence among survivors at 36 months (range: 20–28; Table 6).

Table 6

Projected Health Outcomes for Newborn Screening for Pompe Disease Compared With Clinical Identification for a Cohort of 4 Million Children (US Population), Infantile-Onset Cases Only

	Newborn Screening	Clinical Detection	Cases Averted
Key health outcomes, for infantile onset cases only[a]
Projected deaths, 36 months	0 (0–1)	13[b] (9–19)	13 (8–19)
Projected survivors ventilator-free, 36 months	40 (39–40)	14 (12–19)	26 (20–28)

Classic and nonclassic infantile-onset.

Includes 12 deaths associated with classic infantile-onset and one death associated with nonclassic infantile-onset Pompe disease.

In the absence of screening, the projected number of deaths at 36 months was 13 (9–19) for clinical detection compared with 0 to 1 deaths for infantile-onset cases identified by newborn screening. The projected number of ventilator-free survivors at 36 months among infantile-onset cases by newborn screening was 39 to 40 compared with 12 to 19 with clinical detection (Table 6).

Discussion

Using decision analytic modeling, we were able to project the relative benefits of newborn screening for Pompe disease compared with usual case detection. This illustrates how even in the absence of complete data, inferences can be made to assist with a policy decision. We were able to describe the likely number of infantile-onset Pompe disease cases that would be identified each year as well as associated health benefits defined as deaths averted and cases of ventilator dependence avoided. We were also able to quantify the number of possible late-onset cases that would be identified through newborn screening. Given the heterogeneity of this form, early detection of late onset might be considered either a harm or a benefit. Individuals might not develop health problems for decades, but have to contend with labeling and uncertainty, while some individuals could benefit from earlier diagnosis and treatment, and avoidance of diagnostic odysseys. This was a point of discussion during the consideration of Pompe disease for addition to the RUSP.

Given the evidence base for this rare condition, it was challenging to apply the decision analysis framework in some areas. For example, initial discussions with experts included discussion of modeling a longer time horizon than 36 months, but this was not feasible due to the lack of longer-term evidence. There was also discussion of including other markers of disease progression other than ventilator dependence, but insufficient data were available regarding other important health outcomes. In the context of types of Delphi approaches used to inform the decision analysis model, we utilized both a “classical” and “decision” Delphi[20] to identify model structure, care pathways, and key markers for disease progression.

Lack of data also required the use of expert opinion and assumptions to develop ranges for sensitivity analyses. Given the scant evidence base associated with many rare conditions, this is likely to be the case for many conditions considered for addition to the uniform screening panel. Sensitivity and scenario analyses should be conducted to explore the robustness of the results. As is the case for most decision analysis models, some parameters will be less well-supported by the evidence and a wider range of possible parameter values should be considered. Another issue is whether complete ascertainment of unscreened cohorts and their outcomes can be assumed, and this issue cannot necessarily be resolved addressed with expert opinion. The goal is to inform decisions with the best evidence available at the time and to conduct sensitivity analyses across a range of plausible parameter values (here as determined by expert panel) when evidence is limited.

One of the goals of the CRW in developing decision models is to ensure transparency in the decision modeling process. Consistent with this goal, the CRW includes two liaison members from the ACHDNC who are invited to attend all meetings of the CRW as well as all expert panel meetings. This provides an opportunity for ACHDNC input and to ensure that their questions are addressed in the presentation and report to the ACHDNC. Another tenet for accomplishing our goal of transparency is to aim for simple models, to avoid the “black box” perception of simulation models and foster a clear understanding of the modeling analysis for ACHDNC members.

Using a simpler modeling approach also comes with drawbacks. One-way sensitivity analyses were conducted on all variables to develop the ranges for the results presented in Tables 4 and 5. These ranges present the upper and lower bounds across all of the univariate analyses and as such, present a “best case”/“worst case” set of scenarios. We elected not to conduct a probabilistic assessment given the evidence base and the objective to use a simplified modeling approach.

We did not project long-term outcomes for non–infantile-onset cases of Pompe disease. During the scoping process for the decision model, we chose to focus on outcomes for early infantile cases, consistent with the objective of the newborn screening program to avert deaths and severe sequelae in childhood. Using the modeling results we were able to characterize the estimated number of cases of all forms of Pompe disease that newborn screening would detect. Similar to other newborn-screened conditions, the observed incidence based on newborn screening is higher than observed with clinical identification only.[23] The number of possible late-onset cases projected annually under newborn screening was 94 cases, of which many go undetected by clinical identification. Some cases of the infantile form might be missed clinically if the diagnosis is not established before death.

Since this analysis was completed, additional follow-up on screened and treated patients in Taiwan confirms the benefits of earlier detection and treatment for newborns with infantile-onset Pompe disease.[24,25] For screening outcomes, pooled data from New York state on infants screened during 1 October 2014 to 25 May 2016, from 6 months of screening in Missouri, and from almost 4 months of screening in Illinois, suggests the incidence of infantile-onset cases detected through newborn screening is toward the upper end of the range projected in this analysis.[26-28] The number of probable and possible late-onset cases of Pompe disease from these three states suggests the true incidence of late-onset Pompe disease could be as much as five times higher than was modeled. That is an indication of the structural uncertainty that is inherent in modeling the occurrence of late-onset cases of a spectrum disorder in the absence of years of actual screening data from multiple jurisdictions.

An important limitation is the absence of modeling of the harms of newborn screening. Of particular relevance to late-onset Pompe disease is “psychosocial harm from diagnostic or prognostic uncertainty in diagnosis, or degree or age of onset of disease manifestations.”[29] In this situation, we were able to quantify the likely magnitude of the number of cases that would fall into this category of prognostic uncertainty. The group of newborns identified with possible or probable late-onset Pompe was a topic of considerable discussion for the ACHDNC. Since adding Pompe disease into the uniform screening panel, a Pompe Disease Newborn Screening Working Group has begun to recommend guidelines to inform decision-making challenges in follow-up care for newborns identified with late-onset Pompe disease.[14] The consideration of potential harms and costs resulting from screening is anticipated to be more formally incorporated into future condition reviews.[29] In addition, we considered treatment would be available to all identified patients. If there were barriers to access for enzyme replacement treatment or some parents decided not to pursue treatment for their child, the estimated health benefits of screening and earlier treatment would be lower than reported here. The decision analysis reports the number of expected false positive screens. While there is anecdotal evidence of parental concerns, previous research on the harms associated with a transient false positive result suggest there may be a short-term increase in parental anxiety but have not documented long-term harms.[29-31]

Data from pilot screening programs in other countries may often represent the best data available but these data should be evaluated for their generalizability to a US population. For some conditions including Pompe, it is important to consider the racial/ethnic composition of the population and how that might relate to screening outcomes. For example, data from Taiwan on pilot screening for Pompe is anticipated to include a higher number of false positives because pseudodeficiency is more common in Asian populations. The model relied on US data for estimation of the number of false positives to avoid any potential bias.

We did not model costs or cost-effectiveness of newborn screening. The scope of the decision modeling for the condition review process was limited to estimation of population health outcomes, consistent with the evidence criteria that were used by the ACHDNC.[2,4] Although any evidence on costs or cost-effectiveness that is identified as part of the evidence review is evaluated and presented to the ACHDNC, published evidence is not typically available.[5] The Newborn Screening Saves Lives Reauthorization Act of 2014 for the first time directed the ACHDNC to consider “cost” along with “public health impact” in recommending new conditions. Subsequently, a Cost Analysis Work Group was commissioned by HRSA to develop methods to project the cost of expanding newborn screening for additional conditions. The authorizing legislation also set a constraint of 9 months for the completion of evidence reviews, including the public health system and cost assessment and decision modeling. Given the time and resource constraints, it was determined that it would only be feasible to assess in most cases the direct cost to health departments of adding a condition to the screening panel. In particular, a formal cost-effectiveness analysis is not feasible with the 9-month time constraint.

We recognize that the lack of a formal cost-effectiveness analysis is a limitation. The availability of cost-effectiveness evidence could provide important information on the tradeoffs between costs, health benefits, and harms that may not be fully captured by the current review process. However, we also recognize that doing so would require dedicated resources and a considerably longer time frame than is allowed in the current process. Because of the constraints of data and time, cost-effectiveness analyses of conditions previously added to the RUSP have been conducted retrospectively.[32,33] Some of the authors have undertaken cost-effectiveness modeling of newborn screening for Pompe disease and other conditions through a multiple-year process with separate funding, independent of HRSA and the ACHDNC. The results of those additional analyses are not yet available.

The evidence review process used to inform ACHDNC decisions explicitly includes decision analytic modeling.[5] While other advisory committees may include the results of previously conducted modeling studies as part of an evidence review, it is novel to incorporate a modeling analysis directly into the review process. Moreover, the updated evidence review process for the ACHDNC recognizes the value of allowing for an inclusive evidence review process for newborn screening in order to incorporate data that might be considered “lower quality” by other standards and to incorporate these data into the decision modeling analysis. This is highly relevant to rare conditions, or diseases for which treatments are new or emerging.

In summary, the incorporation of decision modeling into the condition review process confirmed that most of the health benefits from screening for Pompe disease are experienced by patients identified with classic infantile-onset Pompe disease. There would also likely be health benefits for a small number of children with the infantile-onset form without cardiomyopathy that would be identified and treated much earlier under a newborn screening program compared with clinical identification only. The modeling analysis also provided quantitative estimates of the number of possible late-onset cases that would likely be identified under a newborn screening program. This is a group for which there could be harms as well as benefits with the initiation of a newborn screening program. The decision modeling analysis allowed for the estimation of population-level health outcomes for a proposed screening program and has now been incorporated into the review of five conditions that have been evaluated by the ACHDNC.