Newborn screening timeliness quality improvement initiative: Impact of national recommendations and data repository.

BACKGROUND: Newborn screening (NBS) aims to achieve early identification and treatment of affected infants prior to onset of symptoms. The timely completion of each step (i.e., specimen collection, transport, testing, result reporting), is critical for early diagnosis. Goals developed by the Secretary of Health and Human Services' Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC) for NBS timeliness were adopted (time-critical results reported by five days of life, and non-time-critical results reported by day seven), and implemented into a multi-year quality improvement initiative (NewSTEPS 360) aimed to decrease the time to result reporting and intervention.
METHODS: The NBS system from specimen collection through reporting of results was assessed (bloodspot specimen collection, specimen shipping, sample testing, and result reporting). Annual data from 25 participating NBS programs were analyzed; the medians (and interquartile range, IQR) of state-specific percent of specimens that met the goal are presented.
RESULTS: The percent of specimens collected before 48 hours of life increased from 95% (88-97%) in 2016 to 97% (IQR 92-98%) in 2018 for the 25 states, with 20 (80%) of programs collecting more than 90% of the specimens within 48 hours of birth. Approximately 41% (IQR 29-57%) of specimens were transported within one day of collection. Time-critical result reporting in the first five days of life improved from 49% (IQR 26-74%) in 2016 to 64% (42%-71%) in 2018, and for non-time critical results from 64% (IQR 58%-78%) in 2016 to 81% (IQR 68-91%) in 2018. Laboratories open seven days a week in 2018 reported 95% of time-critical results within five days, compared to those open six days (62%), and five days (45%).
CONCLUSION: NBS programs that participated in NewSTEPs 360 made great strides in improving timeliness; however, ongoing quality improvement efforts are needed in order to ensure all infants receive a timely diagnosis.

Introduction

Newborn screening

Newborn screening (NBS) is a public health program that aims to identify newborns at risk for serious life-altering disorders in the first week of life. The NBS process is composed of multiple components (Fig 1) that must work in a coordinated and efficient manner to allow for early medical intervention before significant and irreversible damage occurs.[1] Hospital staff, midwives, and other clinical personnel collect, package, and ship the dried blood spot NBS specimen through commercial or private couriers to be delivered to a state, regional, or private NBS laboratory. Once the specimen is received at the laboratory, testing is completed and results are reported to the appropriate medical personnel who confirm or rule out a diagnosis and initiate the required intervention as appropriate.

Fig 1

Newborn screening process, illustrating specimen collection through result reporting.

Newborn screening (NBS) is a complex system that involves the collection of specimens at birthing facilities, transportation of specimens to the NBS public health laboratory for testing and communicating results to health care providers and families. Each step needs to occur in a timely manner in order to prevent infant mortality and morbidity. NewSTEPs 360 supported state/territorial NBS programs to address challenges associated with the pre-analytical and analytical phases of the NBS process by implementing various activities, including 1) providing education to birthing centers and midwives about the importance of timely collection and shipment of specimens; 2) shortening transit time by optimizing shipping methods; 3) expanding laboratory operating hours to decrease the time from specimen receipt to results reporting; 4) improving the efficiency of laboratory workflows; and 5) developing a health information technology infrastructure to improve the transmission of electronic demographic information, laboratory orders, and results between the NBS laboratory and health care providers.

Timeliness in newborn screening

Timely identification of newborns affected by core disorders on the Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC) Recommended Uniform Screening Panel (RUSP) is critical.[2, 3] The ACHDNC has the mission to reduce morbidity and mortality due to heritable disorders in newborns and children, and provides recommendations to guide and strengthen the newborn screening system. While early detection has always been the goal of NBS, the expansion of the list of screened disorders in the late 1990s and early 2000s to include those identified via tandem-mass spectrometry with a short pre-symptomatic window has led to an increased urgency to detect affected newborns as quickly as possible.[4, 5] NBS timeliness recommendations were first published in 2006 by the American College of Medical Genetics and Genomics (ACMG) and included specifications that all specimens should arrive at the NBS laboratory within three days of collection, and that results be reported within two days of specimen receipt and within five days of specimen collection.[1]

In 2013, based on public comment, the ACHDNC decided to review policies and practices relating to timeliness of NBS in the United States. In support of this work, the Society for Inherited Metabolic Disorders (SIMD) classified 16 of 35 disorders included on the RUSP as time-critical, requiring immediate medical attention. [6] Based on methodologies in practice, published literature and expert opinion, in 2015, the ACHDNC developed five timeframes for conducting newborn screening (Table 1). [7]

Table 1

Newborn screening timeliness goals from the Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC) [7].

To achieve the goals of timely diagnosis and treatment of screened conditions and to avoid associated disability, morbidity and mortality, the following time frames should be achieved by NBS systems for the initial newborn screening specimen:

• Presumptive positive results for time-critical conditions should be communicated immediately to the newborn’s health care provider but no later than five days of life.

• Presumptive positive results for all other conditions should be communicated to the newborn’s health care provider as soon as possible but no later than seven days of life.

• All NBS tests should be completed within seven days of life, with results reported to the health care provider as soon as possible.

In order to achieve the above goals:

• Initial NBS specimens should be collected in the appropriate time frame for the newborn’s condition but no later than 48 hours after birth.

• NBS specimens should be received at the laboratory as soon as possible; ideally within 24 hours of collection.

NewSTEPs 360

In February 2015, the Health Resources and Services Administration (HRSA) funded a collaborative improvement and innovation network to support multidisciplinary teams in improving newborn screening timeliness. This project was called NewSTEPs 360. Supplemental funding to support this project was also provided by the Cystic Fibrosis Foundation. Under NewSTEPs 360, NBS programs were convened to identify and overcome barriers to timely NBS through technical and financial assistance. NewSTEPs 360 was built upon the foundation of the HRSA-funded Newborn Screening Technical assistance and Evaluation Program (NewSTEPs)[8], which included access to the NewSTEPs Data Repository.

The NewSTEPs Data Repository collects data on NBS system components with the goal of supporting quality improvement initiatives and providing comparative data at the state, regional, and national levels. NBS programs that voluntarily enter data into the repository have access to their own data plus de-identified, aggregate data from other participating programs. Data elements collected in the repository include NBS program information (e.g., disorders screened, fees, policies, program structure, etc.), quality indicators (QI) for each stage of the NBS process at the programmatic level, and case data [9] on infants with a confirmed diagnosis of a disorder detected by NBS.

NewSTEPs adopted a panel of eight quality indicators that measure newborn screening program performance that were developed by the broader newborn screening community, including newborn screening laboratorians, follow-up specialists, and clinical providers.[10] A subset of the panel of QIs that reflect timeliness outcomes were collected as part of NewSTEPs 360 (Table 2). Each NBS program participating in NewSTEPs 360 was assigned a continuous quality improvement (CQI) coach who met with the NBS program team monthly to identify challenges and opportunities for improvement in this subset of QIs.

Table 2

NewSTEPs Data Repository quality indicator 5: NBS timeliness from collection to reporting results.

Excerpted from NBS quality indicator panel. [10].

Quality Indicator	Description
QI 5a.i	Percent of first dried blood spot specimens collected in the specified time intervals, in units of hours, from birth.
QI 5b.i	Percent of first dried blood spot specimens received at the NBS laboratory in the specified time intervals, in the unit of days, from specimen collection.
QI 5c.i	Percent of dried blood spot specimens with out-of-range results for time-critical disorders reported in the specified time intervals, in units of days, from laboratory receipt.
QI 5c.ii	Percent of dried blood spot specimens with out-of-range results for non-time-critical disorders reported in the specified time intervals, in units of days, from laboratory receipt.
QI 5c.iii	Percent of first specimens with normal or out-of-range results for any disorder reported in the specified time intervals, in units of days, from laboratory receipt.
QI 5d.i	Percent of dried blood spot specimens with out-of-range results for time-critical disorders reported in the specified time intervals, in units of days, from birth.
QI 5d.ii	Percent of dried blood spot specimens with out-of-range results for non-time-critical disorders reported in the specified time intervals, in units of days, from birth.
QI 5d.iii	Percent of first specimens with normal or out-of-range results for any disorder reported in the specified time intervals, in units of days, from birth.

This study summarizes the impact of implementing quality improvement efforts in participating NBS programs to attain national timeliness recommendations for newborn screening. We evaluated the timeliness of initial specimen collection, delivery from the birthing center to the NBS laboratory, reporting of results for both time-critical and non-time-critical disorders, and the overall reporting of all NBS results. In a subset of programs, we also assessed the timeliness of diagnosis and medical intervention. Finally, we analyzed the impact of individual program activities to improve timeliness.

Methods

Participants in NewSTEPs 360

Twenty-eight US state and territorial NBS programs were selected to participate in NewSTEPs 360 via two rounds of a competitive application process. State newborn screening programs applied through an internet-based application process, identifying the challenges within their programs, and proposed corresponding quality improvement initiatives. Baseline data were required from all applicants. Sixteen applicants representing 20 states were selected for participation in 2015 and a second cohort of eight programs joined in 2016. Participating programs actively engaged in a continuous quality improvement framework to improve timeliness by developing individual improvement projects at the programmatic level. Funding for NewSTEPs 360 was provided by HRSA (UG8MC28554, 9/1/15–8/31/18, no-cost extension through 8/31/19). Supplemental funding was provided by the Cystic Fibrosis Foundation (SONTAG16Q10). The infrastructure for NewSTEPs is funded through HRSA (U22MC24078). NewSTEPs activities were deemed to be public health quality improvement and not human subject research by the Colorado Multiple Institutional Review Board.

Data collection

NewSTEPs data repository and data security

The repository is a centralized web-based platform that can be accessed by registered users via a 128-bit secure socket layer (SSL) encryption. Registration for the NewSTEPs repository is open to all interested parties, however access to state specific data elements is restricted to individuals working in the state newborn screening system. NewSTEPs requires that NBS programs have a signed Memorandum of Understanding (MOU) with the Association of Public Health Laboratories (APHL) in order to enter Quality Indicator (QI) and case data into the repository. Review of QI and case-level data are limited by role-based access control that was assigned at the individual NBS program level, whereas review of programmatic NBS program information (i.e. operating hours, policies and procedures, state demographics) is available to the public.

Quality indicator data

NewSTEPs 360 participants were required to provide monthly data for the QIs associated with timeliness,[10] which was aggregated at the annual level for cross-year comparison. To further encourage data entry and accuracy, CQI coaches frequently reviewed the data using visualization tools and discussed progress or barriers during the monthly or bi-monthly team coaching calls.

Quality indicator benchmarks

Benchmarks were adapted directly from the ACHDNC timeliness goal recommendations (https://www.hrsa.gov/advisory-committees/heritable-disorders/newborn-screening-timeliness.html). Few newborn screening programs were able to meet the ambitious recommendation stated by the ACHDNC the NBS specimens should be received at the laboratory as soon as possible, ideally within 24 hours of collection. In response to this, NewSTEPs created an additional benchmark of two calendar days to assess time elapsed from specimen collection to receipt at the laboratory as an intermediary step. For purposes of our analysis, we equated 24-hours to the next calendar day. Assessing timing of specimen receipt per calendar day is in better alignment with the typical newborn screening laboratory approach. Programs typically test specimens after scheduled shipments have arrived on a given day, shipments arriving earlier in the day may not be tested earlier than those arriving right before the scheduled testing time, making calendar day a more meaningful metric than hours of delivery.

Further, an additional metric was added to assess the time from specimen receipt at the laboratory to results being reporting. This added metric helps identify opportunities for improving laboratory processes that could affect overall timeliness. The additional benchmark was calculated based upon the benchmarks set by the ACHDNC for other timeliness outcomes. The calculation of specimen receipt to reporting results are provided in Table 3.

Table 3

New metric added to calculate time elapsed from specimen receipt at the NBS laboratory to reporting results.

	ACHDNC Timeliness Goals			New Metrics
	Birth to Reporting results (A)	Birth to Collection (B)	Specimen Collection to Receipt at Lab(C)	Specimen receipt to reporting (D = A-B-C)
Time-Critical	5 Days	2 Days	1 Day	2 Days
Non-Time-Critical	7 Days	2 Days	1 Day	4 Days

Case data

The NewSTEPs Data Repository collects basic demographic and diagnostic information on all newborns with a disorder diagnosed through NBS in the US. Continuous timeliness measures were collected on each confirmed case to understand the factors that lead to shortened intervention times Case data is collected in the year following the birth of an infant to allow for the diagnostic process to be completed; cases entered for 2015–2017 by NewSTEPs 360 programs were included in this analysis.

NewSTEPs has implemented the following definitions for intervention and diagnosis, with disorder-specific definitions available[11]:

Time to medical intervention: The first time a medical professional acts to change the course of care for an infant. Intervention may occur via phone or clinic visit. This may also include the date a decision was made NOT to change course for the infant.

Time to diagnosis: The time elapsed from birth until a biochemical or molecular test result on a specimen taken from the infant that confirm the NBS result reported.

Strategies employed to improve timeliness

NewSTEPs 360 guided participating NBS programs through CQI activities via training, personalized coaching, and interactive learning sessions between NBS programs. To support team development and growth, a Plan-Do-Study-Act (PDSA) personality tool was developed to support team growth and team members’ roles within their programs (S1 File). The tool is a short quiz that helps a team to determine if members naturally affiliate with one functional component of the PDSA cycle to better understand how the team functions together; subsequent discussions led teams to identify strategies to maximize team productivity when engaging the PDSA cycle. Programmatic activities varied (Table 4) from educational strategies for birthing facilities and health care providers to increasing courier services and operating hours (Timeliness Toolkit for Expanding Newborn Screening Services–S2 File) and improving health information technology (HIT) systems (Building Block Guide—S3 File). Successes and failures were shared within the participating programs to facilitate the continuous quality improvement environment.

Table 4

List of strategies participating NewSTEPs 360 programs used to improve NBS timeliness.

* Each Strategy is Linked to a Corresponding QI Solution in Fig 1.

Specimen Collection (QI Solution 1)

Educated birthing facility staff and midwives/community-based providers on NBS timeliness (i.e., collection and shipment of blood spot specimens, reducing unsatisfactory specimens, and completion of the NBS card to minimize missing demographic information).

Educated health care providers on the timeliness of follow-up and diagnosis.

Specimen Shipment (QI Solution 2)

Educated birthing facility staff and midwives around courier pick-up hours and location.

Educated couriers on the importance of NBS specimens and timeliness of pick-up to drop-off at the NBS laboratory.

Expanded specimen pick-up to include weekends and/or holidays for all or some birthing facilities; utilize label/sticker to indicate weekend or holiday delivery.

Implemented new courier and/or changed courier route to reduce delivery time from pick-up to delivery at the NBS laboratory.

Implemented centralized “drop-off” locations for out-of-hospital births (i.e., FedEx, local/county health departments or neighboring hospitals); provided UPS labels for midwives.

Changed pick-up location in hospitals to reduce “handoffs.”

Built-in contract monitoring practice for couriers; establish cut-off times for specimen delivery.

Specimen Receipt (QI Solution 3)

Expanded operating hours to include weekends and/or holidays so that the laboratory is not closed for more than two consecutive days.

Modified shift hours so that laboratory staff is available to accession specimens upon delivery; modify cut-off times for specimen delivery to align with hours of operation.

Lab Testing (QI Solution 4)

Implemented alternative testing methodologies, workflows, and/or algorithms to improve time from receipt to reporting results.

Improved laboratory information systems to minimize demographic errors and/or link specimens.

Hired quality improvement and/or data analytic staff

Results Reporting (QI Solution 5)

Implemented electronic ordering and/or electronic result reporting via HL7 interfaces.

Developed web portal for birthing providers or physicians for more timely access to NBS results.

Implemented electronic mechanisms for demographic data entry (e.g., optical character recognition, printing labels for newborn screening blood spot specimen card).

Other Strategies

Changed regulations for blood spot collection to 24 to 48 hours after birth.

*Additional tools available in the NewSTEPs Resource Library [12]

Statistical analysis

Monthly quality improvement data reported for January 2016 –December 2018 were converted to annual metrics and were included in the analysis. Timeliness Quality Indicator (QI) data were collected for the purposes of program improvement on a national level and are not intended for formal statistical analysis. Each participating state newborn screening program provides data for this repository with the intent of informing decisions to improve newborn screening systems. NewSTEPs 360 was a quality improvement initiative that was not powered to detect statistical differences. Further, small cell sizes for individual improvement categories may result in spurious significant results. As a result, descriptive statistics and graphical displays were created, presenting the changes in the percent of specimens meeting ACHDNC benchmarks for time elapsed from birth to specimen collection, collection to laboratory receipt, laboratory receipt to reporting out NBS results, and birth to reporting out NBS results. Additional investigations of the timeliness indicators were completed, stratified by days of operation and type of laboratory (local state laboratory vs. external [regional or private]). Differences in individual case data were tested using non-parametric Wilcoxon-ranked sum tests, and a significance level of 0.05 was set. Data were analyzed using SAS version 9.4 (Cary, NC), and displayed using Tableau Desktop (Seattle, WA, copyright 2019). Results of the analysis do not display state or territory names with the intent to protect NBS programs from the release of sensitive information.

The Colorado Multiple Institutional Review Board determined that the newborn screening quality improvement initiaitives led developed through NewSTEPs are not human subject research.

Results

Timeliness quality indicators

Twenty-eight newborn screening programs participated in NewSTEPs 360; 25 provided complete data for birth to specimen collection (2016–2018), and a subset of those programs provided data for each of the other timeliness QIs (Table 5). Participating programs that provided data for the three years are presented, although some programs were not able to provide data for all of the QIs due to systems challenges and laboratory information management systems structure that did not allow for data collection or retrieval at the program level. Progression toward meeting the ACHDNC timeliness goals for all QIs was achieved in most programs, demonstrated through individual trajectories representing the percent of specimens that met the goal (Fig 2A–2C). The median of all programs for each indicator demonstrated improvement in all indicators across all three years, described in detail below.

Table 5

Percent of newborn screening dried blood spot specimens from each program that met the timeliness goals, by year; medians of the programs’ results are reported; N is the number of programs that provided data for the specified quality indicator.

For example: in 2016, half of the programs reported that at least 95.1% of the specimens met the goal of birth to specimen collection.

QI Timeliness Measure	Time Frame	N	2016		2017		2018
			Median %	IQR	Median %	IQR	Median %	IQR
Birth to specimen collection	48 Hours	25	95.1%	88.1% - 97.4%	96.4%	90.8% - 97.8%	97.0%	92.4% - 98.3%
Specimen collection to receipt at lab	1 Day	19	40.0%	28.6% - 52.5%	39.4	30.4% - 56.4%	41.8%	28.9% - 56.5%
Specimen collection to receipt at lab	2 Days	19	74.3%	67.8% - 86.6%	79.6%	69.9% - 88.7%	80.9%	70.3% - 88.45%
Receipt to Reporting Results
Presumptive positive* for time-critical disorders	2 Days	16	65.5%	38.0% - 89.9%	69.7%	50.2% - 88.4%	75.8%	50.5% - 90.4%
Presumptive positive* for non-time-critical disorders	4 Days	15	80.2%	56.9%-93.9%	90.0%	72.4% -95.1%	93.5%	67.3% - 96.3%
All (normal and presumptive positive results)	4 Days	19	90.3%	69.1% - 98.8%	90.8%	83.0% - 99.2%	94.2%	88.3% - 99.3%
Birth to Reporting Results
Presumptive positive* for time-critical disorders	5 Days	16	48.9%	25.8% - 73.8%	48.8%	34.3% - 71.5%	63.5%	42.5%-71.0%
Presumptive positive* for non-time-critical disorders	7 Days	15	64.4%	57.8% - 77.9%	75.9%	67.1% - 86.0%	80.9%	68.0% - 90.7%
All (normal and presumptive positive results)	7 Days	18	88.9%	68.8% - 96%	87.6%	78.4% - 95.7%	89.5%	84.8% - 98.2%

* Presumptive positive indicates with high probability that the infant may have the disorder; however confirmatory diagnostic testing is required.

Fig 2

Percent of newborn screening dried blood spot specimens that achieved timeliness goals for collection and receipt at the testing laboratory.

Data are presented for each state program individually, with box plots overlaid to demonstrate national trends. Box and whisker plots display the median of the percent of specimens that met the benchmark for each program, with upper and lower quartiles, and range. Panel A: Percent of specimens collected within 48 hours of birth, Panel B: Percent of specimens received at the laboratory within one day of collection (next calendar day), Panel C: Percent of specimens received at the laboratory within two calendar days of collection.

Specimen collection and transportation

Programs successfully collected initial specimens within the first 48 hours of birth, with stepwise increases demonstrated each year (Fig 2A, Table 5; median of the programs’ percent of specimens that met benchmark in 2016: 95.1%, 2017: 96.4%, 2018: 97.0%). In 2018, more than 90% of specimens were collected in the 48-hour time frame in 19 programs (n = 19/25, 76%) and more than 95% were collected within 48 hours in 14 programs (n = 14/25, 56%). Specimens deemed to be unsatisfactory for analysis by the NBS laboratory were flagged for recollection as they could result in delayed testing and subsequent reporting of results. The percent of specimens deemed unsatisfactory for analysis by state laboratories was 1.3% in 2016, 1.3% in 2017, and 1.5% in 2018 (medians of all programs). Further, NBS cards that were submitted without complete essential demographic information could have also delayed testing and reporting of results. Variable definitions between NBS programs and changes in the definitions of the required information within state programs made these data difficult to summarize for programmatic trends.

The program median for the time from collection to receipt at the NBS laboratory (Fig 2B, Table 5) on the next calendar day was 40.0% in 2016, 39.4% in 2017 and 41.8% in 2018. Allowing two calendar days after collection to receipt (Fig 2C), the program median of specimens which met the guideline increased to 74.3% in 2016, to 80.9% in 2017 and 81% in 2018.

Reporting of NBS results

The percentage of specimens with reporting times that met the benchmarks improved both in individual state trajectories and in program medians over all three years (Fig 3A, Table 5). The timely reporting of NBS results for time-critical disorders within two days of receipt at the laboratory improved from a program median of 65.5% in 2016 to 75.8% in 2018. Similarly, the program median for non-time-critical result reporting within four days of laboratory receipt improved from 80.2% in 2016 to 93.5% in 2018. Reporting of all NBS results from laboratory receipt improved from 90.3% in 2016 to 94.2% in 2018.

Fig 3

Percent of newborn screening dried blood spot specimens with results reported within the recommended timeframe.

Data are presented for each state program individually, with box plots overlaid to demonstrate national trends. Box and whisker plots display the median of the percent of specimens that met the benchmark for each program, with upper and lower quartiles, and range. Panel A: Percent of specimens with results reported out for time-critical results within two days of receipt at lab (top), non-time-critical reported out within four days of receipt (middle), and all results reported out within four days of receipt (bottom); Panel B: Percent of specimens with results reported out for time-critical results within five days of birth (top), non-time-critical reported out within seven days of birth (middle), and all results reported out within seven days after birth (bottom).

The elapsed time from birth to result reporting showed improvements in each category as well (Fig 3B, Table 5). Time-critical results reported within five days of birth started at 48.9% in 2016 and increased to 63.5% by 2018; reporting of non-time-critical results within seven days of birth improved from 64.3% to 80.9%. The program median for reporting all NBS results within seven days of birth did not demonstrate change over this period (88.9% in 2106 to 89.5% in 2018).

Timeliness data for cases with a confirmed diagnosis

The 25 participating NBS programs that provided data for NewSTEPs 360 reported 1,713 cases with a confirmed diagnosis of a disorder identified by newborn screening for the years 2016–2018 (288 time-critical cases; 1,425 non-time-critical cases). Individual specimen collection times are consistent with ACHDNC timeliness goals (Table 6). The median report time for time-critical disorders (five days) was earlier (p < 0.0001) than non-time-critical disorders (seven days), and both are in alignment with the timeliness goals. The resulting intervention and diagnosis times are earlier for time-critical disorders than non-time-critical (p<0.0001).

Table 6

Timeliness metrics for newborns identified with a disorder on the newborn screening panel.

	All Infants (n = 1,713) median (IQR)	Infants with a Time-Critical Disorder [6] (n = 288) median (IQR)	Infants with a Non-Time-Critical Disorder (n = 1,425) median (IQR)
Collection of Specimens (hours)	28 (24–40)	28 (25–38)	28 (24–41)
Receipt at Lab (days from birth)	3 (3–4)	3 (3–4)	3 (3–4)
Result Release (days from birth)	6 (5–8)	5 (4–7)	7 (5–8)
Intervention (days from birth)	11 (6–26)	6 (4–9)	13 (7–29)
Diagnosis (days from birth)	18 (9–39)	12 (7–33)	20 (10–40)

Individual-level data from diagnosed cases demonstrate that at least 50% of the specimens were collected, received, and results were reported within the ACHDNC recommended period. Intervention and diagnosis occur earlier in infants with time-critical disorders compared to infants with non-time-critical disorders, reflecting the expedited nature of laboratory processes within the laboratories related to time-critical disorders.

Laboratory operating hours

We found that laboratory operating hours are a critical factor associated with improved specimen delivery times, timely testing, and efficient reporting of results. Each state reported wheteher they were open 5, 6, or 7 days, along with the activites performed on those days. During the NewSTEPs 360 program, two participating NBS programs increased the number of days their laboratories were open, and multiple programs added or adjusted the hours of operation to align with the delivery of samples. By 2018, six of the 25 NBS laboratories were open five days a week, 13 were open six days a week, and six were open all seven days. Activities performed on a given day of the week by laboratories vary (Table 7) due to staff training, availability, and internal policy decisions. For example, programs may report being open 7 dyas a week while not reporting non-time-critical results due to an agreement with clinical specialists to wait until regular business hours to avoid prolonged waiting times for families with a presumptive positive for non-time-critical disorders. These internal decision may impact the timelienss of individual reporting, but no links to clinical outcomes can be made.

Table 7

Laboratory weekend operations vary across newborn screening programs, categorized by the number of days a laboratory reports testing dried blood slot specimens*.

NBS Lab Activity	Laboratory Reported Testing 7 days/week				Laboratory Reported Testing 6 days/ week				Laboratory Reported Testing 5 days/week
	All Days	M-F and Saturday	M-F and Sunday	M-F only	All Days	M-F and Saturday	M-F and Sunday	M-F Only	All Days	M-F and Saturday	M-F and Sunday	M-F Only
Accessioning / Recording Specimen Receipt	3/6	3/6	0/6	0/6	0/12	12/12	0/12	0/12	0/6	0/6	0/6	6/6
Courier Operations	3/6	2/6	0/6	1/6	3/12	4/12	5/12	0/12	1/5	2/5	0/5	2/5
Demographic Data Entry	3/6	3/6	0/6	0/6	0/12	8/12	4/12	0/12	0/6	0/6	0/6	6/6
Receiving Specimens	3/6	3/6	0/6	0/6	0/12	11/12	1/12	0/12	1/6	2/6	0/6	3/6
Molecular Testing	1/6	1/6	2/6	2/6	0/12	5/12	0/12	7/12	0/4	0/4	0/4	4/4
Reporting Non-Time-Critical Results	4/6	1/6	0/6	1/6	0/11	3/11	0/11	8/11	0/6	0/6	0/6	6/6
Reporting Time- Critical Results	5/6	1/6	0/6	0/6	2/12	9/12	0/12	1/12	0/6	1/6	0/6	6/6

*Not all labs reported activities for all categories; total numbers reported are reflected

The percentage of specimens reported out within the ACHDNC recommended benchmarks were improved in laboratories with seven days operations compared to those with five or six-day operations (Fig 4). The median percent of programs with specimen results reported out for time-critical disorders within five days of life was greater than 80% in all years for laboratories open seven days a week, while the median in laboratories open six days a week did not reach 65%, and in those open five days a week the median failed to reach 50% of specimens. The median percent of programs with specimen results reported in a timely manner increased for all laboratories regardless of operating days for both non-time-critical disorders and all results in seven days; however, laboratories open seven days reported results earlier than those open six or five days.

Fig 4

Newborn screening programs that are open seven days report a higher percentage of results to medical providers in the recommended time period.

Box and whisker plots display the median of the percent of specimens that met the benchmark for each program, with upper and lower quartiles, and range. Panel A: Percent of specimens with results reported out for time-critical results within five days of life at programs open five days (left), six days a week (middle) and seven days a week (right). Panel B: Percent of specimens with results reported for non-time-critical results within seven days of life at programs open five days (left), six days a week (middle) and seven days a week (right). Panel C: Percent of specimens with all results reported within seven days of life at programs open five days (left), six days a week (middle) and seven days a week (right).

Most NBS programs participating in NewSTEPs 360 have a laboratory housed within their state public health department (n = 18, recorded in 2018), while others contract with external laboratories (n = 7, recorded in 2018). The external laboratory may be managed by a private or commercial entity, or it may be housed within another state public health department. The percent of specimens with results reported within the recommended time frames from birth for both individual state laboratories and external laboratories for 2016–2018 demonstrate improvement for time-critical, non-time-critical, and all result reporting (Fig 5) suggesting that both state and private labs can achieve the same success in newborn screening timeliness.

Fig 5

Newborn screening programs utilizing external laboratories report a higher percentage of results to medical providers in the recommended time period.

Box and whisker plots display the median of the percent of specimens that met the benchmark for each program, with upper and lower quartiles, and range. Panel A: Percent of specimens with results reported out for time-critical results within five days of birth for state labs (left) and external labs (right); Panel B: Percent of specimens with non-time-critical results reported out within seven days of birth for state labs (left) and external labs (right); Panel C: Percent of specimens with all results reported out within seven days after birth for state labs (left) and external labs (right).

Discussion

The overarching ACHDNC timeliness goals are designed to achieve the earliest diagnosis and intervention for infants with time-critical and non-time-critical disorders identified through newborn screening. [7] The introduction of national timeliness goals, paired with a continuous quality improvement program has led to improved times in reporting results to the clinical community, and earlier intervention of affected infants for NBS programs which participated in NewSTEPs 360. NewSTEPs 360 has demonstrated that NBS programs can make progress toward reaching these goals on a population level in a relatively short time through an organized, focused quality improvement effort tailored to the needs of individual states; however, there is room for system improvement.

State efforts to improve timeliness

During NewSTEPs 360 participation, state programs improved the percentage of specimens collected within ACHDNC’s recommended collection time of before 48 hours of life, the percentage of specimens received within two days of collection, and the percentage of results reported out by recommended guidelines. Programs achieved this through different approaches, including (1) implementing educational campaigns with birthing facilities, (2) increasing laboratory hours of operation and workforce schedules, (3) expanding courier service to deliver specimens to the NBS laboratory, (4) changes in laboratory testing methods, (5) using electronic ordering and results reporting with birthing centers, and (6) changes in regulations to require earlier collection. However, the trajectory of improvement and percent improvement varied among participants.

The timely collection of a newborn screening specimen at the birthing facility allows for earlier analysis and reporting. Regulations in three participating states were changed to reflect the shorter national guidelines of 24–48 hours for collection, and remarkable improvements were seen in those states. Additionally, participating NewSTEPs 360 programs developed educational materials, videos, online modules and in-person training sessions to ensure the staff collecting the specimens were knowledgeable about the importance of proper and timely collection and shipping.

NBS laboratories have historically operated during normal working hours on weekdays. However, the increased urgency of many of the new disorders added to the newborn screening panel has changed the paradigm.[4] Many NBS laboratories have shifted their work days to include Saturdays and/or Sundays and extended or modified operating hours throughout the week. Programs within NewSTEPs 360 pursued changes in operating hours, seeking additional funding, increased fees, and modified work schedules for employees. Continued efforts to increase operating hours so that babies receive the same services independent of the day-of-the-week they were born will decrease the risk of tragic outcomes for individual families. [12]

Most NBS programs are still working to achieve the goal of specimens being delivered to the newborn screening testing laboratory within one or two calendar days of collection. This has been accomplished by individual states through improved shipping from birthing centers, expanded courier systems, increased communication with the couriers, and increased laboratory operating hours to accept specimens. Based upon the largest gaps in timeliness identified in NewSTEPs 360, the best potential for timeliness gains includes increasing the number of days that laboratories are open, adding weekend and holiday couriers, and improving courier services for the transportation of specimens from birthing facilities to newborn screening laboratories.

Improvements in laboratory processes internal to the program were implemented across participating programs with the goal of improving timeliness. For example, one program identified a delay in reporting due to the timing of hemoglobinopathy results, which delayed the reporting of all results. The program changed their incubation and workflow process so that all results could be reported in a timely manner. In another program, a concerted effort was placed on improving the demographic data entry from the dried blood spot cards to improve timely data acquisition and reporting.

Health Information Technology (HIT) solutions hold promise for continuing to improve newborn screening timeliness. Many programs have instituted electronic solutions for data sharing, including electronic orders to improve demographic data transfer, electronic transfer of data, result reporting, and electronic faxing of results. One program implemented electronic ordering of dried blood spot tests, decreasing the time to verify information and initiate testing, initially in four hospitals, then more broadly across the state. The Building Blocks guide provides guidance to NBS programs to implement HIT solutions that can improve timely orders and reporting of results (S3 File).

Increased data entry into repository through utilization

One change that was seen as part of NewSTEPs 360 was an increase in data entry in the NewSTEPs data repository. As part of NewSTEPs 360, the repository was configured so that participating states could enter Quality Indicator data monthly vs just yearly. Further, CQI coaches encouraged monthly data entry and tracking. Options to upload data to ease manual entry were also provided, including direct upload of comma-separated-values files (.CSV), and direct assistance with data manipulation within states. NewSTEPs 360 participants utilized real-time data analysis in partnership with their quality improvement coach. The NewSTEPs Data Repository and infographics have been utilized by NBS programs to advocate for additional resources at the local level. For example, programs shared the NewSTEPs 360 data infographics with program leadership to demonstrate the improvements in timeliness metrics that were gained from adding operating hours or couriers, including weekend/holiday couriers. Conversely, other programs were able to demonstrate that they lagged behind the other participating programs and identified resource needs that could help to improve outcomes.

Unintended consequences

While improving timeliness in newborn screening was the ultimate goal of the NewSTEPs 360 program, timeliness efforts may have unintended consequences. Analytic cutoffs have typically been developed based upon age-based normal ranges for infants who are 24–48 hours of life, and testing infants earlier may impact the accuracy of the tests. Decreasing the accuracy of the screen may result in a high number of specimens flagged for follow-up testing, more infants sent for diagnostic testing, and an increase in false negatives, although this was not reported in this study. Further, earlier screening may limit the time to educate parents in the hospital prior to collecting the screen.

NBS programs have limited resources. Additional demands on staff to meet timeliness goals can limit the time that programs have to improve other program outcomes. Further, changes in programmatic and individual performance expectations may impact staff morale, which in turn affects staff retention. State programs may need to advocate for additional resources to meet timeliness goals, and the other requirements of the NBS program to meet the needs of its newborn population and provide the best outcomes for newborns with a disorder identified by NBS.

Limitations

The results of this analysis are limited to NewSTEPs 360 funded programs, yet most NBS programs are engaged in activities to achieve ACHDNC timeliness goals and beyond. States NBS programs applied to participate in the NewSTEPs 360 program, potentially introducing a selection bias as they may not be a representative sample of all NBS programs Additionally, NBS program variation in NBS data collection may limit interpretation of QI timeliness data. For instance, the ACHDNC timeliness goals apply to first specimens collected, but some programs were unable to differentiate between first and subsequent specimens, which can result in longer reported timeframes than programs reporting data for first specimens only. Some programs also complete second-tier testing to improve the specificity of the screen, but potentially delaying the final result reporting. In addition, not all programs were able to collect the necessary time stamp of specimen receipt at the laboratory electronically, resulting in limited data reporting for some of the outcomes. There is also significant variability in the definitions of required data elements on the dried blood spot card, making interpretation across programs difficult. Finally, only a subset of programs provided case level data, limiting the generalizability of the results.

Conclusions

Newborn screening is one of the most successful public health programs in the US.[13] While states have clear authority with regard to NBS program oversight and monitoring, there is a federal role in supporting states in the implementation of the various components of the newborn screening system and ensuring timely diagnosis and management. The ACHDNC, public health departments, clinical specialists, birthing facilities, midwives, primary care providers, and parents have partnered to improve the newborn screening system. Improving timeliness of reporting of results has been a critical focus.

The individualized approach within NewSTEPs 360 allowed coaches to customize the support provided to the state newborn screening program and, whenever possible, connect one program with another who had shown success in an area. We believe that this structure strengthened the effectiveness of the program. In addition, the NewSTEPs Data Repository played a key role in the success of programs because it allowed (1) participating programs and CQI coaches to identify areas of needs, (2) the NewSTEPs 360 leadership to identify and meet educational needs of the larger group, and (3) the newborn screening community to see the gains made in timeliness.

The achievements of the NBS programs participating in this continuous quality improvement project in partnership with NewSTEPs 360 are the result of the ongoing support by the broader newborn screening community and its commitment to the newborns it serves. Continued success will depend upon that network of support.

(PDF)

Click here for additional data file.

(PDF)

Click here for additional data file.

(PDF)

Click here for additional data file.

17 Nov 2019

PONE-D-19-27133

Newborn screening timeliness quality improvement initiative: Impact of national recommendations and data repository

PLOS ONE

Dear Dr Marci K. Sontag and co-workers,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Some minor issues like the abstract's structure, shortening by deleting repetitions and some context for non-Americans.

We would appreciate receiving your revised manuscript by Dec 23 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Jacobus P. van Wouwe, MD PhD

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. For studies reporting research involving human participants, PLOS ONE requires authors to confirm that this specific study was reviewed and approved by an institutional review board (ethics committee) before

the study began. Please provide the specific name of the ethics committee/IRB that approved your study, or explain why you did not seek approval in this case.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

3. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4.  Thank you for stating the following in the Financial Disclosure section:

The project described in this article was funded by the Department of Health and Human Services, Health Resources and Services administration under Cooperative Agreement # UG8MC28554. (MKS, JIM, SM, CY, RS, SE, SS, JO)

The funders played a role in decision to publish and Preparation of the manuscript.

We note that one or more of the authors are employed by a commercial company: CI International.

1. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

2. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

Additional Editor Comments:

The reviewers gave some practical suggestions how to improve your well written manuscript on this relevant subject. The structure of your abstract, shortening the discussion by deleting repeats, and the suggestion to add some context for the international community will further improve your manuscript. The examples the reviewers have pointed out seem easy to answer and I do hope to see your revision being submitted soon! This manuscript needs to get published! Congrats.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Newborn screening timeliness quality improvement initiative: Impact of national recommendations and data repository

This paper provides an overview of quality improvement initiatives taken on by state newborn screening programs to meet timeliness goals set by theSecretary of Health and Human Services’ Advisory Committee on Heritable Disorders in Newborns and Children. Using a data repository, the outcomes of these initiatives could be tracked and monitored between an individual state and the aggregate of all participating screening programs.

Thank you for giving me the opportunity to review this paper. Overall, I am favorable for the publication of this important summary of state efforts to meet national recommendations. Below are my comments and suggestions for edits within each section of the paper.

Abstract:

Lines 67-69: Consider starting the abstract with the goal of newborn screening “Early identification and treatment of affected infants prior to the onset of symptoms”. Then define the necessary and time critical steps to accomplish the goal (ie…specimen collection, transport, testing, and reporting). The words “newborn screening activities” make it seem optional or less significant. These are steps are critically important to accomplishing the mission.

Lines 77-78: Give a bit more detail on the necessary steps (ie…specimen collection, transport, testing, and reporting). This will help lead into the results summary in line 79-90.

Line 91: This is the first time you mention NewSTEPs 360. Please define NewSTEPs 360, perhaps in the background or the methods section of the abstract.

Introduction:

Line 101: Did you want to mention that clinics also collect newborn screening specimens, in addition to hospitals and midwives?

Line 104: Consider rewording to “medical personnel to facilitate additional testing to confirm or exclude a diagnosis and also initiate the required appropriate intervention.” I see facilitation of additional testing as being the primary goal (first step). Intervention (although absolutely important under certain conditions) may or may not be needed depending upon the results.

Figure 1: I struggled with figure 1. Your figure legend talks about NewSTEPs 360 and then lists five categories of activities performed by states to improve timeliness, but your figure only shows the process of newborn screening. I would suggest removing this text from the legend OR adding details to the figure to describe the five categories.

Line 171-177: Consider rewording the last paragraph a bit to reflect each pieces of your review. You “summarized” the quality improvement initiatives. You “evaluated” the timeliness data for initial specimen collection, delivery from the birthing center, etc, over a certain time period. You “correlated” or “analyzed” or “assessed” the impact of individual program activities to improve timeliness.

Methods:

Lines 193-203: Consider rewording. State first what is available to the public, followed by what is available to registered users and then finally what states have access to after providing the MOU. The paragraph is a bit confusing.

Line 210-224 and table 3: Did you want to use the benchmark of two calendar days for the timing between “specimen collection to receipt at lab”? Or only one day in transit, as shown in table 3?

Line 210: You mentioned in line 210 that you have added two additional timeliness benchmarks, what are the two benchmarks (2 days in transit and time to reporting?), but one is a modification to a metric and the other is a new metric? Please consider rewording the paragraph to provide clarity.

Line 247: You refer to PDSA as a personality tool. I really thought it was a tool to assess a change that is implemented. The word personality confused me as well as how this tool could support team growth and team member’s roles.

Results:

Table 5: Define the “N” as the number of programs that provided data for a give category. Maybe in a footnote?

Table 5: Please clearly differentiate between reporting of results “presumptive positive for time critical disorders” “presumptive positive for non-time critical disorders”, and reporting of “all (normal and presumptive positive) screening results” OR reporting of “all normal results”?

Line 323: I just want to make sure the difference between each of the three categories (time critical/non-time critical/all) is clearly communicated. Plus…maybe this in inherently obvious…but I think it is necessary to state again…that the reporting of presumptive positive NBS results doesn’t mean that the patient is affected with the disease.

Figure 2: Image is not clear/fuzzy. And it could also be larger. But maybe this will be addressed during the formatting process.

Line 344: “did not demonstrate change” OR “did demonstrate change”. It appears that the percentage when up slightly…but maybe this is not a significant change?

Table 7: This table is a bit confusing. For the six laboratories open 7 days a week, it is correct to read that 3 laboratories accession specimens all seven days and 3 laboratories accession specimens ONLY on Saturday? For the laboratory only open Monday through Friday, why do they have activities such as receiving specimens on Saturday? I believe I know what you mean to convey…but it is not inherently clear.

Line 407-408: The last sentence regarding the overall median for all results reported measures, comparing external versus state lab, is hard to appreciate from figure 5. It appears that the median would be closer to 80% for external labs. Maybe I would remove this sentence or clarify.

Discussion:

General comment: I really appreciate how you linked specific programmatic activities to the potential improvement seen within certain metrics.

Line 521: Include within the limitations section of the discussion how second tier (or even third tier) testing (molecular or biochem) performed by laboratories may impact the time to reporting, even for critical conditions.

Reviewer #2: Thank you for this interesting work. As time is of essence for many neonatal screening conditions, this work is very relevant for the stakeholders involved.

As suggestions I would like to put forward that I think the manuscript might be in use of extra context. The relevance of the topic is undisputed, but if the benchmark and the international context is presented more clearly, the reader would be able to grasp the relevance better. For example:

- how many cases are missed due to lack of timeliness? Which is hard to make concrete, because it would also require post-mortem genetic testing?

- what was the starting point of timeliness; there is mentioning of increasing time-critical disorders, but is this due to the type of disorders added to the RUSP or the revision of what a time-critical disorder is?

- how is this embedded or does it compare to international systems? Not all countries have birthing facilities / different facilities / deliveries at home, and the time of collection also differs quite a lot between countries.

- was an implementation theory or model used? How did all parts fit into that?

- Why was the selection based on a competative application process?

- How is the quality and standardization of data in the repository ensured?

Furthermore, I think the manuscript could be quite a bit shorter on the results if the tables are made more legible. Especially tables 5 and 6 are hard to read, because as a reader you are not always sure what you are looking at. Especially combined with the figures and the text, it seems there is a lot of double information, but the sources (how many labs answered which questions, i.e. how was dealt with missing values) are not always clear while potential stratification and/or combination of the variables is not made. Perhaps some information is better to put into tables instead of figures?

Taking the above into account, I think it is important and interesting work, but to be an international scientific publication it seems to be in need of a better handle on highlighting only or more of the really interesting results to the reader; what tables and figures are really relevant?, and better embedding in international scientific literature, unless it is more or less solely aimed at US readers, but still evidence-based implementation theories / model could be applied.

Reviewer #3: Dear authors,

I have written this manuscript with pleasure and I feel it is suitable for publication in PLOSONE.

This is a well-written paper on the continuous improvement of the primary process of newborn screening. It is a showcase of how data concerning this process should be handled and how they can be used to monitor continuous improvement, providing the necessary elements of a PDCA cycle. While scientifically, the data may not be top-notch, for the field of neonatal screening it is an example, also eliciting the ‘machinery’, in terms of centralized databases and secure access, and predefined criteria and definitions, that is needed to make this work.

You make the necessary selections (in terms of handling cards without demographic data or cards that were flagged for recollection) to come to sensible outcome measures.

I have very few issues with this paper.

Line 180

Particpants in Newsteps. Selecting states is not selecting others. Can you explain what states were not selected, why, and whether this could introduce selection bias? (e.g. at limitations of study).

Line 407

overall median was observed. Median of what?

Line 491 -500.

I would like to know whether at the site of the laboratory, these CQI reports were generated automatically, querying the databases of the individual laboratories.. This information is important as it superior to manual transfer of data. If the data is entered manually in the files to upload in the repository, is it curated or checked at the site of the repository? This also applies on ln 204-section Quality Indicator data.

The discussion is quite lengthy and contains repeats of what was already said in the reminder of the paper. Moreover, the discussion is used to describe developments within the programs (e.g. a lengthy description of the efforts made to aIso work on weekend days feel that the authors can be more concise and ( Lines 512-520This is true, but does not relate to the study results.) . I feel the Conclusions section is a double of what has been said before in the paper, especially the discussion.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

24 Jan 2020

Specific Comments from Reviewer #1:

Abstract:

• Lines 67-69: Consider starting the abstract with the goal of newborn screening “Early identification and treatment of affected infants prior to the onset of symptoms”. Then define the necessary and time critical steps to accomplish the goal (ie…specimen collection, transport, testing, and reporting). The words “newborn screening activities” make it seem optional or less significant. These are steps are critically important to accomplishing the mission.

AND

Lines 77-78: Give a bit more detail on the necessary steps (ie…specimen collection, transport, testing, and reporting). This will help lead into the results summary in line 79-90.

Response: Thank you for this comment. We have adjusted the abstract to reflect these changes in both the background and methods section of the abstract.

• Line 91: This is the first time you mention NewSTEPs 360. Please define NewSTEPs 360, perhaps in the background or the methods section of the abstract.

Response: We have added NewSTEPs 360 in both the background and methods section of the abstract.

Introduction:

• Line 101: Did you want to mention that clinics also collect newborn screening specimens, in addition to hospitals and midwives?

Response: The reviewer is correct. While most of the focus of the newborn screening timeliness activities were on the first specimen, clinics do indeed collect specimens. We have modified the introduction to reflect this.

• Line 104: Consider rewording to “medical personnel to facilitate additional testing to confirm or exclude a diagnosis and also initiate the required appropriate intervention.” I see facilitation of additional testing as being the primary goal (first step). Intervention (although absolutely important under certain conditions) may or may not be needed depending upon the results.

Response: The reviewer brings up an excellent point. We believe the initiation of an appropriate intervention to be the ultimate goal for affected infants, and many times a confirmatory diagnosis may come months after an intervention starts. The reviewer accurately notes that classification of infants into affected vs unaffected is critically important. We have rephrased this sentence as suggested, to indicate diagnosis should occur first.

• Figure 1: I struggled with figure 1. Your figure legend talks about NewSTEPs 360 and then lists five categories of activities performed by states to improve timeliness, but your figure only shows the process of newborn screening. I would suggest removing this text from the legend OR adding details to the figure to describe the five categories.

Response: The categories of activities are displayed in the linked boxes under each step in the NBS process. We believe figure one more clearly describes the process and the supporting activities within the process that were implemented within NewSTEPs 360.

• Line 171-177: Consider rewording the last paragraph a bit to reflect each pieces of your review. You “summarized” the quality improvement initiatives. You “evaluated” the timeliness data for initial specimen collection, delivery from the birthing center, etc, over a certain time period. You “correlated” or “analyzed” or “assessed” the impact of individual program activities to improve timeliness.

Response: Thank you, we have made the edits as suggested by the reviewer.

Methods:

• Lines 193-203: Consider rewording. State first what is available to the public, followed by what is available to registered users and then finally what states have access to after providing the MOU. The paragraph is a bit confusing.

Response: This paragraph has been modified to better describe the processes in place for data sharing and data access.

• Line 210-224 and table 3: Did you want to use the benchmark of two calendar days for the timing between “specimen collection to receipt at lab”? Or only one day in transit, as shown in table 3?

Response: This is a great observation. To avoid additional complicated descriptions we have chosen to present one day transit within column C of Table 3. We could have also modified the birth to specimen collection window to be one day, rather than two days. Each of these benchmarks is a bit arbitrary. We wanted to derive a metric to allow NBS programs to assess their performance in laboratory processes, and we did so using the metrics already defined by the ACHDNC, resulting in two days of laboratory testing for time-critical and four days for non-time-critical.

• Line 210: You mentioned in line 210 that you have added two additional timeliness benchmarks, what are the two benchmarks (2 days in transit and time to reporting?), but one is a modification to a metric and the other is a new metric? Please consider rewording the paragraph to provide clarity.

Response: This is an excellent point. We edited the paragraph to reflect that we modified one benchmark and added a new one.

• Line 247: You refer to PDSA as a personality tool. I really thought it was a tool to assess a change that is implemented. The word personality confused me as well as how this tool could support team growth and team member’s roles.

Response: The PDSA tool that we have developed is a tool that does assess personalities, based on the quality improvement technique that you refer to. We have added a sentence that we believe clarifies the use of the tool.

Results:

• Table 5: Define the “N” as the number of programs that provided data for a give category. Maybe in a footnote?

Response: Excellent point. We have added a note to the table legend to reflect the N is the number of programs that provided data for the quality indicator.

• Table 5: Please clearly differentiate between reporting of results “presumptive positive for time critical disorders” “presumptive positive for non-time critical disorders”, and reporting of “all (normal and presumptive positive) screening results” OR reporting of “all normal results”?

Response: Thank you for noting this, we have made the changes as suggested.

• Line 323: I just want to make sure the difference between each of the three categories (time critical/non-time critical/all) is clearly communicated. Plus…maybe this in inherently obvious…but I think it is necessary to state again…that the reporting of presumptive positive NBS results doesn’t mean that the patient is affected with the disease.

Response: The definitions of time-critical, non-time-critical are provided in the background of the document. We have added a sentence to the legend in Table 5 to define presumptive positive results.

• Figure 2: Image is not clear/fuzzy. And it could also be larger. But maybe this will be addressed during the formatting process.

Response: Thank you, we will confirm that the figure is appropriately formatted upon final submission.

• Line 344: “did not demonstrate change” OR “did demonstrate change”. It appears that the percentage when up slightly…but maybe this is not a significant change?

Response: The reviewer is correct, the program median for reporting was not significantly different (88.9 to 89.5% across the 3 years).

• Table 7: This table is a bit confusing. For the six laboratories open 7 days a week, it is correct to read that 3 laboratories accession specimens all seven days and 3 laboratories accession specimens ONLY on Saturday? For the laboratory only open Monday through Friday, why do they have activities such as receiving specimens on Saturday? I believe I know what you mean to convey…but it is not inherently clear.

Response: Thank you for this comment. We have revised the table to make is less confusing. Specifically:

- We added M-F to the columns that indicate that testing occurs on Saturday or Sunday (M-F and Saturday, M-F and Sunday).

- We revised the table to reflect that the number of days a laboratory is open is defined by the number of days the laboratory tests specimens.

• Line 407-408: The last sentence regarding the overall median for all results reported measures, comparing external versus state lab, is hard to appreciate from figure 5. It appears that the median would be closer to 80% for external labs. Maybe I would remove this sentence or clarify.

Response: This sentence was removed as suggested.

Discussion:

• General comment: I really appreciate how you linked specific programmatic activities to the potential improvement seen within certain metrics.

Response: Thank you!

• Line 521: Include within the limitations section of the discussion how second tier (or even third tier) testing (molecular or biochem) performed by laboratories may impact the time to reporting, even for critical conditions.

Response: Thank you for this comment. This is an important word of caution for newborn screening programs, we have added it to the limitation section.

Specific Comments from Reviewer #2:

As suggestions I would like to put forward that I think the manuscript might be in use of extra context. The relevance of the topic is undisputed, but if the benchmark and the international context is presented more clearly, the reader would be able to grasp the relevance better. For example:

� How many cases are missed due to lack of timeliness? Which is hard to make concrete, because it would also require post-mortem genetic testing?

Response: There have been some reports of tragic cases in which an infant has died or had poor outcomes due to a delay in newborn screening reporting, however it is hard to really assess this in a systematic way.

� What was the starting point of timeliness; there is mentioning of increasing time-critical disorders, but is this due to the type of disorders added to the RUSP or the revision of what a time-critical disorder is?

Response: We added more details to the section regarding the expansion of the panel of NBS disorders (in the Introduction, Section on Timeliness in Newborn Screening) to reflect that the change really occurred with the introduction of MS/MS)

� How is this embedded or does it compare to international systems? Not all countries have birthing facilities / different facilities / deliveries at home, and the time of collection also differs quite a lot between countries.

Response: This manuscript is definitely focused on the US systems. Many other countries do not screen for disorders using MS/MS and therefore they do not have the same urgency for early detection. Further, as the reviewer points out, other countries may have more babies born outside of traditional birthing facilities, and may also have newborn screen specimens collected at different times, coinciding with a home-visit from a nurse. These differences are beyond the scope of this manuscript.

� Was an implementation theory or model used? How did all parts fit into that?

Response: No implementation theory or model was used but state activities addressed structural characteristics and the NewSTEPs 360 team provided ongoing implementation support activities (e.g. coaching) and building CQI self-efficacy for the NBS team

� Why was the selection based on a competitive application process?

Response: Participation was based on a competitive process because each program had to demonstrate that they were prepared to make changes in their program and collect data to monitor progress.

� How is the quality and standardization of data in the repository ensured?

Response: The data entered into the NewSTEPs repository are checked via data validation on each field upon data entry, and data inspection by the NewSTEPs staff, and NBS programs are queried to clarify identified outliers or suspected errors.

• Furthermore, I think the manuscript could be quite a bit shorter on the results if the tables are made more legible. Especially tables 5 and 6 are hard to read, because as a reader you are not always sure what you are looking at. Especially combined with the figures and the text, it seems there is a lot of double information, but the sources (how many labs answered which questions, i.e. how was dealt with missing values) are not always clear while potential stratification and/or combination of the variables is not made. Perhaps some information is better to put into tables instead of figures?

Response: We have edited tables 5 and 6 to be clearer. We believe the stratification of the data into the tables and figures is important as there are nuanes in these data that are important in the interpretation of the results of the study.

• Taking the above into account, I think it is important and interesting work, but to be an international scientific publication it seems to be in need of a better handle on highlighting only or more of the really interesting results to the reader; what tables and figures are really relevant?, and better embedding in international scientific literature, unless it is more or less solely aimed at US readers, but still evidence-based implementation theories / model could be applied.

Response: The ACHDNC guidelines are intended for U.S. newborn screening programs, and many of the processes that are described in the manuscript are unique to the U.S.; however, we believe that many lessons can be gleaned for international audiences. We have better described the tables and figures and we believe that the figures are necessary for the reader to understand the nuances of NBS programs in the U.S.

Specific Comments from Reviewer #3:

• Line 180: Participants in Newsteps. Selecting states is not selecting others. Can you explain what states were not selected, why, and whether this could introduce selection bias? (e.g. at limitations of study).

Response: We have added a sentence to the limitations section to indicate that the application to NewSTEPs 360 could introduce a selection bias.

• Line 407: overall median was observed. Median of what?

Response: This sentence was deleted in response to comment from Reviewer 1.

• Line 491 -500.

I would like to know whether at the site of the laboratory, these CQI reports were generated automatically, querying the databases of the individual laboratories.. This information is important as it superior to manual transfer of data. If the data is entered manually in the files to upload in the repository, is it curated or checked at the site of the repository? This also applies on ln 204-section Quality Indicator data.

Response: The reviewer has identified a critical point in the sharing of electronic data. Unfortunately, there are no methods available to automatically query the databases of all of the individual laboratories. The NewSTEPs team has made progress within some states and some of the Laboratory Information Management Systems (LIMS) vendors have individual solutions, this is not available broadly. Data checks are also implemented for data uploaded using automated upload tools.

• The discussion is quite lengthy and contains repeats of what was already said in the reminder of the paper. Moreover, the discussion is used to describe developments within the programs (e.g. a lengthy description of the efforts made to aIso work on weekend days feel that the authors can be more concise and ( Lines 512-520This is true, but does not relate to the study results.) . I feel the Conclusions section is a double of what has been said before in the paper, especially the discussion.

Response: Thank you for noting this. We have shortened both the conclusion and discussion section in light of your suggestions and we believe it reads better in its more concise form.

Submitted filename: Response to Reviewers Sontag January 2 2020.docx

Click here for additional data file.

7 Feb 2020

PONE-D-19-27133R1

Newborn screening timeliness quality improvement initiative: Impact of national recommendations and data repository

PLOS ONE

Dear Dr Marci K Sontag and co-authors,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process and make the clarifications as asked for.

We would appreciate receiving your revised manuscript by Mar 23 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Jacobus P. van Wouwe, MD PhD

Academic Editor

PLOS ONE

Additional Editor Comments (if provided):

Please answer the issues raised by the reviewer to increase the readability of your manucript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The revised manuscript and remarks from the authors addressed my initial concerns. Below are some additional edits for the authors to consider.

Line 102

o Consider rewording to “in the first week of life”, instead of “first days or weeks of life”. Current recommendations target identification within the first week of life.

Line 113 (figure 1)

o Consider adding numbers to the figure to link the legend. For example, labeling the box within the figure “QI Solution 1”, which would link to the “1)” in the legend.

Line 157

o Consider rewording to “…barriers to timely NBS through technical and financial assistance”. I don’t understand what you mean by “utilization of technology”, unless you are referring to the visualization of performance metrics, quality indicators?

Line 210

o State that role-based access control was determined/granted at the individual NBS program level.

Line 220-224

o Wording is a bit unclear and could be made more concise

Line 268 (table 4)

o I “want” to link the various strategies (education, courier, NBS laboratory, etc) to the QI solutions listed in figure 1. It is possible to reconfigure this table to assign each strategy to the five QI solutions, highlighted in table 1.

Line 310 (table 5)

o I think the year should be “2016” instead of “2015”

Line 313 (figure 2)

o I am not sure if showing the lines on the graph for each individual state (connecting between the years and box plots) adds additional value. It may be more “clean” to only show the box and whisker plots. But this is just a preference…

Line 377-379

o These couple of sentences were confusing. The cases with confirmed diagnosis were identified between 2016-2018, so please consider including 2018 data to lines 377 to 379. Is there a different way to visualize the data, rather than text? Another table perhaps? Or maybe just consider removing, as you have the summary information above in table 6.

Line 394

o Spelling error “operations”

Line 394 (table 7)

o I am still struggling with table 7. For example, it doesn’t make sense that a state would report testing 7 days a week…but only receiving specimen on six days (M-F and Saturday) OR only reporting non-time critical results on Monday through Friday.

o Because it is difficult to make sense of the table and because it is somewhat arbitrary (based upon reported practices of a given state), I think I would remove the table.

o The text can remain as is…except maybe to highlight that each state “reported whether they were open 5, 6, or 7 days” and that “activities performed on a given day, vary between states”.

o If you wanted to keep the table, please add additional sentences to the results or discussion that suggest how various activities performed 5, 6, or 7 days a week may have impacted the timeliness.

Line 405

o What do you mean by “laboratories open seven days demonstrated better outcomes than those open six or five days”? What is the “outcome” by which you used to evaluate whether a state performed better.

Line 420-426 and 430-437

o I don’t think it is necessary to include this information on whether it was a state laboratory or an external laboratory, unless you are trying to make the argument that “people might perceive that one is better than another…but in reality, both state labs and private labs can achieve the same success”? Again, just trying to understand the value of including this section.

Line 451-460

o Again, can we link these six different approaches listed to the 5 different QI solutions? Also see Line 268 (table 4) comment above.

Line 463

o Spacing error in text

Line 520

o Instead of the words “consult parents”, maybe consider using “educate parents”.

Lines 522-528

o Wording is a bit unclear and could be made more concise

Reviewer #3: The authors have skillfully taken care of my comments. I thank them for their work and congratulate them with their work.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: Newborn screening timeliness quality improvement initiative_version 2 comments.doc

Click here for additional data file.

11 Mar 2020

• Line 102

o Consider rewording to “in the first week of life”, instead of “first days or weeks of life”. Current recommendations target identification within the first week of life.

RESPONSE: Changed as suggested

• Line 113 (figure 1)

o Consider adding numbers to the figure to link the legend. For example, labeling the box within the figure “QI Solution 1”, which would link to the “1)” in the legend.

RESPONSE: We have made the change to number to QI solutions as suggested, here and throughout the manuscript. Thank you for this suggestion.

• Line 157

o Consider rewording to “…barriers to timely NBS through technical and financial assistance”. I don’t understand what you mean by “utilization of technology”, unless you are referring to the visualization of performance metrics, quality indicators?

RESPONSE: Changed as suggested

• Line 210

o State that role-based access control was determined/granted at the individual NBS program level.

RESPONSE: Changed as suggested

• Line 220-224

o Wording is a bit unclear and could be made more concise

Response: Changed as reflected here: Benchmarks were adapted directly from the ACHDNC recommendations. Few newborn screening programs were able to meet the ambitious 24-hours from specimen collection to receipt by laboratory benchmark set by the ACHDNC. In response to this, NewSTEPs created an additional benchmark of two calendar days to assess time elapsed from specimen collection to receipt at the laboratory as an intermediary step.

• Line 268 (table 4)

o I “want” to link the various strategies (education, courier, NBS laboratory, etc) to the QI solutions listed in figure 1. It is possible to reconfigure this table to assign each strategy to the five QI solutions, highlighted in table 1.

Response: This is a great suggestion. We have modified Table 4 to reflect the numbered QI solutions in Figure 1.

• Line 310 (table 5)

o I think the year should be “2016” instead of “2015”

Response: You are correct. Thank you for catching this.

• Line 313 (figure 2)

o I am not sure if showing the lines on the graph for each individual state (connecting between the years and box plots) adds additional value. It may be more “clean” to only show the box and whisker plots. But this is just a preference…

Response: We have left this figure in its initial format as we believe the changes in individual programs is important in quality improvement projects such as this one.

• Line 377-379

o These couple of sentences were confusing. The cases with confirmed diagnosis were identified between 2016-2018, so please consider including 2018 data to lines 377 to 379. Is there a different way to visualize the data, rather than text? Another table perhaps? Or maybe just consider removing, as you have the summary information above in table 6.

Response: We have removed these sentences as suggested. They do not significantly add to the content.

• Line 394

o Spelling error “operations”

RESPONSE: Changed as suggested

• Line 394 (table 7)

o I am still struggling with table 7. For example, it doesn’t make sense that a state would report testing 7 days a week…but only receiving specimen on six days (M-F and Saturday) OR only reporting non-time critical results on Monday through Friday.

o Because it is difficult to make sense of the table and because it is somewhat arbitrary (based upon reported practices of a given state), I think I would remove the table.

o The text can remain as is…except maybe to highlight that each state “reported whether they were open 5, 6, or 7 days” and that “activities performed on a given day, vary between states”.

o If you wanted to keep the table, please add additional sentences to the results or discussion that suggest how various activities performed 5, 6, or 7 days a week may have impacted the timeliness.

o

RESPONSE: We believe this table is an important table in the context of this manuscript. We have added text in the results section to describe some of the nuances of individual programs, and we have made additional edits as suggested by the reviewer to highlight that the laboratory hours are reported by an individual patient and that daily activities vary between states.

• Line 405

o What do you mean by “laboratories open seven days demonstrated better outcomes than those open six or five days”? What is the “outcome” by which you used to evaluate whether a state performed better.

RESPONSE: The outcome highlighted here is result reporting. We have changed the text to remove the ambiguity. Thank you for identifying this.

• Line 420-426 and 430-437 - I don’t think it is necessary to include this information on whether it was a state laboratory or an external laboratory, unless you are trying to make the argument that “people might perceive that one is better than another…but in reality, both state labs and private labs can achieve the same success”? Again, just trying to understand the value of including this section.

Response: The reviewer brings up a good point. We have modified this section to indicate that both state and private labs can achieve good timeliness outcomes. We believe that this is important to note as there are a number of stakeholders that would like to suggest the privatization of NBS programs.

• Line 451-460

o Again, can we link these six different approaches listed to the 5 different QI solutions? Also see Line 268 (table 4) comment above.

Response: We rearranged the numbering and modified the wording to be consistent with the approaches listed in Figure 1. We have not called it out directly, but it is now in parallel which we believe will be more satisfying for the astute reader.

• Line 463

o Spacing error in text

RESPONSE: Changed as suggested

• Line 520

o Instead of the words “consult parents”, maybe consider using “educate parents”.

RESPONSE: Changed as suggested

• Lines 522-528

o Wording is a bit unclear and could be made more concise

RESPONSE: We modified the paragraph to be more concise.

Submitted filename: Response to Reviewers Sontag March 2020.docx

Click here for additional data file.

16 Mar 2020

Newborn screening timeliness quality improvement initiative: Impact of national recommendations and data repository

PONE-D-19-27133R2

Dear Dr. Marci K. Sontag and co-workers,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Jacobus P. van Wouwe, MD PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Although the line numbers did not match, by the color of the not too many changes it is clear that the reviewers' suggestions have been followed and the manuscript has likewise improved.

Reviewers' comments:

18 Mar 2020

PONE-D-19-27133R2

Newborn screening timeliness quality improvement initiative: Impact of national recommendations and data repository

Dear Dr. Sontag:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Jacobus P. van Wouwe

Academic Editor

PLOS ONE