Novel two-tiered developmental screening programme for Singaporean toddlers: a quality improvement report.

Early identification of developmental delays with timely intervention, especially before the age of 3 years, can improve child development. In Singapore, however, diagnosis and intervention for developmental delays occur at a median age of 44 months. As early detection and intervention depends on an effective developmental screening programme, we aimed to improve the detection of developmental delays before the age of 3 years in a primary care setting. We did this by implementing a novel two-tiered screening programme which uses three standardised screening tools (Parents' Evaluation of Developmental Status, PEDS-Developmental Milestones and Ages and Stages Questionnaire-3). We used quality improvement methods to integrate and optimise this two-tiered programme into the existing 9-month and 18-month screening schedule, with an additional screening at 30 months to replace the pre-existing 36-month screening of the National Child Health Surveillance Programme. A total of three Plan-Do-Study-Act cycles were performed to ensure programme feasibility and sustainability. They focused on adequately training the primary care nurses, targeting an 80% screening rate and aiming for 20 min screening tool administration time per child. We assessed the proportion of children referred to the child development units after positive screening for developmental concerns under the new programme, with a pre-post and with-without intervention comparison, and reviewed the screening rates and screening tool administration time. The proportion of 18-month old children referred for developmental concerns improved from 3.5%-7.1% over a 6-month period. For those who received further assessment by developmental specialists after the two-tiered screening, 100% received a definitive diagnosis of developmental delays, similar to the situation before programme introduction. Our quality improvement efforts facilitated successful integration of the two-tiered programme into the pre-existing screening schedule with minimal impact to the clinic workflow. While we highlight challenges in implementation that need to be addressed, our findings support a potential nationwide adoption of the two-tiered programme. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Problem

Children with developmental delays receive the maximum benefit if they receive appropriate intervention before the age of 3 years.1–3 With the pre-existing model of developmental surveillance in Singapore, however, children with suspected developmental delays first consult a developmental specialist at a median age of 44 months4 to undergo a detailed developmental assessment and diagnosis, and only then receive intervention. One major reason for such a delay is the challenge of accurately identifying developmental delays early in childhood.5 6

Our quality improvement (QI) team agreed that improvements made to the pre-existing model of developmental surveillance would enable earlier identification of developmental delays thereby triggering early intervention. In Singapore, the pre-existing proportion of children seen in primary care who screen positive for potential developmental delays is about 5%. However, it is estimated that the global prevalence of developmental delays for preschool children is around 10%–15%.7 8 Hence, the QI team aimed to increase the proportion of children seen in primary care who screen positive for potential developmental delays by 5% before the age of 3 years by implementing a novel two-tiered developmental screening programme, using three standardised parental-concern based screening tools.

The study was conducted at Punggol Polyclinic, a large public primary care centre (one of eight such facilities) in the Eastern cluster of the Singapore healthcare system. Compared with other primary care centres in Singapore, Punggol Polyclinic serves a young population and cares for the highest proportion of young children with 11.1% of its residents under the age of 4.9 It has its own maternal and child health section, separate from the acute care clinics and the adult chronic disease management clinics. Punggol Polyclinic saw about 5000 developmental assessment visits for children below the age of 4 years in 2018; this accounted for approximately 20% of all such visits across the eight primary care centres in the eastern cluster.10 Similar to all other government funded primary care facilities, the patient turnover is high and the healthcare team is required to cater to a large patient load with time constraints. Under these settings, the primary care nurses conduct the developmental screening within a set period of 20 min per visit per child. The QI team uses the Centers for Disease Control and Prevention Framework for Programme Evaluation11 and QI tools to design and evaluate the programme.

Background

Worldwide, the estimated prevalence of developmental delays is 10%–15%.7 8 Developmental delays have important negative long-term sequelae in cognitive, psychiatric and academic achievements for each child.12 13 Early identification and intervention for developmental delays reaps large socioeconomic benefits ranging from US$30 000 to US$100 000 per child,1 reduces special education needs by 14% and increases childhood average IQ scores by around 6.5 points.14 To optimise the developmental potential of children with developmental delays, early intervention should be implemented during the first 3 years of life while the brain possesses maximum plasticity and remodelling capability.1–3

Parental concern is instrumental in identifying developmental problems.15–18 Being actively involved in the care of their children, parents are highly aware of their child’s developmental strengths and weaknesses. Hence, a developmental history obtained from parents is useful as a prescreening tool in identifying children who would require specialist evaluation.19 The American Academy of Pediatrics (AAP) has recommended standardised developmental screening tools, based on reports of parental concerns, for primary care settings which include Ages and Stages Questionnaire (ASQ) and Parents’ Evaluation of Developmental Status (PEDS).20

In Singapore, there are six touch points at the primary care centres when developmental surveillance is carried out under the National Child Health Surveillance Programme. These occur within the following age ranges: 4–8 weeks, 3–5 months, 6–12 months, 15–18 months, 2–3 years and 4–6 years. Such visits at government funded primary care centres are fully subsidised. Our pre-existing standard of care is based on an age-specific developmental checklist, found within a health booklet that every child born in Singapore owns, which contains an abridged version of the Denver Developmental Screening Test-II (DDST-II).21 However, the DDST-II has been shown to have a wide-ranging sensitivity of 50%–100% and specificity of 25%–69%22–25 resulting in high false negative and high false positive rates, respectively, making it suboptimal as a screening tool. In addition, only about half of the parents reportedly complete this self-administered checklist before every developmental visit.26 If parents failed to do so, the primary care nurse would fill in the checklist by checking with the accompanying caregivers. If there were any developmental concerns identified using this checklist, the child would be referred to one of the two child development units in Singapore for further assessment. An average of a 2-month waiting time to see a developmental specialist at the child development unit may also contribute to the delay for a definite diagnosis and timely intervention.

Measurement

For comparison measurements, we collected aggregate data on the proportion referred to the Department of Child Development (DCD) (among those who screened positive at primary care) between January and December 2018 from Punggol Polyclinic for 9-month and 18-month developmental surveillance visits from the electronic medical records, as well as the proportion of children who defaulted the DCD appointment (among those referred for assessment) and the final diagnosis from the DCD. DCD, located at KK Women’s and Children’s Hospital, Singapore, is the larger of the two child development units in Singapore and receives the vast majority of the referrals from the primary care centres in the Eastern cluster (100% of Punggol Polyclinic referrals in 2018). This served as our retrospective cohort. Data from this retrospective cohort showed that of the 3085 children seen at 9 months and 2074 children seen at 18 months of age, 53 (1.72%) and 73 (3.52%) were referred to DCD, respectively. As standard developmental surveillance was done at 36 months of age, there was no comparison data available for 30-month visits. The proportion defaulting DCD appointment was 50% for the 9-month referrals and 47% for the 18-month referrals in the retrospective cohort. The QI team extracted these comparison data using unique visit and charge codes for developmental assessment visits to ensure specificity and reliability of the aggregate data. In addition, we assigned a data person from the primary care centre to ensure the raw extracted data did not contain any missing information prior to analysis by the QI team.

For the prospective measurements, we collected prospective data after the two-tiered screening was implemented at Punggol Polyclinic. For pre–post intervention analysis, we defined the primary outcomes of intervention as the proportion referred to DCD after positive screening at 9 months, 18 months and 30 months and the proportion who defaulted DCD appointment. To do a with–without comparison, we collected aggregate data on the proportion referred to DCD from Punggol Polyclinic and compared this with the proportion referred from the rest of the government-funded primary care centres in the Eastern cluster, preimplementation and postimplementation of the two-tiered screening. We ensured the quality of the prospective cohort data by regularly monitoring the data integrity of the prospective database before extraction. For both prospective and retrospective cohorts, a child could only be seen at DCD after a referral has been made through the electronic health records, where all our referrals were electronically documented and periodically tracked.

For process measure, we took note that the time constraint due to the high patient turnover was an important consideration, especially when each development assessment clinic slot was allocated an average of 20 min per child. We hypothesised, however, that the new two-tiered screening programme would keep the screening tool administration time within 20 min if the nurses were well trained.

Design

It was clear that the positive screening proportion for developmental delays should be improved so that children could be referred earlier for further assessment and intervention. A multidisciplinary QI team comprising nursing and medical leadership at the primary care centre and paediatricians from the largest paediatric specialist centre in Singapore was formed. The overall objective of the stakeholders was to improve the detection of developmental delays before the age of 3 years under primary care settings via a new screening programme, which uses AAP’s recommended screening tools through the logic model (online supplemental data 1). The QI team agreed on the process and outcome measures. The programme consisted of an initial 6-month planning and training phase for the new developmental screening tools—PEDS,27 PEDS- Developmental Milestones (PEDS-DM)28 and ASQ-329 at the recommended ages of 9, 18 and 30 months, as per AAP’s recommendation. PEDS elicits general concerns regarding learning, development and behaviour and, specific concerns in each developmental domain. PEDS-DM asks parents about their child’s abilities or provides simple tasks for the child to complete (online supplemental data 2).30 A lower than age-equivalent score in each domain is considered failure in that domain. ASQ-3 assesses five developmental domains and responses of ‘yes’, ‘sometimes’ and ‘not yet’ for questions under each domain are scored as 10, 5 and 0 points, respectively. A score of 2 SD below the population mean is indicative of developmental delay in that domain. Although these instruments have not been standardised for Singapore, ASQ which has been recently validated,31 is currently used in many local research projects32 and as a screening tool in DCD.33 PEDS has also been evaluated in Singapore using different languages34 and under a different setting with preschool teachers.35

Initially, the QI team planned to implement all three screening tools for every child. However, after the primary care centre nurses underwent training on the administration of these screening tools, most of them found ASQ-3 to be complicated with long administration time, and were more comfortable using the PEDS and PEDS-DM. On the other hand, ASQ-3 complements PEDS and PEDS-DM in several developmental domains. Thus, the QI team devised a two-tiered screening programme (online supplemental data 3), with graded training for the tier 1 and tier 2 nurses. All children were screened at tier 1 (basic screen) using PEDS and PEDS-DM. We defined a positive screen at tier 1 as having any PEDS predictive concern and failure in any PEDS-DM domain. Children screened positive at tier 1 received further screening at tier 2 (advanced screen) using ASQ-3. We defined a positive screen at tier 2 as a developmental delay in any ASQ-3 domain. Children who screened positive at tier 2 were then referred to developmental specialists at DCD. The developmental specialists used a combination of the Developmental Profile, Third Edition and their clinical assessment as the gold standard of diagnosis. Children screened negative at tier 2 were then given developmental monitoring advice and scheduled for their next age-appropriate developmental assessment visit.

The QI team met every month for 2 hours to discuss the operational and work flow issues consisting of (1) ease of flow of patients from tier 1 to tier 2, (2) time taken for and challenges with administration of the screening tools and (3) workload of the staff involved through the planning and implementation phases.

Strategy

The QI team’s overall objective was to improve the detection of developmental delays before the age of 3 years by implementing a novel two-tiered developmental screening programme into the pre-existing National Child Health Surveillance Programme. We aimed to evaluate this programme using QI methods by measuring the proportion of children referred to DCD after screening positive for developmental delays under the new programme with a pre–post and with–without comparison. Our SMART aim was to improve the positive screening proportion for developmental delays by 5% before the age of 3 years under primary care settings. Due to the lack of comparison group for the new two-tiered screening programme at 30 months, we did a pre–post and with–without comparison for the 9-month and the 18-month screening. We hypothesised that the two-tiered screening would increase the positive screening proportion by 5% without compromising on the usual clinic workflow of the primary care centre.

For the prospective cohort, the baseline demographics (gender, race, maternal education, income status), birth history (birth weight and gestational age) and primary diagnoses of the children referred to and assessed by developmental specialists at DCD were collected. For the retrospective cohort, we were only able to obtain the aggregate number and primary diagnoses of the children referred to and assessed by developmental specialists at DCD. Income was considered low if the household lived in a rental flat or received government healthcare subsidy. Low birth weight and preterm were defined as less than 2500 g of birth weight and less than 37 weeks of gestational age, respectively. Maternal education status was dichotomised to high (defined as possessing a university degree and above) versus low.

As process outcomes, we aimed to (1) train 100% of the primary care nurses at the Punggol Polyclinic with tier 1 screening and 2 senior nurses with tier 2 screening, (2) screen at least 80% of the attendances at the 9 months, 18 months and 30 months developmental assessment visits under the two-tiered programme and (3) keep the average screening tool administration time within 20 min per child. A total of three Plan–Do–Study–Act (PDSA) cycles were performed for the process measures.

PDSA cycle 1

Our first intervention was to get the primary care nurses trained on PEDS, PEDS-DM and ASQ-3 as well as to conduct a 1-month pilot study prior to the actual implementation to find out the impact of introducing a new set of screening tools during the busy clinic hours. The DCD team arranged a comprehensive training programme for the nurses. The training programme’s aim was to enhance the knowledge about the three screening tools and observe the nurses administer these tools at DCD. This direct supervision ensured that the nurses were competent in the administration of the tools before they were certified to do so independently. Subsequently, the QI team introduced a 1-month pilot launch of the two-tiered programme where the general paediatricians observed the nurses integrating the new tools into the pre-existing clinic workflow. This pilot launch allowed further fine-tuning of the new screening workflow and helped to improve the confidence level and competency of the nurses. The nurses were ready for full implementation at the end of the pilot period. This PDSA cycle primarily tested the feasibility of administering the age-specific screening tools in the busy clinic setting and provided feedback to go through to the next cycle of full implementation.

PDSA cycle 2

The two-tiered screening could only be implemented on weekdays (the clinic also runs half days on Saturdays) due to manpower constraints and we aimed to screen at least 80% of those Singapore citizens’ and permanent residents’ children. This was to fulfil the goal of incorporating the two-tiered programme broadly. The screening percentages (number screened by number attending the developmental assessment clinics) were tracked for the 9-month and 18-month visits. We did not track the 30-month developmental screening numbers since the number of children turning up for this developmental assessment appointment was small. For the initial 2 months after implementation in July 2019, the screening percentages were below 80%. This was a result of late appointments made for development screening for which the two-tiered screening could not be successfully carried out in a timely manner. The clinic workflow was then altered such that children scheduled for 9-month and 18-month developmental visits were given earlier appointment slots in the day compared with other routine developmental surveillance visits. With this, the two-tiered screening percentages subsequently improved to greater than 80% consistently for both the 9 months and 18 months (online supplemental data 4).

PDSA cycle 3

As time is a contextual constraint, our third PDSA cycle aimed to keep the average screening time for the two-tier screening to 20 min for long-term feasibility and sustainability. We monitored the average screening time needed during the initial implementation months. With time, the average time taken per screening demonstrated a decrease to a level within our target of 20 min (online supplemental data 5). Continued education and feedback during the monthly meetings ensured that the screening tool administration and referral processes were seamless.

Results

Postintervention outcomes in the prospective cohort were assessed at the end of each PDSA cycles as well as after 6 months of the two-tiered developmental screening implementation.

Increased proportion of children referred for specialist assessment at 18 months

Comparing the pre–post data (before vs after implementation) of the two-tiered screening, at 9-month screening, our retrospective cohort had a higher proportion of referred children compared with the prospective cohort (53/3085, 1.7%, 95% CI (1.3% to 2.3%)2 3 compared with 4/972, 0.4%, 95% CI (0.01% to 0.8%), p=0.003) (table 1). However, there was an increased proportion of children referred at 18 months in the prospective cohort compared with the retrospective cohort (49/687, 7.1%, 95% CI (5.2% to 9.1%) compared with 73/2074, 3.5%, 95% CI (2.7% to 4.3%), p<0.001). A higher proportion of 18 month children was also assessed by the specialist for developmental concerns in the prospective cohort compared with the retrospective cohort (21/687, 3.1%, 95% CI (1.8% to 4.4%) compared with 34/2074, 1.6%, 95% CI (1.1% to 2.1%, p=0.02).

Table 1

The proportion of children referred for specialist assessment in the prospective and retrospective cohorts at ages 9, 18 and 30 months

Age (months)	Retrospective cohort (January–December 2018)				Prospective cohort (July–December 2019)		% difference (95% CI)	P value
	N	N Ref	% (95% CI)	N	N Ref	% (95% CI)
9	3085	53	1.7% (1.3% to 2.3%)	972	4	0.4% (0.01% to 0.8%)	−1.31% (−2.16% to −0.46%)	0.003
18	2074	73	3.5% (2.7% to 4.3%)	687	49	7.1% (5.2% to 1.9%)	3.61% (1.84% to 5.38%)	<0.001
30	NA	51	–	41	4	9.76	–	–

95% CI, calculated using normal approximation (z-statistic); N, Total number of children who were screened for developmental concerns at the primary care center; NA, not available; NRef, Number of children referred for specialist assessment.

In terms of the with–without intervention analysis (figure 1), the referred proportion at 18 months was also significantly higher at Punggol Polyclinic where the two-tiered screening was implemented as compared with other primary care centres who were using the standard-of-care screening (7.8%, 95% CI (6.1% to 9.5%) compared with 3.2%, 95% CI (3.1% to 6.1%), p<0.001). For the 9-month group, the higher proportion of referrals from Punggol Polyclinic compared with other primary care centres was not statistically significant (1.1%, 95% CI (−0.03% to 2.2%) compared with 0.6%, 95% CI (0.1% to 1.1%), p=0.11). In addition, given that the pre–post analysis for 9 months did not reveal significant benefit, the QI team decided to discontinue the two-tiered screening for 9 months.

Figure 1

With–without intervention analysis for two-tiered screening programme at (A) 9 months (B) 18 months. Others: Other primary care centres in the Eastern Cluster of Singapore Healthcare system. H1: First half of fiscal year. H2: Second half of fiscal year. DCD, Department of Child Development.

Proportion of referred children correctly diagnosed with developmental delays

At 18-month screening, among the 687 children screened, 67 (9.8%) screened positive (figure 2). However, only 49 (7.1% of children screened, 73.1% of screened positive) were referred for further assessment as 18 declined specialist referral. Of the 49 referred, 21 (42.9% of referrals) kept their specialist appointment with all 21 resulting in a final developmental delay diagnosis. Hence, for the two-tiered programme, of those referred who kept their appointments, 100% (95% CI 80.5% to 100%) were diagnosed with a developmental delay, which was similar to that of the retrospective cohort’s result of 100% (95% CI 89.7% to 100%). For the prospective cohort, this proportion was only calculated for 18-month group, as the children referred for further assessment in the 9-month (online supplemental data 6) and 30-month groups were too few.

Figure 2

Summary statistics for the two-tiered screening programme for 18-month group.

We compared the prospective cohort’s demographic characteristics (gender, ethnicity, income status, maternal education, birth weight and gestational age) between those who accepted referrals vs declined referrals to a specialist after being screened positive (online supplemental data 7), and those who kept versus missed their DCD specialist appointments (online supplemental data 8), but these comparisons did not show any significant differences.

Comparing primary diagnoses between prospective and retrospective cohorts

Developmental language and speech disorders were the most common diagnoses in both cohorts, followed by pervasive developmental disorder (online supplemental data 9). When compared with the retrospective cohort, there was a larger proportion of developmental language and speech disorders in the prospective cohort with a smaller proportion of pervasive developmental disorder. However, the primary diagnosis distributions of the two cohorts were not statistically different (p=0.17).

Lessons and limitations

The QI team successfully integrated a novel two-tiered developmental screening programme into the pre-existing National Child Health Surveillance Programme, with minimal negative impact to the primary care clinic workflow, while increasing the proportion of 18-month-old children referred for concerns of developmental delays. This study was the first to combine three standardised parental-concern-based screening tools into a single two-tiered screening programme in a primary care setting. Although we demonstrated an increased referral proportion for the 18-month old children, a more holistic assessment of the programme’s effectiveness was limited by the relatively high refusal proportion for onward referral to the specialists at DCD (26.9%) among those who screened positive at the primary care centre and the high proportion who defaulted follow-up (57.1%) at DCD. A similar default proportion was also observed in the retrospective cohort at DCD. We conducted brief follow-up phone call surveys of the parents who missed specialist appointments to ensure continuity of care. Out of the 82% of parents who responded to the phone calls, 40% cited perceived resolution of developmental concerns as the reason for missing the DCD appointment. A review clinic will be scheduled at 6 months after the initial screening, for children who were screened positive but refused referral to or defaulted appointments at the DCD, to reassess their developmental status. This persistent problem has identified an area of concern for another QI project at the primary care centre to address this group of children who refused and defaulted DCD appointments. We also focused our analyses on the 9-month and 18-month screening as there were few 30-month children screened during the 6 months period, as 30-month screening was a newly introduced developmental assessment time point.

We did not evaluate the negative predictive value of the two-tiered screening programme as only those who screened positive were referred for further evaluation. Logistically, given the large number of children seen in the primary care centre, it was not feasible to assess the developmental status of the negatively screened children in more detail. The positive predictive value could not be effectively studied due to the high refusal and default proportion for appointments at DCD, which was not different from the retrospective cohort. However, for those who received formal assessment at DCD, all who had positive screening at 18 months of age were diagnosed to have developmental delays and received intervention.

Another limitation in our study design was the use of a retrospective database as our comparison measurement, which subjects to case ascertainment and data quality between the two cohorts. However, these differences are unlikely to be large as the data collected for the two cohorts were just 6 months apart and we used stringent and consistent criteria to extract the required data from electronic medical records for both the retrospective and prospective cohorts.

Conclusion

Instituting a novel two-tiered developmental screening programme in a primary care setting has increased the proportion of developmental specialist referrals for 18-month-old children with little negative impact to the current clinic workflow. Of those referred who presented to the specialist, all were diagnosed with developmental delays, similar to the situation in the comparison group (the retrospective cohort). In addition, the nursing staff mastered the new screening tools after the initial 6-month period and was comfortable administering them within the allocated time per child, with minimal impact on the usual flow of the busy clinics. These findings imply the potential for the two-tiered programme to be implemented as a nation-wide initiative to improve early diagnosis of developmental delays at Singapore primary care centres, especially for 18-month-old children. The next step for our QI team will be to assess if the higher proportion of screened positive children translates to earlier diagnosis and intervention, as well as to address the high default proportions at DCD.

The BMJ uses the Vancouver style for referencing. There is an example from this paper entitled; Implementing mobile devices to reduce non-rostered workload for junior doctors or click here for a comprehensive guide to referencing.