Brief Report: Training New Zealand Well Child/Tamariki Ora Nurses on Early Autism Signs Using the Social Attention and Communication Surveillance-Revised.

Universal developmental surveillance is considered best practice for early identification of autism. We analysed data from 175 New Zealand Well-Child/Tamariki Ora nurses who attended a 1-day training in developmental surveillance for autism using the social attention and communication surveillance-revised (SACS-R) tool. We used a survey to measure nurses' knowledge of typical development, knowledge of early signs of autism, general autism knowledge, and confidence in identifying and discussing early signs, prior to the workshop, after the workshop, and at follow-up. We measured perceived acceptability of the SACS-R after the workshop and at follow-up. Nurses showed improvements on all measures from pre-workshop to post-workshop and pre-workshop to follow-up. Implementation of the SACS-R across different contexts appears feasible and acceptable.

Autism spectrum conditions (referred to herein as autism) are characterised by difficulties with social communication and the presence of restricted and repetitive behaviours and interests (American Psychiatric Association, 2013). It is estimated that as many as one in 54 children have an autism diagnosis (Maenner et al., 2020). Most children can now be reliably diagnosed by the age of two but the average age of diagnosis remains around 4 years or older (Barbaro & Dissanayake, 2017; Bent et al., 2015; van’t Hof et al., 2020). Early identification is essential in enabling earlier diagnosis and, thus, access to timely evidence-based early intervention (Whitehouse et al., 2019), which promotes child and family health outcomes (Fuller & Kaiser, 2019; Zwaigenbaum et al., 2015).

Universal developmental surveillance is recommended best practice for early identification of autism (American Academy of Pediatrics, 2007). It involves the combined use of clinical observations, standardized autism screening tools, and consideration of parental concerns (Mozolic-Staunton et al., 2020). Child health nurses are ideally placed to conduct developmental surveillance during their regular “well-child checks” in the first 2 to 3 years of a child’s life (Oberklaid et al., 2013). Several studies have examined health professionals’ knowledge of autism and found that it is generally low, regardless of their level of experience (Atun-Einy & Ben-Sasson, 2018; Shrestha et al., 2021Tilahun et al., 2019). Thus, child health nurses may benefit from autism-specific training.

The social attention and communication surveillance-revised (SACS-R) is a well-validated instrument used in universal developmental surveillance for autism (Barbaro & Dissanayake, 2010, 2013; Mozolic-Staunton et al., 2020). These checklists outline key social attention and communication markers associated with a higher likelihood of an autism diagnosis at the ages of 12, 18, and 24 months (Barbaro & Dissanayake, 2010, 2013). They focus exclusively on social attention and communication markers, rather than restricted and repetitive behaviours, as these are the most predictive of an autism diagnosis in this age range (Barbaro & Dissanayake, 2009). Research has been conducted with over 300 Maternal and Child Health (MCH) nurses in Australia who were trained to implement the SACS-R during their routine well-child checks. Barbaro and Dissanayake (2010) found that, when used between 12 and 24 months, the SACS-R had a positive predictive value (PPV) of 81% with estimated sensitivity of 83% and specificity of 99.8%. Mozolic-Staunton et al. (2020) reported similar findings with the SACS-Revised (84% PPV; 99% Negative Predictive Value; 82% Sensitivity; 99% Specificity). This is higher than other common autism screening tools such as the Modified Checklist for Autism in Toddlers (pooled specificity of 51%, sensitivity of 83%, and PPV of 6% in community populations; Yuen et al., 2018) and the autism pathway of the Parents Evaluation of Developmental Status (Mozolic-Staunton et al., 2020).

Nurses appear to view the SACS-R tool and training positively. Barbaro et al. (2011) reported that, immediately following training and at follow-up 6–9 months later, MCH nurses found the SACS-R tool to be useful, positive, and easy to implement. Shreshta et al. (2021) similarly reported that Female Community Health Volunteers in Nepal found the SACS-R tool to be highly acceptable. Following the training, they also improved in their knowledge of typical development and signs of autism, and their confidence in monitoring and referring children with a higher likelihood of autism.

Prior to large-scale implementation, it is important to ensure that tools for developmental surveillance are suited to the needs and context of different settings and countries. In New Zealand (NZ), well-child checks are conducted by Well-Child/Tamariki Ora (WCTO) nurses as a free, universal public health-care service through Plunket and other providers. Therefore, WCTO nurses are ideal candidates to implement developmental surveillance for autism in NZ, although no such programme is currently in place.

The current study aimed to determine whether a 1-day training for WCTO nurses in developmental surveillance for autism using the SACS-R tool was acceptable and feasible in a NZ context. The specific research questions were:

Did nurses’ knowledge of typical social attention and communication development, early signs of autism, and autism more generally improve immediately following the training and at follow-up?

Did nurses’ confidence in administering the SACS-R and in raising concerns with parents improve immediately following the training and at follow-up?

Did the nurses find the training and the SACS-R tool to be acceptable immediately following the training and at follow-up?

Was there any relation between the amount of time participants had spent in their role or as a registered nurse and: (1) their knowledge of autism and typical development, and (2) their confidence in identifying early signs of autism and raising them with parents?”

Method

Ethical Clearance and Informed Consent

Ethical clearance was granted by the Victoria University of Wellington Human Ethics Committee (#27192). The first author explained the study at the start of the workshop, with attendees reading the information sheet prior to consenting. Participation was voluntary and responses were confidential.

Eligibility

To be included participants needed to: (a) have attended one of the workshops, (b) be employed by Plunket or another WCTO provider, and (c) be a registered nurse.

Setting and Context

The first workshop was conducted at the Porirua Campus of Whitirea, Wellington in June 2019, with a further two workshops at the North and South campuses of the Auckland University of Technology, Auckland in November 2019. WCTO nurses work with the majority of NZ infants and conduct checks five times in their first year of life and again at 15–18 months, 2–3 years, and 4 years. All participants at both workshops were invited to participate in the survey, if they met eligibility criteria.

Design

This study used a non-experimental pre-test, post-test, follow-up survey design.

Time-Points

Participants completed the pre- and post-tests surveys on the day of the workshop, while the follow-up survey was distributed 3 months later for the Wellington cohort, and 8 months later for the Auckland cohort. This was due to the initial follow-up falling during a period of Covid-19 lockdown.

Measures

The pre- and post-workshop surveys were primarily completed on paper and the follow-up survey was completed online through Qualtrics. The current survey was adapted from Shrestha et al. (2021). Cronbach’s alpha for the subscales in this study ranged from 0.62 (early signs) to 0.91 (confidence). Fewer questions were included in the Auckland survey than the Wellington survey because, following the Wellington workshop, some nurses reported that the survey was onerous to complete. We retained those questions that were deemed by a developer of the SACS-R (JB), to most accurately reflect the content of the workshop and the tool. Further, for the Knowledge of Early Signs subscale only the Auckland data is reported as the Wellington survey did not include all relevant questions for this subscale. For all other subscales data from both the Wellington and Auckland surveys are reported. Some questions on this subscale were altered for the Auckland workshop because several items related to restricted and repetitive behaviours and were, thus, not deemed to be reflective of overall participant learning. Although restricted and repetitive behaviours were covered in the workshops, the core focus, including all items in the SACS-R checklist, was on social communication. Thus, only those questions included in both the Wellington and Auckland surveys are reported, while the Knowledge of Early Signs subscale is reported for Auckland only as too few questions remained in the Wellington survey once the restricted and repetitive behaviour items were removed. Demographic information was collected in the pre-workshop survey. The survey contained the following subscales (all items are listed in Tables 2 and 3):

Table 2

The percentage of participants who correctly answered each knowledge question at pre-workshop, post-workshop, and follow-up

Item	Pre-workshop (%)	Post-workshop (%)	Follow-up (FU) (%)	DifferencePre to Post (%)	DifferencePre to FU (%)	DifferencePost to FU (%)
Knowledge of typical development
Age when children can speak 5–10 words	55.75	71.34	70.97	+ 15.59	+ 15.22	− 0.37
Age when children engage in pretend play	26.74	48.47	40.32	+ 21.72	+ 13.58	− 8.14
Age when children point to objects	38.73	78.05	54.84	+ 39.32	+ 16.11	− 23.21
Age when children make regular eye contact	84.48	83.85	85.48	− 0.63	+ 1.00	+ 1.63
Age when children respond to name	72.99	62.20	69.35	− 10.79	− 3.63	+ 7.16
Age when children imitate others’ actions	20.69	30.49	24.19	+ 9.80	+ 3.50	− 6.29
Knowledge of early signs (Auckland only)
Key signs
Pointing to share interest	70.52	95.12	91.67	+ 24.60	+ 21.15	− 3.46
Rarely makes spontaneous eye contact	93.68	98.17	100	+ 4.49	+ 6.32	+ 1.83
Does not use age appropriate gestures	72.52	94.35	80.65	+ 21.84	+ 8.13	− 13.71
Behaviours that are not key signs
Delayed gross motor skills	74.05	91.94	93.55	+ 17.89	+ 19.50	+ 1.61
Above average cognitive abilities	65.65	92.74	93.55	+ 27.09	+ 27.90	+ 0.81
Tantrums	75.27	91.46	88.33	+ 16.18	+ 13.05	− 3.13
Hyperactivity	66.09	93.29	90.00	+ 27.20	+ 23.91	− 3.29
General autism knowledge
Correct responses to true statements
Children can be identified by 24 months	48.54	93.29	93.22	+ 44.75	+ 44.68	− 0.07
Occurs more in boys than girls	49.42	92.07	58.33	+ 42.65	+ 8.91	− 33.74
Is a neurodevelopmental condition	70.76	92.59	95.00	+ 21.83	+ 24.24	+ 2.41
Correct responses to false statements
More common in higher SES areas	59.65	84.66	81.67	+ 25.01	+ 22.02	− 3.00
Can be cured	69.46	93.90	98.31	+ 24.44	+ 28.84	+ 4.40
“Just” an intellectual disability	73.53	88.89	88.33	+ 15.36	+ 14.80	− 0.56

SES socioeconomic status

Table 3

Mean Likert Scale Scores and standard deviations for each confidence and acceptability item at pre-workshop (confidence only), post-workshop, and follow-up

	Pre-workshop	Post-workshop	Follow-up (FU)	Mean differencePre to Post	Mean differencePre to FU	Mean differencePost to FU
Confidence
Confidence in identifying signs at 12 months	2.60 (1.24)	5.06 (0.69)	4.77 (0.72)	+ 2.46	+ 2.17	− 0.29
Confidence in identifying signs at 18 months	2.99 (1.26)	5.15 (0.64)	4.91 (0.80)	+ 2.16	+ 1.92	− 0.24
Confidence in identifying signs at 24 months	3.67 (1.25)	5.25 (0.59)	5.15 (0.78)	+ 1.58	+ 1.48	− 0.10
Confidence in identifying signs at 30 months	4.01 (1.28)	5.18 (0.82)	5.17 (0.72)	+ 1.18	+ 1.16	− 0.02
Confidence in discussing concerns with parents	3.55 (1.28)	4.86 (0.70)	4.71 (0.92)	+ 1.31	+ 1.17	− 0.14
Acceptability of the SACS-R tool
Easy to incorporate into practice		5.22 (0.78)	4.78 (0.98)			− 0.44
Will regularly use		5.33 (0.80)	4.74 (1.23)			− 0.59
Help me to understand autism in infancy and toddlerhood		5.62 (0.55)	5.58 (0.05)			− 0.03
Parents will be comfortable with the SAC-R		4.98 (0.66)	4.64 (0.88)			− 0.34
SAC-R is appropriate for Māori whānau		5.23 (0.71)	4.86 (1.06)			− 0.39

Knowledge

The Knowledge of Typical Development subscale included six items regarding the age when children would typically meet early social attention and communication milestones. Participants were required to select the correct age from four age ranges or to select “unsure”. The Knowledge of Early Signs subscale included seven behaviours seen in young children, with participants indicating whether each behaviour was, or was not, a key sign of autism. The General Autism Knowledge subscale included six statements regarding conceptions and misconceptions of autism. Participants indicated whether each statement was true or false or that they did not know. Correct responses received scores of “1” and incorrect responses received scores of “0”. Scores were summed to provide an indicator of overall knowledge for each subscale.

Confidence

This subscale included four items about participants’ confidence in identifying early signs of autism and one item about confidence in discussing concerns with parents. Participants rated their agreement with each statement using a Likert-type scale (1 = strongly disagree; 5 = strongly agree) where higher scores indicated greater confidence. Overall confidence was indicated by participants’ mean scores across all five items.

Acceptability of the SACS-R Tool

This subscale was only administered post-workshop and at follow-up. It included five items about participants’ perceptions of the feasibility and acceptability of the SACS-R tool. Scoring was the same as the Confidence subscale.

Procedure

The workshops were delivered by the senior author (JB) and lasted for 4.5 to 5 h excluding breaks. The workshops covered typical and atypical social attention and communication, the importance of early identification and diagnosis, early signs of autism (including video comparisons of autistic and non-autistic children), administering the SACS-R tool, and discussing concerns with parents. The workshop involved PowerPoint slides, videos, handouts, and group tasks. Local clinicians and representatives from Autism NZ also shared information about referral and local support services. More details of the training are reported in Barbaro et al. (2011).

Data Analysis

Analyses were conducted in SPSS (v. 27). Descriptive statistics were calculated for demographic characteristics, subscales and item-level responses. The subscale scores were not normally distributed, so non-parametric inferential tests were used. Data from Auckland and Wellington were analysed together as Mann–Whitney U tests indicated there were no significant differences in subscale scores between these two locations. As comparatively few participants completed follow-up, Wilcoxon signed rank tests were used to determine the significance of changes in subscale scores from pre-workshop to post-workshop, pre-workshop to follow-up, and post-workshop to follow-up. Here the alpha level was reduced to 0.017 to correct for multiple comparisons. Spearman’s correlations were used to examine the relations between participant demographic characteristics (time in role/as a nurse) and subscale scores. Values from 0.1 to 0.3 were considered weak, values from 0.3 to 0.5 were moderate, and values from 0.5 to 1 were strong (Cohen, 1988). When participants completed ≥ 50% of items for a particular subscale but some responses were missing, data was imputed for the remaining items based on their mean score for that subscale. Twenty-four participants had a least one score imputed and imputation occurred for between 0 and 18 participants across subscales and time points (pre-, post- and follow-up).

Results

Demographic Characteristics

One-hundred-and-eighty-eight individuals completed at least one of the pre and post surveys on the day of the workshop. Except for several clinical leaders and educators who did not intend to use the SACS-R in practice, this number represented all nurses who attended the Auckland and Wellington workshops. However, eleven individuals were excluded because they were not registered nurses, and two were excluded due to incomplete pre-workshop data, leaving 175 participants. Also, only 56 individuals completed the follow-up survey, resulting in a response rate of 32% for the follow-up survey. Table 1 shows the participant demographic characteristics.

Table 1

Demographic characteristics of participants (n = 175)

Characteristic	n (%)/M (SD)	Missing (n)
Gender
Female	175 (100%)	0
Location
Auckland	132 (75.4%)	0
Wellington	43 (24.6%)
Role
Nurse	140 (80.0%)	0
PlunketLine nurse	13 (7.4%)
Clinical leader	15 (8.5%)
Educator	3 (1.7%)
Other	4 (2.3%)
Mean years in role	7.2 years (6.7 years)	22
Mean years registered as a nurse	18.1 years (12.3 years)	5

Knowledge

Figure 1 shows the mean percentage of correct knowledge items for each subscale, while Table 2 shows the percentage of participants who answered each knowledge item correctly.

Fig. 1

Mean percentage of correct items for knowledge of typical development, knowledge of early signs, and general autism knowledge at pre-workshop, post-workshop, and follow-up. Note: **p < 0.001

Wilcoxon signed ranks tests revealed significant improvement in participants’ knowledge of typical development, knowledge of early signs, and general autism knowledge from pre-workshop to post-workshop (Z = 4.542, p < 0.001; Z = 6.722, p < 0.001; and Z = 9.699, p < 0.001 respectively) and pre-workshop to follow-up (Z = 3.266, p = 0.001; Z = 3.794, p < 0.001; and Z = 4.354, p < 0.001 respectively). There was no significant change from post-workshop to follow-up for participants’ knowledge of typical development or knowledge of early signs (Z = − 0.960, p = 0.337; and Z = − 0.288, p = 0.774 respectively). There was a significant decrease in participants’ general autism knowledge from post-workshop to follow-up, although this remained above pre-workshop levels (Z = − 3.486, p < 0.001). Table 2 indicates that the percentage of participants who selected the correct response increased for almost all items from pre-workshop to post-workshop and follow-up and decreased for most items from post-workshop to follow-up across all three knowledge domains.

Confidence

Table 3 shows participants’ mean Likert scale scores for each confidence item and Fig. 2 shows their overall confidence at each time-point. Overall participant confidence increased significantly from pre- to post-workshop (Z = 10.887, p < 0.001) and follow-up (Z = 6.767, p < 0.001), and decreased significantly from post-workshop to follow-up but remained above pre-workshop levels (Z = -2.736, p = 0.006). Participant confidence increased for each item from pre-workshop to post-workshop/follow-up and decreased for all items from post-workshop to follow-up.

Fig. 2

Mean likert scale scores (1 = strongly disagree; 6 = strongly agree) for confidence and acceptibility of the SACS-R tool at pre-workshop, post-workshop, and follow-up. Note: **p < 0.001

Acceptability of the SACS-R Tool

Table 3 shows participants’ mean Likert scale scores for each acceptability item and Fig. 2 shows their overall perceived acceptability of the SACS-R tool at each time-point. Overall acceptability significantly decreased from post-workshop to follow-up (Z = − 3.714, p > 0.001), as did participant agreement with each acceptability item.

Correlations

Spearman’s correlations between each subscale and both time in role and time as a nurse are presented in Tables S1 and S2. There were significant weak positive correlations between time in role and confidence at pre- and post-workshop and between time in role and knowledge of typical development at pre-workshop and follow-up. There was a significant moderate positive correlation between time as a nurse and confidence at pre-workshop and a weak positive correlation between time as a nurse and knowledge of typical development post-workshop. There were no other significant correlations.

Discussion

Universal developmental surveillance plays an important role in early diagnosis and timely access to early intervention. This study was the first to examine developmental surveillance training for NZ WCTO nurses. Our results suggest that the nurses increased their knowledge of typical social attention and communication development, early signs of autism, and general autism knowledge immediately following the training. Their confidence in identifying signs of autism in children aged 12, 18, 24, and 30 months and in raising these concerns with parents also increased. They also generally found the SACS-R tool to be highly acceptable.

These findings align with previous research, which found that nurses and educators had positive perceptions of the SACS-R training/tool and that their confidence in identifying and discussing early signs with parents increased following training (Barbaro et al., 2011; Mozolic-Staunton et al., 2021; Shrestha et al., 2021). Shrestha et al. (2021) similarly found that nurses’ knowledge improved after participation in the SACS-R workshop. The three countries represented in the current and previous research (NZ, Australia, and Nepal) have vastly different demographic characteristics, culture, health systems, and available funding for developmental surveillance. Thus, it is promising that SACS-R training and tool were perceived as feasible and acceptable across all studies.

Nurses’ general autism knowledge, confidence, and perceived acceptability of the SACS-R tool remained relatively high at follow-up but there were some decreases in these three areas compared to post-workshop. This is in contrast to Shrestha et al. (2021), who did not find any decreases from post-workshop to follow-up. Following the initial training, nurses in the Shrestha et al. study participated in monthly meetings in which they discussed cases and issues in implementing the SACS-R, while nurses in the current study did not. These monthly meetings may have been crucial in maintaining nurses’ knowledge and confidence following the workshop. The findings of the current study therefore indicate that while the 1-day training was effective in increasing nurses’ early autism knowledge and confidence, it should ideally be complemented with additional, ongoing support.

Participants who had spent longer in their role or as a nurse were more confident in identifying and discussing early signs, prior to the workshop than those with less experience. Although more experienced nurses scored higher on knowledge of typical development at some time-points, level of experience was not related to nurses’ knowledge of early signs of autism or general autism knowledge. This may be because while many of the nurses had numerous years of experience examining developmental milestones during well-child checks, most had never received any formal autism training. Several previous studies have also found that experienced clinicians often have limited knowledge of autism (e.g., Ben-Sanson et al., 2018; Shrestha et al., 2021; Tilahun et al., 2019). Thus, nurses of all levels of experience and confidence may benefit from autism-specific training.

The current study had some limitations to note. The response rate for the follow-up survey was low, possibly due in part to Covid-19 lockdowns, particularly in the Auckland cohort as there were more strict lockdowns in this area compared to Wellington. The follow-up period was also 3 months in the Wellington sample and 8 months in the Auckland sample, although no significant differences existed between these two locations. The study also lacked a comparison group who did not participate in the training. Further, the data for knowledge of early signs in the Wellington cohort was not useable. Although participant-reported autism knowledge and confidence was related to accuracy of early identification in previous research (Ben-Sasson et al., 2018), we did not directly examine referral rates or the psychometrics properties of the SACS-R within this NZ sample. Thus, it is not clear whether the nurses became more accurate at identifying early signs of autism in their day-to-day practice. We also did not assess the impact of the training on children and families.

The results of this study suggest that NZ WCTO nurses increased their autism knowledge and confidence in identifying and discussing early signs of autism during developmental surveillance training; they also found the SACS-R to be highly acceptable. This suggests that implementation of the SACS-R in a NZ context is feasible and acceptable. Larger scale research on the implementation of SACS-R in NZ, including investigation of its psychometric properties, effects of ongoing training, and referral pathways, is warranted.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 16 kb)