The impact of using digitally-mediated social stories on the perceived competence and attitudes of parents and practitioners supporting children with autism.

A Social Story (SS) is a highly acceptable and widely used intervention by the autism community. Yet, inconsistent implementation of the intervention is reported to be one of the causes of variability in terms of outcome research, particularly in a naturalistic context. This study aimed to (1) investigate whether digitally-mediated social stories (SSs) can improve competence in developing and delivering a SS and thus contribute towards improved implementation, and (2) investigate the impact of the digitally-mediated SS on attitudes towards the SS intervention. Ninety-three participants took an initial pre-engagement survey. Forty-eight of these participants also complete a post-engagement survey. A pre-post design was utilised with the participants who completed both surveys. These 48 participants were invited to develop a digitally-mediated SS with the aim of exploring how digitally-mediated SSs impacted perceived competence and attitudes. Post-engagement data was collected two weeks after the pre-engagement data. Outcomes of this study indicate that both perceived competence and attitudes improved after engaging with digitally-mediated SSs. It is concluded that digitally-mediated SS not only impacted the integrity of how the intervention was delivered but also the beliefs in the participants' capabilities to develop and deliver a SS. Digitally-mediated SS, thus, has the potential to effectively support development and delivery whilst also addressing challenges related to intervention implementation in a naturalistic context.

Introduction

The Social Story (SS) intervention was developed by Carol Gray [1] specifically for children with autism (N.B. There is a lack of consensus on the terminology used to describe autism. In order to respect the diversity of views, both identity-first language as well as person-first language should be used [2]. However, in this paper, whilst acknowledging the diversity of views, for the purpose of stylistic consistency, person-first language–i.e., person with autism–is used throughout). It is an intervention that comprises a highly structured and personalised narrative which describes a situation, skill, or concept in terms of relevant social cues, perspectives, and common responses in a specifically defined style and format [3]. The goal of social stories (SSs) is for a writer (also known as an “author”), usually a parent or a practitioner, to accurately and efficiently share social information with an individual (also referred to as the “audience”) in a descriptive and informative manner [4, 5].

Originally SSs were developed and delivered solely in the form of written text or through the medium of print. Pictures were later introduced after a surge in interest in the intervention led to further research and a better understanding of SSs [6]. Several methods for the administration of the intervention have been used subsequently, which include: digital tablets [7] social robots [8], computer software [9] and also Wearable Immersive Virtual Reality (WIVR) technology [10].

SS is described as a highly acceptable intervention by practitioners in the field of autism [11]. It is also one of the most frequently used interventions by parents of children with autism [12, 13]. Attitudes towards the intervention are reported to be positive [14, 15], which could be a result of the high social validity reported by parents and also by people with autism themselves [16]. Also, this could even be a consequence of the apparent simplicity [17] and/or versatility of the intervention.

There have been a large number of studies illustrating the positive effects of social stories on a number of behaviours. These include anxiety [18], challenging behaviour [19], understanding emotions [20] developing social skills [21, 22], coping with new situations [23] understanding sexuality [24] and increasing task engagement [25], amongst others. These “goals” have been grouped in four categories by Kokina et al. [26]: (i) reduction of inappropriate behaviours, (ii) improvement in social skills, (iii) teach academic or functional skills, and (iv) assist in transition/novel situation/reduce anxiety. However, systematic and meta-analyses highlight great variability in terms of effectiveness and outcomes of the intervention across all four areas [11, 26–30].

Reasons for such variability have been suggested by several researchers. One of the reasons which could be contributing to this variability is the goal which the story is targeting. SS that target reducing negative behaviours are reported to be the most used in research [31], whilst other SS goals, such as managing transitions are reported to only make up around 9% of the literature [26]. Constantin et al. [32] suggest that the audience’s poor or inconsistent understanding of SS could also be another factor which is contributing to this variability. Comprehension checks are recommended by Gray’s original criteria [1], where it is recommended that authors always “Plan for Comprehension”. Constantin et al. [32] argue that comprehension checks are often skipped by parents and practitioners, which means it is unknown whether the child’s understanding of the situation has improved or not. This issue, which is brought forward by Constantin et al. [32] is related to poor procedural integrity of the intervention.

Procedural integrity

Procedural integrity—also known as procedural fidelity or treatment integrity–is essential for empirical testing of intervention efficacy and dissemination of evidence-based practices [33]. A variety of definitions and conceptual models have been proposed to define procedural integrity, which is considered a wide-ranging concept [34]. The definition used in this study is that proposed by Perepletchikova et al. [35], who define Procedural integrity in terms of three components [11, 35, 36]: agent competence (i.e., the knowledge base and skill that the individual who is implementing an intervention exhibits), treatment adherence (i.e., the reliable use of the procedures as specified a priori), and treatment differentiation (i.e., the extent that the intervention is reliably distinguished from another intervention). In their synthesis of the literature, Test et al. [29] and Bucholz [3] suggest that poor procedural integrity could be impacting SS intervention effectiveness, and thus contributing towards the variability in outcomes for SS.

Variability and the use of technology

Digital technology can aid the reduction of variability, and the use of digitally mediated interventions and supports in the field of autism has been on the rise over the past decade [37]. With the development of smartphone/tablet technology and smartphone/tablet apps, their use by children with autism and their families has become increasingly popular [38]. However, Kim et al. (2018) warn that research on digitally-mediated interventions has not kept pace with the volume apps available to support autism. In fact, from a total of 700 smartphone applications, Kim et al. [39] found that only 0.6% of them had any empirical evidence to support their usage. Chen [40] postulates that research regarding the impact and effectiveness of digital technologies for children with autism are inconclusive. Nevertheless, innovative technologies carry great promise. Chen [40] argues that service users and providers still need to understand how to take advantage of the affordance of this technology to provide the best possible support for individuals with autism, their families and professionals who support them (see Riga et al. [41]). Thus, generally, there seems to be agreement that, whilst digital support is a promising area for autism, it still requires further research [7, 39, 42].

More recently, Hanrahan et al. [43] utilised digitally-mediated SS to address the issue of variability in the development and delivery of SS (see also Min et al. [44] and Vandermeer et al. [45]). Using a digital tablet, the authors evidenced the effectiveness of digitally-mediated SS employing a Randomised Control Trial design. When compared with the outcomes of a control group, digitally-mediated SS were found to be effective in producing a beneficial behavioural change in children with autism [43]. Similarly, in a study by Smith et al. [46], teachers used a digital tablet application to develop and deliver personalised digitally-mediated SSs with children with autism over a 4-week intervention period. Outcomes of this study indicated that teachers were able to carry out the intervention with a high degree of procedural integrity. Furthermore, improvements in child understanding as well as decreased anxiety.

In another study, conducted by Smith et al. [47], digitally-mediated SS were used by summer camp teachers to support ten children with autism through the transition to the summer camp. Outcomes of this study were again indicative of appropriate procedural integrity and resulted in significant improvements in appropriate behaviours and reductions in inappropriate behaviours. Consistent with Gray’s criteria, these studies identified a specific goal for the SS intervention and measured intervention success by analysing practitioners’ ratings of the child’s closeness to the target goal [48].

Competence and procedural integrity

Thus, with features such as ensuring goal-setting and comprehension checks, digital technologies can support parents and practitioners to develop and deliver SS to children with autism consistent with Gray’s criteria. However, the effectiveness and integrity of the intervention will still depend on the competence of the author of the story [49]. “Competence” here is defined as the extent to which an author has the knowledge and skill required to deliver a treatment to the standard needed for it to achieve its expected effects [50]. This suggests that competence is directly related to procedural integrity, as it would be difficult to ensure that interventions are implemented as intended without a good knowledge of that intervention. Procedural integrity tends to be higher in interventions led by researchers who are experts in the field of SS, and lower in studies where development and delivery of the intervention is carried out by individuals will less competence in delivering the intervention [51, 52]. Furthermore, improved competence, resulting from direct teaching/coaching of authors (i.e., an individual who is developing and delivering the intervention), tends to lead to positive outcomes in naturalistic contexts where SS intervention is developed and delivered by individuals such as parents and practitioners, rather than by researchers (see Acar et al. [15] and Olçay-Gül et al. [53]).

Aims of the study

With a view to improving procedural integrity, through a pre-post research design, this study aimed to investigate if a digitally mediated SS improved competence ratings of “authors” who are developing and delivering SS in a naturalistic setting i.e., in a context that is outside research confines, where confounding variables cannot be controlled. It also aimed to explore factors that could predict competence outcomes, and also investigate the impact of a digitally mediated SS on the author’s attitude towards the intervention. A final analysis also looked at the different goals identified by authors who used the app to create a SS. The various goals were categorised in pre-defined groups, as described by Kokina et al. [26], in order to examine if there were any differences between categories of SS goals.

The SOFA-app

SSs in this study were digitally mediated through the use of the SOFA-app (Stories Online for Autism). The SOFA-app is an online application, for smart devices (Android/iOS smartphones or tablets), through which social stories can be developed and delivered. It is an app that was co-developed with the autism community [54] that is aimed towards helping authors create and deliver SS reliably and effectively. The SOFA-app (https://SOFA-app.org) is free-of-charge, on the Google-Play store and Apple store.

Relevance of this study

This study is relevant to the autism community as it contributes towards answering the question of high variability in SS research. Furthermore, this study also contributes towards the evidence, or lack of, that is currently available to substantiate the effectiveness of digital supports for autism interventions [37].

Moreover, as a result of the COVID-19 pandemic, digital platforms for interventions are increasing in popularity [55]. Recently, practitioners are more inclined to suggest technological interventions to families that need more accessible services [56]; not only because of the accessibility of digital supports, but also because of the reported affinity with technology children and adolescents with autism are reported to exhibit [57, 58]. Caria et al. [59] hypothesise that this could be “due to the predictable and repeatable nature of technology that can create controlled environments, and which thus appeals to those (particularly children with autism) who feel relieved by stability and routine” (p.1735).

Method

Participants

Ninety-three participants (68 practitioners and 25 parents) completed the baseline measures and were given access to the application. Subsequently, 48 of the initial 93 participants (34 practitioners and 14 parents) used the digitally-mediated SS and completed the outcomes measures. This represents a completion rate of 52%.

The parents that participated in this study all indicated that they were parents of children with autism. The practitioners that participated were Psychologists (coming from the Educational, Clinical, Health, and Counselling fields), Speech-Language Pathologists (SLP), Occupational Therapists (OTs), Teachers, Learning Support Educators (LSEs), Teaching Assistants (TAs), Inclusion Coordinators (INCOs), and Special Education Needs Coordinators (SENCOs) and all worked with children with autism.

Participants were recruited through a non-probability purposive sampling strategy. The recruitment was carried out by emailing invitations to an administrative contact at a programme, in Malta, that provides services for children with autism and their parents. Recruitment was also carried out online through a Massive Open Online Course (MOOC) that focused on evidence-based practices for autism in education. The participants’ characteristics, their experience with SS, as well as their proficiency with digital mobile technology, were identified and are presented in Table 1.

Table 1

Participants’ demographics.

	Completed Baseline Only1		Completed Baseline & Outcome2		All participants3
	n	P(%)	n	P(%)	n	P(%)
Role
Parents	11	24.4	14	41.2	25	26.9
Practitioners	34	75.6	34	58.8	68	73.1
Total	45	100	48	100	93	100
Experience with SS
Extensive experience	35	77.8	35	72.9	70	75.3
Little/no experience	10	22.2	13	27.1	23	24.7
Participant’s Age Range
18 to 25	2	4.4	3	6.2	5	5.4
26 to 35	13	28.9	16	33.3	29	31.2
35 to 45	17	37.8	22	45.8	39	41.9
46 to 55	11	24.5	5	10.4	16	17.2
56 to 70	2	4.4	2	4.3	4	4.3
Participant’s Country of residence
Malta	25	55.6	38	79.2	63	67.7
UK	20	44.4	10	20.8	30	32.3
Perception of one’s proficiency at using an electronic device (mobile or digital tablet)
Proficient	NA4		45	93.8	NA4
Not proficient			3	6.2

P = Percentage, NA = Not applicable.

1 Participants who completed only the baseline measures (n = 45).

2 Participants who completed the baseline and outcome measures (n = 48).

3 Participants from 1 + participants from 2 (N = 93).

4 No baseline or outcome data was collected. Perception of one’s proficiency at using an electronic device was gathered only for participants who completed outcome measures for competence and attitude.

Procedures

The participants were invited to complete a pre-engagement baseline questionnaire evaluating perceived competence and attitude towards social stories. The link for this baseline questionnaire, together with comprehensive information about the study, was made available to potential participants through email, and by posting information together with a link on the MOOC online forum. All of the study procedures were completed using Qualtrics® Software (www.qualtrics.com).

Participants who completed the baseline questionnaire were invited to download a SS app (SOFA-app) on their smartphone or tablet and were asked to complete two tasks: With a particular child with autism in mind; (i) identify a goal for a target behaviour to be addressed by a SS, and (ii) develop an appropriate SS. Practitioners and parents were given two weeks to get acquainted with the app and complete the assigned tasks. Participants were then invited to complete the measures again (i.e., complete a post-engagement survey).

Measures

Initial questions were aimed at establishing the level of experience the participants had with developing and delivering SS. Participants who had experience with both developing and delivering SS were grouped in the “Extensive experience” category. Participants who had experience only with delivering or only with developing a SS were grouped in the “Little/No experience” category. Participants who had never developed nor delivered a SS where also grouped in the “Little/No experience” category. The level of confidence in one’s ability to develop and deliver SS was gauged by asking participants to state their level of agreement, on a 5-point Likert, for the following statements: (i) “How confident are you in your ability to write an effective social story?” and (ii) “How confident are you in your ability to effectively use a social story with a child?”.

Subsequently, the participants were invited to complete a set of baseline questions which were presented in the form of statements. All participants were asked to indicate their level of agreement/level of confidence on a 5-point Likert scale ranging from strongly agree/very confident (5) to strongly disagree/very “unconfident” (1). Participants were asked questions aimed to explore: their confidence rating in their ability to develop and deliver a SS (Confidence Scale), their attitude towards SS intervention (Attitude Scale), and their perceived level of competence in their ability to develop and deliver a SS (Competence Scale). The 48 participants who complete the outcome measures were also invited to rate the perception of their proficiency (proficient or not proficient) of using their particular electronic device (smartphone/digital tablet) and also to rate their experience (User Experience) of using the application to digitally-mediate the intervention. The latter was calculated by computing the mean scores of participants on four questions aimed at gathering information about the users’ experience with the application (see S1 Appendix).

The items used in the Attitude Scale were adapted from the Intervention Rating Profile– 15 [60] and the Abbreviated Acceptability Rating Profile [61]. These measures were developed to explore acceptability of the intervention.

Competence was operationally defined in terms of two aspects [62]: (i) having the knowledge of the intervention (measured through questions C2, C3 & C6) and (ii) having the preparedness to deliver the treatment as intended (measured through questions C1, C4 and C5). Furthermore, the questions comprising the Competence Scale aimed to measure the participants’ perceived competence from a “limited-domain” perspective [62]. Through this conceptual lens, competence is seen as “treatment-specific” and thus is concerned about the specific intervention, i.e., the SS intervention. Thus, questions were intended to elicit the participants’ perceived understanding of goal setting (C4), sentence construction (C3 & C5), SS treatment complexity (C1), SS treatment specificity (C6), and perceived skill required to develop a SS (C2).

Data analysis

For the participants who completed both the baseline and outcome measures (n = 48), Kokina et al.’s [26] coding categories (reduction of inappropriate behaviours; improvement in social skills; teach academic or functional skills; assist in transition/novel situation/reduce anxiety) were used to classify the targets identified by the authors. i.e., the goal of the digitally-mediated SS.

For each participant, we averaged ratings of agreement over 2 “confidence” items, 6 “attitude” items, and 6 “competence” items, and computed summary scores for each scale (see Table 3). For participants who completed the outcomes measures, rating on 4 “user experience” (UE) items were average and computed into one variable. All of the scales have good internal consistency, with a Cronbach Alpha coefficient of 0.75 (Confidence), 0.85 (Attitude), 0.75 (Competence) and 0.82 (User Experience). Baseline and outcome attitude and competence ratings were recorded and analysed. The degree of change registered on competence and attitude scales after using the application was also computed.

The Shapiro-Wilks test for normality was used to detect departures from normality on all computed measures. When analysing all of the baseline data (N = 93), normality assumptions were violated for baseline measures (baseline confidence, baseline competence and baseline attitude). When analysing data obtained from participants who completed both baseline and outcome measures (n = 48), the normality assumption was not violated for baseline competence (p = .082) and outcome competence (p = .247). Change in outcome competence rating was trending towards normality (p = .035). This trend was confirmed by visual inspection of a histogram. Baseline and outcome attitude scores violated the normality assumption.

Thus, in cases where the normality assumption was not violated, parametric tests (paired-sample t-test, Pearson product-moment correlation and simple linear regression) were utilised. This was the case when analysing competence ratings for data obtained from participants who completed both baseline and outcome measures. Non-parametric tests (Wilcoxon signed-rank, Kruskal-Wallis test and Spearman’s rank-order correlation) were utilised when analysing attitude ratings since the data obtained from attitude measures all failed to meet the normality condition. In instances when differences identified in data was found to be statistically significant (p < .05), effect sizes were calculated by using Cohen’s d measure for the magnitude of experimental effect [63].

Ethical considerations

This study was approved by the University of Bath Psychology Research Ethics Committee (PREC, 19–309) and by the University of Malta’s Faculty Research Ethics Committee (FREC, 2393_25072019). All procedures performed in this study were in accordance with the 1964 Helsinki declaration and its later amendments. Data were collected through the use of online questionnaires. In these questionnaires, the participants were informed about the purpose of the study and consent was gathered online. Assurance was given to participants that their involvement in the study was strictly voluntary.

Results

Descriptive statistics that illustrate the basic features of the data in this study are presented in Tables 2 and 3. Table 2 provides a summary of the mean, standard deviation, and median ratings for baseline data gathered with all 93 participants. Table 3 provides similar summary data to Table 2, but only for the participants who completed both baseline and outcome measures (n = 48).

Table 2

Descriptive statistics of baseline measures from all participants (N = 93).

	Total		Parents		Practitioners		Extensive Experience5		Little/no experience6
	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn
Baseline confidence	3.56 (0.63)	4.00	3.52 (0.66)	4.00	3.58 (0.62)	4.00	3.66 (0.59)	4.00	3.15 (0.60)	3.00
Baseline competence	3.74 (0.67)	3.83	3.41 (0.73)	3.67	3.86 (0.61)	4.00	3.91 (0.56)	4.00	3.22 (0.73)	3.33
Baseline attitude	4.55 (0.45)	4.67	4.23 (0.58)	4.17	4.66 (0.32)	4.67	4.62 (0.38)	4.67	4.33 (0.58)	4.33

M = Median; SD = Standard Deviation; Mdn = Median

5 Participants who reported to have extensive experience with developing and delivering SS intervention.

6 Participants who reported to have little to no experience with developing and delivering SS intervention.

Table 3

Descriptive statistics of measures from participants who complete post-engagement survey (n = 48).

	Total		Parents		Practitioners		Extensive experience		Little/no experience
	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn
Competence7
Baseline Competence rating	3.68 (0.76)	3.75	3.30 (0.75)	3.59	3.84 (.71)	4.00	3.88 (0.64)	3.83	3.11 (0.78)	2.83
Outcome Competence ratings	4.10 (0.50)	4.17	4.05 (0.56)	4.09	4.12 (.47)	4.17	4.09 (0.47)	4.17	4.11 (0.59)	4.17
Change* in Competence ratings	0.42 (.79)	0.25	0.75 (0.97)	0.33	.28 (.68)	0.25	0.20 (.66)	0.17	1.00 (0.87)	0.83
Attitude8
Baseline Attitude ratings	4.54 (.43)	4.67	4.27 (0.57)	4.17	4.65 (0.31)	4.67	4.59 (0.36)	4.67	4.37 (0.57)	4.50
Outcome Attitude rating	4.71 (0.37)	4.83	4.71 (0.32)	4.75	4.71 (0.39)	4.92	4.70 (0.40)	4.83	4.74 (0.28)	4.83
Change* in Attitude rating	0.18 (0.46)	0.17	0.44 (0.45)	0.33	0.07 (0.42)	0.00	0.10 (.41)	0.17	0.37 (0.52)	0.33
Baseline Confidence9	3.60 (0.65)	4.00	3.55 (0.64)	4.00	3.61 (0.66)	4.00	3.63 (0.62)	4.00	3.43 (0.79)	4.00
User Experience (UE) ratings10	4.11 (0.67)	4.25	4.20 (0.71)	4.63	4.08 (0.67)	4.25	4.16 (0.54)	4.25	4.00 (0.97)	4.50

M = Mean; SD = Standard Deviation; Mdn = Median

* Change is referring to the mean of the difference between outcome and baseline ratings.

7 Extent to which an author has the knowledge and skill required to deliver a SS intervention to the standard needed for it to achieve its expected effects.

8 Disposition with regards to SS.

9 Belief in one’s ability to deliver SS intervention appropriately.

10 Perception of the “ease of use” of the application used for digital-mediation.

Baseline measures

There was a statistically significant positive correlation, with a medium effect size, between baseline competence and baseline confidence (r .34, n = 87, p < .001), and between baseline competence and baseline attitude (r = .34, n = 93, p < .001). The relationship between baseline attitude and baseline confidence was small (r = .16, n = 93, p > .05) and not statistically significant.

How does role and experience impact baseline measures?

A statistically significant difference was found in baseline competence ratings of experienced (n = 70) and the less experienced participants (n = 23), U = 371.0, z = -3.87, p < .001, r = .40. A statistically significant difference was also present in baseline attitude (U = 568.5, z = -2.14, p = .033, r = .22) and confidence ratings (U = 320.5, z = -3.15, p = .002, r = .33) of experienced participants and less experienced participants. As expected, those with more experience reported more confidence, competence and positive attitudes at baseline.

The impact of role on baseline measures, i.e., the differences in baseline ratings between parents (n = 25) and practitioners (n = 68) was also analysed. A statistically significant difference was registered in baseline competence (U = 556.5, z = -2.5, p = .011, r = .26) and baseline attitude (U = 452.5, z = -3.49, p < .001, r = .36) between the two group of participants, but not in baseline confidence (U = 667.0, z = -.277, p = NS). This indicates that practitioners and parents did not report a significant difference in confidence ratings. However, practitioners reported higher competence and attitude scores at baseline.

Outcome measures

Forty-eight participants (34 Practitioners and 14 parents), from the initial 93, completed the baseline as well as the outcome measures. No significant differences on confidence, competence and attitude measures were obtained between participants who completed baseline measures only, and participants who completed both baseline and outcome measures (all ps >0.85. The means and medians for the outcome measures are in Table 3.

Competence

Does competence change after using digitally-mediated SS? Participants’ (n = 48) competence ratings prior to using the application (M = 3.68, SD = .76) increased after using digitally-mediated SS (M = 4.09, SD = 0.49). The result was indicative of a statistically significant mean increase of 0.42 (SD = .79), BCa 95% CI [0.19, 0.65], t(47) = 3.65, p = .003. A Cohen’s d of .65 indicated a medium effect size.

How do baseline and outcome competence relate to each other? There was a statistically significant, strong negative correlation between the two variables, r(46) = -.79, p < .001, with high levels of baseline competence ratings resulting in a lower change in outcome competence ratings.

How does baseline confidence relate to change in competence? There was a statistically significant, negative correlation, with medium effect size, between the variables, with the resulting degree of change in competence decreasing as baseline competence increased: r(40) = -.33, p = .032.

How does experience with SS impact change in competence? The participants with extensive experience (n = 35) obtained a mean increase of .205 (SD = .112). However this increase was not statistically significant, BCa 95% CI [.004, .434] t(34) = 1.84, p = .074 The participants with little to no experience (n = 13) obtained a statistically significant mean increase of 1.01 (SD = .233), BCa 95% CI [.564, 1.451], t(12) = 4.148, p = .001. A Cohen’s d of 1.45 indicated a large effect size.

The change in competence was higher in the less experienced group (M = 1.00, SD = 0.87) than in the group with extensive experience (M = .20, SD = 0.66). This was statistically significant, 95% CI [-1.27, -.32], t(46) = -3.40, p = .001, with a Cohen’s d of 1.03, indicative of a large effect size.

How does role impact change in competence? Practitioners’ competence ratings prior to using the application (M = 3.835, SD = .713) increased after using the application to develop a SS (M = 4.119, SD = 0.475). The result was indicative of a statistically significant mean increase of 0.284, 95% CI [0.046, 0.523], t(33) = 2.43, p = .021. A Cohen’s d of .47 indicated a trending medium effect size. Parents’ initial competence ratings (M = 3.30, SD = .75) also improved after using digitally-mediated SS (M = 4.05, SD = 0.56). This difference, .75, BCa 95% CI [.28, 1.22] was also statistically significant t(13) = 2.90, p = .020, and represented a large effect size, Cohen’s d = 1.13.

The change in competence, as a result of using digitally-mediated SS, was higher for parents (M = .75, SD = 0.97) than for practitioners (M = .28, SD = .68). However, this result was not statistically significant (95% CI [.13., 1.06], t(18.6) = -1.638, p = .065).

Does “user experience” (UE) relate to change in competence? There was no statistically significant correlation between UE and change in competence, rs(46) = .125, p = .40.

Attitude

Does attitude change with use? Of the 48 participants recruited for the study, the application elicited an increase in attitude ratings in 25 participants post-intervention, whereas 7 participants’ attitude ratings decreased post-intervention. There was a median increase in attitude ratings of .167 from baseline (Mdn = 4.67) to outcome (Mdn = 4.83) ratings. This difference was statistically significant, z = -3.07, p = .002, r = .31.

How do baseline and outcome attitude relate to each other? There was a statistically significant, strong negative correlation between baseline attitude rating and degree of change in attitude ratings after using the app, rs(46) = -.66, p < .001, with higher baseline confidence ratings resulting in a lower change in attitude after using the application.

How does baseline confidence relate to change in attitude? There was no statistically significant correlation between the variables, r(40) = -.134, p = .397.

How does experience with SS impact change in attitude? The app elicited a post-intervention increase in attitude ratings in 18 of the experienced participants and 7 of the less experienced participants. 5 experienced participants and 2 less experienced participants registered a decrease in attitude. Attitude ratings of 12 experienced participants and 4 with little to no experience remained unchanged. There was a median increase in attitude ratings from baseline (Mdn = 4.67) to outcome (Mdn = 4.83) ratings for the experienced group. This difference, .16, was statistically significant, z = -2.228, p = .026, r = .27. The attitude ratings of participants with little to no experience also improved after using the digitally-mediated SS. The results of the signed-rank test indicated that this difference, .33, was also statistically significant, z = -2.322, p = .020, r = .46.

A difference in attitude rating was found between groups of differing levels of experience (Mdn = .16), with participants with less experience registering more attitude improvement (Mdn = .33) when compared to participants with more experience (Mdn = .17). However, this difference was not statistically significant, U = 183.50, z = -1.05, p = .296.

How does role impact change in attitude? An increase in attitude ratings was registered in fourteen of the practitioners, whereas six practitioners’ attitude ratings decreased. There was a median increase in attitude ratings of 0.245 from baseline (Mdn = 4.670) to outcome (Mdn = 4.915) ratings, but this difference was not statistically significant, z = -1.501, p = .133. Parents’ attitude ratings also improved after using the digitally-mediated SS. This difference, 0.33 was statistically significant, z = -2.95, p = .003.

Does “user experience” (UE) relate to attitude change? There was no statistically significant correlation between UE and change in attitude, rs(46) = .047, p = .749.

How does change in outcome competence relate to change in outcome attitude ratings? There was a statistically significant, positive correlation with a medium effect size between the two variables, r(46) = .416, p = .003, with high competence ratings associated with higher attitude ratings towards the intervention.

Goals of SS developed

The goals of the SS developed by parents and by practitioners were classified in terms of the 4 main categories of SS goals described by Kokina & Kern’s [26]: (1) reduction of inappropriate behaviours, (2) improvement in social skills, (3) teach academic or functional skills, and (4) assist in transition/novel situation/reduce anxiety. The researchers independently categorised 47 of the goals/targets identified by the parents and practitioners and obtained 87% interrater agreement. Cohen’s κ was also run to determine if there was agreement between the researchers’ judgement. There was strong agreement between author’s judgements, κ = .885, 95% CI [.777, .993], p < .001. Any discrepancies in ratings were resolved via discussion resulting in 100% agreement. Finally, it was agreed that “reduction of inappropriate behaviours” represented 18.8% of goals; “improving social skills” represented 22.9%; “teaching academic or functional skills” represents 20.8%; and “assisting in transition/novel situation/reduce anxiety” represented 35.4% of goals identified. There were no significant differences between competence-change or attitude-change for any of the 4 groups (all ps > 0.061).

Discussion & implications

Social stories (SS) are a popular, acceptable and widely used intervention for children with autism [12, 13, 64, 65]. Attitudes to SS are positive in both parents and practitioners, although variability in competence, with regards to implementing the intervention, has raised questions regarding effectiveness [15, 51–53]. It has been proposed that digitally-mediated social stories can support the competence of the story writer (author) and improve intervention integrity.

This study found that using digitally-mediated social stories significantly improved perceived competence and attitudes for both parents and practitioners with higher and lower levels of previous experience with the intervention. Importantly, digitally-mediated social stories were found to be beneficial outside of the research context, with parents and practitioners writing social stories in a naturalistic context–in some instances for the first time. Those with the least levels of competence and positive attitudes, perceived the greatest improvement in their competence and positive attitudes. The findings are therefore consistent with previous research which indicates that digitally mediated social stories can reduce the variability in the development of the stories and the delivery of the intervention, and improve effectiveness [43, 46, 47]. This study suggests that this may extend to parents and practitioners with a wide range of levels of previous experience.

This study was one of the first to investigate the impact of digitally-mediated SS in a naturalistic context, specifically from the author’s perspectives. Unlike previous studies where digitally-mediated SS intervention were used in a naturalistic setting [46], in this study researchers did not author the stories. Rather, parents and practitioners were “tasked” with the authoring of the stories. Thus, for the first time, this study shows that parents and practitioners can write the stories themselves.

Baseline and outcome data collected in this study also indicates that the higher the participants’ perceived competence ratings, the more positive the attitude towards SS. Furthermore, outcome data indicates an increase in attitude ratings after using the digitally-mediated SS. Whilst experienced and less experienced participants both registered an increase in attitude, results indicated that there was no meaningful difference between the attitude increase of the two groups. This indicates that experience did not meaningfully impact attitude increase. On the other hand, a statistically significant increase in attitude was registered in parents but not in practitioners. This could indicate that role does impact change in attitude towards SS. Outcomes of this study also indicate that competence ratings improved in both experienced and less experienced participants. Nonetheless, participants with less experience registered more improvement than “experienced” participants. Also, participants with high levels of baseline competence and confidence ratings reported a smaller degree of change in outcome competence ratings; i.e., participants who reported higher baseline competence and confidence also reported the least change in improvement whilst still registering an increase.

Moreover, outcomes of this study also indicate that competence ratings improved in both parents and practitioners who participated in this study. Furthermore, it seems that that parents’ competence ratings improved more than practitioners’ competence ratings. Such improvements in competence also seemed to be dependent on the participant’s levels of experience with the intervention prior to using the application. Thus, this implies that competence–in terms of goal setting, sentence construction, and ratio of descriptive to coaching sentences–can be improved, to various degrees, through digital mediation. This finding is noteworthy, especially in light of the lack of research on the implementation of interventions for children with autism in naturalistic settings. Improved competence and procedural integrity can contribute greatly towards improved effectiveness of SS interventions in a naturalistic setting [50], and also towards decreasing barriers in implementation. To overcome these barriers Kasari and Smith [66] postulate that key components of interventions have to be clear whilst also allowing for flexibility. The issue of flexibility is particularly important for the area of autism, since heterogeneity of presentation is a hallmark of this neurodevelopmental condition [67]. Thus, the clinical features and cognitive differences characteristic of autism imply that an individual with autism requires a specific and personalised intervention [68]. This places parents and practitioners in an ideal and unique position to develop and deliver specific interventions since they tend to be most knowledgeable about an individuals’ unique characteristics. Furthermore, the adaptability of SS intervention also lends itself well to this goal. Actually, in this study, the personalised nature of the intervention was exemplified by how each participant in the study was able to identify a goal which was pertinent to the child/individual they had in mind, adapt the intervention to that individual’s needs, and all whilst being conscious of integrity requirements. This means that SS interventions could be delivered with a higher degree of procedural integrity by individuals who know the “service users” best.

The findings are therefore consistent with previous studies, such as those carried out by Smith et al., [46, 47] and by Constantin et al. [32], who suggest that digitally-mediated SS could be effective in supporting children with autism. This could be achieved by utilising digital-mediation to maximise procedural integrity by improving an “author’s” competence to develop and deliver a SS. Furthermore, similar to Smith et al., [47], this study suggests that digital mediation could contribute towards better accessibility to the intervention for authors, and also reduce consumption of time and effort for both developing and delivering the intervention. Digital mediation of the intervention may therefore address the variability in effectiveness that has been identified in previous studies [11, 26–30].

Such findings are encouraging especially when considering the positive attitude towards the intervention reported in various studies. This study also suggests that the use of a digitally mediated SS could further improve positive attitudes towards the intervention, even when the intervention is used in a naturalistic context. Consequently, this could encourage practitioners and parents to increasingly utilise digitally mediated SS in their everyday interactions with children with autism providing greater intervention consistency across contexts. The accessibility and consistency of use across contexts could potentially lead, in turn, to greater effectiveness of the intervention. Future research can explore this potential through applying the methodology in this study to children with autism, and monitoring the impact on the child’s understanding, or behaviour. Also, future research could focus on obtaining qualitative data which could also elucidate further the participant’s experience of the intervention as well as the medium used to deliver the intervention, for example by explicitly comparing digital and non-digital SS.

Limitations

This study explored authors’ experience of the app, and a limitation is that the stories were not used with children. However, it important to establish for future research that the authors are able to develop the intervention. Another limitation is that the study assessed perceived competence. It was not possible to independently assess the SS that had been written. Again, this can be a feature of future research.

From the 93 participants that showed interest in this study, only 48 finished both the baseline and the outcomes questionnaires, which is another limitation of the study. Furthermore, the low number of parents who participated, compared to practitioners, is a further limitation. Nevertheless, it is difficult to control the number of responses obtained in a study, especially in a population such as that of parents of children with autism who report high burden, psychological distress, and lower social support [69].

Furthermore, it is important to note that 94% of the participants that completed the required activities, together with all the outcome measures, reported that they were proficient in their ability to use digital technology. Only 6% described themselves as “not proficient”. The participants’ extremely high level of proficiency could be revealing of a positive bias towards technology, which could have in turn skewed both their attitude and competence ratings. Also, recruitment invited participation to take part in a study of SS, and it is likely that those with an interest in the intervention were more likely to participate. This will reduce the generalisability of the findings. Though it is interesting to note that some participants had no experience with SS.

No data on the participants’ gender or ethnicity was gathered in this study. However, there currently is no research that indicates that these factors have any impact on the variables and processes that were investigated in this study.

The research design utilised meant that the outcomes could not be compared to other media. Comparison of digital mediation with non-digital media could have added a comparison in changes in competence and attitude outcomes resulting from different modes of delivery. Comparison of competence and attitude outcomes as a result of using other applications, other than SOFA, would have also aided in understanding if the extent to which results of this study were specific to this app. Finally, future research could focus on obtaining data about children’s or adolescents’ perceptions of SS intervention as well as digitally mediated SS. A question which could be addressed is if SS could be developed by the children/adolescents with autism themselves and if they could be supported to do this through digital technology.

Conclusion

Overall, the findings from the present study indicate that using digitally mediated SS could improve, to various degrees, parents and practitioners’ competence and attitudes towards developing the intervention for children with autism. This is the case for those with higher and lower levels of experience with the intervention, with those with lower levels potentially benefitting the most from using digitally mediated SSs.

Practitioners, baseline & outcomes (n = 34), descriptive statistics.

(DOCX)

Click here for additional data file.

Parents, baseline & outcomes (n = 14), descriptive statistics.

(DOCX)

Click here for additional data file.

Questionnaire items.

(DOCX)

Click here for additional data file.

22 Jul 2021

PONE-D-21-16546

The Impact of Using Digitally-Mediated Social Stories on The Perceived Competence and Attitudes of Parents and Practitioners Supporting Children with Autism.

PLOS ONE

Dear Dr. Camilleri,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 05 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Mingming Zhou, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for including your ethics statement: "All procedures performed in this study were in accordance with the ethical standards of the institutional and/or national research committee (University of Bath Psychology Research Ethics Committee – PREC and University of Malta Faculty of Education Research Ethics Committee), and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards."

a) Please amend your current ethics statement to confirm that your named institutional review board or ethics committee specifically approved this study.

b) Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

If you are reporting a retrospective study of medical records or archived samples, please ensure that you have discussed whether all data were fully anonymized before you accessed them and/or whether the IRB or ethics committee waived the requirement for informed consent. If patients provided informed written consent to have data from their medical records used in research, please include this information.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

3. Peer review at PLOS ONE is not double-blinded (https://journals.plos.org/plosone/s/editorial-and-peer-review-process). For this reason, authors should include in the revised manuscript all the information removed for blind review.

4. Your ethics statement should only appear in the Methods section of your manuscript. If your ethics statement is written in any section besides the Methods, please delete it from any other section.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Broadly, the noble goals or contributions of the research can be explained in a straightforward manner. The title already represents the overall picture of the article.

The phenomenon of Digitally-Mediated Social Stories on the Perceived Competence and Attitudes of Parents and Practitioners Supporting Children with Autism as the main figure in the article, can be strengthened again regarding the emergence of the phenomenon.

At the end of the introduction, it is necessary to strengthen the issue of Digitally-mediated social stories and how they can be used in the future for "the new normal" on autism methods. What makes SSs more interesting and urgent to discuss? Can it also be linked to the IoT generation or digital natives? on the other hand, do autistic children also have generations like children in general?

Triggers or lighters regarding the current existence of SSs and their correlation to multidimensional models of autism also need to be sharpened as an effort to lead readers' minds about the direction of the research being carried out.

Reviewer #2: The article presents The Impact of Using digitally-mediated Social Stories on The Perceived Competence

and Attitudes of Parents and Practitioners Supporting Children with Autism.

The article is well written, and sufficient data is provided. However, the study's overall conclusion is difficult to generalize with the children with Autism, as the parents participated and not directly the children. In the case of levels of ASD, every child is different from others within the same level. So the supported data was mandatory about the level of autistic kids (High functioning, low functioning, age); this is a significant limitation. Plus, there should be a reference group as well to compare the results. The details and reason for choosing this specific statistical analysis should also be mentioned. With the current data, the general conclusion is too optimistic.

Reviewer #3: This paper made a subjective self-report investigate on the implementation of using digitally-mediated social story, especially SOFA-app. Both pre and post surveys were collected and analyzed. The result shows that using digitally-mediated SS significantly improved perceived competence and attitudes for both parents and practitioners. For details:

1. In the analysis of the degree of change on competence and attitude in pre and post engagements, whether there exists any difference on the baseline measures using 93 participants (including 45 people who did not complete the post-engagement survey) and baseline using 48 participants who completed both pre and post surveys?

2. As for competence, 6 ratings of competence items are averaged, since COMP1 and COMP6 are reverse scored, is it appropriate to directly average the scores of the 1&6 capability items equally?

3. Objective evaluations on competence are suggested to included, such as how many sentences were constructed on each goal and how much time was spent.

4. Besides currently self-reported rating, manually ratings by other professionals may also include to assess the quality of social story descriptions, such as clarity.

5. Suggest to add a brief description about the process of how to use SOFA-app, I am not sure whether there exists any difference in the ways that different participates use this tool.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

12 Aug 2021

• Issues pertaining to journal requirements:

a. Amend your current ethics statement to confirm that your named institutional review board or ethics committee specifically approved this study.

b. Provide additional details regarding participant consent.

Authors’ reply:

Points a & b, above, have been addressed. In the methods section titled “Ethical considerations” the following has been added:

Ethical considerations

This study was approved by the University of Bath Psychology Research Ethics Committee (PREC, 19-309) and by the University of Malta’s Faculty Research Ethics Committee (FREC, 2393_25072019). All procedures performed in this study were in accordance with the 1964 Helsinki declaration and its later amendments. Data were collected through the use of online questionnaires. In these questionnaires, participants were informed about the purpose of the study and their consent was gathered online. Assurance was given to participants that their involvement in the study was strictly voluntary.

• Reviewer #1 wrote:

At the end of the introduction, it is necessary to strengthen the issue of Digitally-mediated social stories and how they can be used in the future for "the new normal" on autism methods. What makes SSs more interesting and urgent to discuss? Can it also be linked to the IoT generation or digital natives? on the other hand, do autistic children also have generations like children in general? Triggers or lighters regarding the current existence of SSs and their correlation to multidimensional models of autism also need to be sharpened as an effort to lead readers' minds about the direction of the research being carried out.

Authors’ reply:

We have added the following text at the end of the introduction:

Relevance of this study

This study is relevant to the autism community as it contributes towards answering the question of high variability in SS research. Furthermore, this study also contributes towards the evidence, or lack of, that is currently available to substantiate the effectiveness of digital supports for autism interventions (Zervogianni et al., 2020).

Moreover, as a result of the COVID-19 pandemic, digital platforms for interventions are increasing in popularity (Doenyas, 2021). Recently, practitioners are more inclined to suggest technological interventions to families that need more accessible services (Heng et al., 2021); not only because of the accessibility of digital supports, but also because of the reported affinity with technology children and adolescents with autism are reported to exhibit (Hedges et al., 2018; Laurie et al., 2019). Caria et al. (2018) hypothesise that this could be “due to the predictable and repeatable nature of technology that can create controlled environments, and which thus appeals to those [particularly children with autism] who feel relieved by stability and routine” (p. 1735).

With regards to the question posed by Reviewer #1: “Can it also be linked to the IoT generation or digital natives? on the other hand, do autistic children also have generations like children in general?” We feel that the question is indeed a pertinent one. However, answering that question goes beyond the scope of this study. Furthermore, the study’s participants were parents and practitioners, and not the children/adolescents with autism themselves. The questions that were asked to this specific sample were not aimed towards answering the reviewer’s question specifically, but whether digitally mediated SS improved competence ratings of “authors” who are developing and delivering SS in a naturalistic setting.

• Reviewer #2 wrote:

The study's overall conclusion is difficult to generalize with the children with Autism, as the parents participated and not directly the children. In the case of levels of ASD, every child is different from others within the same level. So, the supported data was mandatory about the level of autistic kids (High functioning, low functioning, age); this is a significant limitation. Plus, there should be a reference group as well to compare the results. The details and reason for choosing this specific statistical analysis should also be mentioned. With the current data, the general conclusion is too optimistic.

Authors’ reply:

The study focuses on parents and practitioners’ perceived competence and attitudes. Thus, whilst the reviewer is right to state that “every child is different from others within the same level”, this has little to no bearing on parents’ and practitioners’ competence and attitudes towards social stories. The study aimed to investigate if a digitally mediated SS improved competence ratings of “authors” who are developing and delivering SS in a naturalistic setting. Thus, asking about the children’s “levels” would have raised ethical concerns as it would have been data/information which is surplus to requirements.

The reviewer also requests information about the statistical methods used: “The details and reason for choosing this specific statistical analysis should also be mentioned “. We feel that this information is already substantial and the reasons for using the methods of analysis have been already discussed in the Data Analysis section of the Methodology:

In cases where the normality assumption was not violated, parametric tests (paired-sample t-test, Pearson product-moment correlation and simple linear regression) were utilised. This was the case when analysing competence ratings for data obtained from participants who completed both baseline and outcome measures. Non-parametric tests (Wilcoxon signed-rank, Kruskal-Wallis test and Spearman's rank-order correlation) were utilised when analysing attitude ratings since the data obtained from attitude measures all failed to meet the normality condition. In instances when differences identified in data was found to be statistically significant (p < .05), effect sizes were calculated by using Cohen’s d measure for the magnitude of experimental effect (Cohen, 1988).

Reviewer #2 also stated that the conclusion is too optimistic. To address this, we have made some changes to the conclusion:

Overall, the findings from the present study indicate that using digitally mediated SS could improve, to various degrees, parents and practitioners’ competence and attitudes towards developing the intervention for children with autism. This is the case for those with higher and lower levels of experience with the intervention, with those with lower levels potentially benefitting the most from using digitally mediated SSs.

• Reviewer #3 (no.1) wrote:

1. In the analysis of the degree of change on competence and attitude in pre and post engagements, whether there exists any difference on the baseline measures using 93 participants (including 45 people who did not complete the post-engagement survey) and baseline using 48 participants who completed both pre and post surveys?

Authors’ reply:

This information is included under the heading Baseline Measures in the Results section:

48 participants (34 Practitioners and 14 parents), from the initial 93, completed the baseline as well as the outcome measures. No significant differences on confidence, competence and attitude measures were obtained between participants who completed baseline measures only, and participants who completed both baseline and outcome measures (all p > .05).

• Reviewer #3 (no.2) wrote:

2. As for competence, 6 ratings of competence items are averaged, since COMP1 and COMP6 are reverse scored, is it appropriate to directly average the scores of the 1&6 capability items equally?

Authors’ reply:

The questions the reviewer is referring to are the following:

COMP1 - I believe that social stories are difficult for me to write or create.

COMP6 - Social stories are tools used for entertaining children and not an intervention.

Both of these questions can be considered “reversed-polarity items”. They have been incorporated to see to acquiescence response bias. On the Competence composite score (where all competence items were average), the higher the score obtained, the higher the perceived competence of the participant. If these two items were not reverse scored, the competence score would have not represented an accurate measure of the overall competence items.

• Reviewer #3 (no. 3 & 4) wrote:

3. Objective evaluations on competence are suggested to included, such as how many sentences were constructed on each goal and how much time was spent.

4. Besides currently self-reported rating, manually ratings by other professionals may also include to assess the quality of social story descriptions, such as clarity.

Authors’ reply:

We agree that these are good suggestions. However, the analysis of the stories was not part of the research design as the focus was to investigate perceived competence and attitude. Thus, the stories produced were not recorded.

• Reviewer #3 (no.4) wrote:

5. Suggest to add a brief description about the process of how to use SOFA-app, I am not sure whether there exists any difference in the ways that different participates use this tool.

Authors’ reply:

Including information about the process of “how to use SOFA-app” could be useful for readers to understand further. However, given the space constraints, this information can instead be obtained by following the link https://SOFA-app.org, which can be found under the subheading titled The SOFA-app.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

9 Dec 2021

PONE-D-21-16546R1The Impact of Using Digitally-Mediated Social Stories on The Perceived Competence and Attitudes of Parents and Practitioners Supporting Children with Autism.PLOS ONE

Dear Dr. Camilleri,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jan 23 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Mingming Zhou, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

Reviewer #5: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

Reviewer #5: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: N/A

Reviewer #5: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

Reviewer #5: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

Reviewer #5: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: The authors performed all the required revisions that were requested by the previous round of review.

Reviewer #5: The manuscript is important and well written. I hereby suggest to add the following reference (if found relevant) in introduction/review of literature section: Riga, A., Ioannidi, V., & Papayiannis, N. (2021). SOCIAL STORIES AND DIGITAL LITERACY PRACTICES FOR INCLUSIVE EDUCATION. European Journal of Special Education Research, 7(2). doi:http://dx.doi.org/10.46827/ejse.v7i2.3773

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

Reviewer #5: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

26 Dec 2021

• Reviewer #5 wrote:

The manuscript is important and well written. I hereby suggest adding the following reference (if found relevant) in the introduction/review of literature section: Riga, A., Ioannidi, V., & Papayiannis, N. (2021). SOCIAL STORIES AND DIGITAL LITERACY PRACTICES FOR INCLUSIVE EDUCATION. European Journal of Special Education Research, 7(2). doi:http://dx.doi.org/10.46827/ejse.v7i2.3773

Authors’ reply:

Riga et al.’s (2021) reference is a theoretical piece that highlights how social stories (SSs), particularly digitally-mediated SSs, are useful for the autism community.

As recommended by Reviewer #5, this reference has been included in the introduction/literature review (under the subheading “Variability and the use of technology”).

We hope that this sees to the points raised by the reviewer.

We look forward to your feedback.

Best regards

THE AUTHORS

26th December 2021

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

30 Dec 2021

The Impact of Using Digitally-Mediated Social Stories on The Perceived Competence and Attitudes of Parents and Practitioners Supporting Children with Autism.

PONE-D-21-16546R2

Dear Dr. Camilleri,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Robert Didden

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

5 Jan 2022

PONE-D-21-16546R2

The Impact of Using Digitally-Mediated Social Stories on the Perceived Competence and Attitudes of Parents and Practitioners Supporting Children with Autism.

Dear Dr. Camilleri:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Professor Robert Didden

Academic Editor

PLOS ONE