Protocol for a prospective, school-based standardisation study of a digital social skills assessment tool for children: The Paediatric Evaluation of Emotions, Relationships, and Socialisation (PEERS) study.

BACKGROUND: Humans are by nature a social species, with much of human experience spent in social interaction. Unsurprisingly, social functioning is crucial to well-being and quality of life across the lifespan. While early intervention for social problems appears promising, our ability to identify the specific impairments underlying their social problems (eg, social communication) is restricted by a dearth of accurate, ecologically valid and comprehensive child-direct assessment tools. Current tools are largely limited to parent and teacher ratings scales, which may identify social dysfunction, but not its underlying cause, or adult-based experimental tools, which lack age-appropriate norms. The present study describes the development and standardisation of Paediatric Evaluation of Emotions, Relationships, and Socialisation (PEERS®), an iPad-based social skills assessment tool.
METHODS: The PEERS project is a cross-sectional study involving two groups: (1) a normative group, recruited from early childhood, primary and secondary schools across metropolitan and regional Victoria, Australia; and (2) a clinical group, ascertained from outpatient services at The Royal Children's Hospital Melbourne (RCH). The project aims to establish normative data for PEERS®, a novel and comprehensive app-delivered child-direct measure of social skills for children and youth. The project involves recruiting and assessing 1000 children aged 4.0-17.11 years. Assessments consist of an intellectual screen, PEERS® subtests, and PEERS-Q, a self-report questionnaire of social skills. Parents and teachers also complete questionnaires relating to participants' social skills. Main analyses will comprise regression-based continuous norming, factor analysis and psychometric analysis of PEERS® and PEERS-Q. ETHICS AND DISSEMINATION: Ethics approval has been obtained through the RCH Human Research Ethics Committee (34046), the Victorian Government Department of Education and Early Childhood Development (002318), and Catholic Education Melbourne (2166). Findings will be disseminated through international conferences and peer-reviewed journals. Following standardisation of PEERS®, the tool will be made commercially available. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This study will prospectively recruit and assess a demographically representative sample of children from early childhood, primary and secondary schools from across Victoria, Australia.

We will collect standardisation data on Paediatric Evaluation of Emotions, Relationships, and Socialisation (PEERS®) from children and adolescents of different backgrounds and ages, and data on gold standard measures of social and behavioural status to enable in-depth analysis of the psychometric properties of PEERS.

As with other school-based studies, our sample does not represent all children because we can include only children whose parents return completed consent forms.

We cannot ascertain whether parents returning completed consent forms are more likely to have a child with social difficulties or to have relatively high socioeconomic status.

Introduction

Social interactions are at the heart of human activity.1 Regardless of socioeconomic status (SES), ethnic background and educational or vocational status, a large proportion of human daily life involves social participation—whether it is in person, electronic, or for work, school or play. Appropriate social skills are the basis for the development and formation of satisfying and lasting relationships, which in turn are essential to psychological well-being across the lifespan.2 3 Poor social skills have significant implications for the individual’s ability to participate meaningfully within their environment,3 which has led to a burgeoning interest in better understanding social competence in health and disease.

Social impairment is the primary symptom of a number of conditions, most notably autism spectrum disorder (ASD).4 5 However, social impairment is also observed in many medical and psychiatric conditions including traumatic brain injury,6 attention deficit hyperactivity disorder (ADHD),7 8 learning disabilities,9 10 anxiety disorders,11 epilepsy12 and chronic illness.13 In everyday life, typical examples of social disturbance include social exclusion, avoidance and isolation, lack of satisfying friendships, shyness, bullying, and poor self-esteem.6 14 Additionally, social skills are heavily reliant on social learning, and thus children from disadvantaged backgrounds and dysfunctional families are also at risk for social difficulties.15–17 Social problems elevate the risk of emotional and behavioural difficulties and limit the individual’s ability to participate effectively in their community.3 18 Recently, promising findings are emerging, demonstrating the benefits of early intervention to ameliorate these social problems in some clinical groups, particularly children with ASD.4 Effective intervention, however, requires accurate identification and characterisation of social impairment techniques.19 20 Currently, while a small number of well-validated social skills rating scales are available, there is a dearth of comprehensive, well-validated, ecologically sensitive, child-direct tools available to detect social impairment.

Identification of social impairment first requires an understanding of normal social development, which is underpinned by a complex set of skills, including social cognition (eg, empathy), social communication (eg, understanding humour) and aspects of executive functions (eg, attentional control, decision making).14 Social cognition refers to cognitive processes used to process, perceive, and understand and respond to social stimuli and environments, and includes emotion recognition, theory of mind, and complex derivatives of these such as moral reasoning.14 21 Assessment tools that link social behaviours to these underlying cognitive mechanisms will inform both understanding of the child’s social strengths and limitations and also treatment planning. Current cognitive assessments, however, do not explicitly link to social behaviours, and social skills rating scales are not designed to deconstruct behaviour into underlying cognitive and socioemotional skills. There are few standardised, comprehensive, child-direct tests of social cognition, with those available usually developed for adults (eg, The Awareness of Social Inference Test).22

Many current social assessment tools are symptom-based (eg, inventories of negative behaviour), rather than driven by theories about social development. Further, many have heavy cognitive demands, particularly related to language and working memory, which may also be impaired in children with social difficulties.23 Few available measures have incorporated ‘real-life’ materials, which counter best practice requirements that social assessments should be ‘ecologically sensitive and valid’, that is, as close as possible to real-life experience.20 23 24 Included in the notion of capturing real-life social challenges is requiring the examinee to become part of the social situation, by constructing stimuli that engage the child in social decision making.

Before commencing this project, we conducted a survey with key stakeholders in professional and research domains in the fields of child psychology, neuropsychology and education to ascertain to scope the availability and use of social measures in child assessments. The results identified the lack of a comprehensive, age-normed, child-direct social assessment tool, and indicated clear support for the development of such a tool. This is consistent with findings from a previous National Institute of Health initiative, which similarly described a lack of psychometrically robust measures of social cognition.25 In summary, there is a need for well-validated, theoretically motivated, ecologically valid tests of social skills that capture both behaviours and their underlying neural mechanisms.

To address this gap, we developed the Paediatric Evaluation of Emotions, Relationships, and Socialisation (PEERS®). PEERS® is embedded in a biopsychosocial theoretical framework referred to as SOCIAL (Socio-Cognitive Integration of Abilities model),14 which emphasises the importance of environmental and child factors and brain development and integrity for the development of intact social skills (figure 1). The model highlights that social competence requires intact abilities in several cognitive domains, including high-level attention, executive function and communication in addition to social cognition (eg, empathy, theory of mind, emotion recognition).

Figure 1

SOCIAL: a biopsychosocial model of social competency (from Beauchamp and Anderson14).

Using the SOCIAL framework, we developed PEERS®, a child-direct, interactive, individually administered assessment of social skills delivered via iPad by a trained assessor, and PEERS-Q, a rating scale (teacher, parent, self-report) to complement PEERS® by assessing social skills in daily life. PEERS® comprised 12 subtests, modified and designed from experimental paradigms, to evaluate specific domain from the SOCIAL model (attention/executive function, (social) communication and socioemotional function), focusing on an individual’s strengths and weaknesses in specific social abilities. Previous proof-of-concept work has been completed for some of the subtests (Mind Read,24 Multiple Morals,26 Social Intent), while other subtests have been modified from the social neuroscience literature (eg, Matching Emo, Say What). PEERS® is not developed specifically to be a clinical diagnostic test, but rather aims to describe the social profiles of children identified as at risk for social difficulties and would benefit from referral for tailored interventions.

To optimise ecological validity, most PEERS® subtests are delivered from first-person perspective, using static photographs and video vignettes, and examinees are encouraged to imagine they are embedded in the social situation they are viewing. It has been developed to enable a simple, engaging and intuitive user experience for the participant. In the development, the team applied Design Thinking processes, an Agile Development Methodology incorporating best practice software development processes and game design techniques. These processes and techniques allowed for an enhanced engagement from the participant ensuring a focus on the assessable content for each subtest. Due to the nature of the administration of PEERS®, the application was developed to comply with ‘software as a medical device’ regulatory standards.

PEERS® represents a major advance in psychological testing, as it uses technological advances in information technology software and hardware; is grounded in theoretical and empirical work from neuropsychology and social neurosciences; incorporates methods that are engaging and attractive to young people and simulate the real world; provides professionals with a convenient, low-cost, easy-to-interpret assessment tool; and provides an immediate, comprehensive and actionable ‘social profile’ for young people.

Objectives

The objective is to standardise PEERS® and PEERS-Q with children and adolescents using a population-based sample of participants aged 4–18 years. Specifically, this study aims to collect standardisation data, and a second clinical sample, to generate individualised age-based and sex-based norms, which will be automatically transformed by the app to provide regression-based continuous norming and scaled scores for subtests and an overall social quotient on completion of administration.

Methods

Study design and setting

This study employs a multi-site, cross-sectional design, including normative and clinical samples. Normative population data are collected at Victorian early childhood, primary and secondary schools, commencing in May 2016 and continuing until December 2017. To ensure recruitment of a representative sample, we included schools from a range of demographic backgrounds, based on the Index of Community Socio-Educational Advantage (ICSEA; mean (M)=1000, SD=100).27 Clinical data are being collected through referrals from RCH general paediatrics and child and adolescent mental health outpatient services.

Participants

Participants are children and adolescents aged 4–18 years who are currently attending mainstream schools and are competent in English. For the normative sample, children are invited to participate via their school (mainstream early childhood, primary or secondary). Students who return signed consent forms from their parent/guardian are enrolled into the standardisation study. Children in the clinical sample are referred from either outpatient general paediatric or child and adolescent mental health clinics, and are required to have a formal diagnosis of either a neurodevelopmental or mental health condition.

Parents/guardians (referred to hereafter as ‘parents’) and teachers of participating students are also asked to participate in the study.

Eligibility criteria

Children

For the normative sample, our aim is to recruit a sample that is representative of the general population; thus, we have few exclusion criteria (ie, only English competency and mainstream schooling); however we do collect background information regarding developmental, medical and psychiatric diagnoses in order to describe sample characteristics. For the clinical sample, participants are required to have a formal diagnosis of mental health (eg, anxiety) or neurodevelopmental (eg, ASD, ADHD) condition or an acquired brain injury diagnosis.

Parents

Parents of a child or adolescent at a participating school who are competent in English are eligible to participate.

Teachers

The participants’ classroom teachers are eligible to participate. We define classroom teacher as the teacher who spends the most time on a daily basis with the participant.

Measures

To provide validity (ie, convergent and divergent) data for the child-direct assessment of social cognition and social communication, several additional measures are administered to all participants: intellectual assessment, teacher and parent questionnaires tapping into children’s day-to-day social and emotional functioning in the school, community and home environment, and a parent measure of memory function.

Clinical history and demographics

A background questionnaire was designed specifically for this standardisation study and includes questions relating to sociodemographic factors and medical, developmental and educational history.

Comprehensive assessment of social cognition and social communication

Child-direct assessment

PEERS®, standardisation version, is an individually administered iPad-based test designed for child health professionals to identify strengths and weaknesses in specific social domains. Figure 2 shows an example subtest. The development of PEERS® was based on the biopsychosocial theoretical framework of the SOCIAL model,14 which includes three underlying cognitive domains of social development: attention-executive, social communication and socioemotional skills. These three domains make up the cognitive subscales of PEERS® (table 1). PEERS® comprised 12 basic and complex subtests, each of which focuses on one of these three cognitive domains. All subtests are derived from experimental paradigms from social neuroscience, and modified to reflect real-life situations and developmental expectations.28–32 Basic subtests are designed to capture fundamental social skills, such as emotion recognition or perception. Complex subtests tap higher order social skills such as empathy, moral reasoning and theory of mind.

Figure 2

A screenshot of the Paediatric Evaluation of Emotions, Relationships, and Socialisation (PEERS®) subtest Finding Emo.

Table 1

Paediatric Evaluation of Emotions, Relationships, and Socialisation (PEERS®) subtest descriptions

Domain	Basic skill	Name of subtest	Description	Complex skill	Name of subtest	Description
Attention /Executive	Selective attention	Friend Find	Within an array of distractor photos of other people, search for as many photos as possible that display the same person as the target photo. Assesses selective and divided visual attention for social stimuli (faces).
	Reaction time	Move Fast	Tap the objects as quickly as possible until the time runs out. Assesses motor response speed.
	Information processing	Think Fast	Quickly name the emotion shown on the illustrated face and then tap the face to move on. Assesses speed of information processing for emotion labels.
Social cognition	Emotion perception	Matching Emo	Match illustrated faces displaying different emotions with correct emotion labels. Assesses knowledge and recognition of emotion labels.	Social information processing	Social Intent	Watch brief videos of social situations and respond to multiple choice questions about the scenarios. Assesses intent attribution and responses to social situations.
	Emotion recognition	Finding Emo	Select the photograph in which a discordant emotion is displayed compared with the others in a grid of 4 static face photographs. Assesses facial emotion recognition.	Theory of mind	Mind Read	Watch brief videos of social interactions and answer multiple choice questions about the interactions. Assesses physical and mental-state theory of mind.
				Moral reasoning	Multiple Morals	View three images that tell a social story and say what you would do if you were in the situation. Then, justify your answer by choosing a reason among multiple choices. Assesses moral decision making and reasoning in social situations.
Social communication	Non-verbal gestures	Odd One	Select the photograph in which a discordant emotion is displayed compared with the others in a grid of 4 static body photographs. Assesses recognition of non-verbal emotional gestures.	Social interpretation	Get This	Move the cartoons around so that they tell a logical story. Assesses interpretation of conventional social situations
	Social perception	Social Scenes	Select the photograph in which a discordant social behaviour is displayed compared with the others in a grid of 4 static social scenario photographs. Assesses interpretation of social intent within social scenes.	Prosody	Say What	Listen to spoken sentences and select how the speaker is feeling. Assesses prosody in neutral audio sentences

PEERS® can be administered in full (all subtests) to derive a global social composite (M=100, SD=15) and domain scores (ie, attention/executive, social cognition, social communication; M=100, SD=15), or where specific concerns arise the examiner can administer select subtests (M=10, SD=3) to efficiently and directly test a clinical hypothesis. All scoring is done automatically by the app, and children’s individual responses to subtest items can be reviewed in the results screen.

Surveying social skills in daily life

PEERS-Q, standardisation version, previously known as the Developmental Assessment of Social Competence (DASC),33 is a questionnaire completed by parents and teachers of children in primary and secondary school, and children aged 12 years and older. The standardisation version of PEERS-Q includes five versions (all 73 items)—a parent-rated version for primary school students (5–11 years), a parent-rated version for secondary school students (12–18 years), a teacher-rated version for primary school students, a teacher-rated version for secondary school students and a self-rated version for secondary school students. It provides information regarding (1) mediators of social function—developmental stage, brain integrity, executive skills, personal attributes and environment; and (2) social outcomes—social adjustment, competence and participation. Negatively worded items are reverse-scored, with higher scores indicating better social competence. Initial piloting and development of PEERS-Q involved compilation of items with face validity for tapping social skills; focus groups with health professionals and teachers to refine the item pool; and completion of parent ratings across the target age range (n=144 parents).33 Example items include the following: ‘my child has a close relationship with his/her friends’, ‘my child often irritates or annoys those around him’ and ‘my child is usually able to mend friendships after conflict’. This measure is being validated in the present study (social outcomes component only).

Sample descriptors

Intellectual function

An age-appropriate version of the Wechsler scales is individually administered to assess intellectual functioning, providing a description of the standardisation sample’s overall intellectual ability. The Wechsler scales are widely used measures of intellectual functioning in both typically developing and clinical populations. Psychometrically robust, abbreviated versions of the tests were administered during the assessment to avoid testing fatigue.

Wechsler Preschool and Primary Scale of Intelligence—Fourth Edition (WPPSI-IV).34 In the present study the WPPSI-IV General Ability Index (GAI) is used to estimate IQ in children under 6 years of age (normed for children aged 4–7 years 7 months). This includes four subtests: (a) information (general knowledge), (b) similarities (verbal reasoning, concept formation), (c) block design (non-verbal reasoning) and (d) matrix reasoning (abstract reasoning, problem-solving). The WPPSI-IV GAI demonstrates strong reliability (overall average r=0.95), and correlates satisfactorily with other measures of cognitive and intellectual ability (corrected r=0.84 with the Wechsler Intelligence Scale for Children—Fourth Edition (WISC-IV)).34

Wechsler Abbreviated Scale of Intelligence—Second Edition (WASI-II).35 The WASI-II Full Scale Intellectual Quotient (FSIQ-2) is used to estimate IQ in children above 6 years of age. This involves two subtests: (a) vocabulary (word knowledge, verbal concept formation) and (b) matrix reasoning (visual information processing, abstract reasoning skills). The WASI-II FSIQ-2 has excellent reliability (average reliability coefficient aged 6–16=0.93) and strong concurrent validity (corrected r=0.82 with WISC-IV).35

Social skills

The Social Skills Improvement System (SSIS)—Parent and Teacher Forms36 is the most widely recognised and used questionnaire-based measure of children’s social skills according to a recent systematic review.19 It is administered to determine the convergent validity of PEERS® and PEERS-Q. The SSIS is a questionnaire designed to assess social skills in those individuals aged 3–18 years. Each respondent (teacher, parent and/or student) is asked to rate the perceived frequency (never, seldom, often, almost always) and importance of prosocial and problem behaviours. In the present study, both parent and teacher reports are used. The SSIS has three domains: social skills (communication, cooperation, assertion, responsibility, empathy, engagement, self-control), problem behaviours (externalising, internalising, hyperactivity/inattention, autism spectrum, bullying) and a measure of academic competency (reading, math, parent support, motivation, general cognition). Internal consistency for all subscales is high, with median subscale reliabilities (Cronbach’s alpha; CA) ranging from 0.85 to 0.90 for the teacher form, and from 0.83 to 0.87 for the parent form.36

Behaviour

The Strengths and Difficulties Questionnaire (SDQ)37 measures a parent’s perception of the child or adolescent’s emotional and behavioural status. Parents complete age-appropriate forms (4–10 years 11 months (P4–10) or 11–17 years 11 months (P11–17)), consisting of 25 items rated on a 3-point Likert scale (not true, somewhat true, certainly true), measuring the frequency of positive and negative behaviours. An example item is ‘Considerate of other people’s feelings’. The measure provides a total difficulties score and five subscale scores: emotional symptoms, conduct problems, inattention/hyperactivity, peer problems and prosocial behaviour. All subscales were used in the current study. The SDQ has good concurrent and predictive validity,38 and satisfactory internal consistency (mean CA=0.73).39

Non-social skill assessment

The Observer Memory Questionnaire—Parent Form (OMQ-PF)40 is a 27-item questionnaire assessing parents’ perceptions of their children’s everyday memory function. It asks questions about memory function in everyday scenarios, and beliefs about memory function and non-mnemonic functions. Rated on a 5-point Likert scale, items are summed to yield a total score ranging from 27 to 135. An example item is ‘Does your child recall details of previous conversations?’ The OMQ-PF has high internal consistency (CA=0.92). We expected that, in a sample of typically developing children, the memory skills assessed by the OMQ-PF would not be highly associated with social skills and thus have lower correlations with PEERS® and PEERS-Q.

Procedures

Recruitment

School recruitment

Government, private and Catholic early childhood, primary and secondary mainstream schools across Victoria are selected and contacted to gauge their interest in being involved in the PEERS® project. Selection is based on school size (>100, <1500), the schools’ location from programme base location (<1.5 hours) and SES to promote greater diversity in the recruited sample. Schools are initially contacted via email, and then recontacted by phone if no reply is received after 1 week. At early childhood centres, the centre director is usually the key contact, while at primary schools it is the assistant principal and at secondary schools the well-being coordinator (school counsellor/psychologist). If a school is interested in being involved in the study, a research team member gives a brief overview of the study via phone, and then schedules a face-to-face presentation at the school. These presentations involve one or two members of the research team introducing the rationale of the study, the logistics of how the study would run at the school and a demonstration of PEERS® to either key staff members or the whole school staff, depending on the schools’ preference. Within a week of the presentation, a member of the study team contacts the school via email to determine whether the school has decided to be involved. If the school agrees to be involved, detailed logistics of consent form distribution are then discussed. The school has the option of recruiting only students at specific grade levels (eg, a larger school might choose only one grade) or the whole school. Once these decisions are made, the school is given the necessary materials to start consent form distribution (eg, newsletters, flyers, consent forms).

Participant recruitment

Information statements and consent forms are distributed either electronically via email or in hardcopy (ie, paper) to parents of children in the year levels selected by the school. For electronic consent, the school sends parents an email introducing the PEERS study, with a link to the online consent form. Parents can then read the information statement and complete the consent form online if they are interested in being involved. Electronic responses are automatically received by the study team, minimising administrative burden for the school. If a school prefers paper consent forms, the school is provided with the seven-page information statement and consent form, which they then distribute to children in the identified year levels to take home. Interested parents then complete and return the paper consent form to the school via their child’s classroom teacher, and a member of the study team then collects returned forms from the school. Once completed consent forms, either electronic or paper, are received from a parent, they are signed by a research team member, and a copy is returned to the parent either in hardcopy or electronically, consistent with local ethics requirements. All returned completed consent forms are entered into a participant tracking REDCap database,41 and assigned a unique study identifier. Written consent from participants aged 12 years or older and verbal assent for children under 12 years is obtained on the day of the assessment.

Clinical group recruitment

Children in the clinical group are ascertained from paediatric and child and adolescent mental health outpatient clinics at The Royal Children’s Hospital (RCH). Potential participants are identified via clinic lists or clinician referral, and families are approached and provided with details of the study. Interested parents then complete and return the paper consent form to a member of the study team, and the process described above is also followed for this group, according to the RCH ethics policies.

Assessment procedure

Once consent forms are returned by parents and collated, the study team liaises with the key school contact (or parent) to organise the assessments. These assessments require a separate assessment space for each assessor, and thus the number of assessors a school/clinic can accommodate is often limited by space restrictions. Assessments are completed individually with each participant by trained researchers at the participant’s school, and take approximately 1–1.25 hours depending on the age and speed of the individual.

All participants complete PEERS® and the WASI-II or WPPSI-IV (see table 2). Participants aged 12 years or older also complete the PEERS-Q—Student Form. PEERS® data are collected locally on the iPad, which automatically synchronises to the project server. Student-completed PEERS-Q data are collected using REDCap (a secure online data management system) via iPad. WASI-II35/WPPSI IV34 data are collected using standardised paper forms and later entered into the REDCap database by the assessor. During the assessment, assessors also note down any behavioural observations or feedback from the child on the subtests.

Table 2

Measures involved in the PEERS standardisation project

	Participant	Parent	Teacher
Background questionnaire		•
Paediatric Evaluation of Emotions, Relationships, and Socialisation (PEERS®)	•
Paediatric Evaluation of Emotions, Relationships, and Socialisation—Questionnaire (PEERS®; student/parent/teacher)33	•*	•	•
Wechsler Abbreviated Scale of Intelligence—Second Edition35	•†
Wechsler Preschool and Primary Scale of Intelligence—Fourth Edition34	•‡
Social Skills Improvement System (SSIS; parent/teacher)36		•	•
Strengths and Difficulties Questionnaire (SDQ)37		•
Observer Memory Questionnaire—Parent Form (OMQ-PF)40		•

*Participants 12 years or above only.

†Participants 6 years or older only.

‡Participants 5 years or younger only.

Parent and teacher questionnaires are sent electronically via email on the day of assessment (parent questionnaires) or within approximately 3 months after the child’s assessment (teacher questionnaires) and completed via the REDCap platform, unless a paper version of the questionnaire is requested. If a parent does not complete the questionnaire, REDCap sends automatic reminders every 4 days up to a maximum of four times. If the parent has not completed the questionnaire after four reminders, a research team member makes up to two follow-up phone call attempts to that parent, after which point data are considered missing. Follow-up with teachers is performed via the school contact person if their questionnaires are not completed after the allocated date. The school is reimbursed for the relief teacher time required to replace teachers while they are completing study questionnaires. For parents indicating they would like feedback on their child’s performance, a brief assessment report is given based on their child’s performance on the WASI-II/WPPSI-IV, SSIS and SDQ.

Test–retest reliability data are collected for a subset of the total sample (n=100), with repeat assessment occurring within 1 month of initial assessment.

Data analysis

Sample size

We have calculated the largest number of participants we can feasibly recruit and test within the time and budget constraints of the study. The required sample is 1500 children from mainstream, metropolitan Victorian schools, to represent children who would complete PEERS®. Children ranging in age from 4 to 18 years will be recruited, with an equal distribution of age, race, sex, parental education level and SES across five age bands (4–6, 7–9, 10–12, 13–15, 16–18 years) to collate a standard distribution of scores on the PEERS® subscales and total score. For the major stratifications of sex and age bands, cell sizes of n=150 will accommodate a 95% CI precision of one-third of an SD of the measure.

A clinical sample of 125 children will be tested to provide preliminary discriminant validity data for children with social difficulties. The clinical sample will be aged 4–18 years and will be matched group-wise with the standardisation sample of age, race, sex, parental education level and SES across five age bands (4–6, 7–9, 10–12, 13–15, 16–18 years) to collate a standard distribution of scores on the PEERS® subscales and total score. Clinicians in these clinics will refer patients presenting with diagnoses that commonly involve social difficulties to the study team. The parents of these patients will then be contacted and invited to participate. Assessments will be arranged and completed with children and parents who consent to being involved, either at their homes, the outpatient clinic or the Murdoch Children’s Research Institute (study location). Teacher questionnaires will be given to the parent to pass on to their child’s teacher, along with a reply-paid envelope by which it can be returned. All other questionnaires (parent and study) will be administered in the same method as the school-based sample.

PEERS® and PEERS-Q

Initial analysis of PEERS® and PEERS-Q data from the standardisation study will focus on exploration at an item-based level in order to identify items across all subtests (PEERS®) and subscales (PEERS-Q) that may be deleted from the final versions of the measures. This will include items that demonstrate floor or ceiling effects, those for recording inconsistent responses and those where high inter-item correlations are detected. We expect the final versions of PEERS® and PEERS-Q to be significantly shorter and targeted than the standardisation versions.

In keeping with modern protocols for test standardisation, scored data for each subtest and subscale (table 1) will be derived using regression-based continuous norming with age (and polynomial functions of age) and sex as independent (predictor) variables.28 The explanatory power of other potential predictors (eg, SES) will also be examined. Raw subscale scores will then be expressed on a standardised metric (ie, M=10, SD=3) and as percentile ranks. The uncertainty surrounding these scores, reliability and within-subject variability will be quantified using a combination of classical and Bayesian methods. The critical outcome measure obtained from the PEERS® tool will be a standardised social composite score, similar to an IQ score (M=100, SD=15), with domain scores also provided for the cognitive components (attention/executive, social cognition, communication) and all PEERS-Q subscales. These data will be factor-analysed, using exploratory and confirmatory methods, to check statistical validity of theoretical domains. When data support it, domains may be adjusted to meet these findings. Correlational analyses will be used to add to these findings to develop a screening tool (subset of all subtests) to provide a summary index (M=100, SD=15) of social function to be used where full assessment is not required.

Psychometric properties

One parent-rated ‘global’ social measure will be used to test convergent validity, the SSIS total score (M=100, SD=15), currently the most commonly used questionnaire tapping social function in the 2–18 age range.19 The OMQ-PF will be used to obtain a measure of divergent validity, allowing the PEERS® and PEERS-Q to be compared with a measure unrelated to social competence. Internal consistency of PEERS® domains and PEERS-Q subscales will be determined using the CA statistic42 and associated formulae for the reliability of composites. Inter-rater reliability between the parent and teacher ratings will be calculated using intraclass correlation.43 Reliability data (test–retest) will be used as inputs for quantitative methods that will allow users to quantify change at the individual case level. Discriminant validity testing will also be carried out between the clinical and school samples on PEERS®.

Methodological considerations, learnings and challenges

School recruitment

At the onset of the standardisation study, some challenges were encountered in recruiting schools for the project. School personnel have many demands on their time and resources, and hence participating in the study represents an additional workload. Over time we have learnt the importance of minimising unnecessary study demands on the school. For example, by offering the use of electronic consent forms, schools can distribute forms to families via a single email with responses returned directly to the study team, rather than the school having to distribute and collate numerous paper forms from students. Additionally, efforts are made to minimise contact with the school and schedule assessments as efficiently as possible. At the end of a school’s involvement, we also offer schools a personal development session on social skills, to thank the teachers for their involvement in the study. Schools are also reimbursed for the hire of replacement teachers used while teachers are completing their questionnaires. Also, schools are offered free access to the app following its standardisation, to thank them for their involvement. These initial learnings are now communicated to interested schools at the outset. We find by explaining the minimally intrusive nature of the study to the school, and describing clearly what their involvement would require, schools feel more comfortable and are more likely to want to be involved. Additionally, because PEERS® is an iPad-based task, children tend to it enjoy more than paper and pencil tests, and consequently teachers and parents seem happier for their school/child to be involved. This in turn facilitates recruitment of a more representative sample, by encouraging a diverse range of schools to be involved, regardless of the level of resources they possess.

Participant recruitment

The nature of the present study requires initial contact with potential participants to be made through their school. Only students whose parents return consent forms can be enrolled in the study. This potentially biases the recruited sample, as parents with an interest in being involved may be those with a child who has social difficulties or problems and/or higher SES. It is important in the present study to have a diverse range of SES backgrounds to avoid over-representation of children with social problems in low-SES groups and a restricted range of ‘typical’ social behaviours if the sample is limited. To address this issue and maximise the representativeness of the sample, in all contacts with schools we emphasise that the study is available to all students, whether or not they have social skills problems, and we contacted schools from a range of SES. We also provide consent forms in electronic and paper formats, and use electronic reminders and follow-up phone calls to minimise the burden on interested parents and maximise participation of all students. Regarding diagnosis of social skills problems, we ask parents to provide information on any diagnoses their child might have (eg, medical, psychiatric, neurodevelopmental), such as the kind of diagnosis, age at diagnosis and details of any interventions (including pharmacological) received. This will later enable children with social difficulties to be considered both within the normative sample, to ensure a representative spread of social abilities, and to the normative sample who do not have these diagnoses to investigate the discriminant properties of PEERS®.

Teacher engagement

Student participation rates also depend to a large extent on individual teacher’s engagement in the project. To maximise participation and in turn promote representativeness of the recruited sample, the study team presents the study to key staff members who will be responsible for dissemination of information about the project at the school to ensure that these key stakeholders understand the project and its aims. This process facilitates staff involvement and engagement with the project, which translates to improved participation from their students. We also provide study reminders through the school newsletter to promote greater awareness of the study throughout the school.

Ethics and dissemination

Signed informed consent is obtained from all participating families and children 12 years and older. A brief report comprising results of their child IQ performance and a summary of the parent’s responses on the social and behaviour questionnaires is provided to those who indicate they would like to receive feedback. Referrals and follow-up are offered if a child’s performance is significantly below average (<80) on the IQ screen, or an elevated score on the SDQ is reported by the parent (≥17).

Results from this study will be disseminated at national and international conferences, as well as in peer-reviewed journals. PEERS® will be made commercially available following standardisation and has a patent pending.

Discussion

This study will provide in-depth data on the validity and reliability of PEERS® in children and adolescents aged 4–18 years. It will also advance our understanding of typical development of social function in school-aged children, and enable the development of a theory-driven, reliable and valid (ecological and content) assessment tool for social skills that has broad applicability to clinical populations. Study results will inform the refinement of PEERS®, and generation of a version of PEERS® for use in clinical practice, educational settings and so on. Further, results will be transformed into age-based and sex-based normative reference data to enable interpretation of test performance. This represents the first step in the standardisation of PEERS®. Subsequent steps will include standardisation of PEERS® in other countries.

In addition to the normative population recruited in the present study, we will collect PEERS® data on participants from a variety of clinical groups for whom social impairment is a common feature (eg, ASD, ADHD, traumatic brain injury). This will allow a preliminary assessment of the discriminant validity of PEERS® and its ability to identify strengths and challenges in clinical groups. Given the importance of social function for well-being and quality of life across the lifespan and the need for an engaging and reliable assessment of social skills from a clinical, educational and research perspective, the present study has the potential to provide an evidence-based comprehensive measure of social skills.

As of the end of 2016, the PEERS study has recruited and assessed approximately half of the total sample across 14 primary and secondary schools in Victoria, Australia. Recruitment and assessment will continue into 2017. Following completion of data collection, the psychometric properties of PEERS® will be analysed, and age and gender normative scores developed. Additionally, we endeavour to standardise PEERS® outside of Australia, with international samples (eg, North America, China and Scandinavia).