Development and initial validation of the Odense Child Trauma Screening: a story stem screening tool for preschool and young schoolchildren.

BACKGROUND: Early identification of young children exposed to trauma who are at risk of developing post-trauma symptomatology such as posttraumatic stress disorder (PTSD) or other emotional or behavioral problems is important for allocating appropriate treatment and preventing long-term consequences. However, assessment of young children exposed to trauma is challenging because children may not be able to talk about their trauma or trauma reactions. Story stem tools combine storytelling and play to access the internal world of young children and can be used in the assessment of children exposed to trauma.
OBJECTIVE: To examine reliability and validity of a new story stem tool, the Odense Child Trauma Screening (OCTS). OCTS was developed to screen for play-based behavior and narrative representations indicative of traumatization in preschool and young schoolchildren.
METHOD: Forty-nine Danish children aged 4.5-8.9 years (M = 6.6, SD = 1.2) participated in the OCTS. Participants included a risk sample of 31 children exposed to traumas and a community sample of 18 children. Caregivers were interviewed about child symptoms of PTSD, major depressive disorder (MDD), and reactive attachment disorder (RAD) and answered the Strengths and Difficulties Questionnaire (SDQ). The community sample completed OCTS test-retests.
RESULTS: Interrater reliability proved excellent (ICC = .96-1.00). Test-retest reliability was acceptable (ICC = .66). Significant moderate correlations were found between the OCTS total score and scales of PTSD, MDD and RAD and the SDQ Total Difficulties Scale. The ability of the OCTS to discriminate between children from the risk and community sample was good.
CONCLUSIONS: The study provided preliminary evidence of reliability and validity of the OCTS as a screening tool for young children exposed to trauma. OCTS shows promise as a standardized, age-appropriate informant-based screening measure applicable for clinical assessment.

Introduction

Early detection of children who have been exposed to trauma and who are at risk of post-trauma symptomatology is crucial to allocate appropriate support and treatment and prevent long-term psychopathological outcomes (1, 2). However, assessment of young children exposed to trauma can be difficult, if the child is not able to fully talk about the trauma or verbalize his or her trauma symptoms (3). Also, a core symptom of posttraumatic stress is avoidance of reminders of the trauma (4) which can further challenge the assessment if the child does not want to talk about the trauma. Assessment of trauma symptomatology in early childhood is typically done by observing the child in different settings, for instance during play, and by using adult reports about the child’s mental health. Reliability of adult reports can however be questionable if caregivers are unaware of the child’s experiences or symptoms, if caregivers are traumatized themselves, or if it is the caregivers who caused the trauma (5, 6). Studies have found that parents tend to underreport child symptomatology compared to child-reports of symptoms (7), and especially internalized symptoms may be more difficult than externalized symptoms to notice (6, 8, 9).

Play observations can add to the assessment and individual case formulation but is typically not standardized and rely on clinical experience alone. Story stem assessment tools combine storytelling and play in a structured and controlled setting to access the internal world of young children in an age-appropriate and sensitive manner with the child as informant (10). This paper examines reliability and validity of a new, standardized story stem assessment tool, the Odense Child Trauma Screening (OCTS; 11), that was developed by the authors to screen for behavioural and play-based indicators of traumatization in children aged 4 to 8 years.

Story stem assessment tools

The technique of a narrative story stem assessment tool involves the child being presented with the beginning of a story, i.e. a stem, which the test administrator acts out with affect using a standard set of toys, typically family or animal figures. The story stem consists of an everyday problem with a controlled degree of conflict or distress that is intended to induce emotional arousal in the child (10). When the administrator stops the story stem, the child is asked to continue and play out the story with the figures. When asked to continue the story, the child spontaneously draws on own impulse and mental scripts of social relations and behavior to finish the story (12-14). Story stem tools include several story stems, and the child’s play and narrative representation of feelings and behavior are coded for each story. Use of a story stem tool thus allows for a detailed examination of child mental representation of self, others, and relationships, which have been posited to have a mediating role in how early childhood adversity affects later development (13, 15).

Depending on the purpose of the assessment, different story stem tools with varying story stems and coding schemes can be used. Generally, story stem tools such as the Attachment Story Completion Task (ASCT; 16), the Manchester Child Attachment Story Task (MCAST; 17), the Attachment Doll Play Assessment (ADPA; 18), and the German Attachment Story Completion Procedure (GASCP; 19-21) have been developed to assess attachment in preschool and young school children, while the MacArthur Story Stem Battery (MSSB; 22) and the Story Stem Assessment Profile (SSAP; 5) in addition to attachment, also more broadly assess child mental representations of self, others, expectations of relationships, moral rules, moral emotions, and competences.

Story stem responses and child symptomatology

Story stem tools have been widely applied in research to examine typically developing children (14, 23) and clinical groups of children with externalizing and disruptive disorders (24-26), with anxiety and mood disorders (23, 27-29), and with children exposed to child maltreatment (5, 13), where the link between play and narrative representations in story stems and symptomatic behavior have been investigated.

Specifically, externalizing and disruptive behaviors have been found to be associated with less compliance and with negative narrative themes and representations (24) such as parental injury and role reversal (25) and disorganization (26). Furthermore, negative representations of self and others have been found to significantly predict internalized symptoms and anxiety symptoms (30); and symptoms of depression have been associated with negative adult representations (27) and narrative themes of shame (28).

In samples of children exposed to child maltreatment negative representations of parents and self, representations of a grandiose self (31), avoidant strategies, rejection, death (15), aggression (32), and role reversal (33) have been identified. Furthermore, children who have experienced neglect, but no abuse, have been found to show more passive parents in relation to child figure distress, whereas children exposed to abuse, especially physical abuse, showed narrative representations with anomalous acts (33).

Lastly, disorganized phenomena displayed within the setting of story stem measures of attachment are found more frequently in children with a trauma history of maltreatment compared to typically developing children (15, 34). Disorganized attachment is characterized by disturbed, conflicted, anomalous, or fearful behavior and a lack of a clear and cohesive, organized attachment strategy (35, 36). Disorganized attachment in children is assumed to develop because the caregiver, who should be a source of comfort and protection, is the source of fear, for instance because the caregiver is the source of maltreatment or because trauma-triggered adult behavior in a traumatized parent, is experienced as frightening and unpredictable by the child (37, 38). Disorganized attachment has shown to be a risk factor for psychopathology of both externalizing and internalizing problems, in childhood and adolescence (38). Specifically, MacDonald and colleagues (39) demonstrated in their longitudinal study that a disorganized attachment in children of 12 months of age significantly predicted a higher posttraumatic stress disorder (PTSD) symptom clusters of re-experience and of avoidance in the children at age 8.5 years but disorganized attachment was not associated with higher level of symptoms of anxiety disorders.

Development of a story stem screening for traumatization

Inspired by the findings described above, the authors decided to develop the OCST (11); a story stem screening tool that, within a controlled and structured play-setting, could function as an indirect screening measure of traumatization for children aged 4-8 years, and a tool that would include the child as informant. The OCTS differs from other story stem tools as it is intended as a screening tool and was developed specifically to be utilized in clinical practice for a broad trauma population where the circumstances and nature of the trauma experience of the child for various reasons may be more or less unknown.

Other story stem tools such as the MSSB and the SSAP are characterized by being comprehensive story stem batteries with several story stems applicable for a thorough and detail-rich assessment. The MSSB has been used in various studies on typically developed children and specific clinical groups in which story stems and coding schemes have been developed to fit the aim of the study and the construct under examination (for a review, please see; 12). The SSAP was developed specifically for assessment of maltreated children (5, 40). The MCAST includes fewer story stems and is characterized by rigorous methodology and detailed scoring and classification of attachment, attachment behavior, and attachment strategies. However, in our experience, the use of comprehensive story stem batteries, the detailed level of coding, and the somewhat extensive and time-consuming labor connected to the existing story stem measures is an issue of concern when incorporating the research-based story stem approach into general clinical practice, where children with a broad range of mental health issues are referred.

The development and adaptation of the OCTS administration and coding scheme were therefore designed to include a relatively short story stem administration with five story stems, standardized rules of interaction and follow-up questions, and a relatively short coding scheme and scoring system to be used in cases of suspicion of child traumatization. The scoring was designed drawing on the vast empirical knowledge from other studies with story stem measures describing characteristics of narratives presented by children exposed to potential traumas. For a full review of the empirical findings on which the codes of the OCTS was derived, please see Eriksen & Elklit (41). The OCTS codes comprise categories of the child’s engagement and production of the narrative, nature of the narrative (e.g., narrative coherence), representations of adults and child in the narrative, and disorganized phenomena (42).

Study aim

The present study aimed to provide initial examination of the reliability and validity of the OCTS as a screening tool for behavioural and play-based indicators of traumatization in children aged 4 to 8 years. Specifically, the study set out to examine internal consistency, interrater reliability, and test-retest reliability of the OCTS. Furthermore, the study examined multi-method convergent validity of the OCTS when compared to Danish validated measures, specifically modules of PTSD, major depressive disorder (MDD), and reactive attachment disorder (RAD) from the Diagnostic Infant and Preschool Assessment (DIPA) and the Strengths and Difficulties Questionnaire (SDQ) Total Difficulties Scale and subscales. These measures were chosen because PTSD and RAD are trauma related disorders, and symptoms of depression have been shown to be highly comorbid with PTSD (4, 43). Furthermore, exposure to multiple adverse or traumatic events have been shown to be associated with a broad range of internalizing and externalizing symptoms in children (4). Lastly, the ability of the OCTS to discriminate between a risk sample of children exposed to trauma and a community sample was explored.

Methods

Participants

A total of 49 children participated in the study. Parents and caregivers of 46 children participated in an interview about their child. The study included two samples. Inclusion criteria were Danish children age 4 to 8 years old. Parents or foster parents had to speak Danish or English at a functional level to take part in the interview. The children had to speak and understand Danish at a functional level to participate in the OCTS. For sample 1, the risk sample, the child had to have experienced at least one potentially traumatic event.

Sample 1: Participants were recruited from three child mental health clinics and the Southern Regional Children Center that assess children in cases of suspicion of physical or sexual abuse. Fifty-five children and their parents were invited. For 14 children, one or both parents with child custody declined to participate. Reasons for not participating were mainly the wish to limit further assessment of the child at a time with high level of family crisis, extensive legal case proceedings or ongoing psychological assessment. Parents of eight children agreed to participate but seven did not respond to several contact attempts to schedule interviews or withdrew due to lack of mental energy or resources, and one child did not want to be filmed for the OCTS and was therefore not included in the study. Thirty-three children and their parents and/or foster parents accepted to participate and were included in the study. Two children were excluded as coding was not possible due to one film being deleted by mistake and one child not engaging in sufficient symbolic play for reliable coding. Figure 1 illustrates flow of participant recruitment. The final sample comprised 31 children. Sample characteristics of the children are displayed in Table 1.

FIGURE 1.

Flow of participant recruitment

TABLE 1.

Sample characteristics of the children

	RiskN = 31; n (%)	CommunityN = 18; n (%)
Gender n (%)
Male	19 (61.3)	10 (55.6)
Female	12 (38.7)	8 (44.4)
Age (years)
Min-max	4.5-8.9	4.5-8.7
M (SD)	6.6 (1.3)	6.7 (1.2)
Ethnicity
Caucasian	24 (77.4)	18 (100.0)
Arabic	3 (9.7)	0 (0.0)
Asian	1 (3.2)	0 (0.0)
Mixed	3 (9.7)	0 (0.0)
Diagnosis beforehand n (%)
No	24 (77.4)	15 (100.0)
Yes	5 (16.1)	0 (0.0)
Siblings
Min-max	0-6	0-2
M (SD)	2.0 (1.9)	1.2 (.6)
Age mother (years)
Min-max	21-45	31-46
M (SD)	32.9 (6.4)	38.2 (4.9)
Age father (years)
Min-max	27-54	34-48
M (SD)	38.4 (6.3)	39.6 (4.3)
Danish Citizenship n (%)
Mother	30 (96.8)	15 (100.00)
Father	30 (96.8)	15 (100.00)
Mother job status n (%)
Full-time employed	17 (54.8)	11 (73.3)
Stay-at-home mother	4 (12.9)	2 (13.3)
Studying / job seeking	6 (19.4)	2 (13.3)
On sick-leave	4 (12.9)	0 (0.0)
Father job status n (%)
Full-time employed	16 (51.6)	15 (100.0)
Stay-at-home mother	3 (9.7)	0 (0.0)
Studying / job seeking	2 (6.5)	0 (0.0)
On sick-leave	7 (22.6)	0 (0.0)
In prison	1 (3.2)	0 (0.0)
Dead	2 (6.5)	0 (0.0)
Mother has a diagnosis n (%)	11 (35.5)	1 (6.7)
Father has a diagnosis	13 (41.9)	0 (0.0)
Parents relationship status
Married/partners	8 (25.8)	14 (93.3)
Separated/divorced	18 (58.1)	1 (6.7)
Never in a relationship	4 (12.9)	0 (0.0)
Placed in foster care n (%)	5 (16.1)	0 (0.0)

Information on the community sample are, except for gender, age and ethnicity, based on 15 children because caregiver interviews were missing for three children

Sample 2: was a community sample of children recruited from two elementary schools and a kindergarten. Parents were informed about the study on the schools’ and kindergarten’s intranet and invited to participate. Parents of 19 children consented for their child to participate. All 19 children were included. However, one child was excluded from the sample due to missing data from the OCTS retest due to holiday and missing data on the parent interview. Parents of three children did not participate in the parent interview because after several contact attempts the parents did not call back to schedule interviews. The final sample included 18 children and parents of 15 children.

Procedures

Data collection was conducted between December 2015 and December 2018 in the Southern Region of Denmark. All parents received written and oral information about the study. All parents with child custody provided written consent for them and their child to participate. Children were in an age-appropriate manner given an explanation about the OCTS and the purpose of the study and gave oral consent.

The community sample children participated in OCTS test-retests. Test-retests were administered with 4-5 weeks apart. Parents in the community sample participated in the caregiver interview within two weeks after child participation in the first OCTS. For the risk sample, children and caregivers participated one time. OCTS retests were not conducted with the risk group because the children were in the middle of a psychological assessment and thereafter, if necessary, referred to treatment.

Test administrators were psychologists (n = 6) and psychology students (n = 7). Administrators in the risk sample were required to have clinical experience with children and trauma. All test administrators received training and supervision in administration of the measures. The OCTS was administered as the first measure with administrators being blind for information from the other measures. After the OCTS, two other measures were administered as caregiver interviews. For scoring the OCTS, all coders took part in a two-day training in administration and coding of the OCTS, rehearsed the coding with three to four extra films, and subsequently received supervision on their coding of the rehearsed films to become reliable coders. All OCTS films, both tests and retests, were double coded by six reliable coders, all psychology students, blind to all information about the child. One coding was a priori designated as the primary coding and used for all analyses other than the interrater reliability analysis. In difficult cases consensus ratings were made. The study was approved by the Regional Committee on Health Research Ethics for Southern Denmark (S-20150009).

Measures

Odense Child Trauma Screening (OCTS; 11) is a story stem screening tool for children aged 4-8 years, that was developed to screen for indicators of traumatization based on observations of the child’s behaviour and narratives during a structured and controlled play setting. The OCTS includes five story stems using family figures in their house and an optional animal story stem. The administrator narrates and acts out the story stem and then stops at the most intense time in the story and the child is asked to explain what happens next in the story both verbally and through the use of the figures. The figures include a main child figure and a sibling, both of same gender as the child being tested, and a mother and a father. To ensure a certain amount of displacement between experiences of the child subject and the main child figure the child and administrator agree upon a name for the main child figure that is different to the name of the child being tested.

The story stems in the OCTS are 1) a warm-up baseline story stem with a birthday theme originating from the MSSB, 2) the Bike story stem from the SSAP, 3) the Nightmare story stem from the MCAST, 4) the Burnt Hand story stem adapted from the original Hot Gravy story stem in the MSSB, 5) the Stomach Ache story stem based on the Illness story stem in the MCAST, and an optional 6) Animal story stem adapted from the Little Pig story stem in the SSAP. The warm-up baseline story is without a central conflict and is included to show and teach the child how to play during a story stem test. In addition, the child completion of the baseline story stem will give an indication of the child’s general behavior and play behavior during a story without a central conflict. The remaining story stems all have everyday themes including a central conflict or child distress. The optional story stem with animal figures can be included if the story stems with family figures have not provided enough material to conduct a reliable coding. For the present validity study, the animal story was included in the OCTS tests. Standard rules for administrator-child interaction and follow-up questions are used.

The OCTS is filmed and coded based on the film material. The OCTS includes 27 themes called “codes”. The codes are rated separately for each story stem completion following the directions in the coding manual. The 27 codes are organized into five different categories: ‘Engagement and narrative production’, ‘Nature of the narrative’, i.e. the quality and quantity of the produced narrative, ‘Adult representations’, ‘Child representations’, and ‘Disorganized phenomena’. The category of ‘Engagement and narrative production’ was added to make sure the premise of a story stem methodology is met, i.e. that the child experiences arousal when handed over the story stem and how the child tries to modulate the arousal. The category of ‘Nature of the narrative’ was added to assess difficulties with narrative coherence as this has been found in risk and clinical groups of children (e.g., 24) and to assess how much material the child was able to produce with or without interviewer support. The categories of ‘Adult’ and ‘Child representation’ and ‘Disorganized phenomena’ were added as the codes within these categories have been frequently found in studies of children exposed to different types of potentially traumatic experiences and which have used different story stem assessment tools (41). The categories, their codes and the scores are presented in Supplementary table 1, and a thorough description of scores of each code is described in the coding manual (42). All codes are assigned a raw score on a three-point scale (from 0 = “phenomenon described in the manual not present” to 2 = “definitely present”) except for code 1-4 where a dichotomous rating of 0 or 2 is used. The raw scores are recoded into weighted scores of 0 or 1. The differential weighing is based on the empirical findings that while some representations are primarily seen in relation to traumatization others are also seen in children with other symptomatic behavior such as mood or behavioral disorders (5, 13, 15, 23-33, 41). A partial score for each conflict/distress narrative is calculated by summing up the weighted scores for each story. No partial score is calculated for the baseline story. A total score is calculated by summing up partial scores for each conflict/distress story and dividing by the number of conflict/distress stories (42). A high score on the OCTS indicates that the child screens positive for potential traumatization, and further assessment is needed. The OCTS administration and coding manuals are available in Danish and English. For the present study, the Danish versions were used.

Diagnostic Infant and Preschool Assessment (DIPA; 44) is a semi-structured, diagnostic caregiver interview for assessment of young children covering symptoms of 13 different disorders in self-contained modules. For this study, three modules for PTSD, MDD, and RAD were used. The DIPA has been validated with American, Dutch, and Danish children aged 1-7 (44-46). Symptoms are scored “1” if present and “0” if not present. The Danish version used in this study is based on disorders of DSM-IV and the developmentally modified diagnostic criteria (Research Diagnostic Criteria – Preschool Age; RDC-PA). This version of the DIPA was chosen to examine convergent validity OCTS because this version of the DIPA has been used and validated with Danish children (43, 47). No Danish version of the DSM-5 updated DIPA exists. Follow-up questions were developmentally adapted for children aged 7-8 to match the study sample. Diagnostic algorithms for the three disorders were based on DSM-IV-TR (48) for children aged 7-8. For children aged 4-6 years, the developmentally modified diagnostic criteria, RDC-PA, were used (44, 45). Summed continuous scores for each module were used for analyses. Internal consistency for the total PTSD scale was excellent (α = .91) and acceptable to good for PTSD subscales (α = .75-86), excellent for the depression scale (α = .91), but poor for the RAD total and sub-scales (α = .50-55).

The Strengths and Difficulties Questionnaire (SDQ; 49) is a brief screening questionnaire of child behavior including 25 items each scored on a three-point Likert scale from 0 (not true) to 2 (certainly true). Items are divided into five subscales of hyperactivity, emotional symptoms, conduct problems, peer problems, and prosocial behavior. The first four scales are summed up to a Total Difficulties Scale. The SDQ has been validated in Denmark and other Nordic countries for children aged 5 years and up (50-53) and Danish norms are available (54). In this study, the parent-report version was used. Internal consistency was acceptable to good for all scales (α .78 to .88) except the conduct problem subscale (α = .62).

Statistics

Internal consistency was examined using Cronbach’s alpha. OCTS interrater reliability and test-retest reliability were calculated using intraclass correlation coefficients. Multi-method convergent validity of the OCTS was examined by comparing scores on the OCTS with total amount of symptoms of PTSD, MDD, and RAD from the DIPA and with the scales of the SDQ using Spearman’s rho because scores on the OCTS and DIPA were not normally distributed. Spearman’s rho was used to assess associations between total number of trauma exposures and the total score on the OCTS. Guidelines for interpreting the effect size of the correlations followed Cohen (55): r = 0.10 indicating a small effect, r = 0.30 indicating a medium effect, and r = 0.50 indicating a large effect. Lastly, Mann-Whitney tests were conducted to examine group differences in scores and explore the ability of the OCTS to discriminate between the risk sample and the community sample.

Results

Descriptives

Sample characteristics of participants are shown in Table 1. Of participants from the risk sample, 21 children were recruited from the Southern Regional Children Center and 10 from a mental health clinic.

Table 2 shows the number of children in each sample who had been exposed to the different types of traumas as reported in the DIPA. The children from the risk sample had experienced between 1 and 5 traumas (M = 2.77, SD = 1.2) and the children from the community sample between 0 and 3 traumas (M = 1.07, SD = .8). There was a significant difference in number of trauma between the two samples, t(41) = 4.98, p < .000. In addition to the first 11 trauma types in the DIPA, other traumas or distressing experiences was reported by caregivers for 61.3% of the children (n = 19) from the risk sample and for 26.7% of the children in the community sample (n = 4). In the risk sample other traumas included death of a caregiver or sibling, parental mental disorder, parental alcohol or substance abuse, severely problematic parental divorce, neglect, psychological violence, and placing in foster care or other acute separation from primary caregiver. Eighteen of the 19 children from the risk sample had also experienced at least one of the 11 first trauma types together with the 12th ‘other trauma’. In the community sample, other traumas included severe chronic illness of a close family member and periods of acute separation from primary caregiver. Three out of four children from the community sample had only experienced the 12th ‘other trauma’ while one had also experienced one of the first 11 trauma types categorized in the DIPA.

TABLE 2.

Trauma exposure

Trauma type	Riskn (%)	Communityn (%)
Physical abuse	18 (58.1)	0 (0.0)
Witnessed violence	16 (51.6)	0 (0.0)
Hospitalization or invasive medical procedures	14 (45.2)	8 (53.3)
Sexual abuse	10 (32.3)	0 (0.0)
Accidental burning	4 (12.9)	0 (0.0)
Man-made disasters (fires, wars, etc.)	2 (6.5)	0 (0.0)
Attacked by animal	1 (3.2)	1 (6.7)
Near drowning	1 (3.2)	1 (6.7)
Natural disasters (hurricane, tornado, flood, etc.)	1 (3.2)	0 (0.0)
Traffic accident	0 (0.0)	2 (13.3)
Kidnapped	0 (0.0)	0 (0.00)
Other	19 (61.3)	4 (26.7)

Note. Risk N = 31, community N = 15

Scores on the OCTS, the DIPA, and the SDQ are depicted in Table 3. Thirteen out of 31 children (41.9%) from the risk group displayed symptoms to fulfill the diagnostic criteria for PTSD (i.e. the RDC-PA for 4-6-year olds or the DSM-IV for 7-8-year olds). Eight children (25.8%) fulfilled the diagnostic criteria (RDC-PA) for MDD, and two children (6.5%) fulfilled the criteria for RAD dishibited subtype. Of the ten children who fulfilled the diagnostic criteria for either MDD or RAD dishibited subtype, only one child with MDD did not fulfill the diagnostic criteria for PTSD. For the risk sample, scores on the SDQ Total Difficulties Scale was high. Scores on the subscale of emotional problems were slightly raised, and scores on the subscales of conduct problems, hyperactivity, and peer problems, were bordering “close to average” and “slightly raised” compared to Danish norms. For the community sample, all scores on the SDQ scales were within the normal range.

TABLE 3.

Scores on the OCTS, DIPA, and the SDQ

Scale	Risk, N = 31				Community N = 18
Mdn	M	(SD)	Range	Mdn	M	SD	Range
OCTS total	4.3	5.03	(2.7)	1.80-11.60	1.9	2.40	(1.9)	.20-4.60
Bike	3	4.03	(3.6)	0-14	1	1.83	(1.6)	0-5
Nightmare	5	5.47	(3.3)	0-13	2	2.56	(2.5)	0-9
Burned hand	5	5.30	(3.0)	0-13	2	2.44	(1.8)	0-6
Stomach ache	5	5.31	(3.6)	0-13	2	2.56	(2.1)	0-7
Animal	5	4.93	(3.1)	1-12	2	2.61	(2.1)	0-8
DIPA						N = 15
PTSD	7	7.16	(5.0)	0-18	0	0.00	(0.0)	0
Re-experience	3	3.42	(2.6)	0-8	0	0.00	(0.0)	0
Avoidance	1	1.48	(1.7)	0-6	0	0.00	(0.0)	0
Hyperarousal	2	2.26	(1.9)	0-6	0	0.00	(0.0)	0
MDD	1	3.68	(4.5)	0-15	0	0.00	(0.0)	0
RAD	1	1.29	(1.1)	0-4	0	.13	(.5)	0-2
SDQ							N = 15
Total difficulties	15	16.06	(6.9)	5-33	4	5.13	(3.2)	1-12
Conduct problems	3	3.10	(1.7)	0-7	0	.53	(.7)	0-2
Emotional problems	6	5.45	(2.8)	0-10	0	2.07	(1.9)	0-6
Hyperactivity	5	5.00	(2.8)	1-9	2	2.20	(2.0)	0-8
Peer problems	2	2.52	(2.6)	0-9	0	.33	(.8)	0-3
Prosocial behavior	9	8.23	(1.8)	5-10	9	8.07	(2.5)	2-10

Note: OCTS = Odense Child Trauma Screening; DIPA = Diagnostic Infant and Preschool Assessment; SDQ = Strengths and Difficulties Questionnaire

Five children from the risk sample was placed in foster care. No statistical difference was found between the children placed in foster care and the rest of the risk sample in regard to number of traumas experienced, nor on scores of the OCTS, the DIPA PTSD scales, MDD scale or the RAD scale. A Mann-Whitney test did however demonstrate that the SDQ Total Difficulties Scale was significantly higher for children placed in foster care (Mdn = 24.00) than for the rest of the risk sample (Mdn = 13.50), U = 19.50, z = -2.45, p < .05, r = -.68. Also, the SDQ subscale of hyperactivity was significantly higher for children in foster care (Mdn = 7.00) than for the rest of the risk sample (Mdn = 4.00), U = 26.50, z = -2.09, p < .05, r = -.58.

Reliability

Internal consistency

Cronbach’s Alpha coefficients for the OCTS are reported in Table 4. The total score for the OCTS demonstrated excellent internal consistency. Internal consistency of the partial score for each story ranged from acceptable to good.

TABLE 4.

Reliability of the OCTS (N = 49)

	Internal consistency Cronbach’s α [95% CI]	Items	Interrater reliability ICC [95% CI]
Total		.95 [.93, .97]	135	.98 [.96, .99]
Bike	.85 [.79, .91]	27	.96 [.94, .98]
Nightmare	.80 [.70, .87]	27	1.00 [1.00, 1.00]
Burned hand	.79 [.69, .86]	27	1.00 [1.00, 1.00]
Stomach ache	.84 [.77, .90]	27	1.00 [1.00, 1.00]
Animal	.79 [.70, .87]	27	1.00 [1.00, 1.00]

Note. OCTS = Odense Child Trauma Screening; CI = Confidence intervals; ICC = Intraclass correlation. Interrater reliability are based on double blind coding of 67 films

Interrater reliability

Calculations of interrater reliability for the OCTS are based on 67 films. Results are shown in Table 4. Intraclass correlation coefficient was excellent for the total score of the OCTS and for all partial scores.

Test-retest

The calculation of test-retest reliability was based on a sample of 18 OCTS test-retests. Intraclass correlation for test-retest of the total scores of the OCTS was .67, 95% CI [.15, .87], p = .006.

Validity

Multi-method convergent validity

Multi-method convergent validity of the OCTS as a screening measure of traumatization was examined by comparing the scores of the OCTS to continuous scores on the DIPA scales of PTSD, MDD, and RAD, and scales of the SDQ. Correlation coefficients between OCTS scales and DIPA and SDQ scales are shown in Table 5. Medium, positive significant correlations were found between the OCTS total score and the DIPA PTSD scale, MDD scale and the total RAD scale. Furthermore, there was a significant association between number of traumas and total score of the OCTS, rho = 0.48, p = .001. Medium, positive significant correlations were also found between the OCTS total score and the SDQ Total Difficulties Scale and the subscales of conduct problems, hyperactivity and peer problems.

TABLE 5.

Spearman’s rho correlations between OCTS scales and DIPA and SDQ scales (N = 46)

	OCTS total	Bike	Nightmare	Burned hand	Stomach ache	Animal
DIPA PTSD total	.42**	.27	.32**	.33**	.27	.36**
Re-experience	.42**	.26	.29**	.36**	.28	.40**
Avoidance	.28	.13	.26	.16	.17	.20
Hyperarousal	.39**	.26	.29**	.28	.28	.32**
DIPA MDD	.34**	.26	.27	.25	.22	.29
DIPA RAD total	.31**	.19	.36**	.22	.21	.28
Inhibited	.24	.14	.27	.12	.17	.32**
Dishibited	.19	.07	.27	.23	.20	.11
SDQ total	.41**	.31**	.26	.33**	.29	.41**
SDQ conduct problems	.33**	.15	.31**	.32**	.27	.28
SDQ emotional problems	.21	.05	.13	.11	.13	.34**
SDQ hyperactivity	.39**	.43**	.16	.33**	.29**	.27
SDQ Peer problems	.30**	.26	.19	.29	.07	.33**
SDQ prosocial behavior	.09	-.05	.06	-.05	.23	.07

Note: OCTS = Odense Child Trauma Screening; DIPA = Diagnostic Infant and Preschool Assessment; PTSD = posttraumatic stress disorder; MDD = major depressive disorder; RAD = reactive attachment disorder; SDQ = Strengths and Difficulties Questionnaire.

p < .05 (2-tailed);

p < .01 (2-tailed)

Discrimination between risk and community sample

The Mann-Whitney test demonstrated that the OCTS total score was significantly higher for the risk sample (Mdn = 4.33) than for the community sample (Mdn = 1.90), U = 96.00, z = -3.40, p < .001, r = -.54.

Discussion

The aim of this study was to provide an initial examination of reliability and validity of the new story stem tool OCTS, that was developed to screen for indicators of traumatization in children aged 4-8 years based on child play behavior and narrative representations. The OCTS builds on the knowledge and methodology of existing story stem assessment tools such as the MSSB, the SSAP, and the MCAST but differs in its intended use as a screening measure for implementation in clinical practice for use with a broad trauma population of young children. The OCTS includes four known and widely applied story stems with a central conflict or distress, which we expected to work well with a broad child trauma population to elicit the child–s mental representations and strategies in different areas, and a shorter and more condensed coding scheme inspired by several coding systems from story stem studies.

The present validity study included two samples of Danish children; a risk sample of children exposed to traumatic experiences, where almost half of the children displayed symptoms consistent with a diagnosis of PTSD or MDD and a community sample, where none of the children displayed symptoms of neither PTSD nor MDD. The OCTS demonstrated good internal consistency, excellent interrater reliability, and acceptable test-retest reliability, which shows promise for the utility and implementation of the OCTS, a child-informant screening measure, into a broad range of Danish clinical practices. Furthermore, this study is, to our knowledge, the first to provide test-retest reliability of a story stem test. Other studies have included re-examination of children with story stem measures for other purposes, e.g. Green and colleagues (17) and Hodges and colleagues (15), but not to examine test-retest reliability. Our findings of test-retest reliability of the OCTS with a community sample was acceptable given the small number of children who were retested. To account for a small sample size, 95% CI–s were calculated, and estimated range of outcome were provided. Further exploration of test-rest reliability is therefore needed using a larger sample size. As for now, our findings illustrate that the OCTS can be re-administered to the same children and the results of repeated tests can be deemed reliable, despite the children being familiar with the content and arousal intensity of the story stems.

The study furthermore provided initial examination of the multi-method convergent validity of the OCTS as a screening measure of traumatization by comparing the scores of the OCTS to scores on the DIPA scales of PTSD, MDD, and RAD, and scales of the SDQ. The OCTS total score was significantly correlated with the total PTSD scale and the re-experiencing and hyperarousal subscales as well as the MDD scale and total scale of RAD. Additionally, significant correlations were found between the OCTS total score and the SDQ Total Difficulties Scale and SDQ subscales of conduct problems, hyperactivity, and peer problems. The associations between the OCTS and the DIPA PTSD scales and SDQ Total Difficulties scale were definite but moderate. However, the results are comparable to results of other studies where story stem measures have been compared to adult-reported child symptoms (e.g., 25-30 with correlation coefficients ranging between .12 and .41).

A possible explanation of the moderate association between the measures in the present study may be that the scores of the OCTS are compared to parent-reported symptoms and problems instead of another child-report measure. Indeed, low to moderate correlations between child- and parent-reports of child mental health have been found in various studies (9), and establishing convergent validity of child-informant based measures of traumatization and trauma symptomatology is generally challenged by the lack of age-appropriate self-report measures for this young age group. However, collectively the link between OCTS and DIPA PTSD, MDD, and RAD symptoms, SDQ scales, as well as the significant association between OCTS and number of trauma experiences provides initial evidence of the validity of the OCTS as a screening measure for young children. A high score on the OCTS then calls for further assessment of the child for the results to be confirmed, elaborated on or contradicted, potentially including differential diagnostics.

A note must also be made regarding the findings of non-significant associations between the OCTS and SDQ subscale of emotional problems. Based on findings from other story stem studies (27, 30) and the theoretical foundation of story stems measures in attachment theory, a significant albeit smaller correlation was expected. Possible explanations for the non-significant finding entails a small sample size in the present study, the composition of the sample with children exposed to traumas of primarily abuse or family violence or underreporting of child symptoms by primary caregivers. Futh and colleagues (26) did not find a significant association between disorganization as assessed on the MCAST and mother-reported SDQ emotional problems, and disorganized phenomena are one of the central categories of the OCTS coding system. Future studies on associations between the OCTS and measures of anxiety is warranted to elaborate on this relation.

Furthermore, the ability of the OCTS to differentiate between the trauma sample and the community sample adds to the initial evidence supporting the OCTS as a valid screening measure of play behavior and narrative representations indicative of traumatization in preschool and young school children. To strengthen this evidence further it would be relevant to replicate this finding in a larger sample. Also, the ability of the OCTS to discriminate, not just between a trauma sample and a community sample, but also between a sample of traumatized children and different clinical samples with oppositional defiant disorder, conduct disorder, or mood disorders must be examined. Furthermore, it is imperative to examine this issue in relation to the ability of the OCTS to differentiate between traumatized children and children with developmental disorders in which atypical behavior or narrative content must not be misinterpreted as indicators of the child being traumatized.

In addition to small sample size, other study limitations must be mentioned. Firstly, even though the children from the risk sample were recruited from several mental health facilities from different parts of the Southern Region of Denmark, most of the sample were Caucasian children who had primarily been exposed to abuse or family violence. A more diverse sample in terms of ethnicity and exposure to other non-interpersonal traumas would improve the generalizability of the results. However, the significance of this issue may not be alarming, as the study was a validity study of the OCTS with Danish children, and the majority (87%) of the population in Denmark are of Danish (i.e. Caucasian) ethnicity, and about 6% are immigrants from non-Western countries (56). Secondly, because the OCTS is filmed, we needed written informed consent from all parents with child custody of the children participating in the study as judged by the Regional Committee on Health Research Ethics for Southern Denmark and the Danish Ombudsman. Inevitably, this limited recruitment of children who had been maltreated by a parent with full or joint custody of the child or children for whom one parent with joint custody were unreachable. This can have caused selection bias despite considerable efforts to recruit children who for example were placed in foster care. When children who were placed in foster care were compared to the rest of the risk sample, no significant differences were found on trauma-specific scores from the OCTS and DIPA. Children in foster care did however score significantly higher on the SDQ indicating higher levels of overall problems compared to the rest of the risk group.

Furthermore, in the present study for legal and ethical reasons all administrations of the OCTS in the risk sample was done by psychologists. Administration of the OCTS in the community sample was done by one clinical psychologist and psychology students. The difference in clinical experience of administrators might have influenced our findings and future studies would benefit from having OCTS administrators with the same amount of experience of test administration when examining reliability and validity of the OCTS. Also, for this study all films were coded by psychology students to examine inter-rater reliability. Further examination of inter-rater reliability of coders with long clinical experience and student coders could strengthen a future study and use of the OCTS.

Lastly, potential age and gender differences in scores on the OCTS in were not explored due to the small sample sizes of the risk sample and the community sample. This should be addressed in future studies with larger samples. Indeed, Gloger-Tippelt and Kappler (21) illuminated on such variations related to gender and age within different samples in their study of a pooled analysis of 22 samples where 887 children were assessed with the German story stem measure GASCP. Gloger-Tippelt and Kappler (21) found that girls were 0.4 times less likely to display disorganized narratives compared to boys when controlling for risk status and age. Also, children from risk samples (i.e. children who experienced various types of maltreatment, relationship disruptions, or parental mental disorder) were 5.4 times more likely to display disorganized narratives compared to children from no-risk samples. The effect of risk was not moderated by age or gender.

Clinical significance

The present study provides preliminary evidence of the reliability and validity of the Danish OCTS, which was developed to screen for play-based behavior and narrative representations indicative of traumatization in children 4-8 years of age. The OCTS combines techniques of play observation and strictly verbal interviews in a structured and controlled setting with symbolic play and storytelling to give insight into child mental representations of self and others, emotions, and behaviour in a way that can be psychometrically evaluated.

Despite the previously mentioned limitations, the OCTS shows promise in being a reliable and valid story stem screening tool for young children with a relatively short administration and coding time. Implementation of a story stem measure as the OCTS into Danish clinical context provides clinicians with an age-appropriate, standardized screening tool that can be utilized with children, who may not otherwise be assessed with strictly verbal, structured interview measures. Moreover, the play and storytelling format provides a safe setting that can open up for direct assessment of potentially vulnerable children who for various reasons may hold back during the assessment situation. Importantly, the OCTS can be applied in cases where the nature of the trauma is more or less unknown, or in cases where parents may not be reliable informants about their child–s mental health. Also, when caregiver reports can be applied, a story stem measure as the OCTS can provide valuable information that may go beyond what can be observed and described by caregivers and teachers, such as internalizing problems or child expectations of self and others.