Convergence between observations and interviews in clinical diagnosis of reactive attachment disorder and disinhibited social engagement disorder.

OBJECTIVE: A comprehensive approach is needed for diagnosing disordered attachment behavior due to the multifaceted nature of attachment. Differences between various indicators can pose a challenge for deciding on the proper diagnosis. This study assessed the convergence between clinical interview assessment and observation-based clinical diagnosis, and their linkages with inadequate care.
METHOD: Participating children ( N = 55) had intelligence quotients (IQs) between 50 and 85 and were referred for psychiatric consultation. Data were obtained by structured review of medical records, the Disturbances of Attachment Interview (DAI), and the Clinical Observation of Attachment (COA) procedure.
RESULTS: Of the 18 children identified using the DAI with Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (DSM-5) diagnosis of reactive attachment disorder (RAD) and/or disinhibited social engagement disorder (DSED), only 7 received a clinical DSM-5 diagnosis of RAD and/or DSED. Observed maladaptive attachment behavior in the COA was strongly associated with DAI scores and with clinical diagnosis of DSM-5 RAD and/or DSED. There was a significantly higher prevalence of extremes of insufficient care in children who were classified with RAD by DAI or DSM-5 and/or with DSED by DSM-5 compared to those with no attachment disorder.
CONCLUSIONS: Using structured observation and record assessment leads to more conservative identification of RAD or DSED than using the DAI.

Introduction

While research on attachment based on theoretical and empirical work by John Bowlby (e.g. Bowlby, 1984) and Mary Salter Ainsworth (e.g. Ainsworth, Blehar, Waters, & Wall, 1978) has transformed our overall understanding of socioemotional development and human relationships, progress has been more modest with respect to the etiology, diagnosis, and treatment of attachment-related disorders. The latest revision to a well-known definition of attachment-related disorders was the advent of the Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5; American Psychiatric Association (APA), 2013), 33 years after reactive attachment disorder (RAD) appeared in the DSM-III (APA, 1980). In the DSM-5, RAD is described as a pattern of disturbed social behavior similar to the emotionally withdrawn-inhibited subtype of RAD, as defined in the previous edition of the DSM (DSM-IV) (APA, 1994). In contrast, the pattern previously subclassified as indiscriminate social-disinhibited RAD now qualifies for the diagnosis of disinhibited social engagement disorder (DSED). These changes were the result of theoretical and empirical work on the psychosocial functioning of children with backgrounds that were deemed to be pathogenic for attachment, such as children growing up in orphanages, foster care, and children exposed to maltreatment (Zeanah & Gleason, 2010, 2015).

The indicators of disordered attachment that diagnosticians in clinical practice should take into account have received relatively little research attention. While practice guidelines state that a variety of information sources should be accessed, all of which should yield converging data (Boris, Zeanah, & The Work Group on Quality Issues, 2005; Chaffin et al., 2006; Zeanah, Chesher, Boris, & The American Academy of Child and Adolescent Psychiatry (AACAP) Committee on Quality Issues (CQI), 2016; Zilberstein & Popper, 2014), the extent to which such convergence can be found in practice is largely unknown across clinical populations. As a result, there is little insight into the reliability and validity of diagnoses made using different approaches across populations.

The 2016 practice parameter of the American Academy of Child and Adolescent Psychiatry (Zeanah et al., 2016) for the assessment and treatment of children and adolescents with RAD and DSED (DSM-5) recommended taking a comprehensive history of the child’s caregiving environment (foster care, adoption, institutional care or severe deprivation, and maltreatment), a history of the child’s patterns of attachment behavior plus direct in vivo clinical observation of children with familiar caregivers as well as with a stranger. A Clinical Observation of Attachment (COA) procedure, such as that developed and tested by Boris and colleagues (Boris et al., 2004), was recommended for this purpose. The American Professional Society on the Abuse of Children (APSAC) Task Force issued similar recommendations, adding that the child’s behavior should be assessed across situations, contexts, and caregivers (Chaffin et al., 2006).

Structured clinical interviewing may also be used to collect data across contexts and caregivers, such as with the Disturbances of Attachment Interview (DAI; Smyke & Zeanah, 1999). The DAI has been used as a parent-report measure of disturbed attachment behavior in several studies in young at-risk children (Oosterman & Schuengel, 2007a; Smyke, Dumitrescu, & Zeanah, 2002; Zeanah et al., 2004; Zeanah, Smyke, & Dumitrescu, 2002; Zeanah, Smyke, Koga, & Carlson, 2005) and has been examined on validity and reliability (Gleason et al., 2011; Oosterman & Schuengel, 2007a; Vervoort, De Schipper, Bosmans, & Verschueren, 2013).

Research on clinical observational methods used to assess symptoms of disordered attachment is more limited. Riley, Atlas-Corbett, and Lyons-Ruth (2005) developed an observational measure, the Rating of Infant and Stranger Engagement (RISE) based on the Strange Situation Procedure (SSP; Ainsworth, Blehar, Waters, & Wall, 1978). The SSP was originally developed to assess security of attachment by assessing variations in the way the child related to the parent after separation. In relation to the assessment of indiscriminate behavior, O’Connor and Zeanah (2003) argued that the underlying assumptions and traditional coding methods for assessing security of attachment are not adequate to capture disorders of attachment. Zeanah, Smyke, Koga, & Carlson (2005) and the BEIPCore Group found that 21% of children who were rated as secure on the SSP did not show fully developed attachment behaviors. This illustrates the likely inappropriateness of the traditional SSP coding scheme for children with attachment disorders, since those coding schemes assume the existence of a selective relationship and focus mostly on behavior toward the person assumed to be an attachment figure. To assess disinhibited social engagement, the focus should be not only on the behavior toward the parent but also on behavior toward a stranger. The RISE evaluates both the extent of the infant’s affective engagement with the stranger compared to the caregiver and the extent to which the infant displays nonnormative acceptance of physical contact or response to soothing by the stranger. The RISE has been validated in studies with at-risk home-reared toddlers (Lyons-Ruth, Bureau, Riley, & Atlas-Corbett, 2009) and institutionalized toddlers (Oliviera et al., 2012) by examining the convergence of observed disinhibited behavior of the young child with the (by the parent) reported indiscriminate behavior (in the DAI; Smyke & Zeanah, 1999).

One particular form of disinhibited behavior, namely leaving with a stranger without checking with the parent or caregiver, has been operationalized with the “stranger at the door procedure” (Gleason et al., 2011). Leaving with a stranger was associated with a caregiver report of disinhibited attachment behavior (Gleason et al., 2014). While this behavior can validate the caregiver’s report of disinhibited attachment behavior, the stranger at the door procedure has limited clinical use. Studies using multi-informant diagnoses of disordered attachment are rare. McLaughlin, Espie, and Minnis (2010) examined the reliability of a clinical observation schedule in a brief waiting room procedure. This observation procedure has recently been used with other parent-report measures of disordered attachment and psychopathology as part in a large multi-informant study (N = 1654) to estimate the prevalence of RAD/DSED in children (age 6–8 years) in a deprived population (Minnis et al., 2013). Minnis et al. found a prevalence of RAD (including disinhibited attachment) in this population of 1.4%. Kay, Green, & Sharma (2016) investigated the prevalence of DSED in adopted children (age 6–11 years) using multi-informant cross-context measures, including a researcher observation. Forty-nine percent of the children adopted from out-of-home care were classified with DSED, while 6% of the adopted children with no history of special care or child protection before adoption were classified with DSED.

According to Zeanah, Berlin, and Boris (2011), it is important for clinicians to observe and analyze how children interact with their primary caregiver when they are distressed and their attachment behavioral system is activated. Features that are of interest in the interactions include the presence or absence of proximity seeking, avoidance, resistance, or disorganization in their responses of distress, displays of affection toward the caregiver, controlling behavior toward the caregiver, and the ways in which a child interacts with an unfamiliar adult. Given the complexity of such observational diagnostic assessments, the convergence of the results of such assessments with those of more straightforward assessment procedures, such as use of the DAI, cannot be assumed. Indeed, the two types of assessments could potentially generate conflicting data.

A distinctive feature of RAD and DSED is that etiological factors are implied in the diagnosis itself. Thus, for these diagnoses, it must be established that the aberrant social behavior is reactive to exposure to extremes of insufficient care, such as social neglect and deprivation, frequent changes in primary caregivers, and growing up in caregiving arrangements that limit the formation of selective attachment relationships in early life (APA, 2013). Whether exposure to such environmental conditions is a necessary diagnostic criterion that improves diagnostic validity is disputed (Minnis et al., 2009). However, the AACAP Practice parameter (2016) has decided that “given that DSM-5 criteria require a history of severely ‘insufficient care’, the diagnosis should be questioned in any case in which a history of social neglect cannot be documented.” In the large study of Minnis et al. (2013), information about possible pathogenic care was explored in a computerized parent-report interview with questions about physical abuse and witnessing domestic violence in the context of possible posttraumatic stress disorder (PTSD); other possible information sources about possible extreme insufficient care were not included in the study. Kay et al. (2016) used the Maltreatment Classification System (MCS; Barnett et al., 1993) for rating reported (by the adoptive parents) descriptions of maltreatment and neglect and information from the adoption files. In clinical practice, exposure to different caregiving environments can only be assessed retrospectively. Aside from the specific situations of children who grow up in foster or adoptive care or who are taken into foster or adoptive care from institutional settings, caregiver reporting is often problematic, because current caregivers (such as foster parents) may not be fully informed by previous caregivers and because social and legal implications may discourage caregivers from reporting any maltreatment they committed themselves. Case records can be examined as an alternative to interviewing current caregivers, but the records may also reflect selective or biased reporting by professionals, who often base their information on caregiver accounts. It can be argued that positive evidence for a reactive environment is not a necessary criterion if detailed behavioral study of the phenotype of attachment-related disorders (e.g. Lawler, Hostinar, Mliner, & Gunnar, 2014) reveals clear distinctions with the phenotypes of nonreactive disorders of social behavior, as was found when comparing symptoms of RAD, DSED, and autism spectrum disorder (Giltaij, Sterkenburg, & Schuengel, 2015). Nevertheless, in the absence of standardized and psychometrically sound behavioral tests, the inclusion of converging anamnestic evidence might increase the confidence in RAD diagnosis and protect against false-positive diagnoses, especially in clinical populations that also have a high prevalence of neurodevelopmental disorders that impact social functioning (e.g. Riby, Kirk, Hanley, & Riby, 2014).

To support the development of valid diagnostic assessment procedures that identify children with disordered attachment, the current study explored whether the following methods for determining diagnostic information about RAD and DSED according to the DSM-5 yielded consistent results: semi-structured interviews with caregivers, structured clinical observations of attachment, and independent assessment of extremes of insufficient care.

The study was conducted with a clinical sample of children who were referred for psychiatric consultation to centers that specialize in treating children with low intellectual functioning. Low intellectual functioning is associated with caregiving risk and vulnerability to disorganized, atypical, or disordered attachment (Janssen, Schuengel, & Stolk, 2002). A previous study of this sample that used the DAI revealed that 42% of the children had caregiver-reported symptoms of inhibited and disinhibited behavior (Giltaij et al., 2015); accordingly, this population was expected to show a heightened base rate of disordered attachment, making it feasible to include an observational attachment assessment procedure.

Methods

Participants

The parents of 102 children who had been screened for disturbances of attachment and symptoms of autism spectrum disorder (Giltaij et al., 2015) were asked to participate in a follow-up, which was conducted 2 years after the first one. The participants were recruited through nine Dutch mental healthcare centers that specialize in the psychiatric assessment and treatment of children with developmental disabilities and low intellectual functioning. These centers are scattered throughout 8 of the 11 provinces in The Netherlands. The children were referred for diagnostic consultation mainly not only because of behavioral problems but also because of educational and emotional problems. The inclusion criteria were as follows: age between 5 and 11 years on the date of the assessment; low intellectual functioning with a measured intelligence quotient (IQ) of 50–85 or functional diagnosis of low average, borderline, or mild intellectual disability; and the ability to communicate in Dutch. The centers’ psychiatrists and psychologists were instructed not to preselect cases. The parents of 55 of the 102 children gave permission for the children to participate in the study and returned the informed consent letter. The reasons that were given for not participating in the follow-up study were that the child had been discharged from the mental healthcare center system, there were time constraints, or the parents considered the procedure to be too stressful for the child.

The mean chronological age of the children at follow-up was 10.7 years (SD = 1.8), and 41 (75%) were boys. The mean IQ according to the case files was 72.1 (SD = 9.1), with 20 children scoring in the range of mild intellectual disability (IQ 50–70) and 35 children scoring in the range of borderline to low average intellectual functioning (IQ 71–85). The majority of the children, 43 (78%), had parents who were born in The Netherlands. The demographic characteristics of the participants in the follow-up study correspond with the characteristics from the first study, except for age (Table 1).

Table 1.

Demographic characteristics of participants of the original sample (N = 102) and the follow-up sample (N = 55).

Categories	Characteristics	Original sample	Follow-up sample
Age, M (SD)		8.8 (1.7)	10.7 (1.8)
Gender, %	Boy	71%	75%
	Girl	29%	25%
Cognitive, M (SD)	IQ	71.7 (9.7)	72.1 (9.1)
Ethnicity, %	Dutch	81%	78%
	Non-Dutch, western	3%	4%
	Non-Dutch, non-western	16%	18%

M: Mean; SD: standard deviation.

Ethical approval was obtained from the Vrije Universiteit Medical Centre Medical-Ethical Review Board (protocol number 2006/213). This board is licensed to approve research by the Central Committee on Research Involving Human Subjects, which monitors compliance with Dutch legislation on medical research. The participants received a small present (a voucher and a comic book).

Instruments

Symptoms for RAD and DSED

The DAI (Smyke et al., 2002; Smyke & Zeanah, 1999) is a 12-item semi-structured interview conducted with a primary caregiver or another person who knows the child well, which is designed to assess behavioral symptoms of clinically disturbed or disordered attachment in children. Five items operationalize behaviors that are relevant to DSM-5 RAD: (1) absence of a discriminated, preferred adult; (2) lack of comfort seeking for distress; (3) failure to respond to comfort when offered; (4) lack of social and emotional reciprocity; and (5) emotion regulation difficulties. Three items operationalize the behavioral signs of DSM-5 DSED: (6) not checking back after venturing away from the caregiver, (7) lack of reticence with unfamiliar adults, and (8) a willingness to go off with relative strangers. Interviewers probed the respondent to acquire enough information on the child’s behavior so that during the coding phase they could rate each item as “0” = “no sign of RAD or DSED,” “1” = “somewhat or sometimes a sign of RAD or DSED,” and “2” = “considerable or frequently a sign of RAD or DSED.”

Previous studies have shown strong internal consistency and inter-rater reliability for scales encompassing inhibited (RAD) and disinhibited (DSED) behavior symptoms (Oosterman & Schuengel, 2007a; Smyke et al., 2002; Zeanah et al., 2004; Zeanah et al., 2002; Zeanah et al., 2005). The DAI scales converged with similar measures used in other studies of signs of RAD (Chisholm, 1998; O’Connor, Rutter, & The English and Romanian Adoptees Study Team, 2000) and diverged from measures of aggression, stereotypes, and language development (Zeanah et al., 2002), and autism (Giltaij et al., 2015). Although the DAI was not developed specifically for children with intellectual disabilities, there was good reliability in studies with Romanian orphanage children, among whom the IQs ranged from moderate intellectual disability to normal intelligence (Smyke et al., 2002; Zeanah et al., 2005). Independent psychiatric assessment of attachment disorder was found to be strongly associated with symptoms on the DAI (Gleason et al., 2011; Zilberstein & Popper, 2014).

Based on the work of Gleason and colleagues (2011), the presence of at least three signs of RAD and at least two signs of DSED was considered a positive score on the categorical diagnoses RAD and DSED. Given the validity evidence in their study, this scoring rule was also adopted in the current study. The interviewers were trained by M. Oosterman, who conducted psychometric research on the Dutch version of the DAI (Oosterman & Schuengel, 2007b). N. Boris trained interviewers to score normal and aberrant attachment behavior. Interviews were conducted over the phone and were recorded. The interviewers subsequently determined the scores based on a review of the recording. Inter-rater reliability of the scale scores was good (Cohen’s kappa = .88 for RAD and .98 for DSED).

Structured observation of adaptive and maladaptive attachment behavior

The Clinical Observation of Attachment (COA; Boris et al., 2004; Boris et al., 2005) is a structured observational procedure conducted in an unfamiliar room (50 minutes) with the child, a caregiver, and an unfamiliar adult, who may be the diagnostician. The observational procedure for this study was based on the COA procedure developed by Boris and colleagues (2004). Elements of the COA that may elicit responses through the attachment behavioral system include an unfamiliar room and person, being approached by the unfamiliar person who joins in the child’s play, separation from the caregiver followed by reunion, a sudden and loud alarm sound was then switched on by the unfamiliar person using a hidden remote control of a CD-music system, solicitation of physical contact by the unfamiliar person, and a co-operation task. A behavior transcript was made on the basis of video recordings of the sessions. Video recording and transcript were scored for adaptive and maladaptive attachment behavior using the List of Behavioral Signs of Disturbed Attachment in Young Children (BSDA; Boris, Fueyo, & Zeanah, 1997; Zeanah, Mammen, & Lieberman, 1993) as recommended in the AACAP Practice parameter (Boris et al., 2005; Zeanah et al., 2016). The BSDA list describes eight types of attachment behavior: (1) showing affection to the caregiver, (2) seeking comfort from the caregiver, (3) reliance on the caregiver for help, (4) cooperation with the caregiver, (5) exploratory behavior, (6) controlling behavior, (7) reunion responses, and (8) response to strangers. A five-point scale that ranged from adaptive (score = 1) to maladaptive (score = 5) was used to rate children’s behavior toward their caregiver and toward the unfamiliar person. A sum score of maladaptive attachment behavior was calculated based on the eight behavior types. In the manual of the BSDA, the behavioral expression of the extreme scores (scores 1 and 5) are described. The intermediate scores are related to these extreme scores (mostly adaptive, sometimes adaptive as well as maladaptive, mostly maladaptive). In this study, five master-students Special Education were trained for coding the attachment behavior using the BSDA. The inter-rater reliability was .73. The video recording and transcripts were randomly divided among two coders who coded the video’s independently the observed behavior of the child in the COA. If there was a difference of one point, the coding was discussed until the coders reached consensus. If there was a difference of more than one point, the video recording and transcript were coded a third time by an independent coder with no foreknowledge of the other coders. The coding was then discussed until consensus was met.

Extremes of insufficient care

A research assistant who was not involved with other measurement activities in the study abstracted information from each child’s case files at the mental health centers where the participants were recruited. These medical files contained all the reports (pediatric, psychological, child psychiatric, educational, juridical, and social work) that were used by the clinicians of the mental health center as input for assessment, classification, and treatment of the child. Coding of the files focused on the developmental history of the child, especially information about the first years of the child’s life, as well as descriptions and experiences related to child–caregiver interactions. Extremes of insufficient care were coded positive in cases that showed (1) neglect of the child’s emotional and social needs (references of an environment characterized as highly unstable, reported physical or sexual abuse, as confirmed by interference of childcare reports); (2) repeated changes in primary caregivers (multiple separations from primary caregivers in the first 3 years of age, such as hospital care, adversarial divorce of parents, and frequently changing caregivers); or (3) rearing in institutions with high child-to-caregiver ratios. The inter-rater reliability for extremes of insufficient care was high (intra-class correlation of .93). Given the recent shift from a broader definition of pathogenic care (DSM-IV) to the more narrowly defined extremes of insufficient care in the DSM-5, case records were also coded for exposure to harsh punishment and inept parenting and for exposure to domestic violence. Pakalniskiene (2008) defined harsh parenting as physical punishment and verbal or nonverbal aggression, such as anger outbursts, threats, stony silences, and rejection, thus combining both aspects—the physical and the nonphysical. Harsh punishment is considered as a form of “inept parenting” that includes coercive parent–child communication, dysfunctional disciplining practices, inconsistent control, harsh or violent physical punishment, negative attitudes and reasoning, limited use of praise, support, or warmth, and poor supervision and monitoring (Maccoby & Martin, 1983; Patterson, Reid, & Dishion, 1992; Reid, Patterson & Snyder, 2002; Robins & Rutter, 1990; Stoff, Breiling, & Maser, 1997).

Clinical diagnosis of RAD or DSED

Clinical diagnoses were made independently by two highly experienced clinical psychologists using the DSM-5 criteria. The psychologists considered the assessments of behavior in the COA and the assessments of the attachment history without the DAI results. First, the (COA) observed (maladaptive) attachment behavior on the BSDA was matched with the DSM-5 criteria. Positive evidence for RAD is when the child demonstrates a consistent pattern of inhibited, emotionally withdrawn behavior toward adult caregivers, manifested by both of the following: (1) the child rarely or minimally seeks comfort when distressed, and (2) the child rarely or minimally responds to comfort when distressed, and at least two out of the following: (3) minimal social and emotional responsiveness to others; (4) limited positive affect; and (5) episodes of unexplained irritability, sadness, or fearfulness that are evident even during nonthreatening interactions with adult caregivers. Positive evidence for DSED is when the child shows a pattern of behavior in which the child actively approaches and interacts with unfamiliar adults and exhibits at least two of the next criteria: (1) reduced or absent reticence in approaching and interacting with unfamiliar adults; (2) overly familiar verbal or physical behavior (that is not consistent with culturally sanctioned and with age-appropriate social boundaries); (3) diminished or absent checking back with adult caregiver after venturing away, even in unfamiliar settings; and (4) willingness to go off with an unfamiliar adult with little or no hesitation. It is important to note that the criteria for the two disorders are not mutually exclusive, leaving open the possibility that both the RAD and DSED criteria may apply. However, given that children are prevented from venturing away on their own during the COA, it is unlikely that children may be diagnosed with DSED or DSED/RAD without displaying at least of the active disinhibited forms of social behavior (reduced reticence and/or overly familiar behavior).

Second, all information gained from the attachment history was matched to extremes of insufficient care according to the DSM-5 criteria (C). Third, since insufficient care is a necessary condition for the diagnosis, only children who met both conditions as meeting the DSM-5 criteria received the DSM-5 classification. Fourth, using the DSM-5 criteria A and B for RAD and criteria A for DSED, both psychologists independently coded the observed attachment behavior in the video-recorded COA as diagnosis RAD, DSED, or RAD as well as DSED. The inter-rater reliability was κ = 1.00. Discrepancies in diagnosis were discussed until the consensus was met.

Data analysis

T-tests were conducted to investigate the association between positive diagnoses based on the DAI and the maladaptive attachment behavior scores based on the COA as well as between extremes of insufficient care and maladaptive attachment behavior scores on the COA. Chi-square tests were performed to investigate the convergence between positive diagnoses based on the DAI and extremes of insufficient care. Statistical analyses were performed using the software package SPSS version 21.

Results

Preliminary and descriptive analyses

To test for possible confounding factors for the convergence of the disordered attachment indicators, we tested for associations between indications for disordered attachment on the DAI (inhibited, disinhibited, or none), the scores for maladaptive attachment behavior on the COA, and insufficient care as reported in the records (extremes of insufficient care or no extremes of insufficient care), and the background variables, gender, ethnicity, age, and cognitive functioning. T-tests showed no significant associations (all p > .05).

Based on the DAI with the caregivers, inhibited and disinhibited behavior was below the threshold for a positive categorical diagnosis for RAD or DSED in 37 (67%) of the 55 children (Table 2). Positive DAI diagnoses for RAD were found for 9 children, and positive diagnoses for DSED were found for 15 children. Six children scored positive for RAD and DSED using the DAI.

Table 2.

Frequency of DAI diagnoses of the original sample (N = 102), the nonparticipants in follow-up (N = 47) and the follow-up sample (N = 55).

Categories	Characteristics	Original sample (N = 102, %)	Nonparticipants in follow-up (N = 47, %)	Follow-up sample (N = 55, %)
DAI	No RAD and no DSED	58	47	67
	RAD	5	4	6
	DSED	27	38	16
	RAD and DSED	11	11	11

DAI: Disturbances of Attachment Interview; DSED: disinhibited social engagement disorder; RAD: reactive attachment disorder.

In the structured observation setting (COA), children’s behavior toward their primary caregiver and the stranger was scored according to the eight BSDA signs of adaptive and maladaptive attachment behavior. With this five-point scale, the minimum possible total score is 8 (fully adaptive attachment behavior), and the maximum possible total score is 40 (fully maladaptive attachment behavior). The average score on the BSDA was 17.98 (SD = 6.39), ranging from 9 to 33.

Exposure to extremes of insufficient care was found in the medical records of 12 children (22%). This included repeated changes in caregivers in 10 (18%) cases, living in institutions in 7 (13%) cases, and neglect of their social and emotional needs in 9 (16%) cases. In eight cases, harsh punishment or inept parenting was coded in addition to extremes of insufficient care, and in two cases, harsh punishment or inept parenting was mentioned without extremes of insufficient care. Domestic violence was coded in seven cases in addition to extremes of insufficient care, and domestic violence was coded without exposure to extremes of insufficient care in two cases. The two cases that were positive for harsh punishment or inept parenting did not overlap with the two cases that were positive for domestic violence.

Based on the diagnostic information that was collected, 10 children received a DSM-5 RAD and/or DSED diagnosis: 7 children were diagnosed with RAD and with DSED, 2 children were diagnosed with DSM-5 RAD only, and 1 child was diagnosed with DSM-5 DSED only.

Convergence between DAI diagnoses and COA outcomes

An independent samples t-test found that children who scored positive for RAD or DSED on the DAI (n = 18, M = 21.61, SD = 7.2) showed more maladaptive attachment behavior according to the BSDA than those who scored negative for RAD or DSED (n = 37, M = 16.22, SD = 5.2), t(53) = 3.17, p = .003, d = 0.91. Children who scored positive for RAD only (n = 9, M = 25.78, SD = 5.8) showed more maladaptive attachment behavior than those who scored negative for RAD (n = 46, M = 16.46, SD = 5.3), t(53) = −4.72, p < .001, d = −1.72. Likewise, children who scored positive DSED only (n = 15, M = 22.2, SD = 7.7) showed more maladaptive attachment behavior than those who scored negative for DSED (n = 40, M = 16.4, SD = 5.1), t(53) = −2.70, p = .014, d = 0.82. Figure 1 shows the distribution of the BSDA scores for the DAI scoring RAD and/or DSED (RAD Inhibited and/or RAD disinhibited) and no RAD and no DSED.

Figure 1.

Total BSDA score using COA and DAI screening for RAD and/or DSED and no RAD / no DSED (N = 55).

Convergence between DAI diagnoses and extremes of insufficient care

Extremes of insufficient care were significantly less prevalent in the records of children with a DAI diagnosis of no RAD and no DSED (Fisher’s exact test, p = .043, κ = .19). Significantly more children who scored positive for RAD had extremes of insufficient care compared to children who scored negative for RAD (Fisher’s exact test, p = .017, κ = .36); similarly, significantly more children who scored positive for DSED had extremes of insufficient care compared to children who scored negative for DSED (Fisher’s exact test, p = .011, κ = .36) (Table 3).

Table 3.

Frequency of extremes of insufficient care and no RAD and no DSED, RAD and DSED according to the DAI (N = 55).

Extremes of insufficient care	Extremes of insufficient careTotal	Classification According to DAI
		No RAD and no DSED		RAD		DSED
		Yes	No	Yes	No	Yes	No
Yes	12	5	7	5	7	7	5
No	43	32	11	4	39	8	35

DAI: Disturbance of Attachment Interview; RAD: reactive attachment disorder.

Convergence between COA outcomes and extremes of insufficient care

The mean total score for the observed BSDA behavior of children who were not found to have experienced extremes of insufficient care was 16.07 (SD = 5.14, n = 43); children with records reporting extremes of insufficient care had a mean total BSDA score of 24.83 (SD = 5.86, n = 12). Children who had experienced extremes of insufficient care had a significantly higher total score on the BSDA than those who had not experienced extremes of insufficient care or those with insufficient information in their medical file to determine whether they had experienced extremes of insufficient care: t(53) = 5.07, p < .001, d = 1.66.

Convergence between DAI diagnoses and a DSM-5 classification of attachment disorder

A total of 34 children who were negative for a RAD and/or DSED using the DAI were also independently diagnosed negative according to DSM-5 for RAD and/or DSED. Seven children had positive DAI and an independently positive DSM-5 diagnosis of RAD and/or DSED. Thus, in 41 children (75%) the DAI diagnosis and the DSM-5 diagnosis were consistent. Three children who scored negative according to the DAI had a positive classification for attachment disorder according to the combination of observed behavior (COA) plus a history of exposure to extremes of insufficient care (DSM-5 RAD or DSED). Eleven children who scored positive according to the DAI had a negative DSM-5 diagnosis (Table 4). The predictive validity of the DAI for a DSM-5 diagnosis of RAD or DSED therefore showed only fair strength (Fisher’s exact test, p = .01 κ = .35). For 25% of the children, the DAI was inconsistent with a DSM-5 RAD or DSED diagnosis.

Table 4.

Frequency of the classification according to the DAI and frequency of clinical diagnosis according to the DSM-5 (N = 55).

Classification according to DAI	Clinical diagnosis according to the DSM-5 (including the criterion of evidence of extremes insufficient care)		Total
	No RAD (DSM-5) or DSED (DSM-5) (according to COA and IC)	RAD (DSM-5) and/or DSED (DSM-5; according to COA and IC)
No RAD	34	3	37
RAD inhibited and/or RAD disinhibited	11	7	18
Total	45	10	55

COA: Clinical Observation of Attachment; DAI: Disturbance of Attachment Interview; DSED: disinhibited social engagement disorder; DSM: Diagnostic and Statistical Manual of Mental disorders; IC: Extremes of insufficient care; RAD: reactive attachment disorder.

Convergence between COA outcomes and a clinical (DSM-5) diagnosis of RAD and DSED

The mean total score for the observed BSDA behavior of children who were not diagnosed with a DSM-5 RAD or DSED was 16.0 (SD = 5.03, n = 45); children who only had a DSM-5 RAD diagnosis had a mean total score (BSDA) of 23.5 (SD = .71, n = 2), and the child who only had a DSM-5 DSED diagnosis had a total score (BSDA) of 25. The children with both DSM-5 RAD and DSED diagnoses had a mean total score (BSDA) of 28.2 (SD = 3.72, n = 7). Children with a DSM-5 RAD and/or DSED diagnosis had a significantly higher total score on the BSDA than those without DSM-5 RAD and/or DSED diagnosis: t(53) = −5.07, p < .001, d = −1.77. Figure 2 shows the BSDA scoring distribution for RAD and/or DSED and no RAD and no DSED.

Figure 2.

Total BSDA score using COA and DSM-5 diagnose RAD and/or DSED and no RAD/no DSED (N = 55).

Discussion

The diagnostic steps recommended by Boris et al. (2005) and Chaffin et al. (2006) led to high convergence between indicators of disordered attachment, supporting the validity of diagnoses made of RAD or DSED for children with low intellectual functioning. Using these recommended diagnostic procedures (the association between observation of the attachment behavior in a structured setting and a history of extreme insufficient care, gathered from a comprehensive history of the child’s early caregiving environment) led to a positive clinical diagnosis of RAD and/or DSED according to the DSM-5 criteria in 18% of the cases in this sample of children with low intellectual functioning referred for mental health assessment, despite having been under treatment for 2 years. With a rate of 33% positive RAD and/or DSED diagnosis based on the DAI as a structural clinical interview, the DAI identified a considerable higher number of cases than based on more comprehensive diagnosis using observations.

Observations of maladaptive attachment behavior during the COA procedure using the BSDA proved feasible, reliable, and valid according to associations with the other measures. The mean BSDA score was higher for children who screened positive for RAD and/or DSED using the DAI and for the DSM-5 diagnosed children than for those with negative screens or diagnoses. Still, some children who displayed high rates of maladaptive attachment behavior screened negative on the DAI and did not fulfill the criteria for exposure to extremes of insufficient care, illustrating the importance of using multiple sources of information, even if observations are made. A lack of evidence for insufficient care does not mean that the child was not exposed to it. Caregivers may be unaware of what insufficient care is or may have reasons not to disclose this information during the assessment (shame, guilt, or fear for consequences). Although some studies (e.g. Minnis et al., 2009) did not include the criteria for extremes of insufficient care because the information may be unreliable, in this study the recommendations of the AACAP were followed using the classification criteria of the DSM-5. However, the use of the AACAP increased the risk that false-negative scores on insufficient care may be reported, as the participants’ files may be incomplete.

A surprisingly high number of children were comorbid on RAD and DSED, given the decision that was made for the DSM 5 to define two disorders (RAD and DSED), rather than define two distinct types of inhibited and disinhibited RAD. Past research with the DAI has suggested that so-called passive forms of indiscriminate behavior, such as failing to check with caregiver after venturing and willingness to go off with stranger, may confound comorbid classification if not combined with more active disinhibited social engagement (Zeanah & Gleason, 2010). The DSM-5 criteria for DSED also do not exclude the possibility that children are diagnosed with DSED based on observed “passive” indiscriminate behavior alone, which is on the basis of the two criteria of failing to check back after venturing away and willingness to go off with stranger. However, it is unlikely that comorbidity is accounted for by these passive indiscriminate behaviors only, because children are prevented from venturing away during the COA procedure, requiring that at least one of the other, more active, criteria needs to be fulfilled.

This study had some limitations. The sample of 55 children was modest and originated from a study on inhibited and disinhibited attachment behavior and symptoms of autism (Giltaij et al., 2015). Dropout was considerable (47 out of 102); this may have introduced bias, although analyses of the families who declined participation in the follow-up did not reveal significant differences in gender, ethnicity and cognitive functioning.

Another limitation was that the DAI used in this study was originally developed for administrating disturbed attachment behavior in children until 5 years of age. Although the DSM-5 criteria for RAD and DSED do not distinguish between young and school-aged children, the phenomenology of the behavior may differ. In 2010 (Zeanah & Smyke, 2010), a version of the DAI was presented for school-aged children, adding some adaptations. Relative to the original child version (e.g. Gleason et al., 2011), little is known about the psychometric attributes of the school-age version. Also, the studies using the school-age version are very limited (Vervoort et al., 2013), while other studies on school-aged children used the original version (Oosterman & Schuengel, 2007a, 2007b).

Furthermore, children were only observed with their primary caregiver. Although it is presumed that attachment-related disorders are a characteristic of the child, attachment itself is a relational concept. Nevertheless, the maladaptive attachment behavior in the children scoring positive on the clinical diagnosis RAD and/or DSED is, in this study, only observed in relation with one primary caregiver, and it is therefore insufficient to define it as a within-the-child disorder. Obtaining diagnostic information on disordered attachment within other relationships as well is likely due to increased diagnostic confidence. The AACAP Practice parameter on assessments of RAD and DSED (Boris et al., 2005; Zeanah et al., 2016) recommends serial observations of the child interacting with at least both primary caregivers in a minimum of two or three visits.

Finally, it should be kept in mind that the DSM-5 criteria and diagnostic guidelines refer to disorders that begin in early childhood, while the chronological ages of the children in the current sample ranged from age 7 to 13 years. Previous work has shown that disinhibited attachment behavior is highly stable from age 6 to 11 years (Rutter et al., 2007), but the validity of observed and reported inhibited attachment behavior in school aged children is less clear. Resolution of this issue is hampered by the low frequency of inhibited attachment and RAD in the current sample as well as in other samples.

The classification criteria for the DSM-5 attachment disorders (i.e. RAD and DSED) include observable and distinct social behaviors of children that can be evaluated by clinicians who are trained in attachment theory and in the observation of attachment. The current study supports a best practice for assessing disordered attachment that consists of interviewing the primary caregivers or other adults who are closely familiar with the child, using a checklist of extremes of insufficient care, reviewing a developmental history of the child’s care experiences, and performing a structured observation procedure (COA) with the child and primary caregiver plus a stranger (Boris et al., 2005; Zeanah et al., 2016). Combining these instruments bolsters the validity of the diagnosis and, thus, improves the subsequent treatment plan. These findings suggest that the DAI should be used as a screening instrument with caution, as it cannot replace full clinical assessment for diagnosing RAD and/or DSED. Thus, future research may consider the DAI as a screening instrument. Furthermore, clinicians should always obtain careful histories from as many sources as are relevant about insufficient care. Developing and testing intervention programmes that reduce maladaptive attachment behaviors in the context of promoting secure attachment remain a priority for further research. Although studies have found encouraging results for interventions that focus on secure attachment in high-risk families (Bernard et al., 2012; Moss et al., 2011), with the exception of placement from institutional care to foster care (Smyke et al., 2012), little is known about the potential of treatment to reduce disordered attachment.