Assessment of Unresolved/Disorganized State of Mind in Relation to Attachment: A ROC Curve Study Using the Adult Attachment Interview and the Measure of Parental Style.

Objective: The main objective of the present study was to use the Receiver Operating Characteristic (ROC) curve analysis to identify cut-off points for a self-report measure assessing parental style, i.e., the Measure of Parental Style (MOPS), that are able to discriminate individuals with disorganized internal working models (IWMs) of attachment with adequate accuracy, in terms of sensitivity and specificity. Establishing cut-off points for the MOPS could provide clinicians and researchers with a valuable tool to investigate the role of disorganized IWMs as a link between parental styles and mental health. Method: A sample of 90 university students (mean age = 21.21 ± 2.05, females = 66) was enrolled in the study. We used the Adult Attachment Interview (AAI) to assess disorganized IWM and the MOPS to assess parental styles. Subsequently, we used ROC curve analysis to pursue the objective of the study.
Results: The ROC curve analysis showed that the MOPS total score (i.e., the combination of maternal and paternal dimensions) was able to discriminate individuals with disorganized IWMs from individuals with organized IWMs (AUC= 0.77). Specifically, a score ≥ 25 (Youden index= 0.497) categorized individuals with a sensitivity of 0.69 (69% of participants with disorganized IWMs were correctly identified) and a specificity of 0.81 (19% of participants were incorrectly identified as having disorganized IWMs). Conclusions: Although the AAI has demonstrated high psychometric properties for assessing attachment representations in adulthood, its use is difficult when studies with large samples are to be conducted. As an alternative to the AAI, the MOPS can be used in studies with large populations, but no cut-off has yet been proposed. Here, we have identified cut-off points for the MOPS that are capable of detecting disorganized IWMs of attachment with adequate accuracy, and we suggest that this self-report is a useful brief instrument for detecting disorganized IWMs when time constraints prevent the use of the AAI (e.g., in studies with large samples or epidemiological studies).

Introduction

Many areas of research in disciplines such as psychology, psychotherapy, and psychiatry focus their interest on the role of parenting in mental health and personality development (e.g., Adenzato et al., 2006; Ardito et al., 2004; Bowlby, 1969; Collishaw et al., 2007; Hesse & Main, 2006). This central role is justified, inter alia, by the longitudinal data available to date on the relationship between parental style and short-, medium-, and long-term cognitive and emotional developmental outcomes (e.g., Carlo et al., 2011; Morrell & Murray, 2003; Pettit et al., 1997; Raby et al., 2015; Sulik et al., 2015; van IJzendoorn & Bakermans-Kranenburg, 2019; Whittle et al., 2014).

Several instruments have been proposed to measure parental style, and one of the most commonly used is the Parental Bonding Instrument (PBI) proposed by Parker et al. (1979). Compared to other self-report instruments (e.g., the Adverse Childhood Experiences International Questionnaire; World Health Organization, 4-5 May 2011) that can assess different types of potentially traumatic childhood events (e.g., physical, sexual, emotional abuse/neglect, peer violence, etc.), the PBI is a self-report questionnaire that focuses on two basic parenting dimensions: Care and Over-protection. More specifically, it measures the types of behaviors the individual remembers experiencing from his or her parents during the first 16 years of life and consists of 25 items: twelve items relate to the degree of empathy and caring (care scale) and 13 to the degree of control and protection (over-protection scale). The PBI has demonstrated satisfactory psychometric properties (Parker, 1989) and has been used in numerous studies to demonstrate the relationship between variables assessed with the instrument and various psychiatric disorders, particularly anxiety and depression (Kidd et al., 2022; Kullberg et al., 2020). Nevertheless, the PBI has been criticized (Harris & Brown, 1996) for not capturing experiences of abuse. To overcome this limitation, Parker and colleagues (1997) proposed a refined version of the PBI in 1997, the Measure of Parental Style (MOPS).

The MOPS is a complemented version of the PBI that captures experiences of abuse and of parental separation and loss. These experiences are thought to strongly predispose individuals to psychological distress and psychopathology (Bowlby, 1969). It consists of 30 items that separately assess maternal (15 items) and paternal (15 items) parental styles during the first 16 years of life. The items are grouped into three dimensions for each parent: indifference (e.g., “Ignored me”, “Left me on my own a lot”, “Was uninterested in me”), over-control (e.g., “Over-protective of me”, “Over-controlling of me”, “Critical of me”), and abuse (e.g., “Verbally abusive of me”, “Physically violent or abusive of me”, “Made me feel in danger”). The indifference and over-control MOPS scales correspond to the care and over-protection PBI dimensions, while the newly added scale MOPS abuse has sufficient independence to be considered separately (Parker, et al., 1997). Various studies have shown that the MOPS is a reliable and valid tool for assessing parental style (Parker et al., 1997; Picardi et al., 2013; Rumpold et al., 2002).

As originally proposed by Parker et al. (1997), the MOPS can serve as a measure of the likelihood of exposure to dysfunctional parenting. The importance of assessing dysfunctional parenting is evidenced by the fact that dysfunctional parenting, such as very low care, emotional abuse, and high over-control, has been reported to be a factor of psychopathological vulnerability, affecting development at both neurobiological and psychological levels (e.g., Adenzato et al., 2019; Farina et al., 2021; Measelle et al., 2017; Poletti et al., 2022; Teicher et al., 2016). Various forms of dysfunctional parenting have also been shown to be the strongest predictor of symptom severity, negative prognosis, and worst treatment outcome in several psychiatric disorders, and particularly in affective disorders (e.g., Lima et al., 2010; Lippard & Nemeroff, 2020; McCrory et al., 2017).

Many authors have also emphasized the role of dysfunctional parenting in the etiology of disorganized attachment. For example, highly insensitive care by an attachment figure (Main & Solomon, 1990), helpless-fearful profiles of parenting (Lyons-Ruth & Spielman, 2004), and frightening and threatening parental behaviors (Hesse & Main, 2006) have been linked to the development of disorganized attachment. Moreover, empirical studies showed that parents of children with disorganized attachment were often abusive and neglecting and that severe forms of disorganized attachment can constitute a type of childhood trauma that, along with other forms of childhood adverse experiences, is a risk factor for psychopathology during development and adulthood (Amos et al., 2011; Farina et al., 2019; Liotti, 2004; Velotti et al., 2022).

The processes underlying the development of disorganized attachment have been described by attachment theorists (e.g., Bowlby, 1969; Lyons-Ruth & Jacobvitz, 2016; Main & Solomon, 1986). From the perspective of attachment theory, each child builds interpersonal cognitive/affective schemas or internal working models (IWMs) as a result of ongoing interactions with caregivers. IWMs are mental representations of an individual’s attachment experiences in childhood that are formed from repeated interactions between the child and caregiver and typically remain stable into adulthood (Bowlby, 1988; Bretherton, 1991; Grossmann & Grossmann, 1991; Waters et al., 2000).

Individuals can have organized or disorganized IWMs of attachment. Individuals who have developed organized IWMs show coherent strategies for requesting care and comfort (Ainsworth, 1982; Bowlby, 1988; Main & Solomon, 1990) and coherent states of mind when recalling attachment in adulthood (Hesse, 2008). Conversely, individuals who have developed disorganized states of mind exhibit IWMs characterized by anxious, incoherent, and inconsistent expectations towards attachment figures, concurrent or sequential inconsistent behavioral strategies, and disoriented and/ or disorganized memories of trauma or loss (Hesse, 2008).

Given the role that disorganized IWMs of attachment play in predisposing individuals to significant vulnerability to many mental disorders, it is crucial to have reliable tools for its detection. Several studies (Bakermans-Kranenburg & Van IJzendoorn, 1993, 2009; Ravitz et al., 2010; Sagi et al., 1994) have shown that the Adult Attachment Interview (AAI; George et al., 1996) is an instrument with high reliability and validity to assess adult attachment and the corresponding organized or disorganized IWMs. The AAI is a semistructured clinical interview that focuses on early attachment experiences and their effects (see Method section for more details). This interview classifies individuals’ “state of mind” in relation to their childhood attachment experiences and provides an understanding of the processes that characterize the development of attachment experiences from the first months of life into adulthood.

Although the AAI has repeatedly shown high psychometric properties for assessing attachment in adulthood (e.g., Ravitz, et al., 2010), it is time-consuming because it must be administered, transcribed, and scored by trained professionals. Consequently, although it likely remains the best tool to use in clinical and research settings, its use is often difficult or infeasible when studies with large samples are to be conducted. In these cases, the use of self-report may be an appropriate choice. Unfortunately, although there are several self-report measures of adult attachment in the literature, such as the Adult Attachment Styles (AAS; Hazan & Shaver, 1987), the Adult Attachment Questionnaire (AAQ; Simpson et al., 1996), and the Experiences in Close Relationships-Revised (ECR-R; Fraley & Shaver, 2000), none of these measures is able to assess the construct of disorganization, so they are limited to the secure, anxious, and avoidant styles. The only exception is the 9-item Adult Disorganized Attachment (ADA; Paetzold et al., 2015), which was developed in the field of social psychology in the context of studying romantic relationships and is thus outside the context of the relationship between the development of disorganized IWMs and parental styles examined here.

Therefore, considering the association between dysfunctional parenting and disorganized attachment (Manassis et al., 1999), the main aim of the present study was to add to the existing literature a tool capable of detecting disorganized IWMs of attachment with adequate accuracy in terms of both sensitivity and specificity. To this end, we use the AAI and Receiver Operating Characteristic (ROC) curve analysis to determine appropriate cut-off points for MOPS. Determining these cut-off points could provide clinicians and researchers with a valuable tool to investigate the role of disorganized IWMs as a link between parental styles and mental health.

Material and Methods

Participants

This research is part of a larger prospective study examining the effects of dysfunctional parenting at psychopathological, cognitive, and neurophysiological levels (Adenzato et al., 2019; Farina et al., 2021; Massullo et al., 2022).

For the present study, an a priori power analysis was performed using MedCalc statistical software, version 19.2.6 (MedCalc Software Ltd, 2020). It showed that at a probability level of 0.05, a sample size of 72 (24 positive cases and 48 negative cases) was required to provide an adequate statistical power (1– β= 80%) to detect a satisfactory (i.e., ≥ 0.70) area under the curve (AUC) with a null hypothesis value of 0.50. Thus, a convenience sample of 90 university students were enrolled for the present study. Exclusion criteria assessed using a dichotomous checklist, were: diagnosis/ history of major neuropsychiatric disorders; inability to understand Italian; and refusal to give written informed consent.

Given the low prevalence (i.e., approximately 17%) of unresolved/disorganized attachment in student samples (Bakermans-Kranenburg & van IJzendoorn, 2009) and the time-consuming nature of administering, transcribing, and coding the AAI (Thomson & Jaque, 2017), 296 participants were pre-screened for parenting quality using the PBI quadrants (Parker, et al., 1979), which are known to be associated with attachment styles (Manassis, et al., 1999; Wilhelm et al., 2016). More specifically, of the 296 participants[1], a random sample of 41 participants (i.e., almost twice as many as the 24 positive cases identified by the power analysis) who fell into the third quadrant (i.e., poor care and over-controlling parenting) and/or the fourth quadrant (i.e., neglecting bonding) with both parents (or at least with one parent in the absence of one parent) was included. A random sample of 49 participants who did not fall into these quadrants with both parents (or at least with one parent in the absence of one parent) was also included.

All participants voluntarily participated in the study, were informed of the general purpose of the study, and received neither compensation nor academic credit. The study was approved by the European University’s ethics review board according to the standards of the Declaration of Helsinki.

Self-report Assessment

After providing the written informed consent, all participants were administered the MOPS (Parker, et al., 1997) and a checklist assessing socio-demographic variables (i.e., sex, age, and education level). Participants were tested individually in a quiet room by a trained psychologist who informed them of the purpose of the study and explained how to complete the questionnaire.

The MOPS (Parker et al., 1997) is a 30-item self-report measure that separately assesses maternal (15 items) and paternal (15 items) parental styles during the first 16 years of life. It is the revised version of the Parental Bonding Instrument (Parker et al., 1979), in which items are rated on a 4-point Likert scale (from "not true at all" to "extremely true" i.e., from 0 to 3) and grouped into three dimensions for each parent, as confirmed by principal component analysis (Parker et al., 1997): indifference, over-control, and abuse. Higher scores indicate higher levels of self-reported dysfunctional parenting. In the present study, the Italian version of the instrument was used (Picardi et al., 2013), and the Cronbach's alpha in the present sample was ≥ 0.74 for the MOPS sub-scales.

Adult Attachment Interview

On a different day from the self-report assessment, participants underwent the AAI (George et al., 1996). The interviews were conducted, transcribed, and coded by two clinical psychologists (R.B.A. and L.P., who are certified as reliable AAI coders), using the coding system of Main and colleagues (Main et al., 2003). Participants were classified into one of five categories for overall state of mind with respect to attachment: 1) Secure/Autonomous (F); 2) Dismissing (Ds); 3) Preoccupied (E); 4) Unresolved/Disorganized (U/d); 5) Cannot classify (CC). In contrast to F, Ds, and E (i.e., organized IWMs), individuals classified as U/d show signs of disorientation and disorganization in monitoring reasoning or discourse during discussions of potentially traumatic events such as loss or abuse. Individuals classified as CC instead show a global disruption of attachment strategy, with oscillations between opposite and contradictory mental states (Ds, E) or low coherence, indicating a general inability to adopt an organized stance. Categories U/d and CC are considered disorganized classifications (i.e., disorganized IWMs).

According to previous reports (Barone & Carone, 2020; Massullo et al., 2022), 20% of the AAIs (N= 18) were double-coded. Inter-rater agreement for the two-way analysis (disorganized vs. organized) was 100%, kappa= 1.00; inter-rater agreement for the four-way analysis (F, Ds, E, U/d or CC) was 94%, kappa= .82. All coders were blind to all other measures and participant information.

Statistical analyses

All analyses were performed with SPSS 18.0 Statistical Package for the Social Sciences (IBM, Armonk, NY, USA). In the present study, the Receiver Operating Characteristic (ROC) test procedures (Ruopp et al., 2008) were used to assess the performance of MOPS in categorizing organized (i.e., F, Ds, E) and disorganized (i.e., U/d and CC) individuals according to the AAI classification.

A ROC curve is a two dimensional depiction of test performance (Fawcett, 2006) and the area under the ROC curve (i.e., the probability that a randomly sampled respondent will be correctly assigned to the appropriate group) is considered the key outcome variable (Centor & Schwartz, 1985). The AUC reflects the overall accuracy of the instrument in categorizing individuals with values between 0.70-0.80, 0.800.90, and ≥ 0.90, considered adequate, excellent, and outstanding, respectively (Mandrekar, 2010; Zhang et al., 2020). In the present study the Youden Index (J; Youden, 1950) was also computed in order to detected the optimum cut-off points of MOPS dimensions (Ruopp et al., 2008). This index identifies the thresholds that maximize both sensitivity (i.e., the proportion of subjects who have the target condition and provide positive test results) and specificity (i.e., the proportion of subjects who do not have the target condition and provide negative test results). The non-parametric distribution assumption was chosen.

Differences between the organized and disorganized groups were also examined by chi-square (χ[2]) and Kolmogorov-Smirnov tests for dichotomous and dimensional variables, respectively. The use of nonparametric tests was chosen because none of the examined variables were normally distributed (Shapiro– Wilk test, p< 0.05). To avoid family-wise Type I error, a formal Bonferroni correction was applied by dividing the limit of significance by the number of comparisons (i.e., 12). Therefore, the threshold level for significance was p= 0.05/12 = 0.004.

Results

Fifty-three (58.9%) and thirty-seven (41.1%) participants were classified as organized and disorganized, respectively, according to the AAI coding system. The differences between the groups are shown in . Compared with organized participants, disorganized individuals had higher scores on all MOPS dimensions. However, after a formal Bonferroni correction, only the MOPS total score, the maternal abuse sub-scale, and the maternal and paternal total score remained significantly different between the two groups.

Discriminant validity of MOPS scores

Detailed statistics on the ROC curves are shown in .

The ROC curve analysis showed that the MOPS total score (i.e., the combination of maternal and paternal dimensions) was able to categorize (AUC= 0.77) individuals with disorganized IWMs (N= 352) from individuals with organized IWMs (N= 53) (). Specifically, a score ≥ 25 (Youden index= 0.497) categorized individuals with a sensitivity of 0.69 (69% of all participants with disorganized IWMs were correctly identified) and a specificity of 0.81 (19% of participants were incorrectly identified as having disorganized IWMs).

ROC curve graph for the ability of the MOPS total score (i.e., the sum of the maternal and paternal dimensions) to discriminate between individuals with organized and disorganized IWMs of attachment

When considering the MOPS maternal dimension, a ROC curve procedure () showed that all MOPS maternal sub-scales (AUC ≥ 0.73), with the exception of maternal over-control (AUC= 0.670), were able to classify disorganized individuals (N= 37). Nevertheless, better statistical values in terms of sensitivity and specificity were obtained for the total score of the maternal sub-scales. Specifically, a score ≥ 10 on the maternal dimension (Youden index= 0.476) categorized individuals with a sensitivity of 0.70 (70% of all disorganized individuals were correctly identified) and a specificity of 0.77 (23% of participants were incorrectly identified as disorganized individuals).

ROC curve graph for the ability of the MOPS maternal (panel A) and paternal (panel B) dimension to discriminate between individuals with organized and disorganized IWMs of attachment

Finally, considering the MOPS paternal dimension, a ROC curve procedure () showed that all MOPS paternal sub-scales (AUC ≥ 0.70) could classify disorganized individuals (N= 351). Similar to the maternal dimension, better statistical values in terms of sensitivity and specificity were obtained for the total score of the paternal sub-scales. Specifically, a score ≥ 6 on the paternal dimension (Youden index= 0.460) categorized individuals with a sensitivity of 0.80 (80% of all disorganized individuals were correctly identified) and a specificity of 0.66 (34% of participants were incorrectly identified as disorganized).

Discussion

The main aim of the present study was to determine cut-off points for MOPS capable of detecting disorganized IWMs of attachment. Our results showed that all MOPS sub-scales, with the exception of maternal over-control (AUC= 0.670), were adequate in discriminating (i.e., AUC ≥ 0.70) between individuals with disorganized IWMs and individuals with organized IWMs. However, better statistical values in terms of sensitivity and specificity were obtained for the MOPS total score (i.e., the sum of the maternal and paternal dimensions), which was able to discriminate (AUC= 0.77) between individuals with disorganized IWMs and individuals with organized IWMs with a sensitivity of 0.69 and a specificity of 0.81.

As mentioned previously, the forms of dysfunctional parenting measured by the MOPS significantly increase the likelihood of developing disorganized attachment and predispose individuals to significant vulnerability to various mental disorders (Liotti, 2004). Therefore, the detection of disorganized IWMs of attachment in adults is of great importance to both clinicians and researchers. Indeed, disorganized IWMs of attachment are considered a vulnerability factor for psychopathology in general and specifically for affective and trauma-related disorders (Farina & Meares, 2022; Liotti, 2004, 2009; van Hoof et al., 2019). In samples of families at high risk for psychopathology, the percentage of disorganized attachment can be as high as 80% (Lyons-Ruth & Jacobvitz, 2008). Individuals with a history of disorganized attachment are overrepresented among patients with mental disorders, and regardless of diagnosis, they exhibit relational problems, emotional and behavioral dysregulations, and mentalization difficulties, which complicates their treatment (Farina et al., 2019; Fonagy et al., 1991; Liotti, 2017; Meares, 2012). Regardless of diagnosis, individuals with elements of disorganized attachment tend to have specific therapeutic difficulties caused by the activation of disorganized IWMs in the therapeutic relationship (Farina et al., 2019).

It is very important for researchers to have an agile tool for assessing disorganized IWMs, as it may allow the study of the role that disorganized IWMs play in populations with different mental disorders. In longitudinal or prevention studies, it may also allow the investigation of disorganized IWMs as a risk factor. There is little doubt that the AAI is the best instrument available in the literature for these purposes. However, the procedure of this highly accurate interview is time-consuming and difficult when examining large samples (and indeed, studies conducted with the AAI rarely have sample sizes greater than 100). As an alternative to the AAI, the MOPS can be used in the study of large populations, but no cut-off has ever been proposed that can distinguish between individuals at higher psychopathological risk due to disorganized IWMs and individuals at lower risk due to organized IWMs. In this study, we identified cut-off points for MOPS that are able to detect disorganized IWMs of attachment with adequate accuracy in terms of sensitivity and specificity. Specifically, we have shown that a cut-off score ≥ 25 to the total score of the MOPS provide clinicians and researchers with a rapid but reliable index to investigate the role of disorganized IWMs as a link between parental styles and mental health.

It is important to underline that the correspondence between the AAI classification of the 90 participants involved in the present study and the overall accuracy of MOPS in categorizing individuals (expressed as AUC in the ROC curve analysis) was adequate but not excellent. In our opinion, this finding primarily reflects the fact that even if dysfunctional parenting (assessed by MOPS) may constitute the basis for the development of disorganized attachment (assessed by the AAI), this causal relationship is not linear. Indeed, it has been shown that variables other than dysfunctional parenting should be considered when examining disorganized attachment. In particular, Wazana et al. (2015) examined the combined effects of constitutional risk, molecular genetics, and environmental influences on the development of attachment disorganization, and demonstrated a complex interplay between variables such as birth weight, the dopamine receptor D4 gene, and early maternal care. Therefore, when using MOPS as a proxy for assessing disorganized attachment, researchers should consider that although dysfunctional parenting may play an important role in the etiology of disorganized attachment, other variables may contribute to the process.

The contributions of the study should be considered in light of some limitations. First, we did not examine the psychometric properties of the MOPS in patients with mental disorders, so our interpretations are specific to non-clinical individuals. In addition, there was a high proportion of female participants and young adults in our sample, which limits the generalizability of these results to other populations. Therefore, further studies should be conducted to investigate the performance of MOPS in detecting disorganized IWMs in different groups, especially in clinical samples.

Despite these limitations, to our knowledge, this is the first study ever that provides cut-off scores for the MOPS and examines the accuracy of MOPS in detecting disorganized IWMs. Taken together, our results suggest that the MOPS, particularly the MOPS total score (i.e., the sum of the maternal and paternal dimensions), is a useful brief self-report instrument for detecting disorganized IWMs when time constraints prevent the use of the AAI (e.g., studies with large samples or epidemiologic studies).

Table 1.

Demographic and clinical data of the participants

Variables	Total (N = 90)	Organized (N = 53)	Disorganized (N = 37)	Test	p
Females – N (%)	66 (73.3)	41(77.4 %)	25 (67.6 %)	χ2= 1.07	0.301
Age – M ± SD	21.21 ± 2.05	20.51 ± 1.50	22.22 ± 2.31	Z= 1.42	0.036
Educational Level (years) –M ± SD	14.96 ± 1.48	14.74 ± 1.51	15.27 ± 1.41	Z= 0.98	0.297
MOPS total score –M (SD)	20.83 ± 18.66	13.81 ± 13.88	31.46 ± 20.09	Z= 2.28	< 0.001
Maternal indifference –M (SD)	2.74 ± 4.02	1.36 ± 2.27	4.73 ± 5.06	Z= 1.74	0.005
Maternal over-control –M (SD)	4.74 ± 3.32	3.92 ± 2.95	5.92 ± 3.50	Z= 1.54	0.017
Maternal abuse –M (SD)	2.54 ± 3.65	1.26 ± 2.05	4.38 ± 4.57	Z= 2.07	< 0.001
Maternal sub-scale total score –M (SD)	10.03 ± 9.59	6.55 ± 6.26	15.03 ± 11.29	Z= 2.22	< 0.001
Paternal indifference –M (SD)	4.14 ± 5.11	2.81 ± 4.36	6.14 ± 5.56	Z= 1.58	0.013
Paternal over-control –M (SD)	3.64 ± 3.10	2.81 ± 2.86	4.89 ± 3.08	Z= 1.42	0.035
Paternal abuse –M (SD)	3.05 ± 4.25	1.64 ± 2.81	5.17 ± 5.15	Z= 1.63	0.010
Paternal sub-scale total score –M (SD)	10.82 ± 11.09	7.26 ± 9.22	16.20 ± 11.62	Z= 2.11	< 0.001
Adult Attachment Interview classification
Secure/Autonomous –N (%)	21 (23.3)	21 (39.6)	-	-	-
Dismissing –N (%)	22 (24.4)	22 (41.5)	-	-	-
Preoccupied –N (%)	10 (11.1)	10 (18.9)	-	-	-
Unresolved/disorganized –N (%)	34 (37.8)	-	34 (91.9)	-	-
Cannot classify –N (%)	3 (3.3)	-	3 (8.1)	-	-

Abbreviation: MOPS= Measure of Parental Style

Note: bold indicates significant differences between groups after a formal Bonferroni correction (i.e., p= 0.004).

Table 2.

Detailed ROC curves statistics

Test Variable(s)	AUC	Standard error	p	Confidence Interval
				Lower	Upper	J index – MOPS cut-off (Sensitivity; 1-Specificity)
Maternal sub-scale total	0.74	0.06	< 0.001	0.63	0.85	0.476 – 9.50 (0.70; 0.23)
Maternal indifference	0.73	0.05	< 0.001	0.63	0.84	0.373 – 3.50 (0.49; 0.11)
Maternal over-control	0.67	0.06	0.006	0.55	0.79	0.330 – 5.50 (0.60; 0.26)
Maternal abuse	0.73	0.06	< 0.001	0.62	0.84	0.444 – 2.50 (0.60; 0.15)
Paternal sub-scale total	0.75	0.05	< 0.001	0.65	0.86	0.460 – 5.50 (0.80; 0.34)
Paternal indifference	0.70	0.06	0.001	0.59	0.82	0.345 – 3.50 (0.57; 0.23)
Paternal over-control	0.70	0.06	0.001	0.59	0.81	0.310 – 1.50 (0.86; 0.55)
Paternal abuse	0.70	0.06	0.001	0.59	0.82	0.354 – 3.50 (0.54; 0.19)
MOPS total score	0.77	0.05	< 0.001	0.67	0.87	0.497 – 24.5 (0.69;0.19)

Abbreviation: AUC= Area under the curve; MOPS= Measure of Parental Style; J= Youden index