The Dutch Infant Caregiving Assessment Scales: Psychometric properties in mothers with and without a severe psychiatric disorder.

OBJECTIVES: This study examines the psychometric properties of the Dutch adaptation of the Infant Caregiving Assessment Scales (INCAS). This standardized observation procedure is the first to assess both emotional and instrumental caregiving skills of mothers with a severe psychiatric disorder, during the postpartum period.
METHODS: Mothers with and without a severe psychiatric disorder (N = 123) were observed at home at the infant age of 6 weeks during daily caregiving; changing a diaper, bathing, dressing, and feeding. Recordings of observations were coded independently by trained coders, blind for group membership. Subsequently, the component structure, internal consistency, interrater reliability, and concurrent validity of the INCAS were examined.
RESULTS: Principal component analysis largely confirmed the two a priori defined caregiving domains. The internal consistencies of the emotional and instrumental domains were deemed excellent and good, respectively. The interrater reliability was substantial for the emotional domain and moderate for the instrumental domain. Furthermore, evidence for good concurrent validity of the emotional domain was found. Lastly, significant correlations were found between specific instrumental caregiving skills and maternal neuropsychological functioning.
CONCLUSION: Psychometric findings support the INCAS as a comprehensive and reliable instrument for standardized assessment of caregiving by mothers with a severe psychiatric disorder.

Infant Caregiving Assessment Scales

INTRODUCTION

A substantial number of mothers experience psychopathology around the birth of a child. Maternal psychiatric disorders during and after pregnancy can be categorized in psychotic, mood, anxiety, and personality disorders (Börjesson et al., 2005; Howard et al., 2014; Jones et al., 2014). In the postpartum period, depressive and anxiety disorders are the most prevalent, occurring in around 8%–12% of women (e.g., Goodman et al., 2016; Woody et al., 2017). About five in 1000 women suffer from a puerperal psychotic disorder (VanderKruik et al., 2017). Just under one in six women report disordered personality traits during early pregnancy (Crowley et al., 2020).

Maternal psychopathology significantly increases the risk of maladaptive development of the fetus and the child (Aktar et al., 2019; Stein et al., 2014) as well as the risk of child maltreatment and neglect (Ayers et al., 2019; Li et al., 2016). Moreover, chronicity and severity of maternal psychiatric disorder generally increases the impact on the quality of the mother‐child relationship and on child outcomes (Aktar et al., 2019; Madigan et al., 2018; Suchman et al., 2019).

Effects of maternal psychiatric disorder on child outcome may be mediated by genetic, intrauterine environmental, or postnatal factors, such as parenting or contextual factors (Goodman & Gotlib, 1999). Of these, parenting is an important modifiable factor and a target for intervention (Stein et al., 2014). It has been shown that maternal psychiatric disorder may negatively affect aspects of parenting behavior and parental expression and regulation of affect during parent–infant interaction, such as responsivity, (playful) stimulation, and vocalization (Aktar et al., 2019). Most studies are focused on the emotional aspects of caregiving in mothers with psychiatric disorders and show evidence of reduced maternal sensitivity and responsivity and negative and/or atypical behaviors in mother–infant interactions (e.g., Davidsen et al., 2015; Feldman et al., 2009; Hazell Raine et al., 2020; Lovejoy et al., 2000; Steele et al., 2019).

Since maternal psychiatric disorders are associated with adverse child outcomes, clinicians and researchers need to be able to assess the strengths and difficulties of parenting capacity in mothers with a psychiatric disorder in a reliable and valid way. Several observation instruments have been developed and assess interpersonal or emotional aspects of caregiving during parent‐child interaction across age groups (Lotzin et al., 2015; Mesman & Emmen, 2013). Because these instruments are not developed for mothers with a severe psychiatric disorder, they have limited capacity to capture specific caregiving challenges for this group (Knights, 2015). It is important that caregiving capacity assessment is based on a complete, representative set of behaviors relevant to infant caregiving, preferably in a functional‐contextual observation (Benjet et al., 2003).

Mothers with a severe psychiatric disorder are at an increased risk of coming into contact with child protective services within one year after their infant is born, specifically in the first month, as compared to mothers without a mental health disorder (Hammond et al., 2017). Thus, when assessing caregiving capacity in this group, it is vital that the assessment is suited for young infants (i.e., ≤12 months of age). However, many observation measures are not suited for use in young infants during the early postpartum period (Knights, 2015; Mesman & Emmen, 2013). This hampers the opportunity for early monitoring and preventive interventions in these vulnerable families. Last, most existing observation instruments only focus on emotional aspects of caregiving, while instrumental caregiving is an additional vital part of the caregiving environment (Bradley, 2019; Knights, 2015).

Although emotional aspects of caregiving are evidently important for infant development and mother–infant attachment security, it is important to also consider instrumental aspects of caregiving such as physical nurturance (e.g., sustenance and safety), cognitive stimulation/engagement, and provision and organization (e.g., material and environmental needs; Bornstein, 2019; Bradley, 2019). Instrumental caregiving by mothers with a severe psychiatric disorder may be compromised due to underlying problems with executive functions, including attention, planning, and cognitive flexibility (Barrett & Fleming, 2011; Goldstein et al., 2014). This has been demonstrated in individuals with depression, bipolar disorder, anxiety disorders, and personality disorders (e.g., Dickinson et al., 2017; Hagenhoff et al., 2013; Polak et al., 2012; Rock et al., 2014).

The Infant Caregiving Assessment Scales (INCAS), a two‐step standardized observation procedure, were developed to assess instrumental and emotional caregiving of mothers with schizophrenia (Knights, 2015). The original purpose was to take into account cognitive deficits associated with schizophrenia that were thought to play a role in caregiving challenges experienced by affected mothers. Component structure, internal consistency, interrater reliability, concurrent validity, and associations between the INCAS and several neuropsychological outcomes were examined. This preliminary psychometric study demonstrated that the INCAS were a reliable and valid method to assess maternal caregiving. However, sample size was small (N = 51) and it remained unclear if the INCAS would be suited for mothers with a severe and chronic psychiatric disorder, other than schizophrenia.

The INCAS were adapted in three ways, to be able to assess caregiving in mothers in a heterogeneous psychiatric population for both clinical as well as scientific purposes: (1) the emotional domain was expanded with one scale; (2) the distribution of all scales was modified to better capture variation in suboptimal caregiving, and (3) behavioral indicators were added per subscale to facilitate coding.

The aim of the current study is to examine the psychometric properties of these revised INCAS in mothers with a severe psychiatric disorder and healthy mothers. To include enough mothers with a severe psychiatric disorder, both groups are combined. With this approach, a broad range of maternal caregiving quality was covered, without needing to recruit an extremely large number of randomly selected participants from the general population (Vaughan, 2017). Moreover, the presence of a maternal psychiatric disorder does not automatically imply inadequate caregiving because many mothers with a psychiatric disorder might show optimal or only suboptimal caregiving.

For the psychometric analyses of the INCAS, the component structure will be tested and the internal consistency and the interrater reliability per subscale of the INCAS will be determined. We explored the concurrent validity by comparing INCAS scores to mothers' ratings on the Emotional Availability Scales in a subset of mothers with and without a psychiatric disorder (EA Scales; Biringen et al., 2014). The EA Scales are renowned and widely used to assess the affective quality of mother–infant interaction (Mesman & Emmen, 2013). Relations between INCAS scores and executive functions, as measured with neuropsychological tasks, were explored in the subsample of mothers with a psychiatric disorder.

METHOD

Study design and setting

This psychometric study is conducted within the INCAS study, an observational study on caregiving of mothers with a severe psychiatric disorder and infant development in the first year after birth. Mothers with a severe psychiatric disorder and healthy mothers were recruited during pregnancy between June 2013 and January 2016. For the clinical group, background and clinical information was obtained through questionnaires in the third trimester of pregnancy (T0) and infant characteristics were collected at 6–7 weeks postpartum (T1). All measurements in the healthy group were carried out at 6–7 weeks postpartum. Written informed consent was obtained from all mothers for their own and their infant's participation, and from fathers with legal guardianship where applicable. The study was approved by the Medical Ethics Committee of the Erasmus University Medical Center, Rotterdam (NL42662.078.12).

Recruitment and study population

Clinical group

Mothers with a severe psychiatric disorder were recruited through nine specialized Psychiatry–Obstetrics–Pediatry secondary and tertiary outpatient clinics and mental health care institutions focusing on peripartum psychiatry, throughout the central western part of the Netherlands.

The main inclusion criteria were a severe psychiatric disorder, that is, the presence of any DSM‐IV axis‐I and/or axis‐II disorder, with a duration of service contact of two or more years, and impaired daily functioning as indicated by the Global Assessment of Functioning score (GAF; Jones et al., 1995). Psychiatric diagnoses were checked by a certified interviewer (VC) using the Structural Clinical Interview for DSM‐IV axis I and II disorders (SCID I and II) (First et al., 1997, 2002). The GAF score was based on clinical diagnosis by the treating (resident) psychiatrist at intake (during pregnancy) and duration of service use was based on self‐reported patient history at intake. Exclusion criteria were (impending) outplacement of the infant, maternal insufficient command of the Dutch language, and infant prematurity (<37 weeks).

Seven‐hundred‐fifteen mothers were screened for eligibility after which 258 mothers were approached for participation. Fifty‐nine mothers provided written informed consent during pregnancy. Five mothers were excluded postpartum due to prematurity of their infant. Between T0 and T1 four mothers withdrew their consent. A total of 50 mothers with a severe psychiatric disorder and their infants were included in the analyses.

Healthy group

Healthy mothers were recruited through two general hospitals and several midwifery practices in the second and third largest cities in The Netherlands. During recruitment eligibility was checked by asking if the mother had ever experienced or was currently experiencing psychiatric symptoms. Furthermore, healthy mothers were screened for the presence and severity of psychopathology using the Brief Symptom Inventory (BSI; self‐report) at 6–7 weeks postpartum (Derogatis & Melisaratos, 1983). Mothers with a score above the cut‐off on the General Severity Index (i.e., mean BSI score; GSI >0.68), insufficient command of the Dutch language or a prematurely born infant were excluded from the study.

In total 317 healthy mothers were approached for participation and 79 women provided written informed consent during pregnancy. Three mothers were excluded postpartum based on a GSI score in the clinical range and one mother due to prematurity of her infant. Two mothers were excluded from the current analyses due to procedural and/or technical difficulties. A total of 73 healthy mothers and their infants were included in this study.

Measures

infant Caregiving Assessment Scales

The Dutch INCAS is a standardized observation procedure with two components. First, a healthcare or social work professional (e.g., psychiatric/community nurse) visits the mother and infant at home and instructs the mother to carry out four caregiving tasks: changing a diaper, bathing, dressing, and feeding (order determined by mother). Mother and infant are recorded with a handheld camera during these tasks from preparation (e.g., making a bottle) until completion (e.g., burping the infant). The observer follows mother and infant with the camera ensuring that mother and infant are clearly visible in the center of the picture. Mothers are instructed to perform the tasks as usual, in accordance with their natural routine. During the procedure the observer interferes as little as possible (e.g., quietly present while tasks are being carried out and keeping a comfortable distance from the dyad while making the recording).

Second, the video observation is macro‐coded (i.e., global ratings), preferably by an independent coder without prior knowledge about the dyad, on seven scales of emotional caregiving and six scales of instrumental caregiving, ranging from 0 (i.e., behavior is not or rarely observed and/or behavior is of minimal quality) to 4 (i.e., behavior is consistently observed and/or behavior is of high quality).

The Dutch INCAS differs from the Australian version in several ways. Firstly, a scale on hostile behavior (reversed for ease of interpretation; Non‐hostility) has been added. Previous research has shown that maternal hostility often occurs in mothers with psychiatric problems and is associated with an increased risk for child maltreatment and neglect (e.g., Brown et al., 1998; Suchman et al., 2019). Also, since it is possible for mothers to show affectionate as well as hostile behaviors, indicating that these are two separate constructs, considering both types of behaviors allow for a more comprehensive assessment of emotional caregiving. Apart from this scale, the content and a priori division of the scales into the emotional or instrumental domain is identical to those of the final Australian version. Furthermore, the distribution of the scales was adapted to better capture variation in suboptimal caregiving, since it is especially important to assess elements of suboptimal caregiving which need improvement and targeted intervention. Scores between 0 and 2 represent suboptimal caregiving, a score of 3 represents adequate/good enough caregiving, and a score of 4 represents good to excellent caregiving. Another adaptation concerned the addition of examples for a low or high score on each scale to clarify the content of the scales to facilitate coding (per scale examples are given hereafter).

The seven scales on emotional caregiving are: (1) Affection (parental verbal and/or nonverbal expressions of warmth, endearment, and positive affect toward the infant; e.g., cold attitude toward infant vs. cuddling); (2) Non‐hostility (absence of parental hostile, disapproving, and dismissive statements or behaviors; e.g., actively blaming the infant); (3) Interaction (the adequacy [i.e., attunement vs. intrusiveness] and contingency [i.e., responsiveness] of verbal and/or nonverbal social stimulation and communication of the parent with the infant; e.g., intrusively touching the infant's face vs. attuned to infant's sounds); (4) Empathy (the concern for the subjective experience of the infant and the extent to which the parent is gentle and considerate; e.g., objective handling of the infant vs. gentle care); (5) Adaptability (flexibility and responsivity regarding infant needs and behaviors; e.g., rigidly carrying out caregiving tasks vs. adapting length of task in response to infant signals); (6) Emotion regulation (parental ability to soothe, settle, and temper arousal in a timely and effective manner in response to infant crying and fussing; e.g., applying few strategies to soothe the infant vs. anticipating infant dysregulation by thoughtful planning); and (7) Mentalisation (the extent to which the parent demonstrates understanding of the infant's putative [cognitive and emotional] experiences and intentions as evidenced in verbalisation(s) about the infant's behaviors; e.g., distorted vs. attuned maternal verbalizations about infant mental state) (Meins, 2013; Meins et al., 2003).

The six scales on instrumental caregiving are: (1) Protection (parental ability to provide a safe, secure, hygienic, and health‐promoting environment for the infant; e.g., leaving the infant alone on the changing table vs. laying the infant down safely in the crib); (2) Focus (directing, maintaining and dividing attention and vigilance over the course of the caregiving tasks; e.g., becoming distracted vs. avoiding distraction by external factors); (3) Competence (skill, knowledge, and ability during preparation and caregiving; e.g., proficient approach vs. uncertain, hesitant approach); (4) Provision (the extent to which parents can meet the infant's basic material needs and have adequate materials to carry out caregiving; e.g., no clothes in infant's size vs. use of adequate baby skin care products); (5) Diligence (effort, conscientiousness, thoroughness, and a commitment to task completion; e.g., carelessness vs. multiple attempts to burp the infant after feeding); and (6) Holding (the ease, comfort, and safety of physical handling and control; e.g., lack of vs. adequate support of infant's neck and head while carrying).

For the purpose of the current study, most home observations were carried out and recorded by the researcher (VC) at six to seven weeks postpartum. Subsequently, all video observations were coded independently by two trained coders (six combinations from a group of four graduate students). Coders were unaware of group status and other clinical or background characteristics concerning the mother–infant dyad. Coders were extensively trained and supervised weekly by the researchers (VC and RK). Reliability of the coders was assessed directly after the training and at the end of the coding process to detect possible rater drift. Individual scores were combined into a consensus score according to the following procedure: equal scores were retained, 1‐point differences were averaged, and ≥2‐point differences were discussed by the coders under supervision by the researcher (VC), who remained blind, until a consensus score was reached. A total of 5.6% of all scale scores were discussed for consensus. Difficult to code recordings (5% of coded cases) as indicated by the trained coders, for example, due to culture‐specific practices or procedural challenges, were checked or re‐coded by the researchers (VC and RK). Average emotional and instrumental domain scores were based on the consensus scores.

Emotional Availability Scales fourth edition (EA scales)

The EA Scales assess the affective quality of caregiver‐child interaction and have acceptable psychometric properties (Biringen, 2008; Biringen et al., 2014). The EA Scales are applicable in infancy and have been applied in multiple observational settings in previous studies. Therefore, the EA Scales were selected to explore the concurrent validity of the INCAS emotional domain. The EA Scales comprise sensitivity, structuring, non‐intrusiveness, non‐hostility, child responsiveness, and child involvement. Each component is scored on a 7‐point scale. Scores are summed to obtain an EA composite score.

For the current study, a random subset of the INCAS video observations (clinical n = 20, healthy n = 25) was coded with the EA Scales by two trained and EA licensed child and adolescent psychiatrists (MvL and MLvdB) who were blind to maternal group status (i.e., clinical vs. healthy) and maternal or infant background and clinical characteristics. No differences were found in background and clinical characteristics between video observations of mother–infant dyads which were coded with the EA system and those which were not (data not shown).

Maternal neuropsychological functioning

Neuropsychological functioning of mothers in the clinical group, as a correlate of the instrumental domain, was studied with three computerized tasks from the Vienna Test System NEURO (Schuhfried GmbH): WAFV to assess sustained visual attention, Tower of London‐Freiburg version to assess planning ability (TOL‐F), and SWITCH to assess task cognitive flexibility. These were administered in counterbalanced order (total duration ±50 min) at the 6‐7 weeks postpartum visit.

The WAFV consists of 900 consecutive trials in which a small stimulus is shown in the middle of the screen. Two‐hundred‐fifty stimuli are presented with diminished color saturation. Subjects are asked to respond to these target stimuli as quickly as possible by pressing a reaction button. Internal consistency of the WAFV is high (Sturm, 2012). Sustained attention was operationalized as the (logarithmic) mean reaction time.

Each problem (12 in total) in the TOL‐F consists of two objects representing a start state and goal state. The subject is asked to achieve the goal state within a set number of steps. The internal consistency for the short form of TOL‐F is acceptable (Kaller et al., 2011). Planning ability was operationalized as the number of optimally solved problems.

The SWITCH consists of 160 stimuli (i.e., squares and circles in light and dark grey) and two predictable rules (i.e., “response based on color” or “response based on shape”) according to which the subject must “switch” after every two stimuli (AABBAABB etc.) and give an appropriate response. Reliability of the SWITCH is deemed satisfactory (Gmehlin et al., 2015). Cognitive flexibility was operationalized as the mean reaction times of correct responses during repetition (i.e., AA or BB) and shift (i.e., AB and BA) tasks.

On each task, outliers on the main outcome and on relevant process scores (e.g., missed stimuli or percentage correct) were detected with Z‐scores and subsequently removed. Also, for five or six cases, depending on the specific task, data were missing due to procedural difficulties.

Statistical analyses

Independent samples t‐tests and Chi‐square tests were used to explore differences in background and clinical characteristics between the clinical group and healthy group. An exploratory principal component analysis was carried out to investigate whether the proposed two caregiving dimensions (i.e., emotional and instrumental) emerged from the 13 scales of the INCAS. A varimax rotation was used to ensure maximum differentiation between these caregiving dimensions and to facilitate interpretation. Secondly, the internal consistency of the INCAS emotional and instrumental domain scores were investigated by calculating Cronbach's alphas (n = 122 for both domains due to missing scale scores). Thirdly, intraclass correlation coefficients (ICCs) were calculated to examine interrater reliability, based on a mean rating (of two raters), absolute agreement, two‐way mixed model. The ICCs were calculated for the dyad that coded the largest number of observations (cell sizes ≤15 were excluded from ICC analysis; n = 19 after training and n = 43 after coding process). Lastly, concurrent validity was explored by calculating the Pearson correlations between INCAS scores, EA scores, and maternal neuropsychological performance.

RESULTS

Background and clinical characteristics

Although we combined the clinical and healthy group for the analyses, we describe the two groups separately in Table 1 and tested group differences between mothers with a severe psychiatric disorder and healthy mothers in background and clinical characteristics. Table 1 shows that less mothers with a psychiatric disorder were currently in a relationship or married in comparison with healthy mothers. Also, mothers with a severe psychiatric disorder had a significantly lower educational level and fewer mothers were currently employed than healthy mothers. Infants of mothers with a severe psychiatric disorder had a significantly lower gestational age than infants of mothers in the healthy group.

TABLE 1

Background characteristics clinical mother–infant dyads (n = 50) and healthy mother–infant dyads (n = 73)

	Women with severe psychiatric disorder a	Healthy women b	t	χ2	p
Mother
Age	31.3 (5.9)	32.4 (4.9)	1.12	‐	0.26
Ethnicity, % Dutch	84	82.4	‐	0.06	0.81
Relationship status, % partner	91.7	100	‐	5.70	0.02
Marital status, % married	41.3	60.6	‐	4.05	0.04
Educational level, % high	22.9	47	‐	6.91	0.01
Employment status, % yes	40.4	77.6	‐	16.26	<0.001
Primiparity, % yes	60.4	46.9	‐	2.02	0.16
Infant
Gestational age	39.3 (1.7)	39.8 (1.3)	1.95	‐	0.05
Age at INCAS observation, in weeks	6.7 (1.8)	6.1 (1.2)	−2.14	‐	0.06
Sex, % boys	52	52.1	‐	0.00	1.00

Note: Unless otherwise indicated, values are mean (SD).

Abbreviation: INCAS, Infant Caregiving Assessment Scales.

Sample size varies between 45 and 50.

Sample size varies between 64 and 73.

All mothers in the clinical group met the criteria for one or more DSM‐IV Axis‐I and/or Axis‐2 disorder. Most mothers with an Axis‐I disorder had a mood or anxiety disorder, while psychotic or other disorders occurred less (i.e., 76%, 46%, 6%, and 10%, respectively). Two or more Axis‐1 disorders were present in 46% of mothers. Forty‐four percent of mothers met the criteria for an Axis‐2 (i.e., personality) disorder.

Component structure

Sampling adequacy was verified with the Kaiser–Meyer–Olkin measure and was qualified as “marvelous” (KMO = 91; Hutcheson & Sofroniou, 1999). All KMO values for the 13 individual scales of the INCAS were greater than 0.82, which is well above the acceptable limit of 0.5 (Field, 2013).

The initial exploratory principal component analysis (PCA) revealed two components with eigenvalues greater than 1, which cumulatively explained 63% of the variance. Eigenvalues of these components levelled off after the second inflection on the scree plot. This solution was in accordance with the a priori division of the INCAS into an emotional and instrumental domain. After varimax rotation, the two main components respectively explained 37.1% and 25.9% of the variance (see Table 2). Component loadings above 0.51 were considered statistically significant (Stevens, 2002). For further interpretation of the content of the components (i.e., emotional or instrumental), the difference between the two components loadings per scale were considered (Tabachnick & Fidell, 2013). Six scales of the emotional caregiving domain loaded significantly onto the first component and five scales of the instrumental caregiving domain loaded significantly onto the second component. Two scales, namely Focus and Mentalisation, showed significant loadings on both components. Furthermore, for these scales the difference between loadings on both domains was relatively small, as compared to the other scales. Removing these two scales and repeating the PCA with a varimax rotation yielded similar results. Both scales were then retained in the proposed domain because of their theoretical importance for infant caregiving capacity.

TABLE 2

Component loadings for the principal component analysis with varimax rotation of the Infant Caregiving Assessment Scales (n = 121)

	Rotated component loadings
Scale	Component 1	Component 2
Interaction	0.85	0.30
Emotion regulation	0.85	0.17
Flexibility	0.85	0.22
Affection	0.82	0.35
Empathy	0.77	0.43
Non‐hostility	0.64	0.03
Focus	0.60	0.44
Mentalisation	0.60	0.40
Protection	0.21	0.81
Holding	0.06	0.79
Diligence	0.29	0.73
Competence	0.27	0.67
Provision	0.25	0.54
Eigenvalues	6.67	1.52
Variance, %	37.14	25.87

Note: Component loadings ≥0.51 were considered significant and appear in bold. Component 1 = emotional caregiving; Component 2 = instrumental caregiving. There were missing values for this analysis on one or more subscales due to procedural difficulties (n = 2).

Internal consistency and interrater reliability

The emotional domain had an excellent internal consistency (Cronbach's α = 0.92, n = 122). The internal consistency of the instrumental domain was good (Cronbach's α = 0.82, n = 122).

Interrater reliability (ICC) for the emotional domain directly after the training was 0.48 (see Table 3; range 0.19–0.62; n = 19) and 0.77 at the end of the coding process (range 0.58–0.77; n = 43). The ICC of the emotional domain can be described as good at the end of the coding process (Koo & Li, 2016).

TABLE 3

Intraclass correlations per INCAS subscale after reliability training (n = 19) and the coding process (n = 43)

	Reliability training	Coding process
Affection	0.38	0.68
Non‐hostility	0.60	0.70
Interaction	0.46	0.66
Empathy	0.56	0.61
Flexibility	0.31	0.72
Emotion regulation	0.62	0.58
Mentalisation	0.19	0.77
Emotional domain mean	0.48	0.77
Protection	0.48	0.47
Focus	0.40	0.50
Competence	0.37	0.43
Provision	0.46	0.81
Diligence	0.30	0.24
Holding	0.43	0.57
Instrumental domain mean	0.59	0.59

The ICC for the instrumental domain was considered moderate, with an average of 0.59 (range 0.30–0.52; n = 19) after training as well as at the end of the coding process (range 0.24–0.81; n = 43).

Concurrent validity of the emotional and instrumental domain

The INCAS emotional scales and domain score showed a moderate to strong positive correlation with the individual and overall EA Scales (see Table 4).

TABLE 4

Association between INCAS Emotional scale, domain scores, and the Emotional Availability Scales (n = 45)

	Affection	Non‐hostility	Interaction	Empathy	Flexibility	Emotion regulation	Mentalisation	INCAS emotional domain score
EA sensitivity	0.45**	0.31*	0.64**	0.59**	0.55**	0.61**	0.36*	0.62**
EA structuring	0.51**	0.30*	0.69**	0.64**	0.56**	0.56**	0.46**	0.67**
EA non‐intrusiveness	0.34*	0.23	0.60**	0.65**	0.47**	0.49**	0.34*	0.60**
EA non‐hostility	0.41**	0.37*	0.31*	0.41**	0.44**	0.34*	0.08	0.35*
EA total	0.41**	0.35*	0.64**	0.64**	0.50**	0.58**	0.32*	0.61**

Abbreviations: EA, Emotional Availability subscale; INCAS, Infant Caregiving Assessment Scales.

**p < 0.01, *p < 0.05

Sustained attention was correlated with less focused attention and vigilance during caregiving (see Table 5; Focus r = −0.41, p = 0.007). For planning ability a trend toward significance was found for correlations with Holding (r = 0.30, p = 0.059) and Diligence (r = 0.29, p = 0.086). For cognitive flexibility a trend toward significance was observed for correlations with Focus (r = 0.29, p = 0.058) and Holding (r = −0.26, p = 0.098).

TABLE 5

Association between INCAS Instrumental scale and domain score and maternal neuropsychological performance

	Protection	Focus	Competence	Provision	Diligence	Holding	INCAS instrumental domain score
Sustained attention a	0.14	−0.41*	−0.03	0.05	−0.10	0.03	−0.08
Planning ability b	0.15	0.09	0.12	0.04	0.29 d	0.30 d	0.20
Cognitive flexibility c	−0.15	0.29 d	0.08	−0.05	−0.09	−0.26 d	−0.04

Abbreviation: INCAS, Infant Caregiving Assessment Scales.

n = 42.

n = 44.

n = 43.

associations with a trend toward significance.

*p < 0.01

DISCUSSION

In this study, the psychometric properties of the adapted INCAS were examined in a sample of mothers with a severe psychiatric disorder and healthy mothers, during the early postpartum period.

The a priori defined emotional and instrumental domain structure was largely confirmed. In particular, the instrumental domain seems a valuable addition when assessing caregiving capacity in mothers, explaining a considerable part of variance (i.e., 25%), beyond the emotional domain (i.e., 37%). This demonstrates that the INCAS are a valuable instrument for identifying specific variations in maternal caregiving as compared to existing instruments which focus only on emotional aspects of parenting. In other words, with the INCAS previously undetected instrumental strengths and weaknesses can be observed in a standardized manner, offering healthcare professionals valuable insights. However, it should be noted that the Focus and Mentalisation scales loaded significantly on both the emotional and instrumental caregiving domains and showed a small difference between loadings on each domain. In the Australian version of INCAS, the Focus scale only loaded significantly on the instrumental domain, but the Mentalisation scale showed significant loadings on both domains with a relatively small discrepancy. Our study supports the notion that the Focus scale (i.e., attention for task and infant) has instrumental as well as emotional elements. This scale may include indicators of emotional caregiving, because focus on the task and the infant could be influenced by either mothers' ability to focus on infant signals (i.e., emotional skills) or by maternal attentional processes (i.e., instrumental skills), or a combination of both. Furthermore, the Australian study focused on mothers diagnosed with schizophrenia and revealed specific neurocognitive deficits in this group (Knights, 2015). In our study, mostly women with a mood, anxiety and/or personality disorder were included. This suggests that specific symptomatology might interfere with the extent to which Focus can be considered a purely instrumental scale.

Interrelatedness of maternal executive functions and mentalisation could explain why the Mentalisation scale showed a relatively small difference between the loadings on both domains. Aspects of maternal executive functioning (underlying instrumental skills), such as the ability to inhibit irrelevant information have been negatively associated with the attunement of maternal comments on infant mental states (i.e., emotional skills) before (Yatziv et al., 2018). Given the theoretical and clinical relevance of the Focus and Mentalisation subscales, these subscales were retained in the INCAS. Future research could further explore the specific relevance of these subscales for emotional and instrumental caregiving.

The INCAS emotional domain showed excellent internal consistency, which is in line with other observation instruments of emotional caregiving, such as the EA Scales (e.g., Derscheid et al., 2018). Similarly, we found good internal consistency for the instrumental domain. Together with the findings from the principal component analyses, these results suggest that the INCAS instrumental domain sufficiently captures a distinct and coherent caregiving construct.

The interrater reliability for the emotional domain was moderate after training and increased to a substantial level at the end of the coding process, presumably as the result of intensive supervision. The interrater reliability for the instrumental domain as a whole (with increased reliabilities on specific subscales) was moderate after training and remained as such at the end of the coding process. These findings suggest that reliability, specifically on the emotional domain, increased when our graduate‐level coders became more experienced and further developed their frame of reference. To increase interrater reliability for both domains, it is recommended to train and regularly supervise professionals, with existing knowledge of and experience with parent‐infant interaction and infant care, to assess maternal caregiving using the INCAS. Since, the INCAS observation procedure yields rich and detailed information, it requires existing proficiency to structure and weigh this information to reach strong agreement.

The strong positive associations between the INCAS emotional domain and the EA Scales confirmed the concurrent validity of the INCAS emotional domain. Our exploration of the concurrent validity of the instrumental domain showed that mothers with a psychiatric disorder, who were able to sustain their attention longer during the computerized task, also had a stronger focus on the caregiving task and their infant as measured with the INCAS. We did not find other significant associations. Research suggests that neuropsychological tests that assess executive function have a moderate level of ecological validity when predicting everyday (cognitive) functioning, which might be due to the (differing) environmental cognitive demands that neuropsychological tasks pose in a structured testing environment versus in an everyday situation (Chaytor & Schmitter‐Edgecombe, 2003). However, our findings also suggest that the correlates of instrumental infant caregiving need reconsideration. It is possible that the instrumental domain is not only determined by maternal executive functions, but also by elements of socio‐economic status (captured by the Provision scale), and maternal practical skills and knowledge (captured in the Competence, Diligence and Holding scales). Research has shown that SES and parenting can be linked through various factors including parental mental health and parental knowledge of childrearing and child development (Hoff & Laursen, 2019). Taken together, our findings provide positive preliminary evidence for concurrent validity of the INCAS.

The INCAS have several strengths including the applicability for mothers with a severe psychiatric disorder, the suitability for use with young infants, and the unique focus on emotional as well as instrumental capacities to yield a rich, differentiated, and ecologically valid assessment of maternal caregiving. Furthermore, an important methodological strength of our study is the clinically representative sample, combined with a healthy group, as a means to capture the variation of maternal caregiving behavior.

Our work provides several implications for the use of the INCAS, together with directions for future research. The INCAS could be particularly useful in a treatment setting in which questions arise about maternal parenting capacity. Next to psychopathology, this could extend to parents facing multiple psychosocial and/or substance use problems. It is recommended that an extensive reliability training is completed before INCAS scale scores are used in research or clinical practice and followed up with regular intervision. For the purpose of this validation study, recordings were made by the researcher and were coded afterwards by trained coders, blind to maternal group status. Blind coding is not feasible in clinical practice. Though independent coding might be possible, it is not necessary to arrive at a useful assessment of caregiving capacity. For research purposes, it is possible to use domain scores when identifying (individual) strengths and difficulties is not the primary goal. Because the interrater‐reliabilities of the emotional and instrumental domain scores are higher than those of the individual scales, these could serve as a useful global outcome measure without specification on scale level. Further research is needed into the correlates of the instrumental domain, for example, using measures of everyday executive functioning (e.g., Behavior Rating Inventory of Executive Function; Roth et al., 2014) and SES. Also, future application of the INCAS in clinical practice would benefit from research on the predictive validity in relation to early infant outcomes (e.g., stress regulation) and mother–infant relationship quality (e.g., mother–infant attachment). Potentially, the recording of the INCAS observation could be used to discuss caregiving strengths and difficulties with the mother, as is done in existing video feedback interventions (e.g., van Doesum et al., 2008). It should be noted that when the INCAS observation procedure would be applied in a treatment setting, it is important to have a supportive and non‐judgmental attitude and take the patient's experience of the observation procedure into account given their vulnerable position. As such INCAS could contribute to treatment for mothers with a psychiatric disorder.

In conclusion, the INCAS appear to be a feasible, reliable and comprehensive method to assess maternal caregiving shortly after childbirth. Early assessment with the INCAS could provide healthcare professionals and parents with a valuable insight into specific maternal strengths and difficulties, which can improve the wellbeing of mothers with a severe psychiatric disorder and their children.

CONFLICT OF INTEREST

The authors report no conflicts of interest.

ETHICAL APPROVAL

The study was approved by the Medical Ethics Committee of the Erasmus University Medical Center, Rotterdam (NL42662.078.12).