The role of acquaintanceship in the perception of child behaviour problems.

The role of acquaintanceship with the child on reports of child behaviour by different informants was examined within the framework of a general theory of personality judgment. Mothers of referred children and group-care workers rated videotaped behaviour samples of a well known and an unknown child in the clinic. Independent observers also rated the videotapes. In line with the acquaintanceship hypothesis, mothers were found to perceive more behaviour problems than independent observers when rating well known children but not unknown children. Contrary to the acquaintanceship hypothesis, however, the group-care workers in our study reported more behaviour problems than the other informants regardless of their acquaintance with the children. The clinical and methodological implications of these findings are discussed.

Introduction

Assessment of child behaviour problems greatly relies upon reports of such adult informants as parents, teachers and child care professionals. However, only modest agreement is typically found between the reports of different informants and the question is which informant provides the most accurate information regarding a particular child [2, 15]. A large body of research has further shown both the accuracy of an individual source of information and the degree of consensus among informants to heavily depend upon various situational and informant characteristics [2, 13, 17]. These characteristics include the consistency of child behaviours across situations, the amount of behavioural information available to the informant, motivation on the part of the informant, personality of the informant and acquaintanceship with the child [22].

Some of the most contradictory results with regard to the perception of child behaviour problems concern the impact of acquaintanceship with the child [10, 11]. In some studies, acquaintanceship is defined in terms of a family (i.e., parent–child) relationship, but in other studies a broader range of ‘acquaintances’ such as peers, teachers, or group-care workers has been included. In the present study, we will use the term acquaintanceship in the sense of ‘knowing a person well’ for all types of acquaintance or familiarity with the child, including family relationships. Acquaintanceship has been found to affect perceptions of child behaviour in very different—and apparently inconsistent—manners. For instance, Kendziora and O’Leary [11] reported a tendency for mothers to provide more favourable appraisals of their own children’s behaviour as opposed to other children’s behaviour. Consequently, they concluded that the behavioural appraisals of the mother’s in their study were biased when compared to the appraisals of independent observers. In other studies, no such differential effect of acquaintanceship has been reported [21] or mixed results have been reported [18]. In the study by Snarr et al. [18], for example, the mothers of oppositional boys showed a negative interpretive tendency while the mothers of control children showed a positive interpretive tendency but only for stimuli presenting their own children as opposed to other children. On the basis of these contradictory results, Snarr et al. argued that the biases in maternal ratings of their own children may not be universal but depend on the severity of the behaviour problems and the particular parent–child history.

A more general explanation for the observed variability in child ratings by close acquaintances or strangers can be provided by researchers and theorists in the domains of social and personality psychology. Ample evidence from these domains shows increased acquaintance to clearly affect the accuracy of judgments about personality characteristics of a person. For example, Blackman and Funder [5] showed both interjudge consensus and accuracy, defined as self-other agreement, to be much higher for observers who had known the person being judged for an average of 14 months than for those who were unacquainted with the person being judged and only observed the person on video for 30 min. In terms of Funder’s [7, 8] Realistic Accuracy Model, increased information about the target—greater acquaintance between target and judge—should generally lead to enhanced detection and interpretation of behavioural cues and thus to increased accuracy in the judgment of a person’s personality characteristics.

Blackman and Funder [5] explain the effect of acquaintance on accuracy in personality judgements as follows. Acquaintances have the opportunity to observe a person on various occasions and thus observe and detect more behaviours that are relevant to certain personality characteristics. In other words, acquaintanceship not only leads to more accurate personality judgment but also—as a prerequisite—to the detection of more related behaviours. As a consequence, an observer who knows a person better not only can give a more accurate rating of his personality, but should also be able to give a more accurate rating of his behaviour. Applied to the ratings of behaviour problems of children referred to child psychiatric services or other kinds of child services, the acquaintanceship hypothesis predicts the report of greater amounts of behaviour problems by acquaintances as opposed to strangers. Moreover, this does imply that the child behavioural ratings of acquaintances (such as parents) are not necessarily biased when compared to the ratings of independent observers, but, on the contrary might be accurate.

The acquaintanceship hypothesis provides not only an alternative explanation for the observed variation in maternal perceptions of children’s behaviour but can also help us re-frame one of the most frequently discussed methodological issues in research on observer biases due to informant personality traits—namely, the use of independent observers to provide criterion ratings [17]. Up to the turn of the century most of the research on this topic has been concerned with the demonstration of bias due to maternal depression, depressed mothers were found to report greater child behaviour problems than both non-depressed mothers and other informants serving as criterion raters (e.g., teachers, group-care workers). As Richters [17] has pointed out, however, none of these studies provides convincing evidence of depression-related distortion because the mothers and criterion raters invariably rate different child behaviours in different contexts. According to Richters, carefully validated and independent ratings of the same child behaviours under more or less identical circumstances are needed to provide conclusive evidence of bias in the perception of child behaviour. More recently, a few laboratory studies have indeed been undertaken with video ratings by independent observers not previously acquainted with the child serving as the criterion for evaluation of depression-related distortion of maternal perceptions of child behaviour [14, 20, 21]. These studies have produced mixed results and thus mirror the contradictory results obtained in field studies.

In using judges not previously acquainted with the children being observed as criterion raters the laboratory studies have unintentionally raised some doubts about the use of independent observers to evaluate child behaviour. In all of the laboratory studies, that is, considerable differences were detected between the ratings provided by knowledgeable versus independent observers. In the study by Kroes et al. [14], in fact, the mothers, teachers and group-care workers reported twice as many problem behaviours as independent observers for the exact same children on the same videotape. In the study by Weis and Lovejoy [20], the mothers also reported more than twice as many—both positive and negative—child behaviours as extensively trained independent observers, which led the authors to conclude that “mothers and observers used the scales differently, with mothers reporting higher levels of all behaviours” (p. 223). Youngstrom et al. [21] also found maternal reports of both positive and negative child behaviours to consistently differ from the reports of independent observers for the same children.

The question, however, is whether the differences between the maternal and independent observer ratings are due to differential use of the rating scales or acquaintanceship. This implies that bias in maternal reports of their own children is not convincingly demonstrated and that further experimental research is necessary. In the present study, we propose to approach this issue from a different theoretical angle.

The present study

In light of an hypothesized acquaintanceship effect, the nature of the differences between the ratings provided by mothers and group-care workers who were well acquainted with these children versus independent observers not acquainted with these children will be evaluated. In order to provide convincing evidence for an effect of acquaintance, two different conditions are employed. First, mothers and group-care workers are asked to rate the amount of problem behaviour for video recordings of both known and unknown children. Second, independent observers who did not previously know the children, thus, are also asked to rate the same videos. This design allows us to compare (a) the ratings of known versus unknown children by the same judges and (b) the ratings of judges who were acquainted with versus strangers to the same children. In keeping with the hypothesis of an acquaintanceship effect, we expected the mothers and group-care workers to consistently report higher levels of problem behaviour for the known children when compared to the independent observers but similar levels of problem behaviour for the unknown children when compared to the independent observers.

Method

Participants

The participants were 55 mothers of 43 boys and 12 girls aged 6–13 years (M = 8.9, SD = 1.9) in residential or day-treatment programs at Entréa, a Dutch clinic for the treatment of youth with emotional and behavioural problems. The mean age of the mothers was 38.5 years (SD = 4.8, range of 25–52 years). All of the children were diagnosed with psychiatric disorders such as Attention deficit hyperactivity disorder, pervasive developmental disorder, oppositional defiant disorder or conduct disorder according to the DSM-IV [4]. None of the children met the criteria for mental retardation. The occupational status of the mothers was categorized using the social demographic inventory (SDI) [19]. Along a six-point scale ranging from (1) unskilled labour to (6) academic career, the median SDI score was 3. The mothers signed consent forms and were given a gift with a value of 15 euros after their participation.

The group-care workers for the same children were similarly invited to participate in the present study. To insure consistency of acquaintance with the children, those workers who had known the target child for less than 6 months were excluded from the study. Other group-care workers were not included due to job change, illness or unwillingness to participate. The final sample included 26 (or 80%) of the group-care workers who were initially asked to participate. Some of the 26 group-care workers worked with more than one of the children whose mothers participated in the study, which resulted in 44 worker-child dyads. The mean age of the group-care workers was 33.0 years (SD = 6.77); 81% was female. All of the group-care workers had a college degree in special education. The professional experience of the group-care workers ranged from 1 to 20 years (M = 6.8, SD = 5.1). The group-care workers were also given a gift with a value of 15 euros after their participation. The recruitment procedure is described in greater detail in Kroes et al. [14].

Measures

Direct observation form (DOF)

The DOF [1, 6] was designed to assess the behaviour problems observed in—among other settings—classrooms and group activities. The DOF is easy to use with teacher aids and research assistants after training by an experienced observer [3]. The DOF consists of 96 items, 72 of which have counterparts in the CBCL. Each item is rated along a scale ranging form 0 (=no observed occurrence of the behaviour) to 3 (=definite occurrence with severe intensity or a duration of three or more minutes). The sum of all the items constitutes the total problems score, which we used in our analyses.

The DOF is normally completed after 10 min of live observation. In the present study, which involved the rating of videotapes, an alternative assessment procedure was followed. Rather than complete the DOF form, the informants were asked to sort a deck of 96 cards with the DOF items listed separately on them in two steps directly following the viewing of a videotape. In the first step, the mothers, group-care workers or independent observers were asked to select those DOF items which they definitely did not observe on the videotape. These cards were then removed from the deck and assigned a score of 0 by the interviewers. In the second step, the informants were asked to sort the remainder of the cards into three piles reflecting a DOF rating score of 1, 2 or 3 for the item. Written descriptions of the rating scores were present to facilitate the sorting process. The interviewers subsequently transferred the results of the sorting procedure onto the DOF form. This two-step assessment procedure was designed to promote very deliberate decisions regarding each item and consistent application of the scoring procedure across informants.

Procedure

Videotape recordings were made of the behaviours of the target children, i.e. the children of the mothers who participated in the study. These recordings were made in a standardized setting in the treatment centre which showed one of the target children interacting in playing a board game with three other children (who were also under treatment in the clinic but did not participate in the study). The setting was organized using a video script to both standardize the behaviour samples and elicit a range of naturalistic child behaviours. The video script was tested in a previous study [16]. A 17-min videotaped behaviour sample was obtained for each of the target children. Each mother and each group-care worker observed only one target child on videotape (along with the three other children who volunteered in playing with the target child).

To attain a behaviour sample for a control child, two additional 17-min recordings were made within the same setting for two children, one of each sex. Each observer also viewed a videotape of a male or female ‘unknown’ (control) child, depending on the sex of the ‘known’ child (to control for the influence of sex of the child on variations between familiar and unfamiliar child ratings). These two control children were carefully selected to make sure that none of the professionals (or mothers) participating in the study were familiar with them. The videotapes of the two control children were recorded in the same clinic using the same standardized setting as was used for the known children.

The mothers and group-care workers were asked to watch the videotapes of a (well) known child and an unknown child individually and then assess the behaviour problems observed on the videotapes. The order of presentation for the videotapes of the known and unknown children was counterbalanced. The sex of the unknown child was matched to the sex of the known child, to control for a possible contaminating effect of sex differences on the child behaviour ratings. The sessions were all conducted at the clinic by trained research assistants (N = 5) who were unfamiliar with the target children. The research assistant conducting the session with a particular informant was also unaware of the ratings provided by the other informants for the same child. The mothers and group-care workers were asked to assess the behaviour problems observed on the videotape using the DOF immediately following the viewing of each videotape.

The independent criterion observations for the 55 target children and two control children were performed individually by two undergraduate psychology students with previous training on child observation and assessment. The independent observers were given additional training on the DOF by an experienced observer until an inter-rater Kappa for the DOF items of at least 0.80 on a set of five pilot videotapes was reached. The final inter-rater agreement between the independent observers on the videotapes of the target children was found to be 0.85 (n = 55). After their initial—individual—rating of the target videotapes, the independent observers further discussed any discrepancies until consensus was reached. The independent observers were instructed to review the videotapes as many times as they felt necessary, and the agreed-upon scores were used in all of the statistical analyses.

The mothers, group-care workers and independent observers were all instructed to rate only those behaviours which they had actually observed on the videotape.

Data analyses

The effects of acquaintanceship were evaluated in a repeated-measures ANOVA for the DOF video ratings with informants (i.e., mothers, group-care workers, independent observers) as the within-subjects factor. Planned contrasts were tested for the differences between (1) mothers versus independent observers, (2) group-care workers versus independent observers and (3) mothers versus group-care workers.

Separate analyses were conducted on the DOF ratings of the known and unknown children. For the known children, the analyses involved those video recordings which were rated by all three types of informants. This meant the ratings provided by 44 mothers, 26 group-care workers who rated the same 44 videos (with some group-care workers rating more than one video), and the ratings of the same 44 videos by the independent observers averaged across the two observers per video. For the unknown (or control) children, the mothers and group-care workers were all asked to rate the videotape of the male or female control child depending on the sex of the known child being rated by them. Given that each of the group-care workers rated the videotape of only one control child, the sample size was thus reduced to 26 videos for the analysis of the unfamiliar child ratings. The independent observers rated the videotapes of both the male control child and the female control child. For the analyses of the ratings of the unknown children, each rating of an unknown control child by the mother was matched with the rating control child of the same sex by the independent observers and a possibly differential effect of sex of the unknown child was thus avoided in such a manner.

Results

Descriptive data

The means and standards deviations for the child behaviour ratings (DOF) provided by the mothers, group-care workers and independent observers are presented in Table 1. In order to attain an indication of the level of problem behaviour occurring on the videos, the DOF behaviour problem scores produced by the informants were compared to the DOF scores presented in the Manual for the ASEBA School-age Forms and Profiles [3]. In the Manual, an average total problems score of 9.1 (SD = 4.1) is reported for trained observers rating 10-min samples of classroom behaviour for referred children. In our study, the independent observers assigned an average total problems score of 12.23 to the videotaped behaviour samples for the children well known to the mothers and group-care workers and an average score of 17.88 to the videotaped behaviour samples for the two unknown control children. Taking the difference in the durations of the videotapes into consideration (i.e., 10 min for the normative sample vs. 17 min for the present sample), we can conclude that the independent observers viewed roughly the same level of problem behaviours in the present sample as the trained observers in the normative sample of referred children.

Table 1

Raw scores and ANOVA results for known and unknown child behaviour ratings by different informants

Informant	N	M	SD	Repeated measures (GLM)
				Planned contrast	df	F	p
Known child ratings (DOF)
Mothers	44	25.95	13.95	Mothers versus independent observers	1.43	47.62	0.000
Group-care workersa	44	35.70	20.06	Group-care workers versus independent observers	1.43	68.95	0.000
Independent observers	44	12.23	5.78
Unknown child ratings (DOF)
Mother	26	18.27	16.02	Mothers versus independent observers	1.25	0.02	0.902
Group-care workers	26	34.62	20.34	Group-care workers versus independent observers	1.25	19.01	0.000
Independent observersb	26	17.88	3.17

aMultiple observations per group-care worker

bThe numbers of ratings of male versus female control children by the independent observers was matched with the number of male versus female control children rated by the mothers

Acquaintanceship

The results of the repeated-measures ANOVA to evaluate the influence of acquaintanceship on the ratings of child behaviour problems are also presented in Table 1. The multivariate statistics show significant overall group differences for the known child ratings (F(1,41) = 93,81, p = 0.000) as well as for the unknown control children (F(1,23) = 254,45, p = 0.000). The planned contrasts further show the mothers to report significantly higher levels of problem behaviour than the independent observers in the case of known children but not in the case of unknown children. The acquaintanceship hypothesis is thus confirmed by the mothers in our study. The group-care workers, however, reported significantly higher levels of problem behaviour for both the known and unknown children than the independent observers, which suggests that the acquaintanceship hypothesis is only partially confirmed by the findings for the group-care workers.

Discussion

Drawing on theories of personality judgment, a research design was adopted to examine the role of acquaintanceship on perceptions of child behaviour. On the basis of the acquaintanceship hypothesis, we expected both mothers and group-care workers to report significantly higher levels of problem behaviours for known children than independent observers but similar levels of problem behaviours for unknown children. Mothers rating their own children’s behaviour were indeed found to perceive more behaviour problems than independent observers. In fact, the mothers in our study reported twice as many problem behaviours for their own children as the independent observers, which is in line with the findings of earlier research [20, 21]. We also found mothers to perceive similar levels of problem behaviours for unknown children when compared to independent observers for the same children. Taken together, these results indicate an acquaintanceship effect. However, the acquaintanceship hypothesis is only partially confirmed by the findings for the group-care workers. While these professionals reported more behaviour problems for the known children than the independent observers did, the group-care workers did not report different levels of problem behaviour for the known versus unknown children.

The differential pattern of observation reported for the mothers versus independent observers for the behaviour of known versus unknown children suggests that mothers are indeed more accurate observers of their own children’s behaviour than independent observers. Further evidence suggesting that mothers are generally quite accurate observers of their own child’s behaviour and not biased in their reporting with regard to such is provided by the fact that the group-care workers in our study also reported higher levels of problem behaviour than independent observers for the same sample of known children.

The group-care workers in our study reported the most behaviour problems for all of the children regardless of their acquaintanceship with the children. This suggests that factors other than acquaintanceship or in addition to acquaintanceship may have influenced their perceptions of child behaviour. It is certainly possible that explicit study and professional experience give group-care workers specific knowledge of child behaviour problems and thereby make them more qualified to detect specific behaviour problems than other informants. This capacity is referred to as expertise by Funder [8]. As part of their training and professional practice, moreover, group-care workers may be more skilled at the observation and analysis of videotaped behaviour than mothers or independent observers. Such professionals may be better prepared to detect and interpret minor behavioural cues in addition to obvious cues and report ambiguous or less conspicuous problem behaviours than other observers.

Professional expertise with regard to child behaviour problems may also explain why the group-care workers did not report different levels of behaviour problems for known versus control children. Given that all of the children—including the control children—were being treated in the same clinic, it is reasonable to assume that the level of problem behaviours for the unknown control children did not differ from the average level of problem behaviours for the known children. That is, the similarities in the professional ratings of the known and unknown children by the group-care workers are likely to reflect actual similarities in the levels of problem behaviour for the two groups of children. The professionals presumably drew upon their knowledge of child behaviour problems to evaluate the videotaped behaviour samples for both the known and unknown children. Mothers, in contrast, may only benefit from knowledge of their own children.

It is nevertheless possible that the similarities in the observations of the group-care workers for the known versus unknown children may still reflect some bias as this has been reported in the research literature. Studies of clinical judgments, for example, show a tendency on the part of professionals to adjust their ratings of client behaviour in accordance with information on the clinical status of the client or a phenomenon referred to as anchoring and adjustment [9]. Bias rather than expertise may thus explain the lack of difference in the professional reports of problem behaviours for the known versus unknown children and only additional research will help us unravel the contributions of the different factors to perceptions of child behaviour.

The present findings have some important implications for clinical practice and research on the accuracy of child behaviour reports. To start with, the present findings show mothers to estimate the amount of problem behaviour on the part of their own children more accurately than independent observers. In keeping with the acquaintanceship hypothesis, mothers have greater access to information regarding their children than other informants and are therefore better equipped to detect and interpret critical behavioural cues with regard to their children. This implies that mothers are accurate informants whose information should be used in diagnosis and treatment in clinical practice. In addition, expertise and not acquaintanceship appears to explain the generally greater amounts of problem behaviours reported by the group-care workers. This raises doubts about the use of independent observers who are often used as criterion informants, but who are unfamiliar with the problem behaviours of children in their evaluation. That is, the requirement that independent judges be experienced observers of child problem behaviours should be added to Richters’ [17] recommendation that carefully validated and independent ratings of the same child behaviours in identical settings be undertaken to attain accurate perceptions of child behaviour. In the meantime Richter’s [17] conclusion still holds true as he stated that “…maternal depression–perception associations should be viewed as an occasion for questions, not conclusions, about the accuracy of depressed mothers’ reports” (p. 497).

Yet another important finding is that the professionals in the present study observed more problem behaviours than mothers under controlled circumstances (i.e., when rating exactly the same videotaped behaviour samples). This finding is in contrast with the more common finding that mothers tend to report more behaviour problems than professionals using the same behavioural rating scales but observing the children in different settings [12, 22]. While such field differences are often ascribed to reporting biases on the part of mothers, our findings suggest that the differences between the mothers and the professionals may be due to actual differences in the incidence of problem behaviours depending on the particular situation (i.e., in the home vs. in the institution or school setting). This only strengthens our belief in the accuracy of maternal reports, but more research is needed to test both the acquaintanceship and expertise hypotheses under varying circumstances and in different settings. We suggest a research design in which the status of the client is manipulated via the use of videotaped behaviour samples from both referred and normal children, for example. The professional ratings of these children can then be compared to the ratings provided by their mothers and independent observers in order to gain greater insight into the accuracy of child behaviour assessment and the factors mediating this.