Interviewer BMI effects on under- and over-reporting of restrained eating: evidence from a national Dutch face-to-face survey and a postal follow-up.

OBJECTIVES: To determine the effect of interviewer BMI on self-reported restrained eating in a face-to-face survey and to examine under- and over-reporting using the face-to face study and a postal follow-up.
METHODS: A sample of 1,212 Dutch adults was assigned to 98 interviewers with different BMI who administered an eating questionnaire. To further evaluate misreporting a mail follow-up was conducted among 504 participants. Data were analyzed using two-level hierarchical models.
RESULTS: Interviewer BMI had a positive effect on restrained eating. Normal weight and pre-obese interviewers obtained valid responses, underweight interviewers stimulated under-reporting whereas obese interviewers triggered over-reporting.
CONCLUSION: In face-to-face interviews self-reported dietary restraint is distorted by interviewer BMI. This result has implications for public health surveys, the more so given the expanding obesity epidemic.

Introduction

Eating and dieting behaviors are of major interest to public health research, given the global epidemic of obesity and diabetes (Caballero 2007). In both clinical settings and public health surveys these behaviors are assessed with well-established questionnaires. One such instrument, which is used in the current study, is the Dutch Eating Behavior Questionnaire for assessment of restrained eating (DEBQ-R) developed by Van Strien et al. (1986). The inventory includes questions designed to determine how much an individual thinks about and intends to restrict food intake, termed dietary restraint. Dietary restraint is the conscious attempt to limit caloric intake to regulate body weight. Individuals scoring high on the DEBQ-R are aware of the amount of food they consume and conscious and concerned about what they eat and when.

It is well known that in face-to-face modes of questionnaire administration, readily visible interviewer traits and other personal characteristics of the interviewer may affect responses to survey items and thereby introduce bias in the estimates (Davis et al. 2010). These interviewer effects are particularly operant when respondents are queried about sensitive topics such as substance use (Heeb and Gmel 2001; Dotinga et al. 2005; Lord et al. 2005) and sexual behavior (Chun et al. 2011) and abuse (Dailey and Claus 2001). Surprisingly, it is rare to find studies of interviewer effects on self-reports of eating and eating-related issues such as energy intake and weight control (McKenzie et al. 2002; Sperry et al. 2005). Also, while there is some evidence that the presence or absence of an interviewer influences self-reported body weight in men (Kroh 2005), it is not known whether the interviewer relative body weight, as measured by the Quetelet body mass index (BMI), affects interviewee’s responses and stimulates them to misreport their eating and dieting behaviors.

The present study was undertaken to examine if interviewer BMI is associated with under- or over-reporting in self-reports of restrained eating behaviors, as measured by the DEBQ-R. In order to evaluate potential misreporting we examined data from a national Dutch face-to-face survey. To further investigate this issue we also investigated within-interviewee changes in responses to the interviewer-administered questionnaire versus a self-administered postal follow-up.

Methods

Data were taken from a national Dutch survey conducted by the Radboud University Nijmegen, for which the fieldwork was completed in 2006. Using a random sampling procedure designed to represent the adult population aged 18–70 years inclusive, the cross-sectional study selected 2,176 eligible participants from the target population, 1,212 of whom were interviewed in person, giving a response rate of 56%. For the Netherlands, this is a rather successful figure, not easily exceeded by a national survey with a similar design. A pool of 98 professional interviewers compiled the interviews for the face-to-face study. The 60-minute interviews were conducted at the participants’ homes using computer-assisted personal interviewing.

Following the face-to-face survey, a mail confirmation follow-up study was conducted to evaluate fieldwork procedures and to ensure quality data collection. The 15-minute postal follow-up repeated the face-to-face measures used in the current study and was completed by 504 participants, i.e., 41.6% of the baseline responders.

Restrained eating behavior

To obtain a measure for restrained eating, participants were administered the 10-item, five-point response DEBQ-R scale by Van Strien et al. (1986). The DEBQ-R measures intentions to eat less, maintain or lose weight, and assesses a participant’s degree of involvement in weight control by skipping meals and fasting. The DEBQ-R items were included in the face-to-face interview as a self-completion instrument, with the laptop computer handed over to the participant so that only he or she could see the survey questions and key the responses. For the current analysis a composite (Likert) score was calculated as the unweighted sum of the 10 five-point item scores with a potential range of 10–50. High scores imply more restraint in eating.

BMI and education

Both participants and interviewers were asked their body weight in kilograms and height in centimeters to permit the calculation of their BMI as the ratio of weight in kilograms to height in meters squared. Completed education was obtained using the International Standard Classification of education (ISCED 1997) by UNESCO-UIS (2006).

Statistical analysis

To examine the effect of interviewer BMI on individual DEBQ-R responses, two-level hierarchical models were applied to the face-to-face survey data, with participants nested within interviewers (Snijders and Bosker 1999). The models were estimated using the linear mixed-effects model (MIXED) procedure in SPSS 17 (SPSS Inc., IL, USA). A two-tailed P < 0.05 was used to define a significant association. The estimated intra-class correlation (IC) was calculated as 0.04. While the IC coefficient was small, it nonetheless indicated that the pool of interviewers is a significant source of variance that should be modeled to control for potential biases associated with the nested nature of the data.

Results

Approximately half of the survey participants (53%) were female and the mean age of the sample was 48.1 years (SD = 13.6). Education varied from primary or lower secondary education (37.3%), upper secondary or post-secondary non-tertiary education (32.8%), to first or second stage of tertiary qualification (29.9%). The participant BMI values ranged from 17.0 to 40.4, with a mean of 25.2 kg/m2 (SD = 4.0), and interviewer BMI ranged from 18.1 to 39.6, with a mean of 25.3 kg/m2 (SD = 3.9). Cronbach’s alpha for the DEBQ-R items was estimated to be 0.93 in the face-to-face survey and 0.94 in the postal survey.

Interviewer BMI

A two-level regression model was estimated that entered the participant-level variables female gender, age, education, and BMI, and the interviewer-level variable interviewer BMI as fixed effects. The results are reported in the left-most column of Table 1.

Table 1

Unstandardized regression effects on restrained eating (DEBQ-R) scores in face-to-face survey and postal follow-up and on restrained eating (∆DEBQ-R) difference scores (Netherlands 2006)

Outcomea	Face-to-face survey										Postal follow-up		Difference scores
	DEBQ-R		DEBQ-R underweight, normal weight, interviewers		DEBQ-R normal weight pre-obese interviewers		DEBQ-R pre-obese, obese interviewers		DEBQ-R		DEBQ-R*		∆DEBQ-R
	b	SE	b	SE	b	SE	b	SE	b	SE	b	SE	b	SE
Intercept	−12.67	2.83**	−19.57	5.74**	−9.29	4.19*	−17.08	5.21**	−11.86	4.55**	−4.41	4.02	−6.58	3.77
Female	3.65	0.51**	3.57	0.63**	3.76	0.55**	3.66	0.89**	3.02	0.82**	3.64	0.72**	−0.65	0.66
Age	0.07	0.02**	0.08	0.02**	0.07	0.02**	0.05	0.03	0.09	0.03**	0.12	0.03**	−0.03	0.03
Education	2.77	0.32**	2.93	0.39**	2.78	0.34**	2.41	0.58**	1.79	0.51**	1.68	0.45**	0.10	0.41
BMI	0.83	0.07**	0.76	0.08**	0.82	0.07**	1.00	0.12**	0.85	0.11**	0.83	0.10**	0.01	0.09
BMI interviewer	0.19	0.08*	0.53	0.22*	0.08	0.15	0.25	0.12*	0.22	0.11*	−0.03	0.09	0.22	0.09*
Intercept variance	0.92	1.07	1.02	1.25	0.80	1.11	0.00	0.00	0.00	0.00	0.00	0.00	1.07	1.43
Residual variance	77.18	3.25**	79.82	4.01**	77.37	3.48**	70.05	5.16**	81.95	5.16**	63.72	4.01**	51.75	3.49**
Intra-class correlationb	0.04		0.05		0.04		0.00		0.00				0.04
No. interviewees	1,212		844		1,061		368		504		504		504
No. interviewers	98		70		88		28		89				89
–2ℓℓ	8,720		6,101		7,635		2,608		3,651		3,524		3,430

* p < 0.05; ** p < 0.01

aDEBQ-R refers to the outcome of the face-to-face survey for all (N = 1,212) participants (column 1), for participants interviewed by underweight or normal weight interviewers (column 2), normal weight or pre-obese interviewers (column 3), and pre-obese or obese interviewers (column 4), and for the subgroup (n = 504) that participated in the postal follow-up (column 5). DEBQ-R* (column 6) refers to the outcome of the postal survey and ∆DEBQ-R (column 7) represents the difference score between the two survey modes, i.e., DEBQ-R face-to-face minus DEBQ-R postal score

bIntra-class correlation for null model without covariates

The table reveals that the participant characteristics all had a significant positive effect on the DEBQ-R restrained eating scores. Most important to our study is the finding that adjusted for the participant variables, interviewer BMI was positively associated with variations in DEBQ-R in the face-to-face survey. This implies that participants were more likely to report restrained eating behaviors to obese interviewers than to underweight and normal weight interviewers.

It is hard to determine from this result who of the interviewers—underweight or obese—elicited more valid responses to the DEBQ-R, as there is no gold standard available to validate them. To evaluate this issue in the best possible way, we opted for a twofold analysis approach. First, the effect of interviewer BMI was examined for three partially overlapping subgroups, that is, participants interviewed by (1) underweight or normal weight interviewers; (2) normal or pre-obese interviewers; and (iii) pre-obese or obese interviewers, using the WHO Expert Consultation (2004) BMI cut-off points. These cut-offs (in kg/m2) are: <18.5 (underweight), 18.5 to <25 (normal weight), 25 to <30 (pre-obese), 30 to <35 (obese class I), 35 to <40 (obese class II), ≥40 (obese class III). The results presented in the second to fourth column of Table 1 reveal that interviewer BMI had a positive effect among participants interviewed by underweight or normal weight interviewers and among those questioned by pre-obese or obese interviewers. Among participants interviewed by normal or pre-obese interviewers, however, the effect turned out to be near zero. This finding suggests that underweight interviewers stimulated under-reporting and that obese interviewers induced over-reporting of dietary restraint.

A second approach was to examine the responses to the postal questionnaire and, in particular, intra-individual differences between the face-to-face and the postal survey responses. The postal follow-up was completed by part of the respondents of the cross-sectional study. Although there were no indications that the postal completers were a biased selection of the baseline responders, we re-ran the two-level regression model for the 504 participants who completed the follow-up. The results, reported in the fifth column of Table 1, indicate that the parameter estimates were largely equivalent to the baseline results. Most notable is that the positive effect of interviewer BMI remained statistically significant. We subsequently analyzed the responses to the DEBQ-R in the postal questionnaire benchmark, to confirm the absence of an interviewer effect in this non-interviewer-administered survey mode. To do so, the postal participants were nested within their former face-to-face interviewers. As can be seen in the sixth column of Table 1, the self-administered postal responses were indeed unaffected by interviewer BMI. To investigate within-participant changes the DEBQ-R difference scores were obtained by subtracting the postal survey scores from the face-to-face interview responses. The estimates, displayed in the right-most column of Table 1, indicate that interviewer BMI is the only characteristic that had a positive effect on the ∆DEBQ-R difference scores. This finding again suggests that answers to questions about dietary restraint are related to the interviewer in that underweight interviewers coaxed lower scores on the DEBQ-R in the face-to-face survey than obese interviewers.

The amount of under- and over-reporting predicted by the regression models is graphically represented in Fig. 1. The effect of interviewer BMI on misreporting of the DEBQ-R in the face-to-face survey (N = 1,212) is displayed by the solid line and scaled on the left-hand side axis. The effect of interviewer BMI on the DEBQ-R difference scores (n = 504) is displayed by the dashed line and scaled on the right-hand side axis of ordinates.

Fig. 1

Individual and mean predicted under-reporting (−) and over-reporting (+) of restrained eating (DEBQ-R) in face-to-face survey by interviewer body mass index (BMI) class, according to (N = 1,212) face-to-face mode scores (left-hand side axis, solid line) and (n = 504) face-to-face versus postal survey difference scores (right-hand side axis, dashed line) (Netherlands 2006)

The figure shows that the DEBQ-R scores and the DEBQ-R difference scores yield about the same result as to under- and over-reporting. They both indicate that normal and pre-obese interviewers obtained valid responses to the eating questionnaire in the face-to-face survey, that underweight interviewers stimulated a one-point under-reporting and that obese interviewers triggered a two-point over-reporting in the personal interview. The difference between participants interviewed by underweight and those questioned by obese class II interviewers amounts to approximately three points (i.e., 7.5%) on the DEBQ-R scale.

We additionally performed several sensitivity analyses and supplementary tests. In brief, the results with respect to misreporting obtained by the change score method used here are near equivalent to those obtained using the regression variable method, where the DEBQ-R postal score is regressed on the participant-level variables and interviewer BMI, while controlling for the DEBQ-R face-to-face score (Allison 1990). Also, the models presented in Table 1 include interviewer BMI as a metric variable. Similar results are obtained if it is included as a non-metric grouping variable using the WHO BMI cut-offs. We also examined potential cross-level interactions of the participation characteristics and interviewer BMI to see if interviewer BMI is more influential in some participants. The likelihood ratio tests indicated that the interactions between gender, age and education on the one hand and interviewer BMI on the other are not statistically significant and may be omitted from the regression model without a significant decrease in model fit.

Discussion

There is wide-spread concern about the mis-recording of self-reports of energy intake in health and dietary surveys (Johansson et al. 2001; McKenzie et al. 2002). Such a phenomenon may affect conclusions drawn about the intakes of food or nutrients by populations or subgroups, and about the relationship between such intakes and obesity and diet-related diseases. This study focused on the role of the survey interviewer in misreporting dietary restraint and, in particular, on a virtually neglected issue in interviewer effect research namely the potential response bias introduced by interviewer BMI.

Our results show that self-reports of eating and dieting are susceptible to being influence by the body physique of the interviewer. Participants interviewed by obese interviewers scored about 7.5% higher on our dietary restraint scale than similar participants questioned by underweight interviewers. The findings also indicated that face-to-face surveys administered by normal weight and pre-obese interviewers produce valid responses, that underweight interviewers stimulate under-reporting whereas obese interviewers induce over-reporting of dietary restraint.

The strength of the present study includes the comprehensive nature of our survey in terms of both participant and interviewer counts. We were thereby able to employ a multilevel design to statistically test the effect of interviewer BMI, which many previous studies on interviewer trait effects were unable to address adequately as their interviewer staff was simply too small in number. However, a limitation of the data for this study is that participants and interviewers were not randomly assigned to each other. Interviewer assignment was made on the basis of geographic convenience to cut down time and travel costs of the survey interviews. A rigorous evaluation of interviewer effects would require an interpenetration design where participants are assigned randomly to interviewers with different characteristics (Groves 1989). Due to fieldwork costs considerations such controlled experiments are very rarely employed in nationwide face-to-face surveys, however.

Although we can only speculate about participant’s underlying motives, a plausible explanation for over-reporting is that everybody runs the risk of becoming overweight, that obese interviewers are a potent reminder of that risk (Mann and Ward 2004), and that this dietary reminder aids in the attainment of artificial high levels of restrained eating. Also, it is generally assumed in our culture that people become overweight because they lack self-control around food (De Jong and Kleck 1986). By reporting that they eat minimally, people can try to dispel the impression that they lack self-control, and thereby present themselves to the interviewer in a favorable way. It may be that the desire to present oneself as a minimal eater is strong only in the presence of others who themselves are overweight. In the same vein, there may be a lack of social inhibition of eating in the companionship of underweight interviewers.

These impression management strategies, intended to convey a desired impression to others and perhaps to enhance one’s self-esteem, are a topic worth exploring (Paulus 1984). Meanwhile, according to our results an appropriate advice to researchers involved in health surveys would be to recruit interviewers of normal or pre-obese weight when administrating eating and eating-related questionnaires. If that is easy in theory but impossible in practice, one may collect data on interviewer height and weight and examine the survey data for variations in responses by interviewer BMI. If warranted, the effect should be controlled to avoid bias in the coefficients of interest and to increase statistical power.