Interviewer effects in a survey examining pain intensity and pain interference in nursing home residents.

INTRODUCTION: Face-to-face surveys are applied frequently when conducting research in older populations. Interviewers play a decisive role in data quality, may affect measurement and influence results. This study uses survey data about pain in nursing home residents and analyses, whether affiliation-of-interviewer (internal vs. external to nursing home) and gender-of-interviewer affect residents' responses in terms of interviewer variance and systematically varying pain reports.
METHODS: Overall, 258 nursing home residents with up to moderate cognitive impairment were examined by 61 interviewers about pain intensity and interference applying the Brief Pain Inventory. Interviewer variance was measured using intra-interviewer correlation coefficients (ρ). Two-factorial covariance analysis was applied to analyse whether pain intensity and interference scores differ by interviewer characteristics.
RESULTS: Interviewer heterogeneity accounts for almost one quarter of total variance on average. Interviewer variance is higher for internal and male interviewers than for external and female interviewers. Covariance analyses show significant effects of interviewer characteristics on pain reports. Average pain intensity and interference scores vary considerably by interviewer gender and affiliation. Highest pain intensity was reported towards female internal and male external interviewers; highest pain interference was reported towards male external interviewers.
CONCLUSION: Residents' answers substantially differ in relation to who is assessing pain. There is a risk of imprecise and biased survey estimates on sensitive topics like pain in nursing homes. Interviewer gender and affiliation seem to evoke gender-specific and status-related expectations and attributions which influence residents' response process. Interviewer effects pose a considerable threat to survey data quality in institutionalised older populations.

Interviewer effects are of special concern for survey data quality in nursing home residents.

Interviewers have an influence on older respondents’ response processes.

Pain reports are at risk to be erroneous or biased due to interviewer effects.

Gender- and affiliation-of-interviewer increase interviewer variance and elicit systematically varying pain estimates.

Residents’ gender- and status-related expectations are assumed to influence pain assessment outcomes in pain management practice.

Introduction

Standardised face-to-face interviews are frequently applied in age research to assess attitudes, health states or quality of life of older people [1-3] and are often indispensable to measure health-related outcomes in geriatric settings [4, 5]. Interviewing older persons poses several methodological challenges in terms of recruitment or measurement [1, 6–9]. This becomes especially evident relating to nursing home residents (NHRs), where cognitive impairment (CI), physical limitations or limited audio-visual abilities are rather common [10-12]. Given these population-based constraints, the question–answer processes [13, 14] might be challenging especially for older respondents and introduce specific threats to survey data quality [1, 14, 15]. Here, interviewers play a crucial role, may influence respondents’ provided answers and are a decisive factor for the quality of the collected data [16, 17].

Interviewer effects refer to errors of measurement in terms of deviations between an observed and true value of the phenomena of interest and are attributed to certain interviewer characteristics (e.g. gender, age, behaviour) [17-20]. Two types of interviewer effects are distinguished [18, 21, 22]. Interviewer bias refers to a systematic under- or overestimation of the measures due to errors rooted in the survey design, and thus, leading to similarly biased measures across all interviewers. Interviewer variance refers to differences in the extent to which individual, or a set of, interviewers cause specific errors resulting in an increased variability of responses and reduced precision of estimates. The latter may also cause biased survey outcomes due to systematic differences between interviewers.

An extensive body of literature reports on various interviewer effects [19]. Although older respondents might be particularly susceptible to interviewer effects in health-related surveys, there is still rather limited empirical evidence about this population segment [2, 18, 23]. Early studies found more pronounced interviewer variance with increasing age of interviewer and in male interviewers [24]. Others suggested interviewer experience as a mediating factor for more valid data and higher reporting of psychological symptoms [25]. More recently, Winters et al. [23] found that NHR ratings varied by the interviewer’s previous experience or general knowledge of health care, while neither gender nor age had a significant effect. Beullens et al. [26] found increased interviewer variance in older respondents and that more interviewer assistance was necessary. Other studies observed that older persons reported significantly more health problems towards expert rather than lay interviewers [27] and that study outcomes would have been significantly underestimated if only lay interviewer data had been used [28].

Rationale And Objectives

Interviewer effects are of special concern in surveys with NHRs, especially when sensitive topics are examined, such as pain reports. Pain poses a significant burden in daily living and represents a sensitive topic with extensive subjective relevance [29-31]. A part of a parental study examining pain management in nursing homes (NHs), present research explores the influence of observed interviewer characteristics, i.e. affiliation-of-interviewer (AOI, NH internal vs. external) and gender-of-interviewer (GOI, male vs. female), on pain outcomes. AOI and GOI might elicit role- or status-related perceptions and induce socially desirable responses, i.e. respondents match their responses to what they anticipate the interviewers want to hear [32], especially with regard to sensitive questions relating to cultural norms or expectations [14, 17, 19, 32]. For instance, familiarity between nurses and residents may enhance social desirability. NHRs might attribute internal nurses an ‘expert’-status while stereotyping external interviewers as ‘lays’, eventually affecting reports of health-related information [27, 28]. Gender stereotypes are widely present in Western societies [33] and presumably more pronounced in older populations. Health care institutions, such as NHs, tend to be shaped by hegemonic masculinity that affects interactions, gender constructions and role-related expectations [34, 35]. Such expectations and stereotypes form pain perception and communication, and thereby, might structure the respondents’ editing of pain-related answers [35].

The objectives of this observational study were to examine interviewer effects on the reporting of pain intensity and interference based on self-reports of NHRs and to analyse whether AOI and GOI affect NHR responses in terms of (i) interviewer variance and (ii) biased pain outcomes.

Methods

Survey data come from a registered cluster-randomised controlled trial (c-RCT; UTN: U111-1187-3174) in 15 German NHs, where effects of pain management nursing education interventions on NHR pain situation were evaluated in a sample of 879 NHRs with no, mild, moderate and severe CI. Details and results from the c-RCT have been published elsewhere [36].

Participants

Eligible participants had to be 60 years and older, living in the participating NH, having sufficient German language abilities and providing informed consent. To estimate the ability to self-report in the survey situation, NHRs were initially screened using the Mini-Mental State Examination (MMSE) [37]. NHRs with up to moderate CI (MMSE score 10–30) were interviewed applying standardised interviews (n = 598). We included only those residents who self-reported presence of pain, since their pain intensity and interference were assessed (n = 258) (Supplementary Table S1).

Data collection

Interviewers were health care professionals (e.g. social work, psychiatric nursing), students of nursing or health sciences (i.e. external to the NH) and nurses from the participating NH (i.e. internal to the NH). All interviewers received the same full-day comprehensive interviewer training (e.g. theory, exercises regarding interviewing older persons, procedures, interviewer behaviour) and were supervised during data collection. Interviewers were pragmatically assigned to NHR and study location—i.e. external interviewers were deployed in several NH, and internal nurses in their respective NH. Standardised, tablet-assisted, face-to-face interviewing was applied. Survey questions and answer categories were read aloud and additionally presented as paper versions.

Variables, instruments and outcomes

Residents’ gender, age and documented dementia diagnosis were extracted from the electronic care record system. While ‘dementia diagnosis’ provides general information if any kind of dementia is present, MMSE screening was applied by interviewers to assess current cognitive function, resulting in scores between 0 and 30 points, whereby higher scores represent less cognitive decline [37]. Pain intensity and interference were measured by the validated German version of the Brief Pain Inventory (BPI) [38]. Building on the initial question if any pain was present during the previous 24 hours, BPI measures pain intensity (four items assessing maximum, least, average and actual pain; 0—no pain to 10—worst pain) and interference (seven items assessing the impact of pain on general activities, mood, walking ability, resilience, relations with others, sleep and enjoyment of life; 0—does not interfere to 10—completely interferes) along two scales. BPI items were analysed item by item as well as by computing pain intensity and interference scales representing respondent’s mean scores across the original items. Interviewer affiliation and gender were protocolled in the survey tablet system.

Statistical analysis

All analyses were conducted using IBM SPSS 27. Absolute numbers (n) and percentages (%) of non-missing values were reported for categorical variables; arithmetic means (AM), standard deviation (SD), median (MD) and minimum and maximum (Min, Max) were used to describe metric variables.

Interviewer variance was measured by intra-interviewer correlation coefficients (IIC or ρΙNΤ) drawing on Kish’s analysis-of-variance (ANOVA) framework [39, 40]. In general, ρ expresses the ratio of between-interviewer variance to the total variance in the variable of interest [17, 22]. The larger this coefficient, the more similar are the responses per interviewer, and hence, the more response behaviour can be attributed to differences between interviewers. Interviewer variance cannot be prevented, but most ρ-coefficients rarely exceed 0.05 [41]. To reach sufficient variability of responses, ρINT were calculated for interviewers who conducted at least three interviews (i.e. interviewer workload ≥3). ρ-Coefficients derived from fixed-effects ANOVA are displayed for all BPI items and are summarised in boxplots. Results are presented for total sample and separately for interviewer’s subsamples.

To analyse whether mean scores of BPI scales differ by AOI and GOI, two-factorial covariance analyses (ANCOVA) for each pain outcome were conducted. AOI and GOI were included as main effects as well as interaction terms (i.e. affiliation × gender) and adjusted by NHR’s MMSE, gender, age and pre- vs. post-test (i.e. before vs. after parental study’s intervention). Eta2 (η2) was calculated as a measure of association and type 1 error was set at α = 0.05 (two-sided). Profile plots are used to visualise the estimated, adjusted marginal means.

Results

Interviewer and resident characteristics

Table 1 presents both the interviewer- and resident-level characteristics for the total sample as well as for the subsample data collected by all interviewers with an interviewer workload of n ≥ 3. In total, pain intensity and interference of 258 NHRs were assessed by 61 interviewers. Three quarters were female and one-third was nurses from the respective NH. The average interviewer workload was 4.3 (SD = 2.8). Two-thirds of interviewed NHRs were female, and documented dementia was present in about 20% of NHRs. On average, NHRs were 81.3 (SD = 9.8) years of age and exhibited an MMSE score of 22.6 points (SD = 5.1).

Table 1

Interviewer and nursing home resident characteristics

Sample characteristics	Total sample	Subsample (≥3 int )
Interviewer level	BPI, n = 61	BPI (≥3 int ), n = 36
Gender
Female	45 (73.8%)	29 (80.5%)
Male	16 (26.2%)	7 (19.5%)
Affiliation
External	42 (68.9%)	25 (69.4%)
Internal	19 (31.1%)	11 (30.6%)
Interviewer workload (numberint)
AM (SD)	4.23 (2.8)	6.03 (2.7)
MD, min-max	3.5, 1–13	5.5, 3–13
Resident level	BPI, n = 258	BPI (≥3 int ), n = 217
Gender
Female	166 (64.3%)	151 (69.6%)
Male	92 (35.7%)	66 (30.4%)
MMSE (score)
AM (SD)	22.65 (5.1)	21.72 (5.3)
MD, min-max	23.0, 10–30	22.0, 10–30
Age (years)
AM (SD)	81.29 (9.8)	82.00 (9.5)
MD, min-max	83.0, 60–101	84.0, 60–101
Documented dementia
Yes	59 (22.9%)	56 (25.8%)
No	185 (71.7%)	149 (68.7%)

Notes: Total sample depicts characteristics based on total resident data, Subsample depicts characteristics based on resident data collected by interviewers with a workload of at least three interviews; n, number (% percentage of non-missing values); BPI, Brief Pain Inventory; MMSE, Mini-Mental State Examination; int, number of interviews; AM, arithmetic mean; SD, standard deviation; MD, median; Min, minimum; Max, maximum.

An interviewer workload n ≥ 3 was conducted by 36 interviewers, resulting in a sample of 217 NHRs. The average number of interviews was 6.0 per interviewer (SD = 2.7). Besides that, interviewer- and resident-level characteristics were basically similar.

Subsample sizes by external, internal, female and male interviewers varied between 7 and 44 interviewers and 57 and 198 examined NHRs, respectively. While most sample characteristics proved to be comparable, NHR’s gender and documented dementia proportions differed between the subsamples (Supplementary Table S2).

Interviewer effects in terms of interviewer variance in BPI items

Interviewer variance (ρINT) was computed for the NHR’s data collected by interviewers with an interviewer workload of n ≥ 3. Details for all single items of BPI scales for the total sample as well as for AOI and GOI subsamples are displayed in Supplementary Table S3. In the total sample, the proportions of variance explained by differences between interviewers ranged between 17.6 and 30.5% for the 11 BPI items, exhibiting a mean ρINT of 0.242 (MD = 0.238). Figure 1 depicts this distribution while also presenting boxplots for the AOI and GOI subsamples.

Figure 1

Boxplots for BPI items’ interviewer variance (ρINT).

Interviewer variance varied between external and internal as well as female and male interviewers but was especially pronounced in terms of AOI (Figure 1, Supplementary Table S3). All but one single items’ ρINT were clearly lower within NHRs interviewed by external interviewers (AM = 0.190, MD = 0.196) as compared to NHRs interviewed by internal (AM = 0.288, MD = 0.322) interviewers.

GOI caused less pronounced differences in BPI items’ ρINT coefficients. While the variation of NHR responses due to interviewer heterogeneity was explained by an average 26.5% (MD = 26.8%) within male interviewers, this amounted to an average of 22.2% (MD = 21.2%) for female interviewers. Relating to comparisons of ρINT between the single BPI items, almost all coefficients were higher in the NHR subsamples interviewed by male interviewers.

Interviewer effects in terms of biased BPI intensity and interference mean scores

Two-factorial ANCOVA (Table 2a) shows a significant disordinal interaction between interviewer gender and affiliation on NHR average pain intensity (η2 = 5.3%, F = 12.2, P < 0.001, n = 227). GOI and AOI exhibit a joint and counteracting effect on pain intensity score: NHRs reported the highest average pain intensity when being interviewed by external male (AM = 5.33) or internal female (AM = 5.28) interviewers. The lowest pain intensities were stated when NHRs were interviewed by external women (AM = 4.24) or internal men (AM = 4.29).

Table 2

ANCOVA for BPI scores by affiliation- and gender-of-interviewer

ANCOVA BPI scores by interviewer gender and affiliation
(a) BPI pain intensity score (0–10)		(b) BPI pain interference score (0–10)
INT gender		INT gender
Female AM (SE), n	4.76 (0.2), 173	Female AM (SE), n	4.23 (0.2), 189
Male AM (SE), n	4.81 (0.3), 54	Male AM (SE), n	4.19 (0.3), 56
INT affiliation		INT affiliation
External AM (SE), n	4.78 (0.2), 159	External AM (SE), n	4.61 (0.2), 176
Internal AM (SE), n	4.77 (0.2), 68	Internal AM (SE), n	3.81 (0.3), 69
Gender × affiliation*		Gender × affiliation*
INT (f) × (ex) AM (SE), n	4.24 (0.2), 130	INT (f) × (ex) AM (SE), n	4.22 (0.2), 144
INT (f) × (in) AM (SE), n	5.28 (0.3), 43	INT (f) × (in) AM (SE), n	4.42 (0.4), 45
INT (m) × (ex) AM (SE), n	5.33 (0.3), 29	INT (m) × (ex) AM (SE), n	5.00 (0.4), 32
INT (m) × (in) AM (SE), n	4.29 (0.4), 25	INT (m) × (in) AM (SE), n	3.38 (0.5), 56
Between subject effects		Between subject effects
Factors eta2 (F), P		Factors eta2 (F), P
INT gender	0.0% (0.03), P = 0.862	INT gender	0.0% (0.01), P = 0.914
INT affiliation	0.0% (0.01), P = 0.991	INT affiliation	1.84% (4.51), P = 0.036
INT gender × affiliation	5.26% (12.2), P < 0.001	INT gender x affiliation	2.01% (4.88), P = 0.028
Covariates eta2 (F), P		Covariates eta2 (F), P
NHR gender	1.32% (2.94), P = 0.088	NHR gender	4.10% (10.15), P = 0.002
NHR age	0.35% (0.77), P = 0.381	NHR age	1.19% (2.84), P = 0.093
NHR MMSE score	0.89% (1.97), P = 0.162	NHR MMSE score	0.86% (2.06), P = 0.152
Sample pre/post	2.77% (6.24), P = 0.013	Sample pre/post	1.59% (0.38), P = 0.540
R 2 (R 2 corr ), n	9.9% (7.0%), 227	R 2 (R 2 corr ), n	6.7% (4.4%), 245

Notes: ANCOVA, analysis of covariance; BPI, Brief Pain Inventory; INT, interviewer; NHR, nursing home resident; MMSE, Mini-Mental State Examination; f, female; m, male; ex, external; in, internal; AM, arithmetic marginal mean; SE, standard error; n, sample size; eta2, partial eta-square; F, F-value; P, error probability; check of ANCOVA assumptions demonstrated no relevant model violations; * partially small subgroup sample size.

A combined effect of GOI and AOI on pain interference was found and indicates a weak but significant, disordinal interaction effect (η2 = 2.2%, F = 4.9, P = 0.028, n = 245) (Table 2b). While the mean pain interference score was almost identical for external (AM = 4.22) and internal (AM = 4.42) female interviewers, marginal means differed between external (AM = 5.00) and internal male interviewers (AM = 3.38).

Details for pairwise comparisons of simple main effects are provided in Supplementary Table S4. Figure 2 depicts the estimated, adjusted marginal means by interviewer affiliation and gender for BPI pain intensity (Figure 2a) and BPI pain interference (Figure 2b).

Figure 2

Profile plots for BPI scores by affiliation- and gender-of-interviewer.

Discussion

This study analysed whether affiliation- and gender-of-interviewer influence NHR responses in reporting pain intensity and interference. It was found that a considerable amount of response variability is due to interviewers’ affiliation and gender, and that pain reports vary systematically by joint effects of AOI and GOI. Our findings give rise to potential risks of imprecise and biased survey estimates in NH populations.

Overall, the share of variance explained by interviewer heterogeneity amounts to an average of 24.2%, which represents a comparatively high proportion in general terms [23, 25, 40, 42]. Interviewer variance was especially pronounced in NHR responses interviewed by NH-internal interviewers. Observed interviewer effects might be explained by specific role- and status-related interpretations [28, 33, 35] on the part of NHR, that is, residents attribute certain expectations and meanings to the interviewer. NHRs are dependent on caregivers for their care and may fear negative repercussions if they express undesirable information about perceived care provision [43, 44]. NHR responses might reflect such attributed expectations towards familiar, internal nurses leading to more heterogeneous responses. Furthermore, compared to external interviewers (‘lays’), NHRs might have attributed internal nurses with a higher professional competence (‘experts’) [27, 28, 45], and provided more differentiated but person-specific, hence, varying pain reports. Finally, higher response variability in internal interviewers may suggest that they deviate more often from the predefined standardised interview script [46]. This is plausible insofar as interviews represent a communication opportunity for older persons [9]. The risk of digressing into the usual everyday communication is higher in the case of familiar nurses. Also, nurses usually know ‘their’ residents relatively well, may have individually adapted forms of communication per NHR, and thus, inducing higher variability of responses.

Our findings also show that pain scores differ according to who conducts the interviews. Pain intensity was highest when internal women or external men interviewed and lowest with external women and internal men. Highest pain interference was reported towards external men and lowest towards internal men but did not vary within female interviewers. Again, gender- and status-related expectations and stereotypes seem to elicit socially desirable and role-dependent responses. Our results are consistent with other findings that demonstrate gender-specific communication with physicians and nurses [35], biased observer pain estimations due to gender role stereotypes [47], and varying pain levels by experimenter gender [48]. Individuals who consider themselves to be more masculine show higher pain thresholds and tolerances when providing pain information [49]. At the same time, response patterns of health-related information differ according to whether interviewers are attributed a certain status and topic-related competencies [27, 28, 45]. Hence, male internal nurses may collect lower pain scores as NHRs anticipate higher pain reports to be interpreted as dissatisfaction with received care. External male interviewers may be attributed as representatives of the official research institution and NHRs may be more likely to report high pain scores hoping to initiate improvements in care provision. For female interviewers, these patterns seem to be exactly opposite. Internal female interviewers seem to represent trustworthy and empathetic contact persons within the NH structure, all the while incorporating rather male positions of power, such as physicians or facility managers [34, 35]. External female interviewers may be seen as less trustworthy with low expert status and less power to effect any changes in care provision against the background of gender-induced role expectations.

Our study faces several limitations. This study’s procedures were determined by the overarching project’s study design. Interviewers were chosen pragmatically, do not represent a random sample from a population of interviewers and were not randomly assigned to respondents to avoid confounding of area effects, interviewer and respondent characteristics [17, 18]. For fieldwork feasibility, it was not possible to conduct such an optimal interpenetrated survey design. Some interviewer subsample numbers were low and imbalanced in terms of gender and affiliation. Multi-level analyses to account for clustering of respondents by interviewers including random effects were not feasible for several items due to small sample sizes. This also limits further analyses in ANCOVA models to examine possible moderating factors, such as adding three-way interaction terms using NHR gender or testing gender-related (mis)matches, i.e. if female or male residents are interviewed by female or male interviewers [50]. Besides, age differences between interviewers and respondents might influence responses in interviews [2]. Interviewer age and characteristics like experience, skills, personality traits or attitudes towards pain or gender roles were not assessed but exhibit further explanatory potential to shed more light on the ‘black box’ [26] of the interview process. Although our findings do not suggest that the parental study’s nursing education interventions affected our internal interviewers in applying the standardised survey, we cannot rule out that these might have introduced an additional difference between internal and external interviewers. While we do not expect our results to be biased because we used the MMSE as the current estimate of NHR cognitive function, the proportion of NHR with documented dementia varied between interviewer subsamples, possibly indicating differing interviewer strategies in selecting to-be-interviewed residents. Future studies should also focus on possible interactions of participant selection procedures with interviewer and respondent characteristics.

Given the paucity of methodological research in this specific field, we believe that our findings are of high relevance for both research and practice. While interviewer variance increases standard errors and decreases the precision of survey estimates, reported pain outcomes substantially differ by interactions between GOI and AOI. Anticipating the risk of imprecise and even biased outcomes, interviewer effects should be considered when designing, analysing and interpreting surveys with older persons. Our results also give rise to important implications for pain management in NHs. Medical and nursing decisions rely on regular pain assessment. Depending on the assessor, pain may be over- or underestimated and care provision is unlikely to build upon a resident’s ‘true’ perceived health state. Hence, future studies should also focus on the risk of inadequate care of NHR with pain as an unwitting but maybe far-reaching consequence from potentially biased pain assessment results.

Conclusion

NHR responses are influenced by interviewer characteristics. Self-reports on pain vary according to GOI and AOI. Interviewer characteristics are assumed to evoke gender- and authority-related attributions and expectations and substantially affect study outcomes of self-reported pain intensity and interference. The risk of interviewer effects in face-to-face interviews with older respondents seems to be high and threatens survey data quality in NHRs.

Click here for additional data file.