Ai Rene Ong1, Mengyao Hu1, Brady T West1, John A Kirlin2. 1. 1Institute for Social Research,University of Michigan,426 Thompson Street,Ann Arbor,MI 48104,USA. 2. 2US Department of Agriculture,Economic Research Service,Washington,DC,USA.
Abstract
OBJECTIVE: To understand the effects of interviewers on the responses they collect for measures of food security, income and selected survey quality measures (i.e. discrepancy between reported Supplemental Nutrition Assistance Program (SNAP) status and administrative data, length of time between initial and final interview, and missing income data) in the US Department of Agriculture's National Household Food Acquisition and Purchase Survey (FoodAPS). DESIGN: Using data from FoodAPS, multilevel models with random interviewer effects were fitted to estimate the variance in each outcome measure arising from effects of the interviewers. Covariates describing each household's socio-economic status, demographics and experience in taking the survey, and interviewer-level experience were included as fixed effects. The variance components in the outcomes due to interviewers were estimated. Outlier interviewers were profiled. SETTING: Non-institutionalized households in the continental USA (April 2012-January 2013). SUBJECTS: Individuals (n 14 317) in 4826 households who responded to FoodAPS. RESULTS: There was a substantial amount of variability in the distributions of the outcomes examined (i.e. time between initial and final interview, reported values for food security, individual income, missing income) among the FoodAPS interviewers, even after accounting for the fixed effects of the household- and interviewer-level covariates and removing extreme outlier interviewers. CONCLUSIONS: Interviewers may introduce error in food acquisition survey data when they are asked to interact with the respondents. Managers of future surveys with similarly complex data collection procedures could consider using multilevel models to adaptively identify and retrain interviewers who have extreme effects on data collection outcomes.
OBJECTIVE: To understand the effects of interviewers on the responses they collect for measures of food security, income and selected survey quality measures (i.e. discrepancy between reported Supplemental Nutrition Assistance Program (SNAP) status and administrative data, length of time between initial and final interview, and missing income data) in the US Department of Agriculture's National Household Food Acquisition and Purchase Survey (FoodAPS). DESIGN: Using data from FoodAPS, multilevel models with random interviewer effects were fitted to estimate the variance in each outcome measure arising from effects of the interviewers. Covariates describing each household's socio-economic status, demographics and experience in taking the survey, and interviewer-level experience were included as fixed effects. The variance components in the outcomes due to interviewers were estimated. Outlier interviewers were profiled. SETTING: Non-institutionalized households in the continental USA (April 2012-January 2013). SUBJECTS: Individuals (n 14 317) in 4826 households who responded to FoodAPS. RESULTS: There was a substantial amount of variability in the distributions of the outcomes examined (i.e. time between initial and final interview, reported values for food security, individual income, missing income) among the FoodAPS interviewers, even after accounting for the fixed effects of the household- and interviewer-level covariates and removing extreme outlier interviewers. CONCLUSIONS: Interviewers may introduce error in food acquisition survey data when they are asked to interact with the respondents. Managers of future surveys with similarly complex data collection procedures could consider using multilevel models to adaptively identify and retrain interviewers who have extreme effects on data collection outcomes.