An Investigation of Nonresponse Bias and Survey Location Variability in the 2017-2018 National Health and Nutrition Examination Survey.

Over the past two decades, a steady decline in response rates on national face-to-face surveys has been documented, with steeper declines observed in recent years. The impact of nonresponse on survey estimates is inconsistent and depends on the correlation between response propensity and the survey estimates. To better understand the impact of declining response rates on the 2017-2018 National Health and Nutrition Examination Survey (NHANES), potential nonresponse bias (NRB) was investigated. NRB was assessed using three approaches: (a) studying variation within the respondent set; (b) benchmarking and comparisons to external data; and (c) comparing alternative weighting adjustments. Because NHANES only samples 30 counties in every 2-year cycle, the sample of counties in any given cycle may be an outlier on some characteristics. Such sampling variability may compound the effects of NRB. For this reason, the representativeness of the 2017-2018 NHANES counties was examined by comparing: (a) the characteristics of the 2017-2018 sampled counties with those from prior cycles; (b) each sampled county with the average of all the counties in the sampling stratum from which that county was selected; and (c) the 2017-2018 counties with 5,000 other samples that could have been drawn under the same sample design using a simulation study. The NRB analyses showed that the 2017-2018 NHANES sample had a lower proportion of college graduates and higher-income individuals compared with prior cycles. Additionally, the 2017-2018 NHANES counties had lower proportions of college graduates and lower mean incomes compared with counties from prior cycles and counties not selected in 2017-2018, which exacerbated the effects of NRB. Weighting adjustments used in prior cycles were not sufficient to address the bias in the 2017-2018 NHANES. Instead, enhanced weighting adjustments for education and income reduced the bias resulting from nonresponse and location sampling variability. All material appearing in this report is in the public domain and may be reproduced or copied without permission; citation as to source, however, is appreciated.