Smokers' BMI and perceived health: Does the order of questions matter?

We surveyed 431 daily smokers between November 2014 and March 2015 to examine the impact of the order of questions on the response to a self-reported health question as part of a larger experimental study. We randomized the question order, with some respondents providing their weight prior to self-reporting their health, while others did the opposite. We found that self-reported health outcomes are worse when smokers are first asked to report their weight. However, the order of questions only seems to impact those who are overweight as we did not find evidence that the order of questions affected responses for those with a BMI below 25. These findings suggest that the order of asking self-rated health and weight questions appears to matter, at least for overweight current smokers.

Introduction

There is significant research across multiple disciplines indicating that answers to surveys can be influenced by question order (Perreault, 1975–1976, McFarland, 1981, Lasorsa, 2003, Steenkamp et al., 2010, Schwarz, 1999, Pew Research Center, 2016, Idler and Benyamini, 1997, Jylhä, 2009). Lasorsa (2003) found that a survey that asked participants to rate their interest in politics, as well as assessed their political knowledge, found lower interest in politics from those who first were asked to answer difficult political knowledge questions. The Pew Research Center (Steenkamp et al., 2010) found that people were more likely to say they approved of civil unions for homosexual partners when they were previously asked a question about gay marriage. Another area where question sequence has been important is in reducing social desirability bias. Survey designers have recognized for years that misleading answers could be given on questions where people think there is a “right” way of thinking. Researchers have shown that the wording of questions, the order of questions, and how the research questions are introduced all can impact the answers to survey questions (Schwarz, 1999, Pew Research Center, 2016).

Self-reported health is an independent predictor of mortality and is also commonly used as a general indicator of health status (Idler and Benyamini, 1997). Jylhä (2009) reports that self-rated health has been used to predict mortality in “around one hundred” separate research studies. Age is known to be associated with self-rated health, such that older people have worse self-assessments (Badley et al., 2015, Wagner and Short, 2014, Manderbacka et al., 1999). Body mass index (BMI), a common indicator of overweight and obesity, is often assessed in self-report surveys (usually by asking for self-reported weight and height). Measured (as opposed to self-report) BMI, obesity in particular, was recently shown to predict lower self-rated health status (Cullinan and Gillespie, 2015, Herman et al., 2013).

Some studies have examined how question order might influence self-reported health. Lee and Schwarz (2014) found that different surveys that asked about self-rated health but with different priming questions did not yield different responses when given on a survey written in English. However, those who answered the same questions after they had been translated into Spanish reported different self-rated health across the two surveys when different priming questions were provided.

An open question is how the order of questions on weight and health influences responses to self-reported health. Given the order of questions effects can matter in other contexts, we explored whether self-rated health responses were different based on whether a participant was first asked to provide their weight. Because individual concerns about weight can influence health (Sikorski et al., 2011, Carr et al., 2007), obesity can be socially stigmatizing, and that excess body weight is associated with poor health outcomes (Flegal et al., 2007), we sought to examine whether asking about weight first could influence self-rated health status. Using a sample of daily smokers (who smoked one or more cigarettes daily), the main objective of this analysis is to explore if making one think about their weight (and any positive or negative attitudes that activates) leads to differential responses in self-rated health.

Methods

Study design

Institutional Review Board approval was received both by the Susquehanna University and the Roswell Park Cancer Institute IRB. All survey respondents were smokers who participated in a single-session experimental auction to determine smokers' value for cigarettes and e-cigarettes (for background, see Rousu et al., 2007, O'Connor et al., 2016). Experimental auctions are common in economics and are used to estimate demand for products (e.g., see Rousu et al., 2007, Thrasher et al., 2011). These are actual auctions, for real money, where winners obtain products. The auction results will be analyzed in separate papers.

We used newspaper and radio ads to recruit smokers 18 and older to participate in the experiment. To be eligible, a person needed to be 18 or older (the legal age in both New York and Pennsylvania) and smoke at least one cigarette per day. The experimental auction took about 75 min to complete and, at the end of the experiment, we asked several survey questions to participants, including a question about health. 431 smokers in Buffalo, NY and Selinsgrove, PA completed the in-person survey.

Participants were asked to complete the sentence that started with “Would you say that in general your health is:” and could indicate if they felt their health was excellent (Perreault, 1975–1976), very good (McFarland, 1981), good (Lasorsa, 2003), fair (Steenkamp et al., 2010), or poor (Schwarz, 1999). Participants were also asked to indicate their height and weight. We used the participants' self-reported height and weight to calculate a BMI value (e.g., see NIH, 2016).

We randomly varied the order of these questions to examine whether a person's self-reported health varied based on whether they were first asked to report their weight. Forty-three percent of participants were asked to rate their health first, then provide their height and weight. The remaining 57% of participants were asked to provide their height and weight first, then rate their health. The items were adjacent to each other, and the surrounding questions were otherwise consistent and related to smoking behaviors.

To further examine the impact of the order of questions on self-reported health scores, we used a probit model. For all data analysis, we used SAS version 9.4. If a smoker reported he/she was in good, fair, or poor health (the three lowest categories), we define this as H = 1. To examine the impact of weight, along with demographic and background characteristics on the probability of reporting oneself in good, fair, or poor health (i.e., did not consider their health as excellent or very good), a probit model is used, and is shown in the following equation:where α is an intercept term, W is a dummy variable that represents that the smoker was asked about his/her weight prior to completing the self-reported health question and δ' is the associated coefficient. X is a vector that represents the demographic and background characteristics of smoker i and β' is the associated coefficient vector; ε is the error term.

Results

Demographic characteristics and the response to the self-rated health question are presented in Table 1. Compared to a nationally representative sample of smokers in the National Health and Nutrition Examination Survey (NHANES) 2013–2014, our sample relatively over represents Blacks and men, and underrepresents smokers under 30, but otherwise broadly reflects the smoking population. Overall, the mean participant self-rated health score was 2.67 or between good and very good. Given the Centers for Disease Control and Prevention (2016) reports that a person is considered overweight if they have a BMI above 25, we separately examined those with a BMI of 25 or higher (N = 266, 62%) and below 25 (N = 165, 38%). These are comparable to the proportions among smokers in NHANES (65% above 25, 35% under 25), which defines a smoker as someone who has smoked 100 or more cigarettes in his or her lifetime. As might be expected, those who had a BMI above 25 reported themselves in worse health (self-rated health score of 2.76) than those who did not (self-rated health score of 2.52). This difference was statistically significant at the 1% level using both a t-test or a non-parametric Wilcoxon Rank Sum test. We used both tests to ensure the results are significant regardless of the underlying distribution of the data. Note that of the demographic background characteristics, only the percentage of females that are overweight vs. not-overweight differs statistically (P < 0.05).

Table 1

Demographic and self-reported health characteristics – overall and by weight.

	Overall (N = 431)	BMI under 25 (N = 165)	BMI 25 or greater (N = 266)	Test for difference between BMI < 25 vs. BMI ≥ 25
Race:white	N = 256	N = 100	N = 156	χ2 P-value = 0.420
	60%	61%	59%
Race:black	N = 143	N = 53	N = 90
	33%	32%	34%
Race:other	N = 32	N = 12	N = 20
	7%	7%	7%
Age – under 30	N = 123	N = 61	N = 62	χ2 P-value = 0.000
	29%	37%d	23%
Age – 30 to 50	N = 161	N = 53	N = 108
	38%	32%	41%
Age – over 50	N = 147	N = 51	N = 96
	33%	31%	36%
Female	N = 190	N = 60	N = 130	χ2 P-value = 0.013
	44%	36%c	49%
Smokes pack or more of cigarettes per day	N = 112	N = 42	N = 70	χ2 P-value = 0.840
	26%	25%	26%
Income_below 30 K	N = 124	N = 83	N = 141	χ2 P-value = 0.806
	52%	50%	53%
Income – between 30 K–60 K	N = 59	N = 21	N = 38
	14%	13%	14%
Income – over 60 K	N = 26	N = 11	N = 15
	6%	7%	6%
Income – chose not to reveal	N = 122	N = 51	N = 71
	28%	30%	27%
Moderately or very worried about future quality of life	N = 239	N = 91	N = 148	χ2 P-value = 0.272
	48%	38%	54%
Site – NY	N = 222	N = 81	N = 141	χ2 P-value = 0.502
	52%	49%	53%
Site – PA	N = 209	N = 84	N = 125
	48%	51%	47%
Perceived health (1 = excellent, 2 = very good, 3 = good, 4 = fair, 5 = poor)	2.67	2.52	2.76a	t-Test P-value = 0.008
	(0.90)e	(0.93)	(0.85)
Perceived health when asked about weight first (N = 244)	2.74	2.57	2.83 b	t-Test P-value = 2.21
	(0.87)	(0.90)	(0.85)
Perceived health when not asked about weight first (N = 187)	2.57b	2.48	2.65	t-Test P-value = 0.197
	(0.90)	(0.96)	(0.85)
BMI	28.17	22.34	31.78	t-Test P-value = 0.000
	(6.92)	(2.33)	(6.25)

Surveys took place between November 2014-March 2015.

Difference is statistically significant at the 1% level using a one-sided t-test and Wilcoxon rank sum test.

Difference in scores for self-reported health is statistically significant among those first asked their weight at the 5% level using both a t-test or a Wilcoxon rank-sum test.

Difference in percentage of female that were overweight in our sample is statistically significant at the 5% level using a chi-squared test.

Difference in percentage in age groups that were overweight is statistically significant at the 1% level using a chi-squared test.

Standard deviations in parentheses.

Table 1 also presents the impact of asking smokers about their weight prior to asking them to report their health. Across all participants we find that when asked about their weight first, smokers' report worse health (self-report health score of 2.74) than those who are not first asked about their weight (self-report health score of 2.57). This difference is statistically significant at the 1% level using both a t-test or a non-parametric Wilcoxon Rank Sum test.

However, when examining those with BMIs above and below 25 - we notice a difference. For those with a BMI under 25, i.e., those who are not overweight, the impact of being asked about weight first is smaller and not statistically significant. But there is a larger difference between the self-reported health scores of those who are overweight based on whether they are asked about their weight first (2.83 vs. 2.65, statistically significant at the 5% level). Note: we find similar results if we isolate to compare obese smokers (BMI > 30) against non-obese smokers. Obese smokers reported an average self-reported health score of 3.00 versus a score of 2.74 for smokers who were not obese (statistically significant at the 5% level using both a t-test or a non-parametric Wilcoxon Rank Sum test).

The results of the probit model are presented in Table 2. The results show that those who are overweight (BMI > 25) were more likely to rate their health in the bottom three categories. The coefficient for answering the self-rated health question before reporting weight is only statistically significant in the model that does not contain the interaction term (Model 1). The result that participants who first answer a weight question makes these same participants more likely to report lower health is consistent with the unconditional results. The only other effect we find with the probit model is age was positively correlated with the probability of self-rating health in one of the bottom three categories.

Table 2

Probit model examining the probability that a smoker did not consider themselves in excellent or very good health. (N = 431).

Characteristics	Model 1	Model 2	Model 3
Intercept	0.14	0.05	− 0.45a
	(0.12)	(0.14)	(0.23)
BMI > 25	0.32b	0.47c	0.45b
	(0.13)	(0.17)	(0.17)
Not asked weight before self-rated health question	− 0.20a	− 0.01	0.01
	(0.12)	(0.20)	(0.20)
Interaction: BMI > 25 ∗ not asked weight before SRH		− 0.31	− 0.31
		(0.26)	(0.26)
Female			− 0.11
			(0.13)
Race: non-white			0.05
			(0.13)
Smokes more than one pack daily			0.14
			(0.15)
Age			0.01b
			(0.00)

Dependent variable = SRH – self-reported health.

(Standard error in parentheses).

Surveys took place between November 2014–March 2015.

Statistically significant at the 0.10 level.

Statistically significant at the 0.05 level.

Statistically significant at the 0.01 level.

Discussion

Many studies have shown that the way questions are ordered affects response from surveys. However, to our knowledge, no studies had examined the impact of weight questions on self-reported health. We find that smokers self-report worse health status when they are first asked to report their weight. However, the order of questions only seems to impact those who are overweight. We do not find that the order of questions affected responses for those with a BMI below 25. There is little evidence that smokers differ in BMI from the population at large (in fact, they may be somewhat lower; e.g., see Kaufman et al., 2012), nor that they are more susceptible to ordering effects – so, we believe these findings would generalize for smokers and non-smokers. Still, our study is limited based on our survey size and our use of only smokers. A larger replication study with a representative cross section would be useful. Based on these data, however, it appears that asking about weight among overweight participants may evoke attitudes toward one's own weight (unmeasured in our study) in ways that reduce positive perception of own health status. Future studies could examine this more directly by also assessing weight-related attitudes.

Funding

This work was supported by a grant from the National Institute on Drug Abuse (R21DA036476).

IRB approval was received both by the Susquehanna University and the Roswell Park Cancer Institute IRB.

Conflict of interest

The authors declare there is no conflict of interest.

Transparency Document

Transparency document.