Validation of self-rated overall diet quality by Healthy Eating Index-2010 score among New York City adults, 2013.

OBJECTIVE: Chronic conditions such as cardiovascular disease and cancer can result from a number of diet-related environmental and behavioral factors. Screening for poor diet is helpful in developing interventions to prevent chronic disease, but measuring dietary behavior can be costly and time-consuming. The purpose of this study was to test the ability of a self-rated, single-item measure for evaluating diet quality among individuals and populations.
METHODS: A 24-h dietary recall and single-item self-rated diet quality measure were collected for 485 adults. From dietary recalls, Healthy Eating Index-2010 (HEI) scores were computed and compared with self-rated diet quality. Data were collected in 2013 among adult (18 years and older) New York City residents.
RESULTS: The study sample was 57% female, 47% white, 56% college educated, and 45% in the highest income tertile. The mean HEI score was 56.5 out of a possible 100. Women averaged higher HEI scores compared to men (58.1 vs 54.3, p = .01). There was a modest yet significant correlation between HEI scores and self-rated diet quality (ρ = 0.29, p < .01). Overall, mean HEI score increased as self-rated diet quality improved (from 48.2 for "poor" to 63.0 for "excellent").
CONCLUSIONS: The single-item measure of self-rated diet quality may provide a simple method of identifying those with the worst diet quality. Further investigation of this measure's validity is needed with alternative measures of dietary intake and with health outcomes.

Introduction1

Development of chronic conditions such as cardiovascular disease and cancer can result from a multitude of environmental and behavioral factors. Poor diet is a leading contributor to the occurrence and prevention of such chronic diseases (McCullough et al., 2000, Kant et al., 2000), as well as premature death (Mokdad et al., 2004). In general, recommendations for disease prevention emphasize increased consumption of fruit, vegetables, fat-free or low-fat dairy products, and whole grains, combined with a decrease in consumption of solid fats, added sugars, and refined grains (USDA and HHS, 2010). However, due to the coincident nature of such changes, it is difficult to determine the impact of specific dietary components as they relate to chronic disease prevention (McCullough et al., 2000, Kant et al., 2000). Complex diets contain a variety of nutrients and both beneficial and harmful components; however, there has been research showing that an overall dietary pattern generally consistent with the United States Department of Agriculture (USDA) Dietary Guidelines for Americans (DGA) is associated with decreased mortality risk and improved health outcomes (Kant et al., 2000).

One way to assess diet quality as it relates to DGA adherence is through the use of scores and indices, which can describe both quality and variety of a complex diet (Guenther et al., 2013a, Wirt and Collins, 2009). Refined scoring methods can pinpoint both protective and harmful dietary patterns, while linking them to biomarkers of disease mortality and chronic conditions. Cross-sectional dietary scoring has been used to predict long-term health outcomes, and there is evidence that such tools could be useful to help identify and target those who would benefit from a nutrition intervention, thereby potentially reducing nutrition-related chronic disease (Wirt and Collins, 2009).

The Healthy Eating Index-2010 (HEI) is a single, summary measure of dietary quality which has multiple functions: evaluating compliance to the 2010 DGA, showing changes in diet quality over time, describing and explaining eating patterns of Americans (US Department of Agriculture (USDA), 2013, Guenther et al., 2013a), and assessing overall healthfulness, rather than just singular components, of dietary intake (Guenther et al., 2013a). A validation study determined that it is sensitive enough to detect differences in diet quality between individuals and populations (Guenther et al., 2014). The HEI, which is out of a maximum of 100 points, includes 12 components: total fruit, whole fruit, total vegetables, greens and beans, whole grains, dairy (including soy products), total protein foods, seafood and plant proteins, fatty acids, refined grains, sodium, and empty calories. For the majority of components (i.e. those for which adequate consumption is encouraged), higher intake leads to a higher component score. However, for components meant to be eaten in moderation, such as refined grains, sodium, and empty calories, lower intake results in higher scores (Guenther et al., 2013a). The HEI is amenable to capturing diet quality of Americans because (1) it is appropriate for a wide variety of dietary patterns (Guenther et al., 2013b), and (2) it has the capacity to measure quality of the mix of foods consumed, since dietary intakes are assessed on the basis of nutrient density rather than absolute quantities consumed (Guenther et al., 2014). In public health practice, HEI scores can be used to better understand relationships between eating patterns and health outcomes (Guenther et al., 2013b).

However, collection of detailed dietary data to determine overall consumption patterns can be costly and time-consuming, usually requiring a food frequency questionnaire or 24-h dietary recall. More time- and cost-effective methods to measure diet quality, such as a single-item indicator, would enable public health practitioners and clinicians to more efficiently predict and prevent nutrition-related morbidity. In a similar vein, self-rated general health has been a validated and broadly used health indicator, proven to independently predict morbidity and mortality (Jylhä, 2009). Comprehensive dietary measures undoubtedly reflect true intake with more accuracy than a single-item measure could; however, applying the same logic from self-rated general health to self-rated diet quality, it would seem as though a single-item question could serve as a reliable tool for predicting diet-related health outcomes and/or flagging those with a potential need for dietary intervention. However, limited research has examined the validity of single-item measures, such as self-rated diet quality, in comparison to measured diet quality using HEI-like scales (Milton et al., 2013). The objective of this analysis is to test the construct validity of the single-item survey question against the measured HEI score computed from a sample of 24-h dietary recall surveys collected concurrently.

Methods

Study design

In winter 2013, the New York City Department of Health and Mental Hygiene (DOHMH) administered the NYC Health Survey and 24-Hour Dietary Recall, a cross-sectional study conducted among a representative sample of adult NYC residents. Overall, 2172 respondents first participated in a dual-frame (cellular and landline) random digit dial telephone survey, which was conducted in either English or Spanish. The survey collected information on demographics, Internet access and usage, typical sugary drink consumption, physical activity, general health, and overall diet quality. Self-reported diet quality was assessed through a single-item measure that asked: “In general, how healthy is your overall diet? Would you say excellent, very good, good, fair or poor?”

From these phone surveys, all 2172 participants were asked to participate in a follow-up online dietary recall survey, using the Automated Self-Administered 24-h Recall-2011™ (ASA24) system, which required computer and Internet access. Those without such access were referred to nearby public access options, such as local libraries. Sixty-two percent of phone survey respondents initially agreed to participate (N = 1342); of those, 641 never attempted to complete the recall, and 213 initiated but did not finish the recall. The remaining 488 participants (22.5%) were both recruited to and completed a single online 24-h dietary recall in either English or Spanish. Participants were instructed on how to log into the study website and provided with follow-up support (helpline, available 9 am–5 pm, M–F, or email support) if they encountered any difficulties.

Propensity scores were estimated for the probability that telephone survey respondents would agree to and complete the online dietary recall survey. Significant predictors for completing the ASA24 included education, age, and Internet access and use. Thus, although HEI scores and self-ratings for overall diet quality cannot be generalized to the population, these two measures can be compared within subjects for validation purposes.

New York City Department of Health and Mental Hygiene Institutional Review Board deemed this study exempt from human subjects research classification. Participants who completed the online survey tool were given a $40 incentive, and data were analyzed anonymously.

Dietary recall

The ASA24 system was developed by the National Cancer Institute, Bethesda, MD. ASA24 is a free online tool that allows respondents to enter all foods and drinks consumed during the previous 24-h period from midnight to midnight. The program is based upon the validated USDA Automated Multiple-Pass Method (AMPM), which is used in the National Health and Nutrition Examination Survey (NHANES) (ASA24, 2011a) and has been shown to reduce bias in the collection of dietary intake data (Moshfegh et al., 2008). The ASA24 system has performed well when validated against true dietary intake as well as the AMPM, despite limitations intrinsic in comparing two self-report methods (Kirkpatrick et al., 2014).

The ASA24 database generated from the participant input includes tables detailing food codes and weights for each food reported by each respondent based on the Food and Nutrient Database for Dietary Studies and MyPyramid Equivalents for each food reported based on the MyPyramid Equivalents Database, which can be used to derive HEI scores. The ASA24 Researcher website provides SAS codes and macros for computing total HEI scores (ASA24, 2011b).

Healthy Eating Index-2010

From reported food and beverage intake, we computed HEI scores, which are comprised of 12 dietary component sub-scores that add to 100 maximum points. Higher scores for each dietary component indicate closer adherence to dietary guidelines, both for “adequacy” and “moderation” components.

Statistical analyses

HEI total and component scores were calculated from participant food intake derived from ASA24 output. Amounts of each component, which are linked to the MyPyramid Equivalents Database, were combined to generate information on total fruit, whole fruit, total vegetables, etc. SAS codes and macros available on the ASA24 Researcher website were used to compute individuals' scores. Descriptive statistics, including frequencies, means, standard errors, and confidence intervals were used to characterize participant demographics and enumerate HEI total and component scores. These scores were then compared with self-rated diet quality responses, which were reported as part of a 5-point Likert scale (5 = “excellent,” 4 = “very good,” 3 = “good,” 2 = “fair,” 1 = “poor”). Generalized linear models (proc GLM) and Spearman rank correlation coefficients were used to measure associations between continuous HEI scores and ordinal self-rated diet quality. All analyses were conducted using SAS analysis software version 9.2 (SAS Institute; Cary, NC, USA).

Results

Study population

Overall, 485 of 488 participants both completed the dietary recall and answered the self-rated diet quality question. The sample included more females than males (57% vs 43%), was almost half white (47%), was majority college-educated (56%), and the highest income group was best represented (45%).

Self-rated overall diet quality and HEI score

Overall, 13% of respondents rated their health as “excellent,” 35% “very good,” 35% “good,” 14% “fair,” and 3% “poor” (Table 1). The mean HEI score was 56.5 (SD 16.2), with scores ranging from 16.5 to 93.3. Women earned a higher average total HEI score than men (58.1 vs 54.3, p = .01), as well as higher component scores for total fruit, whole fruit, total vegetables, greens and beans, whole grains, dairy, seafood and plant proteins, fatty acids, refined grains, and sodium. Men had higher component scores for total protein foods, and “empty calories” (solid fat, alcohol, and added sugar) (data not shown). Generalized linear models showed significant differences in mean component scores between those rating their diet as “excellent” versus “poor” for total fruit (p < .01), whole fruit (p < .001), total vegetables (p < .01), and empty calories (p = .02) (Table 2).

Table 1

Demographic characteristics of New York City adults, ASA24 Dietary Recall 2013.

	n	%
Total sample	485	100.0
Overall quality of diet
Excellent	63	13.0
Very good	170	35.1
Good	171	35.3
Fair	68	14.0
Poor	13	2.7
Gender
Male	207	42.7
Female	278	57.3
Age group
18–24	49	10.1
25–44	196	40.4
45–64	200	41.2
65 +	40	8.3
Race
NH white	227	46.8
NH black	110	22.7
Hispanic	98	20.2
NH Asian/Other	50	10.3
Education
HS or less	92	19.0
Some college	119	24.5
College grad +	274	56.5
Poverty/income
< 200% FPL	130	26.8
200%–399% FPL	137	28.3
400 + % FPL	218	45.0

Table 2

Mean daily HEI component scores for NYC adults, overall and by diet quality rating, ASA24 Dietary Recall 2013.

	Max Score	Overall (N = 485)	Excellent (N = 63)	Very good (N = 170)	Good (N = 171)	Fair (N = 68)	Poor (N = 13)	p-value
Total score	100	56.48	63.01	59.88	54.44	48.64	48.22	< .01
Adequacy
Total fruit	5	2.85	3.44	3.15	2.78	1.92	1.80	< .01
Whole fruit	5	2.80	3.43	3.22	2.65	1.90	0.77	< .001
Total vegetables	5	3.40	3.84	3.53	3.42	2.84	2.40	< .01
Greens and beans	5	2.50	2.83	2.50	2.64	1.90	2.21	.39
Whole grains	10	3.43	3.87	3.55	3.40	2.88	2.99	.45
Dairy	10	4.74	4.85	4.50	4.89	4.73	5.20	.75
Total protein foods	5	4.27	4.15	4.31	4.28	4.25	4.32	.69
Seafood and plant proteins	5	2.39	2.96	2.67	2.18	1.88	1.59	.05
Fatty acids	10	5.29	5.61	5.87	4.54	5.34	5.72	.93
Moderation
Refined grains	10	7.11	8.08	7.23	6.93	6.45	6.71	.20
Sodium	10	3.58	4.67	3.82	3.22	2.89	3.28	.20
Empty calories	20	14.13	15.29	15.53	13.51	11.67	11.24	.02

There was a weak yet significant correlation between HEI total score and self-rated overall diet quality, (ρ = 0.29, p < .01), with women's total scores more closely correlating with self-rated overall diet quality than men's (ρ = 0.32, p < 0.01; ρ = 0.25, p < 0.01, respectively) (data not shown). Similar to mean component scores described above, components that were most strongly correlated with self-rated diet quality were whole fruit (ρ = 0.24), empty calories (ρ = 0.23), and total fruit (ρ = 0.22), which may suggest that these items are the ones chiefly evoked when people are asked to rate their diet quality. Additionally, there were moderate yet significant correlations between self-rated diet quality and HEI component scores for total vegetables, seafood and plant proteins, refined grains, and sodium (p-values < 0.01) (Table 3).

Table 3

Mean daily HEI component scores for NYC adults, correlated with overall diet quality rating, ASA24 Dietary Recall 2013.

	Overall (N = 485)
	ρ	p-value
Total score	0.29	< .01
Adequacy
Total fruit	0.22	< .01
Whole fruit	0.24	< .01
Total vegetables	0.16	< .01
Greens and beans	0.07	.14
Whole grains	0.05	.30
Dairy	− 0.03	.54
Total protein foods	0.04	.34
Seafood and plant proteins	0.14	< .01
Fatty acids	0.08	.06
Moderation
Refined grains	0.14	< .01
Sodium	0.14	< .01
Empty calories	0.23	< .01

Overall, mean HEI score increased as self-rated diet quality improved. The average HEI score for those with “poor” self-rated diet quality was 48.2, “fair” averaged 48.6, “good” was 54.4, “very good” was 59.9, and “excellent” was 63.0. Women had higher mean HEI scores at each self-rated diet quality level except “poor.” HEI scores also increased with increases in age group and income level, with higher scores among women than men, except among the 18–24 age group. Overall, those aged 65 and up had significantly higher mean scores than those who were 44 and younger. Educational attainment level did not have a consistent relationship to HEI score, but the highest scores were among those with a college degree or higher. Non-Hispanic whites had the highest mean HEI score, which was significantly higher than that of non-Hispanic blacks. Women had higher mean HEI scores at each level of self-rated diet quality, but these differences were not statistically significant (Table 4).

Table 4

Mean daily HEI scores by sex and demographic characteristics for NYC adults, ASA24 Dietary Recall 2013.

	Overall (N = 485)		Males (N = 207)		Females (N = 278)
Population	Mean HEI	95% CI	Mean HEI	95% CI	Mean HEI	95% CI
Total	56.48	(55.04, 57.93)	54.32	(52.08, 56.57)	58.09	(56.21, 59.97)
Overall quality of diet
Excellent	63.01	(58.92, 67.10)	58.68	(52.27, 65.09)	66.26	(60.95, 71.58)
Very good	59.88	(57.55, 62.21)	57.92	(54.10, 61.74)	61.47	(58.58, 64.36)
Good	54.44	(52.00, 56.87)	52.20	(48.37, 56.02)	55.88	(52.71, 59.05)
Fair	48.64	(45.32, 51.96)	47.33	(42.23, 52.43)	49.62	(45.06, 54.18)
Poor	48.22	(37.69, 58.75)	48.55	(31.96, 65.15)	47.68	(29.20, 66.16)
Age
18–24	50.52	(46.79, 54.25)	51.23	(45.52, 56.93)	49.84	(44.59, 55.10)
25–44	55.52	(53.30, 57.73)	53.49	(49.84, 57.14)	56.80	(53.99, 59.60)
45–64	57.57	(55.21, 59.92)	54.68	(50.96, 58.41)	59.78	(56.77, 62.80)
65 +	63.10	(57.98, 68.21)	59.64	(52.18, 67.09)	66.56	(59.21, 73.91)
Race
NH white	58.23	(56.09, 60.37)	54.72	(51.57, 57.88)	61.71	(58.92, 64.51)
NH black	52.75	(49.68, 55.82)	51.63	(47.09, 56.17)	53.41	(49.25, 57.57)
Hispanic	56.84	(53.63, 60.06)	55.50	(49.10, 61.90)	57.53	(53.80, 61.25)
NH Asian/Other	56.04	(51.84, 60.23)	55.64	(48.83, 62.45)	56.30	(50.65, 61.95)
Education
HS or less	54.97	(51.55, 58.39)	54.13	(49.36, 58.90)	55.74	(50.70, 60.78)
Some college	52.96	(50.27, 55.65)	51.89	(47.71, 56.07)	53.83	(50.24, 57.41)
College grad +	58.52	(56.56, 60.47)	55.57	(52.32, 58.82)	60.49	(58.08, 62.90)
Poverty/income
< 200% FPL	53.26	(50.69, 55.83)	53.31	(49.21, 57.41)	53.23	(49.86, 56.59)
200%–399% FPL	56.35	(53.49, 59.22)	52.61	(47.97, 57.24)	58.79	(55.17, 62.41)
400 + % FPL	58.48	(56.31, 60.66)	55.89	(52.53, 59.26)	60.52	(57.69, 63.35)

Discussion

Though correlations between HEI total scores and self-rated diet quality overall and for both genders were statistically significant, the associations were relatively weak (overall: 0.29, p < .01; men: 0.25, p < .01; women: 0.32, p < .01). Self-rated diet quality responses were fairly normally distributed, with about 70% of respondents rating their diets as either “very good” or “good” which aligned with HEI scores of 59.9 and 54.4, respectively.

Self-report tools are a cost-effective, easily administered method of data collection. However, it is important that such tools are able to screen for the outcome of interest, in this case, diet quality (Milton et al., 2013). A similar single item self-report tool, self-rated general health, has been proven to independently predict morbidity and mortality (Jylhä, 2009) and used as a routine risk assessment indicator in medical practice (Jylhä, 2009, Elfassy et al., 2013). However, to date, little research has been done on the validity of self-rated diet quality compared to measured and scored dietary intake. Our results provide evidence that there is a weak overall association between self-rated diet quality and HEI score. The measure may be appropriate to identify those with the worst diets or to differentiate between those with high versus low scores, but limited in identifying large variance across populations. Utility may be found in its inclusion on population surveillance instruments for estimating prevalence of fair/poor diet quality or for identifying individuals and populations to target for nutrition-related interventions. This may be especially relevant for clinicians who, under the Affordable Care Act, newly offer covered preventive services for adults, including diet counseling for those at risk for chronic disease (healthfinder.gov, 2015).

Further, this study adds to gathering evidence of the potential usefulness of this single-item measure. It has previously been demonstrated that decreasing diet quality as assessed by the self-rated diet quality question corresponds with increasing odds of measured hypertension or of being overweight or obese in a dose–response fashion; those who reported poor diet quality had five times the odds of hypertension (OR: 5.04 [1.91, 13.29]) and four times the odds of being overweight or obese (OR: 3.77 [1.47–9.68]) compared to those reporting excellent diet quality after adjustment in multivariable regression models (Loftfield et al., 2014). Thus, while self-rated diet quality may not correlate strongly with the HEI, it may have utility in identifying those with elevated risk of diet-related comorbidities. Further research linking the self-rated diet quality question to health outcomes and mortality are needed.

It is worth noting that one contributor to higher HEI scores could be health literacy, which has been shown to be associated with increased HEI scores (Zoellner et al., 2011). Although our analysis did not explicitly measure health literacy, we did find that the highest HEI scores were among respondents with a college degree or higher. Despite this finding, health literacy does not necessarily contribute to accurate assessment of one's own diet quality. In an analysis conducted by the USDA's Center for Nutrition Policy and Promotion using the first iteration of the HEI, scores were compared to participant self-rated diet quality, and researchers found similar results to ours. Respondents who rated their diet as “excellent” had a mean HEI of 67.6, while those who rated their diet as “poor” averaged 55.8. The researchers concluded that individuals were fairly accurate in assessing their diet quality (USDA, 1995). However, additional analyses of the single-item question against other indices of diet quality are of interest.

There are several limitations of this study to consider. First, because of substantial day-to-day variability in consumption, usual dietary intake is not typically determined from a single 24-h dietary recall (Basiotis et al., 1987). However, an analysis using the first version of the HEI showed similar results for each index component using 1-day and 3-day dietary intake data (USDA, 1995). In addition, both measures used in this analysis were self-reported rather than measured, which may result in inaccurate outputs. However, because self-reports of both dietary intake and self-rated diet quality were taken from the same individuals, it is conceivable that the accuracy of their responses would be reliable and consistent. In addition, similar results have been observed when the self-rated overall diet quality question responses were correlated with dietary biomarkers (sodium, potassium) measured in 24-h urine samples (Loftfield et al., 2014). Another limitation is that the HEI is typically used to quantify and evaluate population adherence to dietary guidelines; however, we used individual-level data to calculate mean HEI scores, which is not universally accepted as the preferred method for describing such scores (Freedman et al., 2008).

Self-rated diet quality as measured by a single indicator helped to identify those with the poorest diet quality according to the HEI. Though correlations between HEI total scores and self-rated diet quality were statistically significant overall and for both genders, the associations were too weak to make assertions about the validity of this measure as a proxy for measured dietary quality. This measure should be further evaluated among other populations, with nationally representative dietary studies and alternative measures of dietary intake, and assessed in relation to morbidity and mortality from diet-related chronic disease. Despite its limitations, our study demonstrates that there may be utility in the inclusion of the self-rated diet quality question in population health studies in order to identify those with the poorest diet quality. Future research should be conducted to determine if there are meaningful links between low self-rated diet quality and morbidity and mortality from diet-related chronic disease.

Conflict of interest

None.