Examining the Advantages of Using Multiple Web-Based Dietary Assessment Instruments to Measure Population Dietary Intake: The PREDISE Study.

BACKGROUND: Combining traditional dietary assessment instruments has been suggested to improve precision of dietary intake estimates. However, this has not been investigated using web-based 24-h recall (R24W) or a web-based food-frequency questionnaire (wFFQ).
OBJECTIVE: The aim of this study was to compare different combinations of web-based instruments to assess population-level dietary intake estimates (means and percentiles) and their precision, either with or without statistical modeling of within-person day-to-day variations.
METHODS: As part of the cross-sectional PREDISE study, 1025 French-speaking adults completed 3 randomly allocated R24W and 1 wFFQ within 21 d. Crude estimates of intake were generated from either 1 or 3 repeated R24W. The National Cancer Institute (NCI) method was used to account for within-person variation. Usual intakes were modeled from 1 R24W repeated in a subsample (40%) and from 3 R24W, with or without consideration of data from the wFFQ.
RESULTS: Using crude data from 3 R24W increased precision of estimates and modified distribution of intakes compared with using data from only 1 R24W. Using NCI-modeled data from 3 repeated R24W had no impact on the precision around mean intakes but increased precision of low and high percentiles intake estimates compared with NCI-modeled data from a partially repeated R24W. Considering data from a wFFQ in combination with data derived from 3 R24W did not influence the precision of intake estimates of most foods and nutrients.
CONCLUSIONS: The data suggest that relying on repeated measures of food and nutrient intake through R24W is preferable to single assessment when within-person variation is not considered. Data also suggest that when NCI modeling is applied, using 3 R24W only improves the precision of low and high percentiles intake estimates compared with using a partially repeated web-based recall.

Introduction

Large surveys assessing dietary intakes at the population level are crucial to inform nutrition-oriented public health policies as well as to monitor the diet of the population (1, 2). Twenty-four-hour recalls (24HRs) are the dietary assessment instrument of choice in national surveys because they are considered less biased than other instruments, such as food-frequency questionnaires (FFQs) or dietary screeners (3, 4). However, a single 24HR represents dietary intake on a given day only and is influenced by day-to-day variation in intake and random errors, which together reflect within-person variation (5–7). Fortunately, methods have been developed to account for within-person variation and produce usual intake distribution (i.e., longer-term average) based on data from repeated 24HRs (5, 8). The National Cancer Institute (NCI) has developed methods that produce usual intake distributions for everyday and for episodically consumed foods and

nutrients based on data from a single 24HR and a second 24HR repeated in at least a fraction of the population (9, 10). In addition, the NCI methods allow combining data from 24HR and FFQ, which has been proposed to improve estimation of usual intake (11), particularly for episodically consumed foods (12–14).

Technological advances have prompted the development and validation of web-based self-administered 24HRs (15–17), which are often preferred by participants over interviewer-administered (IA)-24HRs (15, 16). Studies suggest that dietary intake estimates obtained from self-administered web-based 24HRs are equivalent to estimates generated by IA-24HRs and hence that their use is appropriate to assess dietary intake at the population level (15, 16, 18–21). One advantage of web-based 24HRs is the ease of collecting repeated data at low cost (22) and possibly increasing quality of dietary assessment (23, 24). To the best of our knowledge, no study has yet examined how the use of repeated web-based 24HRs reduces the variability in the estimate of dietary intakes at the population level. The added value of combining intake data from a web-based FFQ (wFFQ) with data from multiple web-based 24HRs to assess usual dietary intakes based on the NCI modeling approach is also unknown.

Therefore, the aim of this study was to determine how different combinations of web-based dietary assessment instruments influence dietary intake estimates (i.e., means and percentiles of the distribution) and their precision (i.e., variance) at the population level. For this purpose, we used web-based instruments that we developed and validated in the past, including a web-based 24HR (R24W) (17, 25, 26), and a wFFQ (27). Our hypothesis was that repeated R24W provide the most precise estimates of food and nutrient intakes, independent of NCI modeling, and that addition of a wFFQ further increases precision, especially for the most rarely consumed foods.

Methods

Study design and participants

Complete methods of the PREDISE (PRÉDicteurs Individuels, Sociaux et Environnementaux) study have been published elsewhere (28). Briefly, a multicenter cross-sectional and web-based study was designed to examine the association between individual, social, and environmental factors and adherence to Canadian dietary guidelines. Participants were French-speaking men and women from 5 major administrative regions of the Province of Quebec, Canada. To be eligible, participants had to be aged 18–65 y, speak French as the primary language at home, have a computer, have access to the internet, and have a valid e-mail address. Participants who completed all questionnaires were eligible for a draw to win 1 of 40 gift cards and 2 electronic devices. The original study protocol was approved by the ethics board from each participating institution. Finally, the study objectives were not prespecified and are considered exploratory.

Dietary assessment instruments

During a 21-d period, participants were invited by e-mail and phone call to complete the self-administered R24W on 3 separate unannounced occasions selected randomly by an in-house computer algorithm. Details about the development and validation of the R24W have been reported elsewhere (17, 25, 26). Briefly, the R24W was inspired by the automated multiple-pass method of the US Department of Agriculture and uses a meal-based approach to begin the recall (17). The R24W automatically computes Canada's Food Guide 2007 servings (29) and nutrient intakes according to the Canadian Nutrient File 2015 (30). During the same period, the participants also completed a wFFQ previously validated in French-speaking adults (27). The wFFQ is a self-administered web-based questionnaire designed to reflect dietary intakes during the past 30 days. The questionnaire has 136 questions that are based on the Willet semiquantitative FFQ and a previously validated IA-FFQ (31). Several serving sizes were digitally photographed using standardized dinnerware.

Specific foods [vegetables, whole fruits (excluding juices), milk, cheese, yogurt, red meat, fish, nuts and seeds, and sugar-sweetened beverages] and nutrients [total energy, saturated fatty acids, α-linolenic acid, fibers, sodium, potassium, calcium, iron, magnesium, and vitamin D) were selected for analyses. Sugar-sweetened beverages included fruit-flavored drinks, sodas, sport drinks, energy drinks, and sweetened coffee or tea. These foods and nutrients reflect daily to episodic patterns of consumption, and they are of public health interest. All foods are expressed as servings/d, equivalent to servings in Canada's Food Guide 2007 (29), except for total dairy, for which the units are cup-equivalents calculated as servings from milk + servings from cheese + 0.75 times servings from yogurt.

Instrument combinations and assumptions

To account for within-person variation using statistical NCI modeling, at least a partially repeated dietary recall is necessary to estimate within-person variance (5, 9). The repetition of the second dietary recall in 40% of a random sample of participants was chosen according to dietary assessment methodologies for national survey in Canada (32). Assumptions when using the NCI modeling method are 1) that the R24W is an unbiased instrument for measuring usual food intake (i.e., that it is not subject to systematic error) and 2) that there are no true never-consumers of a given episodic food or nutrient (9). The dietary assessment instrument combinations examined are shown in . Specifically, we compared the following combinations and approaches: 1) data from 1 compared with 3 R24W without NCI modeling of the data, 2) NCI-modeled data from 1 R24W plus a second R24W repeated in 40% of the participants compared with NCI-modeled data from 3 R24W in all participants, and 3) NCI-modeled data from 3 R24W compared with NCI-modeled data from 3 R24W and the wFFQ. Crude data from the wFFQ were also calculated for reference purposes but were not formally compared with the other combinations because differences between a wFFQ and a 24HR are beyond the scope of this study.

TABLE 1

Dietary assessment instrument combinations and approaches studied

	Approaches
	Crude data		NCI-modeled data2
Dietary assessment	1×R24W	3×R24W	1×R24WNCI	3×R24WNCI	3×R24W + wFFQNCI
Type of intake	Given day	Within-person mean	Usual	Usual	Usual
No. of R24W	1	3	1 + second repeated in 40%	3	3
wFFQ	No	No	No	No	Yes

1NCI, National Cancer Institute; R24W, web-based 24-h recall; wFFQ, web-based food-frequency questionnaire.

2The NCI method 2.1 was used to account for within-person variation and produce usual intake estimates. Covariates for the NCI method were recall sequence, weekend, and age. The FFQ data were considered only in the 3×R24W + wFFQNCI combination.

Comparison of intake with DRIs

Consistent with DRIs from the Institute of Medicine (33), cutoffs were defined as the upper tolerable level of intake for sodium (2300 mg/d), the adequate intake for α-linolenic acid (1.6 g/d for men and 1.1 g/d for women), the age- and sex-specific estimated average requirements (34) for calcium (ranging from 800 to 1100 mg/d), and the estimated average requirement for vitamin D (10 IU/d). Examining the proportion of participants above or below DRIs using only crude data from 1 R24W would be an improper use of the DRIs (34), but the focus of this study is to compare theoretical results obtained through different combinations of dietary assessment methods.

Sample size and statistical analyses

When using the NCI modeling method, problems with convergence are uncommon with any reasonable sample size (unpublished observation). Therefore, among the 1149 participants who completed at least 1 R24W in the original study (28), only those who completed 3 R24W and the wFFQ (n = 1025) were included in the current study.

Mean estimates of intake generated by crude data combinations (Table 1) were compared using the PROC MIXED procedure in SAS Studio (version 3.6; SAS Institute). Dietary variables were considered as the dependent variables, whereas instrument combination was modeled as the independent variable (fixed effect), adjusting for covariates age group, sex, and region. Participants were considered in the repeated statement as a random effect. Proportions of participants above or below DRI with 95% CI were obtained with exact binomial proportion, and McNemar's exact test was used to assess potential differences among instrument combinations.

The NCI method version 2.1 for SAS (version 9.4; SAS Institute) was used to produce distribution of usual intake (NCI-modeled data; Table 1). Covariates for the NCI method were day sequence (first, second, and third), weekend indicator (1 or 0), and age group. The wFFQ dietary intake data were considered in the NCI modeling in combination with data from 3 R24W (3×R24W + wFFQNCI). The usual intake program provided by the NCI selects an amount model if 90% or more of total observations (all days combined) are >0 (10). Otherwise, a 2-part correlated model is fitted, where usual intake corresponds to amount times probability of consumption (9). The NCI's 2-part correlated model has been developed to estimate usual intake while accounting for nonconsumption days (i.e., recalls with 0 intake reported by some participants) and for positively skewed data such as episodic foods (9). The amount models were selected for all nutrients, and the correlated 2-part model was selected for all foods.

Standard errors and variances were calculated for all estimates of intake using 1) the Taylor series method for crude data and 2) balanced repeated replication for NCI-modeled data. Variance ratios between estimates of intake obtained with the combinations of instruments served as a proxy for precision. Variances of intake from 3×R24W and 3×R24WNCI were compared with variances generated by 1×R24W and 1×R24WNCI, respectively, to assess the effect of repeated R24W on precision of estimates. The variance of intake from 3×R24W + wFFQNCI was compared with that from 3×R24WNCI to assess how addition of data from the wFFQ improves precision of intake estimates. A ratio <1.00 indicates a lower variance (i.e., greater precision around the mean) and a ratio >1.00 indicates greater variance (i.e., lower precision around the mean) compared with the reference categories. All mean ratios and their CIs were generated in the log scale and then exponentiated. In sensitivity analyses, we have excluded individuals likely to be underreporters of energy intake based on the ratio of predicted energy requirements to self-reported energy intake using the method of Huang et al. (35). Details are presented in . Two women who reported being pregnant during the study were also excluded in these sensitivity analyses. A 2-sided α level of 0.05 was used for statistical tests.

Results

Participants

Characteristics of the participants included in this study are shown in . Men and women were equally represented. Most participants were Caucasians (94.2%), never-smokers (54.0%), and a large proportion (44.7%) had a university education. Mean ± SD time to completion was 22 ± 9 min for a single R24W and 41 ± 20 min for the wFFQ.

TABLE 2

Sociodemographic characteristics of the 1025 French-speaking adults included in the analyses

Characteristic	n (%)
Sex
Men	511 (49.9)
Women	514 (50.1)
Age group, y
18–34	365 (35.6)
35–49	295 (28.8)
50–65	365 (35.6)
Administrative region
Estrie	108 (10.5)
Saguenay–Lac-Saint-Jean	95 (9.3)
Capitale-Nationale/Chaudière-Appalaches	378 (36.9)
Montreal	349 (34.0)
Mauricie	95 (9.3)
BMI, kg/m2
Normal (<25.0)	387 (39.5)
Overweight (25.0–29.9)	326 (33.3)
Obese (≥30.0)	267 (27.2)
Ethnicity
Caucasian	966 (94.2)
African/African American	24 (2.3)
Hispanic	18 (1.8)
Other	17 (1.7)
Education
High school or less	252 (24.7)
CEGEP2	314 (30.7)
University	456 (44.6)
Household income, $CAD
<30,000	153 (16.3)
≥30,000 to <60,000	265 (28.2)
≥60,000 to <90,000	187 (19.9)
≥90,000	334 (35.6)
Smoking
Yes	133 (13.0)
Formerly	338 (33.0)
Never	554 (54.0)
Reporting status
Underreporter (rEI:pER ≤ 0.78)	160 (15.6)
Plausible reporter (0.78 < rEI:pER < 1.22)	546 (53.3)
Overreporter (rEI:pER ≥ 1.22)	319 (31.1)

The numbers may not sum to the total number of participants due to missing data. pER, predicted energy requirements; rEI, reporting energy intakes.

CEGEP (Collège d'Enseignement Général et Professionnel) is a pre-university and technical college institution specific to the Quebec educational system.

Value of repeated R24W without NCI modeling (crude data)

As presented in , most mean intake estimates in food servings per day were similar between the 3×R24W and 1×R24W combinations. Differences of small magnitude between 3×R24W and 1×R24W were observed for yogurt (mean difference, −0.03 servings/d; P = 0.01) and sugar-sweetened beverages (mean difference, −0.03 servings/d; P = 0.03). Small differences were also observed with 3×R24W compared with 1×R24W for 7 of 10 nutrients (). Variances derived from 3×R24W compared with 1×R24W were lower for all selected foods and nutrients (Tables 3 and 4). As shown in , using 3×R24W compared with 1×R24W reduced the variance ratios for mean intake of episodically consumed foods (mean ratio, 0.51), daily-consumed foods (mean ratio, 0.55), and nutrients (mean ratio, 0.53), indicating improved precision around mean intake estimates in all cases.

TABLE 3

Comparisons of mean food intake estimates generated by different combinations of dietary assessment instruments in 1025 French-speaking adults

	Crude data			NCI-modeled data2
Servings/d	wFFQ	1×R24W	3×R24W	1×R24WNCI	3×R24WNCI	3×R24W + wFFQNCI3
Vegetables and fruits
Vegetables (e.g., ½ cup)	1.80 (1.73, 1.88)	2.49 (2.34, 2.63)	2.46 (2.36, 2.57)	2.49 (2.41, 2.57)	2.49 (2.41, 2.56)	2.49 (2.42, 2.56)
		Refcrude	[0.53]	Refmodeled	[0.97]	[0.95]
Whole fruits (e.g., 1 fruit)	1.63 (1.56, 1.70)	1.37 (1.27, 1.47)	1.32 (1.24, 1.41)	1.37 (1.28, 1.46)	1.37 (1.31, 1.43)	1.37 (1.30, 1.44)
		Refcrude	[0.71]	Refmodeled	[0.49]	[1.19]
Dairy
Milk, 250 mL	0.63 (0.57, 0.69)	0.63 (0.57, 0.68)	0.63 (0.58, 0.68)	0.63 (0.59, 0.67)	0.64 (0.60, 0.67)	0.64 (0.61, 0.66)
		Refcrude	[0.57]	Refmodeled	[0.75]	[0.57]
Cheese, 50 g	0.95 (0.89, 1.02)	0.65 (0.59, 0.72)	0.63 (0.59, 0.67)	0.65 (0.59, 0.71)	0.64 (0.57, 0.72)	0.64 (0.57, 0.72)
		Refcrude	[0.43]	Refmodeled	[1.48]	[0.98]
Yogurt, 175 g	0.38 (0.35, 0.41)	0.32 (0.28, 0.36)	0.29 (0.26, 0.32)4	0.31 (0.28, 0.34)	0.31 (0.28, 0.33)	0.31 (0.28, 0.33)
		Refcrude	[0.57]	Refmodeled	[0.78]	[1.38]
Total, cup-equivalent/d	1.86 (1.76, 1.96)	1.52 (1.43, 1.61)	1.48 (1.41, 1.55)	1.52 (1.38, 1.66)	1.52 (1.43, 1.60)	1.52 (1.44, 1.60)
		Refcrude	[0.56]	Refmodeled	[0.39]	[0.92]
Meat and alternatives
Fish, 75 g	0.39 (0.36, 0.41)	0.24 (0.20, 0.28)	0.24 (0.22, 0.27)	0.25 (0.20, 0.29)	0.24 (0.20, 0.28)	0.24 (0.20, 0.28)
		Refcrude	[0.39]	Refmodeled	[0.86]	[0.99]
Red meat, 75 g	0.64 (0.60, 0.68)	0.63 (0.56, 0.70)	0.64 (0.60, 0.68)	0.63 (0.52, 0.73)	0.63 (0.51, 0.75)	0.65 (0.51, 0.78)
		Refcrude	[0.39]	Refmodeled	[1.25]	[1.37]
Nuts and seeds, 60 mL	0.19 (0.17, 0.21)	0.18 (0.15, 0.21)	0.16 (0.14, 0.18)	0.17 (0.12, 0.22)	0.17 (0.12, 0.22)	0.17 (0.13, 0.21)
		Refcrude	[0.55]	Refmodeled	[0.89]	[0.95]
Beverages
Sugar-sweetened beverages,	0.36 (0.31, 0.41)	0.35 (0.30, 0.41)	0.32 (0.28, 0.36)4	0.36 (0.29, 0.42)	0.36 (0.30, 0.43)	0.36 (0.29, 0.43)
237 mL		Refcrude	[0.63]	Refmodeled	[1.16]	[0.95]

Values are means (95% CIs). Ratios of variance compared with the reference combination are presented in brackets. A ratio <1.00 indicates smaller variance (i.e., increased precision around the mean) and a ratio >1.00 indicates larger variance (i.e., reduced precision around the mean) compared with the reference method. NCI, National Cancer Institute; Ref, reference method to calculate the ratio of variances; R24W, web-based 24-h recall; wFFQ, web-based food-frequency questionnaire.

The NCI method 2.1 was used to produce usual intake estimates. Covariates for the NCI method were recall sequence, weekend, age, and FFQ, where appropriate. The NCI 2-part model was used for all foods (>10% of data as 0).

The 3×R24WNCI + wFFQ combination is compared with 3×R24WNCI.

P < 0.05 compared with 1×R24W (adjusted for age, sex, and region), as determined by mixed models.

TABLE 4

Comparisons of mean nutrient intake estimates generated by different combinations of dietary assessment instruments in 1025 French-speaking adults

	Crude data			Modeled data2
	wFFQ	1×R24W	3×R24W	1×R24WNCI	3×R24WNCI	3×R24W + wFFQNCI3
Energy
Energy, kcal	2483 (2415, 2550)	2473 (2409, 2537)	2418 (2372, 2464)4	2475 (2409, 2540)	2471 (2409, 2533)	2473 (2411, 2534)
		Refcrude	[0.51]	Refmodeled	[0.89]	[0.99]
Macronutrients
SFA, %E	12.3 (12.1, 12.5)	11.8 (11.5, 12.1)	11.9 (11.7, 12.1)	11.8 (11.4, 12.1)	11.8 (11.5, 12.1)	11.8 (11.5, 12.1)
		Refcrude	[0.52]	Refmodeled	[0.87]	[0.98]
n–3 (ALA), g/d	1.88 (1.82, 1.95)	2.11 (2.01, 2.21)	2.07 (2.00, 2.14)	2.09 (1.97, 2.21)	2.10 (1.99, 2.21)	2.10 (2.00, 2.21)
		Refcrude	[0.48]	Refmodeled	[0.84]	[0.99]
Dietary fibers, g/d	27.9 (27.0, 28.7)	23.4 (22.7, 24.2)	22.7 (22.1, 23.3)4	23.3 (22.3, 24.3)	23.3 (22.4, 24.2)	23.3 (22.4, 24.3)
		Refcrude	[0.63]	Refmodeled	[0.86]	[0.99]
Micronutrients
Sodium, mg/d	3408 (3306, 3510)	3505 (3373, 3637)	3402 (3320, 3484)4	3483 (3217, 3749)	3468 (3210, 3725)	3468 (3209, 3728)
		Refcrude	[0.39]	Refmodeled	[0.94]	[1.02]
Potassium, mg/d	4110 (3999, 4220)	3390 (3309, 3471)	3330 (3264, 3395)4	3389 (3271, 3507)	3388 (3263, 3513)	3391 (3267, 3515)
		Refcrude	[0.66]	Refmodeled	[1.12]	[0.98]
Calcium, mg/d	1406 (1361, 1450)	1188 (1148, 1228)	1155 (1125, 1184)4	1188 (1156, 1221)	1184 (1154, 1215)	1185 (1151, 1218)
		Refcrude	[0.55]	Refmodeled	[0.91]	[1.19]
Iron, mg/d	21.2 (20.4, 22.0)	16.0 (15.5, 16.5)	15.6 (15.3, 15.9)4	15.9 (15.1, 16.7)	15.9 (15.2, 16.6)	15.9 (15.2, 16.6)
		Refcrude	[0.41]	Refmodeled	[0.85]	[1.01]
Magnesium, mg/d	463 (450, 475)	417 (406, 428)	405 (396, 414)4	416 (404, 429)	416 (402, 430)	416 (402, 430)
		Refcrude	[0.64]	Refmodeled	[1.24]	[0.98]
Vitamin D, μg/d	18.4 (16.7, 20.2)	5.3 (5.0, 5.6)	5.4 (5.2, 5.6)	5.4 (5.0, 5.8)	5.4 (4.9, 5.8)	5.4 (4.9, 5.8)
		Refcrude	[0.58]	Refmodeled	[1.16]	[1.00]

Values are means (95% CIs). Ratios of variance compared with the reference combination are presented in brackets. A ratio <1.00 indicates smaller variance (i.e., increased precision around the mean) and a ratio >1.00 indicates larger variance (i.e., reduced precision around the mean) compared with the reference method. ALA, α-linolenic acid; NCI, National Cancer Institute; Ref, reference method to calculate the ratio of variances; R24W, web-based 24-h recall; SFA, saturated fats; wFFQ, web-based food-frequency questionnaire.

The NCI method 2.1 was used to produce usual intake estimates. Covariates for the NCI method were recall sequence, weekend, age, and FFQ, where appropriate. The NCI “amount” model was used for all nutrients (<10% of data as 0).

The 3×R24WNCI + wFFQ combination is compared with 3×R24WNCI.

P < 0.05 compared with 1×R24W (adjusted for age, sex, and region), as determined by mixed models.

FIGURE 1

Mean variance ratios comparing the precision of different combinations of web-based dietary assessment instruments to assess low percentiles (i.e., 5th, 10th, and 25th), mean, and high percentiles (i.e., 75th, 90th, and 95th) of food and nutrient intake estimates of the population when using crude data. Mean variance ratios and 95% CIs were generated in the log scale and then exponentiated. Error bars reflect the 95% CIs of the variance ratios. Daily foods include vegetables, whole fruits (excluding juices), milk, cheese, yogurt, and red meat. Episodic foods include fish, nuts and seeds, and sugar-sweetened beverages. Nutrients include total energy intake, saturated fats, α-linolenic acid, fibers, sodium, potassium, calcium, iron, magnesium, and vitamin D. NCI, National Cancer Institute; R24W, web-based 24-h recall; wFFQ, web-based food-frequency questionnaire.

Proportions of participants above or below DRIs for selected nutrients are shown in . The use of 3×R24W resulted in different prevalence of intake adequacy for all nutrients of interest compared with 1×R24W (all P values < 0.0001). For example, the proportion of individuals exceeding the upper tolerable level of intake for sodium was underestimated by 8.2% when using data from 1×R24W compared with data from 3×R24W, whereas the proportion of individuals failing to meet the estimated average requirement for calcium was overestimated by 5.0% with 1×R24W compared with 3×R24W. Mean reductions in variance (reflected by the variance ratio between 3×R24W and 1×R24W) for low percentiles estimates (i.e., 25th and below) and high percentiles estimates (i.e., 75th and above) are presented in Figure 1. Variances were reduced for both foods and nutrients in low and high percentiles estimates when using 3×R24W compared with 1×R24W. For low percentiles of intake, the reduction in variance was greater for foods than for nutrients, especially for episodic foods. Percentile estimates for individual foods and nutrients are available in and .

FIGURE 2

Proportions of participants above or below relevant DRIs for sodium (A), calcium (B), α-linolenic acid (C), and vitamin D (D) as obtained with different combinations of dietary assessment instruments. Error bars are 95% CIs. AI, adequate intake; EAR, estimated average requirement; NCI, National Cancer Institute; R24W, web-based 24-h recall; UL, tolerable upper intake level.

Value of combining repeated R24W, with or without a wFFQ (NCI-modeled data)

For all foods and nutrients, no clear pattern was observed for differences in variance ratios and, hence, in precision when data from 3×R24WNCI were used to assess population mean intake estimates compared with data from 1×R24WNCI. Indeed, Tables 3 and 4 show that variance ratios increased for some foods and nutrients (e.g., cheese, red meat, potassium, and magnesium) and decreased for others [e.g., fruits, total dairy, and n–3(ω-3)]. Overall, variances derived from 3×R24WNCI were similar to those from 1×R24WNCI, as indicated by mean variance ratios of 0.79 (95% CI: 0.51, 1.24) for daily foods and 0.96 (95% CI: 0.87, 1.06) for nutrients (). Furthermore, combining data from wFFQ with data from 3×R24WNCI had no effect on variances of mean usual food, either daily or episodic, and nutrient intake (Tables 3 and 4, Figure 3).

FIGURE 3

Mean variance ratios comparing the precision of different combinations of web-based dietary assessment instruments to assess low percentiles (i.e., 5th, 10th, and 25th), mean, and high percentiles (i.e., 75th, 90th, and 95th) of food and nutrient intake estimates of the population when using NCI-modeled data. Mean variance ratios and 95% CIs were generated in the log scale and then exponentiated. Errors bars reflect the 95% CIs of the variance ratios. One 95% CI including values over 2.0 was truncated for clarity. Daily foods include vegetables, whole fruits (excluding juices), milk, cheese, yogurt, and red meat. Episodic foods include fish, nuts and seeds, and sugar-sweetened beverages. Nutrients include total energy intake, saturated fats, α-linolenic acid, fibers, sodium, potassium, calcium, iron, magnesium, and vitamin D. NCI, National Cancer Institute; R24W, web-based 24-h recall; wFFQ, web-based food-frequency questionnaire.

Proportions of participants above or below relevant DRIs were similar among all NCI-modeled combinations for the selected nutrients (Figure 2), with minimal effect on the estimates of repeated R24W (3×R24WNCI compared with 1×R24WNCI) or addition of an FFQ (3×R24W + wFFQNCI compared with 3×R24WNCI). However, using 3×R24WNCI reduced variances for foods and nutrients in both low and high percentiles of intake compared with 1×R24WNCI (Figure 3). Finally, addition of the wFFQ to 3×R24WNCI resulted in an increase in variance for low percentiles for both episodically consumed foods (variance ratio: 1.99; 95% CI: 0.94, 4.19) and daily consumed foods (variance ratio: 1.32; 95% CI: 1.11, 1.56). For high percentiles, addition of the wFFQ to 3×R24WNCI was suggestive of a small reduction in variance for episodically consumed foods (variance ratio: 0.86; 95% CI: 0.70, 1.04; Figure 3).

Exclusion of potential underreporters of total energy intake

Excluding data (n = 162) from potential underreporters of energy intake and two women who reported being pregnant during the study did not materially affect mean variance ratios for both the crude and the NCI-modeled analyses compared with results obtained in the entire sample (see details in Supplemental Figure 1).

Discussion

To the best of our knowledge, this study demonstrates for the first time that in the absence of NCI modeling, the use of repeated R24W, 3 R24W compared with 1 R24W, increases the precision of mean population intake estimates as well as low and high percentiles estimates. Also, proportions of participants above or below DRIs when using crude data from repeated R24W (3 compared with 1) were closer to values obtained from NCI-modeled proportions. When applying the NCI method to account for within-person variation, data derived from the R24W repeated only once in a random subsample of the population and from 3 repeated R24W yielded estimates of mean food and nutrient intake with similar precision. On the other hand, NCI modeling using data from 3 R24W increased the precision of estimates for low and high percentiles compared with NCI-modeled data from a partially repeated R24W. Finally, considering data from a wFFQ did not improve precision of usual intake estimates over and above data derived from 3 repeated R24W. These observations were true for patterns of food and nutrient consumption ranging from episodically to daily.

Before the development of the most recent statistical modeling strategies, averaging multiple 24HRs was often the only method to obtain an approximation of usual intakes in a population (5). Although repeated 24HRs can mitigate within-person variation, early results using traditional IA-24HR were considered disputable because of unaccounted residual within-person variation (5, 8, 36). Consistent with these previous observations, we found that proportions of participants above or below DRIs using data from 3 R24W compared with data from 1 R24W were closer to values obtained by NCI modeling. However, averaging crude data from 3 R24W yielded intake distributions that differed from NCI-modeled usual intake distribution (e.g., compared with 1×R24WNCI; ). In summary, these data suggest that mitigation of within-person variation in food and nutrient intake is related to the number of measures obtained via 24HR and not to its mode of administration (web compared with IA).

The value of using repeated 24HR or using combinations of different dietary assessment instruments to estimate usual intakes in a population has been previously documented (12, 14, 37). Carroll et al. (12) used a logistic regression calibration model to assess the added value of repeated IA-24HRs, with or without data from an FFQ, on the quality and precision of food and nutrient intake estimates. The authors reported that 4–6 repeated recalls provided optimal precision for most nutrients and food groups, and addition of data from an FFQ increased precision of intake estimates for episodically consumed food groups. Freedman et al. (37) extended the work of Carroll et al. by investigating the effect of complementing data from repeated IA-24HRs with data from an FFQ on intake estimate accuracy using unbiased intake estimates from recovery biomarkers (energy, protein, potassium, and sodium). They showed that using data from 1 FFQ with 3 or 4 24HRs improved the correlation between estimated nutrient intake and “true” longer-term usual intake compared with other combinations of instruments (i.e., more or fewer 24HR, either with or without FFQ). Using a similar calibration modeling approach, Kipnis et al. (14) found that complementing dietary intake data from 2 IA-24HRs with data from an FFQ increased the precision of the predictor variance for the foods examined compared with data derived from repeated recalls only. Larger gains in precision were observed with episodically consumed foods such as dark green vegetables and fish (14). In the current study, NCI-modeled data from repeated R24W did not yield more precise mean usual intake estimates of foods and nutrients compared with data from 1 R24W repeated in 40% of the population R24W. However, NCI modeling of intake data from 3 R24W provided more precise estimates of low and high percentiles values also compared with data from 1 R24W repeated in 40% of the population. Furthermore, in contrast with previous studies, we have shown that complementing NCI-modeled data from repeated web-based 24 recalls with data from a web-based FFQ had no impact on the precision of usual intake estimates compared with using repeated recalls only. The use of a calibration model at the individual level, the assessment of more specific foods in the study by Kipnis et al. (e.g., dark green vegetables and tomatoes), or simply the use of web-based tools may explain why our results slightly differ from results of previous studies.

One must acknowledge that within-person variation, which includes random error (i.e., precision), is distinct from systematic error (i.e., bias). In other words, although we were able to compare the precision of different combinations of web-based instruments to estimate a population's intake of specific foods and nutrients, this analysis does not provide an assessment of how close these estimates are to “true” intakes (38). Similarly, the NCI modeling method accounts for within-person variation but not systematic error. The R24W used in this study has been previously validated in the context of fully controlled feeding studies (25). We have shown that mean self-reported energy intakes were underestimated by 13.9 kcal compared with actual energy intakes. This does not exclude the possibility that the R24W is affected by systematic error to some extent. An accurate assessment of an individual's diet has been a complex challenge. Promising methods to provide an objective assessment of one's diet include metabolomic-based biomarkers (39–41) and analyses of digital pictures of meals (42).

Strengths and limitations

Strengths of this study include a decent sample size and the use of dietary assessment instruments specifically developed and validated for the population studied (25, 26). Limitations also need to be addressed. First, all questionnaires were completed within a 21-d period. Considering that an individual's diet may vary across seasons or throughout the year, a potentially low within-person variation due to short-term assessment (i.e., 21 d) may have limited our ability to detect meaningful differences. Second, no objective measure of dietary intakes such as those obtained through recovery biomarker was available in our study to assess if the use of a combination of instruments was actually more closely measuring “true” intakes. Our data suggest that complementing data from 24-h recalls with data from FFQ fails to improve precision intake estimates for daily and episodically consumed foods at the population level. However, the possibility remains that intake estimates of episodically consumed foods are less reliable due to violation of 1 assumption of the NCI method (i.e., the presence of true never-consumers). Third, a maximum of 3 R24W was available in this study, and the potential benefit of using more web-based 24HRs on precision of intake estimates remains to be determined. Last, participants in the PREDISE study were required to have access to the internet, and individuals with a high degree of education were overrepresented compared with census data of the same population (28). Diet-conscious individuals may also have been more likely than less diet-conscious individuals to participate in the study. This potential bias needs to be considered when interpreting data from this study.

In conclusion, we showed that assessing dietary intake using 3 repeated web-based 24HR, the R24W, provides more precision around intake estimates compared with estimates generated by only 1 R24W. However, when within-person variation is considered using NCI modeling, repeated R24W only influences precision of low and high percentiles of intake. Considering data from a wFFQ in addition to NCI-modeled data from repeated R24W did not influence the precision of intake estimates of most foods and nutrients, regardless of whether these were consumed daily or episodically. The use of web-based 24HRs for repeated data collection is certainly more efficient than IA instruments, but the gain in precision must be balanced out with the increased burden for participants.

Click here for additional data file.