Reproducibility and relative validity of a food-frequency questionnaire for French-speaking Swiss adults.

BACKGROUND: Due to the distinct cultural and language differences that exist in Switzerland, there is little information on the dietary intake among the general Swiss population. Adequately assessing dietary intake is thus paramount if nutritional epidemiological studies are to be conducted.
OBJECTIVE: To assess the reproducibility and validity of a food-frequency questionnaire (FFQ) developed for French-speaking Swiss adults.
DESIGN: A total of 23 men and 17 women (43.1±2.0 years) filled out one FFQ and completed one 24-hour dietary recall at baseline and 1 month afterward.
RESULTS: Crude Pearson's correlation coefficients between the first and the second FFQ ranged from 0.58 to 0.90, intraclass correlation coefficient (ICC) ranged between 0.53 and 0.92. Lin's concordance coefficients ranged between 0.55 and 0.87. Over 80% of participants were classified in the same or adjacent tertile using each FFQ. Macronutrient intakes estimated by both FFQs were significantly higher than those estimated from the 24-hour recall for protein and water, while no significant differences were found for energy, carbohydrate, fats (five groups), and alcohol. De-attenuated Pearson's correlation coefficients between the 24-hour recall and the first FFQ ranged between 0.31 and 0.49, while for the second FFQ the values ranged between 0.38 and 0.59. Over 40 and 95% of participants fell into the same or the adjacent energy and nutrient tertiles, respectively, using the FFQs and the 24-hour recall.
CONCLUSIONS: This FFQ shows good reproducibility and can be used determining macronutrient intake in a French-speaking Swiss population in an epidemiological setting.

Dietary intake ranks among the most important environmental determinants of chronic diseases (1). Among the several methods to assess dietary intake, food frequency questionnaires (FFQ) are commonly used in epidemiological studies because of their low cost and ease of use (2, 3). The FFQs allow the estimation of an individual's usual food consumption over a defined period of time, and might be sufficient for the estimation of relative risks when ranking of food and nutrient intake are sufficient to categorize participants (4, 5). However, the information collected by an FFQ is usually less accurate compared to other dietary survey methods such as food records or dietary recalls. Hence, FFQs should be checked both for their reproducibility and validity (1–3, 6).

Due to the distinct cultural and language differences that exist in Switzerland, there is little information on the dietary intake among the general Swiss population (7, 8), though a number of specialized tools for specific populations have been developed, such as for adolescents (9), elderly (6, 10, 11), and German speakers (12), or for whole-grain cereal intake (13). Furthermore, since many of the dietary assessment tools have been developed more than 15 years ago, they might no longer be appropriate for the current dietary intake of the Swiss population. With the advent of a major cohort trial underway in the French-speaking region of Switzerland (14), there is a need for an up-to-date dietary assessment tool so that food and nutrient intake can be among the parameters considered when studying health outcomes.

The primary objective of this study was to assess the reproducibility and the validity against 24-hour dietary records of a newly developed FFQ for French-speaking Swiss adults. French-speaking Swiss adults represent about 20% of the Swiss population and the only available FFQ is relatively outdated (15). The use of this new FFQ would thus allow a better assessment of dietary intake among French-speaking Swiss. It could also be applied to the German- and Italian- speaking Swiss populations given the generic availability of most of the foods included, even if there would be generally greater consumption in one part of the country compared to others.

Participants and methods

Participants

The study was approved by the Ethics Commission of Vaud Canton (protocol number 172/08) and was conducted between September and December 2008. Participants were recruited by mail and by poster advertising. All participants gave their written informed consent before starting the study. Inclusion criteria were: ages between 25 and 65 years, no illness or disease requiring a specific diet (e.g. diabetes, food allergies, renal insufficiency), not on a weight reduction diet, an adequate knowledge of French, and a good memory to perform a valid dietary recall. Although diabetics are recommended to follow ‘the same’ diet as healthy persons, we decided to exclude them on the fact that the Swiss Diabetes Association indicated that ‘type 1 diabetes needs, … a specific diet’ and that ‘the first measure to take in case of type 2 diabetes is to change one's lifestyle, by dieting (eating healthier)’ (http://www.diabetesgesellschaft.ch/fr/informations/sur-diabete/, assessed March 30, 2011), thus suggesting that diabetic subjects should have diets distinct from healthy persons.

Food-frequency questionnaire

The food frequency questionnaire was based on previously developed FFQs (6, 15), as well as elements from other FFQ designs (1). This FFQ consists of questions about 126 items: 105 on foods or specific dishes (e.g. fondue, vegetable soup, pizza), five on nutritional supplements, and 16 on beverages (alcoholic and non-alcoholic). The 126 food items were selected from existing Swiss FFQ (primarily Bus Santé, dating from 1990 (15), and an FFQ developed by Wynn-Dumartheray et al. (6) for elderly subjects in the French speaking region of Switzerland). These were cross-checked with the ‘major’ FFQs used in the United States to ensure completeness in terms of food groups. Twenty-four hours recall data confirmed that the FFQ covered the main foods consumed in the region. Foods were classified into 10 groups: dairy products, bread, and cereals; spreads; meat and protein-rich foods (including eggs and tofu); fish and seafood; vegetables; starchy foods (including potatoes and rice); fruits (including berries, canned, dried or in syrup); pastries, sweets and snacks; spices and sauces. For each item, subjects were asked to indicate their intake over the past month, and there were eight frequencies of consumption ranging from ‘never/less than once per month’ to ‘5+ per day.’

In Switzerland, sales of fruit drink (normally <25% fruit juice) are high, and there is considerable confusion between fruit juice and fruit drink. As there was a risk in greatly overestimating fruit intake if ‘fruit juice’ was included as fruit (due to the potentially high volumes consumed), juice was included as a beverage. Supplements were selected as the most common supplements taken – including vitamins C and E, generic multivitamins, and fish oil. The FFQs were filled manually and the data was also handled (keyed in) manually, as our previous experience in another study with optical reading showed an unacceptably high rate of errors.

Twenty-four hour dietary recalls

Participants were surveyed by a single research dietician. The interviews were chronologically based; that is, from the first to the last meal of the previous day. Participants were asked everything they drank and ate the previous day. Food portion sizes were estimated using standard household measures such as cups and spoons and a French food and beverage photo manual (16). While the FFQ gave precise and fixed portions (the same for everyone), during the 24-hour recall, the exact portion was defined with each subject. Participants were asked to be as specific as possible about the type of food and beverage consumed (e.g. skimmed or non-skimmed milk), including type of cooking method used (e.g. roasted, boiled, etc.). When subjects reported eating a personal recipe, the dietician asked the subject complete information on each individual food item (quality and quantity). For example, if the subject reported eating homemade lasagna, then all the different items were calculated separately (pasta, tomato sauce, onions, hard cheese, minced beef, and olive oil). Weight and height in light clothes were measured and the participants were also asked about current tobacco smoking. Twenty-four hour dietary recalls were performed the day after the participants had filled the FFQ, a protocol also used by others (17).

Energy and nutrient intake estimation

Energy and macronutrient intake from the 24-hour dietary recalls was estimated using the PRODI food composition database (Nutri-Science GmbH, Hausach, Germany), which also includes information from the Swiss national nutrient database (http://www.swissfir.ethz.ch/index_EN). Energy and macronutrient intake from the FFQ was estimated using a composite nutrient database based on the Swiss nutrient database and supplemented with data from the French, German, and UK databases when data was not available or appropriate. Notably, cooked meat products are not yet included in the Swiss database, so these values were taken from the databases of neighboring countries. Where a food item included several different foods or types of preparation, all foods were included and an average was taken.

Statistical analysis

Statistical analysis was conducted using Stata version 9.2 (Statacorp, College Station, TX, USA). Descriptive results were expressed as number and (percentage) or as mean±standard deviation for nutrient intake. Reproducibility between the first and the second FFQ was assessed by intraclass correlation coefficients (ICC), where a higher value indicates a lower within-person variation. The natural logarithm of nutrient intake was used to achieve normal distributions (17). Validity of the FFQ was assessed as follows: Pearson's correlation between the FFQ and the 24-hour recall estimations of nutrient intake were calculated. Between and within -person daily variances for each nutrient were computed. The corresponding de-attenuated Pearson's coefficients were then calculated by multiplying the initial Pearson's coefficient by , as the 24-hour recall was repeated twice. Lin's concordance correlation was also computed. It measures how well a new set of observations reproduces an original set and has been reported to be more appropriate than other indices for measuring agreement when the variable of interest is continuous (18). It has also been shown to be robust on as few as 10 pairs of data. Participants were also classified into tertiles of energy and nutrient intake, based on the FFQ and the average of the 24-hour recalls. Proportions of participants classified into the same, adjacent, and extreme tertiles by FFQ and 24-hour recalls were then calculated for energy and each nutrient.

Results

Forty adults (23 men, 17 women, mean age 43.1±2.0 year, mean BMI 24.1±3.6 kg/m2) participated in the study. Participants took on average 22±10 (range: 10–60) min to fill out the FFQ, and this value decreased between the first and the second evaluation: 25±11 versus 20±8 min, respectively (p<0.001). Over 90% of participants considered the questions as easy (93% in the first and 95% in the second evaluations, p=0.64).

Reproducibility

The mean macronutrient intakes of the 24-hour dietary recalls, first, and second FFQ are presented in Table 1). The absolute values estimated by the second FFQ were significantly lower than those estimated by the first FFQ for energy, protein, and carbohydrates, while no differences were found for the other nutrients. Excluding a subject with a very high protein intake did not change the results (data not shown). Crude Pearson's correlation coefficients between the first and the second FFQ ranged from 0.58 for water to 0.90 for alcohol. The ICCs ranged between 0.53 for water and 0.92 for alcohol, and Lin's concordance ranged from 0.55 for water and 0.87 for alcohol. When participants were classified into tertiles of macronutrient intake, the proportion of classification into the same tertile ranged from 40% for carbohydrates to 77.5% for alcohol. On average, over 95% of the participants fell into the same or the adjacent categories (Table 2). Non-parametric correlation coefficients using non-transformed data showed that the values ranged between 0.57 for water and 0.85 for polyunsaturated fatty acids and alcohol.

Table 1

Mean daily intakes of energy and macronutrients estimated from two 24-hour recalls and the FFQs

	Daily intake			Reproducibility
	24 Hour	FFQ1	FFQ2	FFQ2 (% FFQ1)	R	Lin's C	ICC
Energy (kcal)	2022 (634)	2162 (712)	1952 (627) a	90	0.82	0.79	0.72
Protein (g)	71.4 (28.8)	94.3 (36.7)*	83.7 (30.2)*a	89	0.71	0.68	0.60
Carbohydrate (g)	238 (80)	236 (82)	212 (63)*a	90	0.81	0.79	0.68
Fat
Total (g)	76.7 (38.0)	82.2 (33.5)	74.6 (27.1)	91	0.79	0.77	0.67
Saturated (g)	28.1 (18.6)	29.6 (12.4)	26.9 (11.3)	91	0.75	0.73	0.67
Polyunsaturated (g)	22.1 (13.7)	23.4 (10.6)	21.2 (8.7)	91	0.83	0.81	0.74
Monounsaturated (g)	26.5 (13.9)	29.2 (11.9)	26.5 (9.2)	91	0.78	0.76	0.64
Cholesterol (mg)	250 (274)	275 (139)	243 (100)	88	0.74	0.72	0.63
Alcohol (g)	12.2 (32.2)	15.4 (23.5)	15.2 (21.2)	99	0.90	0.87	0.92
Water (mL)	2491 (789)	3015 (783)*	2786 (806)*	92	0.58	0.55	0.53
Fiber (g)	17.9 (5.6)	23.6 (10.7)*	21.1 (8.3)	96	0.78	0.78	0.73

Correlation coefficients computed using log-transformed data. Results are expressed as mean (standard deviation).

FFQ, food-frequency questionnaire; R, Pearson correlation coefficient; C, Lin's concordance correlation coefficient; ICC, intraclass correlation coefficient.

Statistical analysis by Student's paired t-test on log-transformed values between each FFQ and the 24-hour report:

p<0.05.

Statistical analysis by Student's paired t-test on log-transformed values between each FFQ:

p<0.05.

Table 2

Reproducibility of the food frequency questionnaires

	FFQ1 versus FFQ2
	% in the same tertile	% in adjacent tertiles	% in extreme tertiles
Energy (kcal)	55.0	45.0	0
Protein (g)	52.5	45.0	2.5
Carbohydrate (g)	40.0	55.0	5.0
Fat
Total (g)	65.0	35.0	0
Saturated (g)	52.5	45.0	2.5
Monounsaturated (g)	55.0	45.0	0
Polyunsaturated (g)	70.0	30.0	0
Cholesterol (mg)	55.0	45.0	0
Alcohol (g)	77.5	20.0	2.5
Water (mL)	60.0	30.0	10.0
Fiber (g)	57.5	42.5	0

Results are expressed as percentage of participants classified in the same, adjacent, or extreme tertiles of nutrient distribution by the first and the second FFQ (n=40).

FFQ, food-frequency questionnaire.

Validity

Macronutrient intakes estimated by both FFQ were significantly higher than those estimated from the 24-hour recalls for protein and water, while no significant differences were found for energy and the other macronutrients (Tables 1 and 3). Crude Pearson's correlation coefficients between the 24-hour recall and the second FFQ tended to be higher than with the first FFQ (Table 3). The crude Pearson's correlation coefficients ranged between 0.23 and 0.44 for the first FFQ and between 0.32 and 0.57 for the second FFQ. De-attenuated correlation coefficients were higher, but the improvement was modest as the variance ratios were small (Table 3). Lin's concordance correlation coefficients were also higher between the second FFQ and the 24-hour recall than between the first FFQ and the 24-hour recall (Table 3); for FFQ2, the values ranged between 0.31 for cholesterol and 0.54 for water. The classification into tertiles gave similar results for both FFQ: for FFQ1, an average of 43 and 85% of participants classified in the same or adjacent tertile, respectively, the corresponding figures for FFQ2 being 46 and 88% (Table 4).

Table 3

Validity of the FFQs versus the average of two 24-hour recalls

		FFQ1 versus 24-hour				FFQ2 versus. 24-hour
	Variance ratioa	FFQ1 (% 24 hour)	R	C	Rde-att	FFQ2 (% 24 hour)	R	C	Rde-att
Energy (kcal)	0.33	107	0.31	0.29	0.33	97	0.38	0.37	0.41
Protein (g)	0.45	132	0.42	0.34	0.47	117	0.36	0.33	0.40
Carbohydrate (g)	0.37	99	0.23	0.22	0.25	89	0.36	0.33	0.39
Fat
Total (g)	0.48	107	0.41	0.40	0.46	97	0.43	0.43	0.48
Saturated (g)	0.67	105	0.35	0.33	0.40	96	0.40	0.38	0.46
Monounsaturated (g)	0.52	110	0.44	0.42	0.49	100	0.47	0.45	0.53
Polyunsaturated (g)	0.60	106	0.41	0.40	0.47	96	0.32	0.32	0.36
Cholesterol (mg)	0.75	110	0.37	0.24	0.43	97	0.37	0.31	0.43
Alcohol (g)	0.51	126	0.39	0.37	0.44	125	0.34	0.30	0.38
Water (mL)	0.18	121	0.30	0.25	0.31	112	0.57	0.54	0.59
Fiber (g)	0.82	132	0.38	0.31	0.45	118	0.57	0.53	0.68

Correlation coefficients computed using log-transformed data.

R, Pearson correlation coefficient; C, Lin's concordance correlation coefficient; Rde-att, FFQ and 24-hour de-attenuated Pearson's correlation coefficient.

Ratio between within-person and between-person daily variances computed with the two 24-hour recall.

Table 4

Validity of the FFQs versus the average of two 24-hour recalls

	FFQ1 versus 24-hour			FFQ2 versus 24-hour
	% in the same tertile	% in adjacent tertiles	% in extreme tertiles	% in the same tertile	% in adjacent tertiles	% in extreme tertiles
Energy (kcal)	47.5	30.0	22.5	42.5	40.0	17.5
Protein (g)	42.5	45.0	12.5	42.5	50.0	7.5
Carbohydrate (g)	27.5	50.0	22.5	45.0	35.0	20.0
Fat
Total (g)	45.0	40.0	15.0	42.5	45.0	12.5
Saturated (g)	45.0	40.0	15.0	42.5	45.0	12.5
Monounsaturated (g)	42.5	45.0	12.5	37.5	50.0	12.5
Polyunsaturated (g)	42.5	45.0	12.5	42.5	45.0	12.5
Cholesterol (mg)	35.0	50.0	15.0	50.0	35.0	15.0
Alcohol (g)	67.5	25.0	7.5	60.0	30.0	10.0
Water (mL)	37.5	50.0	12.5	52.5	45.0	2.5
Fiber (g)	27.5	60.0	12.5	45.0	50.0	5.0

Results are expressed as percentage of subjects within the same, adjacent, or extreme tertiles.

Discussion

In this study, we assessed the reproducibility and relative validity for energy and macronutrient intake of a FFQ designed for French-speaking Swiss adults. The objective was to obtain an FFQ that could be easily and quickly filled out by a majority of adults. The length of the FFQ (126 items) is similar to other FFQs used for epidemiological studies (6, 17, 19). The average time for filling out the FFQ was less than half an hour, so we believe that this FFQ can be easily applied in an epidemiological setting.

The reproducibility of FFQ was good considering the observed correlation coefficients and ICC being either higher than (17, 19) or within values reported from the literature (20) (Table 5) for FFQ validated with multiple 24-hour recall.

Table 5

Comparison of the reproducibility (R and ICC) between this study and the literature

	Current study		Study 1 (17)	Study 2 (20)		Study 3 (19)
	R	ICC	ICC	Spearman	ICC	R
Energy (kcal)	0.82	0.72	0.60	0.71–0.54	0.84–0.73	0.35
Protein (g)	0.71	0.60	0.39	0.63–0.60	0.77–0.75	0.49
Carbohydrate (g)	0.81	0.68	0.70	0.67–0.67	0.79–0.78	0.24
Fat
Total (g)	0.79	0.67	0.55	0.67–0.64	0.80–0.78	0.56
Saturated (g)	0.75	0.67	0.55	0.65–0.66	0.78–0.80	NA
Monounsaturated (g)	0.78	0.64	0.45	0.65–0.66	0.73–0.72	NA
Polyunsaturated (g)	0.83	0.74	0.57	0.57–0.55	0.72–0.73	NA
Cholesterol (mg)	0.74	0.63	0.48	0.66–0.60	0.78–0.75	0.58
Alcohol (g)	0.90	0.92	0.83	NA	NA	NA
Water (mL)	0.58	0.53	NA	NA	NA	NA
Fiber (g)	0.78	0.72	0.51	0.70–0.72	0.83–0.85	0.42

Study 2: first value men, second value women; NA, not available.

Balancing the reference method with the period covered by the FFQ is a critical factor in the study design, with a need to not have the repeated 24-hour recalls too close together to avoid subjects remembering their previous answers, but not too far apart, to avoid seasonal changes in diet and to keep within the period defined by the FFQ. The time intervals used by other authors vary considerably, from 15 days (21) to 1 year (17, 20, 22–24). Food intake also shows yearly trends (25–27), and we believe that the 1-month period between the two questionnaires is an adequate time interval, to avoid undue influence of seasonal and/or yearly variations. In agreement with some studies (17), it was decided to apply the 24-hour recall the day after the FFQ. This methodology precludes the FFQ to cover the period of the 24-hour recall, thus reducing validity; hence, it is likely that our validity measures could have been better had the 24-hour recall be conducted before the application of the FFQ. Finally, the close timing between the FFQ and the 24-hour recall might lead to interference, unless there is a very long time between measurements – in which case the same period is not being measured.

A higher agreement between the second FFQ and the 24-hour recall has been reported in several studies (2, 17, 22). This could be due to the fact that filling out the first FFQ actually made the participants more conscious of their dietary intake, thus leading to a better reporting of their dietary intake. In this study, the agreement between the first and the second FFQ with the 24-hour recall were rather close, probably because the rather short time between the two assessments did not allow the participants to change significantly their dietary intake. Overall, the agreement was slightly lower than reported in other studies (28, 29), but comparable to others (17, 19, 30). Furthermore, around 80% of the participants were classified in the same or adjacent tertile according to their energy or macronutrient intake, the values being slightly higher for the second FFQ.

The relatively small number of participants and number of reference replicates used in this study are possible limitations. It has been suggested that the minimum sample size for validation studies is 100, namely when the number of replicates of the reference method is small (31). Power analyses indicated that the minimum sample size need to obtain a significant correlation of 0.4 between the FFQ and the 24-hour recall was 30 (17, 19, 32), a value below our sample size of 40. In terms of the number of repeated 24-hour recall measurements needed, several studies have shown that an adequate statistical efficiency is obtained with only two replicate measures to remove the effects of within-person variation (33, 34). Hence, we believe that the two 24-hour replicates were enough to adequately assess the attenuation coefficient in this study.

While overall correlations between the FFQ and 24-hour recalls were acceptable, a probable reason for some of the difference observed is that different nutrient databases were used to calculate the nutrient intakes for the FFQ and 24-hour recall. In constructing the composite nutrient database for the FFQ, when a food item was not adequately covered by the Swiss nutrient database, appropriate foods were substituted from other nutrient databases, and cross-checked to ensure that they were within a probable range for that nutrient. For example, protein in meat differed between the composite database designed for this FFQ and the Swiss nutrient database used for the 24-hour recall, due to cooked meat values being used for the FFQ, but raw meat being used for the 24-hour recall (data not shown). In addition, for some nutrients, missing values were evident in the Swiss database, which were completed in the composite database. Due to the software involved, it was not possible to use the composite nutrient database for the 24-hour recall calculations. Further, it was not possible to have had an exhaustive list of all vitamin supplements available in Switzerland, so a precise estimation of micronutrient intake could not be performed. Although only one participant (2.5%) reported taking vitamin supplements, previous data suggests that the prevalence of nutritional supplement use in Switzerland is considerably higher (35). Hence, further efforts are needed to better assess vitamin and mineral intake in the Swiss population, as this could not be achieved with the current FFQ.

The FFQ was initially designed for specific studies taking place within the French-speaking region. However, the food available in Switzerland is more or less homogenous (from supermarket data), though there are differences in the proportions of different foods eaten. It is thus possible that this FFQ could be applied to the German- and Italian-speaking Swiss populations given the generic availability of most foods included in the FFQ, even if there would be generally greater consumption in one part of the country compared to others.

Conclusions

This FFQ is a valid instrument for determining macronutrient intake in a French-speaking Swiss population in an epidemiological setting, with comparable results to other FFQ validation studies.