Validity and reproducibility of a food frequency questionnaire to assess dietary nutrients for prevention and management of metabolic syndrome in Korea.

Little data exists on metabolic syndrome (MetS) related with intake, especially for the South Korean. The purpose of this study was to develop and evaluate a food frequency questionnaire (FFQ) for nutritional assessment in the population with MetS in South Korea. Randomly selected female participants, mean age 21.9 years (n = 38) were invited to answer the FFQ twice (FFQ1 and FFQ2) over a nine-month interval and to complete twelve-day diet records (DR) during the months between in South Korea. The correlation coefficients for nutrient intake between FFQ1 and FFQ2 varied from 0.253 (niacin) to 0.573 (cholesterol), and the energy intake-adjusted correlation coefficients ranged from 0.187 for protein to 0.662 for iron. The energy intake-adjusted and de-attenuated correlation coefficients for comparison of FFQ1 and the DRs ranged between 0.277 (vitamin B1) and 0.768 (fiber), and between 0.229 (zinc) and 0.859 (fat) for comparison of DRs with FFQ2. The percentages of study subjects classified into the same quartiles in both the DRs and FFQ1 ranged from 15.8% (vitamin B(6)) to 47.4% (calcium), and for the same quartiles in DRs and FFQ2 from 13.2% (vitamin B(1)) to 44.7% (potassium). The FFQ has reasonably good validity and reproducibility. Further research is needed for an assessment of reproducibility and validation of present FFQ in the subjects with MetS.

Introduction

Recently, the reported incidence of metabolic syndrome (MetS) has increased throughout the world, especially in non-Western Asian regions [1]. In South Korea, the prevalence of MetS in 2005 was estimated to be 27.1% for men and 25.6% for women by the Korean National Health and Nutrition Examination Survey (NHANES), by International Diabetes Foundation (IDF) criteria [2]. Although the intakes of various nutrients such as carbohydrates, dietary fiber, and fat have been related to individual components of MetS [3-5], the role of diet in contributing to MetS is not well understood. Because important variables in MetS such as type 2 diabetes mellitus and cardiovascular diseases have long been associated with various dietary factors, it is important to accurately examine the dietary factors connected with MetS.

Accurately estimating long-term food intake habits is essential in diet-disease research. The Food Frequency Questionnaire (FFQ) has been widely used as an epidemiological tool to investigate the association between diet and chronic diseases [6]. The use of inappropriate food lists in the FFQ, however, may result in underestimation of nutrients due to the omission of key items [7]. It has been suggested that the FFQ may need to be validated against target populations because dietary habits vary greatly according to the ethnic, social, and cultural backgrounds of participants [8]. Therefore, validation of the FFQ method is essential, as the use of an FFQ with low validity may result in false associations between dietary factors and diseases or disease-related markers [9]. In particular, it is necessary to develop a specific FFQ for the Korean MetS population because it is thought that MetS may be influenced by relatively varied and complex Korean meals such a bowl of cooked rice with a seasoned mixed soup and multiple side dishes.

In the mean time, a number of FFQs have been developed and used for the last several years in Korea for studies on chronic diseases [10,11]. However, it is rare to find an FFQ that was developed through in-depth analysis of the reliability, validity and seasonal differences of FFQ. This study, as the first step to preparing preventative and management plans for metabolic syndrome in Korean people, aimed at developing FFQ, which is an important tool for accurate nutritional intake assessment of metabolic syndrome patients.

Methods to evaluate and interpret the validity and reproducibility of FFQs have tended to depend on correlation analysis of nutrients and/or foods measured by two or more dietary assessment methods [12,13]. FFQs and diet records (DRs) are practical tools for nutrition investigation [14,15]. Although there is no perfect measure of dietary intake, multi-day DRs extending across all four seasons may be superior to FFQs and have frequently been used as the reference method in validation studies [16]. DRs, which are not dependent on subjects' recall of food items as are FFQs, appear to be a suitable method for validating the performance of FFQ in that errors inherent in these two methods are not correlated [17].

The main objective of this study was to validate and reproduce an FFQ which reflects dietary intakes of the population with MetS. The secondary objective is to develop an FFQ as a nutritional assessment tool for prevention and/or management of MetS, using intakes measured by twelve-day DRs as the reference standard.

Subjects and Methods

Study design and subjects

The target of present study was set as young females of whom it has been reported that dietary intake survey was most accurate. The subjects were 38 female volunteers aged 20-29 years. The mean age of the subjects was 21.9 years old and the body mass index was 20.4. All participants, who were healthy without any disease and who did not take any medications, gave consent to participate in this study. The 100-item FFQ was administered twice at nine-month intervals (the first FFQ [FFQ1] at the beginning and the second FFQ [FFQ2] at the end of the study), and DRs were collected for 3 non-consecutive days during each of the four seasons from March to December 2008.

Food frequency questionnaire

We developed a 100-item FFQ to evaluate the usual dietary intakes of Korean adults with MetS. The selection of the foods (94 items) and dishes (6 items) listed on the FFQ was based on analysis of NHANES results [18]. One hundred foods and dishes which are highly consumed by frequency and amount, or which contribute substantially to nutrition intake in adults, were chosen based upon the food intake amount, frequency, and consumption type of NHANES results. The questionnaire asked about the average frequency and amount of consumption of each food. The food items were listed in food and dish groups: cereals-14 items; potatoes and starches-1; soups-6; sugars-2; legumes-3; vegetables-22; fruits-7; meats and meat products-8; eggs-1; fish and other seafood-12; milk and dairy products-4; beverages and soft drinks-11; oils and fats-6; and seasonings-3. The FFQ used in this study requested participants to estimate their food consumption frequency for each item as nine categories: never or seldom, once a month, 2-3 times a month, one to two times a week, three to four times a week, five to six times a week, once a day, twice a day or three times or more every day [10]. The portion sizes were set as follows: a 1/2 serving size, a serving size, and a 1.5 serving size. One serving size and recipes for 6 dish items of FFQ were based on the basic recipes of CAN-Pro. Color photographs of median-sized food portions were used to improve the accuracy of estimates.

Diet records

As a reference method to assess the validity of this FFQ, we collected information on twelve-day DRs over nine months for each participant. Participants were asked to keep non-consecutive three-day DRs and to include one weekend day or holiday during each of the four seasons in order to capture seasonal and day-of-week variations in food intake. We asked them to provide detailed descriptions of each food, including the weights prepared and proportions consumed. A research dietitian checked the records in a standardized way after completion by the participants.

Statistical analysis

All analyses were done using SAS, version 9.01 (SAS Institute Inc., Cary, NC, USA). Descriptive statistics were computed to describe demographic characteristics and average daily nutrient intakes. Statistical analysis was restricted to twenty nutrients common to the FFQ and diet record databases. Nutrient intake for each food item was calculated based on the consumption frequency and the portion size of each food item. The various nutrient intakes were calculated from the FFQs and DRs by a computer-aided nutrient analysis program for professionals (CAN-Pro 3.0, APAC Intelligence, Seoul, South Korea). The validity and reproducibility of the FFQ were assessed by comparing nutrient intakes, correlations between the intakes derived from the two different dietary survey methods (FFQ versus DRs) and between the two different surveys (baseline versus second FFQ), and agreement of quartile distributions of nutrients. Differences found in each comparison were presented as percentages of the consumption from the DR and were tested by use of a paired t-test. Correlations of nutrient intakes between the DR and FFQ methods were assessed by Pearson's correlation coefficient. To improve normality, the distributions of some abnormally distributed nutrients were log-transformed before analysis. Energy-adjusted nutrient intakes were calculated as residuals from regression analyses, with nutrient intake as the dependent variable and energy as the independent variable [19]. To correct the intra-subject error in the measurement of the DRs, the observed correlation was multiplied by the de-attenuated factor (1+γ/n)½, where γ is the ratio of the intra- and inter-subject variances and n is the number of repeats (here n = 12). Intra- and inter-subject variances were calculated using the SAS Varcomp procedure [6,20]. To measure the degree of agreement, subjects were classified into quartiles based on the nutrient intakes from the two methods, and the percentages of agreement and of complete disagreement were determined. The reproducibility of two FFQs administered nine months apart was represented by Pearson's correlation coefficient.

Results

Daily nutrient intakes

The mean daily nutrient intakes of the twelve-day DRs and FFQs are presented in Table 1. The absolute values estimated by two methods differed for some nutrients. Energy, carbohydrate, dietary fiber, vitamin B1, folate, calcium, phosphorus, and potassium intakes were reported as higher in both FFQs than in DRs. For protein, vitamin B6, and cholesterol the estimated consumptions in FFQ1 were higher than in the DRs, whereas that of vitamin C was lower. For FFQ2, vitamin E was reported lower than in the DRs. The estimated consumptions of vitamin C and E were different between the two FFQs.

Table 1

Mean daily intakes of energy and nutrients estimated from four 3-day DRs and the FFQs

aP < 0.05 for the differences between DR and either FFQ

bP < 0.05 for the differences between FFQ1 and FFQ2

Reproducibility of the FFQ

Pearson's correlation coefficients for comparison between the two FFQs administered at a 9-month interval are shown in Table 2. The correlation coefficients for nutrient intakes varied from 0.253 (niacin) to 0.573 (cholesterol), and the energy intake-adjusted correlation coefficients ranged from 0.187 for protein to 0.662 for iron.

Table 2

Correlation coefficients for comparison between the first and second FFQs

*P < 0.05, **P < 0.01

Validity of the FFQ

Table 3 presents the results of correlations between nutrients intakes obtained from the FFQs and the DRs. Log-transformed Pearson's correlation coefficients for nutrients ranged from 0.216 (protein) to 0.544 (folate) between FFQ1 and the DRs, and from 0.206 (zinc) to 0.388 (fat) between FFQ2 and the DRs. For FFQ1, the energy intake-adjusted correlation coefficients ranged between 0.231 (vitamin B1) and 0.647 (folate), and the de-attenuated correlation coefficients ranged between 0.277 (vitamin B1) and 0.768 (dietary fiber). For FFQ2, the energy intake-adjusted correlation coefficients ranged from 0.210 for zinc to 0.745 for fat and the de-attenuated correlation coefficients ranged from 0.229 (zinc) and 0.859 (fat). De-attenuated correlation coefficients for some nutrients showed little improvement because the ratio of intra- to inter-subject variability was low.

Table 3

Correlation coefficients between nutrients as estimated by twelve-day DRs and by FFQs

1)Nutrient values were log-transformed to improve normality.

2)Nutrient intake were adjusted for energy intake by the residual method.

*P < 0.05, **P < 0.01, ***P < 0.001

Subjects were classified into quartiles according to their nutrient intakes estimated from the DRs and FFQs (Table 4). The percentage of subjects classified into the same quartiles in both DRs and FFQ1 ranged from 15.8% (vitamin B6) to 47.4% (calcium). On average, more than 60% of the subjects fell into the same or adjacent categories. Except for protein, vitamin B1, vitamin E, and cholesterol, the proportion of subjects classified into opposite quartiles in the DRs and FFQ1 was below 8% for all nutrients. The percentage of subjects classified into the same quartiles in the DRs and FFQ2 ranged from 13.2% for vitamin B1 to 44.7% for potassium, and as with FFQ1 more than 60% of the subjects fell into the same or the adjacent categories. Except for vitamin B, vitamin E, iron, and zinc, the percentage of subjects classified into opposite quartiles in the DRs and FFQ2 was below 8% for all nutrients.

Table 4

Percentages of agreement, adjacent agreement and complete disagreement according to quartile classification of nutrients intakes based on FFQ and four 3-days DRs

Discussion

To examine the validity and reproducibility of results for a 100-food item FFQ for MetS investigation, we carried out a relative validity and reproducibility study of energy intake, 19 macro- and micro nutrients measured with our FFQ and compared against reference values from twelve-days DRs in 38 healthy women.

The reproducibility of FFQs has generally been assessed by administering them at two points in time to the same group of people, then using correlation coefficients to assess the association between the two responses [21,22]. An important factor influencing reproducibility is the time interval between the two FFQs- if the interval is short the interviewer may recall the first FFQ and reproducibility will be overestimated, and if the interval is long the dietary pattern may change, resulting in a large error in reproducibility. In the present study, the time difference between FFQ 1 and FFQ 2 was nine months and this was to observe seasonal difference between spring (March) and winter (December). Published reports with similar time intervals between two questionnaires yielded relatively higher correlation coefficient ranges [23]. In our study, however, parameters such as gender composition of the subject group and number of food items were very different. Since there were a relatively large number of nutrients that displayed significant relativity in correlation analysis, it is concluded that this FFQ can be used in nutritional intake assessment regardless of season. However, there was a limitation in this study that analysis of differences among four seasons with the interval of 3 ~ 4 months was not conducted.

One might predict better correlations between FFQ2 and DRs than between correlations between FFQ1 and DRs because of a training effect [24]. Our study, however, did not show that result, and some previous studies have shown that a second FFQ produced lower nutrient estimates. The reason for this observation has not yet been elucidated [10,25].

Most methods to assess and interpret the validity and reproducibility of FFQs have relied on correlation analysis of nutrients measured by two or more dietary assessment methods. Comparisons of percentage agreement in quartile distributions are also often used in the evaluation of the reliability of an FFQ.

Masson et al. [26] suggested that for studies designed to establish the validity of a dietary assessment tool for a range of future epidemiological studies, the use of correlation coefficients above 0.5 is desirable in epidemiological studies. In the present study, some nutrients did not reach that threshold. Our correlation coefficients were, however, similar to those summarized in a review of the validation studies of FFQs using 24-hour recalls as a standard, which ranged from 0.10 to 0.89 [6]. Also, it was reported in a study by Ogawa et al. [12] that with adjustment for total energy and de-attenuation for measurement error with the DRs, correlation coefficients for nutrient intakes ranged from 0.25 from 0.69. Other Korean studies have reported correlations ranging from 0.16 to 0.71, with most in the range 0.3 to 0.5 [11,27,28]. Ahn et al. [10] examined the validity of nutrients assessment by a 103-item FFQ and obtained a range of correlation coefficients against DRs of 0.23 (vitamin A) to 0.64 (carbohydrate), which is similar to our results.

According to previously reported data about validity research in Korea, the observed correlations of nutrient intakes appear to be lower than that reported in western countries. In Masson and co-workers' [26] validity study of a 150-item FFQ with forty female participants, reported correlations between the DRs and FFQ after energy adjustment ranged from 0.37 to 0.84, and several Latin American and European studies obtained validities with similar ranges of correlation coefficients [14,29,30]. The reason for a lower correlation of FFQs in Korea may be related to the types of food items included in the FFQs. The present FFQ was developed mostly based on individual food items (94 items), not on prepared dishes. The seasonings and cooking oils omitted in food-based FFQs might well affect not only the difference of absolute intakes of some nutrients but also the correlation between a structured questionnaire (FFQ) and an open questionnaire for detailed information (DRs). However, the foods listed in this FFQ were selected on the basis of each food's contribution to total dietary intake as indicated by NHANES results. Additionally, we found no evidence that a dish-based FFQ had more precision in assessing dietary intakes, as we had no standard recipe or recipe data for the dishes.

Our findings also indicated that nutrient intakes calculated by FFQ tended to be higher than those calculated by the average of DRs. This finding was consistent with the results of most other studies [10,26,31]. A possible explanation is that subjects may have included some food items more than once when they ate the foods in a mixed dish. Also, a limitation of FFQs is that subjects might overestimate food intake, in part because the answers are taken as multiple choices of frequencies at discrete intervals [6].

Specifically, carbohydrate consumption as assessed by the FFQs was higher than in the DRs. It may be overestimated in the FFQ because rice is the staple food of Korea and grains and noodles were also included among the food items. An overestimation of carbohydrates has also been found in other reports of validation studies of Korean FFQs [10,28].

Subjects were classified into quartiles by their nutrient intakes as estimated from the DRs and FFQs (Table 4). In this study, the average percentage of exact agreement on quartile classifications between the two assessments was about 30%, with more than 60% of the responses classified into the same or adjacent quartiles. Other reliability studies in Korea comparing 24-hour recall and FFQs presented average exact agreements of 28%, which is slightly lower than in our results [32].

In our study, we present reproducibility and validation results for a FFQ for MetS. Relative validity values were rather low for several nutrients, but satisfactorily high figures were obtained with most nutrients for our FFQ. The overall performance of our FFQ appears to be reasonably acceptable for application to a study requiring long-term dietary assessment, as for MetS in the Korean population. However, further research is needed for an assessment of reproducibility and validation of our FFQ in the subjects with MetS. With these results, present FFQ is useful for assessing the normal intakes of nutrients in the population with metabolic syndrome in South Korea.