Development and validity of semi-quantitative food frequency questionnaire as a new research tool for sugar intake assessment among Indonesian adolescents.

Robust evidence has shown that sugar is a major contributor to obesity and Non-Communicable Diseases (NCDs). However, there have not been sufficient tools to estimate sugar intakes. Therefore, developing a new and valid tool to assess sugar intake, based on cultural eating habits, is crucial. The study was done in two phases; the first focused on the development of Semi-quantitative Food Frequency Questionnaire (SFFQ), and the second focused on researching the validity of the questionnaire. Food items in the SFFQ were selected from the latest national survey review, exploratory survey, and food market observation. Forty-nine food items were included in the final SFFQ with five open-ended questions for fruit groups. One hundred and six adolescents aged 15-17 years participated in the study. The total sugar intake among the adolescents was 58.80 g/day (52.7 g sucrose; 1.47 g fructose; 1.49 glucose) which contributed to 11.6% of the total energy intake per day. The reliability analysis showed a good agreement between the two administered SFFQs in a one-month interval. The relative validity results, using 6-days food diaries as a reference method, demonstrated a superior ability to rank individuals into the same and adjacent classification and only <10% gross misclassification in all sugar intakes. The developed SFFQ in turn has been proven to have moderate to good validity and be applicable for a larger epidemiological study.

Introduction

Nutrition-related non-communicable diseases (N-NCDs) are currently becoming a major cause of worldwide morbidity and mortality reported 75% higher than the proportion of mortality due to malnutrition and infectious disease by 2020 [1]. At least 59.5% deaths in Indonesia were caused by NCDs, and mostly one in two deaths was due to cardiovascular diseases. Surprisingly, the prevalence of obesity in adolescents increased from 1.4% in 2010 to 7.3% in 2013, while in adults it increased from 19.7% in 2007 to 26.3% in 2013 [2]. Adolescent obesity needs to be greatly considered because 80% of obese adolescents potentially continue until they are in adulthood [3]. Obesity is a dominant factor of metabolic syndrome [4] and an elevated risk of mortality and premature death [5].

Some reviews and meta-analyses have revealed that sugar had a major contribution to the development of non-communicable diseases (NCDs) including obesity, type 2 diabetes mellitus, hypertension, cardiovascular diseases, and metabolic syndrome [6,7]. Higher consumption of sugar-sweetened beverages (SSBs) in adolescents may also be a predictor of cardio-metabolic risk in adults [8]. Malik et al [9] emphasized both incomplete compensation of liquid calories in SSBs and also increasing fructose metabolism that caused higher glycemic load to promote obesity. Moreover, an increase in fructose intake also led to elevated triglyceride and visceral adipose tissue (VAT) through de-novo lipogenesis. The high glycemic load of fruit juice intakes also affected postprandial blood glucose which in long term induced type 2 diabetes risk [10].

Among Asian countries, Indonesia counts as the third largest user of sugar-consuming countries after India and China [11]. USDA [12] reported white sugar consumption of the Indonesian population increased to 7.15 million metric tons (MMT) over 2019/20 from 7.05 MMT in 2018/19 and would continue increasing to 7.2 MMT in 2021. The National Socio-Economic Survey also reported the visible sugar consumption in Indonesia was 28.2 gr in 2002, 26.2 gr in 2007, and 23.8 in 2009, respectively [13]. The visible sugar consumption of the Indonesian population was slightly higher than 5% of total energy. It is predicted to be higher if combined with invisible sugar consumption from soft drinks or any other beverages. Moreover, the trend of soft drink consumption is increasing with the increase of income [14]. However, the National Socio-Economic Survey did not use direct dietary assessment in assessing common sugar intake but calculated monthly expenditure for sugar consumption, and thus it might be quite unreliable.

In Indonesia, the latest study in 2014 on 24-hour food recall of household in a day assessed general a daily sugar intake which is somehow questionable. It might not represent the real sugar intake among the population since assessing sugar intake is relatively difficult and commonly under reported especially in a population with lower literacy and numeracy [15]. Moreover, there were no reported data about mean daily intakes of different types of sugar from sugary foods and beverages in the Indonesian population. Thus, developing a validated rapid assessment tools to measure daily sugar intake, both total sugar and specific sugar (glucose, fructose, sucrose) is needed to yield future reliable assessment, future prospective research in finding epidemiological evidence on the relationship between sugar intake and N-NCDs, and the national guideline of daily sugar allowance. In addition, developing assessment tools such as semi-quantitative food frequency questionnaire [SFFQ] is useful for a future large-scale national study as it has been proven to be stronger, less time-consuming, more feasible, more affordable, and less burden compared to other commonly used tools [16].

Semi-quantitative Food Frequency Questionnaire (SFFQ) is one of the widely used dietary assessment tools to assess the frequency of food consumption during a specific time [17]. It has also been becoming a common tool used in a large-scale epidemiological study to measure dietary exposure and its association with diseases, and thus the validation of the questionnaire as one of the dietary assessment tools is crucial. Developing a newly specific population dietary assessment tool and proving its validity are also substantial considerations since each population has different demographic characteristics, culture, and dietary patterns. Additionally, improving the validity and reliability of newly developed tools primarily can guarantee the accuracy and precision [18]. However, in Indonesia, SFFQ to assess sugar intake has not been available yet. Therefore, the current research focused on the development and validity assessment of SFFQ.

Materials and methods

SFFQ development

Figure 1 describes the development process of SFFQ in the current study. Three different ways were used to identify foods in the development of SFFQ; those were 1) identified foods containing substantial amounts of sugar content on the published Indonesian food composition database and secondary data analysis; 2) observation in local food outlets and supermarkets, and 3) an exploratory survey. The secondary data analyzed were the Indonesia Total Diet Study latest done in 2014 by the National Institute of Health Research and Development [19]. From the study, there were five most consumed sugary food items i.e., white sugar, candy, syrup, chocolate, and jam, and some were also found in the food composition database and the result of local food market observation. While observation in local food outlets and supermarkets resulted in similar food items as in the food database. An exploratory survey was done using repeated 24-hour food recall to get the list of the most consumed sugary beverages and food in adolescents.

Figure 1

Process of SFFQ development.

Sixty adolescents aged 15–17 years in the exploratory survey were purposively selected and different respondents from the samples in the validity test. Among them, there were 68% of girls and 32% of boys; a half of them lived in urban areas, and the other half lived in rural areas. The exploratory survey results showed more than 20 types of fruits were mostly consumed in a month with great variability. Therefore, SFFQ was developed with open-ended questions for listing the fruit groups only to compensate for seasonal variability. Open-ended questions here refer to a free answer, for example asking what kind of fruits were consumed last month. The food groups included in the developed SFFQ explained the serving size to quantify the sugar intake and photographic examples of each food item. Serving size and photographic examples were derived from the food exchange list and the quantity of common products sold in the market. For example, fruit juices products were commonly sold in 250ml, 300ml, and 1L size. SFFQ was developed in close-ended questionnaires except for fruit groups that allowed respondents to check the frequency of usual intake in day/week/month. Respondents have to choose one frequency of true food intake (a day, week, or month).

After developing the SFFQ, pre-testing was done to twenty adolescents who identified food item[s] rarely consumed or consumed more often to modify and get the actual daily sugar intake before the validation study. During the pre-test, there was a blank space that the respondents had to fill in if they had another sugary food intake that was not listed in the questionnaire. The respondents in the pre-test were not a subset of the exploratory survey group but were from a similar population. A half of them lived in urban areas, and the other half was the population in urban areas; based on sex, 70% of the respondents were girls, and 30% of them were boys.

The first developed SFFQ compiled food items from three different approaches previously mentioned. It had fifty-five food items divided into six food groups and food items which are 1) carbohydrate group (rice, red rice, rice cake, white bread, sweet bread, noodle, tubers, potato, rice vermicelli, corn); 2) sugary packaged food group (cereals, chocolate sprinkle, chocolate jam, fruits jam); 3) sweet snacks group (jelly/pudding, chocolate biscuits, wheat biscuits, wafer, chocolate bars, wafer stick, chocolate paste, candy bars, chewing gum, lapis Surabaya, apem, fried toast, terang bulan, sweet cakes, crackers); 4) beverages group (instant tea, soft drinks, instant coffee, milk, susu bantal, isotonic drinks, fruit drinks, fruit juice, chocolate drink, yogurt, ice cream, green bean juice); 5) fruits group (canned fruits and five open-ended questions for fresh fruits]; 6) additional group (white sugar, palm sugar, syrup, honey, sweet soy sauce, tomato sauce, condensed milk, low-calorie sugar). After the pre-test, the analysis found that the respondents did not consume seven food items including wheat biscuit, wafer stick, susu bantal, low-calorie sugar and Indonesian traditional snacks such as apem, lapis Surabaya, and fried toast. Only one food item consumed was doughnuts. Therefore, seven food items were excluded, and only one item was added in the final SFFQ. The final developed SFFQ contained 49 food items divided into six food groups and was tested for the validation [Figure 2].

Figure 2

Illustrative figure of the Semi-Quantitative Food Frequency Questionnaire (Full SQFFQ form available as Supplementary material).

Sample selection

The number of subjects as amounting to 30 people was permitted for the validation test, but it could lead to a major increase in the width of the corrected confidence interval. Thus, a reasonable size for the validation test would be at least 100 subjects which are adequate for a range of likely validation degrees [20]. In this study, 106 adolescents were recruited proportionally from urban and rural areas which are Surabaya City and Sidoarjo District, located in East Java Province, Indonesia. From the population, the samples were randomly selected. The sample size was calculated based on the correlation between the food frequency questionnaires and assisted food records for comparative validity which was 0.3 [21]. The validity test took a significance level of 5% and power of 80%. Included respondents should be adolescent aged 15–17 years and having no diseases that may influence the ability to keep food record. While, following respondents excluded for the test were pregnant adolescents and subjects who have been previously diagnosed with specific diseases, i.e., type 1 diabetes, kidney disease, and others, and need diet modification. The samples were different from those in the exploratory survey and pre-test.

Reliability and validity

Reliability study was done by administering two SFFQs in specific periods [one month apart]. Based on the literature review, sugar intake is relatively stable over time, and thus a one-month survey timeframe would be appropriate to estimate daily sugar intake as well as to minimize the variation in food intake responses due to the changes over time or memory loss [15,22]. For the validation study, intake references were taken from 6-day food diaries consisting of two times 3 non-consecutive days administered in two different weeks. The food diary record form was also detailed with instructions and a food photograph database book to ease the administration. Previous validation studies for sugar intake found that both 3-day and 7-day food records had a good agreement on the tested questionnaires [23], and a total of 6-non-consecutive-day food record in this study was considered sufficient to validate the developed SFFQ. Reliability was assessed by comparing estimates derived from the first and second SFFQs measured using the paired t-test, Intraclass Correlation Coefficient [ICC], Cronbach-α, and cross-classification. While the validity analysis was done by observing the difference between the results of SFFQ and food diary records using the paired t-test, a weighted kappa, and the Bland Altman. Results of energy and sugar content were computed using the Indonesian food database in Nutrisurvey software [24]. Apart from the Indonesian food database, the nutrient content of all packaged food which was previously unavailable was added to Nutrisurvey.

Ethical approval was obtained from the Committee of Human Rights Related to Research Involving Human Subjects, Mahidol University Central Institutional Board Review [MU-CIRB] and from the Health Research Ethics Committee, Faculty of Public Health, Universitas Airlangga. Prior to starting the study, informed consent was obtained from all study participants.

Results

Subject characteristics and sugar intakes

Baseline characteristics of the participants in the validation study are shown in Table 1. To clarify, the participants were different from those involved in the study of SFFQ development. Sixty percent of the respondents were girls, and the rest were boys with a mean age of 15.6 + 0.7 years. The height and weight of the male subjects were significantly higher than girls. The average Body Mass Index (BMI) of all sexes was 21.8 kg/m2. Respectively, the mean BMI of 22.5 + 5.4 kg/m2 among boys was significantly higher than girls whose mean BMI was 21.2 + 3.2 kg/m2. As many as 50.9% of the respondents had normal BMI, but a lower percentage of underweight was found at 23.6%. Whereas, the percentage of overweight and obese respondents reached 25.5%. The prevalence of obesity was higher in boys than girls at 19. 0% and 7.8%, respectively. For the total body fat, the overall mean was 22.1 + 6.7%, higher in girls than boys. Inversely, the mean total muscle was higher in boys than girls at 35.3 + 3.8% and 27.0 + 1.9%, respectively. Lastly, further analysis found that the values of height, weight, BMI, body fat, and muscle mass were significantly different between sexes.

Table 1

Baseline Characteristics [mean, standard deviation] of Subjects.

	Boys	Girls	Total
Number [n%]	42 [39.6]	64 [60.4]	106 [100.0]
Age [year]	15.7 [0.8]	15.6 [0.5]	15.6 [0.7]
Height [cm]	168.04 [6.1]	154.81 [5.1]	160.05 [8.5]
Weight [kg]	63.66 [16.0]	50.97 [8.3]	55.99 [13.4]
BMI [kg/m2]a	22.5 [5.4]	21.2 [3.2]	21.8 [4.3]
Body fat [%]a	16.9 [6.9]	25.4 [3.8]	22.1 [6.7]
Total muscle [%]a	35.3 [3.8]	27.0 [1.9]	30.3 [4.9]
BMI classification [n%]∗
Underweight [<18.5]	11 [26.2]	14 [21.9]	25 [23.6]
Normal [18.5–23.9]	18 [42.9]	36 [56.3]	54 [50.9]
Overweight [24–26.9]	5 [11.9]	9 [14.1]	14 [13.2]
Obese [≥27.0]	8 [19.0]	5 [7.8]	13 [12.3]
Area of residence [n%]
Urban	15 [35.7]	35 [54.7]	50 [47.2]
Rural	27 [64.3]	29 [45.3]	56 [52.8]

Asian-cut off points [123].

Statistically significant at p < 0.05.

The mean daily sugar intake of the subjects obtained through 6-day food diaries amounted to 58.80 ± 29.1 g/day. Fructose, sucrose, and total sugar intake were significantly higher in girls than boys [Table 2].

Table 2

Nutrient intakes and expenses [mean, standard deviation] of subjects.

Characteristics	Mean [SD]
	Boysn = 42	Girlsn = 64	Totaln = 106
Energy intake [kcal]	2098.2 [460.1]	1976.6 [449.9]	2025.7 [453.6]
Glucose intake [g]	1.30 [1.0]	1.64 [0.9]	1.49 [0.9]
Fructose intake [g]a	1.11 [1.0]	1.73 [1.2]	1.47 [1.2]
Sucrose intake [g]a	52.6 [34.6]	53.11 [22.4]	52.72 [27.7]
Total sugar intake [g]a	57.56 [35.9]	59.62 [23.8]	58.80 [29.1]

Statistically significant at p < 0.05.

Reliability

Paired t-test, displayed in Table 3, was performed to the two SFFQs for estimating sugar intakes in one month apart from each other. SFFQ1 referred to the first administered SFFQ (Week-1), while SFFQ2 referred to the second administered SFFQ (Week-4). A consistent pattern of all types of estimated sugar intakes in the SFFQ2 was displayed. Overall, there was no difference in mean glucose, fructose, sucrose, and total sugar intakes between the two administered SFFQs.

Table 3

Reliability of developed SFFQ using paired T-Test.

Variables	SFFQ1 [n = 106]Mean [SD]	SFFQ 2 [n = 106]Mean [SD]	Mean difference [SD]
Glucose [g]	5.98 [5.3]	5.44 [6.1]	0.54 [6.49]
Fructose [g]	4.77 [6.2]	5.22 [6.2]	0.45 [7.88]
Sucrose [g]	65.12 [40.1]	54.96 [39.2]	10.16 [42.24]
Total sugar [g]	86.98 [52.3]	75.89 [53.4]	11.08 [55.9]

Table 4 demonstrates ICC used for estimating the level of agreement. Results as illustrated in Table 4 explained the reproducibility of all sugar intakes except fructose was good. Furthermore, Cronbach-α was used to measure the internal consistency or the close relationship of a set of items as a group. According to Table 4 below, the fructose intake estimates showed consistent results, while using ICC, the fructose intake estimates showed poor reliability. Furthermore, all types of sugar but fructose intakes showed moderate reliability [Table 4].

Table 4

Level of agreement using intraclass correlation coefficients and cronbach-α

Variables	ICC∗
Glucose	0.530
Fructose	0.322
Sucrose	0.604
Total sugar	0.439
Variable	Cronbach-α∗∗
Glucose	0.533
Fructose	0.322
Sucrose	0.604
Total sugar	0.610

ICC <0.4 poor reliability, 0.40–0.75 good reliability, > 0.76 excellent reliability [25].

α < 0.50 poor reliability, 0.50–0.70 moderate reliability, α 0.70–0.90 good reliability.

Table 5 presents the results of cross-classification of sugar intake quartiles of both SFFQs. The majority of participants (>93%) were classified into the same or adjacent quartiles. All types of sugar intakes (glucose, fructose, sucrose, and total sugar) displayed a small degree of gross misclassification or classification into an extreme quartile. Mostly less than 10% of fructose intakes had the greatest gross misclassification amounting to 6.6%. Kw values for all types of sugars showed fair agreement.

Table 5

Cross-Classification from SFFQ1 and SFFQ2 quartiles of mean sugars intake and weighted Kappa values.

Sugar	Same quartile [%]	Adjacent quartile [%]	Extreme quartile [%]	Kw∗
Glucose	52.8	41.6	5.6	0.349
Fructose	38.7	54.7	6.6	0.222
Sucrose	41.5	54.7	3.8	0.220
Total sugar	36.8	58.5	4.7	0.207

Kw < 0.2 = poor agreement; 0.21–0.40 = fair agreement; 0.41–0.60 = moderate agreement.

Validity

Four different tests i.e., paired t-test, cross classification and weighted Kappa, and Bland Altman were employed to assess the validity. Mean daily intakes of glucose, fructose, sucrose, and total sugar estimated from SFFQ2 and 6-day food diaries are presented in Table 6.

Table 6

Validity of developed SFFQ tested with paired T-Test.

Variables	SFFQ 2Mean [SD]	6D FDRMean [SD]	Mean difference [SD]
Glucose [g]	5.44 [6.1]	1.49 [0.9]	3.94 [5.88]∗
Fructose [g]	5.22 [6.2]	1.47 [1.2]	3.75 [6.11]∗
Sucrose [g]	54.96 [39.2]	52.72 [27.7]	2.25 [42.25]
Total sugar [g]	75.89 [37.3]	58.80 [29.1]	17.10 [53.7]∗

Statistically significant at p < 0.05 based on paired t-test.

Mean intake estimates of all types of sugar obtained from SFFQ2 were higher than those derived from food diaries. From SFFQ2, glucose intakes were significantly 3.6 times higher (p = 0.000). The same finding was pointed in fructose intakes with 3.5 times greater (p = 0.000) as estimated from SFFQ2. Similarly, the total sugar intake was 1.3 times greater (p = 0.000) as estimated from SFFQ2.

Cross-classification of sugar intake quartiles from SFFQ2 and repeated food diaries demonstrated that about 90%–95% of the respondents were classified into the same or adjacent quartiles. All estimated sugar intakes showed a small degree of gross misclassification accounting for <10%. Kw values were varied between sugars; most of the intakes showed moderate agreement, but the glucose intake displayed fair agreement (Table 7).

Table 7

Cross-classification of quartiles of mean sugar intake from SFFQ2 and food diaries tested with weighted kappa values.

Types of Sugar	Same quartile [%]	Adjacent quartile [%]	Extreme quartile [%]	Kw∗
Glucose	34.9	55.6	9.5	0.365
Fructose	27.4	67.0	5.6	0.403
Sucrose	37.7	55.7	6.6	0.415
Total sugar	35.8	59.5	4.7	0.425

fair agreement [0.21–0.40]; moderate agreement [0.41–0.60].

The Bland-Altman plot presented was an assessed agreement between the SFFQ and the repeated food dairies [Figure 3]. The X-axis showed mean intake of sugar resulted from the formula of food diaries + SFFQ/2, while Y-axis showed the difference sugar intakes between food diaries and SFFQ2. The Bland-Altman plot revealed mostly negative mean differences, signifying that SFFQ2 was overestimated when compared to repeated food diaries and was in accordance with the paired t-test results. Moreover, the plot indicated that size differences between two methods for each subject tended to increase as mean sugar intake increased. This trend was seen clearly in the glucose intake [Figure 3a]. For fructose intake, the plot demonstrated a similar trend in which the mean difference was close to the linear line [Figure 3b]. The plot for sucrose and total sugar intakes were more spread if compared to glucose and fructose intakes. The spread around the mean reflected the difference variations in sucrose [Figure 3c] and total sugar intakes [Figure 3d], inferring that the two methods were moderately correlated.

Figure 3

Bland-Altman Plot of Sugar Intake derived from SFFQ2 and Food Diaries; a). Bland-Altman plot of Glucose; b) Bland-Altman plot of fructose; c) Bland-Altman plot of sucrose; d) Bland-Altman plot of total sugar.

Validity of food groups, natural sugar vs added sugar

The developed SFFQ aims to analyze natural and added sugar intakes. However, the validity of each food group was tested for the natural sugar and added sugar intakes (see Table 8). As mentioned before, the developed SFFQ consisted of six food groups; natural sugar was obtained from carbohydrate group and fruit group; while added sugar was obtained from a sugary packaged food group, sweet snack group, beverages group, and additional groups. The mean intakes of natural sugar and added sugar retrieved from SFFQ2 were 5.1 gr/day and 52.7 gr/day, respectively. The paired t-test of food groups revealed no difference. Kw values showed fair to moderate agreement.

Table 8

Validity of food groups containing natural sugar vs added sugar.

	Paired T-Test	Weighted Kappa (Kw)
Food group
Carbohydrate [g]	0.105	0.581
Fruit [g]	0.068	0.213
Sugary packaged food [g]	0.225	0.445
Sweet snacks [g]	0.159	0.451
Beverages [g]	0.241	0.557
Additional sugar [g]	0.119	0.439
Types of sugar
Natural sugar [g]	0.143	0.473
Added sugar [g]	0.192	0.481

Discussion

Dietary findings

This study revealed that the average contribution of total sugar to total energy intakes was slightly higher (>10% of energy intake) than recommended intake from the WHO/FAO Scientific update 2006, Dietary Guidelines for Americans 2005, and WHO 2015 [26,27,28]. The current result is in line with a previous study on sugar intake of Korean children and adolescents having higher intake in girls (54.3 g) than in boys (46.6 g) [29] and another study which found the average sugar intake of female Brazilian adults was higher than males [30]. These findings are consistent with a global trend in the sugar intake differences which might be influenced by several factors such as socioeconomic status [30], different food craving levels, hunger and temptation (external eating), and stress level [31]. However, the current study did not further analyze the influencing factors.

Reliability of developed SFFQ

Th paired t-test analysis demonstrated no difference between two SFFQs administrated over one month. The trend of overestimated mean sugar intakes from SFFQ1 compared to SFFQ2 was observed in the results. The highest overestimation was found in sucrose intake (18%); other than that, overestimation of mean glucose and total sugar intakes were 9% and 14%, respectively. A previous study also found similar results in which FFQ1 was significantly overestimated than FFQ2 [15,32]. Based on the observation, the respondents took 10–15 min more to fill in the SFFQ2, suggesting that they seemed to think more carefully about what they ate months ago. That means estimated mean sugar intakes retrieved from SFFQ2 could represent more actual intake and showed a lower number compared to SFFQ1. Moreover, the respondents possibly changed their dietary intake concerning sugary food and beverages during the research. They could consume less sugary drinks and fewer sweet desserts, which could cause less estimated mean sugar intake from SFFQ2. The trigger of behavior changes among the respondents was increasing awareness about higher sugar intakes that embarrass them while reposting their sugary food and beverages intakes.

The analysis using ICC of developed SFFQ resulted in poor to good reproducibility for estimating a mean daily sugars intake. The highest reproducibility was observed in the sucrose intake, and good reliability belonged to glucose and total sugar intakes. However, poor reliability was still observed in fructose intake possibly due to the use of open-ended questions for fruit group. Willet [20] explained that using an open-ended question approach can detect important contributors to nutrient intake to be measured and unlikely missed; it also eases the respondents to answer if compared to single longer and complex question approach. The researchers considered those reasons in designing the food group and items. However, seasonal changes might cause higher variations in fruit intakes between the two questionnaires, thereby causing poor reliability. Open-ended questionnaires for the fruit group should also list several fruits items including seasonal fruit that have high sugar content to avoid the possibility of seasonal variation and improve its reproducibility. The ICC of the developed tool was considered as good as previous studies which ICC ranged from 0.4-0.7 [15,21,33]. Most of those studies administered the second SFFQ/FFQ at five-month to one-year intervals, while the current study had a shorter interval at one month to re-administer the questionnaire.

The highest internal consistency was found in the mean intakes of sucrose and glucose ranging from 0.228-0.633. In comparison with the current study, a validation study on sugar intake through the administration of FFQ to Ugandan schoolchildren found the Cronbach-α ranged from 0.69 to 0.70 [34], while another SFFQ validation study for young female adults showed Cronbach-α was about 0.58–0.86 [35]. In addition to the previous studies, Cronbach-α of short-term repeatability of FFQ in children in New Zealand ranged from 0.59-0.92 [32]. This difference occurred mainly because every validation study had a different number of food items included in the developed tool and different timeframes of administration between two SFFQs/FFQs depending on the researcher's justification. For example, a higher Cronbach-⟨ value was found in sugar snack FFQ that only contained eight food items and two FFQs were administered one week apart which was much shorter compared to our study, thus resulting in different Cronbach-⟨ value. As in our developed SFFQ, 49 food items were added within a one-month interval because sugar intake is relatively stable over time [15,21]. Strength agreement between two administrations of SFFQs was obtained using cross-classification and weighted kappa. Our developed SFFQ showed a good classification into the same or adjacent quartiles between two administered SFFQs. All estimated sugar intakes retrieved from both SFFQs only showed <7% grossly misclassified which means that two SFFQs administration with one-month interval had a good agreement to classify into quartiles. Based on weighted kappa values, all estimated means sugar intakes had a fair agreement between two administered SFFQs. A short one-month timeframe for re-administered developed SFFQ in our study could explain the good agreement between two SFFQs because it could improve the subject's memory compared to a longer timeframe.

Validity of developed SFFQ

Paired T-Test analysis revealed that SFFQ2, in comparison to repeated food records, significantly overestimated mean glucose, fructose, and total sugar. Estimated mean sucrose showed the lowest mean difference between the two methods. The results were consistent with several studies [33,36,37,38]. Barret's comprehensive SFFQ, particularly, validate 297-food items SFFQ with four 1-week food diaries for 12 months demonstrated SFFQ overestimation of total sugar, fructose, sucrose by 36%, 57%, and 25% respectively [33]. Most of these studies showed overestimated mean nutrient intake assessed using SFFQ/FFQ compared to food diary or 24-hour food recall due to the limited information in FFQ which consisted of fixed multiple choices of intake frequencies at a fixed interval [e.g., 1–3/>3 times/day, 1–6/>6 times/week, or 1–3/>3 times/month] [20]. Two Willet's FFQ also demonstrated a similar trend of energy intake overestimation from FFQ when compared to doubly labeled water among children [39], and another FFQ overestimated several nutrient intakes compared to 1-year diet record of 27 adults [40]. This current study utilized 6-days food records to validate SFFQ2. Six-day food records are deemed to have high daily variations (weekdays and weekend), meaning that the respondents could have higher sugars intake in just a few days out of six days. Therefore, when mean sugar intakes was observed and grouped into six categories, it seemed that mean sugar intake from food diaries was estimated lower than mean sugar intake from SFFQ2. Besides, since food diaries required the respondents to write down all day food consumption within 6 days, the respondents were more likely to only write the dietary record when they ate less to ease the administration and to show less food consumption. This result might include random within and between person error as part of measurement errors in dietary intake assessment due to day-to-day sugar intake fluctuation [20].

The developed SFFQ in the current study had a strong ability to correctly classify individuals into the same or adjacent quartiles of sugar intakes. The finding is similar to those retrieved from previous studies [35,41,42] which used different reference methods to validate; George et al. [35] used 3-day diet records, while the other two studies used 14-day weighed food records and a 24-hour food recall. In contrast, the current study demonstrated better ability to classify individuals into the same and adjacent quartiles of sugar intakes with <10% gross misclassification of than the research by Green et al. [38] who validated SFFQ by using four 1-week food diaries for 12 months with 28% gross misclassification, and Barret et al. [33] studied on the FFQ validation by utilizing 3-day weighed food records with 10% gross misclassification.

Furthermore, both instruments used in the current study conveyed similar measurement errors from a self-report method which could explain that the developed SFFQ performed better cross-classification than previous studies. Compared to other FFQs, 49 food items included in the developed SFFQ equipped with image were considerably fewer because this current study only focused on sugary food, snacks, and beverages, while other FFQs consisted of around 76–297 food items. Fewer food items can be an advantage since it may diminish respondent's fatigue levels while being administered with SFFQ and maintain the accuracy of their answers. Moreover, the developed SFFQ was designed culturally specific to East Java adolescents who were previously involved in the exploratory survey and pre-test. Due to their experience in previous assessments, they were more engaged and understood the questions in SFFQ better and thus could answer them more correctly than other previously used FFQs that had more generalized food and beverages questions [35,38,41,42]. Weighted Kappa values of the developed SFFQ only demonstrated fair to moderate agreement between the SFFQ2 and 6-days dietary records. The weighted Kappa defined a Kw-value ranging from 0.2-0.40 as having fair agreement and a Kw-value of >0.40 as having moderate agreement. Moderate agreement at > 0.40 was found in estimated mean fructose, sucrose, and total sugar intakes. More or less, the current results are similar to other studies showed FFQ that measured sugar intakes resulted in only moderate agreement [15,33,43].

The Bland-Altman plot conveyed that the higher average sugar intake, the higher differences between repeated food records and SFFQ. Furthermore, some cases displayed out of the level of agreement. Further analysis found out of the levels of agreement were similar among 8 subjects. That indicated the validity of the developed SFFQ. A significantly wider limit of agreement was shown in non-glucose fruit, total non-sugary fruit, glucose, fructose, and total sugar intakes. Besides, the number of sample sizes could affect these LoA intervals. Preceding studies also reported a similarly wide range of LoA compared to the current study [15,33,42] as their sample sizes were only 72, 100, and 78 samples which were fewer compared to the current sample size [n = 106]. Another plausible reason such as "outliers" in each study might contribute to result in wider LoA. As previously mentioned, there were several cases out of LoA as seen in the Bland-Altman plot. These outliers should not be withdrawn from the analysis even though it could improve the overall LoA as proposed by Bland [44]. Overall, the current developed SFFQ could be still valid at reporting the mean sugar intakes of a group.

Apart from the questionnaire's limitation, the present SFFQ was developed using three different approaches: exploratory survey, secondary data analysis, and food market observation. Moreover, this study could compile a hundred foods then restricted into several numbers that represent sugar intakes. A limited number of food items included in the SFFQ represent the main sources of sugary food and beverages. This also strengthened the respondent's understanding of food items in the list since the food items were available around their residence and reduced their burden. Moreover, 6-day food records could also capture day-to-day, weekly and daily variations. However, some limitations are also noted. The major limitation is the fact that Indonesia's food composition database, especially its glucose and fructose intake values, was not complete and only available 50–60%, while only data base of sucrose intake value was completely available. Besides, to encounter the limited sugar intake value, the sugar content from packaged food was obtained from its nutrition facts and USDA food composition database to retrieve missing sugar content in specific food items. Moreover, the sugar value of each food item listed in the SFFQ was roughly estimated. For example, many instant tea products were available in the market and had various sugar content, but in the SFFQ, the researchers only labelled them as "instant tea". The sugar content of instant tea in the SFFQ was calculated from the average sugar content of several tea brands, but not specified for each brand. While for food record analysis, sugar intakes were exactly calculated based on its brand. Thus, a rough estimation of sugar content in food items could affect more possible overestimation resulted from the SFFQ compared to the food record. To improve the study, the food items in the SFFQ can be separated based on its sugar content. For instance, food items grouped in instant tea products can be categorized more specifically into less sugar, moderate sugar, and high sugar.

Conclusions

In conclusion, the self-administered SFFQ is a reliable and valid dietary assessment tool for assessing daily sugar intake in the adolescent population. This developed instrument can pose as a simple tool for screening sugar intake either at a hospital or primary healthcare center. Screening of daily sugar intake as basic information can be useful for dietitians or other health professionals to decide further assessment or treatment. Moreover, at the government level, the developed SFFQ can be used in a national diet survey, specifically to assess daily sugar intakes, by giving several improvements considering that the Indonesian population is a multicultural country. Evermore, further studies can use this questionnaire to investigate an association between daily sugar intake and obesity or metabolic disorders, i.e., cardiovascular diseases, metabolic syndrome, or type 2 diabetes mellitus in wider epidemiological research. Knowing the relationship between sugar intake and diseases may encourage the government to set a new public health policy related to sugar intake, especially in the prevention of degenerative diseases.

Declarations

Author contribution statement

Qonita Rachmah: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Wantanee Kriengsinyos: Conceived and designed the experiments; Performed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Nipa Rojroongwasinkul: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Tippawan Pongcharoen: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This work was supported by (Indonesia Endowment Fund for Education) (02-2015).

Data availability statement

The authors do not have permission to share data.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.