Research Priorities for Studies Linking Intake of Low-Calorie Sweeteners and Potentially Related Health Outcomes: Research Methodology and Study Design.

Background: In a world of finite research funding, efforts to prioritize future research topics are increasingly necessary. Objective: The aim of this study was to identify and prioritize the direction of future research in the broad area of low-calorie sweetener (LCS) intake and potentially related health outcomes by using a novel method that incorporates evidence mapping in the Agency for Healthcare Research and Quality's Future Research Needs (FRN) process.
Methods: A diverse expert stakeholder panel was convened and engaged to identify research gaps and prioritize future research needs. An independent research team hosted a number of interactive webinars and elicited feedback through surveys and individual interviews with the stakeholder panel, which included policymakers, lay audience members, health providers, a research funder, individuals with food industry experience, and researchers of several different specialties.
Results: The stakeholder panel generated and ranked a list of 18 FRN questions across 5 broad research areas. Overall, stakeholder panel members unanimously agreed that the research questions that will have the largest public health impact are those that address outcomes related to body weight, appetite, and dietary intake. Although the LCSs included in this FRN project have all been Generally Recognized as Safe by the FDA or approved as food additives, the recurrent concerns and confusions with regard to the "safety" of LCSs by consumers underscore the importance of communicating the science to the general public.
Conclusion: Our project provides evidence that engaging a diverse expert stakeholder panel is an effective method of translating gaps in nutrition research into prioritized areas of future research.

Introduction

A structured approach for examining the results of systematic reviews and prioritizing research gaps from systematic reviews, namely the Future Research Needs (FRN) process, has been developed by the Agency for Healthcare Research and Quality (AHRQ) (1). The current AHRQ FRN process begins by identifying a list of evidence gaps from a systematic review. Next, the systematic review team works with a stakeholder panel to first elaborate on and then consolidate the evidence gaps. Finally, the research questions are prioritized by the stakeholders according to a set of criteria focused on the potential value of the research. Since 2010, the AHRQ has supported studies that develop and prioritize the future research needed by decision makers in a variety of health care content areas (2). However, none to date are related to nutrition topics. Furthermore, nutrition-related systematic reviews only comprise ∼8% of all systematic reviews in the Cochrane Database of Systematic Reviews and, among these, approximately half focus on nutritional supplementation or supplements (3). Due to the anticipated need to consider broader research questions for public health nutrition, we previously modified the ARHQ's FRN approach by integrating an evidence-mapping process (4). Evidence mapping is an emerging evidence synthesis method for describing the volume and characteristics of research in a broad field. This method can be used to identify areas, known as research gaps, in which there is a lack of evidence and further studies may be needed, as well as research-dense areas in which systematic reviews could be pursued. With the use of evidence mapping, one can quickly become informed and identify research gaps without having to conduct a comprehensive systematic review.

The aim of this article was to summarize the processes and results of this new FRN approach by using the published literature on low-calorie sweeteners (LCSs) and selected health outcomes as an example of its application. As concerns increase with regard to a link between added sugars and obesity risk (5, 6), there is increasing interest in LCSs (artificial sweeteners, non-nutritive sweeteners, or high-intensive sweeteners) as a tool to help reduce excess energy intake and body weight. With LCSs being widely used in foods and beverages for decades, the questions on their role in health and whether there are adverse health risks associated with use of LCSs over the course of many years remain unsettled. In the United States, there are 2 regulatory routes allowed by the FDA for LCSs permitted for use in foods: either as “food additives” or as ingredients Generally Recognized as Safe (GRAS); both require rigorous toxicologic review (7, 8). For a food additive, the FDA reviews animal and human studies designed to identify possible toxic effects in any major body system, then evaluates it under conditions of intended use and expected exposure to determine an acceptable daily intake level (9–11) considered to be safe over a lifetime. Currently, aspartame, saccharin, acesulfame potassium, sucralose, neotame, and advantame are approved food additives; and some stevia-based sweeteners (i.e., high-purity stevio glycosides such as stevioside and Rebaudioside A isolated from the leaves of Stevia rebaudiana Bertoni) (12) and Luo Han Guo (also known as swingle fruit or monk fruit) extracts are GRAS (8, 10). Other LCSs are outside the scope of this project.

Methods

The initial scope of work was developed by the sponsor of this project [International Life Sciences Institute (ILSI) North American branch] in a request-for-proposals. Aside from providing initial feedback on our funded research proposal and identifying potential stakeholders, the sponsor was not involved in the later refinement of the scope of work, evidence mapping, or FRN processes. The following 5 areas of research with regard to the potential health effects of LCSs were all considered and included: 1) energy sensing by the brain, 2) gut hormones that may influence energy homeostasis, 3) satiety and taste preference, 4) eating behavior, and 5) body weight and composition. The project consisted of the research team and the expert stakeholder panel. The panel was tasked with overseeing and advising the development of the search strategy, study inclusion and exclusion criteria, and prioritization of FRN research questions. The research team was in charge of data screening, abstraction, analysis, and visualization.

Recruitment of the expert stakeholder panel

The research team collaborated with the project sponsor, the Technical Committee on Low Calorie Sweeteners at the ILSI North American branch, to identify potential stakeholder panel members. We used the “7 Ps of Stakeholder Engagement” approach to establish and organize our panel. This framework highlights 7 stakeholder groups to engage in developing and refining study protocols. The 7 groups are as follows: patients, providers, researchers, policymakers, product makers, payers, and purchasers (13). In our study, panel members were selected to provide a variety of perspectives and broad expertise with regard to LCSs, evidence-based decision making, and health research. Potential project stakeholders were sorted into the following categories: lay audience, policymakers, health providers (physicians, dietitians), research funders, researchers (with expertise in epidemiology, intervention studies, statistics, and taste), and individuals with food industry product-development experience. Our goal was to recruit individuals in each of these categories.

Potential stakeholders were first approached via e-mail invitation. The research coordinator and principal investigator discussed the project in more detail with those who were interested and answered any initial questions through e-mail or phone conversations. Individuals who responded to the invitation with interest were required to submit a curriculum vitae and disclose any potential conflicts of interest; they were asked to disclose financial conflicts of interest >$10,000, as well as any other potential conflicts, for the research team to review before accepting their participation on the panel. Due to their unique expertise, individuals with conflicts of interest were not automatically excluded, but all conflicts were first reviewed by the research team and then shared with ILSI for discussion. ILSI and the research team then reached a consensus about each individual's qualification. The purpose of this vetting process was to screen out any individuals with clear financial interests in the results of this project and to promote transparency in the reporting of our results. To minimize the potential for bias resulting from conflicts of interest, our panelists who had retired from the industry sector were considered “nonvoters” who could not cast votes on the final prioritization of FRN topics, although they still provided their insights and expertise throughout the project. At the beginning of the project (August 2014), 25 potential stakeholder candidates were contacted. Of these, 7 did not respond, 1 declined to participate, and 1 was excluded due to a major financial conflict of interest. Thus, 16 stakeholders were recruited to form the panel. The final panel consisted of 15 individuals. A list of the panel members is presented in .

TABLE 1

Initial FRN stakeholder panel

Stakeholder group	Number of stakeholders in group
Lay audience	2
Policymaker	32
Health provider	3
Research funder	1
Product maker (nonvoters)	2
Researcher (intervention)	2
Researcher (epidemiology)	1
Researcher (statistics)	1
Researcher (taste)	1

FRN, Future Research Needs.

The final stakeholder panel included 15 individuals. One policymaker dropped out for personal reasons.

Evidence mapping

To build a literature database for evidence mapping, the research team developed a search strategy to capture all relevant research articles in the defined scope of work, as described above. Research articles were restricted to randomized or nonrandomized controlled clinical trials or prospective cohort studies involving humans. The search strategy was reviewed, and additional search terms were added by the stakeholder panel. Articles were screened in multiple rounds on the basis of our inclusion and exclusion criteria, which were also developed with the help of the panel. Each article was screened by ≥2 investigators, and discrepancies were discussed and resolved. The full texts of relevant articles were abstracted and an evidence-map database was developed, including information on study design, demographic characteristics, and health status of the participant population; health outcomes of interest; LCSs; and comparisons used in the experiment and findings. The final LCS evidence-map database included 225 studies and was uploaded onto SRDR (Brown University Evidence-based Practice Center), an open public data repository (14). A synthesized summary report of the quantity and characteristics of the published literature in the evidence-map database was presented to the stakeholder panel during a round-table in-person meeting to augment and guide discussions of current gaps in the published literature. The process of evidence mapping and the results are reported elsewhere (15). It is important to note that, although literature gaps may exist, both the importance and feasibility of further research involve judgment of various expert stakeholders, which was the objective of this panel.

Stakeholder panel engagement

Published literature search.

The stakeholder panel was engaged throughout the project. shows a timeline of the overall project, highlighting major project activities. In general, the research team sent out biweekly e-mails and hosted monthly webinars. During the webinars, the research team presented the status of the project. The initial webinar served as an introduction to the project, including evidence-mapping methods and how the findings would be used to inform the FRN project. Subsequent webinars were used to present preliminary findings of our published literature search and abstraction, as well as to refine inclusion and exclusion criteria and add additional search terms. Stakeholders were encouraged to ask questions and make comments, and detailed minutes were kept. Minutes as well as presentation slides were shared with the panel after each webinar and a web page was customized for the project to store and share project documents, as well as to facilitate further discussion. Important project decisions, including study inclusion or exclusion criteria and other search strategy details discussed during project webinars and on the project website, were finalized by using stakeholder feedback collected through a series of online surveys and follow-up e-mails. Specifically, surveys used a series of multiple choice questions to quantify the number of panel members that agreed and disagreed with proposed changes, as well as open-ended questions for panel members to provide additional explanation and suggestions.

FIGURE 1

Timeline of the FRN project. FRN, Future Research Needs.

Development of FRN questions.

In December 2014, the research team convened a 1-d, in-person meeting with the stakeholder panel. At this meeting, the research team presented a draft of the Evidence Mapping Summary Analysis report, which the panel was sent a few days before to review. The summary report was organized into sections by various health outcome categories and provided detailed summaries of the frequencies of various endpoints and other study characteristics in the published literature. These findings were used to spur, as well as to inform with detailed quantitative information, discussion about potential research gaps and future research needs. Stakeholders took turns facilitating discussions on different sections of the report, assigned at the start of the meeting, and conversations were recorded in order to improve the quality of the detailed meeting minutes. The overall goal of the in-person meeting was to facilitate discussion and to document as many different perspectives on research gaps and priorities as possible for later use in refining and prioritizing research questions.

For that reason, citing scientific references or using the Evidence Mapping Summary Analysis report and database were encouraged but not required to support the statements or opinions raised by the stakeholder panel. At the conclusion of each discussion, the panel was encouraged to translate the list of identified gaps in the literature into a preliminary list of research questions.

A final webinar was scheduled after this meeting to revise the list of questions and to ensure that they were fully reflective of the meeting discussions. The final step in refining the list of FRN questions was the administration of a final online survey that gave stakeholders the opportunity to make edits. Panel members were allowed to suggest wording changes, the combination or elimination of certain questions, and to add any questions they felt were missing. E-mails were used to share survey results and resulting project decisions with the panel.

Semistructured interviews

In the period between the final webinar and the survey to refine the list of research questions, the research coordinator and principal investigator conducted semistructured interviews with individual stakeholders. Interviews were recorded and detailed notes were taken from each interview. The purpose of these interviews was to give all stakeholders the opportunity to share their thoughts on our question list, as well as on the more pressing areas for future investigation. Our discussions with each stakeholder started with a simple question: If you had unlimited funding, what research would you do with LCSs? Their answers then led the rest of the conversation about research prioritizations and the state of the science. At the start of each interview, stakeholders were reminded of the area of expertise they were recruited to represent to encourage them to view our discussion from that perspective. For example, policymakers were asked to consider and discuss the questions with regard to their potential impact on public health policy. Fourteen of the 15 panel members completed interviews. One stakeholder, a health provider, could not be interviewed due to scheduling conflicts.

Final FRN question prioritization

Once the list of questions was finalized, the research team designed one final online survey to prioritize research questions with the use of a framework following the Effective Health Care Program Selection Criteria (16), slightly modified for application in the public health nutrition field. Stakeholders were presented with an updated list of research questions, which reflected suggestions and revisions collected from the final webinar, question refinement survey, and semistructured interviews. They were asked not only to rate question priority but also to suggest a categorization scheme for the questions and to indicate any final wording changes they wanted to make. When prioritizing the research questions, stakeholders were asked to rate each research question on a scale of 1 to 5, from lowest to highest, for 4 separate dimensions: importance, desirability, feasibility, and potential impact. The final 18 questions were shown to respondents in random order to avoid question-order bias. The average of the 4 dimension scores each question received was used as the overall priority score, and research questions were ranked in order by average overall priority score for all respondents (). The final deliverable of the FRN project was a list of FRN research questions, identified and prioritized by the members of the stakeholder panel.

TABLE 2

Ranking of FRN questions by priority

Rank	Score, mean ± SD	Question category	Questions
1	4.4 ± 0.8	Physiologic (or clinical) impacts	Q1. Do LCSs aid weight loss and/or weight maintenance?
2	4.2 ± 0.9	Behavioral impacts	Q2. Does LCS consumption modify appetite (hunger, fullness, desire to eat ± prospective consumption) and/or total energy intake and, if so, how?
3	3.9 ± 0.9	Mechanistic questions	Q3. Does the use of LCSs affect insulin secretion, carbohydrate metabolism, or the gut microbiota and its function? If so, where is this happening (cognition, sweet receptors on tongue, receptors in gastrointestional tract, etc.) and does it have any physiologic consequences on health?
4	3.9 ± 1.0	Physiologic (or clinical) impacts	Q4. Are there potential long-term health risks (obesity, diabetes, cancer, CVD, etc.) of LCS consumption in humans? Are certain population groups (diabetics, children, pregnant women, those with genetic disease) more susceptible to the potential health risk(s)?
5	3.8 ± 0.9	Mechanistic questions	Q5. Is LCS sweetness perceived by the brain as energy in the same way as other sweeteners? Do those who are overweight or obese sense LCSs differently than normal-weight people?
5	3.8 ± 1.2	Physiologic (or clinical) impacts	Q6. Are there impacts of LCS consumption during pregnancy on the fetus?
5	3.8 ± 1.1	Impact on dietary intake	Q7. Do LCSs differentially affect long-term food intake, eating frequency, and portion sizes in children, adolescents, and adults? Is there an impact on dietary quality and adherence to recommended dietary patterns?
6	3.6 ± 0.8	Physiologic (or clinical) impacts	Q8. In individuals with diabetes and prediabetes, does chronic consumption of LCSs have an impact on glycemic control, alter glucose transport, or invoke a cephalic phase response?
7	3.6 ± 1.0	Impact on dietary intake	Q9. Does LCS consumption affect consumption of other sweeteners or sugars or total carbohydrate intake? Is the effect different than that from consumption of nutritive sweeteners?
8	3.6 ± 1.0	Mechanistic questions	Q10. Do LCSs affect energy metabolism and fat storage?
9	3.5 ± 1.0	Cross-cutting methodology-related, policy- or evidence-based decision-making questions	Q11. Should study findings be evaluated for each LCS individually or collectively? To which health outcome(s) are the findings from individual LCSs generalizable to the class of ingredients?
10	3.5 ± 1.1	Cross-cutting methodology-related, policy- or evidence-based decision-making questions	Q12. Is LCS intake accurately estimated in current dietary assessment tools?
11	3.2 ± 0.7	Behavioral impacts	Q13. Are there interactions between the combination of fat substitutes and sweetener substitutes on appetite (hunger, fullness, desire to eat ± prospective consumption) and/or total energy intake?
12	3.1 ± 1.0	Cross-cutting methodology-related, policy- or evidence-based decision-making questions	Q14. Is there any variation in how LCSs affect those of different ages, races, and ethnicities?
13	3.1 ± 0.9	Mechanistic questions	Q15. Do individuals with different dietary patterns (high protein vs. high carbohydrate, etc.) affect the metabolism of LCSs differently and, if so, how?
14	3.0 ± 1.0	Cross-cutting methodology-related, policy- or evidence-based decision-making questions	Q16. How do we design a system or methodology to address the differences in existing LCS compounds vs. compounds that will be emerging down the road?
15	3.0 ± 1.0	Physiologic (or clinical) impacts	Q17. Do the effects of LCS consumption on body weight differ by sex? If so, what are the sex-specific mechanisms of the impact of LCS consumption on body weight?
16	2.7 ± 1.0	Cross-cutting methodology-related, policy- or evidence-based decision-making questions	Q18. Has there been a gradual increase in the overall sweetness in our diet?

CVD, cardiovascular disease; FRN, Future Research Needs; LCS, low-calorie sweetener; Q, question.

Results

Of the 16 stakeholders initially recruited to the panel, 1 dropped out during the course of the project due to personal reasons. A total of 13 members across 7 stakeholder groups participated in ≥3 of 4 webinars. The average response rate for the online surveys was 85%. The stakeholder panel remained engaged throughout the project and was frequently encouraged to contact the research team at any time with questions or concerns.

Research gaps identified by published systematic reviews

The 2015–2020 Dietary Guidelines for Americans suggest a reduction in “added sugars consumption to <10% of calories per day” to help the public achieve a healthy eating pattern (17). The Dietary Guidelines for Americans provide a list of replacement options including LCSs and conclude that these LCSs are safe for the general population on the basis of available scientific evidence. The guidelines also stated that “replacing added sugars with LCSs may reduce calorie intake in the short term, yet questions remain about their effectiveness as a long-term weight management strategy” (17).

We identified 4 systematic reviews examining the association between intakes of LCSs and body weight or risk of obesity (18–21). Three of these recommended that future LCS studies focus on long-term interventions in high-quality trial settings. One of these systematic reviews by Rogers et al. (21) suggested switching the research focus to how to best utilize LCSs to achieve specific public health goals.

In-person meeting and research question generation

During the in-person meeting, the stakeholder panel generated a list of 39 preliminary research questions. Although these questions were mostly associated with specific outcome groups (e.g., appetite and/or satiety), there were also a number of cross-cutting questions generated that addressed other topics; examples include how accurately current dietary assessment tools capture LCS intake and whether research specific to one LCS could be generalizable to the entire class of LCSs. Through the final webinar with the stakeholder panel, this initial list was then narrowed down to 26 questions by merging similar ideas into more targeted questions.

Semistructured interviews were used in May and June of 2015, before the final ranking of questions, to collect detailed responses with regard to how different perspectives (i.e., researcher compared with policymaker) affect future research prioritization. Several themes emerged, including the need for more randomized controlled trials, discrepancies between the public perception of LCSs and that of scientists, and the importance of certain questions over others from a public health significance perspective.

The potential impact of LCSs on energy intake and research to better capture current intakes of LCSs was generally identified as the most pressing research topic in the interviews. There was also interest in the benefits and risks of substituting sugar with LCSs in the diets of individuals with diabetes, particularly from our dietitians. Several panel members were interested in efforts to identify a common measurement and endpoint for all LCSs so that they could be referred to collectively as a class of ingredients, despite their unique chemical structures, physical and chemical properties, and taste profiles. It was also discussed that weight outcomes were important due to the continued controversy surrounding the role of LCSs in weight management. Specifically, it was noted that proposed mechanisms linking LCSs to increased energy intake, such as the idea that LCSs may interfere with the body's recognition of calories in response to sweetness or that LCSs may alter the perception of or preference for sweetness, needed to be rigorously tested in humans (20, 22). In addition, our lay consumer stakeholder was most interested in the role of LCSs in weight loss, feeling that LCSs are used in many diet products and marketed as products to assist in weight management. Overall, although researcher stakeholders had a variety of different priorities, most agreed that the potential impact of LCSs on energy intake was the most important for public health impact.

Final question prioritization

Last, the research team administered an online survey to the stakeholder panel (100% response rate) to collect feedback on the list of 26 questions and to give them a final opportunity to propose any questions that could be missing from the list. Again, the votes cast by stakeholders with previous food industry experience were not counted in the calculation of priority scores. The results of the survey helped the team group and merge similar questions to form a list of 18 research questions (Table 2), categorize each into 1 of 5 groups, and rank the questions on the basis of the Effective Health Care Program Selection Criteria (i.e., importance, desirability, feasibility, and potential impact) (16), with slight modifications for application to the field of public health nutrition. The 5 groups were as follows: “physiological (or clinical) impacts,” “behavioral impacts,” “mechanistic questions,” “impact on dietary intake,” and “crosscutting methodology-related questions, policy or evidence-based decision-making.” These 5 conceptual groups show how the 18 FRN questions are linked to one another in a larger framework and may help in the design of future studies ().

FIGURE 2

Conceptual framework of FRN questions. CNS, central nervous system; FRN, Future Research Needs; G.I., gastrointestinal; LCS, low-calorie sweetener; Q, Future Research Needs question (refer to Table 2 for each FRN question).

Discussion

In 2012, a joint scientific statement from the American Heart Association and the American Diabetes Association concluded that, although they considered limiting added sugars in beverages and food to be an important strategy for supporting optimal nutrition and healthy weight, there were insufficient data to determine if the use of LCSs to displace caloric sweeteners reduced the intake of total added sugars or carbohydrates, or if they modified appetite, energy balance, body weight, or cardiometabolic risk factors (23). More recently, the Scientific Report of the 2015 Dietary Guidelines Advisory Committee reviewed the evidence from 3 systematic reviews on the relation between LCS intake and measures of body weight and obesity in adults and children. It concluded that moderate evidence supported replacing sugar-containing beverages with beverages containing LCSs to reduce calorie intake, body weight, and adiposity (24). However, studies of the effects of LCSs on other health indicators, such as cardiometabolic risk factors, behavioral impact, or dietary intake, were not reviewed in the report. Similarly, a 2015 systematic review identified a limited number of studies examining the effects of sugar-sweetened beverages with other beverage alternatives (i.e., water, LCS-sweetened beverages, coffee, tea, 100% fruit juices, and milk) on long-term (defined as ≥4 mo) health outcomes (i.e., type 2 diabetes, stroke, and cardiometabolic risk factors) in both children and adults. Again, although the authors concluded that beverage alternatives may be beneficial for long-term body-weight management, the evidence was inadequate to draw any firm conclusions on other health outcomes (25).

This FRN project aimed to use a novel approach to identify and prioritize the direction of future research in the broad research area of LCS intake and its potentially related health outcomes. Our team previously carried out a similar project with a different group of stakeholders and identified several challenges highlighting the importance of engaging a diverse stakeholder panel in the process (4). This article details how we engaged a multidisciplinary stakeholder panel in elaborating and prioritizing FRN questions, because this methodology shows promise for use in future evidence-based nutrition research decision making. Furthermore, this FRN project showed how evidence-mapping results could be translated into a list of prioritized research questions that are more immediately usable by policymakers, funders, and researchers.

Discussions on FRN questions from the semistructured interviews

Our stakeholder panel members unanimously agreed that the research questions that will have the largest public health impact are those that address body weight, appetite, and dietary intake. When asked what research project they would do if they had no funding limitations, most reported studies with these outcomes. Many indicated that these are the types of questions that will drive health policy and were especially interested in the role of LCSs as either a barrier or facilitator to meeting the federal dietary guidelines for the general public. The question of whether chronic LCS intake has any impact on sweet preference was also deemed to be important to understanding potential impacts on dietary intake. In addition, a few panel members brought up the importance of LCS studies specifically in populations with type 2 diabetes, because this is a group likely to benefit the most from use of these products. Some published systematic reviews also recommend more research on the consumption of LCSs and their association with health risks related to excess body weight (18, 26, 27), as well as on overall dietary patterns (20). Some mechanistic questions of interest were around the sensing of LCSs at oral and gastrointestinal receptors, as well as their potential impact on gut microbiota and peptide signaling. It was generally recognized that the mechanistic “how” questions are important, but that it was more pressing to conduct public health population research on LCSs and these health outcomes.

Question 4 (Table 2) raised concerns about the long-term health risks of LCSs. An issue that came up in several interviews was the distinction between “safety” and “health risks” and the perception among some of the public that LCSs are “unsafe.” Because the scope of this FRN project included only human studies and the stakeholder panel did not include a toxicologist, the panel was not adequately equipped to assess the need for further safety research. However, our panel members thought it was important to clarify that the LCSs included in this FRN project have all been either confirmed as safe for use as a sweetener (food additive) by direct review of research or accepted as GRAS. The panel members knew of no evidence from human studies that these LCSs are unsafe and felt that the public's concern about the “artificial” nature of LCSs was the source of much of the confusion with regard to the safety of LCSs. It was further noted that consumers receive a lot of mixed messages from the media and various nutrition information about “diet” products containing LCSs. It has been suggested that lead scientists should take a more active role in communication, because they can best put the findings in context and help clarify misunderstandings (28).

Many stakeholder panel members also pointed to the need for more intervention studies to look at the relation between LCSs and other health outcomes of interest, as well as their underlying mechanisms. The idea that cross-sectional, observational studies are limited by reverse causality and are partially the reason for controversy around LCSs was mentioned in several interviews. Published systematic reviews have shared these concerns as well (20, 21). Multiple panel members also felt that there were enough intervention trials in the literature to justify that LCSs can be effective for weight loss and maintenance, but that due to public perception and some vocal opposition, more trials may be needed. In addition, it is important to note that the identification of evidence gaps via examination of published studies is limited to the questions addressed; if a question has not been examined in the literature, the need for studies to address it may not yet be identified and should be considered in future projects.

One recurring theme in the stakeholder interviews was the inadequacy of current dietary tools in accurately assessing intakes of LCSs. There was discussion about the weaknesses of reliance on self-reported dietary assessment measures and the need for improvement in nutrition research overall. Researchers on the panel generally felt that we need better information about products containing LCSs, including which sweeteners are used and in what amounts, which is an interest shared by Miller and Perez (20). There was also an interest expressed in collecting stronger data on differences in global intake as well as across different racial or ethnic groups.

Multiple stakeholders expressed an interest in identifying a way to evaluate LCS compounds as a class. For example, when a new LCS is developed with marketing plans, it would be advantageous to have an easy way to compare it with those currently in widespread use. This could be important, especially given the limitation that most sweeteners have not been researched as extensively as aspartame. Furthermore, although stakeholders recognized that this may not be possible, there was interest in seeing a common measurement or endpoint for all LCSs to be able to make a general statement about them as a group. Some endpoints that were proposed were sweetener detection, insulin response, perception of sweetness, or change in dietary pattern. Future research would be needed to assess the accuracy of any statements made about LCSs as a class of ingredients.

Strengths and limitations

The main advantages of this new FRN approach include the transparency of the process and the inclusion of diverse perspectives on how research needs are defined or determined for informing public health policy. We overcame the challenges in engaging a lay audience in the panel discussions by facilitating an online discussion forum and conducting one-on-one interviews. Although we used a novel approach to integrate evidence-mapping findings into the FRN approach, there are several important limitations of this approach. First, it is likely that our evidence-map database did not include all relevant published studies due to the limitations of our search strategies, as we only searched articles indexed on Medline because of resource constraints. Another important limitation is that, unlike systematic reviews, evidence mapping does not provide information on the quality of published studies included. Finally, although we made sure to select a panel with a wide range of expertise, the limited size of our stakeholder panel means that although our findings represent our best efforts to identify future research priorities, it is certain that our panel did not represent all perspectives. For example, as the project progressed, it became clear that the project would have benefited from including a toxicologist on the stakeholder panel.

Conclusions

In summary, the stakeholder panel identified and prioritized future research needs, with research on the role of LCSs in energy intake, weight loss, and appetite in healthy, overweight, and diabetic populations being given the highest priority. The integration of evidence mapping allowed our stakeholders to have a more informed conversation about the state of current research, and the semistructured interviews were invaluable in understanding how our stakeholder panel arrived at the final priority-ranked list of research questions. The recurrent concerns and confusions with regard to the “safety” of LCSs underscore the importance of communicating the science to the general public. Particularly in the nutrition sciences, emerging research studies have the potential to inform consumer choice.

In this FRN project, research questions that were deemed to have the largest public health impact by the stakeholder panel also received higher overall prioritization scores. However, it is important to point out that behavioral and mechanistic outcomes are currently not considered in the evidence-based health care policy decision-making process. The breadth of our FRN questions suggests that public health nutrition policy may need to consider a broader evidence base because of the complex relations between nutrition, lifestyle, and optimal health. The future research questions developed by our panel can serve as an important resource for researchers and research funders. Furthermore, by providing an open evidence-map database of published LCS research (15) that can be used to query what has been done on the basis of a specific research question of interest in a broad area, we hope to contribute to the process of knowledge translation from scientific findings into health practice or policy recommendations.