Evidence-based practice within nutrition: what are the barriers for improving the evidence and how can they be dealt with?

BACKGROUND: Evidence-based clinical research poses special barriers in the field of nutrition. The present review summarises the main barriers to research in the field of nutrition that are not common to all randomised clinical trials or trials on rare diseases and highlights opportunities for improvements.
METHODS: Systematic academic literature searches and internal European Clinical Research Infrastructure Network (ECRIN) communications during face-to-face meetings and telephone conferences from 2013 to 2017 within the context of the ECRIN Integrating Activity (ECRIN-IA) project.
RESULTS: Many nutrients occur in multiple forms that differ in biological activity, and several factors can alter their bioavailability which raises barriers to their assessment. These include specific difficulties with blinding procedures, with assessments of dietary intake, and with selecting appropriate outcomes as patient-centred outcomes may occur decennia into the future. The methodologies and regulations for drug trials are, however, applicable to nutrition trials.
CONCLUSIONS: Research on clinical nutrition should start by collecting clinical data systematically in databases and registries. Measurable patient-centred outcomes and appropriate study designs are needed. International cooperation and multistakeholder engagement are key for success.

Background

The European Clinical Research Infrastructure Network Integrating Activity (ECRIN-IA) project has initiated work to identify threats and barriers to evidence-based clinical practice [1-3].1

We identified common barriers affecting all clinical trials, namely inadequate knowledge and understanding of clinical research and trial methodology; lack of funding; excessive monitoring; restrictive interpretation of privacy law and lack of transparency; overly complex or inadequate regulatory requirements; and inadequate clinical research infrastructures [4]. Furthermore, we identified barriers towards the conduct of clinical research on rare diseases, including the direct consequence of rarity; limited knowledge on the natural history of rare diseases; the need for trial designs adapted to the small population size and clinical heterogeneity; the need for multinational randomised clinical trials and infrastructures to conduct such trials; the need for more sensitive outcomes to quantify clinical benefit; and the need for involvement of all the stakeholders in the study design and conduct [5]. In a related paper, we also assess the specific barriers towards clinical research on medical devices [6]. In the present paper, we assess the specific barriers towards clinical research on nutrition and examine how they can be broken down in order to improve the future evidence base.

Due to the ubiquitous nature of nutrition and the multiple metabolic effects induced by each food, nutrient, or micronutrient component, randomised clinical trials (RCTs) in this area face practical barriers [7]. By nature, some barriers can be difficult to resolve such as making an appropriate placebo for blinding of the interventions. Due to the multiple metabolic impact of nutrients, choosing the primary outcome and the determination of sample size may seem particularly difficult. Nutrition research on products is also complex because it exploits a multitude of bioactive compounds acting on an extensive network of interacting processes. In addition to measuring the variables of interest (assessing health), an extensive description of the trial participants and foods or diets consumed is essential. However, solutions exist to respect general rules of clinical trials despite these barriers.

The present paper gives an overview of the specific obstacles in methodology within nutrition research and provides solutions to these barriers (Table 1). The regulations in this area are also discussed.

Table 1

Barriers to the conduct of randomised clinical trials (RCTs) on nutrition

Specific barriers to RCTs on nutrition	Comments
Testing a nutrient	Should one use the nutrient as an add-on to usual diet, or should the diet be depleted of the nutrient?
Testing a food intervention	How to assess dietary intake
Assessing dietary intake	Combination of diverse methods including the 24-h dietary recall method are likely to be best
Selection of outcomes	Depends on the objective of the study

Barriers as identified by the European Clinical Research Infrastructure Network (ECRIN) panel in the context of the ECRIN-IA project. These barriers are additional to the barriers that affect all clinical trials and trials on rare diseases [4, 5]

Methods

The present study is based on a combination of systematic academic literature searches as well as internal ECRIN-IA communications from 2013 to 2017. In May 2016, the systematic search was performed using The Cochrane Library (Wiley) (Issue 5 of 12, 2016) (including the Cochrane Database of Systematic Reviews (CDSR), CENTRAL, National Health Service Economic Evaluation Database (NHSEED), and Database of Abstracts of Reviews of Effects (DARE, U.S. Library of Medicine)); MEDLINE (Ovid SP) (1946 to May 2016); EMBASE (Ovid SP) (1974 to May 2016); and Science Citation Index Expanded (1900 to May 2016) using different terms covering barriers, evidence-based medicine, and nutrition. The exact search strategy is provided in Additional file 1. A Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram depicting the selection process and a PRISMA Checklist are provided in Fig. 1 and Additional file 2, respectively. Articles were selected if they included valid considerations on how barriers to the conduct of randomised clinical trials (RCTs) on nutrition could affect their number, feasibility, and quality.

Fig. 1

Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) 2009 flow diagram. PRISMA flow diagram depicting the selection process of relevant academic literature

Results and discussion

Search results

Through the electronic searches a total of 4871 references were identified. After 589 duplicates were removed, a total of 4282 references remained, and were further reduced to yield 11 relevant references (Additional file 3).

Randomised clinical trials on nutrition

RCTs are costly and labourious to conduct, but they are essential [8-11]. The RCT is the only design which is able to demonstrate a causal relationship between a dietary change and health outcomes [8-11]. However, testing the effect of a diet or a specific nutrient in a trial is not the same as testing a drug and different obstacles are encountered (Table 1).

Testing a nutrient

If only a nutrient is tested, a placebo can be easily used, and an RCT with blinding can be conducted. Compliance can be checked by counting pills and possibly by assaying the nutrient concentration or related functions.

However, many nutrients occur in multiple forms that differ in biological activity, and there are several factors that can alter the bioavailability of individual nutrients. The nutrient status and the baseline exposure could also cause interference. The nutrient status of an individual or a population can affect the response to nutrient supplementation, e.g., via the gastrointestinal microbiota. For example, if one wants to examine the effect of polyphenol, adding polyphenol to polyphenol-depleted diets will give a good check on polyphenol dose. However, these special depleted diets (depleted in fruit, vegetable, and polyphenol-containing beverages) will have confounding effects and there will be no real control group. Another way will be to add polyphenol in addition to the polyphenols of a normal diet, but then both groups might receive sufficient polyphenol, and a possible effect of polyphenol might be overlooked. Both types of trials will give information, but different information. Background levels of exposures can be difficult to accurately determine due to limitations in currently available assessment methodologies of food intake, incomplete nutrient databases with which nutrient intake estimates are calculated, and temporal changes in exposure. This must be carefully considered in the trial design, including randomisation and subgroup analysis per baseline status. For smaller RCTs (say less than 400 to 500 participants), adjustment according to the baseline level of consumption or nutrient level in blood may be necessary [12].

Testing a food intervention

Food-based interventions are complex; an appropriate placebo for food cannot be designed so RCTs cannot easily be blinded. However, outcome assessors, data collectors, statisticians, and conclusion drawers should be blinded. Furthermore, a specific design with food made with or without the particular nutrient can be conducted.

Thus, food-based nutrition interventions, in contrast to nutrient supplementation, present unique challenges in accurately quantifying the absolute change in intake. Compliance assessment in these trials is more difficult but biomarkers of food intake may be used (alpha-linolenic acid as a proxy for nut intake or some polyphenols, such as urinary tyrosol, for virgin olive oil) [13]. Certain periods of holidays or religious festivities (such as Ramadan) should be avoided to reduce interference from special nutritional patterns during such periods. The seasonal impact on food intake also needs to be taken into account [14]. The optimal length of intervention to have a relevant effect can also be difficult to determine. Here again, particular attention must be paid when testing a food product, the product will not have the same impact if it substitutes another product or if it is added to the usual diet.

The ILSI Europe Functional Foods Task Force guidelines [15] are a relevant source on how to conduct nutrition intervention studies and can help resolve some of those questions.

Assessing dietary intake

Assessing dietary intake is a major challenge in nutrition research despite the large spectrum of research methodologies available.

Quantification of habitual dietary exposure by measurement of food intake combined with food composition is essential for investigating the impact of food on health. The assessment of dietary intake at the population level provides important information on the frequency and distribution of inadequate diets and nutritional status and guides the design of population-based interventions targeting the improvement of dietary habits at the community level. Obtaining reliable data on food consumption (identifying the intake of energy and nutrients) is a key factor and a necessary tool in health promotion and prediction of disease risk [16].

Difficulties are encountered in the assessment of dietary intake. Valid methods are required to assess the effects of a huge range of diverse foods and compounds (macro- and micronutrients, and non-nutritional bioactive compounds) on individual health. A wide variety of dietary survey methods exists, with each one presenting a series of advantages and disadvantages that must be taken into consideration based on the study aims, the degree of precision needed and the available resources. Common methods involving dietary questionnaires are:

Food frequency

24-h recall

Dietary history over a specific period

Food diary associated with photographic food record

Weighed food record

Specific dietary items or patterns

The validity of these approaches has been questioned many times [17]. A volunteer assessing their dietary intake may modify their eating behaviour, thus introducing an another issue in dietary assessment. Dietary intake is difficult to measure and a single instrument that is optimal for all settings does not exist, as each method has its pros and cons or practical difficulties that should be taken into account when selecting the instrument to be administered. For example, a weighed-food record is precise but rather cumbersome for the volunteers and can lead to underreporting. The method used could be improved by using more frequent dietary recall.

Expectations for better assessment of dietary intake are put on new methods, such as research in metabolomics, metagenomics, and natural enrichment of stable isotopes, which could be helpful in terms of the validation of methods and useful in addition to standard methods. Technological devices, such as cellular phones, may also improve the quality of data collected. The majority of the latest-generation technologies offer interesting tools for the process of evaluating dietary intake in epidemiological studies [18-21]. The advantages of digital instruments include: (1) reduction in interviewer bias; (2) reduction in the time and cost of field work; (3) data collection and codification in real time; (4) automatic calculation of daily intake; and (5) highly economic options of capturing food intake online (computer, tablet, and smartphone).

New technologies provide many possibilities for assessing dietary intake in individuals and groups, but are not free of certain limitations: high cost of programme design and acquisition of devices; difficulty of application to certain populations that are not familiar with new technologies; and dependence on access to Internet and on the trial participant’s recall capacity.

Despite the interesting progress and the incorporation of innovative technology into dietary assessment methods, the same flaws remain. As such, the combination of diverse methods could be the best approach, with the 24-h dietary recall being a rather reliable comprehensive and easy-to-use instrument [22-24].

Research over recent years in this field has focussed on estimating the adherence to a determined dietary pattern rather than analysing the individual components of the diet in relation to the health of the population. Many indirect indices have been proposed. These indices summarise the diet by means of a single score that results from a function of numerous components (previously selected on the basis of prior knowledge or scientific evidence) such as food, food groups, or a combination of foods and nutrients [25]. The validity of these indices has been evaluated by examining their relationship with nutrient adequacy and several health outcomes (i.e., nutrient-related diseases). For instance, indices evaluating adherence to the Mediterranean diet, like the Mediterranean food pattern PREDIMED study (MeDiet-PREDIMED) [26], the Mediterranean Adequacy Index (MAI) [27], the Mediterranean Diet Score (MDS) [28], or the Mediterranean Diet Quality Index (KIDMED) for evaluating the quality of the Mediterranean diet in children and adolescents [29], have been successfully applied to studies on life expectancy [30], cardiovascular risk [31-33], cancer [34, 35], hypertension [36], obesity [37, 38], and diabetes [39]. The study of dietary patterns using dietary quality indices to examine interactions between dietary components and their effect on several health variables can be complementary to the analysis of single dietary components [40, 41]. Moreover, these indices are also useful tools to measure food consumption trends and to identify the involved factors, as well as to develop comprehensive public health nutrition recommendations.

Approaches to evidence-based medicine, with its reliance on RCTs and systematic reviews of RCTs [1] have been adapted to nutrition systematic reviews and policy [42-45]. However, there are distinct differences between the evidence that can be obtained for the testing of drugs using RCTs and those needed for the development of nutrient requirements or dietary guidelines [46]. Nutrients tend to work in systems in concert with other nutrients and affect multiple cells and organs; nutrients are homeostatically controlled and thus the body’s baseline nutrient ‘status’ affects the response to a nutrient intervention; a nutrient intervention group cannot be contrasted with a true placebo group (i.e., a ‘zero-exposure’ group). The subtle effect of nutrients and small effect sizes means that far more participants are needed to demonstrate statistical significance depending on the research question and the target population.

Chronic diseases can take decades to develop, so demonstrating a statistically significant and clinically relevant reduction in risk with any intervention requires very long-term trials. Moreover, maintaining adherence to a dietary change all through a trial period is difficult. RCTs present one approach toward understanding the efficacy of nutrient interventions, but the innate complexities of nutrient actions and interactions cannot always be adequately addressed through a single research design.

Outcomes

Outcomes could vary according to the objectives of a study. In mechanistic proof-of-concept studies, they could be very specific and explanatory [47]. The sample size should be calculated according to the variability of the measured primary outcome which could be either a single outcome or a composite outcome.

Secondary outcomes should have enough power (at least 80%) to determine a conclusion; if not, they should only be used as exploratory outcomes [1]. Outcomes should concern nutritional status, relevant markers of functional status, quality of life, morbidity, and mortality. Hard outcomes should always be preferred when assessing the effects of clinical nutrition interventions, whereas intermediate markers might be used in addition to hard outcomes in preventive nutrition.

Numerous experimental nutrition studies have used putative surrogate outcomes and markers of disease risk, but only a few such markers have been validated using hard outcomes [1]. Nutrition research needs better biomarkers of both exposures and outcome. Defining approved panels of outcomes for particular health outcomes would allow easier conduct of systematic reviews and meta-analyses. It is mandatory to have precise outcomes to predefine the moment of their measurement, as is the case for all types of research, and to decide them before commencing RCTs [1].

For dietary supplements and functional food, studies could also target non-diseased populations. Statistically, however, there are challenges in performing studies in a non-diseased, healthy population. In healthy volunteers, the magnitude of functional changes and their clinical relevance are likely to be more difficult to highlight than it would be in diseased participants and the likelihood of demonstrating a statistically significant difference is diminished. Experimental designs including metabolic challenges (such as short-term overfeeding or a fat or fructose load) could be useful to screen for the protective effect of some nutrients. Studies in high-risk populations are encouraged.

The need to accurately capture subtle changes in a multitude of variables creates several challenges. Standardised technology, methodology, and data formats are required for meeting these challenges. Elements of these issues are common to all biological sciences and efforts to produce solutions and best practices for technologies and data handling in these areas are under investigation [48].

Single-centre trials are easier to conduct than multicentre trials and may be done more efficiently with a homogenous population. However, they have less external validity and often do not reach a required sample size. Accordingly, it is often necessary to organise multicentre trials in order to achieve an adequate sample size. In general, results from a multicentre trial will be more convincing than those coming from a single centre.

Solutions to the specific problems of nutrition research could be solved by the extension of good clinical research practice and harmonising procedures.

System studies and observational studies

Observational studies derived from large epidemiological surveys are able to raise hypotheses in terms of the association between dietary patterns and health outcomes [49].

These hypotheses, as well as other hypothesis derived from experimental research, should be the basis for RCTs.

More and more observational and epidemiological studies, which are carried out on a dietary basis, have attempted to associate the intake of particular food group with disease prevalence. The results are rarely unidirectional, and it still remains unclear whether certain food groups may be considered as definitively protective or deleterious. Therefore, a more holistic approach based on cumulative scientific evidence, i.e., data mining, may be needed [50]. This global approach draws updated pictures on the research led on such associations and helps to unravel research needs. For instance, based on meta-analyses and systematic reviews of articles from 1950 to 2013 referring to observational and intervention studies, we were able to identify tendencies related to the health-protectiveness of 17 food groups and beverages towards 10 disease risks [51]. Among these tendencies, some can be considered as strong and may potentially be converted into clear and durable recommendations for public policies related to preventive nutrition and health, while others remain ambiguous.

Another important aspect of nutrition research is that exposure to specific food groups or nutrients may induce metabolic disorders long before any non-communicable chronic disease may be detected in relation to the diet. The origins of these diseases are multifactorial and may result from at least 10 different deregulated metabolic variables, including antioxidant status, acid-base imbalance, increased inflammatory status, impaired carbohydrate/lipid/one-carbon metabolism, impaired functioning of neurons and deoxyribonucleic acid (DNA) transcription, hypertension and/or modified digestive microbiota [52]. Such a systems approach represents an important opportunity to identify early biomarkers of future chronic diseases in association with dietary nutrients. So, technological development allowing rapid and large screening of metabolites, i.e., metabolomics, in plasma or urine samples, will likely be of high value for detecting dietary risk factors in individuals [53].

Regulations

The level of the quality of nutrition research is also dependent on regulations and practices that have been approved by EFSA (The European Food Safety Authority). The authority has indicated a preference for clinical studies to back health claims [13]. Placebo-controlled trials are the ‘gold standard’ with either a parallel or crossover design. RCTs involving whole foods or diets are difficult to conduct, perhaps even more so than nutrient-based trials, but for the same reasons: lack of compliance, no true placebo group, and the impossibility of having a control group. Furthermore, the innate complexities of nutrient actions and interactions cannot always be adequately addressed through any single research design.

Good clinical practice is applicable to interventional research beyond health products, particularly in nutrition, to ensure the reliability and reproducibility of the data and to safeguard the rights and safety of research participants and the confidentiality of information that concern them [54, 55].

The great heterogeneity of national regulations regarding non-drug research mainly depends on the consideration of a dietary supplement as a drug or not and the level of intervention imposed on the patient. There is a need to harmonise the procedures for evaluating nutrition clinical trials in Europe. One of the deliverables of WP6 within ECRIN-IA has been to collect the different national regulation requirements and develop a pan-European human nutrition regulatory and ethics database [56].

For the US Food and Drug Administration (FDA), if an Investigational New Drug (IND) is needed for a clinical investigation, evaluating a dietary supplement is determined by the intent of the clinical investigation. If the clinical investigation is intended only to evaluate the dietary supplement’s effect on the structure or function of the body, an IND is not required. However, if the clinical investigation is intended to evaluate the dietary supplement’s ability to diagnose, cure, mitigate, treat, or prevent a disease, an IND is required under part 312 [54]. The European Commission has now adopted a revision of the Directive on Clinical Trials (2001-20-EC) 17 July 2012 [57], Regulation EU No. 536/2014 which will come into force in 2019 at the earliest [58]. This should facilitate the conduct of multinational clinical trials with particular attention to the acceleration and simplification of licensing and reporting clinical trials procedures. Through this proposed regulation, it will be possible to ensure, in all member states, identical rules for the authorisation, execution, and monitoring of clinical trials.

Conclusions

Nutrition research must apply reproducible and valid methodologies [8, 15]. The methodologies and regulations for drug trials are applicable to trials in the field of nutrition. A consensual approach involving institutional and industrial partners of the clinical nutrition research would be useful for standardisation of these practices on specific aspects such as the choice of a placebo, the design of the test products, the selection of control participants, the choice of outcomes, and sample size estimation. In collaboration with ECRIN, it will be necessary to standardise nutritional and outcome assessment, standardise operating procedures for existing and new methodologies, and standardise data collection, data management and data sharing, which will make studies comparable and interoperable for systematic reviews meta-analyses.

Taking into account the specificities of nutrition studies and developing international collaboration will enhance the quality of nutrition research [1].

Literature search strategy. Description of data: exact search strategy applied for analyses. (DOCX 14 kb)

PRISMA 2009 Checklist. (DOC 63 kb)

Relevant references from the academic literature search. Results from the academic literature search are listed in the form of relevant publications. (DOCX 14 kb)