How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials.

INTRODUCTION: The Mediterranean diet (MD) is a traditional regional dietary pattern and a healthy diet recommended for the primary and secondary prevention of various diseases and health conditions. Results from the higher level of primary evidence, namely randomised controlled trials (RCTs), are often used to produce dietary recommendations; however, the robustness of RCTs with MD interventions is unknown.
METHODS: A systematic search was conducted and all MD RCTs with dichotomous primary outcomes were extracted from PubMed. The fragility (FI) and the reverse fragility index (RFI) were calculated for the trials with significant and non-significant comparisons, respectively.
RESULTS: Out of 27 RCTs of parallel design, the majority failed to present a significant primary outcome, exhibiting an FI equal to 0. The median FI of the significant comparisons was 5, ranging between 1 and 39. More than half of the comparisons had an FI <5, indicating that the addition of 1-4 events to the treatment arm eliminated the statistical significance. For the comparisons with an FI=0, the RFI ranged between 1 and 29 (Median RFI: 7). When the included RCTs were stratified according to masking, the use of a composite primary endpoint, sample size, outcome category, or dietary adherence assessment method, no differences were exhibited in the FI and RFI between groups, except for the RFI among different compliance assessment methods.
CONCLUSIONS: In essence, the present study shows that even in the top tiers of evidence hierarchy, research on the MD may lack robustness, setting concerns for the formulation of nutrition recommendations. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Recommendations for the adoption of the Mediterranean diet (MD) for improved health outcomes are based mainly on randomised controlled trials (RCTs) and their synthesis.

The robustness of RCTs with MD interventions appears to be low to moderate. Similarly, fragility (FIs) and reverse fragility indexes (RFIs) have also been reported among RCTs in other therapeutic domains, including clinical nutrition, anesthesiology, perioperative medicine, etc.

The FI and RFI can be used to improve and promote the science of nutrition.

Background

Since Keys first presented a diet-mortality hypothesis explaining the Seven Countries study results in 1986,1 the Mediterranean diet (MD) has become a dietary pattern of particular interest. Research on the MD has spiralled,2 3 reputed for its health effects, spanning from ameliorated cardiovascular disease (CVD) factors,4–7 to improved pregnancy outcomes,8 ticking all the boxes in the quest for health attainment. For some, the MD is much more than a traditional regional dietary pattern, being regarded as the ‘unicorn’ of diet paradigms, with many clinical practice guidelines endorsing the adherence of the MD.9 10

Apart from many ‘followers’ however, several scientists are also questioning the MD. Some are high-lightening the observational design of the Seven Countries study,11 while others are stressing the limitations of nutritional epidemiology in general,12 often incorporating selective reporting,13 inflated results,14 over-interpretation and skewed perspectives,15 with large flexibility in the performed analyses which can be based οn questionnaires of low reproducibility.16

Subsequently, research designs were improved to minimise bias,14 and the focus shifted to randomised controlled trials (RCTs), situated higher in the pyramid of evidence.17 The worm turned again when the biggest and most promising MD trial to date, the Prevención con Dieta Mediterránea (PREDIMED),18 19 raised concerns over randomisation bias, resulting in its reanalysis.20 Nutrition RCTs were once more in the spotlight, and scepticism was apparent,21 with researchers questioning the suitability of RCTs for nutrition research and the quality of the trials. Most trials tend to report positive findings21; however, statistical significance (P-value) does not ensure the robustness of an analysis and a pledge towards the use of more specific measures was made.22–25

Today, clinical research continues to emphasise the P threshold of 0.05 when interpreting RCT results.26 For this, it is additionally important to evaluate the robustness of RCTs with MD interventions and attain an additional measure of the quality of MD RCTs. Two indexes have been proposed for the evaluation of an RCTs’ robustness,27 namely the fragility index (FI) and the reverse fragility index (RFI), for trials with significant or non-significant findings, respectively. Both indexes can only be calculated on studies with an RCT design and dichotomous primary outcomes.

To assess the robustness of RCTs with MD interventions, the present research-on-research study aimed to identify all RCTs with MD interventions and dichotomous primary outcomes, and calculate their FI or RFI, depending on the significance of the comparisons.

Methods

Research question and search strategy

The present study used a systematic search strategy to answer the question “What is the fragility and reverse fragility index of RCTs assessing MD interventions?” The PICO of the study’s hypothesis was P: human population of any age group or health status, I: MD intervention, C: any comparison other than the MD, a sham diet, other diet or no intervention, O: any dichotomous primary outcome (table 1). To answer this research question, the focus was set on all RCTs examining MD interventions, irrespective of their other characteristics. Similar studies examining the FI/RFI in broad research areas are common in the literature.26

Table 1

PICO strategy of the study’s research question

Population	Randomised controlled trials performed on humans of any age and health status
Intervention	Mediterranean diet
Comparator(s)	Any dietary regime other than the Mediterranean diet, including a sham diet, nutrient supplementation or no intervention at all
Outcome(s)	Any outcome (perinatal, cardiovascular, metabolic or other)

The protocol of the study was published at the Center for Open Science https://osf.io/mnx2c/. A systematic search was conducted on PubMed from inception until 31 August 2019, using the keyword (Mediterranean diet) and the PubMed filter for clinical trials.

Inclusion and exclusion criteria

As the concept of fragility is only applicable to RCTs, only studies with an RCT design were considered eligible.28 In parallel, we searched for trials with dichotomous primary outcomes, as the FI and RFI cannot be calculated in trials with continuous outcomes. Secondary outcomes were not of concern as they are not accounted for when estimating the sample size required for an RCT and should not be used to assess a trial’s robustness.29 All RCTs with MD interventions were assessed for eligibility, despite other possible heterogeneities, as the research question focused on the FI and RFI of MD interventions in general and not in MD RCTs with more homogenous outcomes/samples/designs.

The criteria for inclusion in the present analyses involved (1) RCTs performed on humans, (2) of any age group, (3) irrespectively of any medical diagnosis or health condition, (4) applying MD interventions, (5) compared with no intervention, control diet, or to dietary patterns other than the MD, (6) assessing any dichotomous primary outcome and (7) published in any language.

On the other hand, criteria for exclusion involved trials (1) lacking randomisation, (2) performed on animals, (3) with continuous primary outcomes or (4) with dichotomous secondary outcomes, (5) comparing MD interventions to control diets based on the MD, (6) not including an MD intervention, (7) not reporting the number of events and the sample size in each arm, making it impossible to calculate 2×2 frequency tables, (8) failing to report adequate data to calculate persons–years, (9) trial protocols without results and (10) research performed on animals.

Data extraction

Two researchers (MGG and XT) independently extracted data from the selected RCTs, aided by an additional pair of reviewers (MPN and KG) when deemed necessary. Extracted data involved details regarding the study design, the level of masking (open label/single/double), sample size, protocol registration details, study name/acronym, interventions and comparators, the primary outcomes, the event rates in each arm, the geographical origin of the trial, the randomisation methods used, the level of prevention (primary/secondary) and the methods used to assess intervention adherence. As far as time-to-event outcomes are concerned, extracted data involved the total number of events in each arm over the entire follow-up period of each trial.

Risk of bias

The risk of bias (RoB) of the selected RCTs was evaluated using the Cochrane RoB V.2.0 tool30 by two independent researchers (MPN and KG). Disagreements were resolved via discussion and whenever needed, through the intervention of more experienced researchers (DGG, MGG and DPB).

Calculation of the FI and RFI

The FI was developed as a measure of RCT robustness. It describes the minimum number of patients within the group with the fewest event count needed to change from a non-event to an event, to transform a significant result to a non-significant one.27 It is considered as the measurement of the event count, on which the statistical significance depends.22

For the current analysis, two researchers (XT and MGG) calculated the FI of each RCT, according to Walsh et al.27 In further detail, after extracting the number of events and non-events for each trial arm in 2×2 tables, the additional number of events required to be added in the group with the smaller number of events to make the p value of the Fisher’s exact test ≥0.05 was calculated.

An FI equal to zero describes a highly fragile RCT, as zero participants are required to change from a non-event to an event to reverse a significant finding to a non-significant one.22

On the other hand, in non-significant comparisons (with an FI equal to 0), the RFI was calculated. This was performed via the subtraction of events from the arm with the fewer events, while simultaneously adding non-events to the same arm, keeping the number of total participants constant, until the Fisher exact test two-sided P-value became <0.05. Lower RFIs indicate reduced statistical robustness and increased vulnerability to change from statistical non-significance to significance, with only a minimum number of events. At the moment, there is no recognised cut-off for categorising either the FI or the RFI.26

For the current analyses, 2×2 tables were created in Microsoft Excel and the Fischer’s exact test was used to calculate and verify the FIs and RFIs of the included trials. For one trial,31 32 the reported sample and events in each group were used to calculate the FI, and for another,33 the incidence and the total number of participants allocated in each group were applied in the FI calculations. When more than two interventions were included in one trial, like in the PREDIMED, each arm was compared with the control diet independently, and the FI or RFI was calculated accordingly, for each paired comparison. When the primary outcome was not reported, the first result presented in the abstract was considered as the primary outcome. In RCTs reporting more than one dichotomous primary outcomes, the FI of all three endpoints was calculated accordingly.

Statistical analyses

As the research question was ‘broad’, incorporating all RCTs with MD interventions, an effort to assess differences in RCTs with different characteristics was also performed. Three researchers (KG, MPN and MGG) stratified the selected trials according to blinding, outcome category, sample size, the use of a composite outcome (yes/no) and the method used to assess compliance to the assigned dietary scheme. These categories were used to detect differences in the FI and the RFI between RCTs with different design characteristics and outcomes. As most data did not follow the normal distribution hypothesis, results were presented as medians with their respective IQRs. Group differences were assessed with the Mann-Whitney U test (for comparisons involving two groups) and the Kruskal-Wallis test (for comparisons involving more than two groups). For these analyses, the Jamovi project (V.0.9.5.16) was used. Significance was set at 0.05, unless otherwise specified.

Results

Search results and RCT characteristics

The detailed process of the selection of RCTs fulfilling the study’s criteria is illustrated in figure 1. Published protocols of RCTs lacking the reporting of results, published studies with design issues (cross-sectional, qualitative, or post-hoc analyses), RCTs without dichotomous primary outcomes, and trials lacking a MD intervention, or those with a concomitant MD comparator arm were excluded from the records. A total of 35 distinct publications18 19 31–63 of parallel interventions were identified meeting the predefined criteria (table 2), with those having an original publication and an erratum being counted as one record (five cases in total).18 19 37–39 44 46–48 63 Multiple publications deriving from the same trials, using the same sample size and outcomes, were also counted as one record (three cases in total).31–33 35 36 53 This resulted in 27 distinct RCTs in total, fulfilling the study’s criteria and being included in the present analyses.

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart of the selection of the studies. MD, Mediterranean diet; RCT, randomised controlled trial.

Table 2

Characteristics of the included trials

First author	Origin	Trial name	Design,masking	Registry	Publication(journal, year)	Participants*	Randomisation	Prevention tier	Primary outcome	Intervention	Comparator	Compliance
Al Wattar58	UK	ESTEEM	Pragmatic,Parallel,Single blind	NCT02218931	PLOS Med, 2019	N=1252 inner-city pregnant women with metabolic risk factors (obesity, hypertension, or hypertriglyceridaemia)	1:1 ratio via a password-protected on-line data management system	Primary	Maternal composite outcome,† offspring composite outcome‡	MD high in nuts, EVOO, fruits, vegetables, non-refined grains and legumes, moderate-to-high fish, low-to-moderate poultry and dairy, low intake of red/processed meat, avoidance of sugar, fast food and food rich in animal fat	Usual care (dietary advice)	FFQ and ESTEEM Q
Assaf-Balut59	Spain	St. Carlos GDM prevention study	Parallel,Open label	ISRCTN84389045	PLOS One, 2017	N=1000 normoglycaemic (<92 mg/dL) pregnant women at 8±12 gestational wk	Stratified with permutated block randomisation, by age, pregravid BMI, ethnicity, parity, in a 1:1 ratio and 4–6 blocks	Primary	GDM incidence	MD supplemented with EVOO and pistachios (≥40 mL of EVOO and 25–30 g of pistachios each day)	Standard diet with limited fat intake	MEDAS, DNCT FFQ, urine HXT and serum γ-tocopherol
Assaf-Balut60	Spain	St. Carlos GDM prevention study	Parallel,Open label	ISRCTN84389045	Ann Nutr Metab, 2019	N=697 normoglycaemic (<92 mg/dL) pregnant women at 8±12 gestational wk	Stratified with permutated block randomisation, by age, pregravid BMI, ethnicity, parity, in a 1:1 ratio and 4–6 blocks	Primary	Composite maternofetal outcome§	MD supplemented with EVOO and pistachios (≥40 mL of EVOO, 25–30 g of pistachios every day)	Standard diet with limited fat intake	MEDAS, DNCT FFQ, urine HXT, serum γ-tocopherol
Babio61	Spain	PREDIMED	Parallel, single blind	ISRCTN35739639	CMAJ, 2014	N=5801 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	MetS	MD with EVOO (1 L/wk for the participants and families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS/9-item dietary screener (per arm)
de Lorgeril62	France	Lyon Diet Heart Study	Parallel, single-blind	NR	Circulation, 1999	N=423 consecutive patients who survived a first MI at 6 months of enrolment	NR	Secondary	Composite outcome††	MD: more bread, root, green vegetables and fish, less meat (beef/lamb/pork replaced by poultry), no day without fruit. Butter/cream replaced by canola margarine	Prudent Western-type diet	Diet ‘survey’ (MD group), plasma FA
de Lorgeril34	France	Lyon Diet Heart Study	Parallel, single-blind	NR	Arch Intern Med, 1998	N=605 consecutive patients who survived a first MI within 6 months of enrolment	NR	Secondary	CV mortality, non-fatal MI	MD: more bread, root, green vegetables and fish, less meat (beef/lamb/pork replaced by poultry), no day without fruit. Butter/cream replaced by canola margarine	Prudent Western-type diet	24 hours recall and FFQ
de Lorgeril35 36	France	Lyon Diet Heart Study	Parallel, single-blind	NR	J Am Coll Cardiol, 1996; Lancet, 1994	N=605 consecutive patients who survived a first MI within 6 months of enrolment	NR	Secondary	CV mortality, non-fatal MI	MD: more bread, root, green vegetables and fish, less meat (beef/lamb/pork replaced by poultry), no day without fruit. Butter/cream replaced by canola margarine	Prudent Western-type diet	24 hours recall and FFQ
Díaz-López37 38	Spain	PREDIMED	Parallel,Single blind	ISRCTN35739639	Diabetes Care, 2015; Rev Esp Cardiol, 2019	N=3614 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	New-onset of diabetic retinopathy, nephropathy	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	Urine HXT and plasma ALA proportions
Esposito31 32	Italy	–	Parallel, open-label	NCT00725257	Diabetes Care, 2014; Ann Intern Med, 2009	N=215 men/women (30–75 years) with newly diagnosed T2DM	‘Simple’ randomisation; PC-generated random sequence	Secondary	Initiation of T2DM medication	LCMD	LFD	Diet diaries
Estruch18 19	Spain	PREDIMED	Parallel, single-blind	ISRCTN35739639	NEJM, 2018	N=7447 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 sealed envelopes (pilot phase) and PC-generated random number	Primary	CompositeCV events‡‡	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS/9-item dietary screener, urine HXT, plasma ALA
García-Gavilán33 53	Spain	PREDIMEDReus	Parallel, single-blind	ISRCTN35739639	Clin Nutr, 2018; Am J Clin Nutr, 2018	N=870 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Osteoporotic fractures	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS
García-Layana40	Spain	PREDIMED	Parallel, single-blind	ISRCTN35739639	Nutrients, 2017	N=5802 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Occurrence of cataract surgery	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS, urine HXT and plasma ALA ratio
GenéHuguet41	Spain	Pre Frail 80 Study	Parallel, open-label	NR	J Nutr Health Aging, 2018	N=200 non-institutionalised men/women (≥80 years) fulfilling 1/2 of the Fried65 frailty criteria	Randomised list NOD	Secondary	Reversion to robustness	MD	Standard treatment	MEDAS
Greenberg57	Israel	DIRECT	Parallel, NR	NCT00160108	J Am Coll Nutr, 2009	N=322 men/women(40–65 years) with BMI≥27 kg/m2/T2DM/CHD	Based on sex, age,BMI, history of CHD/T2DM and statins use	Secondary	5% BW loss	Hypocaloric§§ MD based on Willet.Fat: 40% (mainly olive oil and nuts)	(1) Atkins-based LCD. CHO 20 g/d (first 2 months), ≤100 g/d thereafter.Unlimited EI, protein, fat.(2) Hypocaloric§§ LFD based on the AHA. Fat: 20%–30% (SFA 7%–10%, 200–300 mg cholesterol)	127-itemFFQ with 3 portion size pictures for 17 selected items
Marcos-Forniol42	Spain	–	Parallel, open-label	ISRCTN17382091	Eur J Prev Cardiol, 2018	N=127 consecutive patients (≥70 years), with acute coronary syndrome¶¶	1:1 PC allocation with random block sizes of 2	Secondary	Optimal CVD risk factor control***	MD	Standard care	9-item MD score
Martínez-González43	Spain	PREDIMED	Parallel, open-label	ISRCTN35739639	Circulation, 2014	N=7447 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Incidence of atrial fibrillation	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS, urine HXT, plasma ALA ratio
Papadaki44 63	Spain	PREDIMED	Parallel, open-label	ISRCTN35739639	Eur J Heart Fail, 2017	N=7403 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Heart failure incidence	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS
Pintó52	Spain	PREDIMED	Parallel, open-label	ISRCTN35739639	J Nutr, 2019	N=109 consecutive patients, men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Steatosis diagnosis	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS
Properzi45	Australia	–	Parallel, single-blind	ACTRN12612000841875	Hepatology, 2018	N=56 adult patients with NAFLD	In a 1:1 fashion using randomly selected envelope-concealed allocations in blocks of 4	Primary	NAFLD resolution	MD based on foods consumed in traditional Cretan diet, altered to allow for standardisation of protein intake with the control diet. CHO: 40%, fat: 35%–40% (<10% SFA), protein:<20%	LFD (based on NHMRC and AHA). CHO: 50%, fat: 30%, SFA <10%, protein: 20%	Modified Burke diet history, self-assessment of food-group goals, MEDAS
Ruiz-Canela54	Spain	PREDIMED	Parallel, open-label	ISRCTN35739639	JAMA, 2014	N=4991 men/women (55–80 years) PAD-free and CVD-free but with T2DM and/or ≥3 CVD risk factors¶	1:1:1 ratio	Primary	New symptomatic PAD events	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS
Salas-Salvadó55	Spain	PREDIMED	Parallel, open-label	ISRCTN35739639	Arch Intern Med, 2008	N=3923 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	NR but based on the PREDIMED protocol 1:1:1 PC-generated randomisation table**	Primary and secondary	MetS reversion rate and incidence	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS, urine tyrosol and HXT, plasma ALA ratio
Salas-Salvadó 39 46	Spain	PREDIMED Reus	Parallel, open-label	ISRCTN35739639	Diabetes Care, 2011; 2018	N=418 non-diabetic, CVD-free men/women with ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Diabetes incidence	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS
Salas-Salvadó 47 48	Spain	PREDIMED	Parallel, open-label	ISRCTN35739639	Ann Intern Med, 2014; 2018	N=3541 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	New-onset of diabetes	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS/9-item dietary screener, urine HXT, plasma ALA
Sánchez-Villegas56	Spain	PREDIMED	Parallel, open-label	ISRCTN35739639	BMC Medicine, 2013	N=3923 men/women (55–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Depression	MD with EVOO (1 L/wk for the participant and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS
Singh49	India	Indo-MD HeartStudy	Parallel, single-blind	NR	Lancet, 2002	N=1000 men/women (28–75 years) with hypercholesterolaemia, hypertension, DM, angina pectoris, previous MI	By selection of a card from a pile of an equal number of cards for each group	Secondary	Total cardiac events	NCEP prudent diet (fat: 30%, SFA <10%, cholesterol <300 mg/d), >400–500 g of fruit, vegetable and nuts, 400–500 g whole-grains, legumes, rice, maize, wheat, 34 serv of mustard seed/soybean oil	NCEP prudent diet	Weight food records and 24 hours nutrient intakes
Toledo50	Spain	PREDIMED	Parallel, single-blind	ISRCTN35739639	JAMA Intern Med, 2015	N=4282 women (60–80 years) with T2DM and/or ≥3 CVD risk factors¶	1:1:1 PC-generated randomisation table**	Primary	Breast cancer incidence	MD with EVOO (1 L/wk for participants and their families) MD with mixed nuts (30 g/d: 15 g walnuts, 7.5 g hazelnuts, 7.5 g almonds)	Control diet (advice to reduce dietary fat)	MEDAS/9-item dietary screener (per group)
Tuttle51	USA	THIS-DIET	Parallel, open-label(blind PI)	–	Am J Cardiol, 2008	N=101 MI survivors	Sealed envelopes with the allocation sequence, prepared by a PI, placed in a locked drawer	Primary	Free survival†††	MD with fat: 30%–40% (<7% SFA), CHO: 50%, protein: 10%–20%	LFD (AHA step II). Fat:<30%, protein: 10%–20%, SFA <7%, CHO: 55%–60%	Self-reported 3-d food diaries, verified by plasma FA

*Number of initially randomised participants or in the secondary analyses of the Prevención con Dieta Mediterránea (PREDIMED) study, the number of initially randomised participants as stated in the respective papers.

†Maternal composite outcome: gestational diabetes mellitus (GDM) or preeclampsia.67

‡Offspring composite outcome: stillbirth, small-for-gestational age (SGA), or admission to neonatal care unit.67

§Emergency caesarean section, perineal trauma, pregnancy-induced hypertension and preeclampsia, prematurity, large-for-gestational age, and SGA.

¶Smoking, hypertension, elevated low-density lipoprotein (LDL) level, low high-density lipoprotein level, overweight/obesity, or family history of premature coronary heart disease (CHD).

**Concerns regarding randomisation rose post publication.

††cardiac death and non-fatal myocardial infarction (MI).

‡‡MI, stroke, or death from cardiovascular (CV) causes.

§§Hypocaloric, 1200–1500 kcal/day for women and 1500–1800 kcal/day for men.

¶¶ST-elevation MI, non-ST elevation MI and unstable angina.

***Achievement of ≥5 risk factor goals: blood pressure <140/90 mm Hg, LDL <2.6 mmol/L, smoking cessation, body mass index (BMI) <25 kg/m2, physical activity of moderate intensity >30 min/day, 3 days/week (≥6 MET h/wk) and HbA1c <7% in patients with diabetes.

†††Composite of all-cause and cardiac deaths, MI, hospital admissions for heart failure, unstable angina pectoris or stroke.

‡‡‡MI, stroke, or CV death.

AHA, American Heart Association; ALA, α-linolenic acid; BW, body weight; CHO, carbohydrate; CVD, cardiovascular disease; DIRECT, dietary intervention randomised controlled trial; DM, diabetes mellitus; DNCT, diabetes nutrition and complications trial; EI, energy intake; ESTEEM, Effect of Simple, Targeted Diet in Pregnant Women With Metabolic Risk Factors on Pregnancy Outcomes; EVOO, extra-virgin olive oil; FA, fatty acid; FFQ, food frequency questionnaire; HXT, hydroxytysosol; LCD, low-carbohydrate diet; LCMD, low-carbohydrate MD; LFD, low-fat diet; MD, Mediterranean diet; MEDAS, Mediterranean Diet Adherence Screener66; MET, metabolic equivalents; MetS, metabolic syndrome; NAFLD, non-alcoholic fatty liver disease; NCEP, National Cholesterol Education Program; NHMRC, National Health and Medical Research Council; NOD, not-other defined; NR, not reported; PAD, peripheral artery disease; PC, personal computer; PI, principle investigator; ESTEEM Q, ESTEEM questionnaire67; SFA, saturated fatty acids; ST, sinus tachycardia; T2DM, type 2 diabetes mellitus; THIS-DIET, The Heart Institute of Spokane Diet Intervention and Evaluation Trial.

The majority of RCTs were performed in Spain,18 19 33 37–44 46–48 50 52–56 59–61 63 four originated from France34–36 62 two took place in the UK51 58 and Italy,31 32 and single trials were performed in Australia,45 Israel57 and India.49 Most publications belonged to the PREDIMED or PREDIMED Reus trials,18 19 33 37–40 43 44 46–48 50 52–56 61 63 and few referred to the Lyon Heart Study.34–36 62 Two records involved the St. Carlos gestational diabetes mellitus (GDM) prevention RCT,59 60 and others were produced from the Effect of Simple, Targeted Diet in Pregnant Women With Metabolic Risk Factors on Pregnancy Outcomes (ESTEEM),58 The Heart Institute of Spokane Diet Intervention and Evaluation Trial,51 Pre Frail 80,41 Indo-MD Heart Study49 or other trials31 32 42 45 57 (table 2). The sample size ranged from 5645 to 740344 participants.

Given the nature of the intervention (diet), most RCTs were of single-blind masking, and the remaining were open labelled. Regarding the PREDIMED trial, the single-blind masking was disputed by some researchers and further verifications were published by the investigators to support the issue.

Intervention and outcomes

For one trial,41 it was difficult to discern the exact primary outcome. For this specific RCT,41 the first result presented in the abstract (reversion to robustness) was considered as the primary outcome. Accordingly, given that the Consolidated Standards of Reporting Trials (CONSORT)64 guidelines were produced fairly recently, few—mainly older—trials did not have a preregistered protocol, although some had preceding publications detailing the protocol.

The PREDIMED RCTs18 19 33 37–40 43 44 46–48 50 52–56 61 63 evaluated the efficacy of two MD interventions, one with extra-virgin olive oil (EVOO) and one with nuts, in a great variety of health outcomes. In further detail, included PREDIMED RCTs involved the prevention the development of diabetic retinopathy and nephropathy,37 38 CVD,18 19 incidence and reversion of the metabolic syndrome,55 61 liver steatosis,52 depression,56 osteoporosis-related fractures,33 53 peripheral artery disease,54 the occurrence of cataract surgery,40 as well as the incidence of type 2 diabetes mellitus (T2DM),46–48 atrial fibrillation,43 breast cancer50 and heart failure.44 Among the remaining trials, the majority34–36 42 49 51 57 62 investigated the effects of the MD on CVD risk factors. The St. Carlos GDM prevention59 60 and the ESTEEM58 trials used a MD with EVOO and pistachios to investigate maternal and fetal outcomes. The Pre Frail 8041 and Properzi45 trials applied the MD to evaluate frailty65 and non-alcoholic fatty liver (NAFLD) parameters, respectively.

Compliance to the dietary interventions was assessed by the majority of trials using the MEDAS questionnaire,66 food frequency questionnaires (FFQs)34–36 57 58 including the ESTEEM-Q,67 previous day 24 hours diet recalls,34–36 49 diet diaries,31 32 51 diet history,45 diet ‘surveys’62 or other screeners.42 58 In addition, biomarkers indicative of increased MD adherence were selectively assessed, including urine hydroxytyrosol concentrations and plasma α-linolenic acid proportions. When adherence to the control diet differed from that of the intervention group, either a 9-item dietary screener was used, or compliance assessment was not reported in the procedures at all.

A summary of the RoB of the included RCTs is presented in figure 2. For some of the PREDIMED RCTs,18 37 44 46 47 the deviations from the randomisation protocol were considered when assessing the domains of random-sequence generation and allocation-sequence concealment. Many of the PREDIMED RCTs18 19 37–39 44 46–48 63 published errata and reanalyses of their datasets, excluding participants who had deviated from the randomisation protocol; for these, the allocation sequence concealment was considered as adequate, without, however, altering the random-sequence generation domain of the RoB tool, which remained biased. Furthermore, the use of different tools to assess compliance between intervention and controls was also accounted for when assessing the RoB, as it confuted the single-blind masking.

Figure 2

Included randomised controlled trials, investigating the effects of the Mediterranean diet interventions, rated against the Cochrane risk of bias 2.0 tool.30 *Publication excluding participants who had deviated from the randomisation protocol. †Concerns regarding randomisation rose post publication. ‡Personnel blinding was reported; however, compliance assessment indicates inadequate blinding of the intervention personnel.

According to the RoB (figure 2), the majority of RCTs exhibited either unclear, or high overall bias.18 19 33 35 36 38–42 46 47 50 52 54–57 59–61 The fewest concerns were raised with regard to missing outcome data. Among all included RCTs, the ESTEEM58 demonstrated the lowest bias throughout the examined RoB domains.

FI and RFI of the included RCTs

Table 3 details the FI and RFI of all included RCTs. The majority of comparisons33 37–40 43–48 50–52 55 57 58 61 failed to provide a significant result between MD intervention and comparator arms, exhibiting an FI equal to 0. On the other hand, the FI of significant comparisons ranged between 152 and 39.60 The median FI of the RCTs, excluding those with non-significant comparisons, was 5. More than half of the comparisons had an FI <5, indicating that the addition of 1–4 events to the opposite treatment arm eliminated the statistical significance of the RCTs. The most robust results (FI >15) involved publications of the St. Carlos GDM trial,60 the Indo-MD Heart Study,49 the Lyon Heart Study62 and the PREDIMED comparison between MD + EVOO versus control diet, published by Babio et al.61

Table 3

Fragility Index of the included randomised controlled trials

First author	Outcome	Intervention	Intervention arm (n)	Events in intervention arm (n)	Control arm (n)	Events inControl arm (n)	P-value(Intervention vs control)	Fragility Index (FI)	Reverse Fragility Index (RFI)
Al Wattar58	Maternal composite outcome*	MD	486	111	500	143	0.04	2	–
	Offspring composite outcome†	MD	531	92	564	118	0.145	0	7
Assaf-Balut59	GDM incidence	MD + EVOO + Pistachio	434	74	440	103	0.012	4	–
Assaf-Balut60	Composite maternal-fetal outcome‡	MD + EVOO + Pistachio	360	32	337	87	0.0001	39	–
Babio61	Incidence of MetS	MD + EVOO	1982	29§	1934	75§	<0.001	26	–
	Incidence of MetS	MD + Nuts	1885	58	1934	75	0.186	0	7
de Lorgeril62	Composite outcome¶	MD	219	14	204	44	0.0001	17	–
de Lorgeril34	CV mortality	MD	302	6	303	19	0.01	3	–
	Non-fatal ΜΙ	MD	302	8	303	25	<0.001	5	–
de Lorgeril35 36	CV mortality	MD	302	3	303	16	0.004	4	–
	Non-fatal MI	MD	302	5	303	17	0.015	2	–
Díaz-López37 38	Diabetic retinopathy	MD + EVOO	7830	22	6856	32	0.075**	0	2
	Diabetic retinopathy	MD + Nuts	6622	20	6856	32	0.126	0	3
Esposito31 32	Need for T2DM medication	LCMD	108	48	107	75	<0.001	14	–
Estruch18 19	Composite CV events††	MD + EVOO	11 852	96	9763	109	0.024	5	–
	Composite CV events††	MD + Nuts	10 365	83	9763	109	0.024	4	–
García-Gavilán33 53	Osteoporotic fractures	MD + EVOO	291	40	290	37	0.807	0	13
	Osteoporotic fractures	MD + Nuts	289	37	290	37	1	0	16
García-Layana40	Cataract surgery incidence	MD + EVOO	11 728	206	10 633	179	0.681	0	28
	Cataract surgery incidence	MD + Nuts	10 719	174	10 633	179	0.748	0	29
Gené Huguet41	Reversion to robustness	MD	85	14	88	1	<0.001	5	–
Greenberg57	Weight loss	MD (vs LFD)	92	41	91	35	0.454	0	8
	Weight loss	MD (vs LCD)	92	41	89	39	1	0	13
Marcos-Forniol42	Optimal CV risk factor control‡‡	MD	54	34	52	15	<0.001	7	–
Martínez-González43	Atrial fibrillation incidence	MD + EVOO	10 634	72	8851	89	0.014	7	–
	Atrial fibrillation incidence	MD + Nuts	9333	92	8851	89	0.94	0	27
Papadaki44 63	Heart failure incidence	MD + EVOO	11 737	29	9664	32	0.303	0	7
	Heart failure incidence	MD + Nuts	10 279	33	9664	32	0.902	0	14
Pintó52	Presence of steatosis	MD + EVOO	34	3	30	10	0.027	1	–
	Presence of steatosis	MD + Nuts	36	12	30	10	1	0	7
Properzi45	NAFLD resolution	MD	26	3	25	9	0.046§§	0	1
Ruiz-Canela54	New symptomatic PAD cases	MD + EVOO	2539	18	2444	45	<0.001	12	–
	New symptomatic PAD cases	MD + Nuts	2452	26	2444	45	0.023	3	–
Salas-Salvadó55	Reversion of MetS	MD + EVOO	252	53	250	41	0.208	0	6
	Reversion of MetS	MD + Nuts	249	63	250	41	0.015	4	–
	New MetS Incidence	MD + EVOO	157	36	154	36	1	0	15
	New MetS Incidence	MD + Nuts	162	29	154	36	0.266	0	6
Salas-Salvadó39 46	Incidence of T2DM	MD + EVOO	570	14	515	24	0.068	0	1
	Incidence of T2DM	MD + Nuts	598	16	515	24	0.105	0	2
Salas-Salvadó47 48	New-onset of T2DM	MD + EVOO	4990	80	4271	101	0.01	9	–
	New-onset of T2DM	MD + Nuts	4876	92	4271	101	0.126	0	6
Sánchez-Villegas56	Depression	MD + EVOO	7715	88	6096	77	0.018	5	–
		MD + Nuts	6803	59	6096	88	0.003	13	–
Singh49	Total cardiac events	Indo-MD	499¶¶	39	501	76	<0.001	16	–
Toledo50	First invasive breast cancer	MD + EVOO	7031	8	5829	17	0.027	2	–
	First invasive breast cancer	MD + Nuts	5492	10	5829	17	0.253	0	4
Tuttle51	Total outcome endpoints***	MD	51	8	50	8	1	0	7

*gestational diabetes mellitus (GDM) or preeclampsia.

†Stillbirth, small-for-gestational age (SGA) fetus, or admission to the neonatal care unit.

‡Emergency caesarean section, perineal trauma, pregnancy-induced hypertension and preeclampsia, prematurity, large-for-gestational age, and SGA.

§The number of events was calculated by subtracting the baseline n from the follow-up n count.

¶Cardiac death and non-fatal myocardial infarction (MI).

**In the manuscript, the comparison is reported as ‘significant’ based on the multivariable-adjusted model; however, analysis using Fischer’s exact test does not reveal a significant difference.

††MI, stroke, or death from cardiovascular causes.

‡‡Defined as the achievement of ≥5 risk factor goals: blood pressure <140/90 mm Hg, low-density lipoprotein <2.6 mmol/L, smoking cessation, body mass index <25 kg/m2, physical activity of moderate intensity >30 min/day, 3 days/wk (≥6 metabolic equivalents h/wk) and HbA1c <7% in patients with diabetes.

§§In the manuscript, the comparison is reported as significant (p=0.046); analysis using Fischer’s exact test does not reveal a significant difference.

¶¶In the manuscript, table 4 reports that the size of the intervention group (n) was 999; instead, the count reported in the main manuscript text and the other tables was used for the calculation of the FI (n=499).

***All-cause and cardiac deaths, MI, hospital admissions for heart failure, unstable angina pectoris or stroke.

CV, cardiovascular; EVOO, extra-virgin olive oil; LCD, low-carbohydrate diet; LCMD, low-carbohydrate MD; LFD, low-fat diet; MD, Mediterranean diet; MetS, metabolic syndrome; NAFLD, non-alcoholic fatty liver disease; PAD, periphery artery disease; T2DM, type 2 diabetes mellitus.

Table 4

Categorisation of the Fragility and Reverse Fragility Index according to randomised controlled trial design, sample size, compliance assessment method and primary outcomes (n, median and IQR)

	N	Fragility Index	P-value	Reverse Fragility Index	P-value
Masking
Single blind	21	4.0 (2.5–10.5)	0.38*	7.0 (3.3–15.3)	0.68*
Open label	24	7.0 (4.0–12.0)		7.0 (6.0–10.5)
Not reported	2	CNC		10.5 (0.3–11.8)
Outcome categorisation
Perinatal outcomes	4	4.0 (3.0–21.5)	0.34*	7.0 (7.0–7.0)	0.98*
Outcomes related to the MetS and DM	15	11.5 (7.8–17.0)		6.0 (2.5–7.5)
Cardiovascular outcomes	16	5.0 (3.8–8.3)		10.5 (7.0–17.3)
Outcomes related to NAFLD	3	1.0 (1.0–1.0)		4.0 (2.5–5.5)
Other†	9	5.0 (4.3–7.0)		16.0 (13.0–28.0)
Composite outcome
Yes‡	15	6.0 (4.0–16.8)	0.29§	7.0 (7.0–7.0)	0.85§
No¶	32	5.0 (3.3–8.5)		7.0 (3.3–13.8)
Sample size
<1000 patients	20	5.0 (4.0–7.0)	0.64§	7.0 (2.0–13.0)	0.57§
≥1000 patients	27	7.0 (3.5–12.5)		7.0 (6.0–14.8)
Dietary compliance assessment methods
Questionnaires (FFQs, diet scores)/diet history	22	5.0 (2.3–10.8)	0.59*	7.0 (6.3–13.0)	0.035*
Diet recalls or records	7	4.5 (3.3–11.8)		2.0 (1.5–2.5)
Biomarkers	18	6.0 (4.0–11.0)		11.0 (6.0–27.3)

*Based on the Kruskal-Wallis test.

†First incidence of breast cancer, cataract surgery, osteoporotic fractures, return to robustness, depression.

‡Maternal/offspring composite outcome, incidence of metabolic syndrome (MetS), cardiovascular (CV) mortality, composite CV events, optimal CV risk factor control, total cardiac events, total outcome endpoints.

§Based on the Mann-Whitney U test.

¶incidence of gestational diabetes mellitus (GDM), atrial fibrillation, heart failure, type 2 diabetes mellitus (T2DM), depression, first invasive breast cancer, steatosis, non-fatal myocardial infarction, cataract surgery, diabetic retinopathy, osteoporotic fractures, need for T2DM medication, reversion to robustness, weight loss, non-alcoholic fatty liver disease (NAFLD) resolution, new symptomatic periphery artery disease cases, reversion of MetS.

CNC, could not be calculated; DM, diabetes mellitus; FFQ, food frequency questionnaire;IQR, interquartile range.

For the comparisons with an FI=0, the RFI was calculated, ranging between 137–39 45 46 and 2940 (median RFI: 7). Six out of 23 comparisons had RFΙ <5 (median: 4), indicating that the change of 1–4 non-events to events reverses the respective comparisons to statistically significant ones.

Categorisation of the FI and RFI according to study characteristics

Table 4 details the FI and RFI categorisation according to the RCT design, the number of participants, and the primary outcome. When masking was accounted for, no differences were noted in the FI or RFI between trials of different allocation masking.

Primary outcomes of the trials were categorised as perinatal, those related to diabetes mellitus or metabolic syndrome, cardiovascular, NAFLD-outcomes, or other (first incidence of breast cancer, cataract surgery, osteoporotic fractures, return to robustness or depression). This allocation failed to induce differences in the FI and RFI between different outcome categories. Similarly, allocation of the trials to those with composite primary endpoints against all others failed to show differences in the FI and RFI between the two groups.

Again, when sample size and methods used to assess dietary compliance between trials were used to allocate the RCTs, no differences were observed in the FI and RFI, with the exception of the RFI among distinct compliance-methods groups (p≤0.035).

Discussion

The present study revealed that most individual comparisons of RCTs with MD interventions and dichotomous primary outcomes as endpoints fail to demonstrate significant results. In parallel, those with comparisons yielding significant findings appear fragile, with a small number of events needed to change the result from significant to non-significant. Subsequently, the number of robust RCTs investigating MD interventions appears to be limited.

Among the reviewed trials, the St. Carlos GDM60 and the PREDIMED RCT conducted by Babio61 exhibited the highest FIs, indicating that nutrition RCTs can be robust. Both of these trials exhibited high RoB in several RoB domains, suggesting that robustness does not necessarily coincide with low RoB. In the St. Carlos GDM study,60 the reported event rate was high, corresponding to 27.8% and 25.8% of the intervention and control groups, respectively, whereas the Babio61 PREDIMED trial did not exhibit a similar high rate of events (1.5% of the total participants in the intervention arm and 3.9% of those allocated in the control group, respectively). This discrepancy between two RCTs with high FI indicates that the event rate is not the only parameter influencing the FI. According to Gaudino et al 29 the FI, P-values, events and sample size are mathematically related; however, the type of primary outcome might also have an effect on a trial’s robustness. For instance, the St. Carlos GDM study60 used two primary outcomes, the first being the incidence of GDM59 and the second being a composite maternal-fetal score60 and published the trial’s results in two distinct publications. Although both publications reported significant findings, the first exhibited an FI equal to 459 and the latter an FI of 39.60 Composite scores are popular in nutrition research; they are combining distinct outcomes, often resulting in a greater event rate as compared with the use of the ‘component’ outcomes independently. In the present analyses, the use of composite scores did not ensure statistical robustness in all of the trials herein, with many exhibiting low FIs and RFIs (<5).18 19 35 36 58

The Esposito et al 31 32 trial also demonstrated a high FI, indicating a robust outcome. However, in this specific RCT the two diets applied by the trialists were not so comparable. In more detail, the intervention arm adopted a low-calorie MD, whereas the comparator group followed a low-fat diet, without any reported restrictions concerning the energy intake. Thus, the observed effects of the intervention arm, and subsequently the high FI, could well have been the result of the prescribed low-calorie diet, as restricted energy intake leading to weight loss has been shown to delay the development of T2DM and subsequently, improve glycaemic control and various coronary factors.68–72

Additionally, it appears that the majority of evidence on MD interventions with dichotomous outcomes is based on the PREDIMED trial, which had a multiarm design. According to Parmar et al,73 trials concurrently evaluating more than one intervention, like the PREDIMED, have increased chances of finding significant differences even with the use of small sample sizes. Since the FI is based on Fischer’s exact test it can only be applied on 2×2 tables, thus in trials with three parallel arms, distinct comparisons of each intervention with the comparator group were performed for the calculation of the FI/RFI. Exlcusion of the trials with three arms however, did not alter the pooled results herein (Median FI: 5, RFI: 7). Accordingly, separation of the PREDIMED comparisons revealed a lack of a significant effect in approximately half of the comparison pairs. When the PREDIMED comparison arms where grouped together and compared against the other trials, the median FIs and RFIs between groups were similar (5 and 7, respectively for both groups), indicating a similar robustness to the rest of MD RCTs. Apart from disputes concerning the randomisation of the PREDIMED sample and the different reported tools used to assess compliance, Correia74 also noted discrepancies in the medical care offered to the participants, resulting in allocation concealment bias. In parallel, the control group received an intervention of lower intensity for the initial 3 years of the RCT, a corrected problem before completion of the recruitment and analysis of the results.75 Inevitably, however, a different intervention frequency unmasks participant allocation. Additionally, compliance with the low-fat control diet appeared to be a difficult task in the long run, with the mean fat intake of participants reaching 37.4% of the total energy intake 5 years post intervention. Thus, the control diet did not correspond to a low-fat regime but was rather lower in the fat content compared with the two MD interventions (42% of the total energy intake).75 Subsequently, more losses to follow-up were recorded in controls, mainly among participants with a worse CVD risk profile at recruitment.75 This induced further bias towards ameliorated results for the control group, leading to mitigated between-groups differences, and by inference, the bias in the FI. Despite the issues mentioned above, the PREDIMED is an ambitious milestone trial for nutrition research and reanalysis of the data did not reveal differences in the reported results despite the randomisation issues. Given the prolonged intervention duration and the large number of participants, collaborators and outcomes, it is not uncanny that certain aspects of the trial’s design and execution demonstrated issues. Undoubtedly, similar issues might have also been observed in pharmacological trials. On a sidenote, the PREDIMED is probably the only megatrial that has undergone this degree of exhaustive scrutiny, despite the results being unchanged at republication. Moreover, unlike pharmacological trials, the trial aimed in providing evidence to a more traditional and accessible therapy (i.e. diet), without supporting any industry products other than common, ‘healthy’ foods, including olive oil and nuts. According to the authors, these issues should have increased trust to the results. However, for the detailed methodologists, the majority of nutrition research has limitations, whereas for the sceptics, nutrition research is scrutinised for competing against the big Pharma on a pretence of evidence.

For many of the included trials, the calculated low FIs and RFIs were associated with an overall smaller number of events. This problem can be surpassed if greater sample sizes are recruited at baseline, or if we shift the focus towards the execution of pragmatic trials. However, Gaudino29 noted that it is more ethical to power RCTs in order to produce the required level of evidence using the minimum possible number of participants. Enrolling additional participants might result in stronger evidence against the null hypothesis, however, it might violate the equipoise principle.29 On the other hand, findings may produce more contradictory results than similar trials, and may also pose further ethical concerns.29

An important question arising from the present findings is whether we are receiving the reliable data we are craving for, by performing RCTs, or if we are overlooking important flaws of either the nutrition science, or the methodology applied in trials examining the MD. However, the present study did not aim in examining the importance or the effectiveness of the MD as a therapeutic dietary regimen. The low robustness calculated herein indicates that even the best level of primary MD evidence proving causality, namely the RCTs, can fail to reach the standards one would expect. Recently, a study76 assessing the FI of clinical nutrition trials revealed a low FI. According to Zeilstra,77 many nutritional RCTs yield ambiguous results, which is why the RCT design is often considered ‘ill-suited’ for nutritional research.77–80 Additionally, given that most trials are based on different analyses of the same landmark protocol (PREDIMED), bias and limitations of the trial are inevitably reproduced in every publication. Subsequently, any synthesis of related RCTs, although it may present low heterogeneity, carries an inherited risk of extrapolated findings. To nutrition’s defence however, lower median FI compared with that of MD interventions has been reported in perioperative,81 anesthesiology,82 plastic surgery,83 and critical-care medicine84 RCTs, as well as among paediatric orthopedic85 and appendicitis86 trials. Nevertheless, the synthesis of these trials for recommendations formulation consists of a common practice in the fields mentioned above, as in the science of nutrition.

On the flip side, RCTs with MD interventions and continuous primary outcomes demonstrate significant findings while supporting the health benefits of adhering to the MD prototype. However, similarly to the Esposito31 32 trial, control interventions are not always comparable, with a tendency to favour the MD arms. This is why, to verify the health effects of MD adherence and advocate for its prescription, superiority trials with continuous primary outcomes should be performed, comparing the MD to other healthy diet regimens instead of the usual diet of participants or dietary advice only.

Although the current results indicate that as far as trials with dichotomous outcomes are concerned, the evidence on the MD entails some limitations, several other factors must also be considered before treating the MD with contempt. For instance, assessment of the participants’ adherence to the dietary intervention, often relies on short dietary indexes instead of more objective measures, and consists of an important component of a nutrition RCT. Moreover, the Hawthorn effect87 (individuals modify an aspect of their behaviour in response to their awareness of being observed) is apparent in all of nutrition research; thus, compliance and assessment are not always accurate. RCTs are often used to guide clinical practice and are sometimes incorporated in clinical practice guidelines intact or after synthesis, using systematic reviews and meta-analyses. Given the demand for evidence-based nutrition recommendations,88–91 the results suggest that the formulation of recommendations promoting the MD based on RCTs should be performed with caution.76 Thorough examination of the American College of Gastroenterology guidelines revealed that most RCTs used to guide recommendations regarding Crohn’s disease relied on a small number of superior events for ‘securing’ statistical significance.92 Often, the FI coincided with the drop-outs reported in some trials. This is why, reporting the FI has also been suggested for systematic reviews and meta-analyses, to understand the fragility of the presented associations and identify possible misuse of the P-value.93 The present study aimed to pinpoint another issue requiring the attention of scientists when performing nutrition trials, namely the FI. Meticulous care in the trial design, sample size and execution can improve the FI of nutrition trials and aid in upgrading the science of nutrition, as succinctly pointed out by other researchers.94

Another important issue in nutrition research is that often, detailed definitions of the interventions are not reported. This is also the case with the MD. Although the label MD is a generic term used to describe the diet of inhabitants around the Mediterranean basin, according to Trichopoulou,95 what constitutes the MD and its key determinants differs even among ‘experts’ worldwide. Martínez-González96 noted that the discrepancies in the MD definition consist of a major problem, especially for intervention studies. As a result, except for the RCTs included herein which were stemming from the same protocol, like the PREDIMED, the remaining trials have most probably used different definitions of the MD. For instance, Singh and associates49 used a National Cholesterol Education Program modification of the MD, whereas Greenberg et al 57 reported following Professor Willet’s definition of the MD. This indicates that differences may exist even under the same intervention label, and these may well induce inconsistencies and bias in the reported outcomes.97

Undoubtedly, one important limitation of the study stems from the relatively small number of RCTs with a dichotomous primary outcome included in the analyses. However, one should consider that the total number of RCTs examining MD interventions is rather small; additionally, in the present study, RCTs were selected based on a systematic search strategy; thus, the results reflect the actual number of available MD-RCTs fulfilling the study’s criteria and being indexed in the PubMed database. An additional limitation is that the publication of many RCTs predated the CONSORT64 98 guidelines; thus, few important characteristics have not been reported. In parallel, in the case of MD RCTs, as in the majority of nutritional epidemiology, diet adherence and intake rely on not so precise exposure assessments—mainly self-reported information—with an increased potential for confounding.16 99–101

Moreover, due to the small number of retrieved trials, it was not possible to correlate the FI with individual study characteristics, or to perform additional statistical analyses. As already mentioned, the use of broad research topics for the assessment of the FI/RFI, as seen herein with the MD, is common in the literature.26 76 Although such studies result in pooling a greater number of RCTs, they also tend to mix many studies with non-comparable aspects, including participant age, health status, study question, outcomes categories, etc. In an effort to correct the heterogeneity observed in the included trials, we also calculated the FI and the RFI after allocating the RCTs based on sample sizes, masking, or outcomes categories. However, these analyses failed to reveal differences, with the only observed significant finding involving the different RFI among RCTs using different methods to assess dietary adherence. Therefore, in the pooled sample of RCTs included herein, differences in sample size, outcomes categories or masking had a minimal effect on the FI and the RFI. Nevertheless, a larger pool of RCTs might have produced different results.

Limitations of the FI include the fact that its calculation is based on the Fischer’s exact test, which is considered as stricter and more prone to type II errors when compared with the χ2 test. Additionally, as already mentioned, it can only be applied to dichotomous outcomes, whereas the majority of nutrition research tends to examine continuous outcomes. Furthermore, the lack of standardised cut-offs for categorising RCTs as either robust, or fragile, is evident.102 103 According to Andrade,102 the most important limitation of the index concerns the use of the much decried statistical threshold (p<0.05) for determining the significance of a study’s outcome. However, one should consider that the FI uses the same threshold applied in the published RCTs and that additionally, the FI is highly correlated to the P-value of a trial, with a significance closer to 0.05 indicating a lower FI.103 104 Moreover, although Walsh27 suggested calculating the index in time-to-event data—as performed in the current analysis—several researchers raised concerns, claiming that it cannot account for the effect of time.102 Nevertheless, as Charilaou105 promptly noted, the FI can offer a measure of the validity of an RCT, especially in trials where the number of participants lost to follow-up, exceeded the FI of the trial. More recently, in a collective effort to optimise patient care, the routine use of the FI has been recommended for the development of all clinical practice guidelines,28 with incorporation of the results in the GRADE (Grading of Recommendations Assessment, Development and Evaluation) format.

Conclusions

In summary, the present study reveals that, when adhering to good scientific principles, one discerns that even in the top tiers of evidence hierarchy, research on the MD may lack robustness, setting concerns for the formulation of nutrition recommendations in a wider context. A collective effort is required to promote the science of nutrition in an evidence-based manner. Despite the mediocre robustness of RCTs with MD interventions, the findings herein do not overlay on the importance of the MD on health or as a UNESCO accredited intangible cultural heritage. Nevertheless, it appears that our quest for an ideal diet for all could prove horses for courses, and a more personalised approach may be required for both health attainment and ameliorated disease outcomes. As Correia74 noted ‘enthusiasm regarding the MD may not be proportional to the level of evidence’ and this might lead to allegiance bias and an imbalance between expectancies and evidence.