Risk of bias and reporting practices in studies comparing VO2max responses to sprint interval vs. continuous training: A systematic review and meta-analysis.

BACKGROUND: It remains unclear whether studies comparing maximal oxygen uptake (VO2max) response to sprint interval training (SIT) vs. moderate-intensity continuous training (MICT) are associated with a high risk of bias and poor reporting quality. The purpose of this study was to evaluate the risk of bias and quality of reporting in studies comparing changes in VO2max between SIT and MICT.
METHODS: We conducted a comprehensive literature search of 4 major databases: AMED, CINAHL, EMBASE, and MEDLINE. Studies were excluded if participants were not healthy adult humans or if training protocols were unsupervised, lasted less than 2 weeks, or utilized mixed exercise modalities. We used the Cochrane Collaboration tool and the CONSORT checklist for non-pharmacological trials to evaluate the risk of bias and reporting quality, respectively.
RESULTS: Twenty-eight studies with 30 comparisons (3 studies included 2 SIT groups) were included in our meta-analysis (n = 360 SIT participants: body mass index (BMI) = 25.9 ± 3.7 kg/m2, baseline VO2max = 37.9 ± 8.0 mL/kg/min; n = 359 MICT participants: BMI = 25.5 ± 3.8 kg/m2, baseline VO2max = 38.3 ± 8.0 mL/kg/min; all mean ± SD). All studies had an unclear risk of bias and poor reporting quality.
CONCLUSION: Although we observed a lack of superiority between SIT and MICT for improving VO2max (weighted Hedge's g = -0.004, 95% confidence interval (95%CI): -0.08 to 0.07), the overall unclear risk of bias calls the validity of this conclusion into question. Future studies using robust study designs are needed to interrogate the possibility that SIT and MICT result in similar changes in VO2max.

Introduction

There is a growing awareness of a “reproducibility crisis” in preclinical and clinical research, that has widespread societal and financial ramifications., Many research groups,, attribute poor reproducibility to shortcomings in key aspects of study design (e.g., randomization, blinding, and outcome reporting), as inadequacies in these methodological areas compromise internal validity and produce biased results.6, 7, 8, 9, 10, 11, 12 Importantly, quality of reporting is intimately linked with bias: clinical trials that do not report information on bias-mitigating methodologies (e.g., allocation concealment) produce inflated effect sizes compared with trials with adequate reporting. Therefore, interpreting the internal validity of original research requires the assessment of both methodological rigor and quality of reporting.,

Systematic reviews provide an opportunity to evaluate the overall methodological rigor and quality of reporting in studies investigating a given research question. The Cochrane Collaboration bias assessment tool and the Consolidated Standards of Reporting Trials (CONSORT) checklist are robust tools for assessing the risk of bias and reporting quality, respectively. Recent reports found that many systematic reviews in sports and exercise medicine research either do not evaluate the risk of bias or use inferior assessment tools. Furthermore, although several reviews have highlighted poor reporting quality in sports medicine research,19, 20, 21 we are unaware of a study that has systematically evaluated the quality of reporting in exercise medicine research.

A current hot topic in exercise medicine research is determining which mode of exercise training best improves maximal oxygen uptake (VO2max),—a research question with important clinical implications considering the association between VO2max and all-cause morbidity and mortality. Given that a perceived lack of time is a commonly cited barrier to participating in regular, structured physical activity, a large body of work has developed investigating the potency of sprint interval training (SIT) — time-efficient exercise involving repeated supramaximal bouts of exercise interspersed with brief periods of rest — to improve VO2max. A recent meta-analysis demonstrated that SIT elicits similar improvements in VO2max compared with traditional endurance training (herein referred to as moderate-intensity continuous training (MICT)). However, neither this meta-analysis nor any recent meta-analysis examining the effects of SIT on VO2max26, 27, 28 has evaluated the risk of bias or quality of reporting within studies, which limits our confidence in the conclusion that SIT and MICT lead to similar improvements in VO2max. Consistent with the sports medicine literature,19, 20, 21 we speculate that there is a high or unclear risk of bias and poor reporting quality among studies comparing changes in VO2max following SIT and MICT.

The purpose of this systematic review was to test the hypothesis that studies comparing changes in VO2max between SIT and MICT have a high or unclear risk of bias and poor quality of reporting. A secondary purpose was to determine whether bias impacts the overall treatment effect for VO2max responses to SIT vs. MICT. Specifically, we planned to perform 2 meta-analyses: one with all studies that met our inclusion criteria, and a second only including studies judged to have a low risk of bias. The expectation was that this 2 meta-analysis approach would provide greater insight about our confidence in the conclusions derived from current and past meta-analyses. For instance, differences in overall treatment effects between these 2 meta-analyses would indicate that a high or unclear risk of bias impacted the comparison of VO2max responses following SIT and MICT. Systematic evaluations of methodological rigor and reporting quality are likely required for many topics in sports and exercise science, as these issues appear to be widespread.19, 20, 21, 22 We chose to evaluate studies comparing VO2max responses to SIT and MICT, as this topic is clinically relevant, addresses potential barriers to completing regular physical activity, and has a large number of studies that can be included in our analysis—as demonstrated by past systematic reviews.26, 27, 28 Although our systematic review focuses on this specific topic, our discussion provides simple and feasible recommendations applicable to all areas of exercise medicine research.

Methods

The present systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist; and a completed checklist can be found in the Supplementary material (Sheet 4). The study selection process was conducted using Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia).

Eligibility criteria

Studies were included in the present systematic review if they met all of the following inclusion criteria: (1) used human adult participants between the ages of 18 and 65, (2) directly measured VO2max (or peak) using indirect calorimetry (i.e., a metabolic cart), (3) reported VO2max in relative units (mL/kg/min) or in absolute units (mL/min or L/min) with body mass (kg) so that relative VO2max could be manually calculated, (4) reported mean and standard deviation (SD) for changes in VO2max (post-training minus baseline) or VO2max at baseline and post-training, or presented data in a manner that could be extracted using WebPlotDigitizer (WebPlotDigitizer, Pacifica, CA, USA), (5) employed a SIT protocol that was “all-out” or supramaximal (e.g., >100% VO2max or maximal work rate) and interspersed with periods of rest or active recovery, (6) employed a MICT protocol that was continuous and submaximal (e.g., <80% VO2max or maximal work rate), and (7) conducted supervised training for a minimum of 2 weeks. Studies were excluded if they: (1) did not meet all of the inclusion criteria, (2) included non-healthy participants with a specific disease (e.g., cancer, hypertension, type 2 diabetes, etc.); note that we did not consider obesity a disease, and we included studies with obese or overweight participants that were otherwise healthy, (3) included endurance-trained athletes; however, studies in strength-trained athletes were not excluded, (4) employed mixed training protocols (e.g., MICT plus resistance training), or (5) were not in English, not an original research article, or presented previously published VO2max data.

Literature search and study selection

We conducted a comprehensive literature search in AMED, CINAHL, EMBASE, and MEDLINE on August 14th, 2019, and a second up-to-date search took place on August 18th, 2020. The searches included 3 main terms: SIT, MICT, and VO2max. A list of synonyms/related terms for each main term were combined with “OR” (see Supplementary material Sheet 2 for a full list of search terms), and a final single search combined the 3 separate lists with “AND”. Titles and abstracts were extracted from the database searches, and duplicates were automatically removed in Covidence (Covidence, Melbourne, VIA, Australia).

Study selection followed a 2-step process and was independently completed by 2 reviewers (JTB and NP). Both reviewers met in person to justify their decisions during the study selection process and to resolve any initial disagreements. Although a third reviewer (BJG) was available to settle any lasting disagreements, all initial disagreements were resolved during the in-person meetings. First, titles and abstracts were screened to identify studies that appeared to meet eligibility criteria. The 2 reviewers (JTB and NP) also screened relevant previously published systematic reviews,26, 27, 28, in an attempt to identify eligible articles that were not retrieved from the initial literature search. Second, full texts were downloaded for articles that passed the title and abstract screening to determine their eligibility. Third, the 2 reviewers assigned a reason for each study excluded during the full text screening. The final analysis included studies that passed both levels of study selection.

Assessment of risk of bias and quality of reporting

We assessed the risk of bias using the 7 sources of bias and related information outlined in the Cochrane Collaboration Tool. The risk of each source of bias was judged as “high”, “low”, or “unclear”. In brief, studies that reported an adequate methodology for protecting against a given source of bias (e.g., blinding outcome assessors to protect against detection bias) were judged as having a “low” risk of bias. Conversely, studies that reported an inadequate methodology (e.g., randomized participants based on birth month) were judged as having a “high” risk of bias. Studies that did not report information regarding a given methodology were judged as having an “unclear” risk of bias except in cases of reporting bias where studies were judged as having a “high” risk of bias if they did not report publicly registering their trial or if they did not report their methods in a public database/registry.

We assessed quality of reporting by completing the CONSORT checklist for non-pharmacological trials. Each CONSORT item was rated as “yes” (reported) or “no” (not reported), and the elaboration and explanation document was used to help determine the rating for each item. Two reviewers (JTB and NP) independently completed the risk of bias and quality of reporting assessments. Both reviewers met in person to justify their decisions during the assessment of risk of bias and reporting quality process to resolve any initial disagreements. Although a third reviewer (BJG) was available to settle any lasting disagreements, all initial disagreements were successfully resolved by JTB and NP.

Data extraction

Means and SDs for relative VO2max (mL/kg/min) were extracted either by recording values directly from tables/text or by using WebPlotDigitizer (Version 4.4; WebPlotDigitizer, Pacifica, CA, USA)—a data extraction approach with high inter-rater reliability and validity—when VO2max data only appeared in figures. Mean changes in VO2max were either directly extracted from articles or calculated by subtracting the mean baseline value from the mean post-training value. We extracted relative VO2max because many studies did not report VO2max in absolute units35, 36, 37, 38, 39, 40, 41, 42, 43, 44 and because increasing relative VO2max by ∼3.50 mL/kg/min confers an ∼8%–14% reduction in all-cause morbidity and mortality. We also extracted summaries of training protocols and additional participant characteristic data, including self-reported physical activity classification (as reported in papers), age, height, body mass, and body mass index calculated using height and body mass data. For physical activity classification, we recorded the terminology (e.g., “recreationally active”, “inactive”, etc.) that was reported in each study and any details about eligibility cut-offs for physical activity levels (when applicable). Two reviewers (JTB and NP) independently extracted data using a standardized sheet and compared results to verify that correct data were extracted.

Data synthesis

We calculated an effect size (Cohen's d) for each study to compare changes in VO2max between SIT and MICT using Eqs. (1) and (2):, where delta (Δ) refers to mean changes in VO2max following SIT or MICT. SDpooled values were calculated using the SD of change scores (SDΔ) where possible (only 3 included studies reported SDΔ values48, 49, 50). For the remaining studies we calculated SDΔ according to Chapter 16.1.3.2 of the Cochrane Handbook: a correlation of repeated measures of r = 0.89 (calculated using VO2max data from 10 of our previously-published training studies;52, 53, 54, 55, 56, 57, 58, 59, 60, 61 n = 274 participants) and the reported SD for baseline and post-training values were used. Two studies, included 2 SIT groups, and separate effect sizes were calculated for each group. Because effect sizes were calculated by subtracting the mean change in VO2max following MICT from the mean change in VO2max following SIT (Eq. (1)), positive effect sizes indicated a larger increase in VO2max after SIT whereas negative effect sizes indicated a larger increase in VO2max after MICT. As Cohen's d is biased upward for sample sizes less than 20,, the effect sizes for each study were corrected by converting Cohen's d values to Hedges’ g using Eq. (3):

The precision of Hedges’ g effect size estimates were determined by calculating the standard error (SEg) for each Hedges’ g value using Eq. (4) such that 95% confidence intervals (95%CIs) could be constructed around each Hedges’ g estimate (95%CI = g ± (1.96 × SE)):,

To determine whether baseline fitness impacts the comparison of VO2max responses to SIT vs. MICT, we dichotomously grouped studies using an arbitrary threshold of 35 mL/kg/min for baseline VO2max values (calculated as average between SIT and MICT groups). Effect sizes were pooled across all studies within these 2 groups and collapsed across all groups to determine an overall effect by calculating a weighted average Hedge's g and its corresponding SE and 95%CI using Eqs. (5–7), where IVW refers to the inverse variance weight and SE* refers to the standard error of the weighted average effect size:

We also performed a linear regression to determine whether baseline VO2max predicted the Hedges’ g values. To further investigate the impact of sex, we completed 2 additional meta-analyses using male or female participants only. Hedges’ g effect sizes were classified as small (0.2), medium (0.5), and large (0.8), as per Cohen's conventions. A publicly available spreadsheet was used to calculate an I2 statistic in order to quantify the degree of inconsistency (i.e., heterogeneity) of the overall meta-analysis. The degree of inconsistency was considered “low”, “moderate”, or “high” if the I2 statistic was 25%, 50%, or 75%, respectively. Egger's tests are commonly used to detect possible publication bias: the suppression of null or adverse findings in meta-analyses of controlled trials (e.g., equivalency or superiority of placebo). We did not investigate the presence of publication bias because we compared 2 experimental conditions (MICT vs. SIT) rather than comparing the efficacy of experimental conditions against a control. Additionally, we believe that heterogeneity in MICT/SIT intensities, frequencies, and durations, confounds the ability to interpret Egger's test results as evidence of publication bias.

Sensitivity analysis: Does bias impact our meta-analysis?

As recommended by Büttner and colleagues, we planned to perform a sensitivity analysis to determine whether bias impacted our meta-analysis. In brief, a second meta-analysis including only studies identified as having a low risk of bias would be compared to the primary meta-analysis. A difference in the overall estimated effects between these 2 meta-analyses could suggest that biased results impacted the primary meta-analysis. However, as described below, this sensitivity analysis could not be performed because every study included in our meta-analysis was judged to have an unclear risk of bias. We therefore provide an informative discussion on each source of bias included in the Cochrane Collaboration tool and outline recommendations for future work instead.

Results

Study selection

Fig. 1 presents a flow diagram of the study selection process. The literature search retrieved 3859 articles, and Covidence removed 1198 duplicates. Of the 2661 articles that entered title and abstract screening, 2505 of these articles were deemed irrelevant and so were subsequently excluded. Full texts were then downloaded for 156 articles; 132 articles were excluded as they did not meet eligibility criteria. Included were 24 articles from the literature search and 3 additional articles,, identified from previously published systematic reviews. Therefore, 27 articles were part of the final analysis. Participant characteristics and physical activity classifications are presented in Table 1. Several studies did not report information about physical activity eligibility cut-offs, and no study objectively measured physical activity levels (Table 1).

Fig. 1

Flow diagram of the study selection process and number of comparisons included in the meta-analysis. MICT = moderate-intensity continuous training; SIT = sprint interval training; VO2max = maximal oxygen uptake.

Table 1

Characteristics of studies comparing changes in VO2max following SIT and MICT included in the meta-analysis.

Reference	Sample characteristics; physical activity classificationa	Age (year)	n		BMI (kg/m2)		Baseline VO2max (mL/kg/min)		SIT protocol	MICT protocol	Mode	Training frequency (session/week)		Length of intervention (week)
			SIT (female)	MICT (female)	SIT	MICT	SIT	MICT				SIT	MICT
Bailey et al. (2009)97	Healthy; recreationally activeb	21	8 (3)	8 (3)	25.0	22.9	42.0	43.0	4‒7 × 30 s “all-out” bouts against 7.5% BM4 min recovery periods	15‒25 min at 90% VTc	C	3	3	2
Bonafiglia et al. (2016)52	Healthy; recreationally active (<3 h PA/week)	20	21 (12)	21 (12)	23.8	23.7	41.7	42.2	8 × 20 s bouts at 170% WRpeak20 s recovery periods	30 min at 65% WRpeak	C	4	4	3
Burgomaster et al. (2008)116	Healthy; active (≤2 sessions PA/week)	24	10 (5)	10 (5)	23.6	24.5	41.0	41.0	4‒6 × 30 s “all-out” bouts against 7.5% BM4.5 min recovery periods	40‒60 min at 65% VO2peak	C	3	5	6
Cocks et al. (2013)35	Healthy; sedentary (<1 h structured PA/week)	22	8 (0)	8 (0)	24.8	22.7	41.9	41.7	4‒6 × 30 s “all-out” bouts against 7.5% BM2 min recovery periods	40‒60 min at 65% VO2peak	C	3	5	6
Cocks et al. (2016)36	Healthy and obese; sedentary (<1 h structured PA/week)	25	8 (0)	8 (0)	35.9	33.4	33.9	35.1	4‒7 × 30 s at ∼200% WRpeak4.5 min recovery periods	40‒60 min at 65% VO2peak	C	3	5	6
Foster et al. (2015)37	Healthy; relatively sedentary (<2 sessions PA/week)	NR	21 (NR)	19 (NR)	NR	NR	34.0	33.6	8 × 20 s at 170% VO2max10 s recovery periods	20 min at 90% VT	C	3	3	8
Gillen et al. (2016)79	Healthy; sedentary (<600 MET/week)	27	9 (0)	10 (0)	27.1	26.8	31.9d	33.9d	3 × 20 s “all-out” bouts against 5% BM2 min recovery periods	45 min at 70% HRmax	C	3	3	12
Higgins et al. (2016)80	Healthy and overweight; active (≥2 × 30 min sessions PA/week)	20	23 (23)	29 (29)	NR	NR	29.1	26.9	5‒7 × 30 s “all-out” bouts4 min recovery periods	20‒30 min at 60%‒70% HRRc	C	3	3	6
Keating et al. (2014)38	Healthy and overweight; inactive (<3 day PA/week)	43	13 (11)	13 (10)	28.2	28.5	25.3	24.0	4‒6 × 30–60 s at 120% VO2 peak2‒3 min recovery periods	30‒45 min at 50%‒65% VO2peak	C	3	3	12
Kiviniemi et al. (2014)76	Healthy; sedentary (no PA level assessed; VO2peak < 40 mL/kg/min)	48	13 (0)	13 (0)	25.5	26.1	34.7	33.9	4‒6 × 30 s “all-out” bouts against 7.5% BM4 min recovery periods	40‒60 min at 60% WRpeak	C	3	3	2
Kong et al. (2016)39	Healthy and obese; sedentary (<60 min PA/week)	21	13 (13)	13 (13)	25.8	25.5	32.0	32.0	60 × 8 s “all-out” bouts12 s recovery periods	40 min at 60%‒80% VO2peak	C	4	4	5
Macpherson et al. (2011)40	Healthy; recreationally activeb	24	10 (4)	10 (4)	25.2	24.2	46.8	44.0	4‒6 × 30 s “all-out” bouts4 min recovery periods	30‒60 min at 65% VO2max	R	3	3	6
Martins et al. (2016)49	Healthy and obese; sedentary (<1 brisk PA/week or <3 × 20 min light PA/week)	33	13 (NR)	13 (NR)	33.2	33.3	31.1	31.1	SIT: 250 kcals of 8 s “all-out” bouts12 s recovery periods	32 min at 70% HRmaxc	C	3	3	12
		34	13 (NR)		32.4		29.6		1/2 SIT: 125 kcals of 8 s “all-out” bouts12 s recovery periods		C	3
Matsuo et al. (2014)50	Healthy; sedentary (no regular PA)	26	14 (0)	14 (0)	21.3	21.2	43.9	42.0	7 × 30 s at 120% VO2max15 s recovery periods	40 min at 60%‒65% VO2max	C	5	5	8
Mazurek et al. (2014)98	Healthy; untrainedb	NR	24 (24)	22 (22)	21.5	23.0	36.2	36.6	12 × 10 s “all-out” bouts1 min recovery periods	32 min at 65%‒75% HRmax	C	3	3	8
McGarr et al. (2014)41	Healthy; recreationally activeb	24	8 (2)	8 (3)	NR	NR	47.2	47.9	4‒5 × 30 s “all-out” bouts against 7.5% BM4.5 min recovery periods	60‒90 min at 65% VO2max	C	4	4	2
McKay et al. (2009)77	Healthy; recreationally activeb	25	6 (0)	6 (0)	NR	NR	46.0	43.0	8‒12 × 1 min bouts at 120% VO2max1 min recovery periods	90‒120 min at 65% VO2max	C	2.7	2.7	3
Nalcakan (2014)78	Healthy; recreationally activeb	22	8 (0)	7 (0)	25.5	24.5	40.2	40.5	4‒6 × 30 s “all-out” bouts against 7.5% BM4 min recovery periods	30‒50 min at 60% VO2max	C	3	3	7
Schaun et al. (2018)42	Healthy; no information provided	24	15 (0)	14 (0)	23.7	24.7	47.3d	47.3d	8 × 20 s at 130% vVO2max10 s recovery periods	30 min at 90%‒95% HR at VT2	R	3	3	16
Scribbans et al. (2014)59	Healthy; recreationally active (<3 h PA/week)	21	10 (0)	9 (0)	23.2	22.8	48.3	47.6	8 × 20 s bouts at 170% VO2peak10 s recovery periods	30 min at 65% VO2peak	C	4	4	6
Skleryk et al. (2013)71	Healthy and overweight or obese; sedentaryb	39	8 (0)	8 (0)	32.2	35.2	29.7	26.3	8‒12 × 10 s “all-out” bouts against 5% BM80 s recovery periods	30 min at 65% VO2peak	C	3	5	2
Sun et al. (2019)48	Healthy and overweight; inactive	21	14 (14)	14 (14)	26.0	26.5	30.8	31.1	80 × 6 s “all-out” bouts9 s recovery periods	52‒63 min at 60% VO2max	C	3	3	12
Tabata et al. (1996)43	Healthy; physically activeb	23	7 (0)	7 (0)	23.2	24.0	48.2	52.9	8 × 20 s bouts at 170% VO2max20 s recovery periods	60 min at 60%‒70% VO2max	C	5	5	6
Tanisho and Hirakawa (2009)44	Healthy; college lacrosse playersb	19	6 (0)	6 (0)	21.2	21.1	50.8	52.2	10 × 10 s at AnPmax20 s recovery periods	15 min at 70%‒75% HRmax then 1 min step test to volitional fatigue	C	3	3	15
Trapp et al. (2008)72	Healthy; inactive b	22	11 (11)	8 (8)	24.4	22.4	28.8	30.9	15‒60 × 8 s “all-out” bouts against ≥0.5 kg12 s recovery periods	10‒40 min at 60% VO2peakc	C	3	3	15
Zelt et al. (2014)62	Healthy; recreationally active (<3 h PA/week)	23	11 (0)	13 (0)	24.7	25.2	47.9d	44.9d	SIT: 4‒6 × 30 s “all-out” bouts against 7.5% BM4.5 min recovery periods	60‒75 min at 65% VO2peak	C	3	3	4
			12 (0)		25.9		43.5d		1/2 SIT: 4‒6 × 15 s “all-out” bouts against 7.5% BM4.75 min recovery periods		C	3
Zhang et al. (2021)75	Healthy and obese; activeb	21	11 (11)	11 (11)	25.6	25.1	26.7	28.9	All-out: 40 × 6 s “all-out” bouts9 s recovery periods	60% VO2peak until 200 KJc	C	3.7	3.7	12
			12 (12)		26.1		26.4		SIT120: 1 min at 120% VO2peak until 200 KJ90 s recovery periods		C	3.7
Mean ± SD	—	25 ± 7	12 ± 5	12 ± 5	25.9 ± 3.7	25.5 ± 3.8	37.9 ± 8.0	38.3 ± 8.0	—	—	—	3.3 ± 1	3.6 ± 1	8 ± 4

Abbreviations: AnPmax = maximal anaerobic power; BM = body mass; BMI = body mass index; C = cycling; HR = heart rate; HRmax = maximal heart rate; HRR = heart rate reserve; KJ = kilojoules; MET = metabolic equivalents; MICT = moderate-intensity continuous training; NR = not reported; PA = physical activity; R = running; SIT = sprint interval training; VO2max = maximal oxygen uptake; VO2peak = peak oxygen uptake; vVO2max = running velocity at maximal oxygen uptake; VT = ventilatory threshold; VT2 = second ventilatory threshold; WRpeak = peak work rate.

Physical activity levels were determined via self-reporting.

No information reported about eligibility cut-offs for physical activity levels.

Work-matched to SIT protocol.

Data obtained via WebPlotdigitizer.

Risk of bias assessment

Table 2 presents the risk of bias in the 27 included studies. In general, we observed an unclear risk of bias among studies comparing changes in VO2max between SIT and MICT. All 27 studies did not report methods related to adequate allocation concealment, participant blinding, or a priori identification of a primary outcome(s), and therefore had an unclear risk of selection and performance bias and a high risk of outcome reporting bias. (Note that the inability to blind participants is an inherent limitation associated with exercise training studies.) Two studies had a high risk of “other bias” as 1 study imputed group means for missing data — an approach that risks reducing variability and artificially increasing the probability of detecting significance— and the other did not randomize group allocation. Overall, every study included in our meta-analysis was judged to have an unclear risk of bias, and we therefore could not complete a sensitivity analysis to determine whether bias impacted our overall treatment effect.

Table 2

Risk of bias in studies comparing changes in VO2max between SIT and MICT.

	Selection bias		Performance bias	Detection bias	Attrition bias	Reporting bias	Other bias
Reference	Random sequence generation	Allocation concealment	Blinding participants and personnel	Blinding outcome assessment	Incomplete outcome data	Selective reporting
Bailey et al. (2009)97	Unclear	Unclear	Unclear	Unclear	Unclear	High	Low
Bonafiglia et al. (2016)52	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Burgomaster et al. (2008)116	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Cocks et al. (2013)35	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Cocks et al. (2016)36	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Foster et al. (2015)37	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Gillen et al. (2016)79	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Higgins et al. (2016)80	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Keating et al. (2014)38	Low	Unclear	Unclear	Uncleara	Low	High	Highb
Kiviniemi et al. (2014)76	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Kong et al. (2016)39	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Macpherson et al. (2011)40	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Martins et al. (2016)49	Unclear	Unclear	Unclear	Unclear	Unclearc	High	Low
Matsuo et al. (2014)50	Unclear	Unclear	Unclear	Low	Low	High	Low
Mazurek et al. (2014)98	Unclear	Unclear	Unclear	Unclear	Unclear	High	Low
McGarr et al. (2014)41	Unclear	Unclear	Unclear	Unclear	Low	High	Low
McKay et al. (2009)77	Unclear	Unclear	Unclear	Unclear	Unclear	High	Low
Nalcakan (2014)78	Unclear	Unclear	Unclear	Unclear	Unclearc	High	Low
Schaun et al. (2018)42	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Scribbans et al. (2014)59	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Skleryk et al. (2013)71	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Sun et al. (2019)48	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Tabata et al. (1996)43	Unclear	Unclear	Unclear	Unclear	Unclear	High	Highd
Tanisho and Hirakawa (2009)44	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Trapp et al. (2008)72	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Zelt et al. (2014)62	Unclear	Unclear	Unclear	Unclear	Low	High	Low
Zhang et al. (2021)75	Unclear	Unclear	Unclear	Unclear	Unclear	High	Low
Total (unclear or high (%))	26 (96%)	27 (100%)	27 (100%)	25 (93%)	7 (26%)	27 (100%)	2 (7%)

a Unclear which outcome(s) were assessed in a blinded fashion.

b Group means were input for missing individual data.

c Dropout rate >20%.

d No randomization.

Abbreviations: MICT = moderate-intensity continuous training; SIT = sprint interval training; VO2max = maximal oxygen uptake.

Quality of reporting assessment

Table 3 contains the evaluation for select CONSORT items related to key aspects of study design (e.g., randomization procedures, blinding, outcome reporting, and sample size calculations), and the Supplementary material (Sheet 3) presents the evaluation, including all CONSORT items. The high number of “✗” symbols (indicating no reporting or inadequate reporting) in Table 3 highlights an overall poor quality of reporting: 209 “✗” symbols vs. 51 “✓” symbols. Additionally, no study adequately reported CONSORT Items 9 and 10, which relate to allocation concealment and randomization implementation procedures, respectively.

Table 3

Checklist of select CONSORT items to assess quality of reporting in studies comparing changes in VO2max between SIT and MICT.

	CONSORT Item # (see footnote for item topics)
Reference	4A	5B	6A	7A	8A	9	10	11A	13A	17A
Bailey et al. (2009)97	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗
Bonafiglia et al. (2016)52	✗	✓	✗	✗	✗	✗	✗	✗	✓	✗
Burgomaster et al. (2008)116	✓	✗	✗	✗	✗	✗	✗	✗	✗	✗
Cocks et al. (2013)35	✗	✓	✗	✓	✗	✗	✗	✗	✓	✗
Cocks et al. (2016)36	✗	✗	✗	✓	✗	✗	✗	✗	✓	✗
Foster et al. (2015)37	✓	✗	✗	✗	✗	✗	✗	✗	✓	✗
Gillen et al. (2016)79	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗
Higgins et al. (2016)80	✓	✗	✗	✓	✗	✗	✗	✗	✓	✗
Keating et al. (2014)38	✓	✓	✓	✗	✓	✗	✗	✓	✓	✓
Kiviniemi et al. (2014)76	✓	✗	✗	✓	✗	✗	✗	✗	✓	✗
Kong et al. (2016)39	✓	✗	✗	✓	✗	✗	✗	✗	✓	✗
Macpherson et al. (2011)40	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗
Martins et al. (2016)49	✓	✗	✗	✗	✗	✗	✗	✗	✓	✗
Matsuo et al. (2014)50	✓	✗	✓	✓	✗	✗	✗	✓	✓	✗
Mazurek et al. (2014)98	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗
McGarr et al. (2014)41	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗
McKay et al. (2009)77	✗	✓	✓	✗	✗	✗	✗	✗	✗	✗
Nalcakan (2014)78	✗	✗	✗	✗	✓	✗	✗	✗	✗	✗
Schaun et al. (2018)42	✓	✗	✗	✗	✗	✗	✗	✓	✓	✗
Scribbans et al. (2014)59	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗
Skleryk et al. (2013)71	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗
Sun et al. (2019)48	✓	✗	✗	✗	✗	✗	✗	✗	✓	✗
Tabata et al. (1996)43	✗	✗	✓	✗	✗	✗	✗	✗	✗	✗
Tanisho and Hirakawa (2009)44	✗	✓	✗	✗	✗	✗	✗	✗	✓	✗
Trapp et al. (2008)72	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗
Zelt et al. (2014)62	✗	✗	✓	✗	✗	✗	✗	✗	✓	✗
Zhang et al. (2021)75	✓	✗	✗	✓	✗	✗	✗	✗	✓	✗
Total ✗ (%)	16 (59%)	22 (81%)	22 (81%)	20 (74%)	25 (93%)	27 (100%)	27 (100%)	24 (89%)	7 (26%)	26 (96%)

Notes: CONSORT item topics:33 4A = eligibility criteria for participants; 5B = details of whether and how interventions were standardized; 6A = completely defined pre-specified primary outcome; 7A = how sample size was determined; 8A = method used for random sequence generation; 9 = allocation concealment mechanism; 10 = implementation of randomization procedures; 11A = blinding; 13A = participant flow; 17A = results for primary outcome including precision.

✗ Judged to be not or inadequately reported; ✓ Judged to be adequately reported.

Abbreviations: MICT = moderate-intensity continuous training; SIT = sprint interval training; VO2max = maximal oxygen uptake.

Three,, of the 27 studies contained 2 SIT interventions that were both compared to MICT. Therefore, 30 comparisons were included in the meta-analysis with a total sample size of 360 SIT and 359 MICT participants. Fig. 2 presents changes in VO2max, effect sizes (Hedge's g), sample sizes, and percent contribution toward weighted effect size (%weight) for each comparison. Fig. 2 is sorted by mean baseline VO2max: less (Fig. 2A) or greater (Fig. 2B) than 35 mL/kg/min (see Table 1 for mean baseline values). In both baseline fitness groups, the vast majority of 95%CIs crossed zero (11/14 in Fig. 2A; 10/16 in Fig. 2B). Baseline VO2max did not appear to influence Hedges’ g values, as the 95%CI for both weighted effect sizes crossed zero (Fig. 2), and the linear regression was not significant (r2 = 0.07, r = –0.28, p = 0.12). Fig. 2 also presents the overall weighted effect size when we pooled across all studies regardless of baseline VO2max (blue diamond). The mean overall changes in VO2max were +4.6 mL/kg/min following SIT and +4.4 mL/kg/min following MICT, and the weighted average Hedge's g was 0.06 (SE = 0.08) with a 95%CI crossing zero (–0.08 to 0.07). Collectively, these results highlight a lack of superiority between SIT and MICT for improving VO2max regardless of baseline VO2max. Additionally, the I2 statistic (72%) indicated substantial inconsistency of effect sizes across comparisons.

Fig. 2

Forest plot of the meta-analysis comparing changes in relative maximal oxygen uptake (VO2max) following sprint interval training (SIT) and moderate-intensity continuous training (MICT) separated by baseline VO2max: (A) <35 mL/kg/min; (B) >35 mL/kg/min; (C) the overall effect with all studies included. Because effect sizes were calculated as (SIT minus MICT) divided by pooled standard deviation, negative values reflect larger changes in VO2max following MICT whereas positive values reflect larger changes following SIT. The red diamonds represent the overall weighted effect size (Hedge's g) for each baseline fitness group, and the blue diamond represents the overall weighted effect size including all studies. The horizontal points of the diamonds represent the upper and lower bounds of the 95% confidence intervals (95%CIs). The meta-analysis revealed a low–moderate degree of inconsistency (I2 = 38%). Overall changes in VO2max presented as averages, and overall number of participants presented as total sums. Comparison of 1/2 SIT (a) or “all-out” SIT (b) vs. MICT (see Table 1 for details).

Fig. 3 presents forest plots separated by sex, which includes a smaller subset of studies as few included only males,,42, 43, 44,,,,,76, 77, 78, 79 or only females,,,, and only one study reported sex-specific VO2max data. The 95%CIs for the overall Hedge's g values did not cross zero for either sex (Fig. 3). These meta-analyses reveal a possible sex-specific response: females appear to respond more favorably to SIT while males respond more favorably to MICT. However, this interpretation should be made with caution as these meta-analyses were completed with a smaller subset of studies and because both weighted effect sizes were small. Overall Hedge's g values were 0.55 (favoring SIT) for females and –0.32 (favoring MICT) for males.

Fig. 3

Forest plot on subset of studies that reported sex-specific data or included only female or male participants. Forest plot depicts meta-analysis comparing changes in relative maximal oxygen uptake (VO2max) following sprint interval training (SIT) and moderate-intensity continuous training (MICT) separated by sex: (A) females and (B) males. Because effect sizes were calculated as (SIT minus MICT) divided by pooled standard deviation, negative values reflect larger changes in VO2max following MICT whereas positive values reflect larger changes following SIT. The red diamonds represent the overall weighted effect size (Hedge's g) for each baseline fitness group, and the horizontal points of the diamonds represent the upper and lower bounds of the 95% confidence intervals (95%CIs). Overall changes in VO2max presented as averages, and overall number of participants presented as total sums. Comparison of 1/2 SIT (a) or “all-out” SIT (b) vs. MICT (see Table 1 for details).

Discussion

The novel finding of our systematic review was that studies comparing changes in relative VO2max between SIT and MICT—including several of our own,,— had an overall unclear risk of bias and poor quality of reporting. Our meta-analysis revealed that SIT and MICT similarly improve VO2max (Hedge's g = 0.05, 95%CI: –0.10 to 0.20), and this finding is consistent with the previous meta-analysis by Gist et al. (Cohen's d = 0.04, 95%CI: –0.17 to 0.24). However, the overall unclear risk of bias warrants cautious interpretation of these meta-analyses. Specifically, if the presence of bias produced inaccurate effect sizes for each study included in these meta-analyses, then the overall effect size and associated interpretation may also be inaccurate. It is also likely that the substantial inconsistency in these meta-analyses (Fig. 2 and the meta-analysis by Gist and colleagues) is attributable to differences in the frequency, intensity, and duration of MICT and SIT protocols across studies. We refer the reader to recent reviews that discuss this issue in greater detail., Our uncertainty in knowing whether or not bias-protecting methodologies were implemented emphasizes the importance of transparent and full reporting as outlined in the CONSORT guidelines. Although many journals endorse the CONSORT guidelines in the attempt to improve quality of reporting, the success of this approach is incumbent on editors, peer-reviewers, and authors to ensure that submitted manuscripts adhere to these guidelines. Collectively, our findings highlight several major concerns with studies comparing VO2max responses between SIT and MICT and support the need for rigorous risk of bias and reporting quality assessments in future systematic reviews of exercise medicine research.,

The poor quality of reporting (Table 3) meant we had to assign an “unclear” risk of bias for most studies (Table 2) as it is possible that studies protected against sources of bias but failed to report doing so. Devereaux et al. contacted authors of large clinical randomized controlled trials (RCTs) and found that many authors claimed to have performed bias-mitigating methodologies despite not reporting this information in their publications. Although this finding supports the idea that a lack of reporting does not necessarily reflect a lack of methodological rigor in large clinical RCTs, we are unaware of similar evidence in applied exercise science research. In contrast, strong meta-epidemiological evidence demonstrates that studies failing to report measures taken to mitigate bias (e.g., allocation concealment) produce inflated/biased effect sizes. This meta-epidemiological evidence supports a “guilty until proven innocent” approach whereby one should not assume a study protected against biased results unless the bias-mitigating methodologies were explicitly reported. However, additional empirical data supporting this approach is lacking as the majority of meta-epidemiological analyses do not separate studies with poor reporting from those that report performing an inadequate methodology.7, 8, 9, To determine whether a lack of reporting per se is associated with biased results, future work should compare effect sizes from studies with poor reporting vs studies that report performing an inadequate methodology.

Unfortunately, we could not perform a sensitivity analysis to determine the influence of bias on our meta-analysis, as no study was judged to have a low risk of bias. If we could have performed this sensitivity analysis, a different result (e.g., the overall 95%CI lay fully on one side of 0 indicating superiority of either MICT or SIT) might have suggested that biased results impacted our meta-analysis (Fig. 2). The impact of bias was demonstrated by Pildal et al. who found that approximately two-thirds of meta-analyses reporting an overall treatment effect lost this effect when only including studies that reported an adequate allocation concealment method. This finding supports the recent recommendations by Büttner and colleagues that, when applicable, future meta-analyses should conduct a sensitivity analysis to determine whether or not overall effects change when only including studies with a low risk of bias.

We provide a discussion below that describes each source of bias covered in the Cochrane Collaboration tool and makes recommendations to improve the overall methodological rigor of future studies comparing clinical outcomes between SIT and MICT. We also discuss sample size calculations as this aspect of study design was largely overlooked in the studies included in our meta-analysis (CONSORT Item 7A; Table 3).

Selection bias

Selection bias can occur when investigators assign participants to a given intervention group non-randomly. Protecting against selection bias requires generating an unpredictable random allocation sequence and concealing this sequence from all investigators involved in enrolling participants: a process referred to as “allocation concealment”.86, 87, 88 It is unclear whether or not studies included in our meta-analysis overlooked protecting against selection bias (Table 2) because most studies failed to report methodologies related to random sequence generation (CONSORT Item 8A), allocation concealment (CONSORT Item 9), and the implementation of these randomization procedures (CONSORT Item 10) (Table 3). It is imperative that future studies clearly report these procedures to allow researchers to easily assess the adequacy of these methods. Because there are many methods that fail to conceal allocation (e.g., sealed envelopes or randomizing via birth month),,, clear and transparent reporting is the only definitive way to demonstrate that a given study adequately protected against selection bias. For an example of clear reporting of adequate methods for reducing the risk of selection bias, we refer the reader to our recent study, in which we utilized a third party to generate a random allocation sequence using Microsoft Excel and conceal this sequence until group assignment.

Performance bias

Performance bias can occur when participants and/or personnel administering the interventions are not blinded to participants’ group assignments. Protecting against performance bias requires blinding participants as well as personnel to participants’ group assignments., All 27 studies included in our meta-analysis had an unclear risk of performance bias (Table 2), and this finding likely reflects the difficulty of blinding participants and personnel in exercise training studies., In situations where participants and/or personnel cannot be blinded, the CONSORT statement for non-pharmacological trials recommends that researchers report any attempts to limit performance bias (CONSORT Item 11C). However, only 2 studies, reported an attempt to reduce the possible impact of performance bias (Supplementary material, Sheet 3). Future studies comparing changes in VO2max between SIT and MICT should report attempts to reduce the risk of performance bias, such as concealing the study's hypothesis to participants and/or personnel.,

Detection bias

Detection bias, also known as observer or ascertainment bias, can occur when investigators responsible for assessing outcomes (herein referred to as “outcome assessors”) are aware of participants’ group assignments. Protecting against detection bias requires that outcome assessors are blinded to participants’ group assignments. The majority of studies included in our meta-analysis (25/27) had an unclear risk of detection bias (Table 2) as few studies reported whether or not outcome assessors were blinded (CONSORT Item 11A) (Table 3). Given this lack of clarity, we will highlight 2 possible methods for blinding VO2max outcomes assessors that could be reported in original manuscripts (if applicable). First, given evidence that encouragement affects obtained VO2max values, the individual performing the VO2max could be blinded to prevent them from providing unequal encouragement (e.g., more encouragement for SIT participants to align with assessor's belief that SIT is superior). Second, despite the objective nature of obtaining VO2max values (e.g., highest 30-s average), VO2max datafiles can be coded to prevent outcome assessors from manipulating or fabricating data. For an example of clear reporting of methods for blinding outcome assessors, we refer the reader to our recent study, in which we utilized a third party to code samples and data files .

Attrition bias

Attrition bias can occur when participants are lost to follow-up in a non-random fashion between groups (i.e., more dropouts in one group compared to another). When dropout rates are high and/or systematically different between groups, protecting against attrition bias can involve adopting an intention-to-treat (ITT) analysis whereby imputation methods generate data for dropouts so that all randomized participants are included in statistical analyses. The majority of studies included in our meta-analysis (20/27) had low rates of dropout and thus had a low risk of attrition bias (Table 2). Of the 6 studies with an unclear risk of attrition bias, 4 studies,,, did not report the number of dropouts (CONSORT Item 13A), reasons for dropout (CONSORT Item 13B), or the number of participants analyzed (CONSORT Item 16; Supplementary material); 2 studies, did not adopt an ITT analysis despite having dropout rates exceeding 20%, a rate that may introduce bias. Future studies should protect against attrition bias by reporting the number of dropouts and reasons for them and by considering adopting an ITT analysis if dropout rates are high.

Reporting bias

Reporting bias occurs when authors selectively report the results for certain outcomes and withhold the results for others. Protecting against reporting bias requires that researchers report a study's methods, including a list of primary and secondary outcomes, in a public registry or protocol publication before starting data collection. All 27 studies included in our meta-analysis had a high risk of reporting bias as the majority of studies (26/27) did not report their methodologies in a public registry. Although Kiviniemi et al. registered their protocol (CONSORT Item 23; Supplementary material), it was unclear whether or not these authors selectively reported outcomes, as a list of primary and secondary outcomes was not included in their registration file (clinicaltrials.gov: NCT01344928). Collectively, these findings highlight an overall high risk of reporting bias, which emphasizes the need for future work to report the primary outcome(s) a priori in a public registry (see Refs.101, 102, 103 for examples of public registries).

Sample size calculations

Small sample sizes risk generating a type II error (i.e., a false negative). An a priori sample size calculation estimates the sample size needed to statistically detect an expected effect at a pre-determined level of statistical power (i.e., probability of not making a type II error; typically 80%). Performing a priori sample size calculations and subsequently ensuring enrollment reaches the indicated sample size only helps reduce the risk of type II errors if calculations are completed accurately. Sample size calculations should utilize equations and assumptions that match the planned statistical analysis, and should be based on either a clinically meaningful change or an expected effect size and variance derived from previous studies using similar designs, populations, and methods for outcome assessment. Assessing the accuracy of a given sample size calculation requires that researchers report and justify the associated statistical parameters, which include the desired statistical power/type II error risk, alpha level/type I error risk, and the expected effect size and variance. Failing to perform an accurate a priori sample size calculation precludes researchers’ ability to determine whether a reported non-significant result reflects a true finding or a type II error.

The majority of studies (21/27) included in our meta-analysis either failed to report or inadequately reported whether or not they performed an a priori sample size calculation (CONSORT Item 6A) (Table 3). Thus, it is unclear whether or not these studies may have made a type II error when concluding that SIT and MICT are equally effective at improving VO2max. In theory, future studies could perform an a priori calculation to determine the sample size needed to detect a significant difference between SIT and MICT. However, because our overall effect size indicated a lack of superiority between SIT and MICT (Fig. 2), future studies could consider conducting a non-inferiority trial to determine whether SIT and MICT are equally beneficial at improving VO2max (see Refs., for information on non-inferiority trials).

Limitations

Our systematic review focused on VO2max responses to SIT and MICT, and it is therefore unclear whether our findings are generalizable to other areas of exercise medicine research. A popular topic in studies involving endurance athletes is determining the potency of supplementing habitual exercise training with SIT or high-intensity interval training.110, 111, 112, 113, 114 Although García-Pinillos et al. recently evaluated the methodological rigor of studies using high-intensity interval training to augment training load for endurance runners, these authors used inferior assessment tools (i.e., the PEDro Scale and Downs and Black Quality Index) and did not assess reporting quality. This is one of many examples of topics in exercise science research that warrant a systematic evaluation of methodological rigor and reporting quality using the Cochrane Collaboration tool and CONSORT checklist, respectively.

Conclusion

Our systematic review and meta-analysis found an unclear risk of bias owing to poor reporting quality in studies comparing changes in VO2max between SIT and MICT. Given these apparent methodological issues, future studies are encouraged to implement bias-reducing methodologies, as outlined in the Cochrane Collaboration tool, and to follow the reporting recommendations outlined in the CONSORT checklist for non-pharmacological trials. Furthermore, future systematic reviews in exercise medicine research should evaluate and (if possible) account for the risk of bias and reporting quality when synthesizing results in a meta-analysis. Although we focused on studies examining changes in VO2max following SIT and MICT in humans, the methodological and reporting principles highlighted in this review are applicable to all disciplines within exercise and sports medicine research.