Literature DB >> 34520000

Systematic Review of Exercise Studies in Persons with Multiple Sclerosis: Exploring the Quality of Interventions According to the Principles of Exercise Training.

Marit L Schlagheck1, Niklas Joisten2, Annette Rademacher3, Philipp Zimmer1, David Walzik1, Florian Wolf4, Sarah E Neil-Sztramko5, Jens Bansi6.   

Abstract

INTRODUCTION: The objective of this systematic review is to explore the application and reporting of (i) the principles of exercise training in exercise trials, (ii) the components of exercise prescription, and (iii) the adherence towards the prescribed programmes in randomised controlled trials (RCTs) in persons with multiple sclerosis (pwMS).
METHODS: The MEDLINE, CINAHL, SPORTDiscus, PubMed and Embase electronic databases were searched from 1 January 2000 to 16 October 2020. RCTs comprising at least 3 weeks of aerobic and/or resistance exercise intervention in pwMS that reported at least one physiological outcome and were published in peer-reviewed journals were eligible for inclusion.
RESULTS: Out of 52 RCTs included in this review, 58 intervention arms were examined. None applied more than four principles of exercise training. Specificity was addressed by 85%, progression by 33%, overload by 59%, initial values by 26%, reversibility by 0% and diminishing returns by 2% of trials. Fifty-two percent of trials reported all components of exercise prescription, and 3% of trials reported the level of adherence to the prescribed exercise.
CONCLUSION: This systematic review reveals that exercise training principles were not respected in the majority of included RCTs. The weak quality of reported exercise interventions limits the interpretation of the studies' results and potentially leads to an underestimation of 'exercise as medicine' in pwMS. Also, the vague descriptions of exercise prescription and adherence impede the reproducibility of results. Future studies must attend to all principles of exercise training and provide transparent information on the prescribed and performed programmes to develop specific and valid exercise recommendations for pwMS. SYSTEMATIC REVIEW REGISTRATION: CRD42020162671, 28/04/2020, PROSPERO.
© 2021. The Author(s).

Entities:  

Keywords:  Exercise prescription; Exercise therapy; Multiple sclerosis; Principles of exercise training; Systematic review

Year:  2021        PMID: 34520000      PMCID: PMC8571450          DOI: 10.1007/s40120-021-00274-z

Source DB:  PubMed          Journal:  Neurol Ther        ISSN: 2193-6536


Key Summary Points

Introduction

Multiple sclerosis (MS) is a chronic autoimmune disease of the central nervous system affecting approximately 2.8 million people worldwide [1]. Persons with MS (pwMS) typically experience a reduction in physical function (e.g., reduction in motor strength and coordination, sensory dysfunction, visual impairments, spasticity), fatigue, bowel/bladder dysfunction, cognitive deficits and/or depression [2]. Different disease-modifying therapies to prevent relapses or slow progression are under investigation; however, MS is to date not curable. As a consequence, effective rehabilitative strategies are considered a key supportive treatment option to enable pwMS to participate in activities of daily living and maintain their health-related quality of life. The research field of exercise physiology has become increasing differentiated for clinical populations, and in the early 2000s began implementing randomised controlled trials (RCTs) in order to contribute to evidence-based medicine. Apart from pioneers in the late 1990s [3], a growing number of peer-reviewed RCTs investigating exercise training as rehabilitation therapy for pwMS have been published over the last 2 decades. Current evidence suggests beneficial effects of exercise training on disease-specific symptoms such as fatigue or a decline in physical capacity [4, 5], possibly leading to overall improvements in quality of life [6, 7]. Moreover, it is frequently discussed that exercise training may slow disease progression, representing a major direction for future research [8]. Numerous reviews and meta-analyses have examined the efficacy of exercise training interventions with respect to different disease-related endpoints ranging from biological markers to patient-reported outcomes [9-13], reporting mixed and partially contradictory results [14]. These finding may be influenced by the quality and/or dosage of the exercise prescription. As the components included within the exercise intervention are analogous to the dose of a ‘medication’ in a pharmaceutical trial, a detailed review of the quality of exercise intervention prescriptions in RCTs with pwMS is necessary. In exercise training studies, it is important not only to adequately describe the exercise intervention with regard to intensity, type, etc., but also to report to what extent the participants actually met the prescribed exercise components (i.e. adherence). A detailed description of both the prescribed and applied intervention is crucial for scientific reproducibility and the successful transfer into clinical practice. Therefore, this systematic review of exercise studies in pwMS will follow the approach of Campbell and colleagues, who undertook an in-depth study to evaluate the prescription and application of exercise interventions in the field of exercise oncology according to the principles of exercise training and the exercise components frequency, intensity, type and time (FITT) [15-18]. The well-established principles of exercise training represent a core of exercise science, enhancing the chance for successful improvement in performance capacity. The principles of exercise training comprise specificity, overload, progression, initial values, reversibility and diminishing returns (see Table 1) [19]. Disregarding these principles may lead to an inadequate interventional design. For example, if exercise bouts represent an insufficient stimulus, they will not likely provoke any structural or functional adaptions (i.e., progression, overload). Studies might draw false conclusions regarding their outcomes of interest due to the deficient quality of the applied exercise training intervention. Additionally, proof-of-concept investigations might fail to replicate the intervention and its result due to insufficient information regarding exercise prescription and adherence of participants.
Table 1

Exercise training principles

PrincipleCriteria for this reviewExample
Specificity: Training adaptations are specific to the organ system or muscles trained with exerciseAppropriate population targeted and modality selected based on primary outcomeAerobic exercise such as brisk walking is more appropriate for an intervention aimed at increasing cardiovascular fitness than strength training
Progression: Over time, the body adapts to exercise. For continued improvement, the volume or intensity of training must be increasedStated exercise programme was progressive and outlined training progressionIncrease duration of walking program by 5% every two weeks depending on exercise tolerance
Overload: For an intervention to improve fitness, the training volume must exceed current habitual physical activity and/or training levelsRationale provided that programme was of sufficient intensity/exercise prescribed relative to baseline capacityPrescribing intensity in a resistance training program based on % of measured and/or estimated 1-repetition maximum
Initial values: Improvements in the outcome of interest will be greatest in those with lower initial valuesSelected population with low level of primary outcome measure and/or baseline physical activity levelsSelecting a sample with high baseline fatigue levels to participate in an aerobic training program to increase cardiovascular fitness and reduce fatigue
Reversibility: Once a training stimulus is removed, fitness levels will eventually return to baselinePerformed follow-up assessment on participants who decreased or stopped exercise training after conclusion of interventionParticipants who maintained training after a supervised exercise program preserved strength whereas those who stopped exercising returned to baseline
Diminishing returns: The expected degree of improvement in fitness decreases as individuals become more fit, thereby increasing the effort required for further improvements. Also known as the ‘ceiling effect’Performed follow-up assessment of primary outcomes on participants who continued to exercise after conclusion of interventionGains in muscle strength are greatest in the first half of a training program unless the training stimulus continually increases

Table is extracted from [16]

Exercise training principles Table is extracted from [16] The objective of this review is to explore the quality of exercise interventions considering the exercise prescription and adherence in RCTs in pwMS. We seek to expose potential methodological opportunities to improve future exercise intervention designs. Thereby, we aim to enhance the beneficial effects of exercise training in pwMS and to examine why some studies potentially fail. This review provides an in-depth overview of exercise prescriptions in RCTs in pwMS using aerobic and/or strength exercises as intervention.

Methods

A systematic literature review was conducted searching the MEDLINE, CINAHL, SPORTDiscus, PubMed and Embase electronic databases from 1 January 2000 to 16 October 2020 for relevant literature. Terminology related to ‘exercise’ and ‘multiple sclerosis’ was used for systematically searching the above-mentioned databases (see Supplementary Material 1 for full search string). The systematic literature search was conducted in accordance with the PRISMA guidelines. This article is based on previously conducted studies and does not contain any new studies with human participants or animals performed by any of the authors. Articles written in English and published in a peer-reviewed journal were considered for analysis. Further inclusion and exclusion criteria are displayed in Table 2 according to the PICOS format.
Table 2

Overview of in- and exclusion criteria according to the PICOS format

Inclusion criteriaExclusion criteria
Population

• Patients diagnosed with any type of multiple sclerosis (described by the authors as having multiple sclerosis)

• Aged over 18

• None
Intervention• Chronic aerobic or resistance exercise training or a combination of both

• Any other type of exercise intervention (e.g., yoga, tai chi, dancing)

Exercise intervention shorter than 3 weeks

Comparison• Between-group comparison• None
Outcome• At least one relevant physiological outcome related to exercise (e.g., aerobic capacity, muscular strength, functional capacity or body composition)

• Studies focusing on physical activity behaviour change

• Studies only reporting physical activity levels or psychological outcomes

Study design• Peer-reviewed human randomised controlled trials with one control arm (treatment as usual, waitlist, etc.)• Any other study design (e.g., cross-sectional, case, animal, cohort studies, editorial and opinion pieces, books, reviews)
Overview of in- and exclusion criteria according to the PICOS format • Patients diagnosed with any type of multiple sclerosis (described by the authors as having multiple sclerosis) • Aged over 18 • Any other type of exercise intervention (e.g., yoga, tai chi, dancing) Exercise intervention shorter than 3 weeks • Studies focusing on physical activity behaviour change • Studies only reporting physical activity levels or psychological outcomes Two independent reviewers (AR, MLS) screened the titles and abstracts of eligible articles. Each reviewer independently inspected the full text of each article that was included based on abstract and title. Disagreements were solved by consensus or, if required, by the contribution of a third reviewer (NJ). Relevant information of the manuscripts was extracted by the two reviewers (intervention description including sample size [overall and per group], duration of intervention, intervention setting, supervision of training, primary and secondary outcomes, and change in outcomes). Also, precise information about each manuscript’s exercise prescription was recorded according to the FITT criteria including frequency (number of sessions per week), intensity (relative or absolute intensity of exercise), time (duration of a single session) and type of resistance or aerobic exercise. As primary outcome, the application of exercise principles training was rated independently by the two reviewers (AR, MLS) for each intervention arm. For a ‘+’, application of the principle had to be clearly reported within the manuscript. ‘NR’ (not reported) was assigned when the application of the principle was not present. A ‘?’ was assigned when the principle was mentioned but its application was unclear or inconsistent. Similarly, manuscripts were rated based on adequate reporting of the components of exercise prescription and participants’ adherence according to the FITT criteria (secondary outcome) for each eligible intervention arm, respectively. Again, the two lists were compared and, if necessary, disagreement was solved by consensus or with the input of a third reviewer (NJ). Secondary publications were screened to determine whether assigned ratings needed adjustment, but were not interpreted independently. Finally, the number and percentage of studies meeting each criterion (i) for principles of exercise training and (ii) for reporting of exercise prescription and adherence to the prescribed programme were calculated.

Results

The electronic database search yielded 14,118 records. After the removal of duplicates, 6662 manuscripts were screened, from which N = 62 met the inclusion criteria and were included in this systematic review. Within these, results from 52 unique studies were described, and ten manuscripts were identified as secondary publications [7, 20–28] (see Supplementary Material 1 for the full list of included manuscripts). More precise information on the study selection process is outlined in Fig. 1. Eighteen studies (34.6%) involved only aerobic exercises [5, 29–45], 16 studies (30.8%) involved only resistance exercise [46-61], and 18 studies (34.6%) involved a combination of aerobic and resistance exercise [62-79]. Of these, ten studies were multi-armed trials that evaluated different rates of aerobic and resistance exercise [75], various aerobic exercise protocols [5, 79] or different community exercise interventions [67], or compared aerobic and/or resistance exercise to an ineligible intervention arm (such as yoga) [29, 38, 40, 48, 49, 54, 67]. No study compared the effects of aerobic exercise to resistance or combined exercise. This led to a total of 58 eligible intervention arms that were evaluated independently.
Fig. 1

PRISMA flow chart of study selection process

PRISMA flow chart of study selection process The exercise interventions lasted from 3 to 26 weeks, with 11 trials reporting follow-up measures from 5 to 36 weeks post-intervention [35, 41, 50, 52, 55, 57, 59, 65, 66, 71, 77]. Included studies varied in relation to the included MS phenotype, intervention setting (i.e., inpatient rehabilitation, ambulatory rehabilitation, non-rehabilitation settings such as university and fitness centre or home-based) and supervision (i.e., supervised, non-supervised or a combination of both). Supplementary Material 2 provides an overview of the included studies grouped by the training modality with information on the study population, intervention, supervision and primary outcomes measured. Out of 33 trials that defined a primary outcome, fewer than half (48.5%) presented a successful intervention by reporting changes in that outcome, but 50 trials (86.2%) reported favorable changes in at least one secondary outcome.

Application of Exercise Principles

Table 3 displays the rating of the applied principles of exercise training for all included intervention arms. After reviewing secondary publications, in one study [12] the rating of progression was adjusted from ‘?’ to ‘+’ and in another study [78] the rating of initial values was improved from ‘NR’ to ‘+’ as missing information about the rate of progression and recruiting were delivered in later manuscripts. Similarly, in one study [55] the rating of diminishing returns and reversibility was adjusted from ‘NR’ to ‘?’ since follow-up measurements reported in secondary publication did not consider the primary outcome. No trial applied more than four out of six principles of exercise training. Twenty trials (34.5%) addressed half or more (i.e., three or four) of the principles and 35 trials (60.3%) addressed fewer than half (i.e., one or two) of evaluated principles. Three trials (5.2%) did not apply any of the evaluated principles accurately (see Fig. 2).
Table 3

Application of the principles of exercise training and results of included studies

StudySpPrOvIvReDrSignificant between-group results (intervention vs. control group)
Aerobic exercise
 Ahmadi et al. (2013) [29]+??+NRNR↑ Balance, walking endurance, walking speed; ↓ Fatigue
 Baquet et al. (2018) [30]+?+NRNRNR↑ Aerobic capacity (PPO)
 Barclay et al. (2019) [31]+??NRNRNRNone
 Briken et al. (2014) [5]
  Cycling group+?+NRNRNR↑ 6MWT, aerobic capacity (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V{\text{O}}_{{2{\text{peak}}}}$$\end{document}VO2peak)a, cognitive performance [VLMT (learning, delayed recall), TAP (tonic alertness, shift of attention)]; ↓ Depression
  Rowing group+?+NRNRNR↑ Cognitive performance [VLMT (learning, delayed recall)]
 Arm ergometry group+?+NRNRNR↑ 6MWT, cognitive performance [VLMT (learning, delayed recall), TAP (shift of attention)]; ↓ Depression, fatigue
 Dettmers et al. (2009) [32]+NRNR+NRNR↑ Walking distancea, walking time
 Feys et al. (2019) [33]+???NRNR↑ Aerobic capacity (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V{\text{O}}_{2\max }$$\end{document}VO2max)a, brain volume left pallidum, cognitive function (SPART), functional capacity (STS)a, QoL (physical subscale)a, walking abilitya, workload (PPO)a; ↓ fatigue
  Geddes et al. (2009) [34]+??+NRNRNone
  Heine et al. (2017) [35]+NR+??+↓ Fatiguea
  Kargarfard et al. (2018) [36]+??NRNRNR↑ 6MWT, balance, functional capacity (STS), strength (push-up test); ↓ BMI, fatigue
  Mokhtarzade et al. (2017) [37]++++NRNR↑ Adiponectina, aerobic capacity (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V{\text{O}}_{2\max }$$\end{document}VO2max, PPO), QoL; ↓ BMI, body fat percentage, weight, fatigue, leptina, TNF-αa
  Negaresh et al. (2019) [45]+++?NRNR↑ Aerobic capacity (VO2peak), functional capacity (TUG); ↓ Depressiona, fatiguea
  Oken et al. (2004) [38]+NRNR?NRNR↑ QoL (SF-36 energy & fatigue, mental health subscales); ↓ Fatigue
  Sandroff et al. (2016) [43]+++NRNRNR↑ Aerobic capacity (time to exhaustion)
  Schulz et al. (2004) [44]+NR+NRNRNR↑ Aerobic capacity (lactate response), QoL
  Skjerbæk et al. (2014) [39]+NR+NRNRNRNone
  Tollár et al. (2019) [40]+NR??NRNR↑ 6MWT, QoL; ↓ Physical and psychological impact of MS (MSIS-29)a
  van den Berg et al. (2006) [41]+??NR?NR↑ Walking speed
  Zimmer et al. (2018) [42]+NR+NRNRNR↑ Aerobic capacity (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V{\text{O}}_{{2{\text{peak}}}}$$\end{document}VO2peak), cognitive performance (VLMT)a; ↓ MMP2
Resistance exercise
 Aidar et al. (2018) [46]+++NRNRNR↑ Balance, functional capacity (TUG, STS), walking speed, strength (1RM: squat, bench press, leg press, military press, front pulley, lunges)
 Amiri et al. (2019) [47]+?NRNRNRNR↑ Balance, core endurance tests (time until failure), core isometric strength tests (hip abduction, hip external rotation)
 Broekmans et al. (2011) [48]+++NRNRNR↑ Functional reach, isometric knee extensor strength
 Callesen et al. (2019) [49]++++NRNR↓ Fatigue
 Dalgas et al. (2009) [50]+++???↑ Functional capacity score (6MWT, 10-m walking time, SCT, CST)a, isometric knee extensor and knee flexor strengtha
 DeBolt et al. (2004) [51]++NRNRNRNR↑ Leg power (sum of maximal power from right and left leg divided by body weight)a
 Dodd et al. (2011) [52]+?+???↑ Muscle endurance (reverse leg press: number of repetitions at 50% 1RM), strength (1RM: leg press, reverse leg press), QoL (physical health subscale); ↓ Fatigue
 Fimland et al. (2010) [53]+?+NRNRNR↑ Soleus muscle activity (EMG)
 Hosseini et al. (2018) [54]++NR+NRNR↑ Strength (1RM: leg press)
 Jørgensen et al. (2019) [61]+?++NRNR↑ Isometric knee extensor and knee flexor strength, vastus lateralis and biceps femoris muscle activity (integrated EMG)
 Kjølhede et al. (2016) [55]++++??↑ Isokinetic knee extensor and knee flexor strength, walking speed
 Learmonth et al. (2011) [56]+?NR?NR?↑ Physical activity level
 Medina-Perez et al. (2014) [57]+++NR??↑ Knee extensor: isometric strength, maximal torque, muscular endurance
 Medina-Perez et al. (2016) [58]++++NRNR↑ Knee extensor: isometric strength, maximal torque, muscular endurance
 Miller et al. (2011) [59]?NRNR???None
 Moradi et al. (2015) [60]++++NRNR↑ Estimated 1RM strength (seated rowing, chest press, leg extension, leg press), functional capacity (3-min step test, TUG); ↓ EDSS
Aerobic and resistance exercise
 Abbaspoor et al. (2020) [62]++?+NRNR↑ Handgrip strength, IGF-1, walking speed
 Aidar et al. (2017) [63]+NR?NRNRNR↑ Balance, functional capacity (TUG, STS), walking speed
 Bjarnadottir et al. (2007) [64]???+NRNRNone
 Carter et al. (2013) [65]???+??None
 Carter et al. (2014) [66]???+??↑ Physical activity levela, QoL; ↓ Fatigue
 Garret et al. (2013) [67]
  Physiotherapist-led?++?NRNR↑ 6MWT; ↓ Fatigue (physical subscale), physical and psychological impact of MS (MSIS-29)a
  Exercise-instructor-led??NR?NRNR↑ 6MWT; ↓ Fatigue (physical subscale), physical and psychological impact of MS (MSIS-29)a
 Hansen et al. (2015a) [68]+?+?NRNR↓ Exercise blood lactate, RPE
 Hansen et al. (2015b) [69]+?+?NRNRaBlood lactate during exercise testing, exercise HR, RRa
 Magnani et al. (2016) [70]??++NR+↑ Aerobic capacity (VEAT, VEmax, VO2AT, VO2max, PAT, PPO)
 Maurer et al. (2018) [71]+?+???↑ QoL (mobility upper limb subscale)
 Pau et al. (2018) [72]+?++NRNR↑ Cadence, stride length, walking speed
 Paul et al. (2014) [73]?NRNRNRNRNRNone
 Romberg et al. (2004) [74]+?NR?NRNR↑ Walking speeda
 Sangelaji et al. (2016) [75]
  Group 1+++NRNRNR↑ 6MWT, balance, strength [1RM knee flexor (left and right) and extensor (left)], walking speed
  Group 2+++NRNRNR↑ 6MWT, strength (1RM knee flexor (right)),
  Group 3+++NRNRNR↑ Strength [1RM knee flexor (left and right) and extensor (left)]
 Surakka et al. (2004) [76]?+??NR?None
 Tallner et al. (2016) [77]+?+NR??↑ Aerobic capacity (peak expiratory flow), isometric knee extensor and knee flexor strength, physical activity level
 Wens et al. (2015a) [78]
  HIIT group+?+?NRNR↑ Aerobic capacity (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$V{\text{O}}_{2\max }$$\end{document}VO2max, PPO, test duration until exhaustion), isometric knee extensor and knee flexor strength, mean muscle fibre CSA, type IIa CSA, physical activity level
  HICT group+?+?NRNR↑ Mean muscle fibre CSAa, physical activity level; ↓ proportion type IIx fibresa
 Wens et al. (2015b) [79]+?NR?NR?↑ Isometric knee extensor and knee flexor strength strong leg; ↓ HR and lactate response to acute exercise

aIndicates significant between-group effect without reported direction

bIndicates primary outcome

+ clearly reported, ? unclearly reported, NR not reported, Sp specificity, Pr progression, Ov overload, Iv initial values, Re reversibility, Dr diminishing results, 1RM one-repetition maximum, 6MWT 6-minute walk test, BMI body mass index, CSA cross-sectional area, CST chair stand test, EDSS Expanded Disability Status Scale, EMG electromyography, HIIT high-intensity interval training, HICT high-intensity continuous training, HR heart rate, IGF-1 Insulin-like growth factor 1, MMP2 matrix metalloproteinase, MSIS-29 multiple sclerosis impact scale, P workload at anaerobic threshold, PPO peak power output, QoL quality of life, RPE rating of perceived exertion, RR respiratory rate, SCT ascending stair-climbing test, SF-36 Short form-36 health survey, SPART spatial recall test, STS sit-to-stand test, TAP test battery of attention, TNF-α tumor necrosis factor-alpha, TUG timed up & go test, VE pulmonary ventilation at anaerobic threshold, VEmax maximal pulmonary ventilation, VLMT verbal learning memory test, VO oxygen uptake at anaerobic threshold, VO maximal oxygen uptake, VO peak oxygen uptake

Fig. 2

Number of exercise principles applied across all studies

Application of the principles of exercise training and results of included studies aIndicates significant between-group effect without reported direction bIndicates primary outcome + clearly reported, ? unclearly reported, NR not reported, Sp specificity, Pr progression, Ov overload, Iv initial values, Re reversibility, Dr diminishing results, 1RM one-repetition maximum, 6MWT 6-minute walk test, BMI body mass index, CSA cross-sectional area, CST chair stand test, EDSS Expanded Disability Status Scale, EMG electromyography, HIIT high-intensity interval training, HICT high-intensity continuous training, HR heart rate, IGF-1 Insulin-like growth factor 1, MMP2 matrix metalloproteinase, MSIS-29 multiple sclerosis impact scale, P workload at anaerobic threshold, PPO peak power output, QoL quality of life, RPE rating of perceived exertion, RR respiratory rate, SCT ascending stair-climbing test, SF-36 Short form-36 health survey, SPART spatial recall test, STS sit-to-stand test, TAP test battery of attention, TNF-α tumor necrosis factor-alpha, TUG timed up & go test, VE pulmonary ventilation at anaerobic threshold, VEmax maximal pulmonary ventilation, VLMT verbal learning memory test, VO oxygen uptake at anaerobic threshold, VO maximal oxygen uptake, VO peak oxygen uptake Number of exercise principles applied across all studies The principle of specificity was addressed in all 20 aerobic [5, 29–45] (100%), 15 out of 16 resistance [23, 46–49, 51–58, 60, 61] (93.8%) and 14 out of 22 combined [62, 63, 68, 69, 71, 72, 74, 75, 77–79] (63.3%) interventions. Its application was unclear or inconsistent in one resistance [59] (6.3%) and eight combined [64–67, 70, 73, 76] (36.4%) interventions. Regarding specificity, no study was classified with an ‘NR’. The application of progression was properly reported in three out of 20 aerobic [37, 43, 45] (15%), ten out of 16 resistance [46, 48–51, 54, 55, 57, 58, 60] (62.5%) and six out of 22 combined [62, 67, 75, 76] (27.3%) interventions. Ten aerobic [5, 29–31, 33, 34, 36, 41] (50%), five resistance [47, 52, 53, 56, 61] (31.25%) and 14 combined [64–72, 74, 77–79] (63.6%) interventions were assigned an unclear for this principle. The remaining seven aerobic [32, 35, 38–40, 42, 44] (35%), one resistance [59] (6.3%) and two combined [63, 73] (9.1%) trials did not account for the principle of progression in their publications. The principle of overload was addressed by 11 out of 20 aerobic [5, 30, 35, 37, 39, 42–45] (55%), 11 out of 16 resistance [46, 48–50, 52, 53, 55, 57, 58, 60, 61] (68.8%) and 12 out of 22 combined [67–72, 75, 77, 79] (54.5%) trials. For seven aerobic [29, 31, 33, 34, 36, 40, 41] (35%) and six combined [62–66, 76] (27.3%) interventions, the compliance with this principle was unclear. In two aerobic [32, 38] (10%), five resistance [47, 51, 54, 56, 59] (31.3%) and four combined [67, 73, 74, 78] (18.2%) trials, the application of overload was not reported. The principle of initial values was correctly applied in four out of 20 aerobic [29, 32, 34, 37] (20%), six out of 16 resistance [49, 54, 55, 58, 60, 61] (37.5%) and six out of 22 combined [62, 64–66, 70, 72] (27.3%) intervention designs. Its application was unclear or inconsistent in five aerobic [33, 35, 38, 40, 45] (25%), four resistance [50, 52, 56, 59] (25%) and ten combined [67–69, 71, 74, 76, 78, 79] (45.5%) interventions. The remaining 11 aerobic [5, 30, 31, 36, 39, 41–44] (55%), six resistance [46–48, 51, 53, 57] (37.5%) and six combined [63, 73, 75, 77] (27.3%) trials did not report on use of initial values. Out of all included interventions, the principle of reversibility was applied in none of included trials. Reversibility was unclear in two aerobic [35, 41] (10%), five resistance [50, 52, 55, 57, 59] (31.3%) and four combined [65, 66, 71, 77] (18.2%) trials. Eighteen aerobic [5, 29–34, 36–40, 42–45] (90%), 11 resistance [46–49, 51, 53, 54, 56, 58, 60, 61] (68.8%) and 18 combined [62–64, 67–70, 72–76, 78, 79] (81.8%) interventions did not report the application of this principle. The principle of diminishing returns was reported in only one out of 20 aerobic [35] (5%) and one out of 22 combined [70] (4.5%) interventions. Its application was unclear in six resistance [50, 52, 55–57, 59] (37.5%) and six combined [65, 66, 71, 76–78] (27.3%) trials and not reported at all in 19 aerobic [5, 29–34, 36–45] (95%), ten resistance [46–49, 51, 53, 54, 58, 60, 61] (62.5%) and 15 combined [62–64, 67–69, 72–75, 79] (68.2%) trials.

Reporting of Components of and Adherence to Exercise Prescription

Figure 3a illustrates the reporting of components of exercise prescription according to the FITT criteria for the included intervention arms. All four components (i.e., frequency, intensity, time, type) were adequately reported for 16 aerobic [5, 29–31, 34–37, 39–43, 45] (80%), eight resistance [46, 48, 49, 51, 52, 54, 55, 60] (50%) and six combined [65, 66, 68–70, 72] (27.3%) interventions (overall: n = 30, 51.7%). One combined intervention [67] (4.5%; among all interventions included: 1.7%) failed to adequately report any of the four components. The frequency of training sessions was reported for all but one combined [67] (4.5%) intervention arm. The prescribed intensity was unclear in three aerobic [32, 33, 44] (15%), one resistance [47] (6.3%) and six combined [64, 67, 75, 76] (27.3%) interventions. The intensity was not reported for two resistance [56, 59] (12.5%) and three combined [67, 73, 74] (13.6%) interventions. Prescribed time of exercise was unclear in one aerobic [38] (5%) and 11 combined [62, 67, 71, 75–79] (50%) interventions. It was not reported in five resistance [50, 53, 57, 58, 61] (31.3%) and three combined [67, 73, 74] (13.6%) interventions. Prescribed type was unclear in one resistance [59] (6.3%) and three combined [67, 71, 74] (13.6%) interventions, whereas it was not reported at all for two combined [63, 73] (9.1%) interventions.
Fig. 3

(A) Reporting of components of exercise prescription. (B) Reporting of adherence to exercise intervention. The percentage of studies that adequately reported (+), were unclear in reporting (?) or did not report (NR) the component of exercise prescription or adherence

(A) Reporting of components of exercise prescription. (B) Reporting of adherence to exercise intervention. The percentage of studies that adequately reported (+), were unclear in reporting (?) or did not report (NR) the component of exercise prescription or adherence Reporting of adherence to the intervention is displayed in Fig. 3b. Among combined interventions, only two [65, 66] (9.1%) reported adherence to the prescribed intervention according to all FITT criteria (among all interventions included: 3.4%). Conversely, for seven aerobic [5, 29, 32, 37, 44] (35%), four resistance [46, 47, 49, 61] (25%) and 11 combined [62–64, 68–70, 72, 75, 76] (50%) interventions, no component of adherence was adequately reported. The frequency of exercise sessions attended was unclearly described for five aerobic [5, 30, 34] (25%), two resistance [49, 60] (12.5%) and four combined [64, 70, 72, 76] (18.2%) interventions. It was not reported at all in five aerobic [29, 32, 37, 41, 44] (25%), three resistance [46, 47, 61] (18.8%) and seven combined [62, 63, 68, 69, 75] (31.8%) trials. The intensity of exercise executed was unclear in four aerobic [5, 43] (20%) and one resistance [51] (6.25%) interventions. The intensity was not reported in 13 aerobic [29–34, 36–38, 40, 42, 45] (65%), 15 resistance [46–50, 52–54, 56–61] (93.8%) and 20 combined [62–64, 67–79] (90.9%) interventions. The actual time of exercise was unclear for two aerobic [38, 43] (10%) and one combined [76] (4.5%) intervention arms. In 17 aerobic [5, 29–37, 39, 40, 44, 45] (85%), all 16 resistance [46-61] (100%) and 19 combined [62–64, 67–73, 75, 79] (86.4%) trials, the duration of actually performed exercise was not reported at all. The type of exercise completed was unclear in one combined [76] (4.5%) intervention. Eight aerobic [5, 29, 37, 40, 44] (40%), seven resistance [46–49, 52, 59, 61] (43.8%) and 14 combined [62–64, 68–73, 75, 79] (63.6%) trials did not report the type of exercise completed at all.

Discussion

In contrast to frequently published papers that analysed the effects of exercise interventions on distinct outcomes [9-13], this review examined the quality of the interventions themselves applied in RCTs with pwMS. We analysed the application of principles of exercise training and evaluated the reporting of the components of prescribed and actually performed training according to the FITT criteria. Results of this review demonstrate that the existing exercise interventions in pwMS do not consistently address principles of exercise interventions, which may represent a reason for heterogeneous findings across different trials. Likewise, only 52% of trials reported all components of exercise prescription and only 3% of trials sufficiently reported adherence to the prescribed programme.

Application of Exercise Training Principles

The principles of exercise training represent fundamental components for the development of exercise programmes in order to respect physiological aspects of performance and to address all opportunities for improvement [19]. If these principles of exercise training are inconsistently applied in exercise intervention studies, non-significant results might be due to deficiencies in exercise prescription, leading to an underestimation of the true impact of exercise [15-18]. Out of 58 examined intervention arms, none applied more than four principles of exercise training, with reversibility and diminishing returns representing the least addressed principle. Specificity was addressed by 84% of reviewed interventions by prescribing aerobic and/or resistance exercise in accordance with the measured outcomes and study population. Combined studies that were unclear in the application of specificity targeted either aerobic- or resistance-specific outcomes but did not justify the use of a combined training programme [64–67, 70, 73, 74, 76]. Others did not sufficiently describe the content of the exercise prescription [59, 67, 73]. Thus, a rating as to whether the exercise programme was specifically aligned was not feasible. The principle of progression is based on the adaptability of the human body to repetitive stimuli over time. In order to go beyond maintenance and rather aim for further improvements, alterations in the stimulus are needed [19]. Only one-third of interventions adequately implied a progression of the training volume (i.e., intensity, frequency or duration) within the design of the training programme. For example, one resistance study increased the weight lifted by 1–5% when 12 repetitions were previously successfully accomplished [46]. Frequently, tables were utilised to display prescribed training volume and its progression [49, 55, 57, 58, 62]. They contained information about loading (e.g., percentage of maximal voluntary isometric contraction, maximum heart rate or one-repetition-maximum), frequency (sessions per week) and duration (e.g., duration of continuous aerobic exercise or number of repetitions and sets) for each session, week or month. Half of the included trials were unclear in progression, for example, due to a lack of detailed description of its realisation (e.g., underlying criteria, timing and quantity were not reported) [5, 30, 33, 36, 47, 52, 53, 64, 66, 67, 70, 78] or due to its non-systematic implementation (e.g., progression at the discretion of the subjects or trainer’s perception) [29, 31, 56, 65, 71]. In nine interventions a progression in training was not reported at all, even though the training period lasted from 2 to 6 months in some cases [38, 44, 59, 63, 73]. In others, the short time frame of 3–5 weeks [32, 39, 40, 42] might have made an application of progression difficult or even harmful. Neil-Sztramko et al. noted that the timing and rate of progression must be chosen wisely to ensure a high level of safety, motivation and efficacy [16]. In accordance with literature in the field of oncology [15-18], interventions in pwMS implying only resistance exercise reported the principle of progression more accurately than aerobic or combined interventions. To ensure an appropriate stimulus over time in aerobic trials, we recommend a predefined progression of the training intensity in percentage of individuals’ cardiorespiratory fitness (e.g., maximum oxygen uptake or maximum heart rate) and the duration and/or frequency of sessions. The principle of overload indicates that the dose of the applied exercise during the intervention must exceed the exercise load of what the individual participant is already used to in terms of frequency, duration and intensity. Considering the inter-individual variability in the disease course, pwMS are characterised by high heterogeneity in age, prior treatments, physical activity behaviour and physical capabilities [80]. Thus, a prescription based on individuals’ baseline capabilities is crucial [81]. Fifty-nine percent of trials ensured the application of overload by setting the training intensity based on measured initial aerobic fitness or strength. The application of overload was unsure in aerobic and combined trials that based their training intensities of the aerobic part on an age-predicted maximum heart rate of participants [29, 40, 41, 65, 66, 76]. An orientation on formulae such as ‘maximum heart rate = 220 − age in years’ is easy in use but critical in terms of accuracy [82], and is not recommended for universal application since studies in diverse populations such as pwMS are still rare [83]. Others defined intensity based on perceived exertion [31, 63], or referred to a modified Karvonen method by defining the maximum heart rate during a six-minute walk test instead of a progressive maximum exercise test [34]. In the latter, the authors explained their choice in testing with safety precautions. However, it is questionable whether intensity defined in this manner is sufficient, as the individual pacing ability can impact the test results [84]. Moreover, it has been shown that incremental exercise tests to exhaustion are feasible in pwMS [85]. In order to prescribe a suitable training load even though the participants might not reach their individual maximum, Briken et al. considered the aerobic threshold as a submaximal performance index [5]. Further submaximal markers such as the oxygen uptake efficiency slope have been discussed and might represent alternative methods to express cardiorespiratory fitness in pwMS [86]. Eleven interventions did not account for overload at all. For example, in one aerobic trial, playful elements (e.g., biathlon) were implemented for repetitive endurance exercise [32], which made it difficult to control for intensity and individual training load. In another aerobic trial, the prescriptions for intensity and duration were set to ‘be very light to moderate’ and at the discretion of participants themselves [38]. We question whether the intensity specification is sufficient to represent an effective stimulus and believe that individual psychological aspects that are not controlled for (i.e., participant motivation) determine the execution and success of the training. Two resistance trials set the training intensity relative to the participant’s initial body weight [51, 54]. However, this approach does not adequately address the capabilities of an individual, since individuals with the same body weight but different body composition or intramuscular coordination, for example, can have different strength. We strongly recommend that researchers rely on percentages of the individual’s baseline strength (i.e., one-repetition maximum) or cardiorespiratory fitness (i.e., maximum heart rate, maximum oxygen consumption, aerobic threshold) that are assessed according to acknowledged standards (e.g., published by the American College of Sports Medicine) [84] in order to secure overload in training. The principle of initial values posits that those participants who indicate deficits in measured outcomes and fitness at baseline are more likely to experience marked improvements following a training intervention than those who are characterised by high initial values. In this review, less than one-third of included trials respected the principle, by recruiting only participants with low values of the primary outcome [32, 37, 49, 66] or (in case no primary outcome was defined) low initial fitness or physical inactivity [29, 34, 54, 55, 58, 60–62, 64, 65, 70, 72]. Of these, a remarkable number (75%) did not define a primary outcome and therefore attained this principle only by considering participants’ level of activity. One-third of trials were unclear in attaining the principle of initial values, mostly because their inclusion criteria considered the activity level but did not consider initial values of all defined primary outcomes. Schulz et al. considered the previously defined primary outcome in the inclusion criteria and excluded all participants who scored < 14 on the Modified Fatigue Impact Scale (MFIS) [44]. Nonetheless, the chosen cut-off value is not comprehensible, as previous literature published a cut-off value of 38 to determine the presence of fatigue [87]. If participants with high baseline values of the primary outcomes are included, it is more likely that improvements are limited due to the ceiling effect rather than the ineffectiveness of the exercise programme itself. Previous studies with pwMS indicate that participants with low baseline fitness may profit more from typical exercise interventions with regard to the physical outcomes than those with high initial fitness [88]. Further research on an individual basis is needed to determine the influence of baseline fitness on training response and to adjust training prescriptions so that a sufficient stimulus can be provided to all participants [17]. The principles of reversibility and diminishing returns are both associated with the aim of providing long-term effects and require multiple measurements. It is expected that fitness levels will decrease and eventually return to baseline once the training stimulus is removed. In this review, no study complied with the principle of reversibility. Eleven studies included a follow-up measurement after discharge of intervention, but were unsure in evaluating the reversibility of training effects because they either did not re-evaluate the primary outcome [55] or did not record the activity level of the participants over the follow up-period [35, 41, 50, 52, 57, 59, 65, 66, 71]. Thus, one cannot be certain whether recorded changes are actually due to a reduction in physical activity behaviour. To detect potential long-term effects, Tallner et al. offered e-training for three additional months after disclosure of the regular intervention period and even recorded the number of strength sessions executed [77]. The attendance was poor, as only 36% of participants performed 80% or more of the sessions. However, interpretations of changes were only made on a group level, disregarding whether the exercise sessions were executed or not. According to the principle of diminishing returns, it is expected that the degree of improvements will decrease within a period of time as participants become fitter. Especially in longer interventions, more effort is needed to provide a sufficient stimulus over time. Only two studies respected this principle by readjusting the training volume according to results of a re-evaluation of cardiorespiratory fitness after half of the intervention period [35, 70]. Three further studies also conducted an interim analysis, but adjustments of the training volume for the following weeks of intervention were missing [56, 76, 78]. Ultimately, inconsistent application of the principles of exercise in studies with pwMS, as illustrated in this review, can impact the studies’ results. One cannot eliminate the possibility that non-significant or minor effects of interventions were caused ‘only’ by the lack of attention to the principles. The current state of research makes an analysis of long-term effects of exercise difficult, as the principles of reversibility and diminishing returns in particular are rarely considered. An enhanced application and reporting of the principles of exercise in interventions with pwMS will help to detect the true effect of exercise on the disease course and to implement more precise exercise recommendations for optimal and long-term training effects. It must be mentioned that we did not contact the authors but only utilised the manuscripts and supplementary materials to obtain underlying information. Some authors might have applied a principle of exercise training in their study design in an exemplary way but did not adequately report it. Thus, its application was not comprehensible for us, and the principle was rated as ‘not reported’ or ‘unclear’ in this review. Only half of included trials reported all components of exercise prescription, and two trials even failed to address any component adequately. The components that were the least adhered to were intensity and time. Five trials did not report the prescribed intensity [67, 73, 74], and ten trials were unclear in their prescription and/or description [32, 33, 44, 47, 64, 67, 75, 76]. For example, patients were asked to walk at their own comfortable speed [32]. Describing the intensity, especially in unsupervised resistance exercises, with body weight or materials like elastic bands can be a challenge; however, its reporting is necessary for both reproducibility and interpretation of published findings concerning the prescribed training content and quality. For example, in order to understand the application of overload and progression within the training programme, clear information about the applied intensity is required [18]. The duration of the prescribed exercise was especially poorly reported for the resistance part. Typically, only the number of repetitions was presented. However, knowledge about the total time is useful for the purpose of planning future studies and transfer into clinical practice. Even less attention was given to adherence to the prescribed exercise. Whereas 51% of trials reported all four FITT components in the exercise intervention, only 3% of trials reported adherence appropriately. Again, ‘intensity’ and ‘time’ were most often not reported, probably because of more demanding data management required for these two components. Whereas 26% and 35% of trials did not report the components intensity and time of the exercise intervention, more than 91% and 94% failed to report adherence to those components, respectively. Nevertheless, two trials presented adherence to the exercise programme in an exemplary way [65, 66], for example, describing the average number of sessions attended for the supervised and home-based part, respectively, and the type of exercise performed at home. In order to report the actual time and intensity performed, bar graphs were used to display the total minutes achieved at different intensities [66]. For the development of rehabilitation programmes tailored to the individual patient, precise knowledge about the content of prescribed and performed exercise seems to be crucial and warranted. In particular, detailed information about actually performed exercise can provide a foundation for future studies and practitioner understanding, firstly, of the type and extent of exercise that is feasible in pwMS with different disease severity, and secondly, what is necessary to promote health. Also, considering the high heterogeneity in response following exercise interventions in pwMS [88, 89], more valid conclusions can be drawn about individual predictors when adherence is clearly reported and can be considered in further research. A standardised and internationally endorsed consensus statement on exercise reporting (CERT) has been published previously [90]. The CERT, which also covers the FITT criteria for prescription and adherence, represents a guideline that we believe should be closely followed.

Conclusions

In this systematic review covering RCTs on aerobic and/or resistance exercise training in pwMS, no study attended to more than four out of six exercise training principles. Likewise, reporting of prescription and especially adherence to the programme according to the FITT criteria was poor. Thus, (i) interpretation of the present studies and (ii) reproducibility of their results are limited. If the evidence of exercise trials is to correspond to the evidence of pharmacological studies, the standard of the latter must be adhered to. Future studies need to put more emphasis on the adequate design and reporting of interventions. We recommend focusing on all six exercise training principles when designing an exercise programme to enhance the potential of exercise training-induced benefits. We urge authors to report not only exercise prescription but also its adherence in accordance with the FITT criteria in their main manuscript that presents results of the primary outcome or an additional study protocol. Information about adherence to the planned exercise programmes is crucial to determining the true effect of the interventions and replicating results in clinical practice. In order to overcome strict restrictions in word and page count set by scientific journals, the usage of online appendices and supplementary materials is recommended [16]. Below is the link to the electronic supplementary material. Supplementary file1 (PDF 250 kb)
Why carry out this study?
Exercise has been proposed as possibly having beneficial effects on disease progression of multiple sclerosis; however, mixed results are reported.
The principles of exercise training represent fundamental components for the development of exercise programmes in order to respect physiological aspects of performance.
This review examined the quality of the interventions themselves, considering the exercise prescription and adherence in randomised controlled studies in persons with multiple sclerosis (pwMS).
What was learned from the study?
Results of this review demonstrate that the existing exercise interventions in pwMS did not consistently address principles of exercise interventions or adequately report the prescription and adherence to the programme, which may represent a reason for heterogeneous findings across different trials and lead to an underestimation of the rehabilitative benefits of exercise.
Future studies need to put more emphasis on the exercise prescription and reporting of its actual ‘dosage’, especially when exercise is considered as medicine in pwMS.
  68 in total

1.  Natural history of multiple sclerosis symptoms.

Authors:  Ilya Kister; Tamar E Bacon; Eric Chamot; Amber R Salter; Gary R Cutter; Jennifer T Kalina; Joseph Herbert
Journal:  Int J MS Care       Date:  2013

2.  Effects of exercise training on multiple sclerosis biomarkers of central nervous system and disease status: a systematic review of intervention studies.

Authors:  R Negaresh; R W Motl; P Zimmer; M Mokhtarzade; J S Baker
Journal:  Eur J Neurol       Date:  2019-03-14       Impact factor: 6.089

Review 3.  The benefits of exercise training in multiple sclerosis.

Authors:  Robert W Motl; Lara A Pilutti
Journal:  Nat Rev Neurol       Date:  2012-07-24       Impact factor: 42.937

Review 4.  Exercise as Medicine in Multiple Sclerosis-Time for a Paradigm Shift: Preventive, Symptomatic, and Disease-Modifying Aspects and Perspectives.

Authors:  Ulrik Dalgas; Martin Langeskov-Christensen; Egon Stenager; Morten Riemenschneider; Lars G Hvid
Journal:  Curr Neurol Neurosci Rep       Date:  2019-11-13       Impact factor: 5.081

5.  High-intensity interval training combined with resistance training improves physiological capacities, strength and quality of life in multiple sclerosis patients: a pilot study.

Authors:  Pierre Zaenker; Fabrice Favret; Evelyne Lonsdorfer; Guillaume Muff; Jérôme de Seze; Marie-Eve Isner-Horobeti
Journal:  Eur J Phys Rehabil Med       Date:  2017-07-05       Impact factor: 2.874

Review 6.  Review of exercise studies in breast cancer survivors: attention to principles of exercise training.

Authors:  Kristin L Campbell; Sarah E Neil; Kerri M Winters-Stone
Journal:  Br J Sports Med       Date:  2012-10       Impact factor: 13.800

7.  Cardiovascular adaptation in people with multiple sclerosis following a twelve week exercise programme suggest deconditioning rather than autonomic dysfunction caused by the disease. Results from a randomized controlled trial.

Authors:  M G Feltham; J Collett; H Izadi; D T Wade; M G Morris; A J Meaney; K Howells; C Sackley; H Dawes
Journal:  Eur J Phys Rehabil Med       Date:  2013-07-23       Impact factor: 2.874

8.  Which behavioural and exercise interventions targeting fatigue show the most promise in multiple sclerosis? A systematic review with narrative synthesis and meta-analysis.

Authors:  Rona Moss-Morris; Anthony M Harrison; Reza Safari; Sam Norton; Marietta L van der Linden; Federica Picariello; Sarah Thomas; Claire White; Tom Mercer
Journal:  Behav Res Ther       Date:  2019-08-28

9.  Impact of aerobic training on fitness and quality of life in multiple sclerosis.

Authors:  J H Petajan; E Gappmaier; A T White; M K Spencer; L Mino; R W Hicks
Journal:  Ann Neurol       Date:  1996-04       Impact factor: 10.422

10.  Updated systematic review of exercise studies in breast cancer survivors: attention to the principles of exercise training.

Authors:  Sarah E Neil-Sztramko; Kerri M Winters-Stone; Kelcey A Bland; Kristin L Campbell
Journal:  Br J Sports Med       Date:  2017-11-21       Impact factor: 13.800

View more
  1 in total

1.  Reliability of Isometric Muscle Strength Measurement and Its Accuracy Prediction of Maximal Dynamic Force in People with Multiple Sclerosis.

Authors:  Kora Portilla-Cueto; Carlos Medina-Pérez; Ena Monserrat Romero-Pérez; José Aldo Hernández-Murúa; Carolina Vila-Chã; José Antonio de Paz
Journal:  Medicina (Kaunas)       Date:  2022-07-18       Impact factor: 2.948

  1 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.