Literature DB >> 35874861

Telemedicine acceptance and efficacy in the context of preventive cardiology interventions: A systematic review.

Felix Gass1,2, Martin Halle1,2, Stephan Mueller1,2.   

Abstract

Introduction: Telemedicine is being used in an increasing number of healthy lifestyle intervention studies in preventive cardiology. However, the optimal telemedicine-based approach for patients with cardiovascular disease remains unclear. Therefore, the aim of this systematic review is to identify which design features are associated with the acceptance and efficacy of telemedicine in this specific patient population.
Methods: The databases PubMed/MEDLINE, Embase and the Web of Science Core Collection were searched from 5 October 2010 to 5 October 2020. This systematic review only included randomized controlled or quasi-randomized controlled trials with a comparator to a telemedicine-based intervention group and a designated measure of adherence. We adopted a narrative synthesis approach to define telemedical design features, which were clustered into three main categories (social, exercise related and barrier removal) and compared to adherence (graded as good, medium and bad) and primary outcomes (significant improvement, no significant change).
Results: We screened a total of 865 records, of which 14 were included in this review, containing 13 identified design features. In 8 studies (57.1%), adherence was graded as good (4 studies medium, 2 studies bad). A positive primary outcome occurred in 10 (71.4%) studies. Personal contact showed the most pronounced (while not statistically significant) positive association with adherence and study outcomes.
Conclusion: Given the remote nature of telemedical lifestyle intervention studies, including recurring personal contact in the intervention seems to be a key factor in ensuring that adherence levels remain comparable to those seen in centre-based interventions.
© The Author(s) 2022.

Entities:  

Keywords:  Telemedicine; adherence; cardiovascular; intervention

Year:  2022        PMID: 35874861      PMCID: PMC9297471          DOI: 10.1177/20552076221114186

Source DB:  PubMed          Journal:  Digit Health        ISSN: 2055-2076


Introduction

There is a growing body of research postulating various potential benefits linked to a healthy lifestyle and specifically physical exercise in the field of preventive cardiology. Studies show that maintaining a healthy lifestyle at midlife is associated with an increased life expectancy, while also reducing the occurrence and progression of major chronic conditions, such as cardiovascular disease (CVD) and type 2 diabetes mellitus (T2DM). In lifestyle intervention research (i.e. interventions focusing on the variables of physical activity (PA) and nutrition), the clinical outcomes regarding the prevention and progression of chronic diseases can be significant when participants closely adhere to the programmes, yet adherence to exercise interventions over an extended period of time is generally low, particularly in older populations with pre-existing health conditions.[2,3] This issue is even more pronounced during home-based exercise interventions, where adherence tends to be inferior compared to centre-based interventions. Here, telemedicine offers an avenue of addressing this problem. For home-based interventions, telemedicine has been shown to increase patient adherence when compared to studies not utilizing telemedical concepts. However, the typical patient with CVD is of advanced age and often struggles with telemedical concepts, seeing how technology adoption and willingness to work with new and unknown devices is often low in elderly patient populations. Evidently, the optimal telemedical approach in healthy lifestyle intervention studies focusing on CVD patients remains unclear. To our knowledge, there are no studies that assess how telemedicine impacts adherence to healthy lifestyle interventions. Therefore, the aim of this systematic review is to address this research gap by identifying design features that are associated with the acceptance and efficacy of telemedicine in CVD patient populations, further evaluating how adherence is being measured in settings, where the participants are not actually on site during the intervention.

Methods

This systematic review follows best practice recommendations from the Cochrane Collaboration and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. The review protocol was registered in the International Prospective Register for Systematic Reviews with the registration number CRD42020209851. Further, this review employs the definition of telemedicine from the World Health Organization (WHO) : “[Telemedicine is] the delivery of health care services, where distance is a critical factor, by all health care professionals using information and communication technologies for the exchange of valid information for diagnosis, treatment and prevention of disease and injuries, research and evaluation, and for the continuing education of health care providers, all in the interests of advancing the health of individuals and their communities”.

Literature search

The search was conducted using three databases: PubMed/MEDLINE, Embase and the Web of Science Core Collection. Results were limited to English journal articles published from 5 October 2010 to 5 October 2020. The applied search terms varied slightly between databases. The exact search term in PubMed/MEDLINE was: ‘(telemed*[Title/Abstract] OR tele med*[Title/Abstract] OR telehealth [Title/Abstract] OR tele health [Title/Abstract] OR mobile health[Title/Abstract] OR mHealth[Title/Abstract] OR eHealth[Title/Abstract] OR mobile med*[Title/Abstract]) AND (lifestyle[Title/Abstract] OR exercise[Title/Abstract] OR physical activity[Title/Abstract] OR training[Title/Abstract]) AND (intervention[Title/Abstract] OR prospective[Title/Abstract]) AND (cardiovascular [Title/Abstract] OR cardiology [Title/Abstract] OR cardia* [Title/Abstract] OR CVD [Title/Abstract] OR heart [Title/Abstract] OR diabet* [Title/Abstract] OR obese* [Title/Abstract] OR overweight [Title/Abstract]) AND (compliance OR adherence OR acceptance OR efficacy OR adoption)’.

Eligibility criteria

Eligible studies involved participants with or at risk of CVD (as defined by overweight or T2DM). We included only randomized controlled or quasi-randomized controlled trials which had to include a comparator to the telemedicine-based intervention group and a measure of adherence. Studies were excluded if they focused on children and adolescents, pregnant women or other special populations, non-lifestyle-related factors such as medication adherence, or if the chosen adherence measure could not be graded. Furthermore, we excluded studies if part of the intervention occurred after 11 March 2020, the date on which the WHO declared COVID-19 a pandemic, to eliminate the influence that the pandemic ensued on the adherence of ongoing interventions.

Study selection and data extraction

Two authors independently screened titles and abstracts, rating them with either ‘yes’, ‘no’ or ‘unsure’ regarding their potential inclusion in the review. The authors were blinded to each other's decisions, articles only being excluded if both rated them with ‘no’. Both authors then read the remaining articles in full and screened them in detail for the described eligibility criteria. Consensus between authors had to be reached for an article to be included in the review. For each included study we then extracted descriptive data regarding methodology and study design, patient population, intervention components, adherence measures as well as the identified design features. We further extracted specific secondary outcomes, regarding weight loss, PA and exercise capacity.

Data synthesis and risk of bias assessment

This review adopted a narrative synthesis approach as described by Popay et al. to define telemedical design features which are associated with patient adherence and consequently intervention success, following the recommendations by the Synthesis Without Meta-analysis reporting guideline. For further explorative analysis and to allow better quantification of the impact that these design features had on primary study outcomes and adherence, risk ratios (RR) were calculated and tested for significance via two-tailed Fisher exact probability test Meta-analysis was deemed as unfitting due to data heterogeneity. Two authors independently evaluated each individual study and rated the adherence as good, medium or bad. Given the wide range of reported adherence measures and their consequently difficult comparability, the grading process itself was qualitative in nature. However, to be graded as good at least 67% of the utilized adherence measure had to be reached (a cut-off established in prior lifestyle intervention studies.[11,12]) Further, this review utilized version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB 2) to assess the risk of bias in the included studies.

Results

Study selection

A PRISMA flow diagram is shown in Figure 1. The deployed search strategy yielded 865 records, of which 591 remained after the removal of duplicates. After title and abstract screening, 77 articles were read in full, of which 14 [n = 1284, mean number (range) of participants 92 (36–171)] were included in this review (Table 1). Furthermore, we conducted a post hoc literature search adding ‘cholesterol [Title/Abstract]’ and ‘hypertens* [Title/Abstract]’ to the initial search terms (both for Title/Abstract). This yielded an additional 15 articles (11 on hypertension, 4 on cholesterol), of which none would have been included in this review (due to either not meeting the inclusion or meeting our exclusion criteria).
Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram.

Table 1.

Included studies.

TrialParticipants N InterventionIntensityDurationDevice/Program
Avila et al., 2018 14 CR patients- mean age 61.2 years- 11% female- 89% male- mean BMI 27.5 kg/m290IG home: exercise intervention with telemonitoring guidance (weekly emails or phone calls) IG centre: standard in-hospital CR CG: usual careHome: At least 150 min of PA per week (preferably 6–7 days/week) at moderate intensity (70%–80% of HRR) Centre: Three sessions per week totaling 150 min of PA12 weeks- Heart rate monitor (Garmin Forerunner 210, Wichita, USA) - Phone calls- e-mails
Azar et al., 2015 15 overweight- mean age 46.3 years- 0% female- 100% male- mean BMI 34.8 kg/m264 (no power calculation provided)Diabetes Prevention Program-based Group Lifestyle Balance core curriculum 28 At least 150 min of moderate PA similar in intensity to brisk walking per week12 weeks- Web and cloud-based VC software (Blue Jeans™)- Smart scales
Ballin et al., 2020 16 Central obesity- mean age 70.5 years- 50% female- 50% male- mean BMI 28.7 kg/m279Web-based exercise: progressive interval training Supervised exercise: progressive interval trainingThree sessions per week of vigorous intensity PA, duration gradually increasing from 18 min to a maximum of 36 min per session10 weeks- Online platform
Barnason et al., 2019 17 CR patients- mean age 63 years- 30% female- 70% male- mean BMI 35.1 kg/m243Sixmodules with 36 telehealth sessions modelled after the Diabetes Prevention Programme 29  + target caloric goalsAt least 150 min of moderate PA similar in intensity to brisk walking per week12 weeks- Accelerometer (Actigraph GT3X, Pensacola, USA)- Viterion® telehealth device
Bernocchi et al., 2018 18 COPD and CHF patients- mean age 70 years- 8% female- 92% male- mean BMI 28.1 kg/m2112Remote monitoring of cardiorespiratory parameters, weekly phone-calls by nurse & exercise programme monitored by physiotherapistBasic level: 15–25 min of exercise with mini-ergometer without load + 30 min of callisthenic exercises 3 days/week + free walking 2 days/week High level: 30–45 min of mini-ergometer with incremental load (from 0 to 60 W) + 30–40 min of muscle reinforcement exercises using 0.5 kg weights + pedometer-based walking on 3 to 7 days/week4 months- Pulse oximeter (GIMA, Milan, Italy)- Portable one-lead electrocardiograph (Card Guard Scientific Survival Ltd, Rehovot, Israel)- Phone calls- Mini-ergometer- Pedometer
Claes et al., 2020 19 CR patients- mean age 61.4 years- 18.3% female- 81.7% male- mean BMI 27.9 kg/m2120Online platform with regular exercise sessions as basis for personalized lifestyle interventionTraining was set at a heart rate between first and second ventilatory threshold. The goal was 150 min of moderate intensity PA per week6 months- Online platform- e-mails- Text messages- Accelerometer (GT9X Link, Actigraph, Pensacola, USA)- Microsoft band 2 (Microsoft, Redmond, USA)- Motion capturing (Microsoft Kinect)
Dale et al., 2015 20 CR patients- mean age 59.5 years- 18.7% female- 81.3% male- mean BMI 29.5 kg/m2123Core components of CR delivered Via text messages and a supporting website in combination with a pedometer to self-monitor PARecommendations based on ACSM's guidelines for exercise testing and prescription 30 24 weeks- text messages- online platform- pedometer
Fukuoka et al., 2015 21 Overweight and T2DM risk- mean age 55.2 years- 77% female- 23% male- mean BMI 33.3 kg/m26110% body weight loss over 5 months by increasing PA, reducing caloric intake and lowering fat intake. The curriculum was adapted from the Diabetes Prevention Programme 29 with the frequency of in-person sessions reduced from 16 to 6 and group replaced by home-based exerciseEncouragement for moderate-intensity PA (e.g. brisk walking). Long-term goal: increase and maintain step counts to 12000 per day5 months- Omron Active Style Pro HJA-350IT pedometer (Omron, Kyōto, Japan)- Smartphone application
Höchsmann et al., 2019 22 T2DM patients- mean age 57 years- 47.2% female- 52.8% male- mean BMI 33 kg/m236IG: smartphone game – restore a decayed garden CG: exercise programme comparable to the game content to be implemented autonomouslyThe game included workouts and promotion of daily PA that follow the American College of Sports Medicine 31 and European Association for Cardiovascular Prevention & Rehabilitation principles of exercise training 32 24 weeks- Smartphone game
Kaur et al., 2015 23 diabetes patients- mean age 52.7 years- 49.2% female- 50.8% male- mean BMI 28.1 kg/m2120 (no power calculation provided)Three groups – frequency of follow-up: Rare mode: advised 3-monthly follow-ups on OPD basis Moderate mode: advised monthly OPD visits Frequent mode: advised monthly OPD and weekly telephonic consultationNot specified12 weeks- Phone calls- Blood glucose monitor
Kooiman et al., 2018 24 T2DM patients- mean age 56 years- 47.2% female- 52.8% male- mean BMI 32.9 kg/m272Usual care + activity tracker and access to an online self-tracking programmeMinimum of 7500 steps/day or 150 min of moderate to vigorous PA per week12 weeks- Fitbit Zip (Fitbit Inc, San Francisco, USA)- Online platform
Lunde et al., 2020 25 CR patients- mean age 59 years- 22.1% female- 77.9% male- mean BMI 29 kg/m2113Individualized CR follow-up enabled with a smartphone applicationNot specified – depended on baseline assessment12 months- Smartphone application- e-mails
Maddison et al., 2015 26 CR patients- mean age 60 years- 19% female- 81% male- mean BMI 28.7 kg/m2171Personalized, automated text messages via mobile phone aimed at increasing exercise behaviour and supported by a websiteAt least 30 min of moderate to vigorous PA at least 5 day per week24 weeks- Online platform- Text messages
Widmer et al., 2017 27 CR patients- mean age 63 years- 18.3% female- 81.7% male- mean BMI 31 kg/m280Online and smartphone-based CR platform to self-monitor diet and exercise + educational information on healthy lifestylesNot specified3 months- Smartphone application- Online platform

BMI, body mass index; CG, control group; CR, cardiac rehabilitation; CHF, chronic heart failure; CVD, cardiovascular disease; COPD, chronic obstructive pulmonary disease; HRR, heart rate reserve; IG, intervention group; OPD, outpatient department.; PA, physical activity; VC, video conferencing; T2DM, type 2 diabetes mellitus.

Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram. Included studies. BMI, body mass index; CG, control group; CR, cardiac rehabilitation; CHF, chronic heart failure; CVD, cardiovascular disease; COPD, chronic obstructive pulmonary disease; HRR, heart rate reserve; IG, intervention group; OPD, outpatient department.; PA, physical activity; VC, video conferencing; T2DM, type 2 diabetes mellitus.

Risk of bias

The results of the quality assessment are highlighted in Table 2. All of the included randomized controlled trials were assessed as high risk of bias regarding the lack of blinding of participants and personnel. This is however a common problem and difficult to achieve in the field of exercise interventions. The other cases of high risk were assessed in the domain of other biases, where four studies did not register their trials in any online registry and one study only did so retrospectively.
Table 2.

Risk of bias assessment using version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB2).

TrialRandom sequence generationAllocation concealmentBlinding of participants and personnelBlinding of outcome assessmentIncomplete outcome dataSelective reportingOther bias
Avila et al., 2018 14  +  + - +  +  +  + 
Azar et al., 2015 15  + ?- +  + -
Ballin et al., 2020 16  +  + - +  +  + -
Barnason et al., 2019 17 ??-? +  + -
Bernocchi et al., 2018 18  +  + - +  +  + 
Claes et al., 2020 19  +  + - +  +  + 
Dale et al., 2015 20  +  + - +  +  + 
Fukuoka et al., 2015 21  +  + - +  +  + -
Höchsmann et al., 2019 22  +  + - +  +  +  + 
Kaur et al., 2015 23  + ?-? +  + -
Kooiman et al., 2018 24  + ?-? +  + 
Lunde et al., 2020 25  +  + - +  +  + 
Maddison et al., 2015 26  +  + - +  +  +  + 
Widmer et al., 2017 27  +  + - +  +  + 

 + , low risk; ∼, some concerns; - , high risk and ?, uncertain risk.

Risk of bias assessment using version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB2). + , low risk; ∼, some concerns; - , high risk and ?, uncertain risk.

Contents and outcomes of telemedical interventions in preventive cardiology

Table 1 offers a brief description of the main intervention contents of the included studies. Overall, it is evident that telemedicine is a very heterogeneous concept ranging from simple text messages to complex motion capturing. The most common intervention components were: The reported outcomes of the included studies are presented in Table 3. Notably, 10 out of 14 studies (71.4%) reported a significant difference in the primary outcome between groups. Regarding the chosen secondary outcomes, 9 studies reported on changes in weight and of those, 5 (55.6%) reported significant between-group differences. Daily PA was reported by 10 studies, six (60%) reporting a significantly higher change in the intervention group. Lastly, changes in cardiovascular fitness, as assessed by peak V˙O2, were reported by five studies of which two (40%) reported significant between-group differences.
Table 3.

Included studies – outcomes.

TrialPrimary outcomePrimary outcomeWeightDaily PAPeak V˙O2Adherence
Avila et al., 2018 14 Peak V˙O2 +  + good
Azar et al., 2015 15 Weight loss +  + NANAgood
Ballin et al., 2020 16 Visceral adipose tissueNAgood
Barnason et al., 2019 17 Weight loss +  + NAgood
Bernocchi et al., 2018 18 6MWT + NA + NAgood
Claes et al., 2020 19 Daily PA +  + bad
Dale et al., 2015 20 Adherence + NANANAmedium
Fukuoka et al., 2015 21 Weight loss +  +  + NAmedium
Höchsmann et al., 2019 22 Daily PA + NA + NAgood
Kaur et al., 2015 23 Adherence + NANANAgood
Kooiman et al., 2018 24 HbA1c + NAmedium
Lunde et al., 2020 25 Peak V˙O2 +  + NA + good
Maddison et al., 2015 26 Peak V˙O2NA + bad
Widmer et al., 2017 27 CVD-related rehospitalizations and ED visits + medium

 +  , significant improvement; ∼, no significant change; CVD, cardiovascular disease; ED, emergency department; PA, physical activity; V˙O2, oxygen uptake; 6MWT = 6-min walking testand.

online platforms (6 out of 14 studies), activity trackers (6 out of 14 studies), smartphone applications/games (4out of 14 studies), text messages (3 out of 14 studies), e-mails (3 out of 14 studies) and phone calls (3 out of 14 studies). Included studies – outcomes. +  , significant improvement; ∼, no significant change; CVD, cardiovascular disease; ED, emergency department; PA, physical activity; V˙O2, oxygen uptake; 6MWT = 6-min walking testand.

Adherence measures and design features of telemedical interventions

Table 4 shows how adherence was measured in the included studies and the different design features which could be identified. In eight studies (57.1%), adherence was graded as good, in four (28.6%) as medium and in two (14.3%) as bad. Of the included studies, 10 (71.4%) utilized an objective measure of adherence while 4 (28.6%) relied on self-reported measures. Further, only 2 (14.3%) of the included studies took into account some dimension of intensity when analysing adherence, whereas the remaining 12 (85.7%) contented with what could more aptly be described as measures of attendance.
Table 4.

Design features and adherence of telemedical intervention studies.

TrialDesign featuresAdherence
Avila et al., 2018 14 - Run-in period (first three sessions supervised)- Individually determined target heart rate zone corresponding to moderate intensity- Weekly feedback to discuss exercise programme and barriers to adherencehome:- 2.5 sessions per week (range: 12–60 sessions for 12 weeks)- 164 min per week at an average intensity of 46.8% of HRR (76.7 min within the prescribed zone) centre:- 2.0 sessions per week (range: 4–36 sessions for 12 weeks)- 90 min per week at an average intensity of 61.2% of HRR
Azar et al., 2015 15 - Similarity to in-person setting- Software training prior to first session- High-definition web camera with built-in microphone was provided- Group intervention- Group session attendance: 75%- Weight self-monitoring (at least 1/week): 87%
Ballin et al., 2020 16 - Age-appropriate video actors- Informational videos- Individual progression and adjustments- Supervised session before intervention- Provision of suspension band to reduce risk of falling- Detailed instructions on access and use of websiteWeb-based: Self-reported number of completed training sessions  = 85% Supervised: Median attendance rate = 89%
Barnason et al., 2019 17 - One-to-one dietary educationTelehealth session completion:- <5 sessions = 18%- 30–36 sessions = 46%- 36 sessions = 36%
Bernocchi et al., 2018 18 - Exercise equipment provided- Personalised exercise programme- Training videos- The mean duration of nurse and physiotherapist phone contact per patient was 45 min per week- Utilization of Telemedicine Service Centre (HTN, Brescia, Italy) to ensure 24 h/day patient support93% patients performed the prescribed exercises activity sessions/week:- 2.3 sessions = 19%- 4 sessions = 65%- 6 sessions = 16%
Claes et al., 2020 19 - Run-in period (4 weeks)- CPET to determine individual training heart rates- Different exercise modalities available: Exerclass, Exergame, Active lifestyle activity- Exercise sessions automatically adjusted according to enjoyment and exertion as measured via questionnaire- Social connectivity module enabled small groups of remote participants to exercise together by communication via headsets and a live chat function- Motion capturing analysed movements during exerclass and exergame sessions and provided feedback on accuracy of execution- Participants set goals for other lifestyle behaviors and received personalized, automatically generated text messages or e-mails to support goal achievement- Text-based performance feedbackUpload frequency (median number of sessions during 6 months):- Active lifestyle activities = 27- Exerclasses = 14.5- Exergames = 1- 57% of participants set at least 1 extra goal for CVD risk factor modification
Dale et al., 2015 20 - Included coping strategies for modifying illness perception to reduce negative emotions associated with disease- Messages were tailored to preferred time of day and frequency of messages decreased over time- Bidirectional messaging: participants were prompted to submit weekly step counts and to ask questions- Supporting website included tips, graphs displaying pedometer counts and short video messages from role models and medical professionals- If necessary, phone was provided- Participants received training in how to use SMS and Internet- All costs associated with replying to intervention messages were reimbursedSelf-reported composite health behavior score (adherent if ≥ 3 on 4-point scale):- IG: 33% at baseline to 59% at 3 months to 53% at 6 months- CG: 27% at baseline to 37% at 3 months to 39% at 6 months- Significant treatment effect at 3 months but not at 6 months
Fukuoka et al., 2015 21 - Combined telemedicine with in-person sessions- 2-week run-in period- App included daily remainders to track and self-monitor health related information- App delivered interactive intervention content through messages, videos and quizzes (responding to daily message took 1 − 2 min)- Diet and PA assessments at baseline and during run-in period were used to personalize the app by providing individualized goals- If necessary, phone was provided- In person attendance = 85%- Pedometer use = 91.2%- App weight diary = 65.3%- App steps diary = 62%- App calorie diary = 46.9%- App message response = 54.3%
Höchsmann et al., 2019 22 - Garden setting: gardening is among the target group's preferred forms of PA and additionally functions as a metaphor for restoring the players body- Variety: 130 variations of strength, endurance, balance and flexibility exercises -Phone sensors (camera, accelerometer and gyroscope) were used to track PA and execution- Individualization: exercise tests (e.g. 1-min Sit-to-Stand Test; 6-Minute Walk Test) assessed fitness level at baseline and periodically during play. Results were used for algorithm to select appropriate intensity progressionApp usage data:- Average steps per day = 6559- Average steps per day in-game = 1893- Average in-game walking trainings per week = 4.9- In-game walking at a cadence of ≥100 steps/min (moderate-to-vigorous-intensity) = 83.7%- Average strength workouts per week = 6.8- Average minutes of training per week = 143.1
Kaur et al., 2015 23 - Combined telemedicine with in-person sessions- Consultations allowed physician to give advice based on most recent glycemic and clinical dataattendance of OPD visits:- Rare: 95%- Moderate: 87.5%- Frequent: 82.5% Self-monitoring:- Rare: 41.2%- Moderate: 58.3%- Frequent: 90.5% Exercise:- Rare: 10%- Moderate: 20%- Frequent: 55% Diet:- Rare: 20%- Moderate: 35%- Frequent: 60% Scheduled calls received: 75.6%
Kooiman et al., 2018 24 - Incremental activity goals based on individual baseline measurements- To counteract the frequent barrier lack of time’ strategies were provided to integrate PA into the workday- Participants could contact diabetes nurse via the online platform in case of questions- Weekly information about PA, diet (including sample recipes appropriate for diabetes) and training videos were provided- Participants received tailored feedback messages once per week based on steps takenWore activity-tracker > 75% of intervention days and read > 50% of programme content = 82.5%
Lunde et al., 2020 25 - Specific goal setting in application- App provides tasks for goal achievement, the participant choosing on how many reminders per week to receive for each task- Baseline goals developed together with supervisor at baseline based on motivational interview- Participants could contact supervisor at any time, receiving an answer within 2 days- Supervisor monitored goal-achievement and gave feedback, frequency reducing over timeUse of application as defined by amount of answered tasks:- All tasks = 71%- >80% of tasks = 84%- >50% of tasks = 91%
Maddison et al., 2015 26 - Whenever necessary, technical support was provided- Using the intervention to its full extent (maximum dose  =  reading all text messages and using the website) took participants approximately 10 min per week- Frequency of messages was reduced over time- Individualized exercise was provided based on personal preference and baseline fitness level- Exercise duration as well as intensity increased gradually according to symptoms and clinical status- The website included PA monitoring, goal setting and motivational videos- Motivational videos were short (30–120 s) and included other cardiac patients discussing their experiences with exercise- Different role models were used to cover various age and ethnic groupsUsage statistics:- Read some or all text messages = 82%- Viewed some or all videos on website = 57%- On average participants visited website once every 2 weeks
Widmer et al., 2017 27 - Intervention worked both via smartphone and online platform, in case no smartphone was available or participant preferred use of browser- Education on use of online and smartphone intervention components was provided within one week after enrollment- Technical support was provided at all times and inquiries usually answered within 24hCR adherence:- IG: 28 sessions- CG: 25.5 sessions Downloaded app: 19%

CG, control group; CR, cardiac rehabilitation; CPET, cardiopulmonary exercise test; HRR, heart rate reserve; IG, intervention group; OPD, outpatient department; PA, physical activity.

Design features and adherence of telemedical intervention studies. CG, control group; CR, cardiac rehabilitation; CPET, cardiopulmonary exercise test; HRR, heart rate reserve; IG, intervention group; OPD, outpatient department; PA, physical activity. Regarding the impact of adherence on primary study outcomes, out of the 10 positive studies, 7 (70%) was graded as good, 2 (20%) as medium and 1 (10%) as bad with respect to the reported adherence levels. For the four neutral studies, one was graded as good (25%), two as medium (50%) and one as bad (25%). How the various design features impacted the reported adherence levels can be seen in Table 5. Based on the extracted data, we divided design features into three categories:
Table 5.

Design features and adherence of telemedical intervention studies – data synthesis. Two-tailed Fisher exact probability test was utilized to calculate p-values for the observed risk ratios (RR).

Positive primary outcomeGood adherenceAdherence (Primary Outcome + or ∼)
goodmediumbad
Social
Personal contact
 Yes[14,15,17,18,21,23]6/6 (100%)5/6 (83.3%)5 ( + + + + + )1 ( + )
 No[16,19,20,22,2427]4/8 (50%)3/8 (37.5%)3 ( + +∼)3 ( + ∼∼)2 ( + ∼)
 RR (p-value)0.50 (0.08)0.45 (0.14)
Group Setting
 Yes[15,19]2/2 (100%)1/2 (50%)1 ( + )1 ( + )
 No[14,1618,2027]8/12 (66.7%)7/12 (58.3%)7 ( + + + + + +∼)4 ( + +∼∼)1 (∼)
 RR (p-value)0.67 (0.56)1.17 (1.00)
Adaptedness
 Yes[16,20,22,26]2/4 (50%)2/4 (50%)2 ( + ∼)1 ( + )1 (∼)
 No[14,15,1719,21,2325,27]8/10 (80%)6/10 (60%)6 ( + + + + + +)3 ( + ∼∼)1 ( + )
 RR (p-value)1.60 (0.52)1.20 (1.00)
Feedback
 Yes[14,18,19,24,25]4/5 (80%)3/5 (60%)3 ( + + + )1 (∼)1 ( + )
 No[1517,2023,25,27]6/9 (66.7%)5/9 (55.6%)5 ( + + + +∼)3 ( + +∼)1 (∼)
 RR (p-value)0.83 (1.00)0.93 (1.00)
Goal Setting
 Yes[19,21,2426]3/5 (60%)1/5 (20%)1 ( + )2 ( + ∼)2 ( + ∼)
 No[1418,20,22,23,27]7/9 (77.8%)7/9 (77.8%)7 ( + + + + + +∼)2 ( + ∼)
 RR (p-value)1.30 (0.58)3.89 (0.09)
Exercise related
Individual training progression
 Yes[14,16,18,19,22,24,26]4/7 (57.1%)4/7 (57.1%)4 ( + + + ∼)1 (∼)2 ( + ∼)
 No[15,17,20,21,23,25,27]6/7 (85.7%)4/7 (57.1%)4 ( + + + +)3 ( + +∼)
 RR (p-value)1.50 (0.56)1.00 (1.00)
Exercise equipment provided
 Yes[16,18]1/2 (50%)2/2 (100%)2 ( + ∼)
 No[14,15,17,1925,27]9/12 (75%)6/12 (50%)6 ( + + + + + +)4 ( + +∼∼)2 ( + ∼)
 RR (p-value)1.50 (1.00)0.50 (0.47)
Gamification
 Yes 22 1/1 (100%)1/1 (100%)1 ( + )
 No[1421,2327]9/13 (69.2)7/13 (53.8%)7 ( + + + + + +∼)4 ( + +∼∼)2 ( + ∼)
 RR (p-value)0.69 (1.00)0.54 (1.00)
Feedback on Exercise Execution
 Yes[19,22]2/2 (100%)1/2 (50%)1 ( + )1 ( + )
 No[1418,20,21,2327]8/12 (66.7%)7/12 (58.3)7 ( + + + + + +∼)4 ( + +∼∼)1 (∼)
 RR (p-value)0.67 (0.56)1.17 (1.00)
Barrier removal
Run-in-Period
 Yes[1416,19,21,27]4/6 (66.7%)3/6 (50%)3 ( + +∼)2 ( + ∼)1 ( + )
 No[17,18,20,2226]6/8 (75%)5/8 (62.5%)5 ( + + + + + )2 ( + ∼)1 (∼)
 RR (p-value)1.13 (1.00)1.25 (1.00)
Use of videos
 Yes[16,18,20,21,24,26]3/6 (50%)2/6 (33.3%)2 ( + ∼)3 ( + +∼)1 (∼)
 No[14,15,17,19,22,23,25,27]7/8 (87.5%)6/8 (75%)6 ( + + + + + +)1 (∼)1 ( + )
 RR (p-value)1.75 (0.24)2.25 (0.28)
Simplicity
 Yes[20,26]1/2 (50%)0/2 (0%)1 ( + )1 (∼)
 No[1419,2125,27]9/12 (75%)8/12 (66.7%)8 ( + + + + + + + ∼)3 ( + ∼∼)1 ( + )
 RR (p-value)1.50 (1.00)∞ (0.16)
Variety
 Yes[19,22]2/2 (100%)1/2 (50%)1 ( + )1 ( + )
 No[1418,20,21,23,24,26,27]8/12 (66.7%)7/12 (58.3)7 ( + + + + + +∼)4 ( + +∼∼)1 (∼)
 RR (p-value)0.67 (0.56)1.17 (1.00)
a. social, b. exercise related and c. barrier removal. Design features and adherence of telemedical intervention studies – data synthesis. Two-tailed Fisher exact probability test was utilized to calculate p-values for the observed risk ratios (RR). The most pronounced positive association between identified design features and adherence could be observed if the intervention included ‘personal contact’ with the study personnel (i.e. not only communicating via text but also implementing in-person sessions, phone calls, video chats, etc.). Studies that included personal contact reported good adherence levels in five out of six cases (83.3%) compared to three out of eight (37.5%) in the studies which did not include this design feature. Receiving regular feedback on the individual progress during the intervention was associated with good adherence levels in three out of five (60%) studies compared to five out of nine (55.6%) in the remaining studies. On the other hand, allowing participants to set their own goals and thereby independently track intervention progress seemed to be inferior in improving adherence [one out of five (20%) compared to seven out of nine (77.8%) studies]. Regarding the design of the exercise part of the intervention, one feature was providing participants with individual progression instead of using a ‘one size fits all’ approach. This feature took into account the results of baseline assessments (e.g. CPET, 6MWT) to plan incremental exercise adjustments over the course of the intervention. However, this was not associated with an increased adherence. Regardless of whether this design feature was applied, adherence levels were graded as good in four out of seven studies (57.1%). Offering alternatives to standard cardio exercises or strength training with the own body weight via the provision of exercise equipment was a further design feature implemented in two of the included studies. In both studies, adherence levels were graded as good. Another design feature related to the form in which the exercise intervention was delivered. Here, a newly developed smartphone game was implemented in one study, resulting in good levels of patient adherence. Additionally, utilizing phone sensors allowed to deliver specific feedback on exercise execution – a feature that one more study implemented via external hardware. However, in the latter case, the adherence levels were subpar. Removing common barriers towards adherence was another group of identified design features. The implementation of a run-in period was associated with good adherence levels in three out of six studies (50%). Yet, these levels were inferior when compared to studies that did not include this design feature, where we graded five out of eight (62.5%) with good adherence levels. Another identified design feature was the use of educational videos to deliver intervention content, this, however, yielded positive adherence levels in only two out of six (33.3%), compared to six out of eight (75%) studies that did not implement this feature.

Patient and study design characteristics and association with adherence and primary outcomes

Participants weighted mean age and body mass index (BMI) were 59.9 years (range of mean age 46.3–70.5) and 29.8 kg/m2 (range of mean BMI 27.5–35.1), respectively. Given the limited sample of 14 studies, testing for association or correlation between the dependent categorical variables adherence and study outcome as well as the independent continuous variables sample size, age and BMI would not yield meaningful results. However, dividing the included studies into two groups according to sample size, age and BMI allows for a qualitative description of how the adherence levels and primary outcomes are distributed (Table 6). Additionally, we also divided the included studies into two groups regarding their duration, CVD status of the included patients as well as their biological sex distribution.
Table 6.

Patient and design characteristics of included studies and association with adherence and primary outcomes.

GroupPositive primary outcomeGood adherenceAdherence
GoodMediumBad
n < 904/7 (57.1%)4/7 (57.1%)43
n ≥ 906/7 (85.7%)4/7 (57.1%)412
Age < 606/7 (85.7%)4/7 (57.1%)43
Age > 604/7 (57.1%)4/7 (57.1%)412
BMI ≤ 295/7 (71.4%)5/7 (71.4%)52
BMI > 295/7 (71.4%)3/7 (42.9%)34
Duration ≤ 3 months4/7(57.1%)5/7 (71.4%)52
Duration > 3 months6/7 (85.7%)3/7 (42.9%)322
At risk of CVD4/6 (66.7%)4/6 (66.7%)42
Manifest CVD6/8 (75%)4/8 (50%)422
Not predominantly (<66%) male3/5 (60%)3/5 (60%)32
Predominantly (>66%) male7/9 (77.8%)5/9 (55.6%)522

BMI, body mass index; CVD, cardiovascular disease.

Patient and design characteristics of included studies and association with adherence and primary outcomes. BMI, body mass index; CVD, cardiovascular disease. Adherence was graded as good in five out of seven studies (71.4%) in the lower compared to three out of seven (42.9%) in the higher BMI group. Regarding primary outcomes, significant effects were reported by four out of seven studies (57.1%) in the lower compared to six out of seven (85.7%) in the higher sample size group, the contrary being true for age (85.7% lower compared to 57.1% higher). For BMI, no differences could be observed regarding the significance of primary outcomes. Looking at duration, adherence levels were higher in shorter interventions (71.4% compared to 42.9%), while primary outcomes were positive in 85.7% of the longer compared to 42.9% in the shorter studies. Similarly, while primary outcomes were positive in six out of eight (75%) studies with CVD patients compared to four out of six (66.7%) studies with at-risk patients, adherence was higher in the at-risk (66.7%) than in the CVD (50%) patient studies. Regarding the biological sex distribution, both primary outcomes and adherence were positive (respectively rated as good) in 60% of the not predominantly male studies. For the predominantly male group, positive outcomes were reported by seven out of nine (77.8%) and adherence was graded as good in five out of nine (55.6%) studies.

Discussion

Telemedicine is likely to play a key role in future healthcare provision, allowing an avenue to supervise patients at home and thereby not only reducing the burden on study participants but also physicians and scientists. Numerous studies show that telemedicine can significantly reduce both the cost related to upholding an intervention programme, as well as reducing the travel time which is often linked to the participation in intervention studies and especially important for rural patient populations.[33,34] Yet, little is known about the design features which determine the success of telemedical interventions in preventive cardiology. To our knowledge, this is the first systematic review to analyse the associations between design features and the adherence as well as outcomes of telemedical healthy lifestyle interventions. Of the 14 included studies, 8 (57.1%) reported adherence levels which we graded as good. A positive primary outcome occurred in 10 (71.4%) studies. This shows that telemedical interventions can improve outcomes and should be considered as an alternative to supervised exercise programs in patients with or at risk of CVD, even though reimbursement is still an issue in many countries. Looking at the identified design features, it is noticeable that there is a deviation when it comes to the impact they had on adherence when compared to the main study outcomes, as highlighted in Table 5. The RR reveal noticeable divergences – and while none of them were statistically significant (due to the limited study sample), they allow for an explorative comparison of the individual design features. While for features such as personal contact (RR – primary outcome = 0.50, RR – adherence = 0.45) and regular feedback (RR – primary outcome = 0.83, RR – adherence = 0.93) a reduction in risk for a non-significant primary outcome and bad adherence can be observed, other features such as group setting (RR – primary outcome = 0.67, RR– adherence = 1.17) and individual training progression (RR – primary outcome = 1.50, RR – adherence = 1.00) show more inconsistent effects. This could indicate four things: One apparent limitation of commonly used adherence measures in telemedical interventions is that they are often self-reported by study participants.[16,18,20,23] Additionally, most objective measures only evaluate patient attendance without including any measures of intensity or effort.[15-21,23-27] In most cases they capture if interventions are being used, but not to which extent regarding duration, intensity, total energy expenditure or other variables. Additionally, there are studies where adherence measures did not seem to be pre-specified and sometimes arbitrarily chosen. Consequently, there is an apparent need to establish gold-standard adherence measures which are objectively collected and consider variables such as duration and intensity towards the exercise part of the intervention. The intervention had no effect on the primary outcome or the latter was not suitable to detect relevant changes over the observed time period. There were methodological limitations that prevented the detection of statistically significant differences between groups (e.g. the study was underpowered). Adherence did not play a major role in the effect of the intervention, that is, even a low adherence rate was sufficient to induce significant effects. Some adherence measures were chosen in a way, that they did not sufficiently capture the relevant dimensions of participating in the interventions. Looking at the reported adherence levels of the included studies, especially when taking into account the corresponding primary intervention outcomes, the variable of personal contact emerged as the most important design feature in telemedical interventions. While telemedicine can be very practical and facilitate many processes, especially elderly patients, who are often not too familiar with the use of novel technologies, can perceive telemedical intervention concepts as impersonal. This is especially relevant for this review, seeing that the mean age of all participants was 59.9 years. In this age group, having recurring interactions with a dedicated contact person can counteract this problem, serving as a reminder for the patient that even while at home, their progress is continuously monitored. Finding the balance between a simple yet engaging intervention design presents an important challenge. When presented with a choice, patients often prefer receiving intervention content by simple means such as text messages rather than having to learn how to utilize an online platform or a smartphone application.[26,27] Similarly, interventions becoming almost ‘overloaded’ with content and thereby too time consuming could explain, why the design features goal setting (RR – primary outcome = 1.30, RR – adherence = 3.89) and the inclusion of videos (RR – primary outcome = 1.75, RR – adherence = 2.25) seemed to increase the risk for bad adherence levels as well as non-significant study outcomes. When designing a telemedical intervention for a specific population, the technology-focused form of delivery needs to be balanced by trying to keep the contents as age-adapted and appropriate as possible. Currently, a lot remains to be learned on this topic, which is reflected in the results regarding the identified design feature of ‘adaptedness’. Here, studies not including any aspects of adaptedness performed better regarding both primary outcomes and patient adherence, when compared to the studies that did include this feature. Yet, as only four studies even took into consideration any dimension linked to adaptedness (e.g. using age-appropriate actors for educational videos) these results have to be interpreted with caution, given the apparent amount of unexploited potential. Conversely, we argue that there remains huge potential to improve adaptedness in future telemedical intervention studies. For example, one study chose a gardening setting for their smartphone game to improve PA since this activity is among the favorite in the participating age group – and this choice proved to be very successful. Another design feature that requires further attention is the application of a run-in period into the study. Even though we expected this feature to increase adherence levels, this was not confirmed in the present review (RR = 1.25). Again, this might be linked to the limited number of included studies – yet, we argue that similar to designing a successful intervention study, designing an adequate run-in-period requires additional attention and might be more complex than a simple one-off explanation of a technical device to a patient. Therefore, the mixed results in adherence levels of studies that applied this design feature might be explained by the actual procedures of the deployed run-in-periods. Consequently, how to design run-in-periods could be an interesting topic of future research. Looking at the patient and design characteristics of the included studies, there emerge some notable points. Only one study on patients with manifest CVD did not have a majority of male participants, while in at-risk patient populations most studies were balanced regarding the biological sex distribution with only one study being 100% male and another being predominantly female. This further highlights the problem of the generalizability of findings in CVD studies, where included patients are often predominantly male. According to a recent review, adherence to lifestyle intervention trials is higher in males compared to females. In contrast, in the present review, adherence levels were similar between studies with and without predominantly male patient populations. For the primary outcomes, on the other hand, 77.8% of the predominantly male studies reported a positive primary outcome compared to only 60% in the no-predominantly male group. Given the limited number of female participants and the lack of direct comparisons in the included studies, it is not possible to draw meaningful conclusions regarding differences in primary outcomes between men and women. However, it emphasizes the need to consider and evaluate potential gender differences in telemedical interventions. Regardless of sex, long-term adherence is a typical problem in lifestyle intervention trials, which requires further investigations to induce sustainable changes in clinical outcomes. Interestingly, even though adherence does seem to decrease with increased length of the intervention, primary outcomes were positive more often in longer duration studies. This underlines that for many study endpoints (e.g. weight reduction) there is minimum amount of time needed to achieve significant results. On the other hand, the deviation between adherence and outcomes emphasizes the question of whether currently utilized and reported adherence measures are suitable to capture the extent to which interventions are being used. Lastly, due to the exploratory nature and the non-significant findings of this review, no clear conclusion can be drawn about which aspects of a study design are most important. However, we would strongly recommend the implementation of personal contact in every telemedical intervention. Other design features should be considered based on the population and aims of the trial.

Limitations

Due to the limited number of published studies and the large variety of reported adherence measures, it was not feasible to perform a meta-analysis. Therefore, the results should be interpreted as exploratory findings. Also, it was not possible to clearly rank the importance of the identified design features as most studies included more than one feature. It is unclear whether design features really had a causal impact on adherence or outcomes. It is possible that some design features were coincidentally associated with adherence and outcomes or that other factors played a more important role (e.g. methodological limitations). Moreover, this review did not include any studies which focused primarily on the topics of alcohol and smoking cessation, since those face their own and very specific challenges when it comes to patient adherence. Yet, since alcohol consumption and smoking represent two key factors in healthy lifestyle research, results regarding these variables would be of great interest for future research. Additionally, while all included studies pertain to the field of preventive cardiology, differences between the reported primary outcomes and observed patient populations add additional difficulty in analysing intervention adherence levels. Same is true regarding the heterogeneity of the included telemedical interventions. Furthermore, given the qualitative approach as well as the exploratory nature of this review, the limited number of included studies and the non-significant results, all of the reported results have to be treated with a certain degree of caution. Lastly, while we excluded studies if part of the intervention occurred after the beginning of the COVID-19 pandemic to eliminate the influence that it ensued on the adherence of ongoing interventions, the pandemic has also caused a dramatic increase in the use of telemedicine, which led to severe changes in CVD patient care. Analysing how the pandemic impacted the development and implementation of telemedicine in patient care might be of great interest for future research.

Conclusion

Even (or especially) in telemedicine, recurring personal contact seems to be a key factor in ensuring that adherence levels remain comparable to those seen in centre-based interventions. Yet, much remains to be done in the area of adaptedness (i.e. adapting the intervention for a particular target population), where an optimal intervention design for a CVD patient population remains to be established. Lastly, gold standard adherence measures in telemedical interventions need to be established, so that study outcomes are more comparable and meta-analyses can be conducted.
  32 in total

Review 1.  Importance of characteristics and modalities of physical activity and exercise in the management of cardiovascular health in individuals with cardiovascular risk factors: recommendations from the EACPR. Part II.

Authors:  L Vanhees; N Geladas; D Hansen; E Kouidi; J Niebauer; Z Reiner; V Cornelissen; S Adamopoulos; E Prescott; M Börjesson; B Bjarnason-Wehrens; H H Björnstad; A Cohen-Solal; V Conraads; D Corrado; J De Sutter; P Doherty; F Doyle; D Dugmore; Ø Ellingsen; R Fagard; F Giada; S Gielen; A Hager; M Halle; H Heidbüchel; A Jegier; S Mazic; H McGee; K P Mellwig; M Mendes; A Mezzani; N Pattyn; A Pelliccia; M Piepoli; B Rauch; A Schmidt-Trucksäss; T Takken; F van Buuren; D Vanuzzo
Journal:  Eur J Prev Cardiol       Date:  2012-10       Impact factor: 7.804

2.  RoB 2: a revised tool for assessing risk of bias in randomised trials.

Authors:  Jonathan A C Sterne; Jelena Savović; Matthew J Page; Roy G Elbers; Natalie S Blencowe; Isabelle Boutron; Christopher J Cates; Hung-Yuan Cheng; Mark S Corbett; Sandra M Eldridge; Jonathan R Emberson; Miguel A Hernán; Sally Hopewell; Asbjørn Hróbjartsson; Daniela R Junqueira; Peter Jüni; Jamie J Kirkham; Toby Lasserson; Tianjing Li; Alexandra McAleenan; Barnaby C Reeves; Sasha Shepperd; Ian Shrier; Lesley A Stewart; Kate Tilling; Ian R White; Penny F Whiting; Julian P T Higgins
Journal:  BMJ       Date:  2019-08-28

3.  Patient acceptance of a telemedicine service for rehabilitation care: A focus group study.

Authors:  Stephanie Jansen-Kosterink; Marit Dekker-van Weering; Lex van Velsen
Journal:  Int J Med Inform       Date:  2019-01-23       Impact factor: 4.046

Review 4.  Adherence to exercise programs for older people is influenced by program characteristics and personal factors: a systematic review.

Authors:  Alexandra Miranda Assumpção Picorelli; Leani Souza Máximo Pereira; Daniele Sirineu Pereira; Diogo Felício; Catherine Sherrington
Journal:  J Physiother       Date:  2014-08-03       Impact factor: 7.000

5.  VA Telemedicine: An Analysis of Cost and Time Savings.

Authors:  Jack E Russo; Ryan R McCool; Louise Davies
Journal:  Telemed J E Health       Date:  2015-08-25       Impact factor: 3.536

6.  Self-tracking of Physical Activity in People With Type 2 Diabetes: A Randomized Controlled Trial.

Authors:  Thea J M Kooiman; Martijn de Groot; Klaas Hoogenberg; Wim P Krijnen; Cees P van der Schans; Adriaan Kooy
Journal:  Comput Inform Nurs       Date:  2018-07       Impact factor: 1.985

7.  Effectiveness of a Behavior Change Technique-Based Smartphone Game to Improve Intrinsic Motivation and Physical Activity Adherence in Patients With Type 2 Diabetes: Randomized Controlled Trial.

Authors:  Christoph Höchsmann; Denis Infanger; Christopher Klenk; Karsten Königstein; Steffen P Walz; Arno Schmidt-Trucksäss
Journal:  JMIR Serious Games       Date:  2019-02-13       Impact factor: 4.143

8.  Long-term follow-up with a smartphone application improves exercise capacity post cardiac rehabilitation: A randomized controlled trial.

Authors:  Pernille Lunde; Asta Bye; Astrid Bergland; Jostein Grimsmo; Even Jarstad; Birgitta Blakstad Nilsson
Journal:  Eur J Prev Cardiol       Date:  2020-02-28       Impact factor: 7.804

9.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.

Authors:  David Moher; Alessandro Liberati; Jennifer Tetzlaff; Douglas G Altman
Journal:  PLoS Med       Date:  2009-07-21       Impact factor: 11.069

10.  Text Message and Internet Support for Coronary Heart Disease Self-Management: Results From the Text4Heart Randomized Controlled Trial.

Authors:  Leila Pfaeffli Dale; Robyn Whittaker; Yannan Jiang; Ralph Stewart; Anna Rolleston; Ralph Maddison
Journal:  J Med Internet Res       Date:  2015-10-21       Impact factor: 5.428

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.