A systematic review of measures of self-reported adherence to unsupervised home-based rehabilitation exercise programmes, and their psychometric properties.

BACKGROUND: Adherence is an important factor contributing to the effectiveness of exercise-based rehabilitation. However, there appears to be a lack of reliable, validated measures to assess self-reported adherence to prescribed but unsupervised home-based rehabilitation exercises.
OBJECTIVES: A systematic review was conducted to establish what measures were available and to evaluate their psychometric properties. DATA SOURCES: MEDLINE, EMBASE, PsycINFO CINAHL (June 2013) and the Cochrane library were searched (September 2013). Reference lists from articles meeting the inclusion criteria were checked to ensure all relevant papers were included. STUDY SELECTION: To be included articles had to be available in English; use a self-report measure of adherence in relation to a prescribed but unsupervised home-based exercise or physical rehabilitation programme; involve participants over the age of 18. All health conditions and clinical populations were included. DATA EXTRACTION: Descriptive data reported were collated on a data extraction sheet. The measures were evaluated in terms of eight psychometric quality criteria.
RESULTS: 58 studies were included, reporting 61 different measures including 29 questionnaires, 29 logs, two visual analogue scales and one tally counter. Only two measures scored positively for one psychometric property (content validity). The majority of measures had no reported validity or reliability testing.
CONCLUSIONS: The results expose a gap in the literature for well-developed measures that capture self-reported adherence to prescribed but unsupervised home-based rehabilitation exercises. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This study highlights the paucity of reported, validated and reliable self-report measures for unsupervised, exercise-based rehabilitation adherence.

Despite the number and breadth of measures reported, this study reveals only two measures which conclusively possessed any psychometric property.

The study also establishes that the vast majority of measures highlighted in this review had not reportedly undergone any psychometric testing of reliability and validity. However, this does not necessarily mean testing was not conducted.

The lack of reporting regarding tests conducted on a measure does not assume that all measures have poor psychometric properties.

Introduction

Exercise-based rehabilitation improves fitness and functional ability for people with long-term conditions.1 These outcomes are hugely important because they make a substantial difference to people's lives and to the economy. However, prescribed exercise programmes often comprise a part of home-based rehabilitation or self-management for long-term conditions and are typically unsupervised by health professionals. Therefore, it is unclear if any exercise occurs, if people have engaged in enough exercise to obtain the therapeutic benefit or if they are sustaining their exercise levels for long enough to self-manage their condition.2 Finding a way to know what patients are doing and how much they are doing is consequently important and one method that has been used is self-report. This systematic review therefore set out to identify what self-report measures have been used for assessing adherence to home-based unsupervised exercises, as this focused review has not been conducted before.

Self-report measures can overestimate as well as underestimate how much people actually do.3 Individuals’ attitudes and beliefs, coupled with the beliefs of people they interact with, influence intention to exercise,4 as well as actual levels of exercise adherence. Replies to questions asked about adherence may reflect what the person feels is the desired response rather than a true appraisal of their behaviour, giving a falsely positive estimate of adherence.5 6 This may be one reason why unsupervised home-based exercise programmes are deemed ineffective, when in reality ‘an insufficient regimen effect’ has occurred.7

For the purposes of this review adherence is defined as the degree behaviour corresponds with an agreed on recommendation. It is a complex and multidimensional construct that can be affected by a number of factors related to the condition, the person (such as forgetfulness, self-efficacy, attitudes, mood states such as depression and socioeconomic status) and the relationship between the person and healthcare professional.8

While there are self-report questionnaires that have been developed and validated for medication-based adherence,9–11 there appears to be a paucity of psychometrically sound self-report measures for recording adherence in the specific context of prescribed but unsupervised home-based rehabilitation exercises for people with long-term physical conditions. Thus the aims of this systematic review were to: identify self-report measures of adherence that have been used in this context and to critically evaluate the psychometric properties of these measures.

Methods

Selection criteria

The inclusion criteria were kept broad to ensure all studies pertaining to measuring exercise adherence were identified. However, articles had to:

Include participants aged 18 and over;

Use a self-report measure of exercise adherence;

Indicate that the exercise was in relation to an unsupervised home-based exercise programme that was prescribed as part of a rehabilitation programme for someone with a long-term physical condition;

Be available in English.

There were no restrictions on included health conditions or adult subpopulations or study design. Modified versions of measures were included as were papers reporting separate psychometric evaluations of a measure already identified. Where a study used a measure that had previously been reported, only the original citation was included. No limit was made on the type of measure. Studies that used session attendance as a measure of adherence or clinician-reported adherence were excluded as were papers published only as abstracts.

Information sources

Papers were identified from: MEDLINE (1946 onwards); EMBASE (1980 onwards) and PsycINFO (1806 onwards) in the Ovid platform and CINAHL (1981 onwards) in the National Health Service (UK) platform. These searches were originally performed on 19 January 2012 and updated on 27 June 2013. The Cochrane database was searched on 7 February 2013 and updated on 9 September 2013. Studies were limited to those that were published in English involving humans over the age of 18. Manual searching of included studies was also undertaken.

Search strategy

The search strategy included combinations of keywords and MESH terms which were exploded. Truncations of words were used and search terms were prefixed with ‘ti, ab’ to ensure the results would contain these words in the abstract. The strategy was modified for CINAHL and the Cochrane database due to different search platforms and MESH terms. Online supplementary appendix 1 illustrates the detailed search strategy.

Study selection

Titles and abstracts were independently screened for eligibility by two reviewers (JB and VG). Eligible papers were gathered in full text and independently screened by the same reviewers. A third reviewer (SD) facilitated decision-making when there were disagreements.

Data extraction

A data extraction sheet designed by TH for assessing musculoskeletal rehabilitation measures was modified for this study. Data were extracted regarding: the name of the measure, how the measure was devised, a description of the measure, how the measure was scored, the purpose of the study and the number of participants and the population in which the measure was being used. If the information was not evidenced in the papers ‘N/R’ was used to illustrate that the information was not reported. The quality of the measures was assessed using the Quality Criteria developed by Terwee et al.12 Each psychometric property was rated either positive, intermediate, negative or zero (table 1). Data were extracted by one reviewer (JB) and checked by a second (VG).

Table 1

Psychometric properties and quality assessment15 of the measures reported by the included studies

Quality rating	Content validity	Internal consistency	Criterion validity	Construct validity	Agreement	Reliability	Responsiveness	Floor and ceiling effects	Interpretability
Positive*	2	0	0	0	0	0	0	0	0
Intermediate†	6	5	3	18	0	3	0	1	17
Negative‡	9	0	0	3	0	0	0	1	§
Zero¶	44	56	58	40	61	58	61	59	44

*A positive rating for the adherence measure was obtained when tests for the property in question addressed all the criteria to a satisfactory extent.

†An intermediate rating for the adherence measure was obtained when some aspects of the criteria for a positive rating were completed, but not all, or there was doubt about the method or design used to test the psychometric property.

‡A negative rating for the adherence measure was obtained when the property being assessed proved to be non-existent or fell below specified thresholds despite the method and design used to test psychometric property being sufficient.

§No negative option for this construct.

¶A zero rating for the adherence measure was obtained when there was no information in the paper or no evidence that this psychometric property had been considered.

Results

The search identified 2264 citations (figure 1). Fifty-eight papers were included, reporting 58 studies and 61 measures of adherence, of which there were 29 questionnaires, 29 logs/dairies, two visual analogue scales (VAS) and one tally counter. Data from 7424 participants were included. Where reported, there were a total of 2093 men and 2911 women with a mean age of 55.7 years (SD=12.4 years). The study populations included those with cancer,13 14 musculoskeletal,15–37 cardiovascular,38–46 respiratory,47–49 neurological,50 genitourinary51–55 and endocrine conditions,56 in addition to war veterans,57 older people,58–61 those undergoing surgery,62–65 those receiving voice therapy66 and sedentary people.67–70 Online supplementary appendix 2 provides a detailed description of each included study.

Figure 1

Flow diagram outlining the process of selection of papers for the systematic review.

Table 1 provides an overview of each psychometric property and the quality criteria assessment for the included measures. Only two measures achieved a positive rating from the range of psychometric properties, and these were both for content validity.39 60 The Adherence to Exercise Scale for Older People (AESOP)60 was developed using two existing scales, the Self-Efficacy for Exercise and the Outcome Expectations for Exercise scales,71–73 as a basis for developing items that were subsequently evaluated with five older people, modified and re-evaluated with a further five older people. The Heart Failure Compliance Questionnaire39 used qualitative interviews with three patients with heart failure to develop questionnaire items that were tested with six specialist nurses, a sociologist and 10 people with heart failure.

Most measures had no evidence that they had undergone any sort of psychometric evaluation although a small number of researchers had attempted to evaluate some measurement properties but used dubious methods or the property being assessed fell below suitable quality thresholds as determined by Terwee et al.12 In addition, some authors referenced that their measure had established psychometric properties but then modified the scale or used it with a completely different population without re-examining the properties in the revised scale. No studies assessed agreement or responsiveness. Online supplementary appendix 3 provides a detailed account of each measure in terms of psychometric properties and our quality rating.

Discussion

Principle findings

This is the first systematic review to identify and evaluate measures of self-reported adherence to prescribed, unsupervised home-based rehabilitation exercises for a range of health conditions and populations. We found 58 studies reporting on 61 measures and many of the measures shared similarities but almost all lacked any psychometric validation. This is an absurd and messy situation for appraising the benefits of unsupervised home-based exercise rehabilitation.

A few measures had undergone some assessment of measurement properties but these were not considered to meet the quality criteria set by Terwee et al.12 For example, one study19 reported a Pearson correlation coefficient to determine reliability but this is deemed unacceptable due to systematic differences not being accounted for.12 Two measures39 60 were found to have content validity. This is a relatively straightforward property to establish so it is somewhat surprising that more measures did not rate positively for this. Terwee et al12 state that a measure should only be used if content validity is satisfactory. If content validity is not considered in the measures’ construction, it will not be known if the questions are relevant and comprehensive for the target population. Content validity also impacts on floor and ceiling effects and despite the AESOP questionnaire having content validity it was found to have a very strong ceiling effect.

In addition some authors appeared to assume that a measure can be modified and any psychometric properties from the original measure would still stand; however changing a measure may completely undermine any prior assumptions about its validity. This disparity was found when evaluating the internal consistency of one measure,22 23 which was then modified in another study by adding two questions.16 This resulted in Cronbach's α of 0.93 (original measure) and only 0.63 in the modified measure.

Strengths and limitations

This review had clear inclusion and exclusion criteria and used a robust quality criteria tool to assess the reporting of psychometric properties of the measures. Although the quality criteria tool was designed for health status questionnaires and not specifically for adherence measures we believe it was the best tool available.

A limitation was that only papers available in English were included as there were no resources for a translation. This potential publication bias may impact on the generalisability of our review to non-English speaking countries. Another important aspect to note is that just because psychometric testing was not reported or was ambiguous this does not mean that it was not conducted or is not of a high quality. We could have overcome some aspects of this by contacting authors for any unpublished supporting data regarding their measure, if it was available. Although this may have aided our ability to judge the quality of the measure's properties, it would not guarantee that the properties were of a high standard.

Comparison with existing literature

Self-reported medication adherence is perhaps the most advanced in the field with questionnaires having been developed and validated although there remains no gold standard measure.9 A recent review of adherence measures for antihypertensive medication suggested 39% of measures indicated some level of reliability and validity, but 33% had undergone no psychometric testing.74

Alternative methods of assessing adherence to exercise-based rehabilitation do exist and include attendance at appointments,75 although this does not necessarily mean the individual is completing the activities they are meant to be doing. Alternatively adherence could be assessed by others; for example, the Sports Injury Rehabilitation Adherence Scale (SIRAS)76 comprises a therapist or trainer-rated observation of whether a patient has completed their exercises as instructed. Owing to the supervisory element of SIRAS, it is possible that the individual may no longer feel they have a choice to adhere; the constant supervision requires their compliance not their adherence. Conversely in-clinic observations need not be obvious and so could provide insight into an individual's level of motivation to adhere. Either way the in-clinic assessment does not necessarily reflect what happens in an unsupervised environment. In addition to observation by another, objective measurement methods can be used, such as accelerometers to record physical activity.77 However, these also have limitations for assessing adherence, especially longer term or with large clinical groups, as the devices are expensive and require the participant to adhere to wearing them. In addition the devices act as ‘supervisors’ which may result in a false view of adherence as the individual may no longer feel they have the autonomy to choose whether or not to adhere.11 Furthermore these devices do not easily capture the movements of therapeutic exercise. The rapid development of smart phone technology and applications may provide a future solution to this issue albeit still at some cost. At present it is clear that there is no cheap and easily available gold standard measurement of unsupervised exercise-based rehabilitation adherence and so, even with its inherent problems, self-report remains an important option.

Implications for practice and future research

There are a large number of measures that presume to record adherence to prescribed unsupervised home-based rehabilitation exercises but there is a shortage of measures that have been robustly validated. While clinicians generally believe they have some idea as to how adherent their patients are, it is unlikely that their clinical judgement is completely accurate particularly for the unobserved element of an exercise programme. Coupled with the lack of well-developed measures it becomes very difficult for clinicians to determine if an exercise regime being prescribed is ineffective, and the prescription needs adjusting, or if the individual is non-adherent and requires further support to facilitate uptake and maintenance of their exercise programme. A self-report measure that is able to identify patients who require this extra support will be clinically useful however this would also act as a potential confounding factor for measuring adherence (as it could act as a reminder and hence facilitate adherence).

This review has focused on the problems of self-reported exercise adherence measurement and the findings support the urgent need to develop valid and reliable measures that can be used for research purposes, at least in the first instance. It may be possible to develop such measures as suitable adherence assessment tools that will aid clinicians to support patients to undertake optimal exercise doses. Ultimately the best strategy is likely to be a combination of measures across the spectrum of objective, clinician assessed through to patient self-report.