Motor imagery ability assessments in four disciplines: protocol for a systematic review.

INTRODUCTION: Motor imagery (MI) is a very popular and well-accepted technique in different disciplines. Originating from sport and psychology, MI is now also used in the field of medicine and education. Several studies confirmed the benefits of MI to facilitate motor learning and skill acquisition. The findings indicated that individual's MI ability might influence the effectiveness of MI interventions. Over the last two centuries, researchers have developed several assessments to evaluate MI's abstract construct. However, no systematic reviews (SR) exist for MI ability evaluation methods and their measurement properties. METHODS AND ANALYSIS: The SR will evaluate available MI ability assessments and their psychometric properties in four relevant disciplines: sports, psychology, medicine and education. This involves performing searches in SPORTDiscus, PsycINFO, Cochrane Library, Scopus, Web of Science and ERIC. Working independently, two reviewers will screen articles for selection. Then all raw information will be compiled in an overview table-including the articles' characteristics (eg, a study's setting or the population demographics) and the MI ability assessment (psychometric properties). To evaluate the articles' methodological quality, we will use the COSMIN checklist. Then we will evaluate all the included assessments' quality and perform a best-evidence synthesis. Results of this review will be reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. ETHICS AND DISSEMINATION: The SR is based on published data, and ethical approval is not required. This review will provide information on assessment performance and equipment, as well as its main focus and usefulness. Furthermore, we will present the methodological quality of all the included articles and assess the included instruments' quality. Ultimately, this will act as a valuable resource, providing an overview of MI ability assessments for individual clinical settings, treatment aims, and various populations. The SR's final report will be published in a peer-reviewed journal and presented at relevant conferences. PROSPERO REGISTRATION NUMBER: CRD42017077004. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The systematic review will provide an overview of available assessments of Motor imagery (MI) ability in the following disciplines: sports, psychology, medicine and education.

The included articles’ methodological quality will be assessed and reported using the Consensus‐based Standards for the selection of health Measurement Instruments guidelines.

The selected MI ability assessments’ quality will be described.

The review will not be restricted to a specific population, age or disease.

The review will be restricted to literature published in English and German languages.

Introduction

Motor imagery (MI)—the mental representation of an action without engaging in its physical execution—is a widely used technique in many different disciplines (eg, sports, psychology, medicine and education).1 2 The technique of MI has been shown to substantially enhance motor rehabilitation in patients following: stroke,3 4 spinal cord injury,5 orthopaedic surgeries6–8 and sport injuries.9 10 Furthermore, MI is a very popular strategy to enhance psychomotor skills 11–13 or various aspects of performance among athletes14.

MI can be explicit or implicit. Explicit MI is the voluntary active imagination of a movement with conscious mental representation.15–17 Determining the laterality of a picture of a human hand refers to a mental rotation task. The mental rotation task requires individuals to mentally rotate an internal representation of their own body part into the presented part’s position. Unconscious imagining of movement during a mental rotation task is considered implicit MI. Imagined movements can be stimulated mentally, using either the kinaesthetic (sensation of movement) or the visual mode (visualisation of movement)18 and can be viewed from an internal or external perspective.19 20 By using the internal perspective, the movement is imagined from a first-person view, for example, as though viewing through one’s own eyes the body part move. Contrarily, when using the external perspective, it is a third-person view of oneself, similar to an observer watching a movement execution on television.19 20

Several neuroimaging studies have shown that the brain areas active during MI are similar to those active during the actual movement.21–23 Furthermore, recent findings showed that MI activates subcortical structures (ie, the excitability of presynaptic interneurons) without activating alpha-motoneurons.24 This led to the theory that MI facilitates motor activity and excite the movement execution, eventually improving motor function or performance, for example, movement accuracy, gait speed or strength.3 4 7 25 26

However, measuring MI ability is no simple feat because MI is actually a multidimensional construct27 with wide individual differences.28 29

Martin et al 30 showed that an individual’s MI ability can influence his/her effectiveness to achieve intended outcomes. Therefore, it is deemed essential to assess MI abilities prior to MI interventions.14

Over the last two centuries, various assessments were developed to evaluate an individual’s MI ability within different dimensions, for example, vividness or image clarity31–39 and controllability or the ease and accuracy with which an image can be manipulated mentally.40–43 Some assessments can evaluate both dimensions - vividness and controllability.44 However, all these assessments are often used as self-reported questionnaires for subjective MI ability assessments.2 45

Objective assessment methods, on the other hand, record central and peripheral nervous system activities during MI. These methods could be categorised as neurophysiological methods—for example, functional MRI (fMRI), positron emission tomography, electromyography, electro-oculography and electroencephalography (EEG).2

Although MI ability assessments are easy to use and cost-effective,45 they do not allow control of MI ability before or during a clinical experiment.2 Objective assessment methods are more powerful and versatile assessments of the MI duration and the temporal congruence between imagery and the time taken to execute the same movements45 but the assessments cannot evaluate MI qualities, such as the MI perspective, vividness, ease and so forth.2

Thus, no single assessment can determine an individual’s ability or disability to perform MI.46 So, for a comprehensive evaluation of MI ability, including different aspects of MI, a combination of MI ability assessments is recommended.27 45 46 This could include evaluating mental rotation, the temporal congruency of the same movement’s mental and practical performance, for example, the Chaotic Motor Imagery Assessment, and a standardised questionnaire such as the Kinaesthetic and Visual Imagery Questionnaire or Movement Imagery Questionnaire.47

However, the literature is still remiss in providing a systematic review of MI evaluation methods and their measurement properties.2 For example, Di Rienzo et al, McAvinue and Robertson, and Melogno-Klinkas et al mainly focused on evaluating the MI ability in the field of neurology for healthy individuals or athletes.2 17 48 Only two of those reviews reported the assessments’ psychometric properties.17 48 Furthermore, Melogno-Klinkas et al 48 only included assessments designed for a Spanish-speaking community in neurorehabilitation. McAvinue et al 17 summarised self-reported questionnaires for measuring explicit MI restricted to sport and exercise. The review by Di Rienzo et al 2 provided an overview on assessments used for MI ability evaluation but it was limited to the field of neurorehabilitation—for example, patients after stroke, with Parkinson’s disease or spinal cord injury and patients after an amputation in clinical settings.

In contrast, our proposed systematic review will enable clinicians, coaches, teachers and researchers to select a suitable MI ability assessment for their current settings and goals based on information provided regarding the assessment’s focus and quality. Essentially, the review will answer the question: What evaluated MI ability assessments are available in the fields of sports, psychology, medicine and education and what are their psychometric properties? Because of its enhanced utility, the review will provide an overview of the following:

Evaluated MI ability assessments, based on a systematic search-and-selection process, using either explicit or implicit MI in the fields of sports, psychology, medicine and education.

Current levels of evidence for psychometric properties of the selected MI ability assessments.

Necessary equipment and training for the included MI ability assessments.

Methods and analysis

Study design and registration

The study protocol was written and reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) guidelines and the PRISMA checklist.49 For this systematic review, we will follow the recommendations for systematic reviews on measurement properties.50

Selection criteria

Table 1 details the selection criteria applied during the systematic selection process, based on title, abstract and full text.

Table 1

Overview of inclusion criteria

Inclusion criteria
Construct	Motor imagery, mental imagery, mental rehearsal, movement imagery, mental practice, mental training, mental simulation and visualisation.
Field of interest	Sports, psychology, medicine and education.
Target population	Not limited to a specific population: ie, healthy individuals, adults, children and patients. No restriction on age, gender or health status.
Assessment instrument	All assessment methods, standardised assessments or questionnaires, rating tests, congruency tests and a mental chronometry test without limitations on a version or language.
Measurement properties	Reliability parameters: internal consistency, measurement errors, test-retest reliability, inter-rater and intrarater reliability. Validity parameters: content, construct and criterion validity. Responsiveness parameters: SE of measurement, minimal detectable change and standardised response mean.
Publication language	English and German.

Articles will be excluded if the authors only use neurophysiological methods to evaluate MI ability, for example, fMRI, EEG or brain-computer interface technology.

Search strategy

One author (ZS) and a life science librarian from a medical library will carry out the search strategy independently in the defined databases. The following electronic databases will be searched from their inception for relevant articles:

Sports—SPORTDiscus (1892 to current date of search);

Psychology—PsycINFO (1887 to current date of search);

Medicine—Cochrane Library (current issue), Scopus (1996 to current date of search) and Web of Science (1900 to current date of search); and

Education—ERIC (1966 to current date of search).

For each database, the search will include combined terms regarding the construct of interest and the assessment instrument. Furthermore, we will apply and adapt the search strategy proposed for each database (see table 2) to find articles using the Terwee et al.51 measurement properties.

Table 2

Search strategy

Search domain	Search words
1. construct	‘motor imagery’ OR ‘mental imagery’ OR ‘mental rehearsal’ OR ‘movement imagery’ OR ‘mental practice’ OR ‘mental training’
	AND
2. instrument	‘Instrument’ OR ‘measurement’ OR ‘questionnaire’ OR ‘scale’ OR ‘assessment’ OR ‘test’
	AND
3. filter for measurement properties by Terwee et al.51	(instrumentation(sh) OR methods(sh) OR Validation Studies(pt) OR Comparative Study(pt) OR ‘‘psychometrics’’(MeSH) OR psychometr*(tiab) OR clinimetr*(tw) OR clinometr*(tw) OR ‘‘outcome assessment (health care)’’(MeSH) OR outcome assessment(tiab) OR outcome measure*(tw) OR ‘‘observer variation’’(MeSH) OR observer variation(tiab) OR ‘‘Health Status Indicators’’(Mesh) OR ‘‘reproducibility of results’’(MeSH) OR reproducib*(tiab) OR ‘‘discriminant analysis’’(MeSH) OR reliab*(tiab) OR unreliab*(tiab) OR valid*(tiab) OR coefficient(tiab) OR homogeneity(tiab) OR homogeneous(tiab) OR ‘‘internal consistency’’(tiab) OR (cronbach*(tiab) AND (alpha(tiab) OR alphas(tiab))) OR (item(tiab) AND (correlation*(tiab) OR selection*(tiab) OR reduction*(tiab))) OR agreement(tiab) OR precision(tiab) OR imprecision(tiab) OR ‘‘precise values’’(tiab) OR test–retest(tiab) OR (test(tiab) AND retest(tiab)) OR (reliab*(tiab) AND (test(tiab) OR retest(tiab))) OR stability(tiab) OR interrater(tiab) OR inter-rater(tiab) OR intrarater(tiab) OR intra-rater(tiab) OR intertester(tiab) OR inter-tester(tiab) OR intratester(tiab) OR intra-tester(tiab) OR interobserver(tiab) OR inter-observer(tiab) OR intraobserver(tiab) OR intraobserver(tiab) OR intertechnician(tiab) OR inter-technician(tiab) OR intratechnician(tiab) OR intra-technician(tiab) OR interexaminer(tiab) OR inter-examiner(tiab) OR intraexaminer(tiab) OR intra-examiner(tiab) OR interassay(tiab) OR inter-assay(tiab) OR intraassay(tiab) OR intra-assay(tiab) OR interindividual(tiab) OR inter-individual(tiab) OR intraindividual(tiab) OR intra-individual(tiab) OR interparticipant (tiab) OR inter-participant(tiab) OR intraparticipant(tiab) OR intra-participant(tiab) OR kappa(tiab) OR kappa’s(tiab) OR kappas(tiab) OR repeatab*(tiab) OR ((replicab*(tiab) OR repeated(tiab)) AND (measure(tiab) OR measures(tiab) OR findings(tiab) OR result(tiab) OR results(tiab) OR test[-tiab] OR tests(tiab))) OR generaliza*(tiab) OR generalisa*(tiab) OR concordance(tiab) OR (intraclass(tiab) AND correlation*(tiab)) OR discriminative(tiab) OR ‘‘known group’’(tiab) OR factor analysis(tiab) OR factor analyses(tiab) OR dimension*(tiab) OR subscale*(tiab) OR (multitrait(tiab) AND scaling(tiab) AND (analysis(tiab) OR analyses(tiab))) OR item discriminant(tiab) OR interscale correlation*(tiab) OR error(tiab) OR errors(tiab) OR ‘‘individual variability’’(tiab) OR (variability(tiab) AND (analysis(tiab) OR values(tiab))) OR (uncertainty(tiab) AND (measurement(tiab) OR measuring(tiab))) OR ‘‘standard error of measurement’’(tiab) OR sensitiv*(tiab) OR responsive*(tiab) OR ((minimal(tiab) OR minimally(tiab) OR clinical(tiab) OR clinically(tiab)) AND (important(tiab) OR significant(tiab) OR detectable(tiab)) AND (change(tiab) OR difference(tiab))) OR (small*(tiab) AND (real(tiab) OR detectable(tiab)) AND (change(tiab) OR difference(tiab))) OR meaningful change(tiab) OR ‘‘ceiling effect’’(tiab) OR ‘‘floor effect’’(tiab) OR ‘‘Item response model’’(tiab) OR IRT(tiab) OR Rasch(tiab) OR ‘‘Differential item functioning’’(tiab) OR DIF(tiab) OR ‘‘computer adaptive testing’’(tiab) OR ‘‘item bank’’(tiab) OR ‘‘cross-cultural equivalence’’(tiab))

IRT, item response theory; MeSH, medical subject headings; pt, publication type; sh, subheadings; tw, text word; tiab, title/abstract.

Selection of studies

We will upload, store and select the literature search results as figure 1 shows, with the help of a reference management software package, for example, EndNote (version X7; Thomson Reuters, NY, USA). The selection process will entail these steps:

Figure 1

The literature selection process. n=number of references.

For each database, we will create a separate library, while also keeping an original version.

For each library copy, we will create new libraries and subsequently merge them into one library.

We will remove duplicates.

We will select publications based on their title and abstract first.

To confirm final inclusion or exclusion, we will retrieve and screen the full text of the selected publications.

To find more potential articles, we will manually search the selected publications’ reference lists.

We will classify the selected publications by discipline, for example, sports, psychology, medicine and education, based on the articles’ focus and target population.

Two of the authors will work independently to screen all publications for inclusion or exclusion based on a customised selection sheet. We will note the reason for study exclusion in the reference record of the reference management software. In case of disagreement, we will try to reach consensus through discussion. If this should fail, we will consider the opinion of an independent third author. Selection congruency between the two independent reviewers will be measured using Cohen’s Kappa.52

Data extraction

Two authors (CSA and ZS) will carry out the data extraction and a data extraction check independently, using a data extraction sheet specifically developed for this review. We will extract all raw information into Microsoft Excel (V.14.0, 2010, Microsoft, Redmond, California, USA). One researcher will extract data independently from all selected references. A second researcher will check the data extracted from the selected references.

Following de Vet et al. recommendation,50 we will extract the following information:

Characteristics of included articles: first author, year of publication, country of origin, study design and the number and main characteristics of participants (eg, age, gender and target population).

General characteristics of the assessment instrument: name; version; construct of evaluation; number of items; components of MI (kinaesthetic/visual mode or first-/third-person perspective) and subscales; scoring; assessment format; time and equipment needed; examiner qualifications; and costs.

Data on the instruments’ psychometric properties: validity, reliability and responsiveness.

If necessary, the included articles’ authors will be contacted to obtain all the relevant information. A table providing an overview on data extraction will be created for each discipline separately.

Outcomes

The primary outcomes will be: (1) a description of available assessments of MI ability in the fields of sports, psychology, medicine and education; and (2) an evaluation of psychometric properties (the reliability, validity and responsiveness) of the selected MI ability assessments.

The secondary outcome will be to provide an overview of equipment and the training needed for all the selected assessments.

Content comparison

We will provide an overview of each assessment’s content—covering the motor imagery’s ability—using tables to visualise the similarities and differences among several MI ability assessments.

Studies’ methodological quality: the COSMIN evaluation

To assess all the included articles’ methodological quality, we will use the Consensus‐based Standards for the selection of health Measurement Instruments (COSMIN) checklist published by Mokkink et al.53 The COSMIN checklist contains nine domains to assess the following measurement properties: internal consistency, reliability, measurement error, content validity, construct validity (eg, structural validity, hypotheses testing and cross-cultural validity), criterion validity and responsiveness. To complete the COSMIN checklist, we will follow a four-step procedure53:

Determine what measurement properties are evaluated in the article.

If the statistical methods described in the article are based on item response theory (IRT), determine whether the article meets the specified requirements for IRT.

Evaluate the article’s methodological quality with regard to the properties identified in step 1.

Assess the generalisability of the results with regard to the properties identified in step 1.

The domains of the COSMIN checklist contain 5–18 items and each item can be rated as ‘poor’, ‘fair’, ‘good’ or ‘excellent'.53 54 To allocate an overall methodological quality score for each article, we will use the scoring system proposed by the authors of the COSMIN checklist.54 Terwee et al 54 suggest using the ‘worst score counts’principle, which means taking the lowest rating of any item in a checklist domain as the final quality rating for that domain.

Quality assessment of included instruments

Based on the quality criteria for measurement properties proposed by Terwee et al,55 the measurement properties reported in the included articles will be rated as positive, negative or indeterminate, depending on the study design, methods and outcomes (table 3). For the proposed review, we will present an overview of these ratings in a table. Although the aforementioned criteria were developed for health status questionnaires, they can also be applied to performance-based assessments and other measurement methods.50

Table 3

Quality criteria for measurement properties by Terwee et al 55

Property	Rating	Quality criteria
Reliability
Internal consistency	+	Cronbach’s alpha(s) ≥0.70
	?	Cronbach’s alpha is not determined or the dimensionality is unknown
	−	Cronbach’s alpha(s) <0.70
Reliability	+	ICC/weighted Kappa ≥0.70 OR Pearson’s r ≥0.80
	?	Neither ICC/weighted Kappa, nor Pearson’s r are determined
	−	ICC/weighted Kappa <0.70 OR Pearson’s r <0.80
Measurement error	+	MIC>SDC OR MIC is outside the LoA
	?	MIC is not defined
	−	MIC≤SDC OR MIC equals or is inside the LOA
Validity
Content validity	+	All items refer to relevant aspects of the construct to be measured, for the target population and for the purpose of the measurement AND the questionnaire is considered comprehensive
	?	Not enough information available
	−	Not all items are considered relevant for the construct to be measured, for the target population and for the purpose of the measurement OR the questionnaire is considered not comprehensive
Construct validity:  (a) Structural validity	+	Factors should explain at least 50% of the variance
	?	Explained variance is not mentioned
	−	Factors explain <50% of the variance
(b) Hypothesis testing	+	Correlations with instruments measuring the same construct ≥0.50 OR at least 75% of the results are in accordance with the hypotheses AND correlations with related constructs are higher than with unrelated constructs
	?	Correlations solely determined with unrelated constructs
	−	Correlations with instruments measuring the same construct <0.50 OR <75% of the results are in accordance with the hypotheses OR correlations with related constructs are lower than with unrelated constructs
(c) Cross-cultural validity	+	No differences in factor structure OR no important DIF between language versions
	?	Multiple-group factor analysis not applied AND DIF not assessed
	−	Differences in factor structure OR important DIF between language versions
Criterion validity	+	Convincing arguments that the gold standard is ‘gold’ AND correlation with the gold standard ≥0.70
	?	No convincing arguments that the gold standard is ‘gold’
	−	Correlation with the gold standard <0.70
Responsiveness	+	Correlation with changes on instruments measuring the same construct ≥0.50 OR at least 75% of the results are in accordance with the hypotheses OR AUC ≥0.70 AND correlations with changes in related constructs are higher than with unrelated constructs
	?	Correlations solely determined with unrelated constructs
	−	Correlations with changes on instruments measuring the same construct <0.50 OR <75% of the results are in accordance with the hypotheses OR AUC <0.70 OR correlations with changes in related constructs are lower than with unrelated constructs

+, positive rating; -, negative rating.; ?, indeterminate rating; AUC, area under the curve; DIF, differential item functioning; ICC, intraclass correlation coefficient; LoA, limits of agreement; MIC, minimal important change; SDC, smallest detectable change.

Data synthesis

If more than one article is reported on the same assessment and they are homogeneous concerning study population, design, the language, the version of the used instrument and the form of administration, we will perform ‘a best evidence synthesis’ as proposed by van Tulder et al 56 (table 4). We will rate the level of best evidence as ‘strong’, ‘moderate’, ‘limited’, ‘conflicting’ or ‘unknown’. This step should facilitate choosing a suitable assessment.

Table 4

Levels of evidence for the overall quality of measurement properties from van Tulder et al 56

Level	Rating	Criteria
Strong	+++or ---	Consistent findings in multiple studies of good methodological quality OR in one study of excellent methodological quality
Moderate	++or --	Consistent findings in multiple studies of fair methodological quality OR in one study of good methodological quality
Limited	+ or -	One study of fair methodological quality
Conflicting	±	Conflicting findings
Unknown	?	Only studies of poor methodological quality

+, positive rating; ?, indeterminate rating; -, negative rating.

Patient and public involvement

This review will be based on previous published data and no patients or the public will be involved in this review.

Discussion

The proposed review will evaluate available MI ability assessments and their measurement properties across four important disciplines: sports, psychology, medicine and education. There is a need to evaluate the existing assessments systematically across disciplines. This is especially true now for the following reasons:

The benefits and effectiveness of MI have been confirmed and this technique is increasingly applied in different disciplines.1 3 5–8 10–14

The effect of MI interventions depends on the individual’s MI ability and it should be evaluated prior to an MI intervention.14 30

For the last 200 years, experts focused on developing assessments to evaluate MI’s abstract construct.31–43 For these reasons, a systematic evaluation is warranted.

Collecting MI assessments from different disciplines allows and facilitates their cross-disciplinary usage and research. It is assumed that several MI assessment instruments will be attributed to more than one discipline. However, based on our experience, it is difficult for a hemiparetic patient to perform and imagine running or jumping during a MI ability assessment, as is sometimes suggested. Discipline-specific MI assessment might be more sensitive in some cases and thus, it is best not to focus on a movement that cannot be performed at the moment.

This systematic overview could help to select the most suitable MI ability assessment for the treatment aim and population and further evaluate the efficacy of MI training interventions. To date, only a few studies have offered a comprehensive evaluation of the methods by which MI ability has been assessed, much less these methods’ psychometric properties. So far, there have been only a few studies on the assessment methods of MI ability,2 17 48 and for the studies that do exist, their overview is quite narrow and focused on population and language. Only two reviews evaluate the psychometric properties of the included tools17 48 and none of the cited reviews consider whether the MI ability assessments are appropriate for children. Furthermore, in a comprehensive review, Schuster et al 1 reported that some of the included studies used individual, custom-designed assessments of MI ability, which limits the comparison of results with other studies.

Presenting the review results

Results of this review will be reported following the PRISMA guidelines, using flowcharts and tables.57 Information on assessing performance and equipment as well as presenting the included articles’ methodological quality and the included instrument’s quality assessments will offer athletes, trainers, clinicians, teachers and other interested MI user the much-needed tools to quickly determine an MI ability assessment’s focus and its applicability and utility.