Measurement properties of instruments assessing permanent functional impairment of the spine: a systematic review protocol.

INTRODUCTION: Permanent functional impairment (PFI) of the spine is a rating system used by compensation authorities, such as workers compensation boards, to establish an appropriate level of financial compensation for persistent loss of function. Determination of PFI of the spine is commonly based on the assessment of spinal movement combined with other measures of physical and functional impairments; however, the reliability and validity of the measurement instruments used for these evaluations have yet to be established. The aim of this study is to systematically review and synthesise the literature concerning measurement properties of the various and different instruments used for assessing PFI of the spine.
METHODS: Three conceptual groups of terms (1) PFI, (2) spinal disorder and (3) measurement properties will be combined to search Medline, EMBASE, CINAHL, Web of Science, Scopus, PEDro, OTSeeker and Health and Safety Science Abstracts. We will examine peer-reviewed, full-text articles over the full available date range. Two reviewers will independently screen citations (title, abstract and full text) and perform data extraction. Included studies will be appraised as to their methodological quality using the COnsensus-based Standards for the selection of health Measurement INstruments criteria. Findings will be summarised and presented descriptively, with meta-analysis pursued as appropriate. ETHICS AND DISSEMINATION: This review will summarise the current level of evidence of measurement properties of instruments used for assessing PFI of the spine. Findings of this review may be applicable to clinicians, policy-makers, workers' compensation boards, other insurers and health and safety organisations. The findings will likely provide a foundation and direction for future research priorities for assessing spinal PFI. PROSPERO REGISTRATION NUMBER: CRD42017060390. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This systematic review will include a broad range of instruments used to assess permanent functional impairment in individuals with spinal conditions in peer-reviewed articles.

This systematic review protocol is presented in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses Protocols.

A strength of this review is the use of the internationally recognised, validated COnsensus-based Standards for the selection of health Measurement INstruments guidelines to assess the methodological quality of the included studies.

A limitation of this review is language bias, since only studies in English will be included. However, studies published in English that describe cross-cultural validation of instruments from English into other languages will be included.

Introduction

Spinal disorders are the leading cause of work disability, representing around 65% of all musculoskeletal injury/illness claims associated with lost work time, posing a considerable economic, personal and societal burden.1 Internationally, occupational low back pain is the single most prevalent health problem related to absenteeism among workers, responsible for approximately 25% of costs from work-related conditions.2

Spinal disorders such as low back pain are often complex and impact a wide range of multidimensional aspects of health and physical function, such as disability.1 Disability is an umbrella term for impairments, activity limitations and participation restrictions, referring to difficulties encountered in any or all of these three areas of functioning.3 Such complex and interactive processes can be challenging when trying to assess patients’ disability using a dichotomous approach. For example, insurance providers might require thresholds to be set for impairment severity, activity limitations or participation restriction in order to determine eligibility for financial compensation or other benefits.3 4

Permanent functional impairment (PFI) or permanent impairment, is a rating systems used by compensation authorities, to establish an appropriate level of financial compensation.5–9 Evaluation of PFI requires selecting appropriate outcome measures.10 However, the metrics of PFI ratings are not uniformly specified nor universally adopted by workers’ compensation boards, varying in terms of specific PFI rating guides as well as the adoption of function-based criteria.6 9 11 Feasibility, reliability, validity, utility and cost are important aspects that should be taken into consideration when selecting clinical measurement instruments for determination of PFI.12 Although impairment can contribute to limited function and ultimately may have consequences for physical functioning, a clinical examination finding of impairment does not always correspond to a functional loss.13

Instruments used to assess spinal PFI have been critically evaluated, since their measurements are usually based only on impairment.14–18 Poor validity and reliability of non-invasive clinical measurement instruments and the presence of significant measurement error14–17 are criticisms that have led to a number of alternative impairment-based spinal assessment systems such as: American Medical Association Guides’ Diagnosis-Related Estimate,11 Diagnosis-Based Impairment,5 McBride’s method of spinal impairment evaluation19 or Physical Impairment Index.18 20 21 However, these alternative systems for rating PFI present poor descriptions of standardisation and normative values.10 22

The outcome of spinal PFI assessment by compensation bodies can have considerable social, economic and health impacts, yet the reliability and validity of instruments currently available have not been systematically evaluated. To achieve a fair and accurate outcome, clinicians, researchers and government bodies must have access to the most accurate level of evidence regarding methods assessing spinal PFI. To our knowledge, no previous review has assessed the reliability and validity of available instruments for determining PFI of the spine. This manuscript presents the protocol of an ongoing systematic review with the objective to review and synthesise the literature concerning measurement properties of the instruments used for assessing PFI of the spine. Due to the diversity of instruments that might be used to assess PFI of the spine, we anticipate the identification of distinct metrics for measuring PFI will allow specific subgroup analyses for review and ultimate discussion on the strength of support for each instrument.

Methods

Design

This systematic review protocol has been registered with International Prospective Register of Systematic Reviews (registration number CRD42017060390). This protocol will be reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols statement and checklist.23

Eligibility criteria

Inclusion criteria

We will include peer-reviewed articles, with no date limitation, investigating measurement properties of instruments that assess PFI of the spine in individuals with spinal disorders aged from 18 to 65 years. When evaluating PFI, acceptable studies will include measures of impairment (ie, RoM, muscle strength, coordination, endurance and sensation), functional limitation (ie, self-report instruments of physical function and functional performance measures) and permanency of the impairment (ie, duration of the impairment and the likelihood of improvement).3 24

Exclusion criteria

We will exclude letters to the editor, book reviews and short communications. We will also exclude clinical protocols, case reports and series, systematic reviews, meta-analysis, articles not published in English-language, studies intended for screening, diagnosis and prognosis of spinal pathologies as well as studies with specimen-based, cadaver-based, cellular-based, artificial-based and computer-based models. We will also exclude studies of spinal conditions caused by congenital and developmental abnormalities, neoplasm, infection and systemic inflammatory disorders.25 Exclusion will take place at the screening stage, following the literature search, in order to avoid the risk of excluding relevant articles.

Information sources

Studies will be identified by searching electronic databases, scanning reference lists of articles and consultation with experts in the area. A preliminary search plan was developed and pretested by a professional health sciences librarian (CB). The following databases were searched: Medline—OVID (1946 to 20 April 2017); EMBASE—OVID (1947 to 21 April 2017); CINAHL—Cumulative Index to Nursing and Allied Health Literature (1937 to 25 April 2017); Web of Science (1900 to 21 April 2017); Scopus (1995 to 25 April 2017); PEDro (Inception to 22 April 2017); OTSeeker (Inception to 22 April 2017) and Health and Safety Science Abstracts—ProQuest (1988 to 22 April 2017).

Search strategy

To capture all measurement instruments used to evaluate PFI of the spine, our search strategy will focus on combining three conceptual groups of terms: (1) PFI, (2) spinal disorder and (3) measurement properties. The Medline search strategy is presented in table 1.

Table 1

Preliminary search strategy in OVID Medline (1950 to present)

‘Spinal disorder’ search terms	‘PFI’ search terms	‘Measurement properties’ search terms
1. spine/or cervical vertebrae/or exp axis, cervical vertebra/or cervical atlas/or coccyx/or intervertebral disc/or lumbar vertebrae/or sacrum/or spinal canal/or epidural space/or thoracic vertebrae/	33. exp ‘Range of Motion, Articular’/	59. validation studies/
2. (spine or spinal or coccyx or ‘intervertebral disc’ or lumbar vertebrae or sacrum or ‘spinal canal’ or ‘thoracic vertebrae’ or ‘cervical vertebrae’).ab,ti.	34. range of motion.ab,ti.	60. Comparative Study/
3. low back pain/	35. Goniomet*.ab,ti.	61. Psychometrics/
4. back Pain/or neck pain/	36. Joint motion measurement*.ab,ti.	62. psychometr*.ab,ti.
5. back/or neck/or lumbosacral region/or sacrococcygeal region/	37. Electrogoniomet*.ab,ti.	63. clinimetr*.tw.
6. (back or neck or ‘lumbosacral region’ or ‘sacrococcygeal region’).ab,ti.	38. (Movement adj2 evaluation*).ab,ti.	64. clinometr*.tw.
7. Sciatica/	39. Inclinomet*.ab,ti.	65. ‘Outcome Assessment (Health Care)"/
8. Neck Injury/or Whiplash Injuries/	40. Joint flexibility.ab,ti.	66. ‘outcome assessment’.ab,ti.
9. whiplash.ab,ti.	41. Arthromet*.ab,ti.	67. (‘outcome measure’ or ‘outcome measures’).tw.
10. Dorsalgia.ti,ab.	42. Joint mobility.ab,ti.	68. Observer Variation/
11. coccydynia.ti,ab.	43. Osteokinematic*.ab,ti.	69. ‘observer variation’.ab,ti.
12. ((‘cervical vertebrae’ or ‘cervical spine’ or craniovertebral or sacroiliac or verteb* or thoracic) adj2 (symptom* or injur* or disorder* or pain or dysfunction* or problem* or strain* or spain*)).ab,ti.	44. Flexion.ab,ti.	70. Health Status Indicators/
13. discitis.ti,ab.	45. Extension.ab,ti.	71. ‘Reproducibility of Results’/
14. (disc adj degeneration).ti,ab.	46. Rotation/	72. reproducib*.ti,ab.
15. (disc adj prolapse).ti,ab.	47. side bend*.ab,ti.	73. Discriminant Analysis/
16. (disc adj herniation).ti,ab.	48. Work Capacity Evaluation/	74. reliab*.ab,ti.
17. (facet adj joints).ti,ab.	49. Disability Evaluation/and (Occupational Diseases/or Work/or Return to Work/)	75. unreliab*.ab,ti.
18. Intervertebral Disc/or annulus fibrosus/or nucleus pulposus/	50. Occupational Diseases/or Work/or Return to Work/	76. valid*.ab,ti.
19. arachnoiditis.ti,ab.	51. (impair* or disabilit* or abilit* or handicap* or ‘functional severity’ or ‘restriction of function’ or ‘capacity to work’ or ‘functional capacity’ or ‘disability rating’ or ‘impairment rating’ or ‘work fitness’).ab,ti.	77. coefficient.ab,ti.
20. Spinal Fusion/	52. 50 and 51	78. homogeneity.ab,ti.
21. postlaminectomy.ti,ab.	53. ((work* or occupat*) adj4 (capacity or impair* or disabilit* or abilit* or handicap* or ‘functional severity’ or ‘restriction of function’)).ab,ti.	79. homogeneous.ab,ti.
22. Backache*.ti,ab.	54. (evaluation* or assessment* or instrument* or measurement* or tool* or scale* or questionnaire* or test* or determination*).ab,ti.	80. ‘internal consistency’.ab,ti.
23. back injuries/or spinal injuries/or spinal fractures/	55. 53 and 54	81. cronbach*.ab,ti.
24. Spondylitis, Ankylosing/	56. ‘permanent functional impairment’.ab,ti.	82. (alpha or alphas).ab,ti.
25. Spondylitis/	57. (‘work performance evaluation’ or ‘work performance evaluations’).ab,ti.	83. 81 and 82
26. spondylosis/or spondylolysis/or spondylolisthesis/	58. 33 or 34 or 35 or 36 or 37 or 38 or 39 or 40 or 41 or 42 or 43 or 44 or 45 or 46 or 47 or 48 or 49 or 52 or 55 or 56 or 57	84. item.ab,ti.
27. Spinal Cord Compression/		85. (correlation* or selection* or reduction*).ab,ti.
28. (Musculoskeletal adj (symptom* or injur* or disorder* or pain or dysfunction* or problem*)).ab,ti.		86. 84 and 85
29. ((orthopedic or orthopaedic) adj (injur* or problem* or disorder* or dysfunction*)).ab,ti.		87. agreement.ab,ti.
30. Pelvis/or lesser Pelvis/or pelvic floor/		88. precision.ab,ti.
31. (pelvi* adj3 (symptom* or injur* or disorder* or pain or dysfunction* or problem*)).ab,ti.		89. imprecision.ab,ti.
32. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20 or 21 or 22 or 23 or 24 or 25 or 26 or 27 or 28 or 29 or 30 or 31		90. ‘precise values’.ab,ti.
		91. test-retest.ab,ti.
		92. test.ab,ti.
		93. retest.ab,ti.
		94. 92 and 93
		95. reliab*.ab,ti.
		96. (test or retest).ab,ti.
		97. 95 and 96
		98. stability.ab,ti.
		99. interrater.ab,ti.
		100. inter-rater.ab,ti.
		101. intrarater.ab,ti.
		102. intra-rater.ab,ti.
		103. intertester.ab,ti.
		104. inter-tester.ab,ti.
		105. intratester.ab,ti.
		106. intra-tester.ab,ti.
		107. interobserver.ab,ti.
		108. inter-observer.ab,ti.
		109. intraobserver.ab,ti.
		110. intra-observer.ab,ti.
		111. intertechnician.ab,ti.
		112. intertechnician.ab,ti.
		113. intratechnician.ab,ti.
		114. intra-technician.ab,ti.
		115. interexaminer.ab,ti.
		116. inter-examiner.ab,ti.
		117. intraexaminer.ab,ti.
		118. intra-examiner.ab,ti.
		119. interassay.ab,ti.
		120. inter-assay.ab,ti.
		121. intraassay.ab,ti.
		122. intra-assay.ab,ti.
		123. inter-individual.ab,ti.
		124. interindividual.ab,ti.
		125. intraindividual.ab,ti.
		126. intra-individual.ab,ti.
		127. interparticipant.ab,ti.
		128. inter-participant.ab,ti.
		129. intraparticipant.ab,ti.
		130. intra-participant.ab,ti.
		131. kappa*.ab,ti.
		132. repeatab*.ab,ti.
		133. (replicab* or repeated).ab,ti.
		134. (measure* or finding* or result* or test*).ab,ti.
		135. 133 and 134
		136. generaliza*.ab,ti.
		137. generalisa*.ab,ti.
		138. concordance.ab,ti.
		139. (intraclass or intra-class).ab,ti.
		140. correlation*.ab,ti.
		141. 139 and 140
		142. discriminative.ab,ti.
		143. ‘known group’.ab,ti.
		144. ‘factor analysis’.ab,ti.
		145. factor analyses.ab,ti.
		146. dimension*.ab,ti.
		147. subscale*.ab,ti.
		148. (multitrait and scaling and (analysis or analyses)).ab,ti.
		149. item discriminant.ab,ti.
		150. inter-scale correlation*.ab,ti.
		151. interscale correlation*.ab,ti.
		152. error.ab,ti.
		153. errors.ab,ti.
		154. ‘individual variability’.ab,ti.
		155. (variability and (analysis or values)).ab,ti.
		156. (uncertainty and (measurement or measuring)).ab,ti.
		157. ‘standard error of measurement’.ab,ti.
		158. sensitiv*.ab,ti.
		159. responsive*.ab,ti.
		160. ((minimal or minimally or clinical or clinically) and (important or significant or detectable) and (change or difference)).ab,ti.
		161. (small* and (real or detectable) and (change or difference)).ab,ti.
		162. ‘meaningful change’.ab,ti.
		163. ‘ceiling effect’.ab,ti.
		164. ‘floor effect’.ab,ti.
		165. ‘item response model’.ab,ti.
		166. IRT.ab,ti.
		167. Rasch.ab,ti.
		168. ‘differential item functioning’.ab,ti.
		169. DIF.ab,ti.
		170. ‘computer adaptive testing’.ab,ti.
		171. ‘item bank’.ab,ti.
		172. ‘cross-cultural equivalence’.ab,ti.
		173. (‘gold standard’ or ‘criterion standard’ or ‘reference standard’).ab,ti.
		174. 59 or 60 or 61 or 62 or 63 or 64 or 65 or 66 or 67 or 68 or 69 or 70 or 71 or 72 or 73 or 74 or 75 or 76 or 77 or 78 or 79 or 80 or 83 or 86 or 87 or 88 or 89 or 90 or 91 or 94 or 98 or 99 or 100 or 101 or 102 or 103 or 104 or 105 or 106 or 107 or 108 or 109 or 110 or 111 or 112 or 113 or 114 or 115 or 116 or 117 or 118 or 119 or 120 or 121 or 122 or 123 or 124 or 125 or 126 or 127 or 128 or 129 or 130 or 131 or 132 or 135 or 136 or 137 or 138 or 141 or 142 or 143 or 144 or 145 or 146 or 147 or 148 or 149 or 150 or 151 or 152 or 153 or 154 or 155 or 156 or 157 or 158 or 159 or 160 or 161 or 162 or 163 or 164 or 165 or 166 or 167 or 168 or 169 or 170 or 172 or 173
175. 32 and 58 and 174
176. limit 175 to (addresses or autobiography or bibliography or biography or comment or congresses or directory or editorial or festschrift or interactive tutorial or interview or lectures or legal cases or legislation or letter or news or newspaper article or patient education handout or practice guideline or video-audio media or webcasts)
177. 175 not 176

PFI, permanent functional impairment.

The terms for the concept of ‘measurement properties’ were adapted from a search filter validated and published by Terwee et al26 and optimised for Medline. A similar search will be employed for other databases, optimised for database-specific search interfaces. A filter to exclude certain publication types (eg, addresses, editorials, letters and newspaper articles) will be applied, as recommended by Terwee et al.26 References will be imported into DistillerSR software (Distiller SR Evidence Partners, Ottawa, Canada). All duplicate articles will be removed prior to screening. Reference lists of included studies will be scrutinised during the process of data extraction for additional articles meeting our criteria. Identified citations will be downloaded and subjected to the same de-duplication and screening process.

Study selection

Study selection will be undertaken in two steps. First, two reviewers will independently screen all titles and abstracts, and all citations selected by both reviewers will be retrieved for step two: full-text screening. In cases where there is disagreement between the two reviewers, any discrepancies will be resolved by consensus and/or consultation with a third member of the research team prior to making a final decision.

A screening tool will be developed and piloted for the title, abstract and full-text screening stages (see box). Inclusion/exclusion will be determined using the same tool throughout the process, although the definitions may be refined as conflicts arise and are discussed. The same questions will be used for both title and abstract as well as full-text screening, with a focus on sensitivity rather than specificity at these stages.

Is the study written in English?

No—exclude

Yes or uncertain—go to step 2

Does the study deal with humans?

No—exclude

Yes or uncertain—go to step 3

Does the study deal with adults (between 18 and 65 years of age)?

No—exclude

Yes or uncertain—go to step 4

Does the article represent primary study (ie, no letters to the editor, book reviews, published study designs or trial protocols)?

No—exclude

Yes or uncertain—go to step 5

Does the study assess the spine (cervical, thoracic, lumbar spine, including neck and low back)?

No—exclude

Yes or uncertain—go to step 6

Does the study measure permanent functional impairment (PFI; permanent impairment, physical impairment, functional impairment or disability)?

No—exclude

Yes or uncertain—go to step 7

Is the study designed to evaluate measurement properties of measurement instruments/tools (eg, validity, reliability, responsiveness)?

No—exclude

Yes or uncertain—choose one of the following options:

Title and abstract screening stage—include

Full-text screening stage—go to step 8

Does this study investigate individuals with spinal conditions not caused by congenital and developmental abnormalities, neoplasm, infection disorders and systemic inflammatory disorders?

No—exclude

Yes or uncertain—go to step 9

Does this study have clinical measures of impairment (eg, range of motion, muscle performance) or objective measures of function (eg, functional impairment evaluation)?

No—exclude

Yes or uncertain—go to step 10

Does this study have a PFI definition that includes measures of chronic or permanent symptoms’ duration as well as measures of functional limitation?

No—exclude

Yes or uncertain—go to step 11

Does this study use at least one statistical test between the measures listed on question number ‘9’ and on question number ‘10’?

No—exclude

Yes or uncertain—Include

Data extraction process

We will develop data extraction forms, as outlined in table 2, which will be pilot tested on three studies not included in the review. Two reviewers will extract data independently. Any disagreement will be resolved through a third party adjudication. The nature of performance measures employed by included studies will drive the nature of the data extraction, as we anticipate that there may be some heterogeneity in the evaluation of measurement instruments.

Table 2

Draft extraction tool

Item	Definitions, decision rules, guidance and example data
Reference ID	Author, date
Instrument name	The name as described in the article, or as referred to in other articles (eg, ‘Back-EST’ or ‘the Johnson method’)
Instrument description	Will collect sub variables, such as: Equipment required Duration Clinicians performing test (eg, physiotherapist, occupational therapist, physician) Dimensions assessed (eg, range of motion, lifting capacity, etc)
Validity—criterion	Will collect subvariables, such as: Did the article assess this dimension? (yes/no) What was the inferential test used? (eg, intraclass correlation coefficient, kappa) What was the result? (eg, kappa score of 0.7)
Validity—content	Will collect sub variables, such as: Did the article assess this dimension? (yes/no) How was content validity assessed? What were the findings?
Validity—construct	Will collect subvariables, such as: Did the article assess this dimension? (yes/no) How was construct validity assessed? What were the findings?
Reliability—inter-rater	Will collect sub variables, such as: Did the article assess this dimension? (yes/no) What was the inferential test used? (eg, intraclass correlation coefficient, kappa) What was the result? (eg, kappa score of 0.7)
Reliability—intrarater	Will collect subvariables, such as: Did the article assess this dimension? (yes/no) What was the inferential test used? (eg, intraclass correlation coefficient, kappa) What was the result? (eg, kappa score of 0.7)
Responsiveness	Will collect sub variables, such as: Did the article assess this dimension? (yes/no) How was responsiveness assessed? What were the findings?
COSMIN Study Quality Metrics	Will adhere to the COSMIN tool, do not anticipate altering the definitions

These Items may evolve as review progresses and studies are identified. This draft data dictionary describes the selected data extraction variables of the systematic review. The data dictionary would be used to extract data from articles identified as relevant during the screening process. The data extraction database itself would have these items listed in columns with the individual references representing rows such that these data are extracted for every article to form a summary database for later synthesis. Only key variables are presented here. It is anticipated that this list will expand based on the type of data presented in the articles; the definitions guide will also become more specific as a result of independent data extraction and discussion at team meetings.

COSMIN, COnsensus-based Standards for the selection of health Measurement INstruments.

To avoid analysing the same data from multiple articles, we will compare author names, sample sizes and outcomes. In the case of companion papers (eg, reports) of a primary study, all available data will be simultaneously evaluated, to maximise the yield of information for the primary study. Study authors may be contacted for clarifications. If contact with authors is unsuccessful, we will use the most complete and up-to-date data source available, thus avoiding the risk of double counting.

Data items

We will extract information relating to: (1) the measurement properties (ie, reliability, validity and responsiveness) of the measurement instruments for assessing PFI of the spine (eg, assessment of RoM, muscle strength, coordination, endurance and sensation, as well as ability to perform basic daily living and work activities, and duration of the impairment); (2) the type of inferential test and (3) the test results for each article. The extraction process will be tracked using a standardised data extraction form.

Articles will be separated into subgroups based on the instruments used to assess PFI of the spine. The different subgroups will be analysed separately based on impairment-based measures (ie, RoM, muscle strength, coordination, endurance and sensation) and functional limitation assessment (ie, self-report instruments of physical function and functional performance measures). For clarity, we are likely to isolate articles in subgroups to better differentiate the instruments used to assess PFI of the spine, especially when considering that RoM has predominantly been used for assessing impairment of the spine.5 6 9 11

Quality assessment of individual studies

Included studies will be appraised independently by two reviewers as to their methodological quality using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria.27 28 Any disagreement will be resolved by consultation with a third member of the research team. The COSMIN checklist is a consensus-based tool designed to evaluate the methodological quality of studies investigating measurement properties. The instrument shows appropriate levels of agreement27 and, based on its content validity, is a recommended tool for assessing the methodological quality of studies evaluating measurement properties of outcome measures within a systematic review.28

The tool will evaluate the following measurement constructs: reliability; measurement error; content validity; structural validity; hypotheses testing; cross-cultural validity and criterion validity, responsiveness, and interpretability with 5–18 items concerning methodological standards for how each measurement property should be assessed (see table 3). The methodological quality of a study will be considered adequate if all items in a measurement property are considered adequate. Each item is scored on a four-point rating scale (ie, ‘poor’, ‘fair’, ‘good’ or ‘excellent’).

Table 3

COSMIN definitions of domains, measurement properties and aspects of measurement properties

Domain	Measurement property	Aspect of a measurement property	Definition
Reliability			The extent to which scores for individuals who have not changed are the same for repeated measurement under several conditions.
	Internal consistency		Different sets of items from the same instrument.
	Test–retest		Changes are repeated measurement over time.
	Inter-rater		The degree of agreement between raters investigating the same property on the same patient.
	Intrarater		The degree of agreement between repeated measurements of a property on the same patient by the same rater.
	Measurement error		The systemic and random error of a patient’s score that is not attributed to true changes in the construct to be measured.
Validity			How well the instrument under evaluation measures the construct it purports to measure.
	Content validity		The degree to which the score of an instrument is adequate reflection of the construct to be measured.
		Face validity	The degree to which (the item of) an instrument is an adequate reflection of the construct to be measured.
	Construct validity		The degree to which the scores of an instrument are consistent with hypotheses (for instance, with regard to internal relationships, relationships to scores of other instruments or differences between relevant groups) based on an assumption that the instrument truly measures what it is meant to.
		Structural validity	The degree to which the score of an instrument is an adequate reflection of the dimensionality of the construct to be assessed.
		Cross-cultural validity	The degree to which the performance of the items on a translated or culturally adapted instrument are an adequate reflection of the performance of the items of the original version of the instrument.
	Criterion validity		The degree to which measurements are an adequate reflection of a previously used ‘gold standard’.
Responsiveness			The ability of the measurement instrument to detect change over time in the construct to be measured.
Interpretability			The degree to which one can assign qualitative meaning, that is, clinical or commonly understood connotations— to an instrument’s quantitative scores or change in scores.

COSMIN, COnsensus-based Standards for the selection of health Measurement INstruments.

In the context of this review, validity, in general, defines how well the instrument under evaluation measures the construct it purports to measure. Criterion validity is the degree to which measurements are an adequate reflection of a previously used ‘gold standard’. Content validity is an adequate reflection of the construct to be measured; construct validity is based on an assumption that the instrument truly measures what it is meant to and structural validity implies the scores of an instrument is an adequate reflection of the dimensionality of the construct to be assessed. Reliability refers to the extent to which scores for individuals who have not changed are the same for repeated measurement under several conditions. These include using different sets of items from the same instrument (internal consistency); over time (test–retest); by different assessors on the same occasion (inter-rater) or by the same assessors (ie, raters or responders) on different occasions (intrarater). Responsiveness is the ability of the measurement instrument to detect change over time in the construct to be measured.27

Planned methods of analysis

Where it is possible, mean values of statistical analysis (eg, Cronbach’s alpha, intraclass correlation coefficient, SE of measurement, smallest detectable change, effect sizes, etc) will be calculated from pooled data from methodologically similar studies and the results statistically summarised via meta-analysis into forest plots with estimates of heterogeneity. In addition, sensitivity analysis will be pursued by comparing results from studies with high-quality and low-quality ranking. We will then weight such meta-analyses using both the study’s sample size and their quality assessment as determined by the COSMIN checklist.28 However, we predict some heterogeneity will be identified in the various assessments of spinal PFI measures, which will likely make a meta-analysis difficult to apply.

In the event meta-analysis is not possible, descriptive and narrative syntheses will be presented in text and table formats, with the intent of summarising and discussing the sample and methodological aspects, as well as the findings regarding measurement properties of the included studies assessing PFI in individuals with spinal conditions. Tables will provide general and comparative information of these heterogeneous and disparate studies (ie, authors, country and population parameters, such as age, gender distribution, setting etc), and will summarise reliability, validity and responsiveness data with associated study quality indicators (COSMIN checklist).

In order to determine the best available method for measuring PFI of the spine, each identified instrument will be ranked using a range of measurement performance metrics identified in the COSMIN checklist. The findings will be presented and possible hypotheses for the results will be generated and discussed. In addition, gaps in the literature will be identified and discussed.

Ethics and dissemination

A substantial number and variety of spine impairment ratings are calculated internationally and used for financial award decisions, in both legal and compensation contexts.29–34 However, a systematic review in this area to support the use of instruments currently being used to determine PFI is lacking. The application of reliable and valid measurement instruments for assessing PFI of the spine is considered crucial.

This systematic review offers a feasible means for synthesising the evidence specific to spinal PFI assessment; and our results will likely provide unique insights concerning the breadth and depth of literature in the area. Outcomes of this review will be applicable to clinicians, policy-makers, worker’s compensation boards and health and safety organisations. In particular, findings will likely provide a foundation and direction in terms of research priorities for assessing PFI of the spine. Summarising the nature and strength of the evidence regarding the reliability, validity and responsiveness of spinal PFI measures will also inform future research and policy in this field.