Evaluation of the psychometric properties of patient-reported and clinician-reported outcome measures of chemotherapy-induced peripheral neuropathy: a COSMIN systematic review protocol.

INTRODUCTION: Chemotherapy-induced peripheral neuropathy (CIPN) is a poorly understood side effect of many antineoplastic agents. Patients may experience sensory, motor and autonomic symptoms, negatively impacting quality of life. A gold-standard assessment methodology has yet to be determined, limiting efforts to identify effective agents to prevent or treat CIPN. METHODS AND ANALYSIS: This is a protocol of a systematic review of psychometric analyses of CIPN Clinician Reported Outcome Measures (ClinROM) and Patient-Reported Outcome Measures (PROM) among adults receiving, or who had previously received chemotherapy for cancer. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) quality ratings will be compared across studies and across ClinROMs and PROMs. Studies reporting psychometric proprieties of CIPN ClinROMs and/or PROMs among adults aged ≥18 years will be eligible for inclusion, with no restriction on language or year of publication. MEDLINE, Embase, CINAHL and APA PsycINFO databases will be searched from inception to 31 December 2021. Study characteristics, measurement properties of the ClinROMs and/or PROMs and the CIPN definitions will be extracted. The Synthesis Without Meta-analysis guideline will be used to guide data synthesis. The COSMIN Risk of Bias checklist will be used by two independent raters to assess methodological quality. Subgroup analyses by age, chemotherapy type, and study timing in relation to the delivery of chemotherapy will be carried out where data are available. An adapted version of Outcome Measures in Rheumatology filter 2.1 will be used to provide a best-evidence synthesis of CIPN ClinROMs and PROMs and to recommend a CIPN assessment tool for clinical and research settings. ETHICS AND DISSEMINATION: Ethical approval is not necessary to be obtained for this systematic review protocol. Results will be disseminated to clinicians and policy-makers by publication in a peer-reviewed journal and by presenting at relevant conferences. PROSPERO REGISTRATION NUMBER: CRD42021278168. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This proposed study will be the most up-to-date systematic review of chemotherapy-induced peripheral neuropathy (CIPN) Clinician Reported Outcome Measures (ClinROMs) and patient-reported outcome measures (PROMs) and the first to use COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology to assess the methodological quality of studies reporting psychometric properties of CIPN ClinROMs.

The proposed study is also the first to use the Outcome Measures in Rheumatology filter 2.1 methodology following COSMIN analysis of CIPN ClinROMs and PROMs facilitating the recommendation of one or more ClinROMs and/or PROMs for assessing CIPN, based on the available evidence.

Subgroup analyses by age group, chemotherapy type and the timing of the study in relation to the delivery of chemotherapy will be undertaken to gain a better understanding of potential subgroup differences in the psychometric proprieties of CIPN ClinROMs and PROMs.

Limitations may be related to the heterogeneity of patients who have CIPN and to the lack of consensus about the definition of CIPN.

As the COSMIN methodology does not consider patient-reported experience measures, they are not included in this review, potentially impacting the identification of a tool that could improve patient health outcomes.

Introduction

Chemotherapy-induced peripheral neuropathy (CIPN) is a challenging and common side effect of many antineoplastic agents with significant negative impacts on quality of life.1–3 Patients may experience an array of symptoms such as numbness, tingling, pins-and-needles, burning, motor weakness and/or balance disturbance.1 A meta-analysis has suggested that 68.1% (95% CI 57.7% to 78.4%) of patients have CIPN symptoms in the first month after completing chemotherapy and 60.0% (95% CI 36.4% to 81.6%) of patients have symptoms 3 months after finishing chemotherapy.3 4 Thirty per cent (95% CI 6.4% to 53.5%) of patients report persistent CIPN symptoms 6 months or later after treatment.4 Wide CIs suggest substantial uncertainty in this estimate, potentially due to variability in measurement tools used in these studies.5 6 CIPN may lead to dose reductions, delayed treatment or early discontinuation in 2%–58% of people, with unclear effects on disease status and survival.7–13 Unfortunately, there are no known preventative agents, and only one pharmacologic treatment, duloxetine, with sufficient evidence supporting its use when the outcome is pain intensity.14 15 However, treatments for the myriad other symptoms associated with CIPN are unavailable.

Early, frequent and standardised assessment of CIPN is important to understand the development and evolution of symptomatology and to prevent its negative impacts.3 14 This may be achieved via use of Clinician-Reported Outcome Measures (ClinROM) or Patient-Reported Outcome Measures (PROM). ClinROMs are assessment tools and techniques used by clinicians of observable signs, behaviours or other manifestations of CIPN.16–19 PROMs allow patients to self-report the presence, intensity, frequency and/or the impact of symptoms and signs of the sensory, motor and/or autonomic features of CIPN. CIPN ClinROMs and PROMs must be valid, or measure what they purport to measure,20 reliable (ie, consistent), reproducible and stable across raters,21 and responsive to change following administration of the first and subsequent cycles of neurotoxic chemotherapy.19 Unfortunately, there is no agreement on a gold-standard CIPN assessment tool, preventing an adequate understanding of the magnitude of the problem of acute and chronic CIPN, its presentation, development and trajectory, risk factors and impacts.19 22 Importantly, this may also contribute to difficulty identifying effective preventative agents and treatments.14 22 23

It is also unclear whether CIPN tools have similar validity and reliability across neurotoxic chemotherapy types, during versus after chemotherapy delivery, or across the adult lifespan.3 For example, while there may be heterogeneity in CIPN experience due to biopsychosocial factors (eg, genetics, symptom appraisal),4 8 CIPN may also differ according to chemotherapy type with potentially different effects on symptom presentation.3 It may also be possible that the psychometric properties of the tools may differ depending on whether CIPN is measured during treatment (eg, acute symptom presentation) vs after treatment (eg, chronic symptom presentation). Additionally, ageing is accompanied by non-uniform biopsychosocial changes which can influence health and illness.24 In the case of CIPN, age-related morphological changes in the peripheral nervous system, including decreased nerve conduction velocity and slowed axonal regeneration and reinnervation,25–30 may interact with damage to the peripheral nervous system caused by neurotoxic chemotherapy,3 which may also result in differences in symptom presentation. This is important because cancer is primarily a disease of older people.31 32 Therefore, it may be possible that in the upcoming years, with the ageing population,33 there will be many older adults at risk of experiencing CIPN who will require valid and reliable assessment strategies that are able to differentiate normal age-related changes from those associated with CIPN.31 34

Although numerous reviews of CIPN ClinROMs and PROMs have been published,6 35–39 none to date has used methodology that allows for a quantitative assessment and comparison of the methodological quality of psychometric studies or the quality of the CIPN assessment tools, themselves. This is likely a key factor in our inability to identify a gold standard CIPN tool. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) Risk of Bias checklist was developed to quantitatively assess the methodological quality of studies reporting psychometric properties and the ClinROMs and PROMs evaluated in these studies.16 40 41 Validity, reliability and responsiveness are evaluated using 10 boxes that collect information about measurement properties, including measure development, content validity, structural validity, internal consistency, cross-cultural validity/measurement invariance, reliability, measurement error, criterion validity, hypothesis testing and responsiveness. Assessment criteria in each box depend on the type of analysis performed. The scoring of each question within a box can vary according the measurement propriety assessed.40 The rating scale includes four possible scores, ranging from very good, adequate, doubtful, to inadequate.40 The score for each box is calculated based on a ‘worst score counts’ method which only considers the lowest rating within the box. This rigorous method allows for an objective evaluation and quantitative scoring of the quality of studies that report psychometric properties of CIPN ClinROMs and PROMs and the tools themselves. No studies to date have applied the COSMIN methodology to assess the methodological quality of studies reporting psychometric properties of CIPN tools and the overall quality of ClinROMs and/or PROMs.

There are significant gaps in our knowledge about how to best assess CIPN. These knowledge gaps affect our capacity to relieve CIPN symptoms in all adults undergoing neurotoxic chemotherapy. A critical first step to improving our understanding of CIPN across the adult lifespan is to ensure that the assessment tools are valid and reliable. An assessment of the methodological quality of studies reporting psychometric properties of CIPN tools and the quality of ClinROMs and PROMs using the COSMIN method would help to identify gaps and weaknesses in their psychometric properties and potential areas for improvements to existing CIPN measures. Such an analysis would substantially improve the CIPN literature. From a research perspective, it would advance the field toward identification of a gold standard CIPN assessment method, which could have important implications for future trials testing novel preventive and treatment agents. From a clinical perspective, it would provide healthcare providers with valid and reliable assessment tools for early detection and monitoring of CIPN symptoms among younger and older adults undergoing all neurotoxic chemotherapies.

Objectives

The primary objective of this study is to perform a systematic review of studies reporting psychometric properties of CIPN ClinROMs and PROMs among adults receiving, or who had previously received chemotherapy for cancer and to compare COSMIN quality ratings across included studies and the quality of ClinROMs and PROMs, separately. The secondary objective is to perform subgroup analyses of studies by age group (younger adults (18–59 years ±10 years) vs older adults (≥60 years old ±10 years)), chemotherapy type and the timing of the study in relation to the delivery of chemotherapy (eg, during treatment vs after treatment), where data are available.

Methods and analysis

This study protocol was prepared with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) and PRISMA literature search extension (PRISMA-S) guidelines.42 43 The PRISMA-P checklist is available in online supplemental appendix 1.

Population: Adults aged over 18 years receiving or who had previously received chemotherapy for cancer.

Intervention(s), exposure(s): Any type of PROM or ClinROM used to assess CIPN.

Comparator(s)/control: Not applicable.

Main outcome(s): Establishing what is the most appropriate, valid and reliable CIPN PROM and ClinROM using the COSMIN methodology.

Additional outcome: Recommend of one or more ClinROM(s) and/or a PROM(s) for assessing CIPN in clinical and research settings using the Outcome Measures in Rheumatology (OMERACT) filter 2.1 methodology.

Eligibility criteria

All studies reporting on the psychometric proprieties of CIPN ClinROMs and/or PROMs among adults aged ≥18 years will be eligible for inclusion. There will be no restrictions on publication or year. Articles published in French will be translated to English by bilingual team members (PBM, LRG, DT and MEC).44 For articles written in other languages we will use Google Translate, as recommended.44 All types of cancer and all types of chemotherapy will be considered. Only empirical, peer-reviewed articles that report on measurement proprieties of CIPN ClinROMs and/or PROMs will be included. Case studies, study protocols, published conference abstracts and systematic reviews will be excluded. Studies reporting data from neonatal and/or paediatric populations combined with data from adult populations will be included if it is possible to isolate the data from the adult subgroup. If it is not possible, these studies will be excluded. Studies will be excluded when an adult proxy is used to assess CIPN symptoms of paediatric patients. Patients who do not have cancer or who do not receive any antineoplastic agents will be excluded. Any studies using animals as their population will also be excluded

Information sources

MEDLINE, Embase, CINAHL and APA PsycINFO databases will be searched from inception to 31 December 2021. Search terms were developed for (1) the population (adults receiving or who had previously received cancer treatment), (2) CIPN and (3) psychometric properties. The COSMIN search filter for PROMs and ClinROMs was used for the latter conceptual block. Search strategies were verified by a health science librarian with systematic review experience. References of all identified texts will be examined to identify additional studies not identified in the database search.

Search strategy

Search strategies are available in online supplemental appendix 2.

Data management

All references will be imported into Endnote V.20. Duplicates will be removed using Bramer’s deduplication method in Endnote.45 Then, references will be exported into Covidence in order to carry out the study selection.46

Study selection process

A two-stage process for study screening will be used. Two independent raters (PBM and LAR) will screen titles and abstracts of articles to assess whether they meet the eligibility criteria. Full-text screening will be conducted by the two independent raters (PBM and KM). The number of included and excluded studies at each step will be presented in a PRISMA flow chart. Reasons for the exclusion of full-text articles will be recorded in Covidence and presented in the PRISMA flow chart. A third rater (LRG) will resolve all disagreements at each screening phase. Percentage agreement between raters will be calculated at each phase to report interrater reliability.47

Data collection process

For each study included in the review, data relating to study characteristics (target population, mode of administration, recall period, subscale(s) and number of items, response options, range of scores, original language and available translation) and measurement properties of the PROMs and the ClinROMs (instrument development, content validity, structural validity, internal consistency, cross-cultural validity/measurement invariance, reliability, measurement error, criterion validity, hypothesis testing for the construct validity and responsiveness) will be extracted by two people and compared for consistency (PBM and MEC). CIPN definitions reported by the authors will also be extracted to examine and compare conceptual definitions across studies. Data extraction tables will be adapted from those designed by the COSMIN group. Corresponding authors will be contacted in the event of missing data (descriptive data, missing information about the items and the scoring of each tool). Three contact attempts will be made over the period of 1 month. In the case of non-response, data will be coded as missing.

Quality assessment

The COSMIN Risk of Bias checklist will be used by two independent raters (PBM, MEC) to assess the methodological quality of the included studies and the quality of ClinROMs and PROMs.41 The first step consists of assessing the quality of each study describing psychometric properties of CIPN ClinROMs and PROMs using an Excel spreadsheet proposed by the COSMIN group.48 In this step, each of the 10 measurement properties considered by the COSMIN system is given four possible scores: very good, adequate, doubtful and inadequate.48 A summary table which presents the quality rating of each study will be included in this review once the COSMIN Risk of Bias checklist is completed. The second step involves rating each study describing psychometric properties of CIPN ClinROMs and PROMs against the criteria for good measurement properties.48 Each rating has a possible score of sufficient (+), insufficient (−), inconsistent (±) or indeterminate rating (?) according to published criteria.48 The third step involves determining an overall quality score of all studies reporting on a given ClinROM or PROM by considering cross-study ratings.48 The quality of evidence is determined by using a modified Grades of Recommendations, Assessment, Development and Evaluation approach with a possibility of 4 grades: high, moderate, low, very low.48 Psychometric data from ClinROMs will be considered separately from psychometric data from PROMs. We will rely on data for steps 1 and 2 from a recently completed systematic review and COSMIN analysis of CIPN PROMs (PROSPERO 2020 CRD42020210405; Li, Park & Rutherford, Manuscript in progress) for any studies overlapping across the two reviews.

Subgroup analysis

The COSMIN Risk of Bias checklist will be completed and analysed to compare methodological quality ratings from studies reporting psychometric properties of CIPN tools and the quality of ClinROMs and PROMs across age groups (younger adults (18–59 years ±10 years) vs older adults (≥60 years old ±10 years)), the type of chemotherapy received, and the timing of the study in relation to the delivery of chemotherapy (eg, during treatment vs after treatment), where data are available. Age subgroups are based on cut-offs used in previous studies in cancer with clinically relevant outcomes.49–51 We will allow for the cut-off between younger and older groups to vary by 20 years to account for methodological differences in the categorisation of younger and older age groups across studies.

Recommendation of a ClinROM and PROM

While the COSMIN method is the only available method to quantitatively assess the methodological quality of studies reporting psychometric properties and the ClinROMs and PROMs, it does not provide a method to recommend the use of a ClinROMs or PROMs. In this review, we will address this limitation by using the OMERACT filter 2.1, based on the findings of the COSMIN analysis, to determine whether it is possible to recommend the use of one or more ClinROMs or PROMs, and if so, make recommendations.52 53 Data issued by COSMIN on construct validity, test-retest reliability and longitudinal construct validity (ie, responsiveness) and additional information on clinical trial discrimination and thresholds of meaning which are not covered in COSMIN will be used to complete the COSMIN-OMERACT Good Measures Checklist by two independent raters.52 This methodology permits recommendations based on a standardised, rigorous and transparent process.52 Although this methodology has not been used in systematic reviews of the psychometric properties of CIPN measures, two previous reviews have used the OMERACT filter 2.1 in conjunction with the results of a COSMIN analysis to make recommendations about dermatological measures.54 55

The OMERACT filter 2.1 (table 1) has three pillars, four questions, seven measurement properties and one answer.52 Truth, discrimination and feasibility represent the pillars.52 A set of questions are associated with each pillar which are organised in an algorithm.52 Each question has a possibility of 4 scores: Red, amber, green and white.52 Red means ‘stop, do not continue’, amber means ‘a caution is raised but you can continue’, green means ‘go, this question is definitely answered affirmatively’, and white means an absence of evidence and the evaluation must stop.52 An overall score that ranges between do not endorse, provisionally endorse, or endorse is issued once the four questions of the algorithm are answered.52 The seven included measurement properties map onto seven of the COSMIN measurement properties. We will adapt the OMERACT filter 2.1 to include all 10 COSMIN Risk of bias checklist measurement properties.

Table 1

OMERACT filter 2.1 definitions, questions and measurement properties

Pillars	Definition	Questions	Measurement properties
Truth	The ability of the outcome measurement tool to measure what is intended52	‘Is it a match with the target domain?’	Construct validity, reliability
		‘Do the numeric scores make sense?’	Content validity, face validity
Discrimination	The ability of the outcome measurement tool to discriminate different situations of interest52	‘Can it discriminate between groups of interest?’	Test–retest reliability, longitudinal validity/responsiveness, ability to discriminate in Randomized Controlled Trial (RCT)/comparative research setting, threshold of meaning
Feasibility	The practicality of the outcome measurement tool (time, cost, burden)52	‘Is it practical to use?’	Access, training, translation, length, cost, burden

OMERACT, Outcome Measures in Rheumatology.

Data synthesis and best-evidence synthesis

The synthesis without meta-analysis (SWiM) guideline will be used to guide data synthesis for the included studies.56 SWiM includes nine items that report key features in the methods, results and discussion of every study, such as grouping studies for synthesis (item 1), standardised metric used for synthesis (item 2), synthesis methods and their limitations (items 3 and 9), criteria used to prioritise results for summary and synthesis (item 4), heterogeneity in reported effects (item 5), certainty of evidence (item 6), data presentation methods (item 7) and a summary of the synthesis (item 8).56

Patient and public involvement

A patient partner and coauthor (MB) participated in the development of the review protocol and will participate as a research team member throughout the review process, contributing to interpretation of the findings, manuscript drafting and revisions.

Ethics and dissemination

An ethical approval is not necessary to be obtained for this systematic review protocol. The results will be disseminated to clinicians and researchers by publication in a peer-reviewed journal and by presenting at relevant conferences. This systematic review will support researchers and clinicians to use the best measure to assess CIPN.

Discussion

The planned study will be the most up-to-date systematic review of CIPN ClinROMs and PROMs and the only one to use the COSMIN risk of bias tool with CIPN ClinROMs, allowing for a quantitative evaluation and comparison of the methodological quality of studies reporting psychometric properties of CIPN tools and the quality of ClinROMs and PROMs. Subgroup analyses by age group, chemotherapy type and the timing of the study in relation to the delivery of chemotherapy will be carried out to determine whether the psychometric proprieties of CIPN ClinROMs and PROMs are the same across clinically relevant subgroups. This systematic review will also be the first to use the OMERACT filter 2.1 methodology to facilitate the recommendation of one or more ClinROMs and/or PROMs for assessing CIPN in clinical and research settings, based on the available evidence.

Limitations may be related to the heterogeneity of patients who have CIPN and to the lack of consensus about the definition of CIPN. High heterogeneity in the experience of CIPN may be due to multiple factors, (eg, biological, the prescribed chemotherapy regimen and the timing of development of CIPN and its manifestations over time). Our planned subgroup analyses may help to mitigate this limitation. We will also analyse CIPN definitions used across studies to examine conceptual definitions, and to compare similarities and differences across studies and how this might contribute to any observed heterogeneity. In addition, the COSMIN methodology does not currently consider all types of measurement like patient-reported experience measures (PREM). PREMs aim to explore the patient’s experience of care from their own perspective.57 This limitation may have an impact on the identification of a tool that could improve patient health outcomes and the quality of care for those undergoing chemotherapy or receiving CIPN treatment and the economic impact of CIPN.57 58 Additional research is needed to identify and evaluate the quality of CIPN PREMs. Expanding the COSMIN methodology to include PREMs would be an important future research direction.