Validation and application of a core set of patient-relevant outcome domains to assess the effectiveness of multimodal pain therapy (VAPAIN): a study protocol.

INTRODUCTION: Multimodal pain therapy (MPT) has been established accounting for biopsychosocial consideration in diagnostic and therapy. MPT seems to be effective, but comparability of studies is limited due to diversity of study designs and outcome measurements. The presented study aims to develop a core outcome set consisting of a minimum of outcome measures deemed necessary for medical and therapeutic decision-making, which must be measured in all clinical trials and non-randomised intervention studies. METHODS AND ANALYSIS: The study consists of several parts. First, the development and recommendation of preliminary core outcome domains will be based on results of a systematic review and structured online surveys. Participants of the expert panel are representatives of methodological, medical, physiotherapeutic, psychotherapeutic profession and patients suffering from chronic pain (n=25). Subsequently, candidate instruments to measure preliminary core outcome domains will be recommended by these experts. Therefore, systematic reviews on measurement properties of preliminary outcome measures will be conducted and finalised in a consensus meeting. Consented instruments and lacking psychometric properties of relevant instruments will be addressed and validated in the following part, a prospective multicentre study in multimodal pain centres on approximately 300 patients with chronic pain. Based on all previous results, a core outcome set for MPT measured in effectiveness studies and daily recordkeeping will be finalised by consensus. Statistical analyses will be performed according to methodological standards (COSMIN). ETHICS AND DISSEMINATION: The methods and procedure of the study are developed in compliance with the ethical principles of the Helsinki Declaration and Good Epidemiologic Practice. Recruitment of study participants will require approval of the study by the responsible ethics committee and signed informed consent from each participant. Pseudonymised data will be used for statistical analysis. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Introduction

In Europe, approximately 11 million patients suffer from chronic pain,1 which is defined as recurrent or permanent pain of at least 3 months with considerable impairment of physical and psychological functioning. Pain management approaches underwent a significant change of paradigm during the past decades. Usually common monomodal therapy is focused on single treatment approaches such as pharmacological treatment, physical therapy or invasive therapy. Mayer and Gatchel2 first developed a multiprofessional treatment approach in pain management consisting of physical exercises and a behavioral-psychological approach (functional restoration, FR). Based on this initial description of multidisciplinary pain management, a task force of the German IASP Chapter defined multimodal pain therapy (MPT) as the simultaneous, contextual, temporal and coordinated, comprehensive strategy to treat chronic pain patients. It incorporates different somatic, functional and psychological treatment approaches with identical aims of the therapeutic team.3 The improvement of objective and subjective function of the patient with increased controllability and the sense of competence is the central aim.3 There is convincing evidence of effectiveness of both programmes, FR and MPT2 4–7 in chronic pain so far. However, the comparability of those studies is limited due to the diversity of study design and outcome measurement methodology. The current lack of standardised and valid outcome measures across trials is a barrier to perform meta-analyses on the efficacy and effectiveness of MPT/FR and thus to practice evidence-based pain management. Despite ongoing discussions concerning appropriate outcome measures of pain therapy during the past years,8–10 a multiperspective consensus on the most relevant outcomes for MPT/FR is lacking.

The development of a core outcome set (COS) has been pioneered by OMERACT (Outcome Measures in Rheumatology11). Since then, outcome domains have recently been defined for different medical conditions such as chronic pain in general, eczema, psoriatic arthritis and ankylosing spondylitis.10 12–14 Despite ambitions to prepare guidance on the development of COS, there is limited standardised instruction in the literature regarding methodological approaches.15 16

It is observable that the development of a COS begins frequently with a systematic review of all outcomes reported in clinical trials and a subsequent multidisciplinary expert panel to vote for relevant outcome domains which should be assessed in clinical trials. Less common, recommendations for valid and reliable measurement instruments for each suggested outcome domain are described.17

As defined by Boers et al,18 outcome measures should adequately meet the criteria of truth (ie, validity; measure what they intend to measure), discrimination (ie, reliability and sensitivity to change; discriminate between situations) and feasibility (ie, be applied and interpreted easily) in order to be meaningful and relevant. According to the notion that studies are only as credible as their outcome measures18 and that measures have to be validated on target population,19 there is a high need to clarify the relevance and validity of outcome measures for MPT/FR.

With the initiative of IMMPACT (initiative on methods, measurement and pain assessment in clinical trials) the discussion about COS in the therapy of chronic pain has been established following the example of OMERACT.9 IMMPACT (a group of clinicians, researchers, representatives of a patient self-help organisation, of governmental agencies and pharmaceutical industry) defined COS for all forms of chronic pain therapies but focused on medication and clinical trials,9 not on comprehensive therapy approaches such as MPT/FR. Another initiative has done similar work considering low back pain.20 Multidimensional tools such as TOPS (Treatment Outcomes in Pain Survey instruments21) or the standard questionnaire of the German Pain Society22 have been developed but those processes were not consensus driven. Further to ensure translation from research into health provision, it seems to be necessary to have a specific amount of domains which cover COS in both settings, clinical trials and daily recordkeeping. This should help to clarify the differences between controlled and natural therapy condition and therapy outcome.

After defining important outcome domains by patients with chronic pain23 it became obvious, that these domains did not match with the core set of outcome domains originally defined by IMMPACT.10 This gap between clinicians/researchers’ opinion and the opinion of the patients’ has not been closed yet, nor has been conducted a content validation of the COS on target population in general (chronic pain patients). According to COSMIN,19 a COS has to be validated in target population and in (therapy) setting as well. Therefore, there are two main reasons to validate a COS in MPT/FR

MPT/FR patients have a long experience of ongoing pain with many unsuccessful therapeutic approaches. Pain has generalised and interferes with most of live domains.

Primary aim of MPT/FR is not reduction of pain in first place but focuses on general improvement of physical and psychological functioning.

The lack of validation in general, the existing gap between patient perspective on COS in chronic pain trials and the special character of MPT/FR do not allow to just take over any COS without investigation. Therefore, the VAPAIN-initiative tries to enlighten these questions in the setting of MPT/FR and under the perspective of content validity by both patients and clinicians.

Aims of the study

We propose an international, multiprofessional evidence-based and evidence building consensus study to define patient-relevant COS measurements for MPT/FR which includes systematic reviews and systematic procedures (consensus) together with inclusion of patient groups to enhance quality and acceptance of results and recommendations.

The overall aim of this study is to develop an evidence-based consensus on a COS measurement tool for MPT/FR to assess the effectiveness of interventions and to be used in daily recordkeeping.

To achieve the overall aim, the following specific objectives need to be answered successively:

Development and recommendation of a preliminary core set of outcome domains for effectiveness studies and daily recordkeeping,

Definition of specific reliable and valid instruments to measure the preliminary developed and recommended domains for evaluation of the effectiveness of MPT/FR,

Validating the preliminary core set of outcome domains and core set of measurement instruments in case of lacking information concerning psychometric properties in a multicentre study in the setting of MPT/FR,

Finalisation of core set of outcome domains inclusive instruments and additional evaluation of further questions.

Methods and analysis

Given the complexity of the four specific aims, a multistep process using a multimethod approach is necessary. The steps of the study are shown in figure 1. The process of achieving consensus on the final COS is presented in figure 2.

Figure 1

Presentation of the multimethod approach of VAPAIN and chronology of the several steps. COS, core outcome set; FR, functional restoration; MPT, multimodal pain therapy.

Figure 2

Process of consensus and criteria of decision to include outcome domains and measurement instruments into COS. COS, core outcome set; MPT, multimodal pain therapy.

Step 1: Aim—development and recommendation of a preliminary core set of outcome domains

To achieve this aim, a systematic review on effectiveness studies using established evidence-based methods24 and a subsequent online survey will be conducted.

Methods

Systematic review on published outcomes in effectiveness studies of MPT/FR in chronic pain

To systematically search for incorporated domains in randomised controlled trials and longitudinal non-randomised studies in the matter of MPT/FR, the following inclusion criteria will be applied:

Population: chronic pain of at least 3 -month duration; patients of all ages;

Intervention: MPT/FR/comprehensive pain therapy/rehabilitation;

Comparison: not applicable;

Outcome: all outcomes (syn.: end points) to be measured in assessment of chronic pain as well as applied instruments. Study design: clinical trials and longitudinal non-randomised studies with at least n=15;

Publication type: full-text articles published in English or German language.

Literature search: The systematic literature search will be performed in MEDLINE, EMBASE and AMED. The electronic search will be completed by hand search of articles cited in thematically relevant reviews.

Screening and data extraction: Screening of titles/abstracts as well as eligible full texts will be done independently by two researchers. All disagreements will be resolved in consensus meetings. Study characteristics and outcomes will be extracted by using standardised evidence tables. Outcome domains which are mentioned in at least 10% of the included studies will be extracted.

Critical appraisal: This first systematic review will focus solely on broad information of used and published outcome domains in effectiveness studies of MPT/FR in chronic pain. The function of this review will be exploratory and structuring only. Appraisal of study quality and of outcome or effectiveness of MPT/FR is not intended.

Online consensus on preliminary COS to assess effectiveness of MPT/FR in chronic pain

In relation to the derived literature-based outcome domains, an online survey will be conducted to recommend important outcome domains for a COS.

Participants: In order to allow multiprofessional collaboration and to represent different stakeholder groups, the complete consensus process with structured iterative discussion and consensus voting will consist of 25 experts with methodological, clinical, medical, functional and psychological expertise as well as patient representatives.

Study procedures: Important pain societies (eg, European Federation of IASP-Chapters, International Association for the Study of Pain) and stakeholder groups will be invited to nominate experts of outcome measurement in general and of outcome measurement in chronic pain in particular to participate.

All two rounds of the preliminary consensus process will be performed online.

First round: Based on the above described systematic review, panel members will be asked to rate the importance of the patient-relevant outcome domains to assess the effectiveness of MPT/FR on a nine-point Likert scale in the context of (1) effectiveness studies and (2) daily recordkeeping. Scores of 1–3 will represent a region where participants believe the domain is not important; 4–6 a region of equivocal value; and 7–9 a region where they feel the domain is important.25 Participants will be asked to list additional outcome domains they consider as potentially relevant to include into the COS, and additionally name the number of lowest/highest reasonable amount of domains included in the COS.

Second round: Participants will receive feedback on their own response along with the group opinion for each domain from the previous round. Respondents will be able to submit new scores or leave their scores unchanged and also rate domains which were added by participants in the first round. Further, they are asked to order domains from highest on to lowest interest to be included in the COS for effectiveness studies and daily recordkeeping.

Definition of preliminary consensus)

Online survey round 1: Consensus that an outcome domain is important for MPT/FR and shall be discussed as potential core outcome will be defined as a median score of ≥7 (see figure 2). The amount of minimum number of necessary outcome domains in COS in clinical trials resp. daily recordkeeping will be estimated by median.

Online survey round 2: Preliminary consensus that an outcome domain is supposed to be part of the COS will be defined by frequency of announcing by participants voting ‘yes/no’.

Output

The output of this consensus part is a preliminary consensus of a core set of outcome domains for MPT/FR in (1) effectiveness studies and (2) daily recordkeeping.

Step 2: Aim—recommendation of candidate instruments to measure the preliminary core outcome domains from step 1

Systematic reviews on measurement properties of preliminary COS

The objectives of this review are to systematically assess psychometric properties of instruments to measure the previously defined core outcome domains. Articles in which a relevant scale depending on the particular domain (see step 1) was published first (inauguration articles) and articles which subsequently investigate the measurement properties of relevant scales are eligible. To be included into the systematic review following inclusion criteria will be applied.

Population: chronic pain of at least 3 -month duration; patients of all ages

Intervention: not applicable

Comparison: not applicable

Outcome: all outcome domains rated as core outcome domains in step 2

Study design: validation studies (with at least one of the following measurement properties: content validity, internal consistency, reproducibility (test-retest reliability, inter-observer reliability), responsiveness/sensitivity to change, interpretability and acceptability/ease of use)

Publication type: full-text articles published in English or German language

Literature search: The systematic literature search will be performed within our review team in MEDLINE and EMBASE from inception until 15 July 2014. The precise PubMed search filter for finding studies on measurement properties developed by Terwee et al26 will be used to identify relevant articles.

Study selection and data abstraction: Screening of titles/abstracts as well as eligible full texts will be done independently by at least two reviewers. All disagreements will be resolved in consensus meetings. Study characteristics will be extracted independently by using standardised evidence tables. Evidence tables will include: reference, geographical location, setting, study type, key characteristics of study paricipants, name of scale, domains and items measured.

Methodological assessment: The COSMIN checklist27 28 will be used to evaluate the methodological quality of included studies.

Rating of scale quality: The predefined criteria of rating scale quality are in accordance with the recommendations of the COSMIN group29 and the OMERACT filter.31

Generating recommendations for the use of identified instruments: For each instrument identified in the review, a standardised recommendation for usage or required future validation work will be made depending on the scale quality and on the methodological quality of included studies.

Consensus meeting on (candidate) instruments

Presence meeting: A consensus conference for group discussion and final voting will be scheduled in Dresden (Germany) and all participants of step 1 will be invited. In this final round, according to the results of the online surveys, the experts vote for domains being part of the COS. Discussion of results will be moderated by an experienced moderator. Further resulting from the systematic reviews from step 2, the experts will be asked to discuss and recommend particular instruments for specific domains. At the end of the meeting, recommendations on a feasible amount of domains and specific instruments to evaluate effectiveness/daily recordkeeping of MPT/FR are expected. Additionally, recommendations on important gaps considering psychometric properties of instruments and questions to be clarified by the following study/further research will be given.

Definition of consensus: Consensus that a domain/instrument is important for MPT/FR will be defined by at least 70% of all members voting ‘yes’ and not more than 20% voting ‘no’.

Output of step 2 is a consensus on core domains and instruments. We further expect the nomination of candidate instruments which shall be addressed for further questions of research to assess their psychometric properties during the next phase of the study—the validation of the set of instruments in a multicentre study.

Step 3: Aim—validation of a preliminary COS of measurement instruments to assess effectiveness of MPT/FR in chronic pain from step 2

Prospective multicenter study in four German comprehensive pain centers and focus groups

The core set of outcome domains and measurement instruments recommended by the final voting of expert panel as described at the end of step 2 will be applied and further investigated according to validity, reliability, responsiveness and interpretability as highly recommended psychometric properties by COSMIN19 in the setting of MPT/FR. Instruments lacking of at least one of the relevant psychometric properties or which have not been investigated in the setting of MPT/FR will be taken into step 3. To ensure complete content validity additionally to the quantitative part of validation via questionnaire, a qualitative part will be conducted. The validation part will be registered as complete study protocol.

Study population: All patients fulfilling the following inclusion criteria: age ≥18 years, chronic permanent or recurrent pain for at least 3 months, medium or high impairment of physical and psychological functioning, sufficient capability to tolerate physical activation, and sufficient skills in German language assessed by an interdisciplinary team in one of four cooperating German comprehensive pain centre. Further to be included into the qualitative validation study (focus groups), patients have to have completed a MPT/FR programme and have to be discharged at least for 3 months and not more than 9 months.

Quantitative validation study

Sample size calculation: The prospective cohort study will be exploratory and the choice of outcome measures is yet to be determined. Such a situation does not apply to typical sample size calculations, but we expect a required sample size of about 190 patients (expected effect sizes of f2=0.15 (medium effect as usually expected for criterion of clinical important amount of change) for multiple regression analyses under the condition of α=0.05 and 1−β=0.80 (as commonly defined)). Considering a dropout from at least 20% and missing data from at least 20%, we will recruit a total number of 270 patients.

Procedure: All consecutive patients eligible for MPT/FR will be asked to participate in the prospective cohort study at admission in one of the four participating centres (Mainz, Erlangen, Dresden, Leipzig). Study participants will be recruited over a 1-year period and followed for 9 months. Data collection by means of patient and physician questionnaires will be done at baseline (admission), discharge, booster (a repetition therapy after some weeks off therapy) and at 9 months follow-up. The patients’ questionnaire will cover information on sociodemographic characteristics and the patient-reported outcome measures as defined in steps 1 and 2 of the project. The physicians’ questionnaire will cover information on physical, functional and psychological status of the patient, the patients’ history of chronic pain and previous and current treatments.

The decision of variables matching the criterion of good quality27 will be supported by multitrait-multimethod approach.30 Data collection will be performed by each participating centre from admission to follow-up.

Analysis: In accordance with the recommendations of the COSMIN group,19 the following properties of outcome measures will be considered: validation (content), reliability (internal consistency, test-retest), interpretability and responsiveness.

Aspects of validity and interpretability will be estimated by several forms of regression analyses (multiple, hierarchical, etc) and correlation matrix to picture multitrait-multimethod matrix.30 Internal consistency will be performed via Cronbach's α,19 test-retest reliability, if necessary, via correlation analysis. Responsiveness/sensitivity to change will be investigated by the construct approach recommended by De Vet et al19 for studies with no gold standard in measurement of changes by estimating correlations of change between two similar instruments according to specific hypothesis. Concerning the effect sizes, f2 for regression and determination coefficient for correlation will be estimated. If there is overlap in variables, there will be an estimation of semi partial correlations and R2 change in regression.

Methods against bias

Reduction of recall bias: We chose a prospective design to prevent information (recall) bias. The use of validated assessment tools and continuous monitoring/data quality assurance will prevent other sorts of information bias.

Reduction of selection bias: The inclusion of consecutive patients at four multimodal pain centres will reduce selection bias. A detailed documentation of denial or willingness to participate in the study by the patients enables to control differences of these groups. Possible differences between those who fill in the complete study and those who miss at least one of the assessments will be controlled.

Reduction of interpretation bias: Gender, education, age and the degree of chronicity will be controlled (adjusted) and considered during analyses (by means of stratified analysis to explore the presence of interaction/effect modification).

Missings: Quality assurance measures (training of staff and monitoring) will be undertaken to assure high quality of data and to minimise the occurrence of missing data. Missing outcome data in follow-up visits will be assumed to be missing at random. Based on our previous experiences, the follow-up rate is expected to be at least 80%. In case of missing questionnaires, patients will be reminded once per letter, later per telephone call. Missings will be declared as two missed items in one instrument; replacing one missing item of an instrument by mean or median depends on the instructions of the manual.

Qualitative validation study

Because of the nature of quantitative approaches regarding patient’s perspective on content validity and usability of the instruments, there will be an additional qualitative validation realised by focus groups (moderated group discussions, concerning issues of content validity, responsiveness and usability of candidate/core instruments in the presented study). There will be eight focus groups (two in each of the participating centres) all consisting of eight patients, a moderator and a transcript writer. In order to make the different focus groups comparable, the moderation is guided by preassigned central questions. The output of the discussion will be visualised on presentation boards and summarised in written forms. Both steps of documentation will be validated by the members of the focus group to ensure internal validity. The results of these groups will support interpretability of the results of the quantitative study and close the gap between clinicians’ and patients’ perspectives.

Evidence of validity and reliability of core set of outcome domains and measurement instruments in MPT/FR will be the result of step 3. New issues of use and psychometric properties of measurement instruments will be identified and shall be addressed in the following step 4, the finalisation of the COS of domains and measurement instruments. The perspective of patients will be included to ensure overlap between experts’ and patients’ opinion about important outcome measurement of effectiveness of MPT/FR.

Step 4: Aim—finalisation of the COS of effectiveness and daily recordkeeping in MPT/FR by consensus

After completing the study and assembling of all concerning results, a finalisation by the panel members will be held to approve the preliminary COS or/and to develop new questions on specific issues arising from the process.

Ethics and dissemination

Ethical and legal considerations

The methods and procedure of the study are developed in compliance with the ethical principles of the Helsinki Declaration and Good Epidemiologic Practice. Recruitment of study participants will require approval of the study by the responsible ethics committee and signed informed consent from each participant. Only pseudonymised data will be collected in case report forms, used in data management and throughout analyses. The study has been approved by the local ethical committee (EK 105032015).

Qualitative assurance and safety

Pseudonymised data will be used for statistical analysis. To ensure quality of study and analyses, there will be an advisory board consisting of at least six experts of each subject (psychological, medical, functional, methodological and consumer, please see online supplementary appendix 1) who are not involved into the process of study. Meetings will be held at least three times during the complete study. Manual plausibility checks and statistical analyses on data consistency will be done to ensure data validity.

Use and implementation of the results

Because of the multiprofessional nature of the study including different stakeholders (physicians, psychotherapists, physiotherapists, methodologists, consumers) and recruiting centres, we are confident that the research results will be accepted and implemented in routine care. Further, it will enable comparability between facilities and study results and therefore enhance decision-making for healthcare providers.

Time frame

Start of part 1 was 1 January 2014 (see figure 1). Conduction of the consensus process will start in April 2014. The final meeting is scheduled for November 2014. Ethical commitments of the validation study in part 2 will be finished by December 2014/January 2015. Start of recruitment for validation study will be in January/February 2015. Final follow-up questionnaires will be collected until June 2016 and end of study will be at the end of 2016 (tables 1 and 2).

Table 1

Abbreviations

COS	Core outcome set
COMET	Core outcome measures in effectiveness trialshttp://www.comet-initiative.org/
COSMIN	Consensus-based standards for the selection of health measurement instrumentshttp://www.cosmin.nl/
FR	Functional restoration
IASP	International association for the study of painhttp://www.iasp-pain.org/
IMMPACT	Initiative on methods, measurement, and pain assessment in clinical trialshttp://immpact.org/
MPT	Multimodal pain therapySynonymous: interdisciplinary, comprehensive
OMERACT	Outcome Measures in Rheumatologyhttp://www.omeract.org/
RCT	Randomised controlled trial
TOPS	Treatment outcomes in pain survey instruments

Table 2

Glossary

Term	Definition
Outcome	An outcome is any identified result in a domain arising from exposure to a causal factor or a health intervention31
Domain	A concept to be measured, a further specification of an aspect of health (such as health related quality of life, illness impact, pain intensity)31
Core domain set	For studies of health interventions, the minimum set of domains and subdomains necessary to adequately cover all relevant concepts of a specific health condition within a specified setting. Describes what to measure31
Core outcome measurement set	The minimum set of outcome measurement instruments that must be administered in each intervention study of a certain health condition within a specified setting to adequately cover a corresponding core domain set. Describes how to measure31
Consensus (methods)	Consensus methods derive quantitative estimates through qualitative approaches and aim to determine the extent to which experts or lay people agree about a given issue. They consist of following features: anonymity, iteration, controlled feedback, statistical group response25
MPT	MPT is defined as a simultaneous, contextual, temporal and coordinated comprehensive strategy to treat chronic pain patients integrating different somatic and physical, as well as psychological, treatment approaches provided by all relevant professions (physician, psychotherapist, physiotherapist, nurses, co therapist) with identical and consensual therapeutic aims3
	FR consists of an ‘sports medicine’ approach to restore physical capacity and a cognitive ‘crisis intervention’ technique for dealing with psychosocial issues in the patient suffering from chronic (spinal) disability. FR teams consist of physical, occupational, and psychological therapists, nurses, and physicians2

FR, functional restoration; MPT, multimodal pain therapy.