Performance and quality indicators for the management of non-cancer chronic pain: a scoping review protocol.

INTRODUCTION: Chronic pain is a public health problem of epidemic proportion in most countries with important physical, psychological, social and economic consequences. The management of chronic pain is complex and requires an integrated network approach between all levels of the healthcare system and the involvement of several health professionals from different disciplines. Measuring the performance of organisations that provide care to individuals with chronic pain is essential to improve quality of care and requires the use of relevant performance and quality indicators. A scoping review methodology will be used to synthesise the evidence on performance and quality indicators developed for non-cancer chronic pain management across the continuum of care. METHODS AND ANALYSIS: The following electronic databases will be searched from 2000 onwards: Cochrane Effective Practice and Organisation of Care (EPOC) Review Group Specialised Register; Cochrane Library; EMBASE; PubMed; CINAHL; PsycINFO; ProQuest Dissertations and Theses. All types of studies will be included if these are concerned with performance or quality indicators in adults with chronic non-cancer pain. In addition, searches will be conducted on provincial, national and international health organisations as well as health professional and scientific associations' websites. A qualitative descriptive approach will be used to describe characteristics of each indicator. All identified indicators will be classified according to dimensions covered by Donabedian and the Triple Aim frameworks. ETHICS AND DISSEMINATION: The scoping review findings will inform the development of a performance measurement system comprising a list of performance indicators with their level of evidence which can be used by stakeholders to evaluate the quality of care for individuals with chronic non-cancer pain at the patient, institutional and system level. The results will be disseminated via several knowledge translation strategies, including 2 stakeholder meetings, publication and presentation at conferences. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

This study protocol will provide needed information to support chronic pain quality improvement initiatives.

Stakeholders, who will be the knowledge users of the study results, will be actively involved in the study.

Identifying quality improvement domains where developed indicators are lacking has the potential to improve patient care.

No formal assessment of included studies using standardised tools.

It may be challenging to identify indicators across levels of care and covering all non-cancer chronic pain conditions.

Introduction

Chronic pain (CP) is a public health problem of epidemic proportion in most countries.1 In fact, CP is very prevalent with estimates in the adult population varying from 11.5% to 55.2% internationally.2 3 Although CP is recognised to be a global health problem, management of CP is distributed unequally across the population and globally, with evidence that people living in adverse socioeconomic circumstances are far from receiving optimal evaluation, treatment and care.4 Furthermore, racial and ethnic minorities consistently receive less adequate treatment for CP than the white population.5 There are also substantial gender differences in the clinical experience of CP as women are substantially at greater risk for many clinical pain conditions.6

CP is recognised as a multidimensional and complex phenomenon that may have severe consequences on the physical, psychological, social and economic dimensions of the lives of sufferers and families.2 Indeed, the unpredictable fluctuation of the intensity of pain has a significant impact on the family and social functioning of the individual. Persons suffering from CP reported greater tensions and breakdown of conjugal relationships as compared with healthy individuals.7 8 Regarding social functioning, CP sufferers consistently report avoidance and withdrawal from social contact that may lead to additional psychological distress such as feelings of frustration, anger, guilt and despair.7 8 Other negative consequences of CP include reduced quality of life, increased rates of depression and an increased risk of suicide.1 9 Moreover, CP can lead to job loss or reduced work responsibilities, and to a significant decrease in productivity.1 10 For example, in Canada, the direct healthcare costs were estimated to be more than $6 billion a year, and productivity costs related to job loss and sick days were $37 billion per year.10 In the USA, the direct and indirect costs related to CP are estimated to be over $100 billion annually.11 Furthermore, prevalence of CP increases with age2 3 and demographic research suggests that CP conditions will become even more of a health problem and socioeconomic burden in the coming years since the population is ageing and consequently, usage of the healthcare system will increase.1

CP is defined as ‘pain that has persisted beyond the normal tissue healing time, usually taken to be 3 months’.12 There are many categories of CP, including musculoskeletal, neuropathic, headaches and ‘other’, with many subtypes within these groups.13 Patients with CP require a multidimensional therapeutic approach that provides simultaneous assessment and management of somatic, behavioural and psychosocial components of CP that interact in a complex manner that results in a particular expression of CP in each individual.2 3 Consequently, the type, intensity, frequency and prognosis varies greatly among persons with CP. Hence, given the complexity of CP management, patients receive care at all levels of the healthcare system (primary, secondary and tertiary) and require the involvement of many health professionals from different disciplines (anaesthesiologists, pain specialists, nurses, psychologists, physiotherapists, etc).13 14

There is a trend in Canada10 and elsewhere15 to organise care around an integrated continuum of services in which patients progressively receive more complex and specialised care according to need, and in a coordinated manner between primary, secondary and tertiary providers. In an effort to improve quality of care, measuring the performance of organisations providing care to patients with CP is essential to evaluate the extent to which intended results have been achieved, and to create a more efficient and effective healthcare system.16 Implementation of performance and quality indicators (QIs) can be used by organisations and policymakers to assess the gap between actual and targeted performance in order to improve outcomes, increase health system accountability, and reduce the gap between actual and optimal patterns of care17 18 based on evidence-based guidelines for CP. Understanding how indicators may or may not function across different sociodemographic subgroups is also an important consideration to reduce disparities in care. The identification of existing indicators is the first step for the development of a performance measurement system comprising a core set of reliable, valid, useful and actionable indicators.19 20

As part of the provincial action plan of the centres of expertise in CP in Quebec, Canada, the clinicians, administrators and decision-makers want to define QIs to evaluate and improve quality of care for CP across the continuum of care. Therefore, the purpose of this scoping review is to synthesise the evidence on performance and QIs developed for non-cancer CP management regardless of country, clinical setting or level of care (primary, secondary or tertiary care). More specifically, we aim to (1) identify performance or QIs developed for CP care; (2) map the dimensions covered by these indicators according to the conceptual frameworks we will use (see Conceptual framework section for description) in order to identify the quality domains which have benefited from more attention or those that have been undervalued; (3) examine the evidence base in support of these indicators; and (4) produce recommendations about the use of these indicators when sufficient evidence exists and identify the domains where developed indicators are lacking.

Conceptual frameworks

Donabedian's framework (2005) for examining health services and evaluating quality of care, and the Triple Aim framework21 will be used as a guide for synthesising the literature and determining which dimensions of performance were favoured and which lack developed indicators. In Donabedian's framework, the three components of healthcare quality are structure, process and outcome. The structure includes all the factors that affect the context in which care is delivered and includes equipment, human resources, as well as organisational characteristics such as staff training and payment methods. The process contains all actions and activities relating to how healthcare is delivered. These can include diagnosis, treatment, preventive care and patient education. The outcome is the result or effect of healthcare on patients or populations, including the changes to health status, behaviour or knowledge as well as patients’ satisfaction and health-related quality of life. The Triple Aim is a framework developed by the Institute for Healthcare Improvement that describes an approach for optimising health system performance. According to this model, development or reorganisation of healthcare delivery that focuses on three critical dimensions simultaneously will potentially optimise health system performance. These three dimensions are:

Improving the patients’ experience of care (including quality and satisfaction);

Improving the health of populations;

Reducing the per capita cost of healthcare.

Table 1 shows in more detail the core dimensions and constructs that will be used to categorise performance and QIs.

Table 1

Conceptual framework for categorisation of performance and quality indicators

Triple Aim framework	Donabedian framework
	Structure	Process	Outcome
Population health Health/functional status Disease burden Risk status Mortality	Indicators	Indicators	Indicators
Patient experience Safe Efficient Effective Timely Patient centred Equitable	Indicators	Indicators	Indicators
Per capita cost Total cost per patient per month Hospital and emergency department Utilisation rate	Indicators	Indicators	Indicators

Methods and analysis

We will employ the scoping review methodology described by Arksey and O'Malley22 and further clarified by Levac et al23 which comprises six stages. For the purpose of this review, a performance indicator is defined as a ‘unit of information, which reflects the performance of the health care system in maintaining or increasing the well-being of its target population’.24 Furthermore, a performance indicator is a measure of the structure, process of care or outcome that is useful at one or more levels of the health system (patient, organisation, community, regional or provincial), as a support for planning, management or providing quality improvement in healthcare. A performance indicator can also measure a specific dimension of performance (eg, safety, mortality, etc). Performance indicators focus on desired outcomes or processes of care that are evidence-based.25

Stage 1: identifying the research question

The research team defined the concepts, target population and have drafted one main research question for the scoping review which is the following: ‘What are the patient, institutional and system level indicators that are currently in use or proposed for measuring quality of care across the continuum for individuals with chronic non-cancer pain?’

Stage 2: identifying eligible studies

Studies concerned with performance or QIs in adults (18+ years) with chronic non-cancer pain receiving any treatment or assessment in primary, secondary or tertiary care setting will be included in this review. Studies with the following types of non-cancer CP will be included: musculoskeletal (eg, back pain, arthritis, fibromyalgia, etc); neuropathic (eg, phantom limb pain, diabetic neuropathy, etc); headaches and other (eg, haemophilia, irritable bowel syndrome, etc). Studies that include a mix population of adult patients with chronic non-cancer pain and cancer CP will be included if results for non-cancer pain patients are reported separately.

Studies reporting indicators related to acute pain (eg, postsurgical pain, acute pregnancy/labour pain, etc), paediatric pain and cancer CP will be excluded considering that these patients have different healthcare needs; thus, assessing performance of these healthcare services would require different measures.

We will include any type of study design (eg, randomised control trials, systematic reviews, case studies, quasi-experimental studies, mixed-methods studies, clinical guidelines, qualitative studies, audits or quality assessment reports to assess management of non-cancer CP). Studies published in English or French 2000 onwards will be included to ensure results are relevant to the healthcare context. However, limiting the search to these languages only may result in bias towards English-speaking and French-speaking countries.

Search strategy and information sources

We will search for publications in the following electronic databases: the Cochrane Effective Practice and Organisation of Care (EPOC); Review Group Specialised Register; the Cochrane Library; EMBASE; PubMed; CINAHL; PsycINFO; ProQuest Dissertations and Theses (a search strategy in PubMed is shown in online supplementary appendix A).

We will hand search the reference lists of included studies, and review proceedings and abstracts from relevant conferences on CP held in the past 3 years. We will also search for grey literature in provincial, national and international health organisations’ websites, and in health professional and scientific associations’ websites. More specifically, the following websites of organisations/associations will be searched: the NICE; INESSS; CIHI; International Association for the Study of Pain; American Pain Society; Canadian Pain Mechanisms, Canadian Pain Society, Canadian Pain Coalition, Canadian Neuropathy Association, Diagnosis and Management Consortium, Pain Alliance Europe, European Pain Federation. Another source for grey literature will be the internet using search engines like Google and Yahoo. Given the large number of results that this search strategy may yield, we will analyse only the first 50 results from each search engine to identify publications about performance indicators developed for non-cancer CP. Literature search strategies will be developed and conducted by an information specialist using Medical Subject Headings (MeSH) and text words related to non-cancer CP performance indicators. This strategy will be peer reviewed by another information specialist to assure that the proposed search strategy is accurate and sensitive enough to capture most of the relevant literature.26

Stage 3: study selection

Study selection will be performed in four major stages.27 First, search results will be merged and duplicates will be removed by using reference management software (EndNote V.X5). Second, a data extraction form based on the eligibility criteria described above will be developed by the research team. Third, a pilot test of this data extraction form will be performed: two reviewers, with expertise in CP management, will independently screen the first 25 titles, abstracts and grey literature of retrieved publications according to the eligibility criteria by using the data extraction form. Agreement for study inclusion between the two reviewers will be calculated with the κ statistic.27 If agreement is inferior to 75%, reasons for disagreement will be explored, eligibility criteria will be refined and clarified accordingly, and pretesting will be repeated until the mean inter-rater reliability is satisfactory (κ >0.75). Fourth, all eligible studies and those classified as unclear (needing more information) will be reviewed as full-text articles by each reviewer independently to determine if all inclusion criteria are met and if the article is to be included or not in the study. Inter-rater agreement will be again calculated on a random sample of 25 publications. Disagreement on study eligibility will be discussed and resolved by consensus. If disagreement remains, a third reviewer, also knowledgeable in CP management, will be available to resolve discrepancies.

Stage 4: data extraction process

A more detailed data extraction form for eligible studies will be developed by the research team. Two reviewers will use the form to extract data from the first 15 eligible publications. Then they will meet to compare consistency of data extraction and coding. Thus, clarification and an update of the extraction form will take place as an iterative process until the research team reaches consensus on the final form. All eligible studies will be reviewed as full-text articles by the same two reviewers independently according to this detailed extraction form. The reviewers will meet at the beginning, during the middle and at the end of the review process to compare and discuss their extraction results. If they disagree, a third reviewer will resolve the discrepancies.

Extracted data will include the following variables: authorship, year of publication, country, status of publication (ie, published or grey literature), journal, study design (eg, randomised controlled trial, qualitative study, mixed-methods studies, etc), indicator description including numerator and denominator when available, study setting where the indicator was used (primary, secondary or tertiary), data source from where the indicator was collected, purpose of the indicator, level at which the indicator was used (patient level, organisational level or population level), measurement properties of the indicator (validity, fidelity if available) and authors’ recommendations regarding the indicator. Other variables may be added by the researchers’ team when revising and updating the form after analysis of the 15 first eligible publications.

Stage 5: data synthesis

A quantitative and a qualitative synthesis will be performed for all identified indicators. The quantitative synthesis will comprise numerical counts such as number of indicators by setting, by level of use and per dimension according to the conceptual framework. A deductive content analysis will be performed as indicators will be classified according to the dimensions covered by the conceptual framework described above. All indicators which could not be classified according to the included performance dimensions will be grouped and analysed by the research team by categorising these into new performance dimensions. Furthermore, a qualitative description approach28 will be used to describe characteristics of each indicator (eg, definition of indicator, source of data collection, frequency of collection and reporting, audience for reporting, format of reporting). This overall synthesis of published performance indicators will allow us to describe the current state and trends of performance measurement in non-cancer CP management across the continuum of care (primary, secondary and tertiary), and help identify gaps in performance domains not addressed by the literature.

Stage 6: stakeholder consultation

This phase of the recommended methodology for conducting a scoping review offers opportunities to involve stakeholders at different steps of the review process.23 Besides providing feedback on the research process, stakeholder involvement is an ideal first strategy to start translating findings among potential knowledge users. We will initiate contact with stakeholders at the beginning of the review process and after preliminary results are available. Groups of stakeholders who will be contacted include representatives of researchers in CP, decision-makers (hospital, regional and provincial levels), healthcare professionals (eg, physical therapists, psychologists, doctors, nurses, etc) and clients from CP programmes. Stakeholders will be identified through CP programmes listed on the web and via the research team's network. Persons suffering from CP will be contacted via the Quebec Association of Chronic Pain. Consultations will be held through meetings using a blended format, that is, face-to-face and internet technology (videoconference) to allow for broad participation and to reduce costs. Special attention will be paid to assure that all groups of stakeholders are equally represented. The goal of the first consultation will be to solicit stakeholders’ feedback on our approach. Specifically, we will validate our research question, refine or develop additional research questions that can be addressed by this scoping review, validate the data extraction form for eligible studies, gather stakeholders’ input to align better with their information needs concerning performance indicators in CP management, and gather suggestions for appropriate studies to include in the scoping review. The goal of the second meeting will be to validate preliminary results and get suggestions as how to best disseminate our results to various stakeholder groups. All meetings will be audio-recorded, and an inductive thematic analysis will be performed.28

Dissemination and ethics

We will ensure the dissemination of our research findings through several strategies. We will engage stakeholders at the beginning and at the end of the research project through a consultation process to identify optimal methods and content, and to disseminate the results and identify knowledge translation strategies to better align with stakeholders needs. We will also ask stakeholders to disseminate the results across their networks. Furthermore, we will present results of the scoping review at national and international conferences, publish these in a peer-reviewed journal and propose to the stakeholders an adapted content (eg, plain language conclusions) to be posted on their websites.

A scoping review of published articles is a secondary analysis and does not require ethics approval. However, the project will be submitted to an ethics committee for approval of the consultation phase and informed consent will be obtained from stakeholders participating in the project prior to any data collection.

Conclusion

This synthesis of published performance indicators will provide a comprehensive evaluation of current performance measurement in non-cancer CP management across the continuum of care (primary, secondary and tertiary), and will identify gaps in the literature. More specifically, the outputs will be a list of performance indicators with their level of evidence and application across the continuum of healthcare. The indicators will be classified using the Triple Aim framework, and can be used by stakeholders to evaluate the quality of care for individuals with chronic non-cancer pain at the patient, institutional and system level.