Validity of Algorithms for Identification of Individuals Suffering from Chronic Noncancer Pain in Administrative Databases: A Systematic Review.

BACKGROUND: Secondary analysis of health administrative databases is indispensable to enriching our understanding of health trajectories, health care utilization, and real-world risks and benefits of drugs among large populations.
OBJECTIVES: This systematic review aimed at assessing evidence about the validity of algorithms for the identification of individuals suffering from nonarthritic chronic noncancer pain (CNCP) in administrative databases.
METHODS: Studies reporting measures of diagnostic accuracy of such algorithms and published in English or French were searched in the Medline, Embase, CINAHL, AgeLine, PsycINFO, and Abstracts in Social Gerontology electronic databases without any dates of coverage restrictions up to March 1, 2018. Reference lists of included studies were also screened for additional publications.
RESULTS: Only six studies focused on commonly studied CNCP conditions and were included in the review. Some algorithms showed a ≥60% combination of sensitivity and specificity values (back pain disorders in general, fibromyalgia, low back pain, migraine, neck/back problems studied together). Only algorithms designed to identify fibromyalgia cases reached a ≥80% combination (without replication of findings in other studies/databases).
CONCLUSIONS: In summary, the present investigation informs us about the limited amount of literature available to guide and support the use of administrative databases as valid sources of data for research on CNCP. Considering the added value of such data sources, the important research gaps identified in this innovative review provide important directions for future research. The review protocol was registered with PROSPERO (CRD42018086402).

Introduction

Health administrative databases are commonly used to conduct epidemiologic, pharmacoepidemiologic, and pharmacoeconomic research and are indispensable to enrich our understanding of health trajectories, health care utilization, and real-world risks and benefits of drugs among large populations [1-5]. However, the validity of studies conducted with these data sources relies greatly on the accuracy of the diagnostic information used to create or characterize cohorts of patients suffering from particular health disorders [6-11]. In fact, many administrative databases used for research purposes are by-products of physician billing claims in the context of which it is diagnostic codes are entered, for example, according to the International Classification of Diseases (ICD codes) [12, 13]. Such codes are sometimes misclassified, and it has been shown that their validity varies according to the characteristics of patients, clinical conditions, health care encounters, physicians, and their billing practices [14, 15]. Moreover, the validity of a given algorithm may vary from one data source to another due to variability of database completeness across jurisdictions (e.g., remuneration methods, diagnostic code types and specificity, number of diagnostic fields) [6]. Using validated algorithms to identify and characterize specific patient populations is thus a priority in order to reduce bias in administrative database studies [10, 11].

In the field of chronic noncancer pain (CNCP), longitudinal population-based studies representative of the real-world clinical context are clearly needed [16-18]. For example, such studies are important complements to randomized clinical trials that study the benefits and risks of pain pharmacotherapy (e.g., larger sample sizes, possibility to study long-term effects, greater external validity). This is especially important in a context where CNCP treatment is characterized by off-label prescribing, polypharmacy, and multimorbidity [19-25], profiles that are often not represented in clinical trials. Using health administrative data for such purposes represents an attractive and efficient strategy. A large number of studies have used various ICD coding algorithms to identify individuals suffering from CNCP or to adjust for the presence of CNCP as a comorbidity [12, 26–52]. However, the validity of these algorithms has not always been established before their use. For instance, a previous systematic review of algorithms used to identify various health conditions in US and Canadian administrative databases suggested that only 17.5% of studies report on the validation of the algorithms used [15].

Considering that CNCP is commonly under-reported, underdiagnosed, and under-recognized in primary care settings [18, 53–57], possibly resulting in an increased likelihood of diagnostic code misclassification in administrative databases, we wonder if such data sources are valid for research on CNCP. To the best of our knowledge, evidence about the validity of case-finding algorithms for commonly studied CNCP conditions such as back pain or neuropathic pain has never been compiled and synthesized. The objective of this systematic review was to document validation studies of algorithms for the identification of individuals suffering from nonarthritic CNCP using health administrative databases.

Methods

This study is among a series of systematic reviews of validated methods for identifying various chronic diseases using health care administrative data that have been conducted by the Quebec SUPPORT Unit (Support for People and Patient-Oriented Research and Trials) as part of its mandate to implement strategies to facilitate access to and use of health research data. This work was conducted according to the Preferred Reporting Items for Systematic Review and Meta-Analysis protocols (PRISMA-P) 2015 statement [58] and recommendations specific to reviews of test accuracy [59]. The study was registered in the PROSPERO international database of systematic reviews (CRD42018086402) and can be accessed at: https://www.crd.york.ac.uk/prospero/display_record.php? RecordID=86402.

Eligibility Criteria

To be included in the review, studies had to be original investigations reporting on the validity of algorithms for the identification of CNCP cases in administrative/claims data (studies about the quality of computerized medical records studies were excluded). Peer-reviewed journal articles and reports published in English or in French were eligible, and validation studies could be conducted against various types of reference standards (e.g., disease-specific registries, medical chart review, patient self-report) and about various types of codes (e.g., ICD-9, ICD-10, ICD-10CM, ICD-10CA, non-ICD codes). Commonly studied CNCP conditions such as back pain, neck pain, neuropathic pain, complex regional pain syndrome, fibromyalgia, headache, and migraine [60] were considered. In past years, several systematic reviews have been published about the validity of algorithms for the identification of individuals suffering from rheumatoid arthritis, osteoarthritis, systematic lupus erythematosus (SLE), and other rheumatic conditions such as lupus nephritis, polymyalgia rheumatica, ankylosing spondylitis, Sjögren syndrome, and vasculitis [61-64], and algorithms with acceptable diagnostic accuracy measures were reported. Therefore, retrieved studies focusing solely on these conditions were excluded from the present systematic review. As fibromyalgia is sometimes but not always classified as a rheumatic condition [65], it was included in our review. Neurodegenerative disorders (e.g., amyotrophic lateral sclerosis and other motor neuron diseases, multiple sclerosis, Parkinson’s disease, or Guillain-Barre syndrome) or abdominal painful conditions (e.g., irritable bowel syndrome, ulcerative colitis, Crohn’s disease, colonic ischemia, gastro-oesophageal reflux disease, primary sclerosing cholangitis) are not systematically considered CNCP [60, 66]. Articles about such conditions were therefore excluded (a posteriori). If a study reported on the validity of many health conditions, only algorithms related to nonrheumatic commonly studied CNCP conditions were reviewed.

Information Sources and Search Strategy

Studies were identified on March 1, 2018, by searching the following computerized databases without any dates of coverage restrictions: Medline (EBSCO; PubMed for the past two years), Embase (Ovid), CINAHL (EBSCO), AgeLine (EBSCO), PsycINFO (EBSCO), and Abstracts in Social Gerontology (EBSCO). The search strategy was developed in collaboration with experienced medical librarians, a pain epidemiologist, and a primary care physician and included several synonyms for 1) commonly studied CNCP conditions, 2) validation studies, and 3) administrative databases (Supplementary Data). Different types of CNCP conditions, defined by the International Association for the Study of Pain (IASP) Task Force for the Classification of Chronic Pain, were represented in our search strategy [60]. All citations were entered in the citation management software Zotero, and duplicates were removed.

Study Selection

Using the Rayyan web application, the whole screening and selection process was achieved by two independent trained reviewers who met and resolved disagreements with a third party if needed. First, titles and abstracts of all citations retrieved from electronic databases were screened with the aim of identifying articles fitting the prespecified eligibility criteria. All abstracts identified by the reviewers in the abstract screening phase were then assessed in full text for inclusion. The reference list of studies included in the present review was also scanned for potential nonretrieved original investigations (snowball citation searching). At the end of the process, the final list of pain conditions and articles included in the review was validated by a pain epidemiologist and a primary care physician, who were not previously involved in the study selection process.

Data Collection Process

Using a pilot-tested standardized extraction form, data collection was achieved by one reviewer and then validated by a second reviewer (who resolved disagreements with a third party if needed). A tool containing detailed definitions of each piece of information/variable to be extracted was used by the reviewers to better standardize data collection. In the context of our study, obtaining/confirming data from investigators was not needed.

Data Items

For each study meeting the selection criteria, the following information was retrieved: authors, title of the study, state/province and country where it was conducted, administrative database to be validated, reference standard used, study population characteristics and size, and CNCP conditions under study. Moreover, each algorithm was described in detail, including its content, types of codes used, and inclusion of pharmacy claims data in the algorithm. For each algorithm, the following measures of diagnostic accuracy and their respective 95% confidence intervals (95%CI) were extracted when available: 1) sensitivity (SEN): probability that a patient is identified as a CNCP case in the administrative database given the presence of CNCP according to the reference standard, 2) specificity (SP): probability that a patient is not identified as a CNCP case in the administrative database given the absence of CNCP according to the reference standard, 3) positive predictive value (PPV): probability of suffering from CNCP according to the reference standard given that the patient is identified as a CNCP case in the administrative database, 4) negative predictive value (NPV): probability of not suffering from CNCP according to the reference standard given that the patient is not identified as a CNCP case in the administrative database, and 5) kappa coefficient: degree of agreement between the administrative database and the reference standard corrected for chance. When available, measures of diagnostic accuracy were extracted according to sex and age subgroups. All of these statistics ranged between 0 and 1 (0–100%), with higher values indicating better validity/accuracy of an algorithm [67]. To the best of our knowledge, there is no consensus regarding specific cutoffs indicating what can be considered poor vs high SEN and SP values. We chose 60% and 80% arbitrary cutoffs to ease our interpretation.

Risk of Bias

The type of reference standards used in the validity studies was the main aspect considered in terms of quality of reviewed studies. In fact, the assessment of an algorithm’s validity is based on the premise that the reference standard against which it is tested represents the truth—which is not always the case. Although retrospective medical chart review is a widely applicable research methodology, such routinely collected data were not originally intended for research purposes and may be lacking in quality (misclassification and missing information) [68, 69]. One can hypothesize that the quality of clinical information about pain contained in medical charts can vary from one setting to another (primary care vs tertiary care pain clinic). As for self-reported diagnoses, their validity is not always optimal and is variable across chronic health conditions and patient populations [70-78]. Clinician-confirmed diagnoses collected in the context of disease-specific registries were thus considered a high-quality reference standard in comparison with primary care medical chart review or patient self-report. The reporting of key measures of diagnostic accuracy (SEN, SP, PPV, NPV) and external validity of results (e.g., algorithms validated in several databases) were also considered when looking at the quality of reviewed studies.

Synthesis of Results

The results of included studies were described narratively and combined in tables in order to describe 1) the characteristics of the various studies and 2) the algorithms and measures of diagnostic accuracy reported for each. The quantity and quality of available evidence were also depicted in a summary table.

Results

The study selection flow diagram is shown in Figure 1. A total of 1,840 articles were identified, but most were excluded based on their title and abstract. From the remaining 98 full-text articles, 75 did not meet the selection criteria. Combining the remaining articles retrieved from the electronic search (N = 23) and those identified through their reference lists (N = 12), a total of 35 journal articles were considered. After the exclusion of 29 articles about neurodegenerative disorders or abdominal painful conditions [79-107], only six studies were found to report on the validity of algorithms for identification of commonly studied CNCP: back disorders in general [108], complex regional pain syndrome [108], fibromyalgia [108-110], headache/migraine studied together [111], low back pain [108], migraine [111, 112], neck/back problems studied together [108], painful diabetic peripheral neuropathy [113], and painful neuropathic disorders in general [108].

Figure 1

Study selection flow diagram.

Included Studies

Characteristics of the included studies are presented in Table 1 (in alphabetical order according to the last name of the first author). Two studies (33.3%) were published in the five years preceding the electronic search. Studies were all conducted using US (3/6 = 50%) or Canadian administrative databases (3/6 = 50%). Back or neck pain algorithms were only tested in a Canadian context. A given study could focus on more than one CNCP condition and test more than one algorithm for a given CNCP condition.

Table 1

Characteristics of the studies included in the systematic review (N = 35)

Authors	Title of the Study	Country	Administrative Database	Reference Standard Used for the Validation Study	Study Population	Chronic Pain Conditions Studied
1. Hartsfield et al. 2008	Painful diabetic peripheral neuropathy in a managed care setting: Patient identification, prevalence estimates, and pharmacy utilization patterns	Colorado, United States	Kaiser Permanente Colorado Diabetes Registry	Medical chart review	Patients aged 18 years and over (N = 300)	Painful diabetic peripheral neuropathy
2. Katz et al. 1997	Sensitivity and positive predictive value of Medicare Part B physician claims for rheumatologic diagnoses and procedures	Massachusetts, Colorado, Virginia, United States	Medicare Part B physician claims	Medical chart review (rheumatologists in separate practices)	Patients (N = 342)	Fibromyalgia (and rheumatic conditions excluded from the present review)
3. Kolodner 2004	Pharmacy and medical claims data identified migraine sufferers with high specificity but modest sensitivity	United States	Health plan enrollment and medical group files (southeastern Michigan)	Patient self-report (telephone interviews)	Patients aged 18–55 years (N = 8,579)	Headache, migraine (and other chronic conditions excluded from the present review)
4. Lacasse et al. 2015	Is the Quebec provincial administrative database a valid source for research on chronic non-cancer pain?	Quebec, Canada	Quebec Universal Health Insurance Database	Quebec Pain Registry (tertiary care provincial registry)	Patients (N = 561)	Neck and back problems, back disorders, lower back pain, complex regional pain syndrome, fibromyalgia, painful neuropathic disorders
5. Marrie et al. 2012	The incidence and prevalence of fibromyalgia are higher in multiple sclerosis than the general population: A population-based study	Manitoba, Canada	Manitoba Universal Health Insurance Database	Medical chart review	Patients (N = 430)	Fibromyalgia
6. Marrie et al. 2013	The utility of administrative data for surveillance of comorbidity in multiple sclerosis: A validation study	Manitoba, Canada	Manitoba Universal Health Insurance Database	Medical chart review	Patients (N = 430)	Migraine (and other chronic conditions excluded from the present review)

Tested Algorithms

For the purpose of this review, each algorithm was assigned a unique identification number (ID). The detailed description of algorithms and measures of diagnostic accuracy reported for each are presented in the Supplementary Data (in alphabetical order according to the CNCP condition studied). Overall, the six included studies reported measures of diagnostic accuracy for 99 algorithms designed to identify CNCP cases in administrative/claims data. Reported accuracy measures were accompanied by their respective 95% CIs in the great majority of cases (96/99 algorithms). Only one study reported measures of diagnostic accuracy across males and females [111].

Tested algorithms designed to identify commonly studied CNCP conditions focused on back disorders in general (N = 2; ID: 1 and 2), complex regional pain syndrome (N = 4; ID: 3 to 6), fibromyalgia (N = 34; ID: 7 to 40), headache/migraine studied together (N = 12; ID: 41 to 52), low back pain (N = 3; ID: 53 to 55), migraine (N = 38; ID: 56 to 93), neck/back problems studied together (N = 2; ID: 94 and 95), painful diabetic peripheral neuropathy (N = 2; ID: 96 and 97), and painful neuropathic disorders in general (N = 2; ID: 98 and 99). Such algorithms were diverse in terms of the number of health care encounters and time window considered. For all of the CNCP conditions mentioned above, pharmacy claims data were not considered in the algorithms, except for headache and/or migraine. The four key measures of diagnostic accuracy (SEN, SP, PPV, NPV) were reported for the great majority of algorithms.

As shown in the Supplementary Data, several algorithms designed to identify patients’ suffering from CNCP showed a ≥60% combination of SEN and SP values (≤40% chances of false negatives and false positives): back disorders in general (ID: 2), fibromyalgia (ID: 17 to 19, 23 to 40), low back pain (ID: 55), migraine (ID: 60), and neck/back problems studied together (ID: 95). Algorithms designed to identify patients suffering from complex regional pain syndrome, headache/migraine studied together, painful diabetic peripheral neuropathy, and painful neuropathic disorders in general did not reach the 60% SEN and SP cutoff. Only algorithms designed to identify fibromyalgia cases and tested in one study reached an ≥80% combination of SEN and SP values (ID: 17–19, 23–25, 30, 31, 36, 37) [110]. A summary of the quantity and quality of the available literature is presented in Table 2. The most frequently used reference standard was medical chart review (4/6 = 66.7%), followed by disease-specific patient registries (1/6 = 16.7%) and patient self-report (1/6 = 16.7%).

Table 2

Summary of the available literature about algorithms for identification of commonly studied CNCP

Classic/Commonly Studied CNCP	Multiple Different Algorithms Were Tested	Quality of Reference Standards Used	Key Measures of Diagnostic Accuracy Were Calculated for Tested Algorithms	Diversity of Administrative Databases in Which Algorithms Were Tested	At Least 1 Algorithm Showed ≥60 Combination of SEN and SP	Same Algorithm Showed ≥60% Combination of SEN and SP in >1 Database	At Least 1 Algorithm Showed ≥80% Combination of SEN and SP
Back disorders in general	–	Pain-specific patient registry	Yes	–	Yes		No
	N = 2			N = 1
Complex regional pain syndrome	–	Pain-specific patient registry	Yes	–	No		No
	N = 4			N = 1
Fibromyalgia	++	Pain-specific patient registry and medical chart review	Yes (except for 2 algorithms)	+	Yes	No	Yes
	N = 34			N = 3
Headache/migraine studied together	+	Patient self-report	Yes	–	No		No
	N = 12			N = 1
Low back pain	–	Pain-specific patient registry	Yes	–	Yes		No
	N = 3			N = 1
Migraine	++	Patient self-report + medical chart review	Yes	±	Yes	No	No
	N = 38			N = 2
Neck/back problems studied together	–	Pain-specific patient registry	Yes	–	Yes		No
	N = 2			N = 1
Painful diabetic peripheral neuropathy	–	Medical chart review	Yes	–	No		No
	N = 2			N = 1
Painful neuropathic disorders in general	–	Pain-specific patient registry	Yes	–	No		No
	N = 2			N = 1
Other important CNCP conditions (e.g., chronic postsurgical pain, chronic post-traumatic pain, phantom limb pain)

In the table, shaded cells = not applicable.

SEN = sensitivity; SP = specificity.

Discussion

To our knowledge, this study is the first attempt to synthesize evidence about the validity of algorithms to identify individuals suffering from nonarthritic CNCP in health administrative databases. Based on our results, a very limited amount of literature is available to support the use of administrative databases as valid sources of data for research on CNCP. However, our results provide valuable information to yield a number of key findings, identify research gaps, and make several recommendations for future research:

Key Finding 1

Few studies (N = 6) examined the validity of algorithms to identify individuals suffering from commonly studied nonrheumatic CNCP conditions in administrative databases [108-113]. This finding is surprising and could be explained by many factors, including 1) the presence of a publication bias (only studies with results that are statistically or clinically significant are published [114]), 2) the lack of awareness that using validated algorithms is a priority to reduce bias in database studies [10, 11], and 3) the lack of awareness of the value of observational research using administrative data in the community of pain researchers. Considering the added value of such data sources in pain research, new CNCP case-finding algorithms should be developed, and well-designed validation studies should be conducted and published.

Key Finding 2

All studies were conducted in US or Canadian administrative databases. This was not a surprising result considering that they are internationally recognized health databases and have long been used by epidemiologists, pharmacoepidemiologists, and health economists [1, 3, 115, 116]. It is also consistent with other systematic reviews about the validity of algorithms for the identification of individuals suffering from osteoarthritis (100% conducted in the United States or Canada) [63] or rheumatic diseases in general (83% of included studies were conducted in the United States or Canada) [64]. This finding could imply that little is known about the quality of claims databases for pain research in many other countries. We, however, have to keep in mind that studies about the quality of computerized medical records were not included in the present review and that such databases are extensively used for research purposes outside of North America (e.g., UK’s General Practice Research Database [GPRD]) [117]. The fact that back or neck pain algorithms were not tested in US databases was surprising considering the significant burden of these conditions [118].

Key Finding 3

For many CNCP conditions, very few different algorithms were tested for accuracy (≤4 algorithms for each of the following conditions: back disorders in general, complex regional pain syndrome, low back pain, neck/back problems studied together, painful diabetic peripheral neuropathy, or painful neuropathic disorders in general).

Key Finding 4

Pharmacy claims data were not included in the commonly studied CNCP algorithms, except for headache and/or migraine. This was expected as, contrary to conditions such as arthritis where prescription claims are often used in case-finding algorithms [61], many pharmacological treatments used for CNCP are not specific to a particular type of syndrome (e.g., opioids, nonsteroidal anti-inflammatory drugs) or not specific to pain management (e.g., antidepressants, anticonvulsants, cannabinoids). However, it would be important to test and publish about the contribution of adding pharmacy claims to the equation, no matter if the results are positive or not. Key findings 3 and 4 could be explained by selective reporting and publication of results. Our findings emphasize the need for more studies aiming to develop, test, refine, and publish case-finding algorithms.

Key Finding 5

The diversity of administrative databases in which case-finding algorithms were tested is limited. In fact, none of the specific combinations of codes, time windows, and number of health care encounters was tested in more than one database. As the validity of a given algorithm could vary from one data source to another because of the variability of database completeness across jurisdictions [6], this constitutes an important limitation of available evidence.

Key Finding 6

Some algorithms designed to identify patients suffering from CNCP showed a ≥60% combination of SEN and SP values and could be useful (back disorders in general, fibromyalgia, low back pain, migraine, neck/back problems studied together). When selecting the optimal CNCP algorithm for an administrative database study, researchers should, however, assess the relative importance of SEN, SP, PPV, and NPV and prioritize the accuracy measure that is most relevant to their research question [119]. Misclassification of CNCP cases can significantly impact a study’s internal validity. For example, it could affect researchers’ capacity to correctly control for confounding, a constant challenge in observational designs [3]. Also, it is necessary to select algorithms with very high SEN for prevalence studies, as this approach minimizes the number of false negatives [119, 120]. On the other hand, if the goal is to select a CNCP cohort for upcoming studies, one might risk missing some cases and assume the loss of external validity.

Key Finding 7

Only some algorithms designed to identify fibromyalgia cases reached ≥80% combination of SEN and SP values (ID: 17–19, 23–25, 30, 31, 36, 37). They were, however, tested in only one study/database and validated against medical records [110]. Possible explanations for the absence of comparable positive results in the two other studies that tested fibromyalgia algorithms are 1) the shorter case identification time windows (one year or less in the Lacasse et al. and Katz et al. studies [108, 109] vs two to five years for algorithms showing a ≥80% combination of SEN and SP values in the Marrie et al. study [110]) and 2) the reference standards used (of lower quality in the Marrie et al. study vs the Lacasse et al. study). In the future, time and resources should be invested to achieve replication of findings before concluding on the validity of an algorithm.

Key Finding 8

Algorithms designed to identify patients suffering from complex regional pain syndrome, headache/migraine studied together, painful diabetic peripheral neuropathy, and painful neuropathic disorders in general did not reach acceptable SEN and SP cutoffs. This finding can perhaps be explained by the fact that CNCP is commonly under-reported, underdiagnosed, and under-recognized in primary care settings [18, 53–57]. The challenges surrounding the diagnosis of neuropathic pain [121] and the fact that only one diagnostic code per medical visit is recorded in some administrative databases could thus explain the lack of diagnostic accuracy [108].

Key Finding 9

The quality of the reference standard used in the studies aimed at the validation of algorithms to identify individuals suffering from commonly studied CNCP conditions was variable, but many used medical chart review. When choosing a validated CNCP algorithm, researchers should be aware of the potential impact of the reference standard’s quality on the estimation of accuracy measures.

Key Finding 10

Measures of diagnostic accuracy of few algorithms were presented across males and females (only in one study of headache and/or migraine). Growing attention is given to the importance of considering sex and gender in health research [122-125]. Considering the relevance of these health determinants to the pain experience [126, 127], such subgroup stratification of validity results should be achieved when possible.

Key Finding 11

According to the available literature, the diagnostic accuracy of case-finding algorithms for CNCP (all types considered) or specific conditions such as chronic postsurgical pain, chronic post-traumatic pain, or phantom limb pain has never been investigated. An earlier study showed that as few as 0–0.36% of patients who were enrolled in a chronic pain registry had an ICD-9 pain code (307.8, 338, 338.2, or 338.4) recorded in administrative databases [108]. In another study, an algorithm combining pain-related ICD-9 codes, opioid medication, and pain scores was shown to be valid for the identification of individuals suffering from chronic pain in primary care electronic records [128]. Although this algorithm was applied to Canadian health insurance databases [129], its validity in such administrative claims was never evaluated. One more time, this emphasizes the need for studies aimed at the development/validation of new case-finding algorithms, including ways to identify patients no matter what type of CNCP they are suffering from.

Study Limitations

Despite the development of a thorough search strategy, about one-third of studies were retrieved through reference lists of articles included in the present review. This demonstrates the difficulty of identifying studies about chronic pain case-finding algorithms from electronic searches and a possible lack of consistent terminology in the scientific literature. Further studies should look at the state of the situation and the development of recommendations to that effect. Resource constraints brought us to exclude a review of gray literature and an assessment of the quality of reporting of included studies, for example, using an appraisal tool such as the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool [130]. However, important quality components of validation studies of case-finding algorithms were considered in the interpretation of our results, such as the quality of the reference standard used for validation, the reporting of all key measures of diagnostic accuracy, and the external validity of studies. As stated, our search strategy was designed to capture studies about commonly studied CNCP conditions. Another limit of our paper is the exclusion of studies about rheumatic conditions (because systematic reviews reporting acceptable diagnostic accuracy measures were already available [61-64]). In fact, many recent studies have been published about the validity of arthritis case-finding algorithms [131-138].

Conclusions

A small quantity of algorithms with fair diagnostic accuracy is available to identify patients suffering from commonly studied CNCP conditions such as back and neck pain, fibromyalgia, and migraine. However, their diagnostic accuracy should always be interpreted depending on the intended purpose and considering the absence of evidence regarding the replicability of findings across studies and databases. According to the available literature, several CNCP conditions have never been the subject of validated algorithms (algorithms with poor diagnostic accuracy or not developed/tested at all). The present investigation informs us about the limited amount of literature available to support the use of administrative databases as valid sources of data for research on CNCP. Considering the added value of such data sources, the above-mentioned research gaps provide important directions for future research. It should be noted that linking administrative databases with other data sources containing valid pain-related data (e.g., survey data, patient registries) is a valuable option until more evidence is gained in the field.

Click here for additional data file.