Validated methods for identifying individuals with obesity in health care administrative databases: A systematic review.

BACKGROUND: Health care administrative databases are increasingly used for health studies and public health surveillance. Cases of individuals with obesity are selected using case-identification methods. However, the validity of these methods is fragmentary and particularly challenging for obesity case identification.
OBJECTIVE: The objectives of this systematic review are to (1) determine the case-identification methods used to identify individuals with obesity in health care administrative databases and (2) to summarize the validity of these case-identification methods when compared with a reference standard.
METHODS: A systematic literature search was conducted in six bibliographic databases for the period January 1980 to June 2019 for all studies evaluating obesity case-identification methods compared with a reference standard.
RESULTS: Seventeen articles met the inclusion criteria. International Classification of Diseases (ICD) codes were the only case-identification method utilized in selected articles. The performance of obesity-identification methods varied widely across studies, with positive predictive value ranging from 19% to 100% while sensitivity ranged from 3% to 92%. The sensitivity of these methods was usually low while the specificity was higher.
CONCLUSION: When obesity is reported in health care administrative databases, it is usually correctly reported; however, obesity tends to be highly underreported in databases. Therefore, case-identification methods to monitor the prevalence and incidence of obesity within health care administrative databases are not reliable. In contrast, the use of these methods remains relevant for the selection of individuals with obesity for cohort studies, particularly when identifying cohorts of individuals with severe obesity or cohorts where obesity is associated with comorbidities.

INTRODUCTION

Health care administrative databases are extremely useful in many research settings. , They are increasingly being used in epidemiological and health care costing studies, for chronic disease surveillance, as well as for the study of burden of disease, the study of health service effectiveness, compliance and utilization and to examine population health status and its determinants. , , , , , , Health care administrative databases are comprehensive, inexpensive and time saving. When combined with population survey data, they can establish the prevalence and incidence of various diseases. In general, these databases include information on medical procedures such as medical visits, emergency room visits, hospitalizations, prescription drugs and associated costs. In order to identify specific populations within these databases, case‐identification methods are utilized. These case‐identification methods may take the form of identification codes of the International Classification of Diseases (ICD) system, a drug code or a code related to the use of health services.

Given the rising burden of obesity as well as the fact that obesity is an important risk factor for numerous other chronic diseases, it would be particularly useful to rely on validated methods for identifying individuals with obesity in health care administrative databases. Previous studies have shown, however, that the use of case‐identification methods to identify individuals with obesity in health care administrative databases may lead to an imperfect estimate of the prevalence of obesity. , , , Previous studies have shown that the coding of obese cases in health care administrative databases is inconsistent, incomplete or underreported. , , , , In addition, the case‐identification methods used in recent studies have led to variable performance depending on the existence of other comorbidities, the severity of obesity and the type of health care administrative data. , , , , , , Accurate case‐identification methods for health care administrative databases is essential to minimize classification errors, which can represent a significant form of bias related to the use of administrative databases.

One way to assess the accuracy of case‐identification methods is to compare this with a reference standard. For example, data on obesity in the health care administrative database could be compared with clinical data to determine their accuracy. If both the clinical data and the health care administrative database report that the patient suffers from obesity, there is high concordance (validity).

The objective of this systematic review is to determine case‐identification methods used to identify individuals with obesity within health care administrative databases and to report the validity of these case‐identification methods when compared with a reference standard. No systematic review of this kind has been conducted to the authors' knowledge.

METHODS

Search strategy, data source and selection criteria

This review is one of a series of systematic reviews that aims to assess the case‐identification methods used to identify chronic diseases in health care administrative data. This series of systematic reviews were conducted by the Quebec Support for People and Patient‐Oriented Research and Trials (SUPPORT Unit) as part of its mandate to facilitate access to and use of health care administrative data. The PRISMA‐P guidelines as well as production standards of the systematic reviews of the National Institute of Excellence in Health and Social Services were followed in the execution of this review. ,

Data sources

Six bibliographic databases were systematically searched (Medline, CINAHL, PsycINFO, AgeLine, Social Gerontology and Embase, January 1980 to June 2019). PubMed was searched from July 2017 to June 2019 to capture the most recent citations, which may not have been indexed in bibliographic databases. Sources of grey literature from both Quebec provincial organizations (Institut national de santé publique du Québec, Institut national d'excellence en santé et services sociaux, Régie de l'assurance maladie du Québec, Institut de la statistique du Québec and Institut de recherche et d'informations socio‐économiques) and Canadian organizations (Canadian Institute for Health Information and Health Canada) were also reviewed.

Selection criteria

Studies were included if they met all of the following criteria: (1) reported at least one case‐identification method for identifying individuals with obesity in health care administrative databases, (2) validated the case‐identification method(s) against a reference standard, (3) reported at least one of the following key measures of diagnostic accuracy: sensitivity, specificity, positive predictive value (PPV) or negative predictive value (NPV), (4) were written in English or French. To ensure comparability, studies were excluded if only kappa values were reported or if regression models were used to identify cases of obesity. Articles that were not associated with an original study or detailed report (e.g., a research protocol and commentary) were also excluded. Combinations of the following three concepts were used:

the health condition of interest (e.g., obesity, overweight and body mass index [BMI]);

administrative health care data (e.g., administrative database and medico‐administrative data);

case‐identification method validation measures (e.g., sensitivity, specificity, PPV and NPV).

The search strategy for each of the databases is available in Appendix A.

Data management and extraction

The identified articles were imported into a bibliographic management software. All results were combined, and duplicates were removed. Data extraction was carried out by two members of the research team (RD and ECC). In order to ensure systematic and standardized collection, the extraction of information was accompanied by a set of specifications that described in detail the collection of each of the variables of interest. In addition, the assistants received prior training on the type of information collected and how to collect it.

Extracted data were entered into a standardized data extraction spreadsheet designed specifically for the objectives of this study. This spreadsheet included the following information: the definition of obesity utilized in the article, the obesity case‐identification method used, the reference population and the performance of the case‐identification method used (sensitivity, specificity, PPV, NPV, Cohen's kappa and area under the curve [AUC]). This review followed a similar approach to what has been used to validate other disease case‐identification methods for administrative health care databases. , , ,

The primary reviewer was responsible for the initial extraction while the secondary reviewer was responsible for validating the information collected in the data extraction spreadsheet. In the event of nonconsensus, a third member of the research team (AMC) was consulted.

Evaluation of the validity of obesity case‐identification methods

A case‐identification method refers to the method utilized for identifying specific cases of a disease or other health marker in a health care administrative database. Different terminologies are used for health diagnosis depending on the data source and country. This may include codes for medical procedures, interventions, drugs prescribed or diagnostic codes.

The accuracy of case‐identification methods to identify individuals with obesity in health care administrative databases was evaluated by comparing these methods with a reference sample. A reference sample may be medical records, clinical data, self‐reported outcomes or treatment registries. The primary validation statistics of interest was based on sensitivity, specificity, PPV and NPV (Table 1). However, for completeness, some studies have reported AUC or Cohen's kappa coefficient.

TABLE 1

Indicators to assess the validity of case‐identification methods for identifying obesity cases in health care administrative data

Term	Theoretical definition	Definition in context
Sensitivity (Se)	TP/(TP + FN)	Proportion of individuals with a diagnosis of obesity in the reference sample who were identified with a code for obesity in the health care administrative database
Specificity (Sp)	TN/(TN + FP)	Proportion of individuals without a diagnosis of obesity in the reference sample who were not identified with a code for obesity in the health care administrative database.
Positive predictive value (PPV)	TP/(TP + FP)	Probability that individuals identified as suffering from obesity using the case‐identification method are actually affected by obesity in the reference sample
Negative predictive value (NPV)	TN/(TN + FN)	Probability that individuals who have not been identified as affected by obesity in the health care administrative database are not affected by obesity in the reference sample.

Note: TP: true positives are those that have the disease and are tested as positive by the screening/diagnostic test. FN: false negatives are that have the disease and are tested as negative by the screening/diagnostic test. TN: true negatives are the ones that are correctly identified as not having the disease being detected by the test. FP: false positives are the ones that are incorrectly identified had the disease being detected by the test.

Risk of bias analysis

An evaluation of the quality of studies was conducted by two authors (OSS and LI) using a modified version of the Quality Assessment of Diagnostic Accuracy Studies 2 evaluation tool for studies validating administrative data to assess risk of bias within and across studies. This tool is summarized in Appendix B. The risk of bias tool consists of four key domains covering (1) patient selection, (2) the administrative database, (3) the reference standard and (4) flow of patients through the study and timing of the administrative database and reference standard. Each domain was assessed in terms of the risk of bias (graded as high, low or unclear), and the first three domains were also assessed in terms of concerns regarding applicability.

RESULTS

Study selection

The PRISMA study selection flow diagram is shown in Figure 1. A total of 1771 articles were identified. Of these, 1447 unique articles remained after deduplication. Each of the titles and abstracts was screened, and 47 articles were selected for full‐text review. Of these, 14 met the inclusion criteria. Three additional articles, identified by the snowball method or in the grey literature, were added to these articles. A total of 17 original studies that report validated methods for identifying obesity individuals in health care administrative databases were included in this systematic review (Table 2).

FIGURE 1

Flowchart of study selection and review

TABLE 2

Characteristics of the studies included in the systematic review (n = 17)

Authors (year), Country	Population characteristics and study period	Reference standard
Caplan et al., 2018 22 United States	Patients aged 20–89 years old enrolled in a Medicare Advantage Prescription Drug or in a commercial health plan offered by Humana and having a documented V85 code for obesity, 2010–2015	Medical Chart review
McLynn et al., 2018 31 United States	Patients who underwent elective posterior lumbar fusion with or without interbody graft at a large academic hospital, 2013–2016	Medical Chart review
Ammann et al., 2017 10 United States	Adults aged 20 years and older with commercial or Medicare Advantage insurance health plan, 2013, 2014 and 2016	Medical Chart review
Chiu et al., 2017 32 United States	Veterans of the VA Greater Los Angeles Health Administration with an ICD‐9 diagnosis of diabetes mellitus with visits to the eye clinic, 1999–2016	Medical Chart review
Peng et al., 2017 16 Canada	Patients aged 18 years and older from four adult teaching hospitals in Alberta, Canada, 2003	Medical Chart review
Nickel et al., 2016 21 United States	Privately insured women aged 18 to 64 years old who underwent mastectomy for breast cancer, 2004–2011	Medical Chart review
Lau et al., 2015 20 United States	Patients who underwent primary total joint arthroplasty from three high volume total joint arthroplasty centres, 2010–2014	Medical Chart review
Lloyd et al., 2015 15 United States	Elderly aged 65 years and older from the National Health and Nutrition Examination Survey, 1999–2004	Clinical measurements
Samuel et al., 2015 33 United States	Patients aged 18 years and older with proximal tibia fracture, 2011–2012	Medical Chart review
Golinvaux et al., 2014 19 United States	Patients aged 18 years and older from a large tertiary care medical centre who spent at least one night in the hospital as an inpatient, 2013	Medical Chart review
Martin et al., 2014 12 Canada	Patients aged 18 years and older from the Alberta Provincial Project for Outcomes Assessment in the Coronary Heart Disease (APPROACH) registry, 2002–2008	Treatment Registry
Bozic et al., 2013 12 United States	Patients who underwent total joint arthroplasty from three high‐volume institutions, 2009	Medical Chart review
Andrade et al., 2011 14 United States	Women aged 12 to 49 years old who delivered an infant in a hospital or whose pregnancy ended in an induced abortion or miscarriage, 2006–2008	Medical Chart review
Kuhle et al., 2011 11 Canada	Children aged 10–11 years old from the Children's Lifestyle and School Performance Study, 2002–2004	Clinical measurements
Quan et al., 2008 13 Canada	Patients aged 18 years and older from four adult teaching hospitals in Alberta, Canada 2003	Medical Chart review
Varas‐Lorenzo et al., 2008 34 Canada	Saskatchewan Health beneficiaries aged 40 to 84 years old eligible for outpatient prescription drug benefits with a confirmed diagnosis of acute coronary syndrome (ICD‐9 410–411), 1999–2001	Medical Chart review
Yasmeen et al., 2006 23 United States	Women aged 10 to 55 years old who were discharged from a nonfederal licensed acute care hospital in California, after giving birth, 1992–1993	Medical Chart review

Study and population characteristics

The characteristics of the included studies are presented in Table 2. All studies were carried out in or after 2006. The majority of studies were conducted in the United States (n = 12/17) and the rest were conducted in Canada (n = 5/17). Three studies included only female populations, , , one study included only children and one study included only people aged 65 years and older. The remaining studies focused on mixed populations. All studies were conducted according to a retrospective cohort design with the exception of Golinvaux et al., which was a cross‐sectional survey.

Case‐identification methods

All studies used the ICD codes to identify individuals with obesity. No study identified individuals with obesity in health care administrative data based on any other codes, such as codes related to the prescription of drugs or the use of health services. The specific ICD codes utilized are outlined in Table 3.

TABLE 3

Administrative codes and classification of obesity

ICD edition	Obesity ICD code
ICD‐8	Obesity: 277
ICD‐9 codes	Obesity: 278.x, V85.3x, V85.4x Severe obesity: 278.01, V85.4x.
ICD‐9‐CM	Underweight: 783.22, V85.0. Normal weight V85.1; Overweight 278.02, V85.21–V85.25; Obese 278.00, 278.01, 278.03, V85.30–V85.39, V85.41–V85.45; Severe obese (class II) 278.01, V85.35–V85.39, V85.41–V85.45; Severe obese (class III) 278.01, V85.41–V85.45.
ICD‐10 codes	Obesity: E66.x, Z68.3x, Z68.4x. Severe obesity: E66.01, Z68.4x
ICD‐10‐CM	Underweight R63.6, Z68.1; Normal weight Z68.20–Z68.24; Overweight E66.3, E68.25–Z68.29; Obese E66.09, E66.1, E66.2, E66.8, E66.9, Z68.30–Z68.39, Z68.41–Z68.45; Severe obese (class II) E66.01, E66.2, Z68.35–Z68.39, Z68.41–Z68.45; Severe obese (class III) E66.01, E66.2, Z68.41–Z68.45.

Reference standard

Obesity case‐identification methods (ICD codes) were compared with a reference standard to determine the level of congruence between health care administrative data and other sources of health data. Of the 17 studies reviewed, two compared case‐identification methods with clinical measurements (Table 4), 14 compared against medical records (Table 5) and one compared against a treatment registry (Table 6) to validate the obesity case‐identification method. No studies have compared health care administrative data with self‐reported obesity. All three reference standards defined adults with obesity as those having a BMI greater or equal to 30. In the case of children, obesity was defined using appropriate BMI cut‐off points on CDC growth charts.

TABLE 4

Clinical measurements as gold standard (n = 2 studies)

Author, year	Years of data collection	Codes used	Case identification method definitions	N	Se (95% CI)	Sp (95% CI)	PPV (95% CI)	NPV (95% CI)	Characteristics
Lloyd et al., 2015	1999–2007	ICD‐9‐CM: 278.0, 278.00, 278.01.	≥1 claim‐based diagnosis code for obesity
			All	3554	18.4 (‐)	97.3 (‐)	73.6 (‐)	74.2 (‐)	BMI ≥ 30
					34.2 (‐)	95.8 (‐)	47.9 (‐)	92.7 (‐)	BMI ≥ 35
			Diagnosis of diabetes	520	34.4 (‐)	97.5 (‐)	91.7 (‐)	65.2 (‐)	BMI ≥ 30
					54.3 (‐)	93.3 (‐)	68.2 (‐)	88.6 (‐)	BMI ≥ 35
			Diagnosis of CHF	434	25.9 (‐)	96.5 (‐)	81.1 (‐)	69.8 (‐)	BMI ≥ 30
					51.9 (‐)	94.0 (‐)	54.2 (‐)	93.4 (‐)	BMI ≥ 35
			Diagnosis of COPD	237	38.8 (‐)	96.4 (‐)	76.1 (‐)	84.1 (‐)	BMI ≥ 30
					58.8 (‐)	93.0 (‐)	45.6 (‐)	95.8 (‐)	BMI ≥ 35
			Diagnosis of depression	177	25.2 (‐)	93.7 (‐)	57.6 (‐)	78.8 (‐)	BMI ≥ 30
					58.6 (‐)	93.2 (‐)	43.7 (‐)	96.2 (‐)	BMI ≥ 35
Kuhle et al., 2011	2002–2004	ICD‐9: 278. ICD‐10: E66–E68.	≥1 diagnosis code for obesity as a primary or secondary diagnosis from either a physician visit or a hospital stay.	3230	7.4 (5.3–9.9)	99.7 (99.4–99.9)	‐	‐

Abbreviations: BMI, body mass index; CHF, congestive heart failure; CI, confidence interval; COPD, chronic obstructive pulmonary disease; ICD‐9, International Classification of Diseases, 9th Revision; ICD‐9‐CM, International Classification of Diseases, 9th Revision, Clinical Modification; ICD‐10, International Classification of Diseases, 10th Revision; N, number; NPV, negative predictive value; PPV, positive predictive value; Se, sensitivity; Sp, specificity.

TABLE 5

Medical records review as gold‐standard (n = 14 studies)

Author, year	Years of data collection	Codes used	Case identification method definitions	N	Se (95% CI)	Sp (95% CI)	PPV (95% CI)	NPV (95% CI)	Characteristics
Ammann et al., 2018	2013, 2014, 2016	ICD‐9‐CM codes a ICD‐10‐CM codes b	≥1 ICD‐9‐CM or ICD‐10‐CM diagnosis code for obesity during the calendar years 2013, 2014 and 2016.	1 116 283 c	10.7 (‐)	‐	49.0 (‐)	98.8 (‐)	BMI < 18.5
					3.7 (‐)	‐	89.6 (‐)	73.9 (‐)	BMI 18.5–24
					6.0 (‐)	‐	73.4 (‐)	67.6 (‐)	BMI 25–29
					25.2 (‐)	‐	92.4 (‐)	68.0 (‐)	BMI ≥ 30
					22.8 (‐)	‐	89.1 (‐)	‐	BMI ≥ 35
					34.8 (‐)	‐	69.1 (‐)	‐	BMI ≥ 40
	2013, 2014		≥1 ICD‐9‐CM diagnosis code for obesity during the calendar years 2013 and 2014.	676 989 c	8.6 (‐)	‐	59.0 (‐)	‐	BMI < 18.5
					2.7 (‐)	‐	89.2 (‐)	‐	BMI 18.5–24
					4.3 (‐)	‐	69.9 (‐)	‐	BMI 25–29
					21.8 (‐)	‐	92.0 (‐)	‐	BMI ≥ 30
	2016		≥1 ICD‐10‐CM diagnosis code for obesity during the calendar year 2016.	439 294 c	13.9 (‐)	‐	42.1 (‐)	‐	BMI < 18.5
					5.2 (‐)	‐	89.9 (‐)	‐	BMI 18.5–24
					8.6 (‐)	‐	76.5 (‐)	‐	BMI 25–29
					30.2 (‐)	‐	92.9 (‐)	‐	BMI ≥ 30
Caplan et al., 2018	2010–2015	ICD‐9‐CM V85.0, V85.1, V85.2x, V85.3x, V85.4, V85.4x	≥1 claim‐based code for obesity documented during the index period.	207	‐	‐	90.3 (86.3–94.4)	‐	MAPD, all BMI
					‐	‐	71.0 (55.0–87.0)	‐	MAPD, <18.5
					‐	‐	93.8 (85.4–100)	‐	MAPD, BMI 18.5–24
					‐	‐	97.4 (92.5–100)	‐	MAPD, BMI 25–29
					‐	‐	96.9 (90.9–100)	‐	MAPD, BMI ≥ 30
					‐	‐	97.0 (91.1–100	‐	MAPD, BMI ≥ 35
					‐	‐	85.0 (73.3–96.1)	‐	MAPD, BMI ≥ 40
				21	‐	‐	91.1 (87.3–94.9)	‐	Commercial, all BMI
					‐	‐	75.9 (60.3–91.4)	‐	Commercial, BMI < 18.5
					‐	‐	87.8 (77.8–97.8)	‐	Commercial, BMI 18.5–24
					‐	‐	93.5 (84.9–100)	‐	Commercial, BMI 25–29
					‐	‐	97.2 (91.9–100)	‐	Commercial, BMI ≥ 30
					‐	‐	93.0 (85.4–100)	‐	Commercial, BMI ≥ 35
					‐	‐	97.1 (91.4–100)	‐	Commercial, BMI ≥ 40
McLynn et al., 2018	2013–2016	ICD‐9 codes d ICD‐10 codes e	≥1 ICD‐9 or ICD‐10 code for obesity during the study period.	796	42.5 (‐)	99.2 (‐)	‐	‐	All
					54.1 (‐)	‐	‐	‐	DM (yes)
					38.3 (‐)	‐	‐	‐	DM (no)
					49.6 (‐)	‐	‐	‐	ASA class III (yes)
					33.3 (‐)	‐	‐	‐	ASA class III (no)
					69.2 (‐)	‐	‐	‐	Post‐op VTE (yes)
					41.6 (‐)	‐	‐	‐	Post‐op VTE (no)
					57.9 (‐)	‐	‐	‐	Major AE (yes)
					40.9 (‐)	‐	‐	‐	Major AE (no)
					53.8 (‐)	‐	‐	‐	Any AE (yes)
					40.3 (‐)	‐	‐	‐	Any AE (no)
Chiu et al., 2017	1999–2016	ICD‐9 codes not specified	≥1 diagnosis code for obesity at the follow up time point	100	50 (45–92)	95 (88–98)	91 (81–97)	63 (54–71)
Peng et al., 2017	2003	ICD‐10‐CA codes not specified	≥1 diagnosis code for obesity.	4007	89.6 (77.3–96.5)	‐	91.5 (79.6–97.6)		All records
				3891	18.3 (14.2–22.9)	‐	84.3 (73.6–91.9)		Among records without a status of death
				105	30.0 (6.7–65.2)	‐	75.0 (19.4–99.4)		Among records with a status of death
Nickel et al., 2016	2004–2011	ICD‐9‐CM: 278.00, 278.01, 278.03, 649.10–649.14, 793.91, V85.30–V85.39, V85.41–V85.45.	≥2 provider or outpatient facility claims for obesity spaced ˃30 days apart or ≥1 inpatient claim for obesity during the 1‐year period before the mastectomy	174	7.14 (‐)	100 (‐)	100 (‐)	69.41 (‐)
			≥2 provider or outpatient facility claims for obesity spaced ˃30 days apart or ≥1 inpatient claim for obesity during the 1‐year period before the mastectomy and the 1‐week period after the mastectomy	174	12.50 (‐)	100 (‐)	100 (‐)	70.66 (‐)
			≥1 provider or outpatient facility claim for obesity or ≥1 inpatient claim for obesity during the 1‐year period before the mastectomy and the 1‐week period after the mastectomy.	174	17.86 (‐)	99.15 (‐)	90.91 (‐)	71.78 (‐)
Samuel et al., 2015	2011–2012	ICD‐9: 278.00, 278.01.	≥1 diagnosis code for obesity.	32 411	8.9 (‐)	‐	‐	‐
Lau et al., 2015	2010–2014	ICD‐9: V85.xx	≥1 diagnostic code for obesity.	315	100 (‐)	‐	‐	‐	THA
					‐	100 (‐)	‐	‐	THA and BMI ≥ 40
				442	91.5 (‐)	‐	‐	‐	TKA
					‐	90.6 (‐)	‐	‐	TKA and BMI ≥ 40
		ICD‐9: 278.xx	≥1 diagnostic code for obesity.	315	67.7 (‐)	‐	‐	‐	THA
					‐	50.0 (‐)	‐	‐	THA and BMI ≥ 40
				442	65.6 (‐)	‐	‐	‐	TKA
					‐	78.1 (‐)	‐	‐	TKA and BMI ≥ 40
Golinvaux et al., 2014	2013	ICD‐9: 278.00	1 diagnosis code for unspecified obesity	2075	19 (‐)	97 (‐)	70 (‐)	76 (‐)
		ICD‐9: 278.01	1 diagnosis code for severe obesity	2075	48 (‐)	99 (‐)	81 (‐)	95 (‐)
Bozic et al., 2013	2009	ICD‐9‐CM: 278.00, 278.01, V85.3, V85.4.	≥1 diagnostic code for obesity during the study period.	1350	76.4 (‐)	91.8 (‐)	49.2 (‐)	‐
Andrade et al., 2011	2006–2008	ICD‐9‐CM: 278, 278.0, 278.00–278.02, 649.10–649.14.	≥1 diagnosis code indicating obesity or overweight status during the 1‐year period prior to end date of pregnancy.	18 312	33 (32–35)	99 (99–99)	93 (92–96)	‐
Quan et al., 2008	2003	ICD‐10‐CA codes not specified	≥1 ICD‐10‐CA code for obesity (original coding).	4008	18.6 (‐)	99.7 (‐)	83.8 (‐)	93.1 (‐)
		ICD‐9‐CM codes not specified	≥1 ICD‐9‐CM code for obesity recoded from the original coding.	4008	24.6 (‐)	99.3 (‐)	75.9 (‐)	93.6 (‐)
Varas‐Lorenzo et al., 2008	1999–2001	ICD‐8: 277. ICD‐9: 278.	≥1 code for obesity recorded in the hospital or medical services databases at hospital admission for confirmed ACS	431	43.9 (34.3–53.8)	86.2 (81.4–90.2)	25.5 (19.1–32.0)	35.7 (31.6–39.2)
Yasmeen et al., 2006	1992–1993	ICD‐9‐CM: 278	≥1 code for obesity	1611	12 (‐)	‐	62 (‐)	‐	Unweighted estimates
					11 (‐)	‐	49 (‐)	‐	Weighted estimates
		ICD‐9‐CM: 646.1x	≥1 code for obsessive gain weight	1611	5 (‐)	‐	29 (‐)	‐	Unweighted estimates
					3 (‐)	‐	19 (‐)	‐	Weighted estimates
		ICD‐9‐CM: 278, 646.1x	≥1 code for obesity or obsessive gain weight	1611	14 (‐)	‐	83 (‐)	‐	Unweighted estimates
					13 (‐)	‐	71 (‐)	‐	Weighted estimates

Abbreviations: ACS, acute coronary syndrome; BMI, body mass index; CI, confidence interval; ICD‐8, International Classification of Diseases, 8th Revision; ICD‐9, International Classification of Diseases, 9th Revision; ICD‐9‐CM, International Classification of Diseases, 9th Revision, Clinical Modification; ICD‐10‐CA, International Classification of Diseases, 10th Revision, Canada; ICD‐10‐CM, International Classification of Diseases, 10th Revision, Clinical Modification; N, number; NPV, negative predictive value; PPV, positive predictive value; Se, sensitivity; Sp, specificity; TKA, total knee arthroplasty; THA, total hip arthroplasty.

person‐years.

ICD‐9‐CM codes: Underweight: 783.22, V85.0. Normal weight V85.1; Overweight 278.02, V85.21–V85.25; Obese 278.00, 278.01, 278.03, V85.30–V85.39, V85.41–V85.45; Severe obese (class II) 278.01, V85.35–V85.39, V85.41–V85.45; Severe obese (class III) 278.01, V85.41–V85.45.

ICD‐10‐CM codes: Underweight R63.6, Z68.1; Normal weight Z68.20–Z68.24; Overweight E66.3, E68.25–Z68.29; Obese E66.09, E66.1, E66.2, E66.8, E66.9, Z68.30–Z68.39, Z68.41–Z68.45; Severe obese (class II) E66.01, E66.2, Z68.35–Z68.39, Z68.41–Z68.45; Severe obese (class III) E66.01, E66.2, Z68.41–Z68.45.

ICD‐9 codes: Obesity: 278.x, V85.3x, V85.4x Severe obesity: 278.01, V85.4x.

ICD‐10 codes: Obesity:E66.x, Z68.3x, Z68.4x. Severe obesity: E66.01, Z68.4x.

TABLE 6

Registry as gold‐standard (n = 1 study)

Author, year	Years of data collection	Codes used	Case identification method definitions	N	Se (95% CI)	Sp (95% CI)	PPV (95% CI)	NPV (95% CI)	Characteristics
Martin et al., 2014	2002–2008	ICD‐10: E65–E68.	≥1 diagnostic code for obesity in any position in the 25 diagnosis coding fields.
			Overall	17 380	7.75 (‐)	98.98 (‐)	65.94 (61.38–70.51)	80.84 (‐)	2002–2008
			Year	17 380	8.24 (‐)	98.49 (‐)	58.82 (48.36–69.29)	80.41 (‐)	2002
					9.48 (‐)	98.58 (‐)	61.25 (50.57–71.93)	82.16 (‐)	2003
					8.69 (‐)	99.14 (‐)	73.13 (62.52–83.75)	80.02 (‐)	2004
					5.81 (‐)	99.34 (‐)	69.05 (55.07–83.03)	80.60 (‐)	2005
					6.44 (‐)	99.53 (‐)	76.92 (63.70–90.15)	81.38 (‐)	2006
					5.87 (‐)	99.14 (‐)	63.41 (48.67–78.16)	80.64 (‐)	2007
					9.37 (‐)	98.78 (‐)	66.67 (54.74–78.59)	80.67 (‐)	2008
			Age	17 380	7.58 (‐)	99.03 (‐)	68.57 (‐)	79.27 (‐)	<55 years old
					7.63 (‐)	98.77 (‐)	63.64 (‐)	79.19 (‐)	55–65 years old
					8.02 (‐)	99.06 (‐)	69.49 (‐)	80.08 (‐)	65–75 years old
					7.76 (‐)	99.08 (‐)	60.00 (‐)	85.85 (‐)	≥75 years old
			Gender	17 380	9.84 (‐)	98.79 (‐)	76.26 (‐)	73.52 (‐)	Female
					6.14 (‐)	99.06 (‐)	56.48 (‐)	84.11 (‐)	Male
			Indication for coronary catheterization	17 380	8.77 (‐)	98.92 (‐)	68.99 (‐)	79.81 (‐)	Stable angina
					7.85 (‐)	98.92 (‐)	63.19 (‐)	82.01 (‐)	MI
					6.88 (‐)	98.85 (‐)	63.74 (‐)	78.28 (‐)	Unstable angina
					6.52 (‐)	99.54 (‐)	74.19 (‐)	83.87 (‐)	Other
			Diabetes	17 380	6.18 (‐)	99.19 (‐)	62.55 (‐)	82.86 (‐)	No
					11.39 (‐)	98.08 (‐)	70.76 (‐)	73.05 (‐)	Yes
			CVD	17 380	7.49 (‐)	98.98 (‐)	65.07 (‐)	80.85 (‐)	No history
					10.94 (‐)	99.00 (‐)	74.36 (‐)	80.73 (‐)	History
			CHF	17 380	7.64 (‐)	98.98 (‐)	65.73 (‐)	80.74 (‐)	No
					8.46 (‐)	98.99 (‐)	67.24 (‐)	81.48 (‐)	Yes
			Hypertension	17 380	7.80 (‐)	99.18 (‐)	63.87 (‐)	85.27 (‐)	No
					7.73 (‐)	98.86 (‐)	66.78 (‐)	78.36 (‐)	Yes
			Hyperlipidaemia	17 380	7.83 (‐)	98.87 (‐)	65.11 (‐)	79.91 (‐)	No
					7.49 (‐)	99.27 (‐)	68.82 (‐)	83.27 (‐)	Yes
			Prior MI	17 380	7.43 (‐)	98.99 (‐)	65.66 (‐)	80.59 (‐)	No
					8.73 (‐)	98.95 (‐)	66.96 (‐)	81.60 (‐)	Yes

Abbreviations: BMI, body mass index; CHF, congestive heart failure; CI, confidence interval; CVD, cerebrovascular diseases; ICD‐10, International Classification of Diseases, 10th Revision; N, number; MI, myocardial infarction; NPV, negative predictive value; PPV, positive predictive value; Se, sensitivity; Sp, specificity.

ICD‐9‐CM codes

ICD‐10‐CM codes

ICD‐9 codes

ICD‐10 codes

Comorbidities

In six of the 17 studies, the case‐identification methods were designed to select individuals with obesity with other specified chronic conditions, including pain, , diabetes, fractures, arthroplasty and acute coronary syndromes. In these cases, obesity was assessed as a comorbidity.

Statistical validity

The statistical validity of the case‐identification methods was assessed using four indicators: sensitivity (n = 16/17), specificity (n = 13/17), PPV (n = 13/17) and NPV (n = 12/17). Three studies also carried out concordance analysis, specifically Cohen's kappa test (n = 2) , and the calculation of the AUC curve (n = 1).

Table 4 compares obesity case‐identification methods in health care administrative databases with clinical measures (n = 2 studies). These methods had overall high performance in terms of specificity, PPV and NPV. On the other hand, the sensitivity values recorded were very low. Even when the analyses were restricted to individuals with other comorbidities (diabetes, depression, chronic obstructive pulmonary disease, congestive heart failure), the maximum sensitivity observed was only 38.8% for a single BMI group.

Table 5 compares obesity case‐identification methods for health care administrative databases with medical records (n = 14). These methods showed low sensitivity values (always less than 50%) except in two studies. , On the other hand, the specificities were always above 50% with values reaching 100%. The PPV fluctuated from 19% to 100%, and NPV fluctuated from and 36% to 99%. The case‐identification methods were more effective in detecting cases of obesity when combined with other comorbidities. The best performances in detecting cases of obesity, considering both sensitivity and specificity, were observed in a group of 50 patients also suffering from diabetes retinopathy (sensitivity = 50%, specificity = 95%).

The only study that compared obesity identification methods with data from a treatment registry (Table 6) reported sensitivity values that were less than 12% regardless of the subgroup considered (age, gender and associated comorbidities). Conversely, the specificity values obtained in this study were high (above 98%). Similarly, the PPV and NPV were also high.

Validation of methods for identifying severe obesity

Among the articles included in this review, six studies evaluated the statistical validity of methods in people suffering from severe obesity. , , , , , In individuals with BMI between 35 and 39 kg/m2, sensitivity varied (depending on the studies and association with other comorbidities) from 22.8% to 58.8%, whereas specificity always exceeded 93%. In individuals with a BMI greater or equal to 40 kg/m2, sensitivity values ranged from 33.3% to 49.6%. The reporting of validation statistics was often incomplete in the case of severe obesity, likely due to the small number of cases.

This review assessed the quality of the included studies using a modified version of the QUADAS‐2 tool. This assessment found that most studies did not include enough information to determine whether the risk of bias was low or high. In cases where enough information was presented, the risk of bias was generally reported as being low.

DISCUSSION

The objectives of this systematic review were to (1) determine the case‐identification methods used to identify individuals with obesity in health care administrative databases and (2) to summarize the validity of these case‐identification methods when compared with a reference standard. Understanding the validity of case‐identification methods for identifying individuals with obesity in health care administrative databases is critical to improving surveillance and research studies that utilize these databases. In order to understand the validity of case‐identification methods, the main statistical indicators (sensitivity, PPV, specificity and NPV) that are used to evaluate the performance of these case‐identification methods in the literature were reviewed. The validity of these case‐identification methods was then summarized and compared with a reference standard, in order to assess the implications of using health care administrative databases for epidemiological surveillance and other research activities.

The sensitivity of the case‐identification method when compared with all three reference standards was generally very low. This means that a low number of individuals who were identified as suffering from obesity in the reference sample were also identified as suffering from obesity in the health care administrative database. This low sensitivity could be explained by the fact that obesity is often not encoded by physicians in health records and registers. This may be due to a lack of time, the fact that obesity is not always considered a disease, there is only place for one primary diagnostic code or that the diagnosis of obesity may be based on subjective observation by the clinician, and not on an objective measure of BMI, thus only capturing cases of severe obesity. Finally, in countries, such as the United States, that rely on reimbursement from insurance companies, obesity‐related comorbidities are often listed as the primary diagnosis as obesity is not reimbursable. It should also be noted that obesity was only considered a disease by the American Medical Association in 2013, which is in the middle of the search period. This may influence the results of this study as well as studies of this issue in the future. The consequence of poor sensitivity of obesity‐identification methods can be twofold. First, it may lead to underestimation of the prevalence of obesity in administrative databases. Second, it may lead to an inaccurate estimation of health care costs associated with obesity from health care administrative databases. To illustrate this point, Kuhle showed that when comparing the costs of health care usage in children with obesity and normal weight children, costs were 16% higher in children with clinically measured obesity. This cost differential increased to 108% when the analysis was based on data extracted from a health care administrative database using case‐identification methods based on ICD codes. Thus, using only obesity‐identification codes to select individuals with obesity in a health care administrative database severely overestimated health care costs for children with obesity. In conclusion, the low sensitivity of the case‐identification methods suggests that they should not be used for monitoring the prevalence or incidence of obesity.

Conversely, PPV was high in the majority of studies. In other words, if an individual is coded as having a BMI greater or equal to 30 in a health care administrative database, they are very likely to have a BMI greater or equal to 30 according to the reference standard. Thus, identification codes of individuals with obesity in administrative data are very accurate. Several authors point to the possible explanation that physicians are more likely to diagnose obesity if multiple, potentially obesity‐related comorbidities are present or alternatively if the person suffered from severe obesity. , , , , , , , Case‐identification methods can therefore be used to identify, with good precision, cohorts of individuals suffering from severe obesity from administrative databases. These cohorts will likely be made up of individuals who are ill or who are affected by severe obesity.

The specificity of case‐identification methods for the identification of individuals with obesity was generally higher than the sensitivity. This means that a large proportion of individuals without a diagnosis of obesity in the reference sample also did not have a diagnosis of obesity in the health care administrative database. This finding confirms the fact that the encoding of obesity in administrative databases is incomplete. Indeed, the weight status of individuals with obesity is rarely entered in administrative databases. This poor completeness of reporting of individuals with obesity leads to a significant bias in the estimation of the negative predictive value. The NPV, which is very poorly reported in these studies, should be interpreted with caution.

The main contribution of this article is the rigorous manner in which the literature has been searched to identify case‐identification methods compared with a reference standard using identified measures of performance. The main limitation of this research is that none of the available reference standards are validated population measures of obesity. They also suffer from many of the same reliability issues as health care administrative databases such as issues with data entry and reliability of entry as were discussed above.

Despite these limitations, some conclusions were able to be drawn. Overall, several factors that are likely to positively affect the accuracy of the case‐identification methods were identified. Case‐identification methods tended to be more accurate when reported with comorbidities associated with obesity and in cases of severe obesity. , , , , , , , , This may be explained by the fact that subjects with severe obesity or comorbidities are more likely to be coded as such in administrative databases because the patient's weight has direct relevance to the disease of interest. This may lead to a positive bias in the coding of obesity for these groups.

CONCLUSION

The performance of case‐identification methods for identifying individuals with obesity in administrative databases was inconsistent. The sensitivity of these methods was usually low while the specificity was higher. Thus, when obesity is reported in health care administrative databases, it is usually correctly reported; however, obesity tends to be highly underreported in these databases. Therefore, case‐identification methods to monitor the prevalence and incidence of obesity within health care administrative databases should be used with caution. Despite this, the use of these methods remains relevant for the selection of cohorts of individuals with obesity for research activities. Further research on how to improve reporting of obesity in health care administrative databases is urgently required.

CONFLICT OF INTEREST

No conflict of interest was declared.

Medline
Obesity	(MH ‘obesity’) OR (MH ‘overweight’) OR obes* OR overweight* OR malnutrition OR ‘body mass index*’ OR ‘BMI’ OR ‘waist‐hip ratio*’ OR ‘waist hip ratio*’ OR ‘waist circumference’ OR ‘abdominal fat’
	AND
Case‐identification method validation	(MH ‘Sensitivity and Specificity’) OR specificit* OR sensitivit* OR ‘predictive value*’ OR ‘positive predictive value*’ OR ‘ppv’ OR ‘negative predictive value*’ OR ‘npv’ OR valid* OR ‘roc curve*’ OR ‘roc’ OR ‘receiver operating characteristic*’ OR ‘auc’ OR ‘area under curve*’ OR kappa*
	AND
Administrative data	(MH ‘Health Information Systems’) OR (MH ‘Billing and Claims’) OR (MH ‘Coding’) OR (MH ‘Databases, Factual’) OR ‘administrative data*’ OR ‘medico‐administrative data*’ OR ‘administrative register data*’ OR ‘health* administrative data*’ OR ‘administrative code*’ OR ‘medico‐administrative code*’ OR ‘health* administrative code*’ OR ‘health* data*’ OR ‘billing data*’ OR ‘billing code*’ OR claim*

Domain	Patient selection	Administrative database	Reference standard	Flow and timing
Description	Describe methods of patient selection	Describe the administrative database and how it was used and interpreted	Describe the reference standard and how it was conducted and interpreted	Describe any patients in the validation cohort who were not found within the reference standard or who were excluded from cross‐tabulation of the administrative data diagnoses results against the results of the reference standard diagnoses
	What is the study question?	Where available, include comment on how coding was done and by whom.	Where available, include comment on quality of the reference standard, including the level of experience of clinicians making the diagnosis, access to diagnostic tests such as physical exam evaluating height, weight, waist circumference measurement and lab tests to check for comorbidities.	Describe the time interval and any interventions between administrative database diagnosis and reference standard diagnosis
Signalling questions (yes/no/unclear)	Was a consecutive or random sample of patients enrolled?	Were the administrative database diagnosis results interpreted without knowledge of the results of the reference standard diagnosis?	Is the reference standard likely to correctly classify obesity?	Was there an appropriate interval between administrative database diagnosis and reference standard diagnosis?
	Did the study avoid inappropriate exclusions?	If a diagnostic threshold was used, was it pre‐specified?	Were the reference standard results interpreted without knowledge of the results of the administrative database diagnosis?	Did all patients receive a reference standard?
				Did all patients receive the same reference standard?
				Were all patients included in the analysis?
Risk of bias: High/low/unclear	Could the selection of patients have introduced bias?	Could the conduct or interpretation of the administrative database have introduced bias?	Could the reference standard, its conduct, or its interpretation have introduced bias?	Could the patient flow have introduced bias?
Concerns regarding applicability: High/low/unclear	Are there concerns that the included patients do not match the study question?	Are there concerns that the administrative database, its conduct, or interpretation differ from the study question?	Are there concerns that obesity, as defined by the reference standard, does not match the study question?