Validation of an Optimized Algorithm for Identifying Persons Living With Diagnosed HIV From New York State Medicaid Data, 2006-2014.

Algorithms are regularly used to identify persons living with diagnosed human immunodeficiency virus (HIV) (PLWDH) in Medicaid data. To our knowledge, there are no published reports of an HIV algorithm from Medicaid claims codes that have been compared with an HIV surveillance system to assess its sensitivity, specificity, positive predictive value, and negative predictive value in identifying PLWDH. Therefore, our aims in this study were to 1) develop an algorithm that could identify PLWDH in New York State Medicaid data from 2006-2014 and 2) validate this algorithm using the New York State HIV surveillance system. Classification and regression tree analysis identified 16 nodes that we combined to create a case-finding algorithm with 5 criteria. This algorithm identified 86,930 presumed PLWDH, 88.0% of which were verified by matching to the surveillance system. The algorithm yielded a sensitivity of 94.5%, a specificity of 94.4%, a positive predictive value of 88.0%, and a negative predictive value of 97.6%. This validated algorithm has the potential to improve the utility of Medicaid data for assessing health outcomes and programmatic interventions.

Abbreviations

acquired immunodeficiency syndrome

All Patient Refined Diagnosis Related Groups

classification and regression tree

Centers for Disease Control and Prevention

case-finding algorithm

Health Home

human immunodeficiency virus

International Classification of Diseases, Ninth Revision, Clinical Modification

negative predictive value

New York State

New York State Department of Health

persons living with diagnosed HIV

positive predictive value

The federal Medicaid program is the largest health insurer of people who have been diagnosed with human immunodeficiency virus (HIV) infection (40%) (1, 2). Consequently, Medicaid data are a robust source of health-care information and are often used to study multiple facets of HIV disease in the United States, including health-care expenses (3–5), health disparities (6), trends in hospitalization (7, 8), and quality of HIV care (9–11). The size of the Medicaid data set allows for high statistical power and subgroup analysis, as well as the study of low-prevalence events, such as HIV diagnosis, with high statistical power (12–14).

While there are many advantages of using Medicaid data to study HIV care outcomes, data are collected for billing rather than clinical or research purposes (14–17). Medicaid researchers have indicated problems with accuracy and reliability when using Medicaid data (16, 18–25), as claims codes often vary by procedure, type of medical encounter, and clinical condition (14, 22–24, 26). A variety of algorithms have been developed to identify likely persons living with diagnosed HIV (PLWDH) within the Medicaid population (4, 7, 9–11, 13, 22, 23, 27–29). The accuracy and reliability of using diagnosis information in Medicaid data are higher for inpatient claims than for outpatient claims (13), but the number of hospitalizations for PLWDH are decreasing (8, 30). Because most Medicaid claims for PLWDH are for outpatient services (13) and not for diagnostic purposes, new algorithms need be sensitive to a variety of procedure and service codes, in addition to diagnosis codes.

Optimal algorithms should share attributes of the Centers for Disease Control and Prevention (CDC)-defined “optimal surveillance system” (31), such as simplicity, flexibility, representativeness, sensitivity, and a high positive predictive value (PPV) (32). Despite the regular use of algorithms to identify PLWDH from Medicaid data (7, 9–11, 13, 22, 27), to our knowledge there are no published reports of an HIV algorithm derived from Medicaid data that has been validated by matching the results to a population-based HIV surveillance system (the gold standard) to test the algorithm’s sensitivity, specificity, PPV, and negative predictive value (NPV). The New York State (NYS) Medicaid program utilizes its own internally developed algorithm to identify HIV-positive persons. That algorithm has not been validated against the NYS HIV surveillance system and has not been published.

NYS has approximately 113,000 PLWDH (33) and more PLWDH on Medicaid than any other state (34). A validated algorithm that accurately identifies PLWDH in NYS Medicaid data would enable researchers and policy-makers to better understand the impact of expanded Medicaid programs for PLWDH (34). Thus, our aims in this study were 2-fold: 1) to develop an algorithm for identifying PLWDH using NYS Medicaid data and 2) to validate the algorithm using the population-based NYS HIV surveillance system.

METHODS

Case identification and validation

Name-based reports for all HIV-related laboratory test results (e.g., CD4 cell counts and percentages, viral load tests, genotype resistance testing) are received by the New York State Department of Health’s (NYSDOH) HIV surveillance system for persons who reside or receive HIV-related care in NYS. This information is supplemented with data from the medical provider and partner report forms. In accordance with NYS public health law, clinicians are required to submit a report for all new or previously unreported diagnoses of HIV infection/acquired immunodeficiency syndrome (AIDS) within 14 days of diagnosis. The surveillance system routinely undergoes quality control and verification processes to ensure quality and completeness of the data. For this analysis, PLWDH were defined as persons with confirmed cases of HIV infection per the revised 2014 CDC definition (35). Only confirmed HIV diagnoses (regardless of disease stage) that met the CDC case definition were included in these analyses (Figure 1).

Figure 1

Identification and evaluation of presumptive persons living with diagnosed HIV (PLWDH) from New York State (NYS) Medicaid data, 2006–2014. CART, classification and regression tree; HIV, human immunodeficiency virus; NPV, negative predictive value; PPV, positive predictive value.

Medicaid encounter and claims data for 2006–2014 were obtained from the NYS Medicaid Data Mart for persons aged 18–64 years who met eligibility criteria for the NYS Medicaid Health Home (HH) Program. To qualify for the HH Program in NYS, individuals must have 2 or more medically documented chronic conditions included in the “major” categories of the 3M (3M Company, St. Paul, Minnesota) Clinical Risk Groups (e.g., alcohol and substance use disorder, mental health, cardiovascular disease, metabolic disease, or respiratory disease) or have a single qualifying diagnosis such as HIV, AIDS, and/or a serious mental illness (36) (Figure 1).

Because of the resource-heavy nature of matching large data sets, it was infeasible to compare all New Yorkers enrolled in Medicaid against the NYS HIV surveillance system; rather, for this study, an inclusive cohort of individuals—designated “potential PLWDH”—were identified from Medicaid-enrolled HH-eligible persons (n = 269,257). Published literature (3–5, 7, 9–11, 13, 22, 27, 28, 37–41) and an internal NYSDOH–AIDS Institute algorithm (42) were used to establish criteria for identification of potential PLWDH in Medicaid data. Criteria for identification of PLWDH in the HH-eligible Medicaid data included: 1) a diagnosis code for HIV (3–5, 7, 9–11, 13, 22, 27, 28, 37–40, 42); 2) rate codes for AIDS hospice care (4, 38, 42), HIV-related outpatient services (4, 37), or an HIV Special Needs Plan (4, 38, 42); 3) a National Drug Code for an HIV antiretroviral medication (4, 13, 37, 38, 40, 43, 44); 4) a procedure code for HIV viral quantification, phenotyping, or genotyping (4, 9, 38, 40, 43, 44); or 5) 3 claims with a 3M Clinical Risk Group between 102 and 105 (45). Individuals were also identified as potential PLWDH if they had a combination of claims with codes for HIV-related outpatient services (4, 5, 13, 37, 38, 42, 44), provider specialties (4, 37, 42), or a diagnosis of an opportunistic infection (4, 42, 44). (See Web Table 1, available at https://academic.oup.com/aje, for a detailed description of the inclusion criteria.)

A potential PLWDH was categorized as a true case if the record could be matched to an existing record in the NYS HIV surveillance system, which was considered the gold standard for this analysis. Potential PLWDH were probabilistically matched to the surveillance system using combinations of name, sex, race/ethnicity, address, date of birth, health-care provider, and identification numbers from Medicaid and the NYS Department of Corrections and Community Supervision. Matching was conducted using IBM InfoSphere QualityStage software (IBM, Armonk, New York). Through probabilistic matching and manual review, 80,899 persons from NYS Medicaid data were confirmed as true PLWDH (Table 1); the software identified 63,047 of these cases as exact matches, and 17,852 were identified through manual review of uncertain matches by trained surveillance staff.

Table 1

Demographic Characteristics of Potential and Presumptive Persons Living With Diagnosed HIV in Medicaid Data, New York State, 2006–2014

Characteristic	No. of Potential PLWDH	No. of Presumptive PLWDH
Total	269,257	86,930
Yeara
2006	63,650	41,136
2007	59,145	40,750
2008	57,475	41,645
2009	61,421	42,857
2010	64,911	44,443
2011	68,344	45,144
2012	72,017	45,446
2013	100,892	47,067
2014	160,725	49,177
Sexb
Female	134,759	32,556
Male	134,495	54,374
Age group, years
Missing data	5,920	4,628
≤19	4,513	1,732
20–24	19,139	4,311
25–29	23,466	5,636
30–39	50,518	16,293
40–49	71,452	30,012
50–59	62,994	19,777
≥60	31,255	4,541
Race/ethnicity
Non-Hispanic white	52,884	11,705
Non-Hispanic black	78,443	38,493
Non-Hispanic Asian/ Pacific Islander	17,037	981
Non-Hispanic American Indian/Alaska Native	409	75
Hispanic	76,585	27,078
Other	13,840	4,376
Unknown	30,059	4,222

Abbreviations: HIV, human immunodeficiency virus; PLWDH, persons living with diagnosed HIV.

a Individuals were often identified in multiple years, so the sum for all years may be greater than the total number of individuals.

b Sex was unknown for 3 persons.

Demographic information was obtained from the Medicaid eligibility data. Age was defined as an individual’s age at which he/she was first identified for inclusion in this study. Race/ethnicity was defined as non-Hispanic white, non-Hispanic black, non-Hispanic Asian or Pacific Islander, non-Hispanic American Indian or Alaska Native, Hispanic, or other. Persons without a race or ethnicity listed were recoded as having an unknown race/ethnicity. Records were obtained for each month in which an individual was eligible for Medicaid. Because a person’s eligibility status can fluctuate, the Medicaid enrollees may have had data in some years and not in others. Therefore, data were evaluated by year.

Statistical evaluation of algorithm performance

Classification and regression tree (CART) analysis was used to predict which of the potential PLWDH had diagnosed HIV (“presumptive PLWDH”). CART statistically determines which factors in a model have the greatest explanatory power (46). The data from 2013 were chosen to develop the case-finding algorithm (CFA) through CART analysis because the 2013 data allowed for both retrospective (2006–2012) and prospective (2014) cross-validation through unique CART analyses. Because of a natural break in the data, nodes that had a match rate of 69% or higher were retained for the CFA (Web Figure 1). The sensitivity, specificity, PPV, and NPV of the CFA were evaluated among the potential PLWDH by comparing the presumptive PLWDH with the gold standard NYS HIV surveillance system. The flexibility and representativeness of the CFA were assessed by stratifying the population by claim year, sex, race/ethnicity, and age.

Ethical considerations

This study was designated as “Exemption 5—Public Benefit or Service Programs” by the NYSDOH Institutional Review Board. Claims data for 2006–2014 were used pursuant to Data Exchange and Application and Data Use Agreements between the NYSDOH Bureau of HIV/AIDS Epidemiology, the NYSDOH Office of Health Insurance Programs, and the National Center on Addiction and Substance Abuse.

RESULTS

Demographic characteristics of potential PLWDH

The potential PLWDH included 269,257 unduplicated individuals meeting the inclusion criteria (Table 1). There were equal numbers of males and females (134,495 and 134,759, respectively; sex was unknown for 3 persons). More than half of the potential PLWDH were non-Hispanic black (29.1%) or Hispanic (28.4%). Persons aged 40–49 years constituted the largest category of the study population (26.5%). The number of people identified for inclusion increased over the study years. Between 2006 and 2012, the annual number of persons identified for inclusion ranged between 57,475 (2008) and 72,017 (2012). Coinciding with the implementation of the NYS HH Program in 2013, the number of potential PLWDH that met inclusion criteria increased to 100,892 in 2013 and 160,725 in 2014.

Use of CART analysis to identify presumptive PLWDH

The 2013 CART model (Web Figure 1) identified 21 unique nodes, each representing a combination of selection criteria. Sixteen of these nodes indicated presumptive PLWDH and were combined to create the CFA (Table 2). The CFA identified “presumptive PLWDH” if a person met the criteria in one of the following categories: 1) 2 outpatient claims with an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (38) diagnosis code or an All Patients Refined Diagnosis Related Groups (APR-DRG) (42) code indicating an HIV diagnosis; 2) 1 inpatient claim with an ICD-9-CM diagnosis code or an APR-DRG code indicating an HIV diagnosis; 3) 1 claim for an HIV Special Needs Plan rate code; 4) 1 claim with a procedure code for viral quantification, phenotyping, or genotyping combined with 1 National Drug Code for an antiretroviral medication and an ICD-9-CM diagnosis code for an opportunistic infection; or 5) 1 claim for an HIV antiretroviral medication plus 1 claim with a rate code for outpatient services, such as an HIV clinic, an HIV medical evaluation visit, or HIV HH case management. By applying these 5 criteria to the whole sample, we identified 86,930 presumptive PLWDH in the Medicaid claims data. As Table 2 demonstrates, 82.6% of these presumptive PLWDH had at least 2 outpatient claims with a code indicating an HIV diagnosis.

Table 2

Components of a Case-Finding Algorithm Used to Identify 86,930 Presumptive Persons Living With Diagnosed HIV in Medicaid Data, New York State, 2006–2014

Component	Total a Component Contribution		Unique b Component Contribution
	No. of Persons	%	No. of Persons	%
2 outpatient claims with an ICD-9-CM diagnosis code or an APR-DRG code indicating an HIV diagnosis	71,776	82.6	33,370	38.4
1 inpatient claim with an ICD-9-CM diagnosis code or an APR-DRG code indicating an HIV diagnosis	27,502	31.6	9,779	11.3
1 claim for an HIV Special Needs Plan rate code	19,755	22.7	1,772	2.0
1 claim with a procedure code for viral quantification, phenotyping, or genotyping combined with 1 National Drug Code for an antiretroviral medication and an ICD-9-CM diagnosis code for an opportunistic infection	14,086	16.2	416	0.5
1 claim for an HIV antiretroviral medication plus 1 claim with a rate code for outpatient services	16,857	19.4	1,210	1.4

Abbreviations: APR-DRG, All Patient Refined Diagnosis Related Groups; HIV, human immunodeficiency virus; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; PLWDH, persons living with diagnosed HIV.

a 46.5% of presumptive PLWDH were captured by more than 1 component of the case-finding algorithm, so the column percentages may not add to 100.

b 53.6% of presumptive PLWDH were captured by only 1 component of the case-finding algorithm, so the column percentages may not add to 100.

Demographic characteristics of presumptive PLWDH identified by the CFA

The CFA identified more males than females (62.6% and 37.5%, respectively) (Table 1). Presumed PLWDH varied by race/ethnicity (non-Hispanic black, 44.3%; Hispanic, 31.2%; non-Hispanic white, 13.5%; non-Hispanic Asian/Pacific Islander, 1.1%; non-Hispanic American Indian/Alaska Native, 0.1%; other, 5.0%). Persons aged 40–49 years were the largest age group identified, constituting 34.5% of the presumptive PLWDH. On average, the CFA identified 44,184 presumptive PLWDH each year between 2006 and 2014, with a slight increase in the yearly number of presumptive PLWDH identified between 2011 and 2014.

Comparison of potential PLWDH and presumptive PLWDH

Demographic characteristics varied between the potential PLWDH in the Medicaid data and the presumptive PLWDH identified from the CART analysis (Table 1). The proportion of females dropped from 50.0% to 37.5%. The proportion increased for persons aged 40–49 years (from 26.5% to 34.5%) and decreased for those aged 20–24 years (from 7.1% to 5.0%), 25–29 years (from 8.7% to 6.5%), and ≥60 years (from 11.6% to 5.2%). The proportions of non-Hispanic black (from 29.1% to 44.3%) and Hispanic (from 28.4% to 31.1%) presumptive PLWDH increased. The proportions of non-Hispanic whites (from 19.6% to 13.5%), non-Hispanic Asians/Pacific Islanders (from 6.3% (from 11.2% to 4.9%) decreased.

Roughly one-third of potential PLWDH and 88.0% of presumptive PLWDH were true cases (Table 3). A higher percentage of true cases were identified among the presumptive PLWDH for each of the demographic categories assessed.

Table 3

Numbers of Potential and Presumptive Persons Living With Diagnosed HIV in New York State Medicaid Data Matched to New York State HIV Surveillance Data, 2006–2014

Characteristic	Potential PLWDH			Presumptive PLWDH
	Total No. of Persons	True Cases		Total No. of Persons	True Cases
		No.	%		No.	%
Total	269,257	80,899	30.1	86,930	76,467	88.0
Yeara
2006	63,650	48,951	76.9	41,136	39,751	96.6
2007	59,145	44,793	75.7	40,750	39,739	97.5
2008	57,475	45,301	78.8	41,645	40,552	97.4
2009	61,421	46,361	75.5	42,857	41,628	97.1
2010	64,911	47,577	73.3	44,443	43,049	96.9
2011	68,344	48,601	71.1	45,144	43,968	97.4
2012	72,017	49,192	68.3	45,446	44,036	96.9
2013	100,892	49,739	49.3	47,067	44,199	93.9
2014	160,725	55,029	34.2	49,177	44,800	91.1
Sexb
Female	134,759	29,445	21.9	32,556	28,115	86.4
Male	134,495	51,454	38.3	54,374	48,352	88.9
Age group, years
Missing data	5,920	4,541	76.7	4,628	4,374	94.5
≤19	4,513	1,672	37.1	1,732	1,597	92.2
20–24	19,139	4,045	21.1	4,311	3,773	87.5
25–29	23,466	5,167	22.0	5,636	4,859	86.2
30–39	50,518	15,321	30.3	16,293	14,483	88.9
40–49	71,452	28,384	39.7	30,012	27,005	90.0
50–59	62,994	18,124	28.8	19,777	17,035	86.1
≥60	31,255	3,645	11.7	4,541	3,341	73.6
Race/ethnicity
Non-Hispanic white	52,884	10,649	20.1	11,705	10,130	86.5
Non-Hispanic black	78,443	34,752	44.3	38,493	34,693	90.1
Non-Hispanic Asian/Pacific Islander	17,037	779	4.6	981	751	76.6
Non-Hispanic American Indian/Alaska Native	409	75	18.3	75	71	94.7
Hispanic	76,585	23,630	30.9	27,078	23,667	87.4
Other	13,840	3,687	26.6	4,376	3,631	83.0
Unknown	30,059	7,327	24.4	4,222	3,515	83.3

Abbreviations: HIV, human immunodeficiency virus; PLWDH, persons living with diagnosed HIV.

a Individuals were often identified in multiple years, so the column percentages may not add to 100.

b Sex was unknown for 3 persons.

Measures of validity for the CFA

The CFA had a sensitivity of 94.5%, a specificity of 94.4%, a PPV of 88.0%, and an NPV of 97.6% (Table 4). The measures of validity were similar between male and female presumptive PLWDH. Sensitivity was 94.0% for males and 95.5% for females, and specificity was 92.8% for males and 95.8% for females. The PPV was 88.9% for males and 86.4% for females, while the NPV was 96.1% for males and 98.7% for females.

Table 4

Validity Statistics for a Case-Finding Algorithm Created to Identify Presumptive Persons Living With Diagnosed HIV in Medicaid Data, New York State, 2006–2014

Characteristic	Total No. of PLWDH	Match to Surveillance Data			Measure of Validity
		No. of Persons	%	95% CI	Sensitivity, %	Specificity, %	PPV, %	NPV, %
Total	86,930	76,467	88.0	87.8, 88.2	94.5	94.4	88.0	97.6
Yeara
2006	41,136	39,751	96.6	96.5, 96.8	81.2	90.6	96.6	59.1
2007	40,750	39,739	97.5	97.4, 97.7	88.7	93.0	97.5	72.5
2008	41,645	40,552	97.4	91.2, 97.5	89.5	91.0	97.4	70.0
2009	42,857	41,628	97.1	97.0, 97.3	89.8	92.0	97.1	74.9
2010	44,443	43,049	96.9	96.7, 97.0	90.5	92.0	96.9	77.9
2011	45,144	43,968	97.4	97.3, 97.5	90.5	94.0	97.4	80.0
2012	45,446	44,036	96.9	96.7, 97.1	89.5	93.8	96.9	80.6
2013	47,067	44,199	93.9	93.7, 94.1	88.9	94.4	93.9	89.7
2014	49,177	44,800	91.1	90.9, 91.4	81.4	95.9	91.1	90.8
Sexb
Female	32,556	28,115	86.4	86.0, 86.7	95.5	95.8	86.4	98.7
Male	54,374	48,352	88.9	88.7, 89.2	94.0	92.8	88.9	96.1
Age group, years
Missing data	4,628	4,374	94.5	93.9, 95.2	96.3	81.6	94.5	87.1
≤19	1,732	1,597	92.2	90.9, 93.5	95.5	95.2	92.2	97.3
20–24	4,311	3,773	87.5	86.5, 88.5	93.3	96.4	87.5	98.2
25–29	5,636	4,859	86.2	85.3, 87.1	94.0	95.8	86.2	98.3
30–39	16,293	14,483	88.9	88.4, 89.4	94.5	94.9	88.9	97.6
40–49	30,012	27,005	90.0	89.6, 90.3	95.1	93.0	90.0	96.7
50–59	19,777	17,035	86.1	85.7, 86.6	94.0	93.9	86.1	97.5
≥60	4,541	3,341	73.6	72.3, 74.9	91.7	95.7	73.6	98.9
Race/ethnicity
Non-Hispanic white	11,705	10,130	86.5	85.9, 87.2	92.3	96.4	86.5	98.0
Non-Hispanic black	38,493	34,693	90.1	89.8, 90.4	95.3	91.7	90.1	96.1
Non-Hispanic Asian/Pacific Islander	981	751	76.6	73.9, 79.2	92.5	98.7	76.6	99.6
Non-Hispanic American Indian/Alaska Native	75	71	94.7	89.6, 99.8	93.4	98.9	94.7	98.6
Hispanic	27,078	23,667	87.4	87.0, 87.8	96.0	93.9	87.4	98.2
Other	4,376	3,631	83.0	81.9, 84.1	93.7	93.1	83.0	97.6
Unknown	4,212	3,515	83.5	82.3, 84.6	85.6	95.3	83.5	95.9

Abbreviations: CI, confidence interval; HIV, human immunodeficiency virus; NPV, negative predictive value; PLWDH, persons living with diagnosed HIV; PPV, positive predictive value.

a Individuals were often identified in multiple years, so the column percentages may not add to 100.

b Sex was unknown for 3 persons.

The measures of validity for presumptive PLWDH were similar when they were compared across racial/ethnic categories (Table 4). Sensitivity ranged from 92.3% among non-Hispanic white presumptive PLWDH to 96.0% among Hispanic presumptive PLWDH. Specificity ranged from 91.7% among non-Hispanic black presumptive PLWDH to 98.9% among non-Hispanic American Indian/Alaska Native presumptive PLWDH. The PPV ranged from 76.6% among non-Hispanic Asian/Pacific Islander presumptive PLWDH to 94.7% among non-Hispanic American Indian/Alaska Native presumptive PLWDH. The NPV ranged from 96.1% among non-Hispanic black presumptive PLWDH to 99.6% among non-Hispanic Asian/Pacific Islander presumptive PLWDH.

The measures of validity were similar across age groups (Table 4). Sensitivity ranged from 91.7% among presumptive PLWDH aged ≥60 years to 95.5% for presumptive PLWDH aged 18–19 years. Specificity ranged from 93.0% among presumptive PLWDH aged 40–49 years to 96.4% among presumptive PLWDH aged 20–24 years. The PPV ranged from 73.6% among presumptive PLWDH aged ≥60 years to 92.2% among presumptive PLWDH aged 18–19 years. The NPV varied from 96.7% among presumptive PLWDH aged 40–49 years to 98.9% among presumptive PLWDH aged ≥60 years.

The measures of validity improved over the study years (Table 4). Sensitivity ranged from 81.2% in 2006 to 90.5% in 2010 and 2011. Specificity increased, from 90.6% in 2006 to 95.9% in 2014. The PPV ranged from 91.1% in 2014 to 97.5% in 2007. The NPV increased from 59.1% in 2006 to 90.8% in 2014.

DISCUSSION

The CFA developed through CART analysis successfully identified PLWDH from the NYS Medicaid data. The CFA was designed with the CDC’s 2014 updated guidelines for evaluating public health surveillance systems and validated using multiple measures of accuracy (sensitivity, specificity, PPV, and NPV). All measures of validity demonstrated that the CFA had a high likelihood of accurately capturing HIV status among persons listed in the Medicaid data.

German et al. (32) defined a flexible system as one that can adapt easily to changing information and stated that flexibility is best evaluated through retrospective analysis. The CFA was created using data from 2013 and retrospectively applied to data from 2006–2012, and then prospectively applied to 2014 data. This time period included the implementation of the Affordable Care Act (47), expanded Medicaid service, and the initiation of the HH Program in NYS (48). The CFA adapted well to these changes; each year, the sensitivity was above 80% and the PPV was above 90%. While the CFA presented in this paper is not as simple as the CFAs used by other researchers (9–11, 13, 22), the CFA presented here was easy to use, easy to understand, and replicable by year. Researchers should be able to use this algorithm to identify PLWDH in their states’ Medicaid data.

The NPV indicated that 98% of PLWDH identified by the CFA as HIV-negative were truly negative (not diagnosed with HIV) based on comparison with the gold standard NYS HIV surveillance system. Similarly, the PPV indicated that 88% of the PLWDH identified by the CFA were truly positive. Additionally, the high specificity (94%) observed with the CFA is important in an algorithm, because specificity and PPV are closely related (49) and a decrease in specificity leads to both a decrease in PPV and an increase in the number of people falsely identified by the algorithm. The overall sensitivity was 95%, indicating that the CFA identified nearly all of the true cases from the potential PLWDH.

These measures of validity varied but remained high for each of the racial/ethnic, sex, and age groups examined. This demonstrated that the CFA works well for different groups and is representative for all of the subgroups tested (Table 4). A representative system better characterizes the epidemiologic features of events and interventions (32).

Strengths and weaknesses

The CFA performed well with regard to important characteristics of a public health surveillance system, including specificity, sensitivity, PPV, and NPV, demonstrating patterns of flexibility and representativeness across several years. However, limitations of this study should be noted. In 2017, Ngugi et al. (50) used content analysis methods to systematically evaluate the data quality of the CDC’s HIV surveillance data reports and identified problems in several areas, including potential incomplete and incorrect gaps in the HIV surveillance data. Because of NYS’s stringent name-based reporting laws, which require all HIV-related laboratory test results (e.g., CD4 cell counts and percentages, viral load, genotypic resistance) for persons who reside or receive HIV-related care in the state to be reported to the NYSDOH, it is thought that the effect of underreporting to the NYS HIV surveillance system is minimal. However, there is potential that PLWDH may not be reported in NYS if 1) they received HIV testing or care under a research protocol approved by an institutional review board, 2) they received anonymous testing, 3) testing and diagnosis were performed in the federal health-care system (i.e., the Veterans Affairs system, the military, or federal prisons), or 4) they received testing from an out-of-state laboratory that failed to report this information to the NYSDOH (51). Even with these limitations, utilizing the HIV surveillance system as the gold standard for the development and evaluation of the CFA is thought to have provided the most accurate picture of PLWDH in NYS.

A second limitation of the CFA is that the analysis did not include all Medicaid data. Therefore, some potential PLWDH who had Medicaid coverage may not have been included in the analysis. However, it is believed that the number of missing cases was minimal, as the initial selection of potential PLWDH was designed to be as inclusive as possible.

In NYS, there were over 6 million people on Medicaid in 2013 (1), of whom fewer than 2% were diagnosed with HIV (34). Increasing the number of true negatives by millions through screening of the entire Medicaid database would increase the NPV and specificity to a level where the estimates would be close to 100%. This would be not only costly but unfeasible in a short period of time. In addition, the annual number of potential PLWDH in this algorithm almost tripled between 2006 and 2014, with a sharp increase in potential PLWDH occurring in 2013 and 2014. This correlates with the increasing number of enrollees in the NYS HH Program and was validated by examining the numbers of HH outreach rate codes by year. This change increased the numbers of true negatives in the analysis but had minimal impact on the number of presumptive PLWDH, adding confidence to the assumption that running the CFA using the entire Medicaid data set would have a limited impact.

Thirdly, the NYS HIV surveillance system and the Medicaid data system collect demographic information differently, making it difficult to compare demographic distributions of the CFA with those of the HIV surveillance system. An example is that the HIV surveillance system collects information on both sex at birth and current sex; however, Medicaid data do not differentiate between these 2 variables. With regard to demographic characteristics that provided a direct comparison, the CFA and the HIV surveillance system yielded similar proportions (data not shown (33)). This adds to the confidence that the CFA was not biased on the basis of individual demographic information.

Fourthly, a limit to the generalizability of the CFA is that it uses NYS-specific Medicaid rate codes. However, the impact of these codes is minimal, as 46.5% of the presumptive PLWDH identified by the CFA had claims in 2 or more of the categories, and 91.8% of the presumptive PLWDH identified using rate codes had claims in 2 or more of the categories. It is also important to note that algorithms developed for internal programmatic purposes may need to consider the inclusion of program-specific billing codes.

Lastly, the CFA used ICD-9-CM diagnosis codes from the Medicaid billing data. As of October 1, 2015, the ICD-9-CM was replaced by the International Classification of Diseases, Tenth Revision, Clinical Modification (52). To properly apply this CFA to Medicaid billing data after September 2015, the ICD-9-CM diagnosis codes would need a cross-walk comparison with International Classification of Diseases, Tenth Revision, Clinical Modification codes.

Implications

Utilizing billing data is an unobtrusive method of analysis; therefore, the privacy risk posed to PLWDH whose information is being used is minimal. The successful derivation and application of the CFA have promising implications for future HIV analyses. Utilizing the CFA could help improve both clinical research and program evaluation, leading to better care for PLWDH. Applied to the whole Medicaid data set, the CFA may have the ability to look for PLWDH who have not been reported to the NYSDOH HIV surveillance system. Identifying these potential gaps in reporting would allow PLWDH to be better surveilled, further bolstering the HIV surveillance system.

Conclusion

Medicaid is a robust data system, containing treatment information not captured in the NYS HIV surveillance system, such as claims for health-care provider visits, tests, and antiretroviral medications. Researchers have employed other algorithms to answer questions using Medicaid data, but those algorithms have not been evaluated with respect to their sensitivity, specificity, PPV, and NPV by comparison with the gold standard, an HIV surveillance system. The CFA described in this paper was evaluated against this gold standard and performed well. It was deemed simple, flexible, and representative and had high sensitivity and PPV. It successfully identified PLWDH in the Medicaid data, improving the utility of those data for assessing health outcomes and programmatic interventions such as HHs. Findings from this paper provide future researchers with a tool for identifying PLWDH through Medicaid data, potentially helping to increase linkage with HIV care and retention in care among PLWDH and decreasing future transmission.

Click here for additional data file.