Evaluation of the American Hospital Association Annual Survey for health services research in emergency medicine.

Objectives: Emergency department (ED) data are often used to address questions about access to and quality of emergency care. Our objective was to compare one of the most commonly used data sources for national ED information, the American Hospital Association (AHA) Annual Survey, with a criterion database: the National Emergency Department Inventory (NEDI)-USA data set.
Methods: We compared the 2015 and 2016 AHA surveys to the following 3 criterion standards: (1) the 2015 and 2016 NEDI-USA databases, which have information about all US EDs, including merged data from (2) Council of Teaching Hospitals (COTH) and (3) the Critical Access Hospital (CAH) program. We present descriptive results about the number of EDs in each data set; total and median visit volumes; locations in rural areas; and COTH, CAH, and freestanding ED (FSED) status.
Results: The AHA survey identified 3893 US EDs in 2015. These EDs had a total annual visit volume of 129,197,493 visits, with a median of 22,772 visits (interquartile range, 8311-47,938). Compared with the NEDI-USA, the AHA included 1433 fewer EDs (-27%; 95% confidence interval [CI], -28% to -26%) and 23,615,163 (-15%) fewer visits. Specifically, AHA was missing 245 (-22%; 95% CI, -24% to -19%) of those located in rural areas, 268 (-20%; 95% CI, -22% to -18%) in a CAH, and 240 (-47%; 95% CI, -51% to -42%) FSEDs. We saw similar results using 2016 data. Conclusions: Although several aggregated results were similar between the compared data sources, the AHA data set excluded many US EDs, including many rural EDs and FSEDs. Consequently, the AHA underreported total ED visits by 15%. We encourage data users to be cautious when interpreting results from any 1 ED data source, including the AHA.

INTRODUCTION

Background

Emergency departments (EDs) play a critical role in the US healthcare system. As EDs open and close, tracking EDs on the national level is challenging. Administrative ED data are often used by policy researchers to address questions about access to care, quality of care, and ED staffing. , , , , , , , , , , , Thus, reliance on inaccurate data sources could hinder or even misinform such research as well as subsequent efforts to improve access to and quality of care.

Importance

One of the most commonly used data sources for national hospital and ED information is the American Hospital Association (AHA)’s Annual Survey database. This database includes many details about hospital and ED characteristics and can easily be linked with other databases (eg, the Centers for Medicare and Medicaid Services [CMS] data ) to investigate associations between facility characteristics and patient outcomes. Although the AHA data set is uniquely able to facilitate important health services research about emergency care, it has several limitations. Notably, the AHA often groups EDs within the same health system under a single identification (ID) number, and it is difficult to attribute facility characteristics to individual EDs.

Goals of this investigation

Our objective was to compare the AHA database to a criterion standard and determine the magnitude and direction of any identified errors. Specifically, we aimed to compare it with the National Emergency Department Inventory (NEDI)–USA database, which was selected as a criterion database because of its comprehensive inclusion of EDs and annual survey response rate of >80% over many years.

METHODS

AHA database

The AHA database is commercially available on an annual basis and contains hundreds of data elements, including ED‐related variables. The AHA collects data directly from >6000 US hospitals via the voluntary AHA Annual Survey (Chicago, IL), with a response rate of >75% each year. For non‐reporting hospitals and for incomplete responses, data are estimated based on the missing hospital's most recent information using various statistical methodologies. We compared commonly used ED data from the 2015 AHA database to (1) the 2015 NEDI‐USA database, which includes information taken directly from (2) the Council of Teaching Hospitals (COTH) and (3) the Critical Access Hospital (CAH) program. To examine the reliability of the findings, we repeated the comparison using the 2016 AHA and 2016 NEDI‐USA databases (including updated COTH and CAH data). This study was reviewed by the Mass General Brigham Human Research Committee and classified as exempt.

Robust public health databases support high‐level health services research regarding emergency department care. This study compared the widely used American Hospital Association survey with the comprehensive National Emergency Department Inventory–USA data set, finding inconsistencies in captured data. Users must understand these limitations when interpreting results from these data sets.

NEDI‐USA database

The NEDI‐USA contains information on all non‐federal, non‐specialty, US hospital‐affiliated EDs open 24 hours per day, 7 days per week year‐round. Briefly, the NEDI‐USA excludes the roughly 200 federal (eg, Veterans Affairs) and all specialty hospitals because they are not open to the public or typically are not prepared to evaluate a range of diseases and general emergency issues (eg, psychiatric hospitals). The NEDI‐USA was created and is maintained by our research team at the Emergency Medicine Network (EMNet). EMNet is a research division of the Department of Emergency Medicine at Massachusetts General Hospital (Boston, MA). As noted previously, we selected the NEDI‐USA as a criterion database because of its comprehensive inclusion of EDs and an annual survey response rate of >80%. ED data from the NEDI‐USA have been compared with those from other national ED sources. On an annual basis, we send a survey to the directors of all EDs open in the prior year. To create the list of open EDs, we combine EDs known to be open in the preceding year and update it with known openings, closures, and name and address changes. We primarily identify these changes via reviewing news briefs that are provided via Google Alerts and by direct communication with hospitals and EDs (eg, from the prior year's survey). We mail the survey up to 3 times until a response is obtained. We follow‐up with non‐responding EDs via telephone to complete the survey by interview. We also allow participants to complete the survey online. The survey collects data on basic ED characteristics, including total visit volume. If an ED reports a visit volume that is >50% greater than or >30% less than the visit volume collected as part of the prior NEDI‐USA data set, we ask if there was a change (eg, construction or closure of a wing) that led to this or if older data need to be updated.

For EDs for which no visit volume can be obtained, we first push forward visit volumes from the prior iteration of the NEDI‐USA. If no visit volume was first obtained during that prior iteration (ie, that visit volume had already been pushed forward once), we impute values using single imputation regression adjusted for the following 4 factors: prior NEDI‐USA ED visit volume, urban influence codes, US region, and freestanding ED (FSED) type. Other important variables incorporated into the NEDI‐USA include teaching hospital status defined as all members of the COTH and participation in the CAH program, created by Congress in 1998 to help improve the healthcare delivery system in rural areas. The current locations for all US EDs can be identified using the publicly available smartphone application, findERnow.

To enable linkage with other data sets, including the AHA and CMS, the NEDI‐USA data have been consolidated. These methods have been described previously, but briefly, each ED in the NEDI‐USA was matched by address with a corresponding hospital/ED in the AHA and CMS. Those that were not individually listed in the AHA were grouped with a hospital/ED that was listed. If it could not be grouped, that ED's data were included in the consolidated data set but without any AHA or CMD ID match or grouping. To consolidate data, values were either added together (ie, for continuous values such as total visit volume) or weighted by visit volume (eg, to assign a categorical value, such as whether a group of EDs is part of a CAH). Using this consolidated data set, we present data about EDs individually listed EDs in the NEDI‐USA but that are (1) grouped in the AHA or (2) excluded completely from the AHA.

Data analysis

Data analysis was performed using Stata 15.1 software (StataCorp, College Station, TX). Both the AHA and NEDI‐USA include FSEDs, which are EDs that are physically distinct from inpatient hospital services. Although hospital‐owned “satellite” FSEDs are present in both the AHA and NEDI‐USA databases, non‐hospital‐affiliated “autonomous” FSEDs are only available from the NEDI‐USA. We included “autonomous” FSEDs in this analysis given their potential to affiliate with hospitals and become satellite FSEDs; they typically do this to become eligible for payments from Medicare. , Importantly, their inclusion also better represents the provision of emergency care across the United States. Thus, for completeness, analyses comparing the availability of FSEDs in both data sets included a comparison of all FSEDs as well as satellite FSEDs only. The AHA has a few variables indicating the presence of an ED. We identified and included all EDs that were coded either as EMDEPHOS = 1 (indicating presence of a hospital‐based ED), FSERHOS = 1 and FSERYN = 1 (indicating the presence of a satellite FSED open 24 hours per day, 7 days per week), or both. The following variables were investigated: number of EDs, both overall and by rural status (defined according to Core Based Statistical Area) ; annual ED visit volume; COTH status; CAH program affiliation ; and total and satellite FSED status. The analysis used descriptive statistics and reported frequencies with proportions and medians presented with interquartile ranges (IQRs). We compared distributions of ED characteristics presented in each data set using chi‐square and Kruskal–Wallis tests as appropriate. Percentage changes relative to the criterion database were calculated with 95% confidence intervals (CIs) to illustrate variation in differences between the AHA and NEDI‐USA.

RESULTS

The AHA survey identified 3893 US EDs compared with 5326 in the NEDI‐USA in 2015 (Table 1). Thus, the AHA had 1433 fewer EDs (−27%; 95% CI, −28% to −26%) compared with the NEDI‐USA. EDs in the AHA had a total annual visit volume of 129,197,493 visits that ranged from 0 to 532,801, with a median of 22,772 visits (IQR, 8311–47,938). EDs in the NEDI‐USA had a total annual visit volume of 152,812,656 that ranged from 1 to 217,153, with a median of 21,200 visits (IQR, 7665–41,975).

TABLE 1

Descriptions of 2015 US EDs by data source

	AHA 2015	NEDI‐USA 2015	P value	Difference (AHA − NEDI‐USA)	Percentage change relative to 2015 NEDI‐USA (95% CI)
All EDs
Total number of EDs	3893	5326	–	−1433	−27 (−28 to −26)
Total number of ED visits	129,197,493	152,812,656	–	−23,615,163	−15 (−15 to −15)
Visit volume range	0–532,801	1–217,153	–	–	–
Median ED visit volume (IQR)	22,772 (8311–47,983)	21,200 (7665–41,975)	<0.001	1572	7 (7–8)
Urban status
Rural	880 (23)	1125 (21)	0.09	−245	−22 (−24 to −19)
Urban	3013 (77)	4201 (79)		−1188	−28 (−30 to −27)
All FSEDs
Total number of FSEDs	276 (7)	516 (10)	<0.001	−240	−47 (−51 to −42)
Total number of FSED visits	18,842,586	7,202,389	–	11,640,197	162 (162–162)
FSED visit volume range	0–339,146	1–83,950	–	–	–
Median FSED visit volume (IQR)	62,747 (37,753–92,650)	10,070 (6555–22,267)	<0.001	52,677	523 (514–532)
Urban status
Rural	18 (7)	9 (2)	<0.001	9	100 (66–134)
Urban	258 (93)	507 (98)		−249	−49 (−54 to −45)
FSEDs
Total number of FSEDs	276 (7)	289 (5)	0.001	−13	−4 (−8 to −2)
Total number of FSED visits	18,842,586	5,886,706	–	12,955,880	220 (220–220)
FSED visit volume range	0–339,146	1–83,950	–	–	–
Median FSED visit volume (IQR)	62,747 (37,753–92,650)	21,000 (13,616–22,690)	<0.001	41,747	199 (197–201)
Urban status
Rural	18 (7)	7 (2)	0.02	11	157 (112–325)
Urban	258 (93)	282 (98)		−24	−9 (−12 to −6)
COTH
Total number of COTH EDs	262 (7)	262 (5)	<0.001	0	0
Total number of COTH ED visits	22,096,621	19,419,722	0	2,676,899	14 (14–14)
COTH ED visit volume range	4583–532,801	7517–183,000	0	–	–
Median COTH ED visit volume (IQR)	76,729 (55,694–102,557)	68,896 (53,227–96,519)	0.051	7833	11 (11–12)
Urban status
Rural	0 (0)	0 (0)	–	0	0
Urban	262 (100)	262 (100)		0	0 (−1 to 1)
CAH
Total number of CAH with EDs	1065 (27)	1333 (25)	0.01	−268	−20 (−22 to −18)
Total number of CAH ED visits	6,778,336	8,661,465	–	−1,883,129	−22 (−22 to −22)
CAH ED visit volume range	0–39,678	72–43,800	–	–	–
Median CAH ED visit volume (IQR)	5184 (2392–9318)	5408 (2233–9467)	0.90	−224	−4 (−5 to −4)
Urban status
Rural	667 (63)	835 (63)	0.995	−168	−20 (−23 to −17)
Urban	398 (37)	498 (37)		−100	−20 (−24 to −17)

Note: Data are provided as n (%) unless otherwise noted.

Abbreviations: 95% CI, 95% confidence interval; AHA, American Hospital Association; CAH, Critical Access Hospital; COTH, Council of Teaching Hospital; ED, emergency department; FSED, freestanding emergency department; IQR, interquartile range; NEDI‐USA, National Emergency Department Inventory–USA.

In the AHA, a total of 880 EDs (23%) were located in rural areas, 262 (7%) were affiliated with a COTH, 1065 (27%) were in a CAH, and 276 (7%) were identified as satellite FSEDs. There were 7 hospitals in the AHA that had a satellite FSED but no hospital‐based ED. In the NEDI‐USA, a total of 1125 EDs (21%) were located in rural areas, 262 (5%) were affiliated with a COTH, 1333 (25%) were in a CAH, and 516 (10%) were identified as FSEDs of either type, with 289 (5%) identified as satellite FSEDs. Thus, the AHA missed 245 (−22%; 95% CI, −24% to −19%) of those located in rural areas and 268 (−20%; 95% CI, −22% to −18%) of those in a CAH. The AHA also had 240 fewer FSEDs of any type (−47%; 95% CI, −51% to −42%) and 13 fewer satellite FSEDs (−4%; 95% CI, −8% to −2%). The AHA was also missing 100% of the autonomous FSEDs.

Comparing FSED characteristics between data sets, there were differences in aggregated results with respect to median FSED visit volume (62,747 in the AHA vs 10,070 in the NEDI‐USA), maximum visit volume (339,146 vs 83,950), and rural location (7% vs 2%; P < 0.001). We again found differences when comparing FSED characteristics in the AHA versus characteristics among satellite FSEDs only in the NEDI‐USA (all P < 0.05; Table 1).

When using COTH and CAH data as the criterion standards, we identified differences in both the AHA and NEDI‐USA data. Specifically, there were 433 total hospitals/EDs listed in the COTH data set, of which the AHA and NEDI‐USA both identified 262 (61%). There were 1333 EDs listed in the CAH data set. Of these, the AHA identified 1065 (80%), whereas the NEDI‐USA identified 1333 (100%; Figure 1).

FIGURE 1

Number of emergency departments in COTH and CAHs by data source. AHA, American Hospital Association; CAH, Critical Access Hospital; COTH, Council of Teaching Hospitals; NEDI‐USA, National Emergency Department Inventory–USA

When repeating this analysis using data for the year 2016, we found similar patterns. Briefly, the 2016 AHA identified 3789 EDs (Table 2). By contrast, the 2016 NEDI‐USA data set identified 5431 EDs. The AHA, compared with the NEDI‐USA, had 1642 fewer EDs (−30%; 95% CI, −31% to −29%). The AHA had a total annual visit volume of 127,667,855 visits that ranged from 0 to 582,064, with a median of 22,968 visits (IQR, 7882–48,381). The NEDI‐USA had 157,334,526 visits (range, 1–217,153), with a median of 20,356 (IQR, 7827–43,000).

TABLE 2

Descriptions of 2016 US EDs by data source

	AHA 2016	NEDI‐USA 2016	P value	Difference (AHA − NEDI‐USA)	Percentage change relative to 2016 NEDI‐USA (95% CI)
All EDs
Total number of EDs	3789	5431	–	−1642	−30 (−31 to −29)
Total number of ED visits	127,667,855	157,334,526	–	−29,666,671	−19 (−19 to −19)
Visit volume range	0–582,064	1–217,153	–	–	–
Median ED visit volume (IQR)	22,968 (7882–48,381)	20,356 (7827–43,000)	0.002	2612	13 (12–13)
Urban status
Rural	860 (23)	1121 (21)	0.02	−261	−23 (−26 to −21)
Urban	2929 (77)	4310 (79)		−1381	−32 (−33 to −31)
All FSEDs
Total number of FSEDs	284 (7)	635 (12)	<0.001	−351	−55 (−59 to −51)
Total number of FSED visits	20,543,930	8,632,705	–	11,911,225	138 (138–138)
FSED visit volume range	0–354,270	1–94,000	–	–	–
Median FSED visit volume (IQR)	64,121 (39,884–94,315)	10,950 (5475–19,859)	<0.001	53,171	486 (478–494)
Urban status
Rural	20 (7)	10 (2)	<0.001	10	100 (69–131)
Urban	264 (93)	625 (98)		−361	−58 (−62 to −54)
Satellite FSEDs
Total number of FSEDs	284 (7)	405 (7)	0.95	−121	−30 (−35 to −25)
Total number of FSED visits	20,543,930	7,376,827	–	13,167,103	179 (178–179)
FSED visit volume range	0–354,270	1–94,000	–	–	–
Median FSED visit volume (IQR)	64,121 (39,884–94,315)	19,840 (11,208–20,000)	<0.001	44,281	223 (221–226)
Urban status
Rural	20 (7)	7 (2)	<0.001	13	186 (124–398)
Urban	264 (93)	398 (98)		−134	−34 (−39 to −29)
COTH
Total number of COTH EDs	257 (7)	260 (5)	<0.001	−3	−1 (−3 to 0)
Total number of COTH ED visits	22,755,176	19,490,932	–	3,264,244	17 (17–17)
COTH ED visit volume range	5563–582,064	4260–197,546	–	–	–
Median COTH ED visit volume (IQR)	78,718 (55,259–107,886)	70,000 (52,425–92,165)	0.02	8718	13 (12–13)
Urban status
Rural	0 (0)	0 (0)	–	0	0
Urban	257 (100)	260 (100)		−3	−1 (−3 to 0)
CAH
Total number of CAH with EDs	1046 (28)	1335 (25)	0.001	−289	−22 (−24 to −19)
Total number of CAH ED visits	6,609,589	8,786,609	–	−2,177,020	−25 (−25 to −25)
CAH ED visit volume range	0–38,832	1–40,150	–	–	–
Median CAH ED visit volume (IQR)	5183 (2372–9343)	5400 (2291–9600)	0.49	−217	−4 (−5 to −4)
Urban status
Rural	660 (63)	838 (63)	0.87	−178	−21 (−24 to −19)
Urban	386 (37)	497 (37)		−111	−22 (−26 to −19)

Note: Data are provided as n (%) unless otherwise noted.

Abbreviations: 95% CI, 95% confidence interval; AHA, American Hospital Association; CAH, Critical Access Hospital; COTH, Council of Teaching Hospital; ED, emergency department; FSED, freestanding emergency department; IQR, interquartile range; NEDI‐USA, National Emergency Department Inventory–USA.

In the AHA, there were 860 (23%) EDs located in rural areas, 257 (7%) affiliated with a COTH, 1046 (28%) CAH, and 284 (7%) identified as satellite FSEDs. All hospitals in the AHA with satellite FSEDs also had hospital‐based EDs. In the NEDI‐USA, there were 1121 (21%) EDs located in rural areas, 260 (5%) affiliated with a COTH, 1335 (25%) identified as a CAH, and 635 (12%) identified as FSEDs. The AHA missed 261 (−23%; 95% CI, −26% to −21%) EDs located in rural areas and 289 (−22%; 95% CI, −24% to −19%) of those in a CAH. The AHA again had fewer FSEDs of any type (−55%; 95% CI, −59% to −51%) and fewer satellite FSEDs (−30%; 95% CI, −35% to −25%). The AHA again missed 100% of autonomous FSEDs.

In 2016, we again observed differences between FSEDs identified in the AHA versus in the NEDI‐USA (all and satellites only) with respect to median visit volume (64,121 among the AHA FSEDs vs 10,950 among all NEDI‐USA FSEDs vs 19,840 among NEDI‐USA satellite FSEDs; P < 0.001; Table 2), maximum visit volume (354,270 vs 94,000 [all FSEDs]), and rural location (7% vs 2% [all FSEDs]; P < 0.001). Patterns among COTH and CAH EDs were similar to what we found in 2015.

In 2016, there were again 433 total hospitals/EDs listed in the COTH data set, of which the AHA identified 257 (59%) and the NEDI‐USA identified 260 (60%). There were 1337 EDs listed in the CAH data set. Of these, the AHA identified 1046 (78%), and the NEDI‐USA identified 1335 (99.9%; Figure 1).

Consolidated data set

The 2015 consolidated data set included 4658 observations. There were 149,969,604 total visits to these EDs, ranging from 1 to 476,187, and a median of 20,388 (IQR, 7000–45,255; Table 3). There were 1103 (24%) located in rural areas, 1324 (28%) in a CAH, and 214 (5%) identified as affiliated with satellite FSEDs. Using the consolidated data set, 241 (56%; Figure 1) of the COTH hospitals/EDs were identified and 1324 (99%) of the CAH hospitals/EDs were identified.

TABLE 3

Description of 2015 US EDs by data source using the consolidated data set

	AHA 2015	Consolidated database	EDs individually listed in NEDI‐USA but grouped in AHA 2015	P value	EDs listed in NEDI‐USA but excluded from AHA 2015	P value
Total number of EDs	3893	4658	663	–	251	–
Total number of ED visits	129,197,493	149,969,604	18,548,062	–	1,676,751	–
Visit volume range	0–532,801	1–476,187	1–160,300	–	1–62,757	–
Median ED visit volume (IQR)	22,772 (8,311–48,938)	20,338 (7000–45,255)	22,267 (15,368–37,606)	0.24	6676 (3500–6676)	<0.001
Urban status
Rural	880 (23)	1103 (24)	20 (3)	<0.001	11 (4)	<0.001
Urban	3013 (77)	3555 (76)	643 (97)		240 (96)
All FSEDs	276 (7)	434 (9)	287 (43)	<0.001	228 (91)	<0.001
Satellite FSEDs	276 (7)	214 (5)	287 (43)	<0.001	3 (1)	<0.001
COTH	262 (7)	241 (5)	13 (2)	<0.001	0 (0)	<0.001
CAH	1065 (27)	1324 (28)	8 (1)	<0.001	1 (0.4)	<0.001

Note: Data are provided as n (%) unless otherwise noted.

Abbreviations: AHA, American Hospital Association; CAH, Critical Access Hospital; COTH, Council of Teaching Hospitals; ED, emergency department; FSED, freestanding emergency department; IQR, interquartile range; NEDI‐USA, National Emergency Department Inventory–USA.

We found that EDs that were individually listed in the full NEDI‐USA data set, but grouped in the AHA, were more likely to be FSEDs (43%; Table 3). These EDs also were less likely to be in a COTH hospital (2%), be in a CAH (1%), and be rural (3%; all P < 0.001). EDs individually listed in the NEDI‐USA but excluded completely from the AHA had a lower median visit volume (6676; IQR, 3500–6676). They were more likely to be FSEDs (91%), but less likely to be satellite FSEDs (1%). None were in a COTH hospital, and they were less likely to be in a CAH (0.4%) or rural (4%; all P < 0.001). There were 145 EDs listed in the AHA that were not represented by the consolidated data set.

The 2016 consolidated data set included 4699 observations. There were 155,942,949 total visits to these EDs, ranging from 1 to 572,983, and a median of 21,000 (IQR, 7500–47,005; Table 4). There were 1104 (23%) EDs located in rural areas, 1331 (28%) in a CAH, 475 (10%) identified as any type of FSED, and 252 (5%) identified as affiliated with satellite FSEDs. Using the consolidated data set, 237 (55%; Figure 1) of the COTH hospitals/EDs were identified and 1331 (99.6%) of the CAH hospitals/EDs were identified. EDs individually listed in the NEDI‐USA, but grouped with another ED in the AHA, were more likely to be an FSED (53%) and were less likely to be rural (2%; all P < 0.001). Those excluded completely from the AHA were also more likely to be an FSED (90%) and less likely to be rural (4%; all P < 0.001). There were 143 EDs in the AHA that were not represented by the consolidated data set.

TABLE 4

Description of 2016 US EDs by data source using the consolidated data set

	AHA 2016	Consolidated database	EDs individually listed in NEDI‐USA but grouped in AHA 2016	P value	EDs listed in NEDI‐USA but excluded from AHA 2016	P value
Total number of EDs	3789	4699	725	–	279	–
Total number of ED visits	127,667,855	155,942,949	19,480,406	–	2,170,930	–
Visit volume range	0–582,064	1–572,983	1–156,225	–	1–340,051	–
Median ED visit volume (IQR)	22,968 (7882–48,381)	21,000 (7500–47,005)	19,877 (14,201–35,000)	0.98	5475 (3163–7714)	<0.001
Urban status
Rural	860 (23)	1104 (23)	13 (2)	<0.001	12 (4)	<0.001
Urban	2929 (77)	3595 (77)	712 (98)		267 (96)
All FSEDs	284 (7)	475 (10)	382 (53)	<0.001	250 (90)	<0.001
Satellite FSEDs	284 (7)	252 (5)	382 (53)	<0.001	20 (7)	0.84
COTH	257 (7)	237 (5)	11 (2)	<0.001	1 (0.4)	<0.001
CAH	1046 (28)	1331 (28)	2 (0.3)	<0.001	0 (0)	<0.001

Note: Data are provided as n (%) unless otherwise noted.

Abbreviations: AHA, American Hospital Association; CAH, Critical Access Hospital; COTH, Council of Teaching Hospital; ED, emergency department; FSED, freestanding emergency department; IQR, interquartile range; NEDI‐USA, National Emergency Department Inventory–USA.

LIMITATIONS

The current analysis has several limitations. Information in both the AHA and NEDI‐USA is self‐reported by individuals at the respective healthcare facilities. Also, although we identify differences between the AHA and NEDI‐USA data sets, some of those differences may be attributed to how these databases define an ED (eg, exclusion of federal EDs in the NEDI‐USA but not in the AHA). However, even with these differences, it is clear that the AHA does not include a large proportion of general EDs in its annual data set.

DISCUSSION

When reviewing 2 consecutive years of national data, we found that the AHA was similar to the other criterion databases but did have some important limitations. Specifically, the AHA was missing 27% to 30% of all individual EDs as well as 15% to 19% of all ED visits. In particular, the AHA identified fewer rural EDs, CAHs, and FSEDs compared with the NEDI‐USA. We found differences in EDs individually listed in the NEDI‐USA compared with the AHA, and these differences were pronounced when comparing EDs individually listed in the AHA with those excluded completely from the AHA, suggesting selection bias in the type of EDs individually listed and included in the AHA. There were few EDs listed in the AHA and not in the NEDI‐USA, and these were EDs that did not meet the NEDI‐USA inclusion criteria (ie, they were federal or specialty EDs).

The grouping of EDs in the AHA, although beneficial for linkage with other data sets (eg, CMS), may lead to biased conclusions. For example, although there are areas of the country (eg, rural areas in some states) that have experienced a high number of ED closures in the past 2 decades, the overall number of US EDs has consistently risen each year since 2001. , However, despite the increase in the actual number of US EDs, the number of EDs in the AHA database has decreased every year since 2008. Because the AHA often groups EDs under a single ID number by hospital network, this gives the appearance that EDs are closing over time, and this phenomenon was previously reported as if it reflected ED closures only. On the contrary, the NEDI‐USA has found an increased number of EDs each year since its creation in 2001. As health mergers continue, , the number of EDs in the AHA may continue to decrease without actually indicating ED closures.

Along those lines, although the AHA has a variable to indicate whether EDs are affiliated with FSEDs, it does not individually list these FSEDs separately from their hospital affiliate. Furthermore, autonomous FSEDs, or those that operate independently of hospital ownership, are not included in the AHA database. Given that autonomous FSEDs make up 36% of all FSEDs and that all FSEDs make up 13% of US EDs, this consideration is important in making conclusions about the landscape of US emergency care. We previously demonstrated that the number of FSEDs has increased over time, and there were 745 FSEDs open in 2018. , Furthermore, as described previously, autonomous FSEDs sometimes choose to affiliate with hospitals and therefore become satellite FSEDs. There were many autonomous FSEDs that became satellite FSEDs through this process in 2016. The increase in missing satellite FSEDs in 2016 compared with 2015 (−30% vs −4%) suggests that these new satellites are not represented in the AHA data set. Through ongoing NEDI‐USA database maintenance, although we have identified that a subset of FSEDs has closed since then (eg, because of the COVID‐19 pandemic), there are hundreds of FSEDs that remain open as of mid‐2021.

Notably, both the AHA and NEDI‐USA data sets were missing many EDs that are members of COTH. This is likely due in part to the fact that COTH lists health systems in addition to naming individual hospitals and EDs. Because of this, the COTH list includes duplicates (eg, an entire health system plus an individual hospital within that same health system). There are approximately 100 duplicates in both the 2015 and 2016 COTH lists. In addition, the COTH lists include some federal hospitals/EDs, which are not included as part of the NEDI‐USA. On the other hand, the CAH lists each hospital/ED individually, and there was near‐perfect concordance between the CAH list and CAH‐identified EDs in the NEDI‐USA data set. As EDs are grouped together in the consolidated data set, the number of identified COTH and CAH‐identified EDs decreases given that multiple COTH or CAHs can be grouped within a single observation.

The AHA has unique, detailed information about EDs unavailable elsewhere. Furthermore, data sets with information about patient outcomes and other quality and performance measures can be linked with the AHA, including data from the Agency for Healthcare Research and Quality's Healthcare Cost and Utilization Project and Emergency Department Benchmarking Alliance. This allows for investigations of the associations between hospital and ED characteristics and patient outcomes and other performance metrics. Although we have identified limitations of the AHA data set, it may be appropriate to use in many analyses—and depending on the research question of interest (eg, one focused on hospital‐based EDs only)—may include little bias. On the other hand, we encourage caution when investigating ED characteristics that may not be well represented in the AHA, such as FSED status. Furthermore, we encourage caution when using AHA data for geographic analyses, as a single data point in the AHA may actually represent multiple hospital or ED locations.

In conclusion, the AHA is a robust data set that can be used to answer important health research service questions in relation to emergency care. That said, this data set has limitations, most notably the missingness of nearly 30% of all US EDs. Moreover, the EDs that are missing (eg, up to 25% of rural EDs and up to 55% of FSEDs) are systematically different from those that are included. We encourage data users to be cautious when interpreting results from any 1 ED data source, including the AHA.

CONFLICT OF INTEREST

None of the authors report any potential conflicts of interest.

AUTHOR CONTRIBUTIONS

All authors conceived and designed the study. Krislyn M. Boggs and Ashley F. Sullivan collected the data. Janice A. Espinola provided statistical advice, and Jingya Gao analyzed the data. Krislyn M. Boggs drafted the manuscript, and all authors contributed substantially to its revision. Krislyn M. Boggs and Carlos A. Camargo, Jr. take responsibility for the paper as a whole.