A Multistate Comparison Study of COVID-19 Cases Among Accredited and Nonaccredited Nursing Homes.

Widely acknowledged is the disproportionate number of COVID-19 cases among nursing home residents. This observational study examined the relationship between accreditation status and COVID-19 case rates in states where the numbers and proportions of Joint Commission accredited facilities made such comparisons possible (Illinois (IL), Florida (FL), and Massachusetts (MA)). COVID-19 data were accessed from the Centers for Medicare & Medicaid Services (CMS) Nursing Home Compare Public Use File, which included retrospective COVID-19 data submitted by nursing homes to the Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network. The outcome variable was the total number of nursing home-identified COVID-19 cases from June 2020 to January 2021. Joint Commission accreditation status was the independent variable. Mediating factors included state, and county-level case rates. Increases in the county rate had a significant association with higher nursing home COVID-19 case rates (p < .001). After adjusting for county case rates, no differences were observed in the mean group case rates for accredited and nonaccredited nursing homes. However, comparing predicted case rates to actual case rates revealed that accredited nursing homes were more closely aligned with their predicted rates. Performance of the nonaccredited nursing homes was more variable and had proportionally more outliers compared to accredited nursing homes. Community prevalence of COVID-19 is the strongest predictor of nursing home cases. While accreditation status did not have an impact on overall mean group performance, nonaccredited nursing homes had greater variation in performance and a higher proportion of negative outliers. Accreditation was associated with more consistent performance during the COVID-19 pandemic, despite being located in counties with a higher prevalence of COVID-19.

Introduction

Due to the age and comorbidities of nursing home residents, nursing homes are exceptionally vulnerable settings during the COVID-19 pandemic. Widely acknowledged is the disproportionate number of COVID-19 cases and related deaths among nursing home residents (Chidambaram, 2020; Gorges & Konetzka, 2020; Sugg et al., 2021). To ensure appropriate tracking, response, and mitigation of COVID-19 in nursing homes, the Centers for Medicare & Medicaid Services (CMS) mandated nursing homes to enroll and begin reporting COVID-19 cases into the Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) (CMS, 2020a). This study was conducted to explore the relationship between accreditation status and COVID-19 case rates. Performance improvement is the cornerstone of Joint Commission accreditation, with requirements designed to help organizations ensure the highest level of patient safety and quality of care. Specific standards related to infection control, emergency management, and patient care-related processes were hypothesized to be particularly relevant during this public health emergency. To test this hypothesis, this study compared COVID-19 case rates within nursing homes that were located in states where the numbers and proportions of Joint Commission accredited and nonaccredited facilities made such comparisons possible (Illinois (IL), Florida (FL), and Massachusetts (MA)).

Methods

Study Design

This observational study used cumulative data from the CMS Nursing Home Compare Public Use File (downloaded in January 17, 2021), which included retrospective COVID-19 data submitted by nursing homes to the CDC NHSN Long-Term Care Facility COVID-19 Module.

Data Sources

COVID-19 case rates were obtained from the CDC's NHSN Nursing Home COVID-19 Public Use File. The COVID-19 case rate data included in this dataset was self-reported in two phases, an initial submission phase, and a subsequent weekly submission of data. The initial submission phase required nursing homes to report data by May 31, 2020 with cases starting the week of May 8, 2020. It also invited nursing homes to voluntarily report any cumulative cases observed since January 2020. Weekly submissions were required after May 31. This approach for the initial submission may have presented challenges for some nursing homes, based upon data entry omissions and the number of records that failed NHSN data quality check. As a result, this study excluded data submitted from the initial submission phase and relied exclusively on the weekly submission of data.

A Joint Commission dataset was utilized to identify accredited nursing homes in the states of IL, FL, and MA. These specific states were selected because a sufficient number of nursing homes within the state were accredited, and thus could be compared to nonaccredited facilities in the same states. Nursing homes were identified by their CMS Certification Number (CCN), classified as accredited or nonaccredited, then matched to the CMS Nursing Home Compare dataset. Community prevalence of COVID-19 was determined using county-level COVID-19 case data obtained from the New York Times GitHub site which maintains a daily count of COVID-19 cases reported by state and county health departments.

Study Population and Data Collection

To be included in the study sample, nursing homes must have submitted data to NHSN that met the CMS data quality criteria (CMS, 2021b). Although NHSN provided instructions and data dictionary definitions of required data elements, many were somewhat vague and subject to interpretation (CMS, 2021b). Based on the distribution of the number of weeks nursing homes submitted data, those that submitted 30 or more times (or 85% of total possible weekly submissions) during the 35-week timeframe were included in the study sample.

Variables

The outcome variable was COVID-19 case rate, represented as the number of cases, over a 35-week period (June 2020–January 2021), divided by the number of occupied beds, and then standardized per 1,000 residents. Potential mediating factors included state, and county-level COVID-19 case rate (included as a continuous variable), which was defined as total COVID-19 cases within the county per 100,000 people, based upon the 2019 U.S. Census populations for each county. Joint Commission accreditation status was the independent variable of interest.

Statistical Analysis

Descriptive statistics were calculated for all the variables used in the study by accreditation status and state. The analysis of these data needed to account for two features: an excess amount of nursing homes with zero case rates and systematic unobserved factors that resulted in overdispersion (excess variability) of the rates. The model used in the analysis was a zero-inflated Poisson negative binomial (ZIPNB) model which incorporated these two features. The ZIPNB model fits a logistic regression model using those with zero cases as a binary indicator and a negative binomial Poisson model to model the count data for the nonzero cases. The county case rate, accreditation status and state were used as covariates in both models. Residuals were calculated to further explore the fit of the model.

Results

Descriptive statistics of nursing homes in the study sample by accreditation status and state are shown in Table 1. There were 1,719 nursing homes in our study, 42.2% (n = 725) of which were accredited (FL = 329, IL = 177, MA = 219) and 57.8% (n = 994) were nonaccredited (FL = 342, IL = 504, MA = 148). Of the 725 accredited nursing homes in our study, 1.8% (n = 13) reported having no COVID-19 cases. Of the 994 nonaccredited nursing homes, 4.0% (n = 44) reported having no COVID-19 cases during the study period. Across all three states in our study, fewer accredited nursing homes reported having no COVID-19 cases compared to nonaccredited nursing homes.

Table 1.

Descriptive Statistics of Nursing Home Sample by State, County, and Accreditation Status.

Variable	M	SD	Mdn	% with Zero Cases
COVID-19 cases per 1,000 residents
Illinois, n = 681
Accredited, n = 177	300.1	234.4	252.9	0
Nonaccredited, n = 504	241.0	236.4	176.0	3.8
Florida, n = 671
Accredited, n = 329	230.7	192.7	178.5	1.2
Nonaccredited, n = 342	211.6	197.2	155.1	4.1
Massachusetts, n = 367
Accredited, n = 219	378.6	371.5	242.6	4.1
Nonaccredited, n = 148	365.0	363.4	240.4	7.4
County-level COVID-19 cases per 100,000 residents
Illinois, n = 681
Accredited, n = 177	1231.0	378.2	1488.6
Nonaccredited, n = 504	532.9	540.1	247.6
Florida, n = 671
Accredited, n = 329	208.9	155.0	144.2
Nonaccredited, n = 342	251.1	188.5	148.2
Massachusetts, n = 367
Accredited, n = 219	1328.3	407.5	1308.2
Nonaccredited, n = 148	1352.9	408.3	1308.2

The univariate analysis showed that nursing homes in our sample located in MA had the highest mean COVID-19 case rate for both accredited nursing homes (M = 378.6, SD = 371.5, Mdn = 242.6) and nonaccredited nursing homes (M = 365.0, SD = 363.4, Mdn = 240.4). Nursing homes in our sample located in FL had the lowest mean COVID-19 case rate for both accredited nursing homes (M = 230.7, SD = 192.7, Md = 178.5) and nonaccredited nursing homes (M = 211.6, SD = 197.2, Md = 155.1). In IL, accredited nursing homes tended to be located in counties with higher case rates, although this pattern was not present in the other two states.

Analysis of the Zero COVID-19 Case Rates (Zero Model)

As shown in Table 2, an increase in the county-level COVID-19 case rate was significantly associated with a decrease in the probability of a nursing home having no COVID-19 cases (p < .001). Nonaccredited nursing homes were statistically more likely to have zero case rates (p = .003). At the state level, using the state of Florida as a reference, nursing homes in MA were more likely to have zero case rates (p < .001), but no differences between IL and FL were observed (p = .69).

Table 2.

Covariates (County, State, Accreditation Status) in Both Zero and Count Models.

	Estimate	SE	P-value
Zero Model
Intercept	−4.079	0.397	<.001
County Rate	−0.002	0.000	<.001
Nonaccredited	1.099	0.376	.003
State: Illinois	0.154	0.386	.69
State: Massachusetts	2.536	0.536	<.001
Count Model
Intercept	−1.524	0.049	<.001
County Rate	0.000	0.000	<.001
Nonaccredited	−0.041	0.052	.43
State: Illinois	0.028	0.065	.67
State: Massachusetts	0.261	0.100	.009

Note. State comparisons use the state of Florida as the reference. Because our study zero model is on the logit scale, the intercept for the zero model represents the predicted log odds of the probability of zero cases, log(p/(1−p)), when all the covariates are zero and where p is the predicted probability of a zero case. The exponent of the intercept for the count model represents the predicted number of COVID-19 cases when all covariates are zero.

Comparison of Nonzero COVID-19 Case Rates (Count Model)

As with the previous model, increases in the county COVID-19 rate had a significant association with higher nursing home COVID-19 case rates (p < .001) (Table 2). After adjusting for county rates, mean case rates for accredited and nonaccredited nursing homes were not statistically significantly different (p = .44). State COVID-19 case rates were statistically significantly higher for MA when compared to FL (p = .009), and IL rates were not significantly different from FL (p = .67).

Using the COVID-19 county rates to predict nursing home case rates, the predicted case rates for accredited nursing homes were found to be higher than the predicted case rates for nonaccredited nursing homes (Accredited Rate = 293.4, Nonaccredited rate = 248.6, Wilcoxon test p < .001). This implies that accredited nursing homes tend to be located in communities with a greater prevalence of COVID-19.

By calculating the standardized difference between the predicted COVID-19 case rate and the observed COVID-19 case rate for each nursing home (i.e., Pearson residuals), outliers were identified among accredited and nonaccredited nursing homes. Figure 1 shows the expected range of the standardized residuals is between −2 and 2. For nursing homes with large positive standardized residuals (i.e., residuals above 2), the observed COVID-19 case rates were significantly higher than their predicted rates. Of the 1,719 nursing homes in our sample, there were 65 outliers (3.8%). The majority of nursing homes with standardized residuals over 2 were nonaccredited (n = 43 of 994, 4.3%). Accredited nursing homes had significantly fewer outliers with standardized residuals over 2 (n = 22 of 725, 3.0%, Fisher's exact test p = .040). These differences were more extreme for the 15 nursing homes with a residual greater than 3. Twelve of the 15 highest outliers (residuals > 3) were from nonaccredited nursing homes (n = 12 of 994, 1.2%, Fisher's exact test p = .047), whereas accredited nursing homes had significantly smaller proportion of outliers over 3 (n = 3 of 725, 0.41%).

Figure 1.

Residuals by predicted COVID-19 case rate, state, and accreditation status.

Note. Scatterplot of residuals versus predicted COVID-19 case rates. Plot of the standardized Pearson residuals from the zero-inflated Poisson negative binomial (ZIPNB) model by the predicted case rates from the model, identified by accreditation status (pink dot denotes accredited nursing home, blue dot denotes nonaccredited nursing home) and split out by state (FL = Florida, IL = Illinois, MA = Massachusetts).

Discussion

The strongest predictor of COVID-19, for both accredited and nonaccredited nursing homes in our sample, was the community prevalence of COVID-19. As one would expect, nursing homes located in counties with a higher prevalence of COVID-19 had higher COVID-19 case rates. This observation is consistent with previous studies which found that when the surrounding population has a high COVID-19 rate, nursing homes in that area are at greater risk of experiencing higher COVID-19 rates (Bagchi et al., 2021; Chatterjee et al., 2020). This finding underscores the challenge nursing homes face, which is the possibility of staff and visitors, as well as residents transitioning to and from the facility, being asymptomatic carriers of COVID-19, unknowingly bring the virus into the nursing home (Bagchi et al., 2021).

While the mean COVID-19 case rates among accredited and nonaccredited nursing homes were comparable after correcting for state and county case rates, the proportion of outliers differed. Greater variability in case rates was noted among nonaccredited nursing homes, despite the fact that relatively more nonaccredited nursing homes were located in counties with lower COVID-19 case rates. This was most apparent in IL where nonaccredited nursing homes were generally at lower risk (i.e., lower COVID-19 county case rates) but had a greater proportion of higher than expected COVID-19 case rates. In contrast, accredited nursing homes in our sample were more likely to be located in areas with high county COVID-19 case rates compared to nonaccredited nursing homes. Despite this, accredited nursing homes had fewer residuals over 2 indicating that their predicted COVID-19 case rates were reasonably aligned with their observed case rates. Based on the observed influence of community prevalence of COVID-19 on nursing home case rates, one would expect that accredited nursing homes in our sample would have higher case rates (compared to nonaccredited nursing homes). Where accredited nursing homes performed as expected, given their location in higher risk areas, nonaccredited nursing homes performed less predictably.

An explanation for why a higher proportion of nonaccredited nursing homes underperformed (i.e., had COVID-19 case rates that far exceeded expected rates), in comparison to accredited nursing homes is illusive. Previous studies have found that TJC accredited nursing homes, when compared to nonaccredited nursing homes, are more likely to make resident safety a top priority, more likely to allocate resources to support this priority, and more likely to be focused on systems and processes that promote optimal quality of care (Wagner et al., 2012a, 2012b, 2012c). Another study found a consistent pattern of superior performance among nursing homes accredited by TJC when compared to non-TJC accredited nursing homes across a broad range of indicators in the Nursing Home Compare dataset (Williams et al., 2017). While the results of this study are generally consistent with differences observed in previous studies, the specific factors that may have contributed to more consistent COVID-19 case rates, or that prevented failures which led to unexpectedly high rates, are not known.

Limitations

There are several limitations to this study. Key factors not included in our multivariate model such as nursing home case mix, ownership type, socioeconomic status, race/ethnicity of residents and availability of supportive resources may have a significant impact of COVID-19 rates in a nursing home's resident population. The sample also only included three states which had different geographical footprints. Given the influence of county case rates, it is difficult to know how much of an impact these geographical differences may have had. Finally, this study relied upon the weekly data submission of COVID-19 cases after May 31, 2020. If reliable data from January 2020 could have been included in the analysis it is possible that results may have differed—especially for those nursing homes that were located in hot spots very early in the pandemic.

Conclusion

Nursing homes face numerous multifaceted challenges as they work to manage the impact of the COVID-19 pandemic. Consistent with other studies, this study revealed that the community prevalence of COVID-19 is the strongest predictor of COVID-19 cases in nursing homes. While accreditation status was not associated with differences in overall COVID-19 case rates, accreditation may have had some impact on variability in performance. The greater degree of consistency among accredited facilities is consistent with findings from previous studies that report accredited facilities are more likely to make resident safety a top priority, more likely to allocate resources to support this priority, and more likely to be focused on systems and processes that promote optimal quality of care. Further research is warranted to better understand the interaction of accreditation and other factors, such as ownership status, on the prevalence of COVID-19 among nursing home residents and staff. These important insights might help inform preparedness efforts for future emergency situations that may emerge.