COVID-19 outbreaks in care homes during the first wave: are Care Quality Commission ratings a good predictor of at-risk homes?

BACKGROUND: COVID-19 placed a significant burden on the care home population during the first wave. Care Quality Commission's care quality ratings are given to all English care homes and could be used to identify whether care homes are at risk of COVID-19 outbreaks.
METHODS: An audit was performed, April 2020, of Liverpool care homes to identify associations between COVID-19 status and care quality ratings from the Care Quality Commission. Univariable logistic regression was performed to identify whether the Care Quality Commissions ratings were associated with a care home experiencing a COVID-19 outbreak, or a home having asymptomatic cases.
FINDINGS: Over half of the care homes (53.2%, n=41), had laboratory-confirmed clinical cases and 39 (95.1%) of these were reported as outbreaks. A small number of care homes (10.4%, n=8) had asymptomatic cases and over a third had no clinical or asymptomatic cases (36.4%, n=28). There was no significant difference between the overall Care Quality Commission rating of Liverpool and English care homes (p=0.57). There was no significant association between any of the Care Quality Commission rating domains and the presence of COVID-19 outbreaks and/or asymptomatic cases.
CONCLUSIONS: During the first wave of the COVID-19 pandemic, Care Quality Commission ratings were not associated with COVID-19 outbreaks or asymptomatic cases in care homes. Infection prevention and control components of Care Quality Commission ratings need to be strengthened to identify care homes at a potential risk of infectious disease outbreaks that may require targeted support. Further large-scale studies will be required to test the findings from this study.

Introduction

Liverpool, located in the North West of England, is one of the main cities in the United Kingdom. The city is governed by Liverpool City Council (LCC) who are responsible for a population of approximately 494,800. The population is young, with an average age of 38 and only 14.6% being 65 years old and over, compared with the rest of England (average age 40, 18.8% over 65) [1]. Liverpool has public health challenges as it is the fourth most deprived local authority in the country. Fifteen percent of the population are multimorbid, life expectancy is lower than the national average, and there are large health inequalities in the population [1,2]. With an ageing population, these issues may be exacerbated as an increasing demand is placed on Liverpool’s care homes.

The first laboratory-confirmed COVID-19 case, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), in Liverpool was reported on the 2nd March 2020 [3]. The first COVID-19-related death in Liverpool was registered in the week commencing the 16th March 2020 [3,4]. By the end of May 2020, in Liverpool, 1630 cases were confirmed (329.4 cases per 100,000 population), alongside 550 deaths (111.2 deaths per 100,000 population) [4]. Of these deaths 25.6% had occurred in care homes; by comparison 29.6% of COVID-19 deaths in England had occurred in care homes (Figure 1 ).

Figure 1

Epidemic curves of daily cases and weekly deaths of COVID-19 in Liverpool. The dashed line represents when the current study was performed.

Care homes have been identified as a high-risk setting for COVID-19. Care home residents are often older with comorbidities, which are both key risk factors for increased COVID-19 mortality [[5], [6], [7]]. The high burden of mortality in care homes during the first wave of the current COVID-19 pandemic compels health and social care commissioners and providers to have in-depth understanding of the risk factors that are critical for preventing any future negative impacts on care home residents.

In England, the use of agency staff has been found to increase the risk of COVID-19 infection in staff and residents [8]. It is unclear whether this is related to agency staff regularly moving between homes, which increases the risk of virus transmission, or whether it reflects a quality issue because agency staff are less likely to be familiar with an individual care home’s infection prevention and control (IPC) procedures. The use of agency staff may also signify wider resourcing stresses on the care home in question. Failure to implement IPC processes by care home staff has been identified as a key cause of infectious disease outbreaks in care homes [9]. Additionally, high prevalence of infection in staff, and homes not providing staff with sick pay, were associated with COVID-19 infection in residents [8]. In the USA, larger care homes and urban cares homes were associated with an increased risk of cases [10].

One potential risk factor for COVID-19 outbreaks in care homes is the overall quality of care delivered (including IPC) to residents in the home. In California (USA), research found that the Center for Medicare and Medicaid Services star rating, a care quality standard for nursing homes in the USA, had a significant impact on the number of COVID-19 cases and deaths in nursing homes [11]. Nursing homes with poor quality ratings had a significantly higher probability of having COVID-19 cases and deaths. In contrast, a far bigger study covering 30 American states and over 9000 care homes found no association between care quality ratings and COVID-19 status [10].

In England the Care Quality Commission (CQC) are the independent regulator of health and social care services [12]. They inspect and regulate care homes by maintaining a register and undertaking regular monitoring and inspection visits. These inspections aim to provide an overall judgement of the quality of care in care homes by assigning ratings. Action plans are developed if there are areas of concern, and they can take further action, such as fines or prosecution if there is no improvement. Each care home’s CQC rating is made up of five domains [13]. The ‘safe’ domain assesses whether residents are protected from abuse and avoidable harm. The ‘effective’ domain assesses whether residents’ care, treatment and support achieve good outcomes and that quality of life is maintained. The ‘caring’ domain assesses whether staff treat residents with compassion, kindness, dignity and respect. The ‘responsive’ domain assesses whether the services offered are organized and meet residents’ needs. The ‘well-led’ domain assesses the leadership, management and governance of the home and ensures that it provides high-quality individual care, whilst encouraging learning and promoting an open and fair culture.

The five domains are used to assess the quality of a care home. Although there is no specific domain that considers IPC, it is possible that IPC would be partly considered under the ‘safe’ and other relevant domains. For example, the ‘effective’ domain is crucial to IPC, as many of these measures are dependent on the skills and training of care home staff. IPC measures are critical in preventing outbreaks of infectious diseases in care homes [9]. Therefore, CQC ratings could be an essential indicator for identifying care homes at increased risk of COVID-19 outbreaks.

Establishing an association between CQC rating(s) and the risk of COVID-19 outbreak(s) would enable pre-emptive measures to be put in place within care homes. The aim of this study was to determine whether CQC ratings are associated with COVID-19 outbreaks in care homes.

Methods

Care homes in England are required by Schedule 1 of The Health and Social Care Act 2008 (Regulated Activities) Regulations 2014 to register with the CQC [14]. Care homes within the LCC region were identified through the CQC website and validated by public health officials within LCC [15]. On the week commencing 27th April 2020, LCC identified all care homes that had experienced/reported a single case or outbreak of COVID-19.

Care homes report symptomatic residents and staff to the Liverpool IPC Team. This team assesses information about the clinical presentation and testing results before categorizing the care home’s COVID-19 status using the following Public Health England (PHE) definitions. An outbreak in a care home is defined as “two or more cases which meet the case definition of possible or confirmed case, within a 14-day period among either residents or staff in the care home” [16]. At the time of this study, a possible case of COVID-19 in a care home was defined as any resident (or staff) with symptoms of COVID-19 (high temperature or new continuous cough), or new onset of influenza-like illness or worsening shortness of breath. A confirmed case of COVID-19 is defined as any resident or staff with laboratory confirmed diagnosis of COVID-19.

On the 27th April 2020, as part of LCC’s COVID-19 surveillance programme, all homes who had not notified the local IPC team of a possible/confirmed case or an outbreak of COVID-19 were offered testing of all their residents for COVID-19. Consequently, all care homes in Liverpool were categorized into three groups: those who had experienced cases or outbreaks, those with confirmed asymptomatic cases, and those without any cases (symptomatic/asymptomatic). The overall CQC rating of each care home, and the respective domain scores, were extracted from the CQC’s website [15] and subsequently linked to LCC’s data. To understand the representativeness of LCC care homes in relation to English care homes (also extracted from the CQC website), Fisher’s exact tests were performed to compare the overall CQC ratings and each respective domain.

Univariable logistic regression was performed to identify whether CQC ratings were associated with a care home experiencing a COVID-19 outbreak. If variables were found to be significant, multivariable logistic regression models were created with variables selected using a backward stepwise selection. The same analytical process was performed to identify whether CQC ratings were associated with asymptomatic care homes identifying laboratory-confirmed cases. Statistical significance was defined as P<0.05 and all analyses were carried out using R version 3.60.

Results

At the point of assessment, the week commencing 27th April 2020, 1386 cases of COVID-19 (280.1 cases per 100,000 population) had been identified in Liverpool, and 430 Liverpool residents had died (86.9 deaths per 100,000 population). Of these, 100 (23.3%) had occurred in care homes [4]. There were 86 CQC-registered care homes in the LCC region with capacity to care for 3458 residents. Of these homes, four had not been inspected yet by the CQC as they opened in 2019, and two were temporarily closed. Three further homes were excluded from the study as they had less than three residents each. It was therefore possible to analyse 77 (89.5%) of the care homes within LCC. Of these, 11 (14%) had been assessed by the CQC in 2020, 43 (55.8%) in 2019, 18 (23.4%) in 2018, and five (6.5%) more than two years ago.

Nearly half (36, 46.8%) of the care homes had no clinical cases of COVID-19 and 41 (53.2%) homes had laboratory-confirmed clinical cases. Of the homes with laboratory-confirmed clinical cases, 39 (95.1%) had been reported as having outbreaks. The residents from homes with no clinical cases were tested for COVID-19 to confirm the presence of any asymptomatic cases. Consequently, eight (22%) of these homes had asymptomatic cases present. The CQC ratings for the care homes are summarized in Table I .

Table I

Care Quality Commission (CQC) ratings for care homes in Liverpool City Council (LCC) and England

CQC rating domain	Percentage and number of care homes in each CQC rating								Fisher’s exact test P
	Inadequate		Requires improvement		Good		Outstanding
	LCC	England	LCC	England	LCC	England	LCC	England
Overall	1.3% (1)	1.4% (212)	18.2% (14)	16.0% (2410)	79.2% (61)	78.2% (11796)	1.3% (1)	4.4% (658)	0.57
Safe	1.3% (1)	1.5% (223)	29.9% (23)	9.9% (2767)	68.8% (53)	79.8% (12035)	0	0.3% (51)	0.08
Effective	0	0.6% (89)	13.0% (10)	11.6% (1750)	84.4% (65)	86.1% (12987)	2.6% (2)	1.7% (250)	0.71
Caring	0	0.2% (32)	7.8% (6)	5.0% (754)	87.0% (67)	90.4% (13627)	5.2% (4)	4.4% (663)	0.48
Responsive	0	0.3% (51)	14.3% (11)	9.9% (1500)	85.8% (66)	83.4% (12580)	0	6.3% (945)	0.04
Well led	1.3% (1)	2.1% (320)	24.7% (19)	20.5% (3093)	72.7% (56)	73.4% (11070)	1.3% (1)	3.9% (493)	0.62

Based on the overall CQC rating there was no significant difference between LCC and English (N = 15,076) care homes (P=0.57). The only CQC domain where there was a significant statistical difference was the ‘responsive’ domain. LCC care homes were more likely to receive negative scores than positive scores (14.3% vs 85.7%) compared with homes in the rest of England (10.2% vs 89.8%, P=0.04).

Many of the domains and ratings had one or less care homes matched to them, therefore, to allow analysis, the ratings were aggregated into positive or negative ratings. Univariable analysis showed that there was no association between the overall CQC rating, or any of the CQC domains, and the presence of symptomatic confirmed COVID-19 cases in care homes (Table II ). Due to this, multivariable models were not created. The only variable of potential interest was ‘responsive’, where care homes with a ‘bad’ score had 4.78 (95% CI 1.13–32.96, P=0.06) times higher odds of having experienced an outbreak, however this was not statistically significant.

Table II

Univariable analysis of Care Quality Commission (CQC) ratings associations with COVID-19 outbreaks, and the presence of asymptomatic COVID-19 confirmed cases in Liverpool Care Homes

Variable	Outbreak present odds ratio (95% CI)	P	Asymptomatic COVID-19 cases present odds ratio (95% CI)	P
Overall CQC score (all references are ‘good’ scores)	2.00 (0.63–7.05)	0.25	2.78 (0.31–20.82)	0.32
CQC Domain – safe	1.73 (0.65–4.78)	0.28	1.00 (0.13–5.62)	1.00
CQC Domain – effective	Not calculable	N/A	Not calculable	N/A
CQC Domain – caring	Not calculable	N/A	Not calculable	N/A
CQC Domain – responsive	4.78 (1.13–32.96)	0.06	Not calculable	N/A
CQC Domain – well led	1.45 (0.52–4.20)	0.48	2.76 (0.45–15.77)	0.25

CI, confidence interval.

Univariable analysis showed that there was no association between the overall CQC rating, or any of the CQC domains, and the presence of laboratory-confirmed COVID-19 cases in asymptomatic care homes (Table II). Multivariable analysis was therefore not performed.

Discussion

This is the first study to describe care home COVID-19 outbreaks and their relationship with care quality standards in an English local authority. Care homes in England have been severely impacted by COVID-19. By the 27th April 2020 more than half of Liverpool care homes with a CQC rating had possible or confirmed cases of COVID-19. Of these care homes, 95% presented as outbreaks. In England, by the end of June, 56% of care homes had reported at least one confirmed case of coronavirus, with 11% of all residents and 4% of all staff testing positive [8]. This study highlights that the English care home CQC rating system is not associated with COVID-19 presence. Therefore, it raises an important question whether CQC is assessing the correct information for keeping residents safe from infectious disease outbreaks.

This study was possible due to the close links between the LCC public health team, adult social care, and the care homes of Liverpool. This enabled accurate categorization of care homes and robust data collection. Care homes were evenly distributed throughout the LCC region, with no obvious visual clusters of outbreaks or asymptomatic homes (data not presented to maintain anonymity of care homes). The majority of care homes participated in the testing of care homes without outbreaks (N = 34), which ensured that this research was representative of all the care homes in Liverpool. Overall, the CQC ratings of Liverpool care homes were similar to those of the rest of England. The proportion of homes that were rated poorly in the ‘safe’ domain, were higher in Liverpool (31.2%) compared with the rest of England (11.4%). It would be reasonable to expect this domain to include some assessment of IPC measures, and if so efforts to improve IPC in Liverpool may improve the overall ‘safe’ domain rating. In contrast, the majority of care homes (92.2%) had been given positive ratings for the ‘caring’ domain. This shows that most residents were treated with compassion, dignity and respect, something that would be greatly needed during a period of high stress and worry, such as the COVID-19 pandemic.

The only domain which showed a statistical difference to England (P=0.04), where Liverpool scored worse, was the ‘responsive’ domain. This measures how organized the services were and how well they met the residents’ needs. Interestingly, this was the only CQC domain that was potentially associated with a COVID-19 outbreak in Liverpool; negatively rated homes were 4.78 (95% CI 1.13–32.09, P=0.06) times more likely to experience an outbreak than positively rated homes. It is plausible that care homes that are less ‘responsive’ are more likely to have staffing issues and perhaps rely more on agency staff. However, this is more likely to be due to the relatively small sample compared with the rest of the country, and without multivariable analysis it is difficult to show that this domain is independently associated with outbreaks. This study needs to be repeated on a larger scale to have enough power to identify this. There is the potential that disorganized services, that are not adequately meeting resident’s needs, increase the risk of COVID-19 transmission thus leading to an outbreak.

This study found that CQC ratings are not associated with either a care home having an outbreak of COVID-19 or having asymptomatic confirmed cases of COVID-19. The current CQC ratings are not an appropriate predictor of risk for COVID-19 outbreaks. It should be recognized that a third of care homes had ratings that were two or more years old. Therefore, these ratings may not be reflective of the quality of care delivered in the care home at the time of the study. It is also possible that some care homes have improved, and others declined, in their quality of care since the inspection. Undertaking CQC inspections regularly and more frequently would ensure that ratings are more reflective of the true quality of care in care homes.

Our findings are consistent with other studies that have found that care quality ratings were not associated with risk of infectious disease outbreaks. A study of norovirus outbreaks in Cheshire and Merseyside found that CQC rating was not associated with an increased risk of norovirus outbreak [17]. But as with this study, the CQC ratings were often found to be significantly out of date when compared with the time of the outbreak. Our findings are also supported by a widespread American study that found no association between their care quality rating system and the presence of COVID-19 [10].

Risk factors associated with COVID-19 in care homes that have been previously identified include: use of agency staff, urban location, and size of home [7,8,10]. The CQC inspection reports and ratings can easily identify these risk factors [13]. Work has yet to be performed to explore how various IPC policies, their strictness, and adherence, have impacted the risk of a COVID-19 outbreak(s) in care homes. We suspect poor use and compliance of IPC would be associated with outbreaks, as seen with other infectious diseases [9]. The unfamiliar nature of the virus for care home staff meant that even care homes with good IPC practices may have struggled to control virus transmission. All care homes in Liverpool were provided with written IPC guidance specific to COVID-19, however, the lockdown restrictions made it challenging to assess the extent to which care homes adhered to guidance. Without including an explicit assessment of IPC, public health authorities do not know how care homes are performing in this regard.

Many infectious diseases lead to deaths and hospitalizations in care homes and place a significant burden on the staff and residents [18,19]. In England the fourth most prevalent cause of death in care homes is influenza [19]. In a four-year period in the North West of England, a rate of 37.1 norovirus outbreaks per 100 care homes per year has been calculated, with a hospitalization rate of 8 per 1000 resident cases, and a death rate of 3 per 1000 resident cases [17]. By including a new IPC assessment and domain as part of the CQC ratings, care homes that require additional support to manage the risk associated with infectious diseases would be identified. This would enable public health partners working in community IPC to offer more targeted support to care homes which are performing badly. In the current emergency, brought on by the COVID-19 pandemic, following each assessment, it would provide local authority Directors of Public Health with a list of care homes that may require additional support and intervention. The provision of a new IPC domain would have the additional benefit of helping raise IPC standards outside of the pandemic, potentially reducing the frequency and severity of other infectious disease outbreaks, and thereby improving the health and welfare of English care home residents.

The main limitation of this study was its cross-sectional design. It is unknown whether the findings would have differed if it had been performed at a different time point. However, as seen in Figure 1, the study started after the peak of the first wave of COVID-19 in Liverpool, and when England’s government lockdown was at its strictest [20]. Since the study finished, LCC has confirmed that none of the asymptomatic care homes have gone on to have COVID-19 outbreaks. Thus, we believe our results to be representative of the situation in Liverpool care homes. The lack of testing in care homes at the beginning of the pandemic in England, and the similarity of COVID-19 symptoms (fever and cough) to those of other illnesses, may have resulted in an under-reporting of outbreaks by care homes. The identification of outbreaks was also reliant on proactive reporting of symptomatic cases by care homes to the IPC team at LCC. Although all care homes were notified about this process at the start of the pandemic, some care homes were more familiar with this process than others. Despite this study being the first to include the overwhelming majority of care homes in a local authority (89.5%), it is still relatively small (N = 77) compared with the number of variables under assessment. This resulted in limited statistical power, and only univariable analysis was possible. We were therefore unable to identify whether any CQC domains were associated with COVID-19 outbreaks or asymptomatic cases independently of each other. This study needs to be repeated in other local authorities, or nationally, to help support our proposal of the addition of a new IPC domain in the CQC ratings. Further work is required to identify modifiable risk factors that reduce the likelihood of COVID-19 outbreaks in care homes. Specific focus should be placed on which IPC practices being implemented in care homes are associated with reduced COVID-19 transmission.

In conclusion, CQC ratings are not associated with the likelihood of a care home experiencing a COVID-19 outbreak in Liverpool during the first wave of COVID-19. It appears that CQC ratings are unlikely to predict which care homes experience outbreaks or are in need of support. In addition to strengthening the IPC component of the CQC ratings, a further large-scale study on this topic could provide wider evidence to test these findings. Stronger evidence on this will not only be beneficial in the current pandemic but will assist in all future infectious disease outbreaks in care homes, not least seasonal influenza.