A prospective multicentre external validation study of the Liverpool Peritonsillar abscess Score (LPS) with a no-examination COVID-19 modification.

OBJECTIVES: Our primary aim was to validate the Liverpool Peritonsillar abscess Score (LPS) externally in a new patient cohort. Our secondary aim was to modify the LPS in the light of the COVID-19 pandemic to produce a no-examination variant for use in this instance.
DESIGN: Prospective multicentre external validation study.
SETTING: Six different secondary care institutions across the United Kingdom. PARTICIPANTS: Patients over 16 years old who were referred to ENT with any uncomplicated sore throat such a tonsillitis or peritonsillar abscess (PTA). MAIN OUTCOME MEASURES: Sensitivity, specificity, positive predictive value and negative predictive value for both the original LPS model and the modified model for COVID-19.
RESULTS: The LPS model had sensitivity and specificity calculated at 98% and 79%, respectively. The LPS has a high negative predictive value (NPV) of 99%. The positive predictive value (PPV) was slightly lower at 63%. Receiver operating characteristic (ROC) curve, including the area under the curve (AUROC), was 0.888 which indicates very good accuracy.
CONCLUSIONS: External validation of the LPS against an independent geographically diverse population yields high NPV. This may support non-specialist colleagues who may have concerns about mis-diagnosing a PTA. The COVID-19 modification of the LPS has a similar NPV, which may be of use where routine oral examination is to be avoided during the COVID-19 pandemic.

There are increasing numbers of inpatient admissions with peritonsillar abscesses (PTA) and deep neck space infections

The Liverpool Peritonsillar abscess Score (LPS) is a predictive quantitative score of five variables to be used as an adjunct in assessing patients with sore throats

The LPS model has a high sensitivity and negative predictive value (98% and 99%)

The LPS may be used as a tool to assist non‐specialist colleagues in recognising PTA

A modified LPS which removes the need for oral examination still has high sensitivity (94%) and a negative predictive value (98%) which may be of particular use during the COVID‐19 pandemic

INTRODUCTION

Background

Peritonsillar abscesses (PTA, also known as quinsies) are a frequently occurring presentation to emergency departments and represent a common reason for referral to an ENT service. Management options include drainage of the abscess and antibiotic therapy. Without intervention, PTA can develop into deep neck space infections (DNSI) which carry significant morbidity and mortality. Patients who undergo drainage of their PTA have been shown to experience quicker resolution of symptoms.

Reports suggest that there are increasing numbers of inpatient admissions to secondary care with PTA and DNSI. It is therefore important for both non‐specialists and specialists to arrive at the correct diagnosis in order to facilitate early and appropriate treatment. We previously reported the development of a predictive score for peritonsillar abscess through a prospective multicentre observational study (Liverpool Peritonsillar abscess Score, LPS).

The LPS is an additive threshold score consisting of five variables with the aim of predicting the likelihood of PTA in adult patients. Internal validation of the LPS from the initial study sample produced a high sensitivity and negative predictive value (NPV; 96% for both). The LPS functions very much like the Wells score (which estimates the probability of venous thromboembolism), and its strength lies in its NPV. We theorise that the PPV is lower because a proportion of patients with tonsillitis will have trismus, and because the LPS was developed using aspirate‐proven PTA (rather than peritonsillar cellulitis).

Our aim in developing the LPS was to support non‐specialists in preventing a missed diagnosis of PTA and to facilitate remote triage by secondary care services and to avoid any need for significant extra resources or funding in the process.

Covid‐19 modification

During the current COVID‐19 pandemic, ENT UK has issued updated guidance, reserving oral examination only for severe cases of sore throat. A validated predictive score not reliant on oral examination may therefore be useful to the non‐specialist when triaging or referring a suspected PTA.

Objectives

Our primary aim was to validate the Liverpool Peritonsillar abscess Score (LPS) externally against a separate cohort of patients through a multicentre prospective observational study. Our secondary aim was to modify the LPS in the light of the COVID‐19 pandemic to produce a no‐examination variant for use in this instance.

MATERIALS AND METHODS

Reporting standards

This study and manuscript adhere to the STROBE reporting guidelines for observational studies.

Ethical considerations and institutional review

This project was registered with the clinical governance departments at six secondary care sites: Bradford Royal Infirmary (coordinating), Royal Liverpool University Hospital, Arrowe Park Hospital, Stepping Hill Hospital, Wythenshawe Hospital and Royal Stoke Hospital, all in the United Kingdom. After collection at each site, truly pseudo‐anonymised data (unique study number) were shared with the coordinating site while complying with information governance and data protection procedures.

Sample size estimation

When developing the LPS, we based our previous sample size estimate on the need for a certain number of events per variable (EPV). However, in the external validation of a multivariable predictive model, statistical modelling suggested the need for "at least 100 events and ideally 200 (or more) events". We therefore set our sample size at 200.

Definitions

In line with our previous work, we defined a PTA as clinical suspicion based on typical symptoms and signs with positive aspiration of pus from the peritonsillar space. Non‐PTA sore throat, including tonsillitis and peritonsillar cellulitis, was defined either on senior review (ENT specialty registrar ST3 or more senior) or when there had been suspicion of PTA but a negative aspirate.

Inclusion and exclusion criteria

Data were collected prospectively from August 2018 onwards until we met our sample size requirement. We included consecutive patients over 16 years old who were referred to the ENT service for assessment of sore throat such as tonsillitis or PTA. Patients were assessed in the local emergency department or surgical assessment unit according to standard local protocol. We did not apply any length of stay criteria.

We excluded any patients under 16 years old. We also excluded any patients referred with, or who were subsequently proved to have, supra‐ or epiglottitis or a DNSI.

Data sources

Data quality and uniformity were ensured through the use of a single data collection proforma across all sites. Investigators were briefed on the use of the proforma. Data were collated locally before being pooled using unique study numbers. Investigators scored patients both at the point of referral from either General Practitioners or non‐specialists in the local emergency department, and on assessment by an ENT doctor. The final diagnosis was confirmed both by the presence of pus on aspiration and senior review.

The LPS COVID‐19 modification was developed to assist in risk stratification of PTA during the COVID‐19 pandemic. Since uvular deviation was the only variable requiring oral examination with a tongue depressor, we removed it from the parent LPS and examined the classification function of the modified score.

Statistical methods

Data from each collection site were collated prospectively and entered into Excel workbooks (Microsoft Office 365, Microsoft, Redmond, WA, USA). We analysed sample demographics and reported summary measures. We also calculated the classification function and area under the receiver operating characteristics curve (AUROC). Statistical associations were determined using the Mann‐Whitney U test for skewed continuous data, and the significance level (α) was set at 0.05.

The Liverpool Peritonsillar abscess Score (LPS)

In this study, we used the previously reported final iteration of the LPS (Figure 1).

Figure 1

The Liverpool Peritonsillar abscess Score (LPS). This is an additive score with a cut‐off value of 4. Scores of 0‐3 predict no PTA whereas the likelihood of PTA increases with scores of 4‐8. The LPS is validated for assessing the risk of peritonsillar abscess in patients ≥16 years. It does not replace clinical judgement and appropriate PPE should be used. COVID‐19 modification: Clinicians may remove the uvular deviation component to avoid examining the mouth; there is no change to the cut‐off value (see Results for classification function)

RESULTS

Demographics and frequency of PTA

We included 205 patients referred with sore throat (Table 1). Of these, 54 patients were diagnosed with PTA based on the presence of aspirated pus. There was no significant difference in the median age of patient with or without a PTA (P = .029).Overall, 44% of patients were men and were significantly more likely to have PTA (P < .001).

Table 1

Demographics of patients with and without PTA

	PTA n = 54	Non‐PTA n = 151	P‐value
Median age (years)	27 (range from 17 to 63)	32 (range from 16 to 85)	.029
Male gender (percentage)	65	37	<.001

Accuracy of external validation

In this external validation study, the LPS continues to have a high negative predictive value (NPV; 99%) and sensitivity (98%). Its specificity and positive predictive value (PPV) are lower at 79% and 63%, respectively.

A receiver operating characteristics curve, including the area under the curve (AUROC), was calculated for the LPS. This showed that model accuracy was very high (0.888; Figure 2).

Figure 2

Receiver operating characteristics (ROC) curve for the external validation sample. The area under the curve (AUROC) is 0.888, indicating very good accuracy

The LPS COVID‐19 modification (removal of the uvular deviation component) appeared to have similar classification function based on these data. Sensitivity and NPV remained high at 94% and 98%, respectively (Table 2).

Table 2

Classification function of the LPS COVID‐19 modification (where uvular deviation has been removed). The cut‐off value remains 4

		Observed PTA
		Positive	Negative
Predicted PTA	Positive	53	31	PPV 62%
	Negative	1	120	NPV 98%
		Sens 94%	Spec 79%

DISCUSSION

Synopsis of key results

When tested against an independent geographically diverse population, the LPS demonstrates high NPV and sensitivity, as well as overall accuracy (AUROC). We hypothesise that a lower specificity and PPV are because some patients with tonsillitis will have trismus, and because we defined PTA using definite aspiration of pus in order to reduce subjectivity (Table 3). Since statistical analysis of the LPS has yielded consistent findings across both internal and external validation, further iterative development of the LPS is not indicated.

Table 3

Classification function of the LPS following external validation

		Observed PTA
		Positive	Negative
Predicted PTA	Positive	53	31	PPV 63%
	Negative	1	120	NPV 99%
		Sens 98%	Spec 79%

Abbreviations: Sens, sensitivity; Spec, specificity; PPV, positive predictive value; NPV, negative predictive value.

We have developed a COVID‐19 modification to the LPS, which may be of use where clinicians are trying to reserve oral examination for severe cases during the COVID‐19 pandemic. This modified score also demonstrates high NPV and sensitivity.

Limitations and strengths

This observational study benefits from collection of prospective data over a broad geographical area with a pre‐determined sample size. Cohort characteristics are comparable to many other studies of patients with sore throats, and age and gender distributions are similar to the original observational study used to develop the LPS. , The classification function and AUROC yield high degrees of negative prediction and accuracy, respectively. In developing the LPS, we have tried to use objective measures to ensure that the resulting score is highly reproducible in the hands of many different clinicians, including non‐specialists.

In this study, we have categorised diagnosis into discrete categories of "PTA" or "non‐PTA" sore throat to ensure reproducibility and to reduce subjectivity: this is in line with our previous methodology. This is likely to mean that the LPS does not differentiate between peritonsillar cellulitis and tonsillitis (in fact, it is likely that this is also reflected in the relatively lower specificity and PPV of the LPS). The stated aim of the LPS, however, is to avoid a missed diagnosis of PTA rather than to differentiate between tonsillitis and peritonsillar inflammatory disease. In the UK, most PTA is treated with a combination of drainage and antibiotics. By contrast, both peritonsillar cellulitis and tonsillitis can be treated medically with antibiotics and supportive advice.

It is also possible that some of the patients diagnosed as having non‐PTA sore throat had PTAs which did not yield an aspirate, either due to small size or due to inexperience on the part of the admitting clinician. However, no patient, once formally diagnosed with non‐PTA sore throat, required a repeat trial of aspiration. These factors may therefore affect relatively small number of patients and are unlikely to affect the reliability of the score.

Comparison with other studies

Almost in parallel with our original study, Spiekermannet al reported a PTA predictive score which included six variables. Score parameters included presence of halitosis, uvular oedema, unilateral swelling of the palatal arches and trismus, as well as presence of an inflammatory biomarker in both serum and saliva (S100 A8/A9 complex). The study was retrospective in nature and included a smaller sample of 75 patients (24 acute tonsillitis; 51 peritonsillar inflammation). The authors aimed to determine whether they could predict if a patient with peritonsillar inflammation would benefit from medical treatment or "abscess relief" with the primary aim being able to differentiate between peritonsillar abscess from cellulitis. Demographics and key definitions were similar between that study and our own. The medical context, however, appeared to be different in that a significant number of patients underwent abscess tonsillectomy, which is now rarely performed in the UK. In addition, the use of S100 A8/A9 complex as a biomarker for peritonsillar inflammation is not in widespread use at present, is limited to the research sphere and requires additional laboratory resources and funding.

Generalisability

External validation now provides a measure of confidence in the reliability and utility of the LPS. The LPS does not seek to alter the threshold for admitting patients with sore throat (when they cannot swallow enough to remain hydrated and to take medication). Instead, its clinical applicability lies in the referral interface between primary or emergency care and ENT, and in stratification of a patient's PTA risk.

The primary method of transmission of COVID‐19 is thought to be via droplet spread: clinicians may be at greater risk when examining the oropharynx. Updated ENT UK guidance attempts to limit clinician exposure during patient assessment, and the COVID‐19 modification of the LPS could be used to complement this guidance.

CONCLUSION

In this study, we have validated the LPS externally in a new patient cohort, yielding findings consistent with our original work in which this score was developed. We have demonstrated that LPS is a robust and easy‐to‐use tool for colleagues seeking to stratify the risk of PTA in their patients.

CONFLICTS OF INTEREST

None.