Covid tests in secondary schools: A statistical cause célèbre.

Sheila Bird offers a personal account of the debate over rapid coronavirus screening in secondary schools, and the efforts to reinstate confirmatory PCR tests.

Annie Spratt/Unsplash.com

When the time came to reopen secondary schools and colleges in England in March 2021, following a second wave of the Covid‐19 pandemic, the government announced that rapid tests for coronavirus infection would be offered to students. This mass testing strategy was intended to help schools and colleges identify and isolate asymptomatic cases, thereby curtailing wider outbreaks.

A group of statisticians, however, was concerned that the tests might result in a high number of false positives. Worse, the government had ruled out offering confirmatory tests for secondary students testing positive via rapid tests at school, meaning that uninfected students – and their classmates and families – might have to isolate unnecessarily, missing valuable days of education (and work) in the process. Sheila Bird – one of this group of concerned statisticians – picks up the story.

22 February 2021

The Department for Education (DfE) announced today that schools and colleges in England would reopen to all pupils from 8 March. In a bid to detect asymptomatic cases of Covid‐19, the DfE said that all secondary school and college students would, over the course of two weeks, “take three Covid‐19 tests as they return to the classroom” and that “After the initial programme of three tests in school or college, students will be provided with 2 rapid tests to use each week at home” (bit.ly/3aPTpn7). The tests to be used are lateral flow tests (LFTs), which can be self‐administered, with results readable within 30 minutes.

5 March

The Royal Statistical Society (RSS) Covid‐19 Task Force issued a statement of alerts about the proposed use of LFTs for voluntary asymptomatic screening in schools (bit.ly/3dftoPE). “[W]hile the usual concern with LFTs is false negatives, when infection‐prevalence is low there is also a risk that the majority of ‘positive’ tests could be false positives”, the statement said. An accompanying illustration showed that if there were 2 asymptomatic infections per 1,000 persons tested, and the LFT's sensitivity (the percentage of those with asymptomatic infection who actually test positive) was 40% and specificity (the percentage of those uninfected who actually test negative) was 99.8%, a high proportion of LFT positives would be expected to be false positives. (The 40% sensitivity figure came from an earlier evaluation of mass asymptomatic screening in Liverpool (bit.ly/2QoIYiP).)

Government guidance stated that if a student obtained a positive LFT result through home testing, they would be required to confirm the result by a polymerase chain reaction (PCR) test. Unbeknown to those of us on the Task Force, in January 2021, when the prevalence of Covid infection was four times higher than in early March, Public Health England (PHE) had decided to dispense with the need for confirmatory PCR tests for LFT‐positive results, including those obtained via in‐school tests.

6 March

I had agreed to be interviewed about the Task Force statement on BBC Radio 4's flagship morning news programme, Today. The presenter, Nick Robinson, interviewed a father whose son had already had his first LFT and had tested positive. Wisely, the school advised the family, who were self‐isolating with their son, to obtain a PCR test. The child was PCR negative. The father then contacted the NHS Test and Trace service to ask that the family's period of self‐isolation be suspended, and to rescind any tracking of close contacts that had been initiated as a result of the false‐positive LFT result. The father was told that his son would not be allowed back to school and that the entire family must serve out their full 10 days of self‐isolation. Robinson turned to me to ask what I thought. “I am shocked!”, I said.

7 March

During an interview on the BBC's Andrew Marr Show, Dr Susan Hopkins, deputy director of PHE, was asked why secondary pupils were being denied confirmatory PCR tests for LFT‐positive in‐school tests. Her defence was that “the validation of those [LFT] tests in real‐life scenarios suggests that it's 99.9% – at least – specific, which means that the risk of false positives is extremely low – less than one in a thousand – and we would expect that that would be the same risk with PCR tests”. This argument, however, failed to contradict the RSS Task Force's contention that a high proportion of LFT positives in 2 million LFT‐screened secondary pupils in mid‐March would be false positives due to low infection prevalence.

8 March

The Today programme interviewed Vicky Ford, the Minister for Children, who made matters worse by suggesting that LFT‐positive in‐home tests would also be denied PCR adjudication – which would be in defiance of the conditions on pupils' home use of LFTs, as set out by the Medicines and Healthcare products Regulatory Agency (MHRA). As it was a lovely morning, I had been out walking and missed the interview, to which my stepson, Dr Tom G. Bird, alerted me. I and others swung into action so that, by around 3 pm, the Prime Minister's office had issued a prompt correction to Ford's misleading statement.

9 March

England's chief medical officer, Professor Chris Whitty, appeared before the House of Commons Select Committee on Science and Technology. He was asked by the committee chairman to explain the confusion over confirmatory PCR tests, as the chairman and other Members of Parliament were receiving letters on the matter from their constituents. Indeed, not only Members of Parliament but also RSS fellows had received emails from families who intended to withdraw permission for their children to take part in asymptomatic LFT screening for fear that false positives would see them isolated unnecessarily if a confirmatory PCR‐negative test was to be ignored.

Whitty was asked: “If a child tests positive at school with a lateral flow test and then subsequently tests negative with a PCR test, does the PCR test trump the lateral flow test? Can they continue their education?”

Whitty avoided giving an answer lest he “add to the confusion” as the matter had been “hammered out” between the DfE and PHE, he said (bit.ly/3g3GQIg). The Science and Technology Committee has persisted in seeking clarification, including on the basis (in legislation or by regulations) that a negative PCR test result does not overrule positive LFT results for pupils in England, even though it does in Scotland and does for road hauliers in England.

12 March

Members of the RSS Covid‐19 Task Force and Diagnostic Testing Working Group wrote to the MHRA to alert it to the “potential harm” that might result because “PCR‐adjudication is absent from the current asymptomatic screening by use of lateral flow tests in secondary schools” (bit.ly/3dWtGtX).

Our letter explained: “In secondary schools, the implications of LFT‐false positive extend to the child's or teacher's family and their school bubble; and to the households of the children in the bubble who need to take time out to look after their children who are suddenly back at home.” The number of people affected by one false‐positive LFT result “could easily extend to 100”, said the letter. It also warned that “the 10‐day quarantine imposed on pupils and families because of LFT‐false positives during 8–19 March will impact on children's schooling for the next two weeks; and on families' willingness for their children to participate in twice‐weekly at‐home‐screening after 19 March 2021.”

13–14 March

By the end of the first week of in‐school testing, several newspapers were reporting that, in their patch, an entire school year of 100–200 pupils had been sent home on account of one or two positive LFT results at pupils' second screening.

17 March

I wrote to Sir David Norgrove, chairman of the UK Statistics Authority, to suggest that England's secondary pupils, parents and head teachers, let alone the public, had a right to openness about the background data which informed the planning assumptions made by the DfE and PHE about both the consent rate for, and performance of, asymptomatic LFT screening of young people.

For example, in January, when the prevalence of Covid infection was high and schools were ordered to close, the children of key workers were still able to attend secondary school and some of these children were screened for infection using a particular type of LFT (Innova). At that time, up to 27 January, confirmatory PCR tests were offered to students who tested LFT positive. Hence, NHS Test and Trace must hold data both on pupils' consent rate for LFT screening and on the proportion of LFT‐positive pupils who subsequently tested PCR negative. (Belatedly, on 10 March, PHE had released new estimates for the Innova LFT's specificity: 99.9%, it said, but “could be as high as 99.97%” (bit.ly/2RrJGwh).)

I also urged that the public deserved an explanation for why confirmatory PCR testing was now denied to those who tested positive in school.

Moreover, since the NHS Test and Trace service had failed to publish key data on LFT screening either in secondary schools in January or from universities in December 2020, data collection should have been specifically designed to ensure rapid reporting on the uptake of and results from the first round of LFT screening of secondary school students on their return in March. Uptake and results needed to be reported separately for the second and third LFT in the planned trio of tests because, by then, pupils had attended class and any LFT positive could lead to self‐isolation by an entire class, not just those testing positive.

I asked the UK Statistics Authority to encourage official statisticians to consider that efficient data acquisition remains a key part of statistical science and to make their voice heard in how performance monitoring of infection control and other policies is designed. Late evidence is wasted evidence.

18 March

Weekly statistics for the NHS Test and Trace service were published, covering 4–10 March – which included the first few days of back‐to‐school tests (bit.ly/3deaM2v). For this period, the observed count of positive LFT tests per 2 million secondary pupils was 960. This was somewhat of a surprise, being much lower than expected. If we assume that the prevalence of asymptomatic Covid infection was 2 per 1,000 in early March, we would expect 4,000 asymptomatic infections among 2 million secondary pupils being screened, and 1,600 true positives from Innova LFTs (with sensitivity at 40%, as it was in the Liverpool mass testing). And if the specificity of LFTs was 99.97% (as claimed in the PHE preprint on 10 March, based mainly on adults), the remaining 1,996,000 uninfected secondary pupils would be expected to yield 599 false positives – and so 2,199 positive LFTs overall (Figure 1(a)).

Figure 1

(a) Expected frequency tree when asymptomatic Covid‐19 prevalence is 2 per 1,000. (b) Expected frequency tree when asymptomatic Covid prevalence is halved to 1 per 1,000. (c) Expected frequency tree when asymptomatic Covid prevalence is halved again, to 1 per 2,000.

Even if we halve the assumed prevalence of asymptomatic infection to only 1 per 1,000, and if specificity is accepted as 99.97%, we would still expect 1,399 positive LFTs per 2 million secondary pupils (Figure 1(b)). Hence, either the prevalence of asymptomatic infection was less than 1 per 1,000 between 4 and 10 March (see Figure 1(c)), or the sensitivity of LFTs for asymptomatic secondary pupils is substantially lower than the 40% observed in Liverpool, because too few LFT positives were observed overall.

Regrettably, information on PCR confirmations that were in some way linkable to the observed LFT positives was not disclosed by the time that secondary schools had completed their required three tests in two weeks for all students, and so we do not know how many of those LFT positives were PCR positive versus negative. The RSS Covid‐19 Task Force had anticipated that over half the PCR adjudications could be negative.

23 March

I submitted this account of the debate over LFT screening in secondary schools and the lack of confirmatory PCR testing to Significance, fearing that the battle was lost, but hopeful that we might learn from the experience. I concluded by saying: “There must be no repetition of the infection control fiasco visited upon England's secondary schools, which has been a statistical cause célèbre.”

29 March

An email arrived in my inbox with a link to a press release from the Department of Health and Social Care (DHSC). “Government reintroduces confirmatory PCR testing for assisted testing”, read the headline (bit.ly/2QAQgkk). This was welcome news. The press release stated that: “Recent analysis by NHS Test and Trace shows lateral flow tests (LFD) have a specificity of at least 99.9%. This means that for every 1,000 lateral flow tests carried out, there is fewer than one false positive result. Despite this, at times of low prevalence, the probability of a false positive from an LFD is higher, so we are mitigating this by asking people to confirm a positive LFD result with a PCR test.”

Perhaps most importantly, the announcement went on to say that people whose PCR test is negative will be allowed to stop isolating, along with their families and contacts, provided that the PCR test is taken within 2 days of a positive lateral flow test.

The battle was won after all.

29 April

Another hard‐won victory! After a six‐week wait, the DHSC finally published NHS Test and Trace statistics matching LFT results for secondary pupils to results from confirmatory PCR tests (see Table 1). Fewer than half of the positive in‐school LFTs received PCR adjudication – which is unsurprising given that confirmatory PCR tests were discouraged and (at that time) would be ignored anyway. But, of those that did seek PCR adjudication in the first week of in‐school tests (4–10 March), 38% of 571 PCR‐adjudicated LFT positives were PCR‐positive (95% CI: 34 to 42%), while in the second week (11–17 March) 45% of 462 PCR‐adjudicated LFT positives were PCR‐positive (95% CI: 41 to 50%).

Table 1

Summary of NHS Test and Trace statistics published by the Department of Health and Social Care on 29 April, reporting numbers of LFTs, positive LFTs, and matched PCR adjudications for in‐school positive LFTs among secondary school students, 4–17 March. Source: bit.ly/3eAhm42.

NHS Test and Trace reporting week	4–10 March monitored (week 1)	11–17 March monitored (week 2)
Pupils in secondary schools (excluding colleges and 16–19 schools): PCR adjudication of monitored LFT positives
Non‐void LFTs	2,770,267	3,788,938
Positive LFTs	1,331	1,766
Positives per million LFTs	480	466
Assisted/monitored subset of in‐school LFTs
Assisted/monitored positive LFTs	1,215	1,089
Matched PCR adjudications (% adjudicated)	581 (48%)	469 (43%)
Non‐void matched PCR adjudications	571	462
PCR positive on adjudication	219	209
% PCR positive on adjudication (95% CI)	38% (34% to 42%)	45% (41% to 50%)

Thus, our concerns about false positives were not misplaced – though we do not yet know how many days of school attendance were needlessly missed. My estimate is that there were nearly 400 unwarranted quarantine years as a result of second or third LFTs being positive in the second week of in‐school screening when PCR adjudications were discouraged and ignored.

Disclosure statement

The author served on the RSS Working Party on “Official Statistics: Counting with Confidence”, chaired the RSS Working Party on Performance Monitoring in the Public Services, is a member of the RSS Working Party on Diagnostic Tests, and chairs the RSS/DHSC Panel on Test and Trace. Since mid‐January 2021, the author has served on PHE's Testing Initiatives Evaluation Board.