A clinical case definition of post-COVID-19 condition by a Delphi consensus.

People with COVID-19 might have sustained postinfection sequelae. Known by a variety of names, including long COVID or long-haul COVID, and listed in the ICD-10 classification as post-COVID-19 condition since September, 2020, this occurrence is variable in its expression and its impact. The absence of a globally standardised and agreed-upon definition hampers progress in characterisation of its epidemiology and the development of candidate treatments. In a WHO-led Delphi process, we engaged with an international panel of 265 patients, clinicians, researchers, and WHO staff to develop a consensus definition for this condition. 14 domains and 45 items were evaluated in two rounds of the Delphi process to create a final consensus definition for adults: post-COVID-19 condition occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset, with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis. Common symptoms include, but are not limited to, fatigue, shortness of breath, and cognitive dysfunction, and generally have an impact on everyday functioning. Symptoms might be new onset following initial recovery from an acute COVID-19 episode or persist from the initial illness. Symptoms might also fluctuate or relapse over time. A separate definition might be applicable for children. Although the consensus definition is likely to change as knowledge increases, this common framework provides a foundation for ongoing and future studies of epidemiology, risk factors, clinical characteristics, and therapy.

Introduction

As of Dec 3, 2021, more than 263 million confirmed cases of COVID-19 and more than 5·2 million deaths have been reported to WHO, although estimates of 2020 greatly surpass these figures. However, the natural history, clinical course, and long-term consequences of this new disease are still not completely understood.

Most patients with COVID-19 return to their baseline state of health after acute infection with SARS-CoV-2, but a proportion report ongoing health problems. The number of people affected with late sequelae after the acute COVID-19 episode remains unknown. Persistent symptoms are reported to be more prevalent in women, and risk of persistent symptoms is reported to be linearly related to age.3, 4 These effects appear to occur irrespective of the initial severity of infection, and are often linked to multiple organ systems. One study found that up to 70% of individuals at low risk of mortality from COVID-19 have impairment in one or more organs (ie, heart, lungs, kidneys, liver, pancreas, or spleen) 4 months after initial COVID-19 symptoms.

In September, 2020, and in response to requests from Member States, the WHO Classification and Terminologies unit created International Classification of Diseases 10 (ICD-10) and ICD-11 codes for post-COVID-19 condition. Over the course of the pandemic, several definitions of post-COVID-19 condition have been proposed, including long COVID or long-haul COVID (appendix p 3). Absence of both a single terminology and a clinical case definition have been repeatedly signalled as drawbacks to advance on epidemiological reporting, research, policy making, and clinical management of affected patients. Standardisation of nomenclature and clinical case definition is required to facilitate global discussion and streamline research methods, management strategies, and policies. The objective of this Review is to establish the domains and variables for inclusion into a standardised clinical case definition for post-COVID-19 condition.

Methods

Study design and participants

This Review is a prospective, Delphi consensus-seeking exercise and mixed, iterative survey of internal and external experts, patients, and other stakeholders (the research protocol is available as a preprint). The Delphi method is a structured communication technique originally developed as a systematic, interactive, forecasting method that relies on a panel of experts.8, 9 It has been widely used for research and has certain advantages over other structured forecasting approaches.10, 11

The primary users of the clinical case definition for the post-COVID-19 condition will include patients, relatives and caregivers, clinicians, researchers, advocacy groups, policy makers, health and disability insurance providers, and media. We therefore aimed to have a diverse representation of participants, including clinicians with expertise in a variety of disciplines such as quality improvement and research, patients who have had COVID-19 and its mid-term and longer-term effects, researchers, policy makers, and others from countries representing all WHO regions and World Bank income levels. There were no specific exclusion criteria for participants. A statement explaining implied consent was on the title page of the survey, with consent to participate in the survey implied by answering and returning the surveys. Participants could withdraw at any time.

Study procedures

Participants were identified from internal WHO stakeholder lists and through active engagement with WHO networks. Participant list sources included clinician and patient researchers who had attended a previous WHO global webinar on post-COVID-19 condition, members of the WHO COVID-19 Clinical Characterization and Management research working group, members of the WHO COVID-19 Clinical Network, members of the LongCovidSOS patient group, and clinicians and patients nominated by WHO officers.

Participants were invited via an online recruitment letter soliciting participation and engagement, along with an explanation of study objectives, instructions, and outputs. The survey contained listed options regarding domains and variables to consider in the definition, which were initially kept as broad and comprehensive as possible. The domains and variables were followed by a series of questions relating to these variables with eventual values or thresholds related to each. Survey responses were anonymous and tabulated by groups only. Registration of panelists and the actual Delphi questionnaire (appendix pp 4–37) were accessible on the DelphiManager website.

All questions were evaluated on a nine-point Likert scale, from 1 (least important) to 9 (most important) and participants were asked to choose the level of importance for each variable in the definition. Whenever there was a value in the DelphiManager rating column that was something other than the Likert scale values of 1–9, the system coded as −9, the value allocated when an outcome had not been rated; or 10, the value allocated to the “Unable to rate” option.

The first round of the Delphi exercise lasted 14 days, and participants were sent two reminders to complete the online survey. Participants had the opportunity to add comments for each item and to add new domains or variables. After review of the data from the first round, the WHO working group created a second round questionnaire, removing items that received low scores, clarifying terms and uncertainties in wording, and adding a new domain suggested by participants. The second round was done 5 weeks later, used a modified questionnaire based on iterative feedback and consensus during round one, and lasted 8 days, again with two reminders. In round two, participants from round one were provided with the number and percentage of respondents having chosen that answer, and a reminder of their individual answer in round one. In round two, participants had the opportunity to add comments for each item.

Statistical plan

The optimal sample size for a Delphi exercise is not well defined; however, there is general acceptance that 12–18 participants should be included in each group. We sought views from a diverse sample of stakeholder groups and WHO regions (African region, region of the Americas, Eastern Mediterranean region, European region, South-East Asia region, and Western Pacific region). We defined five stakeholder groups: patients, patient-researchers, external experts, WHO staff, and others (advocacy groups, policy makers, health and disability insurance, and media). Considering that participants could be included in more than one category and that response rates might vary by group, we sent out participation invitations to a broad group of individuals identified through WHO resources. Although we sought a minimum number of participants per group, no maximum was established.

Primary and secondary endpoints

The primary objective was to achieve consensus on the importance of the variables and values included in the definition. Consensus was achieved on a question if 70% or more of the responses fell within 7–9 on the nine-point Likert scale (appendix p 2). Disagreement was considered to occur if 35% or more of the responses fell within both of the two extreme ranges of possible options on the Likert scale (1–3 and 7–9). All other combinations of panel answers were considered as partial agreement. For each question, consensus proportions were considered on the basis of the number or percentage of respondents (excluding the category “Not my area of expertise”). Therefore, the denominator for the consensus included only participants with knowledge and expertise for that specific question. Participant responses, including baseline and demographic characteristics, were analysed with basic statistics such as mean (SD), median (IQR), and range. Responses on all other domains were analysed in proportions and illustrated with histograms.

Results

On May 11, 2021, 747 invitations were sent by email. There were 265 respondents in round one (demographics in table ), with 241 complete responses and 24 incomplete responses by May 24, 2021. In round two there were 195 respondents, with 178 complete responses and 17 incomplete responses by June 21, 2021 (figure 1 ). In round one, there were 61 (23%) patients, 18 (7%) patient-researchers, 138 (52%) external experts, 33 (12%) WHO staff, and 15 (6%) were in the “Other” category. There were 115 (43%) female participants, 147 (55%) male participants, one (<1%) non-binary participant, and two (1%) who preferred not to say in round one, with ages ranging from 20 years to 90 years or older, but most participants were in their 40s. Responses were received from participants in countries representing all WHO regions and World Bank income groups (figure 2 ). The subset of participants in round two did not differ from round one (table).

Table

Demographic characteristics of participants

	Round one (N=265)	Round two (N=195)
Stakeholder group
Patient	61 (23%)	47 (24%)
Patient-researcher	18 (7%)	13 (7%)
External experts	138 (52%)	103 (53%)
WHO staff	33 (12%)	22 (11%)
Other*	15 (6%)	10 (5%)
Gender
Woman	115 (43%)	86 (44%)
Man	147 (55%)	107 (55%)
Non-binary	1 (<1%)	0
Prefer not to say	2 (1%)	2 (1%)
Age, years
20–29	16 (6%)	11 (6%)
30–39	53 (20%)	42 (22%)
40–49	86 (32%)	63 (32%)
50–59	73 (28%)	52 (27%)
60–69	32 (12%)	22 (11%)
70–79	4 (2%)	4 (2%)
90 or older	1 (<1%)	1 (1%)
WHO region
African	9 (3%)	8 (4%)
Americas	53 (20%)	36 (18%)
Eastern Mediterranean	7 (3%)	4 (2%)
European	94 (35%)	70 (36%)
South-East Asia	10 (4%)	8 (4%)
Western Pacific	19 (7%)	18 (9%)
Country not specified	73 (28%)	51 (26%)
World Bank income group
High income	140 (53%)	110 (56%)
Upper middle income	37 (14%)	22 (11%)
Lower middle income	13 (5%)	10 (5%)
Low income	2 (1%)	2 (1%)
Country not specified	73 (28%)	51 (26%)

Data are n (%).

Included journalists and policy makers.

Figure 1

STROBE flowchart of participation in the two Delphi rounds

Figure 2

Distribution of participants worldwide

Distribution of participants in May and June, 2021.

After the first round of responses, three of the 14 domains were excluded and a new domain added on the basis of participant comments; thus there were a total of 12 domains in round two (figure 3 ). Similarly, on the basis of the comments and after further discussion of the working group on results that reached borderline significance based on predefined thresholds, the domains were expanded with variables, such as new symptoms, for 45 items (appendix pp 38–42). During subsequent revision, two domains that did not fully reach prespecified thresholds were included in the clinical case definition after panel discussion—namely, “a minimum time period (in months) from the onset of COVID-19 to the presence of symptoms” and “duration of symptoms”. Similarly, the “new onset” nature of symptoms was expanded to incorporate relapsing and fluctuating through patient–panel feedback. A clinical case definition was built and further expanded with those domains, thresholds, and values, and wording was trimmed in a dedicated quantitative and qualitative discussion with patients and patient-researchers (panel ). Post-COVID-19 condition occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset of COVID-19 and with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis. Common symptoms include fatigue, shortness of breath, and cognitive dysfunction, but also others (appendix pp 38–42), and generally have an impact on everyday functioning. Symptoms might be new onset after initial recovery from an acute COVID-19 episode or persist from the initial illness. Symptoms might also fluctuate or relapse over time. A separate definition might be applicable for children.

Figure 3

Domains that achieved consensus by participants in each Delphi stage

Post-COVID-19 condition occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset of COVID-19 with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis. Common symptoms include fatigue, shortness of breath, and cognitive dysfunction (other symptoms are listed in the appendix [p 4] and published literature), and generally have an impact on everyday functioning. Symptoms might be new onset after initial recovery from an acute COVID-19 episode or persist from the initial illness. Symptoms might also fluctuate or relapse over time.

A separate definition might be applicable for children.

Notes

There is no minimum number of symptoms required for the diagnosis; symptoms involving different organs systems and clusters have been described.

Definitions

Fluctuate: a change from time to time in quantity or quality

Relapse: return of disease manifestations after period of improvement

Cluster: two or more symptoms that are related to each other and that occur together; they are composed of a stable groups of symptoms, are relatively independent of other clusters, and might reveal specific underlying dimensions of symptoms

Discussion

By means of a Delphi process involving two rounds, we identified domains and variables to be included in a clinical case definition of post-COVID-19 condition—the name proposed by WHO ICD-10 diagnosis code U09 in September, 2020. A definition based on these domains was created.

This clinical case definition of post-COVID-19 condition is intended to be applied in the community and health-care setting to optimise recognition and care of individuals with post-COVID-19 condition; it is not meant to replace other terms but be used in a more standardised way. This definition was obtained by a robust, protocol-based methodology (Delphi consensus), engaging a diverse group of representative patients, caretakers, and other stakeholders from many areas. This definition is compatible and consistent with previous suggestions available elsewhere (appendix p 3), but is likely to change as new evidence emerges and our understanding of the consequences of COVID-19 continues to evolve. To date, there have been several attempts to define different COVID-19-related topics and outcomes,15, 16, 17, 18 but existing definitions do not take into account presentations in low and middle-income countries and often miss domains that are relevant to various groups of stakeholders. To our knowledge, ours is the first Delphi exercise to define post-COVID-19 condition with these stakeholder inputs.

Strengths of this study include a robust protocol-based Delphi method and inclusiveness and representation of participants from five diverse stakeholder groups, from countries representing all WHO regions and World Bank income groups. We aimed to surpass current controversies on the naming of the condition (eg, chronic COVID-19 syndrome, late sequelae of COVID-19, long COVID, long haul COVID, long-term COVID-19, post COVID syndrome, post-acute COVID-19, and post-acute sequelae of SARS-CoV-2 infection [or PASC]) by using the WHO terminology of post-COVID-19 condition. We acknowledge patients' activism done under the different names for this condition.19, 20

Our study has several limitations. English language was selected for practicality issues, but subsequent Delphi exercises should include other languages. Response rates in both rounds could have been greater, which is not unexpected, given its conduct during a pandemic. Best practices for enhancing response rates were integrated throughout, including introductory messages and reminder emails. Responses from the African and Eastern Mediterranean regions were especially sought and obtained, but their overall proportions were lower than from other regions. Additionally, participants older than 70 years are under-represented. Wording of some domains and addition of new items and values were modified from round one to round two. Our protocol called for only two rounds of consensus-building. Items addressing the timing and duration of symptoms did not reach prespecified criteria for consensus, and should be interpreted in the light of this limitation. Overall, as several pathophysiological mechanisms are in place and interplay during and after acute infection, and different trajectories for recovery after COVID-19 exist, producing a single, universal definition that might work well for clinical, research, policy and advocacy grounds, and for all care levels and severities might be overly ambitious. The definition presented in this Review (panel) might be considered a description based on opinion of those individuals participating, and difficult to operationalise in practice. Not only timing and duration, but the symptoms are susceptible to subjectivity and bias of participants. We strongly support that this is open discussion in an organised way and integration of emerging evidence, such as prospective cohort trials, should help to advance this field.

As mentioned, this proposal of a clinical case definition is probably temporary, as new data continue to emerge. Initial reports describing post-COVID-19 condition were from small patient samples with an inherently short follow-up, and are likely to be subject to bias, which will be unravelled in ongoing meta-analyses. New research is exploring the use of electronic health records from representative samples of patients identified in primary care and elsewhere. The use of comparator samples of individuals fully recovered after acute infection is envisaged. By use of cluster analysis and other mathematical tools to establish specific symptoms and their minimum number, they all could be formally identified and eventually clustered for different phenotypes. Notably, time thresholds from onset of infection or the duration of these symptoms possibly will be established.27, 28

Conclusion

COVID-19 will remain a challenge for the foreseeable future. Many pending answers surrounding COVID-19 and its sequelae remain, with new questions constantly being formulated.30, 31, 32 This global definition of post-COVID-19 condition will help to advance both advocacy and research, but will probably change as new evidence emerges and our understanding of the consequences of COVID-19 continues to evolve.

Search strategy and selection criteria

References for this Review were identified through searches of PubMed for articles published from Jan 1, 2020, to Aug 30, 2021, with the terms “COVID-19”, “post-COVID-19 condition”, “chronic COVID-19 syndrome”, “late sequelae of COVID-19”, “long COVID”, “long haul COVID”, “long-term COVID-19”, “post COVID syndrome”, “post-acute COVID-19”, and “post-acute sequelae of SARSCoV-2 infection (PASC)”. Cross-referencing of relevant published articles were also identified through searches in the authors' personal files and several WHO Expert Panels. Articles resulting from these searches and relevant references cited in those articles were reviewed. Articles published in English, but also abstracts in non-English languages, were included.

For more on the WHO coronavirus (COVID-19) dashboard see https://covid19.who.int/

Declaration of interests

We declare no competing interests.