Challenges in the diagnosis of paediatric pneumonia in intervention field trials: recommendations from a pneumonia field trial working group.

Pneumonia is a leading killer of children younger than 5 years despite high vaccination coverage, improved nutrition, and widespread implementation of the Integrated Management of Childhood Illnesses algorithm. Assessing the effect of interventions on childhood pneumonia is challenging because the choice of case definition and surveillance approach can affect the identification of pneumonia substantially. In anticipation of an intervention trial aimed to reduce childhood pneumonia by lowering household air pollution, we created a working group to provide recommendations regarding study design and implementation. We suggest to, first, select a standard case definition that combines acute (≤14 days) respiratory symptoms and signs and general danger signs with ancillary tests (such as chest imaging and pulse oximetry) to improve pneumonia identification; second, to prioritise active hospital-based pneumonia surveillance over passive case finding or home-based surveillance to reduce the risk of non-differential misclassification of pneumonia and, as a result, a reduced effect size in a randomised trial; and, lastly, to consider longitudinal follow-up of children younger than 1 year, as this age group has the highest incidence of severe pneumonia.

Introduction

Pneumonia, the most severe manifestation of acute lower respiratory infection, is the leading cause of death in children younger than 5 years outside of the neonatal period, with several well recognised risk factors (table 1 ).

Table 1

Established risk factors for pneumonia in children

	Prevalence	Mechanisms
Not exclusively breastfeeding (children aged 0–5 months)	61% of children aged 0–5 months globally3	Suboptimal maternal antibody transmission; suboptimal nutrition
Underweight (weight-for-age <–2 SD)	20·2% of children <5 years in low-income and middle-income countries.4	Poorly characterised immune deficiency
Stunting (height-for-age <–2 SD)	32·0% of children <5 years in low-income and middle-income countries4	Poorly characterised immune deficiency
Severe wasting (weight-for-length <–3 SD)	3·5% of children <5 years in low-income and middle-income countries4	Poorly characterised immune deficiency
Zinc deficiency	7·5–30% globally, all ages5	Impairs various immune functions, including the integrity of respiratory cells during lung inflammation or injury
Exposure to household air pollution	Approximately 40% worldwide6	Impairment of respiratory tract defence mechanisms, local oxidative stress, and inflammation
Non-vaccination	Haemophilus influenzae type b (30%); measles (2 doses) [36%]; Pneumococcus (3 doses of a conjugate vaccine) pertussis (as DTP3)[14%].7	Disease-specific immunity

SD= standard deviation. DTP3= Diphtheria-tetanus-pertussis vaccine.

Improvements in socioeconomic status, child nutrition, HIV control, and the availability of conjugate vaccinations for Streptococcus pneumoniae and Haemophilus influenzae have reduced pneumonia incidence; however, a substantial burden of disease still remains due to other common and preventable risk factors. For example, household air pollution is an important risk factor for acute lower respiratory infections in children (with a population attributable fraction of 52%) and accounts for 39·1 million disability-adjusted life years lost and 455 000 deaths in 2014. Nonetheless, intervention trials have struggled to show an association between a reduction in exposure to household air pollution and decreased pneumonia incidence.10, 11

Important challenges exist in assessing pneumonia in field settings. A Comment in the Lancet Global Health recognises challenges in the implementation of WHO guidelines for the management of childhood pneumonia. In intervention trials, pneumonia case definitions with poor diagnostic accuracy can lead to an underestimation of the effect of interventions on pneumonia. The choice of a passive or active surveillance approach and the frequency of surveillance visits can lead to missed cases or skew case detection towards milder episodes.

This Review summarises the discussions between investigators from the ongoing Household Air Pollution Intervention Network (HAPIN) trial (NCT02944682) and external experts. The evidence we present helped to inform the case definition and surveillance approach in the HAPIN trial.

Epidemiology and burden of disease

Burden

Annually, pneumonia causes approximately 700 000 to 900 000 childhood deaths worldwide.15, 16 In 2016, pneumonia was responsible for 13–16% of all deaths in children younger than 5 years.15, 16 The worldwide burden of pneumonia mortality is concentrated primarily in a few countries: Afghanistan, Angola, Bangladesh, Chad, China, Democratic Republic of the Congo, Ethiopia, India, Indonesia, Niger, Nigeria, Pakistan, Somalia, Sudan, and Tanzania. These 15 countries accounted for 70% of all pneumonia deaths worldwide in 2015. One review paper estimated that, in 2011, 1·3 million cases of pneumonia were fatal, and that 81% of these deaths occurred in the first 24 months of life. Childhood mortality attributed to pneumonia decreases rapidly with age, from approximately 67% of all deaths at 6 months to 14% at 18 months, and reaches a plateau of 6% between 30 and 54 months of age. Incidence decreases more gradually with age: approximately 39% observed at 6 months, 22% at 18 months, 19% at 30 months, 13% at 42 months, and 7% at 54 months. As a result, pneumonia outcome studies might find cases with higher frequency and greater severity by focusing on the first year of life.

Patterns of incidence and severity of pneumonia have also changed over time, with large reductions observed since the early 2000s. Absolute mortality due to acute lower respiratory infections in children aged younger than 5 years has decreased by 37% from 2005 to 2015, whereas incidence has declined more slowly. Possible explanations for these trends include a shift toward a higher proportion of non-severe cases, better access-to-care, and improved case management.

Aetiology and vaccine coverage

In 2015, the global proportion of pneumonia deaths avoided if exposure to pathogens were to be eliminated was 55·8% for S pneumoniae, 8·3% for H influenzae, 5·2% for respiratory syncytial virus, and 1·4% for influenza. Pneumococcal conjugate vaccines have been effective in reducing pneumococcal disease both directly, by protecting individual children, and indirectly, by preventing transmission of disease-causing serotypes to susceptible unimmunised people.2, 17 Vaccines have targeted the most high-burden pathogens but some causes of pneumonia are not preventable by use of vaccines. Pathogens not preventable by vaccines include non-vaccine-type S pneumoniae, Staphylococcus aureus, Salmonella, Klebsiella pneumoniae, Mycoplasma pneumoniae, Chlamydophila pneumoniae, Mycobacterium tuberculosis, and viruses other than influenza, such as respiratory syncytial virus, parainfluenza viruses 1–3, human metapneumovirus, adenovirus, coronavirus, and bocavirus.

Vaccine coverage within a community will influence pneumonia prevalence, incidence, severity, and causes. Regions with higher vaccine coverage might experience lower overall incidence of disease and fewer cases of severe pneumonia.18, 19, 20 Furthermore, pneumococcal conjugate vaccine immunisation could also result in decreased mortality related to viruses such as influenza or respiratory syncytial virus, given the high risk of bacterial coinfection. As vaccine coverage increases, vaccine preventable burden declines and other organisms not included in the vaccines become more important. For example, greater coverage with Haemophilus influenzae type b vaccination relative to pneumococcal conjugate vaccines could be associated with proportionally higher population attributable fractions due to pneumococcal pneumonia.

Pneumonia case definition

The definition and classification of pneumonia, both severe and non-severe, is a well-recognised challenge. No existing classification systems is recognised as the gold standard. Moreover, many of the existing classifications for pneumonia favour empirical treatment with antimicrobials. Specifically, the use of different case definitions between studies results in varying patterns of false positives and negatives (ie, varying degrees of misclassification across studies). Misclassification of some pneumonia cases will always occur because no case detection strategy is perfect, but some strategies will lead to different misclassification errors than others.

Non-differential misclassification of pneumonia will happen when a case definition or diagnostic approach is similarly applied to both study groups in an intervention trial. For example, a pneumonia case definition that does not have appropriate sensitivity or specificity might misclassify pneumonia cases but typically this misclassification occurs to a similar degree in both the intervention and control groups. This type of error generally biases results toward the null hypothesis.

By contrast, differential misclassification of pneumonia occurs when either the case definition or diagnostic approach differs between study groups. For example, differential misclassification of pneumonia can occur when participants in one group are more likely to present for medical attention than patients in the other group, leading to a greater number of cases diagnosed in the first group. In pneumonia field trials in which the intervention is not masked, this type of error might be more common with passive case-detection approaches that rely on the participant or family to notify study staff of any pneumonia symptoms. For instance, in a household air pollution intervention trial in which the intervention households have a gas stove and receive regular fuel tank deliveries (whereas the control households do not), study personnel interact with the intervention households more frequently. This type of error can also occur in studies with active case-detection in which research staff might selectively recognise cases more frequently in the control households than in intervention households, or vice versa. Differential misclassification of the outcome can alter odds ratio results, but the magnitude and direction of the alteration depends on a variety of factors and can be unpredictable.22, 23

Variations between case definitions can also make results less comparable between studies. Currently, no single definition of pneumonia exists that is universally accepted. Pneumonia can present with symptoms that are similar to other childhood illnesses, such as malaria in endemic regions, severe anaemia, and bacteraemia.24, 25, 26, 27, 28 The use of standardised diagnostic algorithms can help to address heterogeneity. For example, the WHO pneumonia case definition is one of the most commonly used in low-income and middle-income settings outside of hospitals. Intended for health workers to avoid missing pneumonia cases and provide appropriate treatment quickly, the definition used by WHO has been adapted since its inception in the 1980s and is now part of the integrated management of childhood illnesses (IMCI) guidelines. Updated in 2014, the revised WHO classification divides pneumonia in children aged between 2 and 59 months into two categories: first, pneumonia defined as cough or difficulty breathing with tachypnoea or lower chest wall indrawing and, second, severe pneumonia defined as pneumonia plus any general danger sign, or either cough or difficulty breathing plus any general danger sign. The IMCI guidelines recommend that treatment should include outpatient therapy with oral amoxicillin for pneumonia, and referral and parenteral antibiotics for severe pneumonia.

The case definition used by WHO has been lauded as straightforward, generalisable, and easy to implement in resource-poor settings.21, 31 The high-sensitivity approach was adopted to substantially reduce childhood mortality by capturing the majority of pneumonia cases and providing rapid treatment. Indeed, this approach has been proven effective: a meta-analysis of nine community trials found that implementation of WHO case management led to reductions in pneumonia mortality of 42% (95% CI 22–57) in children aged less than one month, 36% (20–48) in children aged less than one year, and 36% (20–49) in children aged between 0–4 years of age. Such high sensitivity comes at the cost of specificity, failing to distinguish between bacterial and viral causes, and capturing a range of diseases other than pneumonia as false positives. In general, case definitions with low specificity can lead to a non-differential misclassification of outcomes and, as a result, a reduced effect size in a randomised trial.

Both the RESPIRE trial in Guatemala and the cooking and pneumonia study in Malawi show how a pneumonia case definition might fail to convey the true efficacy of a particular intervention (ie, a reduction in the amount of household air pollution).10, 11 The RESPIRE study analysed the effect of a reduction of household air pollution on pneumonia incidence in children aged between 0–18 months according to a variety of definitions, and obtained varying results. For instance, there was no significant reduction in either fieldworker-assessed WHO-defined pneumonia (rate ratio [RR]=0·91, 95% CI 0·74–1·13, p=0·39) or physician-diagnosed pneumonia based on history and clinical examination (RR=0·78, 0·59–1·06, p=0·10) in children in the intervention group compared with controls. Significant reductions were, however, documented in both fieldworker-assessed WHO-defined severe pneumonia (RR=0·56, 0·32–0·97, p=0·04) and physician-diagnosed severe pneumonia, which was defined as pneumonia with hypoxaemia (RR=0·67, 0·45–0·98, p=0·04). Severe pneumonia was a more specific outcome, especially physician-diagnosed severe pneumonia, which included pulse oximetry, an objective assessment strategy. However, physician-diagnosed pneumonia in any study is difficult to standardise and replicate.

In the cooking and pneumonia study, the primary outcome was non-severe pneumonia (according to WHO criteria) in children less than 5 years, which was identified by clinical staff at local health facilities. Other outcomes included the incidence of all pneumonia diagnoses (including those not meeting WHO criteria), severe pneumonia as defined by WHO criteria, and deaths in children less than 5 years. The trial showed no association between use of cleaner-burning biomass stoves and pneumonia by any definition in children. Although the cooking and pneumonia study did include WHO-defined severe pneumonia and an oxygen saturation cutoff of less than 90%, there were only a small number of cases. Furthermore, the cooking and pneumonia study has yet to report the amount of exposure to household air pollution. In both RESPIRE and the cooking and pneumonia study, low diagnostic specificity, combined with potentially insufficient exposure reduction, could have led to non-significant results and an underestimation of the power needed to document efficacy.

With inadequate power, a trial is at risk of a false-negative finding. The choice to study severe pneumonia requires a larger sample size when compared with pneumonia of any type of severity to maintain power because of the smaller numbers of severe pneumonia cases. Conversely, this issue is somewhat counterbalanced by the anticipated greater effect size (farther from the null) for severe pneumonia versus all cases of pneumonia, as severe pneumonia is a more specific outcome which could be more clearly associated with a lowering of household air pollution. In the RESPIRE trial, for example, the relative risk for severe pneumonia was 0·67, whereas the relative risk for all pneumonia was 0·78. Table 2 shows the respective power calculations needed for these two outcomes.

Table 2

Differences in sample size for a trial by severity of pneumonia

	Incidence	Relative risk to be detected	Sample size needed in treatment group and in control group (combined)
All pneumonia	17 episodes per 100 child-years	0·78	1440 children
Severe pneumonia	5 episodes per 100 child-years	0·67	2300 children

In five studies, the incidence of WHO-defined severe pneumonia or severe-pneumonia with radiographic confirmation was approximately five episodes per 100 child-years in children below two years of age.11, 33, 34, 35, 36 The RESPIRE trial documented a relative risk of 0·67 for children with severe pneumonia in households with improved cookstoves, compared with those using traditional stoves. Using this information, we can calculate the necessary sample size using a difference in two proportions (assuming pneumonia events are independent). If the intervention and control groups are of equal size, 2300 children must be followed in their first two years of age to achieve 80% power with a conventional significance level of 0·05. A smaller sample size is acceptable for pneumonia of any severity because the number of cases will be higher. Although existing data are scarce, three previous studies have found that overall pneumonia incidence is about 3·4 times higher than that of severe pneumonia in children under three years of age (ie, 17 pneumonia episodes per 100 child-years vs 5 severe pneumonia episodes per 100 child-years).37, 38, 39 Under this assumption, and assuming a relative risk of 0·78 from RESPIRE and a 1-year follow-up, 1440 children are needed in each group for all pneumonias.

In research studies, investigators should also consider appropriate clinical elements and ancillary diagnostics to improve the specificity of the case definition.

Diagnostic methods

Clinical symptoms and danger signs

The majority of pneumonia case definitions currently in use, such as the WHO definition, are primarily based on clinical signs and symptoms. Two reviews40, 41 have examined the diagnostic utility of a variety of clinical signs and symptoms (including cough, fever, poor feeding, cyanosis, grunting, nasal flaring, tachypnoea, and lower chest wall indrawing) against radiologically confirmed pneumonia according to any criteria. The first review found that the features with the highest diagnostic accuracy included a respiratory rate higher than 50 breaths per min (positive likelihood ratio of 1·90, 95% CI 1·45–2·48), grunting (1·78, 1·10–2·88), lower chest wall indrawing (1·76, 0·86–3·58), and nasal flaring (1·75, 95% CI 1·20–2·56); however, there was significant heterogeneity in sensitivity and specificity in the studies reviewed. Similarly, the second review reported that signs of increased work of breathing, such as grunting (positive likelihood ratio of 2·7, 95% CI 1·5–5·1), nasal flaring (2·2, 1·3–3·1), or chest indrawing (1·9, 1·2–2·5) had better diagnostic accuracy than radiographically confirmed pneumonia. This study, however, did not examine combinations of clinical signs or stratify by age. A limitation of these reviews is that the studies included enrolled children with a higher probability of pneumonia before testing due to inclusion criteria of respiratory symptoms (or even a suspicion of the presence of pneumonia by the treating physician). It should also be noted that although these results indicate that some signs and symptoms are better diagnostic features than others, none of them have a high diagnostic accuracy. Fever and tachycardia, two common manifestations of pneumonia, are non-specific and are variably present. This fact, taken together with the study heterogeneity, suggests that there is no one clinical criterion with high enough sensitivity and specificity to be relied on for a diagnosis of pneumonia and could help to explain why studies focused on clinical symptoms and signs are largely negative. However, another article, suggests that WHO guidelines amend their danger signs for pneumonia to include signs of severe respiratory distress (ie, grunting, nasal flaring, head nodding, tracheal tugging, intercostal retractions, severe tachypnoea), along with hypoxaemia and moderate to severe malnutrition (defined as either weight-for-age or weight-for-height less than 2 z-scores below the median of the WHO child growth standards, or mid-upper arm circumference less than 12·5 cm for children 6–59 months of age). These signs have yet to be validated or recommended by WHO.

Although case definitions generally consider respiratory symptoms and signs for pneumonia as acute in nature, there is no consensus on the number of days since symptom onset until a diagnosis can be made. It is likely that symptoms of pneumonia develop over a period of several days, with nearly all sick children presenting within 14 days. In the pneumonia aetiology research for child health (PERCH) study, for example, the median duration of symptoms in 967 children in seven low-income and middle-income countries with alveolar consolidation on chest radiography was 3 days (IQR: 2–6). Accordingly, for children to be defined as having pneumonia, we recommend a case definition that restricts the clinical presentation to within 2 weeks of symptom onset.

Respiratory rate

Respiratory rate is typically defined as the number of breaths taken per min, and tachypnoea is defined as a higher than normal respiratory rate. Although there have been many studies measuring respiratory rates of different populations in upper-middle income countries, the reference ranges for respiratory rates in children in other settings are not based on evidence. For example, WHO provides tachypnoea cutoffs across a broad range of ages (respiratory rate of ≥60 breaths per min for children who are <2 months of age, ≥50 breaths per min for children who are between 2 months and 11 months of age, and ≥40 breaths per min for children who are 12–59 months of age), although other factors are hypothesised to affect respiratory rate. For example, baseline respiratory rates of individuals at high altitudes have been found to be significantly higher when compared with those at sea-level, but exactly how much higher respiratory rates are for children under the age of 2 years at altitude is unclear. As tachypnoea is a predictor of pneumonia in children under 2 years of age (particularly febrile children), inappropriate reference ranges for normal respiratory rates can lead to an inaccurate determination of the presence or absence of tachypnoea and, thus, inaccurate pneumonia diagnoses.47, 48

Although respiratory rate is a classifying factor of the WHO definition of pneumonia, it can be difficult to measure in a standardized way. Respiratory rate is typically counted manually in low-resource settings, with breaths per min counted using timers or counting beads.49, 50, 51 Manual measurement, although often the reference standard, can be imprecise and is affected by intra-observer variation as it requires focused concentration and can be more difficult with a crying, irritable, or moving child. Automated respiratory rate counters are more common in well-resourced settings and several models exist that use a variety of technologies, such as indirect effects on cardiovascular physiology and blood flow, thoracic effort and motion, tidal volume, and exhaled breath humidity. A systematic review evaluating studies of 14 automated and three manual counting devices found that direct comparison of devices was difficult in studies with small sample sizes and inconsistent reference standards and methods.

Arterial oxyhaemoglobin saturation

Pulse oximeters are devices that measure peripheral arterial oxyhaemoglobin saturation (SpO2) non-invasively. They are inexpensive, portable, and, with adequate training and supervision, can be reliably used with children at all levels of the health system in low-resource settings, including by lay community health workers at the household level. Hypoxaemia due to pneumonia occurs because of ventilation-perfusion mismatch in the lungs. An SpO2 measurement of less than 90% is a well recognised indicator of pneumonia severity and mortality in children. In hospitalised children in low-income and middle-income countries, hypoxaemia is also associated with higher odds of WHO-defined alveolar consolidation on chest radiography. Furthermore, a cohort study in a paediatric emergency department in Boston found that a SpO2 measurement of less than 92% was the strongest predictor of radiographically-confirmed pneumonia.

Although pulse oximetry has the potential to be useful for the diagnosis of pneumonia at the community level, there are several key considerations. First, the majority of evidence for the diagnostic utility of SpO2 measurement for radiographic disease (the current reference standard) is hospital-based, and the effectiveness of using SpO2 for home-based surveillance is unknown. Second, reference values for SpO2 in healthy children are not well established, especially considering varying altitudes.55, 56 WHO uses a SpO2 threshold of less than 90% for severe pneumonia and as an indicator for severe disease requiring oxygen supplementation and referral. This threshold will probably misclassify some children at higher altitudes, where a lower threshold for SpO2 might be more discriminatory.57, 58 Third, the prevalence of hypoxaemia in pneumonia is low and, although highly specific, it could be affected by a low positive predictive value. Finally, although pulse oximetry facilitates standardisation of severe pneumonia, it does not provide any information or indication about the cause (or causes) of pneumonia.

Lung auscultation

Lung auscultation is a diagnostic procedure that could improve diagnostic specificity. Inspiratory lung crackles are believed to represent the equalisation of distal airway pressures caused by the abrupt opening of collapsed alveoli and adjacent airways. The likelihood of radiographic pneumonia increases in the presence of crackles on auscultation.44, 61 Although traditional acoustic stethoscopes are inexpensive and portable, the implementation of lung auscultation in low-resource settings is limited by difficulties in achieving reliable, reproducible interpretations of lung sounds. Furthermore, lung auscultation requires specialised training to differentiate sounds, as well as the presence of a quiet examination area. These challenges are exacerbated in children with inconsistent breathing patterns and variable cooperation between examiners. WHO guidelines for frontline health-care providers and community health workers do not include lung auscultation in the diagnostic criteria for child pneumonia.31, 63 Although digital stethoscopes and automated lung sound analysis are emerging areas with the potential to overcome these educational and interpretation limitations,64, 65 additional research is needed before field implementation is feasible.

Host-response biomarker testing

Host-response biomarkers, such as C-reactive protein, interleukin(IL)-6, and procalcitonin, have been increasingly used to assist the diagnosis of a variety of infectious diseases, including pneumonia and sepsis.66, 67, 68, 69, 70, 71 A meta-analysis of 6708 adults with acute respiratory infection found that the use of procalcitonin to guide therapy was associated with a 17% reduced risk of mortality and a 2·4-day reduction in antibiotic exposure compared with controls, without increasing adverse outcomes. However, data in children are scarce, and early studies have conflicting results.73, 74, 75, 76 An analysis of 532 children who were hospitalised with community-acquired pneumonia in the USA found that lower procalcitonin concentrations (<0·25 ng/mL) were associated with a reduced risk of typical bacterial detection, suggesting that procalcitonin-guided therapy could identify children who do not require antibiotic treatment.

Few studies in low-income and middle-income countries have investigated the consistency of host-response biomarkers in childhood pneumonia diagnoses.68, 69, 78, 79 Certain conditions complicate the definition of an optimal threshold of host-response biomarkers for diagnosis. For example, not all bacterial infections cause host-response biomarkers to rise, or to rise to the same degree. Furthermore, host-response could be affected by conditions such as malnutrition and immunosuppression. Two studies compared biomarker testing for pneumonia (as defined by WHO) and radiographical findings.68, 69 Erdman and colleagues studied how biomarker testing using C-reactive protein and Chitinase 3-like-1 predicted radiological findings in 155 children who had pneumonia (as defined by WHO) in Tanzania. Valim and colleagues selected 80 children with pneumonia (as defined by WHO), along with ten healthy controls, and identified how combinations of biomarkers (including haptoglobin, tumour necrosis factor receptor 2 or IL-10, and tissue inhibitor of metalloproteinases 1) classified patients by bacterial, malarial, and viral causes. The study by Erdman and colleagues (sensitivity=93·3%, 95% CI 76·5–98·8%; specificity=80·8, 72·6–87·1%) and the study by Valim and colleagues (sensitivity=96%, 78–99, specificity=86%, 74–94) found biomarker testing to be highly sensitive, but less specific.68, 69 Although point-of-care tests do not exist, these studies suggest that host-response biomarker testing for presumed causes of pneumonia might be able to improve diagnostics in resource-limited settings.68, 69

Aetiology detection

Identification of a specific cause of pneumonia can reduce heterogeneity in pneumonia phenotypes, provide confirmation of bacterial or viral causes of pneumonia, and identify mixed causes. Aetiological studies done using microscopy, standard microbiological cultures, serology, antigen detection or molecular methods could assist in both the diagnosis and treatment of pneumonia. Moreover, the causes of pneumonia can be ascertained in sputum; blood; urine; nasopharyngeal, oropharyngeal or nasal swabs, nasopharyngeal or nasal wash, and by nasopharyngeal, lung, or pleural fluid aspirates. Novel molecular methods can detect multiple pathogens from a single sample.

However, some limitations require consideration. Heterogeneity in sample type, approaches for sample collection, and in diagnostic methods can make it difficult to interpret findings or compare across studies. Indeed, a review of studies analysing the causes of childhood pneumonia between 2000 and 2010 showed that there was no standard approach to specimen collection and laboratory techniques, further compounded by multiple case definitions, which affect the interpretation of findings. For example, microbial culture of blood samples provides a specific diagnosis, but has an overall low yield. Expectorated sputum samples are obtained directly from the respiratory system but could also carry asymptomatic colonisers, making it difficult to identify the causative pathogen. Induced sputum requires specialised equipment and can be onerous to collect, with evidence showing that it does not provide additional diagnostic information when using standard microbiology or molecular methods beyond that obtained from nasopharyngeal or oropharyngeal samples.83, 84 Serological tests can have variable sensitivities. Lung or pleural fluid aspirates are invasive and difficult to collect. Finally, swabs are easier and faster to collect than washes or aspirates, but aspirates could be more sensitive. Nasopharyngeal swabs are more invasive than nasal swabs but could have a better yield,85, 86 and the distribution of pathogens between these two types of swabs could be very different. Future studies should consider a standardised approach for both sample collection and laboratory processing, as described by PERCH investigators.

Studies that investigate the causes of pneumonia could help to reduce heterogeneity in pneumonia phenotypes by identifying whether the underlying infection is predominantly bacterial, viral, or a combination of both. One potential complication with these studies is that multiple pathogens are often found in the respiratory tract of both sick and healthy children. For example, PERCH investigators found an average of 3·8 pathogens in all children with severe pneumonia (3·9 pathogens in those with a positive chest X-ray) and 3·6 pathogens in healthy controls, so it can be difficult to disentangle the root cause. However, even with the best of methods, it is common to not find any pathogens in a substantial proportion of pneumonia episodes.

Imaging for pneumonia diagnosis

Consolidation and interstitial patterns, visualised using chest radiography or lung ultrasound, are characteristic features of pneumonia, although many guidelines only require imaging to diagnose pneumonia in ambiguous or hospitalised cases. For instance, the Paediatric Infectious Diseases Society does not recommend the use of chest radiography in community acquired pneumonia unless the child is hypoxaemic, in respiratory distress, has failed initial treatment, or is hospitalised. Imaging has been limited based on concerns of exposure to radiation and cost; point-of-care imaging with lung ultrasound has the potential to alleviate these concerns.

Chest radiography

Chest radiography has historically been considered to be the imaging standard for diagnosing paediatric pneumonia. However, despite longstanding experience and established, up-to-date guidelines, chest radiography is an imperfect standard. Not all cases show evidence of consolidation or interstitial patterns, particularly early in disease. One study showed that, in severely malnourished children, autopsies positive for pneumonia can have normal chest radiography images immediately before death. Also, implementation of radiography equipment is expensive and can be technically challenging in extreme climates or for facilities with an inconsistent power supply; issues common to resource-poor settings. Image quality and technician training also affects interpretation. Training and maintaining study staff that can reliably interpret images is labour intensive, necessitating close adherence to a standardised protocol and frequent re-standardisation and quality control to mitigate high inter-reader variability. In general, inter-reader agreement is higher for consolidations and lower for interstitial patterns.

As radiography uses ionising radiation, it must be used judiciously. Not having follow-up imaging, however, can reduce case ascertainment in research studies, especially if children present to care early in the disease course when radiographic abnormalities might not be apparent. This issue is of particular importance in studies that use frequent household surveillance, which are predisposed towards earlier disease detection. Chest radiography is not universally available, especially in low-resource settings, it can be expensive to upkeep, and does not provide information about the causes of the pneumonia. This problem is further exacerbated by the increasing recognition that childhood pneumonia is multi-pathogenic. However, the presence of lobar consolidation, which has been recommended as an endpoint for vaccine studies, might be indicative of bacterial disease.

Lung ultrasound

Lung ultrasound is an emerging tool used in the diagnosis of pneumonia in both adults and children.92, 93 Unlike chest radiography, lung ultrasound is a rapid, point-of-care diagnostic test that carries no radiation. Lung ultrasound has shown strong diagnostic validity in various age groups.92, 93 Studies in children have shown a sensitivity of 96% (95% CI 94–97) and a specificity of 93% (90–96%) for radiographically confirmed clinical pneumonia of all severities.93, 94

A study in Lima, Peru, showed that lung ultrasound is easy to learn in resource-limited settings. Standardised training methods were used to achieve inter-reader agreements as high as 0·77 (95% CI 0·74–0·81). Additionally, diagnostic criteria for pneumonia in lung ultrasound have a sensitivity of 88·5% and a specificity of 100% compared with chest radiography. In a randomised controlled trial, lung ultrasound was shown to reduce chest radiography use without increasing cases of missed pneumonia or adverse events when used in a busy emergency department. Furthermore, inter-reader reliability is generally higher in lung ultrasound, but it can differ based on approaches to standardised training and has the potential to skew the accuracy of diagnoses.

Compared with lung ultrasound, the definition of pneumonia by WHO, which is based entirely on clinical signs and symptoms, has been shown to lack adequate sensitivity and specificity. A definition of pneumonia that combines imaging with clinical signs and symptoms could improve accurate case detection. There is still a need for large multicentre studies to validate the use of lung ultrasound for the diagnosis of pneumonia in children.

Surveillance

Pneumonia studies rely on surveillance systems to capture cases. Therefore, investigators should consider specific characteristics of surveillance systems, such as passive versus active, frequency, and setting, as these qualities can affect outcome ascertainment (panel 1 ).

Active

Study personnel prospectively identify cases according to pre-set definitions by contacting individuals or providers directly

More resource-intensive and time-intensive

Easier to standardise

Can be home-based or facility-based

Passive

Investigators retrospectively gather information from existing records

Relies on other providers, so requires standardisation of data collection

Must consider provider willingness to participate and their level of training

Less resource-intensive

Is usually facility-based

Facility-based surveillance systems

More convenient for study staff in active surveillance

Resources needed for a complete evaluation are readily available

Relies on participant to come to facility

Barriers to care (including distance between hospitals and homes, absence of transportation, inability to pay, perceived unimportance of symptoms, and having a low confidence in the health-care system) can lead to missed cases

Feasibility affected by number of facilities required for thorough surveillance

Home-based surveillance systems

More convenient for participant

Could allow for more thorough capture of events as field workers can take all measurements and there is no need for coordination with health providers

Can be intrusive

More resource-intensive

More frequent assessment

More intrusive

Might capture fewer severe cases if surveillance leads to early intervention

Less frequent assessment

Might miss events due to recall bias

Surveillance systems are classified as passive or active according to how investigators obtain information about cases. Passive surveillance systems retrospectively gather information about cases from existing records. These systems are often less resource-intensive because of their reliance on pre-existing health infrastructure. However, the reliance on external health providers requires standardisation of methods for collecting data. Before selecting a passive surveillance approach, investigators must consider the quality of existing medical records (which might be scarce, incomplete, or easily lost due to disaster in resource-limited settings), as well as provider willingness to participate, their level of training, and availability to consistently participate in case reporting.

Conversely, in active surveillance systems, study personnel identify cases according to pre-set definitions by interacting with potential cases directly. Although active surveillance is more resource-intensive and time-intensive, and can be more intrusive, it allows for more complete outcome ascertainment provided the study population is adequately captured. Unless extensive efforts are put in place to standardise passive surveillance, active surveillance also allows for more uniform data collection as study personnel are performing diagnoses.

When conducting active surveillance, the frequency of assessment is important to consider. Infrequent follow-up can miss important events and might be affected by poor recall. For example, a community-based longitudinal study in Lima, Peru, found that visits done twice weekly resulted in lower documented prevalence of clinical symptoms than daily visits, demonstrating that symptom misplacement was more likely to occur as the recall period increased.

However, increased frequency of visits can also bias the result by capturing fewer severe cases. In clinical or field trials where surveillance might lead to referrals, early case detection can lead to earlier treatment, thereby reducing the number of severe cases that develop. In the example of the HAPIN trial, whereby severe pneumonia is a primary outcome, active surveillance would possibly reduce the primary outcome. Frequent visits can also be intrusive, leading to decreased compliance with study activities. One must balance these potential sources of bias when deciding frequency of follow-up.

Whereas passive surveillance systems are generally only implemented in health facilities, active systems can be implemented in health facilities, in the home, or in other settings. This choice of surveillance setting can have trade-offs. Home-based assessments are convenient for participants and are likely to lead to outcomes being captured more thoroughly, but are resource-intensive, intrusive, can lead to the Hawthorne effect, or bias results towards milder cases.

Facility-based surveillance can be logistically easier for study personnel, both in terms of transportation and also the resources available for a complete evaluation. However, this approach relies on participants to visit designated facilities, which can be impeded by a variety of barriers (for example, large distances between homes and health facilities, an absence of transportation, an inability to pay, perceived unimportance of symptoms, and having a low level of confidence in the health-care system). The feasibility of facility-based surveillance is affected by the number of facilities required to achieve a complete catchment area. Cases might be missed in settings where participants can present to facilities or informal health providers outside of the surveillance network, or where they do not seek care at all. Finally, facility-based surveillance systems are more likely to under-report milder outcomes, and thus yield lower overall estimates of pneumonia incidence with a greater proportion of severe cases.

Although empirical data to inform choices on surveillance methods for childhood pneumonia are scarce, several examples from the literature are instructive. Pneumococcal vaccine trials have generally taken the approach of facility or laboratory-based active surveillance.100, 101 This approach is appropriate for studies with a primary outcome of culture-confirmed pneumococcal pneumonia, but might be insensitive to pneumonia owing to other causes. This approach also tends to result in a relatively greater capture of severe cases compared with non-severe cases.

Le Roux and colleagues37, 99 present an interesting comparison of two surveillance methods for severe pneumonia: a passive facility-based surveillance system and the active facility and home-based method used in the Drakenstein child health study of a South African birth cohort. They found that the passive facility-based system yielded consistently lower incidences than active surveillance. Possible explanations for this finding included the possibility of under-reporting cases of pneumonia by health facilities due to low motivation or insufficient time, under-detection by health providers due to insufficient training, or failure of patients to present to health-care facilities in the study area. The proportion of severe pneumonia cases was slightly higher in the active cohort than in the passive system (23% vs 18%), but the difference was not statistically significant. There was concern for under-reporting of severity assessments in the passive surveillance system, which could have led to an artificially low incidence of severe pneumonia.

A different approach was adopted for the cooking and pneumonia study, which assessed for pneumonia via passive, facility-based surveillance. Study staff affixed a sticker to the Malawi Ministry of Health and Population health passport (ie, a patient-held medical record system) of all participants, which included a description of the trial and space for providers to document symptoms of pneumonia. This information was then recorded by study personnel every three months. Community health providers and family members were asked to call or text study staff after pneumonia episodes. Although this approach elegantly used pre-existing infrastructure, it could be difficult to replicate in other settings, and might have been subject to under-reporting if a health provider did not see the sticker or neglected to fill out study documentation, or if the participant's health documentation was not presented or lost.

A combination of approaches could yield higher sensitivity. Case-finding in RESPIRE combined active and passive approaches done in various settings to detect childhood pneumonia. Field workers visited participant homes weekly, and study physicians in local community clinics evaluated children presenting with symptoms, reviewed hospital records, and did verbal autopsies (a method of gathering information about symptoms and circumstances for a deceased individual to determine cause of death) for all deaths. This multi-faceted strategy yielded high sensitivity for pneumonia and higher referral compliance in the intervention group than the control group. This approach, however, might have registered a lower incidence of severe pneumonia because of a weekly visit schedule that treated pneumonia cases early. Moreover, this approach could be too cumbersome or unfeasible to implement in larger trials or in studies with larger catchment areas.

In summary, surveillance methods can influence the incidence of pneumonia detected during a study and dictate the type or severity of pneumonia cases identified and the evolution to severe pneumonia. Surveillance systems should also align with study aims to balance sensitivity and specificity and avoid bias.

Recommended case definition

We summarise potential approaches for the diagnosis of severe pneumonia in field trials and ancillary testing methods in table 3 . Importantly, many of these approaches are subject to heterogeneity, have suboptimal specificity, and require careful standardisation of protocol.

Table 3

Advantages and disadvantages of existing diagnostic methods for paediatric pneumonia

	Description	Advantages	Disadvantages
Field assessment methods
WHO case definition	Acute (≤ 14 days) non-severe pneumonia: cough or difficulty breathing with tachypnoea or lower chest wall indrawing; severe pneumonia: pneumonia plus any general danger sign. Another definition of severe pneumonia that follows the guidelines is acute (≤14 days) episode of cough or difficulty breathing with either a general danger sign or hypoxaemia.30	High sensitivity, easy to implement in resource-poor settings; provides a standardised approach that is comparable with other studies.	Low specificity, fails to distinguish between bacterial and viral causes,21 can lead to non-differential misclassification of outcome, danger signs are subjective and difficult to identify and the diagnosis of chest indrawing requires standardised training.
Physician diagnosis of pneumonia	Integrates clinical knowledge, evidence-based knowledge and local practices for the identification of pneumonia. Chest indrawing and general signs of respiratory distress and general danger signs.	Follows common clinical guidelines and practices, benefits from an expert diagnosis.	Highly subjective to inter-observer heterogeneity, cannot be replicated, might need an adjudication panel for consensus.
Respiratory rate	Number of breaths taken per min. If higher than normal (defined by pre-specified cut-offs), considered to be tachypnoea.	WHO cutoffs are age-specific: ≥60 breaths per min for children <2 months of age, ≥50 breaths per min for children between 2 and 12 months of age, and ≥40 breaths per min for children between 12 and 59 months of age.46	Difficult to standardise measurement and inadequate reference ranges for a variety of settings (ie, high altitude).
Ancillary diagnostic methods
Arterial oxyhaemoglobin saturation	Measured using pulse oximeters to determine hypoxemia (typically a SpO2 measurement of <90%) that occurs in pneumonia because of ventilation-perfusion mismatch in the lungs.53	Inexpensive, portable, and, reliable in a variety of settings.52 Provides objective diagnostic criteria that is highly specific when combined with respiratory signs and symptoms, and a well recognised indicator of pneumonia severity and mortality in children.46	Unknown utility in home-based surveillance studies, reference values for SpO2 in healthy children are not well established (especially at varying altitudes), low sensitivity, does not provide any information or indication about the causes of pneumonia.
Chest auscultation	Inspiratory lung crackles represent the equalisation of distal airway pressures caused by the abrupt opening of collapsed alveoli and adjacent airways.	Likelihood of radiographic pneumonia increases with crackles44, 61 and provides objective diagnostic criteria when done correctly.	Difficult to achieve reliable, reproducible interpretations of lung sounds, requires specialised training, quiet examination areas (especially with children), does not provide any information about the causes of pneumonia.
Host-response biomarker testing	Used to guide therapy based on the cause, or causes, of pneumonia. Examples include CRP, procalcitonin, chitinase 3-like-1, haptoglobin, tumour necrosis factor receptor 2 or IL-10, and tissue inhibitor of metalloproteinases 1.	Might identify children who do not require antibiotic treatment, past studies have shown high sensitivity and accuracy.68, 69, 94	Hard to implement in resource-poor settings, point-of-care tests do not exist, not all bacterial infections affect host-response biomarkers, host-response might be affected by malnutrition and immunosuppression.
Aetiology detection	Ascertained in sputum, blood, urine, nasal, nasopharyngeal or oropharyngeal swab, nasal or nasopharyngeal wash, and by nasopharyngeal, lung, or pleural fluid aspirates using microscopy, standard microbiological cultures, serology, antigen detection, or molecular methods.	Can provide confirmation of bacterial or viral causes of pneumonia, and identify mixed causes.81 This method might also help to reduce heterogeneity in pneumonia phenotypes by identifying if the underlying infection is predominantly bacterial, viral, or instances where there are a mixture of causes.	Reliable findings require standardised sample types, approaches for sample collection, and diagnostic methods. Different sample types have varying sensitivities for different pathogens. Some (such as pleural fluid) are invasive to collect.
Chest radiography	Visualises consolidation and interstitial patterns.	Widely available, well developed standard for image interpretation. Historically, this method is the gold standard for diagnosing pneumonia.66 Allows for comparisons between countries, regions, and time periods.	Exposure to radiation, not all cases show evidence of consolidation or interstitial patterns (particularly early in disease). Requires standardised training and maintaining study staff to reliably interpret, and quality control to mitigate high inter-reader variability. Expensive equipment requires power supply.
Lung ultrasound	Visualises consolidation and interstitial patterns.	Rapid, point-of-care diagnostic test without radiation,94 and strong diagnostic validity shown in various age groups.92, 93	Further validation needed in large studies. Not all cases show evidence of consolidation or interstitial patterns (particularly early in disease or in instances of malnourishment). Requires standardised training and maintaining study staff to reliably interpret, and quality control to mitigate high inter-reader variability.

WHO=World Health Organization. SpO2= peripheral capillary oxygen saturation. CRP=C-reactive protein. IL=interleukin.

Our recommended definition for severe pneumonia is summarised in figure 1 . First, assess respiratory symptoms and if cough or difficulty breathing is present (reported or observed), then continue with assessment for clinical signs. Fever and tachycardia are common manifestations of pneumonia, but have low specificity and present variably. We recommend determining the presence of tachypnoea (either defined by IMCI age-specific criteria or by using an age-specific upper limit of normal thresholds of a reference population), chest indrawing, any other sign of severe respiratory distress, or hypoxaemia before continuing for severity. We recommend defining pneumonia as severe if a child has one of the general danger signs delineated by WHO or hypoxaemia because both are widely used in both clinical and research settings. Consider using signs of severe respiratory distress as an indication of difficulty breathing. Adjudication panels should be used for imaging interpretation and physician-diagnosis. Finally, ancillary diagnostic testing for causes, host-biomarker response testing, and chest auscultation can be difficult to implement in low-resource settings and are subject to heterogeneity.

Figure 1

Severe pneumonia diagnostic flow chart for field trials in resource-poor settings

Severe pneumonia diagnosis involves a combination of respiratory symptoms, clinical signs, and severity. The solid grey boxes indicate our recommended approach for diagnosing severe pneumonia in field trials, while the dotted orange boxes indicate additional symptoms and signs that are commonly available in clinical practice but not recommended for diagnosis of severe pneumonia in field studies. Both cough and difficulty breathing can be based on report or observation. Observed difficulty breathing is defined as any abnormal breathing pattern not limited to tachypnoea, chest indrawing, wheeze or noisy breathing, or other signs of respiratory distress. Hypoxaemia can be used as a clinical sign, marker of disease severity, or ancillary test. Severe respiratory distress includes any of the following: head nodding, persistent nasal flaring, grunting, stridor while calm, tracheal tugging, intercostal retractions, pronounced lower chest wall indrawing, very fast breathing for age. General danger signs include inability to drink, vomiting everything, convulsions, lethargy or unconsciousness, severe malnutrition, or stridor in a calm child. Opacification on imaging refers to the finding of a primary endpoint pneumonia on chest radiography or lung ultrasound.

Based on the strengths of diagnostic criteria and need for an objective approach to identify cases, we recommend that pneumonia outcome studies consider using a standardised case definition, as proposed by WHO, rather than one that depends on a physician diagnosis. We also recommend defining tachypnoea as respiratory rate either by the IMCI definition (≥60 breaths per min for children <2 months of age, ≥50 breaths per min for children between 2 and 11 months of age, and ≥40 breaths per min for children between 12 and 59 months of age) or by further refining it by using age-specific upper limit of normal thresholds for the reference population. The latter might be more appropriate in high-altitude settings.

Furthermore, we recommend using general danger signs as defined by WHO to identify severe cases of pneumonia and to increase the specificity of a pneumonia diagnosis (panel 2 ). One potential limitation of using general danger signs as proposed by WHO is that they could be too nonspecific; children with severe respiratory distress might not necessarily present with a general danger sign, and children with other illnesses (such as severe diarrhoea or malaria) could be incorrectly classified as having severe pneumonia by meeting criteria for one or more general danger signs. This limitation could be mitigated by using more specific signs of severe respiratory distress and by adding hypoxaemia and moderate-to-severe malnutrition to the list of danger signs; however, further validation of these clinical signs is warranted. We also recommend testing for malaria and other coinfections following standard guidelines, if relevant to the local context.

General danger signs for children younger than 2 months *

Unable to drink or breastfeed

Unable to feed well

Vomits all food

Convulsions

Lethargic or unconscious

Not moving or moves only when stimulated

Cyanosis

Stridor

Grunting

Severe chest indrawing

Fast breathing (≥60 breaths per min)

Fever (38°C or above)

Low body temperature (<35·5°C)

General danger signs for children aged 2 months or older*

Unable to drink or breastfeed

Vomit all food

Convulsions

Stridor

Lethargic or unconscious

Finally, we recommend that a case definition based on respiratory symptoms and signs that have presented acutely (<14 days before disease presentation) and ancillary testing with either pulse oximetry or chest imaging. We also recommend defining hypoxaemia as a SpO2 measurement of less than 93%, and further refining this measurement for children at higher altitudes by using the lower limit of normal for the reference population. For our trial, we plan to use a combination of respiratory symptoms (cough or difficulty breathing), respiratory signs (tachypnoea and presence of chest indrawing), the presence of a general danger sign to identify severity (ie, severe pneumonia as defined by WHO), and ancillary testing with pulse oximetry and chest imaging.

Discussion

When designing randomised field trials and observational studies that intend to study pneumonia outcomes, researchers must carefully consider how the local epidemiology and vaccine coverage, pneumonia case definition, ancillary diagnostics, and surveillance methods might affect their study findings. In this Review, we have provided examples of how each of these factors affect the detection of pneumonia cases. Previous antibiotic treatment and household air pollution reduction trials have failed to show an association between pneumonia and the exposure of interest.10, 11, 104, 105, 106, 107, 108, 109 Many of these trials used the WHO definition of pneumonia, which is solely based on clinical symptoms and signs, is highly sensitive, but also has a low specificity. Although effective in informing treatment guidelines that largely reduced childhood mortality,21, 32 use of the WHO definition could lead to reduced effect size and study power in field trials.21, 29 In fact, severe pneumonia is less likely to be confused with another disease than non-severe pneumonia, and more accurately assess the efficacy of interventions against pneumonia.110, 111, 112

As pneumonia prevalence is highest in the first year of life, pneumonia outcome studies might find more cases with greater severity by focusing on a shorter follow-up period. The existing disease burden could also complicate pneumonia diagnoses. Other conditions can present with similar clinical signs and symptoms which can mimic the signs and symptoms of pneumonia. For instance, malaria can have overlapping clinical symptoms with pneumonia, and rapid antigen testing for plasmodium falciparum spp might be necessary in regions that are holoendemic for malaria to help differentiate these diseases. Therefore, we recommend that rapid testing for malaria be done in malaria endemic regions. Similarly, anaemia and fever can increase respiratory rate and lead to false-positive diagnosis of pneumonia.48, 113 Furthermore, complicated bacterial pneumonia (eg, a pleural effusion), underlying chronic disease (eg, human immunodeficiency virus infection), or respiratory infections from non-bacterial causes (eg, tuberculosis) might complicate interpretation of study findings, and investigators should either exclude these participants or account for these coinfections in their analyses. These factors might complicate data collection, and investigators should either exclude these participants or account for these coinfections in their analyses. In addition, tracking treatment responsiveness to first-line antibiotics can be informative and is recommended. If feasible, children that fail treatment might be investigated for other potential bacterial or non-bacterial causes, or underlying comorbidities, that could have altered expected treatment responsiveness.

Investigators should obtain baseline data about severe and non-severe pneumonia incidence and vaccine coverage specific to their site before planning any pneumonia intervention trial, as this information is important for selecting study sites and for doing sample size calculations. Studies intending to capture severe pneumonia cases might encounter difficulty reaching a sufficient sample size in regions with comprehensive vaccine coverage. Further, vaccine coverage should be reported to better understand the generalisability of the study setting.

Ancillary diagnostics methods, such as imaging, pulse oximetry, host-response biomarker, and aetiology testing, might allow for a more objective determination of severe pneumonia outcomes. For example, lung ultrasound has been shown to have a high specificity and sensitivity for pneumonia diagnoses and does not have any ionising radiation as compared with standard imaging approaches, such as chest radiography and CT.92, 93, 114 Lung ultrasound might be more affordable to implement and maintain in low-resource settings, but a formal cost-effective analysis needs to be done to substantiate this. Moreover, in some countries like India, ultrasounds are strictly regulated following fears of abortions done after fetal sex identification. A major concern with existing studies that assess the validity of lung ultrasound is the absence of a safe and effective gold-standard for the diagnosis of paediatric pneumonia. Chest radiography, although generally considered as the gold standard for the identification of pneumonia, has poor diagnostic validity when used to assess pneumonia in children.

Indeed, the absence of a true gold standard for pneumonia diagnosis might be one of the greatest challenges to trials assessing pneumonia outcomes. Some of the limitations met when using only one ancillary approach to identify pneumonia can be lessened with the combination of multiple diagnostic tests. For example, one study found that the area under the receiver operating characteristic curve (a summary measure of the accuracy of a quantitative diagnostic test) increased from 0·62, when using the WHO pneumonia criterion, to 0·85 when using a combination of signs and symptoms, auscultation, pulse oximetry, and lung ultrasound.

Investigators should also carefully select thresholds for diagnostic procedures. The cutoff points used by WHO for both respiratory rate and oxyhaemoglobin saturation have been criticised for not having an appropriate age-specificity or altitude-specificity.45, 55, 56 Future pneumonia trials should consider doing formative research that calculates the thresholds for respiratory rate and oxyhaemoglobin saturation at varying altitudes and age ranges.

Whenever incorporating ancillary diagnostics into a case definition, the equipment and technologies used should be thoroughly profiled and, ideally, standardised.87, 116 These factors are both particularly important when study personnel are not doing these ancillary tests or diagnostic procedures themselves, such as in passive surveillance. Investigators should aim to homogenise the equipment and diagnostic procedures available across health facilities. Doing so requires adequate training of all personnel interacting with study participants. Organising and conducting such trainings might not be feasible for study sites with many facilities and physicians. Otherwise, diagnoses might not be generalisable.

A combination of active and passive, home-based and facility-based surveillance uses the existing resources of the health infrastructure while still allowing for a standardised diagnosis and more complete outcome ascertainment. Large trials with extensive catchment areas, for example, could benefit from facility-based surveillance for primary data collection combined with home visits from study staff to reinforce health-seeking behaviours and ascertain health-care visits or hospitalisations that were missed, with the idea that the severe cases are less likely to be forgotten between visits. The passive surveillance aspect would rely on mothers of participating children, community health workers, and health-care providers in the study region to contact study staff when the child develops symptoms or presents for care. Although passive surveillance relies on external agents to alert the study team of potential cases, those children would be promptly referred to hospitals where they will receive a standardised evaluation by study personnel.

Finally, surveillance strategies in trials that assess pneumonia outcomes should account for existing care seeking behaviours and potential barriers that vary between study sites (figure 2 ). In areas with low health-care use, the promotion of healthcare-seeking behaviours and the identification of missed cases during periodic interviews is important. A characterisation of healthcare-seeking behaviour requires understanding the types of healthcare facilities available, the levels of health-care facilities at which patients can present for care, referral and treatment patterns, and modes of transportation between homes and healthcare facilities. Investigators must understand the local context regarding available healthcare-seeking behaviours, as well as barriers to care. For example, caretakers might not bring a sick child to any healthcare facility because of insurance status or if they distrust, or if they have little confidence in, medical personnel. Children with severe disease might not make it to a referral hospital if they have insufficient funds or resources for further transportation from a health post or centre. Lastly, the cost of health care at the referral facility might impede the child from receiving appropriate care. As with any community-based trial, there is rarely a universal approach. Pneumonia diagnosis and treatment differs largely between countries and even between clinicians and health-care professionals and, therefore, investigators must consider the local context when determining case definitions and surveillance systems.

Figure 2

Example of a framework of health-seeking behaviour for pneumonia

This figure describes the locations at which families seek care for their sick child, the process of referral between levels of care, and transport. The arrows represent potential modes of transportation (walking, public transportation represented with a bus icon, or private taxi or car represented with a car icon) between home and a health facility, or between health facilities. This example health system comprises of health posts (small remote outposts, generally with one health provider and minimal equipment, available during limited hours), health centres, and hospitals. In this hypothetical setting, health posts rarely refer to health centres, instead they refer directly to hospitals.

To address challenges in the diagnosis of pneumonia in field trials, the upcoming HAPIN trial incorporated expert recommendations made at the “Challenges in the diagnosis of pneumonia in community-based intervention field trials” workshop held in March, 2017 (panel 3 ). We chose to design a trial assessing pneumonia outcomes with severe pneumonia in children under 1 year of age as the primary outcome. Children with possible severe pneumonia will be identified in select health facilities by study staff. Existing care practices will not be disrupted but the study staff will be on call at the health facility to diagnose participants using HAPIN standardised equipment. Home visits will provide opportunities to capture missed pneumonia cases and reinforce behavioural messaging about seeking care. To reduce outcome misclassification and increase specificity, the primary case definition will include the definition of severe pneumonia provided by WHO plus objective findings (ie, hypoxaemia or consolidation detected by an imaging modality). To allow results from HAPIN to be comparable with previous studies, the definition of severe pneumonia provided by WHO will be a secondary outcome. We will also seek to define local thresholds for oxyhaemoglobin saturation and respiratory rate at altitudes 2500 m above sea level.

The choice of case definition and degree of misclassification of all cases of pneumonia will affect sample size calculations. Using case definitions that are sensitive, but not specific, will decrease statistical power to detect a difference.

The use of objective assessments, such as chest radiography, lung ultrasound, and arterial oxyhaemoglobin saturation, is recommended to identify pneumonia and avoid inconsistencies in interpretation between study staff. Quality control and assurance should be implemented to ensure as little measurement error as possible. Improve diagnostic specificity by using a standard definition to define the pneumonia primary endpoint when using imaging and a panel of readers who have received standardised training.

We recommend testing and adopting new technologies to aid the standardisation of data collection and quality control for case definitions (ie, automated respiratory rate counters, pulse oximeters, auscultation, and ultrasound imaging procedures).

Combinations of clinical signs and symptoms, and ancillary diagnostics are likely to yield a more specific case definition, but requires further research.

Surveillance strategies can greatly affect case detection and severity. Home visits could miss cases if they are held too infrequently. Conversely, home visits could decrease the number of severe cases due to early intervention if held too frequently. For successful surveillance at health facilities, we recommend coordinating with other referring facilities and study participants for communication with study staff.

We recommend a more comprehensive assessment of age-specific upper thresholds of respiratory rate to determine tachypnoea and lower thresholds of oxyhaemoglobin saturation to determine hypoxaemia at different altitudes.

The HAPIN trial, the Ghana randomised air pollution and health study (GRAPHS), and the Nepal trials will inform the literature on the nature of the relationship between various definitions of pneumonia and household air pollution exposure in young children. The HAPIN definition focuses on severe pneumonia diagnosed by field workers using objective criteria (figure 1). The GRAPHS trial uses a physician assessed definition of severe pneumonia in the first year of life, whereas the Nepal trials use the incidence of acute lower respiratory infection during the first 36 months of age, determined by maternal report and field worker assessment for more severe cases.

As study objectives differ based on the type of intervention and study setting, the pneumonia case definition and surveillance strategy should be carefully considered before implementing pneumonia outcome studies.

Search strategy and selection criteria

In March, 2017, the Bill & Melinda Gates Foundation hosted a workshop entitled “Challenges in the diagnosis of pneumonia in community-based intervention field trials”, which brought together a group of pneumonia experts and HAPIN investigators to discuss challenges in including pneumonia as a primary outcome in intervention field trials. We invited all authors to the second workshop. The workshops aimed to assist the investigators in developing field instruments and standardised approaches to identify severe pneumonia cases in young children for the upcoming Household Air Pollution Intervention Network trial.For our Review, we searched PubMed, Web of Science, and Google Scholar with the search terms “pneumonia”, “epidemiology”, “diagnosis”, “surveillance”, “definition”, and “clinical”, up to June 7, 2019. We included relevant articles written in English that studied pneumonia in children via observational or randomised controlled trials and excluded incidence studies of pneumonia or severe pneumonia in adults, and identified key recommendations, knowledge gaps, and research opportunities. However, this was not a systematic review. We also consulted the experts who participated in the workshops and asked them to provide relevant literature.