Applicability of evidence from previous systematic reviews on immunotherapy in current practice of childhood asthma treatment: a GRADE (Grading of Recommendations Assessment, Development and Evaluation) systematic review.

OBJECTIVE: Because most children with asthma now use inhaled corticosteroids (ICS), the added benefit of immunotherapy in asthmatic children needs to be examined. We re-assessed the effectiveness of subcutaneous (SCIT) and sublingual immunotherapy (SLIT) in childhood asthma treatment focusing on studies with patient-relevant outcome measures and children using ICS.
METHODS: We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to systematically search and appraise the evidence using predefined critical patient-relevant outcomes (asthma symptoms, asthma control and exacerbations). We searched to retrieve systematic reviews and randomised controlled trials on immunotherapy for asthma in children (1960-2017). We assessed the quality of the body of evidence with GRADE criteria.
RESULTS: The quality of the evidence for SCIT was very low due to a large risk of bias and indirectness (dated studies in children not using ICS). No effect of SCIT was found for asthma symptoms; no studies reported on asthma control. For asthma exacerbations, studies favoured SCIT. We have little confidence in this effect estimate, due to the very low quality of evidence. For SLIT, quality of the evidence was very low due to a large risk of bias, indirectness and imprecision. The outcome 'asthma symptoms' could not be calculated due to lack of standardisation and large clinical heterogeneity. Other predefined outcomes were not reported.
CONCLUSION: The beneficial effects of immunotherapy in childhood asthma found in earlier reviews are no longer considered applicable, because of indirectness (studies performed in children not being treated according to current asthma guidelines with ICS). There was absence of evidence to properly determine the effectiveness or lack thereof of immunotherapy in asthma treatment in children with ICS. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This study is the first review evaluating immunotherapy in asthmatic children using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, focusing more on clinically relevant than on statistically significant differences in patient-relevant outcomes.

By using GRADE, we identified indirectness in previous systematic reviews in this field, which highlights a lack of applicable evidence.

A strength of the study is the use of predefined clinically relevant patient outcomes, rather than statistically significant differences.

A general limitation of a systematic review is the use of aggregated data, that, in theory might mask potential specific results.

This study has focused on critically appraising earlier evidence for nowadays practice, rather than endeavouring to be complete.

Introduction

Asthma affects 10%–15% of school-aged children. For children with persistent asthma, all international guidelines recommend daily controller treatment with inhaled corticosteroids (ICS), and reliever medication (short-acting β-2-agonists) as needed.1 2 Although many children achieve complete asthma control using this effective and safe treatment,1 some need additional treatment to obtain disease control.3 4 Identification and treatment of comorbidities in children with problematic severe asthma is part of the stepwise approach to improve asthma control in these children.5 6

The most common of these comorbidities in children with asthma is allergic rhinitis,5 symptoms of which occur in 60%–80% of asthmatic children.7 8 Allergic rhinitis shares a common pathophysiological pathway with asthma, which has been described as the united airway concept.9 Allergic rhinitis is associated with worse asthma control in children, and accumulating evidence suggests that treatment of allergic rhinitis with intranasal steroids improves rhinitis and asthma symptoms in these patients.7 10 11

When symptoms of allergic rhinitis cannot be sufficiently controlled with nasal corticosteroids and oral antihistamines,9 12 immunotherapy can be considered as additional treatment.13 Subcutaneous immunotherapy (SCIT) requires repeated injections with an allergen extract and is available for allergens such as grass and tree pollen and house dust mite. After disappointing results of low-dose preparations in drops, effective high-dose sublingual immunotherapy (SLIT) has now become available with grass pollen allergen extract in a daily sublingual tablet.14 15 A Cochrane systematic review (SR), first published in 2000, and last updated in 2010, reported beneficial effects of immunotherapy in children with asthma.16 Multiple studies in this latter review, however, were performed before or in the 1980s, when most children with asthma were not using ICS.

As part of the update of the Dutch paediatric guideline on childhood asthma, we evaluated the literature on the added value of SCIT and SLIT in childhood asthma.17 Our structured clinical question was to assess whether immunotherapy (subcutaneous or sublingual), as an add-on to usual care with daily ICS, improves asthma outcomes in children (6–12 years) and adolescents (>12 years) with persistent asthma and sensitisation to relevant aeroallergens (grass or tree pollen, house dust mite or combinations), with or without symptoms of allergic rhinitis.

Methods

We used GRADE approach (Grading of Recommendations Assessment, Development and Evaluation) to appraise and summarise the body of evidence. GRADE is an internationally approved standard for managing complex evidence reviews.18 In contrast to former grading systems, GRADE focuses on the quality of the total body of evidence, instead of judging single studies. Another important characteristic of GRADE is that predefined outcomes with thresholds for clinical relevance are being used.19 In earlier grading systems, the evidence was summarised using outcomes reported in studies, not necessarily being outcomes a guideline development group would be interested in.20 GRADE avoids the use of surrogate or intermediate outcomes, and uses outcomes and differences that are more clinically relevant to patients instead. Starting from a SR, for each outcome the quality of evidence can be downgraded or upgraded, for instance based on risk of bias, inconsistency, indirectness, possible publication bias and dose–response relation.

The guideline development group included an epidemiologist, paediatric respiratory physicians, paediatricians, an allergist, an ear-nose-throat specialist, a family physician, a lung function technician, a youth public healthcare physician and patient representatives. The guideline development group predefined clinically relevant outcomes and divided these into critical (contributing to the overall quality of evidence), important (also relevant to the content of the guideline) and not important outcomes. For each outcome, a minimal clinically important difference was defined a priori. The outcomes taken into account in our literature review are summarised in table 1, with corresponding minimal clinically important differences.21–24

Table 1

Patient-relevant outcomes and clinical relevance

Outcome	Importance	Minimal clinically important difference
Asthma symptoms	Critical	ACT: 3, c-ACT: (2–)3*
(Severe) exacerbations	Critical	NNT, n=10
Asthma control	Critical	ACT: 3, c-ACT: (2–)3*
(Disease-specific) quality of life	Important	PAQLQ: 0.5 (scale 0–7)*
Change in FEV1	Important	>5% predicted

*Or comparable differences on other valid scales representing this outcome.

ACT, asthma control test; c-ACT, child asthma control test; FEV1, forced expiratory volume in 1 s; NNT, number needed to treat (to prevent one exacerbation); PAQLQ, Paediatric Asthma Quality of Life Questionnaire.

We applied a sensitive search strategy to retrieve all available evidence addressing the clinical question, focusing on SRs about asthma and immunotherapy in children. Literature searches were performed in March 2012 for the guideline (from 1960 onwards), and updated in April 2015 for the purpose of this review. A second update, including an expansion of the searching scope, was performed in June 2017 (see online supplementary table E1). In the original search, we searched in the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness and the Cochrane Central Trial Register. In the 2017 update, we also searched for SRs in the Medline and Embase databases (again from 1960 onwards). Two reviewers (EJG and MKT) independently screened the abstracts using predefined inclusion criteria: methodology (SRs), patients (children with allergic asthma) and SCIT and/or SLIT as an intervention. Animal studies, conference abstracts and studies published in languages other than English, Dutch and German were excluded. Differences between reviewers were resolved by consensus. Selected abstracts were critically appraised with respect to study population, intervention and methodological aspects (eg, systematic search and selection, inclusion of randomised controlled trials (RCTs)), which led to a further selection. An expert in the field (HdG) judged the selection for completeness.

All included studies were summarised in evidence tables by two reviewers (EJG and MKT). SRs were critically appraised using the AMSTAR checklist (A Measurement Tool to Assess Systematic Reviews).25 AMSTAR scores range from 0 to 11, a higher score indicating better quality (less bias). The Jadad scale was used to assess the methodological quality of each included RCT.26 This score ranges from 0 to 5, a higher score denoting a better quality. All eligible studies together defined the body of evidence, of which the quality was determined (per relevant outcome and overall quality) and GRADE profiles were created. Results from SRs and RCTs were pooled, if possible, in meta-analyses using RevMan V.5. We calculated standardised mean differences for continuous outcomes, because of the usage of different symptom scales in the underlying studies. We calculated risk ratios for dichotomous outcomes, to compare the probability of these outcomes between the intervention and control groups. In the meta-analyses we used random effects models, because of the possibility of generalisation of the outcomes for different allergens, and tested the difference between intervention and control with the inverse variance method, since this method is typically used in meta-analyses to combine the results of independent studies. We reported 95% CIs (predefined significance level: 0.05). Conclusions were drawn, based on quality and content, per outcome and discussed in the expert group until consensus was reached.

Patient involvement

The guideline development group included patient representatives who helped defining our clinical question, approved outcome measures and assessed its clinical relevancy. The burden of interventions and patient considerations were assessed as part of GRADE evaluation. Patients were not directly involved in this SR since we reviewed published literature.

Results

Literature search and selection

Screening of titles and abstracts yielded 83 eligible studies, 10 of which fulfilled the inclusion criteria.16 27–35 After examining these 10 papers in full, five more studies were excluded (figure 1).

Figure 1

Literature search and selection. RCT, randomised controlled trial; SCIT, subcutaneous immunotherapy; SLIT, sublingual immunotherapy.

Experts in the guideline working group confirmed that no relevant publications were missed. The updated search (April 2015) revealed 43 additional studies, of which 2 fulfilled the inclusion criteria.36 37 Full text examination resulted in exclusion of these two studies. The extended and updated search in June 2017 resulted in 177 hits, of which 6 were selected to full paper study.38–43 These studies were SRs in the field of SCIT and/or SLIT in children with asthma. Most of the included RCTs in these reviews had already been included in the 2015 search. We only added RCTs of those reviews to our meta-analyses that have not been included earlier. As a result, we added one study.44

Results of SCIT

Description of studies

We retrieved one Cochrane SR on the effectiveness of SCIT in patients with asthma, including 90 RCTs with a total of 3792 patients.16 This was a high-quality review (AMSTAR score 10/11). Fourteen of the included RCTs were performed in children exclusively; another 24 included children and adults. In a few studies the age inclusion criteria were not clear. The characteristics of these review are summarised in an evidence table (see online supplementary table E2).16 29 32 Only nine RCTs included in these reviews reported on our predefined outcomes in children.45–53 In these nine studies different allergens or combinations were studied (house dust mite (3), dog dander (1), grass pollen (1), mould (1), grass pollen/house dust mite (1) and tailored combinations (2)). Two RCTs published after the 2010 Cochrane review were retrieved.34 35 In the first, the clinical efficacy of house dust mite-specific SCIT in 20 asthmatic children was compared with no intervention in 20 others; patients were followed up for 6 months.34 In the other, the effects of allergen-specific SCIT on corticosteroid dose in asthmatic children were evaluated.35 Details of all included RCTs are summarised in the evidence table (see online supplementary table E3).34 35 45–53

Quality of the evidence

Little information was given about the included studies in the Cochrane review; for example, follow-up was not stated. There were also other concerns about the quality of the literature, for example, not all studies were double blind and placebo controlled, and randomisation procedures were poor. Therefore we re-analysed the individual paediatric studies in the Cochrane review, plus the added studies.34 35 Jadad scores of the single studies are presented in table 2.

Table 2

Jadad scores of RCTs on SCIT

	Randomisation*	Blinding†	Withdrawals‡	Total
Adkinson et al 45	1	1	1	3
Altintas et al 46	1	1	1	3
Dreborg et al 47	1	–	–	1
Hill et al 48	1	–	–	1
Johnstone et al 49	2	1	–	3
Johnstone et al 50	2	1	1	4
Price et al 51	1	1	–	3
Tsai et al 34	1	–	1	2
Valovirta et al 52	1	–	1	2
Warner et al 53	1	1	1	3
Zielen et al 35	1	1	1	3

*1 point if randomisation is mentioned; 1 additional point if the method of randomisation is appropriate; −1 point if the method of randomisation is inappropriate.

†1 point if blinding is mentioned; 1 additional point if the method of blinding is appropriate; −1 point if the method of blinding is inappropriate.

‡1 point if the number and the reasons for withdrawal in each group are stated.

RCT, randomised controlled trial; SCIT, subcutaneous immunotherapy.

The quality of the body of evidence for all critical and important outcomes was very low (table 3), mainly due to large risk of bias and indirectness. The large risk of bias was caused by a lack of allocation concealment, lack of information on follow-up and loss to follow-up. The reason for downgrading for indirectness was the publication year of the underlying studies; populations and interventions were considered inapplicable to current clinical practice.

Table 3

GRADE evidence profile SCIT

Quality assessment							Number of patients		Effect		Quality	Importance
Number of studies	Study design	Risk of bias	Inconsistency	Indirectness	Imprecision	Other considerations	SCIT	No intervention	Relative (95% CI)	Absolute (95% CI)
Asthma symptoms (assessed with: asthma symptom scores)
5*	RCT	Very serious†	Not serious	Serious‡	Not serious	None	136	286	–	Standardised mean difference −0.04 (95%CI −0.42 to 0.33)	⨁OOO VERY LOW	CRITICAL
Exacerbations (assessed with: symptomatic deterioration)
5*	RCT	Serious§	Not serious	Very serious¶	Not serious	None	64/253 (25.3%)	92/153 (60.1%)	Risk ratio 0.7 (0.31 to 0.72)	343 fewer per 1000 (95%CI −397 to −265)	⨁OOO VERY LOW	CRITICAL
Asthma control: not reported
–	–	–	–	–	–	–	–	–	–	–	–	CRITICAL
Quality of life: not reported
–	–	–	–	–	–	–	–	–	–	–	–	IMPORTANT
Lung function: not reported
–	–	–	–	–	–	–	–	–	–	–	–	IMPORTANT

*Studies in Cochrane review: Abramson et al16 and Tsai et al34.

†The underlying studies had a quite large risk of bias, due to lack of allocation concealment, problems with blinding and lack of information on follow-up (and lost to follow-up).

‡We downgraded for indirectness, because the included studies are quite old and maintenance medication may have changed probably; thus, study populations may differ from nowadays patients with moderate to severe asthma.

§We downgraded for risk of bias, because of problems with blinding and lost to follow-up.

¶We assessed very serious indirectness, because most included studies for this outcome are very old, and carried out before the ICS era; thus, patients nowadays differ from study populations.

GRADE, Grading of Recommendations Assessment, Development and Evaluation; ICS, inhaled corticosteroids; RCT, randomised controlled trial; SCIT, subcutaneous immunotherapy.

Critical outcomes

Asthma symptoms

Four small studies carried out in children only reported this outcome in the Cochrane review.16 We extracted these results from the Cochrane review and updated these with the results from Tsai et al.34 Results are presented in figure 2.

Figure 2

Meta-analysis of SCIT versus placebo, outcome asthma symptoms. IV, inverse variance; Random, random effect model; SCIT, subcutaneous immunotherapy; Std, standardised.

The meta-analysis showed no significant effect of SCIT on asthma symptoms.

Asthma exacerbations

Five studies (published 1961–1984) in the Cochrane review, carried out in children only, reported this outcome.16 No relevant studies of sufficient quality were published afterwards. Our meta-analysis included 253 patients on immunotherapy and 153 on placebo. The pooled risk ratio was 0.47 (95%CI 0.31 to 0.72), favouring immunotherapy (see figure 3). The absolute risk reduction was 35%, giving a number needed to treat of 3.

Figure 3

Meta-analysis of SCIT vs placebo, outcome asthma exacerbations. IV, inverse variance; SCIT, subcutaneous immunotherapy.

No studies reported results on the critical outcome asthma control.

Important outcomes

No studies reported results on quality of life or lung function (forced expiratory volume in 1 s (FEV1)).

Results of SLIT

Description of studies and quality of the evidence

We retrieved two SRs on SLIT in patients with asthma.29 32 The updated search in 2017 resulted in the addition of one RCT.44 The characteristics of the SRs are summarised in evidence table (see online supplementary table E2). The quality of the reviews was moderate; both had an AMSTAR score of 7/11. Weaknesses included the absence of an ‘a priori design’, exclusion of grey literature, not assessing the likelihood of publication bias and not mentioning conflicts of interest in one review,29 and the absence of an a priori design, no information about excluded studies, too firm conclusions compared with the weak evidence and not assessing the likelihood of publication bias in the other.32 One review included both children and adults, and patients with asthma and/or rhinitis.29 Because of the quality concerns of both existing SRs, we set out to perform a meta-analysis of the original studies that fulfilled our selection criteria. Jadad scores of selected studies, as well as an overview of the outcomes of those studies, are presented in table 4.24 29 32 44 54–67 Study characteristics are summarised in the evidence table (see online supplementary table E4).44 54 55 57 58 60 63–65 We rated the quality of evidence to be very low, due to a large risk of bias, imprecision and indirect evidence.

Table 4

Summary of quality and outcome measures of selected RCTs in reviews Calamita et al, Penagos et al and added Pham-Thi et al 29 32 44

Review	RCT	Eligible	Jadad score				Reported on asthma symptoms	Reported on exacerbations	Reported on asthma control	Reported on disease-specific quality of life	Reported on lung function (FEV1)
			Randomisation*	Blinding†	Withdrawals‡	Total
Calamita29	Bahceciler et al 54	Yes	1	1	1	3	+	–	–	–	–
	Hirsch et al 55	Yes	2	1	1	4	+	–	–	–	–
	Niu et al 24	No, conference abstract
	Novembre et al 56	No, Italian language
	Pajno et al 57	Yes	2	1	1	4	+	–	–	–	–
	Pajno et al 58§	Yes	2	1	1	4	–	–	–	–	+
	Rodriguez Santos et al 59	No, Spanish language
	Rolinck-Werninghaus et al 60	Yes	1	2	0	3	+	–	–	–	–
	Yuksel et al 61	No, Spanish language
Penagos32	Bahceciler et al 54	Overlap with Calamita
	Caffarelli et al 62	No, children with asthma not separately analysed
	Hirsch et al 55	Overlap with Calamita
	Ippoliti et al 63	Yes	1	1	0	2	+	–	–	–	+
	Niu et al 64	Yes	1	1	1	3	+	–	–	–	+
	Pajno et al 65	Yes	2	1	0	3	+	–	–	–	–
	Rolinck-Werninghaus et al 60	Overlap with Calamita
	Tari et al 66	No, Spanish language
	Vourdas et al 67	No, children with asthma not separately analysed
Pham-Thi44		Yes	2	1	1	4	+			+	+
Total							8	0	0	1	4

*1 point if randomisation is mentioned; 1 additional point if the method of randomisation is appropriate; −1 point if the method of randomization is inappropriate.

†1 point if blinding is mentioned; 1 additional point if the method of blinding is appropriate; −1 point if the method of blinding is inappropriate.

‡1 point if the number and the reasons for withdrawal in each group are stated.

§Same patients as Pajno et al 57. 

FEV1, forced expiratory volume in 1 s; RCT, randomised controlled trial.

Eight of the included studies reported on asthma symptoms. Different symptom scores were used, none of them standardised or validated. Clinical differences in asthma scores were not defined and most studies reported improvement in the treatment group as well in the control group. We were not able to compile a meta-analysis of the results of the individual studies, because of the use of various symptom scales in the included studies. Since studies did not report results in a clearly comparable way, reporting the results of the individual studies was considered unreliable.

Other critical outcomes

No studies reported results on the critical outcomes exacerbations and asthma control.

Quality of life

Pham-Thi et al published results on quality of life using Childhood Asthma Questionnaires.44 The authors reported a difference in severity between SLIT and placebo in the younger population (age 6–11 years), but not in older children (age 12–16 years). It is not stated whether this difference is clinically relevant.

Lung function

Four studies reported results on lung function (FEV1). One of the studies reported no numeric data on lung function.64 One study reported no variance (SD), and no comparison of the baseline data.63 The two remaining studies reported on FEV1 percentage predicted,58 and reported no significant differences between treatment groups, neither at baseline nor at follow-up.

Discussion

Summary of main results

Our GRADE SR showed no evidence of a significant difference in asthma symptoms between SCIT and placebo in children with allergic asthma, but some evidence for a significant and clinically relevant reduction in asthma exacerbations was found in SCIT-treated children. We have little confidence in the effect estimate, however, due to a large risk of bias and indirectness. Thus, the true effect of SCIT on exacerbations and asthma symptoms in the target population of interest is likely to be substantially different from the estimate of effect. There was absence of evidence on the effects of SCIT on lung function, asthma control and quality of life in children with allergic asthma. There was no evidence for a beneficial effect of SLIT in reducing asthma symptoms and exacerbations, quality of life and lung function in children with allergic asthma. Our review does not address the efficacy of immunotherapy in children regarding complaints of allergic rhinitis and/or conjunctivitis, without having asthma.

Quality of the evidence/GRADE methodology

The overall quality of the evidence about the effectiveness of SCIT and SLIT was very low. This implicates that our confidence in the effect estimates is very limited. The true effect of SCIT and SLIT on patient-relevant asthma outcomes in children with asthma may be substantially different from our estimates of the effect. We cannot conclude that the possible desirable effects of SCIT and SLIT outweigh the undesirable effects (eg, influence on quality of life, adverse events or increased resource expenditure), nor can we reject that hypothesis. Our concerns about the quality of evidence are based on (very) serious risk of bias and indirectness in the underlying primary studies. First, the quality of many studies had to be downgraded because of risk of bias due to lack of allocation concealment, lack of information on follow-up and loss to follow-up. Second, included studies were heterogeneous in the patients included and allergen extracts used, with different dosing regimens and duration being studied, targeting different inhaled allergens. We have concerns about the potential different responses and the generalisability of the evidence. Thirdly, and most importantly, for SCIT, the quality of the body of evidence was downgraded because of indirectness, since patients in the original studies long ago are likely to differ considerably from patients nowadays.

Fourth, different studies used variable definitions of asthma exacerbations. We had to use ‘worsening of asthma’, which may not represent real-life patient-relevant exacerbations. This may decrease the applicability of the evidence. In addition, there were no studies using the predefined important outcomes quality of life and asthma control.

Finally, and most importantly, we have concerns on comparability of patients. Several included studies dated from the 1980s or earlier, when allergic rhinitis treatment with selective antihistamines and nasal corticosteroids was not available. Against the background of the united airway concept, the comorbidity allergic rhinitis in patients in these studies cannot be compared with patients in clinical practice today.9 Similarly, widespread use of ICS was not introduced in childhood asthma treatment until the 1990s.68 Most studies on SCIT in children with asthma were published decades ago, during the pre-ICS era. Although SCIT appeared to be effective in some of the included studies,49 50 we cannot draw conclusions from these findings, because the patients in the described studies represent an incomparable group when compared with the child with asthma in contemporary clinical practice. Specifically, it is unclear whether the beneficial effects found in the SR of earlier studies are applicable to children with asthma treated according to contemporary guidelines with daily ICS controller therapy.16

In our opinion and that of others, GRADE approach is superior to former methods of SRs, because it focuses on predefined patient-relevant outcomes, predefined minimally clinical important differences and because it judges the complete body of evidence. One RCT among paediatricians studied the influence of different guideline grading systems on clinician’s decisions.69 GRADE showed the largest change in direction on the clinical decision. However, the added value of GRADE on guideline implementation or patient care has not been formally evaluated, GRADE approach is still rather complex for non-methodologists.

To formulate recommendations for clinical practice, not only the body of evidence concerning effectiveness of an intervention is important. Recommendations should balance the benefits and harms of the intervention of interest, and take patient preferences and resource use into account. Since (after critical evaluation) no benefits of SCIT and SLIT for children with asthma were determined, we consider it unlikely that the benefits will exceed the harms. Patient preferences were included in the formulation of our guideline recommendations.

Agreements and disagreements with other studies or reviews

Using GRADE and re-analysing data from children with allergic asthma only, we came to different conclusions on the effectiveness of SCIT and SLIT in children with asthma than the authors of the original SRs. We believe this highlights the importance of using GRADE methodology to systematically review evidence for patient-relevant outcomes, not focusing on levels of evidence, but on underlying study validity, precision, directness and applicability in current clinical practice. The 2009 position paper on SLIT describes history, use and applicability of this treatment for allergic rhinitis.70 It positions SLIT in children as a safe and useful therapy above and after more regular treatment for allergic rhinitis. Potential positive treatment outcome for allergic asthma is however mainly based on literature in adults. We show the lack of evidence and lack of applicability of treatment of immunotherapy for asthma in children. Since we have worries on the applicability of evidence in adults on children (who are still developing their immune system), we think further studies that compare immunotherapy for the contemporary treatment of asthma in children are urgently needed to fill in this gap.

Recently, a Cochrane SR on SLIT for asthma found a similar lack of data for important outcomes (eg, exacerbations, symptom scores and quality of life) as we did.71 Contrary to our study, the authors did no separate analysis for adults and children, and patients with asthma were not separately analysed from patients without asthma.

Conclusions

Focusing on predefined patient-relevant outcomes, and critically appraising the body of evidence using original studies and GRADE methodology, our SR on the effects of immunotherapy in children with asthma came to different conclusions than previous SRs. We believe that this underscores the importance of using GRADE methodology in systematically reviewing evidence.

We found absence of valid applicable evidence on improvement of clinically relevant asthma outcomes in children with allergic asthma using SCIT or SLIT. This absence of evidence is due to serious risk of bias, large clinical heterogeneity between studies and most importantly due to lack of applicability because studies were performed in the pre-ICS era.

Since the effect of immunotherapy added to contemporary asthma treatment with daily controller therapy is not clear, the drawbacks of immunotherapy should be considered carefully. SCIT is a complex and intensive form of treatment, associated with a (very) long duration of treatment, and considerable burden to the patient with (monthly) injections under adequate medical supervision due to potential (however rare) dangerous side effects, and may have relatively high costs and resource use. In SLIT the risk of serious side effects is considerably smaller, but the other drawbacks of immunotherapy apply equally to this treatment. In our opinion therefore, when balancing the absence of evidence on a clear beneficial effect of SCIT or SLIT on clinically relevant patient outcomes in children with asthma with the considerable burden and costs of SCIT and SLIT, we do not recommend this treatment to children with asthma until further high-quality evidence from well-designed RCTs in children comparing SCIT or SLIT to contemporary asthma treatment becomes available.