Evidence mapping and quality assessment of systematic reviews on therapeutic interventions for oral cancer.

Purpose: This evidence mapping aims to describe and assess the quality of available evidence in systematic reviews (SRs) on treatments for oral cancer. Materials and methods: We followed the methodology of Global Evidence Mapping. Searches in MEDLINE, EMBASE, Epistemonikos and The Cochrane Library were conducted to identify SRs on treatments for oral cancer. The methodological quality of SRs was assessed using the Assessing the Methodological Quality of Systematic Reviews-2 tool. We organized the results according to identified Population-Intervention-Comparison-Outcome (PICO) questions and presented the evidence mapping in tables and a bubble plot.
Results: Fifteen SRs met the eligibility criteria, including 118 individual reports, of which 55.1% were randomized controlled clinical trials. Ten SRs scored "Critically low" methodological quality. We extracted 30 PICOs focusing on interventions such as surgery, radiotherapy, chemotherapy, targeted therapy and immunotherapy; 18 PICOs were for resectable oral cancer, of which 8 were reported as beneficial. There were 12 PICOs for unresectable oral cancer, of which only 2 interventions were reported as beneficial.
Conclusion: There is limited available evidence on treatments for oral cancer. The methodological quality of most included SRs scored "Critically low". The main beneficial treatment reported by authors for patients with resectable oral cancer is surgery alone or in combination with radiotherapy or chemotherapy. Evidence about the benefits of the treatments for unresectable oral cancer is lacking. These findings highlight the need to address future research focused on new treatments and knowledge gaps in this field, and increased efforts are required to improve the methodology quality and reporting process of SRs on treatments for oral cancer.

Introduction

Oral cancer is one of the most prevalent cancers worldwide. Oral squamous cell carcinoma is the most common cancer occurring in the mouth, with an estimate of 300,000 new cases globally each year; only in the US, there were around 50,000 new cases expected in 2017.1 Oral cancer is posing an ever-increasing threat to global health and represents a growing burden on health services, which is a major problem in some parts of the world, especially in developing countries. Risk factors for oral cancer are frequently associated with lifestyle habits, such as smoking, alcohol abuse, poor nutrition and the use of betel quid.2

Unfortunately, the overall prognosis in these patients is low, with a 5-year survival rate of 50%, which has not changed over the last decades despite the advances in oncology treatment.3 Locoregionally advanced oral cavity cancers are aggressive tumors with high probabilities of relapse after definitive treatment with surgery or radiotherapy. Therefore, a multimodal approach, combining surgery and postoperative radiotherapy or chemoradiotherapy, has been suggested.4,5

Currently, there is a vast published scientific literature proposing a variety of treatment approaches for oral cancer. This fact may hinder knowing the effectiveness of such therapies and when they should be used. Furthermore, some research may be influenced by conflicts of interest. Thus, a critical analysis and a methodological quality assessment of the available evidence are required. In this sense, one of the options to organize and critically assess published studies is systematic reviews (SRs), which summarize the results of the evidence from health care primary studies in order to answer a specific research question.6

Likewise, there are new tools for evidence synthesis, such as evidence mapping, scoping reviews and rapid reviews, which have been developed to help clinicians, patients, researchers and other stakeholders to make evidence-based decisions.7 These new options are appropriate to address issues that may be too extensive for an SR.8

In 2007, the Global Evidence Mapping (GEM) initiative was established as a collaboration of clinical research and policy stakeholders to provide an overview of existing research about traumatic brain injury and spinal cord injury.9 Evidence mapping provides an innovative and visual approach to establish what we know and do not know about the effects of interventions on a thematic area. It can support evidence-informed decision making by facilitating evidence from existing SRs in a user-friendly format.7,10

The aim of this evidence mapping is to identify, describe and organize the current available evidence in SRs regarding therapeutic interventions for oral cancer. This approach purposes to determine the clinical questions assessed in the scientific literature and the corresponding quality of the supporting evidence, as well as to give general information about their claimed effectiveness. This information shall facilitate detecting research gaps and help stakeholders in the decision-making process.

Materials and methods

Study design

This evidence mapping adhered to the PRISMA-Extension for Scoping Reviews.11 It was carried out in accordance with the methodology proposed by GEM,9 adding some previously suggested tasks.12 All methods were specified a priori in a protocol (available on request).

Eligibility criteria

We included SRs published any year, with or without meta-analysis, assessing any therapeutic interventions in patients diagnosed with oral cavity cancer defined by the ICD for Oncology13 with codes C01–C02, C03, C04 and C05–C06. SRs related to head and neck cancer (C00–C14) with cases of oral cancer were included (as long as at least 50% of the participants had oral cavity cancer, or data for this cancer alone were available separately). Included SRs had conducted a comprehensive search in at least two different databases and reported the assessment of risks of bias or quality of their included studies.6 When several articles published by the same team were identified, we considered the most recent publication. Conversely, SRs about prognosis, safety or cost-effectiveness were excluded.

Search strategy

We searched for systematic literature in MEDLINE (via PubMed), EMBASE (via Ovid), Epistemonikos, The Cochrane Database of Systematic Reviews (via The Cochrane Library) and Database of Abstracts of Reviews of Effects and Health Technology Assessments (via The Cochrane Library). The latest search was conducted on October 25, 2018.

We used MeSH descriptor and free text terms for oral cavity cancer, such as “mouth neoplasms”, “oral carcinoma”, “oral cancer”, “oral tumor”, “buccal carcinoma”, and thesaurus terms when available. We adapted the search strategy in accordance with the specific characteristics of each database (Supplementary material 1) with no language restrictions. In addition, a cited reference search was conducted.

SR selection

We managed all retrieved titles and abstracts with the reference manager software EndNote® (Version X7, Thomson Reuters). After removing duplicates, two reviewers (MMA and JVAF) independently screened all titles/abstracts to exclude irrelevant studies. Then, full articles were obtained for a final decision. Detailed reasons for exclusion of any study considered relevant were clearly stated.

Methodological quality assessment

The report of methodological quality for each SR was assessed with the Assessing the Methodological Quality of Systematic Reviews (AMSTAR)-2 tool, a validated 16-item instrument for critically appraising SRs.14 It has an overall rating based on weaknesses in critical domains (items: 2, 4, 7, 9, 11, 13 and 15). Briefly, the overall confidence in the results of the SR is rated in the following four categories: “High”, no or one non-critical weakness; “Moderate”, more than one non-critical weakness; “Low”, one critical flaw with or without non-critical weaknesses and “Critically low”, more than one critical flaw with or without non-critical weaknesses.

Data extraction

General characteristics of the SR: authors, publication year, type of SR (with or without meta-analysis), objective, search date, design and number of included studies, and number of included participants.

Characteristics of research questions: we identified the research questions of each SR based on the aims stated by the authors, the eligibility criteria and the conclusions of the SR. The research questions were drawn using the PICO framework, which specifies the four key components of a well-defined therapeutic question: population, intervention, comparison and outcomes.6 A research question was considered if all the elements of the PICO framework were provided and a conclusion about the direction of the effect was described anywhere in the SR. We extracted details on the population characteristics (eg, adult population, type of cancer, stage and cancer location), the intervention and comparator (eg, type of intervention and comparison broadly categorized as chemotherapy, surgery, radiotherapy, immunotherapy and targeted therapy) and the outcomes.

The conclusions of the SR authors were classified into five categories following previously reported criteria.12 Briefly, the “beneficial” category was used if there were conclusions with evidence of a positive effect and SR authors used a language clearly indicative of a beneficial effect without major concerns regarding the existing evidence. The “probably beneficial” category was used for those conclusions where the evidence base was insufficient to draw firm conclusions despite the positive treatment effect and the reporting suggested a benefit. The “harmful” category was used when the reporting of the conclusions was clearly indicative of a harmful effect. The “no differential effect” category was used for conclusions that provided evidence for no difference between the intervention and the comparator. Finally, the “inconclusive” category was used if the direction of results was different across or within reviews due to conflicting results or limitations of individual studies.

Two authors (MMA and JVAF) independently performed all processes of study selection, methodological quality assessment and data extraction. If there were any disagreements, these were resolved by consensus, and when necessary, an additional reviewer (GUC) participated in the discussion until an agreement was reached. If needed, we contacted the SR authors for clarification or to obtain missing information.

Evidence mapping presentation

We presented the evidence mapping on tables describing the characteristics of the included SRs and on other tables providing the characteristics of all identified PICOs. We performed a narrative description of the PICOs stratified by disease severity (resectable and nonresectable cancers). In addition, we designed a bubble plot where each bubble represents one SR. This chart displays information in three dimensions: 1) the rating of authors’ conclusions represented in the x-axis as “beneficial”, “probably beneficial”, “harmful”, “no differential effect” and “inconclusive”; 2) AMSTAR-2 assessment in y-axis and 3) the number of primary studies included in the SR, which is shown in each bubble and is represented by the bubble size. Each bubble also represents a pie showing the proportion of randomized controlled trials (RCTs) included using a black bold line.

Results

Studies selected

The research yielded 2,547 records after removing duplicates. After title and abstract screening, 127 articles were obtained for final full-text review; 15 SRs15–29 met the eligibility criteria (Figure 1). The list of excluded studies along with exclusion rationale is available in Supplementary material 2.

Figure 1

PRISMA flow diagram detailing the selection process.

Characteristics of the included SRs

Thirteen SRs15,16,18–27,29 included a meta-analysis, and all SRs15–29 were published in English between 2010 and 2018. Nine SRs15,19,22–26,28,29 had focused on oral cavity cancer exclusively, whereas other six SRs16–18,20,21,27 had focused on head and neck cancers, with the oropharyngeal cancer being the most frequent among them. Eight SRs15,17,19,22,24,27–29 assessed surgical interventions, three SRs16,21,25 assessed radiotherapy, three SRs20,23,26 assessed chemotherapy and one SR18 assessed targeted therapy and immunotherapy. SRs included primary studies conducted from 1969 to 2015; the number of patients included in each SR ranged from 309 to 16,767 adult individuals. This evidence mapping included 118 reports of primary studies (Supplementary material 3) with 10,423 participants after considering the overlapping or duplication of studies. These studies included 65 (55.1%) RCTs (n=5,724), 48 (40.7%) observational studies (n=42,396) and 5 (4.2%) controlled clinical trials (n=460). Table 1 shows the characteristics of included SRs.

Table 1

Characteristics of the included SRs

Author and year	Study design	Search date	Objective	Design and number of included studies	Participants (n)	AMSTAR-2 score
Anderson et al, 201515	SRM	Not given	To determine whether a wider pathological margin reduces local recurrence rates in patients with OSCC treated by primary surgery without adjuvant therapy	Cohort: 5	539	Critically low
Baujat et al, 201016	SRM	August 2010	To study the effects of altered fractionation radiotherapy vs conventional radiotherapy on overall survival rates	RCT: 15	6,515	Low
Bessell et al, 201117	SR	February 2011	To determine which surgical treatment modalities for oral cavity and oropharyngeal cancers result in increased overall survival, disease-free survival, progression-free survival and reduced recurrence	RCT: 7	669	High
Chan et al, 201518	SRM	February 2015	To assess the effects of molecularly targeted therapies and immunotherapies, in addition to standard therapies, for the treatment of oral cavity or oropharyngeal cancers	RCT: 12	2,488	High
Ding et al, 201819	SRM	November– December 2017	To compare elective neck dissection with observation or therapeutic neck dissection specifically in patients with early-stage OSCC and clinically N0 neck to explore the potential benefits of elective neck dissection	RCT: 5 Case– control: 1	865	Critically low
Furness et al, 201120	SRM	December 2010	To determine whether chemotherapy, in addition to radiotherapy and/or surgery for oral cavity and oropharyngeal cancer, results in increased overall survival, disease-free survival, progression-free survival, locoregional control and reduced recurrence	RCT: 89	16,767	Low
Glenny et al, 201021	SRM	July 2010	To determine which radiotherapy regimens for oral cavity and oropharyngeal cancers result in increased overall survival, disease-free survival, progression-free survival and locoregional control	RCT: 30	6,536	Low
Gou et al, 201822	SRM	May 2016	To explore the survival rate and disease control in patients with histological evidence of bone invasion and to compare the differences in survival rate and disease control between patients who underwent marginal mandibular resection and those who underwent segmental mandibulectomy	Cohort: 15	1,672	Critically low
Lau et al, 201623	SRM	March 2016	To analyze the effect of induction chemotherapy in OSCC treatment by performing an updated SR and cumulative meta-analysis	RCT: 27	2,872	Critically low
Liang et al, 201524	SRM	April 2015	To access the feasibility of selective neck dissection in oral cancer patients with positive neck nodes	Cohort: 5	443	Critically low
Liu et al, 201325	SRM	June 2012	To compare the efficacy and safety of high-dose rate and low-dose rate brachytherapy in treating early-stage oral cancer	RCT: 1 Controlled trial: 5	607	Critically low
Marta et al, 201526	SRM	January 2015	To assess the effectiveness and safety of induction chemotherapy prior to surgery for untreated OSCC patients	RCT: 2	451	Critically low
Pang et al, 201627	SRM	September 2016	To compare the prognoses outcomes of mandibular preservation method and the mandibulotomy approach in oral and oropharyngeal cancer patients	Cohort: 6	309	Critically low
Tang and Leung, 201628	SR	February 2015	To answer the clinical question, “When should elective neck dissection be performed in maxillary gingival and alveolar squamous cell carcinoma with negative neck nodes?”	Cohort: 10	506	Critically low
Wang et al, 201829	SRM	March 2017	To perform a meta-analysis to compare discontinuous neck dissection with incontinuity neck dissection as a treatment modality for SCC of the tongue and floor of the mouth	Cohort: 8	796	Critically low

Abbreviations: AMSTAR-2, Assessing the Methodological Quality of Systematic Reviews-2; OSCC, oral squamous cell carcinoma; RCT, randomized controlled trial; SCC, squamous cell carcinoma; SR, systematic review; SRM: systematic review with meta-analysis.

The methodological quality of SRs

Ten SRs15,19,22–29 scored “Critically low”, three SRs16,20,21 scored “Low” and only two SRs17,18 scored “High” methodological quality, according to the AMSTAR-2 critical appraisal criteria (Figure 2). The SRs were downgraded mainly because the SR authors did not explain their selection of the study designs for inclusion in the review,16–23,25–29 sources of funding for the included studies were not clearly stated,15,16,19,22–29 there was no reference to a protocol,15,19,22–29 and the list of excluded studies was not provided.15,19,22,24–27,29

Figure 2

Methodological quality of the included systematic reviews.

Abbreviation: AMSTAR-2, Assesing the Methodological Quality of Systematic Reviews-2; PICO, Population–Intervention–Comparison–Outcome.

Characteristics of PICOs from SRs

The evidence mapping of the therapeutic interventions for oral cancer is presented in Figure 3; 30 PICOs were extracted, which focused on two population groups: patients with resectable oral cancers and patients with unresectable cancer.

Figure 3

Evidence mapping of the therapeutic interventions for oral cancer.

Notes: (A) Interventions for resectable oral cancer. (B) Interventions for unresectable oral cancer. Bubble plots where each bubble represents one SR. The number of individual studies included in the SR is shown in each bubble and is represented by the bubble size. Each bubble also represents a pie showing the proportion of randomized controlled trials included with a black bold line. *Two PICOs included this comparison, but the intervention was reported as “beneficial” only in the PICO for patients with positive neck nodes level II. The number of individual studies included in the SR is shown in each bubble and is represented by the bubble size.

Abbreviations: 5-FU, 5-fluorouracil; BCG-CWP, Bacillus Calmette-Guérin-cell wall preparation; CCRT, concomitant chemo-radiotherapy; CT, chemotherapy; END, elective neck dissection; ICT, induction chemotherapy; MTX, methotrexate; PICO, Population–Intervention–Comparison–Outcome; RT, radiotherapy; SR, systematic review.

Patients with resectable oral cancers

Thirteen SRs15,17–24,26–29 were conducted including 18 PICOs. Eight PICOs evaluated surgical interventions,17,19,22,24,27–29 five PICOs assessed chemotherapy,20,23,26 three PICOs assessed radiotherapy17,21 and two PICOs assessed immunotherapy. Eight PICOs were reported as “beneficial”, one PICO as “probably beneficial”, eight PICOs as “no differential effect” and one PICO was reported as “inconclusive” (Table 2).

Table 2

Therapeutic interventions for resectable oral cancer by PICO framework

Systematic reviews	Population	Intervention	Comparison	Outcomes	Primary studies		Conclusion
					RCT	Observational
Anderson et al, 201515	Primary oral cavity cancers	Wider pathological margin (≥5 mm)	Narrow pathological margin (<5 mm)	Local recurrence		Sadeghi 1986, Loree 1990, Hicks 1997, Sieczka 2001	Beneficial
Anderson et al, 201515	Primary oral cavity cancers	Wider pathological margin (≥10 mm)	Narrow pathological margin (<10 mm)	Local recurrence		Hicks 1998	Inconclusive
Bessell et al, 201117Ding et al, 201819	Primary oral cavity cancers, clinically negative neck nodes	Elective neck dissection	Observation/therapeutic delayed neck dissection	Overall survivalDisease-free survivalLocoregional controlRegional recurrencesDeath related to recurrencesOccult lymph node metastasisTotal number of recurrences	D Cruz 2015, Fakih 1989, Kligerman 1994, Vanderbrouck 1980, Yuen 2009	Mirea 2014	Probably beneficial
Bessell et al, 201117Liang et al, 201524	Primary oral cavity cancers, clinically positive neck nodes	Selective neck dissection	Comprehensive neck dissection	Regional recurrenceDisease-specific deathOverall deathOverall survivalDisease-free survivalDisease recurrence	Bier 1994, BHNCSG 1998	Schiff 2005, Patel 2008, Yanai 2011, Shin 2013, Feng 2014	No differential effect
Bessell et al, 201117	Head and neck cancersa, stage T2–T4, N0–N2, M0	RT+surgery	RT alone	Total mortality	Robertson 1998		Beneficial
Chan et al, 201518	Head and neck cancersa	Recombinant interleukin-2 + surgery	Surgery alone	Overall survivalDisease-free survivalAdverse effects	De Stefani 2002		Beneficial
Chan et al, 201518	Head and neck cancersa	Pretreatment with BCG-CWP followed by surgery	Surgery alone	Overall survivalAdverse effects	Bier 1981		No differential effect
Furness et al, 201120	Primary oral cavity cancers	Bleomycin + vincristine + surgery	Surgery	Total mortalityDisease-free survivalOverall survival	Luboinski 1985, Richard 1991		Beneficial
Furness et al, 201120	Primary oral cavity cancers	Post-surgery CT (MTX)	Postoperative RT	Total mortalityDisease-free survival	Bitter 1979		No differential effect
Furness et al, 201120	Primary oral cavity cancers	Post-surgery CT (MTX)	Surgery alone	Disease-free survivalDisease recurrenceTotal mortality	Rao 1994		Beneficial
Furness et al, 201120Lau et al, 201623Marta et al, 201526	Primary oral cavity cancers, stage T1–T4	ICT+surgery±RT	Surgery±RT	Total mortalityLocoregional recurrenceDisease-free survivalOverall survivalDistant metastasis	Szpirglas 1979, Holoye 1985, Luboinski 1985, Pearlman 1985, HNCProg 1987, Toohill 1987, Schuller 1988, Richard 1991, Depont 1993, Di Blasio 1994, Martin 1994, Paccagnella 1994, Volling 1994, Dalley 1995, Maipang 1995, Hasegawa 1996, Szabo 1999, Bossi 2014/Licitra 2003, Zhong 2015/Zhong 2013		No differential effect
Glenny et al, 201021	Head and neck cancersa	Pre-operative RT	Surgery alone	Total mortalityLocoregional control	Ketcham 1969		No differential effect
Glenny et al, 201021	Primary oral cavity cancers, stage T2, N0–N2, M0	Preoperative and postoperative RT	Postoperative RT alone	Total mortalityLocoregional controlDisease-free survival	Bergermann 1992		No differential effect
Gou et al, 201822	Primary oral cavity cancers	Marginal mandibulectomy	Segmental mandibulectomy	Disease-free survivalOverall survivalLocal control		Ash 2000, Totsuka 1991, Ord 1997, Munoz Guerra 2003, O’Brien 2003, Patel 2008, Shaw 2004, Soo 1988, Wald 1983, Werning 2001, Pascoal 2007, Nie 2000, Barttelbort 1987, Dubner 1993, Overholt 1996	No differential effect
Marta et al, 201526	Primary oral cavity cancers, positive neck nodes level II	ICT+surgery±RT	Surgery±RT	Overall survivalLocoregional recurrence	Zhong 2013, Bossi 2014/Licitra 2003		Beneficial
Pang et al, 201627	Primary oral cavity cancers	Mandibular preservation	Mandibulotomy	Surgical marginsSurvival rateRecurrence rateFistula formationFunctionality		Devine 2001, Song 2013, Li 2014, Li 2015	No differential effect
Tang and Leung, 201628	Primary oral cavity cancers, stage T1– T4, with negative neck nodes	Elective neck dissection	Not receiving an elective neck dissection	Cervical metastasis rateOccult cervical metastasisOverall 5-year survival rate		Simental 2006, Montes 2008, Mourouzis 2010, Lubek 2011, Morris 2011, Poeschl 2012, Feng 2013, Brown 2013, Yang 2014, Philip 2014	Beneficial
Wang et al, 201829	Primary oral cavity cancers	Incontinuity neck dissection	Discontinuous neck dissection	Local recurrence		Spiro 1973, Leemans 1991, Tesseroli 2006, Lim 2007, Feng 2015, Hu 2005, Zhang 2007, Guo 2009	Beneficial

Note:

At least 50% of the participants had oral cavity cancer.

Abbreviations: BCG-CWP, Bacillus Calmette-Guérin-cell wall preparation; CT, chemotherapy; ICT, induction chemotherapy; MTX, methotrexate; PICO, Population–Intervention–Comparison–Outcome; RCT, randomized controlled trial; RT, radiotherapy.

Interventions reported as “beneficial” were as follows: 1) the elective neck dissection was better than no elective neck dissection in patients with negative neck nodes in terms of cervical metastasis rate, overall 5-year survival rate and occult cervical metastasis;28 2) the incontinuity neck dissection was better than discontinuous neck dissection in terms of local recurrence;29 3) a wider pathological margin (≥5 mm) was better than a narrow pathological margin (<5 mm) in terms of local recurrence rates in patients with oral squamous cell carcinoma treated by primary surgery without adjuvant therapy;15 4) radiotherapy combined with surgery was better than radiotherapy alone in terms of total mortality;17 5) the use of intra-arterial bleomycin and vincristine combined with surgery was better than surgery alone in terms of overall survival;20 6) post-surgery chemotherapy using methotrexate as chemotherapy drug was better than surgery alone in terms of total mortality;20 7) induction chemotherapy followed by surgery with or without radiotherapy was better than surgery with or without radiotherapy in patients with positive nodules classified as level II, in terms of overall survival26 and 8) the use of recombinant interleukin-2 plus surgery was better than surgery alone in terms of overall survival.18

Patients with unresectable cancer

Six SRs16,18,20,21,23,25 were conducted including 12 PICOs. Nine PICOs assessed chemotherapy,20,23 two PICOs assessed radiotherapy16,21,25 and one PICO assessed targeted therapy.18 Two PICOs were reported as “beneficial”, two PICOs as “probably beneficial” and eight PICOs were reported as “no differential effect” (Table 3).

Table 3

Therapeutic interventions for unresectable oral cancer by PICO framework

Systematic reviews	Population	Intervention	Comparison	Outcomes	Primary studies		Conclusion
					RCT	Controlled trial
Baujat et al, 201016	Primary oral cavity cancers, M0	Altered fractionated RT	Conventional RT	Survival	Marcial 1987, Dische 1997, Horiot 1997, Jackson 1997, Dobrowsky 2000, Fu 2000, Skladowski 2000, Poulsen 2001, Overgaard 2003, Bourhis 2006		Beneficial
Chan et al, 201518	Primary oral cavity cancers	Gefitinib +RT	RT alone	Disease-free survivalAdverse effects	Singh 2013		No differential effect
Furness et al, 201120	Primary oral cavity and oropharyngeal cancersa	MTX+RT	RT alone	Total mortality	Nervi 1978		Probably beneficial
Furness et al, 201120	Primary oral cavity cancers	Cisplatin +5-FU+RT	RT alone	Total mortalityOverall survivalDisease-free survivalRecurrent disease	Lewin 1997, Licitra 2003		No differential effect
Furness et al, 201120	Primary oral cavity and oropharynx cancersa	Bleomycin+RT	RT alone	Total mortalityLocoregional controlDisease-free survival	Shanta 1980, Morita 1980, Parvinen 1985		No differential effect
Furness et al, 201120	Primary oral cavity and oropharynx cancersa	Pepleomycin+RT	RT alone	Locoregional control	Krishnamurthi 1990		Probably beneficial
Furness et al, 201120	Primary oral cavity cancers	Bleomycin	MTX	Tumor regression	Molinari 1982		Beneficial
Furness et al, 201120	Primary oral cavity cancers	Platin+5-FU	Platin + vinorelbine	MortalityDisease-free survivalToxicity	Segura 2002		No differential effect
Furness et al, 201120	Primary oral cavity cancers, stage T2–T4	Induction simultaneous MTX+5-FU	Sequential MTX+5-FU	Total mortality	Browman 1986		No differential effect
Glenny et al, 201021Liu et al, 201325	Primary oral cavity cancers, stage T1–T3, negative neck nodes	High-dose rate brachytherapy	Low-dose rate brachytherapy	Local recurrenceComplicationsTotal mortalityCause-specific survival	Inoue 2001	Inoue 1998, Kakimoto 2003, Umeda 2005, Arrate 2010, Ghadja 2012	No differential effect
Lau et al, 201623	Primary oral cavity cancers	ICT+RT	RT alone	Overall survivalDisease-free survivalLocoregional recurrenceDistant metastasis	Richard 1974, Fazekas 1980, Pearlman 1985, Carugati 1988, Szpirglas 1988,Brunin 1992, Jaulerry 1992, Mazeron 1992, Salvajoli 1992, Martin 1994, Paccagnella 1994, Lewin 1997		No differential effect
Lau et al, 201623	Primary oral cavity cancers	ICT+CCRT	CCRT	Overall survivalDisease-free survivalLocoregional recurrenceDistant metastasisAdverse effects	Chhatui 2015		No differential effect

Note:

At least 50% of the participants had oral cavity cancer.

Abbreviations: 5-FU, 5-fluorouracil; CCRT, concomitant chemo-radiotherapy; ICT, induction chemotherapy; MTX, methotrexate; PICO, Population–Intervention–Comparison–Outcome; RCT, randomized controlled trial; RT, radiotherapy.

The interventions reported as “beneficial” were: 1) altered fractionation radiotherapy was better than conventional radiotherapy in terms of overall survival16 and 2) bleomycin was better than methotrexate in terms of tumor regression.20

Discussion

Evidence mapping is a relatively new tool used to summarize available scientific evidence about a specific topic. However, although there is no standard definition of it or consensus about its components or the methods to be used, there are common characteristics for these types of review.7 In general, it includes a systematic search covering a broad field to identify gaps in knowledge and/or future research needs. It also presents results in a user-friendly format, often a visual figure or graph, or a searchable database.7 Evidence mapping can produce an extensive list of prioritized research questions in a topic area, even in the absence of study retrieval and data extraction. It is a potential springboard for research, policy development and research funding.9

This evidence mapping may be the first one about therapeutic interventions for oral cancer because we found no previous reports. We decided to use this methodology developed by GEM initiative since it is rational and systematic.9 Recently, a report stated that most of the documents that met the common characteristics of evidence mapping referenced this methodology.7 The referenced methodology includes three core tasks: setting the boundaries and context of the topic area in question, searching and selecting relevant studies and reporting on search results and study characteristics.9 Moreover, we added two uncommon components in evidence mapping, which were previously reported: the methodological quality assessment of SRs and the classification of the conclusions as beneficial, probably beneficial, no differential effect, inconclusive or harmful according to the results reported by authors.12 It has been suggested that this approach allows locating the results of one study in relation to other studies with the same comparison on a bubble plot, obtaining a broader outlook of the available evidence and its quality.12

The results of this evidence mapping show that in line with available evidence, there is a sprinkling of SRs about therapeutic interventions for oral cancer, since only 15 SRs focusing on different therapies met the criteria. Moreover, most SRs included a small number of primary studies; thus, it may suggest that the evidence of this issue is limited. However, we wish to highlight that most of the primary studies included in this evidence mapping were RCTs, which is an aspect with clinical relevance because experimental studies are the best design to evaluate the efficacy of new therapeutic options.30 We also highlight that no comparison was reported as “harmful”, which is probably because most RCTs with negative conclusions are seldom published.31

According to methodological quality assessment, most of the SRs scored “Critically low” methodology quality with the AMSTAR-2 tool. This indicates that there is room for a potential improvement of the quality of SRs in this field. Among the domains to improve are the inclusion of an explicit statement indicating that the SR methods were established prior to the conduct of the SR, as well as the inclusion of a report justifying any significant deviations from the protocol; the explanation of the selection of the study designs for inclusion in the SR; the provision of the list of excluded studies and justifying the exclusions; and the reporting of the conflicts of interests, indicating the source of funding or support for each of the included studies. Although the methodological quality assessment is not a core task of an evidence mapping, it has been suggested that any type of review should include this process in order to evaluate the consistency of its conclusions.6,12

In this evidence mapping, the main therapeutic interventions reported by the authors as beneficial for patients with resectable oral cancer are surgery alone or in combination with radiotherapy or chemotherapy, depending on the extent of the disease. These results were based on SRs15,17,18,20,26,28,29 with “Critically low” to “High” methodological quality evaluated with AMSTAR-2 tool. However, these reports should be taken with caution because some SRs15,28,29 only included observational studies. Moreover, despite the fact that some interventions reported by the authors as “beneficial” were based on RCTs,32–39 the majority of these comparisons included just one RCT,32,35,36 some of which had a small sample size.

There were fewer comparisons for patients with unresectable oral cancer than for those with resectable oral cancer. Only two interventions were reported by the authors as beneficial; these found altered fractionated radiotherapy to be superior to other forms of radiotherapy16 and to the use of bleomycin as a chemotherapy drug.20 We wish to emphasize that all comparisons for this population were based on SRs16,18,20,21,23,25 including only RCTs and controlled clinical trials. Nevertheless, these results should be placed in context. Firstly, despite the fact that altered fractionated radiotherapy was reported as a beneficial treatment for oral cancer, there is a previous report40 of the same SR16 that shows the same outcomes, but there are some numeric inconsistencies in the results between these reports, even though the same authors included the same studies in the analysis. For these reasons, we contacted the authors and they clarified that the latest report had probably reclassified patients and provided the most accurate estimates. Secondly, recommending the use of bleomycin was based on only one single RCT41 published long time ago. Thus, nowadays, it is likely that there are other options for chemotherapy. For example, 5-fluoro-uracil, cisplatin, carboplatin, paclitaxel and docetaxel are among the chemotherapy drugs most often used for oral and oropharyngeal cancers; these may be used alone or combined with other drugs.42,43

We were able to identify some research gaps on this topic such as targeted therapy, since just only one RCT44 addressing this topic was included in one SR.18 Moreover, despite a sharp increase in research into molecularly targeted therapies and a rapid expansion in the number of trials assessing new targeted therapies, their value for treating oral cancers remains unclear. The advantage that these therapies may have over conventional chemotherapy is that rather than affecting both healthy and cancerous cells, they target only cancer cells.18 Recently, de Felice and Guerrero Urbano 45 reviewed the published clinical trials about a specific targeted therapy and suggested that it could become a “central player” in head and neck cancers as it offers a potential therapeutic opportunity. Likewise, the same authors claimed that despite the ongoing trials, clinical data are lacking.

This evidence mapping can be used to help with the interpretation of published research syntheses, such as SRs and meta-analyses, and it can also be used as a tool to engage stakeholders. Similarly, it can be used to address future research projects focused on knowledge gaps identified with this evidence mapping, as well as to conduct SRs and RCTs focused on new therapeutic interventions for oral cancer. It is useful to clarify that this evidence mapping does not intend to replace any clinical protocol or guideline. Its aim is to describe the available evidence on therapeutic interventions for oral cancers; thus, any recommendations and practice points should be considered in the context of clinical judgment for each patient, the available alternatives and their risk/benefit ratio, the available resources and other contextual factors.46

Among the strengths of this study, we highlight that a sensitive search strategy was performed, so it is unlikely that any relevant studies were missed. Likewise, two reviewers independently conducted the whole processes of selection, methodological quality assessment and data extraction from the included SRs. All these processes provide reasonable confidence in these results.

Certain limitations in this evidence mapping should be taken into account. Firstly, there were limited SRs comparing therapeutic interventions for oral cancer, and some of them included only observational studies; thus, some bias due to confounding factor may exist in these studies. Secondly, since some SRs had methodological limitations, their conclusions can be subject to bias; therefore, their conclusions regarding the effectiveness of the different interventions could be invalid. However, this is thoroughly reported in our results, so each conclusion can be assessed by the reader including its limitation. Other limitation is the language barrier; all the included SRs were published in English, which eliminated the inclusion into this mapping of available evidence published in any other language.

Conclusion

There is limited available evidence about therapeutic interventions for oral cancer. The methodological quality of most included SRs in this mapping scored “Critically low” quality with AMSTAR-2 tool. The main beneficial therapeutic interventions reported by authors for patients with resectable oral cancer are surgery alone or in combination with radiotherapy or chemotherapy. Evidence for the benefits of treatments for unresectable oral cancer is lacking. These findings highlight the need to address future research focused on new therapeutic interventions and knowledge gaps in this field, as well as increased efforts are required to improve the methodology quality and reporting process of SRs on treatments for oral cancer. The evidence mapping is an adequate and reliable methodology to identify the current available evidence about therapeutic interventions.

Data sharing statement

All data generated or analyzed during this study are included in this published article and its Supplementary materials.