Understanding the use of evidence in the WHO Classification of Tumours: a protocol for an evidence gap map of the classification of tumours of the lung.

INTRODUCTION: There are gaps in the evidence base of tumour classification despite being essential for cancer diagnosis, treatment and patient care. The WHO in charge of the production of an updated international classification, the WHO Classification of Tumours (WCT), aims to adapt evidence gap map (EGM) methodology to inform future editions of the WCT, by providing a visual summary of the existing evidence. METHODS AND ANALYSIS: Bibliographical references used in the WCT fifth edition of Tumours of the Lung (Thoracic Tumours volume) will be used as search results of a literature search. A descriptive analysis of the cited evidence for tumour types and descriptors will be drafted and plotted in EPPI-Reviewer to develop a visual evidence map. The resulting EGM will reflect the number of cited studies in the size of the spheres, and the level of evidence by applying a four-colour code (red=low level evidence, orange=moderate level, green=high level and blue=unclassifiable). Overview of the findings will be provided in narrative form and a report will discuss the overall stage of cited research in the WCT and will include analysis of gaps, under-researched categories of tumour descriptors and pockets of low-level evidence. ETHICS AND DISSEMINATION: No ethics approval will be required as this is a study of previously published material. Findings of the EGM will be published and used to guide editors, stakeholders and researchers for future research planning and related decision-making, especially for the development of future editions of the WCT. PROSPERO REGISTRATION NUMBER: CRD42022302327. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This evidence gap map (EGM) protocol follows methodological approaches applied by the Campbell Collaboration and the International Initiative for Impact Evaluation, adapting them for the first time to the biomedical field.

This EGM will provide an overview of the current use of evidence to inform decisions when classifying tumours.

This EGM will guide editors, researchers, pathologists, stakeholders and other related specialists in producing the evidence for the classification of cancer worldwide.

Systematic reviews will be specially highlighted in this EGM

As the articles written in languages other than the English, French, German or Spanish will not be included, a risk exists to miss relevant articles written in other languages.

Introduction

The challenge of providing a reliable classification of cancer

Classification is essential for cancer diagnosis, and underpins treatment and care of patients with cancer, as well as research into cancer epidemiology, prevention, diagnosis and treatment. The WHO provides an international classification with diagnostic criteria for tumours, which is regularly updated (Thoracic fifth edition published in 2021).1 This WHO Classification of Tumours (WCT), also known as the WHO Blue Books, is an essential resource for pathologists and cancer researchers worldwide, and regarded as the gold standard for the diagnosis of tumours. The WCT covers all organ sites in one edition, providing evidence synthesis of clinicopathological criteria for each tumour type and definitions for the International Classification of Diseases (ICD-O and ICD-11).2

Traditionally, tumour classification has been based on consensus of histopathological criteria,3 4 but new technologies (as large-scale molecular profiling of tumour DNA, RNA, proteins and epigenetic features or artificial intelligence and digital pathology) are transforming the field rapidly, or will in the near future, and it has become increasingly clear that the traditional approach to tumour classification is insufficient to assess all relevant evidence.5 The understanding of cancer at a molecular level, together with advances in radiology, prognostication and several other fields have reached a point where multiple dimensions of information must be taken into account for the classification of each single tumour. This multidimensional model of information showing the wide range of research that provides evidence to inform decisions of the WCT, has been incorporated as a framework to the editorial process of the 5th edition WHO Blue Books (figure 1) and facilitates the structured reporting throughout the classification. This model recognises the relevance of information from numerous fields when classifying tumours, and defines the challenge this represents for the editorial board. Differences in reporting and publishing standards, as well as an important publication overload in each of these fields, makes the accurate and timely inclusion of new evidence in a structured assessment to inform classification of a tumour difficult.5–7

Figure 1

Multidimensional nature of cancer classification. ICD, International Classification of Diseases.

The WCT is an important route for the incorporation of non-commercial knowledge from diagnostic research into patient management, providing international standards and clinical guidance for pathologists and cancer researchers worldwide. Decisions for such a global reference tool need to be informed by the best available evidence, and the risk of incorporating misinformation into the clinical decision pathways has to be minimised. The current editorial process includes an editorial board to revise and correct evident misconceptions and variations in the evidence summaries provided by the contributing authors. Currently, contributing subject experts are asked to perform structured, but non-systematic, literature searches to retrieve the evidence, and to evaluate and summarise it in accordance with their best knowledge. Authors are mostly pathologists recognised as experts in their subspecialty, but very few have any formal training in or experience of the principles of evidence-based medicine (EBM) or clinical epidemiology. Nonetheless, these pathologists are tasked with making an unbiased appraisal of a variety of detailed and complex types of evidence, often within short timeframes. The board must agree on core criteria in every new edition, and the final classification depends on the successful integration of the members’ expertise (clinicopathological experience), with the best available evidence (retrieved from the literature) and the patients’ values—what many would consider the right approach for evidence-based practice. However, additional determinants such as language, geographical differences in clinical diagnostic practice or the dynamics of the consensus meetings, also influence the final outcome. Strengthening the evidence-base of the editorial process is therefore key to maintaining the reliability of the WCT and improve its impact in future cancer management.

The WCT faces several challenges, including the already mentioned information overload, with a vast number of publications from multiple fields providing potentially relevant information, but using very heterogenous methods of collecting and reporting on the same outcomes. Information is dispersed through different databases which makes locating all evidence difficult and once found, a lack of recommendations for the update of the reviewed evidence makes it hard to identify the latest advances supported by adequate evidence. A good starting point to address these problems would be to provide an overview of existing evidence and how it is used,8 9 in order to guide scientists in their efforts to bridge gaps.10 11 No reliable summary of the available evidence, showing uses and gaps has been published so far and the application of reproducible methods such as systematic reviews is uncommon in this field. The WCT has been exploring potential solutions,3 12–19 and preliminary results of an ongoing evaluation have shown that more than 300 parameters are described as ‘unknown’ in the WHO Blue Books, and that often insufficient or inadequate references are provided. Thus, a reliable estimation of the evidence available for the WCT, describing evidence levels of cited literature and gaps is needed. This would allow evaluation of the evidence-base of the classification and facilitate adapted recommendations.

We believe that the adaptation of the evidence gap map (EGM) methodology that is increasingly applied in public health and social sciences to inform policymakers,20 will allow us to provide a visual summary of the evidence relevant to the WCT. Such an evidence map will improve comprehension of the challenges in the different research fields and promote evidence-based methods in diagnostic research and hence, pathology and tumour classification. At the same time, such a map would serve as a unique tool for contributing authors and editors of the WCT, guiding them to apply best available evidence.

EGMs of the WCT are not only important to support the continuous improvement of the accuracy of the classification, but will potentially have wide-reaching impact on pathological diagnosis over time, by promoting evidence-based approaches and endorsing standardisation of reporting in international research. These EGM’s will also serve as the basis for focused efforts in research, guiding future projects and maximising its impact. The ultimate beneficiary will be the patient who is the centre of all cancer diagnoses and treatment.

Exploring the application of EGM methodology to the WHO Classification of Tumours: the scope of our mapping exercise

Major recognition of the multiple dimensions that determine WCT will be achieved by developing new conceptual frameworks, and along with them, new thinking in how to improve the evidence-base of our decisions in cancer classification and the field of pathology. Innovations that may emerge for the 6th edition of the WCT should include tools to assist in the timely assessment of published evidence, new methods to facilitate the synthesis of evidence, as well as the development of standards and recommendations to apply them to different fields. In order to develop such improvements, we need to analyse the current use of evidence in the WCT and also explore new methods, such as the evidence gap mapping, to be able to adapt them to the specific needs of the classification. We propose to perform a mapping exercise to explore both, the current use of evidence in the WCT and a new method to summarise gaps and deficiencies in its evidence base. We aim to develop an EGM that summarises the evidence currently cited in our classification, with special attention to systematic reviews and other high level evidence synthesis.

Producing an EGM is to map existing systematic reviews and primary studies related to a specific review topic. EGMs provide a visual presentation of the available evidence using a framework of two related knowledge categories. This EGM protocol has been developed by adapting guidelines and recommendations from intervention EGMs to the specific field of evidence informing the WCT. We will adhere to the reporting recommendations from RepOrting standards for Systematic Evidence Syntheses21 and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)22 23 whenever they adapt to our EGM. No systematic search of the literature will be conducted in this case since we will be mapping the evidence cited in the WCT.

We will define our ‘columns domain’ as the tumour types and the rows as the tumour descriptors used in the WCT, respecting the original structure of the WCT in the WHO Blue Books. We will use the well-established blue-print of the WCT Blue Books publications to define these thematic domains of our EGM’s. In a standard EGM, the row headings are interventions and the column headings are outcomes. In our exercise, we will adapt this framework and summarise the current use of evidence in the WCT. To maximise the effect of this feedback, our mapping exercise will focus on the most recently updated group of tumours: the tumours of the lung.

Conceptual framework for the EGM

Although the intervention-outcome framework is most commonly used in EGMs of effectiveness studies, we will adapt this concept to adjust to a broader objective and different type of research. We developed a framework based on the publication/reporting structure of the WCT, where we considered three levels of tumour classification categories: organ of primary tumour site, tumour group and tumour type. The rest will fully adhere to the WCT structure in publications and define categories of tumour descriptors as the second domain (figure 2). We will use the frameworks to ensure that the EGM results of tumours of the lung are comprehensive, and explore the full potential of this method for additional mapping of evidence for the WCT.

Figure 2

Framework for the evidence gap map of cited evidence in the WHO Classification of Tumours of the Lung.

Objectives

In line with the aim of promoting a more evidence-based approach for the development of the WCT, our general objective focuses on exploring the application of EGM methodology to this classification. This evidence gap mapping exercise is an attempt to identify and test an appropriate method to provide our authors/editors and the cancer research community with an accurate summary of the available evidence, as well as to inform decisions in future WCT. We also aim to describe the current use of evidence in the WHO Classification of Tumours.

Our specific objective is to map the cited evidence in the last update of the WHO Classification of Tumours of the Lung.

Methods and analysis

All EGMs published to date have been in social sciences and mainly compilations of effectiveness studies,8 9 24–27 though some researchers have started to adapt the method to alternative fields.9 28 Our EGM constitutes a new application of this evidence synthesis method to a different type of research (biomedical, specifically diagnostic research), and to a completely new field (cancer classification and research). We are not aware of any registered protocols or published EGMs on which to model our EGMs, and no other type of evidence synthesis has so far attempted to synthesise evidence used in the WCT. There is one registered scoping review, analysing topics in need of review for future WCT editions,29 and some reports on the challenges of the WCT identified during the 5th edition,3 12 18 19 but none of them are systematic or are formal EGMs.

We will develop an EGM of the evidence cited in the last updated WCT, the WCT of tumours of the lung (A part of the WCT of Thoracic Tumours). This EGM will include the following key components: tumour types, categories of tumour descriptors (table 1) and levels of evidence (figure 3), applied as filters. Population is defined as cancer research, Intervention in this case refers to the tumour descriptor categories and Outcomes are the tumour types.

Table 1

Categories of information revised and synthesised in the WHO Classification of Tumours

Section in the WHO Blue Book
Localisation
Clinical features	Clinical manifestation
	Imaging
	Others
Epidemiology
Aetiology
Pathogenesis
Macroscopic appearance
Histopathology	General features
	Patterns/subtypes
	Immunohistochemistry
	Grading
	Differential diagnosis
	Others
Cytology
Diagnostic molecular pathology	In-situ hybridisation
	Sequencing
	Target therapy
	Others
Staging
Prognosis and prediction

Figure 3

Evidence level categories. RCT, randomised controlled trial.

Search and selection procedure

Since we aim to map the evidence cited in the WCT based on the most recent update of tumours of the lung (WCT fifth edition of Thoracic Tumours), no structured literature search will be performed. Instead, the WCT list of references will be exported and considered as included studies, skipping for this mapping exercise the entire search and selection procedure. Eligibility criteria are reduced to the inclusion of all cited literature in the WCT of the lung and all study designs, publications and references will be included. Studies will be evaluated and an evidence level assigned, based on methodological criteria adapted from the Centre for Evidence-Based Medicine (CEBM) of the University of Oxford30 (figure 3).

Data extraction and coding

The set of included references will be revised by the first reviewer (JdAM) and data on publication year, geographical location of the studies, tumour type and tumour descriptor, as well as research design and assessment of evidence level will be extracted into a standardised form. A second reviewer (SA) will randomly check the data extraction process and a third reviewer (BIIR) will be consulted to resolve disagreements between the primary reviewers. The reference list will be imported into EPPI-Reviewer software and predefined coding for the key domains and filters will be applied to each record by one reviewer (JdAM). A second reviewer (BIIR) will randomly check the coding process and a third reviewer (SA) will be consulted resolve disagreements.

Methodological quality evaluation

Traditionally, evidence synthesis would include an evaluation of the risk of bias or overall methodological quality of the selected studies. However, due to the exploratory character of this study we will limit this evaluation to a formal assessment of the evidence levels using the already described adapted CEBM levels (figure 3), considering mechanistic studies and classifications or grading systems as ‘unclassifiable’ since they may apply as good evidence to some information categories, but would be considered low level evidence for others.

Analysis and presentation

A descriptive synthesis of the cited evidence for tumour types and descriptors will be drafted, including year-wise and geographical distribution (based on reported affiliations) of the cited studies, and with special attention to cited systematic reviews.

The coded citations will be plotted in EPPI-Reviewer to develop a visual evidence map. Studies can be plotted in multiple places in the EGM if they have been cited multiple times and in different sections in the WCT. The resulting EGM will reflect the number of cited studies in the size of the spheres, and the level of evidence by applying a four-colour code (red=low level evidence, orange=moderate level, green=high level and blue=unclassifiable).

In addition, the final report will include a section discussing the overall stage of cited research in the WCT and we will include a part that highlights gaps, under-researched categories of tumour descriptors and pockets of low-level evidence. Final tables and figures may include a PRISMA diagram and visualisations of geographical and publication date distribution of the selected studies.

Roles and responsibilities

Content: the methodological and thematic framework development will be led by BIIR, systematic reviewer of the WCT, supported by SA, a specialised pathologist contributing to the WCT programme and JdAM, a medical epidemiologist in training with experience in epidemiological software solutions.

EGM methods: FC, expert in evidence synthesis with extensive experience in EGMs, will lead the methodology and advise other aspects of the protocol, as well as helping train the participants.

Patient and public involvement

The development of the research question and outcome measures has considered patients needs as part of the WHO Classification of Tumours framework that includes patient representatives in its editorial process. However, no patient will be involved in this study.

Ethics and dissemination

Details of the findings and the EGMs will be published in a separate manuscript.

No ethics approval will be required as this is a study of previously published material. No patient information or material will be used.

Information retrieval: Information retrieval will be supported by the experienced team of the International Agency for Research on Cancer library and managed by the WCT programme in collaboration with Teresa Lee and Latifa Bouanzi.

Sources of support: This work will be integrated into the strategy of the WCT programme, which is self-funded. The Head of the WCT programme, IC, and the Director of the Institute of Laboratory Medicine, German Heart Center of the Technical University Munich, SH, will provide assessment as senior experts in molecular pathology and biomarkers.

Timeline

Review start date: 15 December 2021.

Review finish date: 31 November 2022.

Reporting date: 15 December 2022.

Plans for update: The WCT EVI MAP project has been successfully funded and we will update the EGM at the conclusion of the project cycle, approximately at the end of 2026.