Reporting quality of clinical practice guidelines on head and neck cancer: a systematic review.

Background: Head and neck cancer (HNC) comprises a heterogeneous group of cancers. In view of the distinct biological characteristics and treatment strategies, clinical physicians require high-quality clinical practice guidelines (CPGs) which could provide reliable recommendations on medical practices. We aimed to evaluate the reporting quality of CPGs in the field of HNC.
Methods: We developed rigorous search strategies before searching the domestic and international literature databases (n=568) including Medline (via PubMed), Chinese National Knowledge Infrastructure (CNKI) and Wanfang as well as websites of guideline organizations (n=8) published between January 1, 2018 to July 1, 2021 for appropriate guidelines on HNC. We included all evidence-based guidelines about HNC in English or Chinese. We excluded translations, summaries and interpretations of guidelines, as well as older versions of guidelines if an updated edition was available. Data were extracted and the reporting quality was evaluated by two investigators independently guided by the Reporting Items for Practice Guidelines in Healthcare (RIGHT) checklist.
Results: A total of 21 guidelines complied with the inclusion criteria. Items show distinctions with reporting proportions among seven RIGHT domains. The proportions of reported items in each RIGHT domain were 75.4% for basic information, 63.1% for background, 42.9% for evidence, 55.1% for recommendations, 42.9% for review and quality assurance, 26.2% for funding and declaration and management of interests, and 50.8% for other information. Discussion: The average reporting quality of the recently published guidelines for HNC was moderate. Our research would help optimize the development processes of guidelines, resulting in high-quality guidelines for healthcare professionals. 2022 Translational Cancer Research. All rights reserved.

Introduction

Head and neck cancer (HNC) is a highly heterogeneous disease with distinct histopathological characteristics. Together with the different anatomical locations of origin, these lead to different manifestations and outcomes (1). About 931,931 new cases of HNC occurred and 467,125 patients died because of HNC worldwide in 2020. The majority of them were squamous cell carcinomas which originated from the oral cavity, oropharynx, nasopharynx, larynx, and hypopharynx, and often related to the tobacco and alcohol misuse (2). Its incidence varies greatly in different regions around the world. In recent years, there have been rapid advances in immunotherapy of HNC. Along with this, improvements in standard therapy have enhanced preservation of function and reduced morbidity and mortality (3). In general, patients with early-stage HNC are treated with surgery or radiotherapy as a single modality and have favorable outcomes. However, locally advanced diseases at diagnosis usually require multi-modality therapies. Palliative chemotherapy is the treatment for most recurrent and/or metastatic HNC. Immune checkpoint inhibitors have been rapidly developed in the treatment of advanced HNC in recent years. Many clinical trials are exploring the possibility of combining immunotherapy with other treatment modalities. In addition to the diversity of treatments, the diagnosis, management, and prognosis of HNC also have many indicators. Previous studies have shown that increased infiltration of CD3+, CD8+, CD4+ and Foxp3+ tumor-infiltrating lymphocytes (TILs) are associated with better prognosis in HNC. High CD4+ and CD8+ TIL were significantly associated with improved OS in oropharyngeal cancer, and high CD8+ was significantly associated with improved OS in hypopharyngeal cancer (4,5). Therefore, clinical practice guidelines (CPGs) are used commonly by physicians with gathering a large number of clinical recommendations based on systematic reviews of the literature (6). With rating the quality of evidence and strength of recommendations, CPGs aim to interpret and consolidate evidence from the expansive clinical research literature to establish standards for clinical practice. CPGs have been increasingly used over the past decade worldwide. However, the reporting qualities of CPGs are not satisfactory enough (7,8). In the past, most guidelines were evaluated by the Appraisal of Guidelines, Research and Evaluation (AGREE) II, a validated tool for the systematic appraisal of CPG methodological development and quality (9). However, it is gradually replaced by the Reporting Items for Practice Guidelines in Healthcare (RIGHT) checklist because of the limited value in evaluating some specific types of guidelines (10). RIGHT checklist was developed and launched by a group of international experts with multidisciplinary intellectual background in 2016. And now it has been widely recognized as a standard in the development and evaluation of guidelines. In some recently published studies, some researchers have evaluated guidelines for other types of cancer through the RIGHT checklist. As research progresses and advances, the development of oncology guidelines will become more standardized (11-14). To evaluate the reporting quality of CPGs on HNC published between 2018 and 2021, we analyzed all items reported in guidelines by using the RIGHT checklist. By means of this analysis, we hope our study can not only offer users deeper understanding of CPGs, but also provide objective suggestions for the later development of guidelines. We present the following article in accordance with the PRISMA reporting checklist (available at https://tcr.amegroups.com/article/view/10.21037/tcr-22-52/rc).

Methods

Literature search

We systematically searched Medline (via PubMed), Wan Fang Database and Chinese National Knowledge Infrastructure (CNKI) for CPGs on HNC. In addition to literature databases, guidelines also were developed and published by oncological organizations or government public health agencies. So, the following websites were added to our search scope, such as World Health Organization (WHO), National Comprehensive Cancer Network (NCCN), Guidelines International Network (GIN), Scottish Intercollegiate Guidelines Network (SIGN), the National Institute for Health and Care Excellence (NICE), the European Society for Medical Oncology (ESMO), the American Society of Clinical Oncology (ASCO) and the Chinese Society of Clinical Oncology (CSCO). The search terms for PubMed included Head and Neck Neoplasms [MeSH], cancer*, carcinoma*, neoplasm*, tumour*, tumor*, and malignant*, mouth, nasopharyn*, oropharyn*, hypopharyn*, laryn*, oral, salivary gland, Guideline, Practice Guideline, guideline*, guidance* and recommendation*. We adjusted the format when we were searching through different databases and ended up with a total of 576 records. The search strategies were presented in detail in the Appendix 1.

Selection criteria of guidelines

We limited the publication time from January 2018 to July 2021. And the language of publication should be Chinese or English. If one CPG was updated, we only chose the latest version. The topic was restrict to HNC. Articles of guidelines about other cancers, non-guidelines, expert consensus in Chinese, interpretation or translation versions of guidelines, or protocols of the development would be excluded. Seven guidelines from the Oral Cancer Task Force (OCTF) focused on the same topic of HNC with different perspectives. We integrated and considered them as an independent guideline for higher efficiency.

Guideline selection, information entry and data extraction

We aggregated all guidelines searched through above procedures to move on the study conveniently. The first-time screening needs comprehensive understanding of titles and abstracts. Two investigators (Jiabao Hou and Xuan Wu) completed this procedure independently. Then they verified the identified guidelines to conduct next screening. This part of guidelines would be obtained for full details. We decided whether to include one guideline through the careful reading of the full texts. If one article met the inclusion criteria of our study, we would take it into consideration. Disagreement would be resolved by discussion or further recommendations from an independent expert (Qiming Wang). And next we enriched the RIGHT checklist by reading full texts again. Reporting information of items was inputted and original data was extracted by investigators independently. The required contents consist of different aspects in the guideline including title, year of publication, journal or website of publication, developer, country or region, development procedures, fundings and interests. All data would be checked and verified.

Reporting quality evaluation of guidelines

RIGHT checklist was used in the whole evaluation of guidelines. It includes 7 domains and 22 items. “Basic information” refers to the key content, such as title, year of publication and so on. “Background” includes specialized information. “Evidence” answers the question that what kind of literature did the guideline based on. “Recommendations” has the specific information of recommendations in the guidelines. “Review and quality assurance” includes the process of quality control. “Funding and declaration and management of interests” and “Other information” are also important ingredients (10). There are three statements of each item: “Reported”, “Not reported” or “Not applicable”. When the relevant information was provided in the guidelines, we regarded this item was “Reported”. When the relevant information was not shown, it was marked as “Not reported”. If the item did not apply for a certain guideline, that was “Not applicable”.

Like the steps above, quality assessment of each guideline was conducted by researchers in an independent way. If disagreements arose, they would be resolved by discussion or asking for recommendations from an expert adjudicator (Qiming Wang). The details of reporting quality were presented in the Appendix 1.

Statistical analysis

Every guideline reported different items, the overall reporting rate means that how many items reported in one guideline. Besides the differences among guidelines, we also showed the differences among items and domains. We used SPSS V.26.0. to calculate and analyze overall reporting rate.

Results

Identification of specific guidelines

We searched 576 articles, of which 568 were from electronic databases and 8 were from guideline-related websites, oncology organizations, or other sources. Seven duplicate records were removed. After reviewing the remaining 569 records for titles and abstracts, we excluded 522 records because they were not guidelines or were not last versions.

After full text access and reading of the 47 records which were remained after re-screening, we finally included 27 guidelines. Seven guidelines developed by the Oral Cancer Task Force (India) were combined as a series. So there were 21 reports included after combining the guideline series ().

Figure 1

Flowchart of systematic search. CPG, clinical practice guideline; HNC, head and neck cancer.

Characteristics of included guidelines

The detailed features of each guideline were shown in . One guideline was published in 2018, four in 2019, six in 2020, and ten in 2021. Seven guidelines had reporting rates greater than or equal to 60.0%, twelve guidelines had reporting rates between 30.0% and 60.0%, and only two had reporting rates less than 30.0%. Eighteen guidelines were published in journals, and three were published on websites. Of these 21 guidelines, three were developed by China, four were developed by European multidisciplinary panel of experts, two were developed by France. And one from India, two from Spain, one from the United Kingdom, and five from the United States, leaving three from international collaborations.

Table 1

Characteristic of included guidelines

No.	Title/Reference	Year of publication	Reporting rate	Journal or website publication	Developer	Country or region
1	Chemotherapy in Combination with Radiotherapy for Definitive-Intent Treatment of Stage II-IVA Nasopharyngeal Carcinoma: CSCO and ASCO Guideline (15)	2021	77.1%	Journal	CSCO and ASCO	International
2	Consensus on resectability in N3 head and neck squamous cell carcinomas: GETTEC recommendations (16)	2020	48.6%	Journal	GETTEC	France
3	Diagnosis and Management of Squamous Cell Carcinoma of Unknown Primary in the Head and Neck: ASCO Guideline (17)	2020	80.0%	Journal	ASCO	United States
4	GEORCC Recommendations on Target Volumes in radiotherapy for Head Neck Cancer of Unknown Primary (18)	2018	45.7%	Journal	GEORCC	Spain
5	Guidelines of the French Society of Otorhinolaryngology–Head and Neck Surgery (SFORL), part I: Primary treatment of pleomorphic adenoma (19)	2021	34.3%	Journal	SFORL	France
6	Head and Neck Cancer International Group (HNCIG) Consensus Guidelines for the delivery of postoperative radiation therapy in complex cutaneous squamous cell carcinoma of the head and neck (cSCCHN) (20)	2020	25.7%	Journal	HNCIG	International
7	The EANM practical guidelines for sentinel lymph node localisation in oral cavity squamous cell carcinoma (21)	2019	40.0%	Journal	EANM and IAEA	Europe
8	Head and neck mucosal melanoma: The United Kingdom national guidelines (22)	2020	77.1%	Journal	HNMMGDG	United Kingdom
9	Head and neck cancers (23)	2021	28.6%	Website	NCCN	United States
10	International Guideline on Dose Prioritization and Acceptance Criteria in Radiation Therapy Planning for Nasopharyngeal Carcinoma (24)	2019	62.9%	Journal	MPE	International
11	Management of Salivary Gland Malignancy: ASCO Guideline (25)	2021	88.6%	Journal	ASCO	United States
12	Management of the Neck in Squamous Cell Carcinoma of the Oral Cavity and Oropharynx: ASCO Clinical Practice Guideline (26)	2019	88.6%	Journal	ASCO	United States
13	Nasopharyngeal carcinoma: ESMO-EURACAN Clinical Practice Guidelines for diagnosis, treatment, and follow-up (27)	2021	54.3%	Journal	ESMO and EURACAN	Europe
14	Nasopharyngeal carcinoma in children and adolescents: The EXPeRT/PARTNER diagnostic and therapeutic recommendations (28)	2021	51.4%	Journal	EXPeRT	Europe
15	Oral cavity cancer management guidelines for low-resource regions (29)	2019	60.0%	Journal	AHNS	United States
16	Salivary gland carcinoma in children and adolescents: The EXPeRT/PARTNER diagnosis and treatment recommendations (30)	2021	42.9%	Journal	EXPeRT	Europe
17	SEOM clinical guidelines for the treatment of head and neck cancer (2020) (31)	2021	45.7%	Journal	SEOM	Spain
18	Guidelines for Radiotherapy of Nasopharyngeal Carcinoma in China (2020) (32)	2021	40.0%	Journal	CATRO and CSTRO	China
19	Guidelines of Chinese society of clinical oncology (CSCO) head and neck cancer 2021 (33)	2021	57.1%	Website	CSCO	China
20	Guidelines of Chinese society of clinical oncology (CSCO) nasopharyngeal carcinoma 2020 (34)	2020	57.1%	Website	CSCO	China
21	Practical consensus recommendation developed by India (35-41)	2020	34.3%	Journal	OCTF	India

ASCO, American Society of Clinical Oncology; CSCO, Chinese Society of Clinical Oncology; GETTEC, Groupe Français des Tumeurs de la Tête et du Cou; SFORL, French Society of Otorhinolaryngology–Head and Neck Surgery; HNCIG, Head and Neck Cancer International Group; EANM, European Association of Nuclear Medicine; IAEA, the International Atomic Energy Agency; HNMMGDG, The United Kingdom head and neck mucosal melanoma guideline development group; NCCN, National Comprehensive Cancer Network; MPE, Multidisciplinary panel of experts; ESMO, European Society for Medical Oncology; EURACAN, the European Reference Network for rare adult solid cancers; EXPeRT, European Cooperative Study Group for Pediatric Rare Tumors; AHNS, American Head and Neck Society; SEOM, Spanish Society of Medical Oncology; CSTRO, China Society for Radiation Oncology; OCTF, Oral Cancer Task Force.

General evaluation of reporting quality

The mean value of the reporting rates for these seven domains ranged from 26.2% to 75.4%, with the highest reporting rate for “Basic information” at 75.4% and the lowest reporting rate for “Funding and declaration and management of interests” at 26.2%. The remaining five domains were “Background”, “Recommendations”, “Other information”, “Evidence” and “Review and quality assurance”. Their reporting rates were 63.1%, 55.1%, 50.8%, 42.9%, and 42.9%, respectively (). In the domain of basic information, all guidelines reported “Abbreviations and acronyms”. Sub-items (1a ,1c and 4) had relatively higher reporting rates that were above 90%. As for the background, all guidelines described the primary population that was addressed by the recommendations. However, only one guideline described the suitable application situations. The reporting rates for the remaining sub-items (5, 6, 8a, 9a and 9b) were 71.4%, 76.2%, 47.6%, 52.4% and 66.7%, respectively. In the domain “Evidence”, 61.9% of guidelines indicated that they were based on systematic reviews. The reporting rate of “Assessment of the certainty of the body of evidence” was 66.7%. Ten guidelines described the health questions on which the recommendations were based, yet only three of these described the methodology for outcome selection and classification.

Figure 2

Mean reporting rates of the RIGHT checklist sub-items by domain. RIGHT, Reporting Items for Practice Guidelines in Healthcare.

In the domain “Recommendations”, 85.7% of guidelines provided clear, precise, and actionable recommendations. The rest of sub-items (14a, 14b, 14c and 15) had reporting rates less than 50%.

The reporting rate of “Review and quality assurance” was 42.9%. None of the guidelines reported “Funding sources and roles of the funder”. The reporting rates for the remaining sub-items (19a, 19b, 20, 21 and 22) were 85.7%, 19.0%, 57.1%, 42.9% and 52.4%, respectively ().

Figure 3

Reporting compliance to each sub-item of the RIGHT checklist in the included guidelines (The descriptions of each sub-item are shown in http://www.right-statement.org/home/extensions). RIGHT, Reporting Items for Practice Guidelines in Healthcare.

Subgroup analysis of reporting quality

The reporting rates of guidelines published from 2018 to 2021 were 42.9%, 70.5%, 51.0% and 54%, respectively. The reporting rate of guidelines published in English was slightly higher than that published in Chinese. Among those published in English, five guidelines from the United States had the highest average reporting rate. Reporting rates of guidelines from Europe and international multidisciplinary panel of experts are 48.9% and 55.2%, respectively. However, the reporting rate for Asian countries, including China and India, was only 47.1%.

Discussion

HNC is a highly heterogeneous group of cancers, which originates mainly in the oral cavity, nasopharynx, larynx, and salivary glands. In China, nasopharyngeal cancer is the most common, and the high incidence area is Guangdong province, which is related to Epstein-Barr virus (EBV) and genetic factors. While oral cancer is highly prevalent in Hunan province and is related to areca nut chewing (42). Most patients were already at an advanced stage when diagnosed, resulting in poor treatment outcomes and limited functional recovery. Moreover, the level and quality of medical services vary worldwide. It is of great importance to use more effective, standardized, and transparent methods to develop guidelines.

For a high-quality guideline, the process of its development should include discussions among multidisciplinary experts. Moreover, clear recommendations are essential. These criteria are all reflected in the RIGHT checklists. Only by following such guidelines can clinicians and health professionals play a greater role in the diagnosis, treatment, and management of disease. Therefore, we used RIGHT checklists to objectively evaluate the reporting quality of included guidelines, hoping to help clinicians make the better decisions and to draw the attention of guideline developers.

This is the first evaluation of HNC guidelines based primarily on the RIGHT checklists. Overall, the reporting rate for guidelines was moderate. Of the seven reporting domains, “Basic information” has the highest reporting rate. All guides reported “Abbreviations and acronyms (3)”. However, only 57.1% of the guidelines reported “Executive summary (2)”. Due to the numerous ways to classify and treat head and neck tumors, providing a clear and concise summary of recommendations is easy to use clinically. This makes it very easy for clinicians to find the focus from the many guidelines entries and apply the recommendations. The reporting rate for the remaining sub-items (1a ,1c and 4) was above 90% (). The lowest reporting rate in this area was “The year of publication of the guideline (1b)”, at only 19%. This may be related to the frequency of updates by the guideline development agencies. Of these, all 3 guidelines from China reported this entry, and 2 were updated annually.

The information of background includes brief description of the health problems, aims of the guideline and specific objectives, end-users and settings, target populations and guideline development groups. The reporting rate in it varies widely. All guidelines describe the primary populations which were applicable with certain recommendations. However, only one guideline describes the suitable application situations, 47.6% of the guidelines reported intended primary users. A complete guideline should include the disease and the primary target population. All the guidelines we included met this criterion. With continued advances in radiotherapy and surgical techniques, patients undergo surgery for better organ function preservation. In addition, different populations have different levels of acceptance of post-operative aesthetics. Therefore, taking subgroup populations into consideration is also important. The reporting rate for the remaining sub-items ranged from 50% to 90%.

The information of evidence includes systematic reviews, assessment of the certainty of the body of evidence. High-quality evidence will make guidelines convincing. In the domain “Evidence”, 61.9% of guidelines indicated that they were based on systematic reviews. And 66.7% of guidelines describe the approach used to assess the certainty of evidence. Ten guidelines describe the key questions on which the recommendations were based, yet only three of these describe the methodology for outcome selection and classification. The reason may be that guideline developers caring too much about survival time, while ignoring the adverse effects of drugs, how treatments affect patients’ quality of life and mental health, etc. There are often many controversies in the diagnosis and treatment of HNC. Providing the search strategies and the selection criteria, and describing how the risk of bias was evaluated, would make guideline users treat disputes in a rational way.

The information of recommendations includes explanation for recommendations, evidence to decision processes. Most of the guidelines provided clear, precise, and actionable recommendations. 81% of guidelines indicated the strength of recommendations, the quality of the evidence supporting the recommendation, and separate recommendations for special subgroups. As for explanation for recommendations and evidence to decision processes, the reporting rate for these sub-items is less than 50%. 42.9% of the guidelines reported that the draft guideline underwent independent review and quality assurance process. In forming recommendations, only a few developers considered the preferences and values of the target population, cost and resource utilization, and factors such as fairness and feasibility. This was not only because of the difficulty of collecting this information, but also because of the lack of awareness among developers.

As for the domain of “Funding, declaration and management of interest”, none of guidelines present the certain sources of funding and the role of funders in the different phases of guideline development. One reason for this may be that funders did not influence the development of the guidelines, or it may be that their roles were consistent across the phases. 85.7% of guidelines describe whether they had conflicts of interest for authors to develop recommendations.

Regarding “Other information”, 57.1% of guidelines describe the efficient way for readers to obtain the guidelines and relative documents. 42.9% of guidelines provide suggestions for the later researches. This may be due to the fact that guideline developers were not aware of the discrepancy between current practice and research evidence. The exploration of clinical trials in HNC is rapidly evolving. Guideline developers should describe the gaps between practice and clinical research and provide appropriate advice for future research with foresight.

Strengthens and limitations

This is the first article to evaluate the HNC guidelines using the RIGHT checklist. Our investigators developed a rigorous search strategy, carefully filled in all guideline reporting items, and analyzed the reporting rates with appropriate statistical methods. As the last item in the RIGHT checklist points out: all limitations of the guideline development process should be presented, our study is not perfect in all aspects. First, our inclusion and exclusion criteria could not meet the requirements of all guideline developers because of the different definitions of HNC in different countries and regions. Second, we only included guidelines written in English and Chinese, guidelines in other languages were not included.

Conclusions

We evaluated the HNC guidelines by using the RIGHT checklist and found differences in reporting rates across domains. Most guidelines reported basic information but under-reported funding and evidence. We hope that guideline developers could not only focus on the content, but also take more concern about reporting quality.