Evaluation of the reporting quality of clinical practice guidelines on lung cancer using the RIGHT checklist.

BACKGROUND: In recent years, the number of clinical practice guidelines (CPGs) for lung cancer has increased, but the quality of these guidelines has not been systematically assessed so far. Our aim was to assess the reporting quality of CPGs on lung cancer published since 2018 using the International Reporting Items for Practice Guidelines in Health Care (RIGHT) instrument.
METHODS: We systematically searched the major electronic literature databases, guideline databases and medical society websites from January 2018 to November 2020 to identify all CPGs for small cell and non-small cell lung cancer (NSCLC). The search and extraction were completed using standardized forms. The quality of included guidelines was evaluated using the RIGHT statement. We present the results descriptively, including a stratification by selected determinants.
RESULTS: A total of 49 CPGs were included. The mean proportion across the guidelines of the 22 items of the RIGHT checklist that were appropriately reported was 57.9%. The items most common to be poorly reported were quality assurance (item 17) and description of the role of funders (item 18b), both of which were reported in only one guideline. The proportions of items within each of the seven domains of the RIGHT checklist that were correctly reported were Basic information 75.9%; background 83.2%; evidence 44.5%; recommendations 55.4%; review and quality assurance 12.2%; funding and declaration and management of interests 42.9%; and other information 38.1%. The reporting quality of guidelines did not differ between publication years. CPGs published in journals with impact factor >30 tended to be best reported.
CONCLUSIONS: Our results revealed that reporting in CPGs for lung cancer is suboptimal. Particularly the declaration of funding and quality assurance are poorly reported in recent CPGs on lung cancer. 2021 Translational Lung Cancer Research. All rights reserved.

Introduction

Lung cancer is the leading cause of cancer-related deaths worldwide (1). In 2018, about 2.1 million new diagnoses and 1.8 million deaths related to lung cancer were estimated globally (2). The most common types of lung cancer are non-small cell lung cancer (NSCLC) and small-cell lung cancer (SCLC) accounting for 76% and 13% of all cases of lung cancer, respectively (3). Data from the Surveillance, Epidemiology, and End Results (SEER) program demonstrates that population-level mortality from NSCLC fell in recent years, and this decrease was associated with the use of targeted therapies (3). The clinical outcomes of targeted therapies are superior to those of traditional cytotoxic chemotherapy in patients with oncogenic-driven NSCLC (4). Next to targeted therapies, only recently immune checkpoint inhibitors have improved the outcomes of both NSCLC and SCLC, dramatically (5). The increasing complexity of molecular pathology and the growing number of new therapeutic agents has brought the management of lung cancer patients into an era of precision medicine. Thus, it is important to accurately use the latest available evidence to make best clinical decisions for these patients.

Clinical practice guidelines (CPGs) are collections of statements that include recommendations intended to optimize patient care. These statements are informed by a systematic review of evidence and an assessment of the benefits and costs of alternative care options. The process of developing a CPG is time-consuming and costly, but CPGs may, once published and implemented, have a tremendous impact on clinical decisions and patients’ lives. Many evidence-based guidelines have been developed for the management of lung cancer. Usually, guidelines should be updated are every three to five years (6,7), however, due to rapid diagnostic and treatment changes, oncology guidelines should be updated annually (8). Since the number of CPGs has increased rapidly, growing concern about the variation in their quality becomes evident. The reporting quality is an important factor influencing the quality of guidelines, contributing to the way the recommendations are implemented. Thus, standardized and formal methods are needed to assess the reporting quality of CPGs.

For this reason, the International Reporting Items for Practice Guidelines in Health Care (RIGHT) instrument was developed in 2016 (9). The RIGHT checklist includes seven domains with 22 items to facilitate complete and transparent reporting, and help developers prepare the guidelines. Given the rapid accumulation of new evidence in therapeutic strategies, adherence to these checklists is particularly relevant for guidelines evaluating lung cancer. In this study, we evaluated the reporting quality of guidelines for lung cancer patients, and investigated how the quality in guideline reporting varies across time and by selected characteristics.

Methods

Literature search

We systematically searched Medline (via PubMed), Chinese Biomedical Literature Database (CBM), Wan Fang Database and Chinese National Knowledge Infrastructure (CNKI) to identify CPGs for lung cancer from January 1, 2018 to November 15, 2020. The search strategy combined the following terms: “lung neoplasms”, “lung cancer”, “practice guideline”, “guidance”, and “recommendation”. Language was restricted to Chinese and English. We also searched the websites of the National Institute for Health and Care Excellence (NICE, https://www.nice.org.uk/), National Comprehensive Cancer Network (NCCN, https://www.nccn.org/), World Health Organization guidelines (WHO, https://www.who.int/publications/guidelines/year/en/), Scottish Intercollegiate Guidelines Network (SIGN, https://www.sign.ac.uk/our-guidelines/) and Guidelines International Network (GIN, https://guidelines.ebmportal.com/), as well as Google Scholar as a supplemental source. Additional details of our search strategy are provided in Appendix 1.

Inclusion criteria and exclusion criteria

CPGs were considered eligible if they met the following criteria: (I) the CPG was published in English or Chinese; (II) the focus of the CPG was screening, testing, diagnosis, treatment or management of lung cancer; (III) full-text of the CPG was available and accessible; and (IV) the CPG was the latest version. Interpretations and summaries of guidelines, and draft guidelines not yet formally published were excluded.

Screening

Two authors (YF Ma and QW Zhang) independently identified eligible CPGs and retrieved the full texts and any related supplementary materials (Appendix 1). Discrepancies were adjudicated by a third reviewer (C Xi).

Data extraction of guidelines

Data were extracted independently by two authors (XD Jia and KF Liu) using a standardized electronic form. Disagreements were settled by consultation. The data extracted from each CPG included the name of the first author, publication year, publication language, region/country where the CPG was developed, developers (institution or working group), format of publication (peer-reviewed journal, or website only), impact factor (IF) of the journal according to SCI (Science Citation Index), and the scope/purpose and target population of CPGs.

Reporting quality assessment using the RIGHT checklist

The reporting quality of CPGs was evaluated using the RIGHT instrument, which includes 22 key items categorized into seven domains. Some key items are further divided into two or three sub-items, reaching a total of 35 items. The seven domains are: basic information (6 items), background (8 items), evidence (5 items), recommendations (7 items), review and quality assurance (2 items), funding and conflicts of interest statements and management (4 items), and other information (3 items). We rated each item as “reported” if the relevant information was fully presented, “not reported” if some relevant information was lacking, or “not applicable” if the item was not applicable for evaluating the specific guidelines, based on the protocol of RIGHT instrument. The authors who performed the assessment were trained by a member of RIGHT checklist working group (YF Ma). Two investigators independently evaluated the reporting quality according to the RIGHT statement (JL Lu and YJ Yang). Disagreements between reviewers were resolved through consensus or consulting an independent expert adjudicator (J Kang).

Statistical analysis

We calculated the reporting rates of the RIGHT checklist items for all 35 items separately (percentage of all CPGs that reported the item), for each domain (mean over the reporting rates of all items of each particular domain across all CPGs), and overall (mean over all items and CPGs). All 35 items were weighted equally in the calculation of the domain and overall scores, and those assigned as “not applicable” were included in the denominator. We also present results stratified by the year of publication, language, region/country of origin, and format of publication (peer-reviewed journals categorized by impact factor, or website only).

Results

Identification of specific guidelines

The search yielded a total of 668 potentially relevant records (). Sixty-five records were removed as duplicates, and 536 records were excluded after screening titles and abstracts. After an extensive review of the full texts of the remaining 67 records, a total of 49 CPGs were deemed eligible and included in the appraisal process.

Figure 1

Flow chart of the literature review.

Characteristics of selected guidelines

The characteristics of the CPGs are summarized in . Seven CPGs were established for SCLC (10-16), 20 for NSCLC (17-36), and 22 for lung cancer regardless of histological subtype (37-58). The number of published CPGs increased every year, with a total of 22 CPGs published in 2020. Eight guidelines (16.3%) focused on screening, 6 guidelines (12.2%) on testing, 30 guidelines (61.2%) on diagnosis and treatment, and 5 guidelines (10.2%) on the management of lung cancer. Forty-two CPGs were developed by medical specialty societies, and seven were developed by guideline working groups not associated with any medical society. Medical specialty societies from the United States were the most common guideline developers, accounting for 11 (22.4%) of CPGs. Eight CPGs were developed in China, five in Spain, four in Italy, one in Brazil, one in Canada, one in Japan, one in Saudi Arabia, one in India, one in the UK, five by European multinational collaborations, four by multinational collaborations from other regions, and six by international organizations. Forty-five of the 49 CPGs were published in journals, one on the NICE website only, one on the CSCO website only, and two on the NCCN website only. Guidelines developed in the US tended to be published in journal with higher impact factor (IF), and the three that were published in the highest-impact journal (IF >30) were all developed or endorsed by the American Society of Clinical Oncology (ASCO). Details on the study characteristics are shown in Table S1.

Table 1

Characteristics of the eligible CPGs

Study characteristics	CPGs, n (%)
Total	49 (100)
Histological classification
Lung cancer	22 (44.9)
Non-small cell lung cancer	20 (40.8)
Small cell lung cancer	7 (14.3)
Scope and purpose
Screening	8 (16.3)
Diagnosis and treatment	30 (61.2)
Testing	6 (12.2)
Management	5 (10.2)
Organization of guidelines
Association/society	42 (85.7)
Development working group	7 (14.3)
Language
English	42 (85.7)
Chinese	7 (14.3)
Region/country of origin
Global+	6 (12.2)
Multinational regional (Europe)	5 (10.2)
Multinational regional (other regions)*	4 (8.2)
USA	11 (22.4)
China	8 (16.3)
Spain	5 (10.2)
Italy	4 (8.2)
Others#	6 (12.2)
Reporting year
2018	10 (20.4)
2019	17 (34.7)
2020	22 (44.9)
Journal’s IF
Chinese-language journals without IF	6 (12.2)
English-language journals without IF	4 (8.2)
IF 0–5	16 (32.6)
IF 5–10	9 (18.4)
IF 10–30	7 (14.3)
IF >30	3 (6.1)
Websites only	4 (8.2)

+, International organizations; *, North America, Pan-Asia, Southern Africa; #, UK, Canada, Japan, Brazil, Saudi Arabia, India. CPGs, clinical practice guidelines; IF, impact factor.

Overall analysis of reporting quality

The reporting rates for the seven domains ranged from 12.2% to 83.2%. The “basic information” and “background” domains had the highest reporting proportions (75.9% and 83.2%, respectively). The “recommendations” domain also had a reporting rate above 50%. The “review and quality assurance” domain had the lowest reporting rate (12.2%). The reporting rates for the remaining domains were 44.5% for “evidence”, 42.9% for “funding and declaration and management interests“, and 38.1% for “other information” ().

Figure 2

The reporting rates of the RIGHT checklist domains in the included CPGs. RIGHT, International Reporting Items for Practice Guidelines in Health Care; CPGs, clinical practice guidelines.

The mean overall reporting proportion over all 35 items was 57.9%. The number of reported items ranged from 12 (38.7%) to 31 (92.1%) across the guidelines. All guidelines (100%) adhered to items 3, 7a and 13a, and items 1a, 1c, 6, 7b, 9b and 13b were also reported by more than 90% of the guidelines. Items 17 and 18b had the poorest reporting rate, both being described in only one CPG ().

Table 2

The reporting rates of each RIGHT checklist item in the eligible CPGs (9)

Section/topic	No.	Item	Reported, n (%)	Not reported, n (%)	Not applicable, n (%)
Basic information
Title/subtitle	1a	Identify the report as a guideline, that is, with ‘guideline(s)’ or ‘recommendation(s)’ in the title	45 (91.8)	4 (8.2)	0 (0)
	1b	Describe the year of publication of the guideline	18 (36.7)	31 (63.3)	0 (0)
	1c	Describe the focus of the guideline, such as screening, diagnosis, treatment, management, prevention, or others	46 (93.9)	3 (6.1)	0 (0)
Executive summary	2	Provide a summary of the recommendations contained in the guideline	21 (42.9)	28 (57.1)	0 (0)
Abbreviations and acronyms	3	Define new or key terms, and provide a list of abbreviations and acronyms if applicable	49 (100.0)	0 (0)	0 (0)
Corresponding developer	4	Identify at least 1 corresponding developer or author who can be contacted about the guideline	44 (89.8)	5 (10.2)	0 (0)
Background
Brief description of the health problem(s)	5	Describe the basic epidemiology of the problem, such as the prevalence/incidence, morbidity, mortality, and burden (including financial) resulting from the problem	43 (87.8)	6 (12.2)	0 (0)
Aim(s) of the guideline and specific objectives	6	Describe the aim(s) of the guideline and specific objectives, such as improvements in health indicators (e.g., mortality and disease prevalence), quality of life, or cost savings	47 (95.9)	2 (4.1)	0 (0)
Target population(s)	7a	Describe the primary population(s) that is affected by the recommendation(s) in the guideline	49 (100.0)	0 (0)	0 (0)
	7b	Describe any subgroups that are given special consideration in the guideline	45 (91.8)	4 (8.2)	0 (0)
End users and settings	8a	Describe the intended primary users of the guideline (such as primary care providers, clinical specialists, public health practitioners, program managers, and policymakers) and other potential users of the guideline	42 (85.7)	7 (14.3)	0 (0)
	8b	Describe the setting(s) for which the guideline is intended, such as primary care, low- and middle-income countries, or inpatient facilities	26 (53.1)	23 (46.9)	0 (0)
Guideline development groups	9a	Describe how all contributors to the guideline development were selected and their roles and responsibilities (e.g., steering group, guideline panel, external reviewers, systematic review team, and methodologists)	29 (59.2)	20 (40.8)	0 (0)
	9b	List all individuals involved in developing the guideline, including their title, role(s), and institutional affiliation(s)	45 (91.8)	4 (8.2)	0 (0)
Evidence
Health care questions	10a	State the key questions that were the basis for the recommendations in PICO (population, intervention, comparator, and outcome) or other format as appropriate	22 (44.9)	27 (55.1)	0 (0)
	10b	Indicate how the outcomes were selected and sorted	13 (26.5)	36 (73.5)	0 (0)
Systematic reviews	11a	Indicate whether the guideline is based on new systematic reviews done specifically for this guideline or whether existing systematic reviews were used	31 (63.3)	18 (36.7)	0 (0)
	11b	If the guideline developers used existing systematic reviews, reference these and describe how those reviews were identified and assessed (provide the search strategies and the selection criteria, and describe how the risk of bias was evaluated) and whether they were updated	19 (38.8)	13 (26.5)	17 (34.7)
Assessment of the certainty of the body of evidence	12	Describe the approach used to assess the certainty of the body of evidence.	24 (49.0)	25 (51.0)	0 (0)
Recommendations
Recommendations	13a	Provide clear, precise, and actionable recommendations	49 (100.0)	0 (0)	0 (0)
	13b	Present separate recommendations for important subgroups if the evidence suggests that there are important differences in factors influencing recommendations, particularly the balance of benefits and harms across subgroups	45 (91.8)	4 (8.2)	0 (0)
	13c	Indicate the strength of recommendations and the certainty of the supporting evidence	25 (51.0)	2 (4.1)	22 (44.9)
Rationale/explanation for recommendations	14a	Describe whether values and preferences of the target population(s) were considered in the formulation of each recommendation. If yes, describe the approaches and methods used to elicit or identify these values and preferences. If values and preferences were not considered, provide an explanation	23 (46.9)	26 (53.1)	0 (0)
	14b	Describe whether cost and resource implications were considered in the formulation of recommendations. If yes, describe the specific approaches and methods used (such as cost-effectiveness analysis) and summarize the results. If resource issues were not considered, provide an explanation	22 (44.9)	27 (55.1)	0 (0)
	14c	Describe other factors taken into consideration when formulating the recommendations, such as equity, feasibility, and acceptability	7 (14.3)	42 (85.7)	0 (0)
Evidence to decision processes	15	Describe the processes and approaches used by the guideline development group to make decisions, particularly the formulation of recommendations (such as how consensus was defined and achieved and whether voting was used)	19 (38.8)	30 (61.2)	0 (0)
Review and quality assurance
External review	16	Indicate whether the draft guideline underwent independent review and, if so, how this was executed and the comments considered and addressed.	11 (22.4)	38 (77.6)	0 (0)
Quality assurance	17	Indicate whether the guideline was subjected to a quality assurance process. If yes, describe the process.	1 (2.0)	48 (98.0)	0 (0)
Funding and declaration and management of interests
Funding source(s) and role(s) of the funder	18a	Describe the specific sources of funding for all stages of guideline development.	19 (38.8)	30 (61.2)	0 (0)
	18b	Describe the role of funder(s) in the different stages of guideline development and in the dissemination and implementation of the recommendations.	1 (2.0)	14 (28.6)	34 (69.4)
Declaration and management of interests	19a	Describe what types of conflicts (financial and nonfinancial) were relevant to guideline development.	39 (79.6)	10 (20.4)	0 (0)
	19b	Describe how conflicts of interest were evaluated and managed and how users of the guideline can access the declarations.	25 (51.0)	24 (49.0)	0 (0)
Other information
Access	20	Describe where the guideline, its appendices, and other related documents can be accessed.	8 (16.3)	41 (83.7)	0 (0)
Suggestions for further research	21	Describe the gaps in the evidence and/or provide suggestions for future research.	25 (51.0)	24 (49.0)	0 (0)
Limitations of the guideline	22	Describe any limitations in the guideline development process (such as the development groups were not multidisciplinary or patients’ values and preferences were not sought), and indicate how these limitations might have affected the validity of the recommendations.	23 (46.9)	26 (53.1)	0 (0)

RIGHT, Reporting Items for Practice Guidelines in Healthcare; CPGs, clinical practice guidelines. Details of the RIGHT checklist is available on: http://www.right-statement.org/right-statement/checklist.

Stratified analyses of reporting quality

The overall mean reporting proportions by year of publication were 58.0% in 2018, 58.6% in 2019, and 57.3% in 2020. The quality of reporting was best in guidelines published in journals with IF >30 (overall reporting rate 78.1%); those published in Chinese-language journals without IF had the lowest overall reporting rate (51.9%). The overall reporting proportion in the English-language guidelines was 59.0%, and in Chinese-language guidelines 51.0%. Guidelines from multinational regional collaborations (except Europe), USA and Italy had the highest reporting rates (65.7%, 63.3% and 63.6%, respectively), while the guidelines by China and European multinational organizations had the lowest rate (49.6% and 49.7%, respectively) ().

Figure 3

The reporting proportion of included clinical practice guidelines in the stratified analyses. IF, impact factor.

Discussion

In this appraisal of 49 CPGs for lung cancer using the RIGHT checklist, the overall reporting rate reached 57%, with 18 CPGs having a rate higher than 60%. The guidelines with high reporting rates had some common features: they were often published in journals with high IF, and they were developed by medical specialty societies who are likely to have internal guidelines for systematic, explicit and rigorous guideline development methodology, such as the ASCO Guideline Program (59). Because the RIGHT checklist only addresses reporting of CPGs, assessing the quality of published CPGs was beyond the scope of this analysis. A high reporting rate indicates good reporting quality, but does not imply that the recommendations are in line with the best practice for disease management.

The items in the “basic information” and “background” domains were well reported in the CPGs included in our analysis. Previous studies assessing other diseases have also found high reporting rates in these two domains (60,61). However, some improvements in these areas are still needed. Item 1b [describe the year of publication (in the title)] had the lowest reporting rate within these two domains. Even many guidelines that otherwise adhered well to the RIGHT checklist, such as some guidelines developed by ASCO, ASTRO and CHEST, did not adhere to this item (14,21,58). Interestingly, showing the publication year within the title is important for readers to see immediately if the recommendations are up to date. In addition, less than half of the CPGs provided a summary of the recommendations (item 2), and almost 60% did not describe the selection process, roles and responsibility of the contributors (item 9a).

The “recommendations” domain achieved a reporting rate of 55.4%, which is higher than in CPGs for some other diseases (60,61). However, the items that are associated with explaining the recommendation and describing the decision process remained poorly reported. The aim of all lung cancer CPGs is to synthesize the best expert recommendations, but the applicability of the recommendations is also influenced by many factors, such as racial/ethnic and age disparities, geographic location, and the type of healthcare facility (62-65). Treatment costs also need to be taken into consideration (66). CPG developers should strive to account for these health disparities in order to provide the highest level of cancer care; therefore, this is particularly important for reporting these recommendations.

With very few exceptions, the guidelines performed poorly in the domain “Review and quality assurance”, with a reporting proportion of less than 20%. In particular, only one CPG indicated whether the guideline was subject to a quality assurance process (58). Although 11 guidelines reported limited information about an external review, the details of how the review was executed and how the feedback from the review was utilized were lacking. The reason for poor reporting in this domain is difficult to ascertain. One possible reason may result from the assumption that the readers may consider the review process less important than the adequacy of clinical content in CPGs. The limitations of the journal layout may also devote to poor reporting. However, we believe that improved reporting of the details of the review and quality assurance methods will reinforce the transparency of review process, thereby improving the overall quality of the CPGs.

Only one CPG described the role of the funders in the development, dissemination and implementation of the guideline. This finding may reflect a lack of awareness among CPG developers on the importance of the dissemination and implementation strategies. Adequate dissemination and implementation strategies can improve the behavior of health care providers and consequently patient outcomes (67).

CPGs developed by Chinese institutions and published in Chinese-language journals had generally low reporting quality. This result is consistent with previous studies that have revealed the lower methodological quality of Chinese CPGs (68). The poor quality may raise concerns about the true value of Chinese CPGs. An exception was the Pan-Asian adapted CPG for the management of patients with NSCLC, a guideline initiated by the Chinese Society of Clinical Oncology (CSCO) and European Society of Medical Oncology (ESMO), which was published in a high-impact journal and had a good reporting quality (24). Nevertheless, Chinese developers should strengthen the rigor and applicability in the development of guidelines, to provide better recommendations and medical service in China.

Limitations

Our study has several limitations. First, we only included CPGs published in English and Chinese between the years 2018 and 2020. Second, the RIGHT checklist is intended to assess only the reporting quality. This assessment does, thus, not reflect the quality of the recommendations or the strength of the evidence, or the overall quality of the guideline. Third, we rated all items using a dichotomous scale; bias was inevitable when rating items that in reality were partially reported. However, we do not believe this is a substantial contributor to the inconsistent results across guidelines. Fourth, our review was limited to descriptive analyses. Because of the expected small number of guidelines, we did not conduct a regression analysis to study associations between the characteristics and reporting quality. Finally, we found no previously published studies evaluating the reporting quality of lung cancer guidelines, and we were thus unable to examine whether the launching of the RIGHT instrument improved the reporting quality.

Questions to be further discussed and considered

Question 1: What impact do you think the low reporting quality of clinical practice guidelines on lung neoplasms will have on clinicians and clinical practices?

Expert opinion: Dr. Florian Kocher and Dr. Andreas Seeber

The ultimate goal of CPGs is to guide treatment in clinical practice and therefore have a strong impact on medical decisions. According to an early study, it was assumed that low quality guidelines may cause harm to patients and might waste medical resources (69). It can be assumed that the quality of guidelines, regarding formal aspects and content, have improved within the last decades. In our point of view, the RIGHT standards and AGREE II (70) are powerful tools to improve and measure quality of CPGs. With regard to lung cancer, guidelines we found a mean overall reporting proportion of 57.9%. This implicates that further improvements are necessary to enhance quality. In 2016, Heins et al. (71) evaluated adherence to cancer guidelines in the Netherlands. Interestingly, adherence to the different treatment guidelines ranged widely between the different cancer entities. Adherence to guidelines regarding lung cancer was substantially lower (57%) compared to recommendations on treatment of malignant melanoma (99%). A recently published study evaluated the reporting quality using the RIGHT statement across cancer guidelines of the NCCN (72). The highest reporting proportion was observed for the acute lymphoblastic leukemia guideline (60.0%) followed by melanoma, amyloidosis, B-cell neoplasms, anal cancer, colorectal cancer (58.6%, each). The reporting proportion for NSCLC and SCLC was slightly lower with 55.7% and 52.9%, respectively. To the best of our knowledge, the association between quality of CPGs and their impact on clinical decision making have not been studied so far. However, increased adherence to guidelines due to improvements in GPCs reporting quality is conceivable.

Expert opinion: Dr. Cesare Gridelli

Often this is cause of low adherence to guidelines by physicians with patients undertreated. For instance is well known the low adherence to concurrent chemo-radiation in locally advanced NSCLC or limited SCLC favouring often an inappropriate sequential approach.

Expert opinion: Dr. Mariano Provencio

It is essential to have a high level of quality and to follow the recommendations for the elaboration of clinical practice guidelines, if these do not follow quality control they will lose credibility for the readers.

Expert opinion: Dr. Nobuhiko Seki

Guidelines are the most important indicators for practicing standard treatment.

In terms of ‘Evidence’, if the balance between toxicity and efficacy and the balance between cost and efficacy are not sufficiently examined, only the expectation for the efficacy may be left alone and therefore patients are more likely to suffer from toxicity and cost.

In addition, even if the treatment has not been established as effective, once it is recommended by the guidelines, doctors are more likely to practice the treatment easily, and many patients will continue to be disadvantaged.

Also, from the perspective of ‘Review and quality assurance’ and ‘Funding and declaration and management of COI’, there is an increased risk that treatment recommendations based on misinterpretations or arbitrary judgments will be used as the established guideline.

Expert opinion: Dr. Yusuke Tomita

The low reporting quality of clinical practice guidelines on lung neoplasms may mislead clinicians and may negatively impact on patients’ clinical course

Question 2: What do you think the most important aspects needed for developing high-quality clinical practice guidelines on lung neoplasms are?

We assume that adherence to the STRING statement facilitates development of high-quality CPGs. Therefore, the checklist should be taken into account prior to the developmental process. Another important aspect is a holistic review of the available literature and timely incorporation of new and practice changing results. For this reason, it should be stated whether the information is based on new systematic reviews or already existing literature and should provide the search strategy to improve transparency. With respect to CPGs dealing with lung cancer treatment, a lack of information was observed. To decipher the value of certain recommendations, the level of evidence has to be provided to draw certain conclusions in the clinical routine.

According to our analysis, the year of publication was reported in only 36.7% of guidelines. Thus, it might be challenging to capture the recentness of the included evidence for the readership and might lead to obsolete treatment algorithms. Therefore, the inclusion of the year published is highly recommended. Finally, guidelines should take into account different local capabilities (i.e., low income countries) to allow optimal patient care.

One of the most important aspects is the multidisciplinary team of panelists in particularly in the early stages and locally advanced disease. Some associations (thoracic surgeons, radiotherapists, medical oncologists) have an own vision that need to be shared with other physician categories (including pathologists, radiologists, pneumologists and others). Furthermore, high quality processes should consider assessment evaluations of adherence to guidelines.

Medical journals should reject and be more selective in accepting clinical practice guidelines for publication if they do not follow adequate quality parameters. The importance and influence of guidelines on clinicians is enormous and if they are based on insufficiently supported data, they can cause terrible damage.

I think it is the most important to clarify the ranking of how important the 7 domains included in the RIGHT checklist are in order to develop high-quality guidelines in clinical practice.

In my opinion, ‘Evidence’ domain is the most important part of the RIGHT checklist and is the basis for delivering truly scientifically effective treatments to patients.

I think the next most important domain after ‘Evidence’ is ‘Recommendations’.

If this quality is not maintained, such guidelines will not be useful for patients’ treatment choices in various clinical settings.

In addition, by clarifying the process of discussion up to the decision of the recommendation level in the guidelines, I think doctors are more likely to choose the more appropriate treatment even if they are uncertain about the treatment choice.

However, I was very surprised to see the results of these reporting rates of ‘Evidence’ and ‘Recommendations’.

Of the 12 items included in these 2 domains, 8 items had reporting rates of less than 50%.

On the other hand, in my opinion, the 5 domains other than ‘Evidence’ and ‘Recommendations’ are primarily responsible for supporting the quality of these 2 domains.

Currently, new drugs and new treatment options for lung neoplasms are frequently approved. Therefore, the frequent update of clinical practice guidelines are required.

Question 3: How do you think conflicts of interest in the guidelines should be handled?

The readership of CPGs should be made aware of funding sources and potential conflicts of interest. The role of funding, in particular, was provided in only one guideline. Nevertheless, this topic is important since it improves transparency and helps the readers to acknowledge such conflicts and potential bias. If such information is not provided it might question the integrity and quality of the provided guideline.

On my opinion conflict of interests should be declared by the Guidelines Panelists but not should be a limitation in case of usual fees from companies for advisory board, speaker bureau and consultant activities

The reporting of conflicts of interest should be mandatory and cause for rejection if they are not well documented or if any anomaly is detected. Independence in the preparation of these reports necessarily requires transparency in the reporting of conflicts of interest.

In my opinion, as mentioned in Question 2, conflicts of interest are primarily responsible for supporting the quality of the domains of ‘Evidence’ and ‘Recommendations’. However, there is no change in the importance.

The funding for the development of the guidelines should be provided by the academic society, not by any external organizations.

Persons with conflicts of interest above the specified level should not be involved in guideline development work.

Even if the conflicts of interest falls within the specified range, the committee members involved in the development of the guidelines should publish the conflicts of interest for at least the past three years and be reviewed and approved by the conflicts of interest management committee of the academic society.

Conflicts of interest status of all authors who involved in the guideline development should be disclosed.

Conclusions

In summary, this analysis revealed that reporting in CPGs for lung cancer is suboptimal, although some domains were relatively well reported. In particular, the reporting of the independent review and quality assurance process needs to be improved.

The article’s supplementary files as