Complementary and integrative medicine mention and recommendations: A systematic review and quality assessment of lung cancer clinical practice guidelines.

BACKGROUND: Complementary and integrative medicine (CIM) use is widely sought by those diagnosed with cancer, with up to 50% of lung cancer patients seeking these therapies in the United States. The purpose of this study was to identify the quantity and assess the quality of CIM recommendations in clinical practice guidelines (CPGs) for the treatment and/or management of lung cancer.
METHODS: A systematic review was conducted to identify lung cancer CPGs. MEDLINE, EMBASE and CINAHL were searched from 2008 to 2018, along with the Guidelines International Network and the National Center for Complementary and Integrative Health websites. Eligible guidelines containing recommendations for the treatment and/or management of lung cancer were assessed with the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument.
RESULTS: From 589 unique search results, 4 guidelines mentioned CIM, of which 3 guidelines made CIM recommendations. Scaled domain percentages from highest to lowest were: scope and purpose (82.4% overall, 76.9% CIM), clarity and presentation (96.3% overall, 63.0% CIM), editorial independence (61.1% overall, 61.1% CIM), rigour of development (62.5% overall, 54.9% CIM), stakeholder involvement (66.7% overall, 42.6% CIM) and applicability (29.9% overall, 18.8% CIM). Quality varied within and across guidelines.
CONCLUSION: Guidelines that scored well could serve as a framework for discussion between patients and healthcare professionals regarding use of CIM therapies in the context of lung cancer. Guidelines that scored lower could be improved according to the AGREE II instrument, with insight from other guidelines development resources.

Background

Lung cancer is defined as the uncontrolled growth of cells originating in the lungs. There are two major types of lung cancer, non-small cell lung cancer (NSCLC) and small cell lung cancer (SCLC), which are both treated very differently. NSCLC is found in 80–85% of all lung cancers while SLC is much rarer, comprising only 10–15% of all lung cancers. According to the American Cancer Society, there will be an estimated 228,820 new cases of lung cancer and 135,720 deaths in the United States in 2020. Research has shown that complementary and integrative medicine (CIM) use among lung cancer patients ranges between 20-40%.4, 5, 6, 7 Complementary medicine is defined as a non-mainstream practice or procedure that is not part of standard medical care, used in combination with conventional medicine. Integrative medicine is defined as conventional and complementary approaches brought together in a coordinated way to treat the patient holistically.8, 9

CIM is comprised of a wide range of therapies. Some common CIM therapies used for lung cancer include acupuncture, herbal supplements and vitamins, yoga, meditation, medical cannabis, massage therapy and aromatherapy.4, 10, 11 Mind-body modalities such as yoga and meditation involve integration of breathing and meditation in order to improve aspects disrupted or caused by lung cancer such as sleep, stress and chemotherapy-induced nausea. Acupuncture is mainly used to manage cancer-related symptoms such as nausea, pain and fatigue, as well as increase blood cell count and lymphocyte activity. Many herbal therapies allegedly have anticancer potential through inhibition of cell proliferation, often via the apoptotic pathway. While conventional practitioners are encouraged to work with CIM practitioners when the patient chooses CIM therapies as part of their lung cancer treatment regimen, the majority of such practitioners receive little to no training in CIM.15, 16 This can discourage communication between the patient and clinician regarding CIM, thereby creating barriers in the treatment plan.15, 16

Healthcare professionals rely on the use of clinical practice guidelines (CPGs) as an evidence-based framework for guiding decision-making with patients regarding various therapies. CPG developers optimize patient care by providing recommendations based on a systematic review of current treatment evidence, and an assessment of therapy risks and benefits. This leads to accessible and higher quality outcomes for patients. To our knowledge no study has assessed the quantity nor evaluated the quality of CIM recommendations in CPGs for the treatment and/or management of lung cancer; this is the purpose of the present study.

Methods

Approach

A systematic review was conducted to identify CPGs for treatment and/or management of lung cancer using standard methods and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria. A protocol was registered with PROSPERO, registration number CRD42019132301. CPGs containing CIM recommendations were assessed with the widely used and validated Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument. These CPGs were assessed twice: once to assess the overall CPG, and once to assess only the CIM section of the CPG. AGREE II consists of 23 items grouped in six domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence.

Eligibility criteria

Eligibility criteria for lung cancer CPGs were based on the Population, Intervention, Comparison and Outcomes framework. Eligible populations were adults aged 19 years and older and diagnosed with lung cancer. With respect to interventions, we only included evidence-based CPGs that included recommendations surrounding the treatment and/or management of lung cancer in order to determine whether any mention or recommendations of CIM therapies were included. In determining what constituted a CIM therapy, we referred to the list provided by the NCCIH. There were no comparisons. Outcomes were AGREE II scores which reflect CPG content and format. The following conditions were also applied to define eligible CPGs: developed by non-profit organizations; published in 2008 or later; available in the English language; and either publicly available or could be ordered through our library system. Publications in the form of consensus statements, protocols, abstracts, conference proceedings, letters or editorials; based on primary studies that evaluated lung cancer management or treatment (i.e. clinical trials); or focused on lung cancer curriculum, education, training, research, professional certification or performance were not eligible. It should be noted that only eligible CPGs that contained CIM therapy recommendations were assessed using the AGREE II tool, in order to determine the difference in AGREE II scores between the overall CPG and specifically the CIM sections; only demographic information is reported for eligible CPGs that did not contain CIM therapy recommendations.

Searching and screening

MEDLINE, EMBASE and CINAHL were searched from 2008 to 2018 inclusive on October 12, 2018. The search strategy (Supplementary File 1) included Medical Subject Headings and keywords that reflect terms commonly used in the literature to refer to lung cancer. We also searched the Guidelines International Network, a repository of guidelines using keyword searches restricted based on the eligibility criteria including ‘lung cancer’ and ‘lung neoplasms’. Next, we searched the NCCIH web site which contained a single list of CIM guidelines. HN and ZN screened titles and abstracts from all other sources. HN and ZN screened full-text items to confirm eligibility. JYN reviewed the screened titles and abstracts and full-text items to standardize screening, and helped to discuss and resolve selection differences between HN and ZN.

Data extraction and analysis

The following data were extracted from each CPG and summarized: date of publication, country of first author; type of organization that published the guideline (academic institutions, government agencies, disease-specific foundations, or professional associations or societies); and whether any CIMs were mentioned in this guideline. If CIMs were mentioned in a guideline, the types of CIM mentioned, CIM recommendations made, CIM funding sources, and whether any CIM providers are part of the guideline panel were also data extracted. Most data were available in the guideline; to assess applicability, the website of each developer was browsed and searched for any associated knowledge-based resources in support of implementation.

Guideline quality assessment

Eligible lung cancer CPGs were scored using the AGREE II instrument based on the user manual’s instructions. All three authors participated in a pilot test and independently assessed three CPGs using the AGREE II instrument. Any discrepancies between their scores were discussed and resolved. HN and ZN then applied the AGREE II instrument once to all eligible lung cancer CPGs and twice to those containing mention of CIM recommendations (once for the 'overall' CPG and once for the 'CIM' sections). Six domains covering 23 items were rated using a seven-point Likert scale that ranged from strongly disagree (1) to strongly agree (7) that the item contains the relevant criteria. HN and ZN then rated the whole CPG (1–7) based on the individual item scores, which was used to either recommend, recommend with modifications or not recommend the CPG for clinical use. The modified AGREE II questions used to guide the scoring of the 'CIM' sections of each CPG are found in Supplementary File 2. JYN resolved any differences between the appraiser’s scores. Average appraisal scores were calculated by finding the average rating for all 23 items of a single appraiser followed by averaging this value for both appraisers for a single CPG. Average overall assessments were calculated by averaging both appraisers’ 'overall guideline assessment' for each CPG. Scaled domain percentages were generated to compare the different domain ratings by adding both appraisers’ ratings of the items within each domain and scaling by the maximum and minimum possible domain scores. Average appraisal scores, average overall assessments, and scaled domain percentages were compared for each CPG.

Results

Search results (Fig. 1)

Searches retrieved 819 items, 589 were unique, and 558 titles and abstracts were eliminated, leaving 31 full-text guidelines that were considered. Of those, 4 were not eligible, because a newer guideline update was found, and 1 was an abridged version (full version in German), leaving 26 CPGs eligible for review. Of these 26 CPGs, 4 made mention of CIM therapies, of which only 3 made CIM therapy recommendations.

PRISMA Diagram.

Guideline characteristics (Table 1)

Eligible CPGs were published from 2008 to 2018, with first authors based in USA (n = 9), Canada (n = 5), the Netherlands (n = 1), England (n = 1), Italy (n = 1), Switzerland (n = 1), Scotland (n = 1), Singapore (n = 1), China (n = 1), France (n = 1), Australia (n = 1), Spain (n = 1), and Saudi Arabia (n = 1); additionally one CPG’s first author listed the Netherlands and Belgium.24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49 The CPGs were funded and/or developed by professional associations or societies (n = 26). Four CPGs made mention of CIMs.34, 39, 40, 47 These CIMs included exercise (n = 3), nutrition/dietary supplements (n = 3), social and spiritual support (n = 2), self-care strategies (n = 1), yoga (n = 1), massage therapy (n = 1), acupuncture (n = 1), Tai Chi/Qi Gong (n = 1), hypnosis (n = 1), music therapy (n = 1), meditation (n = 1), mind-body modalities (n = 1) and relaxation (n = 1). Recommendations relating to CIM were made in three CPGs, and related to social and spiritual support (n = 2), self-care strategies (n = 1), yoga (n = 1), massage therapy (n = 1), exercise (n = 1), acupuncture (n = 1), nutrition (n = 1), dietary supplements (n = 1), mind-body modalities (n = 1); only these CPGs were assessed using the AGREE II tool. CIM funding sources were not identified in any of the CPGs, and one CPG included CIM experts as part of the guideline panel. We provide a summary of CIM recommendations made across lung cancer CPGs for the benefit of clinicians and researchers in Fig. 2.

Fig. 2

Summary of CAM Recommendations in Clinical Practice Guidelines for the Treatment and/or Management of Lung Cancer.

Guidelines mentioning CIM without recommendations

One CPG made mention of CIM, specifically citing the evidence for conditioning and relaxation exercise, but did not make recommendations about these therapies. The CPG mentioned this CIM in their explanation of a study consisting of patients with advanced cancer undergoing radiation. The patients were assigned a structured multidisciplinary intervention which included weekly exercise sessions. This study was used to highlight the role of palliative care in stage IV NSCLC and the CIM therapy was only mentioned very briefly and vaguely.

Average appraisal scores, average overall assessments and recommendations regarding use of guidelines: overall guideline (Table 2)

Table 2

Average appraisal scores and average overall assessments of each guideline.

Guideline	Metric	Appraiser 1	Appraiser 2	Average	Standard deviation
CCA 201734 (Overall)	Appraisal score	4.3	4.3	4.3	0.0
	Overall assessment	4.0	4.0	4.0	0.0
CCA 201734 (CIM section)	Appraisal score	3.7	3.2	3.4	0.1
	Overall assessment	5.0	5.0	5.0	0.4
SIGN 201439 (Overall)	Appraisal score	4.8	5.0	4.9	0.2
	Overall assessment	5.0	5.0	5.0	0.0
SIGN 201439 (CIM section)	Appraisal score	3.1	3.5	3.3	0.4
	Overall assessment	3.0	3.0	3.0	0.0
Deng 201347 (Overall)	Appraisal score	5.3	5.3	5.3	0.1
	Overall assessment	5.0	5.0	5.0	0.0
Deng 201347 (CIM section)	Appraisal score	5.3	5.3	5.3	0.1
	Overall assessment	5.0	5.0	5.0	0.0

Average appraisal scores, average overall assessments, and recommendation regarding use for each CPG are shown in Table 2. The average appraisal scores for each of the three CPGs ranged from 4.3 to 5.3 on the seven-point Likert scale (where 7 equals strongly agree that the item is met); all three CPGs achieved or exceeded an average appraisal score of 4.0, and one CPG achieved or exceeded an average appraisal score of 5.0. Average overall assessments for the three CPGs ranged between 4.0 (lowest) and 5.0 (highest), including one CPG equalling or exceeding a score of 4.0, and two CPGs equalling or exceeding a score of 5.0.

Average appraisal scores, average overall assessments and recommendations regarding use of guidelines: CIM sections

Average appraisal scores, average overall assessments, and recommendation regarding use for each CPG are shown in Table 2. The average appraisal scores for each of the three CPGs ranged from 3.3 to 5.3 on the seven-point Likert scale (where 7 equals strongly agree that the item is met); two CPGs achieved or exceeded an average appraisal score of 3.0, and one CPG achieved or exceeded an average appraisal score of 5.0. Average overall assessments for the three CPGs ranged between 3.0 (lowest) and 5.0 (highest), including two CPGs equalling or exceeding a score of 3.0, and one CPG equalling or exceeding a score of 5.0.

Overall recommendations: overall guideline (Table 3)

None of the three CPGs were recommended Yes or No by both appraisers. Appraisers agreed in their overall recommendation for 3 out of 3 CPGs, all stating Yes with modifications.34, 39, 47

Overall recommendations: CIM sections (Table 3)

None of the three CPGs were recommended Yes by both appraisers. Appraisers agreed in their overall recommendation for 3 out of 3 CPGs including two No,34, 39 and one Yes with modifications.

Scaled domain percentage quality assessment (Table 4)

With regards to scaled domain percentages of the overall CPG, scope and purpose scores ranged from 63.9% to 91.7%, stakeholder involvement scores ranged from 62.9% to 72.2%, rigor-of-development scores ranged from 49.0% to 80.2%, clarity-of-presentation scores ranged from 91.7% to 100.0%, applicability scores ranged from 16.7% to 45.8%, and editorial independence scores ranged from 45.8% to 87.5%. With regards to scaled domain percentages of the CIM CPG sections, scope and purpose scores ranged from 63.9% to 91.7%, stakeholder involvement scores ranged from 22.2% to 63.9%, rigor-of-development scores ranged from 38.5% to 80.2%, clarity-of-presentation scores ranged from 36.1% to 100.0%, applicability scores ranged from 16.7% to 22.9%, and editorial independence scores ranged from 45.8% to 87.5%.

Scope and purpose

The objectives for both the ‘overall’ and ‘CIM’ sections were generally well-defined in all but one CPG which lacked clear outcomes and targets, in addition to their health intents. Similarly, the population to whom the CPG was meant to apply for both the ‘overall’ and ‘CIM’ sections was well-defined in all but one CPG which vaguely referenced the intended population as “extensive stage SCLC”. The health questions were clearly defined in all CPGs for the ‘overall’ section, but one CPG did not cover the CIM therapies later recommended and therefore scored lower in their ‘CIM’ section.

Stakeholder involvement

All three CPGs thoroughly detailed the characteristics of the members of the CPG development group for the ‘overall’ sections, including statements of their affiliations, disciplines, institutions, geography and role in the CPG development process.34, 39, 47 However, two of the CPGs did not involve CIM experts as part of their CPG development group which therefore accounted for the lower score in their ‘CIM’ section.34, 39 All three CPGs did not go into detail regarding the views and preferences of the target population for both the ‘overall’ and ‘CIM’ sections.34, 39, 47 Target users of the CPG were well-defined in two of the CPGs for both the ‘overall’ and ‘CIM’ sections,39, 47 which offered specific descriptions of the types of users, (i.e. which type of specialist/health care provider) and how this CPG can be used. The other CPG lacked a clear statement of the target users and vaguely explained how the CPG can be used for both the ‘overall’ and ‘CIM’ sections

Rigor of development

Systematic methods to search for evidence were used in all three CPGs for both the ‘overall’ and ‘CIM’ sections and the full search strategy was almost always clearly described39, 47 with the exception of one CPG that listed databases but lacked time periods searched and search terms. The criteria for selecting evidence was generally well-defined in two of the CPGs for both the ‘overall’ and ‘CIM’ sections34, 47 but missing in one. The strengths and limitations of the body of evidence were clearly defined in all CPGs for the ‘overall’ section, but were missing in the ‘CIM’ section of one CPG. The methods for formulating recommendations were poorly defined in all CPGs for both the ‘overall’ and ‘CIM’ sections with the exception of one which clearly described how consensus was reached. The health benefits, side effects, and/or risks were typically inconsistently defined across the CPGs with one CPG reporting detailed supporting data for both the ‘overall’ and ‘CIM’ sections while the others listed a limited amount, missed the trade-off and only vaguely reflected them within the recommendations.34, 39 For this item, the ‘CIM’ section of one CPG scored lower than its ‘overall’ section All CPGs provided an explicit link between the recommendations and supporting evidence for both ‘overall’ and ‘CIM’ sections.34, 39, 47 Authors linked each recommendation to key evidence paragraphs and/or a reference list. Two of the CPGs mentioned they were externally reviewed by experts prior to publication but were missing outcomes as well as a description of information gathered and reviewers.39, 47 The other CPG did not state it had been externally reviewed. A procedure for updating the CPGs for both the ‘overall’ and ‘CIM’ sections was inconsistently reported among the CPGs as there was often no explicit time interval listed34, 47 and the methodology, if provided, was not detailed.47, 39

Clarity of presentation

Almost all the CPGs offered specific and unambiguous recommendations with the exception of the ‘CIM’ section of one CPG, which provided only a vague statement of the CIM recommendation. All of the CPGs presented a clear description of the different options as well as the clinical situation appropriate for each option for the ‘overall’ section.34, 39, 47 However, the ‘CIM’ sections of two CPGs34, 39 were scored lower as both CPGs provide one CIM therapy each and they were not thoroughly elaborated upon. The key recommendations for both the ‘overall’ and ‘CIM’ sections were easily identifiable within all CPGs34, 39, 47

Applicability

Most of the CPGs vaguely mentioned a few facilitators and barriers to implementation of the recommendations but lacked methodological details and sufficient description as to how they influenced formulation of recommendations,34, 39, 47 accounting for the low score in this item for both ‘overall’ and ‘CIM’ sections. The offering of advice and/or tools on how recommendations can be put into practice for both the ‘overall’ and ‘CIM’ sections was inconsistent among the three CPGs as some provided sufficient additional materials including: summary document,34, 39 practitioner guide, list of additional resources34, 39, 47 and an implementation section,34, 39, 47 but were either vague or did not have resources applicable to their ‘CIM’ section. None of the CPGs adequately addressed the resource implication of implementing the recommendations for both the ‘overall’ and ‘CIM’ sections, although two39, 47 briefly mentioned cost. Two CPGs failed to provide monitoring and auditing criteria with the exception of one CPG for its ‘overall’ section, which stated vague follow-up details pertaining to their non-CIM recommendations.

Editorial independence

Two of the CPGs did not explicitly disclose their funding nor state that the funding body did not influence the contents of the CPG.39, 47 The other CPG stated its primary funding body. Two of the CPGs clearly recorded and addressed the competing interests of the CPG development groups34, 47 while the other CPG stated their conflicts, but failed to declare the extent of their influence on the CPG development process.

Discussion

The purpose of this study was to identify the quantity and assess the quality of CIM recommendations in CPGs for the treatment and/or management of lung cancer in order to identify evidence-based resources that could better facilitate informed decision-making among patients with lung cancer and their healthcare providers. This review identified 26 CPGs published between 2008 and 2018 that included recommendations for the treatment and/or management of lung cancer; four CPGs were found to have mentioned CIM, and three CPGs made CIM therapy recommendations. In assessing the overall CPG, one CPG scored 5.0 or higher in both average appraisal score and average overall assessment, and two CPGs scored 4.9 or lower in both of these metrics.34, 39 In assessing the CIM section of each CPG, one CPG scored 5.0 or higher in both average appraisal score and average overall assessment, and two CPGs scored 3.4 or lower in both of these metrics.34, 39

In this study, the scaled domain percentages for the 'overall' CPGs from highest to lowest were clarity of presentation (96.3%), scope and purpose (82.4%), stakeholder involvement (66.7%), rigour of development (62.5%), editorial independence (61.1%) and applicability (29.9%). The scaled domain percentages for the 'CIM' section of the CPGs from highest to lowest were scope and purpose (76.9%), clarity of presentation (63.0%), editorial independence (61.1%), rigour of development (54.9%), stakeholder involvement (42.6%) and applicability (18.8%). While there are differences, the two sections still have a similar ranking of domains, notably with clarity of presentation and scope and purpose consistently being the highest scoring, and applicability being the lowest. These findings are comparable to a 2015 study which evaluated eight CPGs pertaining to lung cancer screening using the AGREE II tool. Their results showed a similar ranking of domains, as follows: scope and purpose (84.5%), clarity and presentation (76.9%), editorial independence (65.9%), stakeholder involvement (51.1%), rigour of development (50.9%) and applicability (24.9%). As in our study, the quality of each selected CPG varied, therefore there is potential to improve CPG quality across each AGREE II domain.

To our knowledge, no previous studies have assessed the quantity and quality of CIM therapy recommendations in lung cancer CPGs. Thus, this is the first study to assess the credibility and nature of CIM therapy recommendations in lung cancer CPGs. In a previous study lead by JYN, it was found that in 17 complementary and alternative medicine-specific CPGs across various diseases and conditions, the scaled domain percentages were ordered in a similar fashion from highest (clarity of presentation 85.3 %) to lowest (applicability 20.7 %). Similarly, the quality varied within and across this subset of CPGs, therefore, the sub-optimal and variable quality of CPGs is not unique to this study. Additionally, it has been found that CAM recommendations across clinical practice guidelines for a variety of diseases/conditions vary both in quality and quantity.52, 53, 54

By describing the quantity and quality of lung cancer CPGs containing CIM recommendations, this study has revealed that only a small number of CPGs are available to support informed, decision-making among patients and health care professionals in regard to CIM therapies, with only one providing recommendations for a comprehensive list of CIM. While the reason for this finding can be attributed to a range of different factors, it is likely a reflection of the lack of research on CIM therapies, especially given that a lack of high-quality randomized controlled trials exist. Some factors that challenge CIM research include a lack of research training of CIM practitioners and/or lack of training in CIM knowledge across medical researchers, biases against the use of CIM medicine, and a lack of resources/funding to increase CIM knowledge.56, 57 Given the large public interest and increasing use of CIM therapies, it is important that continued and rigorous research continues in this area. In addition to the AGREE II assessment tool, there are numerous principles, frameworks, checklists and criteria to assist CPG developers in generating high-quality CPGs, including those pertaining to CIM therapies.58, 59, 60, 61

Strengths and limitations

Strengths of this study include use of a systematic approach to reviewing treatment and/or management CPGs pertaining to lung cancer, as well as use of the AGREE II instrument for assessing their methodological quality, an international standard for the appraisal of CPGs. Some limitations of this study include the independent assessment of CPGs by two appraisers instead of the recommended four to optimize reliability. This research limitation was mitigated through the implementation of an AGREE II pilot test in which JYN, HN and ZN each appraised three independent CPGs and then reached consensus on how to apply the instrument accordingly. This was conducted in order to standardize scores and resolve any uncertainties in using the tool. Furthermore, all CPGs that included CIM therapy recommendations for lung cancer may not have been identified based on our English-only eligibility criteria, and therefore may not capture CPGs developed in countries where they may be published in other languages (i.e. traditional Asian medicine recommendations in Korean and Chinese CPGs). For example, one abridged CPG with CIM recommendations (albeit general, and not specific to any particular CIM) was found, however, it was not assessed as the methodology used to develop the CPG was only available in the full version, which was published in German.

This study identified 3 CPGs for the treatment and/or management of lung cancer providing CIM therapies recommendations, including social and spiritual support, self-care strategies, yoga, massage therapy, exercise, acupuncture, and nutrition/dietary supplements. Appraisal of these CPGs using the AGREE II instrument indicated quality variation both within and across CPGs. Some of the CPGs that achieved higher AGREE II scores and favourable recommendations could be used by patients and healthcare professionals as a framework for discussion regarding use of CIM therapies. In the future, CPGs that achieved variable or lower scaled domain percentages could be updated and improved according to the AGREE II instrument and/or with insight from the variety of resources available to aid CPG development. The fact that few CIM recommendations are available for use by healthcare professionals to guide informed decision-making and open discussions about CIM use with their lung cancer patients is of concern given that the prevalence of CIM is high across this patient population. This lack of CPG development and research may foster use of CIM for which there are no proven benefits, potential health risks, and/or underuse of potentially beneficial CIM.

Author contributions

Conceptualization: JYN. Investigation: JYN, HN, ZN. Formal Analysis: JYN, HN, ZN. Writing - Original Draft: JYN. Writing - Review & Editing: JYN, HN, ZN.

Conflict of interests

The authors declare that they have no competing interests.

Funding

JYN was awarded a Research Scholarship and an Entrance Scholarship from the Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences at McMaster University.

Ethical statement

This study involved a systematic review of peer-reviewed literature only; it did not require ethics approval or consent to participate.

Data availability

All relevant data are included in this manuscript.

Table 1

Characteristics of eligible guidelines.

Guideline	Country (first author)	Developer	CIM category	Guideline topic
Sun 201824	Canada	Cancer Care Ontario’s Program in Evidence-Based Care, the Lung Cancer Disease Site Group	None	Initial management of small cell lung cancer (limited and extensive stage) and the role of thoracic radiotherapy and first-line chemotherapy
Planchard 201825	France	European Society for Medical Oncology	None	Diagnosis, treatment and follow-up of metastatic non-small cell lung cancer
Majem 201826	Spain	Spanish Society of Medical Oncology	None	Treatment of non‐small cell lung cancer
Swaminath 201727	Canada	Cancer Care Ontario’s Program in Evidence-Based Care, the Lung Cancer Disease Site Group	None	Treatment of patients with stage III (N2 or N3) non-small cell lung cancer
Postmus 201728	England	European Society for Medical Oncology	None	Diagnosis, treatment and follow-up of early and locally advanced non-small-cell lung cancer
Kris 201729	United States	American Society of Clinical Oncology, Cancer Care Ontario	None	Adjuvant systemic therapy and adjuvant radiation therapy for stage I to IIIA completely resected non–small-cell lung cancers
Jazieh 201730	Saudi Arabia	The Saudi Lung Cancer Guidelines Committee	None	Lung cancer management
Hanna 201731	United States	American Society of Clinical Oncology	None	Systemic therapy for stage IV non–small-cell lung cancer
Falkson 201732	Canada	Cancer Care Ontario’s Program in Evidence-Based Care, the Lung Cancer Disease Site Group	None	Radiotherapy with curative intent in patients with early-stage medically inoperable non-small-cell lung cancer
De Ruysscher 201733	Netherlands, Belgium	European Organization for Research and Treatment of Cancer	None	Planning and delivery of high-dose, high precision radiotherapy for lung cancer
CCA 201734	Australia	Cancer Council Australia	Self-care strategies and social support	Treatment of lung cancer
Ellis 201635	Canada	Cancer Care Ontario’s Program in Evidence-Based Care, the Lung Cancer Disease Site Group	None	Systemic treatment for patients with advanced non-small cell lung cancer
Zhi 201536	China	The Council of Cancer Chemo- therapy of Chinese Anti-Cancer Association	None	Diagnosis and treatment of primary lung cancer
SCAN 201537	Singapore	The Singapore Cancer Network Lung Cancer Workgroup	None	Adjuvant chemotherapy in resected non-small cell lung cancer
Baas 201538	Netherlands	European Society for Medical Oncology	None	Diagnosis, treatment and follow-up of malignant pleural mesothelioma
SIGN 201439	Scotland	Scottish Intercollegiate Guidelines Network	Exercise, diet, spiritual support	Management of lung cancer
Socinski 201340	United States	American College of Chest Physicians	Exercise	Treatment of stage IV non-small cell lung cancer
Ramnath 201341	United States	American College of Chest Physicians	None	Treatment of stage III non-small cell lung cancer
Kozower 201342	United States	American College of Chest Physicians	None	Special treatment issues in non-small cell lung cancer
Jett 201343	United States	American College of Chest Physicians	None	Treatment of small cell lung cancer
Howington 201344	United States	American College of Chest Physicians	None	Treatment of stage I and II non-small cell lung cancer
Gould 201345	United States	American College of Chest Physicians	None	Evaluation and management of individuals with solid pulmonary nodules
Fruh 201346	Switzerland	European Society for Medical Oncology	None	Diagnosis, treatment and follow-up of small-cell lung cancer
Deng 201347	United States	American College of Chest Physicians	General CIM	Complementary therapies and integrative medicine in lung cancer
Cheng 201348	Canada	Program in Evidence-Based Care, Cancer Care Ontario	None	Chemotherapy for relapsed small cell lung cancer
deMarinis 201149	Italy	Italian Association of Thoracic Oncology	None	Treatment of advanced non-small-cell lung cancer

Table 3

Overall recommendations for use of appraised guidelines.

	Overall guideline		CIM section
Guideline	Appraiser 1	Appraiser 2	Appraiser 1	Appraiser 2
CCA 201734	Yes with Modifications	Yes with Modifications	No	No
SIGN 201439	Yes with Modifications	Yes with Modifications	No	No
Deng 201347	Yes with Modifications	Yes with Modifications	Yes with Modifications	Yes with Modifications

Table 4

Scaled domain percentages for appraisers of each guideline.

Guideline		Domain score (%)
		Scope and purpose	Stakeholder involvement	Rigour of development	Clarity of presentation	Applicability	Editorial independence
CCA 201734	Overall guideline	63.9	62.9	49.0	97.2	27.0	50.0
	CIM section	63.9	22.2	45.8	52.8	22.9	50.0
SIGN 201439	Overall guideline	91.7	72.2	58.3	91.7	45.8	45.8
	CIM section	75.0	41.7	38.5	36.1	16.7	45.8
Deng 201347	Overall guideline	91.7	63.9	80.2	100.0	16.7	87.5
	CIM section	91.7	63.9	80.2	100.0	16.7	87.5