Improving adherence to lung cancer guidelines: a quality improvement project that uses chart review, audit and feedback approach.

INTRODUCTION: The implementation of evidence-based clinical practice guidelines is one of the most effective interventions for improving quality of care. A gap between guidelines and clinical practice often exists, which may result in patients not receiving appropriate care. This project aimed at improving adherence to lung cancer guidelines at our institution.
METHOD: The records of patients with lung cancer were evaluated for adherence to guidelines by using an auditing tool that was developed to capture pertinent information. The study team collected data about the following variables: compliance with documentation of pathological diagnosis, documentation of disease stage prior to treatment initiation, presentation at thoracic tumour board within 30 days of diagnosis, management course, and management of end of life in terms of early 'no code' initiation, stopping chemotherapy and referral to palliative care prior to 2 weeks of death. Annual audits were performed from 2012 to 2015. Education and discussion with team members to address the deviations were the main interventions to improve adherence.
RESULTS: The baseline measurements were taken in 2012 (49 patients). Histological subtype identification improved from 94% to 100%. Presentation of new cases at the tumour board improved from 35% to 82%. Testing for epidermal growth factor receptor mutation for non-squamous cell lung cancer improved from 77% to 100%. The staging was documented in 100% of the cases.
CONCLUSION: Running audits to monitor adherence to guidelines and discussions with the team have a positive effect on providing consistent evidence-based care for patients with lung cancer.

Problem

Guidelines should provide a framework for managing patients. The implementation of evidence-based clinical practice guidelines is one of the effective mechanisms for improving the quality of care. A gap between guidelines and clinical practice often exists, resulting in suboptimal care and patient safety concerns.

In our practice, we use the National Comprehensive Cancer Network Clinical Practice Guidelines in Oncology (NCCN Guidelines) as a guide for our standard of care. The NCCN is a not-for-profit organisation, aims to advance the overall quality of cancer care by improving many aspects such as improving the care of patient with cancer, cancer research and education. It offers various assets that support the clinicians in their decision-making while managing patients with cancer.1 NCCN Guidelines are one of these resources, which were developed by using an evidence-based consensus process.2 Based on published data, the authors hypothesised that improving adherence to NCCN Guidelines will have positive effects on patients’ management and outcome.3–6

In our institution, as a result of case discussions at the thoracic tumour board (TTB), we realised that we have a guidelines–practice gap, thus we initiated this project aiming to quantify the level of variation in our domain of practice (baseline data), then the next step was to bridge this gap by conducting a generic and specialised educational session about the findings of the audit, and about how to access/use the NCCN Guidelines. We believe that doing so will be reflected positively in improving the quality and safety of the provided services to patients with lung cancer.

Background

As defined by the Institute of Medicine, clinical guidelines are ‘systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.’7

Several studies have shown significant variation in the use of services provided by physicians. One study showed that the frequency with which procedures are performed varies dramatically among doctors, specialties and geographical regions, even after case mix is controlled.8

In the field of oncology, adherence to guideline rates varies across types of cancer. Numerous disease-specific studies had been carried out studying the extent of recommendation compliance with NCCN Guidelines. A study on head and neck cancer reported a high rate (96%) of adherence to the guidelines,9 while other studies reported rates as low as 37% for ovarian cancer3 and 35% for pancreatic cancer.5

Another study examined the relationship between evidence-based guideline adherence and the follow-up monitoring period over 1.5 years. This study had been conducted on patients with non-small cell lung cancer treated in US community oncology practices. It was found that treatment with guideline-based regimens correlated with a significantly longer follow-up monitoring period. Furthermore, it also revealed that the rates of guideline adherence were 75.0% and 61.3% for the first-line and adjuvant treatment groups.10

In investigating clinical variation in the delivery of healthcare, one study listed several sources of variation: among them was the over-reliance on subjective judgement. Physicians may rely on their personal clinical experience as the foundation for the recommendations they make for treating patients.11 Other sources of variation could be related to patients’ factors and preferences. Due to the unique patient and/or care-setting characteristics, there will always be a degree of appropriate variation in the practice of medicine, even for patients with the same diagnoses.12 Thus, the main reason for guideline development is not to eliminate practice variation but to standardise the process, reduce the probability of error and monitoring any variation in practice.

There are two types of methods to measure guideline adherence. Self-reported measures include self-administered questionnaires and face-to-face interviews, and the objective measures include a review of medical records, discharge data, prescriptions, claims data or observation of actual practice.13

A systematic review of studies on adherence to guidelines reported that self-reports are usually subject to bias and should not be used as the sole measure of guideline adherence.13 A systematic review of the Cochrane Study Group studied the effectiveness of clinical audit and feedback concluded that the relative effectiveness of an audit is likely to be greater when baseline adherence to recommended practice is low and when feedback is carried out with greater intensity.14

For that reason, in this project, we are using the medical records review as an objective process measure for guideline adherence.

Baseline measurement

A data collection tool and process were used to measure adherence to the cases of 2012 as a baseline data.

The results of 2012 had shown that compliance with guidelines was significantly low. Only 76.7% of the cases performed epidermal growth factor receptor (EGFR) testing. 95.9% of the cases had been staged before any cancer-directed treatment. Additionally, only 35% of the cases were discussed in the TTB. Furthermore, during the audit process a patient had EGFR mutation and did not receive tyrosine kinase inhibitors (TKI), which is a targeted therapy that attacks specific receptors of cancer cells with less damage to normal cells.15 It is clear from the baseline data that this variation may result in inappropriate care for patients with lung cancer.

The above-reported rates of adherence to guidelines were measured by dividing the number of the patient population receiving the service in question by the number who should have received the service according to the guidelines. Some of the measures we had collected were: (1) percentage of compliance in documentation of pathological diagnosis, (2) percentage of cases with documented disease stage prior to treatment by physicians, (3) percentage of cases discussed in tumour boards, (4) management course, and (5) quality of end of life in terms of early ‘no code’ initiation, stopping chemotherapy 2 weeks before death and referral to palliative care before 2 weeks prior to death.

Design

Concurrent and retrospective data collection methodologies were used to fill the data collection tool, which contained all variables needed to determine the measures. Two independent research coordinators piloted the tool, and feedback was considered to modify the tool. The tool included variables concerning diagnosis such as confirming the pathological diagnosis, identifying the histological subtype of the disease, and molecular testing, for example, EGFR mutations and anaplastic lymphoma kinase (ALK) fusions. Additionally, the tool contained variables related to staging workup, that is, CT scan of the chest and pulmonary function test, as well as the documentation of the disease stage prior to initiation of cancer-directed therapy. The ‘Management’ section of the tool had detailed questions about first, second and third-line treatments received by the patient. Furthermore, data regarding stopping cancer-directed therapy and referring patients to palliative care at least 2 weeks prior to death had been captured in the tool. Since tumour board discussion is an essential function of our patient care, case presentation to the TTB was also included in the tool.

The auditing tool was subsequently modified to have more specific questions that give no room for auditors’ discretions in filling in the form. For example, the management sections were totally changed in 2013. Instead of having general terms, such as ‘Was the first line of treatment accomplished according to guidelines?’, we specifically stated the treatment type, the stage of the disease and the Eastern Cooperative Oncology Group (ECOG) performance status for better presentation of the guidelines.

In 2013, ALK testing had been added to the auditing tool to ensure proper molecular testing to guide patient selection for TKI therapy. The final tool was comprehensive, precise and self-explanatory in a way that assures the consistency of the abstraction process from the medical records across all auditors (figures 1–3).

Auditing tool used in 2012.

Auditing tool used in 2013 and 2014.

Auditing tool used in 2015.

Charts were identified for all patients diagnosed with lung cancer in the same year the audit was performed. Records reviews and abstracting data were done by staff physicians. All data abstracted were reviewed by a senior consultant for data validation.

Presentation of the feedback

On the level of the treating physicians and multidisciplinary team, the results of 2012 were communicated with the members of the multidisciplinary TTB on a biweekly basis to address any variations found during the audit. The members of TTB include oncologists, radiation oncologists, thoracic surgeons, pulmonary physicians, pathologists, radiologists, experts from nuclear medicine, nurses, patient educators, research coordinators, clinical coordinators and a data manager. Having this variety of specialties in one place made the TTB a suitable place to provide an in-depth specialised educational information and discussion about any controversial case. The TTB was also a suitable place for the treating physicians to provide explanation/justification for any case that has guideline non-compliance. Hence, presenting the audit results in such meetings ensures effective communication and cooperation towards better adherence to guidelines.

Additionally, the findings of the project had been communicated to all physicians in the Department of Oncology through departmental educational activities on many levels, for example, on the level of department, the grand rounds, departmental meeting and quality improvement sessions were used as platform to educate the staff about the importance of the project, the progress of the project, and to highlight any findings that need immediate correction or explanation.

Another platform for providing the feedback was the thoracic combined clinic that had been created later in the project. The thoracic combined clinic functions on a weekly basis where all multidisciplinary team meets physically with the patient with lung cancer. The team discusses only critical cases that need immediate actions to be taken rather than waiting for the next available appointments with other clinical departments. The aim of the clinic is to provide the best evidence-based treatment recommendations in a timely manner.

Disseminate the baseline data to all physicians in the Department of Oncology and to the multidisciplinary team who usually provide care outside the Department of Oncology (lung cancer pathologists, radiologists, thoracic surgeon, pulmonary physicians, and so on).

Assure that each new case is presented to the lung cancer tumour board within 30 days of pathological diagnosis, as several deviations could have been avoided by discussing the case in the tumour board.

Improve the documentation process.

The baseline results were shared during our monthly departmental meetings.

Inform TTB members in the biweekly meetings about the results that reflect their performance and discuss how to improve them.

Instruct our cancer registrar and our pathologist to immediately inform the TTB coordinator about any newly diagnosed patient with lung cancer, so we can list the patient’s medical record number in the next scheduled TTB meeting.

A comprehensive progress note template was developed. The aim was to standardise the documentation process and include parameters which are related to oncology practice such as disease stage, patient performance status, and so on. The new progress note was approved and added to the patient’s chart (online supplementary figure 1A: assessment for new patient with lung cancer, and online supplementary figure 1B: assessment for follow-up patient with lung cancer).

In general, the members of the TTB showed enthusiasm about the idea of auditing. Some of them were curious about the deviations and who did it, but the project leader explained to the team that the crucial step is to improve the system as a whole rather than seeing it as individual cases.

Ensuring case presentation to the tumour board requires the coordination and early notification system between the pathologist, tumour registrar and the TTB coordinator.

There is a need to improve the data collection tool (lung cancer auditing tool) to be clear for the auditor and to eliminate any chance of bias or personal subjectivity (judgement).

Physicians should be oriented about the new progress note and the need to start using it instead of the generic progress note that is used by all specialties.

Collaboration between the tumour registry and the Pathology Department was established. The Pathology Department will notify the tumour registry staff of each new diagnosis of lung cancer in order to schedule it for the next TTB meeting.

Monitoring our performance for the cases of 2013 is needed to see if there is any improvement after doing the above corrective actions.

Updating the data collection form in compliance with the most recently published NCCN Guidelines.

Start auditing the lung cancer cases which were diagnosed in 2013 to benchmark it with the baseline data.

Disseminate the results to the TTB members and the departmental staff for discussion and to gather recommendations from all stakeholders.

Update the lung cancer auditing tool to have more specific questions that leave no room for auditors’ bias and that represent the most recently published NCCN Guidelines. These changes were approved by the project team of the treating physicians in the TTB. The management course section was totally changed in 2013. Instead of having general terms/questions, such as ‘Was the first line of treatment accomplished according to guidelines?’, we changed it to direct and specific questions, such as What is the type of treatment? What is the stage of the disease? and What is the ECOG performance?

Identify the 2013 cases and start the auditing process.

Physicians needed orientation about approved guidelines and how they can be accessed.

Percentage of cases discussed in the tumour board increased (table 1).

Table 1

Diagnosis, staging and tumour board discussion measures

Measure	Year
	2012% (n=49)	2013% (n=46)	2014% (n=39)	2015% (n=26)
Histological subtype identified	93.9 (46)	95.7 (44)	94.9 (37)	100 (24) *
Stage documentation	95.9 (47)	97.8 (44)	100 (39)	100 (23) †
EGFR testing‡	76.7 (23)	85.2 (23)	90 (18)	100 (13)
Tumour board discussion	35 (17)	56.5 (26)	51.3 (20)	81.8 (22)§

*As an explanation for the deviation, two cases need more tissue, as it was recommended by the pathologist to collect more tissue, but one patient expired, and one discharged against medical advice.

†One patient died before referral to oncology. One patient discharged against medical advice. One patient lost to follow-up.

‡The denominator for the EGFR is 30 patients in 2012; 27 patients in 2013; 20 patients in 2014; and 13 patients in 2015.

§Two patients clinically deteriorated immediately after the diagnosis and died before referral to oncology and completing staging workup. One patient had Middle East respiratory syndrome coronavirus (MERS-CoV) and expired. One patient signs discharge against medical advice (DAMA) and lost follow-up after the biopsy.

EGFR, epidermal growth factor receptor.

Although the stage documentation had improved in this cycle as a result of the new progress note, staging was still missing for a couple of patients.

The compliance decreased for the early activation of the ‘no code’ status measure and early referral to palliative care.

The management course measures cannot be compared with the baseline results since major changes concerning this section had taken place in the auditing tool. Nonetheless, 100% compliance had been reached in two areas concerning the treatment of stage IV patients: receiving doublet chemotherapy for patients with EGFR wild type and receiving TKI therapy for patients with EGFR mutant.

It was reported that the developed physicians’ progress note is too long, has too many details and is time consuming.

The approved guidelines had been posted on the oncology website and physicians were trained about how to access it.

To ensure better documentation, the completion of the progress note had been included as part of the physician’s annual performance evaluation. Physicians had been informed about the new evaluation process.

Physicians were instructed about guidelines concerning early activation of ‘no code’ status and early referral to palliative care.

The 2013 management course data will serve as a baseline for the next cycle to measure the improvement.

The importance of using the new progress note template was explained to the team. Emphasis had been placed on the completion of the form for better documentation.

Start auditing the lung cancer cases which were diagnosed in 2014 to benchmark our data with 2013.

Feedback should be provided to the multidisciplinary team for better compliance.

Update the auditing tool to include time constraints such as adding the 30-day range for having CT scan of the chest done and the 2-week range for stopping the chemotherapy before death. Such time constraints had been drawn from the American Society for Clinical Oncology, Quality Oncology Practice Initiative standards (ASCO QOPI standards). Additionally, the ALK test had been added to the auditing tool to ensure proper molecular testing to guide patient selection for TKI therapy.

Identify the 2014 cases and start the auditing process.

The compliance in the measure of identification of the histological subtype decreased.

Presentation of cases to the tumour board had decreased to 51%. Reviewing the cases that were not discussed revealed that all of them were stage IV with poor ECOG performance and not fit for treatment.

The majority of patients audited were receiving chemotherapy during the last 2 weeks of their life. Those cases were further reviewed. We found that a high percentage of them were receiving oral therapy such as erlotinib.

Results revealed that the compliance in the management decreased, such as administrating doublet chemotherapy for stage IV patients with EGFR wild type as the first line from 100% to 75%.

Deviations were explored and the results were related to patient preference or poor performance status and one incident of delay in scheduling MRI.

The Pathology Department had been informed about the results for identifying the histological subtype measure for appropriate pathological evaluation.

Physicians had been encouraged to present all cases, regardless of their stage or ECOG performance, to the tumour board just to obtain consensus from all multidisciplinary teams.

The audit tool was revised one more time to incorporate physicians’ professional judgement and auditors’ explanation of the observed deviations from guidelines. The aim for better calculation of the compliance was to document whether the deviation was justified or not.

An arrangement was made with the Medical Imaging Department for having a certain number of slots devoted to patients with cancer to ensure prioritisation and timely treatment.

Start auditing the lung cancer cases which were diagnosed in 2015 to benchmark our data with 2014.

Validate the data collected by auditors.

Identify lung cancer cases diagnosed in 2015 and start the auditing process.

Assign a senior consultant to review the audit case by case and ensure the data had been correctly abstracted from the patient chart.

Histological subtype identification measure reached 100% compliance.

Cases discussed in tumour board meetings increased to 82%.

Stage documentation measure reached 100% compliance.

Validation of the data by a senior consultant was highly significant for engaging consultants in the audit process and hence improves their own practices.

After sharing the results with the multidisciplinary team, they were highly motivated to sustain the high compliance.

Compliance with the referral to palliative care measures continued to decrease to a point that required intervention.

A system had been created where palliative care should be offered alongside the standard care for better assessment of patient condition. The palliative care team is now assessing the patients with lung cancer beginning with the first office visit to the Oncology Department, as the palliative care team is now part of the multidisciplinary team of the thoracic combined clinic. We believe that this arrangement will resolve all issues pertaining to the management of end-of-life measures.

In order to maintain adherence to guidelines the project will continue but in a monitoring phase where only a random sample of lung cancer cases diagnosed each year will be audited.

Results

Baseline audits were performed for 49 patients diagnosed in 2012. Audits were repeated after feedback of baseline performance and after implementation of various interventions. Throughout the years 2013 (46 patients), 2014 (39 patients) and 2015 (26 patients), significant improvement had been made in histological subtype identification (94%–100%) and presentation of new cases at tumour board meetings (35%–82%). Testing for EGFR mutation for non-squamous cell lung cancer also increased from 77% to 100% and staging was documented in 100% of the cases (table 1).

Regarding the management course of patients with lung cancer, 100% adherence to approved guidelines was achieved in 2015 for patients with an early stage of the disease. Also, 100% of patients with stage IV and EGFR wild type were treated according to guidelines (table 2).

Table 2

Management course and end-of-life care measures

Measure	Year % (n)
	2012*	2013	2014	2015
Surgery for stage I, II		75(3 patients out of 4)	NA†	100(3 patients out of 3)
Chemotherapy/radiotherapy for non-resectable stage III		67(4 patients out of 6)	60(3 patients out of 5)	NA
Doublet chemotherapy as first line for stage IV, EGFR/ALK wild type		100(11 patients out of 11)	75(6 patients out of 8)	100(2 patients out of 2)
TKI therapy for stage IV, EGFR/ALK mutant		100(8 patients out of 8)	86(6 patients out of 7)	100‡(1 patient out of 2)
Chemotherapy stopped 2 weeks prior to death	51.35(19 patients out of 37)	10.81(4 patients out of 37)	44.4(4 patients out of 9)	55.6(5 patients out of 9)
‘No code’ activation	83.33(20 patients out of 24)	85.71(12 patients out of 14)	58.8(10 patients out of 17)	22.2(2 patients out of 9)
Transfer to palliative care 2 weeks prior to death§	46(11 patients out of 24)	35(5 patients out of 14)	29.4(5 patients out of 17)	11.1(1 patient out of 9)

*Management course measures had been changed in the auditing tool in 2013.

†No stage I and II patients in this year.

‡The number of eligible patients was 2 only, and one of them was admitted to the intensive care unit (ICU).

§In 2012–2014 we were using old data collection form and the questions were more generic, for example, the first question was ‘Was chemotherapy stopped prior to death?’ The second one was ‘Was patient transfer to palliative prior to death?’ So the specification to measure 2 weeks prior to death was not obtained for this year and it was calculated later to standardise the measure with 2015 data.

ALK, anaplastic lymphoma kinase; EGFR, epidermal growth factor receptor; NA, not applicable; TKI, tyrosine kinase inhibitor.

In terms of management of end-of-life measures, compliance with early activation of the ‘no code’ status decreased from 84% in 2012 to 22% in 2015 with a strong need for improvement. Stopping cancer-directed therapy 2 weeks prior to death and early referral to palliative care measures worsened, but the introduction of the new system of parallel standard care and palliative care for patients with lung cancer is expected to help (table 2).

Because of the low compliance revealed in the PDSA cycle 1 in terms of documentation, a progress note had been created specifically for patients with lung cancer. We collected some physician experience using the new progress note to measure their satisfaction. There was a clear consensus about the user-friendly aspect of the progress note. Some physicians expressed support for the addition of the quality improvement measures, such as adding the question ‘Is the treatment according to clinical practice guidelines?’ at the end of the ‘Plan of Management’ section. In their opinion, such questions serve as a self-audit checklist to ensure compliance with evidence-based treatment management.

The design of the progress note had been optimised in a way that requires both structured and unstructured data entry. Physicians appreciated such design since it incorporates physicians’ preferences for having text-free spaces for documenting their subjective judgement as well as for having structured fields to ensure the consistency of information documented for all patients with cancer.

Additionally, other specialists, such as research coordinators, preferred the new progress note, since abstracting the data for research studies had been easier. Also, in this regard, having a question asking whether the patient is a candidate for clinical trial enhanced the screening and enrolment process for the research unit.

Lessons and limitations

Limitations that faced the project are:

Lung cancer is a rapidly evolving field in terms of treatment. In 2015 and early 2016, several new immunotherapy drugs were approved by the Food and Drug Administration for the treatment of lung cancer. The process of approving, disseminating and incorporating the updated guidelines in our audit process is not moving at the same pace as the revolutions in this field.

The design of the auditing tool in terms of the specific measures collected was modified between cycles, which made it difficult to compare certain measures collected over the years.

Sample size and follow-up time may not be enough to study the survival of the patients with lung cancer as an outcome measure related to adherence to guidelines.

The intervention by itself was limited, as we only used educational activity and changed the physicians’ progress note, which is not the optimal method to proactively minimise any non-compliance to the guideline in the future.

Conclusion

The main aim of the clinical audit is to monitor the variation in the process, trying to minimise it, without ignoring the physicians’ clinical judgement.

Running audits to monitor adherence to guidelines and discussing the findings with the team had a positive effect on providing consistently high-quality care for patients with lung cancer with regard to diagnosis, staging and management. In spite of these improvements, the end-of-life data showed a fluctuated pattern and it needs further investigation to standardise the process.

All in all, we expect that the results are sustainable in the long term as a system with all processes and forms created throughout the project had been developed in electronic format and incorporated into the new electronic medical record to maintain high compliance in all of the measures.

In 2016 the project had been rolled over to cover the top four diseases in oncology: breast cancer, colorectal cancer, acute myeloid leukaemia and hepatocellular carcinoma, in order to improve adherence to approved guidelines.