Impact of Peer Review in the Radiation Treatment Planning Process: Experience of a Tertiary Care University Hospital in Pakistan.

PURPOSE: To evaluate and report the frequency of changes in radiation therapy treatment plans after peer review in a simulation review meeting once a week.
MATERIALS AND METHODS: Between July 1 and August 31, 2016, the radiation plans of 116 patients were discussed in departmental simulation review meetings. All plans were finalized by the primary radiation oncologist before presenting them in the meeting. A team of radiation oncologists reviewed each plan, and their suggestions were documented as no change, major change, minor change, or missing contour. Changes were further classified as changes in clinical target volume, treatment field, or dose. All recommendations were stratified on the basis of treatment intent, site, and technique. Data were analyzed by Statistical Package for the Social Sciences and are presented descriptively.
RESULTS: Out of 116 plans, 26 (22.4%) were recommended for changes. Minor changes were suggested in 15 treatment plans (12.9%) and a major change in 10 (8.6%), and only one plan was suggested for missing contour. The frequency of change recommendations was greater in radical radiation plans than in palliative plans (92.3% v 7.7%). The head and neck was the most common treatment site recommended for any changes (42.3%). Most of the changes were recommended in the technique planned with three-dimensional conformal radiation therapy (50%). Clinical target volume (73.1%) was identified as the most frequent parameter suggested for any change, followed by treatment field (19.2%) and dose (0.08%).
CONCLUSION: Peer review is an important tool that can be used to overcome deficiencies in radiation treatment plans, with a goal of improved and individualized patient care. Our study reports changes in up to a quarter of radiotherapy plans.

INTRODUCTION

Ensuring a high level of clinical quality in the practice of radiation oncology has become increasingly important. The advancing knowledge about diseases and the increasing complexity of radiation planning and delivery methods can make radiation therapy vulnerable to errors.[1] A considerable variation in target volume delineation among radiation oncologists may occur, which can influence local tumor control and can result in undesirable complications in patients.[2] These differences exist despite adherence to well-established organ dose criteria and uniform contouring guidelines.[3] Therefore, a process is required to prevent inadequacies in contouring and radiation treatment planning.

Quality assurance (QA) is a systematic way of determining inadequacies in any process by meeting a required set of specifications or expectations. A QA program provides a unified set of quality indicators that focus on machines, personnel, and patients to ensure safe and effective use of radiation therapy.[4] Peer review is an important measure for QA in radiation oncology.[5] It has been defined as “the evaluation of creative work or performance by other people in the same field to enhance the quality of work or the performance of colleagues.”[6] A meta-analysis has shown that an audit and feedback can be effective in improving professional practice.[7] However, there is a dearth of quality data in this regard in the field of radiation oncology, and guidelines regarding peer review with specific recommendations are few.[8]

Recognizing the need for peer review in the radiation therapy treatment planning process, a simulation review meeting (SRM) was held once a week in the radiation oncology department starting from 2006 to December 2016. All patients for whom radiation therapy was being planned were discussed routinely in the SRM after undergoing the initial planning processes of contouring and field arrangements. The SRM included a review of the planned treatment by all the radiation oncologists, and changes were recommended and made as a result of peer review, with consensus. The objective of this study was to evaluate whether peer review in weekly SRMs influences the radiation therapy treatment planning process.

MATERIALS AND METHODS

This study was conducted from July 1 to August 31, 2016, after approval by the institutional ethical review committee. Patients who were booked for external beam radiation therapy and underwent simulation were discussed routinely in the weekly SRM of the radiation oncology section of Aga Khan University (AKU) Hospital. All patients, regardless of intent of treatment or technique of delivering radiotherapy, were included in the study. Before discussion in the SRM, the contouring of target volume was completed and approved by the primary consultant radiation oncologist. Each plan was reviewed by at least two or three radiation oncologists. All modifications were categorized as no change, minor change, major change, or missing contour (Table 1). Each change was further classified as change in clinical target volume (CTV), treatment field, or dose. Modification frequency was analyzed chronologically and by treatment intent, tumor site, and treatment technique. Intent of treatment was recorded as either curative or palliative. Tumor sites included the brain, head and neck, thorax, abdomen, pelvis, extremity, and others. Treatment techniques were two dimensional, three-dimensional conformal radiation therapy (3D-CRT), and intensity modulated radiation therapy (IMRT). Data were analyzed by Statistical Package for the Social Sciences version 23 (SPSS, Chicago, IL) and are presented descriptively.

TABLE 1

Operational Definitions

RESULTS

The study was conducted for a period of 2 months, during which the radiation treatment plans of 116 patients with cancer were reviewed in the weekly SRM. A median of three consultant radiation oncologists attended the SRM, with a range of two to four, together with radiation oncology residents and medical physicists.

Out of 116 patients, 96 (82.8%) were planned to be treated with curative intent, and 20 (17.2%) were planned to be treated with palliative intent. Head and neck (n = 47 [40.5%]) was the most common reviewed site, followed by the thorax (n = 30 [25.9%]) and pelvis (n = 17 [14.7%]). The most frequent technique used in treatment planning was 3D-CRT, which was used in 71 cases (61.2%), followed by IMRT (n = 30 [25.9%]). A total of 26 changes (22.6%) were recommended in radiation treatment plans. Minor changes were recommended in 15 reviewed cases (12.9%), whereas a major change was recommended in 10 cases (8.6%; Table 2). Most changes were suggested in CTV (n = 19 [73.1%]), with the least changes suggested in the prescribed dose (n = 2 [0.08%]). Additional classification of changes with respect to CTV, treatment field, and dose is presented in Fig 1.

TABLE 2

Frequencies of Treatment Intent, Tumor Sites, Treatment Techniques, and Category of Changes

FIG 1

Classification of changes. CTV, clinical target volume.

Radical treatment plans were recommended for the majority of changes as compared with palliative plans, as shown in Fig 2. Among all the treated sites, the most frequent changes were suggested in head and neck tumors (42.3%). The breakdown of change recommendations for all the treated regions is presented in Fig 3. When categorized by radiation planning techniques, 3D-CRT constituted the majority of the changes (50%), followed by IMRT (38.5%). Technique-based recommendations for change are displayed in Fig 4. There were no changes recommended in the contour of organs at risk.

FIG 2

Categorization of changes with respect to treatment intent.

FIG 3

Categorization of changes with respect to treatment site.

FIG 4

Categorization of changes with respect to treatment technique. 2D, two dimensional; 3D-CRT, three-dimensional conformal radiation therapy; IMRT, intensity modulated radiation therapy.

DISCUSSION

Radiation oncology involves complex sequential processes of consultation, simulation, planning, and delivery of treatment. With the involvement of so many personnel and procedures, each stage is prone to mistakes, which may have a negligible to profound effect on the quality of related outcomes and may eventually compromise patient care. Therefore, to manage these discrepancies, there has been a desperate need for the implementation of a tool or intervention to ensure that patients receive treatment with minimum chances of error that is evidence based and appropriate to their medical condition.[9] It is imperative to note that AKU is accredited by the Joint Commission International as an academic medical center. The compliance to rigorous standards of patient safety and quality of health care is binding.[10] Furthermore, accreditation plays an integral role in establishing methods to evaluate compliance with international standards.

QA aimed at improving outcomes and eliminating errors plays a vital role in the field of radiation medicine. Its importance in radiation oncology is even greater because of the unique nature of the multiple phases involved. Quality control indicators are established in an attempt to ensure consistent, safe, and optimal delivery of radiation, meeting the highest standards of care.[5] Examples of such quality indicators include review of treatment plans and port films by physicians and physicists, chart rounds, and timeouts before treatment delivery.[11] The quality management guidelines focusing on measuring the functional performance of the radiotherapy equipment by measurable parameters have been provided by the American Association of Physicists in Medicine.[12-15] The American College of Radiology, the American Society for Radiation Oncology, and the European Society for Radiotherapy and Oncology have also developed practical guidelines to ensure quality control during radiation delivery.[16,17]

Peer review is a valuable tool that is central to QA programs in health care. It is practiced widely among radiation oncologists and has several dimensions, including review of treatment decision making, planning contours, and other parameters including, but not limited to, prescribed dose, target volumes, technique, and patient set-up.[8] Cancer Care Ontario defines peer review as the evaluation of the components of a radiation treatment plan by a second radiation oncologist.[18] Peer review is generally the only opportunity for radiation oncologists to evaluate each other’s cases and radiation plans. The input of physicists and radiation therapy technologists is highly valued,[19] and the Association of Physicists in Medicine has published guidelines for a peer review process between two qualified clinical radiation oncology physicists.[20] Peer review generally leads to small but potentially significant developments in professional practice and health care outcomes. The success of review and feedback seems to depend on baseline performance and on how the feedback is provided.[21] This becomes more important in developing countries, which may lack expertise.[22]

There are a number of different formats for conducting peer review. Most centers use peer review in multidisciplinary groups that may or may not have a site-specific focus. The timing of peer review also varies among different centers, with a majority performing peer review before the initiation of treatment.[23] The concept of site-specific multidisciplinary team meetings, in which physicians from different specialties and other members of the health care team discuss patients with upcoming decision points, is emerging in our country.[24] However, there is a lack of evidence about formal radiation oncology–specific meetings in Pakistan other than at AKU, where all radiation plans are evaluated by a team of qualified and trainee radiation oncologists.

The goal of highly conformal radiation techniques such as 3D-CRT and IMRT is to increase the likelihood of tumor control by precisely shaping the dose distribution to the target volume. The reliable delivery of high-quality therapy poses a unique challenge. It is essential to account for all possible sources of uncertainties in treatment planning if we are to achieve this goal. It is the preliminary responsibility of the radiation oncologist during treatment planning to precisely delineate the target volumes and normal organs at risk. Inaccurate demarcation may lead to a geographic miss of a high-risk region and hence, reduced local control and poor survival.[25] This also exposes the surrounding normal tissues to excess doses of radiation beyond their tolerance limits, resulting in higher toxicities. This severity of toxicity warrants the use of precise techniques of delivering radiation.[26]

It is not uncommon to have a disagreement among radiation oncologists about the delineation of target structures, which may invariably have an impact on disease-related outcomes. Interobserver variability in target volumes for cancer treatment with radiation therapy has been demonstrated in multiple studies.[27-29] Lee et al[30] performed a study in which they investigated interobserver variation among five radiation oncologists. Each radiation oncologist was asked to contour the prostate independently on axial computed tomography images 1 day after implanting interstitial needles. There were significant interobserver differences in the defined prostate volume, which led to significant differences in dosimetry. Similarly, Logue et al[31] reported significant interphysician variability in producing target volumes and radiation plans for conformal radiotherapy. These skills are enhanced by knowledge about regional anatomy and patterns of failure; above all, the physician’s experience plays an important role in drawing a CTV.[32] There is a need to quantify disagreements, and additional effort to increase interobserver agreement is required.

Several techniques and procedures have been developed to overcome this variation. Contrast agents and metallic markers have been used commonly to demonstrate the anatomy during simulation to precisely delineate the target volume and to increase the probability of tumor control.[33] Valicenti et al[34] presented a study in which seven radiation oncologists delineated the prostate in groups of patients with and without contrast medium. The study concluded that the use of urethral and bladder contrast improved the reliability of localizing the prostate. We routinely use contrast media and markers as per international recommendations.

Inconsistency in contouring seems to be greater when planning for adjuvant radiation therapy in postoperative cases. The introduction of standardized contouring protocols has led to uniformity in delineation patterns among physicians. In a study by Mitchell et al,[35] the radiation plans of three patients who had undergone a prostatectomy were contoured by six radiation oncologists before and after, providing a contouring atlas. A significant reduction in variability in target volume outlining was achieved by adhering to an evidence-based contouring protocol. The importance of such protocols has also been highlighted by Goodman et al,[36] who presented stepwise contouring guidelines and an atlas for the delineation of CTV in the postoperative irradiation of pancreatic cancers. These guidelines have helped physicians better determine areas at risk and minimize dose to normal structures. The planning target volume accounts for the patient set-up error and internal target motion but not for the interobserver variation in defining the CTV.[37]

In our study, 26 plans out of 116 (22.6%) were recommended for changes; this frequency of change is much higher than that reported in contemporary literature.[38-40] All the recommendations, major or minor, were implemented by the primary radiation oncologist before the radiation treatment plan was delivered to the treatment unit. In 2015, the American Society for Radiation Oncology conducted a nationwide survey related to peer review. The results showed that 83% of radiation oncologists were involved in the process, and among those, 90% changed their radiation plans because of peer review, with approximately 7% to 10% of plans being changed.[38] Boxer et al[39] did a prospective real-time audit of 208 patients from June 2007 to June 2008. Eight patients out of the 208 (3.8%) had a change in management strategy recommended. Brundage et al[40] documented the pretreatment peer review of 3,052 treatment plans over a period of 8 years. Out of the 3,052 audited radiation plans, 124 (4.1%) were identified as having errors in radiation planning or prescription. Modifications were made in 98 plans (3.2%) before commencing radiation. The remainder went unchanged because there were slight deviations from usual practice because of individual patient characteristics.

Delineation of target volume has been identified as the most common variable that is prone to any alteration. In our study, CTV was changed in a total of 19 cases (16.4%), which is consistent with the literature.[39,41] In the audit performed by Boxer et al,[39] most of the changes were suggested in target volume coverage (2.9%), as compared with prescribed dose and fractionation schedule. With the use of precise radiation delivery techniques, the chances of marginally missing the target volume are enhanced. Therefore, there is a need to delicately contour the volumes when using highly conformal radiation techniques. A multi-institutional study by Lo et al[41] reported changes in 66.7% of stereotactic body radiation therapy plans for lung cancer after peer review. The authors of this study concluded that inadequate target volume coverage was the main compulsion behind any change recommendation. Our nascent experience with IMRT has forced us to be extra cautious during contouring and also to seek help from colleagues. Although we most frequently use 3D-CRT in our center, data show that IMRT plans resulted in the most changes in CTV (ie, 10 out of 19).

Our study emphasizes the need for peer review and its impact on the planning process, but several shortcomings require additional attention. First, the sample size was too small to draw any meaningful inferences. Second, we did not compare the pre- and postreview changes in the target volumes. It is the usual practice of our radiation oncologists to make changes in the existing contours. This might be helpful in preventing planning on the wrong target volume, but it does not allow us to quantify the difference in the sizes of altered volumes. Cardenas et al[42] found that postreview volumes were larger than the prereview ones. Third, in many instances, after a thorough and detailed discussion in SRM, the treatment strategy was completely changed and the patient was referred to a service other than radiation oncology. Therefore, we have routine discussions about the need to give radiation and at which doses, and we make required changes after a consensus is achieved; however, documentation is lacking in this regard. Now, we have started daily peer review meetings, and we plan to publish results of daily peer meetings with an increased sample size and the addition of new variables.

Peer review is one of the most effective ways of dealing with routine but controversial patient-specific decisions in radiation oncology. It provides an opportunity to standardize clinical practice patterns and to develop uniform treatment planning guidelines, with a goal of improved and individualized patient care. Our study has shown that up to a quarter of contours may change with peer review. More work is required in this area for better treatment plans, especially in developing countries.