Inter-departmental dosimetry audits - development of methods and lessons learned.

External dosimetry audits give confidence in the safe and accurate delivery of radiotherapy. In the United Kingdom, such audits have been performed for almost 30 years. From the start, they included clinically relevant conditions, as well as reference machine output. Recently, national audits have tested new or complex techniques, but these methods are then used in regional audits by a peer-to-peer approach. This local approach builds up the radiotherapy community, facilitates communication, and brings synergy to medical physics.

Introduction

External dosimetry audits give confidence in the safe and accurate delivery of radiotherapy. For almost 50 years, the International Atomic Energy Agency (IAEA) has provided this service to hospitals around the world, mainly in low and middle income countries.[1] Another example is for clinical trials, where the Radiological Physics Centre (RPC) in Houston, Texas (now part of the Imaging and Radiation Oncology Core group) has provided remote audits and credentialing services for over 40 years.[2] Both of these services are offered as a central review of an institution's dosimetry [Figure 1a] giving confidence in the delivery of standard radiotherapy around the world. In recent years, technology in radiotherapy has advanced greatly, and some audits have expanded to test these advanced technologies. This ensures consistency and safety of these treatments, regardless of previous experience or geographical location. Audit, therefore, forms an invaluable part of “reaching the unreachable with advanced technologies.”

Figure 1

Two models of dosimetry audit: (a) Central review (left) or (b) peer-to-peer (right)

In the United Kingdom (UK), inter-departmental dosimetry audits began almost 30 years ago. From the start, they included clinically relevant conditions, as well as reference machine output. Some national audits have been performed by a single group or small collaborations. However, regional audits are also performed regularly using a voluntary peer-to-peer approach with links to the national standards laboratory [Figure 1b]. This local reciprocal approach builds up the radiotherapy community, facilitates communication, and brings synergy to medical physics.

Dosimetry audit is a core aspect of UK safety culture, along with legislation, regulation, education, and multi-disciplinary collaboration. Recently, a national system was established to report radiotherapy errors including near misses and those without clinical impact, as well as the rarer but more significant incidents.[3] Fostering a culture of openness and sharing allows everyone to learn from mistakes, and to hopefully avoid repeating them in the future.

Audits of Basic Dosimetry

The first national audit tested megavoltage photon beams in all clinical centers between 1987 and 1991, covering both cobalt-60 (Co-60) units and linear accelerators. The audit used an ionization chamber in a standard solid water phantom with trapezoid shape and additional low density insert to simulate lung tissue.[4] A co-ordinator was appointed for each region of the country to manage the distribution of the phantom and performance of the audits. In each center reference output was measured and compared to locally measured values. About 97% of beams were within 3%, but 2 centers showed difference >5%. One of these centers involved a major calibration error, discovered by the audit, which is described in Figure 2. Point doses for a typical three field plan to the phantom were also measured and compared to local predicted values. The mean ratio of the auditor to the local center is given in Table 1. Nine out of 62 centers had differences >5% for one or more points. Reasons for these differences included:

Figure 2

Reconstruction of calculation for cobalt-60 calibration error (event number 10)[5]

Table 1

Details of selected UK national dosimetry audits

Lack of correction in the treatment planning system (TPS) for the lung material, or used a value of approximately 0.5 g/cm3 rather than close to 0.3 g/cm3;

Small errors in the TPS source data, or bugs in the software itself, which were discovered and corrected;

Unidentified or uncorrected drifts in the machine output, for example, with a wedge applied, which led to additional routine quality assurance (QA) tests.

One of the outcomes of errors discovered by this first national audit was the establishment of an inter-departmental audit network, co-ordinated by the Institute of Physics and Engineering in Medicine (IPEM, the UK professional body for medical physics). Eight regional groups were established in 1993 with the aim of repeating dosimetry audits on a regular basis within the region [Figure 3].[14] A national audit for megavoltage electron beams was performed alongside these between 1994 and 1996, by a graduate student funded as part of their PhD investigation.[6] A total of 156 beams in 52 centers were audited, and all agreed within 5% of local values. During the visits, the auditor also repeated measurements on one photon beam per center. All results were found to agree within 3%, and the standard deviation had reduced from 1.5% in the previous national audit to 1.0% in this one.

Figure 3

Map of the country is showing the division of regional audit groups. Representative radiotherapy centers are shown

The methods used in these national audits were then re-used by the regional groups, run by volunteers without any central funding. Further results have been published for the next decade, including MV photons, MeV electrons, and kV photons, though it is evident that some regions were more active than others.[15161718] The inter-departmental audit network was re-invigorated in 2006 by the establishment of standard protocols and analysis spreadsheets for each audit, and an annual meeting of regional representatives to share their findings and learn about future national initiatives. A further “region” has recently been added to include a network of single-linac private centers that have been starting up around the country over the last 6 years. Other private centers are invited to join their local region.

Audits coordinated nationally but performed through the regions confirm the steady decrease in the spread of results over time [Figure 4], yet these audits have also continued to find some issues. For example, the electron audit performed in 2009–2010 included a clinical situation with a stand-off, bolus, and a cut-out for the field size. Several departments showed differences between auditor and host measurement of >4%, and in three departments were >10%. Reasons for these differences included:

Figure 4

Trends in MV photon audit agreement over time

One center was using historical golden beam data for electron depth doses, but all linacs in the department were on the edge of tolerance for distance-to-agreement at 50%, leading to large percentage errors in dose in the steep portion of the curve;

Errors in the estimated thickness and water-equivalence of different bolus materials.

These errors may also have been detected by remote audits using thermoluminescence dosimeters (TLDs) or other postal dosimeters. However, on-site visits with direct reading dosimeters such as ion chambers have lower uncertainties, allow immediate feedback, discussion, and investigation, and therefore, are more likely to find the root cause of the problem in a timely manner. For this reason, audits in-person by neighboring centers have become greatly valued in the UK, and are often requested ad hoc when a center has commissioned a new machine or technique. These visits are not funded centrally but are usually reciprocated over time, so costs are shared locally. However, this approach relies on the strong link with the central standards laboratory to provide absolute reference dosimetry to each regional group.

In the UK, the National Physical Laboratory (NPL) provides calibration services for secondary standard dosimeters against the national primary standard. From 1994 onwards, this institution also started to perform visits to one center in each region, approximately every 2 years. Because of the rigor of their processes and the direct traceability variation in measured results for these audits is lower. However, it can be seen from Figure 4 that for MV photons at least, the peer-to-peer audits are now achieving a similar level of agreement. This gives confidence in the accuracy of basic MV photon calibration in the UK, which is partly because of the homogeneity of the calibration method. Almost every center in the country uses the NPL to calibrate their secondary standard chambers and follows the same UK code of practice.[19] This code gives direct absorbed dose to water for multiple beam qualities, not just Co-60, and is, therefore, simple to interpolate to the local beam qualities. This should not be taken for granted elsewhere, though. For example, the RPC audit results include 5–6% of megavoltage beams, which exceed their tolerance of 5% or 5 mm every year, and these represent 10–20% of institutions.[2] For IAEA audits between 1969 and 1998, the average agreement ratio worldwide was 1.013 ± 0.088, and even in recent years several results continue to fall outside their tolerance of 5%.[1] Therefore, it is recommended that even audits of advanced techniques should include a measurement of standard output.

Clinical trials provide an excellent opportunity to audit and credential recruiting centers. Audits ensure consistency within the trial itself, and may also support the wider implementation of new technologies.[20] The first trial in the UK to audit participating centers was continuous hyperfractionated accelerated radiotherapy (CHART), for hyperfractionated lung radiotherapy.[21] Visits were made to 15 centers from 1989 to 1990 with anatomical-shaped phantoms. They measured doses to the spinal cord area 10% higher than calculated, but dose to the prescription point was within 4% of the intended value. The next audits were 10 years later, but again helped set the standard for treatment of prostate and breast, respectively.

First, the RT01 prostate trial supported the implementation of conformal radiotherapy across the country. Audits visits were made to 17 centers from 2000 to 2002 using a specially designed pelvis phantom.[2223] The average agreement across all measured points was 0.982 ± 1.040, and prescription points were within 2% of expected values. In addition, a silver prostate insert allowed verification of the geometric localization, with all centers achieving 3 mm accuracy or lower. Ideally, all dosimetry audits should also include end-to-end testing of the treatment technique including verification imaging, which is often overlooked.

Second, the standardisation of breast radiotherapy (START) trials led to a standardization of breast prescription, fractionation, and treatment. Audit visits were performed in 36 centers from 2000 to 2001, again using custom-built phantoms for breast and chest wall verification [Figure 5].[789] Although overall mean agreement was within 2% for all phantoms [Table 1], a number of readings showed differences >4%, for the following reasons:

Figure 5

Evolution of breast phantoms used for the START and IMPORT trials. Two-dimensional phantoms for breast and chest wall (top) were made from a solid water-equivalent material with lung density section in the breast phantom. Three-dimensional phantoms (bottom) were made from water-filled plastic with lung inserts and tubes to allow multiple positions for ion chamber readings

Use of a nonstandard normalization point;

Inadequate correction in TPS for lack of lateral scatter;

Variations in wedge output or beam flatness at nonzero gantry angles not accounted for.

The three-dimensional phantom design was used for the intensity modulated partial organ radiotherapy (IMPORT) trials of breast intensity modulated radiotherapy (IMRT), and used for a further audit series while the original phantom is now being used by centers in India. QA of the START trial also included in vivo measurements with TLDs[10] and audit of match lines between breast fields and nodal fields.[24] It has been shown that the introduction of these trials in the UK has driven widespread adoption of new technology in breast treatment.[25] However, these audits continued to find discrepancies because of the modeling of wedge fields or lung correction in the TPS. Pencil beam and measurement-based planning systems (type A algorithms) gave systematically less accurate readings than collapsed cone or convolution/superposition algorithms (type B algorithms, which include variable lateral scatter).[26]

QA for clinical trials is now co-ordinated by the Radiotherapy Trials QA group (RTTQA, www.rttrials.org.uk), part of the National Cancer Research Institute. Previously, QA was funded for individual trials such as those listed above, but the provision of central grant funding for RTTQA in 2010 allowed credentialing of centers for particular techniques on a national basis. Completion of appropriate QA for one trial can then streamline entry into other similar trials. For example, in the last 5 years, 52 out of 62 current NHS centers have been credentialed for IMRT or Volumetric Modulated Arc Therapy (VMAT) delivery.

Most recently, brachytherapy systems have also been included in national audits. First, between 2012 and 2013, 7 centers using compact kilovoltage X-ray units for intraoperative breast radiotherapy were visited to measure output, isotropy, and depth doses.[27] Second, between 2013 and 2014, high dose-rate (HDR) and pulsed dose-rate units at 45 centers were visited as part of a collaboration between RTTQA and NPL.[28] This audit measured absolute dose from a line source at 2 cm from the source axis, using an ion chamber and alanine in a custom-made solid water phantom. Both these audits found close agreement between the different centers (one standard deviation of 2–3%) and an overall small systematic difference as compared to predicted values (also 2–3%). In both audits, accuracy was dominated by positional uncertainties and there was substantial variation in dose measured around the source.

Audits of Advanced Techniques

As confidence grew in standard dosimetry, the investigation was extended to more complex techniques. These audits were mainly performed on a national level, but methods have been reproduced for subsequent regional use.

The introduction of IMRT into clinical practice in the UK was relatively slow, partly because of resource limitations and caution over potential inaccuracies in treatment planning and delivery. However, the parotid-sparing intensity modulated versus conventional radiotherapy (PARSPORT) trial of head and neck IMRT versus conventional treatment contained a rigorous pretrial QA program, giving confidence to participating centers.[29] Dosimetry audit was performed in 6 centers between 2005 and 2007, testing standard geometrical patterns, as well as a clinically relevant plan. Measurements were performed in a slab phantom with the film for the individual fields at zero gantry angle and the combined plan at planned angles. For the point dose measurements with an ion chamber on the combined plan, the fields were transferred to a commercial head and neck phantom (CIRS 002HN) with the original beam angles maintained. Doses to the primary target were all within 3% of expected, suggesting this value was a reasonable tolerance. Agreement for a representative cord point was slightly poorer (average – 1.3 ± 4.1%), as also found in the CHART trial audits. Gamma analysis of the films showed a mean agreement of 97.7% (3%/3 mm, individual fields) and 97.4% (4%/3 mm, combined plan), with the majority of centers passing at this level.

A national audit of IMRT was performed in 2010 by post, using extended dose range film and alanine, in a local slab phantom with all fields delivered at zero gantry angle.[11] Postal audits are easier to schedule for the host center, but may lack the consistency of procedure, due to the interpretation of instructions. In fact, all the outliers measured during this audit were because of user error, for example in scaling the plan to the 10 Gy level required for the alanine exposure. Issues were also found with reading the dose plane format into the central review software, and some films were damaged in the post or had unclear markings. Following some repeat measurements, all point dose readings were within 5% [Table 1]. For prostate and breast plans, all fields achieved >95% pass at a gamma level of 3%/3 mm (average – 99.2%). For head and neck plans, which are typically more complex, almost all fields (97%) passed >95% at 4%/4 mm (average – 98.2%). Those that failed this level were the most complex or were predominantly in areas of low doses. These results were very encouraging. The postal audit of head and neck IMRT by RPC-Radiation Therapy Oncology Group between 2001 and 2011 found that only 82% of irradiations were able to meet a wider criteria of 7%/4 mm, though the pass rate had increased over time.[2]

IMRT is now used by all UK clinical departments, and for approximately one-third of radical patients; however, the use of VMAT is still increasing. Between 2011 and 2013, a national audit of rotational delivery (VMAT and tomotherapy) was performed by on-site visits to 44 centers[12] using a previously validated Octavius detector array[30] and a virtual TPS test.[31] This audit was in collaboration between IPEM, RTTQA, and NPL, laying the foundations for the UK Dosimetry Audit Network (www.uk-dan.co.uk) formed in 2014 to coordinate the approach of all these auditing bodies. The results of this audit were also encouraging. Point doses showed close agreement with expected [Table 1], and in all but one center, at least one measured plane passed a standard gamma criterion of 3%/3 mm for clinical plans. However, issues continued to be found in basic aspects such as laser and barometer calibration. In addition, the poorer results could be traced to use of too small a minimum leaf gap and lack of couch modeling, along with highly modulated plans. Using a gamma criterion of 3%/2 mm, it was possible to differentiate between different systems, to assess the optimality of TPS modeling. In cases, where both planning software and delivery hardware came from the same vendor (e.g. Varian or Tomotherapy), the median pass rate for all plans at 3%/2 mm was 100% (range: 85–100%). For planning and delivery systems from different vendors, the median pass rate was 96% (range: 67–100%). In both cohorts, some centers achieved very high pass rates, suggesting it is possible but more difficult to optimize mixed vendor solutions, perhaps because of the wider range of parameters that can be tuned.

Finally, two very recent audits in 2013–2014 have investigated end-to-end planning of HDR cervix treatments[13] and delivery of stereotactic ablative body radiotherapy (SABR) lung treatments.[3233] The HDR audit was performed in person at 46 centers alongside the absolute dose audit described in the previous section. A custom-made film stand[34] was placed into a water tank, then scanned and planned using a local ring applicator. Gafchromic EBT3 film was used to measure the delivered dose distribution in two planes. Dose to Manchester system point A agreed closely with expected values [Table 1] with a systematic offset for metal applicators, which are not modeled by the TPS. Average film gamma analysis was 98% at 3%/2 mm and 89–91% at 2%/1.5 mm. No major deviations were found, although in one center the applicator library was misaligned with the imaging of the intra-uterine tip.

For the SABR audit, a commercial thorax phantom was used with alanine and EBT3 film to measure dose at 21 centers, including a range of techniques including conformal fields, VMAT, and Cyberknife. Mean point dose measured with alanine was + 0.2 ± 2.0%, with little difference between algorithms or delivery method except for a few outliers. These were one center using a pencil beam algorithm, and Cyberknife centers using Monte Carlo algorithms, which calculate dose to medium instead of water, so may have had issues with the phantom material. Film results were harder to interpret, with gamma pass rates reduced by the heterogeneity of dose within the high dose planning target volume (PTV) or at lower doses. However, a mean dose within the prescription isodose agreed well with alanine results.

Lessons Learnt for the Future

Advances in technology bring great potential benefits but also added the risk of misuse. In the UK, the next two areas to be considered for dosimetry audit are cranial stereotactic radiosurgery and flattening filter free beams (for which guidance was recently published in this journal).[35] However, we intend to continue checks on basic output and regional audits to maintain high standards in more standard dosimetry. Imaging and clinician outlining also have a great effect on patient treatment accuracy and so programs to verify and standardize these processes are ongoing.

There are various approaches to dosimetry audit: Postal or in person visits, peer-to-peer or central review, and commercial equipment or in-house tools. Regional inter-departmental audits can be built on established methods, and may be a more practical approach to support remote centers. This then benefits the whole radiotherapy community. We can also learn from both the experiences and mistakes of others to provide the best service to our patients.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.