First Report of the Clinical Use of a Commercial Automated System for Daily Patient QA Using EPID Exit Images.

PURPOSE: To characterize the clinical utility of a new commercially available system for daily patient treatment quality assurance using electronic portal imaging detector (EPID) exit dose images. METHODS AND MATERIALS: The PerFRACTION automated quality assurance system was used to acquire integrated EPID images for every field every day for 60 treatment courses for 57 patients. Four thousand seventy-nine field values from 855 fractions were analyzed. Gamma passing rates were computed by the system for each field daily. Passing rates and pass-fail status were recorded by treatment modality (intensity modulated radiation therapy or 3-dimensional conformal radiotherapy) and location. When failures occurred, an attempt was made to determine the reason.
RESULTS: Overall, 23% and 8% of fields failed at 2%/2 mm and 3%/3 mm, respectively. Forty-eight percent and 24% of fields failed at least once during the course of therapy for the 2 tolerance settings. Eighteen percent and 8% of all fractions failed and 60% and 28% of courses failed for the 2 tolerance settings, respectively. Eighteen percent of daily field passing rates were below 75% for 3%/3 mm tolerances. Intensity modulated radiation therapy had higher passing rates than 3-dimensional conformal radiation therapy. For 3%/3 mm tolerances, the fraction fail rate for the brain, extremity, and spine treatment sites failed the least, whereas the abdomen, chest, and head and neck failed more often. The most commonly identified reason for failure was body position change, but the reason for about half the daily field value failures could not be identified.
CONCLUSIONS: This is the first report of the clinical utility of a commercial daily patient treatment quality assurance system using EPID exit images. Variations were found in a clinically relevant percentage of images, and these potentially indicate important treatment variations. Reasons for failures are not always discernable. The system was practical to use because of automation and continues to be used for monitoring of nearly every patient in every field every day.

Introduction

Although radiation therapy departments generally apply extensive quality assurance (QA) checks to most aspects of the planning and delivery process, virtually no dosimetric QA is performed during patient treatments. Daily pretreatment cone beam computed tomography (CT) or kV imaging can help ensure accurate patient body position and pose but does not provide dosimetric data for each treatment. In general, therapists are more focused on the alignment process rather than measuring anatomic changes during image guided radiation therapy (IGRT) from which one could infer dosimetric deviations.

This lack of daily treatment dosimetric QA has long been realized, but until recently there was no practical solution for most radiation therapy departments. At most, in vivo dosimeters are placed on the patient's skin surface for one or more treatments on selected patients to detect any large setup errors. Manual 2-dimensional exit dose measurements with an electronic portal imaging detector (EPID) can be performed, but data processing is too laborious to be practical on a large scale. In 2008, van Elmpt et al reviewed the literature on EPID-based dosimetry, but since then the field has further developed. One large center in the Netherlands has developed an in-house system, which has been in use for several years. With the ubiquitous EPID and the sophistication and computing power of new commercial systems, fully automated EPID-based systems are available that make it practical to perform daily dosimetric QA on every field for every patient every day. This report provides the first description of the daily clinical use of such a system and the findings from more than 4000 daily field measurements. A previous publication described the sensitivity of the system to detect a variety of errors, including dose, multileaf collimator position, intensity modulation, and position of movable couch rails.

Methods and Materials

A TrueBeam linear accelerator (Varian Medical Systems, Palo Alto, CA) equipped with an EPID (Varian AS 1000 flat panel detector) consisting of a detector area of 40 × 30 cm2 with a matrix of 1024 × 768 pixels provided images with a 0.39 mm × 0.39 mm spatial resolution. A high degree of constancy of this imaging system has been verified through institutional monthly QA. The couch top used during treatment was the kVue DoseMax (Qfix, Avondale, PA), which included movable rails. The patient immobilization devices used in therapy were BodyFix and HeadFix (Elekta, Stockholm, Sweden); these devices were always indexed to the couch top. The EPID collected megavoltage x-ray (6 MV) integrated portal images for each field on each day of treatment, which were automatically saved in the Aria database (Varian Medical Systems). These images were automatically retrieved by the PerFRACTION system (Sun Nuclear Corporation, Melbourne, FL, version 1) using an automated query retrieval process. The PerFRACTION system used in this study ran on a dedicated server running embedded Microsoft Windows (Microsoft Corporation, Redmond, WA) with database software and a web interface for configuration and data analysis.

On retrieval of the exit dose images, PerFRACTION automatically calculated the degree to which each field agreed with the baseline image for that field based on user-defined parameters; in this way, the consistency of daily treatment was measured. The baseline images were defined as those from the first fraction where each field was imaged, generally the first fraction treated. Commonly, the first day's treatment undergoes more scrutiny because any setup or other undetected error could propagate a systematic error throughout treatment. In some cases, not all fields could be imaged because of the potential collision of the MV source or imager panel with the patient or couch.

The database of exit dose images used for the purposes of this study was collected retrospectively at Children's Hospital Los Angeles from November 2014 to April 2016. Overall, it contains data for 60 treatment courses from 57 patients. Treatment techniques included 44 courses of intensity modulated radiation therapy (IMRT) and 16 of 3-dimensional conformal radiation therapy (3DCRT). The analysis contained 4079 values from 855 fractions (Table 1). The anatomic treatment location was classified as brain, abdomen, chest, extremity, head and neck, pelvis, or spine (Table 2). For any treatment that had overlap between the classifications, the selection was based on which region best described it.

Table 1

Overview of the analysis of all DFVs, fields, fractions, and courses at both the 2%/2 mm and the 3%/3 mm gamma tolerances

	2%, 2 mm	3%, 3 mm
Total DFV	4079	4079
Total DFV failed	834	340
% DFV failed (M1)	20.5	8.3
Total fields	260	260
Total fields failed	61	21
Field failed (M2), %	23.5	8.1
No. of fields with ≥1 DFV failure	125	63
Fields with ≥1 DFV failure (M3), %	48.1	24.2
Total fractions	855	855
Total fractions failed	152	65
Fraction failed (M4), %	17.8	7.6
Total courses	60	60
Total courses failed	36	17
Courses failed (M5), %	60.0	28.3

Abbreviations: DFV = daily field value; M1-5 = the 5 metrics described in the Methods and Materials.

Table 2

Data acquisition by location and modality

Treatment location	No. of courses	No. of daily field values	No. of fields	No. of fractions
Brain	33	3036	189	496
Abdomen	7	95	13	51
Chest	6	71	10	42
Extremity	2	51	5	25
H&N	7	519	30	117
Pelvis	2	91	5	40
Spine	3	216	8	84
Treatment modality
3DCRT	16	207	29	114
IMRT	44	3872	231	741
Total	60	4079	260	855

Abbreviations: 3DCRT = 3-dimensional conformal radiotherapy; H&N = head and neck; IMRT = intensity modulated radiation therapy.

Analysis of gamma failure rate

The measurement of agreement between the baseline field image and any subsequent field image was performed using the gamma analysis. A pixel within the image was considered to pass if gamma was ≤1. Two different gamma analyses were performed, one with a 2% difference and 2 mm distance to agreement tolerance, and the second with 3% and 3 mm. A dose threshold of 10% was used in all cases.

Five metrics were used to characterize each patient's consistency of daily treatment. The first was the daily field value (DFV), defined as the percent of pixels from each field exit dose image that passed the gamma analysis on a single day of treatment. A DFV was defined as passing if ≥93% of the pixels passed the gamma analysis. Second, a field was defined as passing if the average of its DFVs during the entire course of treatment was ≥93%. Third, a variation on field passing rate, is the percentage of fields that had at least 1 failing DFV during the course of treatment. Fourth, the entire fraction was assessed, defined as all of the DFVs for a single day of treatment, and the 93% threshold was once again used to define pass and fail for the average over all the fields. Fifth, a treatment course was considered to pass if the average of all DFVs from each field over the entire duration of therapy (ie, all fields and all fractions) was ≥93%.

Analysis of reasons for field failures

The PerFRACTION web interface shows both the delivered and expected EPID image for each field for each fraction. In addition, the gamma analysis results image is shown, where green and yellow pixels represent those that passed the gamma analysis and orange represents pixels that failed.

For any field from any treatment course that had at least 1 DFV failure, image analysis was performed to determine the reason. Cone-beam computed tomography, portal images, or kV images were routinely taken for image guidance before each treatment. For static fields, a beam's eye view (or digitally reconstructed radiograph) was viewed in Eclipse (Varian Medical Systems,) and compared to the gamma analysis image in PerFRACTION to determine the location within the field that failed gamma analysis and therefore elucidate potential reasons for failure. For VMAT plans, the field was viewed in beams-eye-view movie mode and compared to the integrated image of the field.

Reasons for failure were classified as body position change (BPC), external device position (EDP), internal anatomic change (IAC), or unknown (U). For fields in which failure was due to more than 1 of these classifications, the reason that was most contributory was assigned. BPC was defined as due primarily to variations in the position of the patient's body from treatment to treatment, and an example is shown in Figure 1. EDP was used to classify failures due to relative shifts in the position of patient immobilization devices from their location in the baseline image. IAC classified failures that were a result of changes in the internal anatomic features of the patient (eg, changes in lung volume or bowel gas, weight gain or loss, tumor shrinkage, or growth). Any of these could affect the attenuation of the beam through the patient and subsequent detection by the EPID (Fig 2). Any DFV failure that could not be ascribed to 1 of the 3 specific classifications was labeled as U. The percent of field failures and the percent of fields containing DFV failure due to each category was calculated, both globally and by location and modality. The P value for the analysis of variance was calculated for failures by location and modality to determine if a significant relationship existed.

Figure 1

Field failure due to body position change. (a) Although there is an external device located within this field, the pattern of pixel failure best matches the shape and location of the shoulder. (b) The failing pixels in the region in which the patient's shoulder is located; slight adjustments in the positioning of the shoulder would alter the amount of radiation detected by the electronic portal imaging detector.

Figure 2

Field failure due to internal anatomy change. The failing (orange) pixels (b) correspond with regions in the lungs in image. (a) The best explanation for gamma failures in this case would be internal changes, such as the density of the lungs or position of the diaphragm.

Results

Gamma failure rates were calculated for both the 2%/2 mm and 3%/3 mm tolerance values by DFVs, total fields, total fields with at least 1 failure, total fractions, and treatment courses (Table 2). There were 834 DFV failures; about 20% and 8% of the DFVs failed at 2%/2 mm and 3%/3 mm, respectively, roughly the same for total field failures. There were 125 (48%) and 63 (24%) fields with at least one DFV failure at 2%/2 mm and 3%/3 mm, respectively. About 18% and 8% of fractions and 60% and 28% of courses failed at 2%/2 mm and 3%/3 mm, respectively.

The gamma failure rates for DFVs, fields, fractions and courses were then analyzed by treatment location and modality (Table 3). By location, chest treatments had the highest failure rates for each of the metrics analyzed (other than the course failure rate), at 2%/2 mm; however, these rates were over a factor of 2 lower at 3%/3 mm and often less than rates for other locations at these larger tolerances. The P value by location was significant at .037. Extremity, pelvis, and spine treatments demonstrated the lowest failure rates across each of the metrics. By modality, IMRT had a lower failure rate than 3DCRT for all metrics (up to 6-fold lower) at both 2%/2 mm and 3%/3 mm. The P value by modality was significant at <.0001.

Table 3

Treatment location and modality failure rate (%) at 2%/2 mm and 3%/3 mm by course, DFV, fraction, and field

Treatment location	Courses failed, %		Total DFV fail rate		Fraction fail rate		Field fail rate
	2,2	3,3	2,2	3,3	2,2	3,3	2,2	3,3
Brain	57.6	21.2	18.1	5.8	13.2	4.0	18.0	5.1
Abdomen	42.9	42.9	44.2	32.1	50.0	42.9	50.0	25.0
Chest	83.3	66.7	53.9	18.5	53.4	18.8	75.0	16.7
Extremity	50.0	0.0	8.3	1.4	11.8	2.9	0.0	0.0
H&N	71.4	42.9	38.7	20.1	35.9	21.4	47.6	27.6
Pelvis	50.0	0.0	7.7	0.0	7.7	0.0	0.0	0.0
Spine	66.7	0.0	11.9	1.7	12.3	1.4	0.0	0.0
Treatment modality
3DCRT	81.3	56.3	56.7	24.7	55.2	29.2	65.6	25.0
IMRT	52.8	18.2	15.0	5.7	11.6	4.4	14.3	4.8
Overall failure rates	60.0	28.3	20.5	8.3	17.8	7.6	23.5	8.1

Abbreviations: 3DCRT = 3-dimensional conformal radiotherapy; DFV = daily field value; H&N = head and neck; IMRT = intensity modulated radiation therapy.

P values for 2%, 2 mm, 0.037 by location, <.0001 by modality.

Analysis of reasons for DFV failures

The analysis of reason for failures was conducted by modality and location using the 2%/2 mm tolerance level. The greatest number of both field and DFV failures, nearly 50%, were classified as U. BPC was the leading cause of failure that could be identified, accounting for 28% of fields having at least 1 DFV failure and 29% of all DFV failures. In contrast, EDP changes were responsible for only 22% of fields having a DFV failure and 16% of all DFV failures. IAC was responsible for the fewest; it accounted for only 10% of the fields with at least 1 DFV failure and 5% of the failed DFVs.

Reasons for DFV failures by modality

The reasons for fields with at least 1 DFV failure were broken down by treatment modality (Table 4). For this analysis, the percentage of the total number of fields within each modality with a DFV failure and the percentage of the total number of DFVs were calculated for each modality across all patients who had a DFV failure for a particular reason.

Table 4

Reason for DFV failures by treatment modality at 2%/2 mm tolerances

Modality	Reason	No. of fields with ≥1 DFV failure	Fields with ≥1 DFV failure, %	No. of DFV failed	Total DFVs failed, %
3DCRT
	BPC	9	31.0	65	31.4
	EDP	0	0.0	0	0.0
	U	3	10.3	9	4.4
	IAC	10	34.5	38	18.4
IMRT
	BPC	26	11.3	179	4.6
	EDP	28	12.2	134	3.5
	U	46	19.9	402	10.4
	IAC	3	1.3	7	0.2

Abbreviations: 3DCRT = 3-dimensional conformal radiation therapy; BPC = body position change; DFV = daily field value; EDP = external device position; IAC = internal anatomic change; IMRT = intensity modulated radiation therapy; U = unknown.

There were marked differences in the distribution of reasons for DFV failure by modality (Table EA; available online at https://doi.org/10.1016/j.adro.2019.04.001.). The most striking contrast is seen in failures due to IAC; more than one-third of 3DCRT fields had at least 1 DFV failure because of IAC, but this reason applied to few fields or DFVs for IMRT. BPC was also an important cause of DFV failure in 3DCRT treatments, responsible for nearly one-third of the fields with a DFV failure. More than 65% of 3DCRT fields had at least 1 DFV failure because of BPC or IAC. Notably, EDP was responsible for no DFV failures in 3DCRT treatments. Ten percent of the total DFVs failed for unknown reasons, whereas BPC and IAC caused failures at rates of roughly 31% and 34%, respectively.

IMRT had a lower overall failure rate than 3DCRT (Table 3). For IMRT, the leading cause was unknown, responsible for roughly 20% of IMRT fields with at least 1 DFV failure, whereas BPC, EDP, and IAC were responsible for approximately 11%, 12%, and 1%, respectively (Table 4).

Reasons for DFV failures by location

Analysis identical to that performed for treatment modality was done for treatment location (Table EA; available online at https://doi.org/10.1016/j.adro.2019.04.001.). For brain treatments, the most frequent cause of DFV failure was unknown, accounting for 11% of total DFV failures and 19% of fields having at least 1 DFV failure. BPC and EDP were responsible for 11% and 14% of brain fields with at least 1 DFV but only about 4% of all brain DFV failures. As might be expected, IAC was not assigned to any failures for brain treatments. Treatments of the abdomen were most likely to fail as a result of IAC by a wide margin, accounting for 46% of fields with at least 1 DFV failure and nearly one-quarter of the total abdomen DFV failures. BPC and EDP were not found to have caused any failures. Chest fields were most likely to have a DFV failure because of IAC (50% of total fields), whereas BPC and U contributed a lesser amount at 20% and 10%, respectively. IAC was also the reason for the greatest percentage of total chest DFV failures (23%). No failures were found as a result of EDP. Twelve percent of extremity site fields failed; all were due to U. DFV failure in treatments of the head and neck were most often due to BPC and U, explaining 13% to 15% of the total number of field failures for this site. Pelvis treatments had few failures, and they were all associated with IAC. In treatments of the spine, approximately 50% of fields contained a DFV failure because of BPC and resulted in 9% of total DFV failures. The only other cause of DFV failure was U, which was responsible for 25% of the spine fields having a DFV failure and the failure of 4% of the total DFVs. No DFV failures for this site were due to IAC or EDP.

Discussion

This study presents the first reported clinical results of Sun Nuclear Corporation's PerFRACTION automated daily patient QA system. The gamma passing rates of more than 4000 fields across 60 treatment courses delivering 855 fractions were computed using the field images from the first treatment fraction of each course as the baseline. As such, these results represent the degree of dosimetric consistency of daily treatment, a metric not previously widely available to clinics. Zhuang and Olch have previously shown the system to be a very sensitive detector of changes in the EPID exit image.

Our findings suggest that even at a relatively generous tolerance of 3% dose and 3 mm distance to agreement, almost one-quarter of the treated fields failed at least once during the course of radiation therapy; fractions failed more than 10% of the time for 12 of 60 courses (20%), and 11 of 60 courses (18%) had an average DFV passing rate of <75%. These failure rates are substantially higher at the tighter tolerance setting. We also found treatment deviations by treatment site and modality.

Perhaps surprisingly, relatively simpler 3DCRT treatments were actually less consistent than IMRT treatments. This may be because of the greater susceptibility to changes in body position or internal anatomy changes (eg, for anterior-posterior treatments of the whole lung where the state of inhalation may change or for opposed lateral whole brain treatments where the degree of flash around the head can vary with head position). These situations may be worsened by having few fields. Thus, although one might be concerned that modulated treatments are riskier than 3DCRT for consistent dose delivery, our results show that the opposite is true. This suggests that patient-related factors are more important than delivery system performance factors.

We carefully examined daily imaging to attempt to discover the reason for failing fields. We categorized reasons for failure by commonly found problems such as body position, external device, and internal anatomy changes, as well as unknown when we could not reasonably assign those categories. In about half of all failures, we were not able to find a reason. This is both troubling and illuminating. Where the reason was not clear even after investigation, one could conclude that PerFRACTION may be uncovering potential errors that otherwise were unlikely to have been noticed but perhaps are not resolvable without deeper investigation. In those cases where one could not discern a reason for the error, one at least could begin to study the magnitude and frequency of these events, which might lead to an understanding of the cause. Although the percentage of fraction failures that actually lead to mitigation of a real error may be modest, most QA procedures routinely practiced are not expected to find many errors, yet they are still recommended. Because this study was performed retrospectively, an investigation of gamma failures during the actual daily treatments was limited to the imaging done at the time. For those using the system prospectively, the likelihood of finding a cause may increase over what we report. For example, if the gamma failures indicate a uniformly higher or lower dose in all or part of the images, weight change or other anatomic changes that would increase or decrease patient thickness might be suspected and could be investigated. Cone beam CT might be added to the IGRT process for such a patient as part of that investigation, and confirmation might suggest the need for replanning. If a mispositioned external device was suspected, one could attempt to confirm that (eg, by careful observation of patient setup) and correct the error. Indeed, we have been using the system clinically since the end of the study and have found gamma failures flagged by the system, investigated them, and found some to be real and with dosimetric impact to the patient. In many cases, mitigation of or at least understanding of the magnitude of the error was possible. In addition, many of the errors, such as anatomy changes or incorrect placement of external devices, would not be addressed by larger planning target volume margins.

One of the reasons for failure that we sought was changes in the position of external devices, generally immobilization devices but also movable couch rails. In most cases, the failure was due to couch shifts derived from daily IGRT, which caused the device to change its position relative to the field center. Typically, the device was in the exit of the field so there was no dosimetric impact. However, if the device is in the entrance, a significant dosimetric impact could occur. This failure mode is more insidious than the others because it is less likely to be noticed on daily IGRT images.

Although the component technology used by PerFRACTION is not novel, nor is the desire to have the information it provides, the near total automation of the system provides a practical means to potentially acquire daily dosimetric QA information for every field every day for every patient. This information fills an unmet QA need, making dosimetric QA an integral part of daily delivery of therapy. The impact in our clinic has been to provide reassurance of error-free treatment courses in those cases where fractions consistently pass and where a clinically relevant percentage of fractions failed, a notification with enough specificity in error magnitude and location to be actionable by the physics staff. The examples shown in Figures 1 and 2 are fairly typical, where some DFV deviations are real but were found to present no patient dose impact, whereas other deviations had important clinical ramifications and needed to be addressed. Although the dosimetric impact to the patient is not described by this system, the medical physicist may be able use the information provided per beam for the direction, location, and magnitude of the change of dose incident on the EPID to approximate the change in the patient dose. Each clinic using such a system will have to determine what error tolerance is appropriate, and this would likely be specific to treatment intent. The added physics workload (investigation of failing fractions) resulting from this system can be adjusted based on the tolerance levels chosen. We estimate this workload to be about 1 hour per week per machine treating 30 patients per day for 3%/3 mm tolerances.

The method of analysis in this study compared the first fraction field images to subsequent images of the same field and thus was a consistency test based on relative differences. New versions of the system calculate the absolute dose in the EPID image and compare that to the absolute dose predicted by an algorithm based on the planning system data for the particular plan. There are reports in the literature regarding noncommercial (non- or semiautomated) systems that perform this same type of analysis.5, 6, 7, 8 The analysis of the gamma passing rates for each field in this study was performed after treatment delivery, but there have been reports of noncommercial systems that perform real-time analysis during field delivery to be able to warn of an error before the full treatment dose is delivered.9, 10, 11 Some systems are capable of calculating the full 3-dimensional dose distribution in the patient based on back-projection of the EPID dose through the planning CT dataset and can then perform a 3-dimensional gamma analysis. Other systems, including newer versions of PerFRACTION, calculate the full 3-dimensional dose from treatment delivery log files or from combinations of log files and cine EPID images.

Conclusions

An automated system for daily patient treatment QA using the EPID to capture an exit dose image has been used clinically for several thousand fields. Gamma analysis demonstrates that it has the capability to detect important changes in patient setup, anatomy, and external device position not readily noticeable otherwise. About half the time the reason for the gamma failure could be found in this retrospective study, but a larger percentage may be explained with a more contemporaneous investigation of each failure. Although the magnitude of dosimetric error in the patient cannot be readily determined by this system, having an automated EPID-based system that can flag potential daily treatment deviations should be considered an important new addition to the QA regimen.