PURPOSE: The importance of evaluating and improving quality in clinical practice is now generally acknowledged. In this study we estimated different sources of variation in radiotherapy planning for breast cancer patients after mastectomy and sought to test the applicability of a reproducibility and repeatability (R&R) study in a clinical context. METHODS: Eleven radiation oncologists planned radiotherapy three times for three different kinds of breast cancer patients without knowing they were handling the same patient three times. Variation was divided into different components: physicians as operators, patients as parts, and repeated measurements as trials. Variation due to difference across trials (repeatability), that across the physicians (reproducibility), and that across the patients (variability) were estimated, as well as interactions between physicians and patients. Calculation was based on the sum of squares, and analysis was supported by various graphical presentations such as range charts and box plots. RESULTS: Some parts of the planning process were characterized by higher and different kinds of variation than the others. Interphysician variation (i.e., reproducibility) was not high but there were some clearly outlying physicians. The highest variation was in repeatability (= intraphysician variation). The major part of the variation was, however, that from patient to patient: 33% of the total in Parameter 1 and 85% of the total in Parameter 2. CONCLUSIONS: R&R studies are applicable and are needed to evaluate and improve quality in clinical practice. This kind of analysis provides opportunities to establish which kinds of patients require particularly careful attention, which points in the process are most critical for variation, which are the most difficult aspects for each physician and call for more careful description in documents, and which physicians need further training.
PURPOSE: The importance of evaluating and improving quality in clinical practice is now generally acknowledged. In this study we estimated different sources of variation in radiotherapy planning for breast cancerpatients after mastectomy and sought to test the applicability of a reproducibility and repeatability (R&R) study in a clinical context. METHODS: Eleven radiation oncologists planned radiotherapy three times for three different kinds of breast cancerpatients without knowing they were handling the same patient three times. Variation was divided into different components: physicians as operators, patients as parts, and repeated measurements as trials. Variation due to difference across trials (repeatability), that across the physicians (reproducibility), and that across the patients (variability) were estimated, as well as interactions between physicians and patients. Calculation was based on the sum of squares, and analysis was supported by various graphical presentations such as range charts and box plots. RESULTS: Some parts of the planning process were characterized by higher and different kinds of variation than the others. Interphysician variation (i.e., reproducibility) was not high but there were some clearly outlying physicians. The highest variation was in repeatability (= intraphysician variation). The major part of the variation was, however, that from patient to patient: 33% of the total in Parameter 1 and 85% of the total in Parameter 2. CONCLUSIONS: R&R studies are applicable and are needed to evaluate and improve quality in clinical practice. This kind of analysis provides opportunities to establish which kinds of patients require particularly careful attention, which points in the process are most critical for variation, which are the most difficult aspects for each physician and call for more careful description in documents, and which physicians need further training.