An evaluation of concordance between head and neck advanced practice radiation therapist and radiation oncologists in toxicity assessment for nasopharyngeal carcinoma patients.

BACKGROUND: Weekly toxicity assessments for patients undergoing head and neck (HN) radiotherapy are essential to ensure that acute side effects are appropriately managed in order for patients to complete their treatment in a safe and timely manner. The incorporation of Advanced Practice Radiation Therapist (APRT) led treatment reviews has been reported for various subsites, but there is currently a lack of published literature regarding this role for patients with HN cancer. The purpose of this study is to assess the concordance of toxicity assessments performed during weekly radiotherapy treatment reviews for patients undergoing HN radiotherapy between the HN APRT and Radiation Oncologist (RO).
METHODS: Twenty-three patients with nasopharyngeal cancer (NPC) under the care of 3 ROs were recruited from June to December 2018; weekly assessments were independently performed by HN APRT and ROs. The HN toxicity assessment was graded according to the Common Terminology Criteria for Advanced Events v4.0. Both assessors were blinded to each other's assessments. The percentage agreement of concordance and agreement level were interpreted by Cohen's Kappa statistic (κ), with the ROs' assessments deemed as the 'gold standard'.
RESULTS: The overall concordance for all graded toxicity assessments between HN APRT and ROs was 78.4%. Xerostomia, dysgeusia, pharyngeal pain and dermatitis assessment were evaluated as 'Good' with agreement ranging from κ = 0.608-0.640 between the HN APRT and ROs while dysphagia scored an 'Almost Perfect' agreement of κ = 0.834. 'Moderate' agreement between the HN APRT and ROs was observed for oral pain and mucositis assessment. A scoring discrepancy of 1 and 2 grades was observed in 21.2% and 0.4% for these two toxicities.
CONCLUSION: There was high concordance in scoring of acute toxicity between the HN APRT and ROs. The results support the continuing involvement of HN APRT in weekly assessments for NPC patients.

Introduction

The nasopharynx is a cuboidal space bounded by the nasal cavity (anteriorly), the soft palate (inferiorly), the skull base (superiorly), posterior pharyngeal wall (posteriorly) and the medial pterygoid plates (laterally), with the anterior-posterior diameter of 2 cm and the height of 4 cm [1]. Despite its small size, nasopharyngeal cancers (NPC) which arises from here, is the most common head and neck (HN) cancer in Singapore [2], [3]. This distinct HN malignancy has an unbalanced geographical distribution with 81% of cases arising in Asia, 9% in Northern Africa and the remaining worldwide [4], [5]. In the endemic region, the most common histology is non-keratinizing undifferentiated carcinoma, which is highly associated with Epstein-Barr virus (EBV) with evidence of infection seen in more than 95% of NPC [4], [5]. Due to its anatomical location and radiosensitivity, radiotherapy (RT), with or without chemotherapy, is the primary modality for radical treatment [2], [4].

More than 85% of patients with NPC present with nodal involvement at time of diagnosis [4], and so the typical treatment fields are extensive, extending from base of skull superiorly to level IV nodal region inferiorly. A dose of 70 Gray in 33–35 fractions over 6.5–7 weeks is required to treat NPC. Advanced RT techniques such as Intensity-Modulated Radiation Therapy (IMRT) is commonly used to minimize dose to adjacent critical structures such as the parotid glands, tongue and constrictor muscles [6]. Despite this, NPC patients typically experience severe radiation induced toxicities. Oral mucositis is a common side effect of HN irradiation and the incidence is increased in patients that require concurrent chemoradiotherapy (CCRT) [2], [7], [8].

Similar to other HN squamous cell cancers, NPC is a fast-proliferating tumour and hence, prolonged overall treatment time is detrimental for local control and overall survival for NPC patients [9]. Poor management of acute side effects is a common, but potentially avoidable reported cause of treatment interruptions [10]. We treat more than 300 patients with radical RT annually at our centre and one of our strategies to minimize treatment interruptions is to monitor patients closely and to quickly provide supportive and symptom relief measures before patients deteriorate. Weekly toxicity assessments are the foundation upon which acute RT side effects are assessed and managed, thus allowing patients to complete their treatment in a safe and timely manner [11].

In 2012, the Advanced Practice Radiation Therapist (APRT) specializing in HN cancer was piloted in National Cancer Centre Singapore (NCCS), with the endeavor to provide seamless care for patients having HN radiotherapy and to elevate the professional profile of radiation therapists [12], [13], [14]. This was also supported from ROs in our centre for trained APRT led treatment reviews, assessment of toxicities and reinforce nutritional recommendation made by the medical team, consistent with findings performed by Shi et al. [15]. The HN APRT underwent advanced site-specific education and 3 years of intensive clinical training by 3 HN ROs. The clinical training included understanding patient history and initial diagnosis, management of various HN sub-sites and radiation induced toxicities of HN treatment. The HN APRT was responsible for leading and developing key departmental HN initiatives through active involvement with the multi-disciplinary care team. With numerous integrated holistic training and comprehensive competency assessments, the role of the HN APRT was successfully expanded to include reviewing patients independently on behalf of ROs, subsequently reducing the overall waiting time for patients. Moreover, it helped to lessen the ROs’ workload, thereby permitting them to focus more on complicated cases, treatment planning and research [12].

The contribution of HN APRT in toxicity assessment for NPC radiotherapy patients was established in 2017 with the key intention to closely monitor patients’ radiation induced toxicity and psychosocial issues pertaining to their treatment. To further support and validate this role delegation, the concordance of toxicity assessment between HN APRT and ROs in patients with NPC patients was conducted to show that HN APRT’s toxicity assessment is comparable to ROs.

Methods

This study was conducted with twenty-three NPC patients under the care of 3 ROs from June to December 2018. All patients were treated with either CCRT or radiotherapy alone with a dose of 70 Gray in 33 fractions over 6.5 weeks. Weekly assessments were performed independently by both the HN APRT and RO-in-charge. To benchmark the best observation of assessment, the ROs’ grading was deemed as ‘gold standard’.

Radiation induced toxicities that are commonly observed in HN radiotherapy were shortlisted from the Common Terminology Criteria for Advanced Events (CTCAE) v4.0. A form adapted from the CTCAE v4.0 was included in the departmental Head and Neck Radiation Therapy Protocol and standardized for the use of patient assessment during RT. The evaluated HN toxicities included xerostomia, oral pain, mucositis, dysgeusia, dysphagia, pharyngolaryngeal pain and dermatitis. The APRT assessed patients every Monday, whereas the ROs assessed patients on Mondays, Thursdays or Fridays depending on their work schedule.

Both assessors evaluated the symptoms of radiation induced toxicities using the method described in Table 1. The HN APRT’s assessment was graded according patients to the CTCAE v4.0 form (Fig. 1). The ROs’ chart documentation was translated by the HN APRT and verified by the ROs to the CTCAE form for comparison. Hoarseness of voice was not included in this study as it is not a common side effect observed among the NPC patients undergoing radiotherapy as compared to other HN cancers.

Table 1

Assessment of symptoms of the head and neck radiation induced toxicities.

Toxicity	Method in assessing radiation induced toxicities
Nutrition	Weight, Amount of food intake
Xerostomia	Subjective degree of dryness and assessment on the stickiness of saliva
Mucositis	Physical examination of the oral cavity, oropharynx region
Oral Pain	The degree of oral pain on pain scale scoring related to mucositis, inflammation of gums, buccal mucosal (at rest, talking or eating/drinking)
Dysgeusia	Subjective degree of taste alteration or loss of taste
Dysphagia	Ease of swallowing function (smooth, difficulty, unable to swallow, choking)
Pharyngolaryngeal Pain	The degree of pain on pain scale scoring when swallowing, require NGT (with/without analgesic intervention)
Dermatitis	Physical examination of the skin

Fig. 1

A sample of a graded HN Acute toxicity assessment form.

The concordance rate and agreement level were interpreted by Cohen’s Kappa statistic, where it was used to measure the inter assessor reliability. According to Landis and Koch’s classification method, the agreement levels of Kappa results are interpreted as follows: value ≤0 indicates No Agreement, 0.01–0.20 as None to Slight, 0.21–0.40 as Fair, 0.41–0.60 as Moderate, 0.61–0.80 as Good and 0.81–1.00 as Almost Perfect agreement [16]. Data analysis was performed using IBM SPSS statistic version 20.0 [17]. The variances in grading the toxicity were observed, categories included perfect concordance, one grade and two grade differences.

Toxicity incidence of midpoint and endpoint were analysed in order to further eliminate ‘zero grading’ between the observers and to discriminate whether the HN APRT was able to recognize the toxicities that arose during RT.

Results

A total of twenty three patients were included in this study, with 10 patients receiving RT alone while 13 patients had CCRT. Patient demographics and disease characteristics are summarised in Table 2. The median age was 52 (range 24–75 years). 74% of the patients were male and 82.6% were of Chinese ethnicity. Patients with AJCC 7th Edition Stage I and II received RT alone while patients with Stage III and Iva/IVb disease received chemoradiotherapy.

Table 2

Demographics and disease characteristic by treatment modality.

		Radiotherapy Alone	Chemoradiotherapy
Number of Patients		10	13
Age at Diagnosis [Median (Range)]		51 (24–72)	53 (31–75)
Gender	Male	6	11
	Female	4	2
Race	Chinese	7	12
	Malay	1	0
	Indian	1	1
	Other	1	0
Disease Characteristic
T Category	1	9	5
	2	0	1
	3	1	4
	4	0	3
N Category	0	2	0
	1	7	3
	2	1	5
	3	0	5
Overall Staging	I	2	0
	II	7	0
	III	1	5
	IVa & IVb	0	8

A total of 721 entries were recorded independently by ROs and HN APRT for the patients categorized under the evaluated toxicities. Xerostomia, dysgeusia, pharyngolaryngeal pain and dermatitis assessment were evaluated as ‘Good’ agreement level (κ = 0.608–0640) between the HN APRT and ROs. ‘Moderate’ agreement was observed for oral pain (κ = 0.578) and mucositis (κ = 0.576) assessment. Dysphagia scored an ‘Almost Perfect’ agreement level with κ = 0.834 (Table 3).

Table 3

Kappa statistic agreement values and percentage of concordance between the head and neck advanced practice radiation therapist and radiation oncologists.

Toxicity	Agreement Level (κ value)	Percentage agreement of concordance
Oral Pain	Moderate (0.578)	76.9
Mucositis	Moderate (0.576)	72.8
Xerostomia	Good (0.606)	79.4
Dysgeusia	Good (0.607)	77.5
Pharyngolaryngeal Pain	Good (0.623)	75.5
Dermatitis	Good (0.640)	77.5
Dysphagia	Almost Perfect (0.834)	89.2

Overall, the HN APRT achieved a high concordance rate of 78.4% with the ROs for the 7 graded toxicities. One grade difference in toxicity assessment between HN APRT and ROs was observed in 21.2% of patients. Two grade differences was only seen in grading of mucositis, where there were 2.9% of patients that were graded differently (Table 4).

Table 4

Average percentage agreement and toxicity assessment grade difference between the head and neck advanced practice radiation therapist and all radiation oncologists.

Toxicity	HN APRT vs ROs
	No difference	1 grade difference	2 grade difference
Dry mouth	79.4	20.6	0
Oral Pain	76.9	23.1	0
Mucositis	72.8	24.3	2.9
Dysgeusia	77.5	22.5	0
Dysphagia	89.2	10.8	0
Pharyngolaryngeal Pain	75.5	24.5	0
Dermatitis	77.5	22.5	0
Average (%)	78.4	21.2	0.4

Table 5 demonstrated the concordance rates with individual ROs who reviewed NPC patients on their respective clinic day. The HN APRT assessed the NPC patients on the same day as the RO 1 and scored the highest agreement of 92.7%.

Table 5

Percentage agreement and grade differences between the HN advanced practice radiation therapist and individual radiation oncologist*

Toxicity	RO 1 (Monday)			RO 2 (Thursday)			RO 3 (Friday)
	same%	1 gradedifference %	2 gradedifference %	same%	1 gradedifference %	2 gradedifference %	same%	1 gradedifference %	2 gradedifference %
Xerostomia	94.6	5.4	0	75.0	25.0	0.0	61.9	38.1	0.0
Oral Pain	89.2	10.8	0	73.3	26.7	0.0	63.6	36.4	0.0
Mucositis	89.2	10.8	0	64.5	31.1	4.0	61.9	33.3	4.8
Dysgeusia	89.2	10.8	0	82.2	17.7	0.0	54.5	45.5	0.0
Dysphagia	97.3	2.7	0	88.6	11.4	0.0	76.2	3.8	0.0
PharyngolaryngealPain	91.9	8.1	0	70.5	29.5	0.0	57.1	42.8	0.0
Dermatitis	97.3	2.7	0	58.7	41.3	0.0	81.0	19.0	0.0
Average	92.7	7.3	0	73.3	26.1	0.6	65.2	31.3	0.7

* Only entries which were assessed by both RO and APRT were included in this table.

Further analysis of the toxicity incidence at midpoint and endpoint of the NPC patients was performed (Table 6). Overall assessment was similar for most of the toxicities, with a high concordance achieved by both observers in identifying ‘grade 3′ toxicities. Minor discrepancy in evaluating xerostomia and dermatitis were observed for the endpoint.

Table 6

Toxicity incidence assessed during midpoint and endpoint of treatment course between the HN advanced practice radiation therapist and individual radiation oncologist*

Week (Midpoint)	RO				APRT
Toxicity	G0	G1	G2	G3	G0	G1	G2	G3
Xerostomia		82.4%	17.6%			76.5%	23.5%
Oral Pain	47.1%	47.1%	5.9%		35.3%	58.8%	5.9%
Mucositis	17.6%	52.9%	29.4%		11.8%	47.1%	41.2%
Dysgeusia		70.6%	29.4%			76.5%	23.5%
Dysphagia	17.6%	58.8%	23.5%		5.9%	70.6%	23.5%
Pharyngolaryngeal Pain	41.2%	47.1%	11.8%		41.2%	41.2%	17.6%
Dermatitis Radiation	29.4%	58.8%	5.9%	5.9%	47.1%	47.1%	5.9%
Week (Endpoint)	RO				APRT
Toxicity	G0	G1	G2	G3	G0	G1	G2	G3
Xerostomia		29.4%	70.6%			5.9%	94.1%
Oral Pain	11.8%	52.9%	35.3%			64.7%	29.4%	5.9%
Mucositis		23.5%	64.7%	11.8%		29.4%	58.8%
Dysgeusia		23.5%	76.5%			11.8%	88.2%
Dysphagia		17.6%	64.7%	17.6%		11.8%	70.6%	17.6%
Pharyngolaryngeal Pain	11.8%	23.5%	52.9%	11.8%	5.9%	47.1%	35.3%	11.8%
Dermatitis Radiation		5.9%	70.6%	23.5%		35.3%	47.1%	17.6%

* Only entries which were assessed by both RO and APRT were included in this table.

Discussion

RT is the main treatment modality for NPC. Studies have shown that CCRT confers a significant survival benefit for patients with locally advanced disease [2], [7]. Most patients undergoing RT for NPC will experience some RT related side effects. These side effects manifest earlier and are increased in frequency and severity for patients receiving CCRT [2], [7], [18]. In the study by Monk et al. [19], an audit of treatment review clinic showed that overall, 93% HN cases required medical intervention during RT. Weekly treatment reviews are crucial for toxicities such as oral pain, mucositis, dermatitis and dysphagia to be recognized and treated before patients become severely ill [18], [19]. This allows patients with ‘Category 1′ (rapidly proliferating) tumour to complete their treatment without interruptions, so that their outcome is not compromised [11].

From the results, there was a significant agreement achieved between the HN APRT and ROs in assessing toxicities in dysphagia, xerostomia, pharyngolaryngeal, dysgeusia and dermatitis with the agreement level of ‘Moderate’ to ‘Almost Perfect’ while there was slight variability in oral pain and mucositis. Oral pain arises as a consequence of development of mucositis, undoubtedly both graded toxicities are inter-related [8]. In the healthcare setting, Kappa statistic above the ‘Moderate’ rating (κ > 0.600) is considered reliable consistency for the inter-rater because it is a more robust measurement and it includes the possibility of agreement occurring by chance [16]. With a high inter-rater assessment and a perfect concordance of 78.4% for grading all toxicities, these results indicate that a properly trained HN APRT can be as competent as ROs in assessing toxicities during HN treatment. A similar quantitative study carried out by Lee et al. [20] validated that the Clinical Specialist Radiation Therapist (CSRT) possessed the clinical competency in assessing and grading side effects for patients receiving breast radiotherapy.

In our department, clinical schedules only allows for the APRT led treatment review clinic to run every Monday whereas the individual ROs weekly clinics were scheduled on Mondays, Thursdays, and Fridays depending on their availability. Indeed, there was a significantly high percentage agreement of 92.7% concordance rate with RO 1 who conducted the review clinic on the same day (Table 4). Subsequently, the clinic day for RO 2 was three days apart, with the concordance rate of 73.3% and followed by reduction to 65.2% of concordance with RO 3 where the consultation was four days apart from the APRT treatment review. Overall, the ‘one grade difference’ was noted to be slightly higher for RO review days on Thursday and Friday. Interestingly, there was an increasing disparity over the week between two assessors which was reflected in the average toxicity grading. This can be explained by the cumulative side effects that occur between Mondays (when assessment was performed by APRT) to Thursday/Friday (when assessment was performed by ROs).

The patients with a ‘two grade difference’ that was observed between the HN APRT and RO 2 and 3 were all undergoing CCRT and the mucositis was likely to be accentuated by chemotherapy as a radiosensitizer [2]. Multiple studies have shown that NPC patients with CCRT potentially have worsening toxicities, therefore resulting in the discrepancy on mucositis occurrence within a few days [2], [7], [21]. This result could suggest that patients on CCRT may require more than just one treatment review a week in order for the toxicities to be recognized early and intervention be initiated. However, in many centres, treatment review is usually performed by the primary ROs, registrars or medical officers (MOs) on a weekly basis. Demand on doctors’ time is increasing due to increased complexity of clinical practices, radiotherapy techniques as well as other competing demands from additional responsibilities in research, education and administration [12]. We propose that a trained APRT may be a viable solution to fill this gap. While the APRT’s role is not to deliver medical treatment or to replace the RO, early accurate recognition of toxicities needing intervention can be escalated to the ROs for medical intervention, leading to improved symptom control, improved quality of life and lesser treatment interruption. Moreover, through a strong collaboration between HN APRT and ROs in reviewing the HN patients, there is an improved communication between the RTs and ROs as well as promoting a collaborative approach in managing patients’ overall wellbeing [12], [14], [19], [22].

Patients undergoing HN radiotherapy will typically manifest radiation induced adverse toxicities gradually over the course of treatment. Thus, our current study also assessed the concordance and agreement levels of the RO and HN APRT in toxicity grading over two time point (mid and endpoint) to assess non-zero grade toxicities. The overall midpoint assessments for toxicity incidence were comparable as well for both raters. Looking more closely at the endpoint of toxicity incidence, a minor divergence was observed for ‘grade 2′ xerostomia and dermatitis. The HN APRT had higher overall rating for xerostomia while the ROs graded dermatitis more extensively (Table 5). The disparity observed for grading xerostomia and dermatitis were likely due to the discrete scoring definition of CTCAE, resulting in some subjectivity of individual’s assessment [20], [23]. This was also seen in study by Lee et al. [20] who reported subjectivity of hyperpigmentation and dermatitis grading due to similar scoring system. In addition, the variability of dermatitis scoring, might suggest that the skin desquamation could have progressed consequentially over the different review days.

There are several limitations for this study. Ideally, all treatment reviews should be performed on the same day to improve validity of comparison. It was decided to take a pragmatic approach so as not interrupt routine clinical practice. Secondly, there was some missing data as the ROs may not have the reviewed the patients timely on a weekly basis due to other commitments. This is precisely the argument to have a trained APRT to fill this gap left by the busy RO. Thirdly, the study number is relatively small and commitment from both reviewers was a limiting factor to extend the duration of the study. Fourthly, ROs’ toxicities were written in chart documentation had to be translated to the CTCAE form for comparison. This could result in a possibility of translational or interpretational errors. In order to mitigate this, any uncertainties in RO’s grading were verified with the RO making the entry. Finally, this study was limited to a specific subset of patients with NPC only. While it is difficult to confidently conclude that the result of this study is generalizable to all HN cancers, we believe that the ability of an APRT to assess toxicities is transferable to other HN subsites.

Conclusions

This study demonstrated a high concordance between APRT and ROs in weekly assessment of toxicities for patients on treatment for NPC. These results support the continuing involvement of the HN APRT in weekly assessments for NPC patients. With a well-structured training program and competency based accreditation, APRT can provide this value added service in radiotherapy departments. This allows early identification of patients needing medical attention thus minimising treatment interruptions. It also allows RO time to be utilised for other responsibilities. Moving forward, our department is keen to explore HN APRT led treatment toxicity assessment beyond NPC to other HN sub-sites.

Funding

Melvin L.K. Chua is supported by the National Medical Research Council Singapore Clinician Scientist Award (NMRC/CSA-INV/0027/2018), National Research Foundation Proton Competitive Research Program (NRF-CRP17-2017-05), Ministry of Education Tier 3 Academic Research Fund (MOE2016-T3-1-004), the Duke-NUS Oncology Academic Program Goh Foundation Proton Research Programme, NCCS Cancer Fund, and the Kua Hong Pak Head and Neck Cancer Research Programme.

Declaration of Competing Interest

The authors of this article report no conflict of interest. Disclosure of potential conflicts of interest:Melvin L.K. Chua reports personal fees from Astellas, Janssen, Bayer, Pfizer, MSD, personal fees and non-financial support from AstraZeneca, personal fees and grants from Ferring, personal fees and non-financial support from Varian, non-financial support from Decipher Biosciences, non-financial support from MedLever, and consults for immunoSCAPE Inc., outside the submitted work.