Validity and Reliability Assessment of the Persian Version of Therapy-Related Symptom Checklist.

Therapy-related symptom checklist for children (TRSC-C) was developed as a symptom assessment tool in children receiving chemotherapy. The objective of the present study was to evaluate the validity and reliability of the Persian version of TRSC-C. This cross-sectional study was conducted in 2013-2014 in Tehran, Iran. TRSC-C was translated using backward-forward approach. The content validity, face validity, and comprehensiveness were investigated based on the opinion of experts. The item content validity index (I-CVI) and scale content validity index (S-CVI) were calculated by the mean approach and inter-rater agreement. The scale was revised based on the comments from a team of five experts, after which it was evaluated by an additional group of four experts. To assess the inter-rater reliability, two raters filled the scale with 29 and 30 patients in the outpatient clinic of Hazrat-e Ali Asghar Hospital. The Cronbach's alpha was calculated and factor analysis was performed. The scores of content validity were analyzed in Excel. Other statistical analyses were performed using the SPSS software version 20.0. Based on the initial assessment, the S-CVI with less conservative approach was 60% for clarity, 33% for relevancy, and 60% for simplicity. After revising the scale, the S-CVI reached 100%. The comprehensiveness and face validity of the scale were appropriate. The scale was inter-rater reliable and the Cronbach's alpha was 0.803. Eleven subscales were found in the TRSC-C. It is concluded that the Persian TRSC-C is a valid and reliable tool for measuring children symptoms. Availability of a valid and reliable checklist is a fundamental step in monitoring the symptoms of patients while receiving chemotherapy.

What’s Known

Assessment of symptoms in pediatric patients with cancer is important. Several checklists are available for such assessment including therapy-related symptom checklist for children (TRSC-C). This checklist has been translated into different languages. However, none of the tools for the evaluation of symptoms in pediatrics has been translated into Persian

What’s New

We assessed the linguistic, face and content validity of the translated TRSC-C. As confirmed by experts, the Persian version of TRSC-C was content valid. Reliability measures including inter-rater reliability and internal consistency were acceptable (Cronbach’s alpha: 0.803).

Evaluation of the construct validity by factor analysis led to 11 factors.

Introduction

Cancer is one of the major causes of mortality among children.[1] Pediatric cancer cure rate is greatly improved at the cost of a high prevalence of symptoms during treatment.[2] In addition, symptoms may raise from the disease,[1,3] co-morbidities,[4] and medical procedures.[3] Moreover, patients’ symptoms are perceived differently by the patients, caregivers and physicians.[5] The perception of parents could be a source of identifying the symptoms in pediatric patients with cancer. However, it is not always a suitable alternative for children’s self-report. It seems that when children describe their problems from their own perspective, the clinicians are better equipped to help them adequately.[3]

Several tools such as advanced symptom management system for young people, memorial symptom assessment scale, symptom distress scale, therapy-related symptom checklist (TRSC), and Rotterdam symptom checklist have been developed for the assessment of symptoms in pediatric patients receiving chemotherapy.[2] TRSC was developed to fill the gaps in the documentation of several symptoms.[6] Having the advantage of ease of completion, it “works well in clinical settings and patient-centered care”.[6] The uncalibrated 23-item child version of TRSC (TRSC-C) was first piloted by interviewing parents or caregivers of children who received chemotherapy.[7] Then, the checklist was enhanced and calibrated to 30-item by including children with cancer.[8]

To the best of our knowledge, none of the tools for the assessment of symptoms in children has been translated into Persian nor validated. So, the present study was designed to evaluate the reliability and validity of the Persian version of TRSC-C.

Patients and Methods

This cross-sectional study was conducted in 2013-2014 in Tehran and was approved by the Research Ethics Committee of Tehran University of Medical Sciences (TUMS).

Questionnaire Description

In order to be comprehensive, each item of the TRSC-C, which was developed by Williams et al.,[7] points to a symptom that is described by at least two child-friendly terms. The severity of symptoms can be graded from 0 to 4 (the highest severity).[8] Other symptoms can be documented under the category “other” (item 31).

Validity Measures

Linguistic validation: A clinical pharmacist and a pediatric oncologist independently translated TRSC-C from English to Persian. Special attention was paid to select culturally suitable and child-friendly terms. Then, another clinical pharmacist and an epidemiologist assessed translations while incorporating them into one translation. Following back-translation by two other clinical pharmacists, the original and the back-translated scales were compared. Items in the back-translation that were deemed different from the original version (items 11, 13, 14, 16, 20, 25, 26, and 29) were again re-translated and back translated. The document was then considered as the final version.

Face and content validity: Content validity was examined by requesting a number of professionals to rate (from 1: Inappropriate to 4: Quite appropriate) the relevance, clearness, and simplicity of each item. The team of professionals included a pediatric oncologist, two pediatric oncology fellows, and two nurses at the pediatric chemotherapy ward of Hazrat-e Ali Asghar hospital. This pediatric hospital is affiliated with Iran University of Medical Sciences. Additionally, face validity, comprehensiveness, and necessity for omitting or adding descriptions for symptoms were evaluated.

Item content validity index (I-CVI): This index rates the relevancy, clarity, and simplicity of each item. The agreements between the team of professionals were examined as described in the literature.[9-11]

Scale content validity index (S-CVI): Two methods for the calculation of this index were used, namely the inter-rater agreement and the mean approach.

Inter-rater agreement: This assessment was performed in two ways: Conservative approach and less conservative approach.

Conservative approach: The number of items that all experts rated as “quite appropriate” or “appropriate” was divided by the total number of items.

Less conservative approach: The number of items that the majority of experts (80%) rated as “quite appropriate” or “appropriate” was divided by the total number of items.

Mean approach: The number of items rated as “appropriate” was divided by the total number of items.

The subsequent revision of TRSC-C, based on the results and comments, was re-evaluated by another team of experts in the Children Medical Center hospital affiliated with TUMS. The team of five experts included three nurses and two pediatric oncologists. Unfortunately, one team member did not submit the scale and subsequently, the items rated as “quite appropriate” or “appropriate” by three of the experts were accepted with the conservative approach. The results were then re-analyzed.

Construct Validity

Despite an inadequate number of patients in this pilot study, the exploratory principal component analysis (PCA) was used with some limitations to verify construct validity by the factorial design. Bartlett’s test and Kaiser-Myer-Olkin (KMO) measure were calculated to test the sphericity and adequacy of the sampling. Bartlett’s sphericity test results (chi-square: 844.8, degree of freedom: 465, P<0.001) indicated acceptable correlations of the data to perform PCA. However, the KMO measure of sampling adequacy (0.312) implied a poor sample size. Nevertheless, PCA was performed as it has been suggested that if a factor has four or more loadings of >0.6, the test is reliable regardless of the sample size.[12]

Factors with eigenvalues of >1.00 were retained and items with the loading of >0.4 were included in each factor. All items had adequate loadings and retained in the scale. To optimize the interpretation, quartimax with Kaiser normalization were used for rotation of the extracted components.

Reliability Measures

Inter-rater reliability: To evaluate the reliability, a cross-sectional study was conducted in the outpatient clinic of pediatric oncology in Hazrat-e Ali Asghar Hospital. Patients attend this clinic for follow-up visits or receiving chemotherapy. After obtaining the consent from parents, 5 to 17 years old patients who had received chemotherapy for at least one week and willing to answer the questions were included. Children were excluded if they were unable to communicate independently, had hearing or speaking problems, organic brain syndrome, severe psychiatric disorders, or spoke languages other than Persian. Moreover, patients who received their last chemotherapy earlier than the previous month were not eligible.

Children were asked about their symptoms after their last chemotherapy. To be more understandable, visual schematic faces were used along with the scores for the assessment of severity. The checklist was primarily self-administered. However, similar to Williams et al.,[8] a researcher with/without parents assisted the children who could not read. The first and second rater enrolled 30 and 29 patients, respectively. The patient population was similar and the second rater could fill the scale with patients who had once completed the scale with the first rater.

The internal consistency was calculated by Cronbach’s alpha (minimum acceptable: 0.70) and total item correlations (minimum acceptable: 0.20). The inter-rater reliability was measured by comparing the results of the scales that were filled by patients with raters using the Mann–Whitney U test. Content validity scores were analyzed in Excel (Microsoft Office 2010). Other statistical analyses were performed using the statistical package for the social sciences (SPSS) version 20.0.

Results

Face and Content Validity

The first group of experts commented on the wording of items and recommended additional descriptions for certain items. However, face validity was commented to be appropriate. The mean S-CVI with conservative approach was below the acceptable levels (table 1).

Table 1

Results of the items content validity index (I-CVI), scale content validity index (S-CVI), and the mean approach based on the feedback from the first team of experts

Items		Clarity (%)	Relevance (%)	Simplicity (%)
	I-CVI
1	Loss of appetite	80	80	100
2	Nausea	100	100	100
3	Vomiting	100	80	100
4	Weight loss	60	60	40
5	Sore mouth	100	100	80
6	Difficult swallowing	100	100	100
7	Sore throat	80	80	80
8	Jaw pain	40	60	40
9	Cough	100	80	100
10	Shortness of breath	80	80	80
11	Feeling sluggish	60	60	60
12	Depression	60	60	60
13	Difficulty concentrating	100	100	100
14	Difficulty sleeping	100	100	100
15	Fever	80	80	80
16	Bruising	100	80	100
17	Bleeding	100	80	100
18	Hair loss	100	100	100
19	Skin changes	60	60	60
20	Pain	100	80	100
21	Numbness of fingers and/or toes	80	80	80
22	Constipation	100	100	100
23	Sweating	80	60	80
24	Itching	100	80	100
25	Hard to urinate	100	80	100
26	Afraid	100	80	100
27	Headache	80	80	80
28	Agitation	100	100	100
29	Irritable	100	100	100
30	Difficulty standing/walking	100	100	100
	S-CVI
	Conservative	60	33	60
	Less conservative	83	80	83
	Mean agreement	88.00	82.67	87.33

Based on the comments and data analysis, some parts (items 8, 10, 11, 19, 21, 23, 26, and 30) were modified by substituting or adding child-friendly terms. Some experts mentioned that weight loss is irrelevant due to weight gain as a result of corticosteroids administration. However, it was decided that without deleting weight loss, we add weight gain at the end of the scale. In the second assessment, the S-CVI reached 100% (table 2). As shown in figure 1, based on the feedbacks from the second team of experts, several wording modifications were made for some items (items 1, 2, 5, 10, 12, 19, 20, 22, 23, 25, 27, 29, 30).

Table 2

Results of the items content validity index (I-CVI), scale content validity index (S-CVI), and the mean approach based on the feedback from the second team of experts

Items		Clarity (%)	Relevance (%)	Simplicity (%)
	I-CVI
1	Loss of appetite	100	100	100
2	Nausea	75	75	100
3	Vomiting	100	100	75
4	Weight loss	75	75	75
5	Sore mouth	100	100	75
6	Difficult swallowing	75	75	75
7	Sore throat	100	100	75
8	Jaw pain	75	75	75
9	Cough	100	100	100
10	Shortness of breath	75	75	75
11	Feeling sluggish	100	100	100
12	Depression	100	75	100
13	Difficulty concentrating	75	100	100
14	Difficulty sleeping	100	100	100
15	Fever	75	75	75
16	Bruising	75	100	100
17	Bleeding	100	100	100
18	Hair loss	100	100	100
19	Skin changes	100	100	75
20	Pain	75	75	75
21	Numbness of fingers and/or toes	75	75	75
22	Constipation	100	100	100
23	Sweating	100	100	100
24	Itching	100	75	100
25	Hard to urinate	100	100	75
26	Afraid	100	100	75
27	Headache	100	100	100
28	Agitation	75	75	75
29	Irritable	75	75	75
30	Difficulty standing/walking	100	100	100
31	Weight gain	75	75	75
	S-CVI
	Conservative	60	60	50
	Less conservative	100	100	100
	Mean agreement	90.00	90.00	87.50

Figure 1

The Persian version of therapy-related symptom checklist for children (TRSC-C).

Patients’ characteristics: Amongst the 59 patients who filled the scale with 2 raters, 21 were interviewed twice. The first rater interviewed 30 patients (16 male, mean age: 8.74±3.78 years) comprising of 15 preschool, 8 elementary and 7 guidance school children. The diagnosis of 93% of the patients was acute lymphoblastic leukemia (ALL). The TRSC-C was answered by 24 patients with rater (80%), 3 patients alone, and 3 patients with parents and rater. The second rater interviewed 29 patients (17 male, mean age: 8.54±3.22 years) comprising of 14 preschool, 9 elementary and 6 guidance school children. The diagnosis of 83% of the patients was ALL. The TRSC-C was answered by 20 patients with rater (69%), 3 patients alone and 6 patients with parents and rater.

Inter-rater reliability: Repeated records of items by two interviewers on similar population were not statistically different (P>0.05) (table 3).

Table 3

Reliability statistics of the Persian version of TRSC-C

Items	Total		Rater 1		Rater 2		P value[a]
	Corrected item - total correlation	Cronbach’s alpha if item deleted	Corrected item-Total correlation	Cronbach’s alpha if item deleted	Corrected item-Total correlation	Cronbach’s alpha if item deleted
Loss of appetite	0.32	0.80	0.22	0.79	0.25	0.80	0.07
Nausea	0.52	0.79	0.28	0.78	0.45	0.79	0.76
Vomiting	0.29	0.80	0.27	0.78	0.28	0.80	0.32
Weight loss	0.55	0.79	0.56	0.78	0.46	0.80	0.99
Sore mouth	0.19	0.80	0.05	0.79	0.33	0.80	0.75
Difficult swallowing	0.32	0.80	0.26	0.78	0.24	0.80	0.81
Sore throat	0.36	0.79	0.33	0.78	0.39	0.79	0.69
Jaw pain	0.27	0.80	0.33	0.78	0.34	0.80	0.72
Cough	0.39	0.79	0.20	0.79	0.56	0.79	0.94
Shortness of breath	0.26	0.80	0.24	0.79	0.45	0.79	0.85
Feeling sluggish	0.61	0.78	0.61	0.77	0.50	0.79	0.11
Depression	0.41	0.79	0.55	0.77	0.33	0.80	0.16
Difficulty concentrating	0.24	0.80	0.53	0.77	0.11	0.80	0.71
Difficulty sleeping	0.32	0.80	0.43	0.78	0.20	0.80	0.56
Fever	0.44	0.79	0.23	0.79	0.44	0.79	0.96
Bruising	0.14	0.80	0.14	0.79	0.33	0.80	0.19
Bleeding	0.43	0.79	0.49	0.78	0.20	0.80	0.92
Hair loss	0.45	0.79	0.43	0.78	0.42	0.79	0.81
Skin changes	0.006	0.81	0.05	0.79	0.03	0.81	0.82
Pain	0.17	0.80	0.18	0.79	0.42	0.79	0.77
Numbness of fingers and/or toes	0.23	0.80	0.33	0.78	0.44	0.79	0.35
Constipation	0.27	0.80	0.43	0.78	0.36	0.80	0.27
Sweating	0.47	0.79	0.44	0.78	0.28	0.80	0.69
Itching	0.30	0.80	0.50	0.78	0.10	0.80	0.41
Hard to urinate	0.39	0.79	0.41	0.78	0.23	0.80	0.42
Afraid	0.19	0.80	0.27	0.78	0.26	0.80	0.19
Headache	0.51	0.79	0.59	0.77	0.40	0.79	0.11
Agitation	0.15	0.81	0.01	0.80	0.18	0.81	0.11
Irritable	0.08	0.81	-0.04	0.80	0.33	0.80	0.92
Difficulty standing/walking	0.24	0.80	0.57	0.78	0.22	0.80	0.27
Weight gain	0.05	0.81	-0.16	0.81	0.24	0.80	0.21
Total scale range (mean)	0.006 to 0.189 (0.309)	-	-0.164 to 0.606 (0.314)	-	0.034 to 0.562 (0.315)	-	-

P value is reported for the comparison between the mean severity scores of the completed TRSC-C between the two raters by Mann-Whitney U test.

Internal consistency: The Cronbach’s alpha of the scale for the data collected by the first and second rater were 0.789 and 0.803, respectively. The results of the TRSC-C scales filled by the first and second rater showed that 7 and 4 items failed to reach the total item correlation of >0.20, respectively (table 3). The Cronbach’s alpha for the first interview of all patients (n=38) was 0.803. However, deleting items in cases of low correlation did not result in a considerable effect on the calculated Cronbach’s alpha.

Construct validity: From the PCA, 11 factors with eigenvalues >1.00 were extracted (table 4) which explained 82% of the sample variance. Out of the 31 items, 13 items loaded >0.40 on the first factor, among which 5 items had a correlation of >0.60 with the factor.

Table 4

Rotated matrix of correlations between items and factors with eigenvalues=1.0, percentage of variance, and related Cronbach’s alpha for the Persian version of TRSC-C

Factor and subscales (% of variance, Cronbach’s alpha[a])	Coefficient
Factor 1 (17.540, 0.827)
Sweating	0.804
Feeling sluggish	0.785
Weight loss	0.764
Nausea	0.740
Headache	0.674
Bleeding	0.520
Cough	0.508
Vomiting	0.451
Fever	0.433
Loss of appetite	0.414
Factor 2 (11.817, 0.827)
Difficult swallowing	0.902
Sore throat	0.856
Sore mouth	0.833
Hard to urinate	0.621
Fever	0.453
Factor 3 (9.803, 0.694)
Weight gain	0.838
Shortness of breath	0.837
Numbness of fingers and/or toes	0.634
Hair loss	0.542
Constipation	0.423
Factor 4 (7.699, 0.578)
Bruising	0.898
Vomiting	0.551
Factor 5 (6.753, 0.738)
Depression	0.849
Difficulty concentrating	0.691
Vomiting	0.426
Factor 6 (6.224, -0.666)
Irritable	0.670
Skin changes	-0.811
Factor 7 (5.571, 0.632)
Itching	0.893
Hard to urinate	0.597
Jaw pain	0.594
Factor 8 (4.661, -0.220)
Agitation	0.893
Afraid	0.458
Loss of appetite	-0.400
Factor 9 (4.471, 0.404)
Difficulty standing/walking	0.803
Nausea	0.409
Factor 10 (4.099, 0.531)
Pain	0.907
Constipation	0.620
Factor 11 (3.687, ---)
Difficulty sleeping	0.858

Cronbach’s alpha was calculated on the subscales of the factors with loading>0.40

Comprehensiveness: Neither the experts nor the children provided additional symptoms to the scale. Therefore, we consider the Persian TRSC-C to be comprehensive.

Discussion

The present study was conducted to assess the reliability and validity of the Persian TRSC-C. TRSC-C has the advantage of including items regarding both physical and psychological symptoms as well as its successful use in children as young as 5 years old.[8] With TRSC-C, the presence of symptoms as well as their severity can be evaluated. The original version used the Likert-type scale for the assessment of severity.[8] However, we additionally used visual schematic faces since we consider this approach to be more appropriate for young children who may have difficulty distinguishing the level of severity between “a little bit” and “quite a bit” which was suggested by the scale.

We also examined the linguistic and content validity of the scale. Williams et al. studied the Thai version of TRSC-C in children and parents following linguistic validation. However, they did not report the results despite mentioning that 10 nurses and 12 parents evaluated the appropriateness and convenience of the checklist, respectively.[13] In another study with the Spanish version of TRSC-C, the translation was validated and the ease of completion was evaluated by 5 parents. However, no data was presented regarding the content validity.[14] We also found that the inter-rater reliability of the scale was acceptable. Such assessment has not been performed on TRSC-C in the past. In this study, we did not perform test-retest for the assessment of reliability because of “questionable value” for relatively transient symptoms.[15]

Cronbach’s alpha, which is often applied for the demonstration of the reliability,[16] was 0.803 in our study. Williams et al. reported it as 0.91 for the original scale[8] while it was 0.87 and 0.91 for the Thai[13] and Spanish[14] TRSC-C, respectively. Although the total Cronbach’s alpha was acceptable but the low total item correlations could be an indicator for the subscales. Therefore, we performed PCA that yielded to 11 clusters. Only one study has reported the results of the factor analysis with TRSC-C in which 7 clusters were demonstrated.[8] Similar to our study, Williams et al. noted that the Cronbach’s alpha of some factors were less than 0.7. However, as Williams et al. proposed, we believe that using the scale can be acceptable in patients due to the acceptability of the Cronbach’s alpha of the total scale.[8] It should be mentioned that none of the studies on translated TRSC-C had performed PCA.

To the best of our knowledge, this is the first study to evaluate the psychometric properties of a symptom checklist for pediatric cancer patients in Iran. It should be noted that some aspects of the psychometric properties of the scale have not been performed in previous studies on translated TRSC-C.

There are a couple of limitations in the present study. In the absence of a gold standard for the assessment of symptoms in children undergoing chemotherapy, we could not measure the criterion validity. Moreover, in examining the content validity, the recommendations and comments made by the team of experts are subjective and perhaps prone to bias.[17] Additionally, the sample size of this pilot study was small whereas the factor analysis requires larger samples.

Conclusion

The present study has demonstrated that the Persian version of TRSC-C is a valid and reliable tool and can be used for the documentation of the symptoms in pediatric cancer patients.