Translation and Validation of the Indonesian Version of the Adverse Drug Reaction Severity Level Instruments in Colorectal Cancer Patients.

Introduction: Assessment of the severity of adverse drug reaction (ADR) is very rare in Indonesia. The severity of ADR can describe how serious this affects the clinical condition of the patient. In Indonesia, there are no instruments used to measure the severity of ADR. Purpose: This study aims to translate, pilot test, and validate Hartwig instruments for measuring the severity of ADR in colorectal cancer patients in Indonesia. Patients and
Methods: The translation method was used forward-backward technique from English to Indonesian, then being retranslated from Indonesia to English. The instrument of Indonesian version was used to assess the severity of ADR as the effect of chemotherapy. The assessment was conducted to 10 colorectal cancer patients by 30 health workers. The test validity was done based on content validity ratio (CVR) and content validity index (CVI); meanwhile, the test reliability was based on intraclass coefficient correlation (ICC).
Results: All of the results of CVR present a value of >0.33, while the range of CVI moves between 0.8 to 1.0, which declares that the instrument is valid. The satisfactory alpha value for reliability is 0.996 with signification of 0.197 (p > 0.05) based on ANOVA analysis. Meanwhile, the ICC value of 0.896 indicates a good reliability among raters.
Conclusion: Indonesian version of Hartwig Instrument can be applied in measuring the severity of ADR caused by chemotherapy in colorectal cancer patients.

Introduction

Pharmacovigilance is a whole activity about detection, assessment, understanding, and prevention of adverse drug reaction (ADR) or other problems associated with drug use. The safety of drug circulating should be continuously monitored due to the limited safety information in the drug development phase (clinical study). This monitoring is carried out through the activity of pharmacovigilance. The purpose of pharmacovigilance is to detect unknown drug safety issues, detect an increased frequency of occurrence of ADR, identify risk factors, quantify risks, communicate drug safety information and prevent the occurrence of drug safety risks.1 The drug must be efficacious, safe, and qualified so that it can be marketed and used by patients. Studies conducted during drug research and development provide reliable information about its efficacy, however, information about its security may be less reliable. Such detailed information can only be obtained through its use in the general population under standard practice conditions.2

Pharmacovigilance is a process of identification and response to a problem in the administration of drugs.3 Potential risks or toxicity due to drug use is a problem of particular concern for patients, doctors, circulation permit holders, and regulatory authorities. ADR are often the cause of medical problems, which sometimes lead to hospitalize even being the cause of death of patients and even the cause of morbidity and mortality worldwide.4 Furthermore, in recent years, many drug products have been withdrawn from circulation as a result of undetected risks when drug products have been approved for market.5

It is important to know the possibility of ADR which how much-unexpected reaction is occurs in patients due to the influence of the drug. Assessment of the severity of ADR is very rare in Indonesia. The severity of ADR can describe how serious this affects the clinical condition of the patient. Instruments used to measure the severity of ADR in patients generally use common terminology criteria for adverse events (CTCAE) version 5.0, descriptive terminology that can be used for reporting adverse events (AE).6 There are three criteria for assessing the severity of ADR. It is determined as high severity if it causes death or requires hospital treatment, moderate severity if it requires a change in treatment management and mild severity if it does not require a change in the treatment.7 The severity of mild ADR is usually defined as an unnecessary reaction; i.) changes in the drug regimen, and ii.) specific antidote/treatment. Moderate ADR is intended for those require drug regimen and/ or anti-dote/ treatment replacement to relieve ADR and limit the daily activities. ADR that severe has the potential for lifelong threatening reactions, require hospitalization, and result in significant disability.8 Another instrument that can be used to assess the severity of ADR is the Hartwig Instrument,9 which assesses the severity of ADR based on its effects. By assessing the severity of ADR in patients, health workers can anticipate the occurrence of ADRs in the future, especially in moderate to severe ones. In Indonesia, there are no Indonesian version of instruments used to measure the severity of ADR, therefore the authors are interested in translating, testing, and validating questionnaires in patients at high risk of ADRs, one of which is colorectal cancer patients undergoing chemotherapy.

The prevalence of cancer in Indonesia based on the results of basic health research of the Ministry of Health of the Republic of Indonesia increased from 1.4 per mil in 2013 to 1.8 per mil in 2018.10 Based on Globocan, in 2012, the incidence of colorectal cancer in Indonesia is 12.8 per 100,000 adults, with mortality of 9.5% of all cancer cases,10 which makes it rank third in Indonesia. Chemotherapy is the first option in advanced cancer with palliative purposes. Chemotherapy can lead to the occurrence of ADR as the result of combination of several drugs and its repetition administration in multiple cycles. One of the ADR that can occur as the effect of the use of 5-FU (5-Fluorouracil) is stomatitis and esofagopharyngitis, diarrhea, anorexia, nausea and vomiting, ulcers and gastrointestinal bleeding, leukopenia (leukocytes <3500/μL), or decreased leukocytes, thrombocytopenia (platelets < 100,000/μL), and rare effects can include palmar-plantar erythrodysesthesia syndrome or ankle syndrome, and alopecia.10 The results of a study at Seoul National University Hospital (SNUH), Seoul, Korea, among 357 colorectal cancer patients, there are 284 patients (79.6%) who experienced ADR that occurred in the 5-Fu regimen. Such ADRs are neutropenia 57.7%, alopecia 7.6%, nausea/ vomiting 28.3%, diarrhea 28 5%, hand-foot syndrome 3.9%.11

The nausea and vomiting occur commonly in the FOLFIRI regimen than FOLFOX and XELOX. Severe diarrhea occurs only with the FOLFIRI regimen. Mucositis is more common occurring because of XELOX, also the FOLFOX regimen is associated with the same level of mucositis as XELOX. Neuropathy occurs more common because of XELOX followed by FOLFOX and FOLFIRI. Hair loss is more common occurring because of FOLFIRI followed by XELOX and FOLFOX.12 The ADRs that occur in patients who get a FOLFOX regimen are peripheral neuropathy as much as 4 (9%) and in the group that got leucovorin chemotherapy 5-fluorouracil, there are relatively fewer ADRs namely nausea and vomiting in 2 (4%).13

Over the past decade, several studies have shown that drug-related morbidity and mortality are among the major health problems. In the United States, ADR is estimated to be the fourth to sixth cause of death. ADR results in death in several thousand patients each year. For example, in some countries, the percentage of patients hospitalized due to ADR is more than 10% (Norway 11.5%, France 13.0%, and England 16.0%).14 In a review study of the incidence of ADR that causes patients to be admitted to hospital and deaths reported less than half the number of incoming studies, the incidence of hospitalization due to ADR ranged from 6 to 14%.15 ADR are a significant health problem in primary health care.8 Identification of ADR needs to be done, given the presence of ADR in patients. Chemotherapy is thought to present a risk of worsening the patient’s condition, which further decreases the quality of life. In previous studies, it is mentioned that a decrease in quality of life is associated with death.16

This study aims to translate, pilot test and validate Hartwig instruments in measuring the severity of ADR in colorectal cancer patients in Indonesia. The authors determined the cancer patients as the subject of this pilot testing because of the assessment of severity of cancer has already had a standard instrument, CTC AE, therefore it can be used as the comparison for this study. We hope that the results of this study can be considered to be used and implemented to strengthen ADR management in Indonesia.

Materials and Methods

This section is presented in three parts: Forward-Backward translation, pilot testing of instrument, and statistical analysis. The ethics of this study has been reviewed and declared ethically feasible by the Research Ethics Committee of the Faculty of Medicine, Padjadjaran University (Number: 760/UN6.KEP/EC/2020) and the Health Research Ethics Committee at Gunung Jati Hospital, Cirebon (Number: 077/LAIKETIK/KEPKRSGJ/X/2020). The guidelines outlined in the Declaration of Helsinki was used. Moreover, every participant received an informed consent.

Step 1: Forward-Backward Translation

The instrument of this study was a questionnaire form by Hartwig, which has been used for assessing the severity of ADR. Forward-backward translation procedure was used to translate the Hartwig instrument, which was referred to several articles17,18 that have been modified by researchers. Translation from the native language (English) to Indonesian was done by two Indonesians who are English experts with the criteria as English Lecturers in a Language Institute of Higher Education. The results were compared, then a discussion was held to resolve the issue of differences in translation results between the two experts, afterwards the conclusion was agreed. The translated translations were translated back into English by two Americans who can speak English and Indonesian. The results were compared to the original instrument in English, then final discussion was held to resolve any differences and conclusion are decided.

Step 2: Pilot Testing of Hartwig Instrument

The Indonesian translation of instrument was pilot tested for assessing the severity of ADR. The Naranjo algorithm instrument19 and the WHO Causality instrument20 were used to assess the probability and causality of ADR. The assessment was done to 10 colorectal cancer patients who have received chemotherapy at RS “X” Cirebon and had complaints about ADR. Their cases were summarized in detail, and then were assessed by 30 health workers from 3 hospitals, including doctors, pharmacists, and nurses. Further, validation tests were conducted based on Content Validity Ratio (CVR) and Content Validity Index (CVI) method. All of the 30 health workers as raters were asked whether the instrument can be used to assess the severity of ADR in 10 such cases or not. The instrument was scored by the raters as 1 if it could assess ADR or 0 if it could not.

Step 3: Statistical Analysis

Data from assessment was used in the analysis of validity and reliability of the instrument using SPSS version 15. The test validity was based on Content Validity Ratio (CVR) and Content Validity Index (CVI) method. In the validity of quantitative content, trust was maintained in selecting the most important and correct content, as measured by CVR.21 Meanwhile, the test reliability was evaluated by Intraclass Correlation Coefficient (ICC).

Results

Translation of the Instrument

The original version (English) of questionnaire form to measure the severity of ADR by Hartwig is shown in Table 1. Meanwhile, the Indonesian questionnaire form by Hartwig is shown in Table 1 .

Table 1

The Original Version of Questionnaire Form (English Version)

Severity Assessment
Was the ADR the reason for admission? Yes No
If not the reason for admission, did the ADR cause an increased length of stay (LOS) by at least one day? Yes No
Did the ADR result in the permanent harm to the patient ? Yes No(Explain any Yes response) ?
ADR Severity Assessment Scale (Circle the severity level of the adverse reaction)
Level 1: An ADR occurred but required no change in treatment with the suspected drug
Level 2: The ADR required that treatment with the suspected drug be held, discontinued, or otherwise changed. No antidote or other treatment required. No increase in LOS
Level 3: The ADR required that treatment with the suspected drug be held, discontinued, or otherwise changed, AND/OR An antidote or other treatment was required. No increase in LOS
Level 4:A. Any level 3 ADR which increasing LOS by at least 1 day ORB. The ADR was the reason for admission
Level 5: Any level 4 ADR which requires intensive medical care
Level 6: The adverse reaction caused permanent harm to the patient
Level 7: The adverse reaction either directly or indirectly led to the death of the patient

Notes: Reprinted from Steven C Hartwig, Jerry Siegel APJS. Preventability and Severity Assessment in Reporting Adverse Drug Reactions. Am J Hosp Pharm. 1992; (June), by permission of Oxford University Press.9

Pilot Testing of Instrument

There were 10 colorectal cancer patients who participated and filled out the instrument, which was then analyzed by 30 health workers as the raters. The results present that there are various severity of ADR found, which is presented in narrative form. There are differences in the results in determining the level of severity as presented in Table 2.

Table 2

The Result of Pilot Testing of Hartwig Instrument

No.	Short Narrative of the Case	The Results of Case Assessment Based on Hartwig Severity
		Level of ADR Severity	Number of Raters
1	Mrs. S* vomited 2–3 times during the 10th chemotherapy. Previously, she has given pre-medical therapy according to the protocol for prevention of ondansetron, dexamethasone, and ranitidine. To overcome the vomiting, the doctor instructed to give her anti-emetic therapy again, namely, ondansetron, accompanied by dexamethasone and ranitidine. Then, the complaint of vomiting improved.	Severity 1Severity 3	426
2	Mr. R* complained of any canker sores after the 1st chemotherapy. There was a time when Mr. R* had to go to the emergency room and was treated for 3 days because he could not or had difficulty eating, drinking and walking.	Severity 3Severity 4	327
3	Mrs. S* suffered vomiting during the 2nd chemotherapy, although she had previously been given pre-medication therapy according to the protocol for prevention of ondansetron, dexamethasone, and ranitidine. The doctor instructed to give anti-emetic therapy again, namely, ondansetron, accompanied by dexamethasone and ranitidine to overcome the vomiting complaint. Vomiting has improved.	Severity 1Severity 3	624
4	Mr. AR*, age 29 years, has no comorbid disease, leukocytes occurred unusual after the 2nd chemotherapy. On the 3rd chemotherapy (date 11-3-2020), the result of the leukocyte lab test was 2460/ul, and neutrophils was 10.8%. The result of leukocyte baseline lab test (date 10-13-2020) within the normal limit of 5200/ul, thus the chemotherapy was postponed and given Leucogen® therapy (Filgrastim) for leukocytopenia improvement.	Severity 3Severity 4	426
5	Mr. R* vomited six times at home after the 1st chemotherapy, although during his previous chemotherapy he had been given pre-medical therapy according to the protocol for prevention of ondansetron, dexamethasone, and ranitidine. The patient was taken to the Emergency Room. Then, the patient’s vomiting complaints improved.	Severity 3Severity 4	426
6	Mrs. NN* had anemia (low Hb) during the 5th chemotherapy (normal Hb baseline data), therefore, chemotherapy schedule was canceled. The patient was hospitalized and given transfusion of Packed Red Cells (PRC) to improve her condition.	Severity 3Severity 4	426
7	Mr. AS* experienced nausea during the 2nd chemotherapy, although he had previously been given pre-medication therapy according to the protocol for prevention of ondansetron, dexamethasone, and ranitidine. No therapy was given to address nausea. The patient’s nausea complaint was getting better.	Severity 1Severity 2	282
8	Mrs. F* experienced hypokalemia (from laboratory examination results) during the 6th chemotherapy (normal baseline potassium data), so that the chemotherapy schedule was canceled. The patient was hospitalized to improve her condition.	Severity 4	30
9	Mr. S* suffered a numb hand and tingling during the 3rd chemotherapy. The patient did not get any pharmacological therapy (medication). The complaints were disappeared within a few days.	Severity 1	30
10	Mr. R* was supposed to do the 7th cycle of chemotherapy, but because of his canker sores, he was not present at the hospital for chemotherapy as scheduled, and eventually the chemotherapy schedule was postponed. His canker sores was painful that he had difficulty eating and drinking after the previous chemotherapy. No medication was needed to treat the canker sores.	Severity 1Severity 2	228

Note: * The initials of the patient’s name withheld.

Validity Test

The validity test was conducted by asking 30 raters whether the Hartwig instrument in Indonesian version could be used to assess the severity if ADR. The raters gave score 1 if it can be used to assess or score 0 if it cannot. Afterwards, analyses were done based on CVR and CVI. The results of the assessment and the value of CVR and CVI are listed in Table 3.

Table 3

The Feasibility Assessment of Instrument in Assessing the Severity of ADR as Well as the Value of CVR and CVI

No.	Rater Rating Results		CVR	CVI
	Score 1*	Score 0**
1	26	4	0,73	0,87
2	29	1	0,93	0,97
3	26	4	0,73	0,87
4	26	4	0,73	0,87
5	30	0	1	1
6	28	2	0,87	0,93
7	29	1	0,93	0,97
8	30	0	1	1
9	29	1	0,93	0,97
10	29	1	0,93	0,97
Overall	30	0	1	1

Notes: *: rater gives a score of 1 if the Hartwig instrument can assess the severity of ADR. **: rater gives a score of 0 if the Hartwig instrument cannot assess the severity of ADR.

Reliability Test

The reliability test was based on ICC method to analyze the data from the assessment of severity of ADR. The test was done by 30 health workers with 10 cases and presented as shown in Table 4.

Table 4

The Result of ICC Analysis

No.	Parameters	Value	Interpretation
1	Cronbach’s alpha	0.996	Satisfying
2	ANOVA (signification)	0.197	Meaningless differences between raters (p>0.05)
3	ICC	0.896	Reliability between raters with a good degree of reliability

Discussion

The results of Indonesian translation of the two translators are different in vocabulary and grammar, yet the difference is acceptable as long as it has the same purpose. The examples of the difference between words are “masuk” (enter) and “diterima” (accepted), “peningkatan” (increase) and “penambahan” (addition), “menyebabkan” (cause) and “mengakibatkan” (result), using the word “adalah” (is) and without using “adalah” (is), “reaksi yang merugikan” (adverse reaction) and “reaksi efek samping obat” (side effect reaction of drug), “mengharuskan agar” (requires that) and “mensyaratkan” (requires), “penangkal” (antidote) and “penawar” (antidote), “perawatan” (treatment) and “pengobatan” (treatment), “membutuhkan” (need) and “memerlukan” (need). These differences have been resolved and agreed by using the same words and sentences.

The final Indonesian translation was translated back to the original language (English) by two English native translators who can communicate and understand Indonesian. The translation results were compared to the original instrument in English. The assessment is carried out based on vocabulary and grammar, where there are similarities and differences in the sentence of the original instrument. The examples of differences are “Has/Did” and “Was”, “cause of hospital admission” and “the reason for admission”, “ADR prolong hospital admission (Length of Stay, LOS) by at least 1 day” and “increased length of stay (LOS) by at least one day”, “ADR caused permanent damage to patients” and “ADR result in the permanent harm to the patient”, “ADR occurred but the course of treatment with the suspected medication was not changed” and “An ADR occurred but needed no change in treatment with the suspected drug”. The translation was confirmed to both translators and stated that the difference in translation results with the original instrument is acceptable due to their same meaning. This is because there are differences in the choice of vocabulary and grammar.

From each case assessed by 30 raters, there are differences in the choice of severity of ADR in 8 cases, exactly at case 1, 2, 3, 4, 5, 6, 7 and 10. While the other 2 cases have the same severity assessment in cases number 8 and 9. The different perception in assessing cases lead to the differences in the determination of severity level of ADR. As in case number 1, four raters who gave a score of 1 considered giving anti-emetic drugs because it is appropriate for premedication procedures. However, the majority, 26 raters, considered that the re-emetic treatment is out of the premedication procedure if vomiting nausea still occurred in patients after chemotherapy. In cases, number 2 and 3, as many as 3 raters provide an assessment in the form of level 3 and the majority of 27 raters assess as level 4. After held a discussion, the raters who chose level 3 consider that needing another treatment for the ADR, whereas the patients should be hospitalized through emergency room. The same case happens at case 4, 5, and 6. In the case number 7, there are two raters gave level 2 as the assession, while majority (28 raters) gave level 1. After held a discussion, the rater who assessed level 2 consider that in the additional information regarding ADR, there is no need an antidote or other treatment. In case number 10, as many as two raters gave level 1, because they consider that canker sore does not need to be treated as long as the chemotherapy schedule is not withheld. Meanwhile, the others as many as 28 raters gave level 2.

In assessing the severity of ADR using the Hartwig instrument, each rater may have a different perception due to a different apprehending of the case. Therefore, it is necessary to write and present a clear, detailed, and complete case, so that the assessment process is more accurate.

In the 10 cases assessed by 30 raters, there is a value of 1 (the instrument can assess the severity of the ADR) and a value of 0 (the instrument cannot assess the severity of ADR). The majority of raters gave a value of 1, furthermore, all raters agreed that the Indonesian version of Hartwig instrument could assess the severity of ADR. CVR and CVI values are calculated to determine the validity of this instrument. The result of CVR value in case number 1 until 10 has a value of 0.733; 0.933; 0.733; 0.733; 1000; 0.867; 0.933; 1000; 0.933; and 0.933, respectively, while the total score is 1000. All of the CVR results have value of > 0.33 (n=30)22 and >0.2523 indicating that this instrument is valid. Meanwhile, the result of CVI in case number 1 until 10 have a value of 0.87; 0.97; 0.87; 0.87; 1; 0.93; 0.97; 1; 0.97; and 0.97 respectively. All cases have a CVI value above 79% (0.79), which means appropriate21,24. Overall, the average value of CVI is 1.00. The result of CVI moves between 0.8–1.0 and ≥ 0.9 indicating that the instrument can be declared valid.24

The reliability test using ICC shows satisfactory alpha reliability value of 0.996. The result of ANOVA analysis shows that there is no difference in assessment between raters with the significance of 0.197 (p>0.05). The ICC value of 0.896 shows a reliability between raters with good reliability degree.25 Based on the ICC selection and report guideline for reliability testing, the basis for evaluating the level of reliability using the general instruction is as shown in Table 5.

Table 5

Reliability Test Decision-Making Basis

ICC Value	Interpretation
< 0.5	Poor Reliability
0.5–0.75	Moderate Reliability
0.75–0.9	Good Reliability
> 0.90	Excellent Reliability

Note: Data from Koo and Li.26

The Hartwig instrument can be translated into Indonesian and used to assess the severity of ADR as one of the effects of chemotherapy in colorectal cancer patients. This instrument can make it easier for health workers to assess the severity of ADR. It is simpler than other instruments such as the Common Terminology Criteria Adverse Event (CTC AE) instrument which assess the severity based on the impact on various organs and their functions in detail.6 While, the Hartwig instrument assess directly clinically impact on patients, in the form of whether there are changes in suspected therapy or not, whether the therapy is with held or discontinued, the need for antidote or antagonist, the addition of long treatment or treatment, requiring hospital treatment, requiring intensive medical care, disability and even that can lead to the patient’s death. In general, those two instruments can be used with their respective advantages. The authors are aware of limitations in this study, the level of severity has not presented in all cases, only severity level of 1, 2, 3, and 4, meanwhile the level of 5, 6, and 7 is none, therefore they have not described in this study.

The results of this study provide an information that the Hartwig instrument in Indonesian version can be used to measure the severity level of ADR of colorectal cancer patients. It will also be useful to assess the severity of ADR in the treatment of other diseases which the time of the treatment is quite long and is likely to cause ADR in various severity, such as patients with HIV AIDS, Pulmonary Tuberculosis and so on. Recognizing the severity of ADR will certainly determine a better ADR management strategy.

Conclusion

Hartwig instrument can be translated into Indonesian and applied in the measurement of the severity of ADR in chemotherapy of colorectal cancer patients. It is necessary to test the measurement of ADR in the treatment of other diseases where the incidence of ADR is quite significant, such as in tuberculosis and HIV-AIDS patients who are on long term treatment. Identifying the severity of ADR is important to know how severe the impact is on patients and to anticipate similar events when recurrence, as ADR can affect the patient’s quality of life. The authors hopes that this instrument can be considered for use in pharmacovigilance management in Indonesia.