Validation of a patient interview for assessing reasons for antipsychotic discontinuation and continuation.

INTRODUCTION: The Reasons for Antipsychotic Discontinuation Interview (RAD-I) was developed to assess patients' perceptions of reasons for discontinuing or continuing an antipsychotic. The current study examined reliability and validity of domain scores representing three factors contributing to these treatment decisions: treatment benefits, adverse events, and distal reasons other than direct effects of the medication.
METHODS: Data were collected from patients with schizophrenia or schizoaffective disorder and their treating clinicians. For approximately 25% of patients, a second rater completed the RAD-I for assessment of inter-rater reliability.
RESULTS: All patients (n = 121; 81 discontinuation, 40 continuation) reported at least one reason for discontinuation or continuation (mean = 2.8 reasons for discontinuation; 3.4 for continuation). Inter-rater reliability was supported (kappas = 0.63-1.0). Validity of the discontinuation domain scores was supported by associations with symptom measures (the Positive and Negative Syndrome Scale for Schizophrenia, the Clinical Global Impression - Schizophrenia Scale; r = 0.30 to 0.51; all P < 0.01), patients' primary reasons for discontinuation, and adverse events. However, the continuation domain scores were not significantly associated with these other indicators. DISCUSSION: Results support the reliability, convergent validity, and known-groups validity of the RAD-I for assessing patients' reasons for antipsychotic discontinuation. Further research is needed to examine validity of the RAD-I continuation section.

Introduction

Time to all-cause discontinuation and rates of all-cause discontinuation have been used as primary outcome measures in frequently cited antipsychotic trials.1,2 As a result, there has been growing interest in approaches for measuring treatment discontinuation.3 The advantage of using all-cause treatment discontinuation as an outcome measure is that it represents multiple factors, including patients’ and clinicians’ judgments of efficacy and tolerability.4–7

However, several limitations of this approach have been noted.3 For example, discontinuation rates do not provide an indication of the specific factors that led to discontinuation. Reasons for antipsychotic discontinuation may include insufficient improvement in positive, negative, depressive, or cognitive symptoms; failure to improve functional status; specific adverse events; financial cost; and a range of psychosocial factors, such as lack of social support.8 When clinical trials do assess reasons for discontinuation, they usually identify only a single general reason for discontinuation, without a measure that can capture the interaction among multiple specific factors leading to discontinuation. In addition, trials do not assess the reasons why patients continue medication. In response to these measurement limitations, the Reasons for Antipsychotic Discontinuation Questionnaire (RAD-Q) was developed to assess clinicians’ perceptions of reasons for antipsychotic discontinuation and continuation.9 A validation study has provided initial support for the reliability and validity of the instrument.10

The purpose of the current study was to examine a patient interview called the Reasons for Antipsychotic Discontinuation Interview (RAD-I), which was developed in parallel with the RAD-Q as a tool for assessing the patient’s perspective of reasons for discontinuation or continuation. Identifying patients’ reasons for discontinuation has been recommended as a key step in the treatment process.8 The patient’s perspective may be clinically important even if it diverges from the clinician’s point of view, and previous research has shown that clinicians and patients with schizophrenia often disagree on factors affecting medication use.11,12 With knowledge of the patient’s reasons for discontinuing an antipsychotic, clinicians may be able to choose a subsequent treatment that is likely to be more acceptable to the patient, which may maximize treatment adherence, delay discontinuation, and thus delay or prevent relapse. For example, if a clinician knew that a specific treatment-emergent adverse event directly led to a patient’s decision to discontinue a previous antipsychotic, the clinician could prescribe a medication with a different adverse event profile for future treatment.

The RAD-I was designed to be administered as a brief semistructured interview. Based on the patient’s responses to open-ended questions, the interviewer completes a series of items assessing the same constructs as the RAD-Q in three content areas: treatment benefits, adverse events, and distal reasons other than direct effects of the medication. In a qualitative cognitive debriefing study, 15 patients and four interviewers indicated that the RAD-I was clear, easy to complete, and comprehensive.9 In the current study, analyses examined individual item responses, as well as reliability and validity of the domain scores in a larger sample. In addition, agreement between clinicians and patients regarding reasons for discontinuation and continuation was assessed by examining concordance between the RAD-Q and RAD-I domain scores.

Methods

Study design and sample selection

The current analysis focuses on the RAD-I. Analyses focusing on the RAD-Q have been presented previously.10 Data were collected in the USA at 17 clinical sites (three Veterans Administration hospitals; seven university hospitals or medical centers; three private health care organizations; two private medical research organizations; and two private psychiatric inpatient/outpatient services). Patients diagnosed with schizophrenia or schizoaffective disorder completed a series of self-completed and interviewer-administered measures in a single study visit. Each patient’s treating clinician completed the RAD-Q and the RAD-I was administered to patients by trained interviewers. Patients were categorized as either discontinuation patients or continuation patients. Discontinuation patients were required to have discontinued at least one antipsychotic within the past 7 days. Continuation patients were required to have been treated for at least 2 months with an antipsychotic that they did not intend to discontinue.

Interviewers who administered the RAD-I must have had at least 6 months’ experience working with patients who have schizophrenia. It was not necessary that the interviewer had previously participated in treating the patient, and interviewers were not required to be licensed to prescribe medication. To be eligible for completion of the RAD-Q in this study, clinicians must have been licensed to prescribe medication, and they must have been directly involved with prescribing the medication being rated on the RAD-Q. Clinicians who completed the RAD-Q were not eligible to be the patient’s interviewer. The study protocol was approved by an independent ethics review committee, and all participants provided informed consent.

Approximately 25% of the sample at each site was randomly selected for analyses of inter-rater reliability. For this subgroup of patients, a second rater observed the interview and independently completed the RAD-I scoring sheets based on these observations. The interviewer and the second rater did not discuss their ratings.

Measures

The RAD-Q

The RAD-Q was designed to assess the clinician’s perceptions of the reasons for discontinuing or continuing antipsychotic medication for the treatment of patients with schizophrenia or schizoaffective disorder. Clinicians complete either the discontinuation section or the continuation section, depending on the patient’s treatment status. Reasons for discontinuation/continuation are conceptualized within three distinct domains: (1) treatment benefits, (2) adverse events, and (3) distal reasons other than direct effects of the medication. Treatment benefits items were designed to assess five areas of potential benefit (positive symptoms, negative symptoms, mood, cognition, functional status). Adverse events items assess two types of adverse events (immediate life-threatening safety issues, adverse effects that are not immediately life-threatening), and respondents are given space to list and rate multiple adverse events within these two categories. Distal reasons for discontinuation or continuation include cost, difficulty negotiating the health care system, inadequate social support, and patient’s lack of insight. Respondents are also given space to list up to three additional reasons for discontinuation or continuation that are not captured in any of the RAD-Q items.

For each item, clinicians first answer a yes/no question to indicate whether the item is a reason for discontinuation or continuation. For items marked “yes,” a follow-up question is asked to determine how much the item contributed to the decision to discontinue or continue. This follow-up question is on a four-point scale ranging from “a minor reason” to “a primary reason.” Items marked “no” receive a score of 0, while items marked “yes” are scored according to the response on the four-point scale. Higher scores on each item represent more important reasons for discontinuation or continuation. Three RAD-Q domain scores (treatment benefits, adverse events, distal reasons) are assigned based on the highest score of any individual item within each domain.

The RAD-I

The RAD-I was developed in parallel with the RAD-Q, with items and domains assessing the same factors described for the RAD-Q above. The RAD-Q and RAD-I were developed and refined based on literature review, patient interviews, and expert panel input.9 The RAD-I is administered by trained interviewers in three steps: (1) open-ended questions about reasons for discontinuation; (2) interviewers record patients’ initial responses and complete yes/no questions for a series of items representing reasons for discontinuation (a “yes” response indicates that the patient believes the item is a reason for discontinuation); and (3) follow-up questions in which interviewers ask the patient to rate the importance of each reason on a three-point Likert scale. Items on the scoring sheets assess the same reasons for discontinuation as the RAD-Q items (RAD-I items are listed in Tables 1 and 2). For patients continuing their current treatment regimen, a parallel interview assesses reasons for continuing an antipsychotic. Interviewers are instructed not to rate items based on their own understanding of treatment decisions because the RAD-I was designed to capture the patient’s perspective, even if the patient’s views are inconsistent with those of the interviewer or treating clinician. At the end of the interview, patients report the primary reason for discontinuation or continuation, and primary reasons are subsequently categorized into one of the three domains.

Table 1

Descriptive statistics: RAD-I discontinuation items

RAD-I discontinuation items	Ratings of all 81 patientsa Mean (SD)	Frequency of “yes” responses to each itemb n (%)	Average scores for items with yes responsesc Mean (SD)
Treatment benefits
Did not improve positive symptoms	0.6 (1.1)	18 (22.2%)	2.5 (0.7)
Made positive symptoms worse	0.3 (0.8)	8 (9.9%)	2.6 (0.7)
Did not improve negative symptoms	0.2 (0.7)	6 (7.4%)	2.7 (0.5)
Made negative symptoms worse	0.1 (0.6)	4 (4.9%)	2.8 (0.5)
Did not improve mood	0.3 (0.8)	9 (11.1%)	2.6 (0.5)
Made mood worse	0.1 (0.6)	4 (4.9%)	2.8 (0.5)
Did not improve cognition	0.2 (0.7)	6 (7.4%)	2.7 (0.5)
Made cognition worse	0.1 (0.6)	4 (5.0%)	2.5 (0.6)
Did not improve functional status	0.2 (0.6)	6 (7.4%)	2.3 (0.5)
Made functional status worse	0.2 (0.8)	8 (9.9%)	2.5 (0.8)
Adverse events
Side effectsd	1.9 (1.4)	53 (65.4%)	2.9 (0.4)
Distal reasons
Financial cost for the patient	0.1 (0.5)	2 (2.5%)	3.0 (0.0)
Financial cost for other payers	0.0 (0.3)	1 (1.2%)	3.0 (−)
Difficulty negotiating the health care system	0.0	(0.0) 0 (0.0%)	–
Problems with transportation	0.0 (0.0)	0 (0.0%)	–
Social support	0.0 (0.4)	2 (2.5%)	2.0 (1.4)
Another person told this patient to stop taking the medication	0.4 (1.0)	12 (14.8%)	2.6 (0.7)
Lack of insight into condition (eg, patient does not believe that he/she has schizophrenia)	0.1 (0.5)	2 (2.5%)	3.0 (0.0)
Patient believed he/she no longer needed the medication because he/she was now “better”	0.0 (0.3)	1 (1.2%)	3.0
Unable to form a therapeutic alliance or make a connection with members of the treatment team	0.0 (0.0)	0 (0.0%)	–
The patient wished to try an antipsychotic new to the market	0.1 (0.3)	3 (3.7%)	1.7 (0.6)
Potential interactions with another medication	0.1 (0.4)	2 (2.5%)	2.5 (0.7)
The patient developed a new medical condition, and this antipsychotic may have exacerbated this condition	0.1 (0.5)	2 (2.5%)	3.0 (0.0)
Other reasonse	0.5 (1.1)	16 (19.8%)	2.8 (0.7)

Notes:

Each item is scored as follows: 0 = not a reason for discontinuation; 1 = minor reason; 2 = somewhat important reason; 3 = very important reason;

this column summarizes responses to this yes/no question: “In the interview, did the patient mention this item [in response to questions regarding reasons for discontinuation]?”;

this column presents the mean scores only for patients who mentioned the item as a reason for discontinuation;

for this item, patients could report up to five adverse events and rate each one individually. In the current table, the response was considered “yes” if the patients reported at least one adverse event marked. In addition, the score of this item represents the greatest level of importance indicated for any side effect listed as part of this item;

for the last three items, interviewers could record up to three “other reasons” mentioned by the patient, but not captured in any of the other items. The importance of each of these “other reasons” was then rated individually. In the current table, the response to these three “other reasons” items is considered yes if the interviewer recorded at least one “other reason.” The score in this table represents the greatest level of importance indicated for any reason listed in these three items.

Abbreviations: RAD-I, Reasons for Antipsychotic Discontinuation Interview; SD, standard deviation.

Table 2

Descriptive statistics: RAD-I continuation items

RAD-I continuation items	Ratings of all 40 patientsa Mean (SD)	Frequency of “yes” responses to each itemb n (%)	Average scores for items with yes responsesc Mean (SD)
Treatment benefits
Benefits for positive symptoms	1.8 (1.4)	26 (65.0%)	2.7 (0.6)
Benefits for negative symptoms	0.4 (1.0)	7 (17.5%)	2.4 (0.5)
Benefits for mood	0.8 (1.3)	12 (30.0%)	2.7 (0.5)
Benefits for cognition	0.7 (1.2)	9 (22.5%)	2.9 (0.3)
Benefits for functional status	1.4 (1.3)	22 (55.0%)	2.5 (0.7)
Adverse events
No noticeable side effects	0.3 (0.9)	4 (10.0%)	3.0 (0.0)
Absence of one or more specific side effects that are important for this patient	0.3 (0.8)	6 (15.0%)	2.2 (0.4)
Mild, tolerable side effects	0.4 (0.8)	8 (20.0%)	1.9 (0.6)
Distal reasons
Financial cost for the patient	0.2 (0.7)	2 (5.0%)	3.0 (0.0)
Financial cost for other payers	0.0 (0.0)	0 (0.0%)	–
The patient is willing/able to negotiate the health care system to obtain this medication	0.1 (0.3)	1 (2.5%)	2.0
There are no problems with transportation (eg, getting to the pharmacy to refill medication)	0.0 (0.0)	0 (0.0%)	–
Social support	0.2 (0.7)	3 (7.5%)	2.3 (1.2)
Another person told this patient to continue taking the medication	0.5 (1.1)	8 (20.0%)	2.5 (0.9)
The patient has formed a therapeutic alliance or made a connection with members of the treatment team	0.4 (1.0)	7 (17.5%)	2.4 (0.8)
The patient has already tried other antipsychotics that have not been as effective and/or tolerable	0.6 (1.1)	8 (20.0%)	2.8 (0.5)
Other reasonsd	0.5 (1.0)	8 (20.0%)	2.4 (0.7)

Notes:

Each item is scored as follows: 0 = not a reason for continuation; 1 = reason of minor importance; 2 = somewhat important reason; 3 = very important reason; 4 = a primary reason;

this column summarizes responses to this yes/no question: “In the interview, did the patient mention this item [in response to questions regarding reasons for continuation]?”;

this column presents the mean scores only for patients who mentioned the item as a reason for continuation;

for the final three items, clinicians were provided with space to report up to three “other reasons” and rate each one individually. In this table, the response to these three “other reasons items” is considered yes if the clinician reported at least one reason marked with a “yes” response. The score of these items in this table represents the greatest level of importance indicated for any reason listed as part of these items.

Abbreviations: RAD-I, Reasons for Antipsychotic Discontinuation Interview; SD, standard deviation.

Items marked as “no” (ie, not mentioned as a reason) by the interviewer are scored as 0. For items marked “yes,” the score is based on the response to the follow-up question, with a three-point response scale ranging from 1 (“a minor reason”) to 3 (“a very important reason”). Higher scores on each item represent more important reasons for discontinuation or continuation. The RAD-I follows the same scoring approach as the RAD-Q, which has been discussed previously10 – three domain scores (treatment benefits, adverse events, distal reasons) are assigned based on the highest score of any individual item within each domain.

The Positive and Negative Syndrome Scale (PANSS) for Schizophrenia

The PANSS was developed to evaluate the severity of psychopathology of adults with schizophrenia and related psychotic disorders.13,14 Higher scores indicate greater symptom severity.

The Clinical Global Impression – Schizophrenia Scale (CGI-SCH)

The CGI-SCH scale was developed to evaluate symptoms (positive, negative, depressive, cognitive, and global) in patients with schizophrenia.15 Each of the five items is answered on a 7-point scale, with higher scores indicating greater symptom severity. This measure was completed by clinicians after completing the RAD-Q, as well as by interviewers immediately following the patient’s study visit. When completing the CGI-SCH, the clinicians did not know the patients’ responses to the RAD-I or PANSS.

Adverse events checklist

Patients completed an adverse events checklist which provided a list of possible side effects and asked whether they had experienced any of these as a result of the medication rated on the RAD-Q. For all side effects answered with “yes,” patients were asked to rate how bothersome the side effect was on a 3-point scale ranging from “not at all” to “very much.”

Statistical analysis

Reliability of the RAD-I

Inter-rater reliability of the RAD-I domain scores was assessed in the subgroup of patients for whom the scoring sheets were completed independently by the interviewer and an observer. Three estimates of agreement were computed: percent agreement, kappa, and weighted kappa.16,17 When interpreting kappa and weighted kappa, the strength of agreement between raters was considered moderate for values ranging from 0.41 to 0.60, good for values of 0.61 to 0.80, and very good for values of 0.81 to 1.0.18,20

Validity of the RAD-I

Two types of construct validity were examined: known-groups validity and convergent validity. Known-groups validity (ie, the extent to which an instrument distinguishes among groups of participants known to differ on a relevant dimension) of the three RAD-I domain scores was assessed by categorizing patients into three groups based on their reported primary reasons for continuation or discontinuation: (1) treatment benefits, (2) adverse events, and (3) distal reasons. Then, each of the RAD-I domain scores were compared among these three groups of patients using an analysis of variance (ANOVA) with Scheffé’s post hoc pairwise comparisons. It was expected that RAD-I domain scores would differ among these three groups. For example, patients reporting adverse events as the primary reason for discontinuation were expected to have a greater adverse events domain score than the other two groups.

Known-groups validity was also examined by comparing RAD-I domain scores among subgroups of patients categorized based on clinician-rated CGI-SCH overall symptom scores. Known-groups validity of the RAD-I discontinuation adverse events domain score was further examined by categorizing patients into two groups based on their responses to the adverse event checklist: (1) patients who were bothered “very much” by at least one adverse event and (2) patients who were not bothered “very much” by any adverse event. Then, the adverse events domain scores of these two groups were compared with a t-test.

Convergent validity of the RAD-I benefits domain score was assessed by conducting Spearman correlations with the PANSS, the clinician-completed CGI-SCH, and the interviewer-completed CGI-SCH. It was expected that correlations between the benefits domain of the RAD-Q and these other scales would be positive for discontinuation patients and negative for continuation patients. Correlation coefficients were interpreted based on Cohen’s guidelines19 suggesting that a coefficient of 0.10 to 0.29 is small, 0.30 to 0.49 is moderate, and greater than 0.50 is large.

Patient–clinician concordance

To assess the degree to which patients and clinicians agree on reasons for discontinuation and continuation, Spearman correlations examined concordance between the RAD-Q and RAD-I. In addition, clinicians’ and patients’ primary reasons for discontinuation or continuation were categorized into one of three areas (benefits, adverse events, distal reasons), and the percentage of agreement between patients and clinicians was computed.

Results

Patient and clinician characteristics

Characteristics of the 121 patients (81 discontinuation, 40 continuation) and their treating clinicians have been reported previously in the study focusing on validation of the RAD-Q.10 The patient characteristics are summarized in Table 3. All patients had a diagnosis in their medical chart beginning with the DSM-IV code 295, representing schizophrenia (58.7%) or schizoaffective disorder (41.3%). Mean PANSS scores for discontinuation and continuation patients were roughly in the “moderately ill” range, based on interpretation guidelines proposed by Leucht et al.21

Table 3

Patient demographic and clinical characteristics

Demographic and clinical characteristics	Total sample (n = 121)	Discontinuation patients (n = 81)	Continuation patients (n = 40)
Age (mean, SD)	41.6 (11.3)	39.8 (10.9)	45.4 (11.4)
Gender (n, %)
Male	81 (66.9%)	55 (67.9%)	26 (65.0%)
Female	40 (33.1%)	26 (32.1%)	14 (35.0%)
Race (n, %)
White	84 (69.4%)	57 (70.4%)	27 (67.5%)
Black	14 (11.6%)	8 (9.9%)	6 (15.0%)
Asian	5 (4.1%)	3 (3.7%)	2 (5.0%)
Hispanic	14 (11.6%)	11 (13.6%)	3 (7.5%)
Othera	4 (3.3%)	2 (2.5%)	2 (5.0%)
Living situation
Group home	17 (14.0%)	12 (14.8%)	5 (12.5%)
Alone	47 (38.8%)	29 (35.8%)	18 (45.0%)
Spouse	15 (12.4%)	11 (13.6%)	4 (10.0%)
Family or friends	42 (34.7%)	29 (35.8%)	13 (32.5%)
Employment status (n, %)
Employed full-time	8 (6.6%)	4 (4.9%)	4 (10.0%)
Employed part-time	25 (20.7%)	14 (17.3%)	11 (27.5%)
Skilled training	4 (3.3%)	4 (4.9%)	–
Unemployed	15 (12.4%)	9 (11.1%)	6 (15.0%)
Retired	4 (3.3%)	3 (3.7%)	1 (2.5%)
Homemaker	2 (1.7%)	1 (1.2%)	1 (2.5%)
Student	1 (0.8%)	–	1 (2.5%)
Disabled	61 (50.4%)	45 (55.6%)	16 (40.0%)
Other	1 (0.8%)	1 (1.2%)	–
Clinical diagnosis from chart (n, %)
Schizophrenia	71 (58.7%)	43 (53.1%)	28 (70.0%)
Schizoaffective disorder	50 (41.3%)	38 (46.9%)	12 (30.0%)
Duration of schizophrenia or schizoaffective disorder	17.7 (10.5)	16.7 (10.6)	19.8 (10.1)
Psychiatric comorbidities (n, %)
Bipolar disorder	13 (10.7%)	9 (11.1%)	4 (10.0%)
Depression	10 (8.3%)	5 (6.2%)	5 (12.5%)
Anxiety disorder	12 (9.9%)	8 (9.9%)	4 (10.0%)
Personality disorder	7 (5.8%)	4 (4.9%)	3 (7.5%)
Substance abuse	5 (4.1%)	3 (3.7%)	2 (5.0%)
PANSS total score (mean, SD)	74.9 (20.9)	78.4 (20.8)	67.8 (19.6)
CGI-SCH overall severity (clinician rated)	3.7 (1.1)	3.9 (1.1)	3.2 (0.9)
CGI-SCH overall severity (interviewer rated)	3.6 (1.0)	3.8 (1.0)	3.3 (0.8)

Notes:

For continuation, “other” includes two patients who marked both American Indian/Alaska Native and White. For discontinuation, “other” includes one patient who marked both American Indian/Alaska Native and White, and another patient who marked American Indian/Alaska Native, Asian, and White.

Abbreviations: PANSS, The Positive and Negative Syndrome Scale for Schizophrenia; SD, standard deviation; CGI-SCH, The Clinical Global Impression – Schizophrenia Scale.

Among the 81 discontinuation patients, the discontinued medications were quetiapine (n = 18), ziprasidone (n = 14), risperidone (n = 12), aripiprazole (n = 10), olanzapine (n = 7), haloperidol (n = 6), paliperidone (n = 5), clozapine (n = 4), fluphenazine (n = 2), prochlorperazine (n = 1), thioridazine (n = 1), and thiothixene (n = 1). Among the 40 continuation patients, the continued medications were clozapine (n = 8), quetiapine (n = 8), olanzapine (n = 7), aripiprazole (n = 5), risperidone (n = 4), ziprasidone (n = 3), fluphenazine (n = 2), haloperidol (n = 2), and paliperidone (n = 1).

The 21 interviewers administering the RAD-I had a diverse range of educational degrees, including bachelor’s degree (n = 6); master’s degree (n = 4); registered nurse, licensed practical nurse, or other nursing degree (n = 3); doctor of medicine (n = 2); master’s and nursing degree (n = 2); bachelor of medicine, bachelor of surgery (n = 1); doctor of philosophy (n = 1); doctor of pharmacy (n = 1); and a high school diploma (n = 1). The interviewers had worked with patients with schizophrenia for a mean of 8.1 years and saw a mean of 7.7 patients with schizophrenia each week.

A total of 40 clinicians completed the RAD-Q. Some of these clinicians completed the instrument for multiple patients in the study. Of the 40 clinicians, 34 (85.0%) had a doctor of medicine degree, three (7.5%) had a doctor of osteopathic medicine degree, and three had a physician assistant degree. The clinicians had worked with patients with schizophrenia for a mean of 15.0 years and saw a mean of 17.2 patients with schizophrenia each week.

RAD-I discontinuation section: individual items

Table 1 presents mean scores for the individual items of the three RAD-I discontinuation domains: treatment benefits, adverse events, and distal reasons. Each patient reported at least one reason for discontinuation, with a mean of 2.8 (standard deviation [SD] = 1.8; range = 1 to 9) reasons across the 81 patients. Adverse events were the most frequently cited reasons for discontinuation (mentioned by 65.4% of patients). The most frequently mentioned reasons relating to treatment benefits were insufficient improvement in positive symptoms (22.2%) and mood (11.1%). Although none of the treatment benefit items were mentioned as frequently as adverse events, patients who did mention insufficient benefits generally considered them to be important reasons for discontinuation. For example, among the small number of patients saying that the medication caused worsening of positive symptoms, negative symptoms, or mood, the mean item scores ranged from 2.6 to 2.8 on a scale of 0 to 3 with higher scores representing reasons of greater importance. Most distal reasons were rarely cited, although several distal reasons such as financial cost were considered to be very important by some patients. Twelve patients (14.8%) said they stopped taking the medication because another person told them to do so.

RAD-I continuation section: individual items

Table 2 presents mean scores for the RAD-I continuation items. Every patient was able to state at least one reason for continuing the antipsychotic. On average, patients reported a mean of 3.4 (SD = 1.8; range = 1 to 9) reasons for continuing their current antipsychotic medication. The most commonly cited reasons for continuation were benefits for positive symptoms (65.0%), benefits for functional status (55.0%), and benefits for mood (30.0%). Adverse events were cited less frequently as reasons for continuation. Eight patients (20.0%) stated that mild, tolerable side effects were a reason for continuation, while four patients (10%) stated that they were continuing in part because the treatment had no noticeable side effects. Commonly cited distal reasons for continuing treatment included: another person telling the patient to continue (20%); the patient had already tried other antipsychotics that were not as effective and/or tolerable (20%); and a therapeutic alliance or connection with members of the treatment team (17.5%). No patients mentioned problems with transportation or financial costs for payers other than themselves as reasons for continuation.

RAD-I domain scores

In the first column of Table 4, the RAD-I domain scores are presented for the total sample of 81 discontinuation patients and 40 continuation patients. Mean scores suggest that, from the patient’s perspective, adverse events were the strongest reason for discontinuation, with a mean score of 1.9 on a scale with a possible range of 0 to 3. However, insufficient treatment benefits and distal reasons also clearly played a role in any decision to discontinue, receiving mean scores of 1.3 and 1.2, respectively. Treatment benefits were the strongest reason for continuation, with a mean score of 2.7, followed by distal events (1.5), and adverse events (0.9).

Table 4

RAD-I domain scores for the total sample and for subgroups of patients categorized based on their primary reason for discontinuation or continuation

RAD-I domain scores	Total sample Mean (SD)	Three groups of patients categorized based on primary reason for discontinuation/continuation			ANOVA models comparing the three groupsa
		Benefits Mean (SD)	Adverse events Mean (SD)	Distal reason Mean (SD)	Overall F valueb	Significant pairwise comparisonsc
Discontinuation	(n = 81)	(n = 19)	(n = 44)	(n = 12)
Benefits	1.3 (1.4)	2.6 (0.8)	1.1 (1.3)	0.3 (0.6)	19.8	A***, B***
Adverse events	1.9 (1.4)	1.2 (1.3)	2.7 (0.9)	0.2 (0.6)	38.9	A***, B*, C***
Distal reasons	1.2 (1.4)	1.1 (1.4)	0.8 (1.3)	2.7 (0.7)	10.6	B**, C***
Continuation	(n = 40)	(n = 33)	(n = 3)	(n = 3)
Benefits	2.7 (0.8)	2.8 (0.7)	2.7 (0.6)	2.0 (1.0)
Adverse events	0.9 (1.2)	0.9 (1.1)	2.0 (1.7)	0.7 (1.2)
Distal reasons	1.5 (1.4)	1.3 (1.5)	2.0 (1.0)	2.0 (1.0)

Notes:

ANOVA with Scheffé’s post hoc comparisons. Because so few patients cited adverse events or distal reasons as the primary reason for continuation, cell sizes were not large enough to conduct ANOVAs for the continuation patients;

all overall F values were statistically significant (P < 0.001);

statistically significant pairwise comparisons: A = benefits vs adverse events; B = benefits vs distal reasons; C = adverse events vs distal reasons.

P < 0.05;

P < 0.01;

P < 0.001.

Abbreviations: RAD-I, Reasons for Antipsychotic Discontinuation Interview; ANOVA, analysis of variance; SD, standard deviation.

Inter-rater reliability

A total of 29 patients were randomly selected for analysis of inter-rater reliability (Table 5). For one of these patients, the second RAD-I was not completed because a second rater was not available at the time of the interview. Therefore, inter-rater reliability analyses of the RAD-I were conducted with a sample of 28 patients (17 discontinuation and 11 continuation).

Table 5

Inter-rater reliability of the RAD-I: percent agreement, kappa, and weighted kappa

RAD-I domain scores	Total n	n (%)Agreement	Kappa	Weighted kappa
Discontinuation
Benefits	17	17 (100.0)	1.00	1.00
Adverse events	17	17 (100.0)	1.00	1.00
Distal	17	13 (76.5)	0.64	0.58
Continuation
Benefits	11	10 (90.9)	0.63	0.84
Adverse events	11	10 (90.9)	0.85	0.77
Distal	11	10 (90.9)	0.85	0.87

Abbreviation: RAD-I, Reasons for Antipsychotic Discontinuation Interview.

The 17 pairs of raters for discontinuation patients had matching scores for the benefits and adverse events domains, yielding kappas of 1.0. For the distal domain score, 13 of the 17 pairs of raters (76.5%) had the same score, with a kappa of 0.64. For all three continuation domain scores, percent agreement for the 11 pairs of raters was 90.9%, yielding kappas in the good to very good range (0.63 to 0.85).

Convergent validity

The RAD-I discontinuation benefits score demonstrated convergent validity through correlations with measures of schizophrenia symptoms (Table 6). The consistently positive correlation coefficients indicate that greater symptom severity was associated with greater likelihood of patients citing treatment benefit as an important reason for discontinuation. The correlation with the PANSS total score was in the large range (r = 0.51; P < 0.001). All other correlations were in the moderate range and statistically significant (P < 0.01), including correlations with the PANSS positive and negative symptom scales (r = 0.33 and 0.40), clinician-rated CGI-SCH (0.35 to 0.41), and interviewer-rated CGI-SCH (0.30 to 0.46).

Table 6

Convergent validity of the RAD-I benefits domain: Spearman correlations with the PANSS and CGI-SCH

Measures of symptom severity	RAD-I benefits domain score
	Discontinuation (n = 81)	Continuation (n = 40)
PANSS scales
Positive symptoms	0.33**	0.06
Negative symptoms	0.40***	0.08
Total score	0.51***	0.09
Clinician-rated CGI-SCH
Positive symptoms	0.35**	0.12
Negative symptoms	0.41***	−0.02
Total score	0.40***	−0.08
Interviewer-rated CGI-SCH
Positive symptoms	0.30**	0.06
Negative symptoms	0.41***	0.05
Total score	0.46***	0.07

Notes:

P < 0.01;

P < 0.001

Abbreviations: RAD-I, Reasons for Antipsychotic Discontinuation Interview; PANSS, The Positive and Negative Syndrome Scale for Schizophrenia; CGI-SCH, The Clinical Global Impression – Schizophrenia Scale.

In contrast, the RAD-I continuation benefits domain generally appeared to have little association with symptom measures. Correlation coefficients had absolute values ranging from 0.02 to 0.12.

Known-groups validity

The three RAD-I discontinuation domain scores demonstrated known-groups validity by distinguishing among patients categorized based on primary reasons (Table 4). Insufficient treatment benefit was cited as the primary reason by 19 patients, adverse events by 44 patients, and a distal reason by 12 patients. There were statistically significant differences among these three groups in all three RAD-I domain scores. Patients with insufficient treatment benefit as the primary reason had a significantly higher treatment benefits domain score (mean = 2.6) than the patients with adverse events or distal reasons as the primary reason (means = 1.1 and 0.3; P < 0.001). The adverse events and distal reasons domain scores followed the same pattern.

Results for the three continuation domain scores were also generally in the expected direction (Table 4). For example, patients with adverse events as the primary reason had a greater adverse events domain score (mean = 2.0) than either of the other two groups (means = 0.9 and 0.7). However, differences between groups were difficult to interpret because of the small groups of patients who specified adverse events and distal reasons as primary reasons for continuation.

Patients were also categorized based on the clinician-rated CGI-SCH overall symptom score: normal/minimally ill/mildly ill, moderately ill, and markedly/severely ill (Table 7). The RAD-I discontinuation benefits domain score demonstrated known-groups validity, as scores increased with greater CGI-SCH symptom severity. Adverse events appear to contribute similarly to the decision to discontinue among the two groups of lesser severity. However, among the markedly/severely ill group, adverse events were a significantly less important factor in the decision to discontinue. Distal reasons appeared unrelated to symptom severity, with no notable differences among the three symptom severity groups.

Table 7

Known-groups validity of the RAD-I discontinuation domain scores: ANOVA by Clinician-Rated CGI-SCH overall score

RAD-I discontinuation domain scores	Three groups of patients categorized based on clinician-rated CGI-SCH overall score			ANOVA models comparing the three groups1
	Normal or minimally/mildly ill (n = 25)Mean (SD)	Moderately ill (n = 33)Mean (SD)	Markedly or severely ill (n = 23)Mean (SD)	Overall F value	CGI-SCH rating P value	Significant pairwise comparisons2
Benefits	0.4 (1.0)	1.6 (1.4)	1.8 (1.2)	8.83***	0.0003	A**, B**
Adverse events	2.3 (1.3)	2.2 (1.2)	0.9 (1.3)	9.03***	0.0003	B**,C**
Distal	1.2 (1.5)	1.1 (1.3)	1.4 (1.4)	0.48	0.62

Notes:

ANOVA with Scheffé’s post hoc comparisons;

statistically significant pairwise comparisons: A = Normal or minimally/mildly ill vs moderately ill; B = Normal or minimally/mildly ill vs markedly or severely ill; C = Moderately ill vs markedly or severely ill.

P < 0.01;

P < 0.001.

Abbreviations: RAD-I, Reasons for Antipsychotic Discontinuation Interview; ANOVA, analysis of variance; CGI-SCH, The Clinical Global Impression – Schizophrenia Scale; SD, standard deviation.

Two groups of patients were compared in a t-test examining known-groups validity of the RAD-I discontinuation adverse events domain score. These two groups differed in the degree to which they were bothered by adverse events, as indicated on the adverse event checklist. Discontinuation patients who were bothered “very much” by at least one adverse event (n = 60) had a mean RAD-I adverse events domain score of 2.2 (SD = 1.3). Patients who were not bothered “very much” by any adverse event (n = 17) had a mean score of 1.2 (SD = 1.4). The difference between these mean scores was statistically significant (t-value = 2.8; P < 0.01). These two groups were not significantly different in their RAD-I benefits or distal reasons domain scores.

Patient–clinician concordance

Spearman correlations examined the concordance between patients’ and clinicians’ perceptions of reasons for discontinuation or continuation, as represented by the three domain scores of the RAD-I and RAD-Q. Correlations between the patient and clinician discontinuation scores were statistically significant and in the moderate to large range, with correlation coefficients of 0.37 (P < 0.001) for benefits, 0.58 (P < 0.001) for adverse events, and 0.32 (P < 0.01) for distal reasons.

Correlations of continuation scores were substantially lower, and none were statistically significant. The coefficients of 0.27 for benefits and 0.28 for distal reasons were in the small range. However, the correlation coefficient of −0.08 for the adverse events domain suggests no meaningful concordance between patient and clinician ratings of the importance of adverse events in the decision to continue treatment.

Discussion

The RAD-I is the first measure designed to assess patients’ understanding of reasons for antipsychotic discontinuation, and current findings provide initial support for the measure’s reliability and validity. In the subgroup of 17 discontinuation patients with two available raters, there was good inter-rater reliability. Results also support the validity of the RAD-I discontinuation domain scores, as indicated by associations with patients’ primary reasons for discontinuation, clinician-rated symptoms, interviewer-rated symptoms, and patient-rated adverse events. These findings suggest that the RAD-I may be added to clinician-reported measures of discontinuation in order to assess the patient’s perspective. The individual items of the RAD-I provide a detailed description of patients’ views, while the domain scores may be used to summarize and quantify reasons for discontinuation reported by groups of patients with schizophrenia.

The current study suggests that, despite their serious psychiatric condition, patients were generally able to provide reliable and valid data regarding reasons for discontinuation. The RAD-I included an item which interviewers could use to report that the patient did not know any reasons for discontinuation. However, this item was never used because all patients were able to state at least one reason in response to the open-ended questions of the RAD-I. In addition, interviewers were trained to report when a patient’s response appeared to be influenced by delusional thinking, and no interviewers reported this problem. Furthermore, the association of the RAD-I benefits domain score with the clinician-rated CGI-SCH suggests that patients have a reasonable understanding of how their psychotic symptoms contribute to treatment discontinuation. Given that the RAD-I appeared to yield valid data in this initial study, assessment of patient-reported reasons for discontinuation can be recommended for future studies.

In clinical trials of antipsychotics, the RAD-I can be used to clarify reasons for discontinuation. Previous studies have often reported the reason for discontinuation as “patient’s decision” without further clarification.2 In these situations, it is clear that the patient directly contributed to the decision to discontinue, but the assessment tools could not provide a specific explanation of this treatment decision. The RAD-I may be used to identify these specific reasons, thus providing a more precise understanding of patients’ reasons for discontinuation than has previously been available. This information could eventually help clinicians improve adherence and effectiveness by choosing treatments that are best suited for individual patients.

Although the discontinuation section of the RAD-I performed well in all analyses, the validity of the continuation section was not consistently supported. The relationships of the RAD-I continuation domain scores with primary reasons for continuation were generally in the expected direction (Table 4), but results for the adverse events and distal domains are not fully interpretable due to the small number of patients citing adverse events (n = 3) or distal reasons (n = 3) as primary reasons for continuation. A more serious concern is that the benefits domain was not associated with measures of symptom severity (Table 6). There are several possible explanations for the failure to demonstrate these expected relationships. It is possible that patients’ reasons for continuation are genuinely independent of observers’ and clinicians’ ratings of symptom severity. If so, then different criterion measures would be necessary for validating the continuation section of the RAD-I. Alternatively, it is possible that some patients do not have an accurate understanding of why treatment is being continued. While the discontinuation section of the RAD-I can be recommended for use in clinical trials, more research is needed to examine the psychometric properties of the continuation section. Although it may be important to understand why patients continue antipsychotic medication, currently available data do not yet support the use of the RAD-I for this purpose.

Data from the RAD-I and RAD-Q allow for comparisons between patient and clinician perceptions. The magnitude of correlations between patient and clinicians (r = 0.32 to 0.58) suggests that the two perspectives are related, but distinct. One clear difference was apparent. On the RAD-Q, clinicians most frequently cited insufficient treatment benefits as reasons for discontinuation,9 which is consistent with previous studies.22–25 However, RAD-I results suggest that patients may have different priorities, as side effects were cited much more frequently than insufficient treatment benefits as a reason for discontinuation (Table 1). Therefore, clinician-based assessment of reasons for discontinuation may not provide a complete picture of these complicated treatment decisions.

One similarity between patient and clinicians is that both tend to report multiple reasons for discontinuation. Clinicians provided a mean of 4.1 reasons on the RAD-Q,10 while patients provided a mean of 2.8 reasons on the RAD-I. Previous studies assessing antipsychotic discontinuation have generally categorized patients based on a single reason for discontinuation.1,2 However, current results suggest that these studies may not be capturing the multidimensional nature of treatment discontinuation.

A strength of the current sample is that patients were recruited from a diverse range of geographical and clinical settings across the USA, and all patients were treated in usual care, rather than a controlled clinical trial. Still, these results should be considered only the first step in RAD-I validation because the sample size was relatively small, particularly for assessment of inter-rater reliability and assessment of continuation patients. Another limitation is that the RAD-I was required to be completed within 7 days of discontinuation. Some patients completed the instrument on the day of discontinuation, while others were interviewed later within the prespecified time window. It is not known whether patients’ perceptions changed during the days following discontinuation.

Two limitations of the RAD-I itself should also be acknowledged. First, the symptoms of schizophrenia may interfere with patients’ ability to accurately report reasons for treatment decisions. In future studies, interviewers can specify when patients’ perceptions appear delusional, and researchers may decide whether or not to include data from these patients in the final analysis. Regardless of the accuracy, however, the patient’s perspective is important to the extent that it may influence treatment adherence and persistence. Another limitation of the RAD-I is that it does require interviewer training. All interviewers in the current study participated in a 90- to 120-minute training session. Interviewers who had experience of administering symptom measures such as the PANSS were accustomed to using their own judgment when recording responses. In contrast, the goal of the RAD-I is to capture the patients’ own perceptions, regardless of whether the interviewer believes these perceptions are correct. Therefore, these training sessions were necessary in order to ensure that interviewers understood this novel aspect of the RAD-I.

The current study provides encouraging initial support for using the RAD-I to understand patients’ reasons for discontinuing an antipsychotic. Although time to all-cause discontinuation is increasingly used as a global indication of effectiveness, no studies have reported reasons for discontinuation based on a standardized assessment of the patient’s perspective.1,2 The RAD-I addresses this gap as a tool for systematically describing and quantifying the patient’s perspective of the multiple factors simultaneously leading to discontinuation. If the RAD-I is implemented in clinical trials and naturalistic studies, the resulting data may lead to a greater understanding of patients’ experience with antipsychotic medications.