Validation of a treatment satisfaction questionnaire in non-Hodgkin lymphoma: assessing the change from intravenous to subcutaneous administration of rituximab.

BACKGROUND: A subcutaneous (SC) formulation of rituximab (MabThera®/Rituxan®) has been developed that could reduce administration time and improve patient satisfaction with treatment. The Rituximab Administration Satisfaction Questionnaire (RASQ) was created to assess patients' perceptions and satisfaction with rituximab SC (RASQ-SC) or rituximab intravenous (RASQ-IV). We assessed the content validity and psychometric properties of RASQ in patients with non-Hodgkin lymphoma.
METHODS: Face and content validity of RASQ-SC and RASQ-IV were qualitatively assessed using 60-minute combined concept elicitation and cognitive debriefing interviews. Psychometric validation of RASQ (item performance and reliability) was assessed quantitatively against the established Cancer Therapy Satisfaction Questionnaire (CTSQ), using questionnaire data from the PrefMab (NCT01724021) and MabCute (NCT01461928) clinical studies.
RESULTS: RASQ-IV demonstrated excellent coverage of concepts relevant to patients' (n=10) own treatment experiences and no new concepts were identified. Patients' expectations of rituximab SC were conceptually consistent with items included in the RASQ-SC, suggesting that the tool is also conceptually adequate. In 1,051 patients from PrefMab and MabCute, correlations with domains such as "RASQ: Physical Impacts" and "CTSQ: Feelings About Side Effects", "RASQ: Physical Impacts" and "CTSQ: Satisfaction With Therapy", and "RASQ: Satisfaction" and "CTSQ: Satisfaction With Therapy", achieved moderate-to-high correlations (>0.4) for convergent domains and <0.3 for divergent domains.
CONCLUSION: This study supports the qualitative face and content validity and psychometric validity of RASQ-IV and RASQ-SC. Minor revisions were made to the questionnaires to enhance clarity and aid consistent reporting.

Introduction

Non-Hodgkin lymphoma (NHL), a heterogeneous group of lymphoproliferative malignancies, is one of the leading causes of cancer death in the USA and Europe. NHL accounted for 3.2% of new cancer cases and 2.8% of cancer deaths in Europe in 2006, and almost 5% of all cancers in the USA, with an estimated 70,130 new diagnoses (38,160 males and 31,970 females) in the year 2012. NHL can be classified into aggressive (ie, fast growing) or indolent (ie, slow growing) types, with diffuse large B-cell lymphoma (DLBCL) and follicular lymphoma (FL) being the most common subtypes (25%−30% and 20%−25% of all NHL cases, respectively).1,2

Rituximab (MabThera®/Rituxan®) is a chimeric anti-CD20 monoclonal antibody that is the current standard-of-care for patients with previously untreated or recurrent B-cell malignancies, including FL3 and DLBCL.4 Rituximab is typically administered intravenously (IV) over a period of 90 minutes to 6 hours, and represents the longest part of a treatment cycle. This length of administration may be inconvenient for patients, have a psychosocial impact, and could increase health care resource utilization associated with the longer clinic time for induction.5–7

A subcutaneous (SC) formulation of rituximab has been developed. This formulation reduces administration time to ~5 minutes, thereby increasing the convenience of treatment for patients living with B-cell malignancies, and reducing health care resource utilization.8,9 SC administration can benefit health care providers and patients by reducing drug administration time and associated costs.10,11 The SC formulation of rituximab incorporates recombinant hyaluronidase (rHuPH20), which breaks down hyaluronic acid reversibly in the interstitial tissue, thereby increasing dispersion of the injected fluid.12–14 The advantages of rHuPH20 formulations have been demonstrated for agents (morphine and ceftriaxone) in other settings,14,15 as have the general benefits of switching from IV to SC administration.11,16,17 However, at the time of writing this paper, a literature search had revealed that no existing single instrument comprehensively assessed, from the patient’s perspective, the relative benefits of the two administration routes for rituximab (IV vs SC). Therefore, there was an unmet need to develop a new, patient self-reported measure to assess the burden, impact, and preference for different routes of administration in the context of IV and SC administration of rituximab.

Objectives

There were two objectives of this research. The first was to conduct qualitative interviews to confirm the face and content validity of the newly developed Rituximab Administration Satisfaction Questionnaire-intravenous (RASQ-IV) and subcutaneous (RASQ-SC; henceforth RASQ measures). The second objective was to evaluate the psychometric properties to explore whether the RASQ measures were acceptable multidimensional instruments for the measurement of treatment satisfaction and impact in patients with NHL.

Methods

The first step was to utilize findings from a literature review to develop a conceptual framework of patients’ IV/SC treatment experiences. This conceptual framework included two concepts, namely “treatment satisfaction” (covering subconcepts of “overall preference/satisfaction”, “convenience”, “confidence”, and “bothersomeness”) and “impact of treatment administration” (covering subconcepts of “physical impact”, “psychological impact”, “impact on activities of daily life”, and “impact on the interaction with health care providers”) (Figures 1 and 2). Guided by this framework, IV-specific and SC-specific questionnaires were drafted containing items pertaining to these concepts and subconcepts. These are referred to as the RASQ-IV and RASQ-SC. Items were selected based on a review of the literature and existing patient-reported instruments (eg, Functional Assessment of Cancer Therapy,18 the Cancer Therapy Satisfaction Questionnaire (CTSQ),19,20 and the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events).21 The resulting draft questionnaires included 19 items each and used a variety of response scales, including a five-point Likert scale, categorical scales, and a dichotomous scale.

Figure 1

Core concept is rituximab IV/SC treatment satisfaction.

Abbreviations: CTSQ, Cancer Therapy Satisfaction Questionnaire; IV, intravenous; SC, subcutaneous; TSQM, Treatment Satisfaction Questionnaire for Medication.

Figure 2

Core concept is impact of rituximab IV/SC administration.

Abbreviations: CTSQ, Cancer Therapy Satisfaction Questionnaire; FACT-G, functional assessment of cancer therapy; IV, intravenous; PRO-CTCAE, patient-reported outcomes version of the common terminology criteria for adverse events; SC, subcutaneous.

Following initial item generation, the next stage involved documenting the questionnaires’ face and content validity (based on US Department of Health and Human Services guidance).22 Establishing content validity is a critical step in scale construction,23,24 and its documentation has been a formal regulatory standard since 2006.22 A task force of the International Society for Pharmacoeconomics and Outcomes Research has also provided good-practice guidance for the development and validation of patient-reported tools.25,26 Thus, the aim was to develop and validate new treatment satisfaction and impact questionnaires using methods that meet accepted standards for patient-completed measures.22–24,27 Two phases were conducted to assess the RASQ measures: 1) face and content validation (qualitative) and 2) psychometric evaluation (quantitative). The qualitative phase included ascertaining the content validity of the RASQ measures via face-to-face interviews and finalizing the item content prior to psychometric testing. The quantitative phase optimized data collected within two clinical trials on the RASQ measures to understand the psychometric properties of the newly developed questionnaires.

Qualitative phase

Study population and data collection

US English-speaking patients aged ≥18 years with CD20+ DLBCL or indolent B-cell lymphoma were recruited to the study by their treating physician. Other selected inclusion criteria were: adequate induction therapy (eight cycles) with rituximab IV as first-line treatment; Eastern Cooperative Oncology Group performance status ≤2; and physician confirmation that the patient had the physical, cognitive, reading, and linguistic skills to participate actively in a 60-minute interview. Patients were excluded if they had received second-line treatment for DLBCL or indolent B-cell lymphoma, had been diagnosed with human immunodeficiency virus-associated lymphoma, or had an uncontrolled physical or psychiatric illness.

Sampling quotas and targets were used during recruitment to provide a clinically and demographically diverse sample. All patients provided their written informed consent to participate in the study, and their verbal consent was obtained before any study-related activities began. Ethical approval for the study was provided by Copernicus, a centralized independent review board in the USA.

Interview procedures: concept elicitation and cognitive debriefing

After enrollment, each patient took part in a 60-minute, face-to-face, in-depth qualitative interview, combining both concept elicitation (CE) and cognitive debriefing (CD) (Figure 3). The purpose of CE was to gather information about patients’ experiences of rituximab treatment, whereas CD allowed their understanding of the instructions and response options to be explored. A structured interview guide was used to facilitate discussion between the patient and interviewer. Sessions were led by a trained interviewer who was experienced in conducting qualitative research and had been involved substantially in the development of the interview guide.

Figure 3

Outline of interview methodology.

Abbreviations: IV, intravenous; min, minutes; NHL, non-Hodgkin lymphoma; RASQ-IV, Rituximab Administration Satisfaction Questionnaire-intravenous; RASQ-SC, Rituximab Administration Satisfaction Questionnaire-subcutaneous; SC, subcutaneous.

Concept elicitation

The CE portion of the interview was conducted first and took ~25 minutes to complete. The interviews began with open-ended questions relating to their treatment experience and specifically any factors that had affected their adherence with rituximab IV. As patients did not have experience of rituximab SC, they were presented with a 1-minute video showing the administration of rituximab SC. They were then asked to describe their perceptions and expectations of treatment with rituximab SC based on their experience with rituximab IV as well as drawing on the information presented in the video.

Cognitive debriefing

Following CE, patients participated in the CD interview, which took ~35 minutes to complete. Patients were asked to complete both RASQ measures, by means of a “think-aloud” exercise.28 Half of the patients completed the RASQ-IV first while the other half completed the RASQ-SC (Figure 3). The “think-aloud” exercise involved the patient completing each questionnaire and verbally giving their thoughts as they read each instruction and completed each item. The interviewer prompted the patient throughout to ensure they shared all their thoughts. The benefit of this method was that it allowed access to patients’ spontaneous thoughts as they completed the instrument, thus identifying aspects that patients did not appear to understand correctly. Patients were then asked detailed questions about the content of the questionnaires; for example, the clarity of wording and the relevance of the concepts measured.

Of note, none of the patients had received rituximab SC and therefore it was not possible for them to assess the content validity of the RASQ-SC based on their own real-life experience. Thus, debriefing on this questionnaire focused mainly on its face validity, ie, exploring the patients’ understanding of the instructions and the wording of the items and response options.

Patients also undertook a task designed to explore the relevance of the satisfaction conceptual framework (Figures 1 and 2). Patients were asked to organize cards, labeled with each of the subconcepts and concepts from the questionnaires’ conceptual frameworks, to reflect their perception of the relationship between the subconcepts based on their treatment experiences.

All interviews were recorded and transcribed verbatim before being analyzed qualitatively using ATLAS.ti (version 7.0, Scientific Software Development GmbH, Berlin, Germany). A coding scheme was developed to score statements from the interviews; the statements were then sorted according to domains and used to inform modifications to the draft questionnaires.

Quantitative phase

The study population for the quantitative phase was derived from two clinical trials. The PrefMab (NCT01724021) study was a Phase IIIb, prospective, multicenter, multinational, open-label, randomized study in 743 adult patients with previously untreated CD20+ DLBCL or CD20+ FL. Patients randomized to arm A received one cycle of rituximab IV, followed by three cycles of rituximab SC, followed by four cycles of rituximab IV after interim staging. Patients randomized to arm B received four cycles of rituximab IV, followed by four cycles of rituximab SC after interim staging. Of the total population, 334 patients completed the RASQ-IV at cycle 4 and 291 completed the RASQ-SC at cycle 8 (considering any treatment sequence, 619 patients completed the RASQ-SC and 622 completed the RASQ-IV overall).

The MabCute (NCT01461928) study was a Phase IIIb, prospective, multicenter, multinational, open-label, randomized study in ~700 adult patients to evaluate maintenance therapy with rituximab SC continued until progression compared with observation only in patients with relapsed or refractory indolent NHL who completed and responded to induction and initial 2-year maintenance therapy. Of the total population, 92 patients completed the RASQ-IV or the RASQ-SC.

Psychometric evaluation

Psychometric analyses were outlined in a statistical analysis plan and performed on RASQ-IV and RASQ-SC and related domains to determine the reliability and validity of the two questionnaires. Specifically, the following psychometric measurement properties were examined: item performance (item and scale distribution [ie, missingness <10%, floor and ceiling effects <20%]), reliability (internal consistency reliability [Cronbach’s α≥0.70]), and validity (including concurrent validity [r≥0.40]). Tests and criteria are summarized in Table 1.

Table 1

Summary of tests and criteria

Property	Definition/test	Criteria for acceptability
Item performance	Data quality; assessed by completeness of data and score distributionsFrequency and percentage of missing data per itemFloor/ceiling effectsResponse option frequencies	Items missing in more than 10% of responsesCeiling effect: % of responses in the highest response category > (100/the number of response options on an item) %, eg, 20% for a five-response ordinal scaleFloor effect: % of responses in the lowest response category > (100/the number of response options on an item) %, eg, 20% for a five-response ordinal scaleUtilization of all response options
Reliability	Internal consistency reliability: extent to which items in a domain measure the same concept; assessed by Cronbach’s α and inter-item correlations assessed by Spearman’s or Pearson’s coefficient	Cronbach’s α≥0.70 was considered to indicate internal reliabilityInter-item correlation (Pearson or Spearman coefficient items of r>0.80 should be highlighted as potentially redundant)
Validity (hypothesis testing)	Evidence that the questionnaire measures a single concept, that items can be combined to form a summary score, and that domains measure distinct but related conceptsConcurrent validity: hypotheses based on criterion measure(s)Known groups differences: ability of a scale to differentiate among known groups	Concurrent validity: Spearman’s coefficient values of <0.3, 0.31–0.59, and >0.6 were considered to indicate low, moderate, and high levels of correlation, respectively, with acceptable ranges for convergence identified at ≥0.4

Convergent validity was assessed by looking at correlations with the “RASQ: Physical Impacts”, “psychological impacts”, “impact on activities of daily living”, “convenience”, and “satisfaction” domains and the three similar domains of the CTSQ (expectations of therapy, feelings about side effects, and satisfaction with therapy) at cycle 4 for the PrefMab study population. Moderate correlations (>0.4) were expected to confirm convergent validity, while weaker correlations were expected to confirm divergent validity.

All analyses, unless otherwise specified, were conducted using Statistical Analysis Software (SAS) version 9.1.3 (SAS Institute Inc, Cary, NC, USA).

Results

Study population

In total, ten patients were recruited to the study (Supplementary materials). Most patients were female (n=7; 70%) and Caucasian (n=8; 80%), with a mean age of 60.8 years (standard deviation: 12.3). Seven patients had indolent B-cell lymphoma and three had DLBCL. The time from diagnosis of NHL ranged from <1 year to >10 years, and the disease stage ranged between stages 1 and 4.

Concept elicitation interview results

An overview of the CE results is shown in Supplementary materials. CE analysis revealed that treatment administration route was an important component of the patient’s disease and therapeutic experience. Of particular note, patients highlighted the inconvenience associated with receiving an infusion that typically lasted several hours. Attending infusion appointments was also noted as adversely impacting patients in terms of the burden associated with traveling to the hospital as well as the emotional impact of attending their infusion appointment. In contrast, the potentially positive impact of rituximab SC was also described by patients (after viewing the video of this method of administration) in terms of improved convenience of a short administration time and reduced emotional impact.

The analysis also supported the conceptual relevance and comprehensiveness of the RASQ-IV. A comparison between the concepts related to IV administration spontaneously elicited from patients and those measured by the RASQ-IV revealed that all concepts were covered by the questionnaire, and no additional concepts were mentioned. “Confidence” and “impacts on activities of daily living” were the only concepts that were not elicited in every patient interview.

Similarly, a comparison between the CE concepts mentioned in relation to SC administration and those measured in the RASQ-SC demonstrated good concept coverage (Supplementary materials). Overall, the RASQ-SC provided broad coverage of the potential benefits of rituximab SC administration based on patients’ CE discussion. Patients mentioned the concepts of “convenience”, “physical impacts”, and “impacts on activities of daily living” most frequently. More conceptually distal concepts, such as “confidence”, “bothersome”, and “psychological impacts” were reported less frequently, possibly because patients found it difficult to discuss these hypothetically. None of the patients mentioned a “perceived impact of SC administration on the interaction with their nurse or doctor”.

Cognitive interview results

The majority of patients found the questionnaires easy to understand and respond to. Patients did not identify any missing concepts of importance. “Overall treatment preferences/satisfaction”, “impact of rituximab IV/SC administration”, and “interaction with nurse/doctor” were reported as being relevant to all patients (n=10) and “psychological impacts” and “treatment impact on daily living activities” were reported as being relevant to around half of the patients (Supplementary materials). The items and responses were generally well understood and interpreted consistently and correctly.

Revisions to questionnaires based on interview findings

Several minor amendments to the RASQ measures were made on the basis of the interview findings from the qualitative phase assessments and input from clinical experts (Supplementary materials). Changes to the wording of some items and response options were implemented to enhance clarity and aid consistent reporting. In addition, five items were deleted (leaving 17 items) as patients reported that there was conceptual overlap between these and other items in the questionnaires that were more clearly worded (rationale for these deletions is shown in Supplementary materials). The revised RASQ measures are presented in the online supplementary material (Supplementary materials). Scoring is completed by summing each of the five domains: physical impact, psychological impact, impact on activities of daily living, convenience, and satisfaction (most items offered five response options) where lower scores are more positive.

A total of 1,051 patients were included in the evaluation; however, only 556 of these patients provided data based on early cycles for the quantitative phase of the study: 294 at cycle 4 from PrefMab and 262 from MabCute. The demographic data of these patients are summarized in Table 2. There was a very little difference in the total numbers of males (279) and females (277). Patients in MabCute were typically older than those in PrefMab, with median ages of 65 and 59 years, respectively (respective means, 64.9 and 57.9 years). The majority of patients were White (440; 79.1%).

Table 2

Demographic characteristics of patients in the PrefMab and MabCute clinical studies who participated in the psychometric evaluation of RASQ

Characteristic	Patients, n (%)
	PrefMab (n=294)	MabCute (n=262)
Sex
Male	152 (51.7)	127 (48.5)
Female	142 (48.3)	135 (51.5)
Missing/no response	0	0
Age, years
Mean	57.9	64.9
SD	13.1	10.8
Median	59	65
Range	18–80	36–91
Ethnicity
White	216 (73.7)	224 (85.5)
Asian	64 (21.8)	0
American Indian/Alaskan native	2 (0.7)	0
Black	2 (0.4)	1 (0.4)
Native Hawaiian or other Pacific Islander	1 (0.3)	0
Other	8 (2.7)	9 (3.4)
Missing/no response	1 (0.1)	28 (10.7)

Abbreviations: RASQ, Rituximab Administration Satisfaction Questionnaire; SD, standard deviation.

Psychometric results

For this study, psychometric evaluation of the RASQ measures included item performance, reliability, and concurrent validity with the 16-item CTSQ at cycle 4 for the PrefMab study and cycle 3 for the MabCute study.

Item performance

In the PrefMab study, the RASQ response rate was demonstrated with a low frequency of missing responses (<10%) across items with the exception of item 12 (“length of time to get your SC injection”) and item 18 (“preference for treatment option”), which were each slightly above the criteria for acceptability at 10.9%. While there were no observed floor effects, ceiling effects were above the desired 20% criteria for the majority of items with the exception of item 12, which was 11.8% at ceiling, and item 15 (“lost or gained time for other things”) at 16.8%.

In the MabCute study, the RASQ response rate was demonstrated with a low frequency of missing responses (<10%) across all items. Similar to the PrefMab study, there were no observed floor effects; ceiling effects were above the desired 20% criteria for the majority of items with the exception of item 1 (“satisfaction”), which was 18.5% at ceiling, item 8 (“confident in treating disease”) at 17.4%, item 10 (“convenient to get”) at 13.0%, and item 12 (“length of time to get your SC injection”) at 4.3%.

Reliability

Due to the mixed rating scales included on the RASQ measures, Cronbach’s α coefficients (≥0.70) were conducted, initially, on the overall measure. For both the PrefMab and MabCute studies, internal consistency reliability of the standardized correlation between items was above the criteria supporting the overall reliability (PrefMab =0.791; MabCute =0.756). Similar results were observed on each of the measurable domains with the exception of the “impacts on activities of daily living” and “physical impacts” domains in the MabCute dataset, which were marginally below the recommended criteria for internal consistency reliability (α=0.651 and α=0.669, respectively). There were no items with inter-item correlations >0.80.

Validity

Correlations with domains such as “RASQ: Physical Impacts” and “CTSQ: Feelings About Side Effects”, “RASQ: Physical Impacts” and “CTSQ: Satisfaction With Therapy”, and “RASQ: Satisfaction” and “CTSQ: Satisfaction With Therapy” were moderate (r=0.489, r=0.480, and r=0.501, respectively). With respect to divergent validity correlations, moderate correlations (>0.4) were observed between the “CTSQ: Expectations of Therapy” and all domains of the RASQ (<0.3). The full correlation results for convergent and divergent validity are presented in Table 3.

Table 3

Correlation results for convergent and divergent validity

RASQ measure and domains
Parameter	Threshold for acceptability	Overall RASQ	Treatment satisfaction	Convenience	Physical impact	Psychological impact	Impact on activities of daily living
Stage 1: item response and dimensionality analysis
Classical statistics
Complete data	≥80%	(P) 94.76%
		(M) 99.56%
Floor range	≤9%	(P) 0.0%–78.6%
		(M) 1.1%–69.6%
Ceiling range	≤9%	(P) 0.0%–56.9%
		(M) 0.0%–52.2%
Inter-item correlation	<0.80	(P) 0 items
		(M) 0 items
Stage 2: psychometric testing of final RASQ (reliability, construct validity, and responsiveness)
Reliability
Internal consistency	≥0.7	(P) −0.79	(P) −0.13	(P) −0.99	(P) 0.70	(P) 0.77	(P) 0.78
		(M) 0.76	(M) 0.18	(M) −1.01	(M) 0.67	(M) 0.77	(M) 0.65
Construct validity
Concurrent validity, CTSQ;	>0.4	Expectations of therapy	0.188 (0.0014)	0.147 (0.0159)	0.291 (<0.001)	0.227 (0.001)	0.110 (0.0667)
PrefMab only;		Feelings about	0.328 (<0.001)	0.356 (<0.001)	0.489 (<0.001)	0.394 (<0.001)	0.365 (0.001)
Spearman’s		side effects	0.501 (<0.001)	0.410 (<0.001)	0.480 (<0.001)	0.376 (<0.001)	0.291 (<0.001)
coefficient (P-value)		Satisfaction with therapy

Abbreviations: CTSQ, Cancer Therapy Satisfaction Questionnaire; M, MabCute; P, PrefMab; RASQ, Rituximab Administration Satisfaction Questionnaire.

Discussion

The RASQ measures were developed to address an unmet need for a self-reported patient instrument that can assess the burden, impact, and preference for IV and SC administration routes of rituximab. After minor revisions to the questionnaires to improve clarity and consistency of wording, the current findings confirm the qualitative face and content validity of the RASQ, along with its psychometric validity.

Although the RASQ measures were developed using recognized methodology,22–24,27 our study had several limitations. Patients did not have experience of receiving rituximab SC (which was in development at the time of study), and only a video sequence showing the SC administration could be presented. This may have limited the meaning of item 16 in the final instruments (Supplementary materials), which addressed preference for rituximab IV or SC. In addition, the RASQ items could be considered to reflect a mixture of a “scale”-type and “index”-type questions, according to the definition of Streiner, and this may limit the interpretation of the instrument.29

The RASQ is primarily intended for use within clinical studies, and more generally to quantify satisfaction levels dependent on the route of rituximab administration. The findings of the PrefMab study (patients with previously untreated NHL) support that preference for route of administration is likely to have clinical relevance. In PrefMab, 80% of patients preferred rituximab SC (assessed by the Patient Preference Questionnaire) compared with 10% who preferred rituximab IV (10% expressed no preference).30 As use of rituximab expands outside of oncology, further validation work will be needed to determine what modifications may allow the RASQ to be used across these different disease areas.

The ability of a therapy to achieve its desired clinical outcomes is its most important attribute. However, there is a need for therapies that maintain this critical outcome and are least distressing and impactful on patients’ daily lives. The RASQ-IV and RASQ-SC may help to understand and further quantify this relationship for rituximab by effectively assessing patient’s perceptions and satisfaction with either the IV or SC routes, and may be adapted in future to make similar assessments for other monoclonal antibody treatments and therapeutic areas.