Validation of QUALAS-T, a health-related quality of life instrument for teenagers with spina bifida.

INTRODUCTION: We aimed to develop and validate a self-reported QUAlity of Life Assessment in Spina bifida for Teenagers (QUALAS-T).
MATERIAL AND METHODS: We drafted a 46-question pilot instrument using a patient-centered comprehensive item generation/refinement process. A group of 13-17 years olds with spina bifida (SB) was recruited online via social media and in person at SB clinics (2013-2015). Healthy controls were recruited during routine pediatrician visits. Final questions were identified based on clinical relevance, factor analysis and domain psychometrics. Teenagers with SB completed the validated generic Kidscreen-27 instrument.
RESULTS: Median age of 159 participants was 15.2 years (42.0% male, 77.4% Caucasian), similar to 58 controls (p ≥ 0.06). There were 102 online and 57 clinic participants (82.8% of eligible). Patients, parents and an expert panel established face and content validity of the 2-domain, 10-question QUALAS-T. Internal consistency and test-retest reliability were high for the Family and Independence and Bladder and Bowel domains (Cronbach's alpha: 0.76-0.78, ICC: 0.72-0.75). The Bladder and Bowel domain is the same for QUALAS-T , QUALAS-A for adults and QUALAS-C for children. Correlations between QUALAS-T domains were low (r = 0.34), indicating QUALAS-T can differentiate between distinct HRQOL components. Correlations between QUALAS-T and Kidscreen-27 were also low (r ≤0.41). QUALAS-T scores were lower in teenagers with SB than without (p <0.0001).
CONCLUSIONS: QUALAS-T is a short, valid HRQOL tool for adolescents with SB, applicable in clinical and research settings. Since the Bladder & Bowel domains for all QUALAS versions are the same, Bladder and Bowel HRQOL can be measured on the same scale from age 8 through adulthood.

INTRODUCTION

Spina bifida (SB) is a common congenital anomaly of the central nervous system, affecting multiple organ systems, including bladder and bowel function [1]. One of the goals of SB management is optimizing SB-specific health-related quality of life (HRQOL), or the individual's perception of the impact of SB on his or her physical and psychosocial functioning [2, 3]. This is particularly important in teenagers with SB as they transition to more independent adult care. One of the challenges to this approach is that no clinically useful and validated tools exist for this age group, especially ones that incorporate bladder and bowel domains [4]. Several small, single-institutional studies published to date used limited and poorly validated or non-validated instruments.[4-10] Generic HRQOL instruments validated for healthy teenagers[11-14] were not developed to capture the impact of SB on HRQOL or to detect clinically meaningful differences in HRQOL [2].

A clinically relevant HRQOL instrument focuses on HRQOL specific to SB, and not physical function, is short and has excellent psychometric properties [2, 4]. It should be developed with input from multiple stakeholders: teenagers, their parents, caregivers and healthcare providers, and should accommodate teenagers' cognitive and emotional development [2, 4]. Our goal was to develop and validate a clinically useful, self-reported, disease-specific, health-related QUAlity of Life Assessment in Spina bifida for Teenagers (QUALAS-T).

MATERIAL AND METHODS

The study protocol received Internal Review Board approval (IRB 9470) and followed the patient-reported outcome instrument development guidelines from the Federal Drug Administration [15]. The study protocol was similar to that used to develop and validate QUALAS versions for adults (QUALAS-A) [16] and children (QUALAS-C) [17].

Eligibility and exclusion criteria

Adolescents (13–17 years old) with a history of myelomeniongocele or lipomeningocele requiring newborn spine surgery were recruited either online via social media, or in person at outpatient multidisciplinary SB clinics. A questionnaire was administered between January 2013 and August 2015. Non-eligibility criteria included: a primary diagnosis other than spina bifida (primary tethered cord, sacral agenesis, medullary lipoma, anorectal malformation, spinal trauma, tumor), poor self-reported English proficiency, developmental delay interfering with comprehension of questions, or surgery in the last month. Eligible participants were excluded if they did not complete the questionnaire. Eligibility and exclusion criteria remained unchanged for all phases of this study. Healthy controls without SB were recruited at two local pediatrics clinics during routine checkup visits.

Phase 1. Item generation

QUALAS-T was developed using a comprehensive, patient-centered item generation and refinement process (Figure 1). Items (questions) were generated from a series of 10 semi-structured interviews with teenagers with SB and their families, a review of published instruments as well as the opinions of 20 national and international experts in SB care (see Acknowledgments). The item generation process was stopped when no new items were generated, yielding a comprehensive list of 147 items covering themes of cognition, independence, emotional impact, social interactions, romantic concerns, educational, work and leisure activities, mobility, healthcare interactions, toileting, as well as urinary and fecal continence. This exhaustive list was reduced to 90 items by an expert panel, which eliminated similarly worded and redundant items, while maintaining the themes.

Figure 1

Creation of the QUAlity of Life Assessment in Spina bifida in Teenagers (QUALAS-T). SB – Spina Bifida.

Phase 2. Development of pilot instrument

The 90 items were rated on importance by 6 teenagers with SB and 6 parents from our center's SB clinic. Ratings were made on a 3-point Likert scale (not important, somewhat important, very important). After review by an expert panel, 46 items with the highest rankings per theme were selected to create a more manageable and representative pilot QUALAS-T. This item reduction approach has been successfully used in other studies in the SB population [8, 9, 18]. Some items were reworded to ensure comprehension and clarity based on feedback from the 12 individuals rating the questions and a Health Literacy Educator at our institution's Family Education Center (see Acknowledgments). Readability was assessed by the Flesch Kincaid Grade Level test [19]. The pilot QUALAS-T was self-administered. Questions were close-ended and numbered to avoid omission. Similar to several other pediatric instruments, questions reflected the last 4 weeks [9, 12, 20]. Responses used a 5-point Likert scale (never, almost never, sometimes, almost always, always), an approach successfully used in other instruments [8, 9, 11, 12, 20]. Items focusing on similar themes were grouped together to simulate the final QUALAS-T instrument.

Phase 3. Further participant recruitment

A demographic questionnaire and the pilot QUALAS-T were administered anonymously to a sample of teenagers with SB and controls in a cross-sectional survey. Teenagers with SB also completed the validated general Kidscreen-27 instrument [11]. In order to open the study to teenagers from a variety of social and healthcare setting, participants were recruited through local and national organizations via social media (see Acknowledgments) and at an outpatient multidisciplinary SB clinics. Consent was obtained online (computer-based) or in person (clinic-based). Two-week test-retest reliability was assessed by readministering the QUALAS-T to participants recruited in clinic. Teenagers with SB and controls received $5 incentive payments. Incentive payments were increased to $25 in the final 6 months of the study to encourage enrollment.

Phase 4. Refinement of instrument

A factor analysis on 46 items would require at approximately 230 participants (5 participants/item) for adequate statistical power [21, 22]. The study was closed after 32 months of recruitment despite not reaching 230 participants. To further refine the instrument, we performed a principal axes method of factor analysis using varimax (orthogonal) rotation [23, 24]. The number of extracted factors/domains was based on five criteria described previously: scree plot inflexion point, multiple models with different numbers of factors and 5 rotations, extracted factors having ≥3 variables with loadings ≥0.4, clinical plausibility/ relevance, and statistical properties of each domain [16, 23, 24]. Based on these criteria, the final QUALAS-T instrument consisted of two domains of 5 items each. QUALAS-T is scored 0–100, where higher values signify higher HRQOL, and takes less than 5 minutes to complete. In the event only four responses are provided for a 5-item domain, we suggest considering these four items in the domain [25].

Phase 5. Internal validation

We assessed different types of validity to ensure that the 10-item QUALAS-T measures what it intends to measure [26]. A review by patients, families and experts established face and content validity of QUALAS-T. Factor analysis additionally assessed construct validity. For each domain score, we calculated the mean, standard deviation (SD), median, range and the percentage of participants scoring minimum (floor) and maximum (ceiling) values. Reliability, or reproducibility, was assessed using internal consistency (Cronbach's alpha: 0.7–0.9 signifying good consistency without redundancy) [26] and two-week test-retest reliability (Intraclass Correlation Coefficient, ICC: ≥0.7 indicating acceptable reliability) [27].

Conceptual independence or redundancy of QUALAS-T domains was assessed with convergent and divergent validity. Convergent validity evaluates the degree to which QUALAS-T scores converge with other instruments measuring similar outcomes. On the other hand, divergent validity evaluates the degree to which QUALAS-T scores diverge from those measured by dissimilar instruments. We calculated Pearson correlation coefficients (r) among QUALAS-T domains and with Kidscreen-27 domains. To further assess divergent validity, domain scores between the teenagers with SB and controls were compared using a t-test. To quantify effect size, we used a previously established method of dividing the mean difference between teenagers with SB and controls by the SD of the control population [28, 29]. Several distribution-based approaches were used to determine what minimally important difference could be considered clinically significant. Estimated point differences were determined using 1/2 SD, [30] internal consistency and test-retest reliability, as previously described [31, 32]. For each domain, we selected the most conservative, largest point difference calculated by the three methods as the minimal important score difference. A critical p = 0.05 was used (v9.4, SAS Institute, Cary, NC).

RESULTS

Phases 1 and 2. Development of pilot instrument

The 46-item pilot instrument was developed in a multifaceted, patient-centered fashion (Figure 1). Ten semi-structured interviews lasted a mean of 20 minutes.

Phase 3. Demographics

Of 110 clinic patients screened, 68 (61.8%) met eligibility criteria, and 57 (83.8%) were ultimately enrolled (Figure 2). Of 124 online participants, 124 (100.0%) met eligibility criteria, and 102 (82.3%) were enrolled. We noted no significant differences in gender or age between eligible teenagers who were and were not enrolled.

Figure 2

Enrollment of teenagers into the study Teenagers were enrolled at an outpatient multidisciplinary spina bifida as well as online via social media. Enrollment commenced in January 2013 through August 2015 (32 months). Healthy controls without SB were recruited from two local pediatrics clinics during routine checkup visits.

Median age of 159 participants was 15.2 years (42.0% male, 77.4% Caucasian), similar to 58 controls (p ≥0.06) (Table 1). Half of participants were community ambulators (55.0%) and 71.7% had a ventriculoperitoneal shunt. Most participants performed clean intermittent catheterizations (77.4%) and 78.6% reported daytime urinary dry intervals of at least 4h. Participants lived in 30 states of the United States, encompassing 83% of the country's population in 2011 [33].

Table 1

Factor analysis of responses from 159 teenagers with spina bifida

Item	Factor 1	Factor 2
Having future children	63*	6
Finding a partner	60*	9
Bother by others helping	59*	-1
Bother by family helping	59*	-5
Doing what others can do	54*	26
Bowel wait bother	14	64*
Worry about pads being noticed	26	62*
Bowel leak bother	7	61*
Urine problems stop you from fun things	37	51*
Urine leak bother	37	40*

Factor loading value is greater than the geometric mean (root mean square) of all values in the matrix.

A similar analysis was performed on 36 other items assessed (data not shown). Each row represents an item scale coded on a scale of 1 = never to 5 = always. Factor loadings in the table are multiplied by 100 and rounded to the nearest integer.

Phase 4. Factor Analysis

A break in the scree plot slopes on unrotated and varimax rotated factor analyses of 46 items suggested two meaningful factors, followed by a decreasing slope of minor factors (Table 1). Similar findings were obtained on the other 5 rotations. These two factors (Family and Independence, Bladder and Bowel) accounted for 46.9% of the total variance.

A review by patients, families and experts established content and face validity of the final 10-question QUALAS-T. The Flesch Kincaid Grade Level test indicated a fifth grade reading level. Properties of the QUALAS-T domain scores were calculated, without evidence of floor or ceiling effects (Table 2). Each of the two domains had robust internal consistency (Cronbach's alpha: 0.76–0.78) and test-retest validity (ICC: 0.72–0.75). The mean two-week test-retest changes were small for each domain (+1.9 to +4.3). Missing data was acceptably low for each domain (Family and Independence: 6.9%, Bladder and Bowel: 0.6%).

Table 2

Domain characteristics of the self-reported QUALAS-T in adolescents with and without spina bifida

Adolescents with spina bifida
Domain	N	% missing	Mean (SD)	Median (Range)	% Scoring Minimum	% Scoring Maximum	Cronbach’s alpha	Test-retest reliability (ICC)
Family and Independence	148	6.9	57.0 (23.5)	60.0 (0.0-100.0)	2.0	4.1	0.76	0.75
Bladder and Bowel	158	0.6	68.5 (25.3)	70.0 (5.0-100.0)	0.0	15.8	0.78	0.72
Adolescents without spina bifida
Domain	N	% missing	Mean (SD)	Median (Range)	% Scoring Minimum	% Scoring Maximum	Mean difference for adolescents with SB* (p-value)	Effect size vs. adolescents with spina bifida**
Family and Independence	55	5.2	86.1 (18.8)	95.0 (20.0-100.0)	0.0	36.4	-29.4 (<0.0001)	-1.6Large (-)
Bladder and Bowel	58	0.0	96.7 (12.0)	100.0 (20.0-100.0)	0.0	87.9	-28.3 (<0.0001)	-2.4Large (-)

Abbreviations: QUALAS-T: QUAlity of Life Assessment in Spina bifida for Teenagers; SD: Standard Deviation; ICC: Intraclass Correlation Coefficient.

t-test.

Effect size was calculated as difference in means (spina bifida-no spina bifida) divided by the standard deviation of group without spina bifida (0.2: small, 0.5: moderate, 0.8: large effect)

Correlations between QUALAS-T domains were low (r = 0.34), indicating that QUALAS-T differentiates between two distinct HRQOL components (Table 3). Correlations between QUALAS-T and Kidscreen-27 domains were also low (r ≤0.41), supporting the concurrent use of these instruments.

Table 3

Interscale correlations between QUALAS-T domains and the Kidscreen-27 summary scores

Instrument	Domain	QUALAS-T		Kidscreen-27
		Family and Independence	Bladder and Bowel	Physical Well-being	PsychologicalWell-being	Autonomy and Parent Relation	Social Support and Peers	School Environment
QUALAS	Family and Independence	1.00	–	–	–	–	–	–
	Bladder and Bowel	0.34	1.00	–	–	–	–	–
Kidscreen-27	Physical Well-being	0.16 d	0.21 c	1.00	–	–	–	–
	Psychological Well-being	0.41	0.28 b	0.37	1.00	–	–	–
	Autonomy and Parent Relation	0.23 c	0.31 a	0.31 a	0.47	1.00	–	–
	Social Support and Peers	0.13 d	0.12 d	0.32 a	0.46	0.38	1.00	–
	School Environment	0.24 c	0.11 d	0.37	0.42	0.49	0.34 a	1.00

QUALAS-T – QUAlity of Life Assessment in Spina bifida for Teenagers

p < 0.0001 for all correlations, except where marked:

p ≤ 0.001

p ≤ 0.01

p ≤ 0.05

p ≥ 0.10

QUALAS-T scores for controls were significantly higher than teenagers with SB for both domains (p <0.0001), each with a large effect size (≥1.0) (Table 2). Similar to teenagers with SB, missing data was low for each domain (Family and Independence: 5.2%, Bladder and Bowel: 0.0%). Using several distribution-based approaches, we calculated the minimally important difference to be ≥15 for both domains (range: 11.7–13.4).

DISCUSSION

We present a novel, validated HRQOL instrument developed specifically for teenagers with SB. QUALAS-T is composed of items relevant to teenagers with SB, their families and clinicians, including items relating to intimate relationships and a comprehensive domain focusing on bladder and bowel dysfunction. Rather than treating QUALAS-T scores as absolutes, scores may be best compared longitudinally to the same person over time, or between individuals.

QUALAS-T is the third and final of a set of short, age-specific HRQOL instruments meant to be self-administered by people with SB. Both QUALAS-C (Children: 8–12 years old) and QUALAS-T have two domains of 5 items each [17]. QUALAS-A (Adults: ≥18) has three domains of 5 items each[16]. All three questionnaires contain unique age-specific items addressing issues relating to esteem and independence, with QUALAS-A also containing items on sexuality and relationships (Table 4). All three QUALAS questionnaires share the same Bladder and Bowel domain, allowing Bladder and Bowel HRQOL to be tracked with the same questions starting at 8 years old. Low rates of missing data for all QUALAS instruments suggest they are not difficult to complete. Finally, for each QUALAS domain, we have established minimally important differences between two scores to reflect clinically significant differences.

Table 4

Comparison of the three QUALAS questionnaires

Item	QUALAS-C 2.0 (8-12 years old)	QUALAS-T (13-17 years old)	QUALAS-A (18 years old and older)
Domain	Esteem and Independence*	Family and Independence	Esteem and Sexuality
Embarrassment about your look	X		X
Dealing with health problems	X
Urine catheterization bother	X
Bother by family helping	X	X
Bother by others helping	X	X
Doing what others can do		X
Finding a partner		X
Treated differently because of health problems			X
Having future children		X	X
Bother by sexual in/activity			X
Future sexual satisfaction			X
Domain	Bladder and Bowel	Bladder and Bowel	Bladder and Bowel
Worry about pads being noticed	X	X	X
Bother by urine leak	X	X	X
Urine problems stop you from fun things	X	X	X
Bother by bowel leak	X	X	X
Bother by waiting for bowel movement	X	X	X
Domain			Health and Relationships*
Overall health			X
People saw you for more than health problems			X
Able to do fun things			X
Time with friends			X
Close friendships outside family			X

For this domain, the minimally important difference between two scores that represents a clinically significant difference was ≥10. This difference was ≥15 for the remaining domains.

Over the last 20 years, three other SB-specific HRQOL instruments have been developed [8, 9, 10]. Among these, only the Parkin et al. instrument has a patient-reported version for teenagers 13–18 years old [8]. Similar to the other two other instruments, the questionnaire by Parkin et al. is lengthy (47 items) and does not comprehensively address bladder and bowel care. Finally, it is unclear what score change, or difference, on any of these instruments can be considered clinically relevant.

Another reason for developing QUALAS-T was that generic HRQOL instruments developed for teenagers without SB, such as Kidscreen, [11] fail to capture aspects of HRQOL important to teenagers with SB. As the concept of HRQOL may vary between individuals with and without SB, QUALAS-T scores of ‘healthy controls’ are therefore likely of no clinical relevance and were used only to calculate validation statistics in this study.

Our study has several limitations. Despite an aggressive recruitment strategy over a prolonged 32-month period, we recruited only 2/3 of the ambitious 230 teenagers planned for a well-powered factor analysis. Nonetheless, this large sample of adolescents with SB is one of the largest in the literature and, we believe, did not impair the statistical analysis, as the results were consistent on all analyses performed.

Study participants may have had fewer developmental and functional limitations than the general SB population. In addition, selection bias of eligible participants is unlikely to have played a significant role, as eligible adolescents who were and were not enrolled had similar characteristics. While we did not use anchor-based methods to calculate minimally important differences, distribution- and anchor-based methods have been shown to give comparable results [34]. Finally, since this was validation study, risk factors for lower HRQOL were not investigated.

CONCLUSIONS

QUALAS-T is a short, validated tool for evaluating HRQOL in adolescents with SB, making it a useful instrument in both clinical and research settings. Since the Bladder and Bowel domains for QUALAS-T and QUALAS-A are the same, this aspect of SB-specific HRQOL can be measured on the same scale after age 13 and into adulthood.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.