Test-retest reliability of the Quebec user evaluation of satisfaction with assistive technology 2.0-Korean version for individuals with spinal cord injury.

[Purpose] The study purpose was to investigate the test-retest reliability of the Quebec User Evaluation of Satisfaction with assistive Technology 2.0-Korean version (QUEST-K) for individuals with traumatic spinal cord injury.
[Subjects and Methods] Seventy participants with spinal cord injury were enrolled in this study. The participants were assessed using the QUEST-K in two sessions conducted 3 days apart. This study also utilized the spinal cord independence measure III and modified Barthel index for determining their validity concurrently with the QUEST-K.
[Results] The test-retest reliability (intraclass correlation coefficient 0.855) of the QUEST-K was high, indicating a good agreement. The results of the QUEST-K testing were not substantially positively correlated with those of the spinal cord independence measure III (r = -0.075) and Modified Barthel Index (r=-0.138).
[Conclusion] The test-retest reliability of the QUEST-K was high in individuals with spinal cord injury. The QUEST-K could be a reliable measure for evaluating the satisfaction with assistive technology during functional performance in patients with spinal cord injury. The ability of the QUEST-K to detect actual changes is acceptable for research and clinical settings.

INTRODUCTION

Spinal cord injury (SCI) is one of the most serious injuries of the central nervous system, resulting in severe disabilities that affect independent functional activities in the individual’s personal and social life1, 2). Most individuals with SCI generally rely on assistive technology (AT) as the primary means of accomplishing functional activities (e.g., standing and walking at home and outside). This particularly applies to the functional activity of walking since an improvement is gait performance is an important rehabilitation goal for independent living. AT is used in clinical settings for regaining functional activities because it facilitates independent walking with assistive devices, delays functional decline, and reduces the patient’s dependence on personal assistance3). Previous studies have reported that assistive devices enhance the ability of patients with SCI to participate in daily activities and allow community dwelling1). Therefore, the dependence on AT has been increasing in individuals with SCI. An evaluation of the individual’s performance and satisfaction with AT is important.

The Quebec User Evaluation of Satisfaction with assistive Technology (QUEST) 2.0 was developed as an outcome measurement test to evaluate a person’s satisfaction with a wide range of AT devices. Intended to serve as a clinical and research instrument, this rating scale provides practitioners with a means of collecting satisfaction data to document the real-life benefits of AT and to justify the need for AT devices4). However, the use of the QUEST-Korean version (QUEST-K) will not be appropriate for evaluating satisfaction with AT in individuals with SCI before its reliability is tested in this population. The rating scales used for evaluating of user satisfaction with AT should be adapted to the society in terms of language, policies, and sociocultural aspects, and their validity and reliability should be established.

The study purpose was to evaluate the test-retest reliability of the QUEST-K in patients with SCI. This study also investigated correlations between the QUEST-K, spinal cord independence measure III (SCIM-III), and modified Barthel index (MBI).

SUBJECTS AND METHODS

Forty individuals with SCI participated in this study. The general characteristics of the subjects are shown in Table 1. The following inclusion criteria were used: 1) definitive clinically supported diagnosis of SCI and 2) use of an assistive device (manual wheelchair, powered wheelchair, or scooter). This study excluded patients unable to communicate. Data collection was performed for members of the Korea Spinal Cord Injury Association. The study was approved by the human research ethics committee of Sahmyook University, and all parents of the children agreed to participate in this study and voluntarily signed the consent form.

Table 1.

General characteristics of the study participants (N = 70)

Characteristics	Participants
Gender (male/female)	55/15
Age (years)	40.9 ± 11.2
Height (cm)	172.4 ± 7.7
Body weight (kg)	66.0 ± 10.9
Injury type (complete/incomplete)	29/41
ASIS (A/B/C/D/E)	29/9/9/15/8
Injury level (cervical/thoracic/lumbar)	34/21/15
Post-duration (years)	31.1 ± 58.6
Assistive device
Manual wheel chair	63
Electrical wheel chair	2
Crutch	3
Cane	5
Total number of assistive devices	1.3 ± 0.6
Total number of prescription drugs	2.4 ± 2.3

Data were collected following a two-session protocol. The QUEST-K, SCIM-III, and MBI were assessed for test-retest reliability and concurrent validity in the first session. The QUEST-K was then assessed for test-retest reliability in the second session 3 days later. The testing times in the first and second sessions were 45 min and 15 min, respectively. The tests were administered in a quiet and well-organized therapy room, and participants were allowed a rest period between each test in the first session.

To investigate the test-retest reliability of the QUEST-K in SCI patients, three clinical measures (QUEST-K, SCIM-III, and MBI) were evaluated by means of clinical interview. The QUEST 2.0 was developed by Demers and coworkers and refers to a person’s positive or negative evaluation of those distinct dimensions of the AT device that are influenced by one’s expectations, perceptions, attitudes, and personal values. The evaluation focuses on how satisfied the person is with specific features of the AT device as well as certain characteristics of the services related to the AT4). The QUEST-K, which was translated from by the QUEST 2.0 by An and Kong, consists of an 8-item device domain and a 4-item service domain, and is one of the few standardized instruments designed to measure user satisfaction with a broad range of AT devices. Each item is rated on a five-point ordinal scale graded from 1 (not satisfied at all) to 5 (very satisfied). The QUEST-K yields three scores: device, services, and total QUEST-K, which are calculated by averaging the valid responses to assigned items. For the purpose of identifying the sources of user satisfaction or dissatisfaction, a space for comments is provided next to each item. Finally, the three most important items for the device being assessed are identified5).

Developed by Catz and colleagues in 1997, the SCIM is a widely used measurement tool of daily activities and functional activities in persons with SCI. SCIM-III contains items on 19 daily tasks grouped into three subscales, which are graded for increasing difficulty requiring higher ability. The tool showed good reliability and validity in patients with SCI6). The Barthel index, developed by Barthel and modified by Grander and colleagues in 1979, is a tool for measuring performance in activities of daily living (ADL). MBI contains 10 items describing ADL and mobility, and each item is graded on a scale of 0–10 points. MBI has demonstrated excellent inter-rater and intra-rater reliabilities7).

Collected data were analyzed using PASW version 18.0 for Windows (SPSS Inc., Chicago, IL, USA), and the significance level was set at p < 0.05. The test-retest reliabilities of the total and sub-item scores of the QUEST-K were statistically evaluated using intraclass correlation coefficients [ICC (2,1)]. ICCs ≥0.80 were considered to indicate a good agreement. This study analyzed Pearson’s correlation coefficients to determine correlations between the total scores of the QUEST-K, SCIM-III, and MBI.

RESULTS

Table 2 shows the mean scores for the assistive devices and related services obtained with the QUEST-K test, as well as the total scores of the QUEST-K, SCIM-III, and MBI tests. The ICC for the test-retest reliability of the total QUEST-K scores was found to be high at 0.855 (95% confidence interval [CI], 0.766–0.910). The device and service items had good reliabilities (0.837 and 0.847, respectively). The Cronbach’s alpha coefficients for the device, service, and total scores were 0.837, 0.847, and 0.855, respectively, indicating high internal consistency (Table 3).

Table 2.

Mean values, standard deviations (SD), and minimal/maximal scores of the clinical measures (N = 70)

Variables	Minimal score	Maximal score	Mean ± SD
QUEST-K
Assistive devices	14	40	24.3 ± 5.3
Related services	6	20	11.6 ± 2.7
Total	22	60	35.9 ± 7.4
CIM-III	0	98	61.1 ± 23.7
MBI	0	100	68.9 ± 31.8

Table 3.

Test-retest reliability of the QUEST-K in the participants (N = 70)

Test items	Cronbach’s alpha	ICCb intra-rater	95% CIa
Assistive devices	0.837	0.837	0.584–0.816
Related services	0.847	0.847	0.754–0.905
Total score	0.855	0.855	0.766–0.910

aCI: confidence interval; bICC: intraclass correlation coefficient

Table 4 illustrates the relationship between the QUEST-K, SCIM-II, and MBI. The assistive device score of QUEST-K was significantly correlated with the related services score of the QUEST-K (r = 0.637) and the total score of the QUEST-K (r = 0.955). The related services score of the QUEST-K was significantly correlated with the total score of the QUEST-K (r = 0.905). However, the results of QUEST-K did not show a high correlation with those of SCIM-III and MBI.

Table 4.

Correlations among QUEST-K, SCIM-III, and MBI (N = 70)

	Assistive devices, QUEST-K	Related services, QUEST-K	Total score, QUEST-K	SCIM-III	MBI
Assistive devices, QUEST-K		0.637**	0.955**	−0.096	−0.144
Related services, QUEST-K	0.637**		0.817**	−0.077	−0.150
Total score, QUEST-K	0.955**	0.817**		−0.075	−0.138
SCIM-III	−0.096	−0.077	−0.075		0.905**
MBI	−0.144	−0.150	−0.138	0.905**

**p < 0.01

DISCUSSION

The purpose of the present study was to evaluate the test-retest reliability of the QUEST-K in individuals with SCI. The main findings: 1) QUEST-K showed good intra-rater reliability, 2) the results of the QUEST-K did not show substantial correlations with those of SCIM-III.

People with SCI lose lower limb function and overall aerobic fitness because of the loss of the central nervous system control of musculoskeletal function8). The purpose of AT is to reduce common complications, battle disabilities, facilitate functional activities, and improve the quality of life in individuals with SCI. Clinical care should include examination and treatment based on the individual’s impairment and activity limitations before recommending assistive devices, and it should facilitate acceptance of such devices and comprise re-examination and follow-up care afterwards9). QUEST 2.0 was developed to ensure proper fit, function, appearance, and service of assistive devices during the follow-up care. QUEST 2.0 has been mainly used in adults and is a pioneer scale for satisfaction measurement in the field of rehabilitation AT. Previous studies have reported good reliability and validity. This study examined the test-retest reliabilities of the scores of device and service subscales and total scores of QUEST-K. The study results indicate that AT users in Korea has specific concerns regarding satisfaction and suggest the QUEST-K has a high test-retest reliability in individuals with SCI.

This study found no significant associations between the results of the QUEST-K, SCIM-III, and MBI tests. There are some limitations to this study. In particular, the participants had different impairment levels (e.g., the cervical, thoracic, and lumbar levels). Functional disabilities in individuals with SCI differ according to impairment levels. Therefore, patients homogeneous in terms of impairment level will need to be recruited to investigate the relationships and importance of satisfaction with assistive devices in future studies. Users of assistive devices in different cultures may have different concerns related to satisfaction. QUEST-K is a reliable tool for measuring satisfaction in individuals with SCI using various kinds of assistive devices.