The Use of Task-based Cognitive Tests for Defining Vocational Aptness of Individuals with Disabilities.

OBJECTIVES: The purpose of this study was to demonstrate the use of task-based cognitive tests to detect potential problems in the assessment of work training for vocational rehabilitation.
METHODS: Eleven participants with a normal range of cognitive functioning scores were recruited for this study. Participants were all trainees who participated in a vocational training program. The Rey Complex Figure Test and the Allen Cognitive Level Screen were randomly administered to all participants. Responses to the tests were qualitatively analyzed with matrix and scatter charts.
RESULTS: Observational outcomes derived from the tests indicated that response errors, distortions, and behavioral problems occurred in most participants. These factors may impede occupational performance despite normal cognitive function. These findings suggest that the use of task-based tests may be beneficial for detecting potential problems associated with the work performance of people with disabilities.
CONCLUSION: Specific analysis using the task-based tests may be necessary to complete the decision-making process for vocational aptness. Furthermore, testing should be led by professionals with a higher specialization in this field.

Introduction

Vocational rehabilitation can help a disabled person return to premorbid life and vocation after disease or injury. Rehabilitation can also help develop functional abilities for work performance, and thereby help achieve final goals in the rehabilitation process of disabled people. It is important to prepare the specific skills for performing social activities and participation in their own way without great difficulties. Vocational rehabilitation is a good option to improve quality of life through productive activities that facilitate social reintegration as a workman and meet their role in society [1,2].

In our country, individuals with disabilities encounter many difficulties trying to obtain specific opportunities for effective vocational training in clinical settings and keep healthy patterns in their social life. In general, vocational training adopts a uniform method without recognizing personal differences despite the variety of individual factors such as the type of disability, symptomatic severity, and intellectual or educational level of the disabled person [3]. Selection for vocational rehabilitation should be based on precise evaluation of an individual's physical and cognitive functions and detection of obstructive elements for vocational training, which are prerequisite to supporting the development of intervention strategies for occupational capability [4].

Detailed analysis of professionals with the best knowledge in this field contributes to finding private problems of the disabled who feel a difficulty in performing occupational tasks and thus establishing better intervention strategy for occupational performance [5]. Somesthetic impairments, cognitive and perceptive disabilities, and emotional and psychological problems, along with the lack of social skill development have been commonly known as essential factors that limit occupational performance in clinical settings [6]. In particular, cognitive dysfunction is a contributing factor that impedes the performance of routine daily activities such as schoolwork, occupational activities, and social participation [7]. Such concepts highlight the involvement of cognitive evaluation as an important individual feature because of its influence on work performance and training process in the field of vocational rehabilitation.

A survey on the economic activity of persons with disabilities reported low employment rates for those who are disabled: 45.4% for individuals with physically disabled, 39.6% for individuals with blindness; 31.0% for people with hearing loss; 10.4% for persons with a brain lesion; 23.4% for those with a mental illness; and 23.4% for persons with an intellectual disorder [8]. It is important to highlight that employment rates for people with cognitive impairments such as a brain lesion or a mental or intellectual disorder are lower than those of disabled persons with other impairments in South Korea [4]. Cognitive function is considered a vital component for occupational execution and maintenance. In a previous study, job assessors and vocational teachers at a center for the disabled indicated that cognitive rehabilitation appeared to be determined by several key factors (listed in order of priority): situational awareness and judgment, problem-solving ability, the ability to transfer learning strategies to various situations, attention, and the ability to follow instructions [4]. The research indicates that optimal performance of vocational training requires well-organized executive functioning based on situational awareness and problem-solving ability.

To the best of our knowledge, there has been little attention paid to determining accurate evaluation approaches in vocational rehabilitation. In our country, especially, almost no studies include actual evaluation of cognitive impairments and effective intervention for recovery based on occupational performance. Although many professional providers of cognitive rehabilitation services have focused on the importance of specific training to enhance executive functions, vocational trainees rarely obtain adequate information about it in clinical settings. Accordingly, the aim of the present study was to illustrate the necessity of task-based cognitive tests in addition to routine clinical tests for cognitive function in order to establish optimal training programs in vocational rehabilitation.

Materials and methods

Participants

Eleven disabled people participating in a vocational training program at the Korea Employment Agency for the Disabled volunteered for this study (10 men and 1 woman; average age = 33.91 ± 10.78 years). Inclusion criteria were as follows: (1) no cognitive impairment [i.e., > 24 points on the Korean version of the Mini-Mental State Examination (MMSE-K)] [9]; (2) intact hand function; (3) sufficient capacity to engage in vocational training; (4) no orthopedic problems; and (5) agreement for participation of the study. Participants were classified into three disability types based on evaluation criteria from the Korea Research Institute for Vocational Education and Training: (1) four participants had a physical disability; (2) four participants had a brain lesion; and (3) three participants had a mental illness. Participants had been previously trained in machinery (3 participants), design (5 participants), and electronics (3 participants). Prior to the beginning of the study, a detailed description about the procedure and the aim of the study was provided to the participants, and a signed consent form for participation was obtained.

Measurements

The Rey Complex Figure Test (RCFT) was developed by Rey in 1944, and is widely used to investigate visuospatial perception, visual memory, and graphical reflection of executive function [10]. It is divided into two parts: graphic copy of the figure with the model (RCFT-C) and memory recall of the same figure (RCFT-R). The RCFT-R consists of immediate recall and delayed recall with 20-minute intervals, with a scoring range of 0–36 points each. This study used the scores of the copy and immediate recall tasks. The complexity and the abstract character of the figure (18.5 cm × 12.6 cm) are provided for scoring the test. For the scoring of the RCFT-C, participants were asked to copy the figures while looking at them. RCFT-R scores were determined by having participants draw the figures with their recall from immediate memory. The RCFT tests have been reported to be acceptable for clinical use [11-13]. Based on age-specific data obtained from previous studies [14], scores are classified as: above level (higher than the average of age-specific scores) and below level (lower than the age-specific average).

Based on Allen's cognitive level representing the Allen Cognitive Disabilities Model, the Allen Cognitive Level Screen (ACLS), developed by Allen in 1985, is a dynamic, task-based assessment of new learning and problem solving ability used to obtain a quick measure of global cognitive processing capabilities, learning potential, and performance abilities [15]. The level of cognitive function is demonstrated by performing three types of leather stitching tasks. The tool consists of one leather plate, two Perma-Lok needles, and two waxed leather strings. According to instructor guidelines, assessments were conducted by getting participants to perform a running stitch, whipstitch, and single cordovan stitch sequentially, with increasing difficulty level of lacing. Scores range between 3.0 points and 5.8 points, where higher scores indicate higher cognitive levels [16]. Scores can be used to illustrate Allen's cognitive level. Reliability for this test was sufficiently high for clinical use [17].

Procedures

The RCFT and ACLS were randomly performed to avoid biased results caused by learning effects from repeated testing and fatigue. Coin tosses were used for randomization. Testing procedures were performed by three professionals with > 8 years of experience in this field and who were blinded to the purpose of the study. Testers recorded the frequency of response errors and shape distortions observed during testing, in addition to evaluating the tests' scores.

Analytic methods

Data collected from the task-based tests (RCFT and ACLS) are reported below using descriptive statistics (e.g., mean and standard deviation). For qualitative analysis of the tests, response errors, shape distortions, and behavioral problems observed during testing were recorded for each participant. To maximize the benefits of visual analysis, scatter charts were created to identify individual data points from the tests and classify the level of each score. A matrix chart was developed to describe the response errors and shape distortions of each participant observed during testing.

Results

Results of the RCFT-C and -R, and the ACLS scores

Average scores on the RCFT-C were 33.64 ± 2.80 and 15.50 ± 9.01 on the RCFT-R (Table 1). In the scatter chart, the RCFT-C scores of four participants (36.36%) were below the age-specific average as were the RCFT-R scores of seven participants (63.64%), suggesting potential problems while performing occupational tasks (Figure 1). On the basis of the cognitive level, determined using the ACLS scores, three participants (72.73%) were Level 4 and eight (27.27%) were Level 5. The average score was 5.09 ± 0.43 (Table 1; Figure 2).

Table 1

Results of the task-based tests.

Tests	Sub-items	N (%)	Mean scores, Mean ± SD
RCFT-C	Above level	7 (63.64)	33.64 ± 2.80
	Below level	4 (36.36)
RCFT-R	Above level	4 (36.36)	15.50 ± 9.01
	Below level	7 (63.64)
ACLS	Level 4	3 (72.73)	5.09 ± 0.43
	Level 5	8 (27.27)

ACLS = Allen Cognitive Level Screen; RCFT-C: Rey Complex Figure Test-copy; RCFT-C = Rey Complex Figure Test-recall.

Figure 1

Scores of the Rey Complex Figure Test (RCFT)-copy (C) and -recall (R) of the participants. White and black marks indicate scores above and below the age-specific average, respectively.

Figure 2

Allen Cognitive Level Screen (ACLS) scores of the participants. Level 6: No cognitive impairment, Level 5: Mild cognitive impairment, and Level 4: Moderate cognitive impairment. Numbers indicate the ACLS scores of each participant. 5.8: Needs specific guidance for safety issues; 5.2: Unable to understand written instructions; 5.0: Unable to multi-task; 4.8: Pays attention to only one step of the task at a time; 4.6: Increased risk in using electronic equipment; and, 4.4: Unable to control own activities.

Qualitative analysis using the matrix chart

Based on the results of the RCFT and ACLS, response errors, distortions, and behavioral problems of each participant are summarized in the matrix chart (Figure 3). In the chart, transverse items indicate the response errors, distortions, and behavioral problems categorized from each test. Most participants had problems in more than four items. In particular, poor space recall of the RCFT was observed in all participants, and poor benefit from mistake and poor prediction of the ACLS were observed in nine participants. These findings imply that memory function and error correction may not be adequate during work performance. This may lead to a decrease in the degree of work completion.

Figure 3

Analysis matrix presenting the response errors, distortions, and behavioral problems observed while performing the Rey Complex Figure Test (RCFT and Allen Cognitive Level Screen (ACLS).

Discussion

Given that the final goal in rehabilitation of individuals with disabilities is to maximize their ability to maintain a life pattern without others' help, vocational rehabilitation should be considered an inevitable option for successful rehabilitation. The present study demonstrates the need for employing task-based cognitive tests. Our results support the practical use of task-based tests (e.g., RCFT and ACLS) that can provide additional benefits over routine clinical tests such as the MMSE-K in determining cognitive function.

Our study recruited vocational trainees with a normal range of cognitive function in the MMSE-K, a measure commonly used in the field of vocational rehabilitation. However, scores on the RCFT and ACLS in the present study suggest that potential problems may limit the learning process and work performance in participants. In addition, the scatter chart and analysis matrix results for the RCFT, which assesses problem-solving ability [11], and the ACLS, which evaluates task-based executive function [15], indicate that the MMSE-K may not detect all potential problems in performing occupational tasks such as response error, distortion, and behavioral problems. Again, as shown in the matrix, the use of the task-based cognitive tests (RCFT and ACLS) may be helpful for detecting response errors and behavioral problems objectively, even when cognitive function scores are in a normal range. The clinical features of the tests provide background knowledge for these results [13]. In particular, the RCFT and ACLS have been shown to be strongly correlated with the Wisconsin Card Classification Test, which examines executive function and problem-solving ability [18,19], and the task-oriented subtests of the Wechsler Adult Intelligence Scale, which includes arranging blocks, adjusting shapes, and adjusting order [20]. These previous findings support the use of task-based cognitive tests in determining the performance capacity for occupational tasks. Given that executive function and problem-solving ability directly influence occupational performance, our results suggest that the use of these tests is necessary for selecting trainees and creating training strategies in the field of vocational rehabilitation.

Kim and coworkers [21] reported a strong correlation between the MMSE-K and ACLS; however, our results indicate that the ACLS can provide insights that may not be obtained from the MMSE-K alone. Their study focused on the cognitive function of stroke patients with cognitive impairment; therefore, their results require the understanding of the relationship between the tests depending on personal cognitive function level. However, in populations that are classified with normal cognitive function (i.e., > 24 points on the MMSE-K [9]), possible problems in cognitive function such as response errors and behavioral problems may be better detected by using the ACLS, which requires executive function and problem-solving ability.

Generally, participants who were above Level 5 on the ACLS may be able to learn new skills requiring work performance, although they sometimes need to be assessed to determine whether they require monitoring for safety reasons and for adequate problem solving. However, participants at Level 4 may need daily assistance that involves monitoring their safety, removing hazardous objects in their environment, and establishing problem-solving strategies for new problems arising from any changes in the environment [16]. The outcome of the RCFT and ACLS showed that four participants may have some problems in executive functions and problem-solving abilities, leading to difficulty in performing occupational tasks independently and learning new skills for work [19]. As shown in the matrix chart, most participants had poor space recall and ability to learn from mistakes, which directly hinder performance of occupational tasks. Furthermore, poor predictive ability may reduce performance. Overall, results suggest that the use of the MMSE-K alone might be less effective for selecting trainees and determining appropriate training options based on vocational aptness.

Although this study offers useful information for vocational rehabilitation of individuals with disabilities, we acknowledge several limiting factors that could be improved with further studies. First, the small sample size may limit the generalizability of our results. Second, we did not consider individual characteristics of participants in the use of the tests; therefore, caution is needed in understanding our results. Finally, the fidelity of the participants to the tests may have had an influence on the results [22]. Participants involved in the present study were recruited at a vocational training center, and they all had functional homogeneity for the training despite different types of disability. Our results highlight the necessity of evaluating additional executive factors in the performance of occupational tasks in order to obtain valuable information to select suitable training fields of persons with disabilities. It should be an important option for choosing new trainees. Tests used in the selection process of trainees need to assess executive function and problem-solving ability, and thus the use of the RCFT and ACLS may be more helpful when evaluating individuals with disabilities.

Accurate cognitive evaluation of individuals with disabilities is a prerequisite to match an appropriate training field in vocational rehabilitation. The present study aimed to demonstrate the importance of using task-based cognitive tests in identifying executive factors necessary for performing occupational tasks in the field of vocational rehabilitation. Qualitative analysis revealed the possibility of response errors and behavioral problems, which were not found in the results of routine clinical cognitive function tests. These findings suggest that we need to include task-based cognitive tests when selecting training fields for vocational rehabilitation of people with disabilities, particularly when the diversity of individual cognition levels and learning levels in occupational performance is considered. During testing, participation of professionals with higher specialty in this field may be valuable for the decision-making process for vocational aptness of the disabled. Further studies will be needed.

Conflicts of interest

All authors declare no conflicts of interest.