The paced auditory serial addition test for working memory assessment: Psychometric properties.

Background: The paced auditory serial addition test (PASAT) was primarily developed to assess the effects of traumatic brain injury on cognitive functioning. Working memory (WM) is one of the most important aspects of cognitive function, and WM impairment is one of the clinically remarkable signs of aphasia. To develop the Persian version of PASAT, an initial version was used in individuals with aphasia (IWA).
Methods: In this study, 25 individuals with aphasia (29-60 years) and 85 controls (18-60 years) were included. PASAT was presented in the form of recorded 61 single-digit numbers (1 to 9). The participants repeatedly added the 2 recent digits. The psychometric properties of PASAT including convergent validity (using the digit memory span tasks), divergent validity (using results in the control group and IWA group), and face validity were investigated. Test-retest reliability was considered as well.
Results: The relationship between the PASAT and digit memory span tests was moderate to strong in the control group (forward digit memory span test: r= 0.52, p< 0.0001; backward digit memory span test: r = 0.48, p< 0.0001). A strong relationship was found in IWA (forward digit memory span test: r= 0.72, p< 0.0001; backward digit memory span test: r= 0.53, p= 0.006). Also, strong testretest reliability (intraclass correlation= 0.95, p< 0.0001) was observed.
Conclusion: According to our results, the PASAT is a valid and reliable test to assess working memory, particularly in IWA. It could be used as a feasible tool for clinical and research applications.

↑ What is “already known” in this topic:

The paced auditory serial addition test (PASAT) is a wellknown tool to examine cognitive processing, in particular attention and working memory (WM), in different neurological dysfunctions.

→ What this article adds:

The Persian version of PASAT is a valid and reliable tool to assess WM, particularly in individuals with aphasia.

Introduction

Working memory (WM) is a brain function that obtains necessary representations of high-level cognitive tasks and daily living activities. This cognitive system is used for manipulation and temporary information storage (1-3). The WM system stays active and relevant for a short period of time (4, 5) and keeps stimuli available in the absence of external cues (6). The Baddeley and Hitch (3) WM model proposed a speech- or articulation-based format for WM and a format for representing and maintaining visual/spatial information for temporary storage and manipulation using these 2 subsystems. Subsequently, a multidimensional cache was added to this model; and using this new part, phonological and visuospatial information was connected (1).

WM can be adversely affected in various situations including neurological and psychological disorders as well as aging (7). Aphasia is one of the neurological disorders that impair WM capacity and function (8). Two WM components, namely, the phonological loop and central executive system are impaired in aphasia (9-11). The phonological loop, which is of particular interest in aphasiology, is responsible for rehearsing verbal information and recycling it to refresh its memory traces (12). Cognitive science suspects that the phonological loop is important, and even necessary, for the development of language (13). The central executive system controls the WM system and is responsible for focusing, dividing, and switching attention. Also, this system activates information for long-term memory (2). WM limitations have negative effects on the ability to make decisions on aspects of rehabilitation in individuals with aphasia (IWA) (14).

In the last 4 decades, numerous neuropsychological assessments have been designed to evaluate WM (15-17). The paced auditory serial-addition test (PASAT) was developed in 1974 to evaluate the speed of information processing in patients with traumatic brain injury (TBI) (18). Thereafter, PASAT has been used to investigate cognitive processing including attention and WM in different neurological dysfunctions (19-21). PASAT imposes an extensive load on WM capacity. It has been known as a useful tool for WM assessment (20, 22). When an examinee performs the PASAT, s/he attempts to provide a response after every stimulus; the cognitive demands involved in working memory include active maintenance and control of task-relevant cognitive operations (23).

During PASAT administration, a series of single-digit numbers (1 to 9) are presented randomly, and the participant repeatedly sums the 2 recent digits. For example, if the digits are 3 and 5, the participant would sum them and respond with 8. When the next digit is presented, for example, 2, the participant would respond with the correct sum of the 2 most recent digits, ie, 7. Several versions of PASAT have been developed including versions that apply different interstimulus intervals (ISI) (24, 25). Also, some versions utilize visual modality, while the others employ auditory modality (26). PASAT's psychometric properties have been investigated in previous studies. Gronwall’s study (18), which is one of the first studies to evaluate PASAT normative test data, was conducted on adults. Afterward, normative data were presented for the different versions of the PASAT (23, 27-32) by study of different variables such as age and IQ (23, 27), or age and education (28). Furthermore, a computerized version of the PASAT has been developed (33); its psychometric characteristics were reported in English as well as other languages such as French (34), but these data were not reported in Farsi. Moreover, psychometric properties were reported on individuals with different disorders such as multiple sclerosis (MS) (35, 36) and TBI (22, 37).

WM intact function is crucial in different aspects of daily life. Therefore, using objective tools for WM assessment is an essential part of clinical practice. The theoretical framework of cognitive impairments in IWA has been identified in the literature (38). Limitations in inhibition, blending, and speed of information processing have been suggested in many theories; these limitations could lead to limited information resources or weakness in allocating existing information resources (39). However, the details of language-information processing could be different among IWA (40).

Despite WM impairments in IWA (8) and capability of PASAT in identifying WM impairment, no study has investigated WM in IWA using PASAT. The present study was the first to take this step. Prevalence of stroke and subsequent aphasia could lead to WM impairment; considering the effects of these impairments on the communicative skills (41), examining the psychometric properties of PASAT in IWA comprises an integral part of WM assessment and management (42). Therefore, the present study aimed at investigating the psychometric properties of the PASAT Persian version and its ability to assess WM in IWA.

Methods

Participants

A total of 110 participants aged 18 and 30 (23.6±3.51 years) years were placed in 3 groups and examined in the present study; of them, 70 (35 females) were in the control group. The participants reported no history of neurological disorders; 25 (8 female) IWA aged 18 to 60 years (49.52±9.65 years) participated in the present study (43). Inclusion criteria for IWA were restricted to mild and moderate aphasia to avoid heterogeneity in the language profile. Those IWA with damaged left hemisphere due to ischemic stroke documented by MRI or CT, were considered. Participants with Broca aphasia, a history of 6 to 40 months postonset time and without hemiplegia were considered. Exclusion criteria included history of other neurological disorders, severe naming disorder (44), severe impairment in auditory perception using the Persian version of Western aphasia battery (at least 6 out of 10 scores (43)), severe apraxia using Persian apraxia test (45), and severe dysarthria that limited speech articulation ability (46). Arithmetic abilities were also assessed, then, those IWA who had deficits in performing arithmetic were excluded (47). Good hearing sensitivity for speech communication was required. Furthermore, 15 control participants (5 females) aged 18 to 60 years (49.73±6.74 years) without aphasia were matched according to age, sex, and educational level to the IWA group and included in the study. This last group was included in the study to determine the discriminate validity of the PASAT. All participants were right-handed, monolingual, and native speakers of the Persian language. The least educational level was high school diploma (48). The ethical principles of the Declaration of Helsinki were followed throughout the study. Moreover, the ethics committee of Iran University of Medical Sciences (IUMS) approved the study protocol (ethical code: #93/D/105/5226). Participants were completely aware of the study’s content and provided an informed consent before taking the tests.

PASAT design: Our Persian PASAT version consisted of a set of 61 single-digit numbers presented verbally to the participants. To investigate WM, a 3-second interstimulus interval (ISI) (21, 49) was used on the control group based on a pilot study.

Psychometric properties

The psychometric properties of the PASAT including face validity, construct validity, and reliability were investigated. Convergent validity and divergent validity were used to assess the construct validity of the measurement procedure. To determine convergent validity, the correlations between PASAT and both forward and backward digit memory span test (DMST) (50) were determined. DMSTs have previously been validated in the Persian language. The DMSTs are subtests of the revised version of the Wechsler adult intelligence scale (WAIS-R) (51) that have been used in many aphasia studies (52-54). Comparison between the results of PASAT in IWA and the control group was used to obtain divergent validity. The same rater performed the PASAT for 20 control participants on 2 separate occasions to determine reliability. There were 3-week intervals between the test and retest (55). To assess test-retest reliability, the intraclass correlation coefficient (ICC) was applied as a reliability coefficient from zero to 1 based on the analysis of the variance (56). The ICC expresses the proportion of variance in an observation due to between-subject variability in the true scores (57). Values above 0.80 are considered the evidence of excellent reliability (58).

Tests

To assess WM, forward and backward DMST (50) as well as PASAT were examined. Participants were instructed to practice on a short-form of PASAT to be familiarized with the test. Participants received instruction on the PASAT until they understood the task and completed the practice items successfully (33).

Procedure

PASAT: Recorded high-quality auditory stimuli were presented via an external speaker at the most comfortable level (MCL) for the listener in PASAT administration. During the presentation, participants summed the 2 last digits recited and provided their response. To prevent the effect of any possible naming disorders or speech intelligibility problems on the results among IWA, digits (2 to 18) were printed on a piece of paper in a random order. Participants were instructed to respond to the PASAT by pointing to the printed digits instead of responding verbally (59). The scoring method was based on the total number of correct responses during test administration. Participant received a score of 1 for every correct response, and a score of 0 for every incorrect response. The final score was the total number of the correct responses. If the participants could not respond to 5 stimuli continuously, the test was terminated.

Forward digit memory span test (50): The examiner provided each participant with instruction explaining that the numbers would be spoken 1 second apart by the examiner and that the participants should repeat them in the same order in which they were presented. Increasing the test stimuli during each series elevated test complexity. The forward DMST began with 3 numbers and progressed to 8. The participant had 2 opportunities to recall the numbers at each test level. If the correct answer was not obtained in either of the 2 opportunities, the test was stopped at that level.

Backward digit memory span test (50): The test was administered under the same conditions and instructions as the forward DMST. The only difference was that the participant had to repeat each series of numbers in reverse from the last number to the first one. Corresponding to the forward DMST, the backward DMST complexity was enhanced by increasing the number of test digits. Initially, 2 numbers were presented and increased to a maximum of 7 numbers. The same administration and scoring methods were followed in DMSTs. The participants were scored based on the numbers of correct responses (maximum score was 12).

IWA were asked to respond by pointing to printed digits on paper. So, verbal errors due to naming or fluency problems were avoided in DMSTs.

At the end of the performance, to assess the face validity of the PASAT, we asked the participants to respond to some questions.

Statistical analysis

The normality assumption of the variables was investigated using the Kolmogorov–Smirnov test. To compare the results of the 2 control groups (aged 18 to 30 and 18 to 60 years), independent t test was used. A power analysis was also performed. All statistical analyses were conducted using SPSS Version 22.0; and p<0.05 was considered as statistically significant.

Results

Normality assumption was held for the control (p= 0.74) and IWA (p = 0.061) groups. The descriptive statistics for the PASAT, forward DMST, and backward DMST are presented in Table 1.

Table 1

Descriptive Statistics of PASAT, Forward and Backward DMSTs in the Three Groups

Measure	Groups	N	Mean	SD	Min	Max
PASAT	Control, 18-30 years	70	53.76	4.94	41	60
	Control, 18-60 years	15	54.27	4.81	45	60
	Aphasia, 18-60 years	25	10.64	13.14	0	37
Forward DMST	Control, 18-30 years	70	8.01	1.74	5	12
	Control, 18-60 years	15	8.13	1.6	5	11
	Aphasia, 18-60 years	25	3.92	1.41	2	7
Backward DMST	Control, 18-30 years	70	7.41	1.58	4	11
	Control, 18-60 years	15	7.60	1.63	5	11
	Aphasia, 18-60 years	25	2.95	1	1	5

DMST: digit memory span test, PASAT: Paced Auditory Serial-Addition Test, SD: standard deviation

Validity

Convergent evidence: The correlation between the PASAT and DMST scores were determined. For clinical significance with the elucidation of correlational investigations, a cut-off of 0.30 was utilized; ie, only relationships of roughly 0.30 or larger were considered evidence for a relationship between variables (22). The correlational effect size was characterized by Cohen’s criteria (60), where r = 0.10, r = 0.30 and r = 0.50 are considered to be small, medium, and large correlations, respectively. The correlations between the PASAT and both the forward and backward DMSTs in the IWA and unmatched control groups were significant (Table 2).

Table 2

Correlation between the PASAT and DMST Scores in IWA and Unmatched Control Group

Groups	Tests	n	r	p
Control, 18-30 years	PASAT and forward DMST	70	0.52	< 0.001
	PASAT and backward DMST	70	0.48	< 0.001
Aphasia, 18-60 years	PASAT and forward DMST	25	0.72	< 0.001
	PASAT and backward DMST	25	0.52	0.006

DMST: digit memory span test, IWA: individual with aphasia, PASAT: Paced Auditory Serial-Addition Test

Divergent evidence: Comparisons between the control and IWA groups revealed significant differences in all the test results (Table 3).

Table 3

Comparison between IWA and Matched Controls in PASAT and Forward and Backward DMSTs

Tests	p	Partial η 2	Power
PASAT	p< 0.001	0.84	1.000
Forward DMST	p< 0.001	0.85	1.000
Backward DMST	p< 0.001	0.91	1.000

DMST: digit memory span test, IWA: individual with aphasia, PASAT: Paced Auditory Serial-Addition Test, Partial η2: effect size estimate

Reliability: The test-retest means and ICC results for the PASAT, forward DMST, and backward DMST are summarized in Table 4.

Table 4

Reliability of PASAT, Forward and Backward DMST in Control Group

Measure	Test (mean)	Retest (mean)	ICC (lower bound-upper bound)
PASAT	53.35	54.50	0.95 (0.84 - 0.98)
Forward DMST	8.25	8.70	0.81 (0.52 - 0.93)
Backward DMST	7.85	8.35	0.91 (0.78 - 0.97)

DMST: digit memory span test, ICC: intra-class correlation coefficient, PASAT: Paced Auditory Serial-Addition Test

Discussion

Psychometric properties of the PASAT in Persian were established based on the data from 85 controls without aphasia and 25 IWA. The PASAT has been determined to measure attention (61, 62), and WM (63). Importance of cognitive functions, especially WM, in successful completion of the PASAT was suggested by previous studies (64). In previous studies, DMSTs were used as WM assessment, so the relationship between PASAT and DMSTs would show the validity of the PASAT as a WM assessment tool. In the present study, a strong relationship between PASAT and the forward DMST and a moderate relationship between PASAT and backward DMST were found in the control group. Also, this study showed strong relationships between these variables in IWA. Moderate to high relationship between the backward DMST and the total DMST score (backward DMST plus forward DMST) (21) as well as a moderate relationship between PASAT and WAIS-R were observed in previous studies (65). So, our findings confirmed those of the previous studies. WM contains several functions; digit memory span tests are designed to measure some WM aspects; and the forward DMST assesses verbal short-term memory (66). Components are updated during information encoding through forward DMST (67). On the other hand, the backward DMST involves manipulation by reordering components. The backward DMST was used to evaluate other WM tests because of its ability to assess the functions of encoding, storing, manipulating, organising, and recalling information from WM (16). Therefore, the DMSTs contain switching and manipulation components (67). With regards to the relationships between the PASAT and the forward and backward DMSTs, evidence for the efficacy of the Persian version of the PASAT in WM assessment was shown in the present study. The moderate relationship between PASAT and backward DMST in controls was due to the possibility of different processes required in PASAT and DMST. Difficult tasks elicit different brain resources. The PASAT is a difficult task, thus, the participants might use other brain processes to respond correctly. Also, it is claimed that during the PASAT administration, other resources such as attention, general intellectual ability, and arithmetic ability might be involved (22). Therefore, these resources might lead to better results in the PASAT compared to the backward DMST. However, further studies are required to elucidate the potential resources involved in the PASAT administration.

In case of divergent validity, significantly lower score of IWA than the control group reflected that the PASAT presumably has a high capability to distinguish IWA from a matched control group. This was the first study to use PASAT to evaluate WM in aphasia. Future studies could provide more evidence concerning PASAT advantages and limitations for WM assessment in aphasia and other language disorders.

With respect to reliability, ICC is the most commonly reported reliability measure in the literature. Also, ICC provides information about the measure’s ability to differentiate among individuals. This technique is most appropriate for investigating differences between groups of patients (57). In the present study, ICC was higher than 0.80 in the PASAT and both forward and backward DMSTs. This strong reliability indicates that PASAT could provide the same reliable results as forward and backward DMSTs. Our findings confirmed previous reports of PASAT’s reliability; ie, an ICC range of 0.76 and 0.95 was shown in previous studies (22, 68). In addition, the test-retest coefficients generally fell in the 0.90 to 0.97 range (69, 70). In our study, PASAT showed good face validity as a WM test based on the participants’ opinions. No information regarding PASAT face validity has been collected in the past studies.

To examine different types of validity, the use of a variation of WM tests is suggested for future studies. We used same ISI for presenting PASAT stimuli in all groups. Investigating the effect of ISI on the response of IWA could be an interesting subject for future studies. Aphasia group was not separated based on the type and severity due to small sample size. Such effects could be addressed in the next studies as well. In the present study, total correct (TC) scoring was applied to compare the effect of different scoring methods (ie, based on errors and no responses). Moreover, measurement of reaction time might provide further information in the future studies.

Conclusion

Overall, the Persian version of PASAT can evaluate WM with high reliability and moderate validity. It could also be a suitable clinical application for WM assessment in aphasia and can be used in both clinical and research settings. Examining other aspects of PASAT in aphasia is suggested in future studies.

Acknowledgments

This study was part of a Ph.D. dissertation project in speech and language pathology (SLP), which was approved by Iran University of Medical Sciences (Grant #93/D/320/2691). The cooperation of all participants in this study is appreciated.

Conflict of Interests

The authors declare that they have no competing interests.