Developing the Persian version of the homophone meaning generation test.

BACKGROUND: Finding the right word is a necessity in communication, and its evaluation has always been a challenging clinical issue, suggesting the need for valid and reliable measurements. The Homophone Meaning Generation Test (HMGT) can measure the ability to switch between verbal concepts, which is required in word retrieval. The purpose of this study was to adapt and validate the Persian version of the HMGT.
METHODS: The first phase involved the adaptation of the HMGT to the Persian language. The second phase concerned the psychometric testing. The word-finding performance was assessed in 90 Persian-speaking healthy individuals (20-50 year old; 45 males and 45 females) through three naming tasks: Semantic Fluency, Phonemic Fluency, and Homophone Meaning Generation Test. The participants had no history of neurological or psychiatric diseases, alcohol abuse, severe depression, or history of speech, language, or learning problems.
RESULTS: The internal consistency coefficient was larger than 0.8 for all the items with a total Cronbach's alpha of 0.80. Interrater and intrarater reliability were also excellent. The validity of all items was above 0.77, and the content validity index (0.99) was appropriate. The Persian HMGT had strong convergent validity with semantic and phonemic switching and adequate divergent validity with semantic and phonemic clustering.
CONCLUSION: The Persian version of the Homophone Meaning Generation Test is an appropriate, valid, and reliable test to evaluate the ability to switch between verbal concepts in the assessment of word-finding performance.

Introduction

Word retrieval is an ability composed of two components: Lexical knowledge, and a search process within existing lexical knowledge (1). Word-finding is usually evaluated through picture naming tasks such as Peabody Picture Vocabulary Test (2), in which each picture corresponds to one word. In picture naming tasks, examinees are supposed to search in their brain to identify the required lexical item; and when it is identified, the search is finished. Therefore, this is a type of mental activity which mostly addresses word knowledge, but executive search strategies receive less attention (1).

Verbal fluency is another test which is used frequently in neuropsychological assessments (3). It has two subtests, semantic fluency and phonemic fluency, in which the participant is asked to generate as many words as possible in 60 seconds. The words are within a semantic category in the semantic fluency subtest, or start with a specific letter in the phonemic fluency subtest (4). Unlike picture naming tests, verbal fluency covers both the lexical storage and the ability to access word storage by its two components which are known as clustering and switching (1).

Clustering represents the production of words within the semantic or phonemic subcategories, and switching reflects the ability to shift between clusters (1,3).

Verbal fluency test with its qualitative evaluation method (clustering and switching) was adapted to the Persian language by Ebrahimipour et al. in 2008. This test was used to evaluate clustering and switching in the semantic and phonemic fluency performances in patients with multiple sclerosis (MS), Parkinson’s disease (PD), and amyotrophic lateral sclerosis (ALS). The psychometric properties of the Persian verbal fluency test were also determined and normalized for Persian-speaking children and adults aged 7-69 years. In short, the Persian Verbal Fluency Test is a valid and reliable instrument which can reveal verbal fluency deficits in patients with MS, PD, and ALS (5-8).

Homophone meaning generation is a test assessing retrieval flexibility. In this test, the examinee is asked to generate multiple meanings for each homophone. The English version of this test includes eight homophones: Tick, tip, slip, form, plain, bored, right, and sent (9,10). It seems that HMGT requires more searching strategies than the previous two tests (i.e., picture naming and verbal fluency) since after the first retrieval attempt, the examinee needs to continue to search using more complicated techniques (1,9,10). Warrington (2000) suggests that HMGT measures the ability to switch between alternative verbal concepts, and patients with anterior brain lesions were found to be more impaired on the HMGT than patients with posterior lesions (9).

There is also a Hebrew language version of HMGT. Kavé et al. (2007) translated and adapted this test to the Hebrew language and she also studied the correlation between HMGT and verbal fluency. She maintained that HMGT is highly correlated with the total score, switching, and clustering in semantic and phonemic fluency tasks except phonemic cluster size (11). One year later, Castner et al. used the English version of HMGT to evaluate homophone meaning generation in patients with Parkinson’s disease. They proposed that these patients had fewer scores on HMGT than healthy controls (12). Moreover, Kavé et al. (2010) evaluated word finding abilities in 207 normally developing Hebrew-speaking children aged 8-17 using picture naming, phonemic and semantic fluency, and HMGT. She found no sex effect, but found a positive correlation between age and these three tests. This correlation was weakest for the naming test and strongest for the HMGT. Therefore, the more executive demands required by a test, the steeper the slope of performance increase on this test (1).

The evaluation of word finding performance and searching strategies to find a proper word requires an appropriate instrument. This instrument should be equally natural and acceptable in the target language and culture. The main purpose of this study was to provide a valid and reliable tool for clinicians to evaluate homophone meaning generation in Persian-speaking adults. Therefore, the focus of this study was on adapting HMGT to Persian language and ensuring the adequacy of the test in the Persian-speaking population. The influence of demographic characteristics on the Homophone Meaning Generation performance in normal Persian-speaking adults was also examined.

Methods

Participants

Ninety Persian-speaking individuals participated in this study. All participants (45 males and 45 females) were literate, aged 20-50 years, right-handed and acquired Persian as their first language. The mean age of the participants was 34.30 years with a standard deviation of 7.98 with the mean (SD) years of formal education of 13.73 years (±3.27). Demographic information is presented in Table 1. In this table, participants were divided into six groups to illustrate the sample distribution based on age and formal education.

Table 1

Demographic Information of the Two Groups

Participants	Age (Years)
	20-30	31-40	41-50
Education ≤12 Years	(n=11)12.22%	(n=14)15.55%	(n=12)13.33%
Education >12 Years	(n=20)22.22%	(n=17)18.88%	(n=16)17.77%

Exclusion criteria were as follows: History of head trauma in the last 10 years, stroke, and epilepsy, history of other neurologic or psychiatric diseases, alcohol abuse, severe depression and history of speech, language, or learning problems.

The Persian version of the Beck Depression Inventory-II (BDI-II-Persian) was used to screen depressed participants. Participants with the score of 29-63 in BDI-II-Persian were labeled as severely depressed and were excluded from the study (13,14).

Procedure

The Persian version of the verbal fluency test was administrated to examine the construct validity (convergent and divergent validity) of the Persian version of HMGT. Convergent and divergent validities were estimated using correlation coefficients. The verbal fluency test was composed of semantic fluency and phonemic fluency subtests. The semantic fluency subtest had two tasks (i.e., categories), and the phonemic fluency subtest had three (i.e., categories). In the semantic fluency subtest, participants were asked to say as many words as possible from the animal and fruit categories; and in the phonemic fluency subtest, they had to say as many words as possible that begin with the letters (/s/ ‏س ,/a/ ‏آ, /f/ ‏ف).

The time limit for each task was 60 seconds. The following scores were obtained in the phonemic and semantic fluency tests: (1) Number of words produced, excluding errors and repetitions; (2) Mean cluster size, and (3) Number of switches. Detailed rules for scoring cluster size and switches for both fluency tasks have been provided in Troyer's article published in 1997 (3,6).

In addition to verbal fluency test, the participants were evaluated by the homophone meaning generation test. For this purpose, the HMGT was adapted to the Persian language. First, all Persian homophones, each having at least three possible meanings (5), were extracted from the Dictionary of Contemporary Persian (15). Then 16 homophones were selected based on their frequency, and evaluation was made by four professional experts (PhD linguists). The 16 selected homophones were ‏گاز /gaz/, ‏دوش /duŝ/, ‏سیر /sir/, ‏تن /ton/, ‏پارتی /parti/, ‏پاس /pas/, ‏شور /ŝur/, ‏ترک /tærk/, ‏وجه /væǰh/, ‏مینا /mina/, ‏بار /bar/, ‏باز /baz/, ‏تیر /tir/, ‏بوم /bum/, ‏هوا /hæva/, and ‏قرار /qærar/. The possible number of meanings for the homophones ranged from 3 to 8 with the mean (SD) of 5.31 (1.77). The selected homophones were randomly mixed in one list, and the same list was administrated to all participants. Each examinee was tested individually, and all responses were written verbatim. There was no time constraint to proceed to the next stimulus word. Every distinct meaning was given one point, and the sum of distinct meaning generated for 16 homophones was considered the total test score.

Statistical Analysis

The reliability of this test was studied through test-retest, inter-raters, internal consistency, and split-half analysis. The test-retest interval was within 14 days (in a subsample of 30 subjects). To evaluate the inter-rater reliability, the Persian HMGT was scored the same day by two examiners whose degrees of expertise differed. Both were speech-language pathologists, one with a PhD degree and the other with a Bachelor’s degree. In the split-half analysis, all items were randomly divided into two sets of homophones. Each half was labeled, A and B, and then were given to different examinees in different orders. Moreover, Content validity and construct validity of Persian HMGT was determined. An acceptable CVR (Content Validity Ratio) value for eight professional experts (four speech-language pathologists and four PhD linguists) was 0.7 (16); and the meaningful Alpha level was considered 0.8 and higher.

Finally, the influence of sex, age and education on the score of Persian HMGT was analyzed. The mean and standard deviation were calculated for each item. The effect of sex on the Persian HMGT was examined using independent t-test and the effect of age and education on the Persian HMGT was studied by the Pearson correlation coefficients. SPSS v.20 was used for data processing.

This survey was conducted according to the ethical rules of the National Public Health Institute, and the investigations were performed in accordance with the Declaration of Helsinki. Moreover, this study was approved by The Ethics Committee of Iran University of Medical Sciences. Also, it should be noted that informed consent was obtained from the participants and they were ensured about the anonymity of their information.

Results

A Kolmogorov- Smirnov test applied to the distribution of HMGT scores revealed that HMGT scores were normally distributed (p= 0.52). Among the homophones, the participants produced the least number of different meanings for the homophone ‏ترک / tærk/, and the most number of different meanings for the homophone ‏باز /baz/. In other words,‏ترک / tærk/ was the most difficult, and ‏باز / baz/ was the easiest homophone. None of the homophones were left unanswered. In total, the mean (SD) of the HMGT score in the 90 examinees was 37.72 (5.90). Table 2 illustrates the number of possible meanings and the mean number of the produced meanings by the participants in each Persian homophone.

Table 2

The Number of Possible Meanings per Target on the Persian HMGT and the Mean Number of Distinct Meanings Generated Per Target across Healthy Controls

Target	No. of Possible Meanings	Mean no. of Distinct Meanings Generated
‏گاز / gaz/	4	2.55
‏دوش / duŝ/	4	2.73
‏سیر / sir/	4	2.34
‏تن / ton/	3	2.48
‏پارتی / parti/	3	1.95
‏پاس / pas/	8	2.81
‏شور / ŝur/	7	2.34
‏ترک / tærk/	3	1.61
‏وجه / væǰh/	6	1.67
‏مینا / mina/	4	2.38
‏بار / bar/	8	2.27
‏باز / baz/	6	2.83
‏تیر / tir/	7	2.75
‏بوم / bum/	5	2.65
‏هوا / hæva/	7	1.97
‏قرار / qærar/	6	2.31

Reliability

The internal consistency of scores on the HMGT was examined using Cronbach’s Coefficient Alpha (α) which was 0.80 for the items designed to assess the ability to generate meanings of the homophones. Inter-item correlation indicated a positive relationship between items (r=0.22-0.57). Moreover, when each homophone was excluded from the analysis, the alpha reliability of the scale was not improved. Table 3 displays item-total correlation coefficients (correlation between each item and total score), and alpha values when each item was excluded.

Table 3

Item-Total correlation coefficients and Cronbach’s alpha when each item was excluded

Items	Item-total correlation	Cronbach’s alpha if item was excluded
گاز / gaz/	0.33	0.80
دوش / duŝ/	0.36	0.80
سیر / sir/	0.22	0.80
تن / ton/	0.36	0.80
پارتی / parti/	0.43	0.79
پاس / pas/	0.53	0.78
شور / ŝur/	0.45	0.79
ترک / tærk/	0.43	0.79
وجه / væǰh/	0.36	0.80
مینا / mina/	0.35	0.80
بار / bar/	0.55	0.78
باز / baz/	0.46	0.79
تیر / tir/	0.32	0.80
بوم / bum/	0.57	0.78
ھوا / hæva/	0.42	0.79
قرار / qærar/	0.32	0.80

The test-retest analysis indicated high reliability (α=0.98, r=0.97). Furthermore, there was a considerable agreement between the two scorers on the HMGT total score (intra class correlation coefficient=0.99, standard error of measurement=1.71). Reliability was further demonstrated through a Guttman split-half reliability coefficient of 0.75. The items in the form of A were /gaz/, /duŝ/, /sir/, /ton/, /parti/, /pas/, /ŝur/, and /tærk/, and form B composed of /væǰh/, /mina/, /bar/, /baz/, /tir/, /bum/, /hæva/, and /qærar/.

Validity

In addition to reliability, validity of HMGT test was also examined. CVR of the items were all above the cut-off value presented in Lawshe Table (0.75); and therefore, all items were retained for the next stage. In addition to the CVR, CVI (Content Validity Index) was calculated. The results revealed that all the items were accepted since the CVI values were greater than 0.79.

In order to examine the construct validity, convergent and divergent validity were analyzed. In other words, the correlations of HMGT with semantic and phonemic total score, clustering and switching were examined. This correlation was significant for the total score and switches in both semantic and phonemic fluency tests, but not for semantic or phonemic clustering at the 0.01 level as presented in Tables 4 and 5.

Table 4

Correlation of HMGT with the Semantic Fluency Scores

	Semantic Fluency	Semantic Switching	Semantic Clustering
HMGT	R=0.54	R=0.36	R=0.22
	p<0.001	p<0.001	p=0.03

Table 5

Correlation of HMGT with the Phonemic Fluency Scores

	Phonemic Fluency	Phonemic Switching	Phonemic Clustering
HMGT	R=0.65	R=0.58	R=0.05
	p<0.001	p<0.001	p=0.44

Effects of Demographic Characteristics

Sex had no significant effect on HMGT scores (p=0.59). There was also no significant correlation between age and performance on the homophone generation test (r=0.00, p=0.99). However, the correlation between HMGT score and level of education was highly significant (r=0.28, p=0.006).

Summary of Measurement Characteristics of the Persian HMGT

The reliability of the Persian HMGT was estimated using Cronbach’s alpha and split-half method. The alpha value obtained (0.80) indicates that the Persian HMGT has an acceptable level of reliability. Furthermore, inter-rater and intra-rater reliability was also high. These results are consistent with those of the study conducted by Crawford and Warrington (10).

Furthermore, content and construct validity of the Persian HMGT were determined in this study, and it can be concluded that it is a highly valid test like its English version (10). Three demographic characteristics were also controlled and studied in the Persian HMGT. Sex and age had no effect on this test; however, education was positively correlated with participant’s performance in this test. These findings are consistent with the previous study conducted by the test developer (9).

Discussion

The first aim of this study was to adapt English HMGT to Persian version and to establish the reliability and validity of this instrument. Then the nature of the test and its required strategies were investigated. Generally, there was a positive correlation between HMGT and semantic and phonemic fluency tests in a group of Persian-speakers. As a result, some similarities between underlying processes in these three tests are predictable.

HMGT was originally designed by Warrington (2000) to evaluate executive function as a measure of frontal lobe dysfunction (9). It is hypothesized that HMGT requires two processes: Directed search and flexibility, and manipulation of various conceptual representations (9-11). In fact, finding different meanings of a homophone requires searching within and between existing conceptual categories (11).

Based on previous studies, semantic fluency is highly dependent on lexical knowledge, and phonemic fluency is deeply affected by mental flexibility (17). These findings can unfold this significant correlation between HMGT and fluency tests since HMGT requires both lexical knowledge (like semantic fluency) and mental flexibility (like phonemic fluency). Moreover, all these three tests are sensitive to frontal lobe dysfunction, making this correlation even more sensible and predictable (9,18).

Qualitative analysis of fluency tests further clarifies the underlying word finding processes in fluency tests. Troyer posits that clustering reflects the semantic organization in the brain, and switching is an executive function, including strategic search, shifting, and mental flexibility (3,19). Our findings revealed that clustering and switching scores in both fluency tasks correlate positively with HMGT scores. However, HMGT score is more highly related to the number of switches than to the mean cluster size across the two fluency tasks. The present results are consistent with the study conducted by Kavé et al. in 2007. She and her colleagues found the same pattern except in correlation between HMGT and phonemic mean cluster size which was not correlated with the HMGT at all. Similarly, this correlation in the present study is too weak that it can hardly be considered positive (r=0.05, p=0.44) (11).

Switching between semantic and phonemic subcategories requires strategic search, shifting, and mental flexibility. Thus, the correlation between switching and homophone meaning generation is reasonable since both of them are considered an executive function (3,9,19,20). However, the correlation between the HMGT and the semantic and phonemic cluster size was smaller than the correlations with the number of switches especially for phonemic clusters. Phonemic clusters are made based on similarities of sounds, while semantic clusters manifest organization of knowledge into conceptual categories. This semantic organization is also required in homophone meaning generation, but the similarities of sounds cannot help meaning generation (3,9,19,20). Consequently, HMGT was more correlated with semantic clustering than phonemic clustering. This finding is in agreement with that of Kavé et al. (2007) study which showed that the shared conceptual representation component is necessary for successful performance on both semantic clustering and homophone meaning generation (11). However, this correlation is relatively low (r=0.22; p=0.03); and this may be due to the difference between semantic clustering and homophone meaning generation.

A semantic cluster is composed of words belonging to a similar conceptual subcategory (3), while Persian homophones’ meanings are mostly not semantically related. For instance, the Persian homophone‏شیر / ŝir/ means lion, water taps, and milk. This does not apply to English homophones since many of them may have semantically related meanings. As a case in point, one word can be both a noun and a verb. As a result, it seems that semantic processes required in semantic clustering are not completely the same as the activation of semantic representations in HMGT in the Persian language.

This study suffers from all of the limitations inherent in any nonrandomized sampling. In this context, the sample size of the data set available was not large enough to permit the normalization process. These troublesome methodological issues remain to be dealt with in future researches.

Conclusion

In conclusion, this study proposes that Persian HMGT is a valid and reliable instrument which shares many underlying mental processes with verbal fluency tests. HMGT is an easy test to score, user-friendly and efficient in assessing word finding performance, especially lexical search and manipulation of various conceptual representations with no time constraints. Based on these evidences, we suggest that the Persian HMGT be used for language evaluation in clinical settings and research studies.

Acknowledgment

This study was supported by Iran University of Medical Sciences. We would like to acknowledge Mr. Payam Mihanparast for his help with data collection and compilation. We also would like to thank volunteers for their involvement in this study.