Development and psychometric evaluation of the Persian version of the Phoneme Recognition Test: A central auditory processing measure.

Objectives: The present study aimed to evaluate the psychometric properties of the Persian version of the Phoneme Recognition Test (P-PRT) in normal subjects and cochlear implant (CI) users. Material &
Methods: This study includes developing the Persian phoneme recognition test (PRT), determining its validity and reliability, and comparing the results of a control group versus CI users. The test reliability was examined through a test-retest with an approximately five-week interval. In the present survey, 363 subjects were investigated in three stages. The face validity evaluation stage was conducted on 40 subjects. The psychometric properties of the P-PRT were evaluated in 323 individuals (225 normal subjects and 98 CI users). The test-retest reliability was examined in all the 225 subjects in the control group and 40 CI users.
Results: The results confirmed the face validity of the P-PRT. No significant differences were observed between the two genders in terms of performance in the P-PRT. Significant differences were observed between the control and CI groups. Evaluating the test-retest reliability suggested perfect reliability (r>0.9) in both groups. Significant differences were observed in the P-PRT between the adults and the 7-year-old subjects compared to other age groups.
Conclusion: The P-PRT can be used as a valid and reliable test for clinically evaluating phoneme recognition abilities and monitoring the rehabilitation progress.

Introduction

Phonemes play a fundamental role in language and speech formation and successful auditory processing (1). They have been conventionally used in clinical auditory diagnosis and rehabilitation irrespective of the presence or absence of hearing loss. According to Luria et al. (1970), reading and writing abilities, such as doing dictations, are significantly associated with auditory phoneme processing abilities (2). Investigating the use of phonemes in scoring speech tests has shown that speech recognition tests based on phonemic examinations provide audiologists with more information compared to that obtained from scoring individual words as a whole. Using phonemes in speech recognition tests encouraged further examinations of speech tests and eventually led to emphasizing phonetically-balanced lists in word recognition tests (3).

According to Luria (1986), the auditory cortex in the left hemisphere (i.e., Brodmann areas 42, 22, and possibly 41) is the region of the brain that processes phonemic data (4). The findings obtained by Luria were evaluated in the Buffalo model of auditory processing, and reported by Katz to be consistent with the decoding (abbreviated as DEC) subcategory in the Buffalo model (1). The Buffalo model is a well-known and widely-used model of auditory processing (5). This model is based on many years of research to determine the signs on the Staggered Spondaic Word (SSW) test [X] for various sites of lesion to help localize brain dysfunction. These concepts and findings have been adapted and expanded to understand Central Auditory Processing (CAP) disorders. Currently, the SSW is the most widely used CAP test by audiologists working on Central Auditory Processing Disorders (CAPD) (6,7). This may be because of the sensitivity of the test and its accuracy in identifying the diagnostic categories (8). Out of 20 indicators of SSW, seven cases are related to the decoding category (associated with phonemic processing) (9). Due to the importance of scoring patient responses in the test and analyzing phonemic errors using the Phoneme Error Analysis (PEA), a special attention should be paid to the role of phonemes in this model (10). The other test of the Buffalo model, the Phonemic Synthesis Test, is based on recognizing, analyzing, synthesizing, and memorizing the phonemes (11).

In 1996 and 1997, Katz developed and introduced the Phoneme Recognition Test (PRT) for examining phoneme perception (12). The PRT was developed to investigate phoneme perception abilities, specifically in the cochlear implant (CI) users.

Masters et al. (1998) reported that disruption in the decoding subcategory at the phoneme level is the basis for auditory processing difficulties in CI users (13). Patients with Auditory Processing Disorder (APD) and CI users share a quite similar problem, as in the case of inability to comprehend a foreign language despite listening to and hearing the sound. The lower speed of speech processing observed in CI users can also be explained by the inability and inadequate capacity of their brain to process large volumes of new sounds over an extended auditory frequency range (13). The recorded version of the PRT was developed to fix problems, such as the variations observed in the results when using a live sound caused by its uncontrollable nature (14). The original version of the PRT can be used to randomly present a total of 34 English phonemes to patients, including 20 consonants and 14 vowels. Each phoneme in the test list is presented to a patient at least twice to ensure the consistency of response. (15).

Also, the important role of phonemes in evaluating patients with APD, normal individuals, and especially CI users should be considered. Given that there is no Persian version of the measure to be used in Iran, the present study was conducted to develop the Persian version of PRT (P-PRT). In addition, we aimed to evaluate the psychometric properties of the developed P-PRT, generate normative data in normal subjects, and investigate the performance of CI users in this test.

Materials & Methods

The present study was conducted in four stages: 1) developing the P-PRT, 2) determining the validity and reliability of the test, 3) generating its normative data, and 4) investigating the performance of CI users in the P-PRT. The data obtained from each stage were successively collected. The study was approved by the Medical Ethics Committee of Tehran University of Medical Sciences (code: IR.TUMS.FNM.REC.1398.140). All adult subjects and the parents of the participating children signed written consent forms, and they were ensured that they could withdraw from the study at any stage.

Developing the P-PRT

The voice of a native Persian male speaker sitting in an acoustic chamber was used to record the phonemes of the P-PRT. A Neumann TLM microphone and a pop filter were used to record the phonemes. The pop filter was used to attenuate the popping sounds of phonemes such as /p/ caused by the effect of fast-moving air on the microphone. The microphone’s output was entered into Audition CC Software (Adobe, 2019) installed on an Apple MacBook Pro. The speaker was requested to avoid pronouncing each phoneme for more or less than one second. After sequentially pronouncing all the Persian phonemes, he presented and recorded the instruction section for the patients. The narrator provided all the necessary instructions to the respondents at the beginning of the test.

In case of peak clipping when recording the phonemes, the speaker was asked to narrate the item again. In case of background noise between the phonemes, necessary modifications were made through Audition CC. The final output underwent loudness normalization and was saved with a.WAV format.

The Persian language comprises 29 phonemes, including six vowels and 23 consonants, five of which (i.e., /g/, /z/, /t/, /s/, and /h/) are homophones. In other words, in contrast to Arabic, in which these consonants differ in terms of both speech and hearing, in Persian, they differ only in writing. Given the homophony of these phonemes, their common spoken forms were recorded.

Therefore, the P-PRT comprised 28 items, each of which was repeated once in the list. The test phonemes were successively randomized twice to randomly distribute the items in the final list. Accordingly, 56 items were prepared as the final list of the P-PRT to examine the psychometric properties of this test.

Face validity

Forty subjects were included in the face validity evaluation stage. The recorded version was presented to five audiologists, five speech therapists, five linguists, ten normal individuals, and 15 CI users to report any problems associated with phoneme narration, recording quality and features, as well as audibility and fluency of the instruction (16,17). The final file of the test was developed and prepared for psychometric evaluations after qualitatively investigating the provided comments and making the necessary modifications in the file.

Evaluating the psychometric properties of the P-PRT

Study population and procedures

To investigate the psychometric properties of the P-PRT, ensure its standardization process, and determine the performance of the CI users and normal individuals in the test, the P-PRT was administered on 323 subjects (152 males and 171 females) aged at least seven years. This population consisted of 225 normal subjects (105 males and 120 females) and 98 CI users (47 males and 51 females). To ensure the integrity of the peripheral auditory system before administering the P-PRT, auditory thresholds for air and bone conduction were investigated at the octave and half-octave frequencies of 250 to 8000 Hz, and speech evaluations were investigated in the normal group. The subjects with an auditory threshold of at least 15 dB HL and a speech discrimination score of at least 96% were included in the study. Tympanometry was performed with a 226-Hz probe to assess the middle ear, and the subjects with type A tympanogram were included. To evaluate the auditory processing system in the normal subjects, the Persian Phonemic Synthesis Test (P-PST) and the Persian Staggered Spondaic Words test (P-SSW) were used, and those with a normal performance based on the normative data ​​of these tests were included in the study (18,19).

The inclusion criteria for the normal control group comprised being monolingual, Persian-speaker with a normal IQ, and being right-handed confirmed by the Edinburgh handedness inventory (20). For the CI group, the inclusion criteria comprised being right-handed, being monolingual, and with unilateral CIs in the right ear for at least one year. Also, all the CI users knew the Persian alphabet well and had undergone complete routine auditory training after cochlear implantation. The functional integrity of the CI was confirmed by the audiologists at the Tabassom Cochlear Implant Center, Ahvaz, Iran, using warble-tone audiometry through loudspeakers in the acoustic chamber. The exclusion criteria comprised fatigue, unwillingness to continue participation in the study, and middle ear infections between the test and retest sessions.

Test administration and scoring method

A MacBook Pro connected through a 3.5-mm auxiliary cable to an Inventis-Piano audiometer was used to run the test. The audiometer output was connected to a loudspeaker placed one meter away from the subjects in the acoustic chamber or a quiet room with no resonance. The intensity presented was selected according to the comfort level determined by the subjects. The P-PRT was performed on 323 subjects, including 225 normal subjects (105 males and 120 females) and 98 CI users (47 males and 51 females). The correct responses were specified as a dot (.) in each row. In case of no responses, a dash (-) was inserted, and incorrectly-pronounced phonemes were also inserted as pronounced in the relevant place on the score sheet. The obtained score was eventually reported as a percentage and noted at the end of the score sheet. Therefore, the raw score was reported as 0 to 56, and the final score as 0% to 100%.

Discriminant validity

The nonparametric Mann-Whitney U test was used to investigate the discriminant validity of the P-PRT, and evaluate the effect of gender on the performance and score achieved in the test. This test was also used to compare performance between the controls and CI users.

Reliability

To investigate the test-retest reliability of the P-PRT, a retest was performed on all the 225 subjects in the normal group and the 40 subjects in the CI group five weeks (32±4 days) after the first test. During this interval, the subjects did not participate in any auditory rehabilitation programs, did not consume medications to increase attention and concentration, and developed no middle ear infections; in addition, no significant changes were observed in their overall health status.

Normative data

The mean score of the subjects in normal control group (n = 225) was used to obtain the normative data of the P-PRT. The normal limit was determined using ±1 standard deviation. Moreover, the Kruskal-Wallis test was used to determine the significant differences between different age groups.

Data analysis

The data obtained were analyzed in SPSS software V. 24. The scores obtained from the P-PRT were reported as mean raw scores and percentages. The Kolmogorov-Smirnov test was used to investigate the normal distribution of the data, and face validity was examined qualitatively. The nonparametric Mann-Whitney U test was also used to examine the score differences between the controls and CI users and the performance differences between the two genders. Furthermore, the Kruskal-Wallis test was used to compare the performance of the subjects in different age groups and the Spearman test to examine the test-retest reliability with an interval of approximately five weeks.

Results

Study population

The present study was conducted on 363 subjects in three stages. The face validity evaluation stage involved 40 subjects, including five audiologists, five speech therapists, five linguists, ten normal individuals, and 15 CI users. The psychometric properties of the P-PRT were evaluated in 323 subjects (152 males and 171 females) aged at least seven years old. This population consisted of 225 normal subjects (105 males and 120 females) and 98 CI users (47 males and 51 females). Table 1 presents the demographic information of the subjects participating in the stage of evaluating the psychometric properties.

Table 1

Demographic information of the participants in evaluating psychometric properties

N	Gender	Group	Age (year)
17	Female	Control	7
20	Male
8	Female	CI
7	Male
13	Female	Control	8
13	Male
7	Female	CI
7	Male
16	Female	Control	9
11	Male
3	Female	CI
6	Male
11	Female	Control	10
11	Male
4	Female	CI
8	Male
10	Female	Control	11
16	Male
12	Female	CI
5	Male
18	Female	Control	12
19	Male
9	Female	CI
7	Male
35	Female	Control	Adult
15	Male
8	Female	CI
7	Male
120	Female	Control	Total
105	Male
51	Female	CI
47	Male

Abbreviation: CI: cochlear implant

Evaluating the psychometric properties

Validity

Face validity

According to the qualitative results obtained at this stage and the comments of the subjects, the necessary modifications were made until the reported problems were solved and the subjects were satisfied. Finally, according to all the subjects, the instructions were completely comprehensible, and all the 56 phonemes were reported as flawless and noiseless, which confirmed the face validity of the final P-PRT.

Discriminant validity

Discriminant validity was investigated in the P-PRT, and the Mann-Whitney U test was used to analyze the findings, examine the effect of gender on the performance of subjects in the test, and compare the performance between the normal subjects and the CI users.

Discriminant validity in investigating the effect of gender on outcomes

Our findings showed no significant differences between the males and females in their performance in the P-PRT [confidence interval (CI)=0.95]. Table 2 presents the results of the Mann-Whitney U test in different age groups, as well as in all the participants by gender.

Table 2

Comparing the effect of gender in different age groups and all the normal subjects

Age	Gender	N					Mann-Whitney U test	P-value
7	Female		1720				150.50	0.557
	Male
8	Female					1313	89.00	0.840
	Male
9	Female				1611		90.500	0.904
	Male
10	Female			1111			66.500	0.699
	Male
11	Female		1016				53.500	0.165
	Male
12	FemaleMale		1819				188.00	0.620
Adults	Female		3515				276.500	0.745
	Male
Total	Female		120105				7012.0	0.132
	Male

Abbreviation: N: number

Discriminant validity in investigating differences in performance between the normal subjects and CI users

The results of the Mann-Whitney U test showed significant differences in performance in all the age groups and the entire sample between the normal subjects and the CI users participating in the P-PRT (P<0.05). Table 3 presents the statistical results obtained in these two groups.

Table 3

Performance in the entire study population and its comparison between the normal subjects and cochlear implant users by age

Age	Group	N					Mann-Whitney U test	P-value
7	Control		3717				0.000	0.000
	CI
8	Control					2614	0.000	0.000
	CI
9	Control				279		0.000	0.000
	CI
10	Control			2212			0.000	0.000
	CI
11	Control		2617				0.000	0.000
	CI
12	ControlCI		3716				0.000	0.000
Adults	Control		50 15				0.000	0.000
	CI
Total	Control		22598				0.000	0.000
	CI

Abbreviation: CI: Cochlear implant

Reliability

The Spearman test was used to investigate the relationship between the test and retest scores in all the 225 normal subjects by age group. Reliability was confirmed in all the age groups (r≥0.9). Assessing the test-retest reliability of P-PRT scores resulted in r=0.885 in the 7-year-olds, r=0.923 in the 8-year-olds, r=0.948 in the 9-year-olds, r=0.991 in the 10-year-olds, r=0.982 in the 11-year-olds, r=0.951 in the 12-year-olds, and r=0.916 in the adults (all P-values<0.0001).

The Spearman test used to investigate the results obtained from the P-PRT after five weeks among 40 CI users confirmed reliability (r≥0.9) in all the age groups, and found r=0.939 in the 7-year-olds, r=0.944 in the 8-year-olds, r=1.000 in the 9-year-olds, r=0.998 in the 10-year-olds, r=1.000 in the 11-year-olds, r=0.996 in the 12-year-olds, and r=0.968 in the adults (all P-values<0.0001).

Therefore, the reliability of the P-PRT was confirmed in the CI users and the normal subjects of any age.

Normative data

Table 4 presents the normative data ​​of the P-PRT by age group obtained by administering the test on 225 normal subjects aged at least seven years old. The Kruskal-Wallis test was used to compare the scores between different age groups. The results showed significant differences at least between two age groups (CI=0.95, Chi-square=85.62, df=6, P<0.0001). The modified Bonferroni model was used for the pairwise comparison of different age groups. According to the results, adults and people aged seven years old were found to be significantly different from the other age groups in the P-PRT (Table 5).

Table 4

Normative data of the P-PRT by age

Age (year)	Statistic Parameters	Score (%)
7	Mean	52.54 (93.82)
	SD	1.14 (2.04)
	1-NL	51 (91.07)
8	Mean	54.34 (97.04)
	SD	1.12 (2.01)
	1-NL	53 (94.64)
9	Mean	54.22 (96.82)
	SD	1.31 (2.34)
	1-NL	53 (94.64)
10	Mean	54.18 (96.75)
	SD	1.33 (2.37)
	1-NL	53 (94.64)
11	Mean	54.30 (96.97)
	SD	1.01 (1.80)
	1-NL	53 (94.64)
12	Mean	54.70 (97.68)
	SD	1.50 (0.84)
	1-NL	53 (94.64)
Adulthood	Mean	55.34 (98.82)
	SD	0.68 (1.22)
	1-NL	55 (98.21)

Abbreviation: SD: standard derivation; NL: normal limit

Table 5

Pairwise comparison of age in the P-PRT score

Age groups compared	Test statistic	Std. Error	P
7-8	-72.593	16.166	.000
7-9	-69.041	15.989	.000
7-10	-67.235	17.00	.000
7-11	-67.651	16.166	.000
7-12	-87.270	14.687	.000
7-Adults	-124.979	13.699	.000
8-9	3.551	17.357	0.837
8-10	5.358	18.299	0.769
8-11	4.942	17.520	0.777
8-12	-14.677	16.165	0.363
8-Adults	-52.386	15.273	0.000
9-10	1.806	18.143	0.920
9-11	1.390	17.357	0.936
9-12	-18.229	15.988	0.254
9-Adults	-55.938	15.086	0.000
10-11	-0.416	18.299	0.981
10-12	-20.035	17.006	0.238
10-Adults	-57.744	16.161	0.000
11-12	-19.619	16.165	0.224
11-Adults	-57.328	15.273	0.000
12-Adults	-37.708	13.698	0.005

Discussion

The present study was conducted to develop the P-PRT and investigate its psychometric properties for clinical use in a Persian-speaking population. Given the specific application of this test in diagnosing phoneme processing deficits in CI users, a group of CI users with different ages was matched with a control group to compare performance between the two groups.

The PRT can constitute an appropriate criterion for evaluating auditory processing abilities, especially in the phonemic (decoding) domain, in normal individuals and CI users. Given the fundamental role of phonemes in the correct formation of speech and language, evaluating phonological abilities using the PRT is crucial for diagnosing phonological processing disorders, determining and designing proper and effective rehabilitation programs, and monitoring the progress through rehabilitation (4).

Katz et al. (2004) justified the development of a recorded version of the PRT by explaining the potential effects of confounding factors associated with the live sound on the results during the repeated administration of the PRT (15). Proper recording and controlled features in presenting phonemes were also found to result in a definite criterion for measuring changes in patient behaviors.

The present study was performed to confirm the face validity of the P-PRT by firstly investigating the linguistic features specific to Persian. All the homophones were pronounced as a single phoneme based on linguist comments. The recording was performed in the studio using the voice of a male speaker with a standard Persian dialect who was fully aware of the characteristics of individual phonemes and their effects on phoneme recognition. Recording distortions and additional sounds and noise were also examined, and recording features were investigated by playing the audio file for the experts, the CI users, and the normal subjects after normalizing the loudness. The required modifications were performed until ensuring that the problems were solved, and all the participants unanimously confirmed the face validity of the recorded test.

Confirming face validity can play a key role in evaluating the phoneme recognition skills of patients (20). Parkin and Dankowski (1986) found inherent changes in live sounds and the distortion caused by low-quality recording with a low face validity to be major problems in evaluating phoneme recognition abilities in the patients (14). They found distortion to be a more important problem requiring to be properly addressed. Using recorded test items was highly recommended in other studies and tests for evaluating CI users (21). The present findings confirmed the face validity of the P-PRT.

Investigating the psychometric properties of the P-PRT found no significant differences between the males and females, which is consistent with the findings obtained in a study on the original version of this test, suggesting no differences in the normative data of the two genders (13).

Investigating the phonemic abilities of CI users constitutes an important and specific application of the PRT. In this regard, Katz administered this test for three purposes as follows: 1) investigating the overall ability to identify spoken voices in CI users, 2) detecting their potential errors more accurately, and 3) investigating changes caused by rehabilitation as a criterion for rehabilitation effectiveness in CI users (15).

Given the importance of the clinical application of this test in CI users, the present study investigated the discriminant validity of this test in them (22–24). Our findings suggested significant differences between the control group and the CI users in all the age groups. It is worth noting that a significant difference occurred while all the CI users had benefited from complete routine rehabilitation programs after cochlear implantation. Moreover, the functional integrity of the CI device was confirmed in all the users before entering the study.

In line with this study, Valimaa et al. reported the phoneme recognition and confusions with multichannel CIs; they reported that by 24 months after the switch-on of the implant, the subjects scored 71% in consonant recognition with multichannel CIs (25).

Based on the findings of the present research and other studies, it is recommended that phonemic processing abilities be addressed more seriously as a function of the central auditory nervous system and a basis for speech and language comprehension in rehabilitation after cochlear implantation rehabilitation (26–30). It was reported that the temporal envelope cues from a limited number of channels are sufficient to support high levels of phoneme and sentence recognition in quiet (31–33), but not for speech recognition in a competing voice (34–36).

Speech carries acoustic cues in both spectral and temporal domains (37). For CI listeners who have only limited spectral discrimination, temporal cues are especially important (38), but even modern CI devices can only crudely encode these spectral and temporal cues (39).

In line with the present findings, Katz reported a woman with a CI that was unable to hear the /h/ sound; she was only able to feel that air came out of the speaker’s mouth when she was asked to repeat the presented sound, although the CI had provided her with the /h/ sound audibility. In other words, the /h/ sound inaudibility still persisted two years after cochlear implantation, as her brain could not process this sound, and implantation had formed no appropriate engrams or labels of the /h/ sound even in audible conditions. According to Katz, despite being heard as words in CI users, phonemes are not processed unless individual phonemes are first presented separately from the others and appropriately labeled in the central auditory system in a way that they can be first identified and then differentiated and comprehended, and ultimately manipulated and combined at a word-formation level.

Investigating the results of evaluating the reliability of the P-PRT confirmed its reliability in both the normal subjects and the CI users, which plays a key role in interpreting the results obtained before and after rehabilitation.

The potential effect of learning the test through repeated evaluations on the rehabilitation-associated changes is a major issue in phoneme recognition assessments. The reliability coefficient of over 90% obtained in all the age groups in both the study groups ensured that learning did not affect the P-PRT results. Therefore, repeating the P-PRT seemed to be ineffective in the results of the rehabilitation used.

The present study focused on the performance of different age groups in the P-PRT. Given the reduction observed in the age of the patients with CIs, the normative data of this test were required to be investigated in a wide age range involving the young and adults. In contrast to the study by Katz and Fletcher et al., which did not report the normative data of the original version of the PRT in children, a strength of the present study was reporting these data in subjects aged at least seven years old (15). Our findings showed that the 7-year-olds and the adult group were significantly different from the other age groups in terms of phoneme recognition abilities, and these abilities were found to be positively associated with age, mainly due to two reasons, i.e., the nervous system maturation and the effects of formal rehabilitation, the environment, and the experience. The combined effect of these factors appears to have improved the results in the normal subjects; nevertheless, a lack of improvement in the performance of the adult CI users compared to that of the normal adults can confirm the need for paying more attention to auditory processing rehabilitation in CI users through administering programs such as the Phonemic Training Program and Phonemic Synthesis Program (40,41).

In Conclusion

The present study developed a valid and reliable P-PRT. Our findings suggested that P-PRT can be used in clinical evaluations, diagnosis, and rehabilitation in children and adults. Using this test, along with other similar tests for evaluating phoneme recognition abilities, can help evaluate and diagnose potential disorders of phonemic processing, determine a rehabilitation roadmap, and ultimately monitor rehabilitation outcomes in CI users.

Recommendations

Today, less attention is paid to auditory processing rehabilitation in Iranian CI users, and almost all the rehabilitation programs are dedicated to the peripheral auditory system. According to the findings of present study, it is recommended that the effects of auditory processing rehabilitation be investigated in CI users. Also, the Persian Phonemic Training and Persian Phonemic Synthesis programs can be integrated with the rehabilitation programs of CI users, and the results of their effectiveness can be studied (40,41).

Using a single test alone appears ineffective in accurately studying phoneme recognition skills in patients. Despite the validity and reliability of the P-PRT, this test should be used along with other tests measuring phoneme recognition and phonemic processing abilities, including the Persian Phonemic Synthesis Test, to accurately diagnose and monitor rehabilitation (19).

Author,s Contribution

F.F. conceived the manuscript and revised it. S.SH., N.R, N.S., S.J, E.N, M.T, and M.K., done experimental analysis and wrote the manuscript and prepared tables.

Conflict of interest

The authors declare no conflict of interest. All procedures involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or compared ethical strand.