Validation of revised reading the mind in the eyes test in the Indian (Bengali) population: A preliminary study.

BACKGROUND: Social cognition deficits are common in clinical populations but there is a dearth of standardized social cognition assessment tools in India. Theory of mind (ToM) is an important aspect of social cognition which is often assessed with the revised reading the mind in eyes test (RMET-R). However, we do not have a statistically validated version of the test for the Indian population. AIM: This study aims to assess the acceptability, reliability, and validity of the Bengali version of the RMET-R.
MATERIALS AND METHODS: We administered the RMET-R to 23 patients with chronic schizophrenia (SCZ), 22 patients with bipolar disorder, and 104 healthy controls (HCs) to evaluate the reliability and validity of the instrument in the Indian (Bengali) population.
RESULTS: We obtained moderate internal consistency (Cronbach's alpha = 0.6) and test-retest reliability (intraclass correlation coefficient = 0.64, P < 0.001). Positive correlations were found between RMET-R and Wechsler picture arrangement (r = 0.60, P < 0.001), picture completion (r = 0.54, P < 0.001), and comprehension subtests (r = 0.48, P < 0.001). Patients with SCZ (M = 49.7, standard deviation [SD] = 16.5) scored significantly lower than HCs (M = 68.9, SD = 13.8) (P = 0.008; Cohen's d = 1.3) on the RMET-R. Thus this tool could discriminate patients who are reported to have Theory of Mind deficits from healthy controls.
CONCLUSION: The Bengali version of the RMET-R is a reliable and valid tool for assessing first-order ToM insofar as the original RMET-R measures this construct. Copyright:

INTRODUCTION

Understanding others' perspectives is crucial for successful social interaction. Different cognitive processes work in tandem to enable the human brain to understand others' perspectives –how they think and feel.[1] Theory of mind (ToM) is one of the most important aspects of social cognition. The American Psychological Association dictionary of psychology[2] defines ToM as the ability to understand that “others have intentions, desires, beliefs, perceptions, and emotions different from one's own and that such intentions, desires, and so forth affect people's actions and behaviors.”

Advanced ToM is reported to influence peer relationships in children and has been strongly associated with greater social popularity.[3] Conversely, a deficit in ToM has been associated with social cognition deficits in clinical populations. Cotter et al.[4] reviewed meta-analyses of 30 clinical conditions and stated that there are “significant deficits among individuals with a wide range of clinical conditions in their ability to identify emotions from facial expressions and to successfully complete ToM tasks.”[4] Therefore, a deeper insight into social cognition deficits in developmental, neurologic, and psychiatric populations can help clinicians design specific and transdiagnostic therapeutic measures.

However, assessing social cognition in a diverse population such as India's is quite a challenge especially in the absence of culturally appropriate tests. Consequently, the assessment of social cognition has received very little attention in Indian clinical studies, although global literature has been emphasizing its importance in recent times.[4] The development of the Social Cognition Rating Tools in Indian Setting[5] has initiated a new drive to delve deep into social cognition deficits in Indian clinical populations.[678] However, ToM is a complex construct involving several cognitive processes, and conversely “ToM” tasks often tap more skills than just ToM. Consequently, we have a range of ToM tasks focusing on distinct processes. A recent two-systems hypothesis[910] holds that ToM involves social perceptual processes (e.g., face processing) as well as reflexive cognitive operations (e.g., language and reasoning). Bora et al.[11] conducted a meta-analysis of 36 studies comprising 1181 patients with schizophrenia (SCZ). This study considered the task variability across the different types of ToM tests and “found that only the individual tasks 'Reading the Mind in the Eyes'[12] and the 'Hinting Task' produced consistent effects.”[11]

Thus, the RMET[12] is not only widely used to assess ToM but is also reported to produce consistent results. Consequently, it has been translated into different languages as an effective tool for evaluating ToM. Although Western tests are considered to be inappropriate in other cultural contexts, some studies posit that basic human emotions are universal and their expressions are conventionalized for all members of the species regardless of culture.[13] Recent studies, however, have documented cultural differences in facial expressions of emotions,[14] especially for complex emotions.[15] Moreover, morphologically similar expressions such as anger and disgust may generate multiple responses from different observers. Hence, care must be taken to assess the acceptability of these facial expressions in a particular culture before employing them in a testing tool.

Aim

In this study, we aim to investigate the acceptability, reliability, and validity of the revised RMET (RMET-R) in the Indian (Bengali) population. Validity will be assessed in patients with SCZ and bipolar disorder (BPD), who are reported to have ToM deficits.[1617]

MATERIALS AND METHODS

Participants

Twenty-three patients with chronic (SCZ; 12 males, 11 females; average age: 36.0, standard deviations [SD] = 11.5; average education: 11.8, SD = 3.4), 22 patients with (BPD; 18 males, 4 females; average age: 32.8, SD = 11.5; average education: 10.8, SD = 3.2), and 104 healthy controls (HCs; 55 male, 49 female; average age: 28.1, SD = 8.5; average education: 15.9, SD = 3.4) participated in the study. Two HCs who scored two SD below the mean in RMET-R were excluded from the study. Hence, the final HC group consisted of 102 participants (HC; 55 male, 47 female; average age: 28.2, SD = 8.6; average education: 16.0, SD = 3.3). The SCZ and BPD patients were selected based on Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria for SCZ and BPD respectively. HCs were selected on the basis of their scores in the 12-question General Health Questionnaire.[18] There were significant differences between the SCZ and the HC groups in age (t (27.7) = −3.08, P = 0.005) and education (t (123) = 5.44, P < 0.001). A significant difference was also found between the BPD and the HC groups in their level of education (t (122) = 6.66, P < 0.001). However, SCZ and BPD groups did not differ significantly in either age or education. All the participants were native Bengali speakers with at least 5 years of formal education and had a reasonable command of the four language skills (reading, writing, speaking, and listening). They had normal or corrected to normal vision and were right-handed as determined by the Edinburgh Handedness Inventory.[19] The participants signed an informed consent form approved by the Ethics committee, Calcutta Medical College and Hospital, Kolkata where the data were collected from June 2013 to December 2014. All the patients and their accompanying caregivers (biologically unrelated) who fulfilled the inclusion criteria were invited to participate in this study. Internees, postgraduate students and faculty members of the Department of Psychiatry, Calcutta Medical College and Hospital, Kolkata also volunteered to participate in the study.

Materials

The RMET-R[12] consists of 36 photographs depicting “the eye-region of the face of different actors and actresses.” The subject taking the test has to choose the target word that best describes what the person in the photograph is thinking or feeling from an array of four different choices. This test is designed to assess first-order ToM (i.e., the ability to understand another person's thoughts).

The Indian adaptation of Wechsler Adult Performance Intelligence Scale (WAPIS)-Picture Arrangement subtest[20] consists of a series of cards that need to be arranged in a correct order to weave a story that makes sense. This test aims to assess reasoning abilities. To weave a perfect story one needs to understand the intentions and the emotions of the human characters depicted on the cards. Consequently, it involves mentalizing and is intricately associated with social cognition.[21]

The WAPIS-Picture Completion subtest[20] is a visual cognition test assessing the ability to perceive visual details. The test consists of 26 pictures with a missing part (e.g., a face without ears). The subject taking the test has to identify the missing part in a very short span of time. It may be noted that visual cognition is intricately intertwined with social interaction.[2122]

The comprehension subtest of the Verbal Intelligence subscale of the PGI-Brain Dysfunction Battery (PGIBBD)[23] consists of 18 questions on social situations. The questions are of increasing difficulty. This test assesses social reasoning and the ability to deal with social rules and conventions.

Procedure

Translation of 'Reading the Mind in the Eyes' Test - Revised

The original English version of the test was independently translated into Bengali by the second author (G.D.) and the seventh author (K.B.). These Bengali versions were independently back-translated into English by the third author (R.A.) and the fourth author (S.S.C.), respectively. The translations were reviewed by an expert team consisting of the fifth author (P.G.), the sixth author (M.K.B.) – a native English speaker, the translators, and a senior psychiatrist of the department not involved in the study. The senior psychiatrist was blind to the specific aims and objectives of the study and declared no conflicts of interest that could affect his judgment regarding this study. He was included in the expert team to control possible selection biases. The expert team resolved the discrepancies and made subtle changes to address the inadequacies in expressions. The members of the expert team were balanced bilinguals with almost equal proficiency in both languages. This version of the test was administered to ten postgraduate student interns of the Department of Psychiatry, Calcutta Medical College. They were asked to give their feedback on the test. After careful analyses of their performances and feedback on the test, a second round of moderations was made. Care was taken to avoid the use of unfamiliar words as far as possible. As the test involved (1) recognition of the emotions depicted in the eyes and (2) mapping mental state lexicon to those expressions, difficulty in comprehension of the words that were given as choices could affect performance. To address this issue a glossary of all the mental state terms used in the test was prepared along with their use in sentences and the participants were instructed to go through the glossary before taking the test.

The final Bengali version of the RMET-R comprising all the 36 items was administered to 104 healthy participants. The instructions and stimuli were presented on a Dell Inspiron 15.6” laptop with four answer choices arranged in four corners of the pictures as in the original version.[12] The data from all the 104 participants were preprocessed in Microsoft Excel 2016. Two participants who scored two SDs below the mean were excluded from the study. The percentage of correct responses of the remaining 102 participants was analyzed. An item-wise analysis revealed that 7 items (2, 7, 19, 23, 31, 32, and 35) did not reach the 50% cut off level. These items were excluded [Table 1]. Hence, 29 items were considered for the final analyses. This new version of the Bengali RMET-R (29 items) was administered to 23 patients with SCZ and 22 patients with BPD. The Picture Completion subtest,[20] the Picture Arrangement subtest[20] of WAPIS, and the comprehension subtest of PGIBBD[23] were administered to all the participants per the guidelines laid down in the respective test manuals. The final data were transferred to SPSS Statistics 21.0, IBM Corp., Armonk, NY, USA for statistical analyses.

Table 1

Percentage of correct responses of the 36-item in descending order

Stimulus number	Percentage of accuracy
Stimulus 5	88.23
Stimulus 9	86.27
Stimulus 29	85.29
Stimulus 34	79.41
Stimulus 30	78.43
Stimulus 21	77.45
Stimulus 6	76.47
Stimulus 3	73.52
Stimulus 22	72.54
Stimulus 24	72.54
Stimulus 11	71.56
Stimulus 14	71.56
Stimulus 8	70.58
Stimulus 16	70.58
Stimulus 36	69.6
Stimulus 12	67.64
Stimulus 1	66.66
Stimulus 15	66.66
Stimulus 10	65.68
Stimulus 13	61.76
Stimulus 20	61.76
Stimulus 4	56.86
Stimulus 28	56.86
Stimulus 33	56.86
Stimulus 25	52.94
Stimulus 27	52.94
Stimulus 18	50.98
Stimulus 17	50
Stimulus 26	50
Stimulus 35*	49.01
Stimulus 2*	47.05
Stimulus 32*	46.07
Stimulus 19*	44.11
Stimulus 23*	44.11
Stimulus 31*	43.13
Stimulus 7*	29.49

*These items were eliminated

RESULTS

Descriptive statistics

The data conformed to the normality and homogeneity assumptions measured by the Shapiro–Wilk test and Levene's test, respectively. The mean percentages of correct responses in HC, BPD, and SCZ were 68.9 (SD = 13.8), 51.4 (SD = 13.2), and 49.7 (SD = 16.5), respectively [Figure 1]. There was no significant difference between RMET-R scores of healthy male (M = 69.4, SD = 14.6) and healthy female (M = 68.2, SD = 12.9) participants (t (100) = 0.425, P = 0.672). There were no significant differences between healthy male (age: M = 27.9, SD = 8.3; education: M = 16.3, SD = 3.7) and healthy female participants (age: M = 28.5, SD = 8.9; education: M = 15.5, SD = 2.8) in either age (t (100) = −0.372, P = 0.71) or education (t (100) = 1.251, P = 0.214). Age (r = −0.34, P = 0.001) correlated negatively and education (r = 0.49, P <.001) positively with RMET-R scores.

Figure 1

Percentage of correct responses of the three groups in the Bengali version of the revised reading the mind in eyes test

Internal consistency reliability

The internal consistency of the 29 items was measured with Cronbach's alpha. We obtained a moderate value (α = 0.6) which is below the traditional threshold (0.7). Usually, a high value of alpha indicates that every item measures something similar to other items in the test. As RMET-R assesses a wide range of emotional expressions, a very high value of Cronbach's alpha was not expected.[24] Considering the broad range of emotions assessed, we accepted this α value.

Test-retest reliability

Twenty-five persons from the HC group were randomly selected for the retest. Retests were done at least after a period of 6 months to control practice effects. The test-retest reliability was assessed using intraclass correlation coefficient (ICC). A moderate degree of reliability was found between the test-retest scores. The average measure ICC was 0.64 with a 95% confidence interval from 0.40 to 0.82, P < 0.001).

Concurrent validity

Picture Arrangement and Picture Completion are reported to be associated with the social cognition factor.[21] Comprehension of social situation questions has been used in numerous studies as a measure of social cognition.[25] Therefore, we used the Indian adaptations of these tests to assess concurrent validity. We found moderate positive correlations between RMET-R (29-item Bengali version) and the Indian adaptation of WAPIS-Picture Arrangement subtest[20] (r = 0.60, P < 0.001), the Indian adaptation of WAPIS-Picture Completion (r = 0.54, P < 0.001), and the Comprehension subtest of the verbal intelligence subscale of PGIBBD (r = 0.48, P < 0.001).

Known group validity

One-way ANCOVA revealed a significant difference in RMET-R scores of the three groups (HC, SCZ, and BPD) after controlling for age and education (F (2, 147) = 5.52, P = 0.005), partial η2=0.07, observed power = 0.85. The covariate age (F (1,147) = 8.30, P =0.005, partial η2 = 0.06, observed power = 0.81) and education were significant (F (1, 147) = 23.4, P < 0.001, partial η2 = 0.14, observed power = 1.0).

Pairwise comparisons with adjustments for multiple comparisons (Bonferroni correction) revealed that scores of the patients with SCZ (M = 49.7, SD = 16.5) were significantly lower than their matched controls' (M = 68.9, SD = 13.8) (P =0.008; Cohen's d = 1.3). The difference between the scores of the patients with BPD (M = 51.4 SD = 13.2) and matched controls (M = 68.9, SD = 13.8) approached significance (P = 0.067; Cohen's d = 1.3).

DISCUSSION

The Bengali version of the RMET-R was found to have acceptable psychometric properties. The average score of the healthy population was greater than that for the original English version[12] which reported 18.8 (SD = 2.5) or 52.2% for men and 21.8 (SD = 1.8) or 58.3% for women. In contrast to the original version, the male and female participants in our study did not exhibit any significant difference in RMET-R scores. This gender equivalence is consistent with Olderbak et al.[26] and some other studies on RMET-R. However, in contrast to Olderbak et al.[26] in this Bengali population, we found a significant correlation between RMET-R and age and education. Age was negatively correlated and education positively correlated with the RMET-R scores. This seems intuitive as education helps to hone cognitive and especially language function[27] and aging can result in cognitive decline.[28]

The internal consistency of the Bengali version of the RMET-R was less than the traditional threshold. However, the RMET-R tests a broad domain of facial expressions and the items probably do not measure a single construct. Exploratory factor analysis by Olderbak et al.[26] yielded a five-factor solution clearly indicating that the RMET-R measures more than one construct. Consequently, the value of Cronbach's alpha (α = 0. 6) was only moderately high.[24] Consistent with our result, several published studies report that RMET-R has poor internal consistency. For example, Vellante et al.[29] obtained a similar alpha value (α = 0. 605) for the Italian version of the test.

We obtained moderate test-retest reliability (ICC = 0.64) which is very close to the Spanish version of the test (ICC = 0.63).[30]

The Bengali RMET-R was found to be correlated with other social cognition tests per our expectations. However, the correlations were not strong, probably as these tests assessed different aspects of social cognition. Social cognition involves different processes, and tests measuring different aspects might not exhibit strong correlations. For instance, the computerized test of primary emotions perception (PEP) and the Brazilian version of the RMET-R were also found to have only moderate correlations (r = 0.43), as the PEP measures cognitive aspects of emotional intelligence involving reasoning ability while the RMET-R assesses the ability to label mental states.[31]

The Bengali version of the RMET-R can discriminate clinical populations from HCs and more importantly can make graded discriminations indicating the scale of the impairment. Patients with SCZ scored significantly less than the controls on RMET-R (P = 0.008) whereas patients with BPD only showed a deficient trend (P = 0.067). This result seems consistent with studies that report a more severe ToM impairment in patients with SCZ compared to patients with BPD.[32]

In sum, the Bengali version of the RMET-R has acceptable psychometric measures. However, the small sample size of the SCZ and bipolar patients, the sampling from a particular region of West Bengal, and the higher average educational level of the participants may limit the scope of the generalizability of the results. Moreover, we deviated from the WHO guidelines laid down for translation and adaptation of English instruments. According to the WHO, the back translation should be done by a native English speaker who has no knowledge of the questionnaire. In this case, however, the instrument was back-translated independently by two bilingual persons who were involved in the study. Nevertheless, the translations were reviewed and approved by the sixth author who was a native English speaker. In addition, the agreement between the two independent translators was very high and this validated the aptness of the translations. Despite these limitations, the high average score of the Bengali RMET-R in a healthy population and large effect sizes observed in discriminating healthy and clinical populations look promising. It may prove to be a useful tool for assessing ToM in the Indian population.

CONCLUSION

The results suggest that the 29-item Bengali version of RMET-R is a reliable and valid tool for assessing emotion recognition in adult healthy and clinical populations. This new tool can be used to explore social cognition in different clinical populations and might enable researchers to explore sociodemographic and cross-cultural differences in emotion recognition.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.