Evaluation of social cognitive measures in an Asian schizophrenia sample.

BACKGROUND: Converging evidence has indicated that deficits in social cognition may manifest as poor functioning; therefore, social cognition has emerged as an important research area and treatment target. However, few studies have examined the psychometrics of multiple social cognition measures in an Asian population. This study aims to evaluate the psychometrics of measures indexing the four core social cognition domains.
METHODS: Schizophrenia outpatients (n = 116) and healthy controls (n = 73) completed a battery of nine social cognitive measures, twice, four weeks apart. Psychometric properties were examined via test-retest reliability, internal consistency, utility as a repeated measure, time administration, and tolerability. Logistic regression was performed to identify psychometrically sound tasks that best discriminated case-control status. PCA was conducted to explore social cognition dimensional structure.
RESULTS: The Bell Lysaker Emotion Recognition Task (BLERT), Penn Emotion Recognition Task (ER40), and The Awareness of Social Inference Test, branch III (TASIT-3) showed strongest psychometrics. The Ambiguous Intentions and Hostility Questionnaire, Hostility Bias subscale (AIHQ-HB) showed slightly weaker properties, requiring further evaluation. The Hinting task, Mini Profile of Nonverbal Sensitivity (MiniPONS), Relationships Across Domains (RAD), Internal Personal and Situational Attributions Questionnaire (IPSAQ), and Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) showed poorer psychometrics in our sample. PCA revealed a two-factor solution comprising social cognition skills and attributional style/bias.
CONCLUSION: Here, we examined the psychometric properties of a comprehensive social cognition battery based on the SCOPE study in an Asian schizophrenia population. Continued evaluation and standardization of social cognitive measures are needed to refine our understanding of this construct in schizophrenia.

Introduction

Schizophrenia is a debilitating condition characterized by the presence of positive symptoms (e.g. hallucinations, delusions), negative symptoms (e.g. apathy, amotivation, anhedonia) and disorganized symptoms (e.g. thought disorder, bizarre behaviors), often resulting in functional impairments (American Psychiatric Association, 2013). Despite the low prevalence of schizophrenia (1%) (McGrath et al., 2008) as compared to other conditions, it is the 8th leading cause of disability-adjusted life years (DALYs) worldwide. In the larger context, schizophrenia also exerts a high social and economic burden on patients, caregivers and the community (Chong et al., 2016).

Given the critical role that functional outcomes play in schizophrenia morbidity, there has been growing interest in factors underlying functional outcome. It is postulated that delineation of these factors may enable the development of targeted interventions. Classically, neurocognition was thought to be a major contributing factor given that cognitive deficits have been consistently reported (Bowie and Harvey, 2006; Fioravanti et al., 2012) and considered to be a hallmark of schizophrenia. However, research has indicated that the variance in functional outcome that could be explained by neurocognitive measures is typically modest, with a majority of studies reporting it to be between 20 and 40% (Green et al., 2000). This suggests that most of the variance in functional outcomes (60–80%) is still unaccounted for (Kurtz et al., 2001; Medalia and Choi, 2009; Wykes et al., 2011). Growing evidence suggests that social cognition influences functional outcomes in schizophrenia and mediates the relationship between neurocognition and functional outcomes (Fett et al., 2011; Halverson et al., 2019; Pinkham et al., 2014).

Social cognition refers to “the mental operations that underlie social interactions, including perceiving, interpreting, and generating responses to the intentions, dispositions, and behaviors of others” (Green et al., 2008). Deficits in this multifaceted construct are heterogeneous (Hajdúk et al., 2018; Rocca et al., 2016), and have been documented throughout disease course in schizophrenia (Savla et al., 2013), early phase of illness (Healey et al., 2016), clinical high-risk individuals (Lee et al., 2015; Piskulic et al., 2016; van Donkersgoed et al., 2015), and first degree relatives (Lavoie et al., 2013); thus, suggesting a putative genetic vulnerability, rather than a state dependent construct.

Social cognition has been established as a multi-dimensional construct that is distinct, though related to general neurocognition (Nuechterlein et al., 2004; Sergi et al., 2007; van Hooren et al., 2008). Factor analytic studies of social cognition in schizophrenia have yielded two-factor (Buck et al., 2016a), three-factor (Mancuso et al., 2011; Mehta et al., 2014) and four-factor solutions (Bell et al., 2009), with disparate dimensional structure of lower level versus higher level social cognitive processes (Mancuso et al., 2011) and social cognition skills versus attributional style (Browne et al., 2016; Buck et al., 2016a). These differences could in part be attributed to the variety of social cognitive measures used.

More recently, in an attempt to standardize the field, the Social Cognition Psychometric Evaluation (SCOPE) study sought to systematically evaluate the psychometric properties of widely used social cognitive measures nominated by a RAND expert panel, for potential use in clinical trials (Pinkham et al., 2018, Pinkham et al., 2016, Pinkham et al., 2014). Measures indexing four core theoretical domains of social cognition were identified: emotion processing, attributional style/bias, social perception, and theory of mind (Pinkham et al., 2014). Of the 11 measures assessed across SCOPE study phases (Pinkham et al., 2018, Pinkham et al., 2016), the Hinting task (Corcoran et al., 1995), Bell Lysaker Emotion Recognition Task (BLERT) (Bell et al., 1997), and Penn Emotion Recognition Task (ER40) (Kohler et al., 2003) showed strongest psychometric properties with association to functional outcomes, and were recommended for use in clinical trials. The Reading the Mind in the Eyes (Eyes) (Baron-Cohen et al., 2001), The Awareness of Social Inferences Task (TASIT) (McDonald et al., 2003), Intentional Bias Task (IBT) (Rosset, 2008) showed weaker but promising characteristics, while the authors caution the use of the Ambiguous Intentions and Hostility Questionnaire (AIHQ) (Combs et al., 2007), Mini Profile of Nonverbal Sensitivity (MiniPONS) (Bänziger et al., 2011), Social Attribution Test-Multiple Choice (SAT-MC) (Bell et al., 2010), Relationship Across Domains (RAD) (Sergi et al., 2009) and Trustworthiness Task (Adolphs et al., 1998).

While the SCOPE study presents an important endeavor towards developing a gold standard social cognitive battery, data was only collected in the United States, and further evaluation of these measures are needed, particularly in different cultural contexts (Hajdúk et al., 2019; Mehta et al., 2011a). Current efforts for social cognition validation in Asian populations have been limited to a specific social cognitive domain or measures differing from SCOPE (Chen et al., 2017; Lee et al., 2018; Lo and Siu, 2017; Mehta et al., 2011b). To our knowledge, here we provide the first evidence on the psychometric properties of a comprehensive list of 9 measures indexing the four core social cognitive domains identified by SCOPE, in an Asian schizophrenia population. Of these, 7 out of 9 measures overlapped with SCOPE. The aims of the present study were to: (1) evaluate the psychometric properties of social cognitive measures, (2) identify measures which showed case-control discrimination, and (3) elucidate the dimensional structure of social cognition in our sample.

Methods

Participants

A total of 189 participants were recruited for this study. Of these, 116 participants have a diagnosis of schizophrenia and 73 were healthy controls. Patients were recruited from outpatient clinics from the Institute of Mental Health in Singapore. Diagnosis was determined as having met the Diagnostic and Statistical Manual of Mental Disorder (DSM-IV-TR) or the International Classification of Diseases (ICD-10) criteria for schizophrenia, as reported in the patient's medical records. All participants were aged between 21 and 55 years and proficient in English. Participants with a history of mental retardation, developmental disability, substance abuse, neurological condition, head injury or color blindness were excluded from the study. In addition, healthy controls were screened to exclude those with a family history of psychiatric conditions (1st degree relatives) and psychopathology history with the Structured Clinical Interview for DSM-IV-TR, non-patient edition (First et al., 2002). This study was approved by the National Healthcare Group's Domain Specific Review Board. Written informed consent was obtained from all participants.

Social cognitive measures

Emotion processing

Bell Lysaker Emotion Recognition Task (BLERT)

The BLERT (Bell et al., 1997) includes 21 audio-visual vignettes of a male actor providing facial, voice-tonal and upper-body movement cues, while sharing one of three work-related monologues. Participants were required to correctly identify one of seven emotional states (i.e. happiness, sadness, anger, surprise, disgust, fear, and no emotion). Performance was indexed by the total number of correct emotions identified (scores ranged from 0 to 21).

Penn Emotion Recognition Task (ER40)

The ER40 (Kohler et al., 2003) is a computerized test which includes 40 photographs of static faces portraying one of five emotions (i.e. happy, sad, anger, fear, and, no emotion). Participants were presented with these photographs and were required to identify the correct emotion expressed. Performance was indexed as the total number of correct responses (scores ranged from 0 to 40).

Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT)

The MSCEIT (Mayer et al., 2003) is a 141-item scale, made up of eight tasks, measuring 4 branches of emotional intelligence: perceiving emotions (PE), using emotions to facilitate thoughts (FE), understanding emotions (UE), and managing emotions (ME). Each subscale consisted of two tasks. A general consensus scoring was obtained for each branch by scoring respondents' answers against the proportion of sample that endorsed the same answer, using the MSCEIT online scoring system.

Attributional style/bias

Ambiguous Intentions and Hostility Questionnaire (AIHQ)

The AIHQ (Combs et al., 2007) measures social cognitive bias through 15 negative social vignettes that varied in intentionality (i.e. situations with ambiguous, intentional, accidental causes). For each vignette, participants were required to provide a reason why the situation occurred, and how they would respond to the situation. Participants also rated three questions on a Likert scale: whether the person carried out the behavior on purpose (1 = definitely no, to 6 = definitely yes), how angry the situation would make them feel (1 = not at all angry, to 5 = very angry), and how much they would blame others for the situation (1 = not at all, to 5 = very much). The open-ended responses were coded by two independent raters (ICC = 0.81) to compute the hostility bias (HB) and aggression bias (AB) scores respectively. A blame score (BS) was computed by taking the average of the three Likert ratings

Internal, Personal and Situational Attributions Questionnaire (IPSAQ)

The IPSAQ (Kinderman and Bentall, 1996) measures an individual's perception of causality in positive or negative social situations. Participants were required to provide a likely causal explanation for 16 positive and 16 negative social situations, then categorize the cause as either internal (i.e. relating to the respondent), personal (i.e. relating to another person), or situational (i.e. relating to the circumstances or chance). Two cognitive bias scores, externalizing bias (EB) and personalizing bias (PB), were obtained. A positive EB score indicates strong self-serving bias (i.e. blaming oneself less for negative than positive situations), and larger PB score indicates greater tendency to attribute negative situation to others rather than situational factors.

Social perception

Relationships Across Domains (RAD)

The RAD (Sergi et al., 2009) measures competence in relationship perception. The abbreviated version consisted of 15 male-female dyadic vignettes representing one of four relational models (i.e. communal sharing, authority ranking, equality matching, and market pricing). Participants were required to answer three yes/no questions regarding the likelihood of behavior occurrence, given the relational model described in the vignette. Performance was indexed as the total number of correct responses (scores ranged from 0 to 45).

Mini Profile of Nonverbal Sensitivity (MiniPONS)

The MiniPONS (Bänziger et al., 2011) is comprised of 64 visual-audio segments which measure the ability to correctly identify emotions, attitudes, and intentions based on nonverbal cues (i.e. face, body, and vocal tones). Participants were required to choose which of two options best described the scenarios based on combinations of nonverbal cues provided by a female Caucasian. Performance was measured as the total number of correct responses (scores ranged from 0 to 64).

Theory of mind

Hinting Task

The Hinting task (Corcoran et al., 1995) measures one's ability to infer intentions based on indirect speech. The task was read aloud and consisted of 10 short vignettes of dyad interactions where each ends with one character dropping a hint. Participants were required to infer what the character truly meant. If an incorrect response was provided, a paraphrased hint was given, allowing the participants to gain partial credit if a subsequent response was correct. Performance was indexed as total correct inferences made (scores ranged from 0 to 20).

The Awareness of Social Inference Test (TASIT)

The TASIT (McDonald et al., 2003) assesses the ability to identify emotional expressions and make inferences based on social cues. The TASIT is comprised of three sections. TASIT-1 is a 28-item dynamic video emotion evaluation test where participants were required to correctly identify one of seven emotions (i.e. happy, surprised, neutral, sad, angry, anxious, and revolted) depicted by actors interacting in an everyday social situation. The TASIT-2 assesses detection of sincere, simple, and paradoxical sarcasm through 15 short videos of actors interacting in social situations. Participants were required to answer four standard questions that unravel conversational meanings on the intentions and beliefs through paralinguistic cues. Similarly, TASIT-3 consisted of 16 short videos of conversational exchanges with enriched information of visual and text cues involving either lies or sarcasm. Participants again answered four standard questions addressing the true meaning of the exchange. Performance was assessed as total number of correct responses (scores ranged from 0 to 28, 0–60, and 0–64 respectively).

Practicality and tolerability rating

Practicality was operationalized as the task administration time (Pinkham et al., 2016). Tolerability was rated on a Likert scale where participants indicated the degree in which they enjoyed the task (1 = very unpleasant, to 7 = very pleasant) (Pinkham et al., 2016).

Procedures

Social cognitive measures were administered to all participants twice; baseline and re-test assessment, 4 weeks apart (M = 28.38 days, SD = 2.79). The 18-item Brief Psychiatric Rating Scale (BPRS) (Overall and Gorham, 1962) was used to assess the severity of psychiatric symptoms in schizophrenia participants at both study visits by trained raters. The order of social cognitive tasks administered was counterbalanced across participants and study visits. For TASIT, an alternate form was administered at the retest visit. The TASIT alternate form was also counterbalanced such that both forms A and B were administered equally across visits. As alternate forms were not available for other social cognitive measures, these tasks were identical across both visits.

Statistical analysis

Test-retest reliability

Test-retest reliability was assessed with Spearman's rho and intraclass correlation coefficient (ICC) with two-way single measure random effect model for absolute agreement. Spearman's rho ≥ 0.6 and ICC ≥ 0.6 were considered acceptable (Koo and Li, 2016).

Internal consistency

Internal consistency was evaluated with Cronbach's alpha. A Cronbach's alpha ≥0.7 was deemed acceptable (Tavakol and Dennick, 2011).

Utility as a repeated measure

The Wilcoxon signed-rank test and Cohen's d were used to assess utility as a repeated measure and practice effect. Paired samples t-test was used to examine time administration differences and tolerability. Floor and ceiling effects were evaluated by identifying the number of participants performing at chance or 100% respectively (Pinkham et al., 2016).

Case-control discrimination

Mann-Whitney U test was performed to examine case-control differences across visits. A logistic regression model with stepwise forward likelihood ratio was also performed to identify psychometrically sound social cognitive tasks that could best discriminate case-control status. Only tasks that showed adequate psychometric properties, as investigated above, were included in the regression model.

Principal components analysis (PCA)

Principal components analysis (PCA) was performed to investigate the dimensional structure of those social cognitive tasks that showed significant association with case-control status. PCA model was tested using the schizophrenia sample only. Eigenvalue >1 and scree plot was used to aid in factor model extraction, and items with a factor loading ≥0.4 were retained. Data suitability for PCA was assessed with the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (Kaiser, 1974, Kaiser, 1970) and the Bartlett's test of sphericity (Bartlett, 1954) A KMO > 0.6 and significant Bartlett's test of sphericity would indicate data suitability for factor analysis (Tabachnick and Fidell, 2007). The direct oblimin rotation was used as factors are likely inter-correlated (Buck et al., 2016a). Finally, Pearson's r was conducted in the schizophrenia sample to examine correlations between extracted components and symptoms using the 5-factor BPRS consensus model (Shafer, 2005).

All social cognitive scores were standardized against healthy controls, adjusted for age and gender, and normalized using Blom inverse rank transformation (Blom, 1958). These normalized adjusted scores were used in the logistic regression and PCA. The unadjusted normalized scores were used to evaluate test-retest reliability. All analyses were performed in IBM SPSS Statistics v23 (IBM Corp, 2015).

Results

The demographics and clinical characteristics are presented in Table 1. A total of 189 participants were recruited (ncase = 116, ncontrol = 73). Of these, 108 schizophrenia patients and 70 healthy controls completed both visits. Clinical characteristics reported for the schizophrenia sample were computed using the 5-factor BPRS model (Shafer, 2005) and daily chlorpromazine equivalent for antipsychotics (Atkins et al., 1997; Barnes and Paton, 2011; Davis, 1974; Leucht et al., 2014) Group differences in years of education (t = −4.66, p < 0.001), ethnicity, χ2 (2, n = 189) = 10.68, p < 0.01, and age (t = 3.90, p < 0.001) were observed.

Table 1

Demographics and clinical characteristics.

	Cases	Controls
	(n = 116)	(n = 73)
Age (years)	38.29 (8.91)	32.78 (10.28)
Gender (male, %)	65 (56.0%)	33 (45.2%)
Ethnicity (%)
Chinese	101 (87.1%)	57 (78.1%)
Malay	6 (5.2%)	14 (19.2%)
Indian	9 (7.8%)	2 (2.7%)
Years of education (years)	14.85 (2.78)	16.77 (2.70)
Medication type, n (%)
None	1 (0.9%)	–
Typical antipsychotics only	22 (19.0%)	–
Atypical antipsychotics only	60 (51.7%)	–
Combinations of typical and atypical	33 (28.4%)	–
CPZ equivalent (mg)	453.81 (426.71)	–
BRPS
Positive symptoms	6.10 (2.94)	–
Negative symptoms	4.61 (1.18)	–
Affect	6.96 (2.61)	–
Resistance	4.04 (1.16)	–
Activation	3.32 (0.72)	–
Total score	25.03 (5.97)	–

Note. Values in cells represent mean (SD), unless otherwise stated. CPZ = daily chlorpromazine equivalent; BRPS = Brief Psychiatric Rating Scale.

Test-retest reliability

Test-retest reliability examined with Spearman's rho using the raw social cognitive scores and ICC using the unadjusted normalized social cognitive task scores showed differences in reliability statistics between cases and controls (Table 2). In cases, most tasks were above 0.6 threshold, while controls generally exhibited lower reliability. In cases, poorer reliability was observed for MiniPONS, AIHQ-AB, IPSAQ, TASIT-1 and TASIT-2. In controls, only the MiniPONS, RAD, and AIHQ showed acceptable reliability. Overall, across both groups, the IPSAQ, TASIT-1 and TASIT-2 showed poorer reliability. These three tasks were removed from subsequent association analyses.

Table 2

Test-retest reliability and internal consistency.

	Cases				Controls
	Spearman's rho	ICC	T1 Cronbach's α	T2 Cronbach's α	Spearman's rho	ICC	T1 Cronbach's α	T2 Cronbach's α
Hinting	0.573⁎⁎	0.608⁎⁎	0.640	0.616	0.505⁎⁎	0.551⁎⁎	0.479	0.294
BLERT	0.727⁎⁎	0.727⁎⁎	0.701	0.747	0.513⁎⁎	0.525⁎⁎	0.544	0.548
MiniPONS	0.557⁎⁎	0.572⁎⁎	0.657	0.652	0.618⁎⁎	0.628⁎⁎	0.562	0.527
RAD	0.796⁎⁎	0.790⁎⁎	0.775	0.801	0.776⁎⁎	0.790⁎⁎	0.727	0.780
AIHQ-HB	0.662⁎⁎	0.694⁎⁎	0.626	0.688	0.615⁎⁎	0.644⁎⁎	0.541	0.428
AIHQ-AB	0.514⁎⁎	0.486⁎⁎	0.683	0.667	0.747⁎⁎	0.739⁎⁎	0.545	0.752
AIHQ-BS	0.632⁎⁎	0.634⁎⁎	0.936	0.952	0.780⁎⁎	0.786⁎⁎	0.933	0.947
IPSAQ-EB	0.613⁎⁎	0.606⁎⁎	0.834	0.850	0.468⁎⁎	0.516⁎⁎	0.764	0.774
IPSAQ-PB	0.227⁎	0.219⁎⁎	0.845	0.818	0.558⁎⁎	0.561⁎⁎	0.821	0.814
MSCEIT-PE	0.671⁎⁎	0.702⁎⁎	0.919	0.938	0.589⁎⁎	0.568⁎⁎	0.893	0.940
MSCEIT-FE	0.688⁎⁎	0.659⁎⁎	0.801	0.844	0.548⁎⁎	0.564⁎⁎	0.756	0.821
MSCEIT-UE	0.778⁎⁎	0.778⁎⁎	0.822	0.827	0.827⁎⁎	0.795⁎⁎	0.718	0.744
MSCEIT-ME	0.716⁎⁎	0.731⁎⁎	0.813	0.828	0.561⁎⁎	0.570⁎⁎	0.725	0.764
TASIT-1	0.383⁎⁎	0.432⁎⁎	0.706	0.707	0.340⁎⁎	0.377⁎⁎	0.514	0.680
TASIT-2	0.481⁎⁎	0.461⁎⁎	0.823	0.778	0.438⁎⁎	0.464⁎⁎	0.798	0.863
TASIT-3	0.657⁎⁎	0.635⁎⁎	0.755	0.726	0.525⁎⁎	0.547⁎⁎	0.802	0.794
ER40	0.585⁎⁎	0.606⁎⁎	0.656	0.752	0.575⁎⁎	0.589⁎⁎	0.508	0.602

Note. BLERT = Bell Lysaker Emotion Recognition Task; MiniPONS = Mini Profile of Nonverbal Sensitivity; RAD = Relationships Across Domains; AIHQ-HB = Ambiguous Intentions and Hostility Questionnaire – Hostility Bias; AIHQ-AB = Ambiguous Intentions and Hostility Questionnaire – Aggression Bias; AIHQ-BS = Ambiguous Intentions and Hostility Questionnaire – Blame Score; IPSAQ-EB = Internal Personal and Situational Attributions Questionnaire – Externalizing Bias; IPSAQ-PB = Internal Personal and Situational Attributions Questionnaire – Personalizing Bias; MSCEIT-PE = Mayer-Salovey-Caruso Emotional Intelligence Test – Perceiving Emotions; MSCEIT-FE = Mayer-Salovey-Caruso Emotional Intelligence Test – Facilitating Emotions; MSCEIT-UE = Mayer-Salovey-Caruso Emotional Intelligence Test – Understanding Emotions; MSCEIT-ME = Mayer-Salovey-Caruso Emotional Intelligence Test – Managing Emotions; TASIT-1 = The Awareness of Social Inference Test – Branch 1; TASIT-2 = The Awareness of Social Inference Test – Branch 2; TASIT-3 = The Awareness of Social Inference Test – Branch 3; ER40 = Penn Emotion Recognition Task.

p < 0.01.

p < 0.05.

Internal consistency

Poor internal consistency as indexed by Cronbach's alpha <0.7 was consistently observed for the Hinting task and MiniPONS across study visits and case-control status (Table 2). These two tasks were also removed from subsequent association analyses. While the AIHQ-HB and AIHQ-AB subscale had Cronbach's alpha <0.7, high Cronbach's alpha >0.9 was observed for AIHQ-BS subscale across cases and controls. Hence, the AIHQ task was retained for further analysis.

Utility as a repeated measure

In cases, statistically significant improvement in performance was found for the Hinting task and BLERT, albeit small effect sizes (Table 3). In controls, differences in performance were observed for the MiniPONS, AIHQ-HB, TASIT-1 and TASIT-2, with more pronounced effect sizes, compared to cases (Table 3).

Table 3

Utility as a repeated measure.

	N	T1		T2		P	Cohen's d	Floor/ceiling effect
		Mean	SD	Mean	SD			T1	T2
Cases (n = 108)
Hinting	108	14.31	3.34	14.98	3.24	0.03	0.20	0/3	0/5
BLERT	108	14.26	3.34	15.44	3.37	<0.01	0.35	1/0	1/3
MiniPONS	107	43.42	5.73	44.36	5.87	0.14	0.16	3/0	4/0
RAD	108	27.92	6.12	28.01	6.38	0.80	0.01	21/0	23/0
AIHQ-HB	108	1.38	0.26	1.40	0.30	0.56	0.07	–	–
AIHQ-AB	108	1.40	0.31	1.37	0.29	0.24	−0.09	–	–
AIHQ-BS	108	2.92	0.68	2.93	0.73	0.76	0.01	–	–
IPSAQ-EB	107	1.67	4.40	1.70	4.49	0.99	0.01	–	–
IPSAQ-PB	107	0.56	0.33	0.52	0.34	0.24	−0.11	–	–
MSCEIT-PE	108	0.46	0.14	0.46	0.15	0.99	0.01	–	–
MSCEIT-FE	107	0.43	0.11	0.42	0.12	0.77	−0.06	–	–
MSCEIT-UE	108	0.43	0.10	0.44	0.11	0.51	0.02	–	–
MSCEIT-ME	108	0.32	0.09	0.32	0.10	0.92	0.00	–	–
TASIT-1	108	19.99	4.08	20.38	4.03	0.28	0.10	0/0	0/3
TASIT-2	108	38.96	7.90	37.49	7.36	0.06	−0.19	17/0	15/0
TASIT-3	108	40.93	7.43	40.94	7.06	0.78	0.00	14/0	15/0
ER40	106	30.19	3.82	30.48	4.70	0.30	0.07	0/0	0/0
Controls (n = 70)
Hinting	70	15.63	2.50	15.91	2.20	0.37	0.12	0/5	0/2
BLERT	70	16.83	2.34	17.24	2.20	0.15	0.18	0/0	0/2
MiniPONS	70	47.80	4.79	49.63	4.47	<0.01	0.39	0/0	0/0
RAD	70	32.93	5.13	33.34	5.47	0.30	0.08	2/0	3/0
AIHQ-HB	70	1.41	0.25	1.34	0.22	0.01	−0.30	–	–
AIHQ-AB	70	1.36	0.21	1.35	0.26	0.56	−0.02	–	–
AIHQ-BS	70	2.85	0.54	2.82	0.54	0.65	−0.06	–	–
IPSAQ-EB	70	1.63	3.97	2.11	3.88	0.29	0.12	–	–
IPSAQ-PB	70	0.44	0.28	0.42	0.27	0.28	−0.07	–	–
MSCEIT-PE	70	0.55	0.10	0.54	0.14	0.54	−0.04	–	–
MSCEIT-FE	70	0.47	0.08	0.47	0.10	0.63	0.03	–	–
MSCEIT-UE	70	0.50	0.08	0.51	0.08	0.34	0.09	–	–
MSCEIT-ME	70	0.36	0.08	0.35	0.08	0.59	−0.12	–	–
TASIT-1	70	22.73	2.73	23.40	3.16	0.03	0.23	0/0	0/3
TASIT-2	70	49.76	6.15	44.67	8.41	< 0.01	−0.69	0/3	3/1
TASIT-3	70	50.26	6.72	50.17	6.72	0.77	−0.01	1/0	0/0
ER40	70	33.57	3.05	33.74	3.25	0.57	0.05	0/0	0/1

Note. BLERT = Bell Lysaker Emotion Recognition Task; MiniPONS = Mini Profile of Nonverbal Sensitivity; RAD = Relationships Across Domains; AIHQ-HB = Ambiguous Intentions and Hostility Questionnaire – Hostility Bias; AIHQ-AB = Ambiguous Intentions and Hostility Questionnaire – Aggression Bias; AIHQ-BS = Ambiguous Intentions and Hostility Questionnaire – Blame Score; IPSAQ-EB = Internal Personal and Situational Attributions Questionnaire – Externalizing Bias; IPSAQ-PB = Internal Personal and Situational Attributions Questionnaire – Personalizing Bias; MSCEIT-PE = Mayer-Salovey-Caruso Emotional Intelligence Test – Perceiving Emotions; MSCEIT-FE = Mayer-Salovey-Caruso Emotional Intelligence Test – Facilitating Emotions; MSCEIT-UE = Mayer-Salovey-Caruso Emotional Intelligence Test – Understanding Emotions; MSCEIT-ME = Mayer-Salovey-Caruso Emotional Intelligence Test – Managing Emotions; TASIT-1 = The Awareness of Social Inference Test – Branch 1; TASIT-2 = The Awareness of Social Inference Test – Branch 2; TASIT-3 = The Awareness of Social Inference Test – Branch 3; ER40 = Penn Emotion Recognition Task.

The RAD showed highest floor effect, where approximately 20% of cases performed at chance. Additionally, approximately 15% of cases also performed at chance for TASIT-2 and TASIT-3. These floor or ceiling performances were not evident in controls.

Administration time ranged from 13 to 50 min for majority of the measures, with exception to the Hinting task, BLERT, and ER40, which required <8 min each (Table 4). Significant shorter administration time was reported for most tasks at visit 2, which could potentially suggest familiarly with tasks. Tolerability ratings were found to be poorer for MiniPONS, RAD and TASIT (Table 4). The ER40 showed the best tolerability rating across groups and study visits.

Table 4

Practicality and tolerability.

	Practicality (administration time in minutes)				Tolerability
	Cases		Controls		Cases		Controls
	T1	T2	T1	T2	T1	T2	T1	T2
Hinting	7.44 (2.03)	6.72 (1.92)**	6.24 (1.42)	5.59 (1.08)**	5.42 (1.43)	5.29 (1.44)	5.66 (1.05)	5.60 (1.16)
BLERT	7.25 (1.02)	7.06 (1.03)	7.13 (1.08)	6.93 (0.82)	5.30 (1.35)	5.33 (1.28)	5.41 (1.14)	5.29 (1.19)
MiniPONS	13.57 (1.47)	14.20 (6.67)	13.29 (0.76)	13.03 (0.61)*	4.83 (1.59)	4.96 (1.47)	4.76 (1.56)	5.01 (1.49)
RAD	17.28 (6.61)	13.85 (4.59)**	14.80 (5.05)	11.26 (4.12)**	4.79 (1.59)	4.91 (1.52)	4.90 (1.29)	5.23 (1.21)
AIHQ	18.90 (7.52)	16.87 (6.13)**	16.46 (6.96)	12.37 (4.58)**	5.21 (1.34)	5.10 (1.51)	5.60 (0.95)	5.44 (1.10)
IPSAQ	18.24 (7.78)	18.01 (8.58)	17.07 (7.34)	14.30 (5.75)**	5.17 (1.50)	5.07 (1.56)	4.99 (1.35)	5.21 (1.30)
MSCEITa	37.21 (15.39)	33.14 (16.10)**	28.16 (10.12)	24.43 (9.58)**	5.17 (1.47)	5.06 (1.35)	5.13 (1.35)	5.26 (1.08)
TASITa	48.84 (6.68)	46.55 (4.61)**	44.7 (4.47)	42.89 (3.18)**	4.79 (1.70)	4.68 (1.58)	5.01 (1.31)	4.90 (1.49)
ER40	3.82 (2.02)	3.53 (1.01)	2.94 (0.83)	2.74 (1.02)**	5.58 (1.35)	5.55 (1.39)	5.87 (0.98)	5.99 (0.96)

Note. All values in cells represent mean (SD). **p < 0.01 and *p < 0.05 represents significant difference in task administration time across study visits. BLERT = Bell Lysaker Emotion Recognition Task; MiniPONS = Mini Profile of Nonverbal Sensitivity; RAD = Relationships Across Domains; AIHQ = Ambiguous Intentions and Hostility Questionnaire; IPSAQ = Internal Personal and Situational Attributions Questionnaire; MSCEIT = Mayer-Salovey-Caruso Emotional Intelligence; TASIT = The Awareness of Social Inference Test; ER40 = Penn Emotion Recognition Task.

Outcome measured includes all branches of task.

Case-control discrimination

Overall, schizophrenia patients performed poorer on all social cognitive tasks (Table 5). To identify psychometrically sound social cognitive tasks that could best discriminate case-control status, a stepwise logistic regression was performed with all social cognitive tasks in the regression model, except the Hinting task, MiniPONS, IPSAQ, TASIT-1 and TASIT-2, as these tasks were identified as having inadequate psychometric properties as reported above.

Table 5

Case-control differences on social cognitive measures.

	Cases (n = 116)		Controls (n = 73)		Control-case		P	Cohen's d
	Mean	SD	Mean	SD	Mean	SE
Hinting	14.12	3.41	15.52	2.58	1.40	0.44	<0.01	0.46
BLERT	14.14	3.33	16.77	2.37	2.63	0.41	<0.01	0.91
MiniPONS	43.00	5.91	47.66	4.81	4.66	0.79	<0.01	0.86
RAD	27.71	6.18	32.85	5.05	5.14	0.83	<0.01	0.91
AIHQ-HB	1.37	0.25	1.41	0.25	0.04	0.04	0.21	0.16
AIHQ-AB	1.40	0.30	1.36	0.20	−0.05	0.04	0.57	−0.18
AIHQ-BS	2.93	0.67	2.86	0.53	−0.06	0.09	0.29	−0.10
IPSAQ-EB	1.59	4.31	1.85	4.19	0.26	0.64	0.52	0.06
IPSAQ-PB	0.56	0.33	0.44	0.28	−0.12	0.04	0.02	−0.38
MSCEIT-PE	0.46	0.14	0.55	0.10	0.09	0.02	<0.01	0.74
MSCEIT-FE	0.42	0.11	0.47	0.08	0.05	0.01	<0.01	0.49
MSCEIT-UE	0.43	0.11	0.50	0.08	0.07	0.01	<0.01	0.75
MSCEIT-ME	0.31	0.09	0.36	0.08	0.05	0.01	<0.01	0.58
TASIT-1	19.82	4.08	22.78	2.72	2.96	0.50	<0.01	0.85
TASIT-2	38.60	7.94	49.79	6.12	11.19	1.03	<0.01	1.58
TASIT-3	40.63	7.42	50.21	6.89	9.58	1.08	<0.01	1.34
ER40	30.03	4.06	33.52	3.01	3.49	0.52	<0.01	0.98

Note. BLERT = Bell Lysaker Emotion Recognition Task; MiniPONS = Mini Profile of Nonverbal Sensitivity; RAD = Relationships Across Domains; AIHQ-HB = Ambiguous Intentions and Hostility Questionnaire – Hostility Bias; AIHQ-AB = Ambiguous Intentions and Hostility Questionnaire – Aggression Bias; AIHQ-BS = Ambiguous Intentions and Hostility Questionnaire – Blame Score; IPSAQ-EB = Internal Personal and Situational Attributions Questionnaire – Externalizing Bias; IPSAQ-PB = Internal Personal and Situational Attributions Questionnaire – Personalizing Bias; MSCEIT-PE = Mayer-Salovey-Caruso Emotional Intelligence Test – Perceiving Emotions; MSCEIT-FE = Mayer-Salovey-Caruso Emotional Intelligence Test – Facilitating Emotions; MSCEIT-UE = Mayer-Salovey-Caruso Emotional Intelligence Test – Understanding Emotions; MSCEIT-ME = Mayer-Salovey-Caruso Emotional Intelligence Test – Managing Emotions; TASIT-1 = The Awareness of Social Inference Test – Branch 1; TASIT-2 = The Awareness of Social Inference Test – Branch 2; TASIT-3 = The Awareness of Social Inference Test – Branch 3; ER40 = Penn Emotion Recognition Task.

The logistic regression model significantly identified four tasks (BLERT, ER40, TASIT-3, and AIHQ-HB) that best discriminated case-control status, ??2(4, n = 183) = 65.01, p < 0.001, Nagelkerke R2 = 0.404 (Supplementary Table 1). The positive and negative predictive values are 76.4% and 70.3% respectively. The sensitivity and specificity of model classification are 82.7% and 61.6% respectively.

Principal components analysis (PCA)

Data suitability for PCA assessed with the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and the Bartlett's test of sphericity indicated that the schizophrenia sample was appropriate for PCA (KMO = 0.655, χ2 (6) = 50.85, p < 0.001).

PCA indicated presence of two components in the schizophrenia sample (Table 6), as indexed by eigen >1 and scree plot explaining 45.44% and 25.19% of the variance. The BLERT, ER40, and TASIT-3 loaded on the first component, while the second component consisted of AIHQ-HB only. The components were named as Social Cognitive Skills and Hostile Attribution Style respectively. Factor scores computed by the summation of raw items loaded onto each component indicated that both components were not inter-correlated (r = 0.001, p > 0.05). Correlations between the components and symptoms found only a significant relationship between the Hostile Attribution Style and the BPRS resistance factor (r = 0.228, p = 0.015), which consisted of items measuring hostility, uncooperativeness and suspiciousness.

Table 6

Principal components of social cognition in schizophrenia.

	Social cognitive skills	Hostile attribution style
BLERT	0.783	−0.021
ER40	0.789	−0.107
TASIT-3	0.761	0.121
AIHQ-HB	−0.003	0.992

Note. Values in bold represents items with factor loading ≥0.4. BLERT = Bell Lysaker Emotion Recognition Task; ER40 = Penn Emotion Recognition Task; TASIT-3 = The Awareness of Social Inference Test – Branch 3; AIHQ-HB = Ambiguous Intentions and Hostility Questionnaire – Hostility Bias.

Discussion

This study identified three social cognitive tasks (BLERT, ER40, TASIT-3) that showed acceptable psychometric properties. The AIHQ displayed slightly weaker psychometric properties, requiring further evaluation. Five tasks showed poorer psychometric properties (Hinting task, MiniPONS, RAD, IPSAQ, and MSCEIT) in our sample. Results were consistent with that reported in the SCOPE study where the BLERT and ER40 possessed the strongest psychometric properties, whereas weaker but promising characteristics were reported for TASIT-3 (Pinkham et al., 2018, Pinkham et al., 2016). None of the tasks identified fell within the social perception domain, similar to findings from the SCOPE study (Pinkham et al., 2018, Pinkham et al., 2016).

Out of the identified tasks, the BLERT and ER40 proved to be most favorable. Both tasks showed little evidence of floor/ceiling effects and had better tolerability ratings and shorter administration time, which are important in clinical implementation. Despite both tasks indexing emotional processing, the ER40 is static in nature given that photographs expressing emotions were presented (Kohler et al., 2003), whereas the BLERT provides dynamic videos with facial and prosody cues which mimic real world situations (Bell et al., 1997). This reconciles the findings on differential association of aspects of social cognition with functional capacity and functional performance (Couture et al., 2006; Mancuso et al., 2011) with the ER40 being associated with functional capacity (Pinkham et al., 2018) while BLERT appeared more proximal to real world functional outcome rated by informants (Pinkham et al., 2016). Together, this suggests that different tasks could be related to different aspects of outcome measures, albeit indexing the same social cognitive domain.

The TASIT-3 showed acceptable test-retest reliability and internal consistency (Davidson et al., 2018; Pinkham et al., 2018). Although a small floor effect was observed in cases, this was not true for the controls. It is posited that this dichotomy could indicate the ability of TASIT-3 to identify graded performance differences across individuals, rather than a psychometric flaw. This was supported by research indicating the ability of TASIT-3 to differentiate clusters of social cognitive impairment within the schizophrenia population (Rocca et al., 2016).

Contrary to the SCOPE study (Pinkham et al., 2016), the present study identified the AIHQ-HB as a potentially valid social cognitive measure assessing the attributional style/bias domain, with acceptable test-retest reliability reported in the controls, and slightly poorer test-retest for the AIHQ-AB in the cases. A possible explanation for this could be that the SCOPE only administered items on ambiguous scenarios (Pinkham et al., 2016); this study administered the full AIHQ which consisted of vignettes on ambiguous, accidental and intentional scenarios. While caution for its use was informed by SCOPE phase 3 result (Pinkham et al., 2016), an extension of their findings indicated that the AIHQ could prove useful in providing additional information on its contributory role to paranoia and hostility symptoms, and its association to interpersonal difficulties (Buck et al., 2016b). More recently, Buck et al. (2017) found that the addition of accidental scenarios, beyond ambiguous scenarios, resulted in stronger contributions to functional capacity for the self-rated domain of AIHQ-BS and modest associations with role functioning for the rater-scored domain of AIHQ-HB. This suggests that the inclusion of both ambiguous and accidental scenarios improves its association to functional outcomes (Pinkham et al., 2016).

The MiniPONS, RAD, IPSAQ and Hinting task demonstrated poor psychometric properties in our population. Of these, the MiniPONS, RAD and IPSAQ replicated previous results that precluded them from recommendations due to lack of association to functional outcomes (Davidson et al., 2018; Pinkham et al., 2018, Pinkham et al., 2016). In contrast, the Hinting task showed less favorable psychometric properties than in previous studies (Davidson et al., 2018; Pinkham et al., 2018, Pinkham et al., 2016). A plausible explanation could be that the Hinting task examines inferential ability through presentation of short vignettes which could be culturally sensitive. Further investigation is recommended, particularly, whether the use of culturally appropriate vignettes would improve the psychometric properties of the task.

Consistent with the literature, the current study demonstrated a clear delineation between social cognitive skills and attributional bias (Buck et al., 2016a; Mancuso et al., 2011; van Hooren et al., 2008). The social cognitive skills factor included tasks comprising the theoretical domains of emotional processing and theory of mind (Browne et al., 2016; Buck et al., 2016a), and that are indexed according to accuracy of performance. The lack of correlation between the two identified factors highlights the conceptual differences between the constructs whereby the social cognitive skills domain requires the accurate identification of other's thoughts and emotions and is closely related to functioning, whereas the attributional style/bias domain refers to a cognitive bias on other's behavior, independent of accurate identification, with stronger relations to symptoms than functioning (Browne et al., 2016; Buck et al., 2016a). This association between the attribution style factor and suspiciousness symptoms was also observed in our sample. This two-dimensional structure of social cognition could be of clinical relevance where individuals with general social cognitive skills deficits undergo remediation based on their functional weaknesses, while individuals with attributional bias could undergo cognitive restructuring interventions targeted at challenging their hostile attributions (Buck et al., 2016a). However, it is noted that the attributional style/bias domain only consist of one item, the AIHQ-HB, suggesting that this component may not be stable. Moreover, the AIHQ-HB was only identified in the stepwise regression analysis but not in the other case control discrimination analyses. Nevertheless, as studies have shown the utility of AIHQ (Buck et al., 2016b, Buck et al., 2017), further psychometric evaluation of AIHQ-HB is recommended.

This study has several limitations. First, this study has unequal case to control ratio, with differences in demographic profiles, and lack of matching for age, education and ethnicity. Even though this is the first study in Asia to look at a wide battery of social cognitive tests based on the SCOPE study, further research should aim to recruit a larger matched control group, given that sample size may have affected the psychometric properties of the social cognitive tests. Second, the schizophrenia sample was composed of relatively stable outpatients which may not generalize to individuals with more severe symptoms. Additional information on patients in acute phase and how it may influence social cognitive performance may prove useful. Third, as a first of its kind study, the tests were administered verbatim, without any amendments to test instructions or items, which necessitated the recruitment of only English-speaking individuals. Given that social cognitive tests may be influenced by culture and concurrently, language, the current findings may not be generalizable to individuals whose first language is not English. Further translation of these social cognitive measures and administration to different populations are warranted. In addition, methodological and/or cultural modification and adaption of tasks that exhibited poorer psychometric properties in this study are vital towards improving our understanding of the tests' utility. Fourth, the study did not include functional and neurocognitive measures which guided the selection of suitable social cognitive tasks for use in clinical trials in the SCOPE study. Subsequent research should look at clarifying the link between these measures and functional outcomes, given that an initial indication of utility has been established.

In conclusion, we provide the first report of the psychometrics of a comprehensive list of SCOPE social cognitive measures in an Asian population. This study partially replicated previous SCOPE findings (Pinkham et al., 2018, Pinkham et al., 2016) where we have identified three psychometrically sound measures (BLERT, ER40, TASIT-3) in our Asian population. The AIHQ showed slightly poorer psychometric properties, requiring further evaluation. While the Hinting task, MiniPONS, RAD, IPSAQ, and MSCEIT showed poorer psychometric properties in our sample, it may be premature to exclude these tasks given that they may yet show associations with functional outcomes and may also benefit from modifications for use in our context. Our data supports a clear separation of the social cognitive skills and attribution style/bias domain. Continued refinements of social cognitive measures are needed to provide clearer insights on the pathophysiology of this construct in schizophrenia and its discriminatory value with aspects of functional outcomes and symptomatology.

The following is the supplementary data related to this article.

Supplementary Table 1

Logistic regression of social cognition measures on case-control status.

Declaration of competing interest

All authors declare that they have no conflict of interest.