Identification and predictive analysis for participants at ultra-high risk of psychosis: A comparison of three psychometric diagnostic interviews.

BACKGROUND: An accurate identification of individuals at ultra-high risk (UHR) based on psychometric tools to prospectively identify psychosis as early as possible is required for indicated preventive intervention. The diagnostic comparability of several psychometric tools, including the comprehensive assessment of at risk mental state (CAARMS), the structured interview for psychosis-risk syndrome (SIPS) and the bonn scale for the assessment of basic symptoms (BSABS), is unknown. AIM: To address the psychometric comparability of CAARMS, SIPS and BSABS for subjects who are close relatives of patients with schizophrenia.
METHODS: In total, 189 participants aged 18-58 years who were lineal relative by blood and collateral relatives by blood up to the third degree of kinship of patients with schizophrenia were interviewed in the period of May 2017 to January 2019. Relatives of the participants diagnosed schizophrenia were excluded. All the participants were assessed for a UHR state by three psychometric tools (CAARMS, SIPS and BSABS). The psychometric diagnosis results included at risk of psychosis (UHR+), not at risk of psychosis (UHR-) and psychosis. Demographic and clinical characteristics were also measured. The inter-rater agreement was assessed for evaluation of the coherence of the three scales. Transition rates for UHR+ subjects to psychosis within 2 years were also recorded.
RESULTS: The overall agreement percentages were 93.12%, 92.06% and 93.65% of CAARMS and SIPS, SIPS and BSABS and CAARMS and BSABS, respectively. The overall agreement percentage of the relative functional impairment of the three groups (UHR+, not at risk of psychosis and psychosis) were 89.24%, 86.36% and 88.12%, respectively. The inter-rater reliability of the CAARMS, SIPS and BSABS total score was 0.90, 0.89 and 0.85. The inter-rater reliability was very good to excellent for all the subscales of these three instruments. For CAARMS, SIPS and BSABS, the kappa coefficient about UHR criteria agreement was 0.87, 0.84 and 0.82, respectively (P < 0.001). The transition rates of UHR+ to psychosis within 2 years were 16.7% (CAARMS), 10.0% (SIPS) and 17.7% (BSABS).
CONCLUSION: There is good diagnostic agreement between the CAARMS, SIPS and BSABS towards identification of UHR participants who are close relatives of patients with schizophrenia. ©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved.

Core Tip: To address the psychometric comparability of the comprehensive assessment of at risk mental state, Structured Interview for Psychosis-Risk Syndrome and Bonn Scale for the Assessment of Basic Symptoms for the assessment of participants who were lineal relative or collateral relatives by blood up to three generations of patients with schizophrenia, 189 participants were interviewed for an ultra-high risk state. The final conclusion was that there is good diagnostic agreement among these three instruments. Also, these three instruments may assess and detect at-risk mental states in these participants reliably and validly.

INTRODUCTION

Indicated preventive intervention brings new hope for impacting the course of psychosis since treatments for psychosis substantially improve outcomes[1,2]. Therefore, an accurate identification of individuals at clinical high risk (CHR) based on psychometric tools to prospectively identify psychosis as early as possible is required to allow preventative screening, diagnosis and interventions[3-5].

During the development of psychiatry, psychometric tools were created, analyzed and confirmed. These tools include the comprehensive assessment of at risk mental state (CAARMS), the Structured Interview for Psychosis-Risk Syndrome (SIPS) and the Basel Screening Instrument for Psychosis for the assessment of “ultra-high risk” (UHR) patients. The Bonn Scale for the Assessment of Basic Symptoms (BSABS) and the Schizophrenia Proneness Instruments are used to assess basic symptoms.

A screening test should identify those potential individuals developing the disease[6], and a prognostic test is necessary for prediction of the future disease development when a patient has ominous signs or symptoms. However, criteria for UHR as a screening test rely on subjectively experienced disturbances of perception, thinking, language and attention[7].

Both the CAARMS and the SIPS can distinguish UHR subjects from large group of individuals with high-risk services for potential UHR symptoms. Moreover, the CAARMS and the SIPS demonstrate the similar construct and criteria, which show same predictive values in the follow-up[8,9]. Meanwhile, the prevalence of the condition would affect the predictive values which are not fixed indicators[6].

However, there is little evidence of a single recognized standard among these instruments for UHR identification in China, especially in participants who are lineal relative or collateral relatives of schizophrenia patients. The development of future large-scale UHR multicenter studies was affected significantly by psychometric uncertainty because of amplifying heterogeneity across individual sites. These concerns and conjecture have never been examined practically.

We present this study of UHR assessment in participants who are lineal relative or collateral relatives by blood up to three generations of patients with schizophrenia by using the CAARMS, SIPS and BSABS. Our principal aim was to address the psychometric comparability of the CAARMS, SIPS and BSABS for these participants. Our secondary aim was to verify the viability and reliability of these three instruments for these participants.

MATERIALS AND METHODS

Samples

For the research group, we included participants who were a lineal relative by blood and collateral relative by blood up to the third degree of kinship of patients with schizophrenia diagnosed in the Affiliated Wuhan Mental Health Center, Tongji Medical College of Huazhong University of Science & Technology from May 2017 to January 2019. All the participants were assessed for UHR by three psychometric tools, including CAARMS, SIPS and BSABS. Participants were recruited from the Wuhan Mental Health Center and were able to be contacted by telephone or an internet homepage.

Procedure and clinical measures

The CAARMS, a semi-structured clinical interview, covers different aspects of attenuated psychopathology or functioning. It consists of 27 items, each item rated in terms of intensity from 0 to 6 and frequency/duration from 0 to 6, which can be classified into seven subscales, including positive symptoms, cognitive change, attention/concentration/emotional disturbance, negative symptoms, behavioral change, motor/physical changes and general psychopathology. Positive symptoms, including delusions, hallucinations and thought disorder, of the CAARMS are used to determine both the UHR criteria and the threshold for psychosis.

The SIPS, a semi-structured clinical interview, consists of six parts, including Family History Questionnaire, The scale of psychosis-risk symptoms, the global assessment of functioning (GAF), schizotypal personality disorder checklist (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition), Summary of SIPS data and Summary of SIPS syndrome criteria. The symptoms score from 0 to 6 in terms of intensity, frequency/duration, influence and degree of conflict.

The BSABS, a semi-structured clinical interview, consists of 92 items classified into six rating scales, including adynamia (A + B), cognitive disorder (C), cenesthesia experience (D), dysfunction of central autonomic nerve (E) and self-protection (F).

All the participants participated in the CAARMS, SIPS and BSABS assessments for early detection of schizophrenia. At the end of the diagnostic interview assessment, the psychometric diagnosis results included at risk of psychosis, not at risk of psychosis, and psychosis. Demographic and clinical characteristics were also measured.

The inter-rater agreement was assessed for the evaluation of the coherence of the three instruments. The transition rates of psychosis from at risk of psychosis individuals within 2 years were also recorded.

Statistical analysis

All statistical analyses were performed using SPSS 22.0 (IBM Corp., Armonk, NY, United States). Intra-class correlation coefficients were calculated to estimate intra-scale reliability, and the kappa coefficient was calculated to evaluate the inter-scale and inter-rater agreement on the diagnosis. Comparisons among groups were examined using the Kruskal-Wallis test, and post-hoc analyses were performed using the Mann-Whitney U test with Bonferroni correction. P < 0.05 indicated statistical significance.

RESULTS

Samples and raters characteristics

The research group consisted of 189 participants who were lineal or collateral relatives of schizophrenia patients who were diagnosed between May 2017 and January 2019.

Of the research group participants, 68 were females (35.98%). The mean age was 35.54 years (standard deviation = 4.15, range = 18-58 years).

Diagnostic comparison of CAARMS, SIPS and BSABS

Diagnostic comparison of CAARMS and SIPS (Table The overall agreement percent was 93.12% (expected agreement by chance: 35.12%), and the kappa was a substantial 0.745 [95% confidence interval (CI): 0.663-0.859]. The analysis weighted for the relative functional impairment of the three groups (at risk of psychosis, not at risk of psychosis and psychosis) was determined. The overall agreement percent was 89.24% (expected agreement: 46.38%), and the kappa was 0.796 (95%CI: 0.681-0.895).

Diagnostic comparison of SIPS and BSABS (Table The overall agreement percent was 92.06% (expected agreement by chance: 31.93%), and the kappa was a substantial 0.728 (95%CI: 0.648-0.825). The relative functional impairment of the three groups was analyzed. The overall agreement percent was 86.36% (expected agreement: 44.27%), and the kappa was 0.759 (95%CI: 0.676-0.854).

Diagnostic comparison of CAARMS and BSABS (Table The overall agreement percent was 93.65% (expected agreement by chance: 34.07%), and the kappa was a substantial 0.767 (95%CI: 0.678 - 0.881). The analysis for the relative functional impairment of all the groups was performed. The overall agreement percent and the kappa was 88.12% (expected agreement: 45.52%) and 0.778 (95%CI: 0.680-0.873), respectively.

Inter-rater correlation coefficients of CAARMS, SIPS and BSABS subscales

The inter-rater reliability of the CAARMS, SIPS and BSABS total score were 0.90, 0.89 and 0.85, respectively. The inter-rater reliability of these three scales ranged from very good to excellent for the seven subscales of CAARMS, six subscales of SIPS and five subscales of BSABS. The kappa coefficient for the agreement on the UHR criteria among three raters of theses three scales were 0.87, 0.84 and 0.82, respectively (P < 0.001) (Table 4).

Table 4

Inter-rater correlation coefficients of the comprehensive assessment of at-risk mental states, the structured interview for psychosis-risk syndrome and the bonn scale for the assessment of basic symptoms subscales (n = 30, P < 0.001)

	ICC
CAARMS Subscale
Positive symptoms	0.93
Cognitive change	0.75
Attention/concentration/emotional disturbance	0.74
Negative symptoms	0.88
Behavioral change	0.75
Motor/physical changes	0.85
General psychopathology	0.93
Total	0.9
SIPS Subscale
Family History Questionnaire	0.9
The Scale of Psychosis-Risk Symptoms	0.8
The Global Assessment of Functioning	0.72
Schizotypal Personality Disorder Checklist (DSM-5)	0.81
Summary of SIPS data	0.92
Summary of SIPS syndrome criteria	0.88
Total	0.89
BSABS Subscale
Adynamia (A + B)	0.87
Cognitive disorder €	0.76
Cenesthesia experience (D)	0.82
Dysfunction of central autonomic nerve €	0.88
Self-protection (F)	0.89
Total	0.85

ICC: Intra-class correlation; CAARMS: Comprehensive assessment of at-risk mental states; SIPS: Structured interview for psychosis-risk syndrome; BSABS: Bonn scale for the assessment of basic symptoms; DSM-5: Diagnostic and statistical manual of mental disorders fifth edition.

Transition rates of at risk of psychosis subjects to psychosis within 2 years

The transition rates of at risk of psychosis to psychosis within 2 years were 16.7% (CAARMS), 10.0% (SIPS) and 17.7% (BSABS) (Table 5).

Table 5

Transition rates of at risk of psychosis participants to within 2 years

	UHR+	Outcomes after 2 years follow-up		Transition rates, %
		Psychosis	No psychosis
CAARMS	12	2	10	16.7
SIPS	10	1	9	10
BSABS	17	3	14	17.7

CAARMS: Comprehensive assessment of at-risk mental states; SIPS: Structured interview for psychosis-risk syndrome; BSABS: Bonn scale for the assessment of basic symptoms; UHR+: At risk of psychosis.

DISCUSSION

An accurate identification of individuals at CHR, which is the preventive intervention for psychosis, will substantially improve the outcomes of these individuals. The use of accurate and proper tools to detect psychosis as early as possible for prognosis is very necessary[2]. The aim of this study was to test the comparability of three psychometric instruments (CAARMS, SIPS and BSABS) most frequently used in China to interview participants who were lineal or collateral relatives of patients with schizophrenia. The results indicated that these three instruments have good psychometric properties, and all were reliable and valid for early identification and predictive analysis in this population. Moreover, there were good inter-rater correlation coefficients of each scales.

A prodromal phase named the at-risk mental state (ARMS) precedes most psychotic disorders. Though it is unclear about the most effective therapies, the remained time of and sustained effects, the potential intervention for ARMS groups may take effect on preventing or delaying the onset of psychosis and then improve the outcome for the reduction of untreated psychosis period. In some earliest UHR studies, transition rates of first year from ARMS to psychosis were about 40%[10,11], while a later study reported the transition rate of ARMS to psychosis as 7%-16% within 2 years[12].

Specific psychometric interviews that assess validated CHR criteria are usually accomplished with prognostic testing[13], such as the CAARMS[14], the SIPS[15] and the BSABS[16]. These instruments make a comprehensive analysis for CHR through age, social function, family history of psychosis, symptom score, frequency and duration of symptoms.

Daneault et al[17] claimed that the development of the SIPS was influenced by the CAARMS. The first aim of this study was to verify the diagnostic comparability of CAARMS vs SIPS and BSABS in 189 participants who were lineal relatives and collateral relative of patients with schizophrenia. There was overall substantial agreement (kappa) between the CAARMS and SIPS, SIPS and BSABS and CAARMS and BSABS. These three instruments show similar psychometric parameters, such as excellent reliability properties. A parallel proportion of true positives over time was shown in a previous study about CAARMS and SIPS[9]. In a recent meta-analysis, help-seeking individuals interviewed with CAARMS and SIPS showed similar excellent prognostic accuracy in ruling out psychosis risk[18].

Different CAARMS or SIPS versions in different countries were compared in previous studies. An excellent reliability was shown in these CHR scales used by trained raters: the overall inter-rater agreement was 0.95, 0.85 and 0.91 for the SIPS[19], the CAARMS[20] and the Schizophrenia Proneness Instruments Adult version[21], respectively. Pelizza et al[22] tested the reliability and validity for the help-seeking population evaluated by the authorized Italian version of the CAARMS. The results indicated that CAARMS version may assess and detect ARMS reliably and validly in an Italian polulation, which may also predict transition to psychosis helpfully. Another study observed that there was overall substantial diagnostic agreement between the CAARMS 12/2006 and the SIPS 5.0 in the identification of UHR subjects[23]. We also observed in this study that the CAARMS and the SIPS had similar reliability and validity when used to interview the special population of close relatives of patients with schizophrenia.

Moreover, reliability of the CAARMS, SIPS and BSABS in this study was assessed by inter-rater reliability and internal consistency. The intra-class correlation coefficients of three scales subscale displayed good to excellent reliability, which was similar to the original validation study[12]. The inter-rater reliability for the overall score was 0.90 for total subscales. These findings demonstrate that all the three instruments can be evaluate the early identification of schizophrenia in the population of lineal relative of these patients clinically. Furthermore, the inter-rater reliability of the UHR inclusion criteria of the three scales was also approving. Cronbach’s alpha coefficient reflecting internal consistency for the CAARMS total score was 0.89.

In this study, a 2-year follow-up showed that the transition rates of UHR to psychosis were 16.7% (CAARMS), 10.0% (SIPS) and 17.7% (BSABS). These rates were slightly higher than in previous studies[10-12]. We hypothesize that the participants selected for this study were at a higher risk of psychosis because they were lineal relatives or collateral relatives in three generations of patients with schizophrenia.

If comparability, viability, reliability and practicability are considered, then we suggest that CAARMS and BSABS are easier and more convenient for interviewing participants who are close relatives including lineal or collateral relatives by blood up to three generations of schizophrenia patients in China.

This study had some limitations. First, a long-term follow-up was not performed. Second, the recruitment type may have impacted the observed substantial agreement between the three instruments[24]. Also, it is possible that the UHR patients who did not meet the SIPS criteria were undetected by the referrers. This may have inflated the observed agreement. In one study, there were significant differences between the CAARMS and the SIPS in other epidemiological samples of non-help-seeking subjects[25].

CONCLUSION

There is good diagnostic agreement between the CAARMS, SIPS and BSABS towards identification of CHR participants who are close relatives of patients with schizophrenia. Also, the three instruments are reliable and valid for assessing and detecting at-risk mental states in these subjects.

ARTICLE HIGHLIGHTS

Research background

Indicated preventive intervention is the new hope for affecting the psychosis progress since treatments for psychosis substantially improve outcomes. Therefore, an accurate identification of individuals at ultra-high risk (UHR) based on psychometric tools to prospectively identify psychosis as early as possible is required to allow preventative screening, diagnosis and interventions. With the development of psychiatry, psychometric tools have been created, analyzed and confirmed. There is little evidence of a single recognized standard among these instruments for UHR identification in China.

Research motivation

In total, 189 participants who were the lineal relative or collateral relatives by blood up to the third degree of kinship of schizophrenia patients were interviewed to identify a UHR state by three psychometric tools, including the comprehensive assessment of at-risk mental states (CAARMS), the Structured Interview for psychosis-risk syndrome (SIPS) and the bonn scale for the assessment of basic symptoms (BSABS), which are the most common instruments in China.

Research objectives

To address the psychometric comparability of the CAARMS, SIPS and BSABS for assessment of close relative of schizophrenia patients and to verify the viability and reliability of these three instruments for these participants.

Research methods

All of the participants were assessed for a UHR state by the CAARMS, SIPS and BSABS. The psychometric diagnosis results included at risk of psychosis, not at risk of psychosis and psychosis. Demographic and clinical characteristics were also measured. The inter-rater agreement was assessed for evaluation of the coherence of the three instruments. The transition rates of at risk of psychosis to psychosis within 2 years were also recorded.

Research results

The overall agreement percentages were 93.12% for CAARMS and SIPS, 92.06% for SIPS and BSABS and 93.65% for CAARMS and BSABS. Moreover, the inter-rater reliability of the CAARMS, SIPS and BSABS total score was 0.90, 0.89 and 0.85, respectively. For all the subscales of these three scales, the inter-rater reliability varied from very good to excellent. The transition rates of at risk of psychosis to psychosis within 2 years were about 16.7% (CAARMS), 10.0% (SIPS) and 17.7% (BSABS).

Research conclusions

It showed a good diagnostic agreement between the CAARMS, SIPS and BSABS in identification of UHR participants who are close relative of patients with schizophrenia. Also, these three instruments are reliable and valid tools for at-risk mental states assessment and detection in these participants.

Research perspectives

The lineal and collateral relatives by blood up to the third generations of schizophrenia patients are clinical high-risk participants. Early detection aids in initiation of preventive intervention and can provide substantially improved outcomes. A multicenter interview and follow-up of these participants by different instruments will provide more experience and value for clinical high-risk participants.

Table 1

Diagnostic comparison between the comprehensive assessment of at-risk mental states and the structured interview for psychosis-risk syndrome outcomes in participants of the research group (P < 0.001)

			SIPS outcomes			Total
			UHR-	UHR+	Psychosis
CAARMS outcomes	UHR-	Count	162	0	0	162
		Ratio, %	85.71	0	0	85.71
	UHR+	Count	4	5	3	12
		Ratio, %	2.11	2.65	1.59	6.35
	Psychosis	Count	1	5	9	15
		Ratio, %	0.53	2.65	4.76	7.94
	Total	Count	167	10	12	189
		Ratio, %	88.36	5.3	6.35	100

CAARMS: Comprehensive assessment of at-risk mental states; SIPS: Structured interview for psychosis-risk syndrome; UHR: Ultra-high risk; UHR+: At risk of psychosis; UHR-: Not at risk of psychosis.

Table 2

Diagnostic comparison between the structured interview for psychosis-risk syndrome and the bonn scale for the assessment of basic symptoms outcomes in participants of the research group (P < 0.001)

			SIPS outcomes			Total
			UHR-	UHR+	Psychosis
BSABS outcomes	UHR-	Count	159	0	0	159
		Ratio, %	84.13	0	0	84.13
	UHR+	Count	6	7	4	17
		Ratio, %	3.17	3.7	2.12	8.99
	Psychosis	Count	2	3	8	13
		Ratio, %	1.06	1.59	4.23	6.88
	Total	Count	167	10	12	189
		Ratio, %	88.36	5.29	6.35	100

SIPS: Structured interview for psychosis-risk syndrome; BSABS: Bonn scale for the assessment of basic symptoms; UHR: Ultra-high risk; UHR+: At risk of psychosis; UHR-: Not at risk of psychosis.

Table 3

Diagnostic comparison between the comprehensive assessment of at-risk mental states and the bonn scale for the assessment of basic symptoms outcomes in participants of the research group (P < 0.001)

			CAARMS outcomes			Total
			UHR−	UHR+	Psychosis
BSABS outcomes	UHR−	Count	158	1	0	159
		Ratio, %	83.6	0.53	0	84.13
	UHR+	Count	3	9	5	17
		Ratio, %	1.58	4.76	2.65	8.99
	Psychosis	Count	1	2	10	13
		Ratio, %	0.53	1.06	5.29	6.88
	Total	Count	162	12	15	189
		Ratio, %	85.71	6.35	7.94	100

CAARMS: Comprehensive assessment of at-risk mental states; BSABS: Bonn scale for the assessment of basic symptoms; UHR: Ultra-high risk; UHR+: At risk of psychosis; UHR-: Not at risk of psychosis.