Psychometric properties of the brief self-report questionnaire for screening putative pre-psychotic states and validation of clinical utility in young adult.

INTRODUCTION: The Brief Self-Report Questionnaire for Screening Putative Pre-Psychotic States (BQSPS), a brief, self-reported screening tool for risk of psychosis, can detect auditory perceptual disturbances significantly associated with perceived need for psychological services among young adults. However, the relationship is largely explained by the existence of neurotic traits, anxiety and depression symptoms.
OBJECTIVE: This study explores possible explanations of previous results from factor structures of the BQSPS and the clinical implications underlying each factor.
METHODS: Construct validity, criterion-related validity, discriminant validity, internal consistency, and test-retest reliability of the BQSPS are determined among young adults (N = 289).
RESULTS: We find that Social Anxiety, Positive Symptoms, and Negative Symptoms are three components in the BQSPS for young adults. Moreover, we find that each component of the BQSPS can be explained by related forms of psychopathology, self-esteem, or personality traits. Finally, the BQSPS can satisfactorily distinguish cases from non-cases using the Symptoms Check List-90-Revised.
CONCLUSIONS: We clarify the clinical implications of each component of the BQSPS and thus expand its clinical utility. The BQSPS has good psychometric properties in young adults from an ethnically Chinese population. Limitations and directions for future research are also discussed.

Introduction

The peak onset for many mental health disorders is young adulthood [1], with the first onset by 25 years of age for 75% of those who will have a mental health disorder [2]. Mental health problems are prevalent among college students, with anxiety disorders being their most common psychiatric problem [3]. Depression is also frequently seen in this population [4], as are various psychotic symptoms [1]. These issues can be related to stressors for college students, including academic load, first-time working, being in a committed personal relationship, and living with others from different cultures and belief systems [5]. Many college students may experience the persistence, increase, or the first onset of mental health and substance use problems [1].

Therefore, developing strategies to identify individuals at high risk of clinical first-episode psychosis is a significant current goal for psychiatric services worldwide [6], especially for young adults. These strategies focus on the early detection of subjects who show only subthreshold symptoms, including positive and negative symptoms or functional difficulties appearing in the prodrome phase of psychosis [7]. Measures for the early detection of people at risk of psychosis have been created and tested during the last two decades [8]. As part of this strategy, the Brief Self-Report Questionnaire for Screening Putative Pre-Psychotic States (BQSPS) was developed [9], with confirmed reliability and validity [8, 10].

The BQSPS targets early and extensive at-risk mental states characterized by subtle symptoms and functional impairments, and it is unlike other questionnaires developed to improve the predictive validity for transition to psychosis [11]. Moreover, in addition to evaluating attenuated positive symptoms, like most screening questionnaires [12], the BQSPS also includes other subthreshold clinical manifestations. Liu et al. suggested two cut-off selections of the BQSPS: (a) respondents checking at least eight items, or (b) those checking three to seven items, including any of three specific items. These two criteria could obtain the largest sensitivity+specificity (0.784+0.705 = 1.489). It has construct validity and can reliably distinguish between clinical and non-clinical samples [9]. Psychometric properties of the BQSPS studies have been verified by two studies. Demmin et al. found moderate to large convergent validity, acceptable internal consistency for each scale, and modest test-retest reliability, recommending its usage for screening psychotic-like experience in college populations [10]. Similarly, Nunez et al. found a stable structure of three correlated factors: social anxiety (SA), negative symptoms (NS), and positive symptoms (PS) in adolescent and young adult subjects [8]. This three-factor model also had the predictive ability for suicidality as an external criterion.

An individual’s personality is increasingly recognized as affecting the possibility of psychopathological developments [13]. For example, self-esteem can substantially affect how psychotic symptoms are formed and maintained, as well as recovery from the illness [14]. In particular, low self-esteem may be both a causative factor and a result of a severe mental disorder [15]. This connection is supported by the twin study of Macare et al., which found substantial genetic overlap between schizotypy and neuroticism [16], and the finding of Goodwin et al. that early neuroticism appearing in adolescence may indicate the later development of psychotic symptoms in adulthood [17]. Longitudinal studies have shown both that neuroticism can increase the risk of psychotic symptoms [18], and conversely that extraversion can help to avoid or mitigate depression and social anxiety [19].

The Symptom Checklist-90-R (SCL-90-R) is a self-rating-scale for assessing general psychopathology and specific symptoms [20], used to distinguish potential psychosis-like pathology as a short screening tool for pre-psychotic states [21]. It has been used as a valid indicator of prodromal episodes [22] and successfully detected a disposition to psychosis [23], by now being a standard measure for susceptibility to psychosis [24].

Since there is a recognized need for developing shorter questionnaires with robust psychometric properties [12, 25] the current study uses exploratory factor analyses to examine the internal structure of the BQSPS and compares the results with Nunez et al. [8]. We then test associations between the BQSPS and related criteria to determine their contributions to the explained variance of the BQSPS. In addition, we use receiver operating characteristic (ROC) analysis to investigate the discrimination of the BQSPS in a young adult population and compare the results with Müller et al. [21]. Finally, we also examine the internal constancy and test-retest reliability of the BQSPS and compare them with Demmin et al. [10].

Materials and methods

Procedure

The Institutional Review Board of Yuli Hospital approved this study (Approval number: YLH-IRB-10502). We had obtained permission from the original copyright holder of the BQSPS before this study began. All participants (N = 289) completed an informed consent procedure and the BQSPS. A subsample (N = 219) also completed the Rosenberg Self-Esteem Scale, Eysenck Personality Questionnaire, and the SCL-90-R at this time. A subsample of the first subsample (N = 70) completed the BQSPS again, two weeks after the first administration.

Participants

We recruited the subjects by stratified random sampling. There were 300 undergraduate students from a representative university in eastern Taiwan who participated in this study. The students represented a wide range of faculties, including Chinese, English, Clinical and Counseling Psychology, Chemistry, Life Science, Electrical Engineering, Computer Science & Information Engineering, Business Administration, Finance, Accounting, Tourism Recreation & Leisure Study, Educational Administration and Management, Special Education, Physical Education and Kinesiology, Music, Art & Design. We eliminated 11 responses due to incomplete answers, leaving 289 participants who participated in the following analysis. There were 107 men (37%) and 182 women, whose ages ranged from 19 to 23 years, with a mean of 20.65 (SD = 0.89). Their education ranged from 13 to 17 years, with a mean of 14.65 (SD = 0.89). We used data from the sample to explore the construct validity and internal consistency of the BQSPS. A subsample by simple random sampling (N = 70) was used to assess the test-retest reliability of the BQSPS. The remaining subjects (N = 219) were used to test the criterion-related validity and discriminant validity of the BQSPS.

Measures

Rosenberg Self-Esteem Scale (RSES)

Rosenberg developed the original Rosenberg Self-Esteem Scale (RSES) [26], which contains ten items, each with a 4-point scale from 1, strongly disagree, to 4, strongly agree. A higher score means that the subject has higher self-esteem. The Chinese version of the RESR also showed good Cronbach’s α (= .85) [27] and construct validity [28]. In the current study, we used the Chinese version of the RSES to explore the criterion-related validity of the BQSPS by measuring participants’ self-esteem.

Eysenck Personality Questionnaire (EPQ)

Eysenck and Eysenck developed the original Eysenck Personality Questionnaire (EPQ) [29]. The original EQS contains five subscales, including psychoticism, extraversion, neuroticism, and lying, a total of 90 items. Lu (1995) developed a Chinese short-form version of the EQS [30] with good Cronbach’s α (= .90) and construct validity [31]. It contains 25 items that pertain to either neuroticism or extraversion factors. In this study, the Chinese short-form version of the EQS was used to explore the criterion-related validity of the BQSPS by measuring participants’ neuroticism and extraversion.

The Symptom Checklist-90-R (SCL-90-R)

The English version of the Symptom Checklist-90 (SCL-90) was created by Derogatis [20] and was revised to SCL-90-R [32]. The original SCL-90-R contains 90 items, and each item has a 5-point scale from 0, which means no symptoms, to 4, which means strong symptoms. The original SCL-90-R had good psychometric properties [33, 34]. Yeh’s Chinese version SCL-90-R also had good psychometric characteristics and norms [35]. Cronbach’s α and test-retest reliability of nine symptom dimensions ranged from.77 to.90 and.70 to.93. The original SCL-90-R included nine symptom dimensions: somatization, obsessive-compulsion, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. These symptoms could be calculated into three indexes: the Global Severity Index (GSI), the Positive Symptom Distress Index (PSDI), and the Positive Symptom Total (PST). A T score for the GSI above 63 points, or the T score of any two symptoms dimension above 63 points, generally indicates a significant clinical psychological problem [36]. The current study uses the Chinese version of the SCL-90-R to explore the criterion-related validity of the BQSPS by measuring participants’ symptoms and severity.

The Putative Pre-Psychotic State Scale (BQSPS)

Liu et al. developed the Putative Pre-Psychotic State Scale (BQSPS) [9]. The BQSPS contains 15 items, with each item answered using "yes" or "no" to minimize the response burden. A "yes" indicates an affirmative response to the item of a deviant experience. In this study, we used the BQSPS to analyze its construct validity, criterion-related validity, discriminant validity, internal consistency, and test-retest reliability.

Statistical analyses

Independent sample t-test analysis, exploratory factor analysis, Pearson correlation analysis, stepwise regression analysis, internal consistency analysis, test-retest reliability analysis, and ROC analysis used the Statistical Package for the Social Sciences (SPSS), version 14.

Results

Because the test sample was found to have good sampling properties (KMO = .80; Bartlett’s test, p < .001), we adopted principal component analysis with an oblique factor rotation. The scree plot and eigenvalues from the initial factor extraction indicated that a three-component solution explained 45.51% of the variance. According to Tabachnick & Fidell, using an alpha level of.01 (two-tailed), a rotated factor loading for a sample size of at least 300 would need to be at least.32 to be considered statistically meaningful [37]. In this study, we retain an item base on its factor loading at least.40.

Table 1 shows that item 7 had loading (< .40) on component 1. But item 7 had the highest loading (.40) on component 2. After adding item 7 into component 2, Cronbach’s alpha improved from.59 to.60. Therefore, we attributed item 7 to component 2. Finally, component 1 has six items, component 2 has five items, and component 3 has four items. The first component is composed of six items from the Social Anxiety Scale of Nüñez et al. [8]. The item loadings range from.48 to.72, while the component explains 24.98% of the total variance. The second component is composed of four items from the Positive Symptoms Scale, with one item from the Social Anxiety Scale of Nüñez et al. The range of item loadings is from.40 to.72, and the component explains 10.68% of the total variance. Four items of the Negative Symptoms Scale from Nüñez et al. had high loading on the third component. The item loadings ranged from.62 to.80, with the component explaining 9.85% of the total variance. The components structure and contents of each component are very similar to Nüñez et al.

Table 1

Exploratory factor analysis of the putative pre-psychotic states scale.

		Componenta
Item number and abbreviated wording		1	2	3
12.	I am poor at returning social courtesies and gestures. (SA)	.72b	-.14	-.04
5.	I am mostly quiet when with others. (SA)	.69b	-.03	.01
1.	I cannot deal with the pressures associated with crowds. (SA)	.64b	.12	.00
11.	I do not have an expressive and lively way of speaking. (SA)	.60b	.08	.08
8.	I feel nervous when giving a speech in front of a large group of people. (SA)	.58b	-.02	-.08
2.	I feel I cannot get close to people. (SA)	.48b	.03	-.39
14.	Do you often pick up hidden threats or put downs from the words or actions of others? (PS)	.12	.72b	-.04
6.	I sometimes become concerned about the loyalty and trustworthiness of friends or coworkers. (PS)	.02	.70b	-.06
13.	When you see people talking to each other, do you often wonder if they are talking about you? (PS)	.06	.62b	.01
15.	Do you hear some sounds, voices, or calls of your name when nobody is around you? (PS)	-.21	.55b	-.01
7.	I tend to keep my feelings to myself. (SA)	.33	.40b	.01
4.	I feel mentally insufficient and easily fatigued while thinking or reading. (NS)	-.14	-.00	-.80b
9.	I cannot focus on a task and need to take frequent breaks while working (studying). (NS)	-.05	-.01	-.70b
3.	I feel lethargic whatever I do. (NS)	.05	.06	-.70b
10.	I always mess up whatever I do. (NS)	.18	.03	-.62b
Variance explained (%):		22.98	10.68	9.85

Note. SA = Social Anxiety; PS = Positive Symptoms; NS = Negative Symptoms.

aComponent 1 = Social Anxiety; Component 2 = Positive Symptoms; Component 3 = Negative Symptoms

bItem has a high loading on the corresponding component.

Table 2 shows the Pearson correlations between scores on the three BQSPS components and scores on the two other instruments (the RSES and the EPQ), with the SCL-90-R in the first sample subgroup (N = 219). As predicted, scores on three components and the total scale of the BQSPS were negatively and significantly associated with scores on the RESE and Eysenck-Extraversion Subscale, confirming the criterion-related validity of the BQSPS. Also consistent with predictions, scores on three components and the total scale of the BQSPS were positively and strongly associated with scores on the Eysenck-Neuroticism Subscale. Table 3 shows that scores on three components and the total scale of the BQSPS were positively and significantly associated with scores on all the symptom dimensions of the SCL-90-R.

Table 2

Pearson correlations between scores on the putative pre-psychotic states scale and other psychological measures, and SCL-90-R (subgroup of first sample).

External measures	Social Anxiety	Positive Symptoms	Negative Symptoms	Total
RSES	-.47***	-.39***	-.52***	-.61***
Eysenck-Extraversion Subscale	-.71***	-.17*	-.20**	-.52***
Eysenck-Neuroticism Subscale	.44***	.53***	.55***	.66***
SCL-90-R
Somatization	.23**	.39***	.37***	.43***
Obsessive-Compulsive	.39***	.46***	.63***	.63***
Interpersonal Sensitivity	.44***	.60***	.58***	.70***
Depression	.38***	.51***	.66***	.66***
Anxiety	.35***	.46***	.54***	.58***
Hostility	.21**	.51***	.49***	.51***
Phobic Anxiety	.34***	.46***	.45***	.54***
Paranoid Ideation	.29***	.63***	.43***	.57***
Psychoticism	.32***	.56***	.50***	.59***

*p < .05,

**p < .01,

***p < .001

Table 3

Stepwise regression analysis of components of the putative pre-psychotic states scale.

Predictor	Social Anxiety		Positive Symptoms		Negative Symptoms
	Model 1	Model 2	Model 1	Model 2	Model 1	Model 2
	Beta	Beta	Beta	Beta	Beta	Beta
SCL-90-R
Somatization	-.02	.03	-.03	.01	-.14	-.08
Obsessive-Compulsive	.23*	.08	-.15	-.27**	.30**	.23*
Interpersonal Sensitivity	.46***	.29***	.30***	.32**	.16	.08
Depression	.14	.07	-.11	-.10	.54***	.42***
Anxiety	.05	.04	-.15	-.15	-.06	-.08
Hostility	-.29**	-.15*	.05	.08	-.05	-.04
Phobic Anxiety	.10	.01	-.03	-.08	.01	-.03
Paranoid Ideation	-.07	-.02	.40***	.42***	-.18*	-.18*
Psychoticism	-.06	.04	.003	.07	-.07	-.05
Self-Esteem		-.09		-.07		-.20**
Extraversion		-.60***		-.02		.02
Neuroticism		.21***		.30***		.14*
R2	.23	.61	.43	.49	.46	.52

*p < .05,

**p < .01,

***p < .001

In the current study, we executed a stepwise regression analysis in two stages. First, we predicted three components of the BQSPS by using symptoms dimensions of the SCL-90-R as predictors. We further added self-esteem, extraversion, neuroticism, and predicted three components of the BQSPS again. Table 3 shows that variance was explained for the two models of each component of the BQSPS and their beta values.

First, we use a ROC analysis to test the utility of criterion (a). The results show that the best cut-off score is above 7.5 scores with.87 of the area under the curve (AUC) (95% CI:.80-.93) (p < .001). The sensitivity and specificity are.80 and 81, respectively. We concurrently compared the results of the ROC analysis of SA, PS, NS with BQSPS (see Table 4 and Fig 1). The results support that criterion (a) is appropriate for the sample in the current study. Second, we compare the utility of criterion (a), criterion (b), and the combination of these two by Chi-Square analysis. Table 5 shows that using criterion (a) has the largest value of sensitivity + specificity. It is noteworthy that using the combined criteria could yield the highest sensitivity to the most significant false-positive and the fewest false negatives.

Table 4

Results of ROC analysis of BQSPS and subscales.

Scale	AUC(95% CI)	Best cutoff	Sensitivity	Specificity
SA	.72 (.63, .82)	≥ 3.5	.55	.827
PS	.79 (.71, .87)	≥ 1.5	.825	.659
NS	.82 (.74, .89)	≥ 2.5	.75	.765
BQSPS	.87 (.80, .93)	≥ 7.5	.80	.81

Fig 1

ROC curves of SA, PS, NS, and BQSPS.

Table 5

Chi-square analysis of the putative pre-psychotic states scale and SCL-90-R.

		Criterion (a)		Criterion (b)		Combined (a)&(b)
		No	Yes	No	Yes	No	Yes	Total
SCL-90-R	No	145	34	108	71	104	75	179
	Yes	8	32	5	35	2	38	40
Chi-square		57.79***		29.96***		36.91***
Sensitivity		.80		.875		.95
Specificity		.81		.60		.58
False Negative		.20		.125		.05
False Positive		.19		.40		.42

***p < .001

As shown in Table 6, Cronbach’s alpha for the BQSPS and its components ranged from.60 to.78 for the main sample in the study. This suggests that internal consistency for the BQSPS varies from acceptable to good. Table 6 also shows the 2-week test-retest reliability, whose values ranged from r = .67 to r = .82.

Table 6

Internal consistency and test-retest reliability of the putative pre-psychotic states scale and its components.

	Internal consistency (α)	Test-retest reliability (r)
	1st Sample (n = 279)	Subgroup of 1st Sample (n = 70)
Social Anxiety	.73	.71***
Positive Symptoms	.60	.67***
Negative Symptoms	.69	.76***
Total	.78	.82***

***p < .001

Discussion

In the current study, we found that there were three components in our sample. The contents of each component were almost the same as the previous study [8], except that item 7 was moved from social anxiety to positive symptoms. However, because item 7 had a high positive correlation with PS and paranoid ideation, neuroticism and interpersonal sensitivity had larger impacts on PS. Item 7 might reflect a person’s behavioural results derived from his paranoid ideation, neuroticism, or interpersonal sensitivity. This verifies that a stable three-component structure of the BQSPS exists in a normal young population across language. We followed Cohen’s suggestion to interpret a correlation of .1 as small, .3 as moderate, and.5 as large [38]. As expected, relationships with convergent scales, including neuroticism and all symptoms dimensions, ranged from small to large. Also, relationships with discriminant scales, including self-esteem and extroversion, ranged from small to large. This demonstrates that the BQSPS has good criterion-related validities that fit the theoretical psychological expectations. The results are comparable to a previous study using established psychosis screens and un-associated questionnaires to determine construct validity of the BQSPS [10].

We also found some selected criteria contributing to the explained variance on SA, PS, NS. First, extraversion, interpersonal sensitivity, neuroticism, and hostility had significant impacts on SA. Together, they explain 61% of the variance of SA. We note that personality dimensions had significantly larger impacts than symptom dimensions on SA. We found that extraversion was protective against social anxiety, as in a previous study [19]. Second, paranoid ideation, interpersonal sensitivity, neuroticism, and obsessive-compulsion had significant impacts on PS. Together, they can explain the 49% variance of PS. Previous review studies consistently found that paranoia is associated with more negative conscious self-concepts [39, 40]. Our data also supported that finding (r = -.38, p < .001). Third, depression, obsessive-compulsion, self-esteem, paranoid ideation, and neuroticism had significant impacts on NS. Together, they can explain 52% of the variance of NS. NS addresses four items referring to feelings of tiredness, lethargy and concentration difficulty. These descriptions are similar to some of the symptoms of a depression episode. Our findings showed that there was a strong association between depression and NS. Finally, we found that neuroticism had a pervasive impact on SA, PS, NS. In summary, the BQSPS can reflect some important predictors for psychopathology.

We used the SCL-90-R to differentiate our sample into the two groups of psychological distress and no-distress. ROC analysis showed that SA (.72), PS (.79), NS (.82), and BQSPS (.87) had acceptable to excellent discriminant validities. The BQSPS had the highest AUC by using the cut-off in Liu et al. [9]. We also found that combining two screening criteria in Liu et al.’s study could increase the sensitivity of the BQSPS to.95. We compared the results with Müller et al. [21]. Müller et al. developed a new screening tool (Self-screen-Prodrome, SPro) to differentiate cases and non-cases, using the SCL-90-R-subscales of psychoticism [PSYC] and paranoid ideation [PARA]≥63 as criteria [21]. They found that the SPro subscale for psychotic risk (SPro-Psy-Risk) could identify cases best, with a sensitivity of 74% and a specificity of 61%. Apparently, the sensitivity of the BQSPS is better than SPro-Psy-Risk’s for identifying cases. We thought this is due to criteria to define cases in the current study are more comprehensive than those of Müller et al. These criteria reflected appropriately aims of the BQSPS, which try to detect early and broadly at-risk mental states characterized by subtle symptoms and functional impairments [9]. The BQSPS had acceptable to good internal consistency and good test-retest reliability, similar to findings of Müller et al. [21].

However, the study has two limitations. First, our subjects were undergraduate college students, and whether the findings could be generalized to other young populations remains to be tested. Second, since the BQSPS did not originally focus on increasing the predictive validity of transition from at-high-risk to psychosis, a longitudinal study is needed to confirm the predictive validity of the BQSPS for the onset of psychosis.

Conclusion

To conclude, this study shows that the BQSPS has good psychometric properties for young adults. We also clarify clinical implications for each component of BQSPS and thereby expand its clinical utility in university settings.

Anonymized dataset.

(XLS)

Click here for additional data file.

13 Apr 2021

PONE-D-20-34283

Psychometric properties of the brief self-report questionnaire for screening putative pre-psychotic states and validation of clinical utility in young adult

PLOS ONE

Dear Dr. Chiang,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit; indeed, both the Reviewers and myself have considered your study of interest for its clinical implications. However, it does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 28 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Paola Gremigni, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that Table 1 may include questionnaire items that may have been previously published. The reproduction of previously published work has implications for the copyright that may apply to these publications. We would be grateful if you could clarify whether you have obtained permission from the original copyright holder to republish these items under a CC BY license. If you have not obtained permission to publish these items please remove them from your manuscript. You may wish to replace the text you have removed with relevant question numbers/ brief descriptions of each item; please be sure to include any relevant references and in-text citations.

3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

1) Please, use "construct validity" instead of "constructive validity".

2) Please, use "components" instead of "factors" as you run PCA not EFA.

3) Please, do not say that factor loadings were significant, as you did not report their p values. Usually, we retain an item based on a pre-established cut-point (e.g. >.30 or >. 40) not based on statistical significance.

4) In the Discussion, a mention of your study limitations is missing; therefore, please, add this part.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Identification of psychosis in the early stage is very much fruitful regarding the clinical outcome. So, this study is a good effort to meet the burning issue.

The study was found well organized, used various appropriate statistical analysis tools, and was well written.

I would like to provide some minor suggestions-

1)The full form of ROC should be mentioned at the beginning

2)Psychometric properties of all Chinese versions should be specifically mentioned (Cronbach’s alpha etc.)

3)The statistical part seemed perfect from my point of view, but require further scrutiny by other experts.

Reviewer #2: I understand that this study was performed to validate the psychometric properties of the BQSPS by comparing the results with 2 other previous studies which had the same objective yet a distinctively different population..

The main objective of the study was feasible and interesting yet in the execution I found myself to be lost ..

As the author compared some other studies which used different scales and although the comparisons might be richer this way ..the main objective is lost in the way and the discussion section became overcrowded and confusing.. focusing on the author’s main objective can make the reader more interested in the results.

Table 1 is not well organized, some points are affirmatives others are questions.

Table 2& 3 are better to follow the same items sequence not to have the reader confused .

Finally, the manuscript should be reviewed by a native English speaker for the quality of language.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Panchanan Acharjee

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

28 Apr 2021

To academic editor

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Answer: We had checked and confirmed that our manuscript meets PLOS ONE’s style requirements.

2. We note that Table 1 may include questionnaire items that may have been previously published. The reproduction of previously published work has implications for the copyright that may apply to these publications. We would be grateful if you could clarify whether you have obtained permission from the original copyright holder to republish these items under a CC BY license. If you have not obtained permission to publish these items please remove them from your manuscript. You may wish to replace the text you have removed with relevant question numbers/ brief descriptions of each item; please be sure to include any relevant references and in-text citations.

Answer: The third author of this manuscript is the original copyright holder of the BQSPS. Without any doubt, we got his permission before this study began. We also clarify this in the manuscript.

3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

Answer: We do not change our Data Availability statement. However, we changed our data sharing method from providing a DOI number into providing Supporting information files in this study.

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Answer: We did not cite any articles that have been retracted. For answering the third point of additional editor comments, we add a new citation, “Tabachnick BG, Fidell LS. Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon; 2007”.

Additional Editor Comments:

1) Please, use "construct validity" instead of "constructive validity".

2) Please, use "components" instead of "factors" as you run PCA not EFA.

3) Please, do not say that factor loadings were significant, as you did not report their p values. Usually, we retain an item based on a pre-established cut-point (e.g. >.30 or >. 40) not based on statistical significance.

4) In the Discussion, a mention of your study limitations is missing; therefore, please, add this part.

Answer: We have revised these minor issues in the manuscript accordingly.

To reviewer 1

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1:Yes

Answer: Thank the reviewer's comment.

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1:Yes

Answer: Thank the reviewer's comment.

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1:Yes

Answer: Thank the reviewer's comment.

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1:Yes

Answer: Thank the reviewer's comment.

5. Review Comments to the Author

Reviewer #1: Identification of psychosis in the early stage is very much fruitful regarding the clinical outcome. So, this study is a good effort to meet the burning issue.

The study was found well organized, used various appropriate statistical analysis tools, and was well written.

I would like to provide some minor suggestions-

1)The full form of ROC should be mentioned at the beginning

2)Psychometric properties of all Chinese versions should be specifically mentioned (Cronbach’s alpha etc.)

3)The statistical part seemed perfect from my point of view, but require further scrutiny by other experts.

Answer: Thank the reviewer's comment. For suggestions #1 and #2, we have revised these minor issues in the manuscript accordingly. We also expected to get a critical suggestion about suggestion 3.

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1:Yes: Panchanan Acharjee

Answer: Thank the reviewer's comment.

To reviewer 2

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2:Yes

Answer: Thank the reviewer's comment.

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2:Yes

Answer: Thank the reviewer's comment.

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2:Yes

Answer: Thank the reviewer's comment.

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2:No

Answer: Thank the reviewer's comment. An experienced academic native English speaker edited the paper.

5. Review Comments to the Author

Reviewer #2: I understand that this study was performed to validate the psychometric properties of the BQSPS by comparing the results with 2 other previous studies which had the same objective yet a distinctively different population..

The main objective of the study was feasible and interesting yet in the execution I found myself to be lost ..

As the author compared some other studies which used different scales and although the comparisons might be richer this way ..the main objective is lost in the way and the discussion section became overcrowded and confusing.. focusing on the author’s main objective can make the reader more interested in the results.

Table 1 is not well organized, some points are affirmatives others are questions.

Table 2& 3 are better to follow the same items sequence not to have the reader confused .

Finally, the manuscript should be reviewed by a native English speaker for the quality of language.

Answer: Thank the reviewer's comment. We had added some sentences to make the paragraph more clear for the readers. Besides, an experienced academic native English speaker edited the paper.

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2:No

Answer: Thank the reviewer's comment.

Submitted filename: Response t Reviewers.docx

Click here for additional data file.

6 May 2021

Psychometric properties of the brief self-report questionnaire for screening putative pre-psychotic states and validation of clinical utility in young adult

PONE-D-20-34283R1

Dear Dr. Chiang,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Paola Gremigni, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

The Authors answered adequately the minor concerns expressed by the reviewers and the academic editor. The statistical approach is appropriate.

8 Jun 2021

PONE-D-20-34283R1

Psychometric properties of the brief self-report questionnaire for screening putative pre-psychotic states and validation of clinical utility in young adult

Dear Dr. Chiang:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Prof. Paola Gremigni

Academic Editor

PLOS ONE