Measurement invariance between online and paper-and-pencil formats of the Launay-Slade Hallucinations scale-extended (LSHS-E) in the Chilean population: Invariance between LSHS-E formats.

Research on the multidimensionality of hallucination-like experiences (HLEs) can contribute to the study of psychotic risk. The Launay-Slade Hallucinations Scale-Extended (LSHS-E) is one of the most widely used tools for research in HLEs, but the correspondence of its paper and online formats has not been established yet. Therefore, we studied the factorial structure and measurement invariance between online and paper-and-pencil versions of LSHS-E in a Chilean population. Two thousand eighty-six completed the online version, and 578 students completed the original paper-and-pencil version. After matching by sex, age, civil status, alcohol and cannabis consumption, and psychiatric treatment received, we selected 543 students from each group. We conducted a confirmatory factor analysis of a four-factor model and a hierarchical model that included a general predisposition to hallucination, explaining the strong relationship between the different types of hallucinations. Both models showed a good fit to the data and were invariant between paper-and-pencil and online versions. Also, the LSHS-E has good reliability in both online and paper-and-pencil formats. This study shows that the online LSHS-E possesses psychometric properties equivalent to the paper-and-pencil version. It should be considered a valuable tool for research of psychosis determinants in the COVID-19 era. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s12144-021-02497-7.

Introduction

Hallucinations are defined as sensory experiences unshared by others without any environmental stimuli (Bell et al., 2010). Although they are frequent in psychotic disorders, they are also present in other medical and neurological disorders (Waters & Fernyhough, 2017). The scientific literature reports that hallucinations are also common in non-clinical populations (Johns et al., 2014) but with different manifestations, i.e., they are less frequent, likely to be brief, and less likely to be associated with distress than those shown in a clinical population (Preti et al., 2014).

Investigating hallucinations in the non-clinical population–so-called hallucination-like experiences (HLEs)–– provides an appropriate methodology to better understand processes and mechanisms underlying clinical symptoms. HLEs have been described as multidimensional phenomena that include different sensory modalities (auditory, visual, olfactory, and tactile) and types (e. g., sensed presence, hypnagogic/hypnopompic hallucinations, enhanced imagination). These dimensions could be differently related to psychotic risk (Siddi et al., 2018) and other disorders (Goghari & Harrow, 2016), hence the importance of research on HLEs from a multidimensional perspective.

The multidimensional framework has fostered the development of several instruments to assess HLEs for clinical and non-clinical populations. One of the most frequently used tools is the Launay-Slade Hallucination Scale (LSHS), developed primarily for the non-clinical population. This scale has been used to describe the prevalence of a wide range of HLEs in the different populations and to define psychopathological risk profiles (e.g. Kråkvik et al., 2015; Levitan et al., 1996; Morrison et al., 2000; Preti et al., 2014; Serper et al., 2005; Siddi et al., 2018, 2019).

The original 12-item version (Launay & Slade, 1981) has been updated to better capture the multidimensional nature of HLEs by changing its true/false response format to a 5-point Likert scale (Bentall & Slade, 1985) and including visual HLEs (Morrison et al., 2000). An extended version of the LSHS (LSHS-E) (LarØi et al., 2004) was later implemented and included olfactory and sleep-related modalities (hypnagogic and hypnopompic). It eliminates items related to hearing the Devil (Larøi et al., 2004) and God’s voice (Larøi & van der Linden, 2005), as these are highly unusual in the non-clinical population. The majority of the factorial studies of LSHS-E in non-clinical populations have identified a four-factor structure (Larøi & van der Linden, 2005; Vellante et al., 2012; Preti et al., 2014). This model is composed of the following factors: intrusive thoughts, vivid daydreams, multisensory HLEs, and auditory-visual HLEs. Studies have reported this four-factor structure in Belgian (Larøi et al., 2004) and Italian samples (Vellante et al., 2012), with subtle variations in the factor-structure among them. Indeed, HLEs are deeply rooted in culture (Larøi et al., 2014); thus the factor structure of a tool aimed at measuring them might vary across countries, and people from different countries might interpret the items differently according to their culture.

The four-factor structure of the LSHS-E was demonstrated to be invariant across countries (Siddi et al., 2019), and between the clinical and non-clinical populations (Siddi et al., 2018), using the online version of LSHS-E. In Siddi et al. (2018) a cut-off score for psychotic risk was also identified for each factor.

In recent years, online studies have shown some advantages over the paper-and-pencil format, such as a quick collection of data, reduced loss of data, increased participant privacy, and decreased effect of social desirability (Brock et al., 2012; Hirai et al., 2018). During the current pandemic, online studies are especially relevant when using best practices of social distancing and quarantine is necessary. However, researchers must prove the perfect correspondence between online and paper-and-pencil formats of a questionnaire each time. Studies warn that a different format can alter psychometric properties and the factor structure of the instruments even if the items are identical (Naus et al., 2009; Vecchione et al., 2012). For example, the heightened perception of anonymity can modify responses, especially if the topics are considered sensitive, such as mental health or drug use (Denniston et al., 2010; Kays et al., 2012). Therefore, on sensitive and culturally driven experiences, such as HLEs, people might interpret evaluation tools differently if they know that their responses will be manually checked by the researcher or clinicians, and might feel freer to report their experiences online than in paper format. A previous study assessing psychotic-like experiences, including hallucinations, found non-invariance between the paper-and-pencil and online version (Vleeschouwer et al., 2014), reinforcing the importance of testing invariance between formats of any scale, especially in sensitive issues.

As far as we know, the psychometric equivalence between the online and paper-and-pencil versions of the LSHS-E has not been tested yet. This instrument can offer a valid and reliable online measure of HLEs and help identify people at risk of psychosis, with the advantages of the online method mentioned above.

In this study, our principal aim was to analyze the equivalence between online and paper-and-pencil formats of the LSHS-E in the Chilean population. First, we examined the factor structure of LSHS-E in the Chilean people in both formats. We expected to confirm the four-factor structure identified in previous studies (Preti et al., 2014; Siddi et al., 2018; Siddi et al., 2019). Second, we tested the measurement invariance between the two versions of LSHS-E.

Materials and Method

This study is part of the ECLECTIC study, which explores the prevalence of HLEs in countries and its correlates from a multidimensional perspective (Siddi et al., 2019).

Participants

The online group was composed of 2086 Chilean college students who filled out an online questionnaire. Six-hundred and thirty students were invited to complete a set of questionnaires in the paper-and-pencil version, 33 of whom refused to participate, and 19 provided incomplete data. Therefore, 578 university students finally completed a set of questionnaires in the paper-and-pencil version. The inclusion criteria mandated that participants be college students and over 18 years old. After matching by sex, age, civil status, alcohol and cannabis consumption, and psychiatric treatment, we included two similar groups of 543 students each. Table 1 summarizes the sociodemographic characteristics of the two groups.

Table 1

Description of the online and paper-and-pencil samples after matching by relevant sociodemographic variables

	Online	Paper-and-pencil
	(n = 543)	(n = 543)
Age, Mean, (SD),	21.07 (2.17)	21.08 (2.15)
Range	18–31	18–30
Gender % (n)
Male	35.72 (194)	35.17 (191)
Female	64.28 (349)	64.83(352)
Marital status
Married/live in couple	1.84 (10)	1.84 (10)
Single	98.16(533)	98.16(533)
In treatment for mental disorder % (n)
No	95.21 (517)	95.21 (517)
Yes	4.79(26)	4.79(26)
Self-reported alcohol consumption% (n)
No	18.05 (98)	18.05 (98)
Yes	81.95 (445)	81.95 (445)
Self-reported cannabis consumption% (n)
No	36.64 (199)	36.36 (192)
Yes	63.36 (344)	64.64 (351)

Procedure

Online data collection is part of the design of the cross-cultural ECLECTIC study that received approval from the ethics committees of all the institutions involved, following the guidelines of the 1995 Helsinki Declaration (as revised in Tokyo in 2004, and further revised in Fortaleza, Brazil in 2013) and respecting legal regulations in each country (Siddi et al., 2019). In the case of Chile, approval was obtained from the ethics committee of Universidad de Concepción and Universidad San Sebastián.

Sociodemographic questionnaire and LSHS-E were adapted to an online format using the Webropol Survey platform, which provided an online survey link. The LSHS-E items of the online survey were identical to the LSHS-E items of the paper-and-pencil version. The online survey was sent to the institutional email addresses of students attending Chilean universities. Email addresses for all students were provided by universities and were not associated with the online survey link, maintaining the anonymity of the participants. Also, an online survey link was available on institutional web pages of universities and social networks. Sociodemographic questions included specific items to identify college students. The paper-and-pencil survey was administered to undergraduate students of different knowledge areas during the first twenty minutes of a regular class at the Universidad San Sebastián. Seniors trained psychology students invited students to participate, answer questions about the study, and explain informed consent during data collection. Responses were anonymous. Informed consent was obtained from both groups (online and paper-and-pencil format). No compensation was offered to the participants.

Measures

Launay-Slade Hallucination Scales-Extended (LSHS-E)

The 16-item LSHS-E (Larøi et al., 2004), as modified by Larøi and van der Linden (2005), was used to explore a wide range of HLEs in the non-clinical population. Factorial analyses (Preti et al., 2014; Siddi et al., 2018, 2019) proposed a four-factor structure for the scale composed of the following factors and items: intrusive thoughts (Items: 1, 2, 3. e.g. Sometimes a passing thought will seem so real that it frightens me); vivid daydreams (Items 5, 6, 7. e.g. The sounds I hear in my daydreams are generally clear and distinct), multisensory HLEs (Items: 11, 12, 13, 14, 15. Including olfactory or tactile modalities, and related to sleep disturbance and sensed presence, e.g., On certain occasions I have felt the presence of someone close who had passed away); auditory-visual HLEs (Items: 4, 8, 9, 10, 16. e.g. I often hear a voice speaking my thoughts aloud).

Respondents were asked to rate each item on a five-point Likert scale, considering the previous five years of their life:

“certainly does not apply to me,”

“possibly does not apply to me,”

“unsure,”

“possibly applies to me,” and

“certainly applies to me.”

Total higher scores correspond to a greater intensity in HLEs. Responders were required to exclude experiences under the influence of drugs or alcohol. Standard translation and back-translation procedures were followed to develop the Spanish version of the LSHS-E (Siddi et al., 2018), which was subsequently adapted to the Chilean context (See S1-Chilean version of LSHS-E as supplementary material).

Statistical Analysis

Participants from the paper-and-pencil group were matched with participants from the online group by gender, age, marital status, alcohol consumption (dichotomized) and marijuana consumption (dichotomized) to reduce biases attributable to unknown third variables. This matching procedure was conducted considering that the HLEs can vary by gender and age, marital status (Kråkvik et al., 2015), and psychiatric treatment (Waters & Fernyhough, 2017). Marijuana use can increase HLEs, and the effect may persist during periods of abstinence (Bechtold et al., 2016). Alcohol, too, could be another confounding factor (Auther et al., 2015).

The goal of matching is to reduce or even eliminate the effect of the matching variables on evaluating the differences between paper and online format, generating groups like that are similar as possible, so it would no longer be needed to consider those variables on subsequent analysis (Ho et al., 2006). If a perfect match was not possible, the closest match was assigned using the propensity score calculated from the logistic regression of group membership on the criterion variables, with the maximum acceptable interval being one standard deviation of the propensities. Using this criterion, all participants in the paper-and-pencil group achieved an online group match. Unequal sample sizes are associated with an artificial reduction in fit index changes and less power to detect non-invariance (Chen, 2007). The formation of groups of equal size allows for a better invariance test, despite the online sample’s reduction. The R package optmatch was used for this procedure (Hansen & Klopfer, 2006).

Descriptive analyses of the data were reported for both the online and the paper-and-pencil formats. Group comparisons were carried out using the Student’s t test with Welch correction. Scale reliability was evaluated for both versions using Cronbach’s alpha and omega coefficient. Confirmatory factor analysis was carried out with the lavaan and psych package running in R, 4th version. We estimated the model using WLSMV, considering the ordinal level of items. Two models were tested: (a) the four-factor model, identified in previous studies as including intrusive thoughts, vivid daydreams, and auditory and visual HLEs (Preti et al., 2014; Siddi et al., 2018, 2019), and (b) a hierarchical model that includes the four factors of the previous model as first-order factors, and a second-order general factor, corresponding to a general proneness to hallucination. This hallucination-proneness would explain the moderate to high correlations among all four-factors. Parameters for fit estimation were the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the root mean square error of approximation (RMSEA), and the Standardized Root Mean Square Residual (SRMR). RMSEA values of 0.06 or lower, SRMR of 0.08 or lower, and CFI and TLI values of 0.95 or higher, are considered acceptable (Hu & Bentler, 1999). Differences between models were evaluated using scaled difference chi-square test statistic (Satorra & Bentler, 2001).

We tested factorial invariance between online and paper-and-pencil versions using Steenkamp and Baumgartner’s guidelines (Steenkamp & Baumgartner, 1998), which require testing the configural, metric, and scalar invariances to allow substantive analysis, such as comparison of factor means, to be correct. Before testing invariance, all models should show a good fit for each group. Configural invariance indicates that the items assigned to factors are the same for both samples, working as baseline models for more stringent models. Metric invariance refers to the equivalence of the factorial loadings across groups. This level of invariance is required to compare regression slopes or change scores on longitudinal studies. To test it, factor loadings are required to be equal between groups. Scalar invariance implies that cross-group differences in the means of the observed items are due to differences in the means of the underlying construct. Scalar invariance for interval and ratio items testing implies constraining the item intercepts so as to be equal between groups. For categorical items, item thresholds should be constrained so as to be equal for both groups.

On hierarchical models, the measurement invariance should be tested for both first-order and second-order factors. The procedure implies testing sequentially: a) configural invariance; b) metric invariance of first-order factors, imposing equality constraints to the factor loadings between first-order factor and items; c) metric invariance of first and second-order factors, imposing equality constraints on all factorial loadings; d) scalar invariance of first-order factors, fixing the thresholds of the items; and e) scalar invariance of first and second-order factors, imposing equality constraints on all first-order factor (Rudnev et al., 2018).

Using the criteria defined by Chen (2007) for adequate sample sizes (N > 300), metric non-invariance should be established for changes in CFI greater than −0.10, along with a change in RMSEA>0.015 or a change in SRMR >0.03. Scalar non-invariance must be confirmed for changes in CFI greater than −0.10, along with a change in RMSEA>0.015 or a change in SRMR >0.01.

Results

Descriptive Analyses

Descriptive analyses of the items on the LSHS-E scale are displayed in Table 2. The online sample reported significantly higher scores compared with the paper-and-pencil sample for the total HLEs (t[1082.5] = 4.19, p < 0.001), and for (a) factor 1, “intrusive thoughts”, (t[1078.7] = 3.58, p < 0.001), (b) factor 2, “vivid daydreams”, (t([1065.5] = 5.63, p < 0.001) and (c) factor 3, “auditory and visual HLEs” (t[1061.4] = 4.01, p < 0.001). No differences were found for factor 4, “multisensory HLEs” (t[1076.9] = 1.29, p = 0.19).

Table 2

Descriptive statistics, reliability and factor loading for LSHS-E items

	Onlinen = 543	Paper-and-penciln = 543
Factor 1 “intrusive thoughts”	M (SD)	M (SD)	t-test	d	λ
1. Sometimes a passing thought will seem so real that it frightens me	1.52 (1.33)	1.27 (1.29)	t(1083.3) = 3.14*	0.19	0.796
2. Sometimes my thoughts seem as real as actual events in my life	1.49 (1.28)	1.40 (1.32)	t(1082.6) = 1.12	0.07	0.713
3. No matter how hard I try to concentrate on my work unrelated thoughts always creep into my mind	2.99 (1.17)	2.69 (1.27)	t(1077.5) = 3.98**	0.24	0.545
Item mean factor 1	2.01 (0.94)	1.79 (1.01)	t(1078.7) = 3.58**	0.22
α / ω factor 1	0.66 / 0.64	0.72 / 0.71
Factor 2 “vivid daydreams”	M (SD)	M (SD)	t-test	d	λ
5. The sounds that I hear in my daydreams are usually clear and distinct	1.21 (1.34)	0.80 (1.06)	t(1030.1) = 5.65**	0.34	0.741
6. The people in my daydreams seem so true to life that sometimes I think that they are	0.67 (1.09)	0.47 (0.89)	t(1042.9) = 3.30*	0.2	0.730
7. In my daydreams I can hear the sound of a tune almost as clearly as if I were actually listening to it	1.16 (1.40)	0.90 (1.28)	t(1075.2) = 3.23*	0.2	0.690
Item mean factor 2	1.01 (0.97)	0.73 (0.86)	t(1069.2) = 5.26**	0.32
α / ω factor 2	0.71 / 0.64	0.80 / 0.72
Factor 3 “auditory and visual HLEs”	M (SD)	M (SD)	t-test	d	λ
4. In the past I have had the experience of hearing a person’s voice and then found that there was no-one there	1.20 (1.44)	0.85 (1.28)	t(1069.0) = 4.2**	0.26	0.705
8. I often hear a voice speaking my thoughts aloud	0.96 (1.37)	0.61 (1.11)	t(1041.4) = 4.58**	0.28	0.628
9. I have been troubled by hearing voices in my head	0.45 (0.99)	0.30 (0.80)	t(1040.5) = 2.81*	0.17	0.704
10. On occasions I have seen a person’s face in front of me when no-one was in fact there	0.79 (1.21)	0.70 (1.11)	t(1075.6) = 1.17	0.07	0.667
16. Sometimes, I have seen objects or animals even though there was nothing there	0.84 (1.24)	0.80 (1.26)	t(1083.7) = 0.51	0.03	0.631
Item mean factor 3	0.85 (0.83)	0.66 (0.73)	t(1069.1) = 4.11**	0.25
α / ω factor 3	0.77/ 0.70	0.78 / 0.71
Factor 4 (“multisensory HLEs” total sample α = 0.80; ω = 0.73)	M (SD	M (SD)	t-test	d	λ
11. Sometimes, immediately prior to falling asleep or upon awakening, I have had the experience of having seen, felt or heard something or someone that wasn’t there, or I had the feeling of being touched even though no one was there	1.70 (1.54)	1.66 (1.50)	t(1083.3) = 0.46	0.03	0.766
12. Sometimes, immediately prior to falling asleep or upon awakening, I have felt that I was floating or falling, or that I was leaving my body temporarily	1.88 (1.54)	1.52 (1.58)	t(1083.2) = 3.74**	0.23	0.506
13. On certain occasions I have felt the presence of someone close who had passed away	0.99 (1.36)	1.12 (1.42)	t(1081.6) = 1.48	0.09	0.552
14. In the past, I have smelt a particular odour even though there was nothing there	1.06 (1.37)	0.96 (1.36)	t(1083.9) = 1.13	0.07	0.623
15. I have had the feeling of touching something or being touched and then found that nothing or no-one was there	1.15 (1.39)	1.11 (1.46)	t(1081.6) = 0.45	0.03	0.809
Item mean factor 4	1.36 (0.96)	1.28 (1.03)	t(1077.7) = 1.34	0.18
α / ω factor 4	0.75 / 0.71	0.81 / 0.76
Total	1.26(0.73)	1.08(0.70)	t(1082.5) = 4.19**	0.25
α / ω total scale	0.89 / 0.86	0.90 / 0.86

d = Cohen’s d; λ = Lambda item loading; α = alpha; ω = omega; * p < 0.01; ** p < 0.001; Welch’s t-test performed

The differences between online and paper-and-pencil correlation matrices are significant using the Steiger test, ×2(120) = 379.25, p < 0.001. Range of correlation for the paper-and-pencil group was between 0.07 and 0.68, and for the online sample was between 0.04 and 0.65. All correlations were significant at 5% level, except for the 13–3 item in the online sample, and 13–5 and 13–7 items in the paper-and-pencil sample. The complete polychoric correlation matrix may be seen in Supplementary data, Table 1, also available at: https://osf.io/4jb6k/

Confirmatory Factor Analyses

Before analyzing factorial invariance, we tested the fit of the four-factor model and the hierarchical model for both the online and paper-and-pencil samples, as shown in Table 3. The Chi-square test showed that no model fits the data as defined, but this is expected with large samples. All models meet the expected values for SRMR on both samples (<0.08). RMSEA in the online sample is acceptable both for the four-factor model, RMSEA = 0.059, and the hierarchical model, RMSEA = 0.062; in the paper-and-pencil sample, RMSEA values are slightly higher than the cutoff, both for the four-factor model, RMSEA = 0.070, and the hierarchical model, RMSEA = 0.071. CFI and TLI values are below the cutoff criteria on all models, with the lower value being TLI = 0.924 for both the four-factor and hierarchical models in the paper-and-pencil sample.

Table 3

Goodness-of-fit indices for the proposed models in the online and paper-and-pencil samples

Online sample (n = 543)
Model	x2	df	p	CFI	TLI	RMSEA[90%CI] p value	SRMR
Four-factor	285.58	98	<0.001	0.948	0.936	0.059 [0.051, 0.068] 0.026	0.058
Hierarchical	307.36	100	<0.001	0.942	0.931	0.062 [0.054, 0.070] 0.007	0.061
Paper-and-pencil sample (n = 543)
Model	x2	df	p	CFI	TLI	RMSEA[90%CI] p value	SRMR
‘Four-factor	360.84	98	<0.001	0.938	0.924	0.070 [0.063, 0.078] <.001	0.070
Hierarchical	369.99	100	<0.001	0.936	0.924	0.071 [0.063, 0.078] <.001	0.073

CFI comparative fix index, TLI Tucker Lewis Index, RMSEA root mean square error of approximation, SRMR Standardized Root-Mean-Square Residual

The scaled chi-square difference test attested better fit of four-factor model both for the online sample, ×2(2) = 22.56, p < 0.001, and the paper-and-pencil sample, ×2(2) = 13.2, p = 0.001. No common set of modification indices could be found across models, so a general misspecification of models could be ruled out. In summary, both models show an adequate fit to data.

In the four-factor model, the factors were all significantly correlated (p < 0.001) in both samples: “intrusive thoughts” was correlated with “vivid daydreams” (paper-and-pencil r = 0.599, online r = 0.733), “auditory and visual HLEs” (paper-and-pencil r = 0.567, online r = 0.688), and “multisensory HLEs” (paper-and-pencil r = 0.467, online r = 0.525). The “vivid daydreams” was correlated with “auditory and visual HLEs” (paper-and-pencil r = 0.861, online r = 0.871) and “multisensory HLEs” (paper-and-pencil r = 0.599, online r = 0.718). Finally, “auditory and visual HLEs” was correlated with “multisensory HLEs” (paper-and-pencil r = 0.834, online r = 0.874).

In the hierarchical model, all factorial loadings of second-order factor on first-order factors were significant (p < 0.001) and strong in both samples: “intrusive thoughts” (paper-and-pencil λ = 0.612, online λ = 0.697), “vivid daydreams” (paper and pencil λ = .839, online λ = 0.898), “multisensory HLEs” (paper and pencil λ = 0.783, online λ = 0.831). The factorial loading of a general factor on “auditory and visual HLEs” was very high, (paper and pencil λ = 0.999, online λ = 0.999,) consistent with high correlations between this factor and the other three factors on the four-factor model.

Table 2 shows the factor loadings of the four-factor model in the total sample. The factorial loads are acceptable, ranging between 0.506 and 0.809.

Scale and Factor Reliability for the Four-Factor Model by Type Version

Reliability was also reported in Table 2. Alphas and omegas were acceptable for the total scale of both versions (online α = 0.89, ω = 0.86; paper-and-pencil α = 0.90, ω = 0.86). Alphas and omegas were acceptable (>0.7) for all scales, except for Factor 1, “intrusive thoughts” in the paper-and-pencil sample (α = 0.66, ω = 0.64).

Measurement Invariance Analysis

The results for measurement invariance analysis are presented in Table 4. For the four-factor model, configural invariance was corroborated; baseline model showed an appropriate fit, with SRMR <0.08. RSMEA = 0.065 and CFI = 0.942, very close to the defined criteria for these indices. Metric invariance was achieved: no difference in fit between configural model and model with fixed factorial loadings between groups was found using scaled difference chi-square test, ×2(12) = 16.77, p = 0.158. Although scalar invariance was rejected in absolute terms using the scaled chi-square test, X2(44) = 120.36, p < 0.001, scalar invariance using relative indices was supported: ΔRMSEA<0.001 and ΔSRMR = 0.001 did not reach the thresholds for rejection of invariance, although ΔCFI = −0.011 was lower than expected, but very close to criteria.

Table 4

Adjustment indices for the invariance analyses by formats for the four-factor model (n = 1086)

Invariance level	X2(ΔX2)	df	p	CFI (ΔCFI)	RMSEA (ΔRMSEA)	SRMR (ΔSRMR)
Four-factor model
Configural invariance	648.34	196	<0.001	0.942	0.065	0.064
Metric invariance	626.46 (16.77)	208	<0.001	0.947 (0.004)	0.061 (0.004)	0.066 (0.002)
Scalar invariance	758.01 (120.36**)	252	<0.001	0.935 (−.011)	0.061 (0.000)	0.065 (0.001)
Hierarchical model
Configural invariance	679.80	200	<0.001	0.939	0.067	0.067
Metric invariance first-order factor	658.72 (17.08)	212	<0.001	0.943 (0.004)	0.062 (−0.005)	0.069 (0.002)
Metric invariance first and second-order factor	630.30 (0.42)	215	<0.001	0.947 (0.004)	0.060 (−0.002)	0.069 (0)
Scalar invariance first-order factor	775.11 (150.15**)	258	<0.001	0.934 (−0.013)	0.061 (0.001)	0.068 (−0.001)
Scalar invariance first and second-order factor	781.32 (8.61)	262	<0.001	0.934 (0)	0.060 (−0.001)	0.068 (0)

CFI comparative fix index, RMSEA root mean square error of approximation. SRMR Standardized Root-Mean-Square Residual. ΔX Scaled difference Chi-Square test statistic (Satorra & Bentler, 2001). ΔCFI Difference in CFI between models. ΔRMSEA Difference in RMSEA between models. ΔSRMR Difference in SRMR between models

The hierarchical model showed a similar invariance pattern to the four-factor model. Baseline model presented an appropriate fit, with SRMR<0.08. RSMEA = 0.067 and CFI = 0.939 showing a negligible difference with respect to the four-factor model. Metric invariance for both first-order factors X2(12) = 17.08, p = 0.147, and second-order factor X2(3) = 0.42, p = 0.936, were achieved using scaled difference chi-square test statistic. Although scalar invariance for first-order factors was not achieved using chi-square test, X2(43) = 150.15, p < 0.001, and ΔCFI = -0.013 is lower than criteria, invariance could not be ruled out, because ΔRMSEA = 0.001 and ΔSRMR = -0.001 did not reach the thresholds for rejection of invariance. Finally, scalar invariance for second-order factor was confirmed with chi-square test, X2(4) = 8.61, p = 0.072.

In summary, measurement invariance was confirmed from the least restrictive model (configural) to the most restrictive model (scalar invariance) for the four-factor and hierarchical models.

Discussion

The present study aimed to analyze measurement invariance between an online assessed LSHS-E self-report scale and the original paper-and-pencil version. Our results indicate that the four-factor structure of the HLEs, assessed with the LSHS-E, is confirmed both for the paper-and-pencil and online groups, using confirmatory factorial analysis. Fit indices were acceptable, and all items had moderate to strong factorial loadings. This internal configuration was invariant between online and paper-and-pencil formats and showed acceptable reliability in both formats. The configurational equivalence demonstrated that the same items are associated with the same constructs in both samples. As for the metric equivalence, the strength of the relationship between each item and its construct or underlying factor was identical in both groups. Furthermore, scalar invariance was corroborated, so differences in means between items reflect the same latent factor differences between the two samples. These results support the idea that the measurement and organization of the construct follow the same structure for both scale formats.

The present study also corroborates the hypothesis of a general factor, proneness to hallucination, that explains the strong correlations between the four factors found in this study and in previous investigation (Preti et al., 2014; Siddi et al., 2019). The hierarchical model shares properties with the four factors model, with only a negligible decrease in fit indices: good fit to data, acceptable reliability, and measurement invariance from configural to scalar invariance.

This hierarchical structure can be interpreted as a general proneness or predisposition to hallucinate that is expressed in different dimensions, all related to each other. Underlying mechanisms could act as a trigger for this predisposition. Cognitive and emotional factors have been associated with hallucination phenomena in clinical and non-clinical populations, as a risk factor of onset, and maintenance of hallucinatory phenomena (Allen et al., 2005; Bristow et al., 2014; Brookwell et al., 2013).

Our study confirms previous findings on the utility of the online LSHS-E as a hallucination risk detection tool (Siddi et al., 2018). The online LSHS-E version is a successful complementary approach to clinical exploration. This instrument offers a valid and reliable online measure of HLEs and helps identify people in need of care. Advantages to the online method include a rapid collection of the data, no human error in transferring the data from paper to computer, and the opportunity to complete the questionnaires outside the laboratory and at any time (Brock et al., 2012).

The intensity of HLEs was greater in the online version than the paper-and-pencil one, except for the Multisensory HLEs factor. These results align with other studies that reported higher scores on online scales than the original paper-and-pencil equivalents in evaluating different symptoms and constructs, including positive psychotic symptoms (Vleeschouwer et al., 2014) and thought problems (Zeiler et al., 2020).

In addition, the prevalence of HLEs may vary according to the method of administration used. For example, they are more frequently reported in a self-administered format than interviews (Linscott & van Os, 2013). Studies suggest that the online version’s greater anonymity may make participants more comfortable when reporting this information compared to the paper format (e.g., Brock et al., 2012).

The higher HLEs scores on the online version, compared to the original paper-and-pencil version, corresponding to three factors of the LSHS-E: (a) intrusive thoughts, (b) vivid daydream experiences, and (c) auditory HLEs. These experiences have been related to hallucination-proneness, and in the case of auditory HLEs, to psychosis risk. For example, subjects with hallucination proneness can ascribe intrusive or uncontrolled thoughts to an external source, a process also associated with auditory hallucination (Morrison & Baker, 2000). Those with auditory-verbal HLEs (voice-hearers) reported more vivid and intrusive thoughts than those that do not have them (Moritz & Larøi, 2008). Auditory-verbal hallucination can increase the risk of conversion to psychosis in people in at-risk mental states (Niles et al., 2019). A study by Siddi et al. (2018) explored discriminant validity and established a cutoff score for the clinical psychotic status of each factor using an online version of the LSHS-E. The authors found that these three factors more successfully differentiated clinical from nonclinical populations, but not the multisensory HLEs factor (Siddi et al., 2018). In our study, the multisensory HLEs factor––which includes sleep-related, tactile, visual, and olfactory HLEs––was the only factor that did not present score differences between formats. This factor includes sleep-related HLEs (hypnagogic or hypnopompic) frequently reported in the nonclinical population (Ohayon, 2000). Moreover, tactile and olfactory HLEs are also described in the non-clinical population (Peters et al., 2016).

Furthermore, visual HLEs are less frequent in psychotic disorders in the absence of auditory HLEs (Ford, 2017). As a result, this factor may have less clinical relevance. Previous studies have shown that university students were more likely to self-disclose a mental disorder in online questionnaires than in paper-and-pencil ones (Kays et al., 2012). Thus, the content of the multisensory HLEs factor can be considered less pathological and distressing than the other three factors and might not affect the willingness of the individual to self-disclose.

Strengths and Limitations

The online LSHS-E can reach a considerable number of people quickly and is, therefore, a valuable tool for measuring dimensions of hallucinations and studying their role in the risk of psychosis. We determined that psychometric indices are similar for the two formats. In the context of lockdown and social isolation, it is especially important to utilize remote evaluation tools that help detect individuals in need of care and deliver life-saving treatment to them. For example, a study that assessed psychotic-like experiences before and after the COVID-19 pandemic lockdown found that depression, loneliness, and perceived stress were related to psychotic-like experiences at follow-up (Hajdúk et al., 2020). The new situation points to the importance of detecting people at risk of psychotic disorder. Likewise, case reports reveal that adults without COVID-19 infection developed a brief psychotic disorder as a reaction to the intense psychosocial stress resulting from the pandemic (Chandra et al., 2020; D’agostino et al., 2021; Valdés-Florido et al., 2020; Zulkifli et al., 2020). It is relevant to incorporate an easy and possibly web-based evaluation of psychotic symptoms when monitoring the population’s mental health and to prevent transition to a severe psychotic disorder.

However, the following limitations must be considered: the inclusion of the university population limits generalization of the findings to other communities, such as those with more limited access to the internet. However, choosing samples of young people at an age associated with a higher risk of psychosis allows us to evaluate the utility of this scale as a psychosis-risk screening tool. Additionally, there is no information on the clinical record of the subjects except for what is self-reported. Furthermore, results are subsumed in a specific Latin-American culture, which may contribute to the prevalence and dimensionality of the HLEs (Larøi et al., 2014). Nevertheless, this study included two large population-based samples allowing formal testing of measurement invariance of the factor structure of the LSHS-E.

Conclusion

This study shows that the online and paper-and-pencil LSHS-E formats are equivalent in their psychometric properties. The internal configuration of four factors in both versions has demonstrated adequate reliability and validity and yielded evidence of the multidimensionality of HLEs. The hierarchical structure of HLEs was also corroborated, so a general proneness to hallucinations could be measured using this scale. Psychometric equivalence between the two versions of the LSHS-E scale validates its use in the online version format, facilitating administration with the same quality and at lower costs. The LSHS-E could become part of online and mobile-based technologies for clinical monitoring and treatment in early psychosis. It has already been demonstrated to be acceptable and feasible (Alvarez-Jimenez et al., 2014; Firth et al., 2016; Kumar et al., 2018).

Item		Respuestas
1	Algunas veces, pensamientos pasajeros me parecen tan reales que me asustan.	0 1 2 3 4
2	Algunas veces mis pensamientos me parecen tan reales como lo que me ocurre verdaderamente en mi vida.	0 1 2 3 4
3	Aunque intente concentrarme en una actividad, me vienen a la cabeza pensamientos no relacionados con lo que estoy haciendo.	0 1 2 3 4
4	En el pasado he oído voces de una persona y después me he dado cuenta de que no había nadie ahí.	0 1 2 3 4
5	Los sonidos que oigo frecuentemente en mis fantasías son claros y nítidos.	0 1 2 3 4
6	Las personas que aparecen en mis fantasías son tan reales que a veces creo que realmente existen.	0 1 2 3 4
7	Cuando sueño despierto puedo oír una melodía tan nítidamente que creo que la estoy escuchando realmente.	0 1 2 3 4
8	Frecuentemente oigo una voz que dice mis pensamientos en voz alta.	0 1 2 3 4
9	Me he sentido molesto por las voces que oigo en mi cabeza.	0 1 2 3 4
10	En alguna ocasión he visto la cara de una persona aunque no había nadie.	0 1 2 3 4
11	Algunas veces, en el momento de quedarme dormido(a) o al despertarme, he tenido la experiencia de ver, oír o percibir algo o a alguien que no estaba allí, o he tenido la impresión de ser tocado(a) por alguien aunque no había nadie allí.	0 1 2 3 4
12	Algunas veces, en el momento de quedarme dormido(a) o al despertarme, he tenido la impresión de flotar en el aire, caer o separarme del cuerpo temporalmente.	0 1 2 3 4
13	Algunas veces he tenido la sensación de la presencia de alguien cercano que ha muerto.	0 1 2 3 4
14	En el pasado he sentido un olor particular aunque no había nada.	0 1 2 3 4
15	He tenido la sensación de tocar algo, o ser tocado por alguien y después descubrir que no había nada.	0 1 2 3 4
16	Algunas veces he visto objetos o animales aunque no había nada.	0 1 2 3 4