Structural validity of a brief scale adapted to measure adolescent spiritual health.

Spiritual health is established as an important protective health asset in child populations. Measurement and assessment of this elusive concept are, however, challenging. Brief and age-appropriate instruments are required for surveys and related population health research. One longstanding model describing child spirituality suggests that scales and measures consider four standard domains describing connections to self, others, nature, and the transcendent. In this validation study, we tested the structural validity and internal consistency of a brief, literacy-level appropriate instrument for adolescents that was based on prior adaptations of this model. The 2018 cross-national study population included 47,180 children aged 11-15 years from 9 countries. Based upon theory, factor pattern matrices, and Scree plots, the exploratory factor analysis best supported the four-factor model, with items organized according to the original four domains. Internal consistency of the items was acceptable (alpha>.7) to good (alpha>.8) within domains, again within each of the 9 countries. The confirmatory factor analysis again supported the four-factor model (by country, SRMR: 0.020 to 0.042; and AGFI and NFI fit: >0.98). Model fit indices for the four-factor model were improved compared with its unidimensional version. Moving forward, our analysis establishes the structural validity and internal consistency of this adapted brief spiritual health instrument to be used in surveys of adolescents.

Introduction

The health of young people has traditionally been modelled in terms of its physical, social and mental domains (WHO, 1948). Inspired by more holistic models of health (P. King & Benson, 2006, pp. 384–398), Indigenous health (M. King, Smith, & Gracey, 2009; Montenegro & Stephens, 2006; Tse, Lloyd, Petchkovsky, & Manaia, 2005) and clinical wisdom (Udermann, 2000), many have proposed “spiritual health” as a fourth domain of health (Chirico, 2016; Vader, 2006). Beyond the peer-reviewed literature, the UN Convention on the Rights of the Child recognizes the importance of an analogous concept (spiritual well-being and development) as an internationally protected right in four of its articles (UN General Assembly, 1989). Irrespective of its label, little is said, however, about what actually constitutes spiritual well-being (or health) and how it is best measured in population health surveys.

Spiritual health is admittedly a challenging concept to measure. Several scales have been developed and tested psychometrically, particularly in children (de Jager Meezenbroek et al., 2012). One assessment framework that was proposed in the 1970's by the National Interfaith Coalition on Aging (1975) has inspired research, both quantitative (Michaelson et al., 2016; Gomez & Fisher, 2003) and qualitative (Hay & Nye, 2006), that measures child spirituality as a multidimensional construct. Existing scales attempt to depict the nature and strength of connections in four domains: to oneself, to others, to nature, and to the transcendent (J. W. Fisher, Francis, & Johnson, 2000; Michaelson et al., 2016). These existing scales are quite lengthy and were typically designed with a literacy level consistent with that of older children and adults (Gomez & Fisher, 2003). Most have been studied within non-general populations of children and been subject to simple psychometric assessments, such as reliability assessments and exploratory factor analyses (de Jager Meezenbroek et al., 2012). Items describing the four domains are typically combined to form an overall scale, assuming unidimensionality of the four domains (Gomez & Fisher, 2003). While appropriate to a degree, such measures are rarely tested via confirmatory factor analyses and validated uniformly across countries and cultures, in order to determine their wide suitability for population health research.

In 2013, our research group introduced a short spiritual health module to a cross-national adolescent health promotion survey. Affiliated with WHO-Europe and now conducted in 50 countries, Health Behaviour in School-aged Children (HBSC) has existed for over 30 years and is one of the most longstanding surveys of its type. For scales to be considered by HBSC, they must be introduced as optional items and tested in a small number of countries. It is suggested that these scales be brief, simple, and widely applicable to children cross-nationally. Rigorous validation is required.

In this brief report, we tested the structural validity and internal consistency of this 10-item spiritual health scale in nine countries. This scale was originally adapted for use in the 2014 HBSC survey from an established instrument (Gomez & Fisher, 2003), and then updated based on peer-review, qualitative review of the items by groups of young people, and initial analyses, both psychometric (Michaelson et al., 2016) and etiological (Brooks, Michaelson, King, Inchley, & Pickett, 2018; Michaelson et al., 2019). Our aim was to evaluate the validity of this scale for cross-national use in HBSC and like population health surveys of children aged as young as 11 years.

Material and methods

Participants

Data (n = 47,180) were obtained from the 2018 cycle of the HBSC study in nine countries: Canada, England, Poland, Scotland, Latvia, Russia, Moldova, Lithuania, and Wales. Following removal of incomplete cases for the spiritual health items of interest, the median sample size was 3516, ranging from 1200 in Russia to 14,396 in Wales. Participants were aged between 11 and 15 years with a roughly equal distribution of boys and girls in all countries. Data collection followed a standardized research protocol which specifies sampling methods and questionnaire design in all participating countries (Inchley et al., 2016). Adolescents completed questionnaires in classroom settings. Participants were recruited via stratified random cluster sampling, with whole school classes as the sampling unit. Questionnaires were translated from English into respective national languages (note: translations of the spiritual health scale and associated items for each of the nine countries are available from the authors, upon request). Appropriate institutional ethical consent was gained in each country, with schools and adolescents giving prior informed consent.

Assessment of spiritual health

The 2018 spiritual health scale involved ten questions covering the four theoretical spiritual health domains (Table 1). Students answered using a 5-point Likert-like set of response items ranging from 1 (“Not at all important”) to 5 (“Very Important”) for each item. Response to these items could potentially be summed into an overall unidimensional score (range 10–50) or domain-specific scores that range from 2 to 10 (2 items) or 3 to 15 (3 items).

Table 1

Items on the 2017/18 HBSC survey comprising the 10-item Spiritual Health Scale.

Theoretical Spiritual Health Domain	HBSC Survey Item“How Important is it for you to … ?”
Others	Be kind to other people
	Be forgiving of others
	Show respect for other people
Self	Feel that your life has meaning or purpose
	Experience joy in life
Nature	Feel connected to nature or wilderness
	Care for the natural world
Transcendent	Meditate or pray
	Feel a connection to a higher power
	Feel a sense of belonging to something greater than yourself

Procedure

Exploratory factor analysis

We first conducted an exploratory factor analysis (EFA) to determine the number of latent factors underlying the 10-item spiritual health scale in the nine countries. Key statistical assumptions for factor analysis were assessed, including scale factorability, sampling adequacy (Kaiser–Meyer–Olkin [KMO]) and multivariate normality of the manifest variables (Field & Miles, 2010; Schreiber, Nora, Stage, Barlow, & King, 2006). Sample size adequacy was also considered. Next, the intercorrelation matrix underwent iterated principal axis factoring (Fabrigar, MacCallum, Wegener, & Strahan, 1999). This extraction method was selected for its increased robustness to normality violations compared to maximum likelihood methods (Fabrigar et al., 1999). Factor retention was determined using multiple criteria, including a visual inspection of scree plots, strength of factor loadings and theoretical knowledge. Following extraction, factors were rotated using a Promax (oblique) rotation to enhance factor interpretability and allow for correlation between extracted factors (Hendrickson & White, 1964). The SAS procedure PROC FACTOR was employed for exploratory factor analysis (SAS Institute, Inc., Cary, NC).

Confirmatory factor analysis

Following EFA, a four-factor latent structure was hypothesized (Fig. 1). A unidimensional model was also examined, as many have posited that spiritual health can be captured as a single latent factor (de Jager Meezenbroek et al., 2012). The fit of these models was assessed in all nine countries via confirmatory factor analysis, using the SAS procedure PROC CALIS (SAS Institute, Inc., Cary, NC). A robust diagonally weighted least squares (DWLS) procedure was selected for parameter estimation. This method uses a weighted least squares estimator with the polychoric correlation matrix, and it is appropriate for ordinal response variables (Li, 2016). Model fit was evaluated through an examination of parameter estimates and model residuals. We also examined several fit indices, including the standardized root mean square residual (SRMR), the adjusted goodness of fit index (AGFI) and Bentler and Bonnet's Normed Fit Index (NFI). There is a lack of knowledge on the statistical qualities of estimates generated using DWLS and the appropriate cut-off values (Xia & Yang, 2019). We therefore used traditional cut-off values in this study (SRMR<0.08, AGFI>.95, NFI>0.9) (Schreiber et al., 2006), but acknowledge that their application to ordered categorical data remains non-established).

Fig. 1

Hypothesized four-factor structure of the 10-item Spiritual Health Scale for confirmatory factor analysis, e = error. Note. One loading for each latent variable was fixed to 1 to provide a reference scale for model identification.

Internal consistency

Cronbach's alpha coefficients were estimated, with values above 0.8 indicating good internal consistency (Nunnally, 1978). Non-standardized alpha coefficients were reported by country for the overall spiritual health scale (if considered as a unidimensional model) as well as for analogous domain-specific scores.

Results

Exploratory factor analysis

Key assumptions were initially assessed. With two exceptions (Latvia, Lithuania) intercorrelations between the ten scale items were at least 0.3 within each country, suggesting reasonable factorability (Field & Miles, 2010). The overall KMO value for each country was high (range 0.82–0.90), and all KMO values for individual items were above acceptable limits (>0.5) (Kaiser & Rice, 1974) suggesting suitable intercorrelation for analysis. Several of the manifest variables had heavily left-skewed distributions (skew > −2), potentially violating assumptions of multivariate normality. The country samples yielded ratios ranging from approximately 120:1 to 1400:1 subjects per manifest variable, well above the minimum ratio (5:1) suggested for a well-powered exploratory factor analysis (Gorsuch, 1983).

Based on theory (NICA, 1975), factor pattern matrices, and scree plots, both two factor and four factor models were considered for retention. While a large proportion of variance (range 82%–90%) was accounted by a two-factor solution in most countries, almost full variance was accounted for in the four-factor solution in each country (Fig. 2). After four factors, subsequent factor solutions had little interpretability as single items were retained. Thus, a four-factor solution was selected, with the four theoretical domain names chosen as factor labels.

Fig. 2

Scree Plots showing eigenvalues from the reduced correlation matrix against the number of common factors for the 10-item Spiritual Health Scale, by country.

Following rotation, questionnaire items loaded as per the latent structure described in Fig. 1. The order of extracted factors varied between countries, but factor loading patterns remained consistent (data not shown). Primary factor loadings for others, self, nature and the transcendent ranged from moderate to high strength (>0.50) across all countries (except for nature in Moldova), and most item cross-loadings on other factors were very low (<0.2). There were slightly higher cross-loadings observed for Moldova and Russia, but these still fell below 0.3. The final total communality estimates were moderately high for each country (range 6.03–7.29), indicating that most items had a fair proportion of variance explained by the extracted factors. Correlations between factors ranged from weak (r = 0.20) to moderate (r = 0.67) in strength.

Confirmatory factor analysis

Selected goodness of fit indices are presented in Table 2. SRMR values ranged from 0.020 (Lithuania) to 0.042 (Wales) suggesting reasonable model specification. Similarly, AGFI and NFI values were each over 0.98 in all countries, strongly supportive of the four-factor model. Standardized model residuals were low and centred around zero, also pointing to good model fit. The standardized parameter estimates were high across all countries, with the majority of values ranging from 0.7 to 0.9. No post-hoc model modifications were made.

Table 2

Goodness of fit indicesa for the four-factor model fit of the 10-item Spiritual Health Scale and Cronbach's coefficient alpha.

	Canada	England	Poland	Scotland	Latvia	Russia	Moldova	Lithuania	Wales
Goodness of Fit Index
Standardized root mean square residual (SRMR)	.034	.038	.025	.038	.031	.040	.027	.020	.042
Adjusted goodness of fit index (AGFI)	.993	.991	.995	.991	.993	.989	.995	.997	.989
Bentler and Bonnet's normed fit index (NFI)	.993	.989	.994	.990	.991	.998	.995	.996	.986
Cronbach's Coefficient Alpha
Four-factor model, by domain
Self Domain	.85	.77	.79	.82	.76	.82	.80	.71	.74
Others Domain	.87	.86	.87	.86	.77	.87	.87	.79	.79
Nature Domain	.83	.84	.80	.86	.75	.81	.65	.79	.79
Transcendent Domain	.87	.86	.83	.87	.81	.85	.81	.81	.81
Unidimensional model (total scale)	.88	.85	.85	.87	.84	.87	.89	.83	.84

Indices are generated from confirmatory factor analysis using a diagonally weighted least squares (DWLS) estimation procedure.

Comparison of findings from the four-factor and unidimensional models too were supportive of the four-factor solution. While the unidimensional factor loadings were smaller and explained less of the variable variance than the four-factor model, loadings remained relatively strong (range 0.42–0.76 for the nine countries). However, the absolute fit indices for the unidimensional model fell outside acceptable ranges in all nine countries, indicating problems with model fit.

Cronbach's alpha coefficients for each of the four domains were estimated by country (Table 2). Domains showed acceptable (alpha>.7) to good (alpha>.8) levels of consistency within each country (Nunnally, 1978). The unidimensional model also had good levels of internal consistency (Table 2).

Note

Supplemental findings (data not shown) are available from the authors, upon request. These include: (1) standardized loadings based on the rotated factor pattern matrix (EFA); (2) standardized and non-standardized coefficients (CFA), and (3) fit indices (CFA) for the unidimensional version of the scale.

Discussion

This cross-national study examined the structural validity and internal consistency of a brief spiritual health scale to be used in adolescent population health surveys. This scale was based conceptually on a longstanding theoretical framework proposed (NICA, 1975), and adapted from an existing factor-analytically derived instrument created for (mainly) older adolescents (Gomez & Fisher, 2003).

Our primary finding was that our adapted scale is best analyzed in four domains characterizing the importance of various connections in the lives of young people. Further, we provide strong psychometric evidence that the scale works in four domains, as supported by exploratory then confirmatory factor analyses, and estimations of internal consistency. Whilst reliability analyses supported the potential for a unidimensional factor solution, model fit indices revealed challenges with unidimensional model fit for the ten-item adapted questionnaire. We concluded that the hypothesized four-factor model consistent with the four theoretical spiritual health domains had the best fit for the observed data.

As the idea that adolescent spiritual health is an important protective health asset gains more acceptance as a valid concept, a practical measurement tool that can be used in adolescent populations across countries and cultures is necessary. To date, most scales describing adolescent spiritual health are quite comprehensive and lengthy and have not been tested psychometrically cross-nationally. Further, most adolescent population health surveys need to assess multiple content areas, and survey length and burden are important issues. For the HBSC study, essential properties of new measures are that they are age-appropriate in terms of literacy, supported by established theory, psychometrically valid, and interpreted consistently across many countries and cultures. This 10-item version of the scale that has been adapted for the cross-national HBSC survey appears to possess such properties.

Limitations of our analysis include the fact that this instrument is brief. This inevitably leads to a loss of some of the depth and richness that would accompany the use of a larger scale. However, the reduction that we have proposed was appropriate for use in large-scale surveys. Second, the abbreviated scale purposefully asks about young people's views of the importance of spiritual health in their lives rather than their own lived experienced, which deviates conceptually from the original intent of the scale (Fisher, 2011). Third, whilst all HBSC country teams are experienced in the translation of questionnaire items from English, there is some potential for interpretive differences and misunderstanding of item wording.

Strengths of this study also warrant comment. While the spiritual health of children is an emergent field of great interest, cross-national studies are rare. Our analysis included a diversity of countries and cultures with various spiritual heritages. Our methods were robust and to a high standard in terms of psychometrics.

Overall, our analysis provides a high level of psychometric evidence in support of the adapted 10-item scale for measuring adolescent spirituality in children aged 11–15 years. The scale appears to perform robustly across countries and cultures, and we would argue that it is most likely suitable for use for children within this age range beyond the nine countries involved in our analysis. The validity of this scale has not been confirmed, however, among children and young people outside of 11–15 years, nor among adult populations. In addition, while there can be overlap between the concepts of spirituality and religiosity in some settings, we do not consider this adapted scale to be measuring connection to formal religion in any sort of comprehensive manner, nor does it provide an in-depth assessment of religious involvement or motivations.

Moving forward, our analysis provides a psychometric foundation for the use of a brief, spiritual health scale to be used in broad, population health surveys of the health of children and adolescents.

Project support/funding

Grant funding for the researchers involved in performing this cross-national analysis was provided by the: 1) Public Health Agency of Canada; 2) (operating grant MOP341188); 3) NIH Health Scotland; 4) , Scotland; 5) the UK Department of and Department of Education. Dr. Inchley is supported by the (MC_UU12017/14) and the (SPHSU14) of Scotland. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. International coordinator of the 2014 HBSC survey was Dr. Jo Inchley, University of St. Andrews, Scotland. The international databank manager was Dr. Oddrun Samdal, University of Bergen, Norway. The surveys included in this study were led by the following Principal Investigators: Canada (William Pickett, Wendy Craig, [the late] John Freeman], Czech Republic (Michal Kalman), England (Fiona Brooks, Ellen Klemera), Israel (Yossi Harel-Fisch), Lithuania (Kastytis Smigelskas), Latvia (Iveta Pudule), Poland (Joanna Mazur and Agnieszka Małkowska-Szkutnik), Republic of Moldova (Galina Lesco), Russian Federation (Anna Matochkina), Scotland (Jo Inchley), Slovakia (Andrea Madarasova Geckova), Wales (Chris Roberts).

Declaration of competing interest

None.