Systematic review of psychometric properties and cross-cultural adaptation of the University of California and Los Angeles loneliness scale in adults.

This systematic review assessed the psychometric properties and the cross-cultural adaptation of the University of California and Los Angeles Loneliness scale (UCLA-LS) in adults. A systematic search of four electronic databases (PubMed, EMBASE, Scopus, and PsycINFO) was conducted from inception until March 2021. We followed the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines for data extraction and evidence synthesis. Eighty-one studies assessed the validity and reliability of the UCLA-LS, translated into many languages, and applied across several countries/societies. Three versions of the 20-item and nine short versions of the UCLA-LS with 3 to 20 questions were identified. High-quality evidence supported the internal structure of the UCLAs: 4, 6, 7 and 10, while low-to moderate-quality evidence supported the construct validity of the UCLAs: 3, 4, 6, 8, 16 and 20. Moderate-quality evidence supported the test-retest reliability of version 3 UCLA-20 with excellent interclass coefficients values of 0.76-0.93. The UCLAs: 4, 6, 7 and 10 had the most robust internal structure and may therefore be the most useful for informing clinicians and social psychologists engaged in assisting those with loneliness. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s12144-021-02494-w.

Introduction

Loneliness is defined as an unpleasant feeling arising from a discrepancy between desired and perceived levels of social connectedness (Perlman & Peplau, 1981). Loneliness can be classified into social loneliness, characterized by the lack of social networks or sense of community, and emotional loneliness, characterized by the absence of a personal, intimate relationship (Perlman & Peplau, 1981). Although inconclusive, loneliness tends to be more common in older adults, patients with chronic diseases, with rates ranging from 21 to 34%, men than women, and vary across different cultures (Lasgaard et al., 2016; Rokach, 2018; Yang & Victor, 2011). However, more conclusive research has shown significant differences between married or having a partner than unmarried and singles (Park et al., 2019; Rokach, 2018; D Russell et al., 1980; Stephan et al., 1988).

In the general population loneliness is a major public health issue associated with poor mental and physical health outcomes (Gerst-Emerson & Jayawardhana, 2015). It has also been linked to poor structural social relationships, including being single and living alone as well as social problems such as alcoholism and suicide, which increase mortality risk (Rico-Uribe et al., 2018; Rokach, 2019; D Russell et al., 1980).

Given the above negative impact of loneliness on mental, physical, and social health, it is important that it be assessed using valid and reliable instruments. The University of California and Los Angeles Loneliness scale (UCLA-LS) is the most used scale for assessing loneliness in the general population. It was developed by Russell and colleagues in 1978 to assess social and emotional dimensions of loneliness (Dan Russell et al., 1978). It has become a key clinical and research questionnaire, and reports of its application have come from several countries/societies. The original scale consisted of 20 negatively worded self-reported questions designed to identify feelings of loneliness in college students (Dan Russell et al., 1978). To reduce possible systematic response bias, it was revised in 1980 to include a mix of ten negatively and ten positively worded items (D Russell et al., 1980). It was revised again (version 3) in 1996 (Russell, 1996) to create a simplified response format for easier use in broader groups of respondents, including older adults, and to be administered by mail, surveys and personal interviews. Participants respond to each item on a 1–4 Likert scale, from “never” to “always” The scale has a possible total score of 20 to 80 points, with a higher score indicating greater loneliness (Russell, 1996).

Several short versions of the UCLA-LS, named by the number of the included questions: UCLA−16 (Faustino et al., 2019), UCLA-11 (Lee & Cagle, 2017), UCLA-10 (Knight et al., 1988; Russell, 1996), UCLA−8 (Hays & DiMatteo, 1987), UCLA-7 (Oshagan & Allen, 1992), UCLA-6 (Neto, 2014; Niu et al., 2018), UCAL-5 (Lamm & Stephan, 1987), UCLA-4 (D Russell et al., 1980), and UCLA- 3 (Hughes et al., 2004) were developed to reduce questionnaire length and increase the response rate in clinical practice and longitudinal studies. The psychometric properties of the 20-item and all short forms of the UCLA-LS have been reported by several studies, and one systematic review (Vassar & Crosby, 2008) summarized the internal consistency reliability coefficients (alpha range, 0.71–0.95) of the UCLA-20 and concluded that the UCLA-20 was reliable for use in the general population (Vassar & Crosby, 2008). However, Vassar & Crosby, 2008 did not assess or report validity results and test-retest reliability of the UCLA-20 and the short versions of the UCLA-LS.

In this systematic review, we aimed to assess the validity (structural, construct, criterion, cross-cultural), reliability (internal consistency, test-retest), and the cross-cultural adaption of the 20-item and the short versions of the UCLA-LS. Such information will assist clinicians, especially those involved in social psychology, to select the most appropriate version of the UCLA-LS to be used in their population.

Methods

The systematic review was conducted according to the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines for systematic reviews (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). The cross-cultural adaptation was assessed using the Guidelines for the Process of Cross-Cultural Adaptation of Self-report Measures (Beaton et al., 2000). The manuscript was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Page et al., 2021) The protocol registration number on PROSPERO is CRD42021241610.

Inclusion and Exclusion Criteria

Inclusion Criteria

Studies describing the development and assessing the psychometric properties (reliability, validity, responsiveness) of the UCLA-LS in adults.

Studies published in a peer-reviewed journal from inception until March 2021,

Studies that report the process of culturally adapting or translating the UCLA-LS into another language and evaluating its psychometric properties.

Exclusion Criteria

Studies that validated the UCLA-LS in children and adolescents,

Interventional studies that used UCLA-LS as an outcome measure,

Non-psychometric studies of the UCLA-LS such as qualitative studies,

Conference publications, abstracts, posters, editorials, and commentaries of the UCLA-LS.

Search Methods for Identification of Studies

We conducted electronic searches in PubMed, EMBASE, Scopus, and PsycINFO databases from inception until March 2021, using the following keywords: Reliability OR validity OR responsiveness OR psychometric properties OR measurement properties OR translation OR cross-cultural adaptation AND UCLA loneliness scale OR UCLA-LS OR University of California, Los Angeles Loneliness Scale. In addition, each of the selected studies’ reference lists was manually searched to identify further studies. An example of the search strategy conducted on PubMed is reported in Online Resource 1.

Study Selection

Two authors (SA and RH) independently carried out systematic electronic searches in each of the four databases. The same authors independently performed the screening of the titles and abstracts of the identified articles. Studies deemed to be relevant to the review were retrieved and assessed for eligibility. Disagreements between the two authors were resolved by discussion until consensus was reached.

Data Extraction

Two authors (SA and AO) performed the data extraction. The extracted data from individual studies included the author, year, country or society, UCLA-LS characteristics, sample size, age, sex/gender, recall period, original language and the available translations, and the psychometric properties of the UCLA-LS (validity, reliability, responsiveness). Furthermore, data on the translation and cross-cultural adaptation process were also extracted to examine each design.

Assessing Methodological Quality

The methodological quality of each selected study was independently evaluated by two authors (SA and AO) using the COSMIN Risk of Bias (RoB) checklist (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). This checklist distinguishes three main domains: reliability, validity, and responsiveness and seven sub-domains: content validity, structural validity, hypothesis testing for construct validity, cross-cultural validity, criterion validity, internal consistency, reliability, measurement error, and responsiveness.

The quality of the individual studies was rated on a 4-point scale as very good, adequate, doubtful, or inadequate, and the overall rating of the quality of each study was based on the lowest rating of any standard (the worst score count principle) (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). The result of each psychometric property was assessed against the updated criteria for good psychometric properties and was rated as sufficient (+), insufficient (−), or indeterminate (?). A detailed description of the quality criteria and the rating system is published (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

The results of the psychometric properties were qualitatively summarized and compared against the criteria for good psychometric properties to rate the overall summary result of the scale as sufficient (+), insufficient (−), indeterminate (?), or inconsistent (±). In case of inconsistent results, the overall rating was based on only high-quality studies and the quality of evidence was downgraded for inconsistency (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Assessing Cross-Cultural Adaptation

The translation methodology of the included studies was categorized according to the Guidelines for the Process of Cross-Cultural Adaptation of Self-report Measures (Beaton et al., 2000). These guidelines indicate that an optimal (accurate) adaptation must include five processes: an initial translation, synthesis of translations, back translations, reviews by the expert committee and finally, the pre-test of the instrument. A detailed description of the cross-cultural adaptation process is published (Beaton et al., 2000).

Hypothesis Testing for Construct Validity

To interpreting the results of hypothesis testing and assessing the quality of evidence for construct validity, the review team formulated a set of hypotheses about the expected relationships (direction and magnitude) of the UCLA-LS and other instruments and variables. These hypotheses were based on the literature and the clinical experiences of the review team. For the convergent and divergent validity, the review team expected strong correlations (r ≥ 0.50) between the UCLA-LS and other loneliness scales (similar construct), moderate correlations (0.30 ≤ r < 0.50) between the UCLA-LS and related constructs such as positive and negative affect, weak or no correlations (< 0.30) for unrelated constructs such as demographics. For discriminative validity, the review team expected non-significant differences between groups except for marital status. These hypotheses are summarized in Online Resource 2. Lastly, the term criterion validity was used only when a short version of the UCLA-LS was compared to the 20-item version, as recommended by COSMIN (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Assessing Overall Quality of Evidence

The overall quality of the evidence was graded as high, moderate, low, or very low, using the modified Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach, which considers methodological quality of studies, inconsistency, imprecision, and indirectness to provide a rating of evidence quality. A detailed description of the evidence quality rating is published (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

The quality of evidence was downgraded by one, two or three levels based on the risk of bias, imprecision, and results inconsistency as shown in Online Resource 3 (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Results

Study Selection

The database and reference list search yielded 575. After removing duplicates, 464 citations were screened, leaving 256 studies for full-text screen for eligibility. Of these, 81 studies were considered eligible (Adams et al., 2004; Allen & Oshagan, 1995; Anderson & Malikiosi-Loizos, 1992; Anderssen et al., 2020; Arimoto & Tadaka, 2019; Auné et al., 2019; Austin, 1983; Balandin et al., 2006; Boffo et al., 2012; Britton & Conner, 2007; Caballer et al., 2020; Chalise et al., 2007, 2010; Chang et al., 2017; Chou, 2005; Çinar & Toker, 2019; Cole et al., 2015; Doğan et al., 2011; Durak & Senol-Durak, 2010; Dussault et al., 2009; Elemo et al., 2020; Elphinstone, 2018; Ergün et al., 2020; Faustino et al., 2019; Gan et al., 2015; Gillespie et al., 2021; Grey et al., 2020; Hackett et al., 2012, 2019; Hagger & Riley, 2019; Hansen et al., 2012; Hartshorne, 1993; Hawkley et al., 2020; Hays & DiMatteo, 1987; Heisel et al., 2016; Hojat, 1982; Hughes et al., 2004; Igarashi, 2019; Igbokwe et al., 2020; Jakobsen et al., 2020; O. Kim, 1999a, 1999b; Y. Kim et al., 2020; Knight et al., 1988; Kong et al., 2015; Kurina et al., 2011; Lamm & Stephan, 1987; Lee & Cagle, 2017; Lukács et al., 2019; Matthews-Ewald & Zullig, 2013; McWhirter, 1990; Nazzal et al., 2018; Neto, 2014; Niu et al., 2018; Oshagan & Allen, 1992; Park et al., 2019; Penning et al., 2014; Pretorius, 1993; Robinson-Whelen et al., 2016; Russell, 1996; Russell et al., 1980; Dan Russell et al., 1978; Saito et al., 2019; Sancho et al., 2020; Shi et al., 2016; Shorey et al., 2019; Solano, 1980; Stancliffe et al., 2014; Stephan et al., 1988; Steptoe et al., 2004; Sun & Chang, 2021; Velotti et al., 2021; Wang et al., 2018; Wang et al., 2001; Wilson et al., 1992; Wittenborn et al., 2020; Wongpakaran et al., 2020; Wu & Yao, 2008; Yu et al., 2020; Zakahi & Duran, 1982; Zarei et al., 2016). The PRISMA flow diagram of the study selection process with reasons for excluding studies is shown in Fig. 1.

Fig. 1

PRISMA flow diagram showing study selection process

Characteristics of Included Studies

The 81 eligible studies were conducted between 1978 and 2021 and included 104,387 participants aged 18–97 years. The study sample size ranged from 20 to 50,054 participants. A summary description of the included studies is presented in Table 1 per version of the UCLA-LS, and Online Resource 4 per study.

Table 1

Characteristics of UCLA-LS per Version

Scale version(References)	Countries / Societies	Population: N, M:F, Age	Scale characteristics: Number of items, Response options, Range of scoring
Original UCLA-20 (Ergün et al., 2020; Gillespie et al., 2021; Dan Russell et al., 1978; Solano, 1980)	USA, UK, Turkey	N = 2372 M:F = 1008:1342 Age, range = 18–70 years	- 20 items, all negatively worded - 4-point scale (1: I never feel this way; 2: I rarely feel this way; 3: I sometimes feel this way; 4: I often feel this way) - 20 – 80 points, high scores indicate greater loneliness
Revised UCLA-20 (Anderson & Malikiosi-Loizos, 1992; Auné et al., 2019; Austin, 1983; Chang et al., 2017; Chou, 2005; Çinar & Toker, 2019; Dussault et al., 2009; Hackett et al., 2012, 2019; Hartshorne, 1993; Hays & DiMatteo, 1987; Hojat, 1982; Hughes et al., 2004; O. Kim, 1999b, 1999a; Y. Kim et al., 2020; Knight et al., 1988; Lamm & Stephan, 1987; McWhirter, 1990; Nazzal et al., 2018; Neto, 2014; Park et al., 2019; Penning et al., 2014; Pretorius, 1993; D Russell et al., 1980; Stephan et al., 1988; Steptoe et al., 2004; Wilson et al., 1992; Zakahi & Duran, 1982)	USA, Iran, Germany, New Zealand, Greece, Zimbabwe, South Africa, Korea, UK, China, Canada, Portugal, Hungary, Palestine, Argentina, Turkey	N = 12,601 M:F = 5680:6403 Age, range = 16–90 years	- Revised version of the original UCLA-LS Russell et al., 1978 - Ten negatively & ten positively worded items - Positively worded items to be reversed before scoring
Version 3 UCLA-20 (Adams et al., 2004; Arimoto & Tadaka, 2019; Balandin et al., 2006; Boffo et al., 2012; Britton & Conner, 2007; Durak & Senol-Durak, 2010; Elphinstone, 2018; Hagger & Riley, 2019; Kong et al., 2015; D. W. Russell, 1996; Saito et al., 2019; Sancho et al., 2020; Shi et al., 2016; Stancliffe et al., 2014; Sun & Chang, 2021; Velotti et al., 2021; Wang et al., 2001; Zarei et al., 2016)	USA, Taiwan, Australia, Turkey Italy, China, Iran, Japan, UK Spain	N = 8355 M:F = 2789:5141 Age, range = 18–98 years	- 20-item, 4-point scale from never (1) to always (4) - 20-80 points, high scores indicate greater loneliness - Consists of 11 negatively worded items and nine positively worded items with reverse scoring - Replaced difficult words with simple ones, added the statement “How often do you feel” at the beginning of each question
UCLA-16 (Faustino et al., 2019)	Portugal	N = 154 M: F = 63:91 Age, range = 61–98 years	-16-item, derived from the Version −3 UCLA-20 - Divided into two factors: social isolation & affinity - 4-point scale from never (1) to often (4)
UCLA-11 (Lee & Cagle, 2017)	USA	N = 3806 M: F = 1598: 2208 Age, mean (SD) = 75 (7) years	- 11-item derived from the Revised UCLA-20 - Seven positively and four negatively worded items - 4-point scale from never (1) to often (4) - 11–44, high scores indicate greater loneliness
UCLA-10 From A (Elphinstone, 2018; Knight et al., 1988)	New Zealand Australia	N = 1736 M: F = 795:941 Age, range = 16–89 years	- 10-item, derived from the Revised UCLA-20 - Each form comprised of five positively and five negatively worded items - Items number 1, 2, 4, 6, 7, 8, 11, 16, 18, & 19
UCLA-10 From B (Elphinstone, 2018; Knight et al., 1988)	New Zealand Australia	N = 1736 M: F = 795:941 Age, range = 16–89 years	- 10-item, derived from the Revised UCLA-20 - Each form comprised of five positively and five negatively worded items - Items number 3, 5, 9, 10, 12, 13, 14, 15, 17, & 20
UCLA-10 (Arimoto & Tadaka, 2019; Elphinstone, 2018; Hansen et al., 2012; Heisel et al., 2016; Russell, 1996; Shorey et al., 2019; Wittenborn et al., 2020)	USA, Canada, Australia, Japan Singapore	N = 2060 M: F = 747:1313 Age, range = 18–95 years	- 10-item, derived from the Version-3 UCLA-20 - Items #: 2, 6, 10, 11, 13, 14, 16, 18, 19, & 20 - 4-point scale from never (1) to always (4) - 10-40 scores, high scores indicate greater loneliness
UCLA-8 (Doğan et al., 2011; Elphinstone, 2018; Gan et al., 2015; Hays & DiMatteo, 1987; Igbokwe et al., 2020; Wang et al., 2018; Wilson et al., 1992; Wu & Yao, 2008)	USA, Zimbabwe, Taiwan, Turkey, China, Australia, Nigeria	N = 4027 M:F = 2194:1831 Age, range = 17–94 years	-8-item, derived from the Revised UCLA-20 - Six negatively and two positively worded items - 4-point scale (1–4) - 8 – 32, high scores indicate greater loneliness
UCLA-7 (Allen & Oshagan, 1995; Elphinstone, 2018; Oshagan & Allen, 1992)	USA Australia	N = 1691 M:F = 659:1032 Age, range = 18–87 years	- 7-item, derived from the Revised UCLA-20 - All negatively worded items (2, 3, 11, 12, 13, 14, & 18)
UCLA-6 (Wu & Yao, 2008)	Taiwan	N = 130 M: F = 58:72 Age, mean (SD) = 20 (2) years	- 6-item derived from the 8-item scale - Items #: 2, 3, 11, 24, 15, & 17
UCLA-6 (Elemo et al., 2020; Nazzal et al., 2018; Neto, 2014)	Portugal Palestine Ethiopia	N = 1749 M: F = 832:917 Age, range = 18–90 years	- Six items derived from the Revised UCLA-20 - Items #: 2, 5, 11, 14, 17, & 18 - Item two should be reversed before scoring - 4-point scale from never (1) to often (4) - 6-24 scores, high scores indicate greater loneliness
UCLA-6 (Niu et al., 2018)	China	N = 484, M: F = 272:212 Age, mean (SD) = 74 (8) years	- 6-item, derived from the 8-item scale (Hays & DiMatteo, 1987) - Items #: 2, 3, 11, 14, 17, & 18
UCLA-6 (Wongpakaran et al., 2020)	Thailand	N = 719 M: F = 280:439 Age, mean (SD) = 28 (8) years	- 6-item, derived from version-3 UCLA-20 - Includes items number 2, 4, 7, 11, 13 & 18
UCLA-5 (Lamm & Stephan, 1987)	Germany	N = 154, M: F = 56:98 Age, mean = 23	- 5-item, derived from the Revised UCLA-20 - Items number 2, 13, 14, 17, & 18
UCLA-4 (Elphinstone, 2018; Hays & DiMatteo, 1987; Russell et al., 1980; Wilson et al., 1992)	USA, Zimbabwe Australia	N = 1873 M: F = 1003:870 Age, range = 17–72 years	- 4 items, derived from the revised UCLA-20 - Items #: 1, 13, 15, & 18 - 4-point scale (from 1 to 4) - 4-16 points, high scores indicate greater loneliness
UCLA-3 (Anderssen et al., 2020; Arimoto & Tadaka, 2019; Caballer et al., 2020; Chalise et al., 2007, 2010; Cole et al., 2015; Elphinstone, 2018; Grey et al., 2020; Hawkley et al., 2020; Hughes et al., 2004; Igarashi, 2019; Jakobsen et al., 2020; Kurina et al., 2011; Lukács et al., 2019; Matthews-Ewald & Zullig, 2013; Robinson-Whelen et al., 2016; Saito et al., 2019; Yu et al., 2020)	USA, Nepal, Australia, Japan, Hungary, Norway Spain, Lebanon UAE, UK, Denmark	N = 79,261 M: F = 23,071:44627 Age, range = 18–97 years	- 3-item, derived from the revised UCLA-LS (How often do you feel lack of companionship?/ How often do you feel left out? & How often do you feel isolated from others?) - 3-point scale: hardly ever (1), some of the time (2), & often (3); 3–9, high scores indicate greater loneliness

UCLA-LS: University of California, Los Angeles- loneliness scale, N: sample size, M: male, F: female, SD: standard deviation, USA: United States, UK: United Kingdom, UAE: United Arab Emirates

Description of UCLA Loneliness Scale

Three versions of the 20-item (original, revised, and version 3) and nine short versions, UCLA-16, UCLA-11, UCLA-10, UCLA-8, UCLA-7, UCLA-6, UCLA-5, UCLA-4, and UCLA-3 of the UCLA-LS were identified in the 81 included studies. The characteristics and assessment of the UCLA-10 and UCLA-6 were divided into sub-versions because each study included a different set of items. A detailed description of each scale is presented in Table 1 and Online Resource 4.

Methodological Quality and Psychometric Properties of the Studies

A qualitative synthesis was used to report the results on the psychometric properties of the included studies. Results of the included studies were summarized based on the assessed psychometric measures (validity and reliability) and the study quality ratings. The UCLA-LS development process was rated as inadequate due to the lack of information on the comprehensiveness and comprehensibility of the instrument, as well as the lack of content validity studies.

Validity of Long and Short Versions of UCLA-LS

Results on validity assessment are presented in Online Resource 5, and the overall quality assessment is summarized in Table 2. The structural validity was assessed for all UCLA-LS versions except the original UCLA-20 (Dan Russell et al., 1978; Solano, 1980) and UCLA-5 (Lamm & Stephan, 1987). Based on the confirmatory factor (CFA) and Rasch analyses, high-quality evidence of sufficient structural validity was available for the UCLA-11(Lee & Cagle, 2017) and UCLA-10 (all sub-versions) (Elphinstone, 2018; Knight et al., 1988; Russell, 1996) on a multidimensional scale (two-factor model) and the UCLA-7 (Allen & Oshagan, 1995; Elphinstone, 2018; Oshagan & Allen, 1992), UCLA-6 (Nazzal et al., 2018; Neto, 2014; Wongpakaran et al., 2020; Wu & Yao, 2008), and UCLA-4 (Elphinstone, 2018) on a unidimensional scale (one-factor model). Low to moderate-quality evidence of sufficient structural validity was available for the Revised UCLA-20 (Çinar & Toker, 2019; Dussault et al., 2009; Hartshorne, 1993; Park et al., 2019; Penning et al., 2014) and version 3 UCLA-20 (Boffo et al., 2012; Durak & Senol-Durak, 2010; Elphinstone, 2018; Russell, 1996; Sancho et al., 2020; Zarei et al., 2016) on a multidimensional scale (three-factor model) and for the UCLA-8 (Doğan et al., 2011; Elphinstone, 2018; Wu & Yao, 2008) on a unidimensional scale (one-factor model). Low to moderate-quality evidence of insufficient structural validity was available for the UCLA-6 (Niu et al., 2018), UCLA-3 (Chalise et al., 2007, 2010; Jakobsen et al., 2020; Matthews-Ewald & Zullig, 2013), and UCLA-16 (Faustino et al., 2019) as indicated by the Rasch and exploratory factor analyses (EFA) (Online Resource 5).

Table 2

Validity results summary and evidence quality of UCLA-LS versions

Scale version	Structural			Convergent & divergent				Discriminative			Cross-cultural			Criterion
	N(n)	RoBRR	GRADE	N(n)	r,conv	RoBRR	GRADE	N(n)	RoBRR	GRADE	N(n)	RoBRR	GRADE	N(n)	RoBRR	GRADE
Original UCLA-20				497 (2)	0.30–0.79	A +	M4	497 (2)	D +	M1
Revised UCLA-20	2525 (5)	Vg +	L5	2837 (11)	0.30–0.71	A +	L5	2275 (6)	A +	L5	1648 (3)	D +	VL1,5	127 (1)	D +	L2
Version 3 UCLA-20 One-dimension Multi-dimensions	248 (1) 3487 (6)	Vg - +	H L5	4445 (4)	0.30–0.71	Vg +	L5	1754 (5)	D +	L1,4
UCLA-16	154 (1)	A -	M1	154 (1)	0.31–0.48	A +	M1	154 (1)	A +	M1
UCLA-11	3706 (1)	Vg +	H
UCLA-10, A (Knight et al., 1988)	758 (1)	Vg +	H	758 (1)	0.34–0.71	A -	L1,4	978 (1)	A +	M1				978 (1)	I +	VL3
UCLA-10, B (Knight et al., 1988)	758 (1)	Vg +	H	758 (1)	0.36–0.74	A -	L1,4	978 (1)	A +	M1				978 (1)	I +	VL3
UCLA-10 (Russell et al., 1996)	1074 (2)	Vg +	H	1322 (3)	0.31–0.70	A -	L5	316 (1)	A +	M1				248 (1)	I +	VL3
UCLA-8	1441 (3)	Vg +	M4	1640 (4)	0.31–0.64	A +	L5	953 (1)	A +	M1				1506 (3)	I +	L2
UCLA-7	1691 (3)	Vg +	H	758 (1)	0.30–0.67	A -	L1,4	619 (1)	D +	VL2,5	933 (2)	D +	M1
UCLA-6 (Wu & Yao, 2008)	130 (1)	Vg +	H
UCLA-6 (Neto, 2014)	1442 (2)	Vg +	H	1442 (2)	0.40–0.66	A +	M4	1442 (1)	A +	L1,4				1442 (2)	I +	L2
UCLA-6 (Niu et al., 2018)	484 (1)	D -	L2	484 (1)	0.34–0.52	A +	VL1,5	484 (1)	A +	L1,4
UCLA-6 (Wongpakaran et al., 2020)	719 (1)	Vg +	H	719 (1)	0.30–0.77	A -	VL1,5	719 (1)	Vg -	H	719 (1)	Vg +	H	719 (1)	I +	VL3
UCLA-5				154 (1)	0.36–0.71	A +	L1,4
UCLA-4	758 (1)	Vg +	H	957 (2)	0.30–0.67	A +	L5	953 (1)	A +	M1				953 (2)	I +	VL2,4
UCLA-3	1672 (4)	D -	M1	10,020 (9)	0.32–0.65	A +	L5	175 (1)	Vg +	M4	9263 (1)	Vg -	H	977 (3)	I +	L2

RR: Results rating: (+) sufficient, (−) insufficient, (?) indeterminate. The quality of evidence was downgraded for: 1. Serious risk of bias, 2. Very serious risk of bias, 3. Extremely serious risk of bias, 4. Serious inconsistency, 5. Very serious inconsistency, 6. Based on structural validity. UCLA-LS: University of California, Los Angeles Loneliness Scale, N: sample size, (n): number of studies included in the decision process, RoB: risk of bias (Vg: very good, A, adequate, D: doubtful, I: inadequate), GRADE: Grading of Recommendations Assessment, Development, and Evaluation (H: high, M: moderate, L: low, VL: very low), r: Pearson’s correlation for convergent validity

+

+

+

+

+

+

+

-

+

+

+

-

+

+

+

+

-

+

+

+

-

+

+

+

-

+

+

+

+

+

+

+

-

+

+

+

+

+

+

+

-

+

+

+

-

-

+

+

+

+

+

+

+

-

+

+

-

+

Construct validity was assessed for all UCLA-LS versions except UCLA-11. Moderate quality evidence of sufficient convergent validity was available for the original UCLA-20 (Dan Russell et al., 1978; Solano, 1980), UCLA-16 (Faustino et al., 2019), and UCLA-6 (Nazzal et al., 2018; Neto, 2014). Moderate quality evidence of sufficient discriminative validity was available for the original UCLA-20 (Dan Russell et al., 1978; Solano, 1980), UCLA-16 (Faustino et al., 2019), UCLA-10 (Knight et al., 1988; D. W. Russell, 1996), UCLA-8 (Hays & DiMatteo, 1987; Wilson et al., 1992), UCLA-4 (Hays & DiMatteo, 1987; Wilson et al., 1992), and UCLA-3 (Chalise et al., 2007; Hawkley et al., 2020; Igarashi, 2019; Matthews-Ewald & Zullig, 2013; Robinson-Whelen et al., 2016). Low quality evidence of sufficient criterion validity was available for the revised UCLA-20 (Anderson & Malikiosi-Loizos, 1992), UCLA-8 (Doğan et al., 2011; Hays & DiMatteo, 1987; Wilson et al., 1992), UCLA-6 (Nazzal et al., 2018; Neto, 2014), and UCLA-3 (Arimoto & Tadaka, 2019; Hughes et al., 2004; Igarashi, 2019), and lastly, high quality evidence of sufficient cross-cultural validity was available for the UCLA-6 (Wongpakaran et al., 2020) (Table 2, Online Resource 5).

Reliability of Long and Short Versions of UCLA-LS

The internal consistency (Cronbach’s Alpha) was assessed for all UCLA-LS versions. Results revealed sufficient internal consistency of all versions except for the original UCLA-20 (Ergün et al., 2020; Gillespie et al., 2021; Dan Russell et al., 1978; Solano, 1980) and UCLA-5 (Lamm & Stephan, 1987) due to lack of studies on structural validity. The quality of evidence ranged from high (UCLA-7 (Elphinstone, 2018; Oshagan & Allen, 1992) and UCLA-6 (Elemo et al., 2020; Nazzal et al., 2018; Neto, 2014)) to very low. Test re-test reliability was assessed for the revised UCLA-20 (Hartshorne, 1993; Hojat, 1982; Stephan et al., 1988), version 3 UCLA-20 (Balandin et al., 2006; Britton & Conner, 2007; Russell, 1996; Zarei et al., 2016), and UCLA-3 (Cole et al., 2015). Moderate quality evidence of sufficient reliability was available for version 3 UCLA-20 (Balandin et al., 2006; Britton & Conner, 2007; Russell, 1996; Zarei et al., 2016) with excellent Interclass Correlation Coefficient (ICC) values of 0.76–0.93. A summary of results for internal consistency and test re-test reliability is presented in Online Resource 6, and the overall quality evidence is summarized in Table 3.

Table 3

Reliability results summary and evidence quality of UCLA-LS versions

Scale version	Internal consistency				Test-retest
	Cronbach’s Alpha	N(n)	RoBRR	GRADE	ICC/ r	N(n)	RoBRR	GRADE
Original UCLA-20	0.82–0.96	2372 (5)	D ?	M1
Revised UCLA-20	0.79–0.90	1474 (4)	Vg +	L6	r = 0.72–0.85	113 (2)	D ?	M1
Version 3 UCLA-20	0.71–0.95	1279 (3)	Vg +	L6	ICC = 0.76–0.93	738 (4)	D +	M1
UCLA-16	0.92–0.93	154 (1)	Vg +	M6
UCLA-11	0.87	3706 (1)	I +	VL3
UCLA-10 (Knight et al., 1988, A)	0.86–0.89	1736 (2)	I +	L2
UCLA-10 (Knight et al., 1988, B)	0.83–0.90	1736 (2)	I +	L2
UCLA-10 (Russell et al., 1996)	0.85–0.97	986 (5)	D +	M1
UCLA-8	0.57–0.90	4027 (8)	Vg +	L4,6
UCLA-7	0.85–0.90	1072 (2)	Vg +	H
UCLA-6 (Neto, 2014)	0.73–0.82	1749 (3)	Vg +	H
UCLA-6 (Niu et al., 2018)	0.91–0.94	484 (1)	Vg +	L6
UCLA-6 (Wongpakaran et al., 2020)	0.72–0.84	719 (1)	Vg +	H
UCLA-5	0.84	154 (1)	D ?	L2
UCLA-4	0.38–0.94	1873 (4)	Vg +	M4
UCLA-3	0.67–0.88	72,111 (17)	Vg +	L4,6	r = 0.62	108 (1)	D ?	L2

RR: Results rating: (+) sufficient, (−) insufficient, (?) indeterminate. The quality of evidence was downgraded for: 1. Serious risk of bias, 2. Very serious risk of bias, 3. Extremely serious risk of bias, 4. Serious inconsistency, 5. Very serious inconsistency, 6. Based on structural validity. UCLA-LS: University of California, Los Angeles Loneliness Scale, N: sample size, n: number of studies included in the decision process, RoB: risk of bias (Vg: very good, A, adequate, D: doubtful, I: inadequate), GRADE: Grading of Recommendations Assessment, Development, and Evaluation (H: high, M: moderate, L: low, VL: very low), ICC: interclass correlation coefficient, r: Pearson’s correlation

?

+

?

+

+

+

+

+

+

+

+

+

+

+

+

?

+

+

?

Cross-Cultural Adaption of UCLA-LS

Table 4 displays the ratings of the translation and cross-cultural adaptations of 12 studies across ten countries and societies including Iran, Greece, Korea, Taiwan, Nepal, Turkey, Italy, Japan, Spain, and Thailand. None of these studies followed 100% of the recommended cross-cultural adaptation guidelines when performing the initial translation, synthesis, back-translation, and expert committee review. However, two studies (Sancho et al., 2020; Zarei et al., 2016) followed the guidelines for the initial translation of version 3 UCLA-LS, and one study (Zarei et al., 2016) included an expert committee to review version 3 UCLA-LS (Table 4).

Table 4

Cross-cultural adaptations of UCLA-LS into different languages

Author & year(Countries/Societies, Languages)	UCLA-LS	Forward translation	Synthesis	Back translation	Expert committee review	Pretesting	Submission to developers or coordinating committee
Hojat 1980 (Iran, Persian)	Revised UCLA-20	?	0	0	0	?	?
Anderson & Malikiosi-loizos, 1992 (Greece, Greek)	Revised UCLA-20	?	0	?	0	?	0
Kim, 1999 (Korea, Korean)	Revised UCLA-20	?	0	?	0	0	?
Wang et al., 2001 (Taiwan, Chinese)	Version-3 UCLA-20	0	0	0	0	?	0
Chalise et al., 2007 (Nepal, Nepalese)	UCLA-3	?	0	?	0	?	0
Wu & Yao, 2008 (Taiwan, Chinese)	UCLA-8	0	0	0	0	0	0
Durak & Senol-Durak, 2010 (Turkey, Turkish)	Version-3 UCLA-20	?	0	?	?	0	0
Boffo et al., 2012 (Italy, Italian)	Version-3 UCLA-20	?	0	?	0	0	0
Zarei et al., 2016 (Iran, Farsi)	Version-3 UCLA-20	+	?	?	+	?	0
Igarashi, 2019 (Japan, Japanese)	UCLA-3	?	0	?	?	0	0
Sancho et al., 2020 (Spain, Spanish)	Version-3 UCLA-20	+	?	?	?	?	0
Wongpakaran et al., 2020 (Thailand, Thai)	Revised UCLA-20	?	0	?	0	?	?

(+): Positive rating; (0): no information available, (?): indeterminate [inadequate methods used or insufficient scoring on subcategories to obtain positive rating]. UCLA-LS: University of California, Los Angeles Loneliness Scale

Discussion

Selecting the most appropriate instrument to assess loneliness and to evaluate the outcome of approaches to its management is essential in clinical research and practice. Information on the methodological quality of such instruments helps inform which outcome measures have the most robust psychometric properties. In this systematic review, we evaluated the psychometric properties and cross-cultural adaptation of the 20-item and all short forms of the UCLA-LS. We synthesized the current evidence from 81 studies and identified three versions of the 20-item and nine short versions of the UCLA-LS. Validity and reliability studies were supportive of the use of the UCLA-LS for assessing loneliness in adults.

Content validity is especially important as it reflects how items are representative of the construct to be measured (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). However, the lack of studies on the scale developmental process and content validity prevented an assessment of content validity. The internal structure of the outcome measure was determined from its structural validity, internal consistency, and cross-cultural validity. To assess the structural validity of a scale, CFA and EFA are the commonly used statistical methods. However, CFA is preferred over the EFA because it tests whether measures of a construct are consistent with a priori hypotheses and confirm the factor structure of a set of observed variables (model fit). Besides, the internal consistency assesses the degree of the interrelatedness among the items on a scale while the cross-cultural validity examines whether respondents from different groups respond similarly to a particular item (measurement invariance) (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). Based on these parameters, high to moderate-quality evidence supported sufficient internal structure of the short versions UCLA-10, UCLA-8, UCLA-7, UCLA-6, and UCLA-4, while low-quality evidence supported sufficient internal structure of the long versions (Revised UCLA-20 and version 3 UCLA-20) of the UCLA-LS, to encourage the use of these versions in loneliness-related research and clinical practice.

Good test re-test reliability reflects the internal validity of the outcome measure and ensures that the repeated measurements are stable over time (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). There were limited and indeterminate reports on test re-test reliability because of unclear test conditions, lack of ICC values, and inadequate measurement intervals between the two administrations of the scale. However, only version 3 UCLA-20 demonstrated moderate-quality evidence of sufficient test re-test reliability with excellent ICC values.

Sufficient construct validity is necessary to ensure that the intended theoretical construct and results are consistent with the hypotheses (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018). Although we noted low- to moderate-quality evidence of sufficient construct validity for some of the scales, the quality of evidence was reduced due to the risk of bias and inconsistency of study results.

Many of the UCLA-LS versions such as the revised UCLA-20, version 3 UCLA-20, UCLA-8, and UCLA-3 were translated into multiple languages (e.g., Chinese, Turkish, Persian, Spanish, and Thai) and culturally adapted across several countries and societies (e.g., Greece, Italy, Japan, Taiwan, Korea, and Nepal). Additionally, the psychometric properties of the translated UCLA-LS versions were evaluated and reported by several studies. However, there was insufficient information about their cross-cultural adaptation because many of these studies were not available in English (Çinar & Toker, 2019; Lamm & Stephan, 1987; Nazzal et al., 2018; Stephan et al., 1988).

The UCLA-LS has been validated for use in individuals with different clinical conditions. For instance, the validity and reliability of version 3 UCLA-20 and UCLA-3 were assessed in patients with cerebral palsy (Balandin et al., 2006), opioid dependence (Britton & Conner, 2007), physical and mental disabilities (Stancliffe et al., 2014), stroke and brain tumors (Hagger & Riley, 2019). It has also been assessed among those with obesity, hypertension and sleep apnea (Chalise et al., 2007; Kurina et al., 2011). Besides, the internal consistency of the original UCLA-20, revised UCLA-20, UCLA-10 and UCLA-8 were assessed in patients with COVID-19 (Gillespie et al., 2021), diabetes (Hackett et al., 2019), hypertension and cardiovascular diseases (Gan et al., 2015), and AIDS (Hansen et al., 2012). However, the insufficient information in these studies precluded a robust conclusion about the suitability of these versions for use in the corresponding clinical populations. Studies among those with chronic medical conditions such as chronic lung disease are lacking despite the high prevalence of loneliness among such patients (Reijnders et al., 2018).

Despite the large number of studies on physical, mental, and social interventions on loneliness (Ma et al., 2020), information on responsiveness has been consistently lacking. Such information is valuable when tracking change with time (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018).

Review Limitations

A limitation of this review is the exclusion of 17 non-English articles, which might have yielded important additional information regarding scale development, content validity, cross-adaptation, and translation. Direct comparisons were limited by the many sub-versions of the UCLA-10 and UCLA-6, thereby preventing clear conclusions regarding their psychometric properties. We were aware of the small risk of publication bias by excluding conference posters and abstracts. However, the results of this systematic review were qualitatively summarized. As a result, no meta-analysis was conducted, and the possibility of publication bias was not assessed.

Conclusion

This is the first systematic review that provides a detailed assessment of the psychometric properties of the 20-item UCLA-LS and its subsequent abbreviated versions, spanning multiple languages, cultures, and social circumstances. High-quality evidence supports sufficient internal structure of the UCLAs: 10, 7, 6, and 4, suggesting that these short versions would best serve those interested in loneliness-related research and clinical practice. Future studies are required to assess the responsiveness, cross-cultural and criterion validity of the UCLA-LS among a broad variety of clinical and social populations. This information will further increase our understanding of the psychometric properties of this outcome measure.

(DOCX 18 kb)

(DOCX 19 kb)

(DOCX 13 kb)

(DOCX 38 kb)

(DOCX 84 kb)

(DOCX 56 kb)