Mental health literacy measures evaluating knowledge, attitudes and help-seeking: a scoping review.

BACKGROUND: Mental health literacy has received increasing attention as a useful strategy to promote early identification of mental disorders, reduce stigma and enhance help-seeking behaviors. However, despite the abundance of research on mental health literacy interventions, there is the absence of evaluations of current available mental health literacy measures and related psychometrics. We conducted a scoping review to bridge the gap.
METHODS: We searched PubMed, PsycINFO, Embase, CINAHL, Cochrane Library, and ERIC for relevant studies. We only focused on quantitative studies and English publications, however, we didn't limit study participants, locations, or publication dates. We excluded non-English studies, and did not check the grey literature (non peer-reviewed publications or documents of any type) and therefore may have missed some eligible measures.
RESULTS: We located 401 studies that include 69 knowledge measures (14 validated), 111 stigma measures (65 validated), and 35 help-seeking related measures (10 validated). Knowledge measures mainly investigated the ability of illness identification, and factual knowledge of mental disorders such as terminology, etiology, diagnosis, prognosis, and consequences. Stigma measures include those focused on stigma against mental illness or the mentally ill; self-stigma ; experienced stigma; and stigma against mental health treatment and help-seeking. Help-seeking measures included those of help-seeking attitudes, intentions to seek help, and actual help-seeking behaviors.
CONCLUSIONS: Our review provides a compendium of available mental health literacy measures to facilitate applying existing measures or developing new measures. It also provides a solid database for future research on systematically assessing the quality of the included measures.

Background

Epidemiology of mental illness

Approximately 70 %-75 % of adult mental health problems and mental disorders start to manifest during adolescence or early adulthood (12-25) [1, 2]. Globally, mental disorders make up about 1/3 of the burden of illness in adolescence and young adulthood [3]. Untreated mental health problems and disorders in adolescents and young adults are strong predictors of poor vocational achievements, problematic interpersonal and family functioning, as well as reduced life expectancy due to associated medical conditions, such as diabetes, heart diseases and stroke, respiratory conditions, and suicide [4-7]. However, despite the great burden of illness incurred by these conditions, research shows, worldwide, between 70 %-80 % of young people and adults do not receive the mental health care they need [8-10]. A recent systematic review [11] of perceived barriers and facilitators for mental health help-seeking indicated that perceived stigma and embarrassment, problems in symptom identification and a preference for self-reliance were the most important intra-personal barriers to mental health help-seeking.

Mental health literacy

Mental health literacy is a significant determinant of mental health and has the potential to improve both individual and population health [12-14]. Evidence shows that improved knowledge about mental health and mental disorders, better awareness of how to seek help and treatment, and reduced stigma against mental illness at individual, community and institutional levels may promote early identification of mental disorders, improve mental health outcomes and increase the use of health services [15-17].

We conceptualize mental health literacy to include 4 domains: 1) understanding how to obtain and maintain good mental health; 2) understanding mental disorders and their treatments; 3) decreasing stigma against mental illness; and 4) enhancing help-seeking efficacy [13, 18]. And therefore, mental health literacy addresses 3 inter-related cocepts: knowledge (knowledge of mental illness and positive mental health), attitudes and help-seeking efficacy. This definition is consistent with the current construct of health literacy defined and promoted by the WHO as an empowerment tool for people to participate in their health care [19].

We located five reviews on the effectiveness of mental health literacy interventions [12, 18, 20–22]. In addition there were three literature reviews describing stigma [23-25] and knowledge measures. However, there has been a lack of comprehensive understanding of current available mental health literacy measures. Thus, there exists a need to conduct a study to help better understand strengths and weaknesses of existing measures and to help shape future development of measures. We conducted a scoping review, a systematic approach to map the literature in an area of interest and to accumulate and synthesize evidence available, to bridge the gap. This current scoping review was guided by Arksey and O’Malley’s work (2005) [26], proposing four purposes: 1. to examine the extent, range and nature of research activity; 2. to determine the value of undertaking a full systematic review; 3. to summarize and disseminate research findings; and 4. to identify research gaps in the existing literature.

We analyzed available mental health literacy measures that focus on four common mental disorders with onset before or during adolescence and young adulthood: Schizophrenia, Depression, Anxiety Disorders, and Attention Deficit Hyperactivity Disorder (ADHD).

Methods

We used our definition of mental health literacy [13, 18] that is composed of 4 constructs addressing three outcomes: mental health knowledge (including knowledge about positive mental health (construct 1) and knowledge about mental illness and treatments (construct 2)), stigma/attitudes towards mental illness, and help-seeking, to define our search scope.

Search strategy

One of the authors of this review and a health librarian designed the search strategies together. We searched PubMed, PsycINFO, Embase, CINAHL, Cochrane Library, and ERIC between 2013 and 2014, and re-ran the search in 2015. We applied four sets of search terms to identify domains of mental health literacy as outlined in Additional file 1.

Inclusion criteria

We included quantitative studies that used, developed, or investigated measurement properties of mental health literacy measures evaluating any one, or combinations of the mental health literacy outcomes: knowledge, stigma/attitudes towards mental disorders, and help-seeking. Study designs included any type of quantitative studies: randomized controlled trials (RCTs), cluster RCTs, quasi-experimental studies; cohort studies; cross-sectional/survey studies, and controlled-before-and-after studies (pre/post tests). Only studies published in English were eligible and non-English publications were excluded at the screening stage. Year of publication and study participants, including their age, were not restricted.

Exclusion criteria

Studies were not eligible if they addressed mental health literacy but did not mention or describe the measure applied in the study. Studies of smoking prevention/cessation and other substance use prevention programs were not included. Studies of suicide prevention interventions that did not address related mental disorders, such as Depression were not eligible. Qualitative studies were excluded.

Data extraction and study selection (Charting)

Two reviewers used the search strategy, and independently searched pre-identified databases. We first screened out irrelevant studies which mostly focused on stigma against HIV/AIDS, cognitive behavioral therapies, substance abuse/smoking, resilience scales, and clinical treatment related studies by reviewing titles and abstracts. We then imported the remaining studies, into RefWorks 2.0 database management software (2001) [27]. Duplicates were removed. We then screened titles and abstracts again and briefly scanned the full text to exclude studies not evaluating target outcomes. All studies that passed this exclusion process were included in the third stage of review for relevance by scanning title, abstracts and the full text for relevancy. At the next stage, we reviewed full-text articles for all the final included studies. Additionally, we added original studies which were referenced in included studies that cited their psychometric properties. We also checked the reference list of included studies for additional studies.

We applied “charting” techniques to conduct data extraction. For scoping reviews, “charting” is a data extraction technique to synthesize and interpret data by “sifting, charting and sorting materials according to key issues and themes” (page 26) [26, 28]. The key themes we followed in the “charting” of our data are the three outcome measures: knowledge, attitudes, and help-seeking, which was the base of the data categorization. We also charted data by year of publication, study location, study type, outcome measures, and types of psychometrics examined (e.g., reliability, validity, and responsiveness/sensitivity to change). The detailed charting process for this review is depicted in Fig. 1.

Fig. 1

Charting process (data extraction process)

A data extraction form, developed in advance, was used for data extraction. We categorized studies into four types based on the extent of how psychometric properties were investigated and reported in the study: validation studies with evaluating psychometrics (any type) and/or responsiveness/sensitivity to change as the major purpose of the study (coded as P); studies evaluating effectiveness of interventions or survey studies evaluating psychometrics (any type) and/or responsiveness/sensitivity to change of the outcome measures (coded as I/P or S/P); studies just reporting but not evaluating psychometrics and/or responsiveness/sensitivity to change of the applied tool (coded as I/? or S/?); and studies mentioning the measurement tool applied but not reporting psychometrics (coded as I or S), including studies that quoted psychometrics from other studies, but did not evaluate it in the current study. We then sorted and defined the data by measures on knowledge, attitudes/stigma towards mental illness, and help seeking respectively, listed authors who first applied the tool, and calculated the number of psychometrics studies for each outcome measurement. In addition, we collated all psychometrics studies in separate tables. Figure 1 illustrates this process.

Once this charting process was completed, we reviewed all included studies, developed and populated tables, and created charts and figures according to the above described typology in an Excel spreadsheet. To help ensure consistency in interpretation and validity of the final results, one of the reviewers read and charted all included studies. Then the second reviewer checked all tables and compared and discussed the results with the first reviewer and they came to a consensus on the interpretation of the results. One methodology expert and two content experts were invited to help make the final decision when consensus was not reached between the two reviewers.

Results

Figure 2 presents the flow chart of the screening process and final included studies. A total of 401 studies1 were identified that met study criteria, including 113 studies containing 69 knowledge measures, 307 studies containing 111 stigma measures, and 91 studies containing 35 help-seeking measures. Measures that modified and applied the concepts of the original ones were not counted as a new measure in our review. Out of the 401 studies, 130 validation studies reported and evaluated psychometrics (reliability, validity and/or the responsiveness/sensitivity to change) of the measures applied (P, I/P, or S/P), including14 knowledge studies (14 measures) (Table 1) [29-42], 102 stigma/attitudes studies (65 measures) (Table 2) [35, 36, 39, 43– 142], and 19 help-seeking studies (10 measures) (Table 3) [35, 101, 143–159]. These 3 tables summarized characteristics of validated studies, however we only listed authors who developed or first applied the measures although we included and summarized study results from other authors. Of these 130 studies, 5 studies also evaluated and reported responsiveness/sensitivity to change. Total number of studies for each sub-category may not necessarily match the total number of included studies because some studies tested more than one measurement tool in one study.

Fig. 2

Search results

Table 1

Psychometrics of knowledge measures

Measures	Developer/Author	Reliability	Validity & responsiveness	Content
1.Knowledge about Schizophrenia Questionnaire (KASQ)	Ascher-Svanum, 1999 [29]	KR-20a = .85, .89	R; CT	S
		r ita = .46; .51
		rb = .83 (p < .005)
2. Knowledge About Schizophrenia Test (KAST)	Compton et al., 2007 [30]	KR-20a = .82 (.45-.78)	CS; CT; CR	S
3. Multiple-Choice Knowledge of Mental Illnesses Test (MC-KOMIT)	Compton et al., 2011 [31]	αa = .68-.75	R; CS; CT	G
		rb = .79 (p < .001)
4. Mental Health Knowledge Schedule (MAKS);	Evans-Lacko et al., 2010 [32]	αa = .65	CT	G
		Lin’s Pc b = .71;
		kb = .57-.87
5. Depression Multiple Choice Question (MCQ)	Gabriel & Violato, 2009 [33]	αa = .68	CT; CV; FA	D
6. Depression literacy scale (D-Lit)	Kiropoulos et al., 2011 [34]	αa = ..88, .92		D
		rb = .78, .80 (p < 0.001)
	Gulliver et al., 2012 [35]	αa = .70
		rb = .71 (p = .02)
7. Anxiety Literacy Questionnaire (A-Lit)	Gulliver et al., 2012 [35]	αa = .76;		A
		rb = .83 (p = .003)
8. Test of Knowledge About ADHD (KADD)	Hepperlen et al., 2002 [36]	αa = .81-.82;	FA; CT	ADHD
9. Knowledge about Depression and Mania Inventory (KDMI)	Kronmuller et al., 2008 [37]	αa = .89 (.76-.81);	CC/CR; CT; D; R	D
		r ita = .36-.43
10. Journey of Hope (JOH) Outcome Survey	Pickett-Schenk et al., 2000 [38]	αa = .75-.83	CS; FA	G
11. Knowledge of Mental Disorders (KMD)	Serra et al., 2013 [39]	αa = .588	CS; FA	G
12. Adolescent Depression Knowledge Questionnaire (ADKQ)	Shelley et al., 2014 [40]	αa = .89	FA	D
13. Mental health disorder recognition questionnaire (MDRQ)	Swami et al., 2011 [41]	kc = .94, .96	CS; CV	G
14. Mental Health Knowledge Questionnaire (MHKQ)	Wang et al., 2013 [42]	αa = .69	FA	G

ainternal consistency reliability: Cronbach alpha (α), Kuder-Richardson 20 (KR-20), item to total correlation (r it) ; btest-retest reliability: weighted kappa (k), Pearson correlation coefficient (r), Lin’s Pc; c inter-rater reliability: weighted kappa (k)

CT Content validity, CS Construct validity, CR Criterion validity, CV Convergent validity, CC Concurrent validity, D Discriminant validity, FA Factor analysis, R Responsiveness

G general knowledge, D Depression, S Schizophrenia, ADHD Attention Deficit Hyperactivity Disorder, A Anxiety

Table 2

Psychometrics of stigma/attitudes measures

Measures	Developer/Author	Reliability	Validity & responsiveness	Content
1. Social Distance (SD)	Bogardus, 1925 [49]
Link Social Distance scale (1987)	Link et al., 1987 [107]	αa = .74; .75; .92	CS; CR/CV; D; FA	A
Bogardus Social Distance Scale (modified)	Angermeyer & Matschinger ., 2003 [44]	αa = .90	FA
Social distance scale	Link, 1983 [104]	αa = .85, .91;	CS
Reported and Intended Behaviour Scale (RIBS)	Evans-Lacko & Rose et al., 2011 [69]	αa = .85;
		kb = .75
RIBS- Japanese (RIBS-J)	Yamaguchi et al., 2014 [141]	αa = .83	CC; FA
		Lin’s Pcb = .71
Social Contact Scale	Jackson & Heatherington, 2006 [91]	α = .55-.75	FA
The Social Supports Acceptance Scale (SSAS)	Mansouri & Dowell, 1989 [114]	α = .80-.94	CS; R
2. Opinions about Mental Illness (OMI)	Cohen & Struening, 1962 [59]		CS; FA	A
OMI	Struening & Cohen, 1963 [128]	αa = .299-.80	FA
OMI in Chinese Community scale (OMICC)	Ng & Chan, 2000 [123]	αa = .87 (.43-.72)	FA
3. Community Attitudes towards Mental Illness (CAMI)	Taylor & Dear, 1981 [134]	αa = .62-.90	CS; D; FA	A
Fear and Behavioural Intentions (FABI)	Svensson et al., 2011 [131]	αa = .80
		kb = .29-.54
Mental Health Attitude Survey for Police	Clayfield et al., 2011 [58]	αa = .87	CS; CV; FA
4. Devaluation-Discrimination tool (DD)	Link 1987 [105]	αa = .73-.83	CS	A
Perceived Discrimination Devaluation (PDD)	Interian et al., 2010 [89]	αa = .80	CS; CV/CR; FA
Public stigma	Moses, 2009 [121]	αa = .76	CV; CS; D
		Kc = .79-.90
Stigma-Devaluation Scale (SDS)	Dalky, 2012 [64]	αa = .87	FA
Depression is a Matter of Will	Aromaa et al., 2010 [45]		CS; FA
5. Depression Stigma scale (DSS)	Griffiths et al., 2004 [79]	αa = .75-.82	DS; CV; D; FA	A
		rb = .86 (p = .001)
6. Attribution Questionnaire (AQ)	Corrigan et al., 2003 [61]	αa = .70-.96	CS; FA	A
AQ-27	Brown, 2008 [53]	αa = .60-.93	CV; FA
		ICCb = .72-.90
r-AQ	Pinto et al., 2012 [125]	αa = .70	FA
7. Internalized Stigma of Mental Illness (ISMI)	Ritsher et al., 2003 [126]	αa = .84-.98	CS; CC; D; FA; P	C
		rb = .92 (.61-.91) (p < .05) ICCb = .78
Parents’ Internalized Stigma of Mental Illness Scale (PISMI)	Zisman-llani et al., 2013 [142]	αa = .61-.78	FA
ISMI Chinese (ISMIS-C)	Lien et al., 2014 [103]	αa = .90;	CS; FA
		ICC = .36-.73
ISMI-10	Boyd et al., 2014 [24]	αa = .75	CT; CC; CS
8. Perceived dangerousness (PD)	Link, et al., 1987 [106]	αa = .85	CS	A
Link Stigma Scale (dangerousness)	Bagley & King, 2005 [46]	αa > .80	CS; CR; D
Dangerousness Scale (DS)	Penn et al., 1994 [124]	αa = .78	CS
9. British Omnibus National Survey (ONS)	Kobau et al., 2010 [100]	αa = .66-.69	CV; CC; FA	A
Changing Mind	Svensson et al., 2011 [131]	αa = .19-.46
		rb:Poor to moderate
10. Self-stigma of Seeking Help (SSOSH)	Vogel et al., 2006 [136]	αa = .88	CS; P; D; FA	C
		rb = .72
11. Self-stigma of Mental Illness (SSMIS)	Corrigan et al., 2006 [63]	αa = .64-.91;	CS; D	C
		rb = .62-.82
SSMIS-Short Form	Corrigan et al., 2012 [62]	αa = .22-.87	CS; D
12. Attitudes to Mental Illness Questionnaire (AMID)	Luty et al., 2006 [108]	rb = .70; .93	CC; FA	A
13. Stigma Scale for Receiving Psychological Help (SSRPH)	Komiya et al., 2000 [101]	αa = .72	CS; CR; FA	D
14. Affective Reaction Scale	Penn et al., 1994 [124]	αa = .86	CS	A
15. Discrimination and Stigma Scale (DISC)	Brohan et al., 2013 [52]	αa = .78;	CV; DV	B
		Lin’s Pc b = .88, 89 (p < .001)
		kb = 0.45-0.89
		Kc = .62-.97
Questionnaire on Anticipated Discrimination (QUAD)	Gabbidon et al., 2013 [72]	αa = .86;	CV	B
		Lin’s Pcc = .81
		kb = .41-.80
16. Mental Illness: Clinician’s Attitudes (MICA)	Kassam et al., 2010 [94]	αa = .79	CV; DV; FA; R	A
		Lin’s Pc b = .80 (p < .001)
MICA-v4	Gabbidon et al., 2013 [73]	αa = .72;	CV; FA
		r it a ≥ .2
17. Day’s Mental Illness Scale (DMISS)	Day et al., 2007 [65]	αa = .71-.86	CS; FA	A
18. ADHD Stigma Questionnaire (ASQ)	Kellison et al., 2010 [98]	αa = .55-.93;	CS; CV; DV; FA	A
		ICCb = .71(.55-.73)
Stigmatization towards Adults ADHD	Fuermaier et al., 2012 [71]	αa = .91 (.61-.87)	CS
19. Rejection Experiences	Link, 1987 [105]	αa = .73-.85	CS; CR; CV; D	B
		Kc = .79-.90
20. Generalized Anxiety Stigma Scale (GASS)	Griffiths et al., 2011 [81]	αa = .86-0.91	CR; CS; CV; D; FA	A
		rb = .55,.58,.91 (p < .0001, .001)
21. Relatives’ opinions toward Schizophrenia	Magliano et al., 1999 [111]	αa = .56-.66;	CS; FA	A
		Kb = .36-.84
Questionnaire on the Opinions About Mental Illness (QO)	Magliano et al., 2004 [112]	αa = .42-.72;	FA	A
		kc = .50-1.0
22. EMIC	Chowdhury et al., 2000 [57]	αa = .66-.76;		A
		Kc = .77-.89
23. Stigma Concerns about Mental Health Care (SCAMHC)	Interian et al., 2010 [89]	αa = .69	CS; CV; CR; FA	D
24. Latino Scale for Antidepressant Scale (LSAS)	Interian et al., 2010 [89]	αa = .66	CS; CV; CT; FA	D
25. Devaluation of Consumer Families Scale	Struening et al., 2001 [129]	αa = .82	CV; FA	A
26. Devaluation of consumers scale	Struening et al., 2001 [129]	αa = .71-.77	CV; FA	A
27. Consumer Experiences of Stigma Questionnaire (CESQ)	Bagley & King, 2005 [46]	αa = .79-.82	CC; CS; CR; D; FA	B
28. Attitudes towards Depression and Its Treatment (ATDT)	Gabriel & Violato, 2010 [74]	αa = .57-.79	CT; FA	A
29. Stigmatization Scale	Harvey, 2001 [83]	αa = .90, .94	CS; D; FA; CR	B
30. Psychiatric Skepticism Scale (PSS)	Swami & Furnham, 2011 [132]	αa = .92;.94	CS; FA	D
31. Emotional Reactions	Angermeyer & Matschinger, 2003 [44]		CS; FA	A
32. Labeling of mental illness	Angermeyer & Matschinger, 2003 [44]	kc = 0.85		A
33. Personal Attributes	Angermeyer & Matschinger, 2003 [44]		CS; FA	A
34. Depression Attitude Questionnaire (DAQ)	Botega et al., 1992 [50]		FA	A
R-DAQ	Haddad et al., 2015 [82]	αa = .84;	CT; CS; CV; FA	A
		ICCb = .62
35. Attitudes Toward psychiatry-30	Burra et al., 1982 [55]	r it a = .10-.64;	CC	D
		Split-half ra = .89, .90;
		ICCb = .51-.87
36. Opening Minds Scale for Health Care Providers (OMS-HC)	Kassam et al., 2012 [97]	αa = .78; .79; .82	CS; FA; CT; R	A
		r it a = -.13-.57;
		ICCb = .66 (p < .001)
37. Stigma Scale	King et al., 2007 [99]	αa = .87 (.64-.87);	CS; CC; FA	A
		kb = .41-.71
Chinese Stigma Scale (CSS)	Ho et al., 2015 [87]	αa = .83 (.58-.84)	CC; FA	A
38. Stigma Experiences Scale	Stuart et al., 2005 [130]	αa = .91	CS	B
		KR-20a = .83;
39. Attitudes Toward Serious Mental Illness Scale-Adolescent	Watson et al., 2005 [139]		FA	A
40. Self reported prejudiced attitudes	Andersson et al., 2010 [43]	αa = .78	FA	A
41. Self-Stigma of Depression Scale	Barney et al., 2010 [47]	αa = .87	CS; CV; FA	C
		ICCb = .63 (p = .000)
42. Employer Attitude Questionnaire (EAQ)	Diksa & Rogers, 1996 [66]		FA	A
43. 15-Item Stigma Questionnaire	Gibbons et al., 2012 [75]	αa = .85;	CC; CS; CV; CT	B
		ICCb = .75
44. Attitudes of Nursing Staff towards Co-Workers Returning from Psychiatric and Physical Illnesses	Glozier et al., 2006 [76]	αa = .76-.88	CS	A
45. Self-Esteem and Stigma Questionnaire (SESQ)	Hayward et al., 2002 [84]	αa = .71-.79;	CS	A
		rb = .63 (p < .0001)
46. Test of Knowledge About ADHD (KADD)	Hepperlen et al., 2002 [36]	αa = .81-.82;	FA	A
47. Beliefs toward Mental Illness (BMI)	Hirai & Clum, 2000 [86]	αa = .91;	CS; CC; FA	A
		r it = (.22 < r < .72)
48. Depression Self-Stigma Scale (DSSS)	Kanter, 2008 [92]	αa = .79-.95;	CS; CC; FA	C
		r it = .44-.83
49. General Attitude Questionnaire	Lam et al., 2005 [102]	αa = .88-.93		A
		rb = .72-.94
50. Secrecy	Link, 1987 [105]	αa = .73-.83	CS	C
51. Withdrawal	Link, 1987 [105]	α = .73-.83	CS	C
52. Attitudes to Severe Mental Illness (ASMI)	Madianos et al., 2012 [110]	αa = .88(.79-.86);	CS; P; FA	A
		rb = .89-.92 (p < .0001)
53. Affiliate Self-Stigma Scale	Mak & Cheung et al., 2008 [113]	αa = .94-.95;	CS; P; FA	C
		r it = .51-.81
Self-Stigma Scale-Short (SSS-S)	Wu et al., 2015 [140]	αa = .95	CC; CS; FA	C
54. Knowledge Test of Mental Illness (KT)	Michaels & Corrigan, 2013 [116]	rb = .50-.70 (p < .05; .001)	CC; CS	A
55. Attitudes Toward Social Competence and Integration of People with Mental Illness	Minnebo & Acker et al., 2004 [117]	αa = .77; .79	FA	A
56. Client Attitude Questionnaire	Morrison & Becker, 1975 [120]	rb = .90; .93		?
57. Libertarian Mental Health Ideology Scale (LMHIS)	Nevid & Morrison, 1980 [122]	αa = .81-.94	CS; FA	D
58. Personal stigma scale	Schneider et al., 2011 [127]	αa = .62-.92	FA	A
59. Child stigma scale	Moses, 2009 [121]	αa = .81	CV; CS; D	C
		kc = .79-.90
60. Beliefs and attitudes toward people diagnosed with psychosis	Serra et al., 2013 [39]	αa = .69	FA	A
61. Stigma of Depression Scale	Vega et al., 2010 [135]	αa = .69	FA	A
62. Perceptions of Stigmatization by Others for Seeking Help (PROSH)	Vogel et al., 2009 [137]	αa = .78-.91	CS; CC; FA	D
		rb = .82 (p < .001)
63. The Stigma Inventory for Mental Illness	Karidi et al., 2014 [93]	αa = .90 (.75, .85);	CT; CS; CC; FA	C
		rb = .80 (p < .001)
64. Peer Mental Health Stigmatization Scale	McKeague et al., 2015 [115]	αa = .80 (.70, .75);	CT; CS; D; FA	A
		rb = .65, 75
65. Endorsed and Anticipated Stigma Inventory (EASI)	Vogt et al., 2014 [138]	r it = .47-.75	CT; CV; D; FA	A & D

aInternal consistency reliability: Cronbach alpha (α), Kuder-Richardson 20 (KR-20), item to total correlation (r it), split-half reliability; btest-retest reliability: intraclass correlation coefficient (ICC), weighted kappa (k), Pearson correlation coefficient (r), Lin’s Pc; c inter-rater reliability: weighted kappa (k), Lin’s Pc

CT Content validity, CS Construct validity, CR Criterion validity, CV Convergent validity, CC Concurrent validity, D Discriminant validity, FA Factor analysis, R Responsiveness, DV Divergent validity, P Predictive validity

A Stigma against mental illness or the mentally ill, B Experienced stigma, C self-stigma, D stigma against help-seeking, treatment; mental health institution or psychiatry, ? not reported

Table 3

Psychometrics of help-seeking measures

Measures	Author/developer	Reliability	Validity	Content
1. Attitudes towards help-seeking scale (with various modified versions)	Fischer & Turner, 1970 [143]	αa = .83; .86	FA; CS	A/H
		r it = -.58 - .56 (p < .0001)
		rb = .73-.89
Attitudes Toward Seeking Professional Psychological Help Scale (ATSPPH)	Fischer & Farina, 1995 [147]	αa = .77-.98;.84;.90	FA; CS; CR
		r it a = .54;
		rb = .80
ATSPPH-SF	Elhai, et al., 2008 [146]	αa = .69; .77-.78; .84	FA; CS
		r it a > .40
		rb = .64 (p = .045)
2. Intention of Seeking Counseling Inventory (ISCI)	Cepeda-Benito & Short, 1998 [144]	αa = .89	FA; CS	I
3. General Help Seeking Questionnaire (GHSQ)	Deane et al., 2001 [145]	αa = .67, .76, .82	FA	I
	Gulliver et al., 2012 [35]	αa = .57-.77;
		rb = .42-.91 (p < 0.001)
	Wilson et al., 2005 [159]	αa = .70-.85;	P; CV; DV
		ICCb = .86-.92
4. Jorm Mental health literacy survey (items on attitudes/beliefs towards treatment)	Jorm, Blewitt et al., 2005 [149]	Kc = 0.15-1.00		A/T
	Jorm, Mackinnon et al., 2005 [150]		FA
	Reavley et al., 2014 [154]		CS
5. Help Seeking Intentions	Lee et al., 2014 [151]	αa = .74, .76	FA	I
6. The New Inventory of Attitudes Towards Seeking Mental Health Services (IASMHS)	Mackenzie et al., 2004 [152]	αa = .87 (.76-.82)	FA; CS	A/H
		rb = .64-.91 (p < 0.01)
7. Help-Seeking Attitude Scale (HSAS)	Nickerson et al., 1994 [153]	αa = .87	CC	A/H
8. Scale of Attitudes Toward Seeking Psychological Help for Secondary Students (ASPH-S)	Sahin & Uyar, 2011 [156]	αa = .85 (.59-.81)	FA	A/H
		r it a = .41-.57
		ICCb = .81
9. Help Seeking Acceptability (HSA)	Schmeelk-Cone et al., 2012 [157]	αa = .84-.88	FA; CS	A/H
		r it a = .81-.85
10. Parental Attitudes Toward Psychological Services Inventory (PATPSI) (based on ATSPPH)	Turner, 2012 [158]	αa = .72-.92	FA; CS	A/H
		ICCb = .66-.90

aInternal consistency reliability: Cronbach alpha (α), item to total correlation (r it); btest-retest reliability: intraclass correlation coefficient (ICC), Pearson correlation coefficient (r); c inter-rater reliability: weighted kappa (k)

CT Content validity, CS Construct validity, CR Criterion validity, CV Convergent validity, CC Concurrent validity, D Discriminant validity, FA Factor analysis, DV Divergent validity, P Predictive validity

A/H Beliefs/Attitudes towards help-seeking, I help-seeking intentions, A/T beliefs/attitudes towards treatment

Study characteristics, such as study participants, locations, publication dates, and tool outcomes are reported in Figs. 3, 4, 5 and 6. Studies were conducted in 32 countries, with the United States of America most commonly, followed by Australia and Canada. Study participants were mainly post-secondary students, especially students in psychology or related professions, followed by the general public, and mental health service users (e.g., patients and their families). Most of the studies (n = 337) were published after the year 2000.

Fig. 3

Study participants by study numbers

Fig. 4

Study sites by study numbers

Fig. 5

Publication dates by study numbers

Fig. 6

Measure content in each outcome (knowledge, attitudes and help-seeking) by study numbers

Knowledge measures

The most widely used knowledge measures (by the number of studies in which the measure was applied) include the Mental Health Literacy Questionnaire (MHLQ) by Jorm and colleagues (1997) [160], Mental Health Knowledge Schedule (MAKS) [32], the World Psychiatric Association (WPA) “Open the Doors” (WPA-OD) questionnaire [161], Depression Literacy Scale (DLS) [79], Knowledge about Schizophrenia Questionnaire (KASQ) [29], Schizophrenia Knowledge Questionnaire (SKQ) [162], and In Our Voices (IOV) knowledge measure [163].

The 69 knowledge measures evaluated general knowledge about mental health, knowledge on specific disorders such as depression, schizophrenia/psychosis, ADHD, and anxiety disorders (Fig. 6). They used different approaches to measure knowledge. Some measures, such as those based on the approach by Jorm et al. (1997) [160] used the recognition of specific mental disorders (e.g., depression or anxiety) from the vignette description of symptoms. Other knowledge measures evaluated factual knowledge about mental illness with the true/false/don’t know approach. This includes fact-based tests on terminology, prevalence, causes, diagnosis, etiology, prognosis, consequences, and course of illness; and knowledge about recognition, support, employment, treatment/help-seeking/controllability, and recovery/coping, etc. (e.g. [29, 32, 33, 41, 79, 161–165]). One tool addressed the ability to distinguish mental illness from neurological or somatic illnesses (e.g. [39]). There were a number of measures combining stigma knowledge and mental health knowledge [95, 166–169]. Finally, some were self-evaluation measures of extent of knowledge (e.g. [170, 171]).

Of the 69 measures, psychometric properties were reported for 26 (38 %). And the rest of the 43 measures (62 %) had no psychometric properties reported. Of 26 measures with reported psychometrics, 14 measures were evaluated for psychometric properties, including 2 measures for responsiveness/sensitivity to change [29, 31]. These 14 measures evaluated general mental health knowledge (6 measures), depression (4 measures), schizophrenia (2 measures), ADHD (1 measure), and anxiety disorders (1 measure) (Table 1). The rest of the twelve measures only reported but didn’t evaluate psychometrics (internal consistency) and therefore we didn’t include them in Table 1.

Most knowledge measures applied self-report multiple choice answers (true, false, I don’t know/not sure), or vignettes with open-ended/closed questions (e.g. [172]), or used Likert-scale statements as self- evaluation formats.

Stigma measures

Of all the stigma measures, the most widely used measures (by the number of studies where the measure was applied) include the Social Distance scale (SD) [49]; Opinions about Mental Health Illness (OMI) [59]; Community Attitudes towards Mental Illness (CAMI, a modified version of OMI) [134]; Devaluation-Discrimination (DD) [105]; Depression Stigma scale (DSS, also called Personal and Perceived Stigma of Mental Illness) [79]; Attribution Questionnaire (AQ) [61]; Internalized Stigma of Mental Illness (ISMI) [126]; and Perceived Dangerousness (PD) [106].

The 111 focus of the stigma/attitudes measures included: 1. stigma against mental illness or the mentally ill, such as social distance (the degree to which people are willing to accept the mentally ill in regular social life), personal stigma (participants’ personal attitudes toward people with mental illness) and perceived stigma (participants’ beliefs about others’ attitudes about mental illness); 2. self-stigma; 3. experienced stigma by mental health service users; 4. stigma against mental health treatment, psychiatry, help-seeking, or mental health care facilities. Further, some measures evaluated stigma against specific mental illnesses, such as depression, anxiety, ADHD, and schizophrenia/psychosis. Eleven studies (7 measures) did not report what aspects of stigma were measured (Fig. 6).

Social distance measures investigated issues such as a person’s willingness to engage the mentally ill in the workplace and the community (e.g., employment, renting, being neighbors, marriage) [46, 69, 106, 108, 124, 173]. Similarly, measures evaluating stigmatizing experiences by the mentally ill focused on challenges people with mental illness experience in family and social life [52, 75, 92, 99, 126, 130].

Measures evaluating personal and perceived stigma covered areas such as authoritarianism, benevolence, mental hygiene ideology, social restrictions to the mentally ill, and etiology [59, 134]. Other measures evaluated components such as stigma related to illness prevalence, consequences, dangerousness/threat, treatment and recovery of mental illness, or the social/family life, social responsibilities, human rights, intelligence [36, 74, 79, 81, 105, 111, 116, 135]. In addition, there were personal and perceived stigma measures focusing on emotional/rejection responses, willingness to help, and disclosure concerns [44, 48, 57, 61, 86, 98, 124].

Self-stigma measures mostly evaluated cognition such as self-esteem, self-confidence, self-satisfaction/concurrence, self-blame; negative emotions such as low pride of oneself, shame, embarrassment, sense of inadequacy, inferiority to others, helpless, pressure; and behaviors such as withdrawal, fear of seeking help, and secrecy [47, 63, 83, 92, 113, 121, 136].

Measures examining stigma against treatment/help-seeking/mental health care/medical model/psychiatry addressed perspectives and emotions. For example, some measures evaluated stigma towards help seeking (e.g., help-seeking as personal weakness; people seeking help being less likeable, disturbed, posed risks to others, and should hide the fact of seeking help) [101, 137]. Other tools [55, 122, 132] investigated stigma toward psychiatry, for example, skepticism towards psychiatry; and stereotypes of psychiatrists, psychiatric hospitals, patients, and psychiatric treatments. Some tools measured emotional responses (e.g., fear, discomfort and embarrassment) to psychological services and mental health care [89, 144].

Eighty one (73 %) articles on stigma tools reported on some psychometrics. Sixty five measures had evidence of reliability (e.g., Crobach’s α; item-total correlations; KR-20; test-retest reliability; inter-rater reliability), validity (e.g, construct; concurrent; discriminant; convergent; predicative), or responsiveness/sensitivity to change (Table 2). Sixteen measures demonstrated only internal consistency, but none included discussions on how this was measured. Of these 81 measures, 48 evaluated stigma against the mental illness/ the mentally ill in general; 11 were self-stigma measures; 6 evaluated personally experienced stigma; and 12 evaluated stigma against mental health treatment (psychological and pharmacological), psychiatry, help-seeking, or mental health care facilities. One tool did not specify what it measured.

Help-seeking measures

Of the 35 help-seeking related measures, the most widely used are: Attitudes towards Help-Seeking Scale (later modified as Attitudes toward Seeking Professional Psychological Help Scale) (ATSPPH) [143, 147]; the mental health literacy questionnaire (MHLQ) that contains items on beliefs towards treatments [160]; General Help Seeking Questionnaire (GHSQ) [145]; and Intention of Seeking Counseling Inventory (ISCI) [174].

These help-seeking measures evaluating help-seeking intentions; beliefs or attitudes towards seeking psychological help for mental health problems or illness; beliefs towards mental health help or treatment in general; actual help-seeking behaviors; help-seeking efficacy (e.g. knowledge about where and how to find help, and who to find help from); self-reported ability to help others; or multiple components such as help-seeking intentions, help-seeking efficacy, and barriers for help-seeking (Fig. 6).

Unlike measures of stigma against help-seeking described above, measures evaluating attitudes towards psychological help-seeking mostly addressed: recognition of need for psychological help; interpersonal openness; confidence in and trustworthiness of mental health practitioners [143]. Measures evaluating beliefs toward treatment mostly evaluated the perceived helpfulness, effectiveness or safety of various interventions [150, 175], or the myths of treatment [176]. One measure [177] added social norm items on perceived attitudes of others (e.g., friends, employer) on depression intervention.

Measures evaluating help-seeking intentions examined willingness, or preferences to seek help from different sources (e.g., friends, families, professionals, religion, or spiritual healers [151, 172, 178-182]. One measure [172] further evaluated 3 extra dimensions of help-seeking intentions: talking to the listed sources; comfort level of talking to these resources; and helpfulness of these resources. Another tool measured intention levels for various emotional/behavioral challenges among college students [144]. Two measures didn’t specify how intentions were measured [161, 183].

Measures addressing help-seeking behaviors evaluated whether help-seeking was sought, and if so, what type of help was sought (formal vs. informal) for both stressful events and mental illness [178, 181, 184, 185].

Ten measures had some psychometric evaluation such as internal consistency, reliability, factor analysis, construct validity, and criterion validity [143, 145, 147, 149–154, 156–159]. Details of the psychometrics of these 10 measures are presented in Table 3. The 10 measures with psychometrics addressed attitudes or beliefs towards help-seeking or treatments, and intentions for help-seeking (Table 3). Two measures reported the internal consistency of the tool [172, 181], but did not discuss how this were measured, and therefore were not included in the table. No psychometric properties were reported on measures of help-seeking behaviors.

Discussion

We identified a number of significant issues for consideration. These are: 1) representativeness of study samples; 2) geographic weighting: 3) adequacy of measurement of mental health literacy (knowledge, stigma, and help-seeking).

Representative samples

Almost half of the studies (n = 185) were conducted among adolescents and young adults, particularly with post-secondary students (n = 117) (Fig. 3) mostly from health related professions, such as psychology, social work, and nursing. This raises concerns about the generalizability of findings as participants are not representative of the general population.

Even within the context of postsecondary education, much less attention (only 9 studies) has been paid to the mental health literacy of educators, who are important role models and youth influencers in addressing mental health literacy [186]. Further research into mental health literacy should take these important factors into account.

Geographic weighting

Research on the measurement of mental health literacy started as early as in late 1950’S but did not bloom until after 2000 (n = 336; 84 %) (Fig. 5). Most studies (Fig. 4) took place in developed countries, especially the United States (n = 170; 42 %). Although there is ethnic diversity in the United States, the United States cannot be seen to represent other cultures. Moreover, different countries have different health systems and this may impact the implementation of mental health literacy approaches. For studies conducted in developing countries, authors either adapted existing measures, or used the conceptual framework from developed countries to create their measures, however, very few discussed the process of translation or the method of cultural adaptation. Therefore, the impact of important contextual factors, such as culture, ethnicity, geographic locations, education and health system, on mental health literacy and its measurement is currently unknown.

Adequacy of measurement

Our analysis suggests that, out of three outcomes of mental health literacy (knowledge, attitudes and help-seeking), most measures evaluated stigma (n = 111), followed by measures that evaluated knowledge (n = 69), and a smaller number of help-seeking (n = 33). Only a relatively small number of measures were validated in any way. Secondly, widely used measures are often not validated. For example, the WPA mental health knowledge questionnaire was applied in 9 studies but no research has been identified to analyze its psychometric properties except for internal consistency.

Given the high proportions of un-validated measures being applied, it was difficult to determine the value of the study results and not possible to conduct cross-study comparisons of different interventions. There is a pressing need to validate these measures before their application.

With the measures that have been validated, there has been no research identified that appraised the quality of psychometric studies, and therefore, we were not able to recommend which measures are better than others. Further, given that the measures included in this review vary in their content, purposes and quality (measurement properties), more advanced research, such as systematic reviews is needed to locate evidence-based measures for use. Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) [187] has been developed to serve this purpose and could be adapted for use in the comparative evaluation of mental health literacy measures.

Further, our review did not identify any measures addressing knowledge of positive/good mental health. Future measures should investigate knowledge on how to obtain and maintain good health as this now is recognized as an important component of mental health literacy.

Our findings indicate that the diagnostic vignette approach is widely used as a measure of mental health knowledge. However, a recent study in which diagnostic vignettes were compared against non-diagnostic vignettes showed an inability of participants to discriminate across “normal” and “ill” categories [188]. Further study to establish the validity of the diagnostic vignette evaluation approach as a measure of mental health knowledge is needed.

The myths and facts approach to measure knowledge has covered a wide range of aspects of mental health. However, we are unable to determine if there are different and developmentally appropriate knowledge components addressed at different points of the life-span among the current available measures.

The plethora of stigma measures, developed from numerous different ideological models (e.g., labeling theory [189]; attribution framework model [61]; cognitive behavioral model [190]; and social stigma model [191], has made evaluation of their validity in addressing stigma/attitudes challenging. The challenge has been to both validate each of the specific models and to determine which model may provide a better explanatory prediction for stigma or attitudes in different groups of people.

Further, only a few measures have targeted people’s emotional responses (n = 8) towards mental illness. This is an important area because stigma is associated with self-experience of unpleasant feelings about mental illness and this may influence how people interact with those with mental illness [9]. Only very recently has research measured the stigma experience of people with mental illness (n = 28 studies). This may provide a more comprehensive picture of how society treats people with mental illness. This may help to provide more concrete and useful information on how stigma interventions should be developed and delivered at both individual and community level.

Despite the challenges discussed above, this review has mapped out how stigma measures were developed and what they intended to measure, and this information may provide researchers and practitioners some guidance on which path to take either in designing their measures, or applying/ adapting existing measures, or developing related interventions or programs in the future.

Help-seeking behaviors are challenging to measure as they are influenced by many factors, such as knowledge about the behaviors, attitudes and beliefs towards the behaviors, social norms, and intentions to perform the given behavior [192]. Most help-seeking measures in this review have focused on attitudes towards help-seeking/treatment (n = 20) and intentions to seek help (n = 11), and very few measures (n = 4) directly measured actual help-seeking behaviors. Further, all 4 help-seeking behavior measures had no psychometric validation.

As Ajzen and Fishbein [191] pointed out, behaviors also may be influenced by self-expressed behavioral control which requires a person to have the skills, capacities, resources, and other important capacities needed to perform the behavior. However, we have not identified any measures to address these factors except for one tool measuring help-seeking efficacy (e.g. knowledge about where and how to find help, and who to find help from) [178].

Limitations

We did not conduct a systematic review of the literature on available mental health literacy measures and therefore we are unable to come to conclusions about the quality of the studies applying the measures. We excluded non-English studies (n = 21 at the title and abstract screening stage) and may have missed important measures in other languages. We did not check the grey literature that includes non peer-reviewed publication or documents/reports produced on all levels of governments and academics, and therefore may have missed some eligible studies. We may also have mistakenly excluded some measures at the first screening stage of reviewing titles and abstracts where measures were not mentioned.

Additionally, although we tried to categorize and interpret measures within the category we attributed them to, some measures may contain items relevant to other categories, however we were unable to distinguish them with available information we have.

Conclusions

Our review provides a compendium of available mental health literacy measurement measures for researchers and practitioners who are interested in applying existing measures or developing new measures that of particular relevance to their work. Because of how we selected eligible studies, our review further automatically forms a comprehensive dataset of current mental health literacy interventions for stakeholders to consider for their use. This review also identifies the many gaps in the field, such as the unbalanced application of knowledge and help-seeking evaluation measures compared to the stigma/attitudes measures, the yet-to-be validated measures in each outcome category, and the lack of measures that measure all components of mental health literacy concurrently. This gap identification could potentially guide future research work in the field. Further, we have conducted a thorough summary and synthesis of the psychometrics properties of included measures, and clarified the need to further investigate the quality of the psychometrics studies. At this stage, most of the measures were created without consultation with the intended participants such as students, teachers, patients or health providers. Future work should focus on joint collaboration across disciplines, between investigators and stakeholders and across more varied demographic and geographic groups.