Construct validity and internal consistency of the revised Mental Health Literacy Scale in South African and Zambian contexts.

AIM: The aim of this study was to evaluate the construct validity and internal consistency of the revised Mental Health Literacy Scale (MHLS) in South Africa (SA) and Zambia.
DESIGN: This cross-sectional study was conducted between October 2018 and December 2019.
METHODS: The study population comprised PHC workers (n = 454) in five districts in SA and Zambia. Principal component analysis (PCA) was used to explore the construct validity, and Cronbach's alpha was applied to measure the internal consistency of the MHLS.
RESULTS: Cronbach's alpha values for three attributes were below the appropriate level, but the value was strong (0.804) for the whole scale. The study found nine components explaining ~59% of the total variance of variables. All MHLS items loaded to main attributes based on the theory of MHL. The results stated that the revised version of the MHLS is a construct valid instrument with strong internal consistency.

INTRODUCTION

Many areas in sub‐Saharan Africa are progressively challenged by severe socioeconomic factors such as poverty, health inequality and low education, which characterize the high burden of mental health disorders (Hanlon et al., 2014; Patel, 2007; Rathod et al., 2017; Stein et al., 2008; World Health Organization, 2017). Mental health promotion has gained attention on the global agenda for Sustainable Development Goals (Target 3.4; United Nations, 2020), as mental health issues have taken the place as a leading cause of disability (GBD, 2017 Disease, & Injury Incidence & Prevalence Collaborators, 2018; WHO, 2017).

The concept of mental health literacy (MHL) may provide a key approach for the sustainable development of mental health. MHL, which originally was determined as “knowledge and beliefs about mental disorders which aid their recognition, management or prevention” (Jorm et al.,1997), accompanies health literacy, which is significant determinant of individuals’ economic prosperity (Kutcher, Wei, Gilberds, et al., 2016; World Health Organization, 2013). Lack of health literacy relates to worse health status and greater needs for health services (Sørensen et al., 2015). Likewise, poor MHL, as public knowledge and understanding of factors of mental disorders, has been one of the major challenges in developing mental health services (Atilola, 2016; Jorm, 2000). Nevertheless, previous studies have stated that more relevant, context‐related and empirically tested research measuring MHL should be conducted (Brooks et al., 2011; Jack et al., 2014; Kutcher et al., 2016).

BACKGROUND

In sub‐Saharan Africa, primary healthcare (PHC) workers play a significant role in delivering and promoting mental health care in their respective societies (Atilola, 2016; Munakampe, 2020; Mwape et al., 2012), but they lack knowledge on mental health‐related issues and hold negative stereotypes towards mentally sick persons (Kapungwe et al., 2011; Mwape et al., 2010). These negative attitudes, knowledge, beliefs and recognition of mental health issues (Jorm et al., 1997; O’Connor et al., 2014) may also reflect PHC workers’ poor MHL (Atilola, 2016). This highlights an obvious need for training of PHC workers of low‐ and middle‐income countries (LMICs) on MHL (Alburquerque‐Sendín et al. 2018; Ganasen et al., 2008; Kapungwe et al., 2011; Kutcher et al., 2017, 2019). Training potentially strengthens health professionals’ skills related to MHL and improves the quality of care in LMICs (Atilola, 2016; Kutcher et al., 2017).

PHC workers’ outcomes of training on the concept of MHL cannot be verified without psychometrically tested, valid research tools (Kutcher, Wei, & Coniglio, 2016). Traditionally MHL researchers have faced difficulties in assessing MHL as a whole concept in terms of recognition, knowledge and attitudes towards mental health; rather, they measured parts of it and its different core components of knowledge and beliefs about mental health with separate vignettes (Jorm et al., 1997; O’Connor & Casey, 2015; O’Connor et al., 2014). Particularly psychometrically robust instruments measuring MHL constructively are lacking (O’Connor & Casey, 2015; O’Connor et al., 2014). Limited use of scale‐based, valid and reliable instruments in MHL‐related studies may also lead to greater limitations in study results (Kutcher, Wei, & Coniglio, 2016).

The aim of this study was to clarify the psychometrics of the Mental Health Literacy Scale (MHLS) (O’Connor & Casey, 2015) in South Africa (SA) and Zambia. Differing from earlier MHL measures, the MHLS has been acknowledged as a psychometrically and conceptually reliable instrument with excellent psychometric properties regarding internal consistency, content and structural validity (O’Connor & Casey, 2015; Wei et al., 2016), and internal and test–retest reliability (O’Connor & Casey, 2015). The MHLS, which includes all attributes of MHL, was recently revised and content‐validated in SA and Zambia using a heterogeneous expert panel with PHC workers and workers and professional research experts (Korhonen et al., 2019). The study revealed the MHLS to have sufficient content validity also in LMICs’ context. Only minor changes were made to the original MHLS by O’Connor and Casey (2015). The MHLS instrument requires further validation to be used in this new context to explore MHL of PHC workers from a cultural perspective and understand their training needs (Atilola, 2016; Rathod et al., 2017) in SA and Zambia. In this study, we evaluate the psychometric properties (DeVon et al., 2007) of the content‐validated version of the MHLS in SA and Zambia. The research questions are as follows: (1) What is the construct validity of the MHLS in SA and Zambia? (2) What is the internal consistency of the MHLS in SA and Zambia?

THE STUDY

Design

This cross‐sectional study (Polit & Beck, 2018) was conducted between October 2018 and December 2019. The study is part of a larger European Union‐funded project, “MEGA–Building capacity by implementing mhGAP mobile intervention in SADC countries” (funding number 585827‐EPP‐1‐2017‐1‐FI‐EPPKA2‐CBHE‐JP). In the MEGA project, PHC workers are trained for screening youths’ and adolescents’ mental health problems using a new mobile application (Lahti et al., 2019).

Method

Consensus‐Based Standards for the Selection of Health Measurement Instruments

The Consensus‐Based Standards for the Selection of Health Measurement Instruments (COSMIN) Study Design checklist 2019 version (Mokkink et al., 2018; Prinsen et al., 2018; Terwee et al., 2018) was used to guide the methodological quality of the study. The COSMIN follows “the worst score counts” principle so that only the lowest rating on each domain is reported. In this study, four out of ten domains of the checklist were applicable in terms of construct validity and internal consistency. Three out of ten domains (content validity, structural validity and internal consistency) were determined to be “very good.” The fourth domain, “general recommendation for the design of a study on measurement properties,” was rated as adequate. The evaluation of content validation was based on our previous findings (Korhonen et al., 2019). Previously, six domains of the COSMIN (internal consistency, reliability, measurement error, content validity, structural validity and hypotheses testing) were reported and adequately assessed during the original development of the MHLS (O’Connor & Casey, 2015).

Participants

PHC workers for this study were selected and invited to participate from the MEGA project research field in the five following regions: Free State, Gauteng and Western Cape Provinces of SA, and Lusaka and Central Provinces in Lusaka, Zambia. The aim was to obtain a culturally diverse sample to identify construct validity and reliable of the revised MHLS in sub‐Saharan Africa. Following the MEGA project protocol (Lahti et al., 2019), participating PHC workers who meet and screen youths and adolescents for mental health issues in their daily clinical practice were targeted. Inclusion criteria were as follows: (a) registered or enrolled nurses or clinical officers working in PHC in the three provinces of SA and two provinces in Lusaka, Zambia, who (b) were able to speak and read English. PHC practitioners or clinical officers who were retiring during the course of the project (2017–2020) were excluded from the study.

Data collection and procedure

Before the data collection, local researchers from the MEGA project research team were trained for the data collection in MEGA project partner meetings. MEGA project researchers approached the study participants, contacting them in three SA and two Zambian districts and a total of 45 clinics between October 2018 and December 2019. The researchers reached the participants, PHC workers, in their working fields during hectic workdays and clinics’ open hours. Supervision by the primary investigators (PIs) from Finland was provided during the whole procedure online and in face‐to‐face project partner meetings in Africa and Europe.

At the beginning of data collection, adequate and comprehensive oral and written information with full disclosure was given to PHC workers in the local clinics to obtain informed consent (Polit & Beck, 2018; World Medical Association, 2013). Participants had the power to voluntarily consent to or refuse participation. Participants’ rights were stated and discussed before the data collection. After the informed consent procedure was carried out, the participants were asked to fill in the revised version of the MHLS with a background questionnaire. Collected data were personally handed to the PIs as original copies and in SPSS data form. In one case, the original copies from the participating university were not able to be handed out to the PIs due to COVID‐19 travel restrictions.

Mental Health Literacy Scale instrument

Originally, the MHLS (O’Connor & Casey, 2015) consisted of all attributes of the concept of MHL suggested by Jorm et al. (1997; see Table 1). Theoretically, the concept is divided into six attributes of recognition, knowledge and attitudes relating to mental health. The MHLS consists of 35 items. The total score of the MHLS is produced by summing all items together (minimum score 35, maximum score 160). Questions 1–15 are answered on a 4‐point scale ranging from 1 (very unlikely/unhelpful) to 4 (very likely/helpful), and Questions 16–35 are answered on a 5‐point scale ranging from 1 (strongly disagree/definitely unwilling) to 5 (strongly agree/definitely willing). The instrument also includes the following reverse‐scored items: Q10, Q12, Q15 and Q20–Q28. No cut‐off points have yet been introduced for the appropriate level of scoring.

TABLE 1

Theoretical framework of Mental Health Literacy Scale by items (Jorm, 2000; O’Connor & Casey, 2015; O’Connor et al., 2014)

	Main theme	Attribute	Item
Mental Health Literacy Scale (MHLS)	Recognition	Ability to recognise specific disorders	Q1–Q8
	Knowledge	Knowledge of risk factors and causes of mental illness	Q9–Q10
		Knowledge of self‐treatment	Q11–Q12
		Knowledge of professional help available	Q13–Q15
		Knowledge of how to seek information	Q16–Q19
	Attitudes	Attitudes that promote recognition or appropriate help‐seeking behaviour	Q20–Q35

Content validation of the Mental Health Literacy Scale in South African and Zambian contexts

Previously, a content validation of the MHLS was conducted using a heterogeneous expert panel method involving professional research experts and clinical experts (PHC workers) in SA and Zambia (Korhonen et al., 2019). The study showed the MHLS to have appropriate content validity in the African context, with only minor changes made to 11 items (Q1, Q5–Q7, Q9–Q10, Q14–15, Q20, Q24–Q25) for better cultural clarity. In addition, for given the changes in the DSM‐5, two items (Q5 and Q8) were updated by suggestion of original authors of MHLS instrument. None of the 35 items of the original MHLS was deleted (Korhonen et al., 2019).

Analysis

The data were analysed using statistical analyses with SPSS26.0. The characteristics of the sample were reported using descriptive statistics. Principal component analysis (PCA; Grove et al., 2013; Maćkiewicz & Ratajczak, 1993; Mishra et al., 2017) was used to explore the construct validity, and Cronbach's alpha (Polit & Beck, 2018; Waltz, Strickland, & Lenz, 2016) was applied to measure the internal consistency reliability of the revised version of the MHLS in SA and Zambia. Methodologically, construct validity indicates the instrument's degree and capability of measuring the formulated constructs in relation to its theoretical background and concepts (DeVon et al., 2007; Polit & Beck, 2018). Using PCA enabled extracting the MHLS variables into smaller groups of components and analysing their maximum variance. The PCA method was chosen because it is particularly suitable for development of a new instrument describing attitudes, beliefs, values and opinions (Grove et al., 2013). The component structure of the MHLS instrument was analysed by PCA with the varimax rotation method using the eigenvalue criterion. Using varimax rotation ensures components to be uncorrelated. Results were considered significant if p ≤ .05. Factor loadings ≥0.30 were accepted (Waltz et al., 2016).

Sum variables based on the theoretical background of the MHLS were formed. These were obtained by adding up the coded answers. The reliability of sum variables was checked by calculating Cronbach's alpha coefficients (≥0.70; DeVon et al., 2007; Gray et al., 2017; Tavakol & Dennick, 2011) by examining through item analysis the compatibility of single questions within the scale. Alphas were also examined for every attribute of the MHL concept to avoid any inflation in interpreting alpha values and respecting the so‐called “tau‐equivalent model” hypothesizing that items measure the same latent trait on the MHLS (Tavakol & Dennick, 2011; Waltz et al., 2016). Missing data were handled by using only complete cases in computing PCA and Cronbach's alpha values. Power analysis was performed to assess sufficient sample size for PCA.

Ethics

This study was conducted according to the basic principles and codes of research ethics. Human dignity, confidentiality, justice and beneficence were strictly respected in every phase, following the relevant legislation in terms of research ethics (American Psychological Association 2017; International Council of Nursing, 2012; Polit & Beck, 2018; World Medical Association, 2013). All necessary research permissions for the study were obtained by MEGA project partner universities from the national health research authorities in the participating regions of South Africa and Zambia between June 2018 and April 2019. Partner institutions signed a data‐sharing agreement and ensured continuous monitoring of data and safety of all participants in the study. Quantitative data gathered at the respective sites were manually transferred to Turku University of Applied Sciences (TUAS) in Finland for secure storage according to the guidelines of the TUAS archives protocol.

RESULTS

Characteristics of participants included in the final study

A convenience sample of PHC practitioners (n = 505) was recruited by the MEGA project researchers, of which n = 454 were included in the final study. After cleaning the data, n = 343 complete answers were obtained with the MHL survey. The majority of these participants were Zambian (53%, n = 181) and female (74%, n = 251). Diploma level was presented regarding the background of professional education in 66% (n = 219) of cases. The most participants were ≤40 years old, the largest age group being 30 years old or younger. The majority of participants (84%, n = 287) lacked continuous professional development (CPD)/training activity or course on mental health issues. A full demography of included participants is presented in Table 2.

TABLE 2

Demography of participants

Country	N	n
South Africa	162
Gauteng Province		28
Free State		73
Western Cape		62
Zambia	181
Lusaka Province		178
Central Province		2
Total	343
Sex
Female	251
Male	89
Total	340
Missing	3
Age by group
≤30	130
31–40	70
41–50	90
≥51	48
Total	338
Missing	5
Level of professional education
Certificate	46
Diploma	219
Degree (BA, MA, PhD)	62
Other	5
Total	332
Missing	11
CPD/Training activity
Yes	53
No	287
Total	340
Missing	3

Results for construct validity and principal component analysis

Nine principal components (PCs) were found to explain approximately 59% of the total variance of the data. The measure of sampling adequacy (Kaiser–Meyer–Olkin test) received a value of 0.807 with Barlett's test of sphericity (χ2 = 3861.245; df = 595; sig = 0.000), which indicated that the PCA was suitable for exploring construct validity. All 35 items of the revised version of the MHLS loaded on the PCs (PC1–PC9). Eleven of the 35 items of the revised MHLS had loadings (valued ≥0.30) on two different PCs and one item on three PCs. However, every item was only grouped on the PC divisions driven by its highest loading value, ranging from 0.446 to 0.832. Communalities of items ranged from 0.416 to 0.768. A scree plot of PCA is presented in Figure 1.

FIGURE 1

Scree plot of principal component analysis

Loadings of PCs aligned with the theoretical background of the MHLS (Table 1). The formulated PCs were named according to the three main attributes of MHL in relation to the recognition, knowledge and attitudes towards mental health. The “Recognition” in relation to the ability to recognise specific disorders loaded on two PCs, (PC2, PC7). The “Attitudes,” for attitudes that promote recognition or appropriate help seeking behaviour (stigma), loaded on four PCs (PC1, PC3, PC5, PC9). Likewise, based on the theory, three different knowledge‐related components (PC4, PC6, PC8) were named as “Knowledge,” in association to knowledge of how to seek information, knowledge of risk factors and causes of mental illness, and the knowledge of professional help available.” As an exemption, seventh component (PC7) included knowledge‐related question (Q11) but were named as “Recognition” by its strongest item in terms of communality. The three components (PC3, PC5 and PC6) consisted mainly of reversed scored items. All the formulated and named PCs related to attributes of the revised version of the MHLS with communalities and cumulative percentages are presented in Table 3.

TABLE 3

Principal components and factor loadings of revised version of the MHLS (Korhonen et al., 2019; O’Connor & Casey, 2015)

Original Item of MHLS	Attributes of Principal Component (PC)	PC 1	PC 2	PC 3	PC 4	PC 5	PC 6	PC 7	PC 8	PC 9	Communalities
31. How willing would you be to make friends with someone with a mental illness?	“ATTITUDES” “Attitudes that promote recognition or appropriate help seeking behaviour (stigma)” Q20 – Q35	0.832									0.768
30. How willing would you be to spend an evening socialising with someone with a mental illness?	0.826									0.713
32. How willing would you be to have someone with a mental illness start working closely with you on a job?	0.817									0.714
29. How willing would you be to move next door to someone with a mental illness?	0.694									0.599
33. How willing would you be to have someone with a mental illness marry into your family?	0.694									0.685
35. How willing would you be to employ someone if you knew they had a mental illness?	0.629									0.695
5. To what extent do you think it is likely that Persistent Depressive Disorder (Dysthymia) is a mental disorder	“RECOGNITION” “Ability to recognise disorders” Q1 ‐ Q8		0.751								0.608
7. To what extent do you think it is likely that the diagnosis of Bipolar Disorder includes experiencing periods of extremely elevated (i.e. high) and periods of depressed (i.e. low) mood		0.728								0.611
3. If someone experienced a low mood for two or more weeks, had a loss of pleasure or interest in their normal activities and experienced changes in their appetite and sleep then to what extent do you think it is likely they have Major Depressive Disorder		0.695								0.543
4. To what extent do you think it is likely that Personality Disorders are a category of mental illness		0.687								0.551
8. To what extent do you think it is likely that the diagnosis of Substance Abuse Disorder can include physical and psychological tolerance of the drug (i.e. require more of the drug to get the same effect)		0.627								0.437
6. To what extent do you think it is likely that the diagnosis of Agoraphobia includes anxiety about situations (e.g. open market place) where escape may be difficult or embarrassing		0.591								0.452
1. If someone became extremely nervous or anxious in one or more situations with other people (e.g. in social gatherings) or performance situations (e.g. presenting at a meeting) in which they were afraid of being evaluated by others and that they would act in a way that was humiliating or feel embarrassed, then to what extent do you think it is likely they have Social Phobia		0.446								0.476
27. If I had a mental illness, I would not seek help from a mental health professional __REVERSE SCORED	“ATTITUDES” “Attitudes that promote recognition or appropriate help seeking behaviour (stigma)” Q20 – Q35			0.770							0.646
28. I believe treatment for a mental illness, provided by a mental health professional, would not be effective__REVERSE SCORED			0.759							0.669
26. Seeing a mental health professional means you are not strong enough to manage your own difficulties__REVERSE SCORED			0.713							0.563
25. If I had a mental illness, I would tell no one. __REVERSE SCORED			0.680							0.554
24. It is best to avoid people with a mental illness so that you don't catch their illness __REVERSE SCORED			0.606							0.542
19. I am confident I have access to resources (e.g. GP, internet, friends) that I can use to seek information about mental illness	“KNOWLEDGE” “Knowledge of how to seek information” Q16‐Q19				0.760						0.656
17. I am confident using the computer or telephone to seek information about mental illness				0.747						0.622
16. I am confident that I know where to seek information about mental illness				0.741						0.622
18. I am confident attending face to face appointments to seek information about mental illness (e.g. seeing the GP)				0.718						0.587
21. A mental illness is a sign of personal weakness __REVERSE SCORED	“ATTITUDES” “Attitudes that promote recognition or appropriate help seeking behaviour (stigma)” Q20 – Q35					0.651					0.637
23. People with a mental illness are dangerous __REVERSE SCORED					0.648					0.479
22. A mental illness is not a real medical illness __REVERSE SCORED					0.538					0.612
9. To what extent do you think it is likely that in general, women are MORE likely to experience some mental illnesses compared to men	“KNOWLEDGE” “Knowledge of risk factors and causes of mental illness” Q9, Q10 “Knowledge of self‐treatment” Q11, Q12 “Knowledge of professional help available” Q13 – Q15						−0.598				0.504
15. Mental health professionals are bound by confidentiality; however there are certain conditions under which this does not apply. To what extent do you think it is likely that the following is a condition that would allow a mental health professional to break confidentiality: If patient's problem is not life‐threatening and professionals want to assist others to better support a patient__REVERSE SCORED						0.576				0.571
12. To what extent do you think it would be helpful for someone to avoid all activities or situations that made them feel anxious if they were having difficulties managing their emotions__REVERSE SCORED						0.565				0.431
10. To what extent do you think it is likely that in general, men are MORE likely to experience an anxiety disorder compared to women__REVERSE SCORED						0.463				0.416
2. If someone experienced excessive worry about a number of events or activities where this level of concern was not warranted, had difficulty controlling this worry and had physical symptoms such as having tense muscles and feeling fatigued then to what extent do you think it is likely they have Generalised Anxiety Disorder	“RECOGNITION” “Ability to recognise disorders” Q1 ‐ Q8 “Knowledge of self‐treatment” Q11, Q12							0.651			0.596
11. To what extent do you think it would be helpful for someone to improve their quality of sleep if they were having difficulties managing their emotions (e.g. becoming very anxious or depressed)							0.643			0.531
14. Mental health professionals are bound by confidentiality; however there are certain conditions under which this does not apply. To what extent do you think it is likely that the following is a condition that would allow a mental health professional to break confidentiality: If a patient is at immediate risk of harm to oneself or others	“KNOWLEDGE” “Knowledge of professional help available” Q13 – Q15								0.776		0.695
13. To what extent do you think it is likely that Cognitive Behaviour Therapy (CBT) is a therapy based on challenging negative thoughts and increasing helpful behaviours									0.582		0.559
20. People with a mental illness could put themselves together if they wanted__REVERSE SCORED	“ATTITUDES” “Attitudes that promote recognition or appropriate help seeking behaviour (stigma)” Q20 – Q35									−0.575	0.538
34. How willing would you be to vote for a politician if you knew they had suffered a mental illness?										0.558	0.710
Total Variance Explained (Rotate PCA, Cumulative %)		11.106	21.411	29.934	37.003	42.134	46.694	50.946	54.961	58.829	Total 59%

Results for internal consistency

Internal consistency reliability was explored by determining Cronbach's alpha for the whole scale and six attributes (Table 4). The number of complete responses varied between the different MHLS attributes, with the whole scale receiving 343 valid responses from the total of 454 participants. Three of the attributes, “Ability to recognize disorders” (Q1–Q8), “Knowledge of how to seek information” (Q16–Q19) and “Attitudes that promote recognition or appropriate help seeking behaviour (stigma)” (Q20–Q35)–met the appropriate level for the alpha coefficient (≥0.70). These questions represented seven out of nine previously formulated PCs. Three of six attributes, representing the majority of knowledge‐related questions (Q9–Q15), fell below the appropriate Cronbach's alpha level. However, Cronbach's alpha for the whole scale was 0.804, which can be seen as a strong indicator of internal consistency reliability. All coefficient correlations for the different attributes and the whole scale are presented in Table 4.

TABLE 4

Internal consistency of variables

Attribute/Principal component (items)	n/N valid(total)	n of items	Cronbach's alpha
The ability to recognise disorders PCs 2,7 (Q1–Q8)	407 (454)	8	0.833
Knowledge of how to seek information PC4 (Q16–Q19)	418 (454)	4	0.793
Knowledge of risk factors and causes of mental illness) PC6 (Q9, Q10)	438 (454)	2	−0.638
Knowledge of self‐treatment divided between PCs 6,7 (Q11–Q12)	434 (454)	2	−0.825
Knowledge of professional help available PC8 (Q13–Q15)	432 (454)	3	0.039
Attitudes that promote recognition or appropriate help‐seeking behaviour (stigma) PCs 1,3,5 (Q20–Q35)	393 (454)	16	0.788
Total items of MHLS	343 (454)	35	0.804

DISCUSSION

This study explored the construct validity and internal consistency reliability of the revised version of the MHLS in SA and Zambia. The MHLS has shown excellent validity overall in previous literature (O’Connor & Casey, 2015; Wei et al., 2016) and sufficient content validity in low‐ and middle‐income contexts (Korhonen et al., 2019). The revised MHLS may provide more information for researchers about PHC workers’ MHL in SA and Zambia.

Our study findings indicate that the revised version of the MHLS (Korhonen et al., 2019) has a good construct validity for measuring MHL among PHC workers in SA and Zambia. The study found nine components explaining approximately 59% of the total variance of data, and all items loaded according to the theoretical base of the MHL concept. The structure of the nine components was mostly in line with three main themes and six theoretical attributes. Only two items of one component (PC7) derived from two different main attributes of MHL, “Recognition” as ability to recognise disorders and “Knowledge” of self‐treatment. These items had somewhat equal, but strong factor loadings. Hence, PCs were able to explain the variance of all items well. Thus, the MHLS has sufficient capability to measure the formulated constructs regarding the MHL concept (DeVon et al., 2007; Polit & Beck, 2018).

Cronbach's alpha, supporting evidence of internal consistency, for the whole scale was a strong 0.804, considering that the original MHLS by O’Connor and Case y (2015) and our revised version (Korhonen et al., 2019) are somewhat new instruments. Previous literature states that internal consistency for a new scale (≤5 years) can range between 0.6 and 0.69. Moreover, the MHLS consists of 35 items, and scales with ≥20 items usually score better in terms of internal consistency (Gray et al., 2017). Alpha's adequacy as an indicator of internal consistency reliability has been critically discussed by several authors (Sijtsma, 2009; Trizano‐Hermosilla & Alvarado, 2016; Waltz et al., 2016), and the value itself cannot been seen as a symbol of hetero‐ or homogeneity of test items (Sijitsma, 2009; Tavakol & Dennick, 2011). Our findings indicate that all items of the MHLS fit together in terms of internal consistency (DeVon et al., 2007) and together with proper factor loadings support the instrument's construct validity (Tavakol & Dennick, 2011).

It should be noted that the MHLS is intended to measure MHL as a single concept but consists of different attributes and subscales, reflecting recognition, knowledge and attitudes towards mental health issues. In this study, three knowledge‐related attributes of the main MHL concept fell below the acceptable level of coefficient correlation (≥0.70). The instrument includes multiple rating scales, some of them with reverse scoring built into them. This was also seen in the theoretical structure and PCA as the clear majority of the three components consisted of reverse‐scored items. Diverse items with a constantly changing rating scale and reversed scoring were particularly used for testing PHC workers’ knowledge and attitudes on mental health issues, but this was found to be a challenge for Cronbach's alpha regarding internal consistency. This may be because PHC workers were unfocused on the study due to the challenge caused by their hectic working environment, where the study was conducted. It is also well reported that heterogeneous groups of items can give lower alpha values for this kind of complex scale (Gray et al., 2017; Waltz et al., 2016), as Cronbach's alpha is traditionally applied to examine items indicating the same latent trait on the scale (tau equivalency), and diverse items may violate that assumption (Tavakol & Dennick, 2011). Nevertheless, our findings indicate that for better reliability, avoiding use of knowledge “traps” such as reverse scoring and multiple scales in development of instruments measuring multidimensional concepts may prevent inflation of alpha values. Moreover, our findings strengthen previous findings and suggestions to determine and report alpha values for different sub‐concepts separately (Gray et al., 2017; Tavakol & Dennick, 2011).

Limitations

This study has methodological strengths and limitations, which should be considered when interpreting the results. First, the psychometrical validity and reliability of the revised MHLS are grounded on the development of the original instrument by O’Connor and Casey (2015). This supports the validation and implementation process of the revised version of the MHLS in SA and Zambia. Second, no “gold standard” has yet been introduced for instruments studying the concept of MHL. Thus, comparison with other instruments relating to other measurement properties as cross‐cultural and criterion validity is limited, and the results may vary with various study populations. Third, the recruitment process of study participants was challenging due the work pressure of PHC workers. The surrounding hectic study environment in the clinics combined with time‐consuming effort for participation might have negatively affected participants’ precision in their study response. Finally, our findings with an adequate sample size and minor changes to the original MHLS may support the use of the revised version in multiple study environments and cultural contexts.

CONCLUSION

Even including multiple attributes, the revised version of the MHLS has been shown to be a construct valid and internally consistent scale when measuring MHL as a larger concept. Considering the mentioned methodological limitations, our findings reveal that the revised version of the MHLS is a convenient instrument for studying African PHC workers’ recognition, knowledge and attitudes related to mental health issues. By understanding MHL among PHC workers, context‐ and cultural‐specific, quality education and training can be provided for planning and implementation of proper care for people suffering from mental health disorders in sub‐Saharan Africa.

CONFLICT OF INTEREST

Authors have no conflicts of interest.

AUTHOR CONTRIBUTIONS

J Korhonen (JK) conceived the study and the manuscript. ML and JK were responsible for the study design. MEGA consortium, JK and ML were responsible for collecting the data. ML and AA contributed to the manuscript by commenting and modifying. J Katajisto led the statistical analysis of data and drafted the manuscript. ML and AA supervised the data analysis and interpretation. The MEGA research team provided overall guidance. All authors read and approved the final manuscript.