The Impact of the Chronic Disease Self-Management Program on Health Literacy: A Pre-Post Study Using a Multi-Dimensional Health Literacy Instrument.

This study assessed the impact of the Chronic Disease Self-Management Program (CDSMP) on different domains of health literacy using a pre-post study design. Participants aged over 16 years and with one or more self-reported chronic diseases were recruited for the CDSMP in western Sydney (a highly diverse area of New South Wales, Australia) between October 2014 and September 2018. Health literacy was assessed pre- and immediately post-intervention using the Health Literacy Questionnaire (HLQ), with differences in mean scores for each HLQ domain analysed using paired sample t-tests. A total of 486 participants were recruited into the CDSMP. Of those, 316 (65.0%) completed both pre- and post-intervention surveys and were included in the analysis. The median age of the participants was 68 years, the majority were female (62.5%), and most were born in a country other than Australia (80.6%). There were statistically significant (p < 0.001) improvements across all nine domains of the HLQ. This is the first study evaluating the potential impact of the CDSMP on improving different domains of health literacy amongst a diverse sample of participants with chronic diseases using a multi-dimensional instrument. The absence of a control population in this study warrants caution when interpreting the results.

1. Introduction

Non-communicable (chronic) disease is a global health problem. Chronic diseases (e.g., cardiovascular diseases, cancer, diabetes, chronic respiratory diseases) are the leading cause of disease burden and mortality worldwide, with the attributable share of total global burden of disease increasing steadily from 44% in 1990 to 61% in 2016 [1]. In Australia, chronic disease accounts for 84–88% of the total disease burden [2].

In the absence of a cure, chronic disease must be managed over time as it evolves with shifting severity, pace, and treatments [3]. Holman has written eloquently about the features of chronic illnesses, emphasising their dynamic and evolving nature and variations in severity over time, which require competent, often daily decision making and management by patients [3]. Given this, developing patients’ self-management skills and confidence has become an increasingly important part of clinical care, and a number of interventions aimed at delivering self-management support and education have now been developed and evaluated [4,5].

Most chronic disease self-management programs have the educational goal of providing individuals with a range of transferable health-related skills. They recognise that by developing these adaptable skills—for example in symptom recognition, treatment management, and health services use—individuals with established chronic diseases are much better equipped to partner in the management of their disease and respond to the dynamic and evolving nature of their chronic condition. Knowledge improvement is seen as necessary but not sufficient to achieve sustained disease self-management. However, outcomes reported in evaluations of chronic disease self-management programs do not always reflect the skills-directed methods and learning theories that underpin them. Although there is heterogeneity in the outcomes measured in trials of intervention efficacy, a systematic review identified that the most frequently assessed outcomes of chronic disease self-management programs were improved knowledge and self-efficacy, healthcare utilisation and clinical outcomes (e.g., anxiety, depression, energy/fatigue, general health, pain) [6].

Health literacy refers to the cognitive and social skills required to gain access to, understand and use information in ways which promote and maintain good health [7]. Although health literacy has been recognised as an important foundation for successful and sustained self-management [8,9,10], few studies have explored the direct impact of chronic disease self-management programs on health literacy skills. In 2013, a pre-post longitudinal evaluation of the Chronic Disease Self-Management Program among middle-aged and older adults was conducted across 17 states in the United States. This study reported that the program led to a statistically significant improvement in health literacy, quantified in this study using a limited measure assessing participants’ confidence filling out medical forms [11].

Health literacy is better understood as a multidimensional construct, moving from a narrow conceptual focus on patients’ skills in reading and writing to also include knowing when and where to seek health information, verbal communication and assertiveness, and skills to retain, process and apply health information in changing circumstances [12]. As understanding and definitions of health literacy have evolved, measurement instruments have been developed to better capture the full breadth of ideas embodied by the concept [13,14]. This has led to the development of more comprehensive measures of health literacy, including the Health Literacy Questionnaire (HLQ) [13] and European Health Literacy Survey [14]. These instruments assess participants’ self-reported capacity to actively manage their health, engage with healthcare providers, navigate the healthcare system and find good health information, among other capabilities.

The health literacy ‘domains’ captured by these more comprehensive instruments can be matched to the content of self-management interventions which teach participants how to actively manage their health, provide participants with health information and include topics related to communication skills, making decisions and working with health professionals and organisations. The evolution of these more comprehensive measures of health literacy present an opportunity to examine the impact of existing chronic disease management programs on transferable health skills necessary for adaptive and sustainable self-management.

This study aimed to examine the potential impact of the Chronic Disease Self-Management Program (CDSMP) on different domains of health literacy using a pre-post study design.

2. Materials and Methods

2.1. Setting

Western Sydney is a highly diverse area of Sydney, New South Wales (NSW) in Australia. The Western Sydney Local Health District (WSLHD) is one of 15 local health districts (LHDs) in the NSW health system and provides public healthcare to more than one million residents across four Local Government Areas in the west of Sydney. WSLHD has the highest urban indigenous population in Australia, 47% of residents are overseas-born, and one in two speak a language other than English at home [15]. Since 2013, WSLHD has hosted a demonstration program to improve the integration of care between primary and tertiary health systems. One important part of this program was to better support the large group in the population who were living with (often complex) chronic disease, and who were frequent users of the healthcare system. This included making the established CDSMP (described below) available for eligible participants across the health district [16].

2.2. Intervention

The CDSMP was developed at Stanford University and has been broadly disseminated across populations and several countries [17,18,19,20,21,22]. The CDSMP comprises structured small-group interventions (2.5 hours each) over six weeks and an accompanying reference book. The program includes aspects encountered throughout the chronic illness trajectory (e.g., fatigue, medication/symptom management, decision-making, communication with providers, and behavioural changes related to nutrition and exercise) [23], and seeks to build self-efficacy for self-management through the mastery of skills (action planning and feedback), symptom reinterpretation, modelling of self-management behaviours, problem-solving strategies, and social persuasion [24] (see Figure 1). The CDSMP is peer-led by trained individuals with a personal experience living with a chronic condition [23].

Figure 1

Structure of 6-week Chronic Disease Self-Management Program.

The content of the CDSMP maps onto the domains of health literacy measured in the HLQ. It teaches participants, for example, how to actively manage their health (HLQ Domain 3) and provides them with information to do so (Domain 2; Domain 8; Domain 9). Topics related to communication skills, making decisions and working with health professionals and organisations may help participants to actively engage with healthcare providers (Domain 6) and feel supported and understood by them (Domain 1), as well as navigate the healthcare system more effectively (Domain 7). The small-group format of the program may also increase perceived social support for health (Domain 4).

The effectiveness of the CDSMP has been demonstrated in studies across age groups and diverse cultural and ethnic backgrounds [23], including improvements in self-efficacy [17,18,25,26,27], health behaviour (i.e., exercise frequency, symptom management, communication with healthcare providers) [27,28,29], health status (i.e., level of self-reported health, health distress, disability, fatigue, quality of life) [25,28,29,30,31], and health service utilisation (reduction in emergency presentations and hospitalisations) [24,29,31]. Qualitative evaluations suggest that CDSMP is acceptable to participants, and the group-based modality helps enhance participation, social connectedness, coping, and the acceptance of chronic condition(s) amongst participants [20,32,33,34].

In this study, the CDSMP program was delivered by trained bilingual and Indigenous community leaders recruited from multicultural health services, local non-government organisations and volunteer networks. To qualify, all leaders completed 24 h of training across four full-day sessions hosted by master trainers. A total of 27 leaders were trained between 2014 and 2016. Each CDSMP session was led by two trained leaders who followed a detailed Chronic Disease Self-Management Leader’s Manual and Fidelity Manual to deliver the program. Programs were delivered in several languages (e.g., Mandarin; Cantonese; Turkish; Tagalong; see Table 1) via ‘on-the-spot’ translation by the group leader. All participants were provided with Living a Healthy Life with Chronic Conditions reference text, either in English or simplified Chinese.

Table 1

Demographic characteristics of the participants (pre-intervention) who completed both pre- and post-intervention questionnaires (n = 316).

Characteristics	n (%)
Age (years)
<50	12 (3.8%)
50–59	39 (12.3%)
60–69	123 (38.9%)
70–79	104 (32.9%)
≥80	29 (9.1%)
No Response	9 (2.8%)
Gender
Female	196 (62.0%)
Male	115 (36.4%)
No Response	5 (1.6%)
Live Alone
Yes	56 (17.7%)
No	235 (74.4%)
No Response	25 (7.9 %)
Country of Birth
Australia	51 (16.1%)
Mainland China	78 (24.7%)
India	71 (22.5%)
Hong Kong	34 (10.8%)
Turkey	33 (10.4%)
Sri Lanka	25 (7.9%)
Other	19 (6.0 %)
No Response	5 (1.6%)
English Spoken at Home
Yes	88 (27.8%)
No	222 (70.3%)
No Response	6 (1.9%)
Education Level
Primary School or Less	32 (10.1%)
High School (not completed)	38 (12.0%)
High School (completed)	66 (20.9%)
TAFE/Trade	42 (13.3%)
Undergraduate	87 (27.5%)
Postgraduate	17 (5.4%)
No Response	34 (10.8%)
Employment
Working / Studying	14 (4.4%)
Home duties	39 (12.3%)
Retired	148 (46.8%)
Ill / Permanently unable to work	10 (3.2%)
Other	17 (5.4%)
No response	88 (27.8%)
Illness (participants could list more than one condition)
Arthritis	120 (38.0%)
Diabetes	94 (29.7%)
Back pain	92 (29.1%)
Heart problems	86 (27.2%)
Depression / Anxiety	68 (21.5%)
Other	160 (50.6%)
No response	28 (8.6%)
Private health insurance
Yes	108 (34.2%)
No	185 (58.5%)
No Response	23 (7.3%)
Have Healthcare Card a
Yes	230 (72.7%)
No	59 (18.7%)
No Response	27 (8.5%)
Attended Hospital Emergency in the Past 12 Months
Yes	65 (20.6%)
No	164 (51.9%)
No Response	87 (27.5%)
Required help completing Questionnaire
Yes	46 (14.6%)
No	247 (78.2%)
No Response	23 (7.3%)
Teaching Group
Arabic	6 (1.9%)
ATSI	26 (8.2%)
Australian and New Zealander	4 (1.3%)
Chinese (Cantonese)	57 (18.0%)
Chinese (Mandarin)	54 (17.1%)
Chinese (Traditional)	9 (2.8%)
English	22 (7.0%)
Filipino (Tagalong)	1 (0.3%)
Hindi	63 (19.9%)
Indian (English)	12 (3.8%)
Multicultural	4 (1.3%)
Tamil	25 (7.9%)
Turkish	33 (10.4%)

a. Health Care Cards are issued by the Australian Department of Human Services to people who live in Australia and already receive government payments, benefits or allowances.

2.3. Participants and Recruitment

We used multiple recruitment strategies, with a non-random selection of the study participants. Individuals were referred to the CDSMP through the WSLHD’s multicultural health services, existing chronic disease services (e.g., Connecting Care; Chronic Care Rehabilitation Service), and via clinicians within the Local Health District and general practice network. The inclusion criteria were individuals who (a) had one or more (self-reported) chronic diseases, (b) were over 16 years of age, and (c) had the (self-reported) ability to attend the 6-week face-to-face CDSMP.

2.4. Outcome Measures

The participants were asked to complete paper-based questionnaires providing demographic information and assessing health literacy across a range of domains using the HLQ, as described below. The data were collected upon commencement (pre-intervention) and at the conclusion (post-intervention) of the CDSMP. Although facilitators were present and could assist in reading difficult words, participants were required to answer questions independently.

Health literacy was measured using the HLQ [13]; a multi-dimensional tool that measures health literacy across 9 distinct conceptual domains (Figure 2) [13,35]. The HLQ has 44 questions, with 4 to 5 questions per domain. Domains 1 to 5 assess the strength of participants’ agreement with a statement on a 4-point ordinal scale (strongly disagree = 1 to strongly agree = 4) while domains 6 to 9 assess participants’ perceived ease in task completion on a 5-point ordinal scale (cannot do or always difficult = 1 to very easy = 5). The median reliability of the HLQ domains is 0.88, with a range from 0.77 to 0.90 [13].

Figure 2

Nine domains of the Health Literacy Questionnaire (HLQ) [13].

The questionnaire takes between 7 to 40 minutes to complete. Participants enrolled in non-English CDSMP groups in this study completed a translated HLQ which were available from the instrument developers.

2.5. Statistical Analysis

Descriptive statistics were used to summarise demographic characteristics and the pre- and post-intervention HLQ scores. The pre-intervention demographic data and mean HLQ domain scores between those who completed the program and those who did not complete the program were compared using chi-square tests and independent sample t-tests (see Table A1). Paired sample t-tests were used to assess whether the HLQ domain mean scores differed pre- and post-intervention. Exploratory multivariable linear regression modelling was utilised to investigate the associations between the HLQ domain score changes and a range of demographic variables.

Table A1

Demographic characteristics of all participants (pre-intervention) (n = 448).

Characteristics	Completed n (%)	Not Completed n (%)	Χ2 (P)
Number of participants	316	132
Age (years)
<50	12 (3.8%)	11 (8.3%)	6.97(0.14)
50–59	39 (12.3%)	15 (11.4%)
60–69	123 (38.9%)	46 (34.8%)
70–79	104 (32.9%)	34 (25.8%)
≥80	29 (9.1%)	17 (12.9%)
Gender
Female	196 (62.0%)	84 (63.6%)	0.09(0.76)
Male	115 (36.4%)	45 (34.1%)
Live Alone
Yes	56 (17.7%)	27 (20.5%)	0.28(0.59)
No	235 (74.4%)	95 (72.0%)
Country of Birth
Australia	51 (16.1%)	36 (27.3%)	54.85(<0.001)
Mainland China	78 (24.7%)	40 (30.3%)
India	71 (22.5%)	21 (15.9%)
Hong Kong	34 (10.8%)	1 (0.8%)
Turkey	33 (10.4%)	1 (0.8%)
Sri Lanka	25 (7.9%)	3 (2.3%)
Other	19 (6.0 %)	26 (19.7%)
English Spoken at Home
Yes	88 (27.8%)	63 (47.7%)	17.01(<0.001)
No	222 (70.3%)	64 (48.5%)
Education Level
Primary School or Less	32 (10.1%)	15 (11.4%)	11.03(0.05)
High School (not completed)	38 (12.0%)	20 (15.2%)
High School (completed)	66 (20.9%)	14 (10.6%)
TAFE/Trade	42 (13.3%)	17 (12.9%)
Undergraduate	87 (27.5%)	38 (28.8%)
Postgraduate	17 (5.4%)	15 (11.4%)
Employment Status
Working / Studying	14 (4.4%)	9 (6.8%)	1.51(0.83)
Home duties	39 (12.3%)	14 (10.6%)
Retired	148 (46.8%)	67 (50.8%)
Permanently unable to work / Ill	10 (3.2%)	4 (3.0%)
Other	17 (5.4%)	6 (4.5%)
Self-reported Illness
Arthritis	120 (38.0%)	54 (40.9%)	7.63(0.27)
Diabetes	94 (29.7%)	37 (28.0%)
Back pain	92 (29.1%)	42 (31.8%)
Heart problems	86 (27.2%)	40 (30.3%)
Depression / Anxiety	68 (21.5%)	25 (18.9%)
Other	146 (46.8%)	58 (43.9%)
None	14 (4.4%)	15 (11.4%)
Private Health Insurance
Yes	108 (34.2%)	31 (23.5%)	4.36(0.04)
No	185 (58.5%)	90 (68.2%)
Healthcare Card
Yes	230 (72.7%)	92 (69.7%)	0.65(0.42)
No	59 (18.7%)	30 (22.7%)
Attended Hospital Emergency in the Past 12 Months
Yes	65 (20.6%)	28 (21.2%)	0.02(0.89)
No	164 (51.9%)	76 (57.6%)
Teaching Group
Arabic	6 (1.9%)	3 (2.3%)	75.59(<0.001)
ATSI	26 (8.2%)	21 (15.9%)
Australian and NZ	4 (1.3%)	4 (3.0%)
Chinese (Cantonese)	57 (18.0%)	8 (6.1%)
Chinese (Mandarin)	54 (17.1%)	33 (25.0%)
Chinese (Traditional)	9 (2.8%)	3 (2.3%)
English	22 (7.0%)	14 (10.6%)
Filipino (Tagalong)	1 (0.3%)	16 (12.1%)
Hindi	63 (19.9%)	14 (10.6%)
Indian (English)	12 (3.8%)	9 (6.8%)
Multicultural	4 (1.3%)	1 (0.8%)
Tamil	25 (7.9%)	5 (3.8%)
Turkish	33 (10.4%)	1 (0.8%)
Required Help with Questionnaire
Yes	46 (14.6%)	21 (15.9%)	0.04(0.84)
No	247 (78.2%)	102 (77.3%)

N.B. In the completer group, there were nine missing values for the variable Age; five missing values for the variable Gender; 25 missing values for the variable Live Alone; five missing values for the variable Country of Birth; six missing values for the variable English Spoken at Home; 34 missing values for the variable Education Level; 88 missing values for the variable Employment Status; 28 missing values for the variable Self-reported Illness; 23 missing values for the variable Private Health Insurance; 27 missing values for the variable Healthcare Card; 87 missing values for the variable Attended Hospital Emergency in the Past 12 Months; 23 missing values for the variable Required Help with Questionnaires; In non-completer group, there were nine missing values for the variable Age; three missing values for the variable Gender; 10 missing values for the variable Live Alone; four missing values for the variable Country of Birth; five missing values for the variable English Spoken at Home; 13 missing values for the variable Education Level; 32 missing values for the variable Employment Status; 7 missing values for the variable Self-reported Illness; 11 missing values for the variable Private Health Insurance; 10 missing values for the variable Healthcare Card; 28 missing values for the variable Attended Hospital Emergency in the Past 12 months; nine missing values for the variable Required Help with Questionnaires.

All data were analysed using IBM SPSS Version 25 and R version 3.5.2, with P < 0.05 considered statistically significant. The expectation maximisation algorithm was used to substitute the missing HLQ values, following the HLQ User Manual; up to two missing values were imputed for domains with four or five questions. Missing data that could not be imputed were excluded pairwise from the analysis.

2.6. Ethics

The approval to evaluate and publish the results of this program was obtained from the Westmead Scientific Advisory QA Committee and the Secretary of the WSLHD Human Research Ethics Committee (Protocol number: 1809-04 QA).

3. Results

3.1. Demographic Characteristics

In total, 486 participants were recruited for the CDSMP between October 2014 and September 2018. Of those, 38 were excluded from the analysis as they did not complete the pre-intervention questionnaire. An additional 132 participants did not complete the post-intervention questionnaire or were removed from the analysis due to indistinguishable pre- and post-intervention questionnaires, leaving a sample of 316 (65.0%) participants included in the analysis (see Figure 3).

Figure 3

Flow of participants through the study.

The pre-intervention demographic characteristics of participants who completed both the pre- and post-intervention questionnaires are summarised in Table 1. Overall, the median age of participants was 68 years. There was approximately twice as many female participants than male participants. The majority were non-native English speakers, did not live alone, had a healthcare card, and were born in Australia, the Greater China regions or India. Health Care Cards are issued by the Australian Department of Human Services to people who live in Australia and already receive government payments, benefits or allowances. Approximately 15% of participants indicated that they required assistance to complete the questionnaires.

Compared to the participants who did not complete the program, those who did were less likely to speak English at home (P < 0.001), and more had private health insurance (P = 0.04). There were also differences related to country of birth (P < 0.001) and teaching groups (P < 0.001). Other demographic characteristics were statistically similar (See Table A1). The participants who did not complete the program also had significantly higher mean scores on two health literacy domains pre-intervention compared to those who completed the program (see Table A2): Domain 7 (Navigating the healthcare system) and Domain 8 (Ability to find good health information). However, these groups did not differ significantly on the other seven domains.

Table A2

Comparison of mean pre-intervention Health Literacy Questionnaire (HLQ) scores for completers (n = 316) and non-completers (n = 132).

HLQ Domain	Completers’ Mean Score (SD)	Non-completers’ Mean Score(SD)	Mean Difference	P-value
1. Feeling understood and supported by healthcare providers	11.95(2.23)	12.18(1.75)	0.23	0.25
2. Having sufficient information to manage my health	11.17(2.31)	11.28(2.32)	0.11	0.65
3. Actively managing my health	15.06(2.53)	15.14(2.25)	0.08	0.73
4. Social support for health	14.70(2.75)	14.83(2.44)	0.13	0.62
5. Appraisal of health information	14.51(2.33)	14.64(2.27)	0.12	0.60
6. Ability to actively engage with healthcare providers	17.36(4.03)	18.05(3.86)	0.68	0.09
7. Navigating the healthcare system	19.87(4.84)	20.89(4.84)	1.02	0.04
8. Ability to find good health information	16.39(4.01)	17.23(3.88)	0.85	0.04
9. Understand health information well enough to know what to do	17.16(3.93)	17.94(4.16)	0.78	0.07

N.B. In the completer group, there were 25 missing values for Domain 1; 11 missing values for Domain 2; nine missing values for Domain 3; 13 missing values for Domain 4; 12 missing values for Domain 5; 19 missing values for Domain 6; 23 missing values for Domain 7; 21 missing values for Domain 8; 16 missing values for Domain 9. In the non-completer group, there were six missing values for Domain 1; 11 missing values for Domain 2; seven missing values for Domain 3; 18 missing values for Domain 4; 12 missing values for Domain 5; 10 missing values for Domain 6; 10 missing values for Domain 7; 9 missing values for Domain 8; three missing values for Domain 9.

3.2. Health Literacy

The participants’ mean pre- and post-intervention HLQ domain scores and the mean domain score changes are presented in Table 2. There were statistically significant (P < 0.001) improvements in mean scores in all nine domains of the HLQ. The effect sizes (Cohen’s d) ranged between 0.32 (Domain 4: Social support for health) and 0.58 (Domain 2: Having sufficient information to manage my health).

Table 2

Health Literacy Questionnaire (HLQ) Domain Score Changes (n = 316). Higher values indicate greater understanding or ability.

HLQ Domain (Range)	Pre-intervention Mean (SD)	Post-intervention Mean (SD)	Mean Score Change (SD)	95%CI	Effect Sizes
1. Feeling understood and supported by healthcare providers (4–16)	11.95(2.23)	12.62(1.89)	0.67(2.16)	(0.43, 0.91)	0.32
2. Having sufficient information to manage my health (4–16)	11.17(2.31)	12.45(2.12)	1.28(2.50)	(1.00, 1.55)	0.58
3. Actively managing my health (4–20)	15.06(2.53)	16.19(2.19)	1.13(2.59)	(0.84, 1.42)	0.48
4. Social support for health (4–20)	14.70(2.75)	15.53(2.45)	0.84(2.81)	(0.52, 1.15)	0.32
5. Appraisal of health information (4–20)	14.51(2.33)	15.67(2.29)	1.16(2.66)	(0.87, 1.46)	0.50
6. Ability to actively engage with healthcare providers (5–25)	17.36(4.03)	18.79(3.69)	1.43(3.76)	(1.01, 1.84)	0.37
7. Navigating the healthcare system (6–30)	19.87(4.84)	22.22(4.55)	2.35(4.68)	(1.83, 2.87)	0.50
8. Ability to find good health information (5–25)	16.39(4.01)	18.37(3.82)	1.99(3.83)	(1.56, 2.41)	0.51
9. Understand health information well enough to know what to do (5–25)	17.16(3.93)	18.71(3.93)	1.55(3.73)	(1.14, 1.96)	0.39

Exploratory multivariable linear regression modelling was conducted (see Table A3) to determine whether the magnitude of change in the HLQ domain scores differed as a function of sociodemographic and health factors. Each HLQ domain score was modelled individually, with all demographic variables shown in Table 1 (except country of birth, due to collinearity with teaching group) included as explanatory variables. The outcomes of these analyses indicated a differential effect of age, gender, living alone, education level, particular chronic health conditions (arthritis, heart problems), and teaching group on the amount of improvement observed across HLQ domains.

Table A3

Association of pre-intervention demographic characteristics and change scores for each domain of the Health Literacy Questionnaire (HLQ) (n = 316).

	Mean Score Change (SD)
Characteristics	Domain 1:Healthcare provider support	Domain 2:Having Sufficient Information	Domain 3:Actively Managing Health	Domain 4:Social Support for Health	Domain 5:Active Appraisal of Health Information	Domain 6:Active Engagement with Healthcare	Domain 7:Navigating the Health Care System	Domain 8:Ability to Find Good Health Information	Domain 9:Understanding Health Information
Age group	P = 0.32	P = 0.06	P = 0.002	P = 0.52	P = 0.40	P = 0.93	P = 0.75	P = 0.62	P = 0.49
<50	1.17 (1.64)	2.33 (2.46)	2.83 (2.82)	0.25 (1.86)	2.58 (2.31)	0.75 (2.67)	3.58 (3.87)	2.50 (1.78)	1.00 (2.17)
50–59	0.69 (3.04)	1.74 (3.32)	2.05 (2.93)	1.23 (3.39)	1.46 (3.43)	2.33 (4.66)	2.69 (5.42)	2.10 (4.38)	1.72 (4.68)
60–69	0.83 (2.15)	1.43 (2.38)	1.41 (2.38)	1.12 (2.86)	1.40 (2.65)	1.45 (3.68)	2.39 (4.50)	2.33 (3.94)	2.10 (3.75)
70–79	0.55 (1.73)	0.90 (2.14)	0.43 (2.30)	0.45 (2.42)	0.66 (2.10)	1.09 (3.32)	2.07 (4.18)	1.66 (3.36)	0.96 (3.03)
≥80	0.14 (2.49)	0.38 (2.64)	0.38 (3.20)	0.34 (3.32)	0.72 (3.36)	1.28 (4.67)	2.34 (6.63)	0.97 (4.69)	0.69 (4.50)
Gender	P = 0.85	P = 0.33	P = 0.84	P = 0.47	P = 0.05	P = 0.46	P = 0.67	P = 0.82	P = 0.27
Female	0.62 (2.16)	1.15 (2.36)	1.09 (2.39)	0.76 (2.55)	0.91 (2.50)	1.38 (3.62)	2.23 (4.32)	1.83 (3.62)	1.20 (3.46)
Male	0.76 (2.21)	1.41 (2.70)	1.20 (2.93)	0.98 (3.24)	1.55 (2.91)	1.44 (4.03)	2.56 (5.33)	2.17 (4.20)	2.03 (4.05)
Live Alone	P = 0.03	P = 0.94	P = 0.21	P = 0.15	P = 0.50	P = 0.84	P = 0.24	P = 0.05	P = 0.49
Yes	1.12 (1.96)	1.61 (2.23)	1.30 (2.71)	1.12 (2.91)	1.25 (2.55)	1.61 (3.96)	3.16 (5.15)	2.59 (3.20)	1.88 (3.90)
No	0.46 (2.12)	1.03 (2.49)	0.93 (2.44)	0.63 (2.68)	0.98 (2.57)	1.30 (3.66)	2.08 (4.54)	1.68 (3.84)	1.33 (3.51)
Country of Birth	N/A a
Australia	0.75 (2.24)	1.96 (2.79)	1.61 (2.55)	0.86 (3.23)	1.41 (2.88)	1.35 (4.42)	2.31 (5.67)	2.37 (4.58)	1.51 (3.95)
Mainland China	0.22 (2.11)	0.21 (2.52)	0.47 (2.44)	0.13 (2.42)	0.29 (2.60)	1.17 (3.96)	1.71 (4.67)	1.04 (4.04)	0.26 (3.87)
India	0.97 (2.40)	1.87 (2.60)	1.04 (3.01)	1.14 (3.17)	1.31 (3.00)	2.07 (4.09)	3.59 (5.16)	3.17 (3.99)	2.69 (4.24)
Hong Kong	0.29 (2.22)	0.88 (2.07)	1.41 (2.20)	0.62 (2.81)	1.65 (2.62)	1.56 (3.64)	2.21 (3.90)	1.94 (3.50)	1.88 (3.04)
Turkey	1.58 (1.89)	2.58 (1.56)	2.45 (2.03)	1.85 (2.41)	2.30 (1.86)	1.21 (1.39)	2.42 (2.97)	1.91 (2.14)	1.70 (2.58)
Sri Lanka	0.76 (1.27)	1.00 (1.50)	0.76 (1.90)	1.04 (2.05)	1.08 (2.20)	−0.28 (2.59)	0.52 (2.65)	0.84 (2.53)	0.80 (2.06)
Other	0.68 (1.86)	0.47 (1.84)	1.00 (2.85)	1.05 (2.63)	0.95 (1.93)	2.84 (3.66)	3.68 (5.22)	2.37 (3.35)	2.63 (3.55)
English Spoken at Home	P = 0.996	P = 0.68	P = 0.52	P = 0.36	P = 0.50	P = 0.19	P = 0.28	P = 0.14	P = 0.85
Yes	0.76 (2.08)	1.72 (2.47)	1.20 (2.44)	0.72 (2.82)	1.32 (2.55)	1.17 (3.98)	2.28 (4.83)	2.20 (4.05)	1.57 (3.48)
No	0.59 (2.08)	1.02 (2.36)	1.05 (2.50)	0.80 (2.67)	1.02 (2.56)	1.40 (3.49)	2.25 (4.45)	1.78 (3.53)	1.41 (3.61)
Education Level	P = 0.63	P = 0.54	P = 0.59	P = 0.43	P = 0.95	P = 0.27	P = 0.31	P = 0.30	P = 0.03
Primary School or less	0.59 (2.21)	1.53 (2.93)	1.31 (2.78)	0.38 (3.44)	1.06 (3.22)	2.19 (4.79)	3.72 (5.65)	2.56 (4.19)	2.91 (3.73)
High School (not completed)	1.32 (2.59)	1.97 (2.33)	1.71 (1.84)	1.58 (2.66)	1.50 (2.30)	1.71 (3.85)	2.37 (4.96)	2.95 (3.57)	1.84 (3.61)
High School (completed)	0.74 (2.23)	1.26 (2.54)	1.00 (2.98)	0.55 (2.93)	1.00 (2.73)	1.38 (4.01)	3.08 (4.65)	1.67 (4.05)	1.76 (4.12)
TAFE/Trade	0.21 (2.32)	0.57 (2.31)	0.45 (2.70)	0.57 (2.86)	0.81 (2.96)	1.24 (3.61)	1.83 (5.06)	1.24 (4.15)	0.31 (3.22)
Undergraduate	0.48 (2.03)	0.94 (2.57)	1.11 (2.67)	0.75 (2.62)	1.08 (2.65)	1.26 (3.77)	1.92 (4.65)	1.85 (3.90)	1.14 (3.75)
Postgraduate	0.65 (1.73)	1.00 (1.62)	0.47 (1.70)	0.88 (2.50)	1.53 (2.40)	0.00 (2.81)	1.35 (3.32)	1.71 (3.26)	1.35 (2.83)
Employment	P = 0.77	P = 0.59	P = 0.29	P = 0.33	P = 0.60	P = 0.06	P = 0.23	P = 0.12	P = 0.23
Working / Studying	0.29 (2.30)	0.93 (2.06)	0.64 (2.82)	0.29 (2.61)	0.86 (2.71)	0.93 (3.15)	1.71 (3.41)	0.71 (2.55)	1.29 (2.05)
Home duties	0.10 (2.43)	0.38 (2.32)	1.05 (2.09)	0.38 (2.68)	0.41 (2.59)	0.69 (3.04)	1.18 (3.97)	0.79 (3.78)	0.67 (3.03)
Retired	0.65 (2.04)	1.03 (2.53)	0.91 (2.46)	0.79 (2.70)	1.03 (2.71)	1.09 (3.80)	2.03 (4.70)	1.72 (3.90)	1.11 (3.58)
Ill / Permanently unable to work	0.80 (3.68)	1.30 (1.83)	1.30 (2.71)	2.10 (3.38)	1.10 (2.64)	3.40 (4.33)	3.00 (3.06)	2.70 (2.63)	2.70 (4.42)
Other	0.88 (1.17)	1.88 (2.37)	2.29 (2.87)	0.06 (2.19)	2.12 (2.06)	1.53 (3.79)	2.94 (5.51)	2.53 (4.47)	1.47 (3.22)
Chronic Disease
Arthritis	P = 0.87	P = 0.53	P = 0.29	P = 0.07	P = 0.16	P = 0.10	P = 0.003	P = 0.07	P = 0.13
	0.66 (2.29)	1.25 (2.64)	1.07 (2.45)	1.08 (2.84)	1.05 (2.61)	1.88 (4.21)	3.04 (5.07)	2.33 (4.17)	1.75 (3.70)
Diabetes	P = 0.10	P = 0.68	P = 0.46	P = 0.11	P = 0.91	P = 0.59	P = 0.65	P = 0.91	P = 0.26
	0.49 (2.21)	1.36 (2.70)	1.31 (2.85)	0.60 (2.87)	1.33 (2.80)	1.18 (4.06)	2.13 (5.12)	2.22 (4.07)	1.67 (3.91)
Back pain	P = 0.85	P = 0.30	P = 0.41	P = 0.16	P = 0.50	P = 0.11	P = 0.05	P = 0.86	P = 0.85
	0.80 (2.16)	1.55 (2.43)	1.46 (2.52)	1.18 (2.93)	1.22 (2.46)	2.26 (4.24)	3.55 (5.67)	2.39 (4.39)	1.82 (3.65)
Heart problems	P = 0.12	P = 0.75	P = 0.0005	P = 0.08	P = 0.03	P = 0.86	P = 0.26	P = 0.65	P = 0.43
	0.86 (2.13)	1.31 (2.56)	1.65 (2.39)	1.31 (2.63)	1.56 (2.87)	1.72 (4.26)	2.79 (4.51)	2.14 (3.73)	1.83 (3.25)
Depression / Anxiety	P = 0.88	P = 0.90	P = 0.53	P = 0.64	P = 0.74	P = 0.62	P = 0.75	P = 0.73	P = 0.28
	0.91 (2.36)	1.69 (2.80)	1.87 (2.32)	1.21 (3.07)	1.41 (2.52)	1.75 (4.17)	3.28 (5.51)	2.51 (4.09)	1.91 (3.64)
Other	P = 0.30	P = 0.93	P = 0.87	P = 0.76	P = 0.69	P = 0.26	P = 0.41	P = 0.36	P = 0.72
	0.83 (2.00)	1.38 (2.56)	1.20 (2.44)	0.80 (2.81)	1.18(2.57)	1.52(3.85)	2.53(4.78)	1.89(3.98)	1.63(3.62)
Private Health Insurance	P = 0.46	P = 0.82	P = 0.86	P = 0.93	P = 0.70	P = 0.62	P = 0.11	P = 0.02	P = 0.27
Yes	0.47 (2.10)	1.11 (2.45)	1.11 (2.47)	0.59 (2.66)	1.06 (2.67)	1.15 (3.65)	1.98 (4.37)	2.13 (3.55)	1.37 (3.43)
No	0.80 (2.22)	1.36 (2.49)	1.14 (2.63)	0.96 (2.87)	1.19 (2.65)	1.57 (3.93)	2.64 (4.98)	1.91 (4.07)	1.66 (3.88)
Healthcare Card	P = 0.06	P = 0.11	P = 0.75	P = 0.02	P = 0.08	P = 0.40	P = 0.48	P = 0.53	P = 0.82
Yes	0.56 (2.12)	1.10 (2.50)	1.08 (2.46)	0.63 (2.66)	0.94 (2.59)	1.29 (3.74)	2.30 (4.50)	1.86 (3.78)	1.36 (3.53)
No	0.92 (2.10)	1.56 (2.25)	0.98 (2.32)	1.27 (2.72)	1.47 (2.39)	1.88 (3.33)	2.64 (4.24)	2.20 (3.30)	2.19 (3.43)
Attended Hospital Emergency in the Past 12 Months	P = 0.67	P = 0.29	P = 0.96	P = 0.69	P = 0.23	P = 0.86	P = 0.60	P = 0.90	P = 0.998
Yes	0.62 (2.24)	0.94 (2.57)	1.22 (2.61)	0.51 (2.94)	0.95 (2.90)	1.22 (4.12)	2.49 (5.11)	1.78 (4.11)	1.45 (3.67)
No	0.54 (2.12)	1.06 (2.39)	1.00 (2.37)	0.73 (2.64)	1.06 (2.50)	1.21 (3.48)	1.91 (4.34)	1.70 (3.68)	1.17 (3.31)
Help with Questionnaire	P = 0.65	P = 0.08	P = 0.50	P = 0.77	P = 0.13	P = 0.33	P = 0.06	P = 0.98	P = 0.43
Yes	1.02 (2.56)	2.04 (3.20)	1.39 (3.09)	0.96 (3.63)	1.59 (3.24)	1.17 (4.55)	2.70 (5.94)	2.54 (4.73)	1.93 (4.41)
No	0.60 (2.13)	1.09 (2.34)	1.06 (2.48)	0.78 (2.63)	1.02 (2.56)	1.51 (3.70)	2.39 (4.53)	1.92 (3.73)	1.49 (3.62)
Teaching Group	P = 0.47	P = 0.04	P = 0.14	P = 0.30	P = 0.04	P = 0.53	P = 0.004	P = 0.21	P = 0.02
Arabic	1.17 (2.23)	1.00 (2.10)	1.33 (2.07)	2.67 (2.80)	1.33 (1.86)	3.67 (1.86)	2.83 (3.19)	3.67 (2.34)	3.17 (2.40)
ATSI	1.12 (2.14)	2.77 (2.42)	1.96 (2.92)	0.96 (2.90)	2.31 (2.53)	2.04 (3.55)	3.69 (5.03)	3.81 (3.81)	2.50 (3.47)
Australian and NZ †	0.00 (0.82)	0.50 (1.00)	0.50 (1.00)	0.25 (1.71)	0.00 (0.00)	−0.50 (2.08)	−2.25 (2.22)	−1.25 (2.75)	−1.50 (1.29)
Chinese (Cantonese)	0.51 (2.11)	0.65 (2.04)	1.49 (2.15)	0.63 (2.60)	1.33 (2.43)	1.74 (3.83)	2.84 (4.34)	2.14 (3.50)	2.00 (3.12)
Chinese (Mandarin)	0.04 (2.08)	0.20 (2.64)	0.11 (2.47)	0.06 (2.39)	0.13 (2.60)	1.35 (4.09)	1.74 (4.87)	1.00 (4.09)	0.15 (4.03)
Chinese (Traditional)	0.11 (1.96)	−0.11 (2.15)	1.00 (2.96)	0.11 (2.26)	0.33 (3.28)	−0.56 (3.40)	−1.33 (3.74)	−0.33 (4.36)	−1.78 (2.39)
English	0.45 (2.52)	1.27 (3.15)	1.50 (2.26)	0.95 (3.81)	0.55 (3.17)	0.82 (5.43)	1.50 (6.28)	1.14 (5.03)	0.73 (4.31)
Filipino (Tagalong) b	0.00 (0.00)	−1.00 (0.00)	−2.00 (0.00)	−3.00 (0.00)	2.00 (0.00)	0.00 (0.00)	−1.00 (0.00)	−3.00 (0.00)	−1.00 (0.00)
Hindi	0.90 (2.49)	1.94 (2.71)	1.05 (3.11)	1.08 (3.33)	1.27 (3.12)	2.27 (4.27)	3.97 (5.44)	3.44 (4.10)	3.17 (4.48)
Indian (English)	0.83 (1.80)	1.33 (2.46)	0.33 (2.64)	1.08 (2.39)	1.00 (2.00)	1.25 (2.80)	1.50 (2.28)	1.33 (3.11)	0.67 (2.87)
Multicultural †	0.75 (1.50)	1.00 (2.00)	1.00 (0.82)	0.00 (0.00)	1.00 (2.00)	1.00 (2.00)	2.25 (2.06)	2.25 (1.50)	1.75 (2.22)
Tamil	0.48 (1.73)	0.88 (1.79)	0.40 (2.25)	0.88 (2.40)	1.08 (2.33)	−0.36 (2.66)	0.24 (2.71)	0.48 (2.71)	0.76 (2.30)
Turkish	1.58 (1.89)	2.58 (1.56)	2.45 (2.03)	1.85 (2.41)	2.30 (1.86)	1.21 (1.39)	2.42 (2.97)	1.91 (2.14)	1.70 (2.58)

N.B. There are 9 missing values for the variable of age; 5 missing values for the variable of gender; 25 missing values for the variable of live alone; 5 missing values for the variable of country of birth; 6 missing values for the variable of speaking English; 34 missing values for the variable of education level; 88 missing values for the variable of employment; 28 missing values for the variable of illness and; 193 missing values for the variable of number of sessions attending; 23 missing values for the variable of private health insurance; 27 missing values for the variable of having healthcare card; 87 missing values for the variable of attended hospital emergency in the past 12 months; 23 missing values for the variable of help with questionnaire. a Country of birth was not included in the analysis of covariance due to collinearity with teaching group. b Less than 5 observations stayed within the group.

4. Discussion

This is the first study to evaluate the impact of the CDSMP on health literacy using a multi-dimensional measure. Analyses showed that the CDSMP was effective in improving all domains of health literacy among a broad cross-section of participants. Statistically significant improvements (corresponding to small to moderate effect sizes) in all nine health literacy domains were observed; and these changes in reported confidence and skills can be matched to the educational content of the intervention. The largest effect sizes were observed for the domains ‘Having sufficient information to manage my health’, ‘Ability to find good health information’, ‘Navigating the healthcare system’ and ‘Appraisal of health information’. The smallest effect size was observed for ‘Feeling understood and supported by healthcare providers’. Exploratory analysis indicated that the differences in health literacy scores according to demographic characteristics were domain-specific, providing a complex picture with no clear patterns. However, there were some notable differences, including that the self-reported ability to ‘actively manage health’ decreased with age; people who lived alone had lower scores on ‘feeling understood and supported by healthcare professionals’; and that, in general, increasing education was associated with higher self-reported understanding of health information.

This study adds to the body of evidence supporting the potential effectiveness of self-management programs for patients with chronic disease [17,24,29,30] by showing that programs such as the CDSMP can support the development of sustainable and potentially transferable health literacy skills immediately post-intervention. Importantly, these findings were observed in a highly diverse cohort of traditionally ‘hard-to-reach’ adults and so provide practical insights into intervention delivery and assessment in this context. While we were able to recruit 486 people into the program using multiple recruitment strategies, 35% did not complete the post-intervention questionnaire and were excluded from the analysis, and those who were included had higher HLQ scores on some domains, and were more likely to have private health insurance. They were, however, less likely speak English at home compared to those who were excluded. High attrition is common in health literacy interventions [36] and it has been acknowledged that work is needed to investigate how to recruit and retain ‘hard to reach’ participants throughout trials [36]. Related to this, our finding that we were able to retain people who spoke languages other than English at home provides some support for the use of in-language programs in diverse settings.

The findings of this study should be considered in light of the limitations of a single-arm pre-post study design with non-random selection of participants, including being unable to account for concurrent events or programs that could have contributed to the change in participants’ health literacy during the intervention period and the possibility of recruiting participants who were highly motivated to improve their health. Given the use of convenience sampling, we do not have any information on the total number of eligible participants, and our sample is not representative of the population in WSLHD or those with chronic disease in Australia. Compared to the total population of Western Sydney Local Health District, for example, our sample had disproportionately more females than males. Although, in Australia, females aged 15 years and over are more likely than males to have one or more chronic conditions, the distribution was more heavily skewed in our sample (62% compared to 36%), compared to population data (57% compared to 51%) [37]. The two main countries of birth (other than Australia) for participants in this study, however, did reflect those of the total population in WSLHD [38]. Future studies applying more rigorous study designs (e.g., matched controls, waitlist-randomised control trial) will help to strengthen evidence for the effectiveness of CDSMP on improving health literacy.

There are also limitations pertaining to measurement instruments used in this study, particularly the use of translated versions of the HLQ. Although the instrument developers note that a “strict protocol is followed for each translation to help ensure each version of the HLQ is linguistically, culturally and psychometrically robust” [39], the validity and reliability of the translated versions of HLQ used in our study have not been reported in a peer-reviewed format. Studies using skills-based (rather than self-report) measures of health literacy may also yield different results. It has recently been argued that self-report measures of health literacy (such as the HLQ) assess participants’ confidence in their capacity to comprehend and use health materials, rather than their actual ability, and that, conceptually, feelings of perceived confidence (i.e. self-efficacy) differ from health literacy (i.e. skills) [39]. In line with this, a recent meta-analysis found that health literacy had a positive impact on the self-care activities of people living with diabetes only for studies which assessed health literacy using self-report measures and not for those which used skills-based instruments [39]. Comparing the effect of the CDSMP on skills-based and self-report measures of health literacy, therefore, represents an important direction for future research.

To continue to address questions about the key mechanisms through which chronic disease self-management programs are effective [40], future research should build on the current study by also assessing intermediate and longer-term impacts and outcomes in addition to the short-term impacts on health literacy. Structural equation modelling could then be used to develop a more comprehensive program logic and evaluate whether short term impacts (e.g., changes in health literacy) mediate intermediate impacts (improved self-management) and longer-term outcomes (e.g., improved clinical indices).

Although there is now evidence regarding the effectiveness of chronic disease self-management programs—including on health literacy skills identified in this study—Harris et al., [41] have detailed some of the challenges related to implementing and sustaining programs such as the CDSMP in practice. They highlight the importance of collaboration between organisations to provide and coordinate self-management programs (e.g., state health services; non-government organisations; primary care providers), as well as the need for referral pathways which are sustained over time and facilitate direct communication [41]. The education of staff on the evidence of the effectiveness of these programs was also identified as an enabler of engagement with self-management support [41]. A model for implementation has been provided (by Western Sydney Local Health District, as described here) which addresses many of the potential threats to sustainability, including through the development of care pathways and shared care plans between primary, community and hospital providers, and the use of community-based care facilitators to assist in care planning, navigation, transitional care between services, and patient education and self-management [16]. Implementation research exploring models such as these represents an important direction for future work in this field.

5. Conclusions

This is the first study to evaluate the potential impact of the CDSMP on improving different domains of health literacy amongst a diverse sample of participants with chronic disease using a multi-dimensional measure. The absence of a control population in this study means that some caution should be applied to interpreting the results.