Simple scoring algorithm to identify community-dwelling older adults with limited health literacy: a cross-sectional study in Taiwan.

OBJECTIVE: Health literacy (HL) is the degree of individuals' capacity to access, understand, appraise and apply health information and services required to make appropriate health decisions. This study aimed to establish a predictive algorithm for identifying community-dwelling older adults with a high risk of limited HL.
DESIGN: A cross-sectional study.
SETTING: Four communities in northern, central and southern Taiwan. PARTICIPANTS: A total of 648 older adults were included. Moreover, 85% of the core data set was used to generate the prediction model for the scoring algorithm, and 15% was used to test the fitness of the model. PRIMARY AND SECONDARY OUTCOME MEASURES: Pearson's χ2 test and multiple logistic regression were used to identify the significant factors associated with the HL level. An optimal cut-off point for the scoring algorithm was identified on the basis of the maximum sensitivity and specificity.
RESULTS: A total of 350 (54.6%) patients were classified as having limited HL. We identified 24 variables that could significantly differentiate between sufficient and limited HL. Eight factors that could significantly predict limited HL were identified as follows: a socioenvironmental determinant (ie, dominant spoken dialect), a health service use factor (ie, having family doctors), a health cost factor (ie, self-paid vaccination), a heath behaviour factor (ie, searching online health information), two health outcomes (ie, difficulty in performing activities of daily living and requiring assistance while visiting doctors), a participation factor (ie, attending health classes) and an empowerment factor (ie, self-management during illness). The scoring algorithm yielded an area under the curve of 0.71, and an optimal cut-off value of 5 represented moderate sensitivity (62.0%) and satisfactory specificity (76.2%).
CONCLUSION: This simple scoring algorithm can efficiently and effectively identify community-dwelling older adults with a high risk of limited HL. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Our health literacy (HL) prediction model comprising personal, contextual, and socioenvironmental factors can be used to formulate health policies for older adults.

This scoring algorithm not only helps clinicians assess and identify the HL level among older adults but also assists researchers to establish early identification for older adults with limited HL.

Prospective trials should examine the implementation and utility of this simple scoring algorithm for the early detection of older adults with a high risk of limited HL in the community.

Introduction

Health literacy (HL), which refers to people’s knowledge and competency required to meet the complex demands of health and healthcare (HC), is crucial and increasingly being recognised in modern society. However, no consensus exists regarding the definition, dimensions and conceptual models of HL. The European Health Literacy Consortium has proposed a conceptual model integrating medical and public health perspectives of HL; the model can be used to determine the different dimensions of HL across different clinical and community settings. They defined HL as the ability to access, understand, appraise and apply health information or services for obtaining appropriate HC and for disease prevention (DP) and health promotion (HP).1 The HL measure has been indicated as the sixth vital sign along with temperature, pulse, respiration, blood pressure and pain level. The importance of HL should be emphasised, so that clinicians and public health workers can enable early and precise access to intervention strategies.2 3 However, the construct of HL is complex and dynamic and encompasses many aspects of individuals’ use of health information and the HC system. Therefore, the European Health Literacy Survey (HLS-EU) Consortium has proposed a theoretical model that integrates medical and public health perspectives of HL and that accounts for its various antecedents including personal, contextual, social and environmental determinants. Therefore, the level of HL indicates health service use, health costs, health behaviour, health outcomes, participation and empowerment among individuals.1

Older adults generally have more chronic illnesses and less formal education than their younger counterparts.4 Moreover, older adults experience unique problems related to physical and cognitive functioning that cause difficulty in finding accurate health information and using appropriate HC services.5 Several national surveys have reported that more than half of older adults have limited HL.6–8 Studies have reported that the level of HL was significantly lower in older people than in younger people.9–11 Moreover, limited HL results in poor health outcomes12 and health behaviours,13 increased HC expenditure14 and health service usage,15 and inadequate empowerment16 and participation.17 Therefore, early and accurate prediction of limited HL among older adults is essential to implement prompt and appropriate HC strategies.

Various HL measures have been developed for older adults.18 A US national survey examined the agreement (sensitivity and specificity) and discrimination (c-statistic) of functional HL derived from common sociodemographic data; this functional HL can serve as a readily calculated HL proxy score.19 A study developed a predictive model by performing multiple regression and validating its efficiency in identifying older people with the highest risk of inadequate functional HL.20 However, most of these studies lack underlying theoretical basis and fail to sufficiently cover the comprehensive dimensions of HL across different clinical environments. Furthermore, to the best of our knowledge, no study has developed a prediction model of self-reported general HL for the early and accurate identification of HL levels in older adults. Therefore, developing a simple algorithm that can be applied in clinical settings to accurately identify older adults with a high risk of limited HL is essential. This study conducted a survey among community-dwelling older adults to identify factors predicting limited HL and constructed an optimal scoring algorithm for predicting HL.

Methods

Patient and public involvement

Figure 1 presents information on participant selection, data set division and analysis procedures. In this cross-sectional study, by adopting the convenience sampling method, we recruited eligible community participants aged ≥65 years from six senior service centres and three health check-up clinics in northern, central and southern Taiwan between June and September 2018. The individuals were excluded from the study if they were found to be cognitively impaired based on the Mini-Cog instrument screening. The survey questionnaire items and self-reported outcome measures were developed following the conclusions of three consensus meetings among multidisciplinary experts of health policy, HL, health education, social welfare, gerontology and geriatric medicine. We either interviewed participants to obtain responses for the questionnaire or asked them to complete a self-administered HL-related questionnaire.21 Trained interviewers explained the objective of the study to respondents before they expressed their willingness to participate in the study. The survey was anonymous, and respondents were allowed to suspend the interview at any time. There was no difference in expected medical treatment whether the individual join or turn down our study. The participants were allowed to withdraw anytime or complete the interview in different time schedule if they felt tired or stressful. The study results were disseminated to the public through a conference presentation hosted by the Taiwan National Health Research Institutes at the end of the study. The minimum sample size was decided from the 10:1 ratio of participants to variables by Everitt’s recommendations.22

Figure 1

Flow chart of participant selection, data set division and analysis procedure.

Procedure

After signing informed consent forms for participation in the study, the participants were either interviewed or asked to complete a self-administered questionnaire including questions on 52 potential predictors including personal, situational, and socioenvironmental determinants and factors related to health service use, health costs, health behaviour, health outcomes, participation and empowerment based on the theoretical model of the HLS-EU Consortium.23

Outcome measures

The 47-item HLS-EU Questionnaire (HLS-EU-Q), developed by the HLS-EU Consortium, was used to assess the comprehensive HL of the study participants. The HLS-EU-Q measures four HL competencies (access, understand, appraise and apply health information) required under three health domains: HC (16 items), DP (15 items) and HP (16 items). Each item assesses the self-perceived difficulty in performing selected health-related tasks on a 4-point scale ranging from ‘very easy’ (4) to ‘very difficult’ (1). Higher scores indicate a higher level of HL. For ease of comparison, the score of each domain (ie, HC, DP and HP) was linearly transformed to a score between 0 and 50 by using a scale adopted in the HLS-EU, which h has been validated to have satisfactory psychometric properties.24 On the basis of the scores, HL was divided into four categories: inadequate (0–25), problematic (26–33), sufficient (34–42) and excellent (43–50).25 26 We dichotomised HL into ‘sufficient’ and ‘limited’ based on a cut-off value of 34, as defined by the HLS-EU.25

Statistical analysis

The dichotomised outcome is defined using the HL level as follows:

To develop a scoring algorithm for predicting a low level of HL, the core data set was divided using stratified random sampling without the replacement method as follows: 85% of the core data set was categorised into the training data set, which was used for training the prediction model to create the scoring rule, and 15% of the data set was categorised as the validation data set, which was used for validating the scoring algorithm (figure 1).27 By Everitt’s recommendations, a minimum of 520 participants should be included in the training data.22 Pearson’s χ2 test was used initially to reduce variables and ensure that variables of crucial dimensions were retained. In addition, Pearson’s χ2 test was used to evaluate the association of the HL level with each of the 52 predictors included in the self-administered questionnaire.28 If the variables were not answered, we would not include the record in the analysis. To select the most relevant predictors, variables with a p<0.1 were included in multiple logistic regressions. Multiple logistic regressions with forward selection were used to examine relationships between limited HL and potential predictors classified into the domains of personal determinants, situational determinants, socioenvironmental determinants, health service use, health costs, health behaviour, health outcomes, participation and empowerment. Furthermore, the potential predictors with a p<0.05 were identified from the multiple logistic regression model.29 30 The formula used for multiple regression was as follows:

where p denotes the probability of limited HL in older adults, α is the intercept of multiple regression, and β is the slope of the specified predictor from 1st, 2nd, …, ith. In multiple regression, the OR was estimated using exp(β) to present relationships between dichotomised HL and predictors, and the estimated ORs were applied to calculate the clinical score for predicting health risk.31 32 To develop a simple algorithm, a previous study developed a new score based on the summation of significant predictors identified from multiple logistic regression. Significant predictors that were positively associated with limited HL (OR >1) were assigned a value of +1, whereas those that were negatively associated with limited HL (OR <1) were assigned a value of −1.31

Separately, 15% of the participants were included to validate the proposed scoring algorithm. On the basis of the algorithm obtained from the training data set, the total score was calculated for each older adult in the test dataset. The overall accuracy of prediction of limited HL using this score was evaluated by calculating sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).33 The model fit was assessed on the basis of McFadden’s pseudo R-square (measuring the reduction in the maximised log-likelihood from the intercept only model) and c-statistic (area under (AUC) the receiver operating characteristic (ROC) curve) values.34 35 A two-sided 95% CI for the AUC was used to denote uncertainty,36 and a p>0.05 in the Hosmer-Lemeshow test was used to indicate algorithm performance. As a graphical illustration of sensitivity–specificity trade-offs at each cut-off, the ROC curve demonstrated the degree to which the HL score could discriminate the level of HL, and the best cut-off value corresponded to a perfect scenario (100% sensitivity and 100% specificity). However, in practice, the optimal cut-off value should maintain a balance between sensitivity and specificity,37 which refers to the number of predictors calculated using the optimal classification threshold in our simple scoring algorithm. All statistical analyses were performed using SAS V.9.4 (SAS Institute).

Results

A total of 648 older adults were recruited. We excluded 12 patients who were found to have mild cognitive impairment by using the Mini-Cog instrument. Figure 1 presents age-specific HL levels. Nearly half (41.6%–46.1%) of the participants had problematic HL. A large proportion of the participants (72.6%) aged ≥81 years had limited HL. Table 1 lists the sociodemographic characteristics of the participants in the training data set (n=552/648) and test data set (n=96/648). Sex, age, educational level, marital status, occupation and monthly income showed similar distribution between the training and test data sets.

Table 1

Sociodemographic characteristics of study participants across the core, training and test data sets

	Core data set (n=648, 100%)		Training data set (n=552, 85%)		Test data set (n=96, 15%)
	Health literacy, n (%)
Variables	Limited	Sufficient	Limited	Sufficient	Limited	Sufficient
Sex
Male	138 (57.3)	103 (47.7)	117 (57.9)	85 (42.1)	21 (53.9)	18 (46.1)
Female	216 (53.3)	189 (46.7)	184 (52.9)	164 (47.1)	32 (56.1)	25 (43.9)
Age (years)
65–70	131 (51.8)	122 (48.2)	111 (51.9)	103 (48.1)	20 (51.3)	19 (48.7)
71–80	151 (51.5)	142 (48.5)	133 (52.4)	121 (47.6)	18 (46.2)	21 (53.8)
≥81	74 (72.6)	28 (27.4)	59 (70.2)	25 (29.8)	15 (83.3)	3 (16.7)
Education level
Illiterate or elementary school	124 (69.3)	55 (30.7)	104 (67.5)	50 (32.5)	20 (80.0)	5 (20.0)
Junior and senior high school	139 (57.2)	104 (42.8)	117 (57.6)	86 (42.4)	22 (55.0)	18 (45.0)
College degree or above	88 (40.2)	31 (58.8)	77 (40.7)	112 (59.3)	11 (36.7)	19 (63.3)
Marital status
Married	240 (51.8)	223 (48.2)	206 (52.2)	189 (47.8)	34 (50.0)	34 (50.0)
Single/divorced/widowed	115 (62.5)	69 (37.5)	96 (61.5)	60 (38.5)	19 (67.9)	9 (32.1)
Past occupation*
Manager or professional	59 (37.6)	98 (62.4)	52 (38.5)	83 (61.5)	7 (31.8)	15 (68.2)
Sales/administration/service	84 (57.5)	62 (42.5)	67 (55.4)	54 (44.6)	17 (68.0)	8 (32.0)
Technical/production/operators/ labourers//forestry/farmer/fisher	86 (66.7)	43 (33.3)	72 (67.9)	34 (32.1)	14 (60.9)	9 (39.1)
Housewife/unemployed	90 (61.2)	57 (38.8)	77 (60.6)	50 (39.4)	13 (65.0)	7 (35.0)
Monthly income (NT$)
No income	108 (59.3)	74 (40.7)	95 (59.8)	64 (40.2)	13 (56.5)	10 (43.5)
<NT$20 000	118 (60.2)	78 (39.8)	95 (57.9)	69 (42.1)	23 (71.9)	9 (28.1)
NT$20 001–NT$50 000	81 (50.0)	81 (50.0)	69 (50.4)	68 (49.4)	12 (48.0)	13 (52.0)
≥NT$50 001	42 (42.9)	56 (57.1)	37 (45.1)	45 (54.9)	5 (31.3)	11 (68.7)

*Denoted missing participants (n=579).

In the training data set (n=552/648), of the 52 variables included in the original questionnaire (online supplemental appendix 1), 24 (ie, 5 personal determinants, 2 situational determinants, 2 socioenvironmental determinants, 1 factor related to health service use, 1 factor related to health costs, 3 factors related to health behavior, 6 factors related to health outcomes, 1 factor related to participation, and 3 factors related to empowerment of HL) associated with the HL level (p<0.1) were identified using Pearson’s χ2 test (table 2).

Table 2

The 24 factors significantly correlated with health literacy level based on Pearson’s χ2 tests in the training dataset (n=552, 85%)

Scope and predictor	Health literacy, n (%)				X2	P value
	Limited		Sufficient
Personal determinants
Medical training (n=544)
Yes	12	(4.0)	23	(9.4)	6.5	0.011
No	287	(96.0)	222	(90.6)
Education level (n=546)
Illiterate or elementary school	104	(34.9)	50	(20.2)	25.8	<0.001
Junior and senior high school	117	(39.3)	86	(34.7)
College degree or above	77	(25.8)	112	(45.2)
Past occupation (n=489)
Manager or professional	52	(19.4)	83	(37.6)	23.6	<0.001
Sales/administration/service	67	(25.0)	54	(24.4)
Technical/production/operators/ labourers/forestry/farmer/fisher	72	(26.9)	34	(15.4)
Housewife/unemployed	77	(28.7)	50	(22.6)
Age (n=552, years)
65–70	111	(36.6)	103	(41.4)	9.4	0.009
71–80	133	(43.9)	121	(48.6)
≥81	59	(19.5)	25	(10.0)
Monthly income (n=542, NT$)
No income	95	(32.1)	64	(26.0)	6.4	0.094
<NT$20 000	95	(32.1)	69	(28.0)
NT$20 001–NT$50 000	69	(23.3)	68	(27.6)
≥NT$50 001	37	(12.5)	45	(18.3)
Situational determinants
Marriage (n=551)
Single/divorced/widowed	96	(31.8)	60	(24.1)	4.0	0.046
Married	206	(68.2)	189	(75.9)
Socioenvironmental determinants
Dominant spoken dialect (n=551)
Taiwanese, Hakka, or other dialect	154	(51.0)	41	(16.5)	71.2	<0.001
Mandarin	148	(49.0)	208	(83.5)
Residential area (n=552)
Taipei city	158	(52.1)	157	(63.1)	6.6	0.010
Other cities	145	(47.9)	92	(36.9)
Health service use
Having a family doctor (n=548)
Yes	16	(5.3)	24	(9.8)	4.0	0.046
No	286	(94.7)	222	(90.2)
Health costs
Pneumonia self-paid vaccination (n=547)
Yes	85	(28.4)	100	(40.3)	8.6	0.003
No	214	(71.6)	148	(59.7)
Health behaviours
Exercise frequency (n=550)
No exercise	63	(20.9)	27	(10.9)	9.9	0.007
Every day	114	(37.7)	107	(43.1)
Weekly or monthly	125	(41.4)	114	(46.0)
Active seeking of health information (n=549)
No	63	(20.9)	22	(8.9)	26.3	<0.001
Sometimes	172	(57.1)	130	(52.4)
Always	66	(21.9)	96	(38.7)
Health examination in past year (n=539)
Yes	168	(56.8)	156	(64.2)	3.1	0.079
No	128	(43.2)	87	(35.8)
Searching online health information (n=548)
Yes	75	(24.9)	107	(43.3)	20.7	<0.001
No	226	(75.1)	140	(56.7)
Health outcomes
Assistance while visiting a doctor (n=549)
Need assistance	50	(16.6)	11	(4.4)	20.4	<0.001
No assistance needed	251	(83.4)	237	(95.6)
Diabetes mellitus (n=549)
Yes	58	(19.21)	34	(13.77)	2.9	0.090
No	244	(80.79)	213	(86.23)
Hypertension (n=549)
Yes	155	(51.3)	101	(40.9)	5.9	0.015
No	147	(48.7)	146	(59.1)
Self-care (n=550)
Dependent	20	(6.6)	4	(1.6)	8.2	0.004
Independent	282	(93.4)	244	(98.4)
Activities of daily living (n=551)
Having difficulty	32	(10.6)	8	(3.2)	10.9	0.001
No difficulty	271	(89.4)	240	(96.8)
Anxiety (n=548)
Yes	63	(20.9)	32	(13.0)	5.8	0.016
No	239	(79.1)	214	(87.0)
Participation
Attending health classes (n=547)
No	189	(62.8)	112	(45.5)	23.1	<0.001
Sometimes	105	(34.9)	111	(45.1)
Always	7	(2.3)	23	(9.4)
Empowerment
Medication (n=548)
Without prescription	19	(6.3)	4	(1.6)	7.3	0.007
With prescription	283	(93.7)	242	(98.4)
Self-management during illness (n=548)
Yes	67	(22.2)	80	(32.5)	7.4	0.007
No	235	(77.8)	166	(67.5)
Seeking a doctor (n=547)
Yes	228	(75.5)	167	(68.2)	3.6	0.057
No	74	(24.5)	78	(31.8)

These 24 factors including personal determinants, situational determinants, socioenvironmental determinants, health service use, health costs, health behaviours, health outcomes, participation and empowerment were entered in multiple logistic regression, as shown in table 2. Limited HL was significantly associated with less health service use or self-paid vaccination in preventive medicine, such as not having a family doctor (adjusted OR (AOR) 1.46, 95% CI 1.00 to 2.14) or not receiving self-paid vaccination (AOR 1.28, 95% CI 1.05 to 1.57). In addition, older adults with poorer health behaviour regarding the items related to searching online health information (AOR 1.24, 95% CI 1.01 to 1.52), less social participation while attending health classes (AOR 1.38, 95% CI 1.12 to 1.70) and poor empowerment of self-management during illness (AOR 1.28, 95% CI 1.05 to 1.57) had limited HL. Moreover, poorer health outcomes in older adults, such as having difficulty in performing daily living activities (AOR 1.58, 95% CI 1.00 to 2.52) and requiring assistance while seeing a doctor (AOR 1.70, 95% CI 1.16 to 2.48), may be associated with limited HL. In particular, older adults whose dominant dialect was a dialect other than Mandarin had a higher odds of having limited HL (AOR 2.13, 95% CI 1.72 to 2.64; table 3). The indicators of model performance revealed a reasonably good fit in the training data set, including an acceptable pseudo R2 value of 0.27 and nonsignificance (p=0.923) in the Hosmer-Lemeshow test.

Table 3

Estimations and statistics of selected predictors as revealed by multiple logistic regression

Determining factors	β	SE	Adjusted ORs (95% CIs)	P value	HL point
Socioenvironmental determinants
Dominant spoken dialect
Taiwanese, Hakka or other dialect	0.76	0.11	2.13 (1.72 to 2.64)	<0.001	+1
Mandarin	Ref.
Health service use
Having family doctors
No	0.38	0.19	1.46 (1.00 to 2.14)	0.049	+1
Yes	Ref.		1.00
Health costs
Self-paid pneumonia vaccination
No	0.25	0.10	1.28 (1.05 to 1.57)	0.016	+1
Yes	Ref.		1.00
Health behaviours
Searching online health information
No	0.21	0.10	1.24 (1.01 to 1.52)	0.039	+1
Yes	Ref.		1.00
Health outcomes
Assistance while visiting a doctor
Need assistance	0.53	0.19	1.70 (1.16 to 2.48)	0.006	+1
No assistance needed	Ref.		1.00
Activities of daily living
Having difficulty	0.46	0.24	1.58 (1.00 to 2.52)	0.052	+1
No difficulty	Ref.		1.00
Participation
Attending health classes
No	0.32	0.11	1.38 (1.12 to 1.70)	0.003	+1
Yes	Ref.		1.00
Empowerment
Self-management during illness
No	0.25	0.10	1.28 (1.05 to 1.57)	0.016	+1
Yes	Ref.		1.00

The goodness of fit is measured by McFadden’s R2=0.27 and p value of 0.92 in the Hosmer-Lemeshow test (n=552, 85%).

HL, health literacy.

Eight variables were selected using the forward stepwise selection method in the multivariate model and were used to calculate the HL score. Cross-validation was performed by including 15% participants (n=96/648), of whom 92 provided all responses for the measurement of prediction accuracy. The overall accuracy in predicting limited HL with various cut-off points is listed in table 4. The optimal cut-off point was considered to be 5, yielding sensitivity and specificity of 62.0% and 76.2%, respectively. By using a score of 5 of 8 to predict the limited HL level, the obtained PPV and NPV were 75.6% and 62.7%, respectively. Figure 2 presents the predictive ability of the scoring algorithm among older adults in the test data set. Figure 3 shows the indicators of model performance revealed reasonably satisfactory performance with an AUC of 0.71 (95% CI 0.61 to 0.81).

Table 4

Overall accuracy of limited health literacy classification with various cut-off points (optimal cut-off=5)

Cut-off value	Test data set (n=96, 15%)*
	Overall accuracy† n (%)	Sensitivity %	Specificity %	Positive predictive value %	Negative predictive value %
1	50 (54.3)	100.0	0.0	54.3	NA‡
2	51 (55.4)	100.0	2.4	54.9	100.0
3	53 (57.6)	90.0	19.0	57.0	61.5
4	57 (62.0)	74.0	47.6	62.7	60.6
5	63 (68.5)	62.0	76.2	75.6	62.7
6	58 (63.0)	38.0	92.9	86.4	55.7
7	44 (47.8)	4.0	100.0	100.0	46.7
8	43 (46.7)	2.0	100.0	100.0	46.2

*Only 92 participants finished all responses in the measurement to present the classification accuracy.

†Agreement between predicted and observed level health literacy (limited or sufficient).

‡NA: not available; denominator is zero.

NA, not available.

Figure 2

Age-specific health literacy levels.

Figure 3

ROC curve and c-statistics of the fitting test in the test data set. The AUC was 0.71 (95% CI 0.61 to 0.81), indicating acceptable discrimination. AUC, area under the curve; ROC, receiver operating characteristic.

Discussion and conclusion

Discussion

To the best of our knowledge, this is the first study to develop a model for predicting the HL level of community-dwelling older adults. This algorithm-based model was well calibrated by integrating HL-related factors from the model of the HLS-EU Consortium and was useful for HL risk prediction among older adults. In addition, this model exhibited a moderate ability for discriminating between older adults with sufficient HL and limited HL.

In this study, we integrated variables associated with both medical and public health perspectives from the aforementioned HL model of the HLS-EU Consortium and proposed a simple scoring algorithm. The scoring system dichotomises older adults into high-risk (cutoff ≥5) and low-risk (cutoff <5) populations to maximise the sensitivity and specificity of the prediction of limited HL. Therefore, early identification using five or more factors from a total of eight significant HL predictors among older adults can be an effective strategy that can be implemented in future clinical practice. On the basis of proposed cut-off points, among the 92 older adults included in the test data set, 63 (68.5%) with a cut-off value of ≥5 were recommended to undergo further HL intervention, although only 31 (62.0%) actually had limited HL, resulting in a PPV of 75.6%. Given the importance of early identification and strategy provision for community-dwelling older adults with a high risk of limited HL, the proposed scoring algorithm can be considered useful in community practice.

A previous study proposed using 80%–90% of data to build a prediction model and 10%–20% of data to validate the model.38 In our study, we followed two previous studies and accordingly divided our data into a training data set (by using 85% of the data) and a test data set (by using 15% of the data).39 40 In addition, we randomly selected 10% (n=64) and 20% (n=128) of the data to examine the reliability of dividing our data into 85% and 15% in our study. Similar validation results were obtained when we included 10% (n=64), 15% (n=96), and 20% (n=128) of the participants. The overall accuracy was 75.0% (48/64), 68.5% (63/96) and 64.8% (83/128) for 10%, 15%, and 20% of the participants, respectively, indicating that the score of 5 was the optimal cut-off point.

This conceptual framework integrating medical and public health perspectives developed by the HLS-EU Consortium is suitable for determining the most relevant determinants of the HL level in older adults. Eight predictors were identified to be significantly associated with the HL level: one socioenvironmental determinant (ie, dominant spoken dialect) and seven HL-related factors including health services (ie, having a family doctor), health cost (ie, self-paid pneumonia vaccination), health behaviours (ie, searching online health information), health outcomes (ie, assistance while visiting a doctor and performing activities of daily living), participation (ie, attending health classes) and empowerment (ie, self-management during illness). The results for the seven identified predictors of HL-related factors are consistent with those of previous studies, for example, having a family doctor,8 costs for self-paid vaccination,41 searching online health information,10 functional status such as difficulty in performing daily activities and assistance while visiting doctors,41 42 participation in health classes43 and self-efficacy in disease management.44 However, our study result revealed that personal and situational factors did not affect HL among older adults. Previous studies have reported that personal determinants, namely age, educational level and working status, as well as situational and environmental determinants, including marriage and residential area, were significantly associated with the HL level.15 19 20 45 This difference might be because personal and situational determinants were the proximal factors of HL that are affected and displaced by a more distal and upstream factor (societal and environmental determinants).46

Some studies have developed a weighted score based on multiple regression to determine the HL level. These studies have used originally estimated beta values or have transformed the beta values into a score of multivariate regression.19 20 47 For example, Miller et al used 20 variables to establish a nonequal weighted HL scoring system for older adults, and their scoring system correctly classified the HL of 73.2% of the participants.20 Our simple scoring algorithm referred to associations (positive and negative) between the HL level and risk factors with equal weighs (+1 or −1) to identify individuals with limited HL.31 Our algorithm is rapid and straightforward to use and can be widely implemented for determining the HL of older adults in clinical practice. Our results from the eight questions (correctly classified the HL of 65.8% of the participants) can help rapidly determine the target intervention tailored for a specific older adult.

Users can rapidly predict limited HL through an evaluation of their HL-related personal, situational and environmental factors as well as health behaviour and outcomes by using our algorithm and thus identify community-dwelling older adults who may require further health assistance. Hospitalisation and mortality due to poor HL in older adults can be prevented through early identification and intervention. Therefore, this assessment tool should be promptly extended to broader communities.

Our study has some limitations. First, in this cross-sectional study, participants were recruited from northern, central and southern Taiwan by using the convenience sampling method. Therefore, potential selection bias might exist. In addition, although we included comprehensive HL-related factors on the basis of a conceptual model of HL, some crucial situational or socioenvironmental factors related to the situational demand or organisational environment might be difficult to measure because information on the organisational environment was not available in our individual interview study. Second, this study relied on the 47-item HLS-EU-Q self-reported questionnaire for determining HL. Additional objective HL assessments might be required to recognise functional HL for preventing the potential for outcome misclassification bias. Third, the high prevalence rate of limited HL (54.9%) in our sample may affect the prediction ability (ie, PPV) of this algorithm when applied in other populations. Therefore, when this algorithm is applied to a population with a lower prevalence of limited HL, older adults with positive results of limited HL may in fact have sufficient HL. Furthermore, we excluded older adults who could not pass the Mini-Cog screening or follow instructions to complete the assessment. Therefore, our model may not be generalisable to the entire population of older adults. Thus, this model is not recommended to be used in individuals with cognitive impairment or dementia, who may have difficulty understanding instructions. Larger population studies with prospective longer-term outcome measures are necessary to validate our study findings.

Conclusion

We proposed a simple clinical scoring algorithm with substantial sensitivity and satisfactory specificity to assess the risk of limited HL among community-dwelling older adults.

Practice implications

This scoring algorithm can not only help clinicians to assess and identify the HL level among older adults but also assist researchers to establish intervention strategies for older adults with limited HL. However, for further population-based application for the early detection of older adults with a high risk of limited HL, prospective trials should examine the implementation and utility of this algorithm in the community.