A psychometric network perspective to oral health literacy: Examining the replicability of network properties across the general community and older adults from Brazil.

OBJECTIVES: To evaluate the replicability of oral health literacy (OHL) network models across the general community and a sample of older adults from Brazil.
METHODS: Data were obtained from two oral health surveys conducted with a total of 1138 participants. OHL was measured using the short form Health Literacy in Dentistry scale (HeLD-14). A regularized partial correlation network was estimated for each sample. Dimensionality and structural stability were examined via exploratory graph analysis. Network properties compared included global strength, edge weights, and centrality estimates. Model replicability was examined fitting the general community model to the older participants' data.
RESULTS: Six dimensions with the exact same item composition were detected in both network models. Only the Receptivity domain in the older adults sample yielded low structural stability. Strong correlations were observed between edge weights (τ: 0.68; 95% CI: 0.62-0.74) and between node strength estimates (τ: 0.63; 95% CI: 0.36-0.89). No statistically significant differences were found for global strength. The fit of the older adults sample to the HeLD-14 network structure of the general community sample was satisfactory.
CONCLUSION: Network models OHL replicated across the general community and a sample of older adults. The psychometric network approach is a useful tool to evaluate the measurement equivalence of OHL instruments across populations.

INTRODUCTION

Oral health literacy (OHL) has been a flourishing field of research in the dental literature for the past 30 years.[ ] The concept of OHL is framed through the broader lens of health literacy (HL) and underlines the importance of understanding basic health information to make informed decisions that ultimately impact oral and general health.[ ] The seminal 2004 Institute of Medicine Health Literacy Framework, later adapted to the context of OHL, set the foundations for relevant scientific contributions in the development of renewed health promotion strategies, mitigation of inequalities in oral health, and the integration of oral and general health approaches.[ ] Evidence has demonstrated that low OHL is an important predictor of poor oral health status, dental anxiety, and other factors related to accessing dental treatments.[ , , ]

OHL and HL are evolving and interconnected constructs.[ , ] Whereas early studies focused on the accessibility and comprehension of written health information, theoretical frameworks and areas of research soon expanded to cover a comprehensive range of topics including barriers to accessing healthcare, health system navigation, processes of decision‐making, and cultural aspects of health services.[ , ] Current critical perspectives to OHL address participants' skills and competencies, but also sociopolitical environments, media messages, levels of engagement in communication processes between patients and health providers, and organizational HL.[ ]

Measuring OHL in a comprehensive way and with meaningful implications for policy development remains a significant challenge. A substantial number of OHL psychometric instruments have been developed based on different frameworks, although most include a limited number of facets of the constructs such as patients' comprehension of clinical terms.[ ] The Health Literacy in Dentistry (HeLD) scale is one of the few available instruments that offer an assessment of OHL across a broader range of skills and competencies, from findings and accessing dental treatment to obtaining social support to go to an appointment.[ ]

The HeLD scale was originally designed for the context of Indigenous Australians, an underserved group with a substantial level of unmet health needs, and has since been translated and adapted to other settings and populations.[ , ] The implications of OHL research are particularly relevant for groups that face a greater burden of oral health conditions and populations with high levels of health disparities. Yet, a disproportionately higher share of OHL studies is conducted in high‐income countries and the majority of tools were developed in English, which raises concerns about a potential lack of cultural and linguistic sensitivity in multicultural and non‐English‐speaking contexts.[ , ] Brazil is a unique case of a densely populated and diverse country with marked social gradients in oral health and a dynamic healthcare system.[ ] A relatively recent expansion of public dental services has contributed to facilitating the access of lower‐income groups to dental treatments, although important barriers remain to be overcome.[ ]

Examining the psychometric properties of OHL instruments in the Brazilian population is key to ensuring the valid and reliable measurement of the construct. The importance of validating and replicating findings regarding OHL instruments are twofold: (1) it ensures that the instrument can be applied to collect OHL data in different countries/cultures and this will make it possible to compare findings (from Brazil and Australia, for instance); and (2) it informs that the instrument structure is robust. Although cultural differences should always be taken into account, good instruments usually have their structure validated over multiple countries. Reliable measurement of OHL data is essential to enable patients' autonomy and promote effective change in healthcare.

In this study, we applied a novel psychometric network perspective to OHL.[ ] This method offers a powerful approach to model, represent, and communicate multivariate patterns of statistical relationships between systems of variables. Thus, this study aimed to evaluate the replicability of HeLD network models across the general community and a sample of older adults from Brazil, comparing the psychometric properties of the instrument across both populations.

METHODS

This is a cross‐sectional study involving a total of 1138 participants. Data sources included two oral health surveys conducted in Piracicaba, a major city located in the state of São Paulo, Brazil:

Six hundred and three community‐dwelling adult patients aged 18 years or older and registered at Primary Healthcare Units (PHUs) within the public health system in Piracicaba city were recruited. Six PHUs were randomly selected considering the different socioeconomic and geographic characteristics. First, a sample of 523 participants (86.7%) was selected through a simple random sampling procedure based on the information system of the designated PHUs. Another 80 participants (13.3%) who attended a Dental Specialty Center were recruited through a convenience sampling strategy.[ ]

A sample of 535 noninstitutionalized older adults aged 60 years or older were recruited. Initially, 480 independent‐living older adults (89.7%) with no medical records of relevant cognitive disorders were selected using a simple random sampling procedure after consulting the primary health care information system of six Family Health Units in Piracicaba. Another 55 participants (10.3%) who received specialized dental treatment at a Dental Specialties Center within the public health system were selected via convenience sampling.[ ]

Piracicaba is one of the largest cities in the state of São Paulo, with a population of roughly 400,000 individuals. The city has a high level of Human Development Index (0.785), slightly above the state's (0.783) and national (0.765) scores.[ ] The São Paulo state concentrates the highest rates of expenditure per capita on private dental insurance in Brazil and the lowest proportion of households registered in the public primary health care system (45% compared with the national rate of 60%).[ , ]

Ethical approval was obtained from the Research Ethics Committee of the University of Campinas. Verbal and written informed consent were obtained from all participants.

Health Literacy in Dentistry scale

The short form of the Health Literacy in Dentistry scale (HeLD‐14) is a 14‐item instrument intended to measure participants' OHL across seven conceptual domains: access, understanding, support, utilization, economic barriers, receptivity, and communication. Answers to each item are recorded using a 5‐point scale (ranging from 0 to 4). Higher scores indicate better OHL (i.e., greater ability in performing tasks related to finding, accessing, and affording dental treatments, communicating and following professional advice, reading and completing dental forms, and obtaining social support). Scores are calculated as the unweighted sum of the respective item responses.[ , ].

Statistical analysis

Dimensionality analysis

The dimensionality and structural consistency of the HeLD‐14 scale was examined across samples using exploratory graph analysis (EGA) available in the EGAnet R package.[ ]. The network model estimated was the Gaussian graphical model and items conditional associations were modeled as a regularized partial correlation network. EGA applies the Walktrap community detection algorithm to detect the number and composition of subnetworks within the overall structure. In other words, the method employs random walks to identify densely clustered communities of items (dimensions) by computing the intra and intercommunity distances between nodes. A simulation study has demonstrated that EGA performs comparably or superior to traditional factor analytical techniques in the identification of the underlying number and composition of dimensions.[ ]

To investigate the structural consistency of the HeLD‐14 dimensions detected by EGA, a bootstrap method was applied to resample 2000 times and the network was estimated on each bootstrap sample to evaluate replicability. This procedure generated a bootstrap sampling distribution of the number and composition of the HeLD‐14 dimensions, allowing the computation of measures of structural consistency and item stability. Structural consistency provides information on the proportion of times that each dimension from the empirical EGA network was retrieved with the exact same item composition in the bootstrap samples (values range from 0 to 1). It measures the homogeneity and interrelatedness of items in each dimension considering the multidimensional structure of the network. Item stability aid to identify items are the source of low structural consistency within dimensions. Values of item stability are interpreted as the proportion of times that a given item was correctly assigned to the same dimension obtained in the empirical EGA network (values range from 0 to 1).[ ] Relatively low stability (<0.8) is indicative that items might be multidimensional and potentially problematic.[ ]

Centrality estimates

To investigate the relative importance (centrality) of each item to the overall network structure, we estimated measures of node strength. Node strength provides information on how strongly connected to the network a given item is. Strength is calculated as the sum of all edge weights connecting a given node. Values were reported as standardized z‐scores to allow comparison between networks.

We investigated the stability of centrality estimates using a bootstrap case‐dropping procedure that calculates the proportion of participants that could be dropped to still yield a correlation of at least 0.7 with the original strength values. It is recommended that centrality stability (CS) coefficients should not be lower than 0.25 and, ideally, should be greater than 0.50.[ ]

Comparison of models

Several network parameters were compared between the HeLD‐14 models of the general community and the older adults sample. Differences in global strength (the network overall connectivity estimated as the sum of all edge weights) and between specific edges were assessed using the network comparison test with 2000 permutations and a significance level of 5%.[ ]

Network models were also compared using Kendall's rank correlations between edge weights from each specific pair of nodes, and between nodes' strength centrality values. Kendall's τ 95% confidence intervals were computed by means of a bootstrap procedure with 2000 iterations. Coefficients range from −1 to +1 and measure the strength of the similarity between values (coefficients near zero indicate weaker relationships).

Confirmatory network modeling

The replicability of the HeLD‐14 network model across the general community and the older adults samples was formally tested using confirmatory network analysis available via the R package Psychonetrics. The method consisted of first estimating the HeLD‐14 network on the data obtained from the general community sample. To avoid spurious edges, the edges from the network model were pruned at the alpha level of 0.05. Subsequently, the adjacency matrix of the network, which provides information on which edges are present or absent, was retrieved. Finally, the adjacency matrix retrieved from the general community sample was used to fit the same HeLD‐14 network model to the older adult data and test if the HeLD‐14 network model replicated across samples.[ ] Model fit was evaluated using fit indicates that calculate the discrepancy between the observed zero‐order correlation matrix and (network) model implied zero‐order correlation matrix. Adequate model fit was indicated by a nonsignificant χ 2 test, comparative fit index (CFI) > 0.95, and root mean square error of approximation (RMSEA) < 0.07.

Visualization

The network models are visually represented by nodes and edges. Nodes correspond to HeLD‐14 items, whereas edges represent the partial correlations between pairs of nodes conditioned for all other nodes in the network. Edge thickness and color indicate the magnitude and direction of the relationships between nodes. The thicker the edge, the stronger is the relationship between the two nodes. Positive edges are plotted as green lines, whereas negative edges are plotted as red dotted lines. Plots were generated using the R package qgraph.

RESULTS

Sample characteristics are presented in Table 1. Overall, the sample comprising older adults had a greater proportion of participants who identified as white, with low educational attainment, and lower HeLD‐14 scores across all domains compared with the general community sample. Mean age was 43.3 ± 17.2 years and 68.1 ± 7.9 years for the general community and the older adult samples, respectively.

TABLE 1

Description of sociodemographic characteristics and HeLD‐14 scores

Variable	General population (n = 603)		Older adults (n = 535)
	N	%	N	%
Sex
Women	401	66.5	317	59.3
Men	202	33.5	218	40.7
Ethnicity
White	370	61.36	402	70.4
Black	62	10.28	46	8.6
Yellow	14	2.32	1	0.2
Mixed	157	26.04	84	15.7
Education (years)
0	5	0.83	46	8.6
1–8	214	35.61	390	73.0
9–11	240	39.93	62	11.6
12+	142	26.63	36	6.7
HELD‐14	Mean ± SD		Mean ± SD
Receptivity	6.35 ± 2.04		6.23 ± 2.13
Understanding	6.31 ± 2.25		5.57 ± 2.85
Support	6.99 ± 1.86		6.63 ± 2.10
Economic barriers	4.43 ± 2.81		3.58 ± 2.75
Access	6.78 ± 1.95		6.54 ± 2.12
Communication	6.45 ± 2.14		5.75 ± 2.40
Utilization	6.30 ± 2.11		6.01 ± 2.29
Score	43.60 ± 10.53		40.35 ± 11.52

EGA identified six node communities with the exact same item composition in both network models (Figure 1). All communities reflected the structure of the HeLD‐14 conceptual domains, except items of Utilization (X3 and X4) and Communication (C5 and C7), which merged into a single dimension. In both samples, the strongest connections emerged between items of the domain Understanding (U2 and U3) (general community r  = 0.72; older adults r  = 0.83). Edge weights presented a strong relationship across models (τ: 0.68; 95% CI: 0.62–0.74).

FIGURE 1

Communities of nodes (dimensions) of the HeLD‐14 questionnaire detected by exploratory graph analysis [Color figure can be viewed at wileyonlinelibrary.com]

The six‐factor model replicated in 71.2% and 67.4% of the bootstrap samples for the general community and older adults, respectively. Nearly all dimensions yielded high structural dimension values (>0.8), suggesting that the corresponding items are stable and consistently identified in the originally assigned dimension (Table 2). The analysis of item stability (Figure 2) revealed that both items of the Receptivity domain in the older adult sample contributed to lowering the overall structural consistency of the respective domain—that is, both receptivity items had a relatively lower replication rate in the assigned node community (71.2%).

TABLE 2

Structural consistency of EGA dimensions

	Access	Understanding	Support	Economic barriers	Receptivity	Utilization/communication
General community	0.977	1.000	0.997	1.000	0.881	0.960
Older adults	0.981	0.981	0.976	0.980	0.713	0.872

FIGURE 2

Stability of HeLD‐14 items [Color figure can be viewed at wileyonlinelibrary.com]

Estimates of node strength are presented in Figure 3. The stability coefficients of the centrality values were adequate (CS >0.5) for the general community model (CS‐coefficient: 0.594; 95% CI: 0.517–0.672) and the older adult model (CS‐coefficient: 0.516; 95% CI: 0.439–0.594). A strong correlation between node strength estimates was observed across samples (τ: 0.63; 95% CI: 0.36–0.89).

FIGURE 3

Comparison of node strength centrality across network models [Color figure can be viewed at wileyonlinelibrary.com]

No statistically significant differences in global strength were detected between the models (general community: 6.41; older adults: 6.51; diff: 0.10; p‐value = 0.562). The maximum difference observed in edge weight across samples was r  = 0.15 (items X3 and X4).

The fit of the HeLD‐14 network structure of the general community participants to the older adult data was deemed poor (χ 2 (68) = 453.03, p < 0.001; CFI = 0.90; RMSEA = 0.097, 90% CI = 0.089–0.11), so we proceeded to examine modification indices. In an incremental process, we identified three parameters that would significantly improve the model fit if they were freed (edges A3‐U3, S1‐U2, and F3‐R2). After modifications of the original hypothesized model, the χ 2 test remained statistically significant (χ 2 (70) = 241.01, p < 0.001). Nevertheless, the fit of all other indices was substantially improved (CFI = 0.97; RMSEA = 0.064, 90% CI = 0.055–0.073). To assess the effect of the model post‐hoc modifications on the network structure, we used EGA to verify whether the number and configuration of dimensions remained unchanged. The confirmatory model with the three freed edges yielded a six‐dimension structure with the exact same item composition of the empirical models. This indicates that the three new edges added to the model were possibly weak and crossloadings, with little impact to the interpretation and validity of the instrument.

After demonstrating the partial measurement invariance of the HeLD‐14 network model across the samples, we compared the distribution of factor scores using Kernel density plots (Figure 4). The factor scores distributions for Access and Receptivity were very similar for the general community and the sample of older adults, whereas higher factor scores for Understanding, Support, Communication/Utilization, and Economic Barriers were observed among the general community.

FIGURE 4

Factor scores kernel density plots of the HeLD‐14 [Color figure can be viewed at wileyonlinelibrary.com]

DISCUSSION

This study evaluated the replicability of OHL network models across the general population and a sample of older adults from Brazil. We used network measures to compare the psychometric properties of the HeLD‐14 instrument across the two groups. Our findings demonstrated that the HeLD‐14 questionnaire presents stable network parameters, including number and composition of dimensions, edge weights, node centrality, and CS. The fit of the older adults HeLD‐14 data to the general community network model was considered acceptable after minor modifications in three edges.

Network science is an emerging field that provides cutting‐edge methods to model statistical interactions between sets of variables as complex network systems.[ ] A novel network psychometric approach has been proposed in the literature, offering advanced techniques to compare network structures and to assess network measurement equivalence, in addition to providing an alternative interpretation of psychological constructs.[ ] To the best of our knowledge, this is the first study to formally assess the replicability of OHL network models across different samples. Our findings contribute to the dental literature by providing evidence that parameters of HeLD‐14 network models are robust, comparable, and present partial measurement invariance. This is also one of the first studies to use confirmatory network analysis in the field of dentistry.

Notably, the EGA detected identical dimension structures in both samples. Six node communities were identified, with theoretical domains Utilization and Communication merging into a single dimension. This finding reflects the interconnectedness between the skills related to being able to communicate with dental healthcare providers and being able to apply the information given by these professionals. In line with our results, a previous study found a consistent pattern of strong item conditional associations within HeLD‐14 theoretical domains.[ ] Nevertheless, the findings of this study demonstrate that the Brazilian version of the HeLD‐14 presents a slightly distinct structure to the originally proposed 7‐factor composition.[ ] The different number of dimensions is likely to result from measuring attributes of the Brazilian version of the instrument or cultural aspects related to how these constructs (most specifically Utilization and Communication) are understood by this population. It is important to note that 10 out of 14 items conformed to the theoretical dimensions originally proposed by the authors of the instrument (Access, Understanding, Support, Economic barriers, and Receptivity). It is not uncommon for psychometric tools to present contrasting structures when applied to populations with different characteristics. For instance, a single‐factor structure was indicated for the Chinese version of the HeLD‐14 and the long version of the instrument has not shown adequate fit to the theoretically driven 7‐factor structure in the Australian population.[ , ]

Assessing the psychometric properties of questionnaires such as dimensionality is essential to confirm whether findings are valid and directly comparable across populations. Based on the replication of the number of dimensions across both Brazilian samples, in addition to the level of stability obtained from the bootstrap analysis, we found evidence that supports the computation of six HeLD‐14 scores as a valid measure of OHL for this population. On the other hand, findings from different versions of the instrument should be compared with caution, particularly when a different number and composition of dimensions is found. Further research is needed to examine the dimensional structure of the HeLD‐14 scale across different countries.

The highest level of discrepancy in factor scores distributions between the general community and the sample of older adults was observed for Economic barriers. Similarly, lower income has been associated with poor oral health knowledge and lower utilization of dental services among older adults.[ , ] We also detected minor variations between the HeLD‐14 network models. The fit improvement of the confirmatory model observed after freeing edges “A3‐U3,” “S1‐U2,” and “F3‐R2” provides evidence that the relationships between these respective items differ across the general community and the older adults' samples. Post hoc model modification is essentially an exploratory procedure and, as any exploratory procedure, it needs to be interpreted with caution since it can capitalize on sampling variation. On the other hand, the use of MIs guided by theoretical and substantive considerations offers relevant practical guidance on how to detect local misspecification. This could lead to a more robust and complex model than the originally hypothesized structure.[ ] Moreover, compared with the general community data, the boundaries of the HeLD‐14 dimensions in the older adults sample are fuzzier (i.e., lower stability values). The Receptivity dimension in the older adults' sample was the only node community with structural consistency below 0.8, which indicates a greater level of item multidimensionality.

From a public health perspective, high‐quality and psychometrically robust tools are essential to assessing population levels of OHL. Comprehensive valid and reliable measures of OHL provide relevant information to foster health promotion practice and policy development, ultimately contributing to tackling oral health inequalities.[ ] A bibliometric analysis of the OHL literature has shown that instrument development has remained one of the main areas of interest over the past decades, despite most tools being excessively restricted to word recognition and reading comprehension.[ , ] The HeLD questionnaire, on the other hand, measures a range of skills and abilities related to accessing and navigating oral health care, communicating with health providers, affording treatments, following professional advice, and obtaining peer support. The underpinnings of the HeLD questionnaire assume that simply transmitting information is not sufficient to improve patients' levels of OHL. Instead, it is centered on an emancipatory approach that recognizes the importance of promoting autonomy and enabling individuals to actively engage in decision‐making regarding their oral health.[ ]

It is important to bear in mind that it was not our intention to evaluate the usefulness of network analysis as a psychometric approach as this has already been established by previous empirical and simulation studies.[ , ] Furthermore, several limitations should be considered when interpreting the study findings. The OHL data used in our analyses are not representative of the Brazilian population. The surveys from which the data were obtained were conducted in one of the wealthiest regions of Brazil, which presents a relatively lower reliance on the public health system compared with other states.[ ] The general community sample included a proportion of participants aged 60 years or older (24.1%). Although the age distribution was markedly different between populations, there was some level of overlap. We believe the populations included in this study are somewhat homogeneous, which may have influenced the replicability of the models. Furthermore, we were not able to assess measurement invariance across genders and other relevant categories since subgroup analysis would substantially reduce the number of participants for the estimation of each model. This would result in unstable network structures and potentially unreliable findings. Future studies may explore the inclusion of relevant variables to HeLD‐14 network models to assess convergent and divergent validity, and to examine whether the stability of the network structure holds.

In conclusion, HeLD‐14 network models presented stable properties for both the general community and older adults. A six‐dimension structure was identified in both populations, with items from theoretical domains Utilization and Communication merging into a single dimension. Network structures were comparable across samples, demonstrating the applicability of the HeLD‐14 in different contexts and population groups. The Brazilian version of the HeLD‐14 instrument presents similar psychometric properties when measuring OHL across groups with different age distributions.