Protocol of the TOHLA instrument: A Test of Oral Health Literacy in Adults.

BACKGROUND: A proper measurement instrument is selected based on the suggested use of the instrument, the target concept of measurement, and features of measurement (e.g. internal consistency, reproducibility, content and construct validity, responsiveness, and interpretability). Concerning the design and features of measurement, there are not any adequately specific standards for instrumentation to measure oral health literacy (OHL). The present study proposes a protocol that attempts to fill this gap by introducing the psychometrics of a standard questionnaire which measures OHL.
MATERIALS AND METHODS: The present research employs a methodological design and is carried out in Tehran, with data collected through interviews that are held face to face. The data collection procedure involves a review of the related literature, cognitive interviews, fuzzy Delphi Method, and focal groups with participants with OHL work experience for item generation. The target participants of this research are the Iranian adult population and experts working professionally in different health domains. For qualitative data analysis, the content analysis strategy and in the instrument Psychometrics COSMIN checklist will use.
CONCLUSIONS: Achievement of the present research will be used to evaluate the capability of the Iranian adult population in searching, processing, and deciding on healthcare services. This instrument will focus on evaluating both clinical and nonclinical settings. The present research can vastly improve our knowledge of the state of OHL in the Iranian adult population. Copyright:

Introduction

Health literacy is an important factor involved in health-related issues.[12] Besides environmental and genetic factors, individuals’ health behavior plays a pivotal role in preventing diseases and promoting health.[3] A number of factors can cause prevalent oral diseases.[456] In spite of great efforts to improve oral health, still a gap exists between oral health knowledge and practical efforts.[7] As a result, it is maintained that a primary factor involved in oral health is OHL.[89] OHL can be defined as “the degree to which individuals have the capacity to obtain, process, and understand basic oral health information and services needed to make appropriate health decisions.”[10]

Worries about low levels of health literacy made investigators seek for practical methods and instruments to measure patients’ health literacy skills. Formerly, researchers and clinicians viewed patient education as a sign of such skills.[1112131415]

Screening individuals with low levels of OHL is demanding since it is hard to gain information about one's reading ability, comprehension, listening, and other required skills. Educational performance is usually used as a proxy, yet it is usually restricted and might overestimate one's level of OHL. This is because literacy is often several grades below the educational level.[16] So far, 19 instruments have been constructed to assess OHL in the world.[17181920212223242526272829303132333435]

Among the existing instruments, 14 were about dentistry and 5 were about oral/dental health. On the one hand, 12 instruments (19 cases) which measured oral/dental health literacy were restricted to word diagnosis test (short form or rapid estimation) and reading skill. On the other hand, only one instrument among all belonged to Iran. This instrument was derived from rapid estimation of adult literacy in dentistry (REALD)-30, REALD-99, rapid estimation of adult literacy measurement (REALM)-D, OHLI, TOHLA in dentistry (ToFHLiD), and CMOHK. It was only a patchwork and its design was not based on the target concept.[35] As there is no comprehensive instrument to measure oral/dental health literacy in Iran, the present research aims to adopt qualitative methodology to develop and validate such an instrument exclusively for Iran to fill the existing gap in literature and pave the way for a comprehensive exploration.

Materials and Methods

Research design

The procedure involved in this study consists of eleven stages, as shown in Table 1. The article will go on with an elaboration of the methodology (participants, materials, procedure, and statistical analysis).

Table 1

The developmental stages of the Test of Oral Health Literacy in Adults initiative

Stage	Process
1	Expert interviews
2	General interviews with research population
3	Item generation and pretesting
4	Fuzzy Delphi Method
5	Pilot-test survey
6	Psychometric analysis of the pilot test
7	Field test on a large scale
8	Psychometric analysis of field test
9	Test-retest method
10	Responsiveness measurement
11	Focus group discussion for questionnaire scoring

Phase I: Development of the Test of Oral Health Literacy in Adults

Participants

Three separate groups of participants are needed to develop the Test of Oral Health Literacy in Adults (TOHLA). The source of including and the participants and their characteristics vary depending on the specific stage of the project, as described below.

Expert interviews

A wide range of experts in oral/dental health needs to be included for health education, promotion, communications psychology, and social dentistry.

General population-based interviews

A sample of about 20–30 participants should be selected along different demographic features such as age, sex, marital status, education, and expertise.

Survey participants

A second round of selection follows of participants meeting the above-mentioned features now included for psychometric analysis.

Inclusion/exclusion criteria

For interviews with experts, the target participants needed to have previous knowledge and experience of OHL. They are also supposed to be actively involved in a field with the relevant use of OHL (at least 5 years of field specialty). The inclusion criteria in the interview phase also included participants’ willingness to participate and have an Iranian nationality. Those incapable of completing the questionnaires on their own are to be excluded from the study. The required age is 18 years or above.

Procedure

The eleven developmental stages summarized in Table 1 follow a procedure as:

A comprehensive review of the related literature is essential in any research project and though listed as a single activity, it enjoys a continuous nature during the research. As for the interview, the semi-structured type is held in Tehran with 17 professionals with different backgrounds and fields of expertise. They include health education, promotion, communications, psychology, and social dentistry. Items at the core of the interview guide and are as follows:

”Which skills and abilities do people need to act in ways that are beneficial for the health of oral and dental and as well as interacting with the healthcare system?”

Define oral/dental health literacy

What are the potential skills and capabilities one can have concerning oral/dental health literacy?

The other questions are supposed to be asked along with the primary questions.

There has been no measurement designed so far which, from the very beginning, includes a variety of potential users as a central idea. Moreover, the idiosyncrasies of the instrument were discussed in interviews too. These included the preferred methods of scoring and interpretation rules. After conduction, the interview content is transcribed and prepared for content analysis to be used to inform the later stages of the project. Exploratory, open-ended, face-to-face, interviews are used in-depth among a sample of participants with various features. The aim of the interviews is to elicit the significant aspects of OHL.

Stage 1 and 2 interviews are then transcribed whose content will be independently explored by a research team. A panel of experts, subsequently, evaluate items for completeness, ambiguity, and repetition.

Predictably, about 20–30 participants should be interviewed in-depth until data satiation occurs, i.e. “No new themes emerge.”[36] All interviews are recorded, transcribed, and prepared for content analysis. Although there is an interview guide, participants can freely go beyond this and address other relevant issues. Then, the concepts are extracted from the primary structure of the questionnaire.

To rate items for completeness, ambiguity, and repetition, a Fuzzy Delphi Method is used. This would impede misinterpretation of initial expert opinion and provides for real reactions. Here are the steps.[37] Primarily, three (written) Delphi rounds are involved with approximately 20 expert members of different backgrounds in health education, promotion, communications, psychology, and social dentistry.

Here is the design of the Fuzzy Delphi procedure for TOHLA:

Three Delphi rounds are followed in the Delphi procedure of TOHLA. These are outlined in Figure 1.

Figure 1

Fuzzy Delphi flowchart

a. Round (1): Issues and Interview Protocol

The extracted themes get approved by the experts, review of the related literature and general population. Thus, “TOHLA instrument” is developed.

b. Round (2): Development and Distribution of Consensus Survey

TOHLA aims to gain a consensus based on the themes proposed by the experts. Experts should agree with a statement. In Round 2, thus, many experts are involved to rate the responses of the survey.

c. Round (3): Fuzzy Delphi Data Analysis.

There are two result indicators in the FDM for decision-making. First, each expert's responses are changed into triangular fuzzy numbers to find their state of mind for each statement. Then, the defuzzification process follows to set the value that indicates respondents’ extent of consensus. As the last section of data analysis, a framework is designed to help the author to continue with the development process [Figure 1].

Phase II: Psychometric evaluation of Test of Oral Health Literacy in Adults

Pilot-test survey

The long version of the instrument, which is the outcome of Stage 3 and 4 will be submitted to a sample of 500 people with a variety of features. The sample size is a function of the assumption that the instrument has about 40 items. As estimates have it, at least five times as many respondents as the number of items are required for psychometric tests to make sense.[38]

With a typical response rate of 60%–70%,[39] this can lead to a sample of about 300–350 questionnaires, which can allow the rigorous testing of the instrument.

Data analysis in Stage 5 helps to develop a shorter version of the questionnaire using established statistical procedures as explained here in the statistical analysis section. Items that enjoy high face validity, strong correlation with the scale total which they contribute to, and an acceptable rate of reliability make up an aspect of the resultant instrument.

The short version of the measurement developed in Stage 6, should be submitted to 900 participants with a variety of features. With a response rate of 60%–70% (as formerly discussed), there will be yield a final sample size of about 540–630 questionnaires as the return rate.

The data obtained from stage 7 are further statistically analyzed to approve the results of the primary survey in Stage 6. Secondary amendments can be made to the questionnaire at this stage.

Then, the previous respondents are supposed to complete the questionnaire again. If they agree to do so, the “TOHLA instrument” is sent to them one a week after receiving the measure of Stage 6 to estimate test–retest reliability. Then, previous respondents are asked to complete the questionnaire finally after 3 months to check the responsiveness of the instrument.

Statistical analysis

The presence of outliers is checked within the data as well as the normal distribution before any further statistical procedure. Shapiro–Wilk test of normality is run for the distribution of the data.[40] TOHLA Items are checked for floor and ceiling effects based on the criteria set by Terwee et al.[41] Then, a factor analysis follows to test construct validity and explore the related dimensions to particular facets of participation and activity. Item-total correlations, which are corrected for probable overlap, are then calculated among items and the total score they contribute which to. To determine which items conform to a hierarchical uni-dimensional structure, Rasch analysis will follow.[42] Other tests of validity (concurrent and discriminant) are run by estimating the Pearson's product-moment[43] or Spearman rank[44] correlation coefficients (proportional to data distribution) between the TOHLA and the other instruments. Cronbach's alpha coefficient test of internal consistency is run to test the reliability[45] along with the test–retest method which uses intra-class correlations.[46] Standardized effect size is also used to check responsiveness to change.[47] To keep confidentiality, the target data are coded analyzed through thr SPSS version 18 (IBM, Armonk, NY, USA). As for qualitative data analysis, the present study enjoys the content analysis strategy introduced by Graneheim and Lundman run in MAXQDA.

Informed consent and discontinuation/withdrawal of participants from the study

Before any special measures taken in the present research, the participants will sign a letter of informed consent. The written and oral versions of participants’ data and informed consent will be provided for participants. It clearly states that they are allowed to exit the study regardless of later care and commitment whenever they wish and for any reason. Why they leave will be recorded by the researcher in a separate file.

Quality assurance procedures

To analyze qualitative data, four criteria by Lincoln and Guba will be used. These scholars maintained that a piece of research should be trustworthy. Trustworthiness involves establishing credibility, transferability, dependability, and conformability.[48]

Test of Oral Health Literacy in Adults scoring in a focus group discussion

When the final version of the instrument is developed, the research team will form a discussion focus group with experts to score each item of the questionnaire and then its overall score. Focus group discussion (FGD) members will make such decisions as: using simple or weighted mean scores, weight of each section of instrument, multiple-choice responses for each item, direct or reversed scoring of each item, setting the overall range of score for the whole instrument and qualitative categorization of scores for each respondent (e.g. low, moderate, and high). Categorization will be done by an expert statistician who uses rock test to set the number of cutoff points experts in health literacy agreed on. The schematic structure of the research procedures is presented in Figure 2.

Figure 2

Test of Oral Health Literacy in Adults construction and psychometrics flowchart

Ethical considerations

The study design obeys the principles of the Declaration of Helsinki[49] and shows to be in agreement with the ethical principles and national norms/standards for conducting medical research in Iran. To respect participants’ rights, certain measures will be taken such as informed consent to take part in the study, permissions to take notes or record voices, deleting audio files after the research, anonymity of all questionnaires, confidentiality of the data, and financial and nonfinancial rights of the whole research team.

Discussion

There has been increasing research on health literacy or OHL. Yet, the main reasons for low oral/dental health literacy are: inadequate sources of knowledge about oral health, difficult guidelines on oral/dental health and inefficient dentists who are incapable of evaluating patients’ literacy needs (2) in developing countries. The majority of health-related content, including drug labels, after-surgery prescriptions and guidelines on oral health in Iran are in English. That is why it is hard for patients to read and comprehend the information content. Moreover, dentists often use specialist language mostly in English, which adds to the difficulty of doctor–patient communication.

The primary instrument to measure oral/dental health is based on health literacy instruments. As an example, the REALD has its basis in the REALM[21] or a similar case such as the ToFHLiD has come out of ToFHLA.[19] Original instruments elicited similar criticism on general health literacy versions as they were mainly word recognition instruments.[4] These instruments were mostly focused on such places as clinics and healthcare centers and particular populations such as patients.[171819202124]. The present research aims to offer a practical and comprehensive instrument to measure all aspects of oral/dental health literacy with a focus on appropriate content for the Iranian adult population. It takes into account settings outside clinics (i.e. home) to avoid selection bias.

Strengths

This research has certain strengths, including the fact that it develops and evaluates a comprehensive instrument exclusively for oral/dental health literacy in Iran for the first time. To this aim, it employs qualitative methods and reviews the related literature and existing relevant instruments. It also takes into account expert viewpoints as well as those of the adult population. It employs fuzzy Delphi technique and considers expert comments to complement the questionnaire. This technique leads to better recognition besides other traditional qualitative methods. This method has been also employed successfully in a body of health-related research. The present research uses FGD or experts (in health education, social dentistry, and statistics) to score the oral/dental health literacy instrument which were mostly absent in other similar investigations.

Implications

Findings of the present research will be used to evaluate the capability of the Iranian adult population in searching, processing, and deciding on healthcare services. This instrument will focus on evaluating both clinical and nonclinical settings. It will also comprehensively evaluate oral/dental health in dentistry as well as the prevention section (oral health). This instrument can be used in developing and performing educational interventions that affect the level of oral/dental health literacy. It will also act as a valid instrument in prospective investigations to measure oral/dental health literacy of the Iranian adult population.

Financial support and sponsorship

This study was financially supported by Shahid Beheshti University of Medical Sciences Presentation at a meeting.

Conflicts of interest

There are no conflicts of interest.