Psychometric Properties of a New Questionnaire Assessing the Mothers' Knowledge and Perception about the Oral Health of Their School Children.

Objectives: The aim of the present study was to develop and evaluate the psychometric properties of an instrument in Persian to assess the mothers' knowledge, and perception about oral health of school children. Materials and
Methods: A sequential exploratory mixed method design consisting of qualitative and quantitative phases was performed. We developed the questionnaire by inductive-deductive method, through a synthesis of literature review and a qualitative study with semi-structured interviews and focus group discussions. Face and content validity of the items were assessed by consulting a panel of 11 experts. In the quantitative phase, an exploratory factor analysis was performed using data from a cross-sectional study with a sample of 303 mothers. Reliability analysis with test-retest approach and Cronbach's alpha coefficient and intra-class correlation coefficient (ICC) was done.
Results: Pre-final version of the scale consisted of 120 items extracted from the qualitative study and literature review. After content and face validity, 92 items were chosen with the greatest agreement between experts, with a content validity index (CVI) >0.8 and content validity ratio (CVR) of 0.59. The final questionnaire covered 62 items. The overall Cronbach's alpha was 0.94 and it ranged from 0.87 to 0.97 for the subscales. The ICC ranged from 0.91 to 0.98 (Cronbach's alpha ≥0.70).
Conclusion: The present study introduced a valid and reliable questionnaire for assessment of the mothers' perception regarding school children's oral health. It can be used as a standardized measure for public health surveillance and evaluation of oral health promotion programs.

Introduction

Oral diseases are associated with particularly high economic burden in disadvantaged groups in developed and developing countries. Children are dependent on their caregivers to receive oral health services. Therefore, caregivers should obtain the necessary information and provide children with preventive and restorative care [1].

According to Kay and Locker [2], it is important to determine the mothers' understanding of oral health problems in order to improve children's oral health and modify their behaviors. Dental caregivers can focus on issues, which need to be addressed in the development of educational contents for the parents and caregivers and prevent pediatric oral diseases and improve oral health.

Observational studies have shown an association between the parental oral health-related knowledge, beliefs, and attitudes and their children's future oral health practices [3-5]. Adequate parental oral health-related knowledge and positive beliefs, and attitudes have a positive influence on the children’s dental and gingival health [6,7]. Previous studies have emphasized on the role of mothers in relation to the children’s oral health habits and status [8,9].

There is a little difference between “knowledge” and “perception”. Knowledge reveals scientific information, while perception shows beliefs formed by individual attitudes, prospects, and cultural factors [10]. The words “attitude” and “belief” are different, but occasionally they may be used in place of each other. A "belief" about something is needed to develop "attitude" towards it [11].

A valid and reliable assessment tool is essential to ensure accurate appraisal of mothers’ perception about children’s oral health, which is an important predictive factor of children’s oral health behaviors, and to develop effective oral health promotion programs.

A review of the related literature shows that currently, there is no comprehensive, valid and reliable instrument in Persian to assess the mothers’ perception about different concepts of children’s oral health, which was the motive for conduction of the present study. Different tools have been designed and used in previous cross-sectional studies to measure the level of mothers’ knowledge and attitude regarding the preschool children’s oral health [12], or assess their perception in controlling children’s sugar consumption [13], or their beliefs about children’s oral health [14]; while, the present study adopted a sequential exploratory mixed-method design to develop a new wide-range questionnaire to explore the mothers’ perceptions. Therefore, the purpose of the present study was to develop an instrument in Persian for assessment of the mothers’ knowledge and perception regarding the oral health of their school-age children and to evaluate and explore the psychometric properties of this instrument.

MATERIALS AND METHODS

Ethics statement:

The study was approved by the Ethics Committee of Tehran University of Medical Sciences (Ethics Number: 9021431004-1). All the participants were informed about the purpose of the study. Oral consent was obtained from the participants. The data were collected anonymously, and all the identifying information was deleted. Ethical considerations were taken into account in all the interviews, as well as the publication.

Study design and setting:

In this study conducted from April to May 2015 in Tehran, Iran, an exploratory, sequential, mixed-method design was applied, based on the guidelines by Waltz and colleagues [15] in two phases of qualitative and quantitative, and the target population included mothers who had first-grade primary school children selected from the primary schools in different districts of Tehran.

Qualitative phase:

A descriptive qualitative approach utilizing semi-structured interviews and focus group discussions with mothers, and a review of literature were undertaken to identify the tool domains. A deductive-inductive approach was used to generate the item pool and domains. Item pool was created initially by the content of the transcripts consisting of perceptions, beliefs, views, and experiences of the mothers regarding children’s oral health (inductive approach). This pool was revised according to the existing literature and supplemented by the emerging new items (deductive approach).

The result of this step was preparation of a comprehensive list of factors related to children's oral health. Then, after preparing the item pool, at this stage, item reduction was done, and the items were combined to form the tool by accessing the psychometric range of the questionnaire.

Questionnaire design (psychometric properties):

Psychometric properties refer to the validity and reliability of a tool. Accordingly, after generating the initial pool, a panel of experts (n=11) with at least 5 years of academic experience in the fields of community oral health (n=7), epidemiology (n=1), and pediatric dentistry (n=3) assessed the content and face validity of the first version of the questionnaire.

Face and content validity:

According to McKenzie et al, [16] a comprehensive procedure for the assessment of content validity was performed through both quantitative and qualitative processes. The experts evaluated the qualitative content validity to revise, eliminate, or add items to the questionnaire. For quantitative review, they were asked to comment on each question to calculate the content validity index (CVI) and content validity ratio (CVR). The results were quantified by the calculation of the CVR and CVI for each item (I-CVI) and scale (S-CVI).

CVR measures the essentiality of an item. To calculate CVR using Lawshe’s approach [17], each question was coded as “essential”, “useful but not essential”, or “non-essential” to determine the significance of items from the primary pool. Given the total number of experts in the panel, according to the Lawshe’s guideline, when there are 11 experts, the CVR must be more than 59% (CVR score ≥59%) to be considered satisfactory and acceptable. According to this rationale, items which were regarded as relatively unimportant were discarded.

For CVI, each question was rated on a four-point ordinal scale along the item-rating continuum [18]: 1, irrelevant; 2, somehow relevant; 3, quite relevant; 4, highly relevant. The formula is I-CVI = (N3,4)/(N), where N3,4 is the sum of the number of experts who indicate each item as quite relevant or highly relevant (scores 3 and 4) divided by the total number of experts (N). Actually, CVI is the ratio of experts agreeing on the relevance of each item regarding simplicity and clarity [19].

When there are 11 experts, the I-CVI must be 81.82, according to Lynn’s guideline which states that an I-CVI score ≥81.82 is to be considered satisfactory and acceptable [20]. The S-CVI was defined as the proportion of items given a rating of either 3 or 4 (moderately and highly relevant) by all the raters involved, divided by the total number of items [15]. Overall, S-CVI ≥ 0.80 is regarded suitable [20].

Next, 10 mothers as laypeople were invited to fill out the questionnaire, to assess the flow of the questionnaire, and to identify the possible difficult-to-understand items. Subsequently, they were interviewed by a researcher to indicate any ambiguity experienced in responding to the questionnaire and interpreting the readability of the items. After the interviews, the subjects were questioned to determine unclear or redundant items. Their feedbacks were considered to generate new items if needed. In fact, for the assessment of the face validity, the grammar, wording, item allocation, syntax, organization, scaling, appropriateness, and logical sequence of the statements were evaluated.

Construct validity:

To perform the quantitative phase of the study, the questionnaire was distributed among 350 mothers; the response rate was 90.86% (n=318). The Tabachnick and Fidell’s [21] suggestion on sufficient sample size for factor analysis is 300 participants.

A multi-stage random sampling method was applied. Firstly, Tehran city was divided into separate strata; north, south, west, east, and central area. Then, one district was selected from each stratum, through simple random sampling. Finally, the random selections of primary schools from each district (two primary schools for boys and girls) as well as one first-grade class from the primary schools included in the random sampling process were conducted.

An exploratory factor analysis (EFA) was performed using principal components analysis to extract the main factors of the questionnaire and identify the latent dimensions of the tool. Actually, EFA examines the internal relationship between variables to discover the classes of variables that are most related to each other for factor detection in studies [22,23].

In order to develop the questionnaire, factor loadings and correlations were examined after eliminating redundancy for the clinical samples. In this way, the highest correlated items in each factor can be used as items to explain each factor or component of the tool. Varimax rotation was conducted to achieve a simpler structure and check the degree of conformity and the naming of the extracted factors. For the improvement of interpretability, rotation is generally applied to reduce low-factor loadings and increase high-factor loadings.

The Kaiser–Meyer-Olkin and Bartlett’s test of sphericity (to test the hypothesis of sufficient correlation among the variables) were used to assess the adequacy of the sample size for the EFA. The number of factors in the tool was determined based on the factors with eigenvalues greater than one and the result of the scree plot. The eigenvalue is a measure of how much of the variance of the observed variables a factor explains [24]. The size and pattern of the factor loadings through the Varimax rotation anti-image correlation matrix helped to decide regarding the number of items underlying the factors [24]. Items were excluded if they met each of the following criteria: (i) an eigenvalue less than 1, (ii) item-total correlation less than 0.3, (iii) the Cronbach’s alpha increased “if item deleted,” and (iv) factor loading less than 0.4 on a single factor [24].

In case similar items showed different factor loadings, they were designated to the most suitable factor with respect to the item’s nature. It means that if a question has an acceptable factor loading (more than 0.4) in two factors, it will be located in a factor which matches more with respect to the item’s nature. For example, bruxism matches more with factor: “the causes of oral disease” by nature, although the factor loading may be acceptable for other items as well.

Test-retest reliability:

By measurement of test-retest reliability and internal consistency, reliability of the final version was confirmed. The Cronbach’s alpha coefficient was determined to measure internal consistency; values above 0.70 were considered acceptable [25].

The test–retest method was applied to evaluate the stability of the questionnaire. Stability means getting the same results from the same person if the test is repeated. For this purpose, 30 mothers completed the questionnaire twice with a two-week interval, then the intra-class coefficient (ICC) [26] was measured. All the questionnaires were coded to merge the test-retest data. Participants were requested to write a unique code on the top of their own questionnaires. ICC with a value of at least 0.4 was considered acceptable [27].

Scoring and weighting:

All the questions were assessed on a 5-point Likert scale, with response categories ranging from “completely agree” to “completely disagree.” A Likert scale is commonly used to measure attitudes, knowledge, perceptions, values, and behavioral changes [28]. It is an important part of surveys and public health research. For instance, it can be used to assess the public knowledge and awareness about a public health issue [29].

To score the questions of the final questionnaire, the weight of each question was indicated based on its factor load and the variance resulting from the factor analysis using SPSS [30]. The domain scores were calculated by summing the multiplication products of response code and the weight of each question.

Results

Questionnaire design:

Based on the results of the qualitative phase and after removing the redundant information, we obtained an initial list of 120 items. The first version of the questionnaire was organized into eight domains: 1, the definition of oral health (7 questions/item); 2, the importance of tooth maintenance and treatment (16 items); 3, the effect of nutrition on children’s oral health (9 items); 4, the role of oral health behaviors (32 items); 5, the causes of dental caries and malocclusion/ crowding (25 items); 6, the symptoms and causes of periodontal disease (7 items); 7, the causes of halitosis (5 items); and 8, the mothers’ role in children’s oral health (19 items).

Face and content validity:

The CVR of the items in the questionnaire was between −0.27 and 1, and the items with a score less than 0.59 were excluded [17]. Based on the experts’ opinion, of the 120 selected items, 31 items were deleted, and the 89 remaining items were checked for CVI.

In the present study, all the items had a satisfactory I-CVI score (≥81.82). The S-CVI of the items in the questionnaire was between −94.69 and 100, and no items were deleted accordingly.

For the face validity and qualitative content validity, we received some suggestions about three questions to reframe each of them into two questions and edit some words. Therefore, after computing the face and content validity, the 92 remaining items were checked for construct validity.

Construct validity:

For the construct validity, the EFA was conducted with a sample of 350 mothers who had a first-grade primary school child (n=318; response rate=90.86%). The univariate outliers were screened, and out-of-range values were identified and recoded as missing data. A univariate outlier is an observation or a measure with an extreme value that located away or disconnected from the majority of the observations [31].

The data of 302 participants who completed the questionnaire were used for the analysis. The mean age of the participants was 34.90±4.76 years (range: 24 to 48 years).

The Kaiser–Meyer-Olkin was 0.86 (greater than 0.6), and Bartlett’s test result was significant (X2=12291.221, df=4186, P<0.001), which indicated the adequacy of the sample size for EFA.

According to the initial EFA, eight factors with eigenvalues greater than one, that jointly accounted for 40.20% of the observed variance, were irrelevant. After Varimax rotation, 26 items which had factor loading less than 0.4 were deleted. Four items were removed from the questionnaire; although they had an acceptable factor loading (>0.4), they did not fit into the other items. Finally, 62 items remained in the questionnaire.

The result of the quantitative EFA showed that eight factors/domains could explain the structure of mothers’ viewpoint on factors related to the oral health of children. These domains in the final 62-item questionnaire included: 1, the definition of oral health (3 items); 2, the importance of tooth maintenance and treatment (10 items); 3, the importance of the teeth (permanent first molars) (5 items); 4, the symptoms and causes of tooth decay and its preventive measures (16 items); 5, the causes of malocclusion/ crowding (6 items); 6, the causes of halitosis and periodontal disease (8 items); 7, common beliefs/misconceptions (7 items); and 8, the role of mothers in maintaining and improving oral health of children (7 items).

Test-retest reliability:

For the test-retest reliability, 30 mothers completed the questionnaire twice with a two-week interval. They were between 21 to 49 years (mean age: 34.87 ± 0.28 years). The majority of them (85%) had a high school diploma or higher level of education, and 25% had a full- or part-time job.

The 62-item questionnaire showed excellent internal consistency based on the reliability analysis (α=0.92), and the internal consistency of domains varied from 0.87 to 0.97. The stability of the scale over time was investigated by calculating the ICC, which was 0.92 for the total questionnaire and varied from 0.91 to 0.98 for the domains. These results indicated appropriate stability of the present questionnaire.

Scoring and weighting:

In the present study, for scoring of the questions, each question was weighted. The final 62-item questionnaire consisted of the following constructs with their possible score ranges: 1, the definition of oral health (3 items, 8 to 40); 2, the importance of tooth maintenance and treatment (10 items, 29 to 145); 3, the importance of teeth (permanent first molars) (5 items, 9 to 45); 4, the symptoms and causes of tooth decay and its preventive measures (16 items, 20 to 100); 5, the causes of malocclusion/crowding (6 items, 6 to 30); 6, the causes of halitosis and periodontal disease (8 items, 9 to 45); 7, common beliefs/misconceptions (7 items, 7 to 35); and 8, the role of mothers in maintaining and improving oral health of children (7 items, 7 to 35).

Accordingly, the total score of the final 62-item questionnaire ranged from 95 to 475. Scores 95 to 133 were considered as the floor and 437 to 475 as the ceiling. The ceiling effects show the percentage of participants who are scored in the top 10% best possible score of the instrument. The opposite is the floor effect [32]. No ceiling and floor effects existed as less than 10% of the participants scored less than 133 or more than 437.

Discussion

In the present study, a new tool to assess the mothers’ perceptions about factors affecting children’s oral health was designed and evaluated using the exploratory mixed-method design. If the researcher seeks to develop an instrument when none is available, this design of combining qualitative and quantitative research components can be of particular usefulness to expand and strengthen a study’s conclusions [33,34]. In addition, when the research problem is qualitative, the exploratory design can be highly efficient [33]. Since each tool/scale is designed for a special population, situation, or special goal, it may be invalid in other cultures or situations. To the best of our knowledge, this is the first comprehensive study designing and validating a tool for mothers’ perceptions about different aspects of pediatric oral health. The questionnaire was shown to be well understood and accepted by mothers. According to test-retest, the responses showed suitable reproducibility.

In the present study, a multi-stage sampling was performed to find a representative sample of mothers. The key strength of the study was that we used a mixed method to obtain an in-depth understanding of the mothers’ perception. The results showed that all the dimensions of the questionnaire, which were based on the qualitative phase, were in accordance with the experiences of the participants in the quantitative part.

All the characteristics of the participants in the qualitative phase were considered for the selection of the participants in the quantitative phase. Also, the time interval from the qualitative to the quantitative phase was less than one year, and the location of the study, as well as the cultural and social structure of the participants, were the same in both phases. Therefore, the generalizability of the data is of value.

According to the values cited by Baumgartner and Chung [27], the ICC values obtained in our study for the total points and questionnaire domains were satisfactory. Therefore, the questionnaire has good validity and reliability. It can be used in different situations by oral health researchers and policymakers. It can adequately assess the knowledge and perception of mothers regarding oral health and present information in different research projects. Although the scale can be accepted as a comprehensive and practical one for Iranian mothers, more evaluation may be necessary for an adjustment in different cultures.

Even though the qualitative phase provided rich data, the study had some limitations. The criterion validity of the questionnaire should be evaluated in future studies. Also, we suggest comparing the children’s clinical oral health outcomes and the viewpoints of mothers. In this study, EFA was conducted to extract the new factor structure from the dataset. We suggest conducting confirmatory factor analysis and discussing model fits in the future.

Different studies have been designed and performed as qualitative or quantitative cross-sectional studies to measure the level of mothers’ knowledge and attitude about the oral health of preschool children [10], to assess their perception in controlling children’s sugar consumption [11], or their beliefs about children’s oral health [12]; while, the present study was a mixed method research which aimed to develop a new wide-ranging questionnaire to explore mothers’ perceptions. Thus, comparison of the findings cannot be quite reasonable; mainly because of the differences in the methodologies.

CONCLUSION

The questionnaire designed in this study is valid and reliable to assess the mothers’ knowledge and perception about the oral health of school-children and their role in this respect. Therefore, it can be used in future research and interventions with a reliable certainty as a valid and sustainable instrument in assessing the knowledge of mothers regarding the oral health of their children.