CRA Grid - A preliminary development and calibration of a paper-based objectivization of caries risk assessment in undergraduate dental education.

CONTEXT: Caries risk assessment (CRA) varies between students and faculty due to the subjectivity inherent in the process as well as in the critical thinking skills required for the processing of information. AIMS: The aim was to develop a paper-based, grid system, CRA-Grid, to objectivize and standardize risk assessment and to assess its rating agreement with the critical thinking process of the teachers. SETTINGS AND
DESIGN: The CRA-Grid was developed and tested in a clinical study, in Indira Gandhi Institute of Dental Sciences, Puducherry.
MATERIALS AND METHODS: Based on the currently available evidence, specific weightages were assigned to the risk factors in an existing CRA form. A 10 × 10 paper grid was created incorporating these weightages. Shading and summing up the respective squares in the grid provided a percentage score. The class interval of percentage indicating the risk status was determined using historical clinical data. After training, the students performed CRA of 57 patients by using the CRA-Grid. Six faculties were blinded to these scores and assessed the risk by critical thinking process. STATISTICAL ANALYSIS USED: Cohen's weighted kappa (k) for inter-rater agreement was run using Graph Pad QuickCalcs.
RESULTS: Cohen's weighted kappa for agreement at 95% confidence interval, between the CRA-Grid and critical thinking process, ranged from "good" to "very good." Mean percentage of agreement of the six faculty was 79.6%; and with caries grid, was 80.5%.
CONCLUSIONS: Risk assessment with the paper-based, objectivized, CRA-Grid matched that done by critical thinking process.

INTRODUCTION

Caries risk assessment (CRA) helps to identify the causative risk factors for dental caries and thus aid in tailor-made management plans.[12345] A CRA tool possesses questions on the social, behavioral, microbiologic, environmental, and clinical variables.[6] Data collected is analyzed by critical thinking as in reason-based systems or by computer programs in algorithm-based systems.[7] Without calibration, the faculty and the students tend to assign caries risk inconsistently.[8] Critical thinking skills required are not similar between an expert and a novice.[9] The variation can also be attributed to the inherent subjectivity and to the multifactorial nature of the disease.[10]

The aim was to develop an objective system and to assess its rating agreement with the critical thinking process.

MATERIALS AND METHODS

Development of the CRA-Grid

The CRA form that has been used by the authors for training in predoctoral dental education program since the year 2000 incorporated a systematic way of data collection and is broadly categorized as four sections, such as data obtained from: (1) History, (2) Habits, (3) Clinical evidence, and (4) Salivary analysis. This form was initially designed based on the previous evidence, modified to a certain extent to suit the local sociocultural issues.[1112] It was modified further with emerging new evidence such as CAMBRA,[13] ADA guidelines,[14] and ICCMS.[15] The content of form was phrased in negativity, without binominal categorization as “Yes” and “No,” as currently being practiced by other systems, and therefore marking any factor indicates risk toward caries. Table 1 shows the CRA form in use.

Table 1

Caries risk assessment form depicting the risk factors along with weightage assigned to them

	weightage in percentage
1. History
1.1 Personal history
1.11 Susceptible age factor (<35 and >60 years)	3
1.2 Past dental history
1.21 Premature extraction due to caries	1
1.22 Multiple restorations due to caries	1
1.3 Medical history
1.31 Medically compromised/physically challenged	1
1.32 Head-and-neck radiation therapy	4
1.33 Medications inducing dry mouth	1
1.34 Medications that are syrup (sugar) based	1
1.4 Social/family history
1.41 Socially deprived (eg. low SE status)	1
1.42 Low dental awareness	1
1.43 Irregular attendee to the clinic	1
1.44 High incidence of caries in siblings/parents	4
2. Habits
2.1 Life style habits
2.11 Smoking/alcohol/pan/tobacco consumption	1
2.12 Consumption of sugary chewing gum/mouth fresheners	2
2.13 Irregular working hours (shift system)	1
2.2 Dietary habits
2.21 Un balanced diet (no 3 meals/day; not balanced nutrition)	2
2.22 Excessive frank sugar intake (sugary beverages/sweets)	6
2.23 Frequent snack intake (>3 times between meals)	4
2.3 Oral hygiene habits
2.31 No use of brush/paste (use of ash/brick powder with finger)	1
2.32 Use of nonfluoridated tooth paste	1
2.33 Irregular or no use of mouth wash (fluoridated or antibacterial)	1
2.34 Irregular or no use of flossing	1
2.35 No brushing or brushing only once daily	1
3. Clinical evidence
3.1 Two/more active carious lesion (incipient or cavitated) visible in X-ray or clinically/grossly destructed tooth	10
3.2 One or more anterior or proximal caries/anterior or proximal recent restorations (done within the past 3 years)	10
3.3 Heavy plaque in cervical/proximal regions	4
3.4 Deep pits and fissures/crooked teeth/exposed root (that compromises oral hygiene)	4
3.5 Defective restorations (secondary caries/marginal ditching/overhangs/open contacts)	4
3.6 More than 4 restorations done within 3 years/orthodontic treatment/prosthesis/missing tooth due to caries (extracted in past 3 years)	4
4. Salivary analysis
4.1 Less than 0.5-0.7 ml/min of stimulated saliva	15
4.2 Less than 0.1-0.3 ml/min of un-stimulated saliva	15
4.3 Less than 5.5 pH of resting saliva	6
4.4 Less than 4 of buffering capacity	6
4.5 More than 106 CFU of Streptococcus mutans count	4
4.6 More than 106 CFU of Lactobacilli count	4

SE: Socio-economic, CFU: Colony forming units

Weightage was given to every factor with regard to their impact on dental caries. To assign the weightage, the authors had mapped their own critical thinking process in risk assessment by using historical database of previously recorded risk assessment forms. They reflectively and analytically, observed how and why they had arrived at specific risk status and how much of consideration they had given for the impact of a specific risk factor or a combination of risk factors, collected in the form. Current evidence on every factor was also taken into consideration for this purpose.

The salivary biomarkers such as low salivary flow, acidic pH, and low buffer have strong evidence linked to caries. The accuracy of salivary tests for mutans streptococci and lactobacilli is less, especially in low caries prevalence. They are more predictive for low caries risk than high caries risk.[16] Thus, more weightage was given to the salivary flow, pH, and buffer than the salivary microbial count. Correlating with salivary flow rate, the history section of the form elicits drug intake that can possibly reduce salivary flow as well the head-and-neck radiation, the latter assuming more weightage.[15]

The past clinical experience such as restorations and extractions for carious reasons assume profound predictive value.[4] This detail is obtained in the past dental history section, as well as in the clinical examination section and has been weighted appropriately. The presence of large cavitated active lesions are considered as prime risk indicators.[13] Therefore, maximum weightage has been given to these factors in the clinical findings.

Dental biofilm being one of the major biological risk factors, has been given considerable weight.[17] Defective restorations and malposed or disfigured teeth that predispose to plaque stagnation have also been considered in weightage assignment. Corroborating the clinical plaque condition, the habit history part of the form, elicits the oral hygiene aids and measures being adopted by the patient.

Dietary intake of sugar has long been incriminated as the “arch criminal” in dental caries.[18] The frequency of intake has been pointed to have profound impact on caries risk, than the quantity of sugar.[19] Thus, the form has significant weightage placed on these factors, including the history section eliciting the use of syrup based medications and use of lozenges/chewing gums.

A recent systematic review concludes that socioeconomic factors such as education, income, and occupation are associated with caries.[20] A consistent pattern of the severity of caries disease was found in patients with the familial-risk groups.[21] These factors have been included in the weightage assignment. However, being confounding factors and not the biological risk factors in dental caries, all these factors have been allotted lesser weight than the other risk indicators and predictors.

Based on the above evidence and the authors’ deliberation on their own critical thinking process, weightages were assigned to the major sections as well as the subfactors in each section, in percentage.

A 10 × 10 grid was made in a graph paper. The outer-most 36 squares represented the data from salivary analysis, the next inner box of 28 squares represented the data from clinical examination, the next inner 20 squares denoted the data obtained from habits, and the inner-most 16 squares denoted data from history. Under each section, the individual parameters occupied the number of squares based on the weightage assigned to them. For example, 1.1, which is susceptible age factor was assigned a weightage of 3% as opposed to 1.21 that denotes history of premature extraction due to caries, for which a weightage of 1% was assigned. Thus 1.1 occupied three squares and 1.21 occupied one square.

The data obtained for a patient, once entered in the Grid by shading the respective squares, when summed up at the end, will provide a percentage of risk factors contributing to dental caries. To finalize the class-interval of the percentage that denotes high/low/moderate risk, the authors had used the CRA-Grid on 100 previously recorded CRA forms that were analyzed by using critical thinking process. Thus, the following values were determined: low risk: <15%; moderate risk: 15%–25%; high risk: 25%–35%; and very high risk: >35%. Figure 1 shows a sample CRA form incorporating the caries grid.

Figure 1

Sample caries risk assessment form incorporating the caries grid and the guidelines

Pilot testing inter-rater agreement between teachers and students

The final year students in an undergraduate program in were trained by the didactic method as well as clinical demonstration, to use the CRA-Grid. They assessed the caries risk status for 57 patients using the Grid. Being a novel concept, this explorative pilot study was intended to be done over a period of 3 months. Thus, the data of the patients who were assessed by the students for risk status within this period have been included in the study. A sample size of 12 has been recommended as a thumb rule for pilot clinical studies.[22]

The score and risk assigned by the CRA-Grid were recorded. The CRA data form was separated from the Grid for blinding the teachers. These sheets were randomly distributed to six teachers for assigning risk status by using critical thinking process. The teachers who had a mean teaching experience of 8 years, had been teaching CRA in the undergraduate program. They had been trained and calibrated by the authors.

Inter-rater agreement between the teachers’ critical thinking process and CRA-Grids’ risk assessment was assessed. Cohen's weighted kappa (k) was run using GraphPad QuickCalcs.

RESULTS

Mean percentage of agreement among the teachers for risk assessment using critical thinking process was 79.6% and that between the teachers’ risk assessment and CRA-Grid scoring by students was 80.5%. Cohen's weighted kappa for agreement at a 95% confidence interval, between the CRA-Grid and faculty, is shown in Table 2. The agreement between the Grid and the teacher's assessment ranged from “very good” to “good.” Figure 2 shows the agreement for high/low/moderate risk between the Grid and critical thinking process.

Table 2

Inter-rater agreement between teachers’ risk assessment through critical thinking and scoring by students using caries risk assessment grid

T versus grid	Weighted κ	SE	95% CI	Strength of agreement
T1 versus grid	0.764	0.089	0.468-0.815	Good
T2 versus grid	0.655	0.095	0.319-0.691	Good
T3 versus grid	0.879	0.074	0.680-0.969	Very good
T4 versus grid	0.646	0.097	0.334-0.713	Good
T5 versus grid	0.797	0.092	0.519-0.880	Good
T6 versus grid	0.630	0.109	0.271-0.697	Good

CI: Confidence interval, SE: Standard error, T: Teachers

Figure 2

Percentage distribution of high/moderate and low-risk status assigned with caries risk assessment-grid and teachers’ critical thinking process (L: Low/M: Moderate/H: High)

DISCUSSION

In this preliminary study, an appreciable agreement was found between the CRA-Grid and the critical thinking process of teachers. This result can be attributed to the process of development, which was based on mapping the thought process of the experts in assessing the risk. This objective model not only captured the complex process of clinical reasoning but also reproduced the outcome of the process. Assignment of the individual risk status as high/moderate/low also matched between the Grid and the critical thinking process. This observation is gratifyingly different from the previous report which stated that students usually err in identifying moderate risk status.[10]

Use of algorithm-based system such as Cariogram reduces variation by objectivization,[1] but it does pose a risk of compromising the training in critical thinking skills. Students tend to conclude directly without understanding the process or the significance of every risk factor.[89] It is vital that they are trained in critical thinking skills and clinical reasoning so that they can emulate the thought process of an expert in decision making.[92324] Such training is best provided in a staged manner as per the Dreyfus skill acquisition model.[25] The CRA-Grid can be considered as a training model toward independent critical thinking skills because the process of manually shading the Grid-enabled the student to understand the impact of every factor on caries risk as well as the “risky” combinations of multiple factors.

It is also stated that scheme-inductive reasoning that is based on sound knowledge is a more effective critical thinking skill, than the hypothetico-deductive reasoning.[26] In accordance with this, the comprehensive, inclusive, and systematic CRA data collection form was educative for the students on the basic knowledge of caries etiology.

Training and implementation of CRA have been reported to be challenging and a slow process, despite extensive training of the students and the faculty.[27]“Skepticism among the students and teachers has been reported as the main hurdle in implementation of CRA in education.[28] However, in this preliminary validation, an improvement in the buy-in by the students as well as the faculty was observed, due to the synchronization and objectivity of using the Grid. Implementation was also smooth due to its simplicity.

It has been suggested that after identifying the potential high-risk patients, the risk can be assessed accurately for specific clinical decisions by identifying the biological risk factors, with a “full-blown model,” that included salivary tests.[2930] Accordingly, a guideline has been provided in this Grid, that salivary analysis may be done for patients with high risk. However, for this preliminary trial, salivary risk assessment tests for the high-risk patients were not included, due to the cost factor.

CONCLUSIONS

From this study, it can be concluded that CRA-Grid can be a simple, effective, and objective tool for risk assessment to be used in dental education. Further studies are required to assess the tool in different educational settings and with a different cohort of teachers.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.