Implementation of post treatment critical evaluation improved the quality of orthodontic care in postgraduate orthodontic clinic: A 10 years comparative study.

OBJECTIVE: The aim of the study was to evaluate the effect of post- treatment critical evaluation on the quality of orthodontic care in a postgraduate orthodontic clinic.
MATERIALS AND METHODS: Orthodontic treatment outcome of 109 consecutively treated cases was evaluated in Phase-I evaluation. Following Phase-I evaluation, PTCE of each case was made mandatory. After 6-years of implementation of compulsory PTCE for each case, orthodontic treatment outcome of all consecutively treated cases (n = 126) was evaluated (Phase-II). The treatment outcome was evaluated by American Board of Orthodontics Model Grading System (ABO MGS) and Subjective evaluation (Visual Analogue Scale, VAS).
RESULTS: Based on the ABO scores, the cases were divided into three grades, that is, Grade-I, Grade-II, and Grade-III. The mean total ABO score was improved significantly in Phase-II evaluation (P < 0.01). The total number of cases in ABO Grade-II were increased significantly (P < 0.01) whereas cases in ABO Grade-I remained comparable. The VAS score was improved from 5.66 ± 0.77 at Phase-I to 6.02 ± 0.99 at Phase-II evaluation (P < 0.01).
CONCLUSION: The implementation of PTCE significantly improved the quality of orthodontic care in a postgraduate orthodontic clinic. CLINICAL SIGNIFICANCE: Grading one's own treatment improves the quality of future treatment.

INTRODUCTION

With the ever increasing importance of clinical effectiveness, audit and quality assurance; it is becoming mandatory to grade one's own treatment results. The idea of individuals grading their own performance is a self-teaching exercise and improves the quality of future treatment.[1] In addition, it is also useful for educational purposes in postgraduate orthodontic programs. The assessment of orthodontic treatment needs and the outcome has traditionally been accomplished using subjective opinion and the experiences of clinicians. However, the variations in criteria used by different orthodontists and lack of reproducibility are limitations of the subjective evaluation. Visual analog scale (VAS) has been the most widely used as a subjective assessment method for the validation of various indices.[234]

In early surveys, various classifications were used as determinants of treatment needs and outcome. However, various authors such as Massler and Frankel,[5] Drakers,[6] Bjoerk et al.,[7] and Summers[8] devised indices to evaluate malocclusion on the basis of objective requirements of occlusal discrepancies. Pickering and Vig[9] were the first to investigate the use of Summers occlusal index as objective grading version to assess the outcome of treatment. Richmond et al.[10] devised peer assessment rating (PAR) indices to evaluate malocclusion on the basis of objective requirements of occlusal discrepancies. The American Board of Orthodontics (ABO) model grading system is presently one of the most detailed scoring systems in use.[11] Comprehensive clinical evaluation as outcome assessment was developed which included facial form, dental esthetics, vertical control, arch form, periodontium management, root structure preservation, and treatment efficiency.[121314] To assess clinical performance, both objective and subjective scoring criteria should be used for the evaluation of treatment outcome. Although there are many studies in the literature mentioning the orthodontic treatment outcome of various bracket systems,[151617181920] various extraction pattern,[21222324] various bracket slot size,[25] type of orthodontic practice, that is, private versus university practice,[26272829] timing of treatment;[14151830] and general practitioner versus orthodontist[31] but, to our knowledge, there are very few studies in the literature mentioning the effect of assessment of treatment outcome on the orthodontic treatment outcome.[13] Thus, the present study was conducted to find out the role of posttreatment critical evaluation (PTCE) implementation on the effect of orthodontic treatment outcome. To eliminate any institutional bias, subjective evaluation was done by external examiners to assess the outcome of orthodontic treatment.

MATERIALS AND METHODS

The study sample comprised of pretreatment and post treatment orthodontic records of 235 consecutively debonded cases in the Department of Orthodontics, Oral Health Sciences Centre, Postgraduate Institute of Medical Education and Research, Chandigarh. Cases whose orthodontic treatment was completed during the period 1999 and 2004 were included in Phase-I (n = 109) evaluation. A total of 278 cases were debonded during this period, out of which 120 cases comprised of compromised debond and incomplete records and 49 cases comprised of cleft and surgical cases. Cases whose orthodontic treatment was completed during the period 2006 and 2010 were included in Phase-II (126) evaluation. The sample was extracted from a total of 328 debonded cases, out of which 108 comprised of cleft and surgical cases, 94 had incomplete records. The final sample had 126 cases. While selecting a case for the evaluation, no attempt was made to select a case on the basis of malocclusion type, gender or extraction pattern. Orthodontic cases having good quality pretreatment and posttreatment records such as study models, photographs, and orthopantomograms (OPGs); and orthodontic cases those had completed comprehensive orthodontic treatment in both the arches were included in the study. Orthodontic cases having good quality pre-treatment and post-treatment records like study models, photographs and orthopantomograms and orthodontic cases those had completed comprehensive orthodontic treatment in both the arches were included in the study.

In order to improve the quality of orthodontic care, PTCE of each case was made mandatory as a part of the clinical protocol after Phase-I study. PTCE included the power point presentation of patient's orthodontic treatment. The power point presentation contained detailed pre and post treatment records which included study models, intraoral and extra-oral photographs, intraoral periapical radiographs, OPG, lateral cephalograms. Stage records were also included wherever required. Detailed history, progress of treatment, number of missing appointments, data from study model, OPG and lateral cephalometric analysis and ABO model grading, superimpositions of pretreatment, stage, and post-treatment cephalograms (to evaluate growth, change in facial profile and incisor molar positions) were also included in the power point presentation. During the presentation, pre- and post-treatment records, data and treatment plan were critically evaluated by all the postgraduate residents and faculty. The presence of any lacunae in the treatment was discussed, and special emphasis was given to improve these lacunae subsequently. Then, 6-years after implementation of compulsory PTCE for each case, the orthodontic treatment outcome of all consecutively debonded cases was evaluated (Phase-II).

The treatment outcome was evaluated by American Board of Orthodontics Model Grading System (ABO MGS)[11] and Subjective evaluation.[32] The ABO Model Grading System included eight criteria to measure the post-treatment finishing of the occlusion. Post-treatment study models and panoramic radiographs were scored by one examiner for eight components of the ABO MGS using the ABO measuring gauge. The intraexaminer calibration was done by repeated ABO scoring by the examiner at the interval of 4 weeks. As per the ABO instruction, a case with <20 points would pass, and the case with 30 points would fail. A case with points more than 20 but <30 would fall in category “undetermined to pass.” Thus, the scores were thus divided into three categories, that is, Grade-I (score 0–20 points), Grade-II (score 20–30 points), and Grade-III (score >30 points).

The subjective evaluation was carried out by three external examiners who were senior orthodontists with a minimum of 7 years of clinical experience and also involved in orthodontic teaching. The external examiners were chosen to avoid any bias in scoring cases from one's own department. Each of the three examiners judged debonded cases independently at different time periods for treatment outcome based on combined assessment of the pre- and post-treatment dental casts and photographs (extraoral and intraoral, pre- and post-treatment photographs). The mean of the VAS score allotted for each case by the three examiners was calculated for the purpose of analyzing the results. The scale used was 11-point (0–10; 0 = very poor, and 10 = excellent) VAS, which was anchored at its two ends with extreme limits of very poor and excellent.[32] The total VAS score was divided into three grades, that is, poor (VAS score 0–4.9), fair (VAS score 5–6.9), and good (VAS score 7–10). The treatment outcome of cases in Phase-I and Phase-II evaluation were noted on a standard Performa. The mean values with standard deviation for the individual components and the total ABO score of the ABO model grading system for the total sample was recorded. The mean of the VAS score allotted for each case by the three examiners was calculated for the purpose of analyzing the results.

Statistical analysis

The statistical analysis was carried out using SPSS-software version-16.0 (SPSS Inc. Released 2007. SPSS for Windows, Version 16.0. Chicago, SPSS Inc.). Descriptive statistics was used. The intra examiner reliability was determined among the various components of the ABO MGS and the total score by using paired t-test and Pearson's correlation coefficient. The Cronbach's alpha test was used to determine the intraexaminer reliability for each of the three examiners. Unpaired t-test was used to compare various components and a total score of ABO model grading system and subjective evaluation between Phase-I and Phase-II. The Chi-square test was used for comparison between Phase-I and Phase-II for different categories of ABO model grading and subjective evaluation. The P value of 0.05 was considered as the level of significant.

RESULTS

The intra examiner reliability among the various components of the ABO MGS and the total score was found to be between 0.888 and 1. The description of cases evaluated during the Phase-I and Phase-II evaluation is summarized in Table 1.

Table 1

Description of cases evaluated during Phase-I and Phase-II evaluation

The mean value for the individual components of the ABO MGS and the total ABO score for the total sample in Phase-I and Phase-II evaluation is described in table 2. The mean scores of alignment, marginal ridge relationship, occlusal contacts, and interproximal contacts were improved in Phase-II evaluation as compared to Phase-I evaluation; and the difference was comparable. The buccolingual inclination, occlusal relationship, and root angulation were improved significantly during Phase-II evaluation (P < 0.001). The score for overjet was improved from Phase-I evaluation to the Phase-II evaluation (P < 0.01). The total ABO score was improved significantly from 30.06 ± 10.54 at Phase-I evaluation to the 26.67 ± 7.77 at Phase-II evaluation (P < 0.01).

Table 2

The mean values for the individual components of the ABO MGS and the total ABO score for the total sample in Phase-I and Phase-II evaluation and their comparison

The distribution of the sample according to ABO score during Phase-I and Phase-II evaluation is described in Table 3. The percentage of cases in the Grade-I of ABO category was similar during Phase-I and Phase-II evaluation (20.10% vs. 20.63%). The percentage of cases in the Grade-II of ABO category was improved significantly from Phase-I evaluation (34.03%) to Phase-II evaluation (50.79%) (P < 0.01). The percentage of cases in the Grade-III of ABO category was decreased significantly during Phase-II evaluation (45.87%) as compared to the Phase-I evaluation (28.57%) (P < 0.01).

Table 3

Comparison of total number of cases falling in each category of ABO model grading system between Phase-I and Phase-II evaluation

The value of reliability for three examiners was found as 0.675 that indicated the good reliability between the examiners. The VAS score for the total samples evaluated in Phase-I and Phase-II is described in Table 4. The mean VAS score for the total sample was improved from 5.66 ± 0.77 at Phase-I to 6.02 ± 0.99 during Phase-II. The percentage of cases in the poor and fair grades was decreased from Phase-I evaluation to the Phase-II evaluation. The percentage of cases scoring good score was increased significantly from 7.34% at Phase-I evaluation to 19.05% at Phase-II evaluation indicating improvement in the orthodontic treatment outcome (P < 0.01).

Table 4

Comparison of mean VAS score and total number of cases falling in each category of VAS grading between Phase-I and Phase-II evaluation

DISCUSSION

The excellence in finishing and the quality displayed at the end of orthodontic treatment constitutes a fundamental goal of orthodontic specialty. During all stages of treatment, one should keep the end results in mind, providing a protected occlusion, better aesthetics (both dental and facial), good periodontal health and long-term stability, which are correlated to proper finishing.[11]

The sample for the study was collected from the records available in the department for a large sample in order to draw generalized conclusions. The gender distribution was not considered while including a case for the study because of uneven distribution of males and females.

The ABO model grading system was used to evaluate the treatment outcome. All the cases were evaluated by the same observer and the intraexaminer reliability value was found as 0.985, which was >0.85 recommended by ABO members.

The results of the present study showed a reduction in the mean total ABO score from 30.06 ± 10.54 to 26.67 ± 7.77. There was an improvement in the score of each component of ABO model grading system. The cases falling in the Grade-II category had also been increased from 33.03% to 50.79% from the Phase-I evaluation to Phase-II evaluation. The implementation of compulsory PTCE was responsible for the improvement in the treatment outcome at Phase-II evaluation. The average total ABO score (26.67) that we observed at the end of Phase-II evaluation was less than the average score of 45.54 ± 18.33 for university group of Illinois and 33.88 for ABO group.[26] ABO score was also better than that observed in the study done by Pinskaya.[12] The results of the present study correspond with the study done in Indiana University School of Dentistry.[13] The better results in our study could be secondary to the implementation of compulsory PTCE for orthodontic residents.

We also considered a subjective assessment by three external orthodontists to compare the overall treatment outcome and treatment quality. In the subjective evaluation, not only study models were evaluated, but also intraoral and extraoral photographs were evaluated to identify the improvement in the profile. As the scoring on the VAS was totally based on the personal opinion and judgment of the external examiners, this evaluation was purely subjective in nature and no standardization in terms of any set criteria or guidelines was carried out. All the three examiners scored each case at the different times. Therefore, an absolutely independent judgment was done by removing the confounding bias related to a discussion among the examiners. Similar to ABO model grading scores, the VAS score was also improved at Phase-II evaluation, and the implementation of compulsory PTCE was responsible for such improvement. However, unlike ABO model grading system, the subjective evaluation scored the treatment results by looking at the change in occlusion, as well as facial appearance, thus it did not have the inherent limitation of objective indices.

The intraexaminer reliability of examiners did not show good reliability. The lack of any objective measures to determine the results explained the low reliability of subjective criteria. Similarly, to our study low reliability values were also reported by Richmond et al.,[10] where they used a 9-point scale for subjective assessment of dental casts with respect to deviation from normal occlusion in the validation study of the PAR index by 74 dentists. To reduce the effect of interexaminer variability the mean score of three examiners was used for the final score in the statistical comparison of results. Thus, the present study showed that the implementation of PTCE improved the quality of orthodontic treatment among orthodontic residents, and the PTCE should be implemented as compulsory in a postgraduate orthodontic clinic.

CONCLUSION

The quality of orthodontic care was improved significantly following the implementation of PTCE in a postgraduate orthodontic clinic.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.