An index of orthognathic functional treatment need (IOFTN).

OBJECTIVE: To design a new index categorizing the functional need for orthognathic treatment.
DESIGN: Laboratory-based study.
SETTING: Records were obtained from two UK hospital-based orthodontic departments. PARTICIPANTS: A panel of four consultant orthodontists, experienced in providing orthognathic care, devised a new index of Orthognathic Functional Treatment Need (IOFTN) with the aid of the membership of the British Orthodontic Society Consultant Orthodontists Group (COG). Twenty-three consultants and post-CCST level specialists took part in the study as raters to test the validity and reliability of the new index.
METHODS: A total of 163 start study models of patients who had previously undergone orthognathic treatment were assessed by the panel of four consultant orthodontists using the new index (IOFTN) and the agreed category was set as the 'gold standard'. Twenty-one consultants and post-CCST level specialists then scored the models on one occasion and two scored 50 sets of models twice to determine the test-re-test reliability.
RESULTS: Kappa scores for inter-rater agreement with the expert panel for the major categories (1-5) demonstrated good to very good agreement (kappa: 0·64-0·89) for all raters. The percentage agreement ranged from 68·1 to 92% in all cases. Intra-rater agreement for the major categories was moderate to good (kappa: 0·53-0·80).
CONCLUSIONS: A new index, the IOFTN, has been developed to help in the prioritization of severe malocclusions not amenable to orthodontic treatment alone. It demonstrates good content validity and good inter-rater and moderate to good intra-rater reliability. As a result of being an evolution of the IOTN, the familiar format should make it easy to determine functional treatment need within daily orthognathic practice.

Introduction

Globally, the disciplines of medicine and dentistry use indices of health widely and these have been developed for many different purposes. Uses include the classification of conditions to aid the understanding of aetiology, risk, prognosis and treatment outcome (Sharabiani ). They can also be used to determine prevalence and or incidences within a population, and therefore, help in the planning and provision of treatment at the individual or population levels. In recent years, their use in the planning of services, particularly within cash-limited, publicly funded health services such as the UK National Health Service, has gained greater acceptance. Our own experience within orthodontics is that indices have been used to prioritize treatment to those most in need and likely to benefit from orthodontic treatment, as well as to monitor the quality of treatment outcome. In the prioritization of treatment provision, we have become familiar with the use of the Index of Orthodontic Treatment Need (IOTN) (Brook and Shaw, 1989), which has been in routine use in NHS primary care in England and Wales since 2006, and somewhat earlier than this in many secondary care settings (Holmes and Wilmott, 1996). This index was developed with the aim of prioritizing the functional need for treatment through its dental health component (DHC) and psychosocial need through the aesthetic component (AC). This was not the first such index used for this purpose in orthodontics, with indices such the Handicapping Labio-lingual deviations index (Draker, I960), the Treatment Priority Index (Grainger, 1967), the Handicapping Malocclusion Assessment Record (Salzmann, 1968) and the Occlusal Index (Summers, 1971) all having been developed earlier than the IOTN. The IOTN itself is a modification of an index previously developed by the Swedish Dental Health Board (Linder–Aronson, 1974).

In the process of developing an index of health, a number of factors must be taken into consideration. The principal factor is its intended purpose, but ease of use in daily practice is also important, since it may involve the collection and interpretation of a large amount of data from which, a single useful indicator is then provided (Arvaniti and Panagiotakos, 2008). In addition, an index should be both valid and reliable. A number of studies have been carried out to assess the validity and reliability of the IOTN, as well as the time taken to use the index. Validity is often measured against expert opinion. In comparison with other orthodontic occlusal indices, the strengths of the IOTN DHC component are, not only its validity and reliability, but also that it is quick and easy to use (Cardoso ). Moreover, the grading also appears to be unaffected by age, at least within the relatively narrow age range of the adolescent patient (Cooper ). As a result, it is not only widely used in orthodontic research (Bellot-Arcís et al., 2012), but is also highly rated by those involved in planning orthodontic service provision within the UK (de Oliveira, 2003).

The IOTN DHC is a straightforward five-point scale, with the greatest need for treatment classified as being group 5 and little or no need for treatment classified as group 1. Within each group, there are well defined descriptors of the features of the malocclusion deemed as indicators of orthodontic need (such as overjet, impacted teeth and missing teeth). The reason the index is quick and easy to apply is that the malocclusion is scored simply on the worst feature. In order to identify this feature in a systematic manner it is suggested the assessor uses the acronym MOCDO (Missing teeth, Overjet, Crossbites, Displacement of contact points, Overbite) (Richmond ).

Currently within the NHS, orthodontic treatment is limited to IOTN DHC groups 4 and 5, and group 3 where the AC is grade 6 or above. Although widely used, there are some limitations of the IOTN. In the case of the AC of the index, it comprises only class I and class II division 1 incisor relationships and there are no class II division 2 or class III incisor relationships. In the case of the DHC, some of the functional indications for orthognathic treatment are not included, or might be classified differently if the malocclusion were not treatable with orthodontics alone. For example, excessive upper labial segment show at rest is not included in the IOTN in the absence of any other occlusal traits, and yet this can lead to potential gingival and periodontal problems and might only be amenable to treatment with combined orthodontic and orthognathic treatment.

With these limitations in mind, it was decided to create a new Index of Orthognathic Functional Treatment Need (IOFTN), using wherever possible the same traits as used in the IOTN DHC, but with modifications and additions to reflect the functional indications of treatment need for orthognathic patients. In this way, it was hoped to create an index that feels familiar to those using the IOTN, is valid, reliable and quick and easy to use. This paper describes the development of this new index.

Materials and methods

Using the IOTN DHC as a starting point, four consultant orthodontists with extensive experience in treating orthognathic patients (AJI, SJC, MTC, NPH) devised a draft IOFTN based on a five-point scale ranging from Very Great Need for Treatment (5) through to No Need for treatment (1). The draft index was then presented at the British Orthodontic Society Consultant Orthodontists Group (COG) Symposium in Bristol during March 2013, following which it was formally circulated to all 280 members of the COG for written comment. Forty-six members replied with written comments and these were then reconsidered at a further meeting of the panel of experts. Modifications were made to the wording of the index to reflect the comments and at this point the index was considered to have face validity.

The panel of four experts then worked in pairs to score 163 sets of start study models using the new index. The sample of models represented various malocclusions that had previously been treated using an orthognathic approach. The scores were then compared and, wherever there was disagreement, the panel discussed them and came to a consensus score for each of the 163 sets of models. At this point, the index (Figure 1) was considered to have content validity, as it was felt that all of the possible facets of the construct of whether or not an orthognathic treatment approach was appropriate for functional reasons, had been considered.

Figure 1

The Index of Orthognathic Functional Treatment Need (IOFTN)

Twenty-one specialist orthodontists with experience of orthognathic treatment, all consultant orthodontists and senior FTTAs were then asked to score the 163 sets of models using the new index, in order to test agreement with the expert panel scores. The scores were then analysed using Cohen’s kappa for inter-operator agreement with the expert panel scores when assigning the patient to one of the five major categories. It also looked at assignment within the major categories to the individual sub-categories (for example, having decided a patient was in category 5, what was the agreement for allocation to the subcategories within the major category). In addition, the percentage agreement with all of the categories scored by the experts was also determined for each of the 21 assessors.

In order to determine the intra-operator agreement, two consultants scored 50 sets of study models on two separate occasions 1 week apart. Agreement was again tested using Cohen’s kappa for the main groups.

Results

The results were analysed using SPSS (IBM SPSS Statistics 22·0; IBM Corp., Chicago, IL, USA) and Stata Version 13 (STATA Corp., College Station, TX, USA). Table 1 is the summary table of the 163 study models, illustrating the number of models in each of the IOFTN categories. The kappa scores for inter-operator agreement with the expert panel scores for the major categories are illustrated in Table 2, and this shows good to very good agreement for all raters. The percentage agreement of the 21 assessors with the expert panel scores for all categories is illustrated in Figure 2 and ranged from 68·1 to 92%. The per cent agreement was over 80 for 16 of the 21 assessors, which can also be considered good. When the agreement for the subcategories within the major categories was compared with the expert panel score, it was also good to very good for all raters (Table 3), 2–5. There were too few models in category 1 for statistical testing as few patients in this category will have undergone orthognathic treatment.

Table 1

Numbers of models in each of the categories of the IOFTN

Category 5	Number of models
5·1	9
5·2	9
5·3	8
5·4	10
5·5	2
5·6	0
5·7	5
Category 4
4·2	12
4·3	14
4·4	2
4·8	10
4·9	8
4·10	10
Category 3
3·3	13
3·4	7
3·9	0
3·10	9
Category 2
2·8	8
2·9	18
2·11	9
Category 1
1·12	0
1·13	0
1·14	1

Table 2

Kappa scores illustrating strength of agreement between the major category scores of each of the 21 assessors with the expert panel major category scores (<0·2 = poor; 0·21–0·40 = fair; 0·41–0·60 = moderate; 0·61–0·80 = good; 0·81–1·00 = very good agreement)

Assessor number	Kappa score
1	0·85
2	0·88
3	0·66
4	0·75
5	0·73
6	0·64
7	0·78
8	0·81
9	0·83
10	0·81
11	0·80
12	0·83
13	0·87
14	0·80
15	0·88
16	0·64
17	0·84
18	0·89
19	0·76
20	0·81
21	0·74

Figure 2

Histogram illustrating the percentage agreement of each of the 21 assessors with the expert panel scores for all of the 23 categories within the IOFTN

Table 3

Kappa scores illustrating strength of agreement for the subcategories within each major category score for each of the 21 assessors with the expert panel scores (<0·2 = poor; 0·21–0·40 = fair; 0·41–0·60 = moderate; 0·61–0·80 = good; 0·81–1·00 = very good agreement)

Assessor	IOFTN 5	IOFTN 4	IOFTN 3	IOFTN 2
1	0·91	0·97	0·94	1·00
2	0·91	0·98	0·93	1·00
3	0·72	0·96	0·91	1·00
4	0·94	1·00	0·93	1·00
5	0·97	0·97	1·00	1·00
6	0·91	0·89	0·82	1·00
7	0·94	0·94	1·00	1·00
8	0·90	1·00	1·00	0·95
9	0·97	1·00	1·00	0·95
10	0·96	0·95	1·00	1·00
11	0·97	0·87	1·00	1·00
12	0·94	1·00	1·00	1·00
13	0·88	0·97	0·88	1·00
14	1·00	1·00	0·89	1·00
15	0·87	0·97	0·94	1·00
16	0·91	0·87	1·00	1·00
17	0·94	0·95	0·93	0·95
18	0·97	1·00	1·00	1·00
19	0·94	0·97	1·00	1·00
20	1·00	0·89	0·71	1·00
21	0·97	0·91	0·87	0·94

The weighted kappa scores for intra-operator agreement were 0·53 for operator 1 and 0·80 for operator 2, showing moderate to good agreement over time for each rater. The percentage agreement for all categories and subcategories was 68 and 76%, respectively.

Discussion

In recent years, there has been a drive to reduce costs within the UK NHS, not only to reduce overall spending, but also to divert money and resources from what are deemed ‘low priority’ treatments, to those deemed to be of higher value and where the evidence to support their use is said to be greater. As far back as 2006, primary care trusts in England responsible for NHS funding within their areas began compiling lists of what they considered low priority treatments. One such list, the Croydon List has received much attention and comprised 34 treatments. Other PCTs compiled much longer lists of over 100 procedures and this prompted the Audit Commission in 2011 to suggest that their implementation in commissioning health could lead to annual savings to the NHS of £500 million (Audit Commission, 2011). Although the audit commission found some commonality in the lists, there was not complete uniformity. What could be considered low priority in one area might automatically receive funding in another, leading to the potential for a ‘postcode lottery’ of access to healthcare. In 2012, the South Central PCTs, in consultation with Solutions for Public Health, investigated the evidence to support the routine funding of orthognathic treatment for reasons of function, sleep apnoea, speech and temporomandibular joint dysfunction. Following this investigation, the southern cluster within the South Centrals area decided that all orthognathic treatment should be considered to be of low priority, except for severe sleep apnoea, cleft lip and palate and following major trauma (HIOW/SHIP Priorities Committee April 2008 to March 2012). The northern cluster also considered it to be low priority and decided not to fund treatment for speech or temporomandibular joint dysfunction, but were prepared to continue funding for functional reasons and sleep apnoea, and provided the patients were categorized as IOTN 4 or 5 (Solutions for Public Health, 2012). It was at about the same time that the Strategic Health Authorities in England were abolished, in line with the introduction of the UK government’s Health and Social Care reforms (Ham, 2012), and the South Central PCTs commissioning intents appeared to have been lost during this NHS restructuring. In the new era, the commissioning of all dental services and for the interim, all oral and maxillofacial services, were now to be commissioned centrally by NHS England and implemented locally by the local area teams. In late 2013, NHS England published its interim clinical commissioning policy for orthognathic treatment. Although this interim policy was withdrawn in March 2013, it had stated the following (British Association of Oral and maxillofacial Surgeons (BAOMS), 2014):

the IOTN must be 4 or 5;

functional symptoms must have an important impact on quality of life, which would normally have become apparent within 5 years of achieving skeletal maturity;

the multidisciplinary team confirms that orthodontic treatment is insufficient by itself to adequately correct these functional symptoms;

patients have reached skeletal maturity;

orthognathic treatment should be low priority on the grounds of insufficient evidence of functional improvement for:

speech problems;

jaw pain, particularly that associated with the temporomandibular joint.

It would seem that the interim guidance was based on the earlier South Centrals PCT work and included the IOTN as a measure of severity and functional need. However, the use of IOTN has limitations as a measure of functional and health need in orthognathic treatment provision. In particular, some severe dentofacial deformities and malocclusions would not be eligible for NHS funding for orthognathic treatment using IOTN. Examples include excessive upper labial segment gingival exposure with evidence of gingival and/or periodontal effects, complete scissor bites or facial asymmetries with marked effects on the occlusal plane. In addition, there was no mention of orthognathic treatment for sleep apnoea.

It is in order to overcome these limitations with the use of IOTN in orthognathic treatment provision that the IOFTN was developed. The new IOFTN has good face and content validity and also demonstrates good to very good inter-operator agreement (0·64–0·88), similar to the IOTN (0·731–0·797) (Brook and Shaw, 1989). This is perhaps not surprising, in that the two indices share a similar format, which clinicians are familiar with. As with the IOTN, the single-most severe trait is used for scoring the patient. It is important, particularly when scoring from study models alone, that additional information is provided; for example, information would be required regarding the degree of upper labial segment exposure where present, or functional effects such as trauma to the soft tissues where there is an increased overbite. This will not be a problem where the IOFTN is used to score the patient at the chairside. Similar limitations also apply to the use of IOTN when scoring more routine malocclusions from study models alone. Once again, the IOFTN also demonstrates moderate to good intra-operator agreement over time (0·53–0·80), not too dissimilar to that observed with the IOTN, with its reported kappa scores of 0·75–0·84 (Brook and Shaw 1989).

After 24 years of service to orthodontics, it is perhaps timely that the application of IOTN in clinical practice is being revisited. The concept that any one index should not be expected to fit all eventualities when deciding on treatment priorities has recently been made in reference to secondary care orthodontics (Cousley, 2013). We therefore feel that the IOFTN is a natural evolution of the IOTN that should be used when setting treatment priorities for combined orthodontic and orthognathic care. It is both valid and reliable and, like the IOTN, is quick and easy to use, thereby fulfilling the essential requirements of an index. However, the IOFTN concerns the functional indicators for orthognathic treatment, and other clinical and psychological indicators will also be important in the assessment of orthognathic patients.

Conclusions

A new index, the IOFTN, has been developed to help in the prioritization of severe malocclusions not amenable to orthodontic treatment alone. The index has face and content validity and has been shown to have good inter and moderate to good intra-operator reliability. As a result of being an evolution of the IOTN, the format is similar to this index and so it should be easy to incorporate within daily orthognathic practice.

Disclaimer statements

Contributors Each of the authors contributed in the following manner: substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; drafting the article or revising it critically for important intellectual content; and final approval of the version to be published. The guarantor is Anthony J. Ireland.

Funding None.

Conflicts of interest There are no conflicts of interest.

Ethics approval Ethical approval was not deemed necessary.