Validation of an Oral Disease Severity Score for use in oral lichen planus.

dear editor, Oral lichen planus (OLP) is a chronic inflammatory condition with a long clinical course and potentially negative impact on quality of life. , The heterogeneous clinical presentation, lack of treatment consensus yet a need to ensure therapeutic efficacy, is compounded by the absence of a standardized and comprehensively validated disease scoring methodology. The Oral Disease Severity Score (ODSS) is a detailed oral scoring system that has been validated for use in oral pemphigus vulgaris (PV) and mucous membrane pemphigoid (MMP). , We have previously demonstrated its usefulness for oral lichen planus. , The aim of this study was to formally validate the ODSS in OLP.

Research ethics approval was obtained (REC15/ES/0038). Sixteen patients (aged 48–78 years) with active OLP (histologically confirmed) were recruited from oral medicine and dermatology clinics at Guy’s Hospital, London. Nine were on systemic treatment [mycophenolate mofetil (five), hydroxychloroquine (nine), low‐dose prednisolone (three)] and seven on topical corticosteroid treatment alone [betamethasone soluble tablets (four) and fluticasone propionate nasules (three)].

Ten clinicians from four UK oral medicine centres participated. All were oral medicine specialists; one additionally a dermatologist. Patients were scored using the ODSS and the Physician Global Assessment (PGA).

The ODSS scores 17 oral mucosal sites. Each unit of site is given an activity score. Further details are given in the Table 1 legend.

Table 1

Scores, inter‐ and intra‐observer reliability for each of the disease severity scoring systems and their individual components

Scoring system a	Range	Mean (SD)	Median (IQR)	Intraclass correlation coefficient (95% CI)
				Interobserver b	Intra‐observer b , c
					Observer 1	Observer 2
ODSS Site	3–19	10.6 (3.5)	10 (8–13)	0.97 (0.93–0.99)	0.92 (0.79–0.97)	0.95 (0.87–0.98)
ODSS Activity	0–34	11.2 (7.2)	10 (5–15)	0.97 (0.95–0.99)	0.87 (0.62–0.95)	0.85 (0.38–0.95)
ODSS Pain	0–8	3.0 (2.3)	3 (1–5)	0.99 (0.99–1.00)	0.99 (0.97–0.99)	0.99 (0.98–0.99)
ODSS Total (0–106)	7–56	24.5 (10.8)	23 (16–30)	0.98 (0.96–0.99)	0.85 (0.63–0.95)	0.92 (0.71–0.98)
PGA (0–10)	0–9	3.4 (1.8)	3 (2–4)	0.96 (0.91–0.98)	0.75 (0.42–0.91)	0.88 (0.69–0.96)

The sites are the outer lip, inner lips, buccal mucosae right/left, soft palate right/left, hard palate right/left, dorsum of tongue right/left, ventrolateral tongue right/left, floor of mouth right/left, oropharynx right/left and the gingivae (divided into 6 segments). Site score 0 (no lesion) or 1 (lesion), buccal mucosa: 1 (≤ 50%) or 2 (> 50%); dorsum of tongue, floor of mouth, hard or soft palate or oropharynx: 1 (unilateral) or 2 (bilateral). Where a site has a score of 2, each site unit is allocated an activity score, which are then added together.

Activity score: 1, mild erythema; 2, marked erythema without ulceration; 3, erosion or ulceration. White asymptomatic lesions are given a site score proportional to the size of the area affected but an activity score of zero.

Pain score: Analogue scale from 0 (no discomfort) to 10 (the most severe pain they have encountered with this condition so far); the patient is asked to provide a score reflecting their pain/discomfort as an average of the preceding week.

Total Score = Site Score + Activity Score + Pain Score (0–10) (maximum 106).

Overall benchmark values = excellent in all instances. Assessment for the level of agreement in terms of the intraclass correlation coefficients followed Fleiss and Altman’s benchmark scales. ,

P‐values < 0.0001 in all instances.

ODSS, Oral Disease Severity Score; PGA, Physician Global Assessment; IQR, interquartile range; CI, confidence interval.

Five were familiar with the scoring systems. Prior to the study, those unfamiliar were sent training slides demonstrating the ODSS and PGA. On the study day, all clinicians met for a detailed discussion of methodologies and a calibration exercise using clinical images. Each patient was scored by all 10 clinicians, with two clinicians rescoring all patients after a 2‐h interval providing 12 sets of scores.

Fifteen subjects were required to achieve intraclass correlations (ICC) of 0.77 for the interobserver reliability. Intra‐observer reliability was tested by two raters performing two replications. This provided 80% power to detect an ICC difference of 0.50 (null value of 0.20). ICC level of agreement for ordinal or continuous measures followed Fleiss and Altman’s benchmark scales and Landis and Koch’s benchmark values for categorical outcomes. , ,

Sixteen patients (15 F : 1 M) with OLP and a mean (SD) age of 65 (8.4) (range 48–78) years were included (Table 1). The mean (SD) total ODSS score of 24.5 (10.6) (range 7–56) and median (interquartile range) 23 (16–30) reflected mild to moderately severe disease. The mean and median ODSS site, activity and pain scores are listed in Table 1. The mean PGA score was 3.4 (1.8) (range 0–9) and median 3 (2–4).

The interobserver ICC (95% confidence interval) for the ODSS total was 0.98 (0.96–0.99). For the PGA, the ICC was 0.96 (0.91–0.98). Intra‐observer agreement between initial scoring and rescoring of the same patients demonstrated an ICC for ODSS total of 0.85 (0.63–0.95) and 0.92 (0.71–0.98). The data pertaining to the inter‐ and intra‐observer for site, activity and pain are presented in Table 1. All were rated as excellent. The PGA ICCs were 0.75 (0.42–0.91) and 0.88 (0.69–0.96).

There was good correlation between the ODSS total score and PGA (0.753, P < 0.0001). The mean (SD) time taken to complete the ODSS (total) was 124 (40) s. The PGA time was not recorded as it took less than 5 s.

This study has demonstrated ODSS to be a valid tool for assessing OLP with excellent intra‐ and interobserver reliability.

Sixteen participants were more than adequate to assess reliability. Using two clinicians to rescore patients resulted in 30 datasets compared with 20 if each clinician had rescored one patient. A minimum 2‐h interval reduced recall bias.

No gold standard scoring system exists for OLP. Many disease severity scoring systems have been proposed, including two, the Modified White–Erosive–Atrophic (WEA‐MOD) and Reticular–Erythematous–Ulceration (REU), that have been partially validated. No other system records as many clinical sites as the ODSS and the clinical descriptors are limited in comparison, which may mask subtle clinical changes.

The ODSS was designed by a consensus of experts and being already validated for use in PV and MMP offers wide clinical application and is quick to use. For use in OLP white asymptomatic lesions are given a site score proportional to the size of the area affected but an activity score of zero. The composite score encompassing 17 oral sites, activity and pain ensures a granular assessment and facilitates accurate monitoring of treatment response. The ODSS is primarily a disease severity scoring system; the subjective element of ODSS does not replace validated patient‐reported outcome measures, such as the Chronic Oral Mucosal Disease Questionnaire. Construct validity was not examined, and this may be an area for further investigation. There was no difference in the reliability of scores between the clinicians familiar with and new to the ODSS. Clinician feedback was positive (‘quick to learn’, ‘easy to use’, ‘accurate’).

We propose that the ODSS be considered for routine recording of sequential disease activity in the clinic as well as in future multicentre studies.

Author contributions

Helen McParland: Conceptualization (equal); investigation (equal); methodology (equal); project administration (equal); writing – review and editing (equal). Priya Thakrar: Conceptualization (equal); data curation (equal); investigation (equal); methodology (equal); writing – review and editing (equal). Ana Nora Donaldson: Formal analysis (lead); methodology (equal). Manoharan Andiappan: Formal analysis (lead); methodology (equal). Richard Cook: Investigation (equal); writing – review and editing (equal). Michael Escudier: Investigation (equal); writing – review and editing (equal). Esther Hullah: Investigation (equal); writing – review and editing (equal). Roddy McMillan: Investigation (equal); writing – review and editing (equal). Jennifer Taylor: Investigation (equal); writing – review and editing (equal). Pepe Shirlaw: Investigation (equal); writing – review and editing (equal). Stephen Challacombe: Conceptualization (equal); investigation (equal); methodology (equal); writing – review and editing (equal). Jane Setterfield: Conceptualization (lead); data curation (equal); formal analysis (equal); investigation (equal); methodology (lead); project administration (equal); supervision (lead); validation (lead); writing – review and editing (equal).