Evaluation of interobserver reliability of Nugent score for diagnosis of bacterial vaginosis.

BACKGROUND: Vaginal discharge is the commonly narrated compliant of the female attendees of sexually transmitted infection clinic, among which bacterial vaginosis (BV) is responsible for one-third of the visits. BV is often diagnosed clinically which warrants laboratory confirmation. AIMS: The study aims to detect the reliability of the Nugent scoring system between observers for the diagnosis of BV.
MATERIALS AND METHODS: This is a prospective study including 177 high vaginal swabs. The gram-stained smears were examined by three independent microbiologists, and the Nugent scoring was performed. Statistical analysis was performed using IBM-SPSS version-22 statistical package for kappa value.
RESULTS: Concordant results were seen in 64.03% of smears, discordant results were given in 4.51% of smears, and partial agreement was observed in 31.63% of smears.
CONCLUSION: Interobserver reliability is good for the Nugent score. The Nugent score is a simple and reliable method for the diagnosis of BV that can be adapted even in the resource poor settings.

INTRODUCTION

Bacterial vaginosis (BV) is a polymicrobial syndrome characterized by loss of normal vaginal flora and acquisition of mixed anaerobic bacteria with an increase in the bacterial load from 1000 to 10,000 times above the normal.[1] The prevalence of BV was found to be 19% among sexually active females in Mysore, India.[2] Common clinical presentations are malodorous homogeneous discharge, pruritus, dyspareunia, and lower abdominal pain.[3] There is a bidirectional association between BV and increased risk of acquisition of sexually transmitted infections[456] including HIV.[7] The numerous diagnostic methods available for BV include Amsel's criteria,[8] Nugent scoring,[9] Hays/Ison system,[10] Spiegel's criteria,[11] Schimdt's scoring system,[12] anaerobic culture,[13] proline aminopeptidase test,[14] gas-liquid chromatography,[15] sialidase activity,[1617] and molecular methods.[18] Because of the complexity of bacterial flora in BV, none of the diagnostic methods are currently better than the standardized Gram's stain.[19] Although microscopic methods are simple to perform, cost-effective, rapid, and reproducible, the reliability between various observers is of concern. This study was carried out to evaluate interobserver reliability in interpreting the microscopy of vaginal smears by the Nugent score method.

MATERIALS AND METHODS

This is a prospective study including nonrepetitive high vaginal swabs received from patients with vaginal discharge in microbiology laboratory from January 2015 to May 2015. Ethical and research clearance was obtained from the Institutional Ethical and Research-Committee. Informed consent was obtained from each patient before the procedure. The total number of high vaginal swabs received during the period was 177. One gram-stained smear prepared from each sample was visualized by three independent microbiologists. They were graded as normal (N), intermediate (I), and BV based on the number of bacterial morphotypes [Table 1].[9] Statistical analysis was performed using SPSS 22 (IBM Corp, Armonk, NY, United States of America).

Table 1

Nugent scoring of gram-stained smear for bacterial vaginosis

RESULTS

Gram-stained vaginal smears were examined by three observers, and results were analyzed using IBM-SPSS version-22 for kappa value. The percentage of concordance and discordance were calculated and tabulated. All the three observers scored the smears concordantly in 113 (64.03%) smears. Completely discordant results were given by three examiners in eight (4.51%) smears [Table 2]. BV was diagnosed in 17.7% of females [Table 3]. Analysis of partial agreement results showed that the agreement between two observers in various combinations (observer 1 and 2, observer 2 and 3, and observer 3 and 1) was good based on kappa values. The percent of agreement were 66.7%, 65.5%, and 59.9%, respectively [Table 4]. The interobserver reliability was good to fair in our study.

Table 2

Correlation of the Nugent score results among three observers (n=177)

Table 3

Distribution of results of complete agreement among observers

Table 4

Percentage and kappa statistics for partial agreement between observers

The weighted kappa statistics was used to measure agreement between the observers. Calculated kappa values of <0.4 are considered to reflect poor reproducibility or agreement, those of 0.4–0.75 reflect good to fair agreement, while >0.75 reflect excellent agreement.[20] Since the calculated kappa values in our study were in the range of 0.4–0.75, the results reflect good agreement between the observers.

DISCUSSION

BV is the most common health problem affecting women.[1] It has been implicated in causing higher rates of late miscarriage, premature rupture of membrane, chorioamnionitis, spontaneous preterm labor, and postpartum endometritis.[21] Among various diagnostic methods available for BV, Amsel's criteria are the gold standard method, and the Nugent score is considered as the reference method for microscopic reading of vaginal smears.[10] Amsel's criteria are based on the clinical assessment with which the presence or absence of BV is detected. On the contrary, the Nugent score is laboratory-based and allows for assessment of alteration in vaginal microbial flora as a continuum rather than a dichotomy.[22] Among the four criteria described by Amsel, amine odor was the least accurate, and character of vaginal discharge had the highest accuracy. The combination of only two criteria (pH and KOH test) showed the highest accuracy, which was even more than all four criteria combined together.[23] Based on the accuracy values, Amsel's criteria can be used as a simple bedside test, but wherever resources are available, it is essential to diagnose BV by other reliable method.

In the present study, high vaginal smears were evaluated for interobserver reliability. The percentage of complete agreement achieved in our study was 64.03%. The complete disagreement was 4.51%, where all the three observers gave an entirely different report for eight smears. The partial agreement where two among the three observers reported similarly was seen in 31.63% [Table 2]. All the three observers scored 7.96% of smears as normal, 32.74% of smears as intermediate, and 17.7% of smears as BV [Table 3]. Based on percentage values, the agreement between observer 1 and 2 and observer 2 and 3 was almost equal, but the agreement between observer 3 and 1 was comparatively lower. However, all three sets of observer combinations fell into good reliability based on kappa values [Table 4]. In a similar study done by Mohanty et al.,[24] complete agreement was found in 76.2%, partial agreement was seen in 22.13%, and complete disagreement was seen only in 1.66%. The inter-rater reliability was good approaching to excellent in their study. Another study by Zarakolu et al. also showed that the agreement between observers for interpretation of the Nugent score was good.[25]

The prevalence of BV in our study was 17.7% which is comparable to the study done by Rao et al.[3] where the prevalence was found to be 17.42%. The intermediate flora was 14.4% which is lower compared to 32.7% in our study. The intermediate score is an additional finding in the Nugent score which is not done with Amsel's and Spiegel's criteria. The problem with intermediate flora is that 30% of it can revert back to normal, 30% can progress to BV, and the rest can remain as such. Because of the uncertainty of intermediate flora, it should be reassessed or considered as BV for treatment based on clinical risk.[2] The disadvantage with Spiegel's criteria is that it involves Bacteroides and Gram-positive cocci in its grading, because of which the reliability is low.[9] Hence, Spiegel's criteria which divide vaginal smears into BV and normal flora are not as popular as the Nugent method.[26]

Examination of smears by all the three examiners showed that about 35.34% of smears were not scored due to insufficient material. The various reasons for nonvisualizable smear are the absence of vaginal discharge, improper collection technique, delayed transport of high vaginal swabs to the laboratory, delay in processing, unskilled smearing technique, skipping of drying, and fixation step during preparation of smear or observer error. In our study, most patients had symptomatic vaginal discharge, but few had complaints predominantly of lower abdominal pain and pruritus; swabs were collected by trained professionals; samples were processed immediately on receipt; smears were prepared and stained by trained technicians; and scoring was performed by qualified microbiologists. Since all the three observers reported nonvisualizable smear in concordant samples, the possibility of observer error was eliminated. The possible reasons for nonvisualizable smear in our study could be insufficient sample or delay in transport of swabs to the laboratory.

According to Mohanty et al.,[24] discrepancies in scoring can occur due to various reasons such as different sampling devices, collection technique, site of collection, homogeneity and thickness of vaginal discharge, different methods of fixation, tendency of the old lactobacilli to lose the Gram-positive nature, misinterpretation of diphtheroids as Gardnerella vaginalis and also due to differences in bacterial density, and image area observed under different microscopes. In this context, Hays/Ison system is the best method as image area, and bacterial density has minimal effect in its interpretation. However, it is difficult to teach Hays/Ison system as compared to the Nugent scoring method.[1] The possible reasons for discordant results in our study can be due to differences in identifying various morphotypes by the observers, differences in bacterial density in various fields, and image area of the microscopes.

The eight smears that were given discrepant results by the three observers were examined by an expert microbiologist and were found to have intermediate score. All the three observers reexamined the slides and scored them correctly. The occurrence of discrepancy with intermediate score again emphasizes the need for reassessing the smears whenever such score is encountered. Limitations of the study are that prestudy refresher training for the Nugent score was not given to the observers before participation. Delay in the transport of samples to the laboratory should have been taken care to minimize the percentage of nonvisualizable smears.

CONCLUSION

Interobserver reliability for the Nugent score is good. After considering the constraints and accuracies for various methods, the Nugent score stands out as a reliable method for diagnosing BV.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.