Classification of the journal category "oral surgery" in the Scopus and the Science Citation Index Expanded: flaws and suggestions.

OBJECTIVES: The aim of this study was to evaluate the journal category "oral surgery" in Scopus and in the Science Citation Index Expanded (SCIE).
MATERIALS AND METHODS: The Journal of Oral and Maxillofacial Surgery (JOMS), The Journal of the Korean Association of Oral and Maxillofacial Surgeons (JKAOMS), and The Journal of Prosthodontic Research (JPR) were selected from the Scopus list of journals as oral surgery journals. Maxillofacial Plastic and Reconstructive Surgery (MPRS) was selected from PubMed as a Scopus oral surgery title. From these titles, 10 recently published articles were collected and used for reference analysis.
RESULTS: The percentage of citations from oral surgery journals was 26.7%, 24.5%, and 40.1% for JKAOMS, MPRS, and JOMS, respectively. In total, 1.1% of JPR's citations were from oral surgery journals and significantly fewer from other journals (P<0.001). The percentage of citations from dentistry journals excluding oral surgery journals was 11.9%, 34.4%, and 15.8% for JKAOMS, MPRS, and JOMS, respectively. For JPR, 80.6% of citations were from dentistry journals and significantly more were from other journals (P<0.001).
CONCLUSION: Selected samples revealed that JPR is incorrectly classified as an oral surgery journal in Scopus. In addition, the scientific interaction among JKAOMS, MPRS, and JOMS was different to JPR in the reference analysis.

I. Introduction

There are numerous indexing services for journals, and most are not recognized by the government. Thus far, Science Citation Index Expanded (SCIE) and the Korean Citation Index (KCI) are currently recognized by the Korean government. Scopus and PubMed are used as additional evaluation tools, but they are not widely accepted in Korea. However, some European countries use the h-index to evaluate individual performance among researchers; these data are provided by Scopus. Accordingly, the influence of Scopus has been expanded. Publishing a manuscript in a high-impact journal may be an advantage for researchers with regard to grants and reputation1. Accordingly, indexed journals with high-impact factors will have more opportunity to publish higher quality articles. Regional journals may lose the opportunity to publish high-quality articles and to be accepted by major indexing services. When this trend is combined with the underestimation of specific fields, scientific advancement can be delayed.

Dentistry journals have been underestimated by Clarivate Analytics and Scopus for several decades in comparison with medicine journals. Oral surgery and dentistry journals have been placed in the same category by Clarivate Analytics2. This practice has also been used with the medicine category as it comprises several fields, including allergy, anatomy, anesthesiology, cardiovascular system, clinical neurology, dermatology, and emergency medicine2. Scopus is much better than Clarivate Analytics because it classifies seven different fields as part of the dentistry category3. However, Scopus also has many flaws in their classification system. For example, the oral surgery category comprises 59 titles, but many may not be classified as an oral surgery title. Of these 59 titles, some no longer publish under the same name, including Dental Cadmos, Mund-, Kiefer- und Gesichtschirurgie, Oral Surgery, and Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology, and Endodontics. The Journal of Maxillofacial and Oral Surgery was indexed to Scopus, but it is not anymore. Because the category dental implantology comprises multidisciplinary fields, it should be classified separately. If these titles are excluded from the oral surgery category, only 26 titles remain in the oral surgery category of Scopus.(Table 1) We also selected titles that do not fall into the oral surgery category of Scopus.(Table 2) SCIE places dentistry, oral surgery, and medicine collectively in the same field2. The total number of journals in this field is 90. Among them, only 10 titles may be classified as oral surgery titles.(Table 3)

Table 1

Journal titles indexed to Scopus and properly classified as oral surgery

No.	Journal title
1	Asian Journal of Oral and Maxillofacial Surgery
2	Atlas of the Oral and Maxillofacial Surgery Clinics of North America
3	British Journal of Oral and Maxillofacial Surgery
4	Cleft Palate-Craniofacial Journal
5	Craniomaxillofacial Trauma and Reconstruction
6	Dental Traumatology
7	International Journal of Oral and Maxillofacial Surgery
8	Italian Oral Surgery
9	Journal of Cranio-Maxillo-Facial Surgery
10	Journal of Oral and Maxillofacial Surgery
11	Journal of Oral and Maxillofacial Surgery, Medicine, and Pathology
12	Journal of Oral Medicine and Oral Surgery
13	Journal of Stomatology, Oral and Maxillofacial Surgery
14	Journal of the Korean Association of Oral and Maxillofacial Surgeons
15	Medecine Buccale Chirurgie Buccale
16	Minerva Stomatologica
17	Operations Research for Health Care
18	Oral and Maxillofacial Surgery
19	Oral and Maxillofacial Surgery Cases
20	Oral and Maxillofacial Surgery Clinics of North America
21	Oral Oncology
22	Oral Oncology Extra
23	Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology
24	Revista Española de Cirugía Oral y Maxilofacial
25	Revue de Stomatologie et de Chirurgie Maxillo-Faciale
26	Revue de Stomatologie, de Chirurgie Maxillo-faciale et de Chirurgie Orale

Table 2

Journal titles indexed to Scopus and improperly classified as oral surgery

No.	Journal title
1	Archives of Orofacial Sciences
2	Contemporary Clinical Dentistry
3	Dental Cadmos
4	Dental Press Journal of Orthodontics
5	International Journal of Periodontics & Restorative Dentistry
6	International Journal of Prosthodontics
7	Journal of Adhesive Dentistry
8	Journal of Advanced Prosthodontics
9	Journal of Indian Prosthodontist Society
10	Journal of Oral Pathology & Medicine
11	Journal of Orofacial Orthopedics
12	Journal of Orofacial Sciences
13	Journal of Prosthetic Dentistry
14	Journal of Prosthodontic Research
15	Orthodontics and Craniofacial Research
16	PROtech
17	Protetyka stomatologiczna
18	Revista Dental Press de Ortodontia e Ortopedia Facial

Table 3

Journal titles indexed to Science Citation Index Expanded (SCIE) that can be classified as oral surgery journals

No.	Journal title
1	British Journal of Oral and Maxillofacial Surgery
2	Cleft Palate-Craniofacial Journal
3	International Journal of Oral and Maxillofacial Surgery
4	Journal of Cranio-Maxillofacial Surgery
5	Journal of Oral and Maxillofacial Surgery
6	Journal of Stomatology, Oral and Maxillofacial Surgery
7	Medecine Buccale Chirurgie Buccale
8	Oral and Maxillofacial Surgery Clinics of North America
9	Oral Oncology
10	Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology

If the titles are from the same category, there should be similarity among research fields. This similarity can be proved by analyzing references1. The hypothesis of this study was that the titles in the same category would have higher interactions. If some titles showed extremely low numbers of citations compared with other titles in the same category, the classification may be incorrect. The purpose of this article was to evaluate the classification system of major indexing services for journal titles in the oral surgery category by analyzing references, through which the relevance between the journal title and category could be determined.

II. Materials and Methods

Four different titles were selected arbitrarily for this analysis. The Journal of Oral and Maxillofacial Surgery (JOMS)45678910111213 was one journal title selected from journals indexed by Scopus and SCIE that was correctly classified as an oral surgery title.(Tables 1, 3) The Journal of the Korean Association of Oral and Maxillofacial Surgeons (JKAOMS)14151617181920212223 was one journal title selected from the journals indexed by Scopus only and was correctly classified as an oral surgery title.(Table 1) One journal title that was selected from journals not indexed by Scopus or SCIE and classified as an oral surgery title was Maxillofacial Plastic and Reconstructive Surgery (MPRS)24252627282930313233. One journal title that was selected from journals indexed by Scopus and SCIE and incorrectly classified as an oral surgery title by Scopus (Table 2) was The Journal of Prosthodontic Research (JPR)34353637383940414243.

The inclusion criteria were (1) being an original article or a review article and (2) having at least 30 references. Editorials, case reports, and technical reports were excluded. Articles were collected beginning on January 1, 2018. When 10 articles that met the criteria were collected, collection was finished. If any journal title that met the inclusion criteria in 2018 had less than 10 articles, the deficiencies were replenished by the articles published in 2017.

The references were classified into three categories (oral surgery, dentistry, and other). Among Scopus- or SCIE-indexed journals, classification as an oral surgery title was based on Tables 1 and 3. For unindexed titles, the terms “surgery” and “oral/maxillofacial” were used as the keywords for determining the classification category. Other titles in the dentistry category were classified as dentistry journals. Any title that was not classified as an oral surgery or dentistry title was classified as other.

After counting the total number of references in each article, the number of citations for each category was recorded. The ratio of citations among each category was calculated by dividing each value by the total number of references. The interaction between the three selected oral surgery titles and titles incorrectly classified by Scopus was analyzed.(Table 2) Additionally, the self-citation ratio was analyzed. Comparison of the four titles was done using ANOVA. For post hoc testing, Bonferroni's method was used. The level of significance was set at P<0.05. Statistical analysis was performed using IBM SPSS Statistics 23.0 (IBM Corp., Armonk, NY, USA).

III. Results

The total number of references for each journal title was 463, 362, 411, and 459 for JKAOMS, MPRS, JOMS, and JPR, respectively. In the case of MPRS, two articles were collected from 2017. The rate of self-citation was 0.2%, 4.1%, 10.2%, and 2.0% for JKAOMS, MPRS, JOMS, and JPR, respectively. JOMS showed a relatively higher self-citation rate compared with the other three titles. When analyzing the citations in JKAOMS, MPRS, and JOMS from the titles listed in Table 2, the percentage of citations was 0.4% for JKAOMS, 4.4% for MPRS, and 0.2% for JOMS.

The percentage of citations from oral surgery journals was 26.7%±18.4% for JKAOMS, 24.5%±22.7% for MPRS, and 40.1%±28.0% for JOMS.(Fig. 1) However, the percentage of citations from oral surgery journals was only 1.1%±1.3% for JPR. This difference among groups was statistically significant (P<0.001). On post hoc test, JPR had a significantly lower value compared with that of JKAOMS and JOMS (P=0.046 and P=0.001, respectively).

Fig. 1

The percentage of citations from oral surgery journals. JPR had a significantly lower value compared with other journal titles (P<0.001). (JPR: The Journal of Prosthodontic Research, JKAOMS: The Journal of the Korean Association of Oral and Maxillofacial Surgeons, MPRS: Maxillofacial Plastic and Reconstructive Surgery, JOMS: The Journal of Oral and Maxillofacial Surgery)

The percentage of citations from dentistry journals excluding oral surgery journals was 11.9%±9.2% for JKAOMS, 34.4%±32.4% for MPRS, and 15.8%±14.0% for JOMS (Fig. 2), while the percentage of citations from dentistry journals excluding oral surgery journals was 80.6%±13.5% for JPR. The difference among groups was statistically significant (P<0.001). On post hoc test, JPR had a significantly higher value compared with the other journal titles (P<0.001).

Fig. 2

The percentage of citations from dentistry journals excluding oral surgery journals. JPR had a significantly higher value compared with other journal titles (P<0.001). (JPR: The Journal of Prosthodontic Research, JKAOMS: The Journal of the Korean Association of Oral and Maxillofacial Surgeons, MPRS: Maxillofacial Plastic and Reconstructive Surgery, JOMS: The Journal of Oral and Maxillofacial Surgery)

IV. Discussion

Considering the increasing influence of major indexing services on the scientific community, extremely few studies have evaluated indexing services. To the best of our knowledge, this was the first research evaluating indexing services in the oral surgery category. According to the selected samples, the current oral surgery category was flawed in both Scopus and SCIE. The interaction pattern based on the reference analysis of oral surgery titles such as JKAOMS, MPRS, and JOMS was significantly different compared with dental journals such as JPR.(Fig. 1, 2) This preliminary study demonstrated that oral surgery journals had different patterns of interaction than dental journals and therefore should be classified as an independent category.

The percentage of citations from oral surgery journals was at least 20% in three oral surgery journals; however, it was approximately 1% for JPR.(Fig. 1) This is considered a major difference and should be considered an error in Scopus classification. Some authors claimed that a high percentage of mutual citations may be evidence of a citation cartel1. However, a journal article having a reasonable number of mutual citations should help determine the journal category. Arbitrary selection criteria were used in this study to classify a journal as an oral surgery journal; the criteria were having the word “surgery” and “oral/maxillofacial” in the title. In this study all three selected titles showed similar performance.(Fig. 1) Thus, the selection criteria for classifying a title as an oral surgery title in this study seemed functional.

In the case of MPRS, the percentage of citations from dentistry journals excluding oral surgery journals was higher compared with the other two titles.(Fig. 2) The reason for this could be that articles associated with dental implants showed higher interaction with dentistry journals, and not oral surgery journals. Yu et al.24 and Kakar et al.25 were associated with implant restorations 87% and 73% of their citations, respectively, were from dentistry journals excluding oral surgery journals. In addition, MPRS includes articles about maxillofacial application of tissue engineering.

The percentage of citations from dentistry journals excluding oral surgery journals was higher in JPR.(Fig. 2) In the case of orthodontic journals, some articles show high interaction with oral surgery journals. One article that was selected from The American Journal of Orthodontics and Dentofacial Orthopedics (AJODO) had 25 references, 10 of which are from oral surgery journals and eight of which are from dentistry journals excluding oral surgery journals44. However, another article45 selected from the same journal did not have any references from oral surgery journals, despite having 55 references. Because there was a high level of variance in topics, collecting big data will be necessary to draw reliable conclusions.

The limitation of this study was that the number of selected samples was small considering the vast number of publications in the field of oral surgery and dentistry. There were no definite title selection criteria for each category. In addition, there were no guidelines about how to manage multidisciplinary fields of oral surgery having close interactions with other dental categories such as dental implantology624 and orthognathic surgery44. In this article, any title including the word “implant” in was excluded from oral surgery titles. In addition, any title including the word “orthodontic” was also excluded from the oral surgery category. These exclusions are not relevant to the clinical practice of oral surgery specialists. If all oral surgery/dentistry titles were included using a cross-reference database, more confirmative results could be obtained. Because this study was a preliminary pilot study, subsequent confirmative studies are needed.

V. Conclusion

Analysis of selected samples demonstrated that some oral surgery titles had low interaction with other oral surgery titles. Therefore, the decision to include or exclude a journal from a particular category in Scopus and SCIE should be reviewed. If errors are discovered, revisions should be made accordingly. In addition, the scientific interaction of oral surgery titles may vary from that for other dental titles. Thus, oral surgery titles should be separated from dentistry titles and established as an independent category.