Levels of Evidence in Plastic and Reconstructive Surgery Research: Have We Improved Over the Past 10 Years?

Levels of evidence (LOE) aid in the critical appraisal of evidence by ranking studies based on limitation of its design. Analyzing LOE provides insight into application of evidence-based medicine. The aim of this study is to determine if the quality of evidence in plastic surgery research has improved over the past 10 years. Systematic review of research published in Plastics and Reconstructive Surgery journal over the years, 10-year period (2008, 2013, 2018), was performed. LOE for each article was determined using the American Society of Plastic Surgeons (ASPS) guidelines. Each level was calculated as percentage of publications per year and compared yearly and between different topics. Eight hundred eighty-four studies were included in the final analysis. The LOE of the research improved over the study period. Level 4 evidence was the most frequent published (50.6%, 447/884), with a decline from 63.2% in 2008 to 41.3% in 2018. Level 1 evidence improved each year and accounted for 2.1% of all research in 2018. Aesthetic surgery was the most frequent published topic with upper limb research demonstrating an 18.5% increase in high-quality evidence over the study period. Increased awareness of evidence-based medicine has improved the quality of plastic surgery research over the past decade. It is vital this continues to provide gold standard patient care.

INTRODUCTION

Advances in technology, improved understanding of disease pathogenesis, and superior interventions have enhanced plastic surgery patients’ outcomes over the past decade.[1,2] To support these advancements, up to date research endeavors are mandatory. The resultant increase in studies has generated an overwhelming amount of evidence. To assist clinicians in critical appraisal of this evidence, a conceptual tool known as evidence-based medicine (EBM) was developed.[3]

Levels of evidence (LOE) is the foundation of EBM. It is a hierarchical appraisal system which grades research (levels 1–5) based on inherent limitations of study methodology.[4] LOE enables clinicians to rapidly appraise evidence before translating into clinical practice.[5] It is also a reliable method of evaluating the quality of evidence published. High-quality research is a prerequisite in maintaining optimal patient care. The aim of this study is to determine if the quality of evidence in plastic surgery research has improved over the past 10 years.

METHODOLOGY

A systematic review of published research articles was performed in Plastic and Reconstructive Surgery (PRS) journal. Articles were selected from this journal, as it is the highest impact factor plastic surgery journal, publishing on a wide variety of plastic surgery topics. To evaluate any trends, articles were initially reviewed from 3 years, covering a 10-year period (2008, 2013, 2018). Editorials, letters, announcements, reflections, book reviews, Continuing Medical Education (CME) articles were excluded from this study. Review articles and laboratory studies (animal, cadaver, basic science) were included in the initial review but excluded from the final analysis, as no LOE can be allocated to these studies.

Each clinical article was allocated an LOE based upon published American Society of Plastic Surgeons (ASPS) guidelines.[6] First, the research aim was broadly divided into 3 categories: therapeutic, risk, and diagnostic. Within these categories, evidence was ranked from 1 to 5 mirroring the hierarchal research pyramid. This ranking is based upon the probability that the research design has reduced the potential bias. Highest quality evidence (level 1) is produced from randomized control trials or systematic reviews/meta-analysis of these. Articles of limited study design with biases, such as expert opinions, are ranked the lowest (level 5).

Two authors (C.M.S., C.W.J.) independently evaluated published research articles. Discrepancies in the assignment of LOE were discussed with the senior author (S.M.C.). Information obtained from the articles included year of publication, topic, study design, and LOE. Each LOE was expressed as percentage of the overall publications that year and to the other years. Study design was also assessed. Further analysis on the different topics published in the journal was performed. The percentage of higher-level evidence (levels 1 and 2) was calculated for each topic and compared over the 10-year period.

RESULTS

Two thousand six hundred sixty-four articles were published in the PRS journal in the years 2008, 2013, 2018. By applying the inclusion criteria, a total of 1,369 articles were reviewed. Review articles and laboratory studies accounted for 14.7% and 20.7% of the remaining articles. These were excluded from the final analysis as they are not part of LOE hierarchy. A LOE rank was applied to 884 articles (2008 = 313, 2013 = 291, 2018 = 280) (Fig. 1).

Fig. 1.

Flow diagram of study methodology. CME indicates continuing medical education; LOE, levels of evidence.

Therapeutic studies were the most frequent research aim, accounting for 83.6% of all research. Level 4 evidence was the greatest level published across the years (50.6%, 447/884). There was a decline in the percentage of level 4 evidence from 63.2% in 2008 to 41.3% in 2018. Twelve publications accounted for level 1 evidence, with 11 of these published in the past 5 years (Table 1).

Table 1.

Percentage of Each Level of Evidence Published per Year

Levels of Evidence	2008	2013	2018
1	0.3	1.7	2.1
2	6.3	11.3	13.6
3	19.6	33.3	34.5
4	63.2	45.5	41.7
5	11.3	8.5	7.9

Case series was the most common study design (36.4%, 498/1,369). This study designed decreased over time from 48.3% in 2008 to 25.5% in 2018. Case–control studies increased from 9.7 % to 19.9% (Table 2).

Table 2.

Evaluation of Each Study Methodology Used in Research, per Year

	2008n (%)	2013n (%)	2018n (%)
Systematic review/meta analysis	5 (1.1)	21 (4.6)	15 (3.2)
Randomized control trials	8 (1.7)	5 (1.1)	18 (3.8)
Cohort study	4 (0.9)	10 (2.2)	8(1.7)
Case–control	44 (9.7)	75 (16.7)	93 (19.9)
Case series	219 (48.3)	156 (34.7)	123 (26.3)
Case report	10 (2.2)	9 (2.0)	9 (1.2)
Expert opinion	25 (5.5)	15 (1.1)	12 (2.1)
Review article	45 (9.3)	61 (15.8)	99 (22.2)
Laboratory study	96 (21.1)	97 (21.6)	90 (19.3)
	n = 453	n = 449	n = 467

Aesthetic (21.6%), breast (17.3%), craniofacial (4.2%), and upper limb surgery were the most frequently published topics in PRS journal. The percentage of higher LOE (levels 1 and 2) published in these topics over the years is seen in Table 3. The largest percentage increase over the 10-year period was demonstrated by upper limb surgery (18.5%).

Table 3.

Percentage Comparison of High-quality Evidence (Levels 1 and 2) per Plastic Surgery Topic, per Year

Topics	2008	2013	2018
Aesthetics	6.3	6.2	15.3
Breast	19.2	13.7	18.7
Craniofacial	1.5	8.3	8.5
Upper limb	11.5	11.1	25

DISCUSSION

The application of EBM involves merging individual clinical experience with the best scientific evidence.[7] The interrogation of EBM into plastic surgery practice has been limited.[8] By evaluating trends in LOE of published research, the utilization of EBM principles can be measured. This study has shown that the LOE in plastic surgery research has improved over the past decade. There has been a growth in levels 1, 2, and 3 evidence, with a reduction in the publication in lower-quality evidence. In 2018, high-quality evidence (levels 1 and 2) accounted for 15.7% of all plastic surgery research. This was marginally lower than orthopedic literature (21.6%),[9] but higher than neurosurgical (10.3%)[10] and maxillofacial research (2%).[11] Within the PRS journal, upper limb and aesthetics surgery demonstrated the largest increase in high-quality evidence over the 10-year period.

Case series are the backbone of surgical research. By evaluating a similar group of patients undergoing a common intervention, this study design replicates everyday surgical practice.[12] The absence of a control group justifiably ranks this design at the lower end of the evidence pyramid. Despite this, case series are vital. They may be the only feasible and ethical study methodology obtainable, as seen with craniofacial surgery.[13] In our study, craniofacial research accounted for the lowest percentage publication (8.4%) of high-quality evidence, with no improvement over the past 5 years. The rarity of craniofacial pathology coupled with a small number of patients makes it difficult to produce higher quality research. Case–control studies are an upgrade from case series, with the addition of a control group significantly reducing study bias.[14] Case –control studies increased in our study period. In a specialty where obtaining high-quality evidence is challenging, the evolution from case series to case–control studies is an important indicator of EBM application.

The concept of LOE was originally described 50 years ago.[15] Yet, its application in plastic surgery research has been underwhelming,[16] with lack of awareness a probable reason.[17] To overcome this, PRS journal, in 2011, made it mandatory for authors submitting manuscripts to attach an LOE rating. This is then displayed as a small pyramidal graphic on the abstract page, providing immediate context for the reader. This editorial policy could account for the greater increase in LOE between 2008 and 2013, in comparison to the past 5 years of this study. Other journals have a similar requirement, but the LOE is allocated by the editorial board, out of fear of authors over infiltrating their own research.[18] However, good interobserver and intraobserver reliability has been reported when grading LOE.[19] By placing the responsibility of LOE ranking with the submitting author, knowledge of EBM has improved along with the quality of evidence published.