An Update on the Level of Evidence for Plastic Surgery Research Published in Plastic and Reconstructive Surgery.

In 2011, Plastic and Reconstructive Surgery (PRS) implemented a level-of-evidence (LOE) pyramid to bring attention to evidence-based medicine and to promote quality of research. The objective of our study was to examine the current, overall quality of plastic surgery research when compared with that of the previous 30 years. Articles from PRS published in 2013 were culled for information, including LOE based on the assigned score from the LOE pyramid. (Animal, cadaver, and basic science studies; reviews; correspondence; and continuing medical education articles were excluded.) The LOE grades were compared with those from 1983, 1993, and 2003. In 2013, 536 articles were published in PRS; of these, 247 met the inclusion criteria and were included in the analysis. The mean LOE in PRS for 2013 was 3.42. For the year 2003, the mean LOE was 4.16; 1993, 4.25; and 1983, 4.42. Analysis of variance indicated significant improvement in research quality over time (P < 0.001). In 2014, 216 of 489 published articles met the inclusion criteria. The mean LOE of PRS articles in 2014 was 3.33, demonstrating continued higher LOE. There was also a decrease in the percentage of level IV and V studies to 47.2% (from 51.4% in 2013), whereas higher quality level I and II studies had increased to 18.1% (from 17.4%). The quality of plastic surgery research has shown a continued upsurge as evidenced by overall improvement in LOE in published articles over the past 3 decades.

Evidence-based medicine (EBM), which incorporates research outcomes into clinical decision making, came into common practice in the 1990s when more resources, such as those described by the Evidenced-Based Medicine Working Group, became available.[1] During the 1960s and 1970s, notable figures, including Dr. Feinstein[2] and Dr. Cochrane,[3] had begun to question the validity of surgical practices that had previously been thought to be effective. After careful assessment, they discovered considerable variation in practice policies from provider to provider and a dearth of reliable research to substantiate many clinical decisions. On the basis of their findings, they stressed the need for additional research trials that could scientifically prove the efficacy of treatments. In 1979, levels of evidence (LOE) were first described by the Canadian Task Force on the Periodic Health Examination.[4] The 1980s then saw a dramatic shift in the practice of medicine across specialties, with more and more clinicians basing care and clinical decisions on sound study outcomes.

LOE is a method of quantifying the strength of scientific studies and identifies potential sources of bias. Over the years, several classifications have been proposed, and although there are subtle differences, they are essentially the same in their grading—with randomized controlled trials carrying the most weight and case reports carrying the least. The most common classification system presently used in the plastic surgery literature grades articles based on a scale from I to V, with I as the highest quality and V as the lowest quality.[5] In 2011, the LOE pyramid was implemented for the plastic surgery literature. Reporting the LOE helps to highlight the quality of the research and the potential for bias so that the reader may prioritize information accordingly.[5] However, the LOE classification system cannot be considered an absolute guide. Not every question can be answered by reports of level I evidence, and not all questions in the practice of plastic surgery are amenable to randomized controlled trials. The best LOE may, in fact, be a level II study.

Plastic surgery is a unique discipline within medicine. Unlike many other specialties, plastic surgery includes an essential element of artistry. Although a universal instrument is yet to be developed that truly captures this aspect of the intervention, great strides have been made with instruments like the Breast-Q[6] and the Face-Q,[7] which capture patient-reported outcomes and are therefore important indicators of aesthetic outcomes. In other ways, we are not unique. Many dated procedures and techniques based on expert opinions and anecdotal evidence continue to be practiced without scrutiny. The need for EBM in plastic surgery is, therefore, no different from that in other medical specialties.[8] By continuously reexamining the data behind practices and outcomes, advances can be made. The purpose of this study was to determine the current, overall LOE in the plastic surgery literature and to ascertain its progression over the past 30 years.

METHODS

We reviewed all the articles published in each 2013 issue of Plastic and Reconstructive Surgery (PRS) and categorized the studies by LOE. Level I evidence consisted of high-quality randomized controlled trials that were adequately powered and the systematic reviews of such studies. Level II publications consisted of lesser quality randomized controlled trials, prospective cohort studies, and systematic reviews of those studies. Level III studies consisted of retrospective comparative studies and case–control studies. Level IV studies were typically of the case-series variety, and level V articles were usually case reports or expert opinions. This corresponded to the journal’s current LOE and followed the therapeutic, diagnostic, and prognostic study evaluation as specified by the journal. Nonclinical articles, animal studies, basic science studies, cadaver studies, review articles, correspondence (letters), and continuing medical education articles were excluded. We also used the same methods for an ad hoc analysis of articles published in 2014.

Methods were consistent with those of earlier work by Loiselle et al,[9] and the current 2013 data were compared with data from 1983, 1993, and 2003 from the previous article.[9]

Statistical Analyses

Data were analyzed by using SPSS software (IBM, Armonk, N.Y.). Data are expressed as means. One-way analysis of variance for repeated measures was used for comparison of group means and to determine the presence of significant differences between group means. The level P < 0.05 was considered statistically significant.

RESULTS

In 2013, 536 articles were published in PRS, of which 247 met the inclusion criteria. There were 9 randomized controlled trials, 2 meta-analyses, 10 systematic reviews, 39 prospective studies, 22 case–control studies, 82 retrospective studies, 66 case series, 10 cross-sectional studies, and 7 case reports. Of the studies, 7 (2.8%) were level I evidence studies, 36 (14.5%) were level II, 77 (31.2%) were level III, 105 (42.5%) were level IV, and 22 (8.9%) were level V (Figs. 1, 2). The mean LOE found in PRS was 3.42. When articles were categorized and compared, there was a significant difference (P < 0.05). However, no significant difference was found in LOE among the topics of publication [breast, 54; cosmetic, 46; hand, 36; pediatric/craniofacial, 46; and reconstructive, 66 topics (P > 0.05)].

Fig. 1.

The distribution (%) of LOE in studies published in PRS in 2013.

Fig. 2.

The number of studies according to LOE that were published in PRS in the past 3 decades.

Data from 2013 were then compared to data collected from 1983, 1993, and 2003 for which the mean LOE was 4.42, 4.25, and 4.16, respectively. An analysis of variance showed a significant year-to-year difference (P < 0.001), indicating an overall shift toward higher quality LOE research (Fig. 3). Although lower quality LOE studies (ie, level IV or V) initially comprised the vast majority of publications, a substantial decrease in these levels has occurred over time: from 92.8% in 1983; 87.6%, 1993; 86.9%, 2003; to 51.4% in 2013. At the same time, level I and II studies, deemed as higher quality, increased more than 5-fold between 1983 and 2013: 3.4% to 17.4%.

Fig. 3.

The mean LOE of studies published in PRS in the past 3 decades.

Of the articles reviewed in 2014, we found that 216 of 489 published articles met our inclusion criteria. There were 12 randomized controlled trials, 3 meta-analyses, 7 systematic reviews, 37 prospective studies, 130 retrospective studies, 14 case series, 3 cross-sectional studies, and 10 case reports. Of the studies, 4 (1.9%) were level I evidence studies; 35 (16.2%), level II; 75 (34.7%), level III; 89 (41.2%), level IV; and 13 (6.0%), level V. The mean LOE of PRS articles in 2014 was 3.33, demonstrating continued higher LOE. There was also a decrease in the percentage of level IV and V studies to 47.2% (from 51.4% in 2013), whereas higher quality level I and II studies had increased to 18.1% (from 17.4%).

DISCUSSION

The results of this study show continued, higher quality research, as measured by LOE, although surgical research still lags behind that of other medical disciplines. For example, even in 1995, 53% of internal medicine decisions were supported by randomized controlled trial evidence,[10,11] whereas only 17% of studies in PRS in 2013 were level I and II evidence. The lag in surgical research in plastic surgery has occurred because of the difficulty in applying more sophisticated study designs to clinical questions in surgery. This lag is also evident in other surgical disciplines. For example, in Diseases of the Colon and Rectum, there were 5 randomized controlled surgical trials in 1990, 13 in 1995, and 17 in 2000.[12]

Medical disciplines continue to make a concerted, progressive effort toward evidence-based practice as a way not only to validate and standardize care but also to ensure optimal outcomes. However, inherent difficulties remain in conducting randomized controlled trials in surgery. Hassanein et al[13] described several common obstacles: frequent lack of equipoise both for investigators and for patients, higher costs of surgical interventions, difficulties in enrolling patients in a placebo control, and the subjective nature of outcomes. It is certainly difficult for surgeons to justify randomizing patients to the investigative arm of a study with unproven outcomes when they have had relatively positive experiences with the technique being used for the control group. Similarly, it may also be challenging to convince patients to subject themselves to an experimental procedure with unknown outcomes when they are aware that a traditionally reliable option exists. Perhaps more than in any other specialty, success in plastic surgery relies more on patient-reported outcomes and surgeon assessment of outcomes than on any numeric metric, which makes quantifying outcomes in this specialty much more difficult. Furthermore, unlike trials designed to determine efficacy of medications, trials of surgical techniques can be difficult to control because the techniques may be hard to reproduce and are dependent upon surgeon experience and skill.

However, those involved in surgical specialties are continuing to improve the quality of published research studies and have made great strides over recent years, especially in neurosurgery but also in orthopedic, otolaryngologic, and ophthalmologic surgery.[14,15] Although plastic surgery lags slightly behind these other specialties, the gap has narrowed substantially. In the past, lower ranked LOE publications, that is, levels IV and V, comprised the vast majority of publications (more than 80% of publications each year). Recently, however, this percentage has decreased substantially (47.2% in 2014), replaced by higher quality research. Before 1980, <5 studies of level I evidence were published per year in all of the plastic surgery literature. That number has since risen to >30 studies per year.[16] Within the aesthetic subcategory of plastic surgery, level I evidence has increased from 0% to 2.5% over the past decade.[15]

As necessary as it is to improve the quality of research studies, one cannot dismiss weaker studies simply because of LOE. Many innovations and discoveries in plastic surgery have come from intrepid pioneers willing to try something new on a small scale. Conversely, a study can be very well conducted yet provide little impact if the question under investigation is of little consequence or interest. Furthermore, one must remember that while a randomized controlled trial may serve as the highest form of evidence, not every question posed can or should be answered with a controlled trial. A question regarding epidemiology may be better answered by a cross-sectional study, and concerns about long-term effects of any intervention should be examined with a longitudinal study, neither of which are often considered level I evidence. Therefore, one must be careful to remember that LOE is not necessarily synonymous with clinical guidelines because practitioner experience, patient characteristics, and research data all contribute to clinical decision making.

A disadvantage of the current study is that it examines only 1 journal in plastic surgery, PRS. We did choose the plastic surgery journal with the highest impact factor, which serves as a proxy measure for the best publication. Generalizations about LOE in other general plastic surgery journals and those with a more specific focus, such as in the disciplines of hand, microsurgery, or craniofacial surgery, can not necessarily be made based on this study. This study also compares data gathered only at 10-year intervals and therefore does not account for year-to-year discrepancies. We were able to include data from 2014 because that data were available at the time we submitted the manuscript, and we thought that it would be interesting to determine whether the tendency toward increased quality continued.

An analysis of the reasons for the increase in the quality of publications is beyond the scope of this article. However, there are several potential factors. For example, substantial changes have been made to the editorial policies of the journal, the instructions to reviewers, education of the plastic surgery community about LOE, education within residency programs, and added research requirements for graduation from residency programs. Also, the addition of new avenues for publication, such as open access journals, raises the question of whether these other types of publications siphon a portion of the lower quality submissions.

As information has become more readily available and consumers have grown more aware of treatment options, the need for guidelines and protocols that correspond to the best outcomes has become increasingly necessary. Applied correctly, EBM can also challenge the current, unsustainable increase in health care costs by eliminating unsubstantiated practices and appropriately allocating resources.[17,18]

CONCLUSION

Our study showed that the quality of research in plastic surgery has seen a continued upsurge as evidenced by an overall improvement in published LOE over the past 3 decades.