OrthoEvidence™: A Clinical Resource for Evidence-Based Orthopedics.

The prevalence of musculoskeletal issues in clinical practice, and the limited focus placed upon musculoskeletal conditions by current electronic summary resources, highlights the need for a resource that provides access to simple and concise summaries of top-quality orthopedic literature for orthopedic surgeons and allied healthcare professionals. OrthoEvidence™ is an online clinical resource that addresses the paucity of adequate evidence-based summary tools in the field of orthopedic surgery. OrthoEvidence™ uses a rigorous, transparent, and unique process to review, evaluate, and summarize high quality research studies and their implications for orthopedic clinical practice. Randomized controlled trials and meta-analyses are identified and reviewed by an expert medical writing team, who prepare Advanced Clinical Evidence (ACETM) reports: one or two detailed pages including critical appraisals and synopses of key research. These timely and targeted reports provide a clear understanding about the quality of evidence associated with each summarized study, and can be organized by users to identify trending information. OrthoEvidence™ allows members to use their time efficiently and to stay current by having access to a breadth of timely, high-quality research output. OrthoEvidence™ is easily accessible through the internet and is available at the point-of-care, which allows treating orthopedic surgeons and allied health professionals to easily practice the principles of evidence-based medicine within their clinical practices..

Introduction

Evidence-based medicine was recently noted as one of the top 15 most important medical discoveries of the past 160 years. Since the term was coined in the early 1990’s by the Evidence Based Working Group at McMaster University,[2,3] evidence-based medicine has seen unparalleled adoption across medical and surgical specialties. It is a method of practice that placed less emphasis on expert opinion and unsystematic clinical observations, instead stressing the impact of evidence derived from clinical research, such as randomized controlled trials (RCTs) and high quality meta-analyses, requiring physicians to make themselves aware of published results before accepting dogma.

Subsequently, there has been an exponential rise in research papers promoting evidence-based practice across many specialties of medicine. In orthopedic surgery, the terminology collectively referred to as Evidence-Based Orthopedic Surgery has also become a standard language of journals and major orthopedic societies. Furthermore, the practice of evidence-based medicine has also evolved from an initial focus purely on the best available published evidence for a treatment to the present emphasis on the importance of patient values and expected outcomes for the treatment of disease.

While the practice of evidence-based medicine may provide advantages in the efficacious treatment of patients, it is also presents a number of challenges. Evidence-based medicine requires the ability to not only locate the best available research (i.e. RCTs and high quality meta-analyses), but to additionally be able to critically appraise the available evidence. These steps can be challenging for clinicians, who do not have formal training in research methodology, as well as the time to access and read full-text published research papers. Despite being published in academic journals, only a small number of high quality RCTs and meta-analyses are read by their target audience. Consequently, important advances in patient care are not being made available to clinicians. As a result, there is a great need for novel methods of summarizing and appraising the literature, to allow the principles of evidence-based medicine to be effectively followed. To address the need of an efficient way to practice evidence-based medicine, multiple interactive electronic resources that allow rapid access to evidence-based summaries of clinical research have been created over the past decade. These websites help physicians and other health care professionals to follow the principles of evidence-based medicine, by offering continuously updated answers to clinical problems using the internet. Examples include First Consult, Up-to-Date®, DynaMed™, Clinical Evidence, and Physicians’ Information and Education Resource (PIER). A recent study that investigated the surgical content across these five point-of-care sites identified an average of 18% surgical content. Specifically, First Consult and PIER contained 28% surgical content as a percentage of their overall total data™ base content. DynaMed’s surgical content was 14% and Clinical Evidence surgical content was 11%, whereas Up-To-Date® contained only 9.5% surgical content. Overall, general surgery, pediatric surgery, and oncology-related surgeries were the most common specialty areas across the databases. Orthopedic surgery content averaged 2.04% (range 0.2% to 4%) of the total database content and averaged 12.1% (range 2% to 20%) of the surgical content. The most common orthopedic subspecialty areas included general orthopedic surgery representing 26% of the orthopedic summaries (range 9% to 36%), trauma representing 24% of the orthopedic surgery summaries (range 7% to 50%), and pediatric orthopedic surgery representing 21% of the orthopedic summaries (range 7% to 25%). The resources that contained the greatest amount of orthopedic surgical content relative to total content included First Consult with 64 summaries (4%) and DynaMed with 126 summaries (2%). These sites do not take into account content for each sub-speciality within orthopedic surgery.

Musculoskeletal injuries and disorders affect millions of people around the world, representing approximately 40% of a typical general physicians practice. Additionally, orthopedic surgery is a prominent speciality area within medicine today, with relevance expanding beyond surgeons and emergency medicine physicians, to allied health care professionals including occupational therapists, physiotherapists, chiropractors, podiatrists, nurses, and medical students. The high prevalence of musculoskeletal problems and the fact that musculoskeletal research is not the focus of other electronic summary resource websites illustrates the need for a website to provide access to simple and concise summaries of relevant top quality musculoskeletal care literature for orthopedic surgeons and other healthcare professionals who treat patients with musculoskeletal disorders.

Materials and Methods

Implementation

OrthoEvidence™ is a clinical resource that has been developed to address the paucity of adequate evidence-based summary tools in the field of orthopedic surgery. It has quickly become a global online source for high quality and timely evidence-based orthopedic clinical research summaries. OrthoEvidence™ was founded in 2009 with the overall goal of providing critical and timely information access to orthopedic surgeons and healthcare professionals and their patients. It has an easy to access and use online format (Supplementary Figure S1). OrthoEvidence™ is a privately own entity funded by institutional and organizational licenses similar to most online resources in its class. OrthoEvidence™ does not use industry funding to advertise currently but industry licenses are made available any companies for access to the full content of the database.

Structure of OrthoEvidence™

Th Advanced Clinical Evidence Report™

OrthoEvidence™ uses a rigorous, transparent, and unique process to review, evaluate and summarize high quality research studies and their implications for clinical practice (Supplementary Figure S2). Each month, over 60 leading orthopedic journals and prominent orthopedic key words are searched for publications of RCTs and meta-analyses. The identified RCTs and meta-analyses are then reviewed by a member of the OrthoEvidence™ medical writing team, who have formal training in research methodology. The medical writers then prepare a corresponding Advanced Clinical Evidence (ACE™) Report. ACE™ reports are detailed one or two page critical appraisals and synopses of the research (Supplementary Figure S3). Specifically, the ACE™ report includes a study overview and answers to the following five main questions, which are highly relevant when evaluating research that may change clinical practice: i) Why is the research needed now? ii) What is the principle research question? iii) What are the important findings? iv) What should I remember most? and v) How will the outcomes affect the care of patients? These five questions were selected for inclusion within the ACE™ reports as they provide an effective way to communicate the vital take home messages from each research study.

An assessment of the evidence’s methodological quality is also conducted with each of the ACE™ reports (Supplementary Figure S4). This quality assessment is based up on the Cochrane Risk of Bias Assessment, a standard tool for assessing the level of bias within RCTs. An overall score from zero to ten is presented and this provides the OrthoEvidence™ member with a clear understanding about the risk of bias associated with each of the studies summarized. This component is vital as many surgeons do not have the formal training and skills to accurately and consistently critically appraise the literature. In addition, the quality of reporting for each study is presented, which is based upon a 20 item checklist (Supplementary Figure S5). The score is a modified Detsky Score which comments on five reporting quality areas: Randomization, Outcome Measurements, Inclusion/Exclusion, Therapy Description, and Statistics. This score provides further information to the membership about the overall comprehensiveness of the original published study. The scores for both the methodological and reporting quality assessments are accompanied by the Fragility Index when applicable. The Fragility Index is a measure of the strength in dichotomous outcomes that have been found to be statistically different, representing the number of events that would need to be changed in order for the significant finding to lose its significance.

The reporting accuracy of OrthoEvidence™ is of the upmost importance to enable for an effective translation of research into clinical practice. A medical expert, who is an orthopedic surgeon, along with the Managing Editor of OrthoEvidence™, verifies each report for accuracy. To further ensure the reliability of each ACE™ report, they are independently verified by the author of the RCT or meta-analysis before they are posted on the OrthoEvidence™ website. The authors often provide information not presented in the original publication, rendering the ACE™ report highly valuable beyond the original publication. When possible, authors associated with each ACE™ report are interviewed, which allows for unique insights, further explanation, and detailed discussion of their research. The interviews are short (i.e. 2 to 5 minutes in length) and the authors are asked to discuss the study rationale, key findings, and future research to complement the ACE™ report. The interviews provide the membership with a greater understanding of the motives behind conducting the research. OrthoEvidence™ members are able to rate the quality of the ACE™ report on a scale of one to ten and provide written comments on the ACE™ report. Members can also subscribe to receive alerts for when comments are added to specific ACE™ reports. This interactive forum allows for discussion of research finding, methodology, and relevance to clinical practice among the OrthoEvidence™ membership. In addition, OrthoEvidence™ provides trending statistics that shows the popularity of the ACE™ reports, so that members can easily access the most frequently viewed reports.

The ACE™ reports may be downloaded into PDF format so that they can be easily saved to a personal computer and printed for use at journal clubs or personal records. They may also be easily shared through email and social media including Facebook and Twitter. For select journals, a link to the journal website is provided so that the user can access the full-text manuscript when applicable.

Timeliness of Advanced Clinical Evidence reports

Timeliness of knowledge dissemination is very important at OrthoEvidence™. The majority of ACE™ reports are made available to the OrthoEvidence™ membership within four to six weeks of being published in an academic journal. This ensures that members have access to the latest research promptly. ACE™ reports from RCTs and meta-analyses presented at high impact orthopedic meetings and conferences is another example of the timeliness of getting important information to members. The ACE™ meeting reports allows the members to be able to easily access the latest orthopedic research. Summaries are posted within a couple of weeks of the meeting to further ensure timeliness of the research.

Advanced Clinical Evidence reviews

With the expanding database, OrthoEvidence™ internally develops ACE Reviews, which combine the results of two or more ACE Reports. They provide a high level look at a relevant topic and provide a recommendation on the best treatment practice. To date 11 such summaries have been completed on topics such as meniscus repair, fracture healing, and visosupplementation.

Comprehensiveness of the database

One-hundred new ACE™ reports are added to OrthoEvidence™ each month from over 60 journals across multiple orthopedic specialties. The orthopedic specialties include arthroplasty, foot and ankle, general orthopedics, hand and wrist, metabolic disorders, osteoarthritis, pediatric orthopedics, physical therapy and rehabilitation, shoulder and elbow, spine, sports medicine, trauma, and tumor. OrthoEvidence™ currently has a database of greater than 3000 ACE™ reports available to its global membership.

Educational features of OrthoEvidence™

OrthoEvidence™ offers numerous educational opportunities to its membership. This includes a series of educational videos on surgical trial methodology which are available to its membership. Each educational lecture is given by an orthopedic surgeon who has epidemiological training and leads a research program in the field. Examples of presentations include The Language of Evidence-Based Medicine, Hierarchy of Evidence, What is a Randomized Trial, and Measuring Outcomes in Surgery. The series provides the OrthoEvidence™ membership with a solid foundation on research methodology. For continuing education purposes, there are quizzes associated with some of the ACE™ reports, which members can complete. Upon the successful completion of the quiz, the member receives a certificate that can be applied to Continuing Medical Education (CME) credits. Finally, OrthoEvidence™ offers monthly journal clubs to its members. The journal club forum provides the membership with the opportunity to critically appraise and discuss recently posted ACE™ reports.

Target audience

To date, OrthoEvidence™’s membership exceeds over 13,000 individuals from over 100 countries and is comprised of orthopedic surgeons and trainees, physicians, physiotherapists, chiropractors, nurses and other allied health care professionals. The membership continues to expand daily allowing for individual, academic institutional, and association memberships. A trial membership is also available.

Discussion

Strengths of OrthoEvidence™

The easy to read ACE™ report allows members of OrthoEvidence™ to quickly and efficiently read the latest and most relevant, high-quality, evidence. Each ACE™ report takes approximately two to five minutes to read, as opposed to the 20 to 30 minutes it would take to read a full journal article. In other words, OrthoEvidence™ removes the steps that often limit orthopedic surgeons when consulting the literature by conducting the search for relevant evidence, summarizing it, and critically appraising it for its membership. OrthoEvidence™ allows its membership to easily compare research studies and draw accurate and timely conclusions on the research’s relevance to their clinical practice.

OrthoEvidence™ allows its members to use their time efficiently and to stay current by having access to a broader range of high quality research output. In attempts to remain up to date on the literature, many surgeons read or review the publications in the top five specialty journals. By doing so, they are missing 60 percent of the relevant published RCTs. OrthoEvidence™ allows its members to expand their horizons and access high quality research from over 250 relevant journals.

OrthoEvidence™ also helps its membership by recommending relevant ACE™ reports to them and sorting the reports by speciality and publication date. Another strength of OrthoEvidence™ is the timeliness of the posting of the ACE™ reports. These reports are posted within a month of their publication, which is much faster than similar evidence-summary sources. One-hundred new ACE™ reports are added each month. OrthoEvidence™ is easily accessible through the internet and is available at the point-of-care, which allows treating orthopedic surgeons to easily practice the principles of evidence-based medicine within their clinical practices.

Limitations

The primary limitation of OrthoEvidence™ is its narrow scope, focusing primarily on RCTs and meta-analyses in the field of orthopedic surgery. OrthoEvidence™ currently does not include ACE™ reports of other types of important research studies including observational studies, non-randomized comparative studies, economic analyses, or any basic science or biomechanical research. In the some circumstances, there may be no-high quality evidence (i.e. RCT) evaluating a research question within the published literature. In such cases, the principles of evidence-based medicine suggest that the surgeon look on the hierarchy of evidence triangle for the next available research design (i.e. prospective comparative studies) (Supplementary Figure S6). In addition, many research questions cannot be answered by an RCT design and other studies designs (i.e. a large observational cohort study to ask about the prevalence of a condition, an economic analysis to determine cost-effectiveness) are more appropriate. In the case of rare conditions, it may not be feasible to complete a RCT, therefore an observational study may be the only available research evidence. In addition, published practice guidelines synthesize the available research and make recommendations to practice. Currently, OrthoEvidence™ also does not provide its members with ACE™ reports on clinical practice guidelines.

Another limitation of OrthoEvidence™ is that it only focuses on research relevant to orthopedic surgery. It does not include studies published that are relevant to other surgical specialties. Finally, the research within OrthoEvidence™ is subject to publication bias, as negative trials may not be published and therefore are not available to the OrthoEvidence™ membership.

Conclusions

OrthoEvidence™ is always expanding, improving, and offering new features to its membership. Future directions may include the addition of ACE™ reports on other study designs (i.e. large observational studies, clinical practice guidelines, and economic analyses), the expansion of the volume of Evidence Reviews and the introduction of educational videos and courses. Additionally, OrthoEvidence™ is also improving its access through the development of a mobile application.

Availability and requirements

Project name: OrthoEvidence™TM

Project home page: http://www.myorthoevidence.com/

Operating system: Linux

Programming language: PHP, Database-MySQL version 5.5.4, Environment-Apache, PHP version-5.4 or higher

Other requirements: Image Magick installed, Imagick PHP library installed

License: None.

Any restrictions to use by non-academics: Membership Fee, access to a limited number of free reports is available.