Global research coaching in orthopedic surgery: seeding for an international network.

Despite the importance of delivering evidence-based health care, orthopedic surgeons have directed fewer efforts towards the generation of such evidence. Even when present, published evidence lacks methodological rigor and is known to be inaccurate. One of the main reasons for the lack of generation of quality evidence, and the low involvement in research among orthopedic surgeons, is the lack of structured research coaching environments where they can learn concepts and hone their research skills. There is a palpable need for a pragmatic and outcome-oriented approach that can equip orthopedic surgeons with effective ways of communicating their research in writing. We describe a pragmatic research coaching program, designed and developed by the Research on Research group, which aims to build a global network of orthopedic researchers trained in streamlined and standardized research methods. We also provide a brief overview of the course principles and tools, and the platforms used in this program.

INTRODUCTION

The importance of knowledge generated through evidence-based medicine (EBM) has been recognized by surgeons,[1] as it helps them make informed decisions based on reliable evidence. The practice of EBM is even more important with the increasing need for orthopedic surgery, such as procedures related to hip fractures and hip and knee arthroplasties.[2,3] However, until now, surgeons have relied on reports and case series,[4] indicating the need for a higher level of evidence, which is currently lacking in the field of surgery.[5] The same applies to orthopedic surgery due to the shortage of standardized clinical trials[6] and to inaccurate information in published literature.[6,7] The lack of involvement of surgeons in research is also reflected by the low number of successfully consolidated funding proposals.[8] It can also be seen that surgery departments only attract funding through considerable subsidies when they have committed surgeons with a demonstrable track record.[9] The deficiency in research involvement is also attributed to the lack of formal coaching in research methodology.[10]

Orthopedic surgeons have directed few efforts towards the generation of evidence, either through the exploration of existing literature,[5] or even through the publication of articles.[11] Some of the reasons described for this limited research participation are: time restriction,[12] lack of relevant knowledge[13] and of specific coaching.[14] Although other areas of medicine have resolved the coaching gap in research by introducing specific programs at residency level,[15] these efforts are still scarce in orthopedic surgery. This emphasizes the need to introduce a structured and streamlined research coaching program that can assist orthopedic surgeons to generate hypotheses that effectively lead to high quality publications. Such a coaching program can potentially increase the productivity of orthopedic surgeons, thereby contributing to the generation of evidence that would afford better evidence-based healthcare.

The aim of this article is to describe an ongoing research coaching program, which was originally developed by the Research on Research group (RoR www.researchonresearch.org) at Duke University Medical Center and implemented worldwide, including implementations at multiple institutions in Brazil. The article also highlights how this coaching program would help to build a global network of orthopedic researchers trained in streamlined and standardized research methods.

COACHING PRINCIPLES

Traditionally, research methods have been imparted through didactic courses and seminars or are disseminated by means of the mentor and apprentice interaction. Although novice orthopedic researchers may obtain theoretical knowledge, practical skills are rarely imparted by this teaching method. The lack of clear goals, of firmly grounded problems and of practical courses reduces the chances of orthopedic surgeons working on independent projects throughout their lives. As an alternative, the global research coaching program in orthopedic surgery conceived by the RoR group not only focuses on this shortcoming, but also helps the participants to learn functional methods that can be used consistently in future research projects. During this program, the orthopedic surgeons become familiar with platforms, tools, and software that facilitate their journey from the elaboration of a scientific question to publication in high quality journals.

To guarantee that the participants not only know the research methods in theory, but also know how to implement them in practice, we draw up a pragmatic coaching plan that uses learning methods based on the problem. Moreover, to achieve a clear goal, and to comprehend the possibility of delays in the conclusion of prospective studies with living beings, the coaching program involves the exploration of existing databases, by means of the analysis of secondary data or by reviewing the existing literature through coaching in systematic reviews and meta-analyses. In this manner, by providing a structured learning environment with a pragmatic focus, we reduce unpredictability and broaden the application of the concepts learned. With each module covered, the participants find themselves closer to the final objective, i.e., a manuscript ready for submission.

To enrich the learning experience and to equip the orthopedic researchers with advanced knowledge in research, instruction in the coaching program is achieved through interaction with the course coordinators, with specialists in the medical area (clinical mentors), with specialists in methodology (distance mentors), with participating researchers and with examples of previous studies. The local instructors serve as main point of contact and interact regularly with the participants. They work close to the participants, answering questions, solving problems and ensuring their continuous progress. The clinical mentor is a specialist physician, for example, an orthopedic surgeon. They provide clinical information and show the clinical significance of the project in several stages of the coaching program. The distance mentor, designated by the Research on Research group, concentrates more on providing methodological information at each step of the project. The participating researchers and their projects serve as case studies and live examples. As a result, the participants not only learn with their own projects, but also through interaction with researchers and mentors, as well as absorbing information from previous examples.

Interaction with colleagues and mentors is important, as the researchers do not work alone. On the contrary, they are part of an interdisciplinary and collaborative research environment. Besides relying on their colleagues for feedback and collaborative projects, to perform their daily research activities successfully, the orthopedic surgeons work together with the research coordinators, editors, managers and data analysts, as well as counting on the support of Information Technology (IT) professionals, among others. The wide diversity of individual coaching and knowledge of research methods among specialists and colleagues, as well as the lack of interdisciplinary communication methods among the team members, results in less research productivity. The research coaching program, besides training the orthopedic surgeon individually, also focuses on modeling their research environment, building an infrastructure of reliable support. As a consequence, it creates a favorable environment for research that facilitates greater productivity, quality and cost-benefit.

The teaching method is just as important as the instructors and mentors in a course. Traditional methods, such as classroom learning, seminars and discussion groups, are neither flexible nor customizable in terms of time, location and scheduling.[16] The orthopedic surgeons who reconcile clinical practice, surgery and research, may not find traditional methods convenient or appealing. Based on this, the research coaching program uses an online teaching, coaching and communication mode. The participants interact in an online environment through emails, calls and videoconference with the instructors and mentors at times that are mutually convenient according to their agendas. Although designed to transmit skills and to effectively train the future generation of biomedical researchers, the costs related to the course and the restricted use of didactic material could become an obstacle for its diffusion and for the benefit of researchers around the world. To avoid this, the research coaching program is based on the open code principle. All the slides, reading material, learning platforms and tools prepared by the RoR group are of gratuitous access under the General Public License -GPL - http://www.gnu.org/licenses/gpl.html). The participants can have access to the teaching material and to contribute towards its improvement during and after the course. As a result, the course becomes highly implementable in other parts of the world and continues to evolve and to improve with time. It is important to note that the coaching program not only focuses on the transmission and expansion of research skills, but also aims to promote innovation, helping the researchers to understand the heuristics of innovation and the methods for increasing the rate of innovation of their research projects. The possibility of an article being published in a high impact journal depends on the scientific question formulated, on the methods used, on the results and conclusions derived from the research project. The orthopedic research coaching program concentrates on guaranteeing characteristics of high quality for high impact publications from the scientific question formulation stage. With each step, the quality of the research project is increased by relevant and regular information from the instructors, clinical mentors and distance mentors.

Besides helping the orthopedic researchers to innovate, communicate and collaborate efficiently, the coaching program also aims to transform them into independent researchers to develop and expand research capacity at the individual, departmental, institutional and national levels. Instead of being limited to the objective of a single project, the course concentrates on sharing a functional and structured method so that the orthopedic surgeons can head, seek or participate independently in multiple research projects in the future.

Finally, complementing the objective of making the researchers independent, it is important to help them to be active participants in communication with statisticians. Although aware of the significance of statistical concepts in evidence-based medicine and of actively participating in research,[13] physicians and other health professionals, at various coaching levels, present limited knowledge and understanding of statistics.[17] Furthermore, they consider statistics and the programming related to this field both a cognitive challenge and a complex task. Consequently, instead of coaching orthopedic surgeons in statistical theory, the research coaching program focuses on teaching its application by means of examples and previous situations. The coaching program aims to train orthopedic surgeons to be able to indicate statistical tests, to communicate with statisticians and also to interpret statistical results.

THE CONCEPT OF COACHING PLATFORMS

The ultimate aim of a researcher should be to clearly communicate his or her science to a community of colleague scientists and, in the globalized world, to the public in general. This is a challenge for the novice researcher, since he or she must communicate a rigorous science in a language that is understandable by multiple audiences. Unfortunately, the writing of a scientific article is not an easy task for most researchers, due to the lack of a structured and agile methodology during the entire process, from the conception or idea of the research to the publication stage.

The concept of coaching platforms arose to fill the gap in the scientific writing methodology. The platforms are composed of standard operating procedures (SOP), templates that serve as guidelines for the writing of the manuscript and communication tools that facilitate discussions. Studies conducted by our group (RoR) ascertain the easiest and most practical methods of moving from the formulation of a question to the writing of the original text for publication. The use of these platforms guarantees quality research and excellent productivity. Some of these platforms address all the aspects of the research project and are listed below.

Formulation of the Scientific Question

The first difficulty encountered by the novice researcher is preparing a formal scientific question to form the basis of the research idea. In general, this idea will come from information that is necessary, but has not yet been answered by previous studies. The impact of the scientific question is enhanced if it attempts to fill the most important and significant information gaps. The second question arises from how this idea should be formalized so as to transmit the complete meaning to the statistician and other collaborators.

A platform called question diagram (QD) was developed by RoR to formulate the scientific question in a structured format. The QD involves several aspects, including the verification of whether the researcher has access to all the necessary variables to answer the proposed scientific question. This will prevent the researcher from investing his or her time in preparing a scientific question, which although interesting, may not have the necessary variables to be answered. Moreover, the QD can also reveal errors and limitations that would otherwise be neglected, leading to difficulties in data analysis or in obtaining the study conclusions. In the QD the researcher will delineate all the study components, such as the variables of outcome, predictors, inclusion and exclusion criteria, stratifications, statistical analysis and simulations of tables and graphs.

The QD content description can be accessed on the page related to the QD guidelines on the website of RoR (https://docs.google.com/document/d/1ldI6lRym0todBDlMIjNdyY1LsaqEh6tvJ8IV3nNkQio/edit?hl=en_US).

All the QD components should be aligned with one another. This alignment guarantees the internal consistency of the QD and avoids errors in the preparation of the study. For example, all the hypothesized conclusions should be answered by variables that are listed and available in the set of data to be used. The same applies to the simulations of tables that need to be delineated with a basis on the conclusions. The variables that will not be used to answer any hypothesized conclusion should not be listed.

With the QD complete, the researcher will have a consolidated scientific question ready to be shared with the study collaborators and, specifically, with the statistician who should understand the terms used in the QD, facilitating the statistical analysis in the right direction. This will allow the statistician to understand the scientific question, avoiding unnecessary contacts with this individual and also reducing the chances of losing important components such as variables or stratifications. In other words, the QD serves as a tool that will facilitate communication with the statistician. This structured format allows statistical programmers to understanding exactly which question the researcher wants to answer and which variable from the database are necessary for the data analysis.

Examples of the QD can be found on the website of RoR.

Research and Review of Literature

Although the researcher should know the information gap that he or she wishes to research, it is necessary to perform a search and complete review of the literature to provide grounds for the scientific question. The researcher generally struggles with these stages due to the large amount of information available and the volume of literature found discourages the review process. Therefore, a precise yet comprehensive search in the literature is necessary to obtain a more focused and manageable sample of articles. Based on the scientific question and on the predicted volume of available information, several literature search tools can be used to facilitate the search, attempting to accurately identify the literature that will be useful for the study. Besides the literature review, it is equally important to organize the compiled literature so that at the end it facilitates the manuscript writing process without the need for further searches in this stage.

To carry out the literature search, the researcher needs to be familiar with some search tools and strategies and with the online databases. The use of appropriate keywords related to the topic and the scientific question are also important for conducting the literature search. The use of these tools not only reduces the cognitive burden associated with literary research, but also makes it less time-consuming and more efficient.

Literature review is an important task, as it aims to identify previous studies that support the arguments and findings of your study. All the sections of the scientific article and even the DQ depend on the literature found. Hence it is important for the relevant literature to be known before starting to write the manuscript. The available studies need to be critically reviewed to formulate specific arguments while preparing the "Introduction" and "Discussion" sections of the scientific article.

The literature review should start with the confirmation that its scientific question has not yet been answered by any published material. At the same time, it is important to verify whether there is any previous article that is similar to your planned study, in terms of topic or methods and also published in the same proposed journal. As remarked previously, the article's arguments need to be backed up by the literature review. At this time, the orthopedic surgeons and certainly other clinical researchers in general face other challenges such as having to summarize the reviewed literature. The literature matrix platform developed by RoR helps in this process.

The researcher needs to be sure that all the fundamental and recent articles related to his or her scientific question were compiled. To conduct the literary research correctly, tools such as PubMed (public interface of MEDLINE), Gopubmed, Ac Academic Google, Jane and the Cochrane library, are essential.

Literature Search Tools

PubMed (http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed)

The researcher can follow this tutorial recommended by PubMed.

GoPubMed (http://www.ncbi.nlm.nih.gov/sites/ntrez?db=pubmed)

Uses semantic technology from the web to classify the results of research in biomedical literature.

Biosemantics Jane (http://www.biosemantics.org/jane/)

This web application is useful for finding journals with similar lines of research, helping in the choice of the correct journal for submission. In addition, researchers can find relevant articles that can be cited in their manuscripts.

Academic Google in Portuguese (http://academico.google.com.br/)

Another Internet search engine in which researchers can find articles related to their topics.

Literature Matrix

The literature matrix is a method for organizing important scientific information compiled from the existing literature. With the large quantity of scientific articles, some important information might be lost if it is not filed correctly. Even the source where the information was found can be lost. Therefore it is possible to save the time that would be spent on going back several times to the same information.

The literature matrix is a simple spreadsheet with four folders: significance of the topic; review of the literature that supports the information gap; favorable and unfavorable methods and results. In each one of these folders, the researcher inputs the bibliography, important remarks and his or her own observations. Thus, while they read the relevant articles, they can gradually complete these folders. When writing the article, important information that has been collected is already organized in the matrix and just needs to be transferred to the article. The literature matrix aligned with each section of the article also helps the researcher to write quickly and objectively.

Example of Literature Matrix

http://spreadsheets.google.com/pub?key=pk3Yq2LCc9VEty5mjtWnGOA&output=html#

Indication and Interpretation of Statistical Findings

Although researchers are not expected to carry out their own statistical analyses, knowledge of data analysis is essential to facilitate communication with those that do carry out this task. People are still unclear about what constitutes the minimum knowledge that allows interdisciplinary communication, not only to avoid communication errors, but also to generate a productive exchange between specialists.

The concept of Information Layers[18] is a platform that aims to help researchers to indicate statistical tests and to interpret the results of statistical analyses. This concept affirms that the minimum knowledge that a clinical researcher needs to have to interact with statisticians should be: (1) understand the necessary variables that are part of the analytical method; (2) understand the most typical graphs and tables generated by the analytical method; (3) know the previous publications in your field that used these methods and (4) know other specialized references of the area that provide a progressive level of understanding in relation to the analytical method. In practice, all this information is provided on a collaborative site (wikipage) that contains all the information for a test on a single page.

For example, imagine that an orthopedic surgeon is discussing the best method for analyzing his data with a statistician. They consult the Question Diagram and the statistician suggests the use of a t test. The orthopedic surgeon then turns the page that contains the layers of information on the t test and verifies that this test requires the result to be a continuous variable, where the predictor is a dichotomous variable. His QD, however, is attempting to establish the association between postoperative infection (yes/no) and presence of preoperative diabetes (yes/no). His reply to the statistician is, therefore, that the t test is not appropriate for this analysis and that they should find an alternative. The statistician then realizes that he did not understand how these variables were coded, and suggests the chi-square test. The orthopedic surgeon checks the information layer document to see the chi-square test, agreeing that the input and output variables are suitable and suggesting a table format that captures how these tests are presented. The orthopedic surgeon also looks at previous publications in the field of orthopedics that employed chi-square tests to confirm that this test has been used in similar situations and, finally, consults some other articles to better understand the statistical test in question. When providing information "in layers", the information layer method allows researchers and statisticians to communicate, without necessarily requiring orthopedic surgeons to have extensive knowledge of the method, yet with the possibility of using a common language.[19]

SCIENTIFIC WRITING

The scientific writing platform is based mainly on two principal concepts: writing and cloud writing platform. The concept of writing platforms is based on the idea that any scientific text can be understood in terms of its content and structure. For example, if we analyze a sentence such as "Low back pain results in substantial costs for healthcare in the USA, reaching more than 90 billion dollars per year".[20] This sentence can be read at two levels. In the content sphere, it describes the fact that low back pain is costly for the USA. In the structural or rhetorical sphere, the main message that the author wants to transmit is that low back pain is a significant problem. In understanding the underlying message that the writer wants to broadcast to his or her readers, the structure allows the writers to create a sequence of arguments to convince their readers and to facilitate the demonstration of solid arguments in scientific, interesting and important terms.

The writing platform method is based on the idea of a block of text, defined as a unit of variable extension (sentence, paragraph or group of paragraphs), with a single rhetoric or purpose of argumentation. For example, a block of text can concentrate on the significance of the topic, followed by another block of text that describes the current gap existing in the literature in this field. The importance of blocks of text is that their sequences determine the extension of the argument presented by the section of a manuscript. Considering the "Introduction" section as an example. A typical "Introduction" section has the following sequence of blocks of text:

1. Significance of the topic as a whole

2. Information gap

3. Review of literature to support the information gap

4. Study objective

This sequence has rhetorical meaning because the block of text on the significance of the topic produces awareness of the topic as a whole, and arouses interest among readers, encouraging them to continue reading the article. The block of text on the information gap points towards the lacuna in knowledge in a particular field, thus creating the need to fill this gap. The block of text block about the literature supporting the information gap stresses the existence of the gap, demonstrating that, although there is information in the areas that surround this hiatus, the hiatus itself already has to be approached. The "Introduction" section then ends with the affirmation of the primary objective of the article, which coincides with the gap that was previously indicated. In aligning the gap and the objective of the article, the "Introduction" section closes a cycle that can now be supported by a solid methodology described in the "Methods" section.

In the research coaching program, the researchers learn how to write the four sections of a scientific article, namely, Introduction, Method, Results and Discussion (IMRD). Although the structure of the Introduction and Discussion sections is common to all the research designs (for example, randomized clinical trials, qualitative studies, among others), the Methods and Results platforms are specific according to the study design. Moreover, although the structure of different journals is standardized, its extension varies and therefore the researchers are encouraged to define a specific journal to which they intend to submit the article before starting to write it.

COACHING FOR A GLOBAL NETWORK

Once the coaching of orthopedic surgeons is in progress and a continuous flow of new researchers is appearing, it is possible to form a research network. This network is facilitated by its coaching in activities such as (1) common methods for the formulation of scientific questions, allowing the sharing of ideas with good communication, (2) common data collection platforms allowing the performance of prospective studies in a systematic manner in countries with different cultural and clinical practice standards, (3) joint writing of scientific articles, providing the authors employ the same argumentation structure regardless of the clinical area or of another previous research coaching program. Although it is an initial form, this global network has already started, as in the various collaborations between the Institute of Orthopedics and Traumatology (IOT - http://www.iot.com.br), in Joinville, Brazil. By means of partnerships for data collection with Duke University, the University of North Carolina, Duke-National University of Singapore and the University of Nebraska, its ability to generate publications of international reach has been substantially increased.

Finally, although the Research Coaching Program is an initial form of standardizing research processes in the field of orthopedics, it in no way exhausts the range of possibilities involving the rationalization of these processes. Other protocols are currently being implemented in distinct areas, such as the capturing of prospective data, patient evaluation, research in the community, research policy, qualitative methods and others (see Attachment).

FUTURE CHALLENGES AND OPPORTUNITIES

Although there is substantial progress with the introduction of an orthopedic surgery research coaching system, there is still an array of challenges and opportunities ahead. Firstly, as the model progresses to different academic programs, it is important to guarantee that this model remains expandable and self-propagating. Although the connection of the coaching with the research coordinators and the central group of Duke University is still a viable model, over the course of time this group will become an overcrowded area. A solution to this problem is the creation of the concept of super coordinators or regional coordinators, who are responsible for the training of new coordinators for new sites, thus allowing rapid diffusion of the methodology and an adequate supply for the growing demand of the program. Secondly, the selection of potential trainees to receive the coaching is crucial for the progress of the program. Although research coaching can be beneficial for physicians, who can both incorporate their recently acquired knowledge into clinical practice, and be good collaborators in research projects, progress in this field is closely linked to the coaching of physicians who view research as a fundamental activity for their careers. The creation of mechanisms of early identification of these individuals is, therefore, essential to provide coaching to physicians with different needs and geared towards their role in the research spectrum. (Chart 1) Thirdly, as the program expands, it is important to commence the mapping of research resources in the community, including, for example, researchers with databases available for collaboration, as well as researchers with specific skills in analytical methods. In mapping these resources, the specialists will be able to establish new partnerships for publications and fundraising, while novice researchers will have access to resources that would otherwise take a long time to be created. A final challenge is that, as new sources of scientific information are produced, more liaisons should be created with health policy formulators so that the scientific information generated by members of this network can maximize the impact of the scientific information on healthcare, ultimately improving quality and access for the population in general.

Chart 1

Research participation categories.

• Professional researcher

• Research participant

• Health professional

CONCLUSION

Substantial progress has been achieved with the creation of a series of research platforms, widely defined as a combination of standard operating procedures, templates and software applicable to different phases of the biomedical research process. With the expansion of this program to other academic programs of orthopedic surgery in Brazil and in other parts of the world, we expect not only to increase the number of researchers with the ability to produce high quality publications that will contribute towards the progress of the area, but also to create a global network of orthopedic research that will enable the execution of research protocols that would be impossible for isolated institutions.