Useful and useless publications measured by bibliometrics and scientometrics in orthopaedic surgery. Are the relevance of a journal and publication metrics useful enough for the scientific promotion of surgeons?

Excellence in science

Science is driven by the publication of novel ideas and experiments in peer-reviewed scientific journals and books [1]. To affect clinical practice, research findings need to be important and be presented properly; the data should be integral and transparent [2]. Universities are trusted to know best about research because it is assumed that academics strive for excellence in science. This is evidence, however huge differences in quality between individuals exist, especially in crowded universities. The end result could be average science and mediocrity [3-7]. One method to guarantee excellence in science is to find, support and allow independent work of scientists in order to promote and validate new ideas. In this way, education and research has the opportunity to stay ahead in the rankings. Yet, in the game of end results citation metrics and notoriety could be biased.

For the past half-century, the impact factor as originally described in 1955 by Dr. Eugene Garfield has emerged as an index of quality and prestige and has been the most prominent method of evaluation for citation metrics [8]. However, one should be aware that there are many limitations and biases to the impact factor and metrics including self-citations, increased number of review articles and limited number of original research articles, as well as the timing of publication [9-14]. While citation metrics may be useful for the qualitative evaluation of journals, the usefulness does not extend to individual articles. In fact, there should be no correlation between the frequency of citation of an individual article and the impact factor of the publishing journal [9, 15]. Moreover, a simple citation metric could be misused to evaluate scientists [16]. Additionally, in the game of the impact factors, the race for publications and citations has also led to numerous cases of scientific misconduct [17]. Therefore, citation metrics should not be used arbitrarily as single methods to assess the quality of a journal and to evaluate scientists. Instead, the quality of a journal should be evaluated by factors based on citations by other journals, visibility and download metrics and authors should be evaluated by thorough analysis of their curriculum vitae, scientific work and international scientific performance [11, 18–20].

Publication factories

In almost every academic discipline, especially in basic research, the number of publications is the most important measurable output. Unavoidably, a scientist or institute’s productivity is considered on the basis of their publications. The mass education, teaching and research programs are an explanation for the increased demand for publications, rankings and citation metrics. In this system, scientific knowledge is replaced by measurable outputs. More and more scientists are currently employed in research but most of them have limited time for developing original research. Dissertations and theses have become mass products, the majority of which add little to the scientific advancement and current literature. In this scenario, attention is directed to mass papers production that although may contribute to personal or institutional academic progress and promotion and easier access to research funds, they hardly contribute to scientific knowledge [3–6, 21]. Importantly, the people employed in research consume a larger portion of their time dealing with research bureaucracy such as research proposals, ethical or administrative approvals, science and human rights protocols, and General Data Protection Regulation (GDPR) instead of performing research and writing the paper. Scientists know about this phenomenon, however, obtaining funds is another concern [3]. In the end, authors’ publications relate to measurable contributions (numbers) rather than increasing knowledge (science) [3]. One recent example is the massive data production on Covid-19 epidemic with hundreds of thousands of papers published in various languages. One may well wonder if this massive media attention ever happened in the past and, if so, if this has been finally proven to be beneficial or appropriate [22-26].

Bibliometrics

Publications are important for the rankings of individual scientists as well as institutions and universities. In applications for a position, the number of publications is usually the most important criterion in the decision process. Therefore, no wonder that scientists do everything to publish as much as possible no matter what cost, importance and novelty of the topic, and format of scientific writing. However, these publications are not of the slightest importance for anybody outside of the academic system; in the medical activity, especially in private practice, most people have limited time to read and integrate modern research or interpret the potential importance to medicine of recent published data. Some research results are not validated in long-term and introduction in current practice requires longer follow-up and comparative studies. One notorious example is the promotion for some minimally invasive techniques and approaches in various surgical operations that are usually promoted by sponsors and medical industries before scientific long-term studies validate the eventual better outcomes with these procedures. The authors present their number of publications in the process of personal promotion, and interestingly, most people are driven to their physicians without knowing, asking or acknowledging their scientific background [3, 23].

To this end, an entirely new science has developed, called bibliometrics or scientometrics that deals only with measuring and comparing the publications’ numbers and citation metrics of scientists and scholars [27-31]. The most important tool of bibliometric research is citation analysis that determines the quantity of citations of a specific journal’s article, and ascertains the scientific effect of articles. Hence, the quantity of citations is used as a quality index of an article. Therefore, an objective method (number of publications and citations) is used to evaluate the prestige of a scientist, a journal or a university without a respective objective method to evaluate the quality of these publications and citations. Is this rationale, fair or ethical [3]?

Useful and useless publications

From a practical point of view, intellectual research is an activity in which researchers find satisfactions from publishing and eventually helping patients [32]. Every paper has a point and if the research is honest – it deserves publication no matter the research area or topic. Definitely, not every intellectual research study will ultimately end in changing practice. Less useful research stimulates also theoretical inquiries that may or may not solve the problems by which they were suggested, but may also open views for future achievements. They may generate important findings. This emphasizes the importance of intellectual work, even in the form of arid research [32] that could be considered by some as useless.

Useful clinical research should address problems with disease burdens and be patient - oriented, it should add to the related literature and the topic and ideally should lead to a change in decision-making and possibly to a change in clinical practice. Importantly, it should be pragmatic, feasible, transparent, repeatable, and economically sustainable [33]. Much clinical research seems to be losing its purpose; researchers could waste time, patients are sometimes recruited without substantial purpose, the results often merely present personal clinical experience. This type of work adds limited value to the current knowledge and science [33]. Quantity wins out quality that may boost academic career but does not improve knowledge. Researchers increase their submitted and published papers, drug and implants companies sponsor redundant clinical trials to promote some products and medical centers earn revenue by running or hosting studies of debatable value [21, 33, 34].

Nowadays, research production in science is much more sophisticated. The actual research rests largely on the shoulders of assistants and graduate students; how do these junior researchers cope? Their low hourly compensations and less practice and experience does not upgrade science; yet, they have more time available and greater flair for knowledge and scientific work that compensate and allow them to improve scientific knowledge. Therefore, in the era of publications factories, guidance for junior researchers on how to write a paper is paramount [35-37].

When a poor value paper is submitted to a journal it could be rejected; in this case, the authors will be disappointed, possibly discouraged, but the effect will be limited in space. However, most original papers are finally published either after the original submission, if the reviewers like the paper, or at the re-submission to another journal; then, the paper is digitally archived and becomes visible to the world to read and appraise [5, 38, 39]. For a useful publication, the authors should search and read the available literature, respect quality, provide a clear and useful rationale on an important topic and argue in their rationale for the utility of the study to the related literature. The writing should be careful edited and address all comments and concerns raised by the editors and the reviewers in the peer-review process. Useless publication could be generated by the ignorance of some writers who do not study the literature and end-up repeating previous reports. Lack of quality could result from using unverified sources, a recent example was observed in the use of Wikipedia as a source of data where the femur neck fracture classification of Friederich Pauwels, notorious German scientist who published his research in 1935 was describes as “Pauwel’s angle” then the author of a research used the name Pauwel instead of Pauwels in all his paper. This could be funny however it reveals that the author did not read the original paper by Pauwels and based his arguments on unverified sources. Correct referencing is important and is a proof that the author is aware of relevant previous publications on the topic. Plagiarism should be carefully considered when reporting published text; [39].

Currently, there are many journals and books of variable quality, and predatory journals that publish low or marginal quality research have appeared [4, 5]. Many of these journals have names that closely resemble those of established publications, making them easily mistakable [5]. In the demand for abundant publications on young researchers, it may be easy to get attracted by some low quality/predatory journals tricked by flattering emails from the journals inviting them to submit a paper or fooled by a name that sounded like a journal they knew. These journals advertise the gold open access route that requires the author to pay a fee if the paper is published. Therefore, in this publication factory, the journals will publish almost anything for a fee, therefore, effortless and useless publications will be increasing [5, 40]. Authors’ attention to disguised websites, and a robust peer-review and editorial system are the solutions to this problem [38, 41, 42].

Even in highly ranked journals flawed and useless articles may get through. Many readers cannot understand all the potential problems of a study, and do not have access to all the data that they need to draw conclusions [38]. In this case post-publication peer-review may be useful [43].

Any submitted work is subjected to a demanding, constructive and objective process of peer-review which is still the best method to validate the quality of the accepted papers, although also flawed with biases [44]. In this regard, some journals have notorious high rejection rates [38, 45]. Although the high number of submissions justifies a high rejection rate, this reflects another bias of the peer-review process; the reviewers are encouraged to reject manuscripts in most cases in order to preserve this quality measure, and solely manuscripts that find favor within the reviewers get published. Moreover, not all reviewers evaluate personally, they could eventually be assisted by a resident or a junior staff member and the result could be debatable for various reasons [38, 46].

At “International Orthopaedics”, we aim for useful publications, after an objective and constructive peer-review process; we are in debt to our reviewers for their honest work. We publish original research papers, as well as educational review and orthopaedic heritage papers. The readers are encouraged to react by letters to the Editor that are generally meaningful and useful in the debate. We are also respectful to our readers by publishing editorials on specific, timely topics, as well as on general orthopaedic practice [47-53]. We are proud that a high percentage of the published papers in the last two years were cited. Those papers are probably meaningful and hopefully useful for the readers. We believe that every paper has a point that deserves attention and we try to help the authors with a constructive and informative peer-review process aiming to improve a submitted paper, accepted or rejected. Papers with a substantially useful point will be published; however some good true papers could be rejected for various reasons. We strongly believe that for the quality of scientific research and publications the peer review process should be strengthened and barred of biases, and the scientific journalism should opt for its reputation and prestige.

The current publication metrics of “International Orthopaedics” comfort us in the pursue of our goals, the main task we keep in mind is to publish true scientific data coming from every place in the world, whatever rich or poor, famous or unknown, notorious or modest places, institutions, hospitals, universities, centers. As official flag carrier of the Société Internationale de Chirurgie Orthopaedique et Traumatologique (SICOT), our Journal became one of the major Journals as ranked by the Journal Citations Report with continuous uprising “Impact Factor” that currently is at 2.854 and with a good ranking in the family of publications on conjunctive tissue and locomotors system, articulations, muscles, bones and the whole marvelous genetic determinants that makes the human being a beautiful moving being. Current scientometric research performed by our Editorial Board is directed to detection and definition of the utility and meaning of each published work including different types of papers, clinical and outcome studies, meta-analysis and systematic reviews, experimental science, biology or laboratory findings, procedural and technical data, comparative operative data, narrative and historical papers as well as readers opinions, letters and meaningful correspondence. Our partnership with Springer Nature allow the use of digital data attached to the papers including educational videos, clinical or surgical procedures, infographics and supplementary statistical material that could support the presentation of research. This material is uploaded frequently by the readers and comfort us in the pursue of the ultimate goal that is to help our authors, reviewers, readers and general public to access good quality scientific material and useful production that would ultimately make proud all of us with a label of quality and with “International Orthopaedics”.