Level of evidence of abdominal surgery clinical research in Saudi Arabia.

OBJECTIVES: To quantify and evaluate the level of evidence (LOE) of Saudi publications in abdominal surgery and correlate the obtained results with that of other similar national and international studies. 
Methods: Study design was a systemic review. Literature search strategy was developed to retrieve available articles between January 2000 and  December 2016 that are related to abdominal surgery utilizing PubMed and Google Scholar. Retrieved articles were analyzed in depth with several parameters, then evaluated using (OEBM) level of evidence scale. 
Results: One hundred and ninety-eight articles met the inclusion criteria. Of these, 50.5% were level III evidence studies. The most common study design was case reports (47%), and academic institutions had the highest rate of publications (47%). 
Conclusion: Saudi research in abdominal surgery published between 2000-2016 are of lower quality and of III and IV LOE, which is in the consistency with other specialties. We emphasize the need for promotion of a national and institutional research studies of I and II LOE with collaboration between different health care institutions.

Evidence-based medicine (EBM) is defined as the practice of medicine that simultaneously integrates the best available high-quality research, the experience of the clinician, and the patients’ values.1 Surgeons in various specialties have become aware of its value2 and strive to implement EBM by producing periodic practice guidelines, to standardize the practice of many surgeons worldwide.2 Many studies have analyzed the level of evidence (LOE) of publications in a variety of surgical specialties such as plastic surgery, orthopedic surgery, neurosurgery, and otolaryngology.1-4 A study by Müller et al5 addressed the issue of LOE in visceral surgery; organs of the digestive tract, endocrine system, and abdominal wall. It is important to mention that the few studies from Kingdom of Saudi Arabia (KSA) focused on the quality of publications in neurosurgery, orthopedics, and plastic surgery.6-8 A recent study that is of great interest to our current study, by Almaghrabi et al9 quantified the quality of Saudi publications in gastroenterology. Our current study differs from Almaghrabi et al that it focuses on topics related to abdominal surgery instead of focusing on specific journals. A question has arisen regarding the quality and influence of Saudi abdominal surgery research; is it of good quality and does it have a good impact in comparison with other specialties? Or does it occupy the same place as others, reflecting low quality and influence? To answer this question, we aimed in this study to quantify and evaluate the level of evidence of Saudi publications in abdominal surgery, correlating the obtained result with that of other similar national and international studies, to determine the status of Saudi publications.

Methods

Search strategy

This study was conducted at King Abdulaziz University (KAU), Jeddah, Saudi Arabia, between November and December 2016. We reviewed the levels of evidence among published clinical studies using search phrases such as abdominal surgery, general surgery, abdominal trauma, hepatobiliary, pancreas, spleen, gastric, duodenum, ileum, colon, rectum, and appendix. Databases were accessed and the following strategy was used: “Search term” and “Saudi Arabia”. The time frame was restricted to the interval between January 2000 and December 2016. Each article was identified by abstract screening, then inclusion criteria was applied, followed by accessing the full-text to retrieved more data for the remaining articles. A schematic representation of the audit process is shown in .

Schematic representation of the review process.

Eligibility criteria

The eligibility criteria included all published papers on abdominal surgery, in English, produced by an author affiliated with a Saudi medically-related establishment, in any journal between January 2000 to December 2016. Also, to be included, the study had to be conducted, at least in part, in KSA.Articles were excluded that dealt with basic aspects of abdominal surgery, editorials, reviews, or in which the study population was based outside KSA.

Information sources

Systemic search was carried out to retrieve each relevant article using both PubMed and Google Scholar by a double-blinded review process, and graded using the Oxford Centre for Evidence-Based Medicine (CEBM), Levels of Evidence Scale ().10

Oxford Center of Evidence Based Medicine level of evidence scale.10

Study selection process

After the audit process, 198 papers in English, related to abdominal surgery, published between January 2000 and December 2016 with the first author associated with a Saudi establishment, were included for final review.

Data items and data collection process

Several parameters were collected from each article, such as the name of the journal, impact factor, publication year, affiliation of the primary investigator, city, study design, population, citation numbers, study title, database, and corresponding sector. Those parameters were collected in a Microsoft Excel spreadsheet.

Others

In order to find similar research for comparison of the results, many databases such as PubMed, Google Scholar, and Embase were accessed, using search terms such as abdominal surgery, general surgery, and intraperitoneal surgery. Those terms were used as follows: “Search term” and “Level of Evidence”. Then, articles were accessed to screen the methods section to find similarities with our methods in order to eliminate bias.

Statistical analysis

Statistical analysis was carried out using Microsoft Excel (Microsoft, Redmond, Washington, USA). Mean and median were calculated for most parameters, along with percentage. Paired data were compared using a 2-sample proportion test. Statistical significance was defined as p<0.05 and a confidence interval of 95%. In order to find the degree of agreements between the 2 reviewers, Kappa value was calculated.

Results

Out of 2284 articles identified in our literature search, only 198 met the inclusion criteria of the current study. Most of the included articles (n=162, 81.8%) were retrieved from both search engines, while 36 (18.2%) were found only on Google Scholar. The degree of agreement between the 2 reviewers was very good (Kappa= 0.988). The number of yearly publications during the period 2014 to 2016 were higher in comparison to the period from 2000 to 2012. In the period 2014 to 2016, the yearly publications were as many as 36 (18.2%), while in 2000 to 2012 there were only 5 to 15 publications per year ().

A graphic demonstration of number of Saudi abdominal surgery publications per year.

Quality of the studies

Level III studies made up 100/198 (50.5%) of the total publications, followed by Level IV (n=97, 49%), and Level II (n=1, 0.5%). No Level I and V studies were identified (). Case reports were the most common type of study design retrieved in our search (n=93, 47%), followed by retrospective studies (n=75, 37.9%), prospective studies (n=21, 10.6%), cross-sectional studies: (n=6, 3%), and case-series (n=2, 1%). One (n=1, 0.5%) randomized controlled trial (RCT) was found. When analyzing LOE per sector, 61.3% of papers produced by academic institutions (n=93) were level III, while 38.7% were level IV. Moreover, of total research output from governmental institutions (n=75), 1.3% were level II, 38.7 level III, and 60% level IV. Military institutions had 48.1% level III of their total research production (n=27), and 51.9% level IV. Finally, private sector (n=3) had 33.3% level III and 66.7% level IV.

Authors’ affiliations

summarizes the top primary affiliations from KSA and their contributions to abdominal surgery research.

Journals’ impact factor

The published abdominal surgery research had impact factors (IF) ranging from 0.082 to 3.8 (Median, 0.73). Ten articles (5.1%) were published in journals that lacked recorded IF. The most frequently used journals were Saudi Medical Journal (n=49, 24.7%), Saudi Journal of Gastroenterology (n=25, 12.6%), International Journal of Surgery Case Reports (n=11, 5.6%), Journal BMJ Case Reports (n=8, 4%), World Journal of Gastroenterology (n=5, 2.5%), Obesity Surgery (n=5, 2.5%), and Annals of Saudi Medicine (n=5, 2.5%). The remaining 90 articles (45.5%), were published in 65 different journals, 1 to 4 articles in each journal.

Articles’ citations

Citation numbers for the publications ranged from 1 to 92 (median 6); only 53 publications (26.8%) had no verified citation numbers.

Discussion

Most Saudi publications were of level III (50.5%). We compared our results with other similar studies with similar specialties and methodology, to eliminate confounding factors and to formulate conclusions that are generalizable; however, such studies are lacking. One local Saudi study by Almaghrabi et al9 where the main focus was Saudi publications in gastroenterology, 80.7% of the articles included represented level IV.9 Other Saudi studies,6-8 similar in the methodology in neurosurgery (91%), orthopedics (86%), and plastic surgery (91%), we found that most of the articles included in those studies were of level IV evidence.6,7,8 All the previously mentioned studies demonstrated a lower LOE then the one found in our current study, in which level III was presented by (50.5%) of all included articles.

With regards to the global literature, one study by Müller et al shared the interest in LOE in surgery publications, in particular, visceral surgery.5 They defined visceral surgery as surgery on organs of the digestive tract, plus surgery of the endocrine system, and the abdominal wall.5 Approximately 68% of the articles were found to be of level IV, which is consistent with other Saudi literature, but different from our current study (). However, it should be recognized that we used a different LOE scale from that used by Müller et al.5 One would hypothesize that academic institutions would produce high quality research, as observed worldwide. In our obtained data, we found no statistically significance difference in LOE between academic institutions and other institutions (p=0.4, CI= -38.11-42.91). Looking at study design, most of the publications in our current study were case reports, which is consistent with Jamjoom et al6 where case reports comprise 47.5%. Case series, and cross-sectional studies were the majority in the other studies of plastic surgery (41.9%) and gastroenterology (33.9%).

The previous results imply that Saudi publications in abdominal surgery are of low quality (low LOE). As might be obvious, high LOE studies are not always feasible to carry out due to ethical, financial, logistic, and other relevant issues. Taking RCTs as an example, McCulloch et al11 argued that randomized trials in surgery are difficult to conduct due to many obstacles; including the lack of proper training in research methodology and epidemiology in general, lack of proper funding, and some technical aspects in surgery that make it difficult to design a trial. One significant obstacle in KSA is the cultural resistance of the public to enroll in clinical trials. Research in KSA faces many difficulties that might have played a role in the observed low quality and frequency. One of these difficulties is the lack of knowledge in proper research methodology, since until recently, many medical schools in KSA have not given much attention to research methodology training for medical students.12 Moreover, residents enrolled in postgraduate programs have no secured research time, such as their counterparts in other countries.12 Most importantly, and due to a highly demanding health care system with tons of clinical work, there is a lack of internal drive to conduct research.12 Therefore, low LOE studies may be the only available studies to conduct, and those can give only a general answer to many clinical questions clinicians face in their daily clinical practice.13 Poolman et al argued that studies labeled as high LOE, are not necessarily of high quality due to the possibility of the presence of methodological flaws.14

We recommend focusing on methodologies of studies in addition to LOE, because a study with low LOE but with well-constructed methodology would likely provide more valuable information on patient care than would a high LOE study with methodological flaws. However, this should not keep us from striving to conduct higher quality research when possible, by collaborative efforts of Saudi centers, in order to provide improved patient care that is tailored to the Saudi population.

Study limitations

One limitation in this study is that the results are not generalizable, due to the absence of studies that are identical in specialty and methodology. Moreover, more databases could have been accessed to increase the number of included articles, to determine a pattern. Articles published in the nineties (1990-1999) could also have been included.

In conclusions, we have observed that LOE of Saudi publications in abdominal surgery has not changed greatly over the period of 16 years. There is a strong need to produce high-level LOE studies, since the characteristics of the Saudi population is different from other populations, and such studies would result in better patient care. This might not be feasible without the national collaboration of various institutions; which can probably be best achieved under the umbrella of a scientific society. Thus, we urge our colleagues to strive for more collaborative research for better patient care tailored for the Saudi population.

Table 1

Top 10 Saudi institutions and their contribution to abdominal surgery literature.

Table 2

Level of evidence (LEO) of Saudi abdominal surgery publications in comparison to data of an international study.