Systematic reviews. Minimizing search bias.

I have enjoyed reading the recent important article of Mansuri et al1 on road traffic collisions. Being a trauma researcher working in the Gulf Region for more than 25 years, I felt that there were some important missing articles in this systematic review. I took the burden to follow the same methodology of the authors so as to find why that happened. The most important part of a systematic review is to find all relevant available evidence if possible. Search bias can have a major impact on systematic reviews.

First, it is very important to use all relevant terms. The authors searched “road traffic accidents” and “Saudi Arabia” as a text term. It is well known for trauma researchers that the British Medical Journal banned the term accidents in 2001, and we follow that advice.2 The authors should have used other alternative terms like crash or collision in their search. We have to acknowledge that PubMed has the ability to automatically search for alternative terms, such as “KSA” for Saudi Arabia, but this advantage does not apply for all other 5 databases, which were used by the authors. That is why it is important to involve a search engine expert to define all possible terms. The search process should be repeated until exhausted, and no further papers are found.

Using the same search methods and period of the authors in PubMed, I could locate 75 articles. Using different, terms, affiliations, combination techniques, and the same period, I could locate 82 articles in PubMed within 2 hours. Comparing both searches, I found 4 relevant articles that should have been included.3-6 Finally, I am aware of a very important paper on driver’s behavior in roundabouts that was not even picked up by the terms used.7 The authors, at least, missed 5 articles out of 34 (15%) in one search engine simply because of search strategy.

It is important to acknowledge that the search bias did not have a major impact on the present review because data were not pooled together. Nevertheless, this will dramatically impact a meta-analysis of randomized controlled trials, especially when the number of studies is small or a study with large number of patients was missed (large weight study). Furthermore, I wonder why the authors required an ethical approval for their study because the data used were research papers and reports of public domains.

Finally, I would like to congratulate the authors for their excellent paper, I personally enjoyed reading it, and I hope that my comments will be useful for future studies.

Thank you very much for showing interest in our article entitled “Road safety and road traffic accidents in Saudi Arabia”. To reiterate, the aim of this paper was to cumulate findings of all road traffic accidents (RTAs) reported in the last 25 years regarding its trends and preventive approaches. Furthermore, this systematic review was planned to propose a standardized surveillance system for RTAs, which is a neglected front in the area in KSA.

Our respectable expert highlighted that 5 articles were missed out in this systematic review. These were picked up in the initial search, however they were not included, as those of Al Ghamdi6 and Bener and Jadaan3 studies’ were intended towards “health system research,” and not biomedical one. Whereas, that of Khan and Mirdad5 studied specific musculoskeletal management success and not distribution of overall traffic injuries. Nofal et al’s study5 had already been discussed in our systematic review and cited as ref 29. In page 420, Table 1 of our study,1 we explained that 8 full text articles’ results and data collection procedures were described according to meta-analysis of observational studies in epidemiology criteria to show deficient surveillance system of information on road crashes.

We appreciate expert’s opinion and his agreement that no robust statistical procedures were required and as well used in this systematic review, which in fact, again signifies the impact of this research. This paper is an eye opener for public health planners that unless a unified system of data collection is used, the problem cannot be justly recognized, or addressed accordingly. We hope to plan systematic reviews with more precision and conviction in the future.

Department of Community & Family Medicine, Taibah University, Al Madinah Al Munawwarah, Kingdom of Saudi Arabia