Systematic Reviews in Occupational Health and Safety: where are we and where should we go?

Systematic Reviews have been introduced to improve the synthesis of available evidence and to reduce bias in the conclusions about a body of evidence. Nowadays, Systematic Review is an established method also in the Occupational Safety and Health (OSH) field. It is the Cochrane Work Review Group that facilitates authors to produce Cochrane reviews of intervention topics in this area. A variety of guidelines used Cochrane Work reviews for underpinning their recommendations. Due to the comprehensive search and reproducibility of the methods of a systematic review, it turned out that systematic reviews can be powerful in changing beliefs. For example, studies published in the eighties advocated the use of back schools. Nowadays, we know that the total body of evidence has changed the traditional view that training in lifting techniques could prevent back pain. 'Sitting is the new smoking' is an eye catching nicely alliterating motto, but it is of course highly overstated. The findings of a Cochrane review of the effects of interventions to decrease sitting at work showed that sitting time can be reduced by a bit less than two hours per day by providing sit-stand desks plus education. However, it is unclear if this is sufficient to counter the effects of sitting. A wealth of evidence on OSH interventions has been collected by international collaboration in the Cochrane Work Review Group. This can be extended to systematic reviews of the effects of exposure of workers to assess to which risks of adverse health effects they are exposed.

Introduction

Systematic Reviews have been introduced to improve the synthesis of available evidence and to reduce bias in the conclusions about a body of evidence. The concept of systematic reviews has been around for about 30 years now. It was especially introduced because it was clear that at that moment in time conclusions about a body of evidence were mainly based on expert opinion and that turned out to be severely biased (14). At around the same time, the Cochrane Collaboration took up the challenge to conduct systematic reviews of evidence of health care interventions. This proved highly successful and as of October 2019 there are more than 8000 Cochrane systematic reviews available in the Cochrane Library (www.cochranelibrary.com).

Cochrane is organised around medical themes. For the Occupational Safety and Health (OSH) field, it is the Cochrane Work Review Group that facilitates authors to produce Cochrane reviews of intervention topics in this area. The group started in 2003 as the Cochrane Occupational Health Field based on an initiative of the Finnish Institute of Occupational Health (FIOH) and then the group’s status was changed to Cochrane Work Review Group in 2010 thanks to the insights of the leadership of FIOH at that time. The Cochrane Work Review Group has started working on a portfolio of 77 OSH related titles of which 61 are maintained as full reviews. This work has been conducted by 404 review authors, numerous editors, reviewers and Cochrane colleagues without whom this could not have been achieved.

In this article we would like to draw on the experiences of more than 15 years of work in Cochrane Systematic Reviews and present some useful examples and project these into the future.

Systematic Review is an established method also in the OSH

In 2003 it wasn’t at all clear if the Cochrane method of systematic review could be applied to OSH topics. There was doubt that there would not be sufficient quality evidence such as Randomised Controlled Trials (RCTs) or that more in general there would not be evaluation research available. However, systematically searching for the evidence of effectiveness of interventions in databases of the research literature revealed that studies were available including RCTs (18). It was also realized that evaluation research in OSH is different from the clinical setting. For example, more interventions are carried out at the group level which makes randomisation of individual participants to intervention and control condition more difficult (6). Also, the setting of a work organisation and the legal context make it more complex to conduct evaluation research because more parties are involved than just a patient and a physician. These have been used as arguments to include more study types than just RCTs such as Controlled Before After (CBA) studies, which are similar to RCTs but in which the participants are assigned to the intervention or control group based on a non-random mechanism such as geographical location (6). Another study design that seems particularly useful for OSH interventions is the Interrupted Time Series (ITS) design (17, 22). Control for changes that always occur is achieved by comparing the trends in the outcome over time before and after the introduction of the intervention. This is only feasible if one can dispose of routinely collected outcome data and the intervention is introduced at a specific moment in time.

The difficulties in conducting systematic reviews of not just interventions but all kind of OSH questions, led the National Institute for Occupational Safety and Health (NIOSH) to assign an investigation into the feasibility of systematic reviews in OSH to the Rand Corporation (5). The outcome was that systematic reviews are feasible, but that for topics other than interventions, such as the effects of exposure, the methods have to be adapted. In addition, the World Health Organization (WHO) has decided that systematic reviews should be conducted as a basis for any recommendation that the organisation will make. The feasibility of this has also been demonstrated by the systematic reviews underpinning the recommendations in the WHO guideline for working safely with manufactured nanomaterials.

Given the authoritative status of WHO, NIOSH and the Rand Corporation, we conclude that the method of systematic review has now been well established in the OSH field.

Systematic Reviews are still not important enough

Like the results of any research, the results of systematic reviews can be used as input for decisions to implement interventions on a larger scale or to improve the quality and extend of an existing body of evidence. We have been impressed by the uptake of Cochrane Work reviews in guidelines and recommendations around the world. The Food and Drug Administration (FDA), the Centers for Disease Control and Prevention (CDC) and NIOSH jointly recommended the use of blunt needles to surgeons for preventing needle stick injuries based on a Cochrane Work review (4). A search for the use of Cochrane reviews in guidelines, yielded a wide variety of guidelines that used Cochrane Work reviews in 2017.

Despite our enthusiasm for the implementation of systematic review methods and their use to underpin recommendations, we are disappointed by the use of systematic reviews by researchers. Given that new researches should add to the already existing body of evidence, nowadays researchers should cite at least one systematic review in the introduction of their article giving a rationale for the new study. However, in practice, we see that many if not most articles still haphazardly cite various studies without being clear what their new study will add. Checking the latest published articles in OSH journals one can come across some papers that cite some studies in the introduction as a rationale for doing new research. It could be unclear if the cited studies cover the whole evidence base and why they were chosen. It could be also unclear what the results of the studies are: some can be simply quoted as being insignificant, for others the authors can give an Odds Ratio (OR) plus 95% Confidence Interval (95% CI).

Systematic Reviews can shock beliefs

As John Ioannidis nicely pointed out: most published research findings are false (7). He meant that most findings from studies that are published do not hold over time when other studies try to replicate these findings. However, the first published stay in people’s minds and lead to ineffective interventions being implemented for a long time (8). Systematic reviews can help here. We have experienced this ourselves as we were firm believers in the hypothesis that teaching workers correct lifting techniques will prevent back pain. This was based on studies published in the eighties of the previous century that advocated the use of back schools (9). Once we had collected all available evidence, it turned out that there were nine RCTs and nine CBAs of which none showed a significant beneficial effect on back pain (23). Among these, there were RCTs published in the highly ranked New England Journal of Medicine (NEJM) and RCTs that were well designed with large sample sizes of almost 2000 workers. We apparently had not taken notice of any of these. The total body of evidence made it impossible for us to maintain our view that training in lifting techniques could be beneficial. Therefore, we realized that systematic reviews can be powerful in changing beliefs.

Sitting is not the new smoking but still an important public health problem

‘Sitting is the new smoking’ is an eye catching nicely alliterating motto, but it is of course highly overstated. The risk of lung cancer in smokers is about 14-fold compared to non-smokers, whereas persons who sit a lot have about 1.3 times greater risk than those who sit a small part of the time (12, 20). Nevertheless, it has been shown that physical activity at work over the past 30 years has decreased more than 25% (12). There is no doubt that being physical active contributes to a healthy life. The decrease in physical activity at work is thus a serious occupational health problem. That was the reason to conduct a review of the effects of interventions to decrease sitting at work (20). After several updates, the review included 10 studies of sit-stand desks that showed that sitting time can be reduced by a bit less than two hours per day by providing sit-stand desks and education about the harms of sitting. However, it is unclear if this is sufficient to counter the effects of sitting. If we assume that the harms are brought about by being inactive, then replacing sitting by standing is not going to help because it entails the same amount of energy consumption. The review did not yield any other interventions that could make workers more active. This means that it seems that we don’t have a solution for this important public health problem yet and that more efforts to find solutions are needed.

Better, more relevant questions are needed for return to work interventions

There is a large variation in the objectives of occupational health services among different countries. In some countries the services of occupational health physicians are mainly geared towards the prevention of occupational diseases, whereas in other countries it is all about rehabilitation and return to work of workers that have a medical condition. Return to work has important social and economic implications both for the individual and the society at large. This is one of the reasons that we have welcomed reviews of return to work interventions in workers with specific medical conditions. One of the main questions here is if interventions are disease-specific or they can be implemented across medical conditions. The course of diseases varies, and this has implications for sick leave and work participation. Cancer is a sudden diagnosis with usually an immediate consequence of sick leave often leading to job loss in the long term (2). On the other hand, back pain is recurrent and does often not lead to sick leave and leads seldom to job loss (24). Therefore, it seems that an approach based on specific diseases is most feasible. This has resulted in about a dozen reviews of return to work interventions. At best these interventions have a small effect and at worst they are not effective at all or have inconsistent results. However, it is difficult to categorize these interventions. Often the authors describe that they undertook a return to work intervention without clearly depicting what they did and why they did it. Most interventions consist of some kind of education and support in addition to medical treatment. This makes it difficult to categorize interventions and it can result in combining apples and oranges. We conclude that there is a need for better underpinning of interventions with theory and looking at a wider range of interventions than just education and support of workers on sick leave.

A good systematic review team is gold and provides realist reviews

Research is a highly competitive environment (16). For systematic reviews, we believe that team work is much more important. Many reviews consist of a sort of mechanical combination of the effects of studies. However, as argued before, a systematic review should be about what works for whom and under which circumstances (15). It helps to better understand an intervention or an exposure if there are people on the review team with different points of view and with different cultural and professional backgrounds. This prevents one from becoming stuck on one point of view only and taking into account a wider range of arguments. As mentioned before, the goal of a systematic review is also to provide input for new research. If one wants to avoid the platitude ‘better quality research is needed’, one has to have a good understanding of the research conducted and what its short comings are. Having a diverse team almost always helps there.

The conduct of systematic reviews should be automated soon

For those readers who have conducted a systematic review, it will be no news that some parts of the review are really repetitive and sleep provoking. For some review, one has to spend hours and hours going through the results of database searches to weed out the irrelevant references (1). In a time of fast developing of artificial intelligence, it is difficult to understand how such repetitive tasks are still not automated. Already in 2013 authors envisaged that soon ‘autonomous agents will sift the evidence continuously and will use their protocols to provide updated reviews on demand’ (21). Even though reasonable on-line tools for systematic reviewing have been developed such as Covidence, Ryann and Destiller, this is still far from an autonomous agent providing reviews on demand. It seems that considerable more investment is needed to get us there (13). Until that time, tedious and boring repetitive work will constitute a big part of the systematic review process.

Big potential for systematic review development in OSH

In OSH reviews the effects of exposure are equally important as reviews of the effects of interventions. Much of the work of OSH professional concerns risk assessment. The evidence for risk assessment is provided by systematic reviews of studies that evaluate the effects of exposures. The methods for these reviews are to a large extent similar to those of reviews of the effects of interventions but they also differ (3). At the moment, reviews of effects of exposure very often lack a proper risk of bias assessment (19). Tools for risk of bias in studies of exposure are still under development but promising (11). Studies of exposures are often more heterogeneous than studies of interventions and it is difficult to properly quantify this. These issues pose still considerable challenges, but we expect that there will be fast developments here which can be very beneficial for OSH.

Applying findings of systematic reviews of OSH topics is still challenging

Systematic reviews seldom conclude that there is high certainty evidence for the effects of an intervention or exposure. This is of course inherent to the type of evidence we are using. Human epidemiological studies almost always suffer from bias and imprecision which make the results less certain. In case of OSH, where we also use evidence from non-randomised studies the picture gets even more blurred because of the weaknesses of the non-randomised studies. For many reviews, the authors propose to conduct better quality studies, but it is questionable if they will ever be conducted. This means that we have to make better use of the existing evidence. Guidance for doing so is already available in the Evidence to Decision framework that the GRADE working group has produced (10). When using evidence to underpin a decision or making a recommendation, GRADE recommends to take also other factors into account such as costs and values and preferences of stakeholders. Let’s take an intervention for which there is low quality evidence. The decision to implement this intervention will be much easier if the intervention is very cheap than when it is very expensive. We therefore suggest complementing the evidence on effectiveness with information on costs and reviews of values and preferences of stakeholders.

Funding for international collaboration is needed

Cochrane Work has been lucky to be funded by FIOH until May 2019. If we see this as a joint effort to collect the available evidence on relevant OSH topics, it makes sense to collaborate internationally. Therefore, we need a global alliance of stakeholders such as national institutes of occupational safety and health to support the collection and implementation of evidence.

Conclusion

In the past 15 years, systematic review methods have been well established in occupational safety and health. A wealth of evidence on OSH interventions has been collected by international collaboration in the Cochrane Work Review Group. This experience can be extended and applied to systematic reviews of the effects of exposure of workers to assess to which risks of adverse health effects they are exposed. International collaboration should continue also in the future.

No potential conflict of interest relevant to this article was reported by the authors