Effectiveness of preventive interventions and Randomised Controlled Trials in Occupational Health: an overview of the last five decades.

INTRODUCTION: Evidence-Based Medicine, as a new scientific paradigm, modified the approach to diagnosis, -treatment and prevention of diseases based on the best available scientific evidence synthesized in systematic reviews since the last decade of the past century. To evaluate its influence, we assessed the trend in the number and proportion of -randomised controlled trials (RCTs) and systematic reviews of preventive interventions in occupational health (OH) over the last five decades.
METHODS: PubMed has been searched using established search filters regarding occupational determinants of diseases, OH preventive interventions, RCTs and systematic reviews. The number of hits were -assessed per decade. We estimated the number of pertinent studies in the systematically recruited samples of retrieved citations.
RESULTS: Over the years, the number of studies concerning the effectiveness of preventive interventions in OH increased 3.5-fold from 986 in 1970-1979 to 3,428 in 2010-2019. RCTs of preventive interventions increased more than 60-fold from 6 in the seventies to 370 in the last decade. Systematic reviews first appeared at the end of the past century with a 30-fold increase (from 4 to 120) over the last three decades. DISCUSSION: The number of high-quality studies, such as RCTs and systematic reviews evaluating the effectiveness of preventive interventions in OH, has increased more rapidly than other studies on this topic. The Evidence-Based Medicine philosophy, diffused by researchers worldwide, has promoted the evaluation of the effectiveness of preventive interventions in OH.

Introduction

Archibald Cochrane, an epidemiologist, and former occupational health physician (1), was one of the researchers that changed our way of thinking and acting in Medicine. Together with the colleagues of the North American Public Health, he stressed the importance that healthcare interventions had to be based on critical summaries of the best available evidence from the scientific literature (2). Following these ideas, at the beginning of the nineties of the past century, the term Evidence-Based Medicine was coined in Canada (3), and the Cochrane Collaboration took the first steps in Great Britain (4). This new scientific paradigm profoundly modified the approach to diagnosis and treatment and promoted the systematic search of evidence of the effectiveness of preventive interventions as well.

It was since the mid-nineties that the evidence-based paradigm began to be enforced in Occupational Health (OH) (5). OH researchers started promoting the evaluation of the effectiveness of preventive interventions and carried out systematic reviews, showing the relevance of applying Cochrane methodology to improve workers’ health. As a consequence, in 2004, the Cochrane Occupational Health Field (currently Cochrane Work) began its activity, performing systematic reviews of preventive interventions (6, 7) and at present also promoting systematic reviews of the effects of exposure of workers to assess to which risks of adverse health effects they are exposed (8).

Among the study designs that can be applied to evaluate the effectiveness of OH intervention, Randomised Controlled Trials (RCTs) are known to provide the best quality of evidence (9). Randomisation is the defining feature of RCTs. Each subject has the same chance to be allocated either to the intervention or to the control group (i.e. alternative intervention/no intervention). As the groups are treated identically apart from the intervention received, the difference in outcomes can be attributed to the intervention under study. Therefore, RCTs stimulate researchers to set up the best possible experimental studies. In the case of human populations, every person is different from each another in terms of genetic background and exposure to environmental factors introducing potential confounding in non-experimental observational studies. In observational studies, we are used to control for confounding both at the design stage and in the analyses, but it is hard to achieve the same control as in an RCT.

To evaluate if there has been an increase, over the last five decades, in the proportion of RCTs and the number of systematic reviews of OH preventive interventions, PubMed citations have been explored up to 2019.

Methods

Retrieval of citations regarding intervention studies in OH

The searches were run in MEDLINE (through PubMed) from 1970 to 2019 on July 2, 2021.

To retrieve citations regarding intervention studies in OH, the most specific PubMed search filter for occupational determinants of diseases (10), and the most specific PubMed filter for occupational health interventions (11) were used.

To limit citations to mostly RCTs, the Cochrane highly sensitive search strategy for identifying RCTs in PubMed (sensitivity- and precision-maximizing version, 2008 revision) was used (12). To search for systematic reviews (including Cochrane reviews), the PubMed systematic reviews filter was applied. The search strategy is described in Table 1. The number of hits per decade is reported according to PubMed ‘date of publication’ filter.

Table 1.

Search strategies developed for PubMed

	Search statement
	The more specific search filter for occupational determinants of diseases (10):
1	(occupational diseases[MH] OR occupational exposure[MH] OR occupational medicine[MH] OR occupational risk[TW] OR occupational hazard[TW] OR (industry[MH] AND mortality[SH]) OR occupational group*[TW] OR work-related OR occupational air pollutants[MH] OR working environment[TW])
	The most specific search filter for retrieving studies of OH interventions (12):
2	(program[TW] OR “prevention and control”[SH]) AND (occupational[TW] OR worker*[TW])
	The Cochrane highly sensitive search strategies for identifying RCTs in PubMed (sensitivity- and precision-maximizing version, 2008 revision) (13):
3	(randomized controlled trial[PT] OR controlled clinical trial[PT] OR randomized[TIAB] OR placebo[TIAB] OR clinical trials as topic[MESH:NOEXP] OR randomly[TIAB] OR trial[TI]) NOT (animals[MH] NOT humans[MH])
	To retrieve citations regarding preventive interventions in OH:
4	#1 AND #2
	To retrieve RCTs that likely evaluates the effectiveness of preventive interventions in OH:
5	#1 AND #2 AND #3
	To retrieve systematic reviews that summarises the effectiveness of preventive interventions in OH:
6	(#1 AND #2 AND systematic[SB]) OR (#2 AND “Cochrane Database Syst Rev”[TA])

Estimate of pertinent citations regarding intervention studies in OH

To estimate the proportion of references pertinent to OH interventions, we evaluated a random sample of citations returned by each search. We calculated the sample size based on an alpha error of 0.05 and a precision level of 90% (13). In the case of search queries retrieving less than 300 abstracts, we assessed all the citations.

Two authors (SM1, SM2) independently assessed the pertinence to: i) OH topic; ii) preventive interventions in OH; iii) intervention studies likely to be RCTs; iv) systematic reviews on OH preventive interventions. A third author (SC) resolved any disagreements.

Results

Table 2 reports the total number of citations included in PubMed over the last five decades, the number and proportion of citations retrieved with our search strategy, the estimated number and proportion of citations pertinent to OH, the number and proportion of RCTs and the number of systematic reviews of OH interventions. Overall, citations pertinent to OH were about 60% of the citations retrieved by the search, OH preventive interventions accounted for 33%, RCTs for 42% and systematic reviews for 52%.

Table 2.

Number and proportion of retrieved citations and estimated number of pertinent citations by decade.

		1970-1979	1980-1989	1990-1999	2000-2009	2010-2019
PubMed citations, n		2,458,012	3,330,441	4,440,722	6,529,127	10,600,966
OH citations, n (% of PubMed citations)	retrieved	23,996 (1.0)	32,099 (1.0)	38,352 (0.9)	44,790 (0.7)	54,837 (0.5)
	estimated	15,117 (0.6)	20,222 (0.6)	24,162 (0.5)	28,218 (0.4)	34,547 (0.3)
OH preventive interventions, n (% of OH citations)	retrieved	3,081 (12.8)	5,323 (16.6)	7,359 (19.2)	9,297 (20.7)	9,521 (17.4)
	estimated	986 (6.5)	1,456 (7.2)	2,944 (12.2)	2,603 (9.2)	3,428 (9.9)
OH preventive interventions likely to be RCTs, n (% of OH preventive interventions)	retrieved	33 (1.1)	54 (1.0)	214 (2.9)	451 (4.8)	712 (7.5)
	estimated	6 (0.6)	16 (1.1)	75 (2.5)	149 (5.7)	370 (10.8)
Systematic reviews of OH preventive interventions, n	retrieved	0	0	5	88	244
	estimated	0	0	4	50	120

Abbreviations: OH, occupational health.

Over the years, the number of citations of biomedical papers in PubMed has dramatically increased. In parallel, the pertinent citations of OH studies have increased as well, but less rapidly. The estimated number of studies of the effectiveness of preventive interventions in OH increased more than threefold from 986 in the decade 1970-1979 to 3,428 in 2010-2019. These studies account for 10% of all OH studies. The OH preventive interventions likely to be RCTs grew exponentially in number and proportion over the decades compared to the citations of preventive interventions. In the 70’s and 80’s, RCTs in OH preventive interventions were only 6 and 16, respectively. Notably, in the last decade (2010-2019) there were 370 (about 11% of all the OH preventive interventions) – an increase of more than 60-fold compared to 1970-1979. Systematic reviews of OH preventive interventions were non-existent in the first two decades under study. Four systematic reviews were retrieved in 1990-1999; the first of these was aimed at preventing back pain in industry (14). In 2010-2019 they reached a value of 120 with a 30-fold increase over the last three decades.

Discussion

The proportion of high-quality studies, such as RCTs, increased over the decades among the studies evaluating the effectiveness of preventive interventions in OH. Remarkably, their number increased more than tenfold among OH preventive interventions (from 0.6% in 1970-1979 to 10.8% in 2010-2019). At the same time, systematic reviews of OH preventive interventions, virtually not existing before the 90’s, are now regularly carried out based on an even larger number of available RCTs and other study designs for OH preventive interventions.

In 2005, the proceedings of 15 congresses of the Italian Society of Occupational Medicine (1989-2003) were screened (15). The articles about the effectiveness of OH preventive interventions accounted for 3% out of the total and no RCTs were identified. In 2012, we hand searched the reports of OH intervention studies published in Italian in peer-reviewed scientific journals between 1990 and 2008 (16): only one RCT was detected out of 25 included studies. Publication bias and language bias likely influenced these figures. However, these results confirm that in the past decades the proportion of RCTs was a small part of the number of studies on evaluation of the effectiveness of OH preventive interventions. In 2006, the colleagues of the Cochrane Work (17) manually searched 16 biomedical journals. The proportion of RCTs accounted for about 20% of the studies evaluating the effectiveness of OH interventions and about 0.4% of the total of published screened articles. These findings, even if based on a restricted number of journals, comply with the results of this overview: in the last two decades, RCTs became frequently used as study design to evaluate the effectiveness of OH preventive interventions.

Practical and ethical concerns can limit the use of RCTs to evaluate the effectiveness of preventive interventions in OH. In addition to that, treatment contamination between intervention and control groups is likely, as a result of the frequent absence of blindness and potential proximity of workers allocated to the intervention and control group.

Cluster RCTs are often used to prevent contamination between intervention and control groups (18). Cluster randomisation is used to allocate groups of subjects (not individual participants) to different interventions. Clusters are usually groups of subjects connected to a common reference centre like students enrolled in a given school, patients of a given General Practitioner, or workers of a given factory. In OH it would be easier to randomise factories than workers; in such a way all the workers of a given factory will receive (or not) the intervention, minimising the risk for contamination between intervention and control groups. An example of cluster RCT is the recent evaluation of the effectiveness of an intervention to reduce the sitting at work (19).

Apart from RCTs, other study designs can be useful to evaluate the effectiveness of OH preventive interventions such as Controlled Before-After (CBA) studies and Interrupted Time Series (ITS). Researchers and practitioners could apply CBA study designs in the case randomisation is not feasible because of (i) ethical considerations or (ii) inability to randomise subjects or clusters. In a CBA study, two groups – not randomised - are usually compared: the intervention group in which the intervention is carried out and the control group where no intervention (or an alternative intervention) is put in place. A comparison group enables researchers to control for changes occurring over time that happen simultaneously with the exposure or intervention of interest. However, these studies have a high risk of bias considering that there is always a risk of unidentified differences between the intervention and control groups that may affect changes in the outcome measure (20). An example of CBA is the study by Risør et al. (21) aimed at evaluating an intervention for patient-handling equipment. ITS studies allow researchers to examine the effect of an intervention as well. Multiple data are collected before and after the intervention: at least three times according to the EPOC (Effective Practice and Organisation of Care) criteria (20). When a comparison group is impractical and the outcome is available on an administrative basis, ITS studies could be a good alternative, as in the case of the study of the effectiveness of workplace inspections in preventing injuries performed by Agnesi et al. (22). Nowadays the Cochrane Work review group includes a broader range of study designs in their systematic reviews including non-randomised studies (i.e. ITS and CBA studies), considering that RCTs are often not available to address questions about the effectiveness of OH interventions (23). Unfortunately, we were not able to explore the amount of CBA and ITS studies of OH preventive interventions carried out in the last decades due to the lack of a specific search strategy.

The present overview was limited to PubMed considering that most of the high-quality intervention studies are indexed in PubMed (24). This enables us to use validated and ready-to-use PubMed search filters (10-12) to develop a proper search strategy to retrieve RCTs of OH interventions or systematic reviews summarising the effectiveness of such interventions. A PubMed search filter tailored to identify non-randomised studies - such as ITS and CBA studies - may help researchers retrieving OH preventive interventions when randomisation is not feasible.

This overview shows that the commitment (in lectures, conferences, seminars, papers, editorials, books) of many researchers promoting and disseminating the evidence-based approach in OH acted like a proper “intervention”. In this hypothetical ITS study design, we might compare the number of citations retrieved in OH in the 70’s and 80’s (i.e. before the intervention) to those retrieved in the last two decades (i.e. after the intervention). All in all, the intervention seems to be “effective” to increase the number of citations in OH intervention studies.

Conclusions

The Evidence-Based Medicine philosophy has promoted the evaluation of the effectiveness of preventive interventions in OH, also with high-quality designs, as RCTs. This study design along with ITS and CBA studies is even more used today, in Occupational Medicine as well, to evaluate the effectiveness of OH preventive interventions.

Conflict of interest:

No potential conflict of interest relevant to this article was reported by the authors