Use of natural experimental studies to evaluate 20mph speed limits in two major UK cities.

INTRODUCTION: Reductions in traffic speed can potentially offer multiple health and public health benefits. In 2016, implementation of 20mph (30kph) speed limit interventions began in Edinburgh (city-wide) and Belfast (city centre). The aims of this paper are to describe 1) the broad theoretical approach and design of two natural experimental studies to evaluate the 20mph speed limits in Edinburgh and Belfast and 2) how these studies allowed us to test and explore theoretical mechanisms of 20mph speed limit interventions.
METHODS: The evaluation consisted of several work packages, each with different research foci, including the political decision-making processes that led to the schemes, their implementation processes, outcomes (including traffic speed, perceptions of safety, and casualties) and cost effectiveness. We used a combination of routinely and locally collected quantitative data and primary quantitative and qualitative data.
RESULTS: The evaluation identified many contextual factors influencing the likelihood of 20mph speed limits reaching the political agenda. There were substantial differences between the two sites in several aspects related to implementation. Reductions in speed resulted in significant reductions in collisions and casualties, particularly in Edinburgh, which had higher average speed at baseline. The monetary value of collisions and casualties prevented are likely to exceed the costs of the intervention and thus the overall balance of costs and benefits is likely to be favourable.
CONCLUSIONS: Innovative study designs, including natural experiments, are important for assessing the impact of 'real world' public health interventions. Using multiple methods, this project enabled a deeper understanding of not only the effects of the intervention but the factors that explain how and why the intervention and the effects did or did not occur. Importantly it has shown that 20mph speed limits can lead to reductions in speed, collisions and casualties, and are therefore an effective public health intervention.

Introduction

Transport related policies and practices impact health through collisions, air and noise pollution and perceived lack of safety (Jacobsen et al., 2009; Morrison et al., 2003; National Institute of Health and Care Excellence, 2017) and by providing (or not) environments conducive to active travel (Winters et al., 2017). Traffic speed, in particular, is a key risk factor in road traffic incidents for collision and injury severity (World Health Organization [WHO], 2017). For pedestrians and cyclists especially, the relationship between speed and injury is even more acute (WHO, 2017). Reductions in traffic speed can therefore potentially offer multiple health and public health benefits. These may include reducing the risk of traffic collisions and the resulting severity of injuries, encouraging greater uptake of physical activity (through increased walking and cycling) and making streets more pleasant and liveable.

On restricted roads in the United Kingdom (UK) (roads in built-up areas), the national speed limit is 30 miles per hour (mph) (UK Public General Acts, 1960). However, implementing 20mph speed restrictions in the UK has become increasingly common (Tapp et al., 2015; Toy et al., 2014). For example, city-wide 20mph speed restrictions were introduced in Portsmouth (Atkins, 2018) and Bristol (Bornioli et al., 2020), and other local authorities have introduced 20mph restrictions on a smaller, more localised scale on a pilot basis (see Cleland et al., 2019 for a summary). Typically, two approaches have been taken. Most commonly physical infrastructure is installed, such as speed bumps or chicanes. These are usually called 20mph ‘zones’ (Grundy et al., 2008). The other approach is to install ‘signs and/or lines’, without any other physical infrastructure. The latter are conventionally referred to as 20mph ‘limits’ (Toy, 2012). A recent review identified 11 published studies globally involving 20mph speed reduction interventions, nine involved zones and two involved limits (Cleland et al., 2019). The review concluded that 20mph ‘zones’ are effective in reducing collisions and casualties; however, there was insufficient evidence to draw robust conclusions on the overall public health effectiveness of 20mph ‘limits’.

In 2016, implementation of 20mph speed limit interventions began in Edinburgh (city-wide) and Belfast (city centre). Edinburgh implemented a city-wide 20mph speed limit network between July 2016 and March 2018. Around 50% of streets in Edinburgh were already 20mph; the aim was for this to be increased to 80% of streets, with the remaining 20% of streets – mostly arterial – maintaining a 30 or 40mph limit. The scale-up of 20mph limits was implemented in four phases across seven areas, with each taking approximately 16 weeks to put in place. When complete there were in total 1572 roads that were 20mph - approximately 771 miles (1240.3 km). Belfast implemented 20mph speed limits on 76 streets in the city centre. This was the part of the city with the highest levels of pedestrian movement, cycle activity and bus facilities. The 20mph streets in Belfast were surrounded by a network of 30mph and 40mph streets. The Belfast city centre scheme came in to force in February 2016 and was implemented in a single phase (Department of the Environment, 2011) but at the same time as a range of other transport infrastructure changes were taking place (www.infrastructure-ni.gov.uk/topics/transport-initiatives/belfast-move).

In 2017, the UK National Institute for Health Research (NIHR) funded concurrent evaluations of the 20mph speed limit interventions in Edinburgh and Belfast (‘Is 20mph plenty for health?’). The aims of this paper are to describe 1) the broad theoretical approach and design of two natural experimental studies to evaluate the 20mph speed limits in Edinburgh and Belfast and 2) how these studies allowed us to test and explore theoretical mechanisms of 20mph speed limit interventions.

Methods

Theoretical approach

We used a theory-based approach (Stame, 2004). This acknowledges that evaluation should make explicit the theories underlying the assumptions about how things work, and the evaluation steps (and methods) should be designed and built based on the best way to test those theories given the resources available and the context in which the evaluation is being conducted. This is done by elaborating assumptions, revealing causal chains, and engaging all concerned stakeholders. A significant part of this study also involved a process evaluation to understand implementation, mechanisms of change, and context (Moore et al., 2015).

Following the Medical Research Councils (MRC) guidance on complex interventions (Craig et al., 2008), considerable preliminary work was undertaken with stakeholders in Edinburgh to develop an initial programme theory. This was used to guide the outcome and process evaluations in both cities (Turner et al., 2018). The model from that preliminary work - outlining the intervention theory, activities and outcomes - is shown in Fig. 1. This programme theory highlights the pathways through which the reduced peak and average speeds (proposed to result from the lower speed limit) are purported to lead to health-related outcomes. Consequently, this study was designed to not only address the question of the effects and impacts of the intervention but also how and why the intervention and effects/impacts occurred or did not occur.

Fig. 1

20mph programme theory, from Turner et al. (2018).

Study design

The research team had to confront a number of challenges:

The pre-implementation and implementation activities took place over an extended time period (2000–2016) and much of it before our research began. These were dynamic continuing processes and occurred in complex systems of multiple actors and organisations, with particular local histories, contexts and vested interests.

What was done in Edinburgh and Belfast was different. The same ends were sought in both cities – to reduce traffic speed – but in order to achieve that goal, the interventions were implemented differently and on different scales.

These were not single interventions at single moments in time. What was implemented in both places consisted of a number of different elements. They involved legislation, signage, public education and enforcement, to varying degrees, and many individuals and organisations participated.

The interventions took place in multiple and very different locales (between the cities and within Edinburgh) at different times.

There were other travel initiatives taking place at the same time, including the introduction of a rapid transit system and extension of city centre bus lane provision in Belfast and active travel promotions in Edinburgh.

The mechanisms leading to the outcomes from the different components (legislation, signage, public education, enforcement) were not known empirically in advance nor was it known which parts of the various activities were more potent or possibly less or ineffective.

Guided by our programme theory, we developed a pragmatic, theory based, mixed-methods evaluation within a natural experimental study. Similar mixed-methods approaches have been used in other transport related natural experiment studies (e.g. the NIHR PHR funded ‘On the buses’ study, Green et al., 2014; and the EPSRC funded ‘iConnect’ study, Ogilvie et al., 2012; Ogilvie et al., 2017).

Data collection

The ‘Is 20mph plenty for health?’ study used a combination of routinely and locally collected quantitative data and primary quantitative and qualitative data, specifically selected to assess the pre-stated theories. The project consisted of several work packages (WP), each with different research foci. Full details can be found in the final project report (Jepson et al., 2021) and publications specific to each component of the work, but in summary:

The objectives of WP1 were to assess the impact of introducing 20mph speed limits on: driver perceptions; public support; perceptions of safety; perceptions of the pleasantness of the environment; traffic speed and volume; the number and type of road casualties; and the number of journeys made by walking or cycling. The data for WP1 were primarily (although not exclusively) already being collected by third parties – the local and national government, external contractors, and Sustrans – and not necessarily for the purpose of evaluating the 20mph limits.

The objectives of WP2 were to understand the implementation processes and activities, barriers and facilitators to successful implementation in Edinburgh and Belfast, and to evaluate how and why behaviour change may, or may not have occurred, predominantly through primary qualitative enquiry with key implementation actors and groups of the population exposed to the speed limit interventions.

The objectives of WP3 were to investigate the factors behind the decisions to implement the schemes in the two cities in the first place and to their possible transferability to other places. This involved documentary analysis and interviews to investigate the political decision-making processes that preceded the implementation of 20mph speed restrictions in the two cities.

The objective of WP4 was to undertake an economic evaluation of the intervention, comprising cost-benefit analysis (CBA) and cost-consequence analysis (CCA), using the range of data collected in the other work packages.

Ethics

Ethical approval was obtained from the Moray House School of Education Ethics Committee at The University of Edinburgh.

Results

Using a theory-based approach, we were able to assess different parts of the programme theory to determine their importance and impact, as well as the overall effects of the intervention. It should be noted that due to a lack of adequate and robust data, we were not able to test all aspects of the programme theory. Within this section we provide an overview of the findings emerging from the key work packages. Full details of the project findings can be found in the final report (Jepson et al., 2021).

Pre-implementation

The evaluation shed light on the many contextual factors influencing the likelihood of 20mph speed limits reaching the political agenda in the first place. While the original programme theory highlighted the role of local stakeholder groups, our research identified a much broader range of factors that determine whether 20mph speed limits reach the political agenda including global and national policy, local histories, politics, bureaucracy, evidence, and the public.

In both Edinburgh and Belfast, it was not the case that a quick decision was made and the intervention was implemented. In both cities there was around a twenty-year history from 20mph limits being recognised as a potential public health and transport intervention, to the point where the schemes became a reality. In neither city were there major landmark events that caused a radical shift in policy. Rather ‘baby steps’ were taken to nudge closer and closer to the idea and the eventual reality over a sustained period of time, such that what unfolded was seemingly inevitable.

During this long lead-in period, different factors were important at various times throughout the discussions. The initial rationale for slower speeds in both cities was road safety, and scientific evidence was critical in the early discussions, although most of the cited evidence was on the link between speed and risk, rather than the effectiveness of 20mph limits per se. As the interventions came closer to being a reality, practical considerations became more pertinent in discussions, such as the potential impact on bus timetabling and fast food delivery times.

A key facilitator in both cities was the shift in the intended design of the speed limit interventions from ‘zones’ to ‘limits’, removing the need for physical infrastructure. This made the intervention considerably cheaper and more feasible to implement at scale, making it more attractive to decision makers. A critical feature in both cities was that the issue never became party political; there were key individuals across parties and other stakeholder groups who were in support of the initiative. It was also important to build public support, by seeking opinion and responding to opposition.

Implementation processes

There were four implementation components of the intervention in both cities: 1) legislation; 2) signage; 3) public education; and 4) enforcement. However, there were substantial differences between the two sites in several aspects related to implementation such as the governance, delivery partners involved, and the scale, timeframe and phasing approach of the various implementation components.

Implementation was broadly considered as being ‘delivered as intended’ in both Edinburgh and Belfast, with few practical issues noted by delivery partners. Only minor amendments to implementation were made during delivery. For example, changes to signage were made in Belfast in response to public feedback that the signs weren't visible enough. Approximately 8 months into implementation, the original signs with a white background were replaced with signs with a yellow background.

A joint and integrated public education and awareness campaign was viewed as an integral component of implementation in Edinburgh. This was delivered from the outset of implementation, with dedicated staffing and funding allocated to this component of implementation. In contrast in Belfast, this component of implementation was delivered on a very small scale with few delivery partners involved. The scheme in Edinburgh was perceived by the general public as being highly visible, attributable in part to the education and awareness activities which took place. In contrast, a lack of awareness of the 20mph speed limit initiative reported by some population groups was an important finding in Belfast.

In both sites, police enforcement was influenced by resources and competing priorities (Jepson et al., 2021). Enforcement was viewed by delivery partners as being implemented as intended, with 20mph speed limits viewed as initiatives that should be self-enforcing. Active enforcement was often reactive in nature in response to public complaints, and then with a focus on the education of drivers. This is in-line with how other speed limits are generally enforced. However, this contrasted with the perception from the general public of what enforcement should consist of. In both Edinburgh and Belfast, enforcement activities were perceived by the general public as being insufficient, and the view was expressed that enforcement should focus more on the issuing of fines.

Behaviour change and outcomes

In both sites, a range of perceptions were described by the general public about the impact of the initiative on driver behaviour (Cleland et al., under review; Williams et al., under review). Some were positive, such as compliance with the scheme, which affected the overall speed on roads. However, some possible unintended consequences of the initiative were perceived – such as a worsening in driver behaviour, increased overtaking and decreased driver attention (Cleland et al., under review). Views from the general public in both Edinburgh and Belfast would suggest that 20mph speed limit initiatives had not directly led to, or contributed to, decision making about active travel choices, but limited data were available to determine whether the 20mph speed limit initiatives led to any increases in journeys made on foot or by bike.

A range of outcomes were anticipated from the outset including reductions in traffic speed, volume, collisions and the severity of casualties. We assessed the impact of the interventions on these outcomes. A summary of the overall observed change in average traffic speed and volume are shown in Fig. 2, Fig. 3 and collision and casualty data are shown in Fig. 4. In Edinburgh, 12 months after the policy change average speeds on affected roads had reduced by 1.34 mph (95% CI 0.95 to 1.72), but the volume of vehicles did not change significantly. The average 7-day volume of traffic decreased by 87 vehicles (95% CI -112 to 286). Across Edinburgh the number of collisions in a year reduced by 367 (40% reduction) with 409 fewer casualties (39% reduction) (Nightingale et al., under review). These reductions equate to 36% (collisions) and 35% (casualties) when adjusted for secular trends. In Edinburgh, road traffic fatality rates decreased by 23%. Thus, even relatively modest reductions in speed led to significantly improved levels of safety.

Fig. 2

Changes in average vehicular speed, with labels indicating the point estimate of change and the accompanying 95% Confidence Interval.

Fig. 3

Changes in average vehicular volume, with labels indicating the point estimate of change and the accompanying 95% Confidence Interval.

Fig. 4

Changes in road traffic collisions and casualties, as observed percentage reductions.

Average speed fell in Belfast by 0.91 mph (95% CI -3.01 to 1.18) though this was not statistically significant. This could be for a number of reasons for example: a) the average speed before the intervention was already close to 20mph; and b) it aligns with data from local practice that suggests signage only produces an approximate 1mph reduction (City of Edinburgh Council, 2013). The average 7-day volume of traffic within Belfast city centre decreased by 133 vehicles (95% CI -252 to −15), with significant increases in objectively assessed liveability, especially in relation to traffic and transportation. The change in average traffic volume may also be due to new bus lanes, the introduction of a Rapid Transit System, and other urban transport changes that happened in Belfast city centre during the implementation of the 20mph speed limits. There was a 2% significant reduction in collisions and a 6% reduction in casualties in Belfast city centre up to 3 years following the introduction of the 20mph speed limits. Road traffic fatality rates decreased by 44.3%.

A broad range of other potential outcomes or unintended consequences emerged from the research, and particularly from the qualitative findings from WP2. For example, while it was initially envisaged that slower speeds would lead to fewer collisions and reduced severity of injuries, and this is indeed what was observed, there was a perception that reduced speeds had the potential to lead to more collisions; people reported they may lead to reduced attention on the road due to concentrating on the speedometer, the potential for people to check their mobile phone more often when travelling at slower speeds, and more pedestrians walking into the street, which all have the potential to increase collision rates (Cleland et al. under review). Further research is required to better understand driver behaviour (and potentially cyclist and pedestrian behaviour) and the myriad of factors that might explain increases or decreases in collisions and casualties. Future research should, where possible, utilise objective measures to overcome differences between perceptions and real changes in behaviours and outcomes.

Economic evaluation

A range of factors meant that the planned economic evaluation was not possible in its entirety, due to data not being available as expected. For example, no data were available on enforcement or maintenance costs, or changes in active travel or emissions. In addition, the limitations arising from the natural experimental design increased the uncertainty in attributing observed effects to the intervention. Therefore, we adopted a decision theoretic approach, asking the questions: how effective would the intervention have to be for the benefits to exceed the costs and how likely is the level of benefits achieved in the Edinburgh and Belfast schemes to have exceeded that?’

We were able to obtain cost data for the Edinburgh scheme, covering design, supervision and project management, construction, awareness raising, and speed and traffic volumes. Using these data on costs and the available casualty data, we were able to explore the potential monetary value of the benefits of the scheme. The cost of the Edinburgh scheme totalled £2.81m in current prices or £2.76m in real prices, calculated using the retail price index (RPI) with 2016 as the base year. This total does not include enforcement or maintenance costs. No mechanisms were put in place to record time spent on enforcement of speed limits specifically in the 20mph areas because it is part of wider speed monitoring activities, and the council confirmed that maintenance costs would be subsumed within general road maintenance budgets.

The monetary value of reduced casualties and deaths would need to exceed the costs of the intervention (£2.76m in 2016 prices). Department for Transport data on average costs of road traffic accidents classified by casualty severity are shown in Table 1. Given the estimated outcomes in terms of reduced fatalities, casualties and collisions, it is likely that the monetary value of these outcomes exceeds the costs of the scheme under a range of plausible scenarios reflecting the uncertainties surrounding the outcomes observed.

Table 1

Average value of prevention per reported casualty and per reported road accident.

Accident/casualty type	Cost per casualty	£ (2016 prices)
		Cost per accident
Fatal	1,841,315	2,053,814
Serious	206,912	237,527
Slight	15,951	24,911
Average for all severities	59,358	83,893
Damage only	-	2211

These conclusions are cautious at this stage. A full economic evaluation would need to consider:

duration of benefit net of secular downward trends in casualties

time profile and discounting of benefits and costs, given the vast majority of costs are incurred up front whereas the stream of benefits in terms of causalities avoided would likely continue into the future

sensitivity analysis, for example, testing the sensitivity of the conclusions to assumptions made about the proportion of the observed changes in collisions and casualties (and liveability) attributed to the 20mph limits and the duration of benefit

including a wider range of benefits in the analysis if and when data, for example on active travel and emissions, become available.

However, many of these factors would be likely to tip the balance in favour of the economic case for 20mph limits.

Discussion

This research has contributed a number of important findings to the evidence base on 20mph speed limits. In both cities, there was about a twenty-year political history over which momentum for 20mph gathered. Once the decision to implement was made, the interventions were largely delivered as intended, albeit with variation in the dose of delivery across different components, particularly public education and enforcement. In Edinburgh, large reductions in collisions and casualties were observed, which is in line with what we would expect from reductions in speed. Reductions in speed, collisions and casualties were also observed in Belfast. It was not possible to determine whether the interventions had any impact on walking and cycling.

This was a challenging project methodologically. Implementing 20mph speed restrictions requires multiple partners and actions, is not easy to do, and determining its effectiveness in realising public health goals is multiplex. We were researching something that was happening in real time, and in ways that were not under our control. As such, observational and natural experimental methods were employed (Craig, 2008) and we integrated evidence and data drawn from a range of sources including routinely and locally collected quantitative data, and primary collected quantitative and qualitative data. The use of these methods brings challenges to the conclusions that can be drawn from the results, although the approach allowed us to conduct a study with high ecological validity.

While drawing on routinely and locally collected data provided historical trends or baselines from before the start of project, as well as resource savings, it also had several disadvantages; for example, in terms of how and where the data were collected. This was the case for the data on walking and cycling; automatic counter data were incomplete due to machine malfunction, and once obtained it transpired that route user intercept survey data were not collected in areas where and when the new speed limits were implemented, meaning effects of the intervention could not be assessed.

An important limitation with which we grappled, was the presence of multiple confounding interventions and factors in the City of Edinburgh including traffic management, road works, a national doubling of the active travel budget, the weather, and the embedding of a new tram system. In Belfast matters were complicated by bus lanes and a rapid transport system being introduced in the city centre around the same time (www.infrastructure-ni.gov.uk/topics/transport-initiatives/belfast-move). Thus, it is not possible to determine the discrete impact of 20mph versus the role that these other complementary actions played. This is a common challenge when attempting to formulate theories of change for complex interventions operating within complex systems (Ofek 2017; Patton, 2010). However, these are the realities that researchers face when undertaking an assessment of an intervention in real life, and not in highly controlled conditions.

Overall, the approach taken to evaluating these large-scale interventions that may affect public health was pragmatic, cost efficient, and provided a nuanced understanding of key aspects of pre-implementation, implementation and post-implementation processes. While the evaluation faced a number of challenges and has associated limitations, we believe that the strengths outweigh the limitations, and further development of such methods is to be encouraged.

Future research directions

There are a number of health-related outcomes that we were unable to gather data on including the impact of the interventions on walking and cycling levels and noise and air pollution. We were unable to assess the impact of the interventions on inequalities or carry out a full economic evaluation. In addition, we did not look at travel times, which is important in economic transport appraisal. These are important areas for future research.

An important finding from the current research is that what is perceived in local communities and what is observed by researchers are not always consistent. Therefore, future research should, where possible, utilise objective measures to assess behaviours and outcomes. This might include direct observation of walking and cycling behaviour as well as utilising sensor data from vehicles to track changes in driver behaviour.

In this study we were able to use similar evaluation approaches in both cities, which aided comparability; although there were differences in the scale and implementation of the two schemes, meaning they are not directly ‘comparable’. A recommended framework for evaluating speed reduction interventions would help to ensure future studies utilise similar methods and tools, facilitating comparability between 20mph limit interventions and also aiding comparability of the effectiveness of limits versus zones. Furthermore, future studies of this kind would benefit from the recruitment of matched comparison communities.

Conclusions

Innovative study designs, including natural experiments, are important for assessing the impact of ‘real world’ public health interventions in real time. This study involved the evaluation of two natural experiments of city-wide and city centre scale interventions to reduce traffic speed. We adopted a theory-based approach to systematically test assumptions and establish mechanistic pathways to explain the links between intervention components and outcomes. Using multiple methods, this project has enabled a deeper understanding of not only the effects of the intervention but the factors that explain how and why the interventions and the effects did or did not occur. Importantly it has shown that 20mh speed limits, which are cheaper to implement than zones, lead to reductions in speed, collisions and casualties, and are therefore an effective and potentially cost-effective public health intervention.

Author statement

All authors conceived the study design. KM and MPK led the draft manuscript. All authors contributed important intellectual content and approved the final manuscript.

Funding

This research was funded by the National Institute for Health Research (NIHR), grant number 15/82/12.

Author contributions

All authors conceived the evaluation design and contributed to data collection, analysis and interpretation. KM and MPK drafted the manuscript. All authors edited the manuscript for important intellectual content and approved the final version.

Declaration of competing interest

RH is a member of the NIHR Public Health Research (PHR) funding panel that funded the study. All other authors declare that they have no competing interests.