Mitigating the mental health consequences of mass shootings: An in-silico experiment.

Background: There is emerging evidence that mass shootings are associated with adverse mental health outcomes at the community level. Data from other mass-traumatic events examined the effectiveness of usual care (UC), (i.e., psychological first aid approaches without triage), and stepped care (SC) approaches, with triage, in reducing the burden of post-traumatic stress disorder (PTSD) in a community.
Methods: We built an agent-based model of 118,000 people that was demographically comparable to the population of Parkland and Coral Springs, Florida, US. We parametrized the model with data from other traumatic events. Using simulations, we then estimated the community prevalence of PTSD one month following the Stoneman Douglas High School (Florida, US) shooting and reported the potential reach, effectiveness, and cost effectiveness of different what-if treatment scenarios (SC or UC) over a two-year period. Findings: One month following the mass shooting, PTSD prevalence in the community was 11.3% (95% CI: 11.1-11.5%). The reach of SC was 3461 (95% CI: 3573-3736) per 10,000 and the reach of UC was 2457 (95% CI: 2401-2510) per 10,000. SC was superior to UC in reducing PTSD prevalence, yielding, after two years, a risk difference of -0.044 (95% CI, -0.046 to -0.042) and a risk ratio of 0.452 (95% CI, 0.437-0.468). SC was also superior to UC in reducing the persistence of PTSD, yielding, after two years, a risk difference of -0.39 (95% CI, -0.401 to -0.379) and a risk ratio of 0.452 (95% CI, 0.439-0.465). The incremental cost-effectiveness of SC compared to UC was $2718.49 per DALYs saved, and $0.47 per PTSD-free day. Interpretation: This simulation demonstrated the potential benefits of different community-level approaches in mitigating the burden of PTSD following a mass shooting. These results warrant further research on community-based interventions to mitigate the mental health consequences of mass shootings. Funding: None.

Evidence before this study

We searched PubMed for studies published until May 20, 2022, using the terms ((Treatment) OR (“Community Mental Health Services”[Mesh] OR “School Mental Health Services”[Mesh] OR “Mental Health Services”[Mesh])) AND (“Stress Disorders, Post-Traumatic”[Mesh])) AND ((mass shooting) OR (“Gun Violence”[Mesh])). We found 11 articles. The literature highlighted the importance of psychosocial interventions following mass shooting events. Overall, studies focused on services at the individual level, supporting survivors through therapy and psychiatric assessments with the goal of increasing self and community efficacy and encouraging hope and belief that recovery from traumatic events is possible. However, the literature on community interventions for mental health support remains limited.

Added value of this study

In this simulation, we suggested the potential benefits of two community-level approaches— psychological first aid and stepped care—in mitigating the burden of posttraumatic stress disorder following a mass shooting.

Implications of all the available evidence

This study highlights the need for further research—particularly research that focuses on improving the reach of interventions—and real-world application of community-based interventions to address the mental health consequences of mass shootings.

Alt-text: Unlabelled box

Introduction

Mass traumatic events are associated with an increased burden of a wide range of mental disorders including posttraumatic stress disorder (PTSD). These psychological consequences can be experienced beyond those who were physically injured or were in direct physical danger during the mass traumatic event. There is ample evidence that the families and loved ones of those directly affected and their communities all experience an elevated burden of mental illness after mass traumatic events., The literature on the psychological consequences of mass shootings is more limited than for other mass traumatic events. However, there is emerging evidence that mass shootings are associated with adverse mental health outcomes—particularly PTSD—among both survivors and their communities.

Despite the potential increased burden of mental illness, there is often substantial unmet need for mental health services in the aftermath of mass traumatic events. For example, following the September 11, 2001 terrorist attacks only 26.7% of persons with severe psychological symptoms received treatment. Such unmet need frequently persists despite efforts to provide services for the affected populations.,

Psychological first aid interventions are often the first line of mental health services implemented in the aftermath of mass traumatic events. Psychological first aid interventions include providing physical and emotional support as well as practical assistance to address immediate needs following a mass traumatic event. Such interventions aim to reduce distress and improve adaptive functioning—often without triage. People who continue to have psychological symptoms after receiving psychological first aid, especially if they persist for over 2 months, may benefit from approaches of higher intensity such as evidence-based trauma-focused cognitive behavioral therapy (CBT) or pharmacotherapeutic therapy. However, evidence suggests that without triage, persons who require high-intensity services may be underserved under psychological first aid interventions. Moreover, psychological first aid services are often offered to direct survivors—and sometimes their families—not to community members in general. Importantly, many do not receive any form of assistance to begin with, not even psychological first aid.

Given these challenges, stepped care (SC) interventions have emerged as a potential alternative to tackle the mental health consequences of traumatic events and improve the reach of mental health services., SC interventions include screening and triage of services to the appropriate level of mental health care, followed by ongoing systematic re-evaluation. The SC approach has clear utility and has been recommended in the context of disasters. To our knowledge, however, there have been no trials examining the reach and effectiveness of SC interventions following mass shootings.

We developed an in-silico experiment to investigate the potential reach, cost effectiveness, and reduction of burden of PTSD in a community following a mass shooting under various what-if treatment scenarios using different mental health services approaches—SC and usual care (UC). In this simulation study, UC refers to a psychological first aid intervention without triage. Our goal was to assess: (1) The reach of either SC or UC in a two-year period; (2) The predicted reduction in the prevalence of PTSD in the full population for either approach; (3) The predicted proportion of PTSD cases that persist over time for either approach; and (4) The incremental cost-effectiveness of SC compared to UC.

Methods

We used agent-based models (ABMs) to simulate the burden of PTSD in the community following the Stoneman Douglas High School (Parkland) shooting. ABMs allow for the development of counterfactual estimates of treatment effects, when given appropriate inputs. The outcomes measures we selected were guided by the population impact framework to accurately capture the predictive values of the proposed treatment scenarios. This framework defines impact by measuring the proportion of participants who received an intervention—i.e. reach—and by the reduction in the incidence or prevalence among those who received the intervention.

Model structure

We built an agent population that was demographically comparable to the population of Parkland and Coral Springs, Florida. We added Coral Spring to the model to be able to have a sufficiently large population to run our model and estimate results. Coral Spring was particularly relevant as this was the area the shooter was apprehended. Using this agent-based population, we simulated treatment scenarios beginning four weeks after the Parkland shooting and ending two years later. We chose four weeks to distinguish between symptoms of acute distress disorder—which arise and resolve by four weeks after a traumatic event—and PTSD.

Agent population

We initialized a population of 118,000 agents to serve as an approximation of the population of Parkland and Coral Springs, Florida. The demographic (sex, age, and race/ethnicity) parameters for this synthetic population were applied using data derived from the 2011-2015 American Community Survey 5-year population estimates for Parkland and Coral Springs, Florida.

Levels of exposure

We designated within our model three levels of exposure to the shooting: primary, secondary, and tertiary. Primary exposure included persons who were injured or present and in danger of being shot at the site of mass shooting. This level included 900 students and 30 adults based on the average number of persons present each day at Stoneman Douglas High School. Secondary exposure included direct family or friends of those primary affected by the shooting. We built an internal social network to connect agents directly affected with agents assigned as family or close friends. Ultimately, secondary exposure included 4725 agents. Tertiary exposure included persons who were not directly affected by the shooting and did not have family or friends affected by the shooting, but were living in the affected community at large, which included 112,189 agents.

PTSD prevalence and symptoms score estimates

We parameterized PTSD prevalence in the past month for different age groups (14-17, 18-34, 35-64, or ≥65 years) from prior studies that used the diagnostic and statistical manual of mental disorders (DSM-IV) criteria to estimate the initial distribution of PTSD prevalence for each exposure level in the aftermath of a mass shootings or other mass traumatic events.19, 20, 21 We selected 14 years as the lower age limit because that is the average age of students admitted to the Stoneman Douglas High School. We then used the sex ratio of PTSD prevalence from a study reporting on primary exposure data to estimate PTSD prevalence by sex for adults within the secondary and tertiary levels. To test the dependence of our results on variation in community PTSD prevalence, we conducted a sensitivity analysis with a different tertiary PTSD prevalence from a survey that examined the community burden of PTSD following another mass traumatic event. Details of the estimates are provided in Supplement.

Guided by the DSM-IV PTSD checklist as the gold standard, we used data from a prior study to estimate the number of symptoms (range, 0-17) corresponding to the demographic parameters of agents in our simulation. Accordingly, we set the cutoff point to identify a true case of PTSD (probable PTSD diagnosis) in our simulation at six symptoms. We used sensitivity and specificity of 0.80 in our model.

Our model assumed that, among those who did not receive care,14% underwent remission without treatment. This proportion corresponds to the mean from a meta-analytic estimate for remission among survivors of a wide range of traumatic events receiving psychotherapy in trial waitlist control conditions. We chose the mean because assaultive violence-related events are among the more potent types of traumatic events.

Among those who recovered, our model assumed that 15% experienced recurrence of PTSD after symptom resolution. We chose 15% to split the difference between recurrence and relapse rates of up to 34% from samples of mixed traumatic exposure, and studies that report lack of relapse when examining the trajectories of PTSD symptoms following events of mass violence. We also conducted a sensitivity analysis that used the highest reported rate (34%) for recurrence and relapse. Details of the estimates are provided in Supplement. Further, among agents who underwent relapse in our model, PTSD symptoms returned to 80% of the initial symptom levels.

Intervention components

As shown in Figure 1, we assessed the reach, treatment effectiveness, and cost effectiveness based on approaches: SC and UC.

Figure 1

Overview of the agent-based model structure and flow. Figure 1 presents a visual representation of the agent-based model structure and flow.

Usual care

In the model, UC included the provision of Skills for Psychological Recovery (SPR) indiscriminately within the affected community. SPR is an intermediate-level intervention (which follows the design of psychological first aid) that aims to reduce distress and improve coping and functioning. The intervention may also lower the need for formal mental health treatment. In our model, five sessions of SPR reduced an agent's symptoms by 20%. We also conducted sensitivity analyses using a reduction in agent's symptoms by 15% and 25%.

Stepped care

In the model, SC included a triage screening step during initial entry to services. Through SC, agents were assessed for PTSD case status; identified cases were offered CBT and non-cases were offered SPR. Per the International Society for Traumatic Stress studies guidelines, CBT took place for eight sessions in the model. A course of CBT reduced an agent's symptoms by 36%. This percentage is based on the reduction in PTSD symptoms experienced in a three-month period by participants in the CBT arm of a randomized clinical trial. In this model, we allowed for up to eight extra sessions for agents who did not recover after the initial eight sessions.

Treatment seeking behavior

We used prior data that aimed to examine determinants of PTSD following the September 11th, 2001, terrorist attacks to estimate probabilities of enrollment in, and use of, SPR and CBT treatments. These probabilities were estimated for cases and non-cases separately, accounting for age, sex, and race/ethnicity.

Outcome measurement

Reach

As an overall measure, we reported the reach of each treatment scenario. We defined reach as the number of agents who accessed treatment per 10,000 in the affected population.

Treatment Effectiveness

We reported treatment effectiveness using both risk differences and risk ratios. These measures were calculated for increments of three-month periods for the duration of the simulation.

Treatment Cost

Based on prior assessment of in-person delivery costs, we set the cost of CBT at $60 per session. Because we did not have real-world data on average costs for SPR, we made an assumption—based on earlier simulations—that costs per person, per group session, would be $15. To examine how cost fluctuation is linked to cost effectiveness, we conducted a sensitivity analysis raising the costs of CBT to $120 per session and SPR to $30 per person per session.

Incremental cost-effectiveness

Using disability-adjusted life years (DALYs) and PTSD-free days as measurements, we computed the incremental cost-effectiveness ratios (ICER) among initial cases for simulated comparison. DALYs provide a standardized measure for estimating years of healthy living lost due to disability. We derived disability weights from the global burden of disease study, specified as mild (0.03), moderate (0.149), and severe (0.523) disease states.

PTSD-free days are the number of days spent by an agent in non-case status throughout the simulation. Our model takes a healthcare cost perspective and within it, we applied the ICER calculations for a horizon of a ten-year time—accounting for a discounting rate of 3% on effectiveness and future cost. Details of ICER calculations are described in Supplement.

Statistical analysis

Each of the interventions was run for 100-time steps, starting four weeks after the shooting and representing the passage of two years after the mass shooting in total. These ABMs were developed using C++ and implemented using Microsoft Visual Studio 2012 (Microsoft Corp). We ran each model scenario 50 times to account for the stochasticity in the modeling process, with mean statistical measures reported. We then computed the 2.5th and 97.5th percentiles across those 50 simulations. Supplement provides an overview of the design concept and detailed protocol for this study including sub-models, pseudocodes, and a more in-depth focus on the design concepts.

Boston University Institutional Review Board waived approval for this study as the parameters used were based on publicly available literature or derived from anonymized publicly available data.

Role of the funding source

There was no funding source for this study.

Results

As shown in Table 1, our simulated synthetic population closely resembled the population distribution in terms of age, sex, race/ethnicity, and education of the 2011-2015 American Community Survey 5-year population estimates for Parkland and Coral Springs, Florida. Our model estimated that the overall PTSD prevalence in the community following the Parkland shooting was 11.3% (95% CI: 11.1–11.5%). In the sensitivity analysis using different a tertiary prevalence, the community prevalence remained high at 8.7% (95% CI: 8.5–8.6%).

Table 1

Comparison of Broward Country ACS estimates with Model synthetic population.

Demographic group	Broward Country, FL population (2011–2015 ACS Estimates) %(N = 1,843,152)	Model Population %(N = 1,728,265)
Sex
Female	51.452	51.142
Male	48.548	48.858
Age
Under 20 years	24.01	24.38
20 to 44 years	33.15	33.38
45 to 64 years	27.83	27.74
65 and over	15.01	14.50
Race/Ethnicity
White (Non-Hispanic)	40.397	40.074
Hispanic	26.976	26.779
Black (Non-Hispanic)	26.904	27.199
Asian (Non-Hispanic)	3.430	3.446
American Indian and Alaskan Native (Non-Hispanic)	0.169	0.237
Some other race (non-Hispanic)	0.454	0.521
Two or more race (non-Hispanic)	1.670	1.743
Education
Up to 12th grade	12.362	12.644
High School	27.832	28.085
Some College	21.766	21.537
Associate degree	9.620	9.641
Bachelor's degree	18.480	18.442
Graduate degree	9.940	9.651

Reach

The reach of SC was 3461 (95% CI: 3573–3736) per 10,000 and the reach of UC was 2457 (95% CI: 2401–2510) per 10,000. The reach of SC was similarly higher than that of UC in the subsequent sensitivity analyses, as shown in Supplement.

Treatment effectiveness

Figure 2 demonstrates the treatment effectiveness of both SC and UC in reducing the population prevalence of PTSD. SC began yielding a lower PTSD prevalence than UC starting at three months (RD, -0.004; 95% CI,-0.007 to -0.002), with the absolute benefit (risk difference) continuing to gradually improve through the end of the two years (RD, -0.044; 95% CI, -0.046 to -0.042). Relative benefit (risk ratio) of SC was clear at three months (RR, 0.963; 95% CI, 0.941 to 0.986), with continued gains through the end of the two years (RR, 0.452; 95% CI, 0.437 to 0.468). As shown in Supplement, the sensitivity analysis assuming a different tertiary PTSD prevalence yielded comparable results.

Figure 2

Treatment effectiveness of stepped care and usual care in reducing PTSD prevalence among the full population. Figure two shows the effectiveness of stepped care (blue color) and usual care (green color) in reducing population PTSD prevalence over two years (base case model). Sensitivity = 0.80 and specificity = 0.80.

Figure 3 demonstrates the treatment effectiveness of both SC and UC in reducing persistence of PTSD. SC was superior to UC in reducing persistence of PTSD: absolute benefit was clear after three months (RD, -0.042; 95% CI, -0.047 to -0.036) and increasing through the end of the two years (RD, -0.390; 95% CI, -0.401 to -0.379). Relative benefit of SC was clear at 6 months (RR, 0.786; 95% CI, 0.743 to 0.832), with continued gains through the end of the two years (RR, 0.452; 95% CI, 0.439 to 0.465). As shown in Supplement, the sensitivity analysis assuming a different tertiary PTSD prevalence yielded comparable results.

Figure 3

Treatment effectiveness of stepped care and usual care in reducing PTSD prevalence among cases. Figure three shows the effectiveness of stepped care (blue color) and usual care (green color) in reducing PTSD persistence among cases over two years (base case model). Sensitivity = 0.80 and specificity = 0.80.

Cost effectiveness

As shown in Table 2, the total cost of the SC intervention was $4,758,260 and the total cost of UC was $943,207. The ICER for SC compared to UC was $2718.49 per DALYs averted and $0.47 per PTSD-free days. The sensitivity analysis assuming higher costs for both SC and UC reported total costs for SC of $9,510,610 and a total cost of UC of $1,885,500. ICER for SC compared to UC was $5429.32 per DALYs averted and $0.95 per PTSD-free days. As shown in Supplement, other sensitivity analyses showed comparable results.

Table 2

Cost-effectiveness with a discounting rate of 3%.

Parameters	Mean cost/Person ($)	Total Cost ($)	DALYS avoided	PTSD-Free Days	ICER($ per DALY averted)	ICER($ per PTSD-Free Day)
Scenario 1: Base Case(CBT cost $60/Session, SPR Cost = $15/Session)
Usual Care	70.93	943,207	3368.07	28,612,300	-	-
Stepped Care	360.58	4,758,260	4771.44	36,665,300	2718.49	0.47
Scenario 2: Sensitivity Analysis with higher costs(CBT cost $120/Session, SPR Cost = $30/Session)
Usual Care	141.85	1,885,500	3370.38	28,594,900	-	-
Stepped Care	720.98	9,510,610	4774.82	36,633,500	5429.32	0.95

The simulation begins 4 weeks following the mass shooting. ICER: incremental cost effectiveness ratios.

Discussion

In this paper, using ABMs assessing the efficacy of SC or UC approaches in the aftermath of the Parkland mass shooting, we demonstrated the potential benefits of different community-level approaches in mitigating the burden of PTSD following a mass shooting. SC was associated with greater reach and higher effectiveness in reducing the prevalence of PTSD in the population. Overall, SC was costlier than UC. The ICER for SC per DALYs saved ($2718.49), and per PTSD-free day ($0.47) were, nonetheless, well below the acceptable cost-benefit value for interventions in the United States (i.e., <$50,000). Sensitivity analyses reinforced these findings, showcasing a range of comparable results when key parameters were changed.

Our findings are consistent with prior studies examining the effectiveness of adopting an SC approach to address the burden of mental health disorders in populations in the aftermath of mass traumatic events. For example, similar to our results of the reach of SC, an effectiveness trial conducted in the Military Health System found that participants getting a variation of SC received significantly more mental health services than participants who received UC following a mass shooting. A simulation study studying individuals with PTSD in the aftermath of a hurricane found that SC is associated with greater reach and higher effectiveness than UC.

In terms of cost effectiveness, one study, in which SC was superior to UC in the treatment of multiple anxiety disorders, reported an ICER of about $4.00 per anxiety-free days. Another study of a collaborative care intervention, which included SC elements, reported an ICER of $8.40 per anxiety-free days. The higher costs found in these studies, compared to the costs from on our findings, may be explained by the difference in settings; these studies were conducted in a hospital/primary care setting compared to our community-based setting. Moreover, these studies included pharmacologic interventions, which are often less cost-effective than psychotherapy.

Our study should be considered in light of its limitations. First, and importantly, this was a simulation. As such, the findings are based on the design and assumptions of our model, including the need to expand the modeling beyond Parkland, the area surrounding the Stoneman Douglas High School, to include Coral Springs to build the model population. Moreover, these assumptions were based on the results of several epidemiological studies; each of the studies have its own limitations. However, the parameters we used to initialize our model have many strengths and afford confidence in the estimates provided by our simulation. Further, we see this simulation as an essential, formative work, paving the way for real-world trials. Our study adds to the evidence base that shifts equipoise before such trials can be implemented in human populations. Second, the treatment scenarios in our model do not account for whether different levels of exposure may require different levels of treatment due to the paucity of literature available to parametrize these assumptions. Third, while strength of social supports can have an effect on the PTSD burden in the community, we did not account for such effect in our model. Fourth, our cost-effectiveness estimates do not account for associated administrative costs and the complex array of consequential, indirect, costs on society and the economy. Moreover, the costs of SPR are largely dependent on assumptions. However, the sensitivity analyses we conducted to check the validity of our model further support our main results, including a second cost-effectiveness analysis with higher costs to allay concerns of cost underestimation. Finally, the model presumes that psychological first aid is implemented on a widespread basis in communities following mass violence, which is rarely if ever the case. Likewise, skills for psychological recovery is not the current standard of care in mass violence communities. Therefore, our assumptions likely overestimate the usual care that individuals actually receive.

Notwithstanding these limitations, these agent-based simulation models show the potential benefits of using community-based interventions for the early treatment of PTSD following a mass shooting. Moreover, our simulation shows that SC yields estimates well within the range of acceptable cost-effectiveness figures in the United States. Overall, to the best of our knowledge, this simulation represents the best available evidence supporting the use of community level interventions following mass shootings. These results warrant further research—particularly research that focuses on improving the reach—and real-world application of community-based interventions to mitigate the mental health consequences of mass shootings.

Contributors

S.M.A., G.H.C., and S.G. contributed to the conception and design of the study. S.M.A. and G.H.C. conducted the literature review for the model parameters. S.T. conducted the data analysis and modeling and S.M.A. and G.H.C. led the data interpretation. S.M.A. wrote the first draft of the manuscript with input from G.H.C. and S.G. All authors contributed to critical revision of the manuscript. S.M.A., G.H.C., and S.T. have accessed and verified the data. All authors approved the manuscript and were responsible for the decision to submit the manuscript.

Data sharing statement

All the relevant data are available in the manuscript and supplementary material.

Declaration of interests

All authors received funding as part of a grant from the U.S. Department of Justice to study the mental health consequences of mass traumatic event that ended in September 2020. However, that funding is not related to this work. L.S also receives unrelated funding from the National Institutes of Health, U.S. Department of Defense, and U.S. Department of Justice.