Computer adaptive testing to assess impairing behavioral health problems in emergency department patients with somatic complaints.

Objectives: To assess: (1) the prevalence of mental health and substance use in patients presenting to the emergency department (ED) through use of a computer adaptive test (CAT-MH), (2) the correlation among CAT-MH scores and self- and clinician-reported assessments, and (3) the association between CAT-MH scores and ED utilization in the year prior and 30 days after enrollment.
Methods: This was a single-center observational study of adult patients presenting to the ED for somatic complaints (97%) from May 2019 to March 2020. The main outcomes were computer-adaptive-assessed domains of suicidality, depression, anxiety, post-traumatic stress disorder (PTSD), and substance use. We conducted Pearson correlations and logistic regression for objectives 2 and 3, respectively.
Results: From a sample of 794 patients, the proportion of those at moderate/severe risk was: 24.1% (suicidality), 8.3% (depression), 16.5% (anxiety), 12.3% (PTSD), and 20.4% (substance use). CAT-MH domains were highly correlated with self-report assessments (r = 0.49-0.79). Individuals who had 2 or more ED visits in the prior year had 62% increased odds of being in the intermediate-high suicide risk category (odds ratio [OR], 1.62; 95% confidence interval [CI], 1.07-2.44) compared to those with zero prior ED visits. Individuals who scored in the intermediate-high-suicide risk group had 63% greater odds of an ED visit within 30 days after enrollment compared to those who scored as low risk (OR, 1.63; 95% CI, 1.09, 2.44).
Conclusion: The CAT-MH documented that a considerable proportion of ED patients presenting for somatic problems had mental health conditions, even if mild. Mental health problems were also associated with ED utilization.

INTRODUCTION

Background

According to the 2019 National Hospital Ambulatory Medical Care Survey, there are approximately 150 million visits to US emergency departments (ED) annually. Visits for psychiatric or substance abuse account for approximately 10% of these visits as of 2016. Importantly, this represents a dramatic rise (approximately 30%) in the number of visits primarily for mental health complaints over the past 2 decades, which has outpaced the rise in overall ED visitation. , , , , , , , ED service delivery costs associated with the most prevalent mental health and substance use diagnoses are in excess of $3.5 billion, with anxiety and depression accounting for $1.7 billion annually. The severity of mental health symptomology is positively correlated with rate of ED use. , However, mental health problems may be missed during ED encounters. In fact, 45% of patients who are seen in the ED for non‐psychiatric primary complaints have comorbid psychiatric disorders, including major depression, general anxiety, substance abuse, and suicidal ideation. , , The ED offers the opportunity to screen for mental health problems.

Importance

The barriers to standard and truly universal mental health screening in the ED are many and include the competing priorities of ED physicians delivering care to critically ill patients in busy and crowded ED environments, limited access to adequate mental health resources for consultation and outpatient care, as well as the training and attitudes of ED clinicians. , These same barriers have limited the approach to widespread mental health screening in the ED and represent a major missed opportunity to provide and/or triage to appropriate care as a significant proportion of ED patients presenting with non‐psychiatric complaints have comorbid mental health issues as noted above. It is noteworthy that many patients who die by suicide often visit the ED or access the healthcare system shortly before their attempt for a non‐psychiatric reason. , Psychiatric screening and subsequent mental health care triage in the ED may have downstream alleviation on the healthcare system.

Goals of this investigation

In this study, we build and expand on the work of Beiser et al who demonstrated the feasibility of using the brief computer adaptive diagnostic screen (the CAD‐MDD ) and depression severity measurement test (CAT‐DI ) for depression assessment in a broad sample of ED patients with non‐psychiatric primary complaints. Whereas the previous study evaluated 1 domain, our study evaluates 5 domains. Our objectives in this study were 3‐fold. First, we aimed to document the prevalence of suicidality, depression, anxiety, post‐traumatic stress disorder (PTSD), and substance abuse in an ED population presenting primarily for somatic complaints. Second, we compared the results of the computer adaptive test‐mental health (CAT‐MH) standardized screening to standard practice ED assessments for suicide, clinician‐reported rating of suspected suicide attempt in the 30 days post‐enrollment, and other validated self‐report screening tools for anxiety, depression, somatic symptoms, and pain. Finally, we examined the association between ED utilization and mental health problems among ED patients.

METHODS

Study design and setting

This was a single‐center prospective observational sample of randomly selected non‐consecutive patients presenting to a 90,000 annual visit Midwestern ED over the course of 10 consecutive months. This study was approved by the Indiana University School of Medicine institutional review board (protocol no. 1904527368), and all participants gave informed consent. Patient enrollment began on May 24, 2019, and ceased on March 10, 2020 with the rapid increase of coronavirus disease 2019 (COVID‐19) cases and resulting stay‐at‐home orders.

Selection of participants

Trained research personnel used the ED tracking system within the electronic medical record (EMR) system (Cerner Firstnet) to screen for potentially eligible subjects presenting to the ED primarily during weekday daytime hours. To reduce sampling bias, research personnel began each 4‐ to 6‐hour screening shift by selecting a random digit between 0 and 9 using an online random number generator. All subjects regardless of triage status, method of arrival, or ED location (except dedicated psychiatric observation rooms) who were on the ED census at that time with the last digit of their age matching the selected number were eligible for screening. Each potentially eligible patient was screened for enrollment sequentially provided they remained on the ED census. This random selection process was repeated each time the list of potentially eligible subjects was exhausted. The treating clinicianss for screened subjects were approached to confirm eligibility criteria prior to patient approach. Exclusion criteria for the study included: (1) age younger than 18 years; (2) being under involuntary detention; (3) lack of or impaired decisional capacity (eg, active psychosis, dementia, developmental delay, intoxication); (4) hemodynamic instability according to the treating clinician; (5) non‐English speaking; and (6) those that may encounter issues with utilization review (eg, prisoner, living out of the state). Therefore, the cross‐section of included patients were ED subjects without overt barriers to providing consent, the majority of whom were presenting with chief somatic complaints. Subjects meeting all enrollment criteria were approached in their private or semi‐private care area for informed consent unless it would interfere with active patient care, or they were no longer on the ED census. The informed consent stated the purpose of the study was to use a “tool called the CAT‐MH to check for a variety of psychological conditions (including depression, anxiety, post‐traumatic stress disorder [PTSD], etc) in patients who need the services of the emergency department…to develop and allot appropriate resources for our patients.” Participants were not compensated for participation.

Additional data sources

ED utilization data was obtained through the EMR and the Indiana Network for Patient Care managed by the Indiana Health Information Exchange, the nation's largest inter‐organizational clinical data repository, to collect data regarding disposition (admit or discharge) for the index visit, discharge diagnoses, and ED utilization in the 12 months before and after enrollment. Note that, first, time‐to‐event analyses were not possible given that ED utilization data were in the form of total number of visits, rather than specific dates. Second, although we intended to examine ED utilization 12 months before and after enrollment, we limited our analyses to 12 months prior and 30 days after enrollment. Research has demonstrated that the COVID‐19 pandemic greatly affected health care use, particularly in EDs, which would have systematically biased our long‐term follow‐up data.

The Bottom Line

Mental health and substance abuse issues are of significant concern in emergency departments (ED), but current screening practices are imperfect. This single‐center observational study tested the use of a computer‐adaptive test to assess for mental health and substance abuse problems in patients presenting for somatic complaints. In this study, the test was able to identify patients that were at risk for suicidality, depression, anxiety, and substance abuse. Patients who were identified as at risk were more likely to have had past visits to the ED (odds ratio [OR], 1.62) and were more likely to have subsequent visits to the ED (OR, 1.63).

Variables

Computer adaptive testing with the CAT‐MH

The CAT‐MH , is a suite of mental health computerized adaptive tests (CATs), that can be self‐administered via any internet capable device (computer, tablet, smartphone), either in the clinic, ED, or remotely via a secure patient portal. The CAT‐MH is unique in that unlike traditional CATs developed for educational assessments that are based on unidimensional item response theory (IRT) and are only appropriate for simple constructs such as mathematical ability, the CAT‐MH is based on multidimensional item response theory and is suitable for the measurement of complex constructs such as depression, anxiety, and suicidality. For a given construct (eg, depression) the CAT‐MH draws symptom items from a large bank of potential items (eg, 389 items for depression) that completely covers the latent variable of interest from the lowest to the highest levels of severity. The CAT begins by selecting an item from the middle of the severity continuum. Based on the item response, typically on a 5‐point Likert scale, a provisional severity score and its uncertainty are computed. Based on that score, the next maximally informative item is selected and administered. Based on the response to the second item, the severity score and uncertainty are recomputed. The process continues until the uncertainty in the estimated score falls below 5 points on a 100‐point scale.

All of the CAT‐MH scales have been validated against structured clinical interview and we included CATs for suicidality (CAT‐SS ), depression (CAT‐DI ), anxiety (CAT‐ANX ), PTSD (CAT‐PTSD—an adaptive version of the 20‐item PCL‐5 ), and substance use disorder (CAT‐SUD ) in our study. See Appendix S1 for additional details. The CAT‐SUD also determines frequency of use of alcohol, sedatives/hypnotics, opioid analgesics, heroin/methadone, and cocaine/amphetamines. The CAT‐SS has been prospectively validated in terms of its ability to predict future suicidal behavior. , , Approximately 10 items are administered for the suicide subscale, 12 for depression, 12 for anxiety, 6 for PTSD, and 11 for substance use. The CAT‐SS, CAT‐DI, and CAT‐SUD have been previously implemented in EDs.

Standard ED suicide screening

During the study period, standard universal suicide screening took place at the enrollment site ED for individuals not presenting with a chief complaint of suicidal ideation. Please see Appendix S2 for the standard screening administered by nursing personnel through an electronic medical record workflow in a private care area, consisting of questions closely aligned with the Columbia‐Suicide Severity Rating Scale (C‐SSRS). Per ED protocol, treating clinicians were informed of anyone who screened positive for suicidal ideation using this tool. For the purposes of this study, these questions will be referred to as the “standard ED suicide screen.”

Additionally, before patient enrollment and for the purposes of this study, the clinician (either attending physician [17%], resident [65%], or advanced practice provider [18%]) was asked to quantify their concern for a future suicide attempt by marking a 100‐mm visual analog scale (VAS) in response to the question: “What is your level of suspicion that this patient will attempt suicide in the next 30 days?” Additionally, the CAT‐MH, as described abo, also screened for suicide risk.

Non‐adaptive self‐administered screening tools

Socio‐demographic data were collected through patient self‐report. Self‐reported comorbidity/past medical history was assessed by asking the participant to indicate if a doctor or another health care worker had diagnosed or treated the participant for 13 medical items in the past 3 years (see Table 1). Chief complaint was transcribed directly from that location in the EMR.

TABLE 1

Frequency distribution of intake variables for entire sample and sample screening intermediate‐high suicide risk on CAT‐MH

	Individuals consenting for CAT‐MH (N = 794)	Individuals screened intermediate‐high CAT‐MH suicide risk (N = 191)
Demographic information	No. (%) a	No. (%) b
Gender
Female	492 (62.0)	125 (65.5)
Male	301 (37.9)	66 (34.6)
Other	1 (0.1)	0 (0)
Ethnicity
Hispanic or Latino	42 (5.3)	11 (5.8)
Not Hispanic or Latino	725 (91.3)	174 (91.1)
Unknown or not reported	27 (3.4)	6 (3.1)
Race
White	413 (52.0)	105 (55.0)
Black or African American	316 (39.8)	65 (34.0)
Asian	3 (0.4)	0 (0)
American Indian/Alaska Native	6 (0.8)	2 (1.1)
Native Hawaiian or other Pacific Islander	0 (0)	0 (0)
More than 1 race	36 (4.5)	15 (7.9)
Other	20 (2.5)	4 (2.1)
Marital status
Single	422 (53.2)	101 (52.9)
Married	224 (28.2)	38 (19.9)
Separated	21 (2.6)	9 (4.7)
Divorced	91 (11.5)	31 (16.2)
Widowed	36 (4.5)	12 (6.3)
Education
Some high school	126 (15.9)	43 (22.5)
High school	230 (29.0)	39 (20.4)
GED	42 (5.3)	12 (6.3)
Some college	210 (26.5)	51 (26.7)
Graduate college	149 (18.8)	38 (19.9)
Graduate/professional school	37 (4.7)	8 (4.2)
Employment
Employed full‐time	318 (40.1)	63 (33.0)
Employed part‐time	86 (10.8)	26 (13.6)
Student and working	19 (2.4)	3 (1.6)
Student and not working	19 (2.4)	4 (2.1)
Homemaker	19 (2.4)	7 (3.7)
Unemployed	99 (12.5)	28 (14.7)
Retired	80 (10.1)	10 (5.2)
Disabled	154 (19.4)	50 (26.2)
Chief complaint
Abdominal pain	167 (21.0)	45 (23.6)
Abnormal lab/test results	3 (0.4)	2 (1.1)
Abnormal vital signs (heart rate, blood pressure, etc)	14 (1.8)	1 (0.5)
Allergic reaction	3 (0.4)	1 (0.5)
Anxiety	3 (0.4)	2 (1.1)
Chest pain	91 (11.5)	17 (8.9)
Ear, nose, and throat complaint	28 (3.5)	6 (3.1)
Eye complaint	7 (0.9)	0 (0)
Facial pain	8 (1.0)	2 (1.1)
Fever/chills/sepsis	12 (1.5)	0 (0)
Other musculoskeletal pain (back, extremity, etc)	105 (13.2)	25 (13.1)
Gastrointestinal problem (vomiting, diarrhea, etc)	51 (6.4)	14 (7.3)
Genitourinary problem (kidney complaints, etc)	14 (1.8)	4 (2.1)
Headache	26 (3.3)	5 (2.6)
Medical screen/evaluation	10 (1.3)	3 (1.6)
Neurological complaint (seizure, focal weakness, etc)	18 (2.3)	4 (2.1)
Obstetric‐gynecological complaint	21 (2.6)	7 (3.7)
Post‐operative complication	4 (0.5)	2 (1.1)
Respiratory problem (cough, asthma, shortness of breath)	78 (9.8)	15 (7.9)
Sickle cell crisis	11 (1.4)	1 (0.5)
Skin/soft tissue infection or complaint	22 (2.8)	6 (3.1)
Substance use	1 (0.1)	1 (0.5)
Trauma (motor vehicle accident, fall, assault, etc)	34 (4.3)	5 (2.6)
Weakness/fatigue and dizziness	44 (5.5)	14 (7.3)
Miscellaneous	16 (2.0)	8 (4.2)
Invalid	2 (0.3)	1 (0.5)
Missing	1 (0.1)	0 (0)
Health history information
Currently have a primary care physician	580 (73.0)	136 (71.2)
History of:
Asthma, emphysema, chronic bronchitis	251 (31.6)	75 (39.3)
High blood pressure or hypertension	341 (42.9)	82 (42.9)
High blood sugar or diabetes	175 (22.0)	46 (24.1)
Arthritis or rheumatism	226 (28.3)	63 (33.3)
Heart attack, angina, heart failure, or other types of heart disease	140 (17.6)	36 (18.9)
Stroke, seizures, Parkinson's disease, or another neurological condition	102 (12.8)	40 (20.9)
Liver disease	40 (5.0)	12 (6.3)
Kidney or renal disease	82 (10.3)	27 (14.4)
Cancer diagnosed or treated in the last 3 years	58 (7.3)	12 (6.3)
Anxiety	323 (40.7)	136 (71.2)
Depression	307 (38.7)	141 (73.8)
PTSD	129 (16.2)	67 (35.1)
Suicidal thoughts or attempt	58 (7.3)	45 (23.6)
Current use of:
Tobacco	269 (33.9)	88 (46.1)
Alcohol ≥4 times/week	99 (12.5)	28 (14.7)
Marijuana	163 (20.5)	73 (38.2)
Cocaine	13 (1.6)	5 (2.6)
Heroin	3 (0.4)	1 (0.5)
Other	13 (1.6)	7 (3.7)

Abbreviation: VAS, visual analog scale.

Based on 794 unique individuals. Rounded to nearest tenth; may not equal 100%.

Based on 191 individuals.

Based on 699 unique individuals.

Based on 166 individuals.

Proportion of individuals in the consenting sample (n = 794) who were rated as “0” suspected suicide attempt risk was 43.45%. Proportion among intermediate‐high risk suicide individuals (n = 191) was 37.70%.

Proportion of individuals in the consenting sample (n = 794) who were rated as “0” suspected suicide attempt risk was 78.46%. Proportion among intermediate‐high risk suicide individuals (n = 191) was 70.16%.

Subsequently, each participant underwent self‐administered screening in 4 domains via a secure tablet. (1) The Generalized Anxiety Disorder 7‐item scale (GAD‐7) is a rapid screening tool for the presence of clinically significant anxiety. , , , The scores range from 0 to 21, with cut points of 5, 10, and 15 corresponding to mild, moderate, and severe anxiety, respectively. (2) The Patient Health Questionnaire 8 item scale (PHQ‐8) is a rapid screening tool for the presence of clinically significant depression. , , , , , , The scores range from 0 to 24, with cut points of 5, 10, 15, and 20 corresponding to mild, moderate, moderately severe, and severe depression, respectively. We used the PHQ‐8 instead of the PHQ‐9 due to suicidality assessment via the C‐SSRS items and little difference in sensitivity and specificity between the PHQ‐8 and 9. (3) The Patient Health Questionnaire‐15 (PHQ‐15) is used to detect participants at risk for somatoform disorders; , scores range from 0 to 30 it has cutoffs of 5, 10, and 15, corresponding to low, medium, and high somatic symptom severity. (4) The Pain Catastrophizing Score (PCS) is a 13‐item self‐administered tool used to quantify a person's thoughts and feelings when they have experienced pain. , The maximum total score is 52 and total scores ≥30 (corresponding to the 75th percentile) are considered clinically significant. In a sample of chronic pain patients who scored ≥30, 70% identified as occupationally disabled, and 66% had moderate depression. The PCS measures rumination and helplessness, which are related to mental health disorders. ,

Statistical methods

For objective 1, we documented the percentage of patients endorsing different categories (eg, normal, mild, moderate or intermediate, severe, or high) of the 5 domains assessed by the CAT‐MH. The categories were pre‐established in the original articles based on thresholds producing high sensitivity for the underlying diagnosis (eg, SCID for DSM‐5) for mild, moderate sensitivity and specificity for moderate, and high specificity for severe). We present this for all the patients, as well as a subgroup of patients who scored in the intermediate to high categories of suicidal risk.

For objective 2, we correlated the CAT‐MH scores with clinician‐reported suspected risk of suicide attempt and self‐reported measures (GAD‐7, PHQ‐8, PHQ‐15, and PCS). We then compared the identification of patients at risk for suicide using the CAT‐MH with the standard ED suicide screen procedure.

For objective 3, we conducted logistic regression analyses between the number of ED visits in the year prior to enrollment and CAT‐MH subscales. The predictors, number of ED visits within the prior year, were dummy coded into 3 categories: (1) 0 visits in the year prior, which served as the reference category; (2) 1 visit prior; and (3) 2 or more visits prior. We analyzed each CAT‐MH subscale separately as a binary outcome, dichotomized suicide and substance use into low risk (0) and intermediate‐high risk (1), depression and anxiety into normal/mild (0) and moderate/severe (1), and PTSD into no evidence (0) and possible/highly likely (1). For depression and anxiety, we included mild symptomatology into the reference (0) group as research has utilized scores ≥10 (moderate) as cut‐off points for a diagnosis of generalized anxiety disorder or major depressive episode. , , , , , , , , , We included the following demographic covariates: age, gender, ethnicity, race, marital status, and employment.

To examine the association between the CAT‐MH subscales and subsequent ED visits within 30 days of ED enrollment date, we conducted logistic regression analyses adjusting for demographics. The predictors were each of the categorical risk levels for the CAT‐MH subscales, which we dummy coded such that the lowest level of risk served as the reference category. Given the reduced sample size in the high‐risk categories, we combined all the categories above low risk. For depression and anxiety subscales, mild was also included in the reference category. We dichotomized the outcome into not present (0) or any ED visit (1) within 30 days. All analyses were 2‐tailed at an alpha‐level of 0.01.

Sensitivity analyses

We conducted 3 sensitivity analyses to examine the potential impact of analytical decisions (ie, categorization of subscales, dichotomization of ED visit recidivism) on the results and interpretation. First, we predicted the continuous CAT‐MH subscales from ED visits in the year prior to enrollment. We included an unstandardized and standardized outcome to examine the magnitude of the effects in terms of a single point increase on the 100‐point scale and in terms of standard deviation units, respectively. Second, when assessing the associations with mental health problems with subsequent ED visits, we utilized continuous, standardized CAT‐MH subscales rather than utilizing categorical CAT‐MH predictors. Third, we conducted ordinal logistic regression to predict ED visits within 30 days after enrollment to examine the association with each CAT‐MH category.

RESULTS

Patient enrollment

In the study, 1854 potentially eligible patients were screened and approached for enrollment. Of those subjects who were approached for enrollment in the study, 828 (44.7%) underwent informed consent and were enrolled. Of those who consented, 97.3% (n = 806) completed the CAT‐MH. Reasons for lack of completion in the remaining 22 subjects included study withdrawal, and excessive interruptions due to clinical care. To conduct a complete case analysis, we dropped an additional 12 individuals with the following missing indicators: demographic variables, use of ED care within 30 days post‐discharge, and the PTSD subscale on the CAT‐MH. We also dropped 1 individual with an extreme outlier of ED care (ie, 75 visits) within 30 days post‐discharge and 2 individuals with invalid ages. The final analytic sample consisted of 794 individuals. See Appendix S3 for the data flowchart. The median time for CAT‐MH completion for the 5 domains was 9.7 minutes (interquartile range [IQR], 5.6 minutes) and mean was 10.8 minutes (SD, 5.5 minutes).

Sample demographics

Table 1 presents the overall breakdown of our population including frequency distribution of demographic, chief complaint (97% somatic), medical history, suicide, admission, and self‐questionnaire (past medical history, GAD‐7, PHQ‐8, PHQ‐15, PCS) of the subjects included in the sample. Additionally, this information is also provided for the subgroup of patients who screened intermediate‐high‐risk on the CAT‐MH suicide severity rating (n = 191) and is presented in the right‐hand column in Table 1. The frequencies of demographic, chief compliant, and medical information were similarly distributed for individuals who screened intermediate‐high suicide risk as for the entire sample. Individuals who screened intermediate‐high suicide risk had a higher proportion of those who endorsed history of PTSD or suicidal ideation, as well as a higher mean of prior ED visits within the past year, as compared to the entire sample.

Objective 1

Table 2 presents the frequency distribution of the CAT‐MH subscales defined both categorically and continuously. The continuous subscales demonstrated acceptable skewness and kurtosis. The proportion of those who scored above the threshold of low risk/no evidence group for the following subscales were: (1) 24.1% for suicide, (2) 12.3% for PTSD, and (3) 20.4% for substance use. The proportion of those who scored above the threshold of normal/mild risk group for the following subscales were: (1) 8.3% for depression, and (2) 16.5% for anxiety. Across all categories, approximately 31.9% (n = 253) of the sample screened above the threshold for at least one CAT‐MH subscale and 20.7% (n = 164) for 2 or more CAT‐MH subscales. Approximately one‐third of the sample population (31.9%) had at least 1 high CAT‐MH subscale and 20% of the sample had at least 2 high subscales scores. For example, 16.5% and 8.3% screened positive for moderate/severe anxiety and depression, respectively. The prevalence of moderate or severe depression in this population (8.3%) by CAT‐MH screening is similar to the prevalence (7%) reported in another ED sample using the CAT‐MH. See Appendix S4 for the frequency distribution of use and abuse of specific substances.

TABLE 2

Frequency distribution of CAT‐MHTM variables for entire sample and those who screened intermediate‐high suicide risk on CAT‐MH

CAT‐MH categorical subscales	Individuals consenting for CAT‐MH a	Individuals screened intermediate‐high CAT‐MH suicide risk b
	No. (%)	No. (%)
Depression subscale
Normal	433 (54.5)	9 (4.7)
Mild	295 (37.2)	120 (62.8)
Moderate	35 (4.4)	32 (16.8)
Severe	31 (3.9)	30 (15.7)
Anxiety subscale
Normal	537 (67.6)	28 (14.7)
Mild	126 (15.9)	52 (27.2)
Moderate	69 (8.7)	54 (28.3)
Severe	62 (7.8)	57 (29.8)
PTSD subscale
No evidence	696 (87.7)	104 (54.5)
Possible	74 (9.3)	63 (33.0)
Highly likely	24 (3.0)	24 (12.6)
Substance use subscale
Low risk	632 (79.6)	73 (38.2)
Intermediate risk	143 (18.0)	106 (55.5)
High risk	19 (2.4)	12 (6.3)

Note: Rounded to the nearest tenth, may not equal 100%.

Based on 794 unique individuals who completed the CAT‐MH study protocol.

Based on 191 unique individuals who screened in the intermediate or high‐risk group on suicide severity subscale on the CAT‐MH.

Objective 2

To examine the overlap between suicide screens, Table 3 presents the cross‐tabulation between the CAT‐MH positive suicide screen (ie, those in the high‐risk suicide severity category) and the standard ED's suicide screen. Ten individuals screened positive on the CAT‐MH suicide screen but only 3 of these screened positive by the standard ED suicide screen. Thus, 7 individuals were not identified via the standard screening. All 7 identified participants were in the high‐risk category on the CAT‐MH suicide subscale. The kappa between the suicide screens is 0.46 (95% confidence interval [CI], 0.13–0.79). Table 4 presents the Pearson correlations between the CAT‐MH subscales and the clinician‐rated suspected risk of suicide attempt rating and self‐reported questionnaires. Each CAT‐MH subscale, including suicidality, was weakly correlated with the clinician rating of suspected suicide attempt, and was strongly correlated with the self‐reported questionnaires indexing of anxiety (GAD‐7), depression (PHQ‐8), pain catastrophizing (PCS), and physical symptoms (PHQ‐15).

TABLE 3

Cross‐tabulation between suicide screen in ED and CAT‐MH

	CAT‐MH suicide screen (No. %)
	Negative	Positive a
ED standard care suicide screening
Positive	0 (0)	3 (0.38)
Negative	760 (95.72)	7 (1.01)
Not recorded	23 (2.90)	0 (0)

Note: Derived from 794 unique individuals. CAT‐MH suicide screen based on suicide severity.

Individuals who screened positive were in the high‐risk suicide severity group.

TABLE 4

Pearson correlation between CAT‐MH subscales, clinician‐rated concern of suicide attempt, and other self‐report measures

CAT‐MH Subscale	Clinician rating of suspected suicide attempt in next 30 days a	Sum of anxiety (GAD‐7) r a	Sum of depression (PHQ‐8) r a	Sum of somatic symptoms (PHQ‐15) r b	Sum of pain catastrophizing (PCS) r a
Suicide subscale	0.11	0.73	0.79	0.50	0.63
Depression subscale	0.14	0.72	0.78	0.53	0.63
Anxiety subscale	0.12	0.76	0.74	0.53	0.62
PTSD subscale	0.12	0.73	0.70	0.49	0.57
Substance use subscale	0.09	0.62	0.66	0.49	0.54

All P values <0.01.

Based on 794 unique individuals.

Based on 699 unique individuals.

Objective 3

Table 5 presents the prediction of dichotomous CAT‐MH subscales from the number of ED visits in the year prior to enrollment. Individuals who had 2 or more ED visits in the prior year had a 62% increase in the odds of being in the intermediate‐high suicide risk category (OR, 1.62; 95% CI, 1.07–2.44) compared to those with 0 prior ED visits. Odds ratios for the other CAT‐MH subscales were the following: depression (OR, 1.55 [0.81–2.94]), anxiety (OR, 1.56 [0.96–2.52]), PTSD (OR, 2.86 [1.63–5.02]), and substance use disorder (OR, 1.49 [0.96–2.31]). Individuals with 1 visit to the ED in the prior year were not more likely to score higher on any of the CAT‐MH subscales compared to those with 0 visits. Appendix S5 presents the point estimates for the covariates whereas Appendix S6 provides the data using log‐linked general linear regression with combined categorical predictors.

TABLE 5

Prediction of CAT‐MH from prior year ED visits using logistic regression

Visits ED year prior	Suicide subscale, OR (95% CI; P value)	Depression subscale, OR (95% CI; P value)	Anxiety subscale, OR (95% CI; P value)	PTSD subscale, OR (95% CI; P value)	Substance use subscale, OR (95% CI; P value)
0	REF	REF	REF	REF	REF
1	0.86 (0.53–1.41; P = 0.56)	1.02 (0.47–2.21; P = 0.96)	1.06 (0.60–1.86; P = 0.84)	1.64 (0.85–3.14; P = 0.14)	1.11 (0.67–1.85; P = 0.69)
≥2	1.62 (1.07–2.44; P = 0.03)	1.55 (0.81–2.94; P = 0.18)	1.56 (0.96–2.52; P = 0.07)	2.86 (1.63–5.02; P < 0.001)	1.49 (0.96–2.31; P = 0.08)

Note: Derived from 794 unique individuals. Demographic variables included are age, gender, ethnicity, race, marital status, and employment. Each CAT‐MH subscale was dichotomized: suicide subscale 0 = low risk, 1 = intermediate‐high risk; depression subscale 0 = normal‐mild; 1 = moderate‐severe risk; anxiety subscale 0 = normal‐mild; 1 = moderate‐severe; PTSD subscale 0 = no evidence, 1 = possible‐highly likely; substance use subscale 0 = low risk, 1 = intermediate‐high risk.

Table 6 presents the prediction of dichotomous ED visits (not present/any) within 30 days from CAT‐MH subscales. Individuals who scored in the intermediate‐high‐suicide risk group had 63% greater odds of an ED visit within 30 days after their enrollment compared to those who scored as low risk (OR, 1.63; 95% CI, 1.09, 2.44). The associations with the other domains assessed by the CAT‐MH were not statistically significant and the confidence intervals around the estimates were quite wide. Appendix S7 presents the point estimates for the covariates whereas Appendix S8 provides the data using log‐linked general linear regression with combined categorical predictors.

TABLE 6

Prediction of ED visits within 30 days using logistic regression with combined categorical predictors

	OR (95% CI; P value)
Suicide
Low risk	REF
Intermediate‐high risk	1.63 (1.09–2.44; P = 0.02)
Depression
Normal‐mild	REF
Moderate‐severe	0.97 (0.52–1.81; P = 0.93)
Anxiety
Normal‐mild	REF
Moderate‐severe	1.12 (0.70–1.78; P = 0.65)
PTSD
No evidence	REF
Possible‐highly likely	1.07 (0.63–1.82; P = 0.80)
Substance use
Low risk	REF
Intermediate‐high risk	0.85 (0.55–1.33; P = 0.48)

Note: Derived from 794 unique individuals. ED visits dichotomized into not present/any (0/1). Demographic variables included are age, gender, ethnicity, race, marital status, and employment.

First, we predicted continuous CAT‐MH subscales from visits to the ED within the 12 months prior to enrollment. The pattern of results was similar to the main analyses. For example, patients who visited the ED 2 or more times in the previous year scored slightly higher (4.6 points, 0.25 SD difference) on the continuous suicide subscale (Appendix S9 Second, we predicted ED admissions 30 days after enrollment from standardized continuous CAT‐MH subscales. The trend of results was similar to the main analyses (Appendix S10). For suicide, each SD increase in the suicide subscale was associated with a 19% increased odds of ED admission. Third, when conducting ordinal logistic regression to predict ED visits within 30 days after enrollment, the results followed a similar pattern as the main analyses, although certain estimates were statistically significant (Appendix S11).

LIMITATIONS

There are several important limitations to mention. Given that this study depended on voluntary participation, it is unclear if those with underlying physical or psychological conditions were more or less likely to volunteer to participate given factors such as a willingness to disclose and seek help or concern regarding persistent mental health stigmas. Although we randomized the possibility of study inclusion, we relied on a convenience sample of individuals presenting to the ED, and approximately 45% of individuals did not consent to be included in the study. The CAT‐MH tool is offered in different languages, but we excluded those who were non‐English speaking as our enrollment staff would not be able to have an informed consent conversation in another language. Additionally, those who were felt to have issues affecting utilization review (eg, out of state residence) were also excluded. In any case, these all represent a form of selection bias, potentially influencing our results. If individuals with mental health conditions are more inclined to decline participation, it is possible that our results underestimate the magnitude of impairing mental health and/or substance use conditions. The opposite would be true if those with mental health conditions were overrepresented in the sample. Although our data cannot determine the direction of bias, the prevalence of mental health symptomatology is comparable to prior research. Our overall enrollment goals (1000 patients) and ED recidivism targets (12 months) were curtailed by the institution of COVID‐19 precautions and stay‐at‐home orders. Increasing enrollment may have aided our ability to examine separate levels of the CAT‐MH subscales. Finally, although we examined ED utilization, we cannot infer a causal relation between mental health/substance use problems and ED use.

DISCUSSION

This prospective observational study represents the most complete use of the validated computer adaptive tool (CAT‐MH) in the ED environment for multidomain mental health screening in a broad random sample of patients presenting to a large urban emergency department primarily for non‐psychiatric complaints. Because a significant proportion of ED patients who are evaluated for non‐psychiatric complaints (eg, chest pain and abdominal pain) have comorbid mental health problems, which often remain undiagnosed during ED encounters, , , , mental health screening is a critical first step in identifying complex symptomatology, triaging care, and potentially preventing ED recidivism.

Estimates suggest that approximately 1% of all US ED visits are related to suicidal ideation. In our enrolled sample, 3 participants screened positive for suicidal ideation according to standard ED suicide screening procedure as described above. Although the CAT‐MH correctly identified each of these individuals and categorized them as high risk, this tool detected 7 additional high‐risk individuals not flagged by standard screening. We cannot determine whether these individuals were false‐positives per other suicide screens, although these individuals were elevated on an underlying continuum of suicidality and likely required clinical attention. Future research is needed to compare the CAT‐MH to other standard suicide practices and predict suicidality to further clarify this finding's implications. Approximately, 22% of individuals scored in the intermediate suicide risk group. Notably, clinician report of suspected risk of future suicide attempt was only very weakly correlated (r = 0.11) with the CAT‐MH index of suicidality. It is possible that patients felt more at ease disclosing their feelings via the tablet administered CAT‐MH rather than directly to the nurse at check in or during the evaluation by the doctor.

One of the aims of our study was to examine the relationship between positive screens for impaired mental health and ED recidivism because mental illness has been shown to be a strong predictor of frequent ED use. , , In our study, compared to those with no visits in the 12 months prior to enrollment, those with 2 or more ED visits were significantly more likely to score in the high ranges for suicide risk, PTSD, and substance use on the CAT‐MH. With regard to subsequent ED use in the 30 days post‐enrollment, suicide risk (intermediate‐high on the CAT‐MH) was associated with an increased likelihood of repeat ED utilization. Taken together, these results are consistent with prior literature showing that approximately 30%–50% of these patients with frequent ED utilization have a mental health or substance use disorder. , , , ,

Clinical implications

Although this study did not directly investigate the feasibility of CAT‐MH implementation in an ED setting nor was the intention to inform decision‐making practices, results from this study have important clinical implications. The median response time was 9.69 minutes—roughly 2 minutes per each of the 5 CAT‐MH domains, which is similar to fixed‐item mental health screening tools such as the GAD‐7 and PHQ‐8 depression. Additionally, of those that consented, the extremely high completion rate (97%) is also quite encouraging in that, despite the busy nature of an ED, less than 3% of participants encountered circumstances which prevented timely completion of the tool. Thus, our results suggest that future studies should explore the implementation of the tool in EDs.

The impact of an adaptive screening tool on ED workflow depends on the resources available, as increased screening will need to be met with increased patient follow‐up. In settings with integrated mental health professionals, individuals who screen in the intermediate/high risk categories can meet with clinicians who can conduct further assessments, brief interventions, and coordinate with further care. In settings with limited mental health resources, clinicians may prioritize learning and implementing brief interventions (eg, safety planning, counseling on access to lethal means) for positive suicide risk and developing an up‐to‐date mental health referral list for patients. If an individual screens as positive suicide risk, receives further assessment, and is held in the ED waiting for a psychiatric bed, an adaptive test could be administered multiple times to determine longitudinal changes in suicide risk. Those who screen in low‐risk categories could receive access to internet‐based interventions or follow‐up adaptive screens to facilitate future referrals.

In the study, we were able to demonstrate the use of the validated multidimensional CAT‐MH to document the high rate of impairing mental health conditions and their associated ED utilization in a population of ED patients presenting largely for somatic complaints. There is a strong association between mental health issues and ED recidivism, which may benefit from screening tools such as the CAT‐MH. Future work should focus on assessing the feasibility of universal screening implementation via the multidomain CAT‐MH, how the CAT‐MH can support clinical decision making in an ED setting, and the association between the CAT‐MH suicide subscale specifically with suicide‐related ED visits.

CONFLICTS OF INTEREST

Robert D. Gibbons is a founder of Adaptive Testing Technologies, which distributes the Computerized Adaptive Test—Mental Health suite of computerized adaptive tests (CAT‐MH is a trademark of Adaptive Testing Technologies); the terms of this arrangement have been reviewed and approved by the University of Chicago in accordance with its conflicts of interest policies. The authors report no additional financial or other relationship relevant to the subject of this article.

AUTHOR CONTRIBUTIONS

Brian M. D'Onofrio obtained research funding. Brian M. D'Onofrio, Paul I. Musey, and David G. Beiser conceived and designed the study. Paul I. Musey and Kelli Peterson oversaw patient recruitment and managed the data. Azhar I. Dalal, Serena Maag, Matthew T. Perry, Alex Card, Max B. Bohrer, Jackson Hamersly, and Setarah Mohammad Nader undertook the recruitment of patients. Lauren M. O'Reilly analyzed the data. Paul I. Musey, Lauren M. O'Reilly, and Brian M. D'Onofrio drafted the manuscript, and all authors contributed substantially to the revision.

Supporting Information

Click here for additional data file.