The Psychometric Properties of the Brief Symptom Inventory in Men under Criminal Justice Involvement: Implications for Forensic Social Workers in Practice Settings.

The purpose of this study was to assess the factor structure and psychometric properties of the original and a revised modification of the Brief Symptom Inventory (BSI) in 259 black and Latino males, aged thirty-five to sixty-seven, who had been released from a New York state prison or a New York City jail. The data were analysed using exploratory factor analysis, principal axis factoring and confirmatory factor analysis. Standardised factor loadings were evaluated at 0.05, model fit was evaluated using the chi-square statistic, and fit indices were examined. Items whose communalities fell below 0.30 were eliminated from the procedure. The findings did not yield the same number of factors as the original BSI, but the revised BSI model fitted the current data better. This modified factor structure reduced the BSI to the nineteen most appropriate items to assess five key common psychiatric symptoms affecting men under community supervision. The results of the current factor structure suggest that the psychiatric disorders experienced by men under community supervision may differ from the populations studied by the original BSI factor structure. Forensic social work ought to examine the psychometric properties of standardised measures for different populations such that appropriate instruments may be specifically targeted and maximised.

Introduction

In the USA, 700,000 individuals are released from state and federal prisons in a given year (West and Sabol, 2010), and approximately nine million individuals are released from local and city jails (Beck, 2006). Formerly incarcerated individuals returning to the community after long periods of incarceration often have difficulties managing the most fundamental tasks for community reintegration including, but not limited to: finding meaningful employment; securing stable housing; reconnecting with families; and accessing mental health care services and treatment (Drucker, 2011; Luther ). Of the 700,000 individuals who are released from US prison systems, approximately two-thirds will be re-arrested, and more than half will return to prison within three years for new crimes or parole violations (Pew Center on the States, 2011).

Many individuals involved in the US criminal justice system have mental health problems (National Commission on Correctional Health Care, 2002; James and Glaze, 2006; Morrissey ). According to James and Glaze (2006), the prevalence of mental health problems is highest among jail inmates, followed by state inmates and federal prisoners. The symptoms commonly observed in individuals who have been incarcerated include, but are not limited to: mania, insomnia, psychomotor agitation, and hostility (Maruna, 2001; Solomon ; James and Glaze, 2006). Chui and Chan (2012) assessed different psychological characteristics among Hong Kong male juvenile probationers (e.g. pro-criminal attitudes, positive and negative affect) and found that the type of crime(s) committed, timing of the onset of delinquent behaviours, the probationers' negative affect and the probationers' propensities for impulsivity, among other elements, were significant risk factors for recidivism. One way to assess the psychological characteristics of individuals with criminal justice backgrounds is to use psychological assessments for criminal justice populations, which detect signs and symptoms of psychiatric disorders. However, to our knowledge, no studies to date have investigated the utility and psychometric properties of a self-report psychological symptom inventory in men under parole or probation (hereafter described as ‘community supervision’).

Assessment of the Brief Symptom Inventory

The Brief Symptom Inventory (BSI) is a standardised screening instrument that evaluates psychological distress and psychiatric disorders, commonly used in a variety of settings with adolescents and adults (Derogatis, 1993). The BSI is a fifty-three-item self-report scale designed to assess psychopathology and psychological symptoms, measuring nine dimensions that can be summed to reflect three global indices: the General Severity Index (GSI), the Positive Symptom Distress Index (PSDI) and the Positive Symptom Total (PST) (Derogatis, 1993). The BSI uses the five-point Likert scale, ranging from 0 (‘not at all’) to 4 (‘extremely’). The original factor was intended for adults and adolescents with a range of psychiatric disorders.

Researchers for several decades have been documenting the significance of the BSI to assess mental health problems among vulnerable populations, including disadvantaged youth (Grisso ; Whitt and Howard, 2012); homeless populations (Velasquez ); patients with cancer (Grassi ); persons with intellectual disabilities (Kellett ; Endermann, 2005); populations with substance use disorders (Royse and Drude, 1984); and adult populations with mental health problems (Derogatis and Melisaratos, 1983). Though the BSI is widely used in juvenile justice populations (Whitt and Howard, 2012), little is known about the psychometric properties to measure current psychiatric or psychological symptoms and the potential utility of the BSI for men under community supervision following their release from jail or prison. Thus, the objective of this paper was to further extend this research on the utility of the BSI in a sample of men under community supervision. The aims of this study were to use exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to determine the factor structure proposed by Derogatis and Melisaratos (1983) among men under community supervision and to compare the current factor structure proposed by Whitt and Howard (2012).

Methods

Study sample and procedures

The data used for this study were collected for a previous research project that focused on cancer–health disparities among men involved in the criminal justice system (Valera ). The study was approved by the Albert Einstein College of Medicine's Institutional Review Board and a Federal Certificate of Confidentiality was obtained. Using a venue-based sampling approach (Muhib ), 259 participants were recruited through word of mouth and flyers placed in addiction treatment centres, re-entry agencies and community-based centres frequented by men under community supervision. The primary study focused on black and Latino men who met the following eligibility criteria: (i) self-identified as a male; (ii) aged thirty-five to sixty-seven years old; (iii) residing in the Bronx, New York; (iv) currently under parole or probation; (v) never been diagnosed with cancer; and (vi) provided informed consent. Recruitment took place over eight months in 2012. Participants underwent an informed consent process and completed individual cognitive survey interviews with a trained interviewer. Cognitive face-to-face cross-sectional survey interviews lasted 90 to 120 minutes, and respondents were compensated $25 for completing the survey.

Study setting

Bronx, New York, is the poorest borough of New York City and also one of the poorest counties in the nation (University of Wisconsin Population Health Institute, 2014). Bronx county has below average labour workforce participation and educational attainment, an unemployment rate of 14 percent (New York State Department Labor, 2014), and disproportionate rates of HIV/AIDS and high rates of chronic health conditions (e.g. heart, disease, obesity, diabetes and asthma) (University of Wisconsin Population Health Institute, 2014).

Measures

The cross-sectional survey focused on demographic variables including age, race, education, geographic location of release (e.g. zip code), facility type (e.g. prison or jail), and outcome variables including smoking behaviours, social support, research knowledge (i.e. the ability to understand certain procedures of a biomedical or clinical research study), cancer cognition, physical health status, attachment, discrimination, and medical mistrust. For the purpose of this study, we reported on the psychometric properties of the BSI proposed by Derogatis and Melisaratos (1983) and Whitt and Howard (2012). We hypothesise that Whitt and Howard's (2012) proposed BSI structure would have a more adequate psychometric structure than the original factor structure of Derogatis and Melisaratos (1983).

Procedures

EFA procedure

Communalities were examined and items whose communalities fell below 0.30 were eliminated from the procedure.

Principal axis factoring (PAF) was used to extract the factors. The screen plot and Kaiser's root eigenvalue were used to determine the number of factors to retain. A non-orthogonal Oblimin procedure was used to rotate the factors.

CFA procedure

Model fit was assessed by interpreting the chi-square statistic and the following fit indices: Comparative Fit Index (CFI)—value of 0.95 and above indicated good model fit (Hu and Bentler, 1999); Tucker-Lewis Index (TLI)—value of 0.95 and above indicated good model fit (Hu and Bentler, 1999); Root Mean Square Error of Approximation (RMSEA)—value less than 0.06 indicated good model fit; value less than 0.08 indicated reasonable fit; value less than 0.10 had poor fit (Brown and Cudeck, 1993); Standardized Root Mean Square Residual (SRMR)—value less than 0.08 indicated good model fit (Hu and Bentler, 1999); Normed chi-square or ratio of likelihood χ2 to degrees of freedom—benchmark was not established, but the lower the number (i.e. below 3.00), the better the fit. EFA was conducted using the Statistical Package for Social Sciences (SPSS) 20 program. The CFA was conducted using AMOS 20 software (Kline, 2010).

Results

Sample characteristics

The participants were black and Latino and released from a New York state correctional prison or New York City's main jail complex—Rikers Island. More than half (60 percent) of the participants were released from a state prison and 40 percent (n = 102) were released from Rikers Island. The participants ranged in age from thirty-five to sixty-seven, with an average age of forty-seven (SD = 6.63). Forty-six percent of participants identified as Latino, 49 percent identified as black and 5 percent selected other race/ethnicity. Nearly 50 percent had been released within one year of completing the research study. The majority of the participants dropped out of high school (65 percent), and only 35 percent completed high school or passed the general educational development (GED) tests (demonstrating sufficient knowledge in core content areas equivalent to that of graduating high school seniors). Details of their demographic characteristics have been published elsewhere (Valera ).

EFA findings of Derogatis and Melisaratos (1983)

As shown in the pattern matrix of Table 1, only twenty-six items loaded onto one of the ten factors. Out of the twenty-six items retained, only nineteen items were retained in the Whitt and Howard (2012) factor structure. Thus, we did not replicate the present study with the original factor structure of Derogatis and Melisaratos (1983).

Table 1

Pattern matrix for the original BSI 53-item measure (Derogatis and Melisaratos, 1983)

Item	Factor
	1	2	3	4	5	6	7	8	9	10
14 Feeling lonely even with others	0.63
16 Feeling lonely	0.56	−0.47	0.66	−0.98
17 Feeling blue	0.49	−0.68	0.74
20 Feelings easily hurt	0.46	−0.60
46 Get into frequent arguments	0.52	−0.46
49 Feeling so restless can't sit still	0.45	−0.67
50 Feelings of worthlessness	0.50
52 Feelings of guilt	0.45
12 Suddenly scared for no reason
28 Afraid to travel on buses, etc.
31 Avoid places because frightening
43 Feeling uneasy in crowds
45 Spells of terror or panic
40 Have urges to beat someone
41 Have urges to break things
9 Thoughts of ending own life
33 Numbness in parts of body					−0.63
37 Feeling weak in parts of body					−0.77	−0.47	0.46	0.42	0.50	−0.42
35 Feeling hopeless about future						−0.53	0.63		0.46
36 Trouble concentrating
3 Someone else can control thoughts
4 Others to blame for your troubles
39 Thoughts of death or dying
21 Feel people are unfriendly
26 Double-check what you do
30 Have hot or cold spells

CFA findings of original BSI (1983) factor structure

The CFA yielded an inadmissible solution, because the covariance matrix was not definitely positive. Inspection of the correlation matrix revealed extremely high correlations between many of the factors (i.e. correlations were between 0.91 and 0.99), thus indicating that the eight factors could be narrowed down to a smaller number of factors. Given that the EFA did not yield the same number of factors as the EFA of Derogatis and Melisaratos (1983) and that the factors in the current study consisted of items from different factors of the original BSI measure, we tested the Whitt and Howard (2012) factor structure with the sample's reported BSI data.

CFA findings of Whitt and Howard (2012)

The chi-square results, displayed in Table 2, indicate that Whitt and Howard's (2012) model was adequate, since the RMSEA was within the reasonable range at 0.07. The SRMR was within the acceptable range at 0.05, and the normed chi-square was also acceptable at 2.25; but the TLIs were not acceptable, as they were both below 0.95. All standardised factor loadings were statistically significant, although a few items had standardised loadings below 0.50 (Hair ). All correlations were statistically significant. However, the correlation between the obsessive–compulsive and hostility constructs was extremely high at 0.89, indicating some collinearity between the two constructs.

Table 2

Chi-square results and goodness of fit indices for the Whitt and Howard (2012) model

Index	Value
Chi-square	583.63
Degrees of freedom	260.00
Sig.	0.00
Normed chi-square	2.25
Tucker–Lewis index (TLI)	0.89
Comparative fit index (CFI)	0.91
Root mean squared error (RMSEA)	0.07
Lower bound of 90% confidence interval	0.06
Upper bound of 90% confidence interval	0.08
Standardised root mean square residual (SRMR)	0.05

Modified Whitt and Howard (2012) factor structure

Although the EFA did not yield the same factor structure, the CFI findings indicate that the Whitt and Howard (2012) model fitted the current sample adequately. We modified Whitt and Howard's (2012) model based on two criteria. First, only indicator variables with standardised factor loadings above 0.50 were retained (Hair ). Second, indicator variables with high modification indices (MI) were deleted, as this was an indication that the variables were cross-loading onto other constructs (Byrne, 2001). After the model was modified, we conducted several tests to assess the convergent and discriminant validity of the constructs. The composite reliability and the average variance extracted were used to measure the convergent validity of the constructs. Constructs are deemed to have convergent validity when the composite reliability exceeds the criterion of 0.70 (Hair ), and the average variance extracted is above 0.50 (Bagozzi, 1994).

Discriminant validity

To assess discriminant validity, we compared the absolute value of the correlations between the constructs and the square root of the average variance extracted by a construct (Tabachnick and Fidell, 2007). When the correlations are lower than the square root of the average variance extracted by a construct, constructs are said to have discriminant validity (Fornell and Larcker, 1981). Based on the two criteria described above, five items were deleted. Three items measuring somatisation were deleted. Item 2 (‘faintness or dizziness’) had a standardised factor loading below 0.50. Item 7 (‘pains in heart and chest’) was highly correlated with item 32 (‘your mind going blank’, which also measures obsessive–compulsion). The MI for its error and 32's error to co-vary was 11.26. The MI for it to load onto item 32 (‘your mind going blank’) was 5.45, and item 23 (‘nausea or upset stomach’) was highly correlated with item 32 (‘your mind going blank’, measuring obsessive compulsion). The MI for its error and 32's error to co-vary was 15.91, and the MI for it to load onto 32 (‘your mind going blank’) was 6.63.

One item measuring phobic anxiety, item 20 (‘your feelings being easily hurt’), was highly correlated with item 16 (‘feeling lonely’, measuring for depression). The MI for its error and 16's error to co-vary was 11.95, and the MI for it to load onto 16 (‘feeling lonely’) was 15.22. One item measuring hostility, item 4 (‘feeling others are to blame for most of your troubles’), was highly correlated with item 12 (‘suddenly scared for no reason’, measuring phobic anxiety). The MI for its error and 12's error to co-vary was 9.35, and the MI for it to load onto item 12 (‘suddenly scared for no reason’) was 4.03.

As shown in Table 3, the normed chi-square index was much lower at 1.88. Similarly, the RMSEA was lower at 0.06, and the SRMR dropped to 0.04. The CFA was acceptable at 0.95, and the TLI was close to acceptable at 0.94. In addition, as shown in Table 4, all standardised factor loadings were significant and above 0.50. The findings in Table 5 reveal that, except for the suicidal ideation construct, the composite reliabilities of the constructs were above 0.70. Furthermore, the average variance extracted value for all constructs was above 0.50. Thus, except for the suicidal ideation construct, all other constructs were deemed as having convergent validity.

Table 3

Chi-square results and goodness of fit indices for the revised Whitt and Howard (2012) models

Index	With suicidal ideation	Without suicidal ideation
Chi-square	291.77	227.54
Degrees of freedom	155.00	125.00
Sig.	0.00	0.00
Normed chi-square	1.88	1.82
Tucker–Lewis index (TLI)	0.94	0.96
Comparative fit index (CFI)	0.95	0.95
Root mean squared error (RMSEA)	0.06	0.06
Lower bound of 90% confidence interval	0.05	0.05
Upper bound of 90% confidence interval	0.07	0.07
Standardised root mean square residual (SRMR)	0.04	0.04

Table 4

Unstandardised and standardised factor loadings for the modified Whitt and Howard (2012) model

Path	B	SE	β	t
Somatisation to:
BSI 33	1.00	–	0.75	–	***
BSI 37	0.73	0.07	0.84	11.16	***
Phobic anxiety to:
BSI 8	1	–	0.8	–	***
BSI 12	1.36	0.11	0.76	12.6	***
BSI 28	1.46	0.12	0.71	11.74	***
BSI 47	1.27	0.11	0.72	11.93	***
Obsessive–compulsive to:
BSI 5	1.00	–	0.72	–	***
BSI 27	0.99	0.08	0.77	11.99	***
BSI 32	0.88	0.07	0.76	11.85	***
BSI 36	1.16	0.09	0.86	13.4	***
BSI 53	0.95	0.08	0.73	11.37
Depression to:
BSI 14	1.00	–	0.86	–	***
BSI 16	1.01	0.06	0.87	17.43	***
Suicidal ideation to:
BSI 9	1.00	–	0.54	–	***
BSI 39	2.12	0.34	0.68	6.23	***
Hostility to:
BSI 6	1.00	–	0.8	–	***
BSI 10	0.92	0.09	0.65	10.67	***
BSI 13	0.83	0.07	0.72	12.05	***
BSI 46	0.72	0.06	0.68	11.3	***

Note. ***p < .001.

Table 5

Convergent validity for the constructs

Construct	Composite reliability1	Average variance extracted2
Somatisation	0.78	0.80
Phobic anxiety	0.84	0.75
Obsessive–compulsive	0.88	0.77
Depression	0.87	0.83
Suicidal ideation	0.54	0.61
Hostility	0.81	0.71

1Composite reliability = (square of summation of factor loadings)/[(square of summation of factor loadings) + (summation of error)].

2Average variance extracted = (summation of the square of factor loadings)/[(summation of the square of factor loadings) + (summation of error)].

The findings in Table 6 show that, except for the obsessive–compulsive construct, the square roots of the average variance extracted for the constructs were higher than their correlations with other constructs. Therefore, except for the obsessive–compulsive construct, all constructs had discriminant validity. Note, however, that the correlation between the obsessive–compulsive and hostility constructs was only 0.01 higher than the square root of the average variance extracted by the obsessive–compulsive construct, indicating that these two constructs appear to be two separate constructs. When a model without the suicide ideation construct was tested, model fit improved significantly, Δχ2 (30) = 64.23, p < .005. Both TLI and CFI values also increased and were now at or above the cut-off criterion.

Table 6

Discriminant validity results for the revised Whitt and Howard (2012) model

Construct	1	2	3	4	5	6
1 Somatisation	0.89
2 Phobic anxiety	0.71	0.86	0.88	0.91	0.78	0.84
3 Obsessive–compulsive	0.73	0.76	0.78	0.71	0.70
4 Depression	0.67	0.72	0.71	0.82
5 Suicidal ideation	0.51	0.64	0.89
6 Hostility	0.68	0.77

The values of the square root of the average variance extracted are on the diagonal; all other entries are the correlations.

Discussion

The present study investigated the psychometric properties of Derogatis and Melisaratos (1983) and Whitt and Howard's (2012) factor structure. The results suggest that the model proposed by Whitt and Howard (2012) may be appropriate and useful for some, but not all, of the psychometric properties. The correlations between the modified BSI items that assessed five common psychiatric symptoms—somatisation, phobic anxiety, obsessive compulsion, depression and hostility—were adequate and fit the current data better than the original BSI structure. The suicidal ideation construct proposed by Whitt and Howard (2012) had minimal convergent validity; it appeared that a model without this construct fits the data better.

Our findings contribute to the mental health symptomology research literature in several ways. First, the current study allows social workers to consider other mental health problems, such as hostility and phobic anxiety that might be significant concerns among men who are currently under some form of community supervision. For instance, feelings of hostility in men under community supervision could be linked to reoffending (Agnew and Brezina, 1997; Maruna, 2001). Feelings of hostility and phobic anxiety are likely to be the result of being overwhelmed by the challenges of community reintegration faced by men under community supervision (National Commission on Correctional Health Care, 2002; Solomon ). Second, this modified factor model reduces the BSI to nineteen most appropriate items to assess five key common psychiatric symptoms affecting men under community supervision. The results of the current factor structure suggest that the psychiatric disorders experienced by men under community supervision may differ from the populations studied by the original BSI factor structure. Given that the EFA did not yield the same number of factors as the EFA proposed by Derogatis and Melisaratos (1983) and that the factors in the current study consisted of items from different factors of the original BSI measure. Thus, it appeared that the original factor structure could not be generalised onto the target population. Third, the present study demonstrates how a tailored nineteen-item BSI for specific populations might be used to assess psychiatric disorders that could provide a framework for appropriate treatment approaches for adult men under community supervision.

Strengths and limitations

While this study is the first step toward disentangling the psychological constructs of the BSI to the non-psychiatric population of men under community supervision, it is not without limitations. The primary limitation of the study was the rather small sample size and therefore, the findings are likely not generalisable to other populations of men involved in the criminal justice system. While the revised Whitt and Howard (2012) model fitted the data better than the factor structure of Derogatis and Melisaratos (1983), the Whitt and Howard (2012) study sample comprised of juvenile justice populations; unlike juvenile justice populations, adult men under community supervision have often served longer sentences, which means their re-entry from prison to their communities often results in more complicated reintegration processes (Petersilia, 2003). Additional psychometric studies are needed to accurately identify specific psychiatric BSI subscales for formerly incarcerated adults.

Implications for forensic social work in clinical settings

The current findings encourage forensic social workers in practice settings with justice-involved populations to start examining the psychometric properties of standardised measures for different populations, such that available assessments and instruments may be specifically targeted and maximised. It is critical for forensic social workers in practice settings to use appropriate health assessment tools and tailor original instruments to aid the transition of former prisoners into the community. Forensic social workers in practice settings often serve as the front line staff and administrators in these settings. Thus, future studies in the areas of clinical social work and forensic social work could also examine the interplay between psychiatric conditions and symptomatology, the social environment, biological factors and mental health in criminal justice populations (Solomon ; Hansen ). Our study builds on the important examination of existing health assessment tools to effectively and accurately diagnose the most vulnerable and marginalised criminal justice populations (Donovan, 2003; McLellan ), in order to create effective mental health treatments and reduce recidivism.