Measuring the economic costs of discrimination experienced by people with mental health problems: development of the Costs of Discrimination Assessment (CODA).

PURPOSE: Stigma and discrimination are faced by many with mental health problems and this may affect the uptake of services and engagement in leisure and recreational activities. The aims of this study were to develop a schedule to measure the impact of stigma and discrimination on service use, employment and leisure activities and to estimate the value of such reductions.
METHODS: A questionnaire, the Cost of Discrimination Assessment, was developed and piloted in a sample people with mental health problems. Costs were calculated and test-retest reliability assessed.
RESULTS: Test-retest reliability was good for most items. A substantial proportion of the sample had experienced negative impacts on employment as a result of stigma and discrimination. Around one-fifth had reduced contacts with general practitioners in the previous 6 months due to stigma and discrimination and the leisure activity most affected was visiting pubs/restaurants/café.
CONCLUSIONS: In conclusion, stigma and discrimination result in reduced use of services and reduced engagement in leisure activities. This represents a welfare loss to individuals.

Introduction

In recent years, the subject of mental health-related stigma and discrimination and its effects on people with mental health problems has been the subject of increasing research interest [1]. At an individual level stigma and discrimination may result in the social exclusion of people with mental health problems because the wider population is less willing to interact with them [2], which can mean that they participate less in valued social and leisure activities, to the potential detriment of their quality of life. Such stigma also impacts on help seeking for mental health and other health problems, making it less likely that help is sought [3]. At a more structural level, these negative attitudes can become expressed in the form of discrimination in relation to civil rights, housing, employment and financial institutions, and the highly stigmatising attitudes towards people with mental disorders can act as a disincentive to invest in mental health services to the same extent as other health care services [2, 4]. Such negative attitudes, particularly the low perceived effectiveness of professional care, can also be detrimental to the visibility and credibility of mental health services [5].

Economic theory suggests that individuals engage in specific activities to maximise their ‘utility’ (which may alternatively be described as ‘wellbeing’). Failure to engage in desired activities would, therefore, result in a ‘cost’ to the individual in the form of non-realised utility or welfare. For example, avoidance of visits to cafes or restaurants because of perceived discrimination will lead to a loss to the individual, although the value of this is hard to determine. This can also extend to engagement with activities where the benefit may be in the future. For example, avoidance of visiting the dentist may not result in an immediate loss, but dental problems in the future would presumably represent a loss. Some effects are more tangible—for example discrimination in terms of employment. Discrimination in relation to mental health problems, therefore, is likely to come with a cost. However, to date few studies have explicitly addressed this [2].

This paper aims (1) to describe a new schedule for recording the economic impact of mental health-related discrimination and stigma, the Costs of Discrimination Assessment (CODA), and (2) to investigate the extent to which beneficiaries or users of projects set up under an anti-stigma programme, called Time to Change (TTC), have been affected by mental health-related stigma and discrimination [6]. Economic impact may include costs, but the main focus is on changes in activities which lead to a welfare loss for individuals. While some psychometric properties of the schedule are addressed it is not intended that this be a clinical instrument that is to be used across studies. Rather it is intended that it be used in its current form where appropriate or that sections of it be used in future studies or indeed that it be adapted where necessary.

Methods

Questionnaire

Development of the CODA was based on issues highlighted in the literature on mental health-related discrimination and stigma, and in focus group sessions with service users. It consists of an interview schedule which aims to establish the impact of mental health-related discrimination and stigma in relation to the following areas: employment opportunities and experience in work, dealing with financial institutions (e.g. mortgage lenders) and housing departments, seeking help from health and social care professionals, contacts with police, support from families, use of private healthcare, and participation in social and recreational activities. The CODA asks questions in ways relevant to each area and invites yes/no responses (e.g. whether the respondent feels they have been discriminated against), free-text responses describing the experience, and quantitative responses (e.g. number of contacts with services). Some questions regarding service use and activities refer to the participant’s whole life to that point while others relate to the previous 6 months. Level of education and illness severity are important considerations in stigma-related research [7], and items relating to these were included. The CODA is included as supplementary material.

Study sites, participants and procedures

Details of local TTC projects have been given elsewhere [8]. A total of 18 TTC local projects mainly from urban (but not inner city) areas around England were visited on different days, and a convenience sample of 108 participants was recruited from among the beneficiaries (defined as those using the service) attending on the day of the visit. Data were not available on those not taking part because interviews were only held with those who made their availability known to the interviewer or staff. Participants were all aged between 18 and 65 years, and had a history of treatment for mental health problems. Of these participants, a subsample of 16 was re-interviewed within 2 weeks of the initial visit to establish the test–retest reliability of the CODA. Local ethical approval was granted and the study was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki. After informing the participants about the purpose of the evaluation, informed consent was obtained and the interview was conducted.

Analysis

The data collected were entered into an SPSS database for analysis. Descriptive statistics concerning socio-demographic and clinical characteristics of the participants were produced, as well as level of involvement with the project, and employment and income. Chi-square tests were used to examine associations between lack of educational qualifications and not having been in paid employment since first contact with services. Descriptive statistics were also produced concerning lifetime and past 6-month experience of stigma and discrimination. Test–retest reliability for questionnaire items was assessed using Cohen’s kappa.

Reduced service use could be seen as representing a saving to the economy, which may be good or bad. However, we here assume that reduced use as a result of stigma or discrimination represents a ‘welfare loss’ (i.e. this is negative) to the individual which can be valued using the cost incurred were this service to be provided. As such, we are assuming that the service use that would otherwise have taken place would have been appropriate (this also means that in the longer term any apparent savings would likely be lost because reduced service use may be reflected in poorer health outcomes). Similarly, foregone leisure/recreational activities can be valued using the price that would have been paid to engage in them. This would again represent a welfare loss to the individual, although not an overall economic loss if we assume that the funds that would have been spent on these activities will be spent in other ways. Costs/values for each service that was used less or more often in the previous 6 months as a consequence of discrimination or stigma were calculated from recognised national sources [9, 10]. Exceptions were for police time (£16.40 per hour taken from unpublished data), dental appointments and contacts with advocates/solicitors (both assumed to cost £50 per contact). Reduced help from family members was valued using an average wage rate of £12.56 per hour [11] with an assumption that a contact with a family member or friend would last 2 h. Values were taken from an internet search and placed on foregone contacts of the following activities: team sports (£4.60 per contact), visits to cinema/theatre (£10.50), visits to art galleries/museums (£5), visits to the gym (£4.60), and visits to pubs/restaurants/cafes (£10.50). These are only meant to be indicative values of these foregone activities. Other activity reductions were specified by the respondent but not costed due to the variability in this information and lack of available data.

Results

Socio-demographic characteristics

Of the 108 participants from the 18 projects, 78 (72 %) were male and 99 (92 %) were White British or White Other. The average age of the sample was 42.8 years (ranging from 22 to 65 years, SD 10.2 years). Regarding highest level of educational qualifications attained, 15 (14 %) had no qualifications, 37 (35 %) had GCSE/CSE/GCE or equivalent, 14 (13 %) had A levels. Sixteen participants (15 %) were educated to first degree level, and six (6 %) had a postgraduate qualification. The remaining 20 participants (19 %) held a range of professional and vocational qualifications.

Clinical characteristics

Twenty-two participants (17 %) had a (self-reported) diagnosis of schizophrenia, schizoaffective disorder, or other unspecified psychotic condition. Forty-one (39 %) had a diagnosis of depression, 11 (11 %) suffered from bipolar affective disorder, 18 (17 %) had an anxiety disorder, and six (6 %) had a personality disorder. One participant (1 %) preferred not to disclose their diagnosis, and two (2 %) reported that they did not know their diagnosis. Diagnoses among the remaining six participants (6 %) included Asperger’s syndrome, temporal lobe epilepsy, and traumatic brain damage.

Median length of time that the participants had been receiving treatment for mental health problems at the time of interview was 177 months (range 8–540 months). At interview, 58 participants (56 %) were being seen by secondary mental health services. Twenty-one participants (36 %) were being seen every 3 months or less, 36 (62 %) were seen more frequently than this, and for one participant it was unclear. Fifty-seven participants (53 %) had been admitted to psychiatric hospital at least once, 32 of whom (56 %) had a history of formal admission. Median length of time since last admission was 54 months (range 2–438 months). Of the seven participants who had been admitted in the 6 months prior to interview, three (42 %) had been formally admitted, and median length of inpatient stay amongst all seven was 8 days (range 2–42 days).

Involvement with the local project

The local projects offered activities for a mean of 4.3 days per week (range 1–7 days), and at the time of the visit the mean length of time that they had been operating for was 32.2 months (range 21–36 months). Mean length of time that the participants had been attending the projects’ activities was 19.9 months (range 3–36 months), and half of the participants had attended 65 % or more of the sessions offered during the time they had been attending the project.

Employment and income

At the time of the interview only two participants (2 %) were in full-time employment. Seventeen participants (16 %) were in part-time work, three (3 %) had jobs but were currently signed off sick, two (2 %) described themselves as self-employed, one (1 %) was in sheltered employment and 30 (28 %) were doing voluntary work. Seven (7 %) had retired due to ill health and two (2 %) had passed retirement age. The remaining 44 participants (41 %) were unemployed. Taking into account all sources of income, 90 participants (86 %) had an income of less than £1,000 per month.

Forty-eight participants (49 %) had held down or successfully applied for at least one job since receiving treatment for their mental health problems, while 13 (13 %) had applied for jobs but never successfully. Thirty-seven participants (38 %) had not applied for any jobs since becoming ill. Among those who had either not applied for any jobs or who had never made a successful application, seven (15 %) attributed this to actual or anticipated mental health-related discrimination, while 22 (48 %) reported that they had been too ill to apply for or hold down a job, and one (1 %) attributed it to both illness and discrimination. The remaining 16 participants (35 %) ascribed their never having applied for, or being appointed to a job that they had applied for, to other reasons.

Of the 89 participants who had been in contact with services for at least 4 years, 42 (47 %) had not been in any form of paid employment since they were first seen by services. Of these, 13 participants (31 %) had no educational qualifications. Chi-square tests revealed no statistically significant relationship between lack of qualifications and not having been in paid employment since first seen by services.

None of the participants without qualifications and who had not been in paid employment since the beginning of involvement with services attributed this to discrimination, while two participants (33 %) attributed it to the severity of their mental health problems and the remaining four cited other reasons unconnected to either illness severity or discrimination. This is in contrast to 15 participants with qualifications (47 %) who cited illness severity, six (19 %) discrimination, and 10 (31 %) other reasons (the remaining participant amongst these cited illness severity and discrimination equally).

Information concerning experience of mental health-related discrimination and stigma in terms of applying for jobs and employment during the last 6 months and over the participant’s lifetime are summarised in Table 1. Not all the items in the CODA applied to all participants and the sample size for those responding ‘Yes’, ‘No’ and ‘Don’t Know’ is noted for each item. For example, not everyone who had worked since developing their mental health problems may have requested a transfer to a new job location or to change jobs in the same location, and if they last worked before the Disability Discrimination Act came into effect in 1995, they would not have asked for allowances and adaptations to be made under this Act. Around one-third of participants had at some time decided not to apply for a job because they felt they would not be hired because of discrimination, and the figure was only slightly lower for the past 6 months. Around one-quarter felt that discrimination had at some time led to colleagues treating them unfairly, and similar proportions felt they had experienced difficulty progressing in their career, changing jobs or maintaining job security as a result of discrimination. Those who worked fewer hours than they wanted to stated on the questionnaires that they worked an average of 14 h per week less than their colleagues.

Table 1

Mental health-related discrimination in job applications and work

Item	Lifetime, N (%) [applicable N]	Previous 6 months, N (%) [applicable N]
Have you ever felt that you have not been hired for a job that you have applied for as a result of discrimination?	14 (21) [68]	2 (13) [5]
Have you ever decided not to apply for a job because you felt that you would not be hired as a result of discrimination?	30 (38) [78]	6 (30) [20]
Have you ever felt that you have been denied a promotion at work as a result of discrimination?	10 (16) [61]	0 [15]
Have you ever felt that you have been denied access to training as a result of discrimination?	12 (20) [61]	0 [14]
Have you ever felt that you have been demoted to a lower position at work as a result of discrimination?	7 (11) [63]	1 (7) [14]
Have you ever felt that you have been treated unfairly by co-workers as a result of discrimination?	16 (26) [62]	0 [15]
Have you ever applied for adaptations or allowances at work for your mental health problems on the basis of the Disability Discrimination Act?	4 (16) [25]	1 (14) [7]
Have you ever felt that you have been made to work more or fewer hours than you wished as a result of discrimination?	6 (10) [63]	0 [13]
If so, have you ever felt that you have been denied a requested transfer to another job location as a result of discrimination?	5 (13) [39]	1 (10) [10]
Have you ever felt that you have had difficulty progressing in your career as a result of discrimination?	15 (24) [62]	1 (6) [16]
Have you ever felt that you have had difficulty in changing jobs as a result of discrimination?	12 (28) [43]	1 (9) [11]
Have you ever felt that you were fired, made redundant or asked to resign as a result of discrimination?	16 (26) [62]	1 (6) [16]

Financial institutions and housing

None of the participants reported experiencing discrimination with regard to mortgage applications (participants who had successfully negotiated a mortgage had done so before the onset of their mental health problems). However, eight participants (12 %) reported having experienced discrimination in relation to housing, particularly with local councils being insensitive to their needs, with four of these participants reporting having experienced discrimination in the previous 6 months. Two participants (7 %) reported experiencing discrimination with regard to driving licence applications or approval, six (14 %) reported experiencing discrimination in the application or approval process of insurance policies (two of these reported discrimination in the previous 6 months), and 15 (21 %) reported experiencing discrimination in relation to other housing or financial situations (typically credit card or loan applications, problems with neighbours, and medical assessments undertaken by the Department of Work and Pensions), two of whom reported having experienced discrimination in the previous 6 months.

Healthcare service use

Data regarding the impact of mental health-based discrimination and stigma on the uptake of healthcare services and help seeking from family and friends is described in Table 2. In terms of lifetime use, reductions were most likely for general practitioners (GPs), psychiatrists, psychologists, police, and family/friends. During the previous 6 months, only one participant reported using a service more frequently because of stigma and discrimination. This participant consulted their GP six times more often than they would otherwise have done in the previous 6 months because of discrimination. Reduced service use was far more common for GPs, with other services left relatively unaffected in the previous 6 months. However, a relatively large proportion had reduced assistance from family/friends as a result of perceived discrimination. Table 2 also shows the average number of reduced contacts for those for whom this was applicable and the average welfare loss (value of missed contacts) across the whole sample. The latter was highest for family/friends, social workers and nurses. The average value or welfare loss of reduced service contacts (as measured by the costs of these had they been received) was £239 per participant over the 6-month period.

Table 2

Impact of mental health-based stigma and discrimination on healthcare service use, and upon participation in social and leisure activities

Service (number who have used the service or felt need to)	Service use affected by stigma or discrimination		Mean (SD) times help sought less because of stigma/discrimination in last 6 months	Mean (SD) value of missed service contacts in the previous 6 months (2010/2011 £s)
	Ever N (%)	Last 6 months N (%)
GP	34 (32)	19 (18)	2.6 (3.0)	15 (50)
Specialist doctor	17 (16)	3 (3)	1.7 (0.6)	7 (43)
Psychiatrist/psychologist	25 (24)	3 (3)	1.7 (0.6)	7 (40)
Psychiatric nurse/CPN	13 (12)	2 (2)	27.0 (35.4)	20 (193)
Dentist	11 (10)	3 (3)	1.3 (0.6)	2 (12)
Complementary/alternative practitioner	5 (5)	0 (0)	–	0 (0)
Patient advocate/CAB/solicitor/etc.	3 (3)	2 (2)	2.5 (0.7)	2 (18)
Social worker	12 (11)	2 (2)	13.5 (17.7)	27 (269)
Counsellor	1 (1)	0 (0)	–	0 (0)
Crisis team	3 (3)	2 (2)	12.0 (2.8)	9 (64)
Police/other emergency services	25 (24)	5 (5)	10.2 (8.1)	8 (44)
Help from family and friends around the home	29 (28)	15 (15)	16.9 (45.0)	63 (449)
Help from family and friends in the community	29 (28)	16 (16)	20.1 (35.4)	79 (389)

One-fifth of participants reported that they had visited pubs/restaurants/cafes less as a result of discrimination. The impact on gym attendance was also relatively high. During the past 6 months the value of foregone leisure/recreational activities was £46 per participant.

Test–retest reliability

Fleiss suggests subdivision of kappa values as follows: over 0.75, excellent agreement, 0.40–0.74, fair to good agreement, and below 0.40, poor agreement [12]. Of the 12 CODA employment items, 11 had a rating of 0.40 or above for lifetime history, and all 12 had a rating of 0.50 for the previous 6 months. The only item not achieving the 0.40 standard concerned not applying for jobs because of discrimination. While the kappa value of 0.36 for this item was quite close to the standard, the fact that it appears to have poor agreement casts some doubt on the strength of the finding discussed above concerning not applying for a job because of anticipated discrimination being the most common form of employment-related mental health discrimination experienced by the participants. Ratings for four out of the five financial institutions and housing items were higher than 0.40 for both the lifetime and 6 month ratings. Interestingly, the only item not to achieve a kappa rating of 0.40 or higher here was the item scoring discrimination in any other housing or financial institution, which suggests that it is too vague and should, therefore, be removed or replaced with more concrete examples (e.g. perceived discrimination in fitness to work assessments carried out on sickness benefit claimants for the Department of Work and Pensions). Of the 12 healthcare items rated, 11 met the 0.40 standard for lifetime ratings and seven met the standard for ratings of the previous 6 months (Table 3). The only item where the ratings were below 0.40 for both ratings was dentist. In one instance, the kappa coefficient of the 6-month rating for GP was 0.01, but this item had a 75 % agreement rate between ratings, which suggests that this might be a chance finding. Out of the six items for participation in social and leisure activities, four met the 0.40 standard for both lifetime and previous 6-month ratings. In this instance the same two items (going to the gym and going on holiday) achieved ratings of less than 0.40 and comparatively low agreement rates, which suggest that these items also require re-examination and possibly clearer definition.

Table 3

Test–retest reliability of employment, financial institutions and housing, healthcare and social and leisure activity items

Employment	Lifetime		Previous 6 months
Item	Kappa	Agreement (%)	Kappa	Agreement (%)
Not hired for job applied for	0.81	88	0.41	75
Did not apply for job because expected discrimination	0.36	56	0.44	69
Denied promotion	0.90	94	0.85	94
Denied training	0.62	75	1.00	100
Demoted	0.67	81	0.71	19
Treated unfairly by workmates	0.71	81	1.00	100
Denied requested allowances/adaptations	0.44	75	0.65	88
Made to work fewer hours than wanted to	0.59	75	0.82	94
Denied requested transfer to different job location	0.59	88	1.00	100
Difficulty progressing in career	0.70	81	1.00	100
Difficulty in changing job	0.50	75	0.82	94
Been fired/made redundant/asked to resign	0.81	88	1.00	100

Discussion

This paper has reported the development of, and results from, a schedule to measure the economic effects of mental health-related discrimination and stigma. The approach taken is different from that used in most economic evaluations in that we have assumed that lost opportunities represent a ‘cost’ (or loss of welfare) to the individual concerned, irrespective of the impact on the wider economy. This was intentional and does not mean that this approach should necessarily be used in economic evaluations of specific interventions. An alternative approach would have been to measure utility using a (not necessarily health related) quality of life measure, but this would not identify the welfare loss due to specific activities foregone.

Few beneficiaries in this sample were in any form of paid employment at the time of interview, which is unsurprising because if the beneficiaries were in paid employment they would be unlikely to be attending the project. However, nearly half of the beneficiaries had been in paid employment at some time during the period that they had been receiving help for their mental health problems. While many of these participants had experienced some form of discrimination at work, comparatively few of those who had not worked since starting to use services blamed stigma and discrimination directly for not having been employed.

While lack of educational qualifications is a significant barrier to gaining employment, and while a higher proportion of the sample as a whole had no qualifications than is the case for the wider community, few of the participants who had been unemployed for a long time had no qualifications. In contrast, Sejersen et al. [13] found that 48 % of those people surveyed who had been unemployed for 4–7 years had no educational qualifications. This suggests that, compared to the wider long-term unemployed population, participants surveyed in the present study who had been unemployed for a similar length of time are better educated, and factors relating to severity of mental health problems and stigma appear to be to blame for their lack of employment. Interestingly, while over two-thirds of the participants with qualifications attributed their lack of success in gaining employment to stigma and discrimination, the same proportion of those without qualifications cited other reasons. Unfortunately, in the interests of simplicity, the CODA schedule used in this pilot study focussed only on the extent to which participants attributed their lack of success in gaining employment to stigma or illness severity, and the role of other factors was not considered. Clearly, attributions of lack of success in seeking paid employment are varied, and more than one reason is likely to be involved. A factor that might account for the low level of paid employment in the sample is variation in regional unemployment rates. While the sample surveyed is geographically diverse, the size of the sample is too small to make such an analysis meaningful, and the bias caused by the nature of the population surveyed means that it is hard to establish a true unemployment rate among the population with mental health problems in the areas served by the local projects. In some parts of England over 90 % of people with severe mental illness are unemployed and such rates are higher than in some other European countries [14, 15].

While participants did report experiencing mental health-related discrimination and stigma in relation to job applications and work, this was not especially prevalent. The most frequently reported experience was of not applying for jobs because discrimination was anticipated. This supports findings elsewhere [16].

The main leisure/recreational impact of stigma and discrimination was to reduce visits to pubs/restaurants/cafes. Valuing these reduced activities is not straightforward and we here have used prices obtained from an internet search as indicators of the ‘welfare loss’. In future work, a more sophisticated methodology would be useful.

While discriminatory experiences in the workplace were not reported as commonly as might be expected, these findings show that mental health-related stigma and discrimination are significant problems affecting over a third of people with mental health problems at work. Employers, therefore, have a clear need to implement measures to prevent this to be compliant with the Equality Act 2010. This makes it illegal to discriminate directly or indirectly against people with mental health problems in public services and functions, access to premises, work, education, associations and public transport [17].

There was reduced lifetime use of a number of services. In the past 6 months this was particularly noticeable for use of GPs and contacts with family/friends. People with mental health problems may lack treatment for physical health problems [18, 19], and have relatively high mortality rates [20-22]. This reduced use of GPs is, therefore, concerning and indicates the need for interventions designed to enhance primary care access. Given that family and friends are the most likely sources of help for people with mental health problems on a day-to-day basis, the reduction in contact with them is important even if in monetary terms the ‘welfare loss’ is not great. It may represent an important loss in social capital and/or social support that, along with reduced access to employment, impede the recovery process.

There are a number of limitations with this study. First, we have developed and described a schedule for recording economic impacts of stigma and discrimination. We do not though have an instrument that has undergone the level of psychometric testing that would be required for a clinical scale. This was not though the intention. Rather we have followed the process by which other schedules, such as the Client Service Receipt Inventory (CSRI) [23], have been developed. The CSRI is a way of recording service use for economic evaluations and has been adapted for most studies and settings in which it is used. Second, we have assumed that not engaging in certain activities represents a welfare loss. This may not be the case, but it can be argued that individuals only engage in activities if the expected utility from doing so is sufficiently high. Third, we have relied on self-report information. This can be inaccurate but it was the only feasible option given the type of data we wished to collect. Fourth, while test–retest reliability was encouraging, this was only assessed for a small number of participants. Fifth, while the study sites were spread across the country, they were mainly in urban, non-inner-city, areas. As such they are not representative of the whole country. In addition, some of the effects of discrimination may be related to underlying economic conditions and these would change over time. Ideally we would repeat this exercise in different areas and at different time points. Finally, while the CODA makes frequent reference to mental health issues, some of the questions refer to just “discrimination” and so it is possible that respondents take into account other forms of discrimination.

Supplementary material 1 (PDF 148 kb)