The utility of novel outcome measures in a naturalistic evaluation of schizophrenia treatment.

BACKGROUND: A number of naturalistic studies have investigated paliperidone palmitate (PP) using proxy measures of effectiveness. An unexplored option is to examine the utility of the mental health clustering tool (MHCT), which is used in UK clinical practice to measure patient well-being and is linked to allocation of resources. This study evaluated the effectiveness of PP using the MHCT, the Health of the Nation Outcome Scales (HoNOS), and, for comparison, more conventional outcome measures.
METHODS: This was a naturalistic, 1-year evaluation of PP (n=50) in schizophrenia as well as a comparator antipsychotic drugs group. Changes in the MHCT cluster-score cost ranking and four HoNOS-derived factors were analyzed using a mixed-model statistical analysis to explore the utility of these measures.
RESULTS: At 1 year, 30 patients (60%) continued PP treatment. The mean "cluster-score cost ranking" (-1.5) and Severe Disturbance factor scores (-1.1) were significantly lower (p-value [adjusted] =0.0003, p-value [adjusted] =0.002, respectively) after 1 year of antipsychotic treatment but no differences were found between PP and the comparator antipsychotic drugs group. Patients prescribed PP were 1.8 times (95% CI 1.1-3.1) more likely to be discharged from hospital than those in the comparator antipsychotic drugs group.
CONCLUSION: PP's continuation rate after 1 year made the study similar to the existing evaluations, and it was possible to prospectively evaluate antipsychotic effectiveness using the novel measures although these did not discriminate between PP and the comparator group. The investigation illustrates that in principle these novel measures are meaningful in naturalistic study designs.

Plain language summary

We tried a new way of finding out if people with schizophrenia get better with medicines. We asked doctors to tell us when people were prescribed a medicine called paliperidone palmitate (PP). This medicine is for people with schizophrenia. When doctors answered special questions about how well people were doing on this medicine, we got a copy of the answers too. After a year, we compared answers for the medicine PP with answers for other medicines. We found that all medicines were making people feel better after a year. Our new way of finding out if people with schizophrenia get better is good. We also found that people with schizophrenia given PP in hospital were sent home sooner than people given other medicines.

Introduction

Paliperidone palmitate (PP) is a long-acting antipsychotic injection that was licensed for the maintenance treatment of adult patients with schizophrenia in 2009 in the USA and 2011 in Europe. A number of naturalistic studies have investigated PP in clinical practice primarily using proxy measures of effectiveness. An early UK example is the year-long follow-up of PP which utilized discontinuation rates, hospital bed days, and admission rates pre- and post-initiation of PP.1,2 Other papers (including clinical trials) have also reported on PP by measuring discontinuation3–5 and/or hospitalization (eg, readmission rates and number of bed days),3,6–11 as well as time to treatment failure12–14 as markers of clinical effectiveness. Another hitherto unexplored option for reporting on the clinical effectiveness of PP in the UK is the use of the mental health clustering tool (MHCT) that incorporates an assessment of patient well-being and is used in routine clinical practice.

The MHCT is a tool used to assess the needs of patients with mental illness. It provides a “global description of a group of people with similar characteristics as identified by a holistic assessment”,15 allowing patients to be “clustered” into groups with the use of a set of question items that can be scored from 0 to 4. There are 21 care clusters of which seven are for psychosis (Table S1). The MHCT is in fact linked to the allocation of resources through Payment by Results (PbR), and in future the National Tariff prices, within the Quality and Outcome Framework (QoF) system of commissioning and funding mental health care in the UK.16 It follows therefore that measuring whether patients move from one care cluster to another (when associated with indicative costs) could be an indication of cost-effectiveness, which is explored as a primary outcome measure of the effectiveness of PP in a year-long naturalistic setting in this study. The “four factors” described below were also incorporated into the current study.

The MHCT incorporates the Health of the Nation Outcome Scales (HoNOS) (Table S2). Speak and Hay17 demonstrated that four clinically meaningful factors could be derived from HoNOS that were sensitive to change over time and could be used within the PbR QoF for working age and older adult mental health services, namely, Personal Well-Being, Emotional Well-Being, Social Well-Being, and Severe Disturbance. The current study sought to include the score for each of these four factors as part of the evaluation of PP effectiveness. This is the first evaluation that uses the MHCT to assess the clinical effectiveness of the introduction of an antipsychotic in clinical practice.

Finally, patient-reported outcomes, including treatment satisfaction, are considered to be important in the assessment of therapeutic interventions, and a link has been shown between treatment satisfaction and an improvement in symptoms and adherence rates in schizophrenia.18 Thus, in line with other studies measuring patient satisfaction with PP,11,19 the medication satisfaction questionnaire (MSQ) was employed as the secondary outcome measure in this study.

Aim of the study

The aim was to evaluate the effectiveness of PP by measuring and comparing (and therefore examining the utility) the MHCT cluster scores using their indicative costs, the four-factor scores derived from the HoNOS, and MSQ scores at baseline and 12 months. We utilized a comparator antipsychotic drugs group and also included hospital bed days and continuation rates as part of our study in line with other studies.

Methods

Compliance with ethical standards

This study was approved by the University of Reading’s Research Ethics Committee through the School Exemptions process reference number 10/13, on August 16, 2013.

This was a naturalistic, prospective service evaluation of PP prescribed for people diagnosed with schizophrenia in Berkshire Healthcare NHS Foundation Trust (BHFT). The decision to prescribe PP was made by a consultant psychiatrist, in consultation with a multidisciplinary team, without the involvement of the study team. The prescriber completed paperwork to indicate the patient’s concurrent medication, general practitioner (GP) contact details, inpatient/outpatient status, diagnosis, and cluster score, which was passed on to a pharmacist coinvestigator (Kate Masters). After initiation of PP, patients followed their usual care pathway. Along with the initial baseline (T0) cluster data, if a patient continued with the treatment with PP at 6 months (T6) and 12 months (T12) further cluster data were inputted into electronic patient records by the care coordinator or a nurse. No separate consent was required for the study which was judged to be a service evaluation and approved by the Trust Clinical Audit Department on February 4, 2015. All adverse effects were recorded and reported to the manufacturer.

Each eligible prescription of PP was recorded on a secure, anonymized spreadsheet, and a researcher (Tamara Tompsett) used the electronic patient medical records to reconcile demographics to include age, gender, ethnicity, diagnosis, and details of the previous 12 months of antipsychotic use against each entry at T0. The prescriptions were monitored via electronic patient records for up to 12 months after initiation of PP. Once a patient was stabilized and discharged from the hospital, responsibility for prescribing medicine was generally passed to their GP although responsibility for monitoring therapy and completion of evaluation documents remained within BHFT. If PP treatment was stopped before T12, the data collected up to that point were entered into the study spreadsheet for statistical analysis. The reason for discontinuation and, if applicable, any new antipsychotic medication prescribed were also recorded. Readmission rates and length of inpatient stay(s) from T0 to T12 and for the previous 12 months were also recorded in order to calculate the mean number of bed days pre- and post-PP prescribing and to calculate time to discharge (TTD) at treatment initiation. For this element, the “index admission” was included in the analyses such that the “mirror” as described by Taylor et al9 was placed at the point of PP initiation.

By June 2015, a total of 78 patients were initiated to PP of which 50 were diagnosed with schizophrenia at the point of initiation. A primary outcome measure was the change in the MHCT cluster-score cost ranking at T12 compared to T0. To derive the cost ranking, clusters 10–17, relevant to patients with psychosis, were ranked from low to high cost using the expected reference cost (their predicted daily cost of care)20 (Table S3). Thus, for example, cluster 11 “ongoing recurrent psychosis (low symptoms)” which was the “cheapest” cluster (cost per day £15.22) was ranked first, while cluster 14 “a psychotic crisis”, the most “expensive” cluster (cost per day £86.23), was ranked eighth.

Another primary outcome measure was change in the four HoNOS-derived factors at T12 compared to T0. The score for each of the items of HoNOS was first retrieved from the original MHCT completed by the health professional assigning the care cluster. Then the individual scores for the items grouped by Speak and Hay17 to form that factor were aggregated on the study spreadsheet to derive the factor score. Thus, for example, for Personal Well-being, the factor score for each case was calculated by summing the scores assigned to each of items 4, 5, 10, and 12 and for Severe disturbance, the factor score for each case was calculated by summing scores assigned to each of items 1, and 6.

The secondary outcome measure was a change in patient medication satisfaction from T0 to T12 measured using the MSQ.21 The prescriber (and later, the care coordinator or a nurse) read out aloud the following question to the patient: “Overall, how satisfied are you with your current antipsychotic medication(s)? 1= extremely dissatisfied, 2= very dissatisfied, 3= somewhat dissatisfied, 4= neither satisfied nor dissatisfied, 5= somewhat satisfied, 6= very satisfied, or 7= extremely satisfied”, before circling the patient’s response on a sheet which was later returned to the researcher (Tamara Tompsett) for entry onto the study spreadsheet.

Trust approval was given to collate anonymized data on a comparator antipsychotic drugs group using information retrieved from the electronic patient database for any inpatient from two acute wards at BHFT between January 2014 and April 2014 with a diagnosis of schizophrenia using the F200 as a search term (from the International Statistical Classification of Diseases and Related Health Problems, ICD 1022). This was a convenience sample chosen to match the first 4 months of PP prescribing at the Trust. Patients were excluded from the comparator group if they were treatment resistant, had changed diagnosis, or were assigned to PP during their admission. Patients were included in the comparator group if there were sufficient HoNOS data available within the electronic patient database over a continuous period of antipsychotic medication for 6 or 12 months (within 15% either side of these time points).

A mixed model was used to analyze the primary and secondary outcomes with “patient” treated as a random effect. Mixed models generalize the linear model methods to account for the presence of a hierarchical structure in the study and allow retention in the analysis of subjects with missing outcomes. A total of 49 patients (one patient died and was excluded from the analysis) with schizophrenia receiving PP were compared against 30 patients in the comparator group (with three being included twice to give 33 outcome measure points) for the primary outcomes. Reinclusion in the PP group was allowed. The MSQs were examined for the PP group only. The following fixed effects were assessed: gender, age, age by gender interaction, time point (0, 6, and 12 months), and drug, and time point by drug. The repeated measurements nature of the records was modeled using an unstructured covariance pattern.

A mixed model for numerical outcomes was used for all four-factor outcomes. The effect measure used in this model was the absolute difference in mean of aggregate scores between evaluation at T0 and evaluation at T12.

A significance level of 5% was used and 95% CI for the true effect measure quantified. As multiple outcomes were analyzed, an adjustment for multiplicity was applied. Adjustments for multiplicity were made using the Holm method for p-values and the Bonferroni method for confidence intervals. The package SAS 9.3 was used to fit the mixed models defined above. A Cox proportional hazards regression was used to model the TTD days for comparing PP to those in the comparator group. Descriptive statistics were used to summarize patient and medication characteristics. Means and standard deviations were calculated for continuous data and percentages for categorical data.

Results

Table 1 compares the demographic profile of 50 people with schizophrenia who continued/discontinued on PP at T6 and T12. Of these, 20 (40%) had discontinued (or were lost to follow up) by T12. This was because 8 (40% of this group) declined to receive a long-acting injection (LAI) with a preference for oral medication, 3 (15%) had raised prolactin, 5 (25%) were due to perceived ineffectiveness (2 of which had ongoing tardive dyskinesia), 1 (5%) experienced drowsiness, 1 (5%) stopped all medications, 1 (5%) was lost to follow up, and 1 (5%) died due to a non-PP related cause. It is important to explain that the cause of death for the patient who died was judged by a coroner to have been due to natural causes not associated with PP use. One patient (5%) was lost to contact after initiation of PP.

Table 1

Demographic profile of 50 people with schizophrenia prescribed PP

Population	Total (N=50a)	Continuation at 6 months (n=35)	Discontinuation before 6 months (n=14)	Continuation at 12 months (n=30)	Discontinuation before 12 months (n=20)
Age (years)
Mean (± SD)	42 (13)	41 (13)	42 (13)	42 (14)	41 (14)
Gender, n (%)
Male	38 (76)	27 (71)	11 (29)	24 (63)	14 (32)
Female	12 (24)	8 (67)	3 (25)a	6 (50)	5 (42)a
Ethnicity, n (%)
Black	2 (4)	0 (0)	2 (100)	0 (0)	2 (100)
White	28 (56)	20 (71)	7 (25)a	17 (61)	10 (36)a
Otherb	19 (38)	15 (79)	4 (21)	13 (68)	6 (32)
Unknown	1 (2)	0 (0)	1 (100)	0 (0)	1 (100)
Care setting, n (%)
Inpatient	47 (94)	32 (68)	14 (30)a	27 (57)	19 (40)a
Outpatient	3 (6)	3 (100)	0 (0)	3 (100)	0 (0)
Treatment 12 months prior to PP, n (%)
Risperidone oral/LAI	24 (48)	17 (71)	6 (25)a	15 (63)	8 (33)a
Risperidone oral/LAI and other drug	9 (18)	7 (78)	2 (22)	6 (67)	3 (33)
None	2 (4)	1 (50)	1 (50)	1 (50)	1 (50)
Others	15 (30)	10 (67)	5 (33)	8 (53)	7 (47)
Comorbid substance use, n (%)
Yes	17 (34)	12 (71)	5 (29)	11 (65)	6 (35)
No	33 (66)	23 (70)	9 (27)a	19 (58)	13 (39)a

Notes:

One lost to follow up (65 years, female, white) and one patient died, therefore percentages do not total 100% due to missing data.

Other ethnicity included 13 Asian or Asian British.

Abbreviations: LAI, long-acting injection; PP, paliperidone palmitate.

Characteristics of patients treated with PP

Patients were aged between 22 and 65 years (mean 42 years). The majority of patients (n=33) were switched from risperidone (oral or LAI) before starting PP (Table 2). Those who stopped PP at T6 (n=14) were switched to clozapine (n=3), aripiprazole (n=3), oral risperidone (n=2), or other antipsychotics (n=4). At T12, further 3 patients were switched to clozapine (total n=6), another to oral risperidone (total n=3), and one patient was discharged to their GP and declined to receive any further medication.

Table 2

MSQ descriptive statistics relating to paliperidone palmitate

Descriptive statistics	T0 (n=50)	T6 (n=35)	T12 (n=30)
Completed MSQ, n (%)	38 (76)	13 (37)	22 (74)
Not completed MSQ
Lack of insight, n (%)	5 (10)	n/a	4 (13)
Refused, n (%)	1 (2)	n/a	n/a
Missing data, n (%)	6 (12)	22 (63)	4 (13)
Least squares mean	3.8 (0.4)	5.3 (0.5)b	5.1 (0.4)c
MSQ score (SE)
Rangea	1–7	2–7	3–7
Minimum value 1, n (%)	8 (21)	2 (15)	3 (14)
Maximum value 7, n (%)	3 (8)	3 (23)	1 (5)

Notes: T0, initial baseline; T6, treatment with PP at 6 months, and T12, treatment with PP at 12 months.

Range 1 (extremely dissatisfied) to 7 (extremely satisfied).

T6, p-value (adjusted) =0.0045, difference =1.5 (SE 0.47).

T12, p-value (adjusted) =0.0043, difference =1.34 (SE 0.4).

Abbreviations: MSQ, Medication Satisfaction Questionnaire; n/a, not applicable; PP, paliperidone palmitate; SE, standard error.

Comparator group patients’ characteristics

Patients of the comparator group were aged between 22 and 68 years (mean 42 years). The data related to 20 male patients (67%), 3 with data that were used twice as they switched antipsychotics, and 10 female patients (33%). The antipsychotics prescribed in the comparator group included aripiprazole (n=6), flupentixol decanoate (n=3), haloperidol (n=1), olanzapine (n=7), zuclopenthixol (n=6), trifluoperazine (n=1), pipotiazine palmitate (n=3), risperidone (n=4), and quetiapine (n=2).

Analysis of MHCT cluster-score cost ranking

The mean cluster-score cost ranking at T0 was 4.2 (standard error [SE] 0.4) and lower at T6 with a mean of 2.9 (SE 0.5) and at T12 with a mean of 2.5 (SE 0.5) for PP. A similar trend was observed in the comparator antipsychotic drugs group. This downward trend in the mean cluster-score cost ranking was statistically significant for T0 versus T6 (p-value [adjusted (adj)] =0.01, difference =−1 [SE 0.4]) and T0 versus T12 (p-value [adj] =0.0003, difference =−1.5 [SE 0.4]). There were no statistically significant differences between PP and the other antipsychotic drugs group in terms of the pattern observed for reduction in the cluster-score cost ranking (Figure 1).

Figure 1

Line graph to compare the least squares mean for the cluster score data of paliperidone palmitate and the other antipsychotic drugs group ranked by monetary costs per day (1 lowest cost to 7 highest cost).

Notes: Paliperidone palmitate (T0, n=48; T6, n=34; and T12, n=30) and other antipsychotic drugs group (T0 n=32, T6 n=21, and T12 n=28). T0, initial baseline; T6, treatment at 6 months; and T12, treatment at 12 months. aT0 versus T6, p-value (adj) =0.01, difference =−1 (SE 0.4). bT0 versus T12, p-value (adj) =0.0003, difference =−1.5 (SE 0.4).

Abbreviations: adj, adjusted; SE, standard error.

Analysis of the four HoNOS-derived factors

The mean Severe Disturbance factor score at T0 was 3 (SE 0.4) for PP and lower at T6 with a mean of 2.2 (SE 0.4) and again at T12 with a mean of 2.2 (SE 0.4). A similar trend was observed in the comparator antipsychotic drugs group. This downward trend in the Severe Disturbance factor score was statistically significant for T0 versus T6 (p-value [adj] =0.0032, difference =−0.85 [SE 0.26]) and T0 versus T12 (p-value [adj] =0.002, difference =−1.1 [SE 0.3]). There were no statistically significant differences between PP and the other antipsychotic drugs group in terms of the pattern of reduction in the Severe Disturbance factor score (Figure 2).

Figure 2

Clustered column chart to compare the least squares mean for the aggregated scores (0–8) of Severe Disturbance for paliperidone palmitate and the other antipsychotic drugs group.

Notes: Severe Disturbance, containing overactive, aggressive, disruptive, or agitated behavior and problems associated with hallucinations and delusions (scales 1 and 6 from HoNOS). Paliperidone palmitate (T0, n=48; T6, n=34; and T12, n=30) and other antipsychotic drugs group (T0, n=32; T6, n=21; and T12, n=28). T0, initial baseline; T6, treatment at 6 months; and T12, treatment at 12 months. aT0 versus T6, p-value (adj) =0.0032, difference =0.85 (SE 0.26). bT0 versus T12, p-value (adj) =0.002, difference =1.1 (SE 0.3).

Abbreviations: adj, adjusted; HoNOS, Health of the Nation Outcome Scales; SE, standard error.

There were no statistically significant reductions in the aggregated mean scores for the Personal Well-being, Emotional Well-being, or Social Well-being factors from T0 to T6 and T12 and no statistically significant differences in the pattern of these scores for PP compared to the other antipsychotic drugs group.

Analysis of the MSQ scores

The mean MSQ score for PP was significantly higher at T6 and T12, indicating a statistically significant improvement in patient medication satisfaction on use of PP. The mean MSQ score at T0 was 3.8 (SE 0.4), which increased to 5.3 (SE 0.5) at T6 and to 5.1 (SE 0.4) at T12. The differences in mean MSQ scores were statistically significant for T0 versus T6 (p-value [adj] =0.0045, difference =1.5 [SE 0.47]) and T0 versus T12 (p-value [adj] =0.0043, difference =1.34 [SE 0.4]) (Table 2).

Medication satisfaction increased in all 76% of the patients who completed an MSQ.

Mean number of bed days

There was a slight reduction in the mean number of bed days for the 30 patients who received PP up to T12 compared with the previous 12 months when they were on a different drug. The mean number of bed days was 63.4 for the 12 months before PP initiation compared with 61.6 days for the 12 months post PP initiation. This equates to a total reduction of 2.8% in the number of inpatient bed days for the 12 months after initiation of PP.

TTD

Of the 50 patients prescribed PP, data for 34 patients were available to calculate the TTD compared to 27 available data sets of hospital admissions in the comparator group (Figure 3). This was because n=9 discontinued prior to their discharge, n=4 were outpatients at time of prescribing, n=2 had no records, and n=1 was in residential care.

Figure 3

Cox PH regression showing the time to discharge of patients prescribed either PP (n=34) or those in the other antipsychotic drugs group (n=27).

Abbreviations: PH, proportional hazards; PP, paliperidone palmitate; PPLAI, paliperidone palmitate long-acting injection.

The hazard for PP compared to the other antipsychotic drugs group is estimated to be 1.8 with 95% CI of 1.1−3.1. In other words, at any time point within the range of observed days to discharge (0.5–447) a patient on PP was 1.8 times more likely to be discharged than a patient on other antipsychotics. The median number of days to discharge for patients on PP was 41 days compared to 63 days for the comparator antipsychotic drugs group.

Discussion

In this year-long follow-up, 60% of 50 patients continued treatment with PP, which made the study similar to the existing evaluations on this measure. Using a mixed-model statistical analysis we found that the mean MHCT cluster-score cost ranking and the HoNOS-derived Severe Disturbance factor score were each significantly lower after 1 year of treatment with either PP or the comparator antipsychotic drugs group. There were no statistically significant reductions in the other HoNOS-derived factors. Patient satisfaction was significantly higher after 1 year of treatment with PP compared to baseline. We found a modest reduction in the mean number of bed days for patients who continued with PP in the year following treatment compared to the previous 12 months. In addition, we found that a patient on PP was 1.8 times more likely to be discharged from hospital than a patient in the comparator group of antipsychotics.

Other studies have reported similar continuation rates (65% and 60%) after 1 year of treatment with PP.1,4 In this respect, our study is comparable to existing literature although it is important to acknowledge that a 40% discontinuation rate is a limitation of this paper. The main reason for PP discontinuation in our study was patient refusal of medication followed by raised prolactin levels and perceived medication ineffectiveness. Other studies have found the reason for PP discontinuation at 1 year to be perceived ineffectiveness more frequently than the experience of side effects and refusal of treatment.1–3 In our study, perceived ineffectiveness was the reason for PP discontinuation in 25% of the cases, whereas patient refusal of treatment accounted for 40%. This difference could be due to chance and the relatively small size of our sample compared to other studies.

For those who did continue with PP for 12 months, 74% completed the MSQ, showing a statistically significant improvement in their medication satisfaction scores with treatment. A review of the literature regarding patients’ satisfaction with antipsychotic medication for schizophrenia highlighted that as well as the more obvious negative effects upon satisfaction of drug side-effects, the lack of involvement in treatment planning or decision making was a major influence on satisfaction ratings.23 The improvement we found in the mean MSQ scores at 12 months (compared to baseline scores) was 1.5 points, which is slightly higher than that reported in other studies: 0.88 points at 21 weeks19 and 0.8 points at 12 months.11 The improvement though, as well as the significant reduction found in the HoNOS-derived Severe Disturbance factor, described below, over the 12-month treatment period suggests that medication satisfaction with PP treatment is aligned to a reduction in the severity of psychotic symptoms.

PP resulted in a significant reduction in the indicative costs associated with MHCT cluster-score cost ranking as well as the HoNOS-derived Severe Disturbance factor score at 12 months, indicating benefit from treatment. This reduction was similar to a comparator group. On the one hand, these findings suggest that the MHCT- and HoNOS-related measures are useable for evaluating antipsychotic effectiveness over a 1-year treatment period. After all, those who continue treatment over a 1-year period would be expected to receive tangible benefit from their medication, and finding a reduction in these scores (and specifically Severe Disturbance by virtue of measuring illness severity) is in line with that expectation. One might infer that PP was at least as good as a multitude of other antipsychotics prescribed for the treatment of schizophrenia according to these novel outcome measures. On the other hand, it could be argued that cluster-score cost rankings and HoNOS-derived factor scores, although useable, are not sufficiently sensitive to discriminate between PP and the other antipsychotics, if such a difference exists. It is a fairly common limitation of naturalistic studies to not include a comparator group. Although our comparator group was small (n=33), it provides a useful indicator that the novel measures of this study are usable with PP as well as other antipsychotics.

One limitation of this evaluation is that the accuracy and reliability of the MHCT cluster and HoNOS scoring were accepted at face value. Across the mental health sector there has been an investment to report data based upon the MHCT in an effort to standardize practices and improve the reliability of this tool as a possible foundation for patients’ mental health funding. However, for the MHCT to be useful, the accuracy with which it is applied needs to be monitored. An error rate of 40% has been observed in an audit of 540 initial or changes to patient clusters made by health professional completing the clustering.24 Nonetheless, it is thought that clusters will remain the mandated currency for mental health care with the payment period being based upon an episode of care, not contacts or bed days.25 The 5-year forward view for mental health in the UK wishes to remove unaccountable block contracts for mental health payments, with providers not rewarded for providing days of care but initiating whole pathways of care.26 This evaluation is important as it found that compared to baseline and after 12 months of consistent antipsychotic treatment, MHCT cluster-score cost ranking and HoNOS-derived Severe Disturbance factor scores can be sensitive to change, in contrast to little change in the number of inpatient bed days over 1 year of treatment with PP when compared to the 1-year period prior to treatment.

The PP continuation rate after 1 year made the study similar to the existing evaluations, while it was also possible to prospectively evaluate antipsychotic effectiveness using the MHCT and the HoNOS as well as the MSQ. Patients who continued with PP treatment over 12 months were significantly more satisfied when compared to baseline satisfaction data relating to their previous medications. Treatment of patients with schizophrenia was shown to result in a significant reduction in the MHCT cluster-score cost ranking and a significant reduction in the HoNOS-derived Severe Disturbance factor scores over 12 months of antipsychotic treatment. The investigation illustrates that in principle these novel measures are meaningful in naturalistic study designs.

Table S1

Key to MHCT Care clusters 10–17 (“psychosis” super cluster)

Care cluster	Description and likely primary diagnosis
Care cluster 10: first episode psychosis	This group will be presenting to the service for the first time with mild to severe psychotic phenomena. They may also have depressed mood and/or anxiety or other behaviors. Drinking or drug-taking may be present but will not be the only problem. Likely to include (F20–F29) schizophrenia, schizotypal and delusional disorders, and F31 bipolar disorder.
Care cluster 11: ongoing recurrent psychosis (low symptoms)	This group has a history of psychotic symptoms that are currently controlled and causing minor problems if any at all. They are currently experiencing a sustained period of recovery where they are capable of full or near functioning. However, there may be impairment in self-esteem and efficacy and vulnerability to life. Likely to include (F20–F29) schizophrenia, schizotypal and delusional disorders, F30 manic episode, and F31 bipolar affective disorder.
Care cluster 12: ongoing or recurrent psychosis (high disability)	This group has a history of psychotic symptoms with a significant disability with major impact on role functioning. They are likely to be vulnerable to abuse or exploitation. Likely to include (F20–F29) schizophrenia, schizotypal and delusional disorders, F30 manic episode, F31 bipolar affective disorder.
Care cluster 13: ongoing or recurrent psychosis (high symptom and disability)	This group will have a history of psychotic symptoms which are not controlled. They will present with severe to very severe psychotic symptoms and some anxiety or depression. They have a significant disability with major impact on role functioning. Likely to include (F20–F29) schizophrenia, schizotypal and delusional disorders, F30 manic episode, and F31 bipolar affective disorder.
Care cluster 14: psychotic crisis	They will be experiencing an acute psychotic episode with severe symptoms that cause severe disruption to role functioning. They may present as vulnerable and a risk to others or themselves. Likely to include (F20–F29) schizophrenia, schizotypal and delusional disorders, F30 manic episode, and F31 bipolar affective disorder.
Care cluster 15: severe psychotic depression	This group will be suffering from an acute episode of moderate to severe depressive symptoms. Hallucinations and delusions will be present. It is likely that this group will present a risk of non-accidental self-injury and have disruption in many areas of their lives. Likely to include, F32.3 severe depressive episode with psychotic symptoms.
Care cluster 16: psychosis and affective disorder (high substance misuse and engagement)	This group has enduring, moderate to severe psychotic or bipolar affective symptoms with unstable, chaotic lifestyles and co-existing problem drinking or drug taking. They may present a risk to self and others and engage poorly with services. Role functioning is often globally impaired. Likely to include (F10–F19) mental and behavioral disorders due to psychoactive substance use, (F20–F29) schizophrenia, schizotypal and delusional disorders, and bipolar disorder.
Care cluster 17: psychosis and affective disorder – difficult to engage	This group has moderate to severe psychotic symptoms with unstable, chaotic lifestyles. There may be some problems with drugs or alcohol not severe enough to warrant care associated with cluster 16. This group has a history of non-concordance, is vulnerable and engages poorly with services. Likely to include (F20–F29) schizophrenia, schizotypal and delusional disorders, and bipolar.

Notes: Adapted with permission from Mental Health Clustering Booklet (V5.0) (2016/17). Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/499475/Annex_B4_Mental_health_clustering_booklet.pdf.1 The ‘F’ codes are defined by the International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10)-WHO Version for 2016.2

Abbreviation: MHCT, mental health clustering tool.

Table S2

Key to individual HoNOS items

HoNOS scales	Title of scales
Item 1	Overactive, aggressive, disruptive or agitated behavior
Item 2	Non-accidental self-injury
Item 3	Problem drinking or drug taking
Item 4	Cognitive problems
Item 5	Physical illness or disability problems
Item 6	Hallucinations and delusions
Item 7	Depressed mood
Item 8	Other mental and behavioral problems
Item 9	Problems with relationships
Item 10	Problems with activities of daily living
Item 11	Problems with living conditions
Item 12	Problems with occupation and activities
Item 13	Strong unreasonable beliefs

Note: Adapted from Health of the Nation Outcome Scales (HoNOS). Available from: https://www.rcpsych.ac.uk/pdf/HoNOS-secure%20Glossary%20v2b%20Feb%2007.pdf. ©Royal College of Psychiatrists 1996.3

Table S3

Cluster-score cost ranking according to financial outlay

Cluster score	Indicative cluster costs value/day (£)	Linear ranking
10	29.06	3
11	15.22	1
12	26.45	2
13	40.57	5
14	86.23	8
15	46.56	6
16	35.84	4
17	53.24	7

Notes: The indicative cluster costs are derived from 2011 to 2012 reference costs. NHS. Payment by Results Guidance for 2013–14; 2013. Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/232162/Mental_Health_PbR_Guidance_for_2013-14.pdf.4