Clinical Validity of Subjective Clinical Prognosis in First Episode Psychosis Schizophrenia Patients: An Analysis of Data from the European First Episode Schizophrenia Trial (EUFEST) Study.

PURPOSE: This study examined the validity of subjective clinical prognosis (SCP), a commonly used clinical tool, in first episode psychosis patients included in the European First Episode Schizophrenia Trial (EUFEST) study. PATIENTS AND METHODS: The study comprised 455 patients from the EUFEST trial (mean age 25.92, SD=5.45; 188 (41.31%) women, 267 (58.69%) men). SCP was classified into three mutually exclusive groups: "good prognosis" (GP) (n=265), "average prognosis" (AP) (n=131), and "poor prognosis" (PP) (n=59). The validity of the SCP was assessed by investigating the differences between the SCP groups and completer or responder status of the patients, during 1 year of the trial.
RESULTS: The proportion of completers was significantly higher in the GP group (64.4%) compared to the AP group (25.6%) (OR=1.62, 95% CI=1.062-2.476, p<0.031) and the PP group (10%) (OR=2.17, 95% CI=1.226-3.853, p<0.009) throughout the whole duration of the trial. In what concerns responsiveness, a significantly higher number of responders were registered in the GP group compared to the AP and the PP groups in the first three months of treatment, but this outcome did not persist afterwards.
CONCLUSION: In terms of its predictive value at first episode schizophrenic patients, SCP seems to be reliable for treatment completion, but has a limited utility in what concerns responsiveness to treatment. This finding suggests the necessity of creating a prediction model potentially including, besides SCP, other measurement-based variables.

Introduction

Prognosis in Medicine represents a substantially important concept for all stakeholders involved in the medical process: patients, patients’ families, clinicians, and policymakers.1–5 In the current medical practice, dominated by the shift to a veritable patient-oriented industry,6 prognosis has also changed, becoming increasingly based on objective determinants.7–10 Although the imperatives of “efficacy, effectiveness and efficiency”11 are also prevalent in the domain of mental health, measure-based prognostic instruments are not commonly used in the routine management of patients.7,8 Subjective clinical prognosis (SCP) is still being frequently utilized, possibly because of the high importance played by individual cognitions,9 emotions,10 or the scientific and therapeutic culture where the clinician developed professionally.12–15 Personal clinical experience in Psychiatry plays an important role16 and could, in its turn, impact the SCP.

Knowing more about SCP’s influence on the clinical psychiatric judgment process is important for improving the quality of care. Once established, SCP may influence planning and the allocation of health services, decisions regarding the duration and intensity of treatment, and the type of communication developed with patients and their families,17 particularly in severe conditions, such as schizophrenia.18,19 Moreover, the decisions related to the treatment of mental or comorbid somatic diseases may be influenced by SCP.18 In this sense, psychiatrists may be inclined towards expressing less hope to patients and their families, if the SCP is poor.20,21

Examining the validity of SCP in such circumstances could be very informative, especially in first episode psychosis patients, where the prognosis may have high variability, from rapid and stable remission to severe and chronic deterioration of health status. In this respect, current literature is scarce, with virtually no trials having investigated the validity of SCP in this particular group of patients. Considering this knowledge gap, the aim of the current study was to assess SCP and its predictive value in first episode schizophrenia patients being part of a large trial, with the twofold intention to collect valuable scientific information and to offer valuable cues for using this prognostic tool in the current care. In this study, we focused on data gathered from EUFEST - the first independently designed trans-European schizophrenia treatment trial. The choice of this trial was largely determined by its high representative potential (due to the large number of subjects involved, its long duration, and multicentricity).22 Additionally, the conditions for running the EUFEST trial resemble the general clinical circumstances in Europe, making the conclusions of this study relevant for European specialists in mental health, and having potential applicability in local care facilities as well.

Patients and Methods

To assess SCP and its clinical validity in first episode schizophrenia patients, we examined the concordance between the patients’ SCP at the baseline and the patients’ outcome during 1 year of the trial. The analysis was separately conducted for the two criteria considered relevant in the SCP: the number of patients completing the trial and their responsiveness to the treatment.

The EUFEST Trial Design

The European First Episode Schizophrenia Trial (EUFEST)23 was a randomized, open treatment trial comparing the effectiveness of five antipsychotics (haloperidol, amisulpride, olanzapine, quetiapine and ziprasidone) in 498 patients with a first episode of psychosis. A total of 50 centers located in 13 European countries and Israel participated in the trial. To be eligible for inclusion in the EUFEST, patients (18–40 years of age) had to meet the Diagnostic and Statistical Manual of Mental Disorders (4th Edition)24 criteria for schizophrenia, schizophreniform, or schizoaffective disorder, as confirmed by the Mini International Neuropsychiatric Interview Plus.25 To expand study generalizability, the EUFEST trial also included patient groups that are often excluded in efficacy trials (ie, patients with substance misuse and patients who could not participate in blinded trials, for safety or other reasons). The primary outcome measure was all-cause treatment discontinuation, while the continuation of treatment across 12 months was the secondary outcome. After randomization, the patients were evaluated bi-weekly for the first 3 months, and then at 3, 6 and 9 months of treatment.

Detailed data regarding the design and methods of the EUFEST trial were reported previously.23,26,27

Current Study

In the current study, we included all patients meeting the EUFEST inclusion criteria, diagnosed with schizophrenia, schizophreniform, and schizoaffective disorder diagnosis and experiencing their first psychotic episode and about whom we had valid information at the baseline.

Participants

From the 498 patients included in the EUFEST trial,28 455 with valid data at baseline were recruited for the study (Table 1). They were divided, according to their initial evaluation at baseline, into three distinct SCP groups: good prognosis (GP), average prognosis (AP) and poor prognosis (PP). At the moment of their inclusion in the EUFEST trial, all study participants received information on the study objectives and an invitation to participate. Decision to participate in the study was endorsed through a written informed consent form. The trial complied with the Declaration of Helsinki and was approved by the Institutional Review Boards of participating centers. The Julius Centre for Health Sciences and Primary Care monitored the trial, according to the ICH-GCP guidelines.29

Table 1

Socio-Demographic Factors and Affiliation to Prognosis Groups

Factors		Good Prognosis (GP) N = 265	Average Prognosis (AP) N = 131	Poor Prognosis (PP) N = 59
Agea		25.86 (5.29)	26.08 (5.82)	25.81 (5.39)
Years of education		12.97 (2.79)	12.04 (3.06)	11.29 (2.39)
Gender	Men	145	81	41
Marital status	Married	36	15	6
Occupation	Yesb	146	48	19
Living status	Family/others	231	112	52

Notes: aMean (standard deviation); baccording to the seven occupations listed in the EUFEST trial protocol (“High executive, major professional”, “administrative personnel, minor professional”, “sales, technician, farmer”, “skilled manual employee”, “unskilled employee”, “student”, “home-maker”, “unemployed”).

Data Collection and Instruments

Baseline data included information about (a) the socio-demographic characteristics of patients (age, gender, years of education, marital status, occupation and living status); (b) their clinical status (symptoms, global assessment of functioning, depression, quality of life), and (c) their cognitive functioning. To collect data pertaining to (b), the following instruments were applied: Positive and Negative Syndrome Scale [PANSS],30 Global Assessment of Functioning [GAF] scale,31,32 Calgary Depression Scale for Schizophrenia [CDSS],33 Manchester Short Assessment of Quality of Life scale (MANSA).34 For (c), a battery of additional tests were applied, including Trail-Making A (TMA) and Flexibility Index, also known as Trail Making-B (TMB),35 Wechsler Adult Intelligence Scale III-The Digit-Symbol Coding (WAIS-III),36 Purdue pegboard test,37 evaluating motor speed and motor coordination, Rey Auditory Verbal Learning Test – learning index (RAVLT-SC) and Rey Auditory Verbal Learning Test – secondary memory-delayed recall (RAVLT-DC).38 All these tests were administered again at 1, 3, 6, 9 and 12 months, according to the EUFEST protocol.

Procedure

Subjective clinical prognosis (SCP) was evaluated at Baseline (0 months), by assigning each patient to one of the following categories: “Best”, “Good”, “Above average”, “Average”, “Below average”, “Bad”, and “Worst”, as recommended by the EUFEST protocol. Subsequently, as the number of patients categorized in certain groups was too low to allow a reliable statistical analysis, the first three groups (“Best”, “Good”, and “Above average”) and the final three groups (“Below average”, “Bad”, and “Worst”) were merged into two new categories, labeled “Good prognosis” (GP) and “Poor prognosis” (PP). The “Average prognosis” group suffered no changes (AP).

In order to fulfill the objectives of the study, affiliation to the GP, AP and PP prognosis groups was considered the independent variable, having a binary nominal value of Yes or No, while responsiveness and study completion were the dependent variables. Responsiveness was measured according to the PANSS manual,30 with tests performed at baseline, 1, 3, 6, 9, and 12 months after the treatment initiation. Considering that the minimum PANSS score is 30, the percentage of reduction in symptom scores was calculated using the following formula: percentage of reduction in symptom scores = [(PANSSbaseline − 30) – (PANSSfollow-up − 30)] × 100/(PANSS baseline − 30). Patients who improved by at least 50% from baseline were considered to be responders and the remaining patients, non-responders.39 Regarding completion status, patients who were able to reach the end of the 12-month period were considered “study completers”, while those who discontinued treatment were labeled as “non-completers”. Both responsiveness and completion statuses were interpreted as binary nominal values (Yes or No), reflecting the patient status in each group.

Statistical Analysis

To measure SCP validity, we tested the associations between the prognosis groups (GP, AP, and PP), as independent variables, and the status of completer and responder, respectively, as dependent variables, using Chi-square tests, odds ratio (OR) and confidence intervals (CI). All statistical tests were run at a level of statistical significance p< 0.05.

Results

Among the 455 patients considered for the study [GP: 265 (58.2%), AP: 131 (28.8%), PP: 59 (13%)], 253 (55.6%) were completers and 202 (44.4%) non-completers. In the completers group, 163 patients (64.4%) had been assigned to the GP group, 65 (25.6%) to the AP group, and 25 (10%) patients to the PP group. The number of completers was significantly higher in the GP group compared to the AP group (OR = 1.62, 95% CI = 1.062 to 2.476, p =0.03). This statistical significance was also met when comparing GPs with PPs (OR = 2.17, 95% CI = 1.226 to 3.853, p = 0.009). This suggests that, in terms of completeness, SCP shows a good predictive value for treatment follow-up

Treatment responsiveness was asserted if a patient had a PANSS score reduction higher than 50% at the end of the 12-month trial. Two hundred and sixty-five patients were considered responders and 190 non-responders (Table 2). When comparing the GP group and the AP group, there were significant differences in the 1st (OR = 1.55, 95% CI = 1.08 to 2.22, p =0.01) and 3rd months of treatment (OR = 1.43, 95% CI = 1.15 to 1.78, p =0.001), whereas when comparing the GP group and the PP group, there were significant differences in the 1st (OR = 3.40, 95% CI = 1.56 to 7.40, p =0.001), 3rd (OR = 1.43, 95% CI = 1.04 to 1.96, p =0.01), 9th (OR = 1.84, 95% CI = 1.06 to 3.20, p =0.04) and 12th (OR = 1.45, 95% CI = 1.11 to 1.90, p =0.001) months of treatment. When comparing the AP and the PP groups, significant differences were observed only for the 12th month of treatment (OR = 1.34, 95% CI = 1.01 to 1.78, p =0.02). These results suggest that in terms of responsiveness, the SCP can be considered reliable in the first 3 months of treatment; however, clinicians should not extensively rely afterwards on this instrument and complement it with other clinical tools for patient evaluation.

Table 2

Associations Between Prognosis Groups and the Number of Responders, by Visit

Visit (Month)	Responders (n)			Significance of Inter-Group Comparisons
	GP	AP	PP	Statistic	GP vs AP	GP vs PP	AP vs PP
1	90	29	6	OR	1.55	3.40	2.19
				CI	1.08–2.22	1.56–7.40	0.96–4.99
				p	0.01	0.001	0.06
3	157	52	23	OR	1.43	1.43	1.00
				CI	1.15–1.78	1.04–1.96	0.69–1.43
				p	0.001	0.01	1.00
6	190	87	35	OR	1.07	1.12	1.04
				CI	0.97–1.18	0.95–1.31	0.87–1.23
				p	0.15	0.08	0.64
9	154	70	26	OR	1.11	1.84	1.13
				CI	0.95–1.28	1.06–3.20	0.86–1.48
				p	0.15	0.04	0.43
12	168	73	24	OR	1.08	1.45	1.34
				CI	0.95–1.22	1.11–1.90	1.01–1.78
				p	0.20	0.001	0.02

Abbreviations: OR, odds ratio; CI, confidence interval; p, statistical significance; SCP, subjective clinical prognosis; GP, good prognosis; AP, average prognosis; PP, poor prognosis; EUFEST, European schizophrenia treatment trial; PANSS, Positive and Negative Syndrome Scale; GAF, Global Assessment of Functioning; CDSS, Calgary Depression Scale for Schizophrenia; MANSA, Manchester Short Assessment of Quality of Life scale; TMA, Trail Making A; TMB, Trail Making-B; WAIS-III, Wechsler Adult Intelligence Scale III-The Digit-Symbol Coding; RAVLT-SC, Rey Auditory Verbal Learning Test – learning index; RAVLT –DC, Rey Auditory Verbal Learning Test – secondary memory-delayed recall.

Discussion

This study assessed SCP and its clinical validity in first episode schizophrenia patients. Concerning this objective, clinicians seemed to be more optimistic than expected, with the majority of patients being assigned to the “Good prognosis” (GP) group (58.2%) and “Average prognosis” (AP) (28.8%) group. The characteristics of the study population in the EUFEST patients (generally young, with first episode psychosis, without substance use disorder, antipsychotic naive or treated less than 2 weeks with antipsychotic medication) may have contributed to directing many clinicians towards an optimistic prognosis. Although the outcome of these patients may not be favorable, only 13% of patients were considered to have a poor prognosis (PP) at baseline. The patients included in the GP group were mostly females, employed, better educated, had lower PANSS scores and better cognitive functioning, quality of life and global functionality.

Regarding SCP validity per se, this has been separately assessed in two directions: completeness and responsiveness to treatment. Concerning completeness, this can be asserted at the end of the 12-month period, as patients in the GP group outperformed patients in both AP and PP groups, and patients in the AP group outperformed those in the PP group. Still, regarding responsiveness, significant differences across the whole trial were only met between the GP and the PP groups; in contrast, they were limited only to the first three trial months, when comparing the GP and the AP groups and only to the last evaluation (at the 12th month), when comparing the AP and the PP groups Taking into account that discriminative value is an important factor in the assessment of the validity of a prognostic tool,40 we can conclude that SCP has a good validity in predicting the completer status, but has a limited validity in predicting the responder status of first episode schizophrenic patients.

Regarding this study outcome, one element which needs to be considered is represented by the impact of other factors on responsiveness, which could account for a part of its fluctuations. Variables such as the effects on responsiveness brought by a certain type of psychosocial intervention, or by the unstable functionality of the family climate, are often impossible to be anticipated by the clinician at the time of the first encounter with the patient. As a result, the idea of a low validity of the SCP, in what concerns responsiveness to treatment, should be regarded with caution. The interactions between SCP and other confounding factors could be better displayed and understood only when analyzing treatment completeness and responsiveness across a longer period of time. This kind of research could also draw more refined conclusions on the real opportunity of using outcomes like continuing treatment or responsiveness to evaluate the prognosis of first episode schizophrenia patients.

Limitations

Our study has a number of limitations. The generalizability of results can be influenced by the procedure of patient inclusion (eg, some clinicians may have had pre-visits with patients, but this was not covered in the EUFEST protocol and it was not considered a restriction in the inclusion criteria). Regarding SCP, an additional limitation is that we cannot be sure of the moment during the baseline visit when the clinician has decided on a certain prognosis. It is also possible that SCP was influenced by the format of the clinical interview performed by the psychiatrist, as a result of the patient’s cognitive status, or other co-existing symptoms (eg, depression). In our study, we did not analyze the impact of the investigator’s clinical experience on the prognosis assessment, nor the distinct influences of country-specific training, administrative requirements, or clinical work conditions, as all these elements were not captured in the EUFEST protocol.

Conclusion

Our study contributes to the construction of a more solid assessment of the SCP as an extensively used clinical instrument for first episode schizophrenic patients. Over a span of 1 year, SCP appears to be more valid in predicting the completer status of first episode schizophrenic patients, than their responsiveness to treatment. Further research, potentially extended on a longer time and including other variables, could test the validity of new models for decision-making in schizophrenia, with the goal of offering a better quality of care.