Retrospective analysis of treatment effectiveness among patients in Mianyang Municipality enrolled in the national community management program for schizophrenia.

BACKGROUND: Over the last six years China has developed the largest community-based service network for persons with serious mental illness in the world (the '686 Project') but the effectiveness of this program has not been assessed in detail. AIM: Compare the characteristics of patients with schizophrenia enrolled in the program whose clinical status has improved with the characteristics of patients whose clinical status has not improved.
METHODS: The records of 3090 patients with schizophrenia in Mianyang Municipality, Sichuan (a community with 60% rural residents) who participated in the 686 Project at any time during 2011 were extracted from the national electronic registry system for the project and the demographic and treatment characteristics of individuals rated by treating clinicians as 'recovered' or 'improved' at the time of their last evaluation in 2011 (n=1866) were compared to those of patients rated as 'unchanged' or 'worsened' (n=1224). The factors considered included gender, age, ethnicity, occupation, education, family economic status, marital status, family history of mental illness, duration of illness, time of enrolled in the 686 Program, and adherence to medication.
RESULTS: In the univariate analysis there were significant differences between the two groups in all variables considered except for gender, ethnicity, and family history of mental illness. The recorded treatment outcome was better in patients who were younger, who had a shorter duration of illness, who were more educated, who came from better-off families, who were more adherent to treatment and who had participated in the program for a shorter period of time. Logistic regression analysis found that patients classified as unchanged or worsened were more likely to be non-adherent to drug treatment, to come from families living below the local poverty line, and to be enrolled in the 686 Program for a longer period of time.
CONCLUSION: Poor treatment adherence and poverty seriously limit the effectiveness of the 686 Program. New approaches to improving adherence and for providing basic financial support to families with a mentally ill member will be needed to enhance the efficacy of the program.

Introduction

Schizophrenia is a severe mental disorder with a lifetime prevalence of 7.2 per 1000 persons that usually starts between the ages of 15 and 35.[1] Due to its protracted course and disability, schizophrenia severely affects the social functioning of the individual, undermines the quality of life of family members, and results in a huge socioeconomic burden for the community.[2]

In order to standardize the management of patients with severe psychiatric diseases, China launched a nationally subsidized program for the registration and treatment of such individuals in 2005: the National Continuing Management and Intervention Program for Psychoses. This project is referred to as the ‘686 Project’ because the funding for the first year of the program was 6.86 million Renminbi. Individuals with schizophrenia, one of the most common severe mental illnesses,[3] have gradually been enrolled in the management network, which is operated by the national public health system. Over the first six years of development the 686 Project has been expanding and evolving to meet the changing needs of patients, their family members and communities. The treatment model is now standardized and data on all cases is kept up-to-date in a national electronic registry system so it is now feasible to conduct a formal assessment of the factors that are associated with successful outcomes among patients enrolled in the program.

Subjects and Methods

The 686 Program

Mianyang is a municipality of approximately 5.22 million people [4] in Sichuan Province, 60% of whom live in rural areas. The 686 Program provides services to permanent residents of the municipality with serious mental illnesses who have received treatment at specialist psychiatric hospitals, general hospitals and community health clinics throughout the municipality. Patients and their guardians identified through this service network are invited to participate in the program; those who provide written informed consent are enrolled. The enrollment of patients in the 686 Program started in 2006 but initially enrollment was quite slow. A renewed effort to expand enrollment and to standardize the services provided by the program started in July 2010. This initiative included the following steps:

Step 1: Medical records including information from a key informant screening questionnaire of mental disorders, emergency room records, and discharge summaries from clinics and community hospitals in Mianyang were scrutinized to identify individuals with serious mental illnesses.

Step 2: Psychiatric specialists reviewed the recorded data for the suspected cases and completed the ‘case finding registration form.’ If the case already had a diagnosis of schizophrenia from a specialist psychiatric service this was considered definitive, other suspected cases were individually assessed by psychiatrists who visited the individuals' homes.

Step 3: Individuals the specialists identified as suffering from schizophrenia or other serious mental disorders were registered in the system and township or community physicians then completed patients' basic personal information forms and supplemental forms and established a health file for the individual.

Step 4: The patient or family were contacted to obtain written informed consent that would allow the details of the patient's condition to be entered into the 686 data system by the township or community physicians. All baseline, treatment, follow-up, and outcome information on these patients is the regularly recorded and updated in the electronic 686 Project Management and Treatment System.

Step 5: Township doctors followed the patients at least once every 3 months and a psychiatrist saw each patient at least once every 6 months to supervise their treatment. Stable patients were primarily followed by township doctors, unstable patients were primarily followed by psychiatrists. Medications were provided free-of-charge by the government.

Diagnostic criteria

The diagnosis of schizophrenia was based on the 3rd edition of the Chinese Classification and Diagnostic Criteria of Mental Disorders.[5] Doctors participating in the review of diagnoses were either psychiatric resident physicians or psychiatrists with more than 5 years of experience.

Instruments

The standardized instruments developed for the national registry program[6] are employed in the 686 Program in Mianyang: a checklist for identifying individuals with abnormal behaviors, an emergency medical treatment record sheet, information sheets on hospitalization of enrolled patients, baseline personal information forms and supplemental forms for severe cases, informed consent forms, assessment forms of dangerous behaviors, and consent forms for release of information.

Program management

The 686 Program is supervised by the Sichuan Provincial Health Department and coordinated by the Center for Chronic Non-Communicable Diseases under the Sichuan Centers for Disease Control. The Expert Committee for the Division of Chronic Non-Communicable Disease's Mental Health Group is responsible for providing expert advice. The program covers all 276 townships within Mianyang Municipality. To ensure that procedures were conducted according to the protocol, the Sichuan Provincial Mental Health Center provided standardized training to all of the over 600 individuals involved in the data collection. Input and quality control components of the project and these individuals had to pass an examination prior to participating in the project.

Quality control

Three different methods were used to ensure quality control. a) The Mianyang Municipal Health Bureau established regulations for assessing the effectiveness of the management and treatment of individuals with serious mental illnesses and incorporated these regulations into the annual evaluation metrics for each county's health bureau. These criteria included the proportion of all mentally ill patients under management; the proportion of managed patients who are taking medication; the level of awareness of mental health issues in the community, and so forth. b) The municipal psychiatric hospital, in accordance with the National Regulations on the Essential Public Health Services for the Management and Treatment of Serious Mental Illness (2011),[7] supervised the mental health services for persons with severe mental illnesses provided at township hospitals and reported the results to the Sichuan Provincial Health Department. And c) the Sichuan Provincial Health Department organized experts to provide technical support and supervision of the project.

Data extraction

The identification of patients included in this analysis is shown in Figure 1. For the current retrospective analysis we abstracted information from the electronic registry system for patients in Mianyang Municipality who received services from the 686 Program during 2011, who were 15 years of age or older, and who met the diagnostic criteria for schizophrenia according to the 3rd edition of the Chinese Classification and Diagnostic Criteria of Mental Disorders (CCMD-3 ).[5] The data collected included gender, age, ethnicity, occupation, education level, family economic status (i.e., below or above the local poverty line), marital status, family history of mental disorders, duration of illness (i.e., time from first onset to last assessment in 2011), time enrolled in the 686 program at the time of the last visit, and treatment compliance and effectiveness at the time of the last clinical assessment in 2011. Treatment effectiveness was recorded by the clinician who evaluated the patient (approximately 40% were psychiatrists and 60% were township or community physicians), covered the time period since the last visit (i.e., from 1 month to 6 months), and was classified as ‘recovered’, ‘improved’, ‘unchanged’ or ‘worsened’ based on the overall judgment of the clinician. These follow-up visits were either conducted in patients' homes or, if the patient was stabilized, at the township hospital.

Figure 1.

Flowchart of case identification

As shown in the flowchart, 35 of the 3125 (1.1%) identified patients were dropped from the program during 2011 due to migration out of the district, death or other causes; so a total of 3090 individuals were included in the analysis.

Statistical analysis

SPSS 19.0 software[8] was used for the analysis. Univariate differences in the characteristics of patients classified as ‘recovered’ or ‘improved’ (hereafter, ‘improved’) and those classified as ‘unchanged’ or ‘worsened’ (hereafter, ‘not improved’) at the time of the last clinical evaluation in 2011 were conducted using t-tests and chi-square tests. Follow-up multiple comparison tests for categorical variables use a Tukey-type multiple comparison method based on an arcsin transformation of the original proportions.[9] Variables that were statistically significant in the univariate analysis were entered into a forward stepwise regression analysis to identify independent factors associated with good outcomes in the program. All statistical tests were two-tailed and the level of significance was set at p<0.05.

The 686 Program has been approved by the Ethics Review Board of the Sichuan Provincial Mental Health Center.

Results

Univariate analysis

The 3090 enrolled patients who completed the follow-up assessments had a mean age of 49 years and a mean duration of illness of 15.1 years. At the time of the final evaluation in 2011 the median duration of participation in the 686 Program was 5 months (interquartile range, 2–8 months), though a small minority of patients had participated in the program since its inception 6 years previously. Farming was the main occupation of 85% (2622/3090) of the participants, reflecting the fact that Mianyang Municipality is primarily a rural community.

With the exception of gender, ethnicity and family history of mental illness, all the other variables showed statistically significant differences between patients classified as ‘improved’ and those classified as ‘not improved’ at the time of the last clinical evaluation in 2011. As shown in Table 1, improved patients were younger, had a shorter duration of illness and had participated in the 686 treatment program for a shorter length of time than patients who were not improved. As expected there was a step-wise relationship between better treatment adherence and clinical improvement. Patients with higher educational status or better family economic conditions were also more likely to be classified as improved. Farmers were less likely to be classified as clinically improved than factory workers and those who had been widowed were less likely to be clinically improved than those who had never married or were currently married.

Table 1.

Comparison of the characteristics of patients with schizophrenia enrolled in the 686 Program in Mianyang, China who were classified as improved or not improved by the treating clinician at the time of the last clinical evaluation in 2011

Characteristic	Improved (n=1886)	Not improved (n=1224)	statistic	p-value	multiple comparisonsa
Age in years (mean, sd)	47.0 (13.5)	50.1 (13.7)	t=6.25	<0.001	—
Years duration of illnessb (mean, sd)	14.3 (11.0)	16.3 (11.8)	t=5.34	<0.001	—
Months in the program (mean, sd)c	11.3 (17.2)	15.6 (21.6)	t=6.12	<0.001	—
Male (n, %)	859 (46.0%)	533 (43.5%)	χ2=1.85	0.183	—
Han ethnicity (n, %)	1837 (98.4%)	1213 (99.1%)	χ2 =2.49	0.143	—
Family history of mental illness (n, %)	72 (3.9%)	40 (3.3%)	χ =0.74	0.432	—
Treatment adherence (n, %)			χ2 =377.62	<0.001	—
[A] does not take medication	257 (32.4%)	536 (67.6%)			C>B>A
[B] intermittently takes medication	783 (64.9%)	424 (35.1%)
[C] takes medication regularly	826 (75.8%)	264 (24.2%)
Occupation (n, %)			χ2=43.50	<0.001
[A] Farmer	1525 (58.2%)	1097 (41.8%)			B,D>A
[B] Worker	22 (81.5%)	5 (18.5%)			D>C
[C] Unemployed or laid-off	110 (64.7%)	60 (35.3%)
[D] Other	209 (77.1%)	62 (22.9%)
Education background (n, %)			χ2=55.12	<0.001
[A] Illiterate	622 (55.4%)	500 (44.6%)
[B] Primary school	626 (61.1%)	398 (38.9%)			D,C,B,A>E
[C] Middle school	428 (67.1%)	210 (32.9%)			D,C>A
[D] High school and above	144 (73.8%)	51 (26.2%)			D>B
[E] Unknown	46 (41.4%)	65 (58.6%)
Family economic status (n, %)			χ2=39.54	<0.001
[A] Below local poverty line	1129 (58.7%)	795 (41.3%)			B>A>C
[B] Above local poverty line	548 (68.8%)	248 (31.2%)
[C] Unknown	189 (51.1%)	181 (48.9%)
Marital status (n, %)			χ2=28.82	<0.001
[A] Married	1194 (62.4%)	718 (37.6%)
[B] Unmarried	506 (60.3%)	333 (39.7%)			A,B>D
[C] Divorced	75 (57.7%)	55 (42.3%)
[D] Widowed	66 (42.6%)	89 (57.4%)
[E] Other	25 (46.3%)	29 (53.7%)

IQR,Interquartile range

aMultiple comparisons use a Tukey-type multiple comparison method based on an arcsin transformation of the original proportions.[9] All intergroup comparisons shown are significant at the p<0.05 level.

bTime from illness onset to last follow-up in 2011

cTime from enrollment in 686 program until last follow-up in 2011; median time (interquartile range) for effectively treated group was 5 (2-8) months and for the not effectively treated group was 5 (3-14) months (Z-value for Mann-Whitney test=-3.21, p<0.001)

Multivariate analysis

All eight statistically significant variables in the univariate analyses were entered as independent variables in the stepwise logistic regression of factors related to clinical improvement. As shown in Table 2, only duration of illness and age did not enter the final model, indicating that the remaining six factors are independent predictors of clinical improvement. Similar to the univariate analysis, being classified as clinically improved at the time of the last clinical evaluation in 2011 was associated with better adherence, better family economic status and shorter times of participation in the 686 Program. There was a clear ‘dose-response’ relationship between adherence and outcome; patients with good adherence had better outcomes than those with fair adherence and they all had better outcomes than those who were non-adherent.

Table 2.

Results of stepwise multivariate unconditional logistic regression analysis of factors associated with clinical improvement for 3090 patients with schizophrenia enrolled in the 686 community management programa

Variables	n	partial regression coefficient B	standard error	Wald value	p-value	odds ratio	95% confidence interval
Months in the 686 Program	3090	-0.014	0.002	43.34	<0.001	0.99	0.98-0.99
Treatment adherenceb				302.24	<0.001
fair adherence	1207	1.293	0.099	169.37	<0.001	3.64	3.00-4.43
good adherence	1090	1.819	0.108	283.61	<0.001	6.17	5.00-7.62
Family economic statusc				20.89	<0.001
below poverty line	1924	-0.420	0.101	17.28	<0.001	0.66	0.54-0.80
unknown	370	-0.550	0.150	13.49	<0.001	0.58	0.43-0.77
Marital statusd				20.21	<0.001
married	1912	0.086	0.094	0.83	0.361	1.09	0.91-1.31
divorced	130	-0.429	0.209	4.22	0.040	0.65	0.43-0.98
widowed	155	-0.577	0.194	8.81	0.003	0.56	0.38-0.82
unknown	54	-0.449	0.314	2.05	0.152	0.64	0.35-1.18
Education backgrounde				15.61	0.004
primary school	1024	0.096	0.096	0.99	0.32	1.10	0.91-1.33
middle school	638	0.261	0.115	5.14	0.023	1.30	1.04-1.63
high school and above	195	0.385	0.2	3.70	0.054	1.47	0.99-2.17
unknown	111	-0.565	0.236	5.71	0.017	0.57	0.36-0.90
Occupationf				9.16	0.027
factory worker	27	0.577	0.528	1.19	0.275	1.78	0.63-5.01
unemployed	170	-0.039	0.187	0.04	0.835	0.96	0.67-1.39
other	271	0.477	0.169	7.99	0.005	1.61	1.16-2.24

aAll eight variables that were significant in the univariate analysis (Table 1) are entered as independent variables; patient age and duration of illness dropped out of the forward stepwise regression. Nagelkerke R for model=0.203.

bcompared with ‘non-adherence’ (n=793)

ccompared with ‘family income above poverty line’ (n=796)

dcompared with the ‘unmarried’ (n=839)

ecompared with ‘illiterate’ (n=1122)

fcompared with ‘farmer’ (n=2622)

The results for the other three variables that were retained in the multivariate model – marital status, educational background, and occupation – were not as clear cut, probably because of the relatively small numbers of patients in some of the categories. Individuals who were divorced or widowed were less likely to be classified as improved than those who have never married, but there was no difference in the outcome classification between those who were currently married and those who had never married. There was a non-significant trend of better outcomes for individuals with higher levels of education. Patients with some education were all more likely to be classified as improved than those who had no education (i.e., illiterate) but this difference was only statistically significant for those with a middle school education. Interestingly, those who had no educational status recorded had worse clinical outcomes than illiterate patients. And individuals with ‘other’ occupations (e.g., self-employed, management positions, etc.) were more likely to be classified as clinically improved than farmers but there were no other differences by occupational status.

Discussion

Main findings

Patients enrolled in the 686 Program classified as having no clinical improvement at the time of the last evaluation in 2011 were more likely to be non-adherent to treatment (primarily mediation), to be members of families that were living below the poverty line, to be widowed or divorced, and to have been enrolled in the program for longer periods of time.

The finding that treatment adherence was the strongest predictor of effectiveness confirms a long list of previous studies in China and elsewhere[10]–[13] in this new, primarily rural, community-based setting in China. The characteristic symptoms of schizophrenia and adverse reactions to many of the commonly used medications for schizophrenia make adherence to recommended treatments a particularly difficult problem in the management of patients with chronic schizophrenia.[14],[15] The lower rate of improvement among patients living in poverty confirms other studies that find better outcomes in patients with better economic conditions.[16],[17] Patients who are divorced or widowed (9% of the current sample) have lost an important component of their social support network (who often help in the supervision of medication) so it is not surprising that these individuals have poorer clinical outcomes.

Perhaps the most unexpected findings were that occupational status was not clearly related to outcome and that outcomes were worse in patients who had been enrolled in the 686 Program for longer periods of time. As reported by others[18] the vast majority of individuals with schizophrenia enrolled in the program had low occupational status (85% of the current sample were classified as ‘farmer’) so the numbers of cases in the other occupational classes are too few to make meaningful comparisons (e.g., there were only 27 subjects in our study classified as factory workers). Moreover, family economic status is related to occupational status of the patient so inclusion of family economic status in the model weakens the independent predictive power of the patient's occupational status.

The 686 Program in Mianyang Municipality started enrolling patients in 2006 so there are some participants who have been in the program for up to 6 years, but most of the current participants have been enrolled since the new effort to enroll patients started in July 2010. As of the end of 2011, 25% of participants had been in the program for 3 months or less, 50% of participants had been in the program for 5 months or less and 75% of participants had been in the program for 8 months or less. We expect that patients who were enrolled before the new initiative to increase enrollment started and those identified soon after the initiative started may have been patients with more severe, intractable problems (who were well known by the local providers) and that those enrolled more recently were less severely affected by their illness. If this is the case, patients who are enrolled for longer periods in the program would be more likely to be clinically classified as ‘unchanged’ or ‘worsened’ and those enrolled for shorter periods would be more likely to be clinically classified as ‘recovered’ or ‘improved’. Further analysis of the severity of symptoms at the time of entry in the program would be needed to assess this hypothesis.

Limitations

The current analysis is primarily limited by the narrow range of variables recorded in the electronic registry system for the 686 Program and by the uncertain validity of some of the variables included in the registry. The most important consideration for the current analysis is the validity of the outcome evaluation and the assessment of adherence by the treating clinicians. The more than 600 clinicians involved in the project were provided training in these determinations, but the decision about how to classify a specific patient at a particular point in time inevitably involves some subjective clinical judgment. There has been no assessment of the inter-rater reliability of participating clinicians regarding these assessments so the accuracy of the determinations of outcome and adherence remains uncertain. Another problem is that the time frame for assessing outcome and adherence is uncertain because the duration of follow-up is so variable between different patients. And the status of patients at the time of the last clinical evaluation at the end of 2011 does not necessarily reflect the outcome and adherence during 2011 or (for the majority of patients who were enrolled for less than 1 year) during the time of enrollment in the program. Moreover, the simultaneous evaluation of clinical outcome and adherence by the same clinician inevitably magnifies the correlation of these variables because clinicians have a strong bias that assumes adherence is essential to good outcomes.

There are several other problems that will need to be addressed in subsequent studies. It is unclear what proportion of all patients with schizophrenia in Mianyang are enrolled in the 686 Program so it is not possible to say how representative the sample is or how generalizable the result would be for all community-based patients. Without data on the relapse rate, re-hospitalization rate (or duration) and re-employment rate it is hard to assess the social important outcomes of the patients. Similarly, without data on the medications used, the dosages and the rates of adverse reactions it is hard to relate the outcomes to the interventions provided. And the ‘time enrolled in the program’ variable generated some unexpected results that were, possibly, related to the highly skewed distribution of enrollment times; it may have been better to convert this to a categorical variable in the analysis (e.g., greater than 1 year, 3-12 months, under 3 months) rather than using it as a continuous measure.

Implications

Despite these limitations this large study is one of the first reports on the efficacy of the 686 Program. The clear relationship of poor clinical outcomes with poor adherence and with poverty is not new but it highlights the urgency to developing creative methods of improving adherence and of providing financial assistance to families that have a member with a serious mental illness. The ultimate success or failure of the 686 Program will depend on its ability to overcome these two major barriers to improving patient outcomes.

This analysis also highlights important problems in the 686 Program that need to be rectified. Outcome measures included in the registry system need to include ‘hard’ variables such as relapses, re-hospitalizations, and full-time employment. Treatment variables need to include medication type, dosage and serious side effects. The reliability of the clinical ratings need to be evaluated and, if lacking, additional training should be provided until the reliability of the ratings is satisfactory. The assessment of outcomes needs to be integrated over time periods (say one year) rather than cross-sectional one-time assessments. And economic data need to be integrated into the information system so that the cost-effectiveness of the program can be assessed and compared to that of other community-based programs for persons with severe mental illnesses. These revisions to the training and data collection system for the 686 Program are essential to enhance the validity of the assessment of its efficacy and, thus, provide the data needed to alter the program in ways that will improve outcomes for patients with severe mental illnesses.