Reporting Inpatients' Experiences and Satisfaction in a National Psychiatric Facility: A Study Based on the Random Forest Algorithm.

Understanding psychiatric inpatients' experiences is important to establish a culture of patient-centric care and promote trust in healthcare. This study aimed to evaluate nine dimensions of patients' experiences and investigate their association with patient satisfaction, revisit intention, and positive word-of-mouth (WoM) recommendation. Cross-sectional questionnaire data from five years of surveying (2016-2020) in the main psychiatric hospital in Bahrain were statistically analyzed, involving 763 psychiatric inpatients with an overall 65.6 ± 17.2 length of stay (days). The findings show that across the five years 2016-2020, the overall reported satisfaction was "very high" (4.75 ± 0.44) with no significant differences between these five years (F [4, 758] = 0.66, p = 0.620). The experience of confidentiality received the highest rating (4.72 ± 0.45). The experiences of ease of access, hospitality quality, and quality of responsiveness to one's needs significantly correlated with revisit intention (p ˂ 0.05). Patients with high satisfaction had greater potential for revisit intention (r [761] = 0.08, p = 0.027), which was associated with WoM recommendation (r [761] = 0.08, p = 0.033). Overall, men were less likely than women to experience convenient access to psychiatric wards. The findings of the Random Forest algorithm indicate the tendency of female patients with short-term stays to demonstrate lower satisfaction rates, and thus innovative approaches are needed when managing these groups' psychiatric problems.

Introduction

Patient satisfaction has been a subject of paramount importance and burgeoning attention to be promoted by patient-centered care practices (1), excellent healthcare management (2,3), and patient feedback-centric research (4). Because of its clear influence on positive patients’ behaviors and trust towards healthcare (5,6), many studies argue that measuring and monitoring patient satisfaction and feedback should be essential in health provision settings (7,8). This argument has been cited in research involving several healthcare specialties, emphasizing that patient satisfaction is a valid indicator of care provider and health system performance and quality (9,10).

The psychiatric care setting is one of the fundamental healthcare domains to which growing evidence trends stressed the importance of surveying patient satisfaction (11–13). Because of its positive spillovers, patient satisfaction in the mental health arena is recognized as an integral part of institutions’ care quality evaluation and improvement (14,15), and as an effective catalyst for the patients themselves for supporting their psychological well-being (16), predicting treatment compliance, and fostering their cooperation and subsequent acceleration of recovery or improved outcomes (17,18). Thus it would be beneficial to identify the antecedents of patient satisfaction in inpatient psychiatric services to provide an empirically informed foundation to promote best care delivery models and responsiveness to this distinct segment of patients.

Despite the significance of measuring inpatient opinion, few studies were undertaken to propose validated instruments or to explore the predictors of patient satisfaction in inpatient psychiatric wards through the lens of patient-reported experience (19). It has been argued that patient satisfaction is determined through experiences of care (20). Primary research exploring and identifying the patient experiences with mental health services or psychiatric hospitalization is still lacking (19–22). Meanwhile, it is imperative to distinguish between patient-reported experience and satisfaction and study the interrelation between these two constructs. Patient-reported experience refers to interaction on specific aspects of care, while satisfaction is about the degree of fulfilling patients’ expectations or needs (21,22). Worth mentioning in this context, and according to our best knowledge, no studies in Bahrain—the country in which this research took place—have been conducted to explore the association between patients’ experiences of care and satisfaction in the inpatient psychiatric ward.

A recent systematic review study has identified a total of 86 articles and examined 75 patient-reported experience measures for mental health care. It has indicated the diversity of experience factors and the necessity to focus on the most relevant ones meeting psychiatric care providers’ needs (21). Alongside the treatment delivery experience (22), many other relational and environmental experience factors have been invaluable and could play a vital role in the mental health setting. In a study, inpatient care experience is linked with the experiences of hospital admission, especially within the first few days of admission; this depends on how the patient is being assured with helpful relationships and elimination of uncertainty sense through clear communication and information (11). Ward atmosphere, including dimensions like “patients’ cohesion,” “experienced safety,” and “therapeutic hold,” is also of potential influence on patient satisfaction and should be assessed carefully through applicable policies (23).

In summary, patient-reported experiences with psychiatric inpatient services were observed to be varied across clinical groups and hospitals. Thus identifying and studying experiences for inpatient psychiatric wards could not be reduced to a kind of simplification, and that how these experiences may influence satisfaction is a complex issue, particularly across different cultural, social, and institutional contexts. Based on that argument, this study assessed selective experience factors and their potential to predict patient satisfaction in inpatient psychiatric care. To further understand the potential consequences associated with patient satisfaction, we empirically examined whether or not a causal relationship exists between patient satisfaction and each of “Word-of-Mouth (WoM) recommendation” and “revisit intention,” which could in return help improve knowledge around the varying observations and discussions regarding the relationship between these factors (24,25).

Methods

Participants & Ethical Considerations

The study was approved by the Psychiatric Hospital Research Ethics Board. Anonymity and confidentiality of data have been managed in accordance with the American Psychological Association's recommendations (26). Participants were 763 psychiatric inpatients admitted to the Psychiatric Hospital, which belongs to the Ministry of Health in the Kingdom of Bahrain. The hospital is a central government facility providing secondary and tertiary mental health services to the citizens and residents in the country since its establishment in the 1930s. It has a capacity of about 300 beds for the inpatients, alongside the mobile community services and multi-specialty clinics serving psychiatric outpatients. As a national center, the hospital plays a crucial role in diagnosing, treating, and rehabilitating psychiatric patients and promoting mental health within the local population.

Sample size and power calculations indicated that the overall sample sizes would provide a minimum power of 80% to detect satisfaction/dissatisfaction areas assuming a common rate of 50% satisfaction with a type I error of 0.05 and type II error of 0.2. Using a self-administered paper questionnaire, the data were collected over five years from 2016 to 2020, during which 203, 184, 193, 126, 57 subjects participated, respectively. The overall sample characteristics are shown in Table 1.

Table 1.

Sample Characteristics (N = 763).

Domain		n (%)
Gender	Male	512 (67.1)
	Female	251 (32.9)
Nationality	Bahraini	635 (83.2)
	Non-Bahraini	128 (16.8)
Diagnosis(according to ICD-10)	Bipolar affective disorder	113 (14.8)
	Major depressive disorder	270 (35.4)
	Schizophrenia	239 (31.3)
	Others	141 (18.5)
Length of stay(M ± SD)	65.6 ± 17.2 days

Survey Development

The “Survey Form” was developed and co-designed by the researchers and advisors of the hospital quality committee. It was released in both Arabic and English language and piloted at the time of its inception on a group of 30 patients whose collected data were not included in this study. Further, expert review and feedback on wording style, clarity, meaning, and briefness were incorporated. It comprises three parts: 1) general information about patient's gender, diagnosis, length of stay (LoS), and nationality; 2) 9 questions to assess patient's experiences with psychiatric services based on the following themes (as independent variables): treatment effectiveness, confidentiality, empathetic interaction, ease of access, hospitality quality, communication quality, responsiveness quality, hygiene quality, food deliciousness—each question was assigned with a 5-point level of quality scale rating from poor to excellent; and 3) three questions related to the dependent variables of patient's satisfaction (rating on a 5-point Likert satisfaction scale), revisit intention (rating on a 5-point Likert agreement scale) and WoM recommendation (rating on a 5-point Likert agreement scale).

Measures

Each of the nine themes of patients’ experiences with psychiatric services was assigned a single measure. These included 1) perceived treatment effectiveness which is about the success of the delivered treatment to solve a patient's problem (27), 2) perceived confidentiality, which is about a patient's confidence that all information being provided will be treated in the strictest secrecy (28), 3) empathetic interaction, relating to the patient's perception of how the clinicians deal in a comfortable and tactful way, empathically and compassionately (29), 4) ease of access which is about the quality of inpatient admission process and how entering the ward was easy and uncertainty less (11), 5) hospitality quality which reflects the standard of entry procedures to the ward, and inducing comfort comparable to guests in a hotel (30), 6) communication quality through which the patient's understanding of his/her condition has been ensured through sufficient and clear information (31), 7) responsiveness quality relates to the healthcare worker’ attention for addressing quickly and smoothly the needs of the patient, embracing patient-centeredness (32), 8) hygiene quality which is simply about patients’ perceptions of the hospital's cleanliness, and 9) food deliciousness which reflects the acceptance of food with pleasure (33). The item measuring patient satisfaction was stated as “Overall, my stay in the ward was satisfactory.” Measuring satisfaction through a single item has been commonly practiced in healthcare management research, despite being argued with validity, sensitivity, and reliability concerns. (34,35). The item of revisit intention was stated as “I would go to this hospital if I needed help in the future” (25), and the item of WoM recommendation “If one of my friends needs help, I will advise him to come to this hospital” (24).

Statistical Analysis

As per the quality standard adopted by the hospital management, the numerical data of completed questionnaires were consistently entered and saved into a Microsoft Excel spreadsheet 2017 version. A sample of 763 questionnaires with complete answers to all the questions was selected for the present study data analysis. Descriptively, the results were presented as percentages and frequency distribution. The arithmetic mean (M) and standard deviation (SD) were reported for continuous variables, and counts and percentages were reported for categorical variables. On the other side, applied inferential tests included t-test and one-way Analysis of Variance for means comparison and Pearson product-moment correlation coefficient r to assess the correlation between variables.

To determine the influencing factors of overall patient satisfaction more objectively, Random Forest (a machine-learning algorithm) has been administered. Its ease of use and flexibility have powered its adoption, as it handles classification and regression problems. The random forest model outperforms standard evaluation approaches (e.g., regression techniques) in terms of evaluation efficiency and accurate dimensionality reduction and correctness (36). Using a double random sampling of samples and variables, the random forest method eliminates the requirement for high sample size and the poor feature selection typical of decision trees. Li et al. (2021) and Simsekler et al. (2021) introduced the random forest model as an effective analysis approach of inpatient satisfaction (37,38), and hence this approach has been replicated in this study. The discriminant analysis was based on computing train, validation, and test sets in Machine Learning. The dataset was split into three parts: training set comprising 64% of the data (used to fit the model), validation set comprising 16% (used to provide an unbiased evaluation of a model fit on the training dataset while tuning model hyper-parameters), and test set comprising 20% (used to run an unbiased evaluation of the final model fit on the training dataset). Total trees of 25 were used within a Random Forest (39). We evaluated the outputs using several fit metrics, including R-squared (R2), Mean Square Errors (MSE), and Root Mean Squared Errors (RMSE).

All statistical analyses were performed in R Statistical Computing version 4.1.1, the package “Random Forest” was used for AI/ML based analyses. A p-value of < 0.05 was considered to be significant.

Results

Patients’ Experiences

Amongst the investigated nine experiences, the highest rating was associated with the experience of confidentiality (4.72 ± 0.45), and that 27.8% and 72.2% reported their experience of confidentiality practices in their encounters with clinical practitioners as “very good” and “excellent,” respectively. However, this factor did not correlate with the dependent variables of this study (p ˃ 0.05). The least rated experience was related to the food deliciousness (3.10 ± 1.08); this factor did not significantly correlate with the dependent variables of this study (p ˃ 0.05). Important to mention that only the experience of treatment effectiveness was significantly associated with satisfaction (p ˂ 0.05) and that three experiences were significantly in correlation with revisit intention: ease of access, hospitality quality, and quality of responsiveness to one's needs (p ˂ 0.05). However, no significant correlation was detected between any of the experiences and dependent variables of the study (p ˃ 0.05) (Table 2).

Table 2.

Patients’ Experiences: Descriptive and Correlation Values (N = 763).

Factor	M ± SD	Rank^	Pearson correlation coefficient r (p-value)
			Satisfaction	Revisit intention	WoM recommendation
Treatment effectiveness	3.79 ± 1.47	4	0.073*(0.043)	0.044(0.228)	0.005(0.892)
Confidentiality	4.72 ± 0.45	1	0.045(0.219)	0.005(0.899)	0.001(0.970)
Empathetic interaction	4.12 ± 0.32	3	0.016(0.650)	0.062(0.088)	0.016(0.661)
Ease of access	3.18 ± 1.08	7	0.000(0.990)	0.113**(0.002)	0.023(0.521)
Hospitality quality	3.61 ± 0.78	5	0.020(0.583)	0.073*(0.043)	0.062(0.088)
Communication quality	3.11 ± 1.08	8	0.030(0.407)	0.023(0.524)	0.001(0.974)
Responsiveness quality	4.13 ± 0.32	2	0.014(0.695)	0.096**(0.008)	0.017(0.646)
Hygiene quality	3.19 ± 1.08	6	0.002(0.965)	0.066(0.068)	0.030(0.416)
Food deliciousness	3.10 ± 1.08	9	0.003(0.938)	0.038(0.290)	0.048(0.185)

Note. Bolded values with * and ** indicate statistical significance p < 0.05 and p < 0.01, respectively.

^. Ranking is from highest rating value (1) to the lowest rating value (9).

Through using t-test, there was no significant difference between the male and female group in all the experiences (p > 0.05), except the experience of “ease of access” at which the men indicated lesser ratings (3.23 ± 1.05) than the women (3.30 ± 1.14), (t [761] = - 2.03, p = 0.043). There was no significant difference between the Bahraini and non-Bahraini groups (p > 0.05) in all rated experiences. Further, ANOVA shows no significant difference between the four groups of diagnosed psychiatric illness regarding all rated experiences (p ˃ 0.05).

Patient Satisfaction, Revisit Intention, and WoM Recommendation

Over the five years 2016–2020, participants reported an overall satisfaction of “very high” (4.75 ± 0.44). There was no significant difference between the participants’ groups for each year regarding their satisfaction levels (F [4, 758] = 0.66, p = 0.620). 25.3% and 74.7% reported their overall satisfaction towards overall hospital experience as “very good” and “excellent,” respectively. Notably, patients with higher satisfaction had more potential for revisit intention than those with less satisfaction (r [761] = 0.08, p = 0.027). Also, patients willing to show positive WoM recommendation had a higher tendency to revisit the hospital (r [761] = 0.08, p = 0.033). However, no significant direct association was found between satisfaction and WoM recommendation (p ˃ 0.05).

Most participants were nearly in a position of high tendency to have a revisit intention (3.90 ± 0.60), or give a positive WoM (3.89 ± 0.58); explicitly 76% and 76.4 agreed and highly agreed to revisit the hospital once needed, and to advise a friend for referring to the hospital, respectively.

Through comparing means of satisfaction, there was no significant difference: 1) between male (4.74 ± 0.44) and female group (4.77 ± 0.43), (t [761] = - 0.795, p = .427), 2) between Bahraini (4.75 ± 0.44) and non-Bahraini group (4.75 ± 0.43), (t [761] = - 0.084, p = 0.933), and also 3) between the different four groups of diagnosed psychiatric illness (F [3, 759] = 0.712, p = 0.545). Similarly, the results of comparing the means for all of these groups sets with reference to revisit intention and WoM recommendation showed no significant differences (p ˃ 0.05).

Demographic Variable Importance

The LoS was the most important predictor (explanatory) variable for the overall patient satisfaction, based on the mean decrease Gini measure (calculated for each predictor variable as the cumulative increase in data purity associated with each decision tree node split)- Figure 1(a). The year of conducting the survey was shown to be the most important variable in predicting the overall patient satisfaction based on the calculated mean decrease in accuracy score- Figure 1(b).

Figure 1.

(a) Values of mean decrease Gini impurity index measuring the importance of each variable; (b) values of mean decrease accuracy; measuring the decrease of model accuracy when variables are dropped.

The random forest regression yielded a validation was used to train based on 488 participants and was validated and tested on 123 and 152 participants, respectively. The model was optimized with respect to the out-of-bag MSE. The graphical representation of both training set and validation sets showed an adequate concordance after a number of trees > 5; a visual representation of the data behavior is depicted in Figure 1. Evaluation metrics showed high modeling sensitivity with MSE = 96.8% and RMSE = 98.4%. The most important variable that led to a total increase in node purity is LOS (short LOS) which explains about 65% of the total variance. Year of surveying and sex explained about 32% and 9% of the variance.

Discussion

This study is of its first kind in the country of Bahrain to assess psychiatric inpatients’ experiences and the potential association of these experiences on patient's satisfaction, revisit intention, and positive WoM recommendation. Further, investigating patient satisfaction over a spectrum of five years, 2016–2020, represents a valuable source of information to insight into how successful the hospital has consistently met its main stakeholders’ needs. Specifically satisfaction across time showed no statistical difference. Our results pointed to a top level of patient satisfaction (4.75 ± 0.44), which is relatively higher than the global mean score for patient satisfaction [4.11 (minimum: 2.0; maximum: 5.0)] with mental health services (14). Alongside, steady satisfaction levels during the five years were observed, showing the impact of continuous quality improvement adopted by the hospital management (10). Despite that positive record, investigated patients’ experiences have received varied ratings between “good” and “very good,” and only the “treatment effectiveness” experience factor was significantly associated with patient satisfaction. Such results establish a scientific alert for exploring the determinants of satisfaction amongst the beneficiaries of psychiatric services in the country. Explicitly, factors outside of patients’ experiences appear to influence patients’ perceptions of their care. For instance, there has been evidence implying that patient satisfaction scores could be positively biased (over-optimistic evaluation) by the patients themselves and that their characteristics could determine how likely they will be satisfied with delivered healthcare (20,40). The present study also validates the relationship between perceived treatment effectiveness and satisfaction construct (41,42), which ultimately urges clinicians to push treatment quality to the front of the priority in healthcare practices, and thus creating the better potential for improved patient's medication adherence (41).

As the experience of confidentiality received the highest rating (4.72 ± 0.45) compared to its counterparts, then it is worth shedding some light on this aspect. Recent research has been steered to gain an in-depth understanding of psychiatric clinicians’ confidentiality practices from both the eyes of service users and the judgment capacity manifested by the practicing clinicians themselves (43–45). Hence, excelling in the confidentiality domain can be linked to the competency of mental health professionals in terms of awareness of ethical issues, mitigation of mental illness stigma, and patient information protection standards (45,46). Additionally, trust in the patient-provider relationship could be a significant element responsible for low patients’ perceived risk to privacy and confidentiality gaps (47). However, in the current situation of published information paucity about psychiatric confidentiality practices, particularly within our Arab social and cultural context, directing future research to this subject would be highly desirable.

The four lowest-ranked experiences that received average scores within the middle of the rating scale represent additional quality evaluation and improvement areas. The first lowest-rated experience points out a diminished deliciousness in prepared meals, a significant drawback that could potentially elicit food intake and malnutrition problems (33). The second lowest-rated experience signifies the importance of paying more attention to the interpersonal communication skills of psychiatric professionals; however, differently from previous studies, we did not find a significant association between communication quality and satisfaction (10), which suggests that there is room for exploring patients’ judgments about outstanding communication. The third lowest-rated experience, which is about admission easiness, should not be interpreted based on mere numerical values as the study sample had not distinguished between involuntary and voluntary admissions, and thus coercion perceptions could vary (48). The fourth lowest-rated experience of ward hygiene quality stresses the need to monitor cleanliness through higher standards and consider additional relevant measures to induce a patient-friendly physical environment (19).

In the context of literature scarcity regarding the sequential relationships of experiences-satisfaction-revisit intention in the psychiatric field, interestingly, this study found that experiences of admission easiness, hospitality quality, and responsiveness have a significant association with patients’ revisit intention, significantly linking to patient satisfaction. This finding necessitates acknowledging that patient satisfaction through its mediating effect could potentially promote patient revisit intention (49). The observed high tendency of this study's participants to revisit the hospital does not only imply the extent of patient's gained loyalty, but beyond that, it could foster establishing the esteem of the psychiatric discipline among the public and positive attitudes towards seeking help from psychiatrists (50), particularly in our Arab region suffering the issue of mental illness stigma (46).

The findings of this study show that men, compared to women, have more potential to experience uncertainty and convenience less when accessing a psychiatric ward. This result can be viewed within the context of gender differences in health-related behavior, in line with previous studies showing that men are less likely than women to seek mental health care (51,52). Regarding the LoS, from a clinical perspective, longer LoS is associated with patient features such as severity and chronicity of the disease and a lack of social support, which may influence satisfaction (53). More research is needed to look at the patient's dispositional attributes and the hospital (or contextual) factors, such as clinician attitudes and behaviors, that may be connected to shorter hospital stays. However, in our study, patients who had an acute episode and stayed for a short period were less satisfied than others. This was reported in previous research, and it was stated that such individuals frequently release themselves after a few days (typically against medical advice) and return when their symptoms deteriorate (54). Future follow-up studies should concentrate on those individuals since low satisfaction may postpone getting help and sticking to a treatment plan. Based on these findings, we propose that inpatients get stigma evaluations (55), psychoeducation (56), and non-pharmacological therapies in addition to medications (57), which may increase patients’ and families’ satisfaction and engagement.

WoM recommendation is a broadly discussed factor in healthcare management literature because of its potent influence on health behavior and service consumption (24). However, limited research has been steered towards investigating this factor within the context of the psychiatric field. The present study provides generic implications of how likely psychiatric patients expressing positive WoM may have a higher tendency for revisit intention. This finding is consistent with Lee's (2005) analyses that patients’ WOM is a valid predictor of revisit intention (25). Overall, the present study's high positive WoM scores could be seen as a catalyst for reassuring mental health seekers and promoting trust in delivered services.

The empirical findings of this study could open new avenues for improving quality metrics, particularly through: 1) converting the different current variables of the study into composite variables based on literature, which could provide broader understanding and quality gaps detection, allow applying multivariate analyses, and greater capacity to benchmark the results with previous research; 2) collecting and recruiting data relevant to group differences according to demographic information such as age, education, and social status, and contextual factors such as admission type (voluntary vs. involuntary), family visits, and rehabilitation programs. Such segmentation could help quality advisors and researchers in modeling and understanding the group differences, hence structuring tailored solutions and programs of quality improvement.

This study has some drawbacks, although providing useful baseline data. First, there is a potential of social desirability bias: the data were gathered using a self-reported approach, leading to over- or under-reporting. Second, recall bias exists; participants may be forgetful, and the information they provided was not cross-checked.

Conclusion

The results of this study indicate that the high satisfaction with services amongst psychiatric inpatients over the years 2016–2020 deems of a potential role in eliciting revisit intentions and subsequent positive WoM recommendations. However, exploring the drivers of overall patient satisfaction within psychiatric care requires further studies with novel research approaches. RFs algorithm manifested predictive capability and indicated the importance of gender, LoS, and year of surveying factors within the context of patient satisfaction. In light of these findings, planning and studying improvement interventions for promoting patients’ experiences and overall satisfaction need to be a key priority in future value-creating projects.