Application of PSO-based LSTM Neural Network for Outpatient Volume Prediction.

In order to study the construction method of long- and short-term memory neural network model, which is based on particle swarm optimization algorithm and its application in hospital outpatient management, we have selected historical data of outpatient volume of relevant departments in our hospital. Furthermore, we have designed and developed the outpatient volume prediction model, which is based on long- and short-term memory neural network. Additionally, we have used particle swarm optimization algorithm (PSO) to optimize various parameters of long- and short-term memory network and then utilized this optimized model to accurately predict the outpatient volume. Experimental observations, which are collected through the results of monthly outpatient volume prediction, show that Root Mean Square Error (RMSE) of the particle swarm optimized LTMN model on the test set is reduced by 48.5% compared with the unoptimized model. The particle swarm optimization algorithm has efficiently optimized the prediction model, which makes the model better predict the trend of outpatient volume and thus provide decision support for medical staff's outpatient management.

1. Introduction

With the development of modern information technology, medical activities, medical research, and other process data are increasingly recorded and stored. These advancements lead to the generation of huge amount of medical big data, which contain a large amount of valuable information, and it is difficult to effectively process it by traditional data processing methods [1-4]. The deep integration of artificial intelligence and medical field is one of the core technologies of medical information in the context of big data. For the accumulated medical management data including outpatient volume history at all levels of treatment institutions, especially our hospital, the potential value of these data can be effectively explored by using data mining methods.

Hospital outpatient statistics reflect the hospital operation and provide an important reference for hospital management. A reasonable and accurate forecast of outpatient volume can help hospital managers optimize the allocation of medical resources, facilitate the hospital's refined management, and be of great significance to improve hospital efficiency and treatment capacity. Hospital outpatient services are extensive, and outpatient attendance statistics are highly random. At the same time, outpatient attendance is influenced by many factors, reflecting both medium- and long-term cyclical trends and obvious seasonal effects [5-7]. The traditional time series forecasting methods, such as gray system model [8], Autoregressive Integrated Moving Average (ARIMA) model [9], and Seasonal Autoregressive Integrated Moving Average (SARA) model [10], were applied to predict the number of outpatient visits. ARIMA model [9], Seasonal Autoregressive Integrated Moving Average (SARIMA) model [10], and so on face more limitations in predicting outpatient volume. With the continuous development of machine learning, especially deep learning techniques [11, 12], the prediction accuracy for time series problems, such as outpatient volume, has been improved. For example, [13] first used the outpatient volume prediction method based on Deep Belief Networks (DBN) to complete the feature extraction of outpatient volume data and improve the accuracy of outpatient volume prediction, [14] classified the historical outpatient volume according to the similarity day principle and established the Extreme Learning, and [15] constructed Baidu index features, time features, and outpatient volume features of search keywords and made more accurate prediction of hospital psychiatric outpatient volume based on XGBoost algorithm.

Recurrent neural network (RNN) is suitable for processing sequential data; therefore, various researchers have used these approaches to construct targeted prediction models. For example, [16] is based on long short-term memory (LSTM) network [17] to study the variation of the prediction accuracy of the model for monthly outpatient visits with different step sizes; [18] is based on a single LSTM network prediction model; the empirical modal decomposition was integrated to preprocess the nonstationary data, and the experimental results showed that the combined model achieved better prediction accuracy than the single model; [19] constructed a SARIMA-LNN based on the SARIMA-LSTM network; the experimental results showed that the combined model achieved better prediction accuracy than the single model; [19] constructed a SARIMA-LSTM-based outpatient volume prediction model and modeled multiple indicators such as holidays, legal days of work, and average maximum temperature comprehensively, which could achieve higher prediction accuracy compared with the mainstream prediction methods.

RNN, especially LSTM, have been proven by many researchers to be suitable for processing long series data [20], and there are more and more studies on the prediction of hospital outpatient volume based on long short-term memory networks. However, the process of long short-term memory network construction involves many hyper parameters determination, and researchers often just set hyper parameters with a priori knowledge or research experience, which lacks certain exploration process and may require repeated manual tuning of parameters in the face of different problems, while automatic neural network hyperparameter optimization using optimization algorithms has been rarely studied.

To address these issues, we have proposed a process for optimizing key hyper parameters of LSTM using a simple and efficient particle swarm optimization (PSO) algorithm. Additionally, particle swarm optimization-based LSTM is used for monthly outpatient volume prediction of chronic disease-related departments in our hospital. The experimental results demonstrate that optimized prediction model predicts the trend of outpatient volume better. The whole modeling and prediction process is easily applied to outpatient management, providing technical support for the refinement of hospital management. The main contributions are highlighted as follows:

An algorithm to optimize key hyperparameters of LSTM which is primarily based on the particle swarm optimization

Prediction model for the monthly outpatient volume prediction of chronic disease-related departments

An efficient outdoor patient management system which is based on state of the art technology

Development of an easy to use software system for doctors' other relevant personals

The remainder of the manuscript is organized as follows.

A brief literature review is presented in the subsequent section, which is followed by a detailed description of the proposed model and its various sections, such as PSO-based LSTM Neural Network Prediction Model Construction. In Section 4, outpatient volume forecast is described in detail. Experimental results and observations along with possible comparative results are presented in the same section as well. Finally, concluding remarks along with future directions are given.

2. Related Work

The prediction flow of the outpatient mainly focuses on the study of outpatient volume time series as well as the influencing factors, using data mining methods [4] for research. Predicting trend of outpatient volume is conducive to the coordinated agreement of medical resources, better serving patients and promoting scientific hospital management decisions. Summarizing the results of outpatient volume prediction research in the past decade, the research on outpatient volume prediction based on time series perspective is mainly divided into three types of methods.

The first type of method is statistical models, and the main statistical models commonly used at present are ARMA and ARIMA models [5]. For example, [6] used the moving average seasonal index method to analyze as well as predict the dynamics of outpatient volume changes in general hospitals, and this type of method has a more desirable effect on the seasonal outpatient volume change prediction. Most of the statistical modeling methods for outpatient volume prediction are for single data analysis, lacking deeper data mining.

The second type of methods is machine learning and neural network related modeling methods, which mainly exploit the nonlinear relationships of the data. For example, [7] used long- and short-term memory networks to study the accuracy of monthly outpatient visits; [8] used deep training networks to mine the hidden relationships of each department's outpatient visits to predict the overall outpatient visits; [9] used Extreme Gradient Boosting (XG Boost) model to predict psychiatric outpatient volume. The third category of methods is multimodel combination prediction. For example, [6] used a combined gray radial basis (Radial Basis Function, RBF) neural network approach to predict outpatient volume. Reference [9] also argued that deep learning-based time series data prediction model can map the data better; [11] used a combined model of kernel limit learning machine and least squares support vector machine to forecast monthly outpatient volume research; [2] used trend fitting and ARIMA to forecast combined outpatient volume; [13] formed a combined model of gray prediction model and ARIMA model after linear weights; [14] used gray model to fit monthly outpatient volume and used BP neural network model to predict outpatient volume by fitting residuals; [15] used seasonal differential autoregressive sliding average model with long- and short-term memory network to predict outpatient volume by fitting residuals; [16] used wavelet analysis and ARMA model to predict outpatient volume in winter in combination. Combined forecasting studies can integrate the advantages of multiple models and are the focus of the current outpatient volume forecasting studies.

The main function of the analysis of outpatient volume influencing factors is to give some of the characteristic attributes that are useful in the selection of characteristics for the prediction of outpatient volume. There are two main types of outpatient volume factors. First is the questionnaire approach, through the questionnaire to summarize the hospital itself and patient conditions influencing factors affecting outpatient volume [17-19], such as the technical level of specialists, service attitude, advanced equipment conditions, occupation, economic income, literacy, and other conditions. Secondly, the analysis was done from the perspective of the type of disease and disease environment factors of outpatient visits. For example, [20] explored the effect of atmospheric pollution and the number of adult internal medicine outpatient visits and found that the increase in the concentration of atmospheric pollutants such as PM2.5 and PM10 was associated with an increase in the number of outpatient visits for respiratory and circulatory diseases. Reference [4] analyzed the lagged effect of weather pollutants on the number of outpatient visits for respiratory diseases in children. In the study of factors influencing outpatient volume, the analysis of hospital and patient factors mostly used statistical methods and the findings were mainly representative of individuals. Studies on the association between environment and specific disease outpatient visits usually make a predetermination based on experience, such as the association between respiratory disease outpatient visits and air pollution, and then conduct a follow-up study.

3. PSO-Based LSTM Neural Network Prediction Model Construction

3.1. LSTM

For the hospital's outpatient volume historical record data, a prediction model is built using deep learning methods to make reasonable predictions of short- and medium-term outpatient volume changes. The outpatient volume of each period in this case belongs to time series data, and RNN in the deep learning method is a kind of neural network used to process series data. LSTM [17], as a special RNN model, mainly adds the long short-term memory unit to the conventional RNN hidden layer, so that it can use the long-range sequence data more effectively and avoid the gradient disappearance problem of conventional RNN when processing long sequence data. The structure of LSTM memory unit is shown in Figure 1.

Figure 1

LSTM memory cell structure.

The LSTM memory unit structure generally consists of input gates, forgetting gates, and output gates. When the gate output is 0, it is equivalent to discarding the information because its multiplication with any state value is 0. When the gate output is 1, it is equivalent to keeping all the information by the neuron. In Figure 1, X (…, t − 1, t, t + 1,…) is the input data at different times, h(…, t − 1, t, t + 1,…) is the corresponding network output, and the hidden state C (t) represented by the long horizontal line above is called the cell state.

In the forward propagation phase, the memory unit at time t receives the input x at that moment with the output h of the hidden layer at the previous moment as the input to the forgetting gate, which determines the information to be discarded versus retained through the activation function layer, output f:

The information that the neuron needs to be retained is implemented by an input gate, which is calculated in two parts. An update value is calculated using the sigmoid function i and a new candidate value is generated using the tanh function , the product of which is used to update the cell state:

The cell state update contains the product of the forgotten gate output and the cell output at time t − 1 plus the product of the input gate output and the candidate value:

The update of the final output gate, that is, the hidden layer output at time th, is determined by

3.2. PSO Algorithm

The LSTM network prediction model based on deep neural networks contains multiple layers of networks, each layer of which contains a certain number of neurons, and the parameters, such as step size of the LSTM network, make it difficult to set and optimize the hyper parameters of this network prediction model. In this paper, heuristic optimization algorithm is used to optimize the hyper parameters in the model, and PSO becomes one of the most commonly used heuristic optimization algorithms because of its simple principle, low computational resource consumption, better convergence, and high computational efficiency.

The algorithm uses a population of particles, each with its own position and velocity. The iterative process of the algorithm is a process in which all particles continuously update their positions in the design space and exchange position information with each other until they search for the global optimum. In the standard swarm optimization algorithm, for a D-dimensional optimization problem, x( denotes the position of the ith nd particle and v( denotes the velocity. p( and p( denote the historical optimal position of the particle and the optimal position found by the whole population, respectively. Each particle will find its own optimal solution and the global optimal solution in the design space through continuous iterations. The whole population is updated with velocity and position by

Equation (5) contains the sum of 3 parts. The first part is the product of the velocity of the previous generation particles and the inertia factor ω; the second part characterizes the individual search behavior of the particles, which motivates them to continuously search for the individual optimal solution; the third part characterizes the information interaction between the particles, which jointly search for the global optimal value, where, c1 and c2 are the individual learning factor and the social learning factor, respectively, and, r1 and r2 are the random numbers within [0, 1].

3.3. PSO Optimization-Based LSTM Neural Network Prediction Model

In this study, LSTM network prediction model is constructed and the key parameters in the LSTM are optimized using the PSO algorithm, and the model is finally applied to hospital departmental outpatient volume prediction. The modeling and optimization process is shown in Figure 2.

Figure 2

PSO based LSTM outpatient volume prediction process.

In LSTM model, the number of neurons in each layer of the LSTM network is set as parameter k, and these parameters are used as the candidate solution vectors for particle swarm optimization. The objective function is set as the outpatient volume prediction error of the LSTM model, and the prediction error is minimized by iterating the algorithm to find a set of optimal parameters of the model, and finally the LSTM model based on particle swarm optimization is obtained for the outpatient volume prediction task.

4. Outpatient Volume Forecast

4.1. Datasets and Data Preprocessing

The outpatient volume projection data was obtained from our hospital and selected from orthopedic outpatient clinics related to Chinese medicine chronic diseases, such as osteoarthritis of the knee. Orthopedics is also one of the largest outpatient departments among all departments, so the management of orthopedic outpatient clinics is very important. The data contains the outpatient volume data of the orthopedic department for each month from January 2012 to October 2018, which is a continuous time series of 82 months in length. Figure 3 illustrates the change in orthopedic clinic volume over this time period. As can be seen from the trend of the fold line, the orthopedic outpatient volume shows a certain cyclical pattern by year, which is due to the significant influence of holidays on outpatient volume, such as the Chinese New Year period, which is a low point of outpatient volume every year, while holidays such as the National Day also lead to smaller outpatient volume in that month.

Figure 3

Hospital outpatient volume from 2012 to 2018.

As shown in Figure 3, this outpatient volume data is a nonstationary series with large fluctuations from a minimum of 20,194 to a maximum of 54,070 visits, with multiple peaks and valleys in each year. Therefore, data preprocessing was performed before making predictions. The data are normalized using (6), which scales the original data equally within [0, 1], where Xnorm is the normalized data, Xmax is the maximum value in the sample, and Xmin is the minimum value in the sample.

After data preprocessing, the dataset was divided into training and test sets. Among them, the last 12 months' (October 2017–October 2018) visit volume was selected as the final test data, and the remaining part was used as the training data. The optimized LSTM model will be trained using the training set, and the test set will be used for outpatient volume prediction.

4.2. Evaluation Indicators

To check the final prediction of the model, the Root Mean Square Error (RMSE) is used to evaluate the prediction results. As shown in (6), RMSE is the square root of the ratio of the square of the deviation of the predicted value from the true value y to the sample size n. The evaluation index reflects the deviation of the predicted value from the true value, is sensitive to large errors, and can better reflect the prediction accuracy:

4.3. Experimental Results and Analysis

In this example, parameters of the LSTM model selected for particle swarm optimization, including the backtracking time step n and the number of neurons in three-layer neural network k1, k2, k3, are four key parameters, and the optimization objective of the particle swarm algorithm is to minimize the prediction error RMSE of the LSTM network.

The key parameters of LSTM with iterations of 1, 25, and 50 are selected to plot the prediction curves of outpatient volume, as shown in Figure 4. The training and test sets are plotted together on the same graph (solid line in Figure 5), and the prediction curves of the model on the training and test sets are plotted separately (dashed and dotted lines in Figure 5). The prediction curves do not completely cover the time range of the original data curves because the backtracking time step n in the LSTM model is greater than 0. The algorithm needs n months of visit data to predict the subsequent months of visits.

Figure 4

PSO iteration curve.

Figure 5

Prediction curve of LSTM model with different number of iterations.

In the initial stage of LSTM model parameter optimization, the model fits the training set data well, but the prediction effect on the test set is poor, and the model has overfitting phenomenon. When the optimization iteration reaches 25, the prediction curve on the test set starts to be close to the test set data, although the training set does not fit as well as it did at the beginning, and when it reaches 50 iterations, the training set prediction curve is very close to the original curve after 2015, and the prediction value on the test set is also closer to the true value.

After the optimization of the particle swarm algorithm, the optimized parameters n = 6, k1 = 69, k2 = 128, and k3 = 58 were finally selected to train the LSTM prediction model. The comparison of the training and testing prediction curves and the true value curves of the optimized LSTM model based on orthopedic outpatient data is shown in Figure 6. As can be seen in Figure 6, the model has increasingly high fitting accuracy in the late training period as the training progresses, and the prediction curves almost overlap with the true value curves, and the model accurately fits the fluctuations of each year. The model also has good prediction results on the test set. Among them, the prediction curve well captures the precipitous drop in outpatient volume in January-February 2018, which is the Chinese new year and therefore causes a decrease in outpatient volume in February 2018. The prediction is especially effective in the rebound phase of outpatient volume, which almost overlaps with the original curve. In addition, the prediction curve also successfully predicted the trend of several peaks and valleys of orthopedic outpatient volume in 2018, as well as the eventual decrease in outpatient volume in October due to the National Day holiday. Overall, the model prediction values in the test set have some errors with the real values, but the errors are controlled within 10%. The model is able to capture the overall trend of outpatient volume and has some application value for predicting the future trend of increase or decrease of outpatient volume.

Figure 6

Outpatient prediction curve of LSTM network based on particle swarm optimization.

Table 1 shows RMSE values of the LSTM on the training set and the test set for different PSO iterations. It can be seen that the LSTM model tends to have a small RMSE on the training set at the initial stage of particle swarm optimization because the optimization goal is to minimize the RMSE on the test set, but the optimization process decreases the RMSE on the test set. The final optimized LSTM model's RMSE on the test set is reduced by 2,927.64 compared to the initial one, which is a significant optimization effect.

Table 1

Prediction error of LSTM model with different PSO iterations.

PSO iterations	Training set RMSE	Test set RMSE
1	267.22	6037.13
25	1025.09	4567.78
50	1043.45	3109.49

4.4. Model Training

As shown in Figure 7, loss functions and accuracies of the training process of LSTM and DNN for the single layer of our model and the baseline scheme are shown, and it can be clearly seen that the LSTM of our model converges faster than the baseline scheme, and the training process is stable. Our method has the best convergence and starts the model training finished at 27k steps, while the latest DNN requires 32k steps. The accuracy of our solution increases linearly and stays in the lead during the training process, although some jitter is within the allowable range. In general, from Figure 7, we can see that our method is fast converging and stable because our different channel settings can learn adaptively, which not only deals with the heterogeneity of multisource data but also makes the model training stable and avoids the problem of gradient disappearance caused by the model being too deep. In contrast, the single-layer LSTM training set has unstable accuracy, so the convergence is poor, and the generalization ability is insufficient, which shows that the design idea of our framework has the effect of reducing overfitting, and our framework has better generalization ability than the single-layer LSTM, and the final training set has the highest accuracy.

Figure 7

Loss and accuracy during model training (a). Model convergence demonstration. (b) Model accuracy display.

4.5. Accurate Outpatient Volume Forecasting

The outpatient volume data provided by our hospital is selected, and 40 sets of data are known, 30 of which are used as the training set and 10 as the test set. The prediction was done by selecting 1700 visits in chronological order from far to near as the training set of the model, predicting 300 steps of visits, and then comparing it with the known 300 visits corresponding value to determine the size of the error value and dividing it into two test sets for easy verification. Set the batch size = 4, the number of neurons to 128, the window size to 350, the training to 6000 times, and the prediction step size to 300, and carry out the simulation. The results are shown in Figure 8.

Figure 8

Prediction of outpatient volume with LSTM.

The results of the outpatient volume prediction for the two groups are shown in Figure 8. It can be seen that the curves of each group are different and fluctuate greatly but show a certain periodicity, and the predictions of the LSTM-VAE model proposed in this study throughout the time period are in good agreement with the real numerical curves, indicating that the proposed model can predict different outpatient volume scenarios and then provide a reasonable basis for resource optimization for railroad managers.

In order to visualize the prediction accuracy of the model in this paper, the predicted and true values are reported as shown in Table 2, and it can be clearly seen that the error values of both are very small.

Table 2

Prediction and true values of different groups (partial).

Group	1	2	3	4	5	6	7	8	9	10
Predicted value	0.635	1.813	3.636	0.612	1.655	2.689	0.425	2.759	0.565	3.235
True value	0.645	1.835	3.635	0.635	1.643	.652	0.598	2.622	0.675	3.335

5. Conclusion and Future Work

Outpatient volume prediction, which is an important part of outpatient management, affects the rational allocation and optimization of medical resources in hospitals. In this paper, we have used deep learning-enabled methods to construct a long- and short-term memory network model for outpatient volume prediction for the monthly outpatient volume records of the orthopedic department. To solve the problem of hyper parameters and difficulty in optimization of network model, a particle swarm optimization-based long- and short-term memory network along with particle swarm optimization algorithm model was developed. The results of the proposed algorithm test show that although there is still room for improvement in the final prediction accuracy, the prediction RMSE of is reduced by 2,928 through particle swarm optimization, and prediction error is reduced by 48.5% compared with the original model. The optimized prediction model has better predicted the trend of outpatient volume changes and has a strong ability to capture the scenario of rapid changes in outpatient volume. In summary, the proposed temporal data prediction process based on optimization algorithm and neural network can be applied to the outpatient volume prediction of various departments in hospitals and can be easily extended to the application of serial data in medical institutions at all levels to provide technical support for medical staff to make better use of medical data.

In the next phase, the study will include more factors influencing outpatient volume in the prediction model, such as holidays, weather, and hospital geographic location, to establish a more comprehensive prediction model and provide more accurate predictions, in order to further explore the practical application value of medical time-series data.