Using Machine Learning to Predict Early Preparation of Pharmacy Prescriptions at PSMMC - a Comparison of Four Machine Learning Algorithms.

BACKGROUND: Patient satisfaction is one of the primary Key Performance Indicator (KPI) goal of health care service, and it creates many reasons for implementing research, plans, and innovations to achieve it for a better quality of life. Cutting Patient waiting time would increase patient satisfaction.
OBJECTIVE: A healthcare framework has been constructed utilizing a machine learning approach to construct an early predicting preparation model of pharmacy prescriptions and the worthiness of changing the outpatient pharmacy workflow.
METHODS: Data sets were retrieved between Januarys and June 2019 from Prince Sultan Military Medical City, Riyadh, KSA, for all patients who visited the clinics or discharged with pharmacy prescriptions. Included (1048575) instances and composed of (11) attributes. The evaluation criteria to compare the four algorithms were based on precision, Recall, True Positive Rate, False Negative Rate, F-measure, and Area under the curve.
RESULTS: Overall, 94.88% of patient's shows at the pharmacy, female represents 58.89% of the data set while male represents 41.1%. RT gives the highest accuracy, with 97.22% in comparison to the other algorithms.
CONCLUSION: The suggestion to change the pharmacy workflow is worth increasing patient satisfaction and overall the quality of the care.

BACKGROUND

Patient satisfaction is one of the main Key performacne Indicator (KPI) goal of health care service, and it creates many reasons for implementing researches, plans, and innovations to achieve a higher level of patient satisfaction to ensure a better quality of life. Patient waiting time has been defined as “the length of time from when the patient entered the pharmacy to the time the patient received his or her prescription and left the pharmacy (1). The time spent is more to get to the examination, treatment, lab examination, insight, or nursing care for specific contamination, infection, illness, pernicious substance (2). Exorbitant waiting times might be one of the severe issues in the hospital’s framework and ought to be tended to as a significant aspect of good administration practice. In a study conducted at the University of Southern California, Los Angeles, the United States, it was shown the relationshhip between the overall satisfaction of patients and waiting time for pharmaceutical services (3). One of the main disfactional between patient was the Long waiting to receive the medications (4). These show the significance of waiting time on pharmacy services and patronage.

Patients have different visit units /time inside the hospital. However, a high level of outpatients goes to the pharmacy inside the hospital for their prescriptions. These patients leave the doctors’ facilities and any of the different units on different occasions, subsequently establishing an arbitrary entry rate at the pharmacy, where the dispensing activity happens (7).

Healthcare framework constructed utilizing machine learning is created with a tremendous number of patient records. With immense development over the Big Data lately made it conceivable to oversee the enormous gathering of record series. Therefore, with the massive data sets, a machine can be assembled which can become familiar with the information, bunch them, order them and extricate them at whatever point it is required. Hence, by utilizing different machine learning calculations, we can prepare a system with the information. It accomplishes something over a typical database framework that must be alluded to as opposed to making an investigation over the information (8).

Machine Learning algorithms perform calculations and learn from information to gain learning to perform forecasts. Health Informatics ponders the effective utilization of probabilistic data for essential leadership (9). Utilize machine learning in the health informatics domain has the best possibilities to enhance the quality, efficacy, and efficiency of treatment and care. Wellbeing frameworks worldwide are stood up to with enormous information in high measurements, where the consideration of a human is unimaginable, and automatic Machine Learning shows enormous outcomes. Nonetheless, at times are defied with complex information, little information, or uncommon occasions, where approaches suffer from insufficient preparation tests (9).

Predictive analytics alludes to creative strategies for investigation created to conquer difficulties related to big data, including an assortment of measurable procedures running from predictive analytics to information mining to machine learning. Since predictive analytics can be utilized in foreseeing many results, they can give pharmacy-informatics specialists a superior comprehension of the dangers for specific prescription-related issues that every patient countenance. This knowledge will empower pharmacists to convey intercessions customized to patients’ requirements. Hence, to exploit these advantages, clinicians should comprehend the fundamentals of big data, machine learning, and predictive analytics (10).

Literature Review

Medication usage is known as critical since it is affecting the patient’s quality of life. Fernando focused the study on assessing the effectiveness of improving the general performance of prescription adherence (11). This was initiated because the effect of the growing use of technology in the health sector signifies the issue of prescription discharge and adherence is of immense relevance presently. This could be achieved by focusing on three main areas that are: Waiting period, patient satisfaction, and the compliance rate.

The selection of participants were from a group of emergency department patients, who were randomly allocated to a control group (received written prescriptions) and a testing group (received an electronic prescription). The follow-ups were taken over a week to receive the required data related to prescription compliance and adherence. In case of non-compliance, the reasons were collected and analyzed.

The procedure, under reliable test conditions, for both groups indicated no correlation of electronic prescriptions with an increase in non-compliance. The data collected was further reviewed to analyze the reasons for non-compliance, such as lack of affordability of medicines and the inability to purchase the medicine. None of these indicated any correlation or causation, with waiting time being a significant factor. Replacement of traditional prescriptions with electronically placed prescriptions, however, indicated a decrease in the waiting time and, consequently, higher customer satisfaction.

The study by Fernando signaled towards not only the areas of improvement and efficiency of introduced E-prescriptions but also related to the explanation for non-compliance (11). The results are significant for future studies and implications of the introduction of E-prescriptions in the health sector where sample collection is limited due to which the extent to which the results could be applied and generalized practically In another study, it shows the efficacy of using machine learning; the study conducted reviewed a significant area of effects of machine learning on medication adherence. This was done concerning the fact that lack of compliance/ adherence to the medication prescribed lead to casual adverse effects and long-term deficiencies, especially in chronic diseases.

The study focused on reviewing the effects of machine learning on adherence rates as compared to the traditional data analysis driven by distribution methods. Thus, the hypothesis was based on shifting away from the established 80% adherence rate, which is classified as feasible, as it lacks historical data to validate its generic application on all diseases. The method taken up for collection was by use of a ‘random and fit survival tree,’ to acquire relevant data and predictors for prevention of hospitalization risk in case of non-adherence.

It was observed as per the study, that the predictors post-identification indicated a varying marginal adherence threshold in various patients. With the initial baseline, 80% established as per prior studies, that was tested against, the adherence rate did not show a specific number or trend but ranged between 46%–94%. Hence, the study indicated that machine learning through the application of algorithms could be useful in not just preventing hospitalizations but also in increasing the understanding of the reasons/causes that might lead to it. Therefore, the study holds immense significance in terms of health sector progression, especially for patient-specific analysis and action as compared to a generic adherence threshold appliance (12).

The study aims to comparatively evaluate the performance of the machine learning-based models in predicting early preparation of pharmacy prescriptions - the worthiness of changing the outpatient pharmacy workflow.

Afolabi et al. (5) studied the waiting time at many hospitals. The study used a workflow analysis method. They grouped workflow into two sub-components “process” and “delay.” A process component involved a staff member actively working on the prescription, while a “delay” component involved the prescription lying idle and waiting for a staff member to work on it and they found that, delay components of the dispensing procedure can account most of the patient waiting time in the hospital. However, another study conducted to analyze the prescription-filling process concluded that no significant additional reductions in waiting times could be achieved (6).

OBJECTIVE

A healthcare framework has been constructed utilizing a machine learning approach to construct an early predicting preparation model of pharmacy prescriptions and the worthiness of changing the outpatient pharmacy workflow.

METHODOLOGY

Process of dispensing medication

Prince Sultan Military Medical City (PSMMC) is in Riyadh city, with tertiary medical services. PSMMC provides healthcare for 2,000 patients per day on average. At PSMMC, Pharmacists are usually under immense pressure to find ways to handle efficiently more prescriptions and provide extended counseling and healthcare services for their patients. On the other hand, patients demand high quality, efficient, and effective pharmaceutical care in a short time. The patient’s medications are dispensed from many pharmacies, yet the average dispensing from the main outpatient pharmacy is 1,300 patients daily. One of the major complaints from the patients is a long waiting time. As a result, many patients left their prescriptions and did not collect their medications. However, it is not good to make sick people wait for an unreasonable length of time.

The process of dispensing medication is multi-dimensional. Each process may be associated with waiting time, adding to the total waiting time to dispense the medicine. These process can be shown in Figure 1.

Figure 1.

Process of Dispensing Medication at PSMMC

The Pharmaceutical services department administration acknowledged the issue and suggested to modify the outpatient pharmacy workflow to start preparing the prescription before the arrival of the patient. Thus, one of the aim of this research study is to utilize machine learning algorithms to improve the effectiveness of preparing pharmacy prescriptions before the patient arrived at the outpatient pharmacy.

Data set and Feature

The data sets were retrieved between January and June 2019 for all patients who visited the clinics or discharged with pharmacy prescriptions. Included (1048575) instances and composed of (11) attributes which are most important characteristics of prescriptions preparation namely Age Group, Gender, Percentage Show of visit ID, Percentage No Show of visit ID, visit Flag, Temperature, Allergy, Pregnancy, Department No., Drug code, and Class. There were a few steps done for data preparation before starting the analysis. All incomplete records were excluded.

Data preprocessing: define each attribute, correlate the relevant attributes, irrelevant symbols, texts removed, anonymization of the patient’s information for privacy and security purposes.

Evaluation criteria

A 10-fold cross-validation method was applied. It is a statistical technique working by partitioning the dataset into ten folds with equal size. Nine folds used for model training and the tenth used for model testing. After the tenth iterations were finished, the ten results averaged into a single estimation (15). For choosing the best model, some of the metrics had been applied are True Positive rate, False Positive rate, Precision, Recall, Area Under the Curve, and F-measure. The metrics were calculated as:

True Positive Rate (TPR): represents the number of patients who were classified as show at pharmacy correctly.

False Negative Rate (FNR): represents the number of patients who were classified as did not show at pharmacy incorrectly.

Precision: represents the percentage of visited patients that classified as “show” and they were “show,” and it is calculated based on formula 1

Precision = TP/(TP+FP) (1)

Recall: represent the percentage of visited patients that classified correctly, and it is calculated based on formula 2

Recall = TP/(TP+FN) (2)

F-score is the representation of the harmonic mean of precision and recall, and it is calculated based on formula 3

2 * (precision*recall) / (precision+ recall) (3)

ROC: Receiver Operating Characteristic (ROC) Curve: It is a graphically way to display true positives versus false-positives across a series of cut-offs, and select the optimal cut off for clinical use. (13)

In this study, the Weka software tool has been used to select the best algorithm with high accuracy results. No. and Class. Then the models to be applied chosen depends on the capabilities of it, J48 can handle nominal, binary, and miss-values classes. For attributes, also can handle binary attributes, date attribute, missing value, nominal attribute, numeric attribute, and unary attribute.

Machine Learning Algorithm

Random Tree is similar to J48, with extra capabilities to handle the numeric classes.

Multilayer Perceptron Class: Binary class, Date class, Missing class values, Nominal class, Numeric class. For Attribute: Binary attributes, Date attributes, Empty nominal attributes, Missing values, Nominal attributes, Numeric attributes, Unary attributes.

AdaBoost Class: Binary class, Missing class values, Nominal class.

Attributes: Binary attributes, Date attributes, Empty nominal attributes, Missing values, Nominal attributes, Numeric attributes, Unary attributes (14).

RESULTS

As mentioned above, the total of (1048575) instances included with 11 attributes was used with four algorithms RT, J48, AdaBoost, and multilayer perceptron (MLP). RT gave higher accuracy with 97.22 % compared with J48 97.05 %, AdaBoost 96.16 %, and for the neural network (MLP) 96.73 %.

DISCUSSION

The main aim of the research study is to create a model to predict an early preparation of pharmacy prescriptions, the majority of patient’s shows at the pharmacy after attending the outpatient clinic. RT gives the highest accuracy in comparison to other models. According to Jenkins Eckel (16) study held in 2012, the average waiting time for the prescription is 40 minutes and mostly 21 minutes consumed before start processing the prescription. So, changing the pharmacy workflow to be preparing the prescription before patient arrival to the pharmacy might increase patient satisfaction and enhance the overall quality of care.

The quality of data is essential in machine learning. Missing data, inaccuracy and inconsistency plays the primary role in the model performance.

Limited evidence researches to support and compared to this study

CONCLUSION

Waiting time is one of the significant factors that affect patient satisfaction and overall health services. Machine learning becomes an essential method for processing and modeling human-related work in multiple disciplines and especially in the healthcare sector. Upon Saudi vision 2030 to lead artificial intelligence (AI) in the region, which determine to build the future based on AI and all the digital capabilities, using ML with available data will have a positive impacct on the quality of services provided to the patients.

Suggest to increase research in this field, increase the availability and accuracy of data.

Suggest to implement the model and increase efficiency in multiple ways.

Table 1.

Described the attributes of the data

Attribute	Description
Age group	18 patients age group example: (1-4,5-9.10-14 … etc)
Gender	0= female,1=male.
Percentage Show of visit ID	Percentage of the show that ID generated for the prescription
Percentage No Show of visit ID	Percentage of not show that ID generated for the prescription
visit Flag	OP= for outpatient clinic, IN= discharge patient.
Temperature	Patient temperature
Allergy	Unknown = there is no allergy specified by patient OR not entered in the patient profile Yes = confirmed allergy by patient
Pregnancy	0= not pregnant,1=pregnant.
Department No.	No. describes the department and specialty of the clinic.
Drug code	Grouped from 1-161.
Class	Either show or not show.

Table 2.

Described the show status

Count	Class
58145	No Show
1089165	Show

Table 3.

describes other attributes show status

Show	No Show
		Gender
644963	32233	Female
444202	25912	Male
		VisitFlag
75210	3483	IN
1013955	54662	OP
		Allergy
951692	51409	Unknown
137473	6736	Yes
		Pregnancy
107772	3006	Unknown
966500	54337	No
14893	802	Yes

Table 4.

describes the algorithms results.

	RT	J48	AdaBoost	MLP
TPR	0.992	0.993	0.974	0.991
FNR	0.390	0.451	0.274	0.471
Precision	0.979	0.976	0.985	0.975
Recall	0.992	0.993	0.974	0.991
F-score	0.985	0.985	0.980	0.983
ROC	0.950	0.963	0.974	0.980