BPLT+: a Bayesian-based personalized recommendation model for health care.

In this paper, we propose an Advanced Bayesian-based Personalized Laboratory Tests recommendation (BPLT(+)) model. Given a patient, we estimate whether a new laboratory test should belong to a "taken" or "not-taken" class. We use the bayesian method to build a weighting function for a laboratory test and the given patient. A higher weight represents that the laboratory test has a higher probability of being "taken" by the patient and lower probability of being "not-taken" by the patient. For the sake of effectiveness and robustness, we further integrate several modified smoothing techniques into the model. In order to evaluate BPLT(+) model objectively, we propose a framework where the data set is randomly split into a training set, a validation input set and a validation label set. A training matrix is generated from the training data set. Then instead of accessing the training data set repeatedly, we utilize this training matrix to predict the laboratory test on the validation input set. Finally, the recommended ranking list is compared with the validation label set using our proposed metric CorrectRateM. We conduct experiments on real medical data, and the experimental results show the effectiveness of the proposed BPLT(+) model.

Background

Large amounts of clinic laboratory test data are collected and stored every day. Therefore, there is an increasing need for analyzing and utilizing the laboratory test data. The problem we are working on in this paper is to recommend laboratory tests for given patients. Health care recommendation problems have drawn researchers' attention for years. However, there are not a lot of studies conducted on the clinic laboratory test recommendation problem.

The medical data we are working on contains several years patients' laboratory test records. Figure 1 shows an example of the data format. Formally, the laboratory test prediction problem can be described as follows [1]: "Given a set of patients P = {p1, p2, ..., pand a set of laboratory tests T = {test1, test2, ... test, each patient phas done tests test, ..., test. If a doctor would like to assign a new test for patient p, which test in T should be chosen?"

Figure 1

An example dataset. The format of the laboratory data sets is presented: the attributes from left to right are SDTE (SERVICE DATE), REQ# (REQUISITION NUMBER), PNUM (PATIENT HEALTH CARD#), PNAM (PATIENT NAME), PSEX (PATIENT SEX), BDTE (PATIENT DATE OF BIRTH), TSEQ (TEST SEQUENCE NUMBER), TEST (TEST CODE), DESC (TEST DESCRIPTION), RSLT (TEST RESULT), NORM (NORMAL RANGE), REXP (RESULT EXPECTED Y/N), EXRS (EXTENDED RESULT Y/N). The patient information in this table is fake due to privacy.

The computer systems have been playing for an important role in health care for years [2-8]. Statistic algorithms [9-12] lead an important role in investigating health care data. [13,14] extracts chemical keywords from a query patent by analyzing word frequency and the word's effect over the data collection. Bayesian learning is a widely used algorithm that shows good performance [15-19]. A semantic-based association rule mining approach is proposed to model the medical query contexts in [20]. Using a novel classifier based on the Bayesian discriminant function, Raymer, M. L. [21] present a hybrid algorithm that employs feature selection and extraction to isolate salient features from large medical and other biological data sets. Martín and Pérez [22] analyze the robustness of the optimal action in a Bayesian decision making problem in the context of health care. [23,24] studies the association between two words by simulating the impact of words in documents in the context of information retrieval. A probabilistic survival model is derived from the survival analysis theory for measuring aspect novelty of genomics data [25]. A mixture markov model is proposed to investigate user navigation patterns so that a personalized recommendation system for each user can be built [26]. In our previous work [1], we propose a laboratory test prediction model, which would objectively determine whether a laboratory test is associated to a patient. This paper is a significant extension to [1].

Smoothing [27] is a technique to create an approximating function that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena. The smoothing techniques have been used in many realms to improve the accuracy [28]. Based on the basic Bayesian algorithm and smoothing techniques, we propose an Advanced Bayesian-based Personalized Laboratory Tests recommendation (BPLT+) model, to investigate the correlation among laboratory tests for each patient. Evaluation is a crucial issue in the health care domain [29]. Some previous health care researchers do evaluation via patient interaction [30] or statistics [31]. We present a metric CorrectRateby employing the idea of Mean Average Precision (MAP) [32] in Information Retrieval domain.

Four unique contributions are presented in this paper. Firstly, we learn the associations among laboratory tests and make personalized recommendations to patients without human interaction. Secondly, we integrate modified smoothing technologies to improve the personalized recommendation model and propose the BPLT+ model. Thirdly, we propose a framework to randomly generate a training data set, a validation input set and a validation label set. Fourth, we use a objective evaluation metric for personalized recommendation systems without patient interaction.

Methods

Bayesian-Based personalized laboratory tests recommendation (BPLT) model

Here we assume that the laboratory tests for a patient have associations among each other. For instance, if a patient is suspected to have diabetes, usually the doctor will assign both Hemoglobin test and Glucose Fasting test for this patient. We can see that there exists an association between Hemoglobin and Glucose Fasting with respect to some hidden information, diabetes in this case. On the other hand, if a patient is assigned Hemoglobin test, then it is very likely that this patient should also take Glucose Fasting test. In this section, we build a model for learning the associations of the laboratory tests, inferring the associations between patients and laboratory tests, and therefore recommending new laboratory tests to the patients. We regard the test recommendation problem as a special classification problem, where a test belongs to either a "taken" or "not-taken" class. We use Bayesian classifier as our basic classifier, and modify it to a personalized ranking model.

Basic concept: Bayesian classifier

A classification problem is the following [33]: given a set of training instances, each described with a set of n attributes and each belonging to exactly one of a certain number of possible classes, learn to classify new, unseen objects. In addition, each attribute has a fixed number of possible values. We use naive Bayesian classifier as our basic classifier in this paper, since it evaluates directly the probability of taking a test and the conditional probability among two tests. Moreover, naive Bayesian is easy to construct and has surprisingly good performance in classification, even though the conditional independence assumption is rarely true in real-world applications [34]. The probability model for a classifier is a conditional model

where F1, ..., Fare attributes, and C is a class variable. By Bayesian criteria, it equals to

The denominator is effectively constant, and the numerator is equivalent to the joint probability model

In naive Bayesian, it assumes the features are conditional independent

Therefore, the probability of a class C given feature F1, ..., Fis

where is a constant.

The weighting function of BPLT model

In this Section, we describe the Bayesian-based Personalized Laboratory Tests recommendation (BPLT) model, which was proposed in our previous work [1]. More details are given in this paper. The purpose of BPLT model is to classify the laboratory tests for individual patients by their personal conditions. In the real world, it is often easier to obtain the patients' previous laboratory tests information. Therefore, the BPLT model recommends additional new laboratory tests to patients, given the previous laboratory tests that the patients have taken.

Suppose we have a set of M laboratory tests T = {test1, test2, ..., test}, and a patient pwho has taken tests T= {test,1, ..., test} where test∈ T for all 1 ≤ i ≤ k. We denote the events that tests in Tare taken by pas F,1, F,2, ...F. For example, if we have 7 tests in T, and phas taken test3, test5 and test7 could be represented as (F,1, F,2, ..., F,7) = (0, 0, 1, 0, 1, 0, 1). Bayesian Classifier is employed to evaluate the association between pa new test test0 where test0 ∈ T and test0 ∉ T. We use Fto represent the event of pshould take t0, and to represent the event of pshould not take t0. By Formula (3), the probability of F,0 given F,1, Fis

The probability of given F,1, F,2, ... Fis

In the BPLT model, we reward the tests with high probability of "taken" and low probability of "not-taken". The correlation between a new test test0 and a given patient pis shown in Definition 1 [1].

Definition 1 The correlation between a new test test0 and a given patient p0 divided by the probability of p0 given F, F,2, ... F.

We can see that higher value of corr(test0, p) indicates that test0 has more association with p. The calculation of corr(test0, p) can be further simplified as follows

Moreover, a test either belongs to a "taken" class or a 'not taken" class. Thus, the following two formulas are held.

from which we can obtain and

Thus and in (5) can be eliminated in corr (test0, p), as shown below

A joint probability for patient ptake both of the tests testand test0 is

The definition of the correlation between test0 and pis

which leads to the following Definition 2 [1].

Definition 2 The weighting function for a laboratory test test0 for a patient p0 and p

where

The new laboratory tests will be ranked in a list according to w(test0, p) for a given patient p. In the later section, we will present the evaluation environments for the laboratory test ranking list.

An advanced model: BPLT+

To have a more robust and better performance model, we further propose an advanced model, BPLT+, by improving the BPLT model using several smoothing techniques. There are two reasons for smoothing BPLT. One reason is that smoothing is a way to deal with noise within the data. Another reason is to avoid the mathematically meaningless. When test0 laboratory test has not been observed in the previous visits, which means α = 0, the first part of formula (6) will become an irrational number. Meanwhile, when the joint frequency of two laboratory tests is zero, which means β= 0, the second part of (6) will become an irrational number. Therefore, we introduce smoothing technologies to further improve BPLT model.

Smoothing techniques

In statistics, smoothing [27] is a technique to create an approximating function that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena. The main purpose of smoothing in this paper is to assign a non-zero probability to the unseen tests and improve the accuracy of test probability estimation in general.

The smoothing techniques are discussed based on the following definitions of a conditional probability [28].

where c(t;p) is the count of a patient taking a test. Here are some commonly used smoothing methods. Since we have defined a ranking problem, which is similar to the problems in Information Retrieval (IR), we use some widely used smoothing methods in language model in IR. The general form of a smoothed model [35] is assumed to be the following:

where Pr(t|p) is the smoothed probability of a test t given the patient with existing tests. Pr(t|C) is the probability of a test t given the whole data set.

A smoothing method may be as simple as adding an extra count to every test, which is called additive or Laplace smoothing, or more sophisticated as in Katz smoothing, where tests of different count are treated differently. Three representative methods that are popular and effective are:

• The Jelinek-Mercer method

where λ is a balancing parameter ranges from 0 to 1.

• Bayesian Smoothing using Dirichlet Priors

where µ0 is a balancing parameter, and µ0 >0. The Laplace method is a special case of this technique.

• Absolute Discounting

where δ ∈ [0, 1] is a discount constant and σ = δ|p|, so that all probabilities sum to one. Here |p|is the number of unique terms in document d, and |p| is the total count of words in the documents.

BPLT+ with smoothing techniques

There are two parts in formula (6) that need smoothing. The first one is the conditional probability β= Pr(F|F). Its smoothed format is as follows:

• BPLT+ with Jelinek-Mercer

• BPLT+ with dirichlet priors

• BPLT+ with absolute discounting

In Jelinek-Mercer BPLT+ and Absolute Discounting BPLT+, we use the existing smoothing method. The smoothing parameters λ, δ are within the range of [0, 1]. In Dirichlet Priors BPLT+, we modify the Dirichlet smoothing technique, by divide both the numerator and the denominator in (10) by , and normalize the parameter µ to the range of 0[1], where .

Another part in formula (6) needs smoothing is , which is a simple division that could be smoothed

via Laplace smoothing as

where θ is a tuning parameter ranges from 0 to 1.

Evaluation environments

Datasets

The datasets in our experiment are obtained from Alpha Global IT [1,36]. Alpha Corporate Group provides laboratory, medical clinic, commercial electronic medical record and practice management software. The data set contains 78 monthly patient's laboratory test results. Our experiments use 6 month results, containing 1,048,575 patients' records, as a key study. Thousands of patients' records and more than 400 laboratory tests are included in our experiments. The data format is the same as the example shown in Figure 1. Our data set contains real patients' information, such as health card ID, age, gender, date of visit, laboratory test ID, laboratory test results. We only use the patient ID and laboratory ID attributes in this paper, and analyze the associations among these laboratory tests. In our future work, we will incorporate more attributes in the laboratory recommendation model.

Validation data and measure

To evaluate BPLT+ models objectively, we divide the data set into three components: a training set, a validation input set, and a validation label set. The data set is firstly randomly split into a training set and a validation set. In this step, we split based on the patients and do not split the records from a same patient. Then for the validation set, we randomly remove one test tfrom each patient p, and store the tin the validation label set. The ranked list returned by BPLT+ will be compared with tfor each patient. To measure such comparison and finally evaluate the effectiveness of BPLT+, we use the following defined CorrectRate[1]. Suppose the returned laboratory ranking list is , CorrectRatevalidates whether tappears in the top ranked tests. The measure is modified from Mean Average Precision (MAP) [32] evaluation metric.

Definition 3 The CorrectRate.

where

n is the number of patients, X is a parameter indicating how many top tests are compared to the golden standard test t*, which is set to be 1 or 3 in this paper.

We present an example to show how the CorrectRateevaluates the model in Table 1. Suppose the laboratory test sets includes 200 tests and there are 5 patients in the validation set. As we have introduced, the BPLT+ model returns a ranked list for each patient. Here ">" represents that the weight of the left-side laboratory test is higher than the weight of the right-side laboratory test. In our example, 2 out of 5 patients have the desired test tranked in the top 1 position of the list, then CorrectRate1 equals 0.4. And 4 out of 5 patients have tappears within the top 3 positions of the returned ranking list, then CorrectRate3 equals 0.8. We can see that the top 3 positions include the top 1 position, so the following statement is always true: CorrectRate1 ≤ CorrectRate3.

Table 1

An example of CorrectRate

	t∗	Recommendation list	X = 1	X = 3
p1	test104	test104 > test5 > test40 > ...	TOP1,1 = 1	TOP1,3 = 1
p2	test30	test30 > test3 > test18 > ...	TOP2,1 = 1	TOP2,3 = 1
p3	test2	test95 > test2 > test34 > ...	TOP3,1 = 0	TOP3,3 = 1
p4	test95	test78 > test19 > test58 > ...	TOP4,1 = 0	TOP4,3 = 0
p5	test198	test92 > test134 > test198 > ...	TOP5,1 = 0	TOP5,3 = 1
All patients	--	--	CorrectRate1 = 0.4	CorrectRate3 = 0.8

This example contains a validation set of 5 patients, their desired laboratory test t*, the recommendation list, and the corresponding evaluation results.

BPLT+ System Framework

The framework of BPLT+ Model is shown in Figure 2. The data set in this framework is abstracted to contain only patient ID and laboratory test ID. The procedures in the proposed framework are described as follows.

Figure 2

BPLT. The procedures for processing the laboratory data and testing the BPLT+ model are shown: (1) the rectangles represent the data sets; (2) the rounded rectangles present the implemented procedures; (3) the ovals show the personalized laboratory model; (4) the lines with arrows determine the directions through the framework.

• Split: First the data set is randomly split into a training set and a validation set.

• Random Remove a test as label: Since it is hard to objectively evaluate the performance of the BPLT+ model, we further randomly remove a test for each visit of the patients from the validation set. These removed tests are regarded as labels of the validation set input. Our ultimate goal is to recommend the missing test for a patient's visit.

• Build training matrix: To avoid duplicate calculating the frequency of a test and the joint frequency between two tests, we build a training matrix out of the training data. This training matrix contains the frequency of co-occurrences of two laboratory tests. For example, if a patient in the training data did test1 and test2 together, then add 1 to F12 and F21. We can see that the training matrix is a symmetric matrix.

• BPLT+ model: The correlation of a given test0 and a patient is calculated based on formula (6).

• Evaluation via CorrectRate: Finally, the evaluation criteria CorrectRateevaluates if the model made the correct recommendations.

Results

We first show the overall performance under different training-validation proportion in Table 2[1]. We randomly take 40%, 50% and 60% of the data out of the raw data set as the training data and keep the rest as the validation data. In general, there is higher performance of BPLT+ model on a larger training data set. This is because the larger training data set contains more information, and more knowledge can be learned. With the development of computer technology, larger amount of medical data will be available in practice. Therefore, we will use 60% of data as training data in the rest of this paper. As we have discussed before, CorrectRate3 is always higher than CorrectRate1. In general, the BPLT+ model has promising performance with an accuracy of 0.7074 for CorrectRate1 and an accuracy of 0.7840 for CorrectRate3.

Table 2

Performance

Percentage of Training Data	CorrectRate 1	CorrectRate 3
60%	0.7074	0.7840
50%	0.6962	0.7837
40%	0.6823	0.7821

The overall performance of BPLT+ with different training-validation proportions.

Then we investigate how the smoothing parameters affect the effectiveness in detail. We first consider smoothing βonly. There are three smoothing technologies utilized to smooth β. They are Jelinek-Mercer BPLT+, Dirichlet Priors BPLT+ and Absolute Discounting BPLT+, with the corresponding parameters: λ, µ, δ ∈ [0, 1]. We conduct experiments on these three methods individually. The change of CorrectRate1 and CorrectRate3 with respect to the parameters are shown in Figure 3, Figure 4, and Figure 5. We can see from the figures that the curve of CorrectRate1 is always below the curve of CorrectRate3, which is consistent as we have discussed Definition 3. With the increasing of parameters from 0.1 to 1, both CorrectRate1 and CorrectRate3 become higher at the beginning due to the incorporating of the smoothing portion. After reaching the maximum value, CorrectRate1 and CorrectRate3 become lower, since the weighing would tend to be more universal when too much smoothing is incorporated. All the smoothing parameters achieve their best performance at the value of 0.2. Comparing among these three methods, Jelinek-Mercer BPLT+ obtains the best performance on both CorrectRate1 and CorrectRate3, which are 0.5569 and 0.6167. When it comes to the average value, Dirichlet Priors BPLT+'s average performance on CorrectRate3 is better than the other two, and Jelinek-Mercer BPLT+'s average performance on CorrectRate1 is the best.

Figure 3

Parameter Sensitivity of . The influence of parameter λ is investigated: (1) the stars represent the performance of Jelinek-Mercer BPLT+ under the evaluation metric CorrectRate1; (2) the circles represent the performance of Jelinek-Mercer BPLT+ under the evaluation metric CorrectRate3; (3) CorrectRate3 is always higher than CorrectRate1; (4) Jelinek-Mercer BPLT+ achieves its best performance when λ = 0.2.

Figure 4

Parameter Sensitivity of . The influence of parameter µ is studied: (1) the stars represent the performance of Dirichlet Priors BPLT+ under the evaluation metric CorrectRate1; (2) the circles represent the performance of Dirichlet Priors BPLT+ under the evaluation metric CorrectRate3; (3) Dirichlet Priors BPLT+ achieves its best performance when µ = 0.2.

Figure 5

Parameter Sensitivity of . The influence of parameter δ is investigated: (1) the stars represent the performance of Absolute Discounting BPLT+ under the evaluation metric CorrectRate1; (2) the circles represent the performance of Absolute Discounting BPLT+ under the evaluation metric CorrectRate3; (3) Absolute Discounting BPLT+ achieves its best performance when δ = 0.2.

We further discuss to smooth the second part of (6), where the Laplace smoothing parameter is θ. As we have discussed before, Jelinek-Mercer BPLT+ has the best performance on both CorrectRate1 and CorrectRate3. We focus on investigating the sensitivity of θ by fixing Jelinek-Mercer BPLT+ with λ = 0.2. The results are shown in Figure 6. We can see that the CorrectRate1 increases while θ is increasing, and the CorrectRate3 decreases a little and then increases. Both of them reach the maximum and tend to be stable when θ is greater than 0.5.

Figure 6

Parameter Sensitivity of . The influence of parameter θ is presented: (1) we use the best smoothing technique for the first part in Formula 6, which is Jelinek-Mercer BPLT+; (2) the smoothing parameter λ is set to be optimal; (2) the stars represent the results of Jelinek-Mercer BPLT+ under evaluation metric CorrectRate1; (3) the circles represent the results of Jelinek-Mercer BPLT+ under evaluation metric CorrectRate3; (4) both metrics reach the maximum and tend to be stable when θ is greater than 0.5.

Conclusions and future work

An Advanced Bayesian based Personalized Laboratory Tests recommendation (BPLT+) model is proposed in this paper. Based on the assumption that hidden association could exist among laboratory tests, we employ a Bayesian approach to build a weighting function for scoring the correlation between a new laboratory test and a patient. To have a more robust and better performance model, we employ several enhanced smoothing technologies into the BPLT+ model. The main purpose of smoothing in this paper is to assign a non-zero probability to the unseen laboratory tests and improve the accuracy of test probability estimation. We integrate existing smoothing techniques in the BPLT+ model. In particular, we use three techniques, Jelinek-Mercer, Dirichlet Priors and Absolute Discounting approaches, to smooth the conditional probability of observing a patient taking an existing test when a new test test0 is given (Formula 12-14). Also we use Laplace method to smooth the log function in the BPLT+ model (Formula 15). We conducted experiments to discuss the performance of the BPLT+ model and the sensitivity of smoothing parameters. We find that BPLT+ is able to make accurate recommendations under proper smoothing parameters.

Further, we propose a novel framework for effectively implementing BPLT+ model and objectively testing personalized recommendation systems without human interactions, shown in Figure 2. Based on the real patients' laboratory test data, we randomly generate a training data set, a validation input set and a validation label set. A training matrix containing the laboratory test statistics is calculated from the training data set and stored. For new patients (the validation input set), instead of processing the original training set, we utilize this training matrix to predict the laboratory test on the validation input set, and compare the ranking results with the validation label set.

There are a few future directions of this research work. As we can see from the data format in Figure 1, we have not make use of all the attributes. In the future, we would like to conduct a comprehensive investigation for the patients' profiles. For example, we can cluster the patients into groups and investigate the similarities of the patients in the same group. We can also analyze the associations among laboratory test results and therefore further enhance our proposed personalized recommendation model. Moreover, we look forward to testing our proposed models in more real applications.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

JZ proposed BPLT+ model, carried on the experiments and drafted the manuscript. JXH supervised the project and revised the manuscript. JXH also contributed in the study design and experiments. XH provides useful feedback. All authors read and approved the final manuscript.