Modification of Case-Based Reasoning Similarity Formula to Enhance the Performance of Smart System in Handling the Complaints of in vitro Fertilization Program Patients.

OBJECTIVES: Eighty percent of in vitro fertilization (IVF) patients have high anxiety levels, which influence the success of IVF and drive IVF patients to quickly report any abnormal symptoms. Rapid responses from fertility subspecialist doctors may reduce patients' anxiety levels, but fertility subspecialist doctors' high workload and their patients' worsening health conditions make them unable to handle IVF patients' complaints quickly. Research suggests that smart systems using case-based reasoning (CBR) can help doctors handle patients quickly. However, a prior study reported enhanced accuracy by modifying the CBR similarity formula based on Lin's similarity theory to generate the Chris case-based reasoning (CCBR) similarity formula.
METHODS: The data were validated through interviews with two fertility subspecialist doctors, interviews with two IVF patients, a questionnaire administered to 17 community members, the relevant literature, and 256 records with data on IVF patients' complaints and how they were handled. An experiment compared the performance of the CBR similarity formula algorithm with the CCBR similarity formula algorithm.
RESULTS: A confusion matrix showed that the CCBR similarity formula had an accuracy value of 52.58% and a precision value of 100%. Fertility subspecialist doctors stated that 89.69% of the CCBR similarity formula recommendations were accurate.
CONCLUSIONS: We recommend applying a combination of the CCBR similarity formula and a minimum reference value of 80% with a CBR smart system for handling IVF patients' complaints. This recommendation for an accurate system produced by the CBR similarity formula may help fertility subspecialist doctors handle IVF patients' complaints.

I. Introduction

Infertility is a problem throughout the world that is experienced by 48.5 million couples. In vitro fertilization (IVF) is a technology used to solve infertility, but the success rate of IVF programs is still low in many countries, such as Thailand (about 30.7%–38.6%), the United States (28%–29%), European countries (17%–25%), the United Kingdom (24%–25%), Australia (19%–20%), and Japan (9%–15%) [1].

The success rate of IVF in Indonesia is also still low (about 29%), and approximately 17% of IVF patients have miscarriages [1]. Despite the low success rate of IVF programs, the number of couples interested in undergoing IVF keeps increasing every year [2,3]. Sixty-eight percent of IVF patients must travel relatively long distances to access IVF services [4].

Eighty percent of patients undergoing IVF have high anxiety levels [2,5,6]. These high anxiety levels prompt 82% of IVF patients to complain about abnormal symptoms in their body to fertility subspecialist doctors through short message services or WhatsApp [2,3]. IVF patients often make these requests regardless of fertility subspecialist doctors’ difficult conditions [7] and demand the best services [2,3]. However, fertility subspecialist doctors: face a heavy workload [8], and their patients may experience worsening health conditions [2,3]. These constraints make fertility subspecialist doctors unable to quickly handle IVF patients’ complaints [2,3], although waiting a long time for responses from a fertility subspecialist doctor increases IVF patients’ anxiety [2,3]. In contrast, a quick response to a complaint may reduce IVF patients’ anxiety levels [9,10]. Since high anxiety levels influence the success of IVF, measures are needed to lower IVF patients’ anxiety [11].

The coronavirus disease 2019 pandemic has required health services to take steps to prioritize the health of IVF patients and fertility subspecialist doctors, making online health systems a necessity [12]. Online systems can be used by anyone, from any location [13]. Previous research has shown that online health smart systems can help doctors with their tasks by quickly handling patients’ complaints [14], facilitating monitoring pregnant mothers’ health [15], and predicting IVF cycles’ success rates [16]. Case-based reasoning (CBR) is widely applied to smart health systems [16].

The accuracy of CBR can be improved through modifications, such as chi-square CBR [17] and combination of CBR with a genetic algorithm [18].

No minimum value for CBR [19] systems has been specified to determine the level of information accuracy. Nonetheless, a general recommendation in the literature is that a similarity value ≥80% is considered relatively accurate, while a similarity value <80% is considered to indicate relative inaccuracy.

The aim of this study was to examine IVF patients’ and fertility subspecialist doctors’ responses to a smart system handling the complaints of IVF patients and to examine the performance of a modified CBR similarity formula and a minimum reference value of 80% in the smart system for handling IVF patients’ complaints.

II. Research Stages

The research was conducted in the following stages:

The data were validated through triangulation [20].

A questionnaire was distributed to the community “Pejuang Tangguh Permata Hati (Community of IVF patients who are struggling to get pregnant through the IVF program at Permata Hati Clinic, Yogyakarta)” to which IVF patients belonged.

A survey was directly conducted in some cities in Central Java Province and Yogyakarta Province in 2018–2019, which found that only four hospitals provided IVF services [4].

Based on the “Regulation of the Minister of Health of the Republic of Indonesia No. 69 Year 2008,” the research data included information on IVF patients’ complaints and how they were handled, without data on IVF patients’ identity. The total research data comprised 256 records.

Based on the “Regulation of the Minister of Health of the Republic of Indonesia No. 30 Year 2019” and the “Regulation of the Indonesian Medical Council No. 87 Year 2020,” we certified that the validator of the results of this research is a member of the Indonesian IVF Association with at least 5 years of experience in handling IVF.

A direct survey in some cities in Central Java Province and Yogyakarta Province in 2018–2019 found that there were only five fertility subspecialist doctors.

The CBR similarity formula was modified by adopting Lin’s similarity theory [21,22], referred to as the Chris case-based reasoning (CCBR) similarity formula.

An experiment used a computer program to compare the performance of the CBR and CCBR similarity formula algorithms.

A confusion matrix was used to measure the smart system’s performance in handling 97 complaints of IVF patients. (10) This research’s scope was the handling of IVF patients’ complaints in Central Java Province and Yogyakarta Province.

The researchers received two ethical clearances from the Faculty of Health, Pekalongan University (No.33/B.02.01/KEPK/I/2022) and the Faculty of Medicine, Duta Wacana Christian University (No.1384/C.l6/FK/2022).

Figure 1 presents the research stages.

Figure 1

Research stages. IVF: in vitro fertilization, CBR: case-based reasoning, CCBR: Chris case-based reasoning.

III. Results

1. Timing of Research Data Collection

The interviews with two fertility subspecialist doctors were conducted at different times (September 2019 to February 2021). The interviews with two IVF patients were conducted in August 2019. The questionnaires were distributed in October 2019 to the members of “Komunitas Pejuang Tangguh Permata Hati,” and 17 members responded. The community coordinator explained that only a few members responded to the questionnaire because not all of the community members were willing to share information.

2. Research Data Processing

Based on the result of crosschecking the fertility subspecialist doctors’ responses with those of the IVF patients, it was found that both groups supported the smart system for handling the complaints of IVF patients. Fertility subspecialist doctors agreed on the requirement that the system’s recommendations must be accurate. IVF patients supported this system since they understood that fertility subspecialist doctors were busy, but still wanted their complaints to be quickly handled. More specifically, 53% of the community members strongly agreed and 35% agreed with the use of the smart system for handling the complaints of IVF patients.

Table 1 presents the results of cross-checking the data sources, with results from the interviews with fertility subspecialist doctors, complaint data, data on the handling of IVF patients’ complaints, and the relevant literature [18,21,23].Objective weights for scoring were obtained from the assessment by experienced fertility subspecialist doctors [18]. Table 2 presents the results of consensus between the two fertility subspecialist doctors for the weight scores used in the experiment. The interval of the weight scores was 0–1 [23].

Table 1

Identification of types of IVF patients’ complaints

Type of complaint	Interview result with fertility subspecialists		Data of IVF program patients’ complaints and complaint handling
	Doctor 1	Doctor 2
Per vaginal bleeding [23]	Existed	Existed	Recorded
Supra-pubic pain [21]	Existed	Existed	Recorded
Fever [23]	Existed	Existed	Recorded
Nausea and vomiting [23]	Existed	Existed	Recorded
Insomnia [27]	Existed	Existed	Recorded

IVF: in vitro fertilization.

Table 2

Complaint weighting

Complaint type	Complaint level	Criteria	Weight score
Per vaginal bleeding	Normal	Normal	0
	Little	Appearing blood spots	0.1
	Medium	Coming out blood volume of about 1 tablespoon	0.5
	Severe	Menstruation-like coming out blood	1
Supra-pubic pain	Normal	Normal	0
	Mild	Able to do normal activities	0.1
	Medium	Inhibiting activities	0.5
	Severe	Unable to do any activity	1
Fever	Normal	Body temperature of <37.2°C	0
	Mild	Body temperature of 37.2°C–37.5°C	0.1
	Medium	Body temperature of 37.5°C–40°C	0.5
	Severe	Body temperature of >40°C	1
Nausea and vomiting	Normal	Normal	0
	Mild	Able to do normal activities	0.11
	Medium	Inhibiting activities	0.5
	Severe	Unable to do any activity	1
Insomnia	Normal	>7 hours	0
	Mild	6–7 hours	0.25
	Medium	5–6 hours	0.5
	Severe	<5 hours	1

3. Modification of the CBR Similarity Formula

CBR is based on records of similar past situations, which are saved on a case-by-case basis and used to solve current problems [19]. If an old case was identical to a new case, its value was 1; otherwise, its value was 0 [24]. The CBR similarity level was measured using the following similarity formula [2]:

where A is a new case, B is an old case, and K is B’s weight value.

Lin’s similarity theory [21,22] states “Examining the similarity and difference between A and B must take different perspective and calculate the level of similarity in each perspective. The optimal similarity value is the average value of similarity value of each perspective.”

There are two perspectives for calculating the level of similarity between A and B. A’s perspective on B is calculated using the following formula:

B’s perspective on A is calculated using the following formula:

An optimal similarity value means the average value of similarity value of A’s perspective on B and B’s perspective on A. Thus, a modification of the CBR similarity formula or CCBR similarity formula is obtained:

where A is a new case, B is an old case, K is B’s weight value, and K' is A’s weight value.

4. CBR Algorithm and CCBR Algorithm

Figure 2 shows the CBR and CCBR similarity formula algorithms, which are applied to the retrieval process, as shown in Figure 3. The experiment tested two algorithms with a minimum reference value of 80% on the smart system for handling the complaints of IVF patients. The system saved 159 data records of IVF patients’ complaints and complaint handling.

Figure 2

Algorithms of the CBR similarity formula and the CCBR similarity formula. CBR: case-based reasoning, CCBR: Chris case-based reasoning.

Figure 3

CBR system flow. IVF: in vitro fertilization, CBR: casebased reasoning, CCBR: Chris case-based reasoning.

IV. Discussion

1. IVF Patients’ Complaints and Fertility Doctors’ Responses

Fertility subspecialist doctors and IVF patients supported the smart system for handling the complaints of IVF patients, provided that the system’s recommendations must be accurate.

2. Accuracy of the Similarity Value of the CBR Similarity Formula and the CCBR Similarity Formula

Figure 4 shows 97 similarity values produced by the CBR and CCBR similarity formulas. The horizontal axis denotes the order of test data, and the vertical axis presents the similarity values.

Figure 4

Similarity values. CBR: casebased reasoning, CCBR: Chris case-based reasoning.

The CBR similarity formula showed better performance than CCBR similarity formula, since 91.8% of system recommendations had a similarity value ≥80%, while this was the case for only 42.3% of the recommendations using the CCBR similarity formula.

The first performance assessment was conducted by comparing the 97 system recommendations of the CBR and CCBR similarity formulas with how the 97 IVF patients’ complaints were handled. The results are presented in Table 3.

Table 3

Performance of the CBR and CCBR similarity formulas

Value ranges	CBR similarity formula		CCBR similarity formula
	Number of similarity values	Number of accurate results	Number of similarity values	Number of accurate results
100%	84.5%	9.8%	9.3%	100%
80%–99.9%	7.2%	16.9%	33.0%	100%
<80%	8.2%	16.9%	57.7%	82.1%

CBR: case-based reasoning, CCBR: Chris case-based reasoning.

The highest similarity value of the system recommendation was inaccurate, despite having a CBR similarity formula value of 100%; thus, a minimum value of 80% cannot be used as a reference threshold for the CBR model with the CBR similarity formula.

All system recommendations in the CCBR similarity formula with a similarity value ≥80% were 100% accurate. The highest inaccurate similarity value was 79.7%; thus, it can be concluded that a minimum value of 80% can be used as a reference to determine the accuracy of the system recommendations made using the CCBR similarity formula.

Based on the minimum value reference of 80%, measurement performance was evaluated using a confusion matrix. In Figure 5, the horizontal axis denotes the classification of accuracy values and precision values for each similarity formula, while the vertical axis presents the values. Accuracy refers to the similarity of all system recommendations to the data on how IVF patients’ complaints were handled, while precision denotes the similarity of system recommendations with a similarity value ≥80% to the data on IVF patients’ complaint handling.

Figure 5

Comparison between accuracy and precision values. CBR: case-based reasoning, CCBR: Chris case-based reasoning.

The confusion matrix showed that the accuracy of the CCBR similarity formula was better than that of the CBR similarity formula. The low accuracy value of CBR (19.59%) is similar to that of 17.32% reported in previous research [25]. The difference in the accuracy value between the CCBR similarity formula and the CBR similarity formula of 32.99% also corresponds to reported in previous research of 35%, with CBR showing lower accuracy [26].

The precision value of the CCBR similarity formula was 100% because all system recommendations with a similarity value ≥80% were declared to be 100% accurate. Thus, this precision value satisfied the requirement of the fertility subspecialist doctors and IVF patients that the system’s recommendations must be accurate.

The validation results of two fertility subspecialist doctors stated that 89.69% of the recommendations for the CCBR similarity formula were accurate.

There was a significant difference between the accuracy value of the CCBR similarity formula produced from the confusion matrix calculation (52.58%) and that produced from fertility subspecialist doctors’ validation. The reason for this difference is that 47.42% of the accurate system recommendations evidently had a similarity value <80%; thus, based on the CBR system flow, the system recommendations could not be directly accessed by IVF patients, but must first be checked by a fertility subspecialist doctor.

The low number of CBR system recommendations declared accurate by the fertility subspecialist doctors can be explained by two factors. First, there were different complaints with the same weight values (Table 2). Second, the CBR similarity formula did not adopt Lin’s similarity theory; thus, no optimal similarity value was obtained [22].

3. Conclusion and Research Opportunities

The CCBR similarity formula and the minimum reference value of 80% applied in the CBR system are capable of providing quality health services [27] through the provision of accurate system recommendations; thus, this approach is capable of helping fertility subspecialist doctors handle IVF patients’ complaints.

The fact that the research data were only collected from hospitals in Central Java Province and Yogyakarta Province means that the research results cannot be taken as the standard method, since further research should use data from many provinces in Indonesia to test the performance of the CCBR similarity formula. The combination of CBR with rule-based reasoning (RBR) can improve the accuracy of smart system output [24]; thus, research needs to be carried out on the combination of the CCBR similarity formula with an RBR model.