Real-world Evidence versus Randomized Controlled Trial: Clinical Research Based on Electronic Medical Records.

Real-world evidence (RWE) and randomized control trial (RCT) data are considered mutually complementary. However, compared with RCT, the outcomes of RWE continue to be assigned lower credibility. It must be emphasized that RWE research is a real-world practice that does not need to be executed as RCT research for it to be reliable. The advantages and disadvantages of RWE must be discerned clearly, and then the proper protocol can be planned from the beginning of the research to secure as many samples as possible. Attention must be paid to privacy protection. Moreover, bias can be reduced meaningfully by reducing the number of dropouts through detailed and meticulous data quality management. RCT research, characterized as having the highest reliability, and RWE research, which reflects the actual clinical aspects, can have a mutually supplementary relationship. Indeed, once this is proven, the two could comprise the most powerful evidence-based research method in medicine.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

Real-world data (RWD) refer to data collected from diversified areas of daily life that are outside the scope of highly controlled randomized control trials (RCTs).1 Examples of RWD are the large quantities of data collected broadly through the use of a diverse range of medical devices by various users, including self-checked glucometer or sphygmomanometer. These real, practical data cannot be obtained through contemporary or traditional clinical trials. A wide range of data already accumulated, including the electronic medical records (EMRs) used in hospitals,234 Korea National Health and Nutrition Examination Survey (KNHANES) data,5 Health Insurance Review and Assessment service data6 or Korea Adverse Event Reporting System (KAERS) database7 are also included in RWD.

EMR, in allowing for the accumulation of a large amount of clinical data, are increasingly used for medical research.89 Worldwide, EMR-based clinical research has been reported to increase the quality of research and satisfaction of clinicians.10111213 Depending on study design and objectives, EMR research may utilize a variety of data: medication prescription, treatment decision making, disease management, and clinical research data. For these reasons, EMR data are recognized as definitive data with the highest reliability among RWD. Analysis of RWD yields real-world evidence (RWE), and many scholars are focusing their interest on RWE research using RWD accumulated broadly, along with the issue of big data. Efforts have been made toward expanding the range of relevant clinical research.

Fundamentally, it is desirable for the new research format, RWE, and the traditional research format, RCT, to be mutually complementary. However, despite the comprehensive reports on the importance of RWE, there is a tendency among scholars of mistrusting the findings of RWE compared with RCT. For many researchers, overcoming a diverse range of biases that can occur in EMR-based RWE research requires trial and error. Therefore, we intend to present opinions on precautions that can be used effectively in executing research using RWE from the clinical rather than technical perspectives.

RWE VERSUS RCTs

RCT conducts research on highly selective populations and is managed in tightly controlled settings.14 RCTs are conducted under the premise that the research outcomes of the chosen sample population represent those of the entire population. This research format is deemed to have the highest reliability. The majority of clinical guidelines are given on the basis of the resultant values deduced from RCTs.15 However, a disadvantage is that participants may fail to reflect the actual clinical site sufficiently to represent the entire population, given the clear set of inclusion or exclusion criteria. This drawback of exclusion has been deemed a bias.1617 That is, there would be difficulty in applying the resultant values of data analysis confirmed in a small number of patients selected in accordance with set criteria in RCTs to diversified actual situations that could occur in the real world. For this reason, the foundation on which to elevate definitive reliability through concurrent execution of RWE research, in addition to RCT, needs to be established; these two should ideally be developed in a mutually supplementary rather than competing relationship. For example, if the RCT concentrates on the efficacy of a drug, the RWE can focus on the epidemiology, effectiveness, safety, or costs of treatment related to the drug (Tables 1 and 2).18 RWE research that can sustain follow-up observation after the completed RCT research holds important significance. Clinical trials and secondary syntheses of evidence-based information can be biased owing to undisclosed financial and nonfinancial conflicts of interests. Therefore, research should be undertaken with the perception that managing these conflicts of interests more thoroughly is crucial to maintaining the essence and credibility of research.19

Table 1

RCTs versus RWE18

Variables	RCTs	RWE
Purpose	Efficacy	Effectiveness
Setting	Experimental setting	Real-world setting
Follow up	Designed	In actual practice
Treatment	Fixed pattern	Variable pattern
Study group	Homogenous	Heterogeneous
Attending physician	Investigator	Many practitioners
Comparator	Placebo/selective alternative interventions	Many alternative interventions
Patient monitoring	Continuous, per protocol	Changeable

RCT = randomized clinical trial, RWE = real-world evidence.

Table 2

Advantages and limitations of real-world evidence compared with RCTs

Advantages
	Less time and cost consumption compared with RCT
		- Possible to shorten the duration of clinical research
		- No period needed for patient recruitment/enrollment
	Presentation of the directionality for RCT
	Research that cannot be done with RCT is possible
		- Safe research on high-risk groups
	Detection of less frequent side effects
	Rapid access/easier information and data retrieval
	Prediction model or high-risk group selection
	Set a foundation on AI
Limitation
	For correct analysis, a massive amount of data should be collected.
	Much time needed for DQM
	Experienced experts needed for the analysis of the massive amount of data
	Lack of privacy/confidentiality/lost data
	Standardized research protocol needs to be established before research
		- High possibility of bias
		- Possible to interpret with bias or incorrectly based on research results

RCT = randomized control trial, DQM = data quality management.

DIRECTIONALITY FOR EMR-BASED CLINICAL RESEARCH

Previous research on the use of EMR data have focused on their convenience and accessibility.20 With the recent accumulation of extensive data in EMR systems, these data can be used in clinical studies. The use of EMR-based RWE addresses the disadvantages of both cohort studies (e.g., follow-up cost, long study periods, and maintenance of consistency during the study period) and RCTs (e.g., loss of participants at follow up, changes in treatment, long study periods, and expense). In EMR-based clinical research, the investigator has direct access to the EMR system and can quickly verify a hypothesis by sampling related variables from the system.

Real clinical practice

Compared with RCT, the core aspect of RWE is its practical nature rather than a tightly controlled set group. EMR include clear diagnoses and prescriptions entered by physicians stored in a database that can be easily accessed and utilized. EMR research can broadly reflect actual practice as it is fundamentally research aimed at the analysis of vast quantities of data already accumulated. More specifically, EMR enables researchers to extract and analyze quickly the side effects,21 progress, and prognosis of drugs22 with low frequency. EMR allows for the quick and systematic collection of data on the effectiveness or side effects that can manifest when a specific drug has been broadly prescribed to unspecified masses. EMR research can practically approach a diverse range of safety issues not detected in three- and four-phase studies.23 In addition to the evaluation of drugs, EMR research is able to evaluate the performance or clinical output of medical staff or hospital. With EMR data, any scholar can evaluate the actual achievement rate of goals of individual medical departments or hospitals.24 For instance, whether prescriptions were made in accordance with or against the guidelines is being disclosed through such research activities. In other words, the one making the prescription is observed and data accumulated on what is being prescribed. Further, the diversification of approaches to data allows the evaluation of the prescription patterns of individuals. Consequently, this system can be expected to enable the most appropriate prescriptions for any given disease. Thus, the possibility of approaching system-based rather than experience-based medicine is extremely high.

Presentation of the directionality for RCT

A long period is necessary to conceptualize RCT for the effects or side effects of drugs that carry social issues. Indeed, there are cases in which clinical research is terminated owing to unexpected results of the RCT executed.25 The organization that conducts clinical research must bear the enormous consequences if the results of RCT research turn out to be different from those anticipated. In such cases, a desirable RWE research model is one that presents the directionality of RCT by forecasting the research results under the concept of a pilot study by securing as extensive a range of data as possible in a short time period.26 The advantages of EMR for pharmaceutical or medical device companies include precise market and customer analyses as well as access to the data of competitors. Further, information on the prescribing patterns of physicians would be useful for establishing marketing plans.

Research not possible with RCT owing to research ethics

This point marks the most powerful advantage that can be done only with EMR. Research can be conducted for all clinical trials only after having acquired the approval of an Institutional Review Board (IRB). New topics of research outside the range of set guidelines cannot be approved by an IRB, and consequently, such research cannot be carried out. However, certain prescriptions that are outside the guidelines may exist within the actual EMR data.22 These are mostly cases in which it was unavoidable to make the said prescription rather than intentionally infringing the guidelines with the determination that there are greater advantages than disadvantages in making the said prescription. As RWE research uses already accumulated data, there is no infringement on the rights and benefits or harm with respect to the physical states of patients. Therefore, research in itself is not impossible. Nonetheless, this aspect requires careful approach. Modification of existing guidelines can be proposed carefully, and additional large-scale RCT research can be proposed for this purpose. If clinical RWD are used well, various controversial issues can be easily studied, which may improve future diagnosis/treatment guideline modalities.

Prediction model based on artificial intelligence

With deep learning becoming the talk of town in recent years, people are becoming increasingly interested in artificial intelligence that utilizes deep learning.27 Deep learning-based artificial intelligence ultimately needs to the utilization of an extensive range of accumulated data. As EMR have been implemented widely in hospitals, we expect a drastic increase in the analysis of accumulated EMR data. To obtain clinically significant data from large hospitals, a clinical data warehouse can be an important tool, whereas EMR will potentially lead to various types of clinical research. However, the majority of actual EMR-based deep learning projects have failed to reach the level initially anticipated. At the heart of the failure is data collection. Various algorithms developed using unprocessed data have frequently failed to perform as anticipated. It is not possible to generalize and deduce outcomes with the results obtained from unprocessed and low-quality data. For these reasons, hospitals have recently commenced improving access to and systematic purification of data.

Unknown information

Previously unknown information that clinical researchers fail to predict in the study planning phase may come to light, and EMR systems might lead to new knowledge and theories.2122

PRIVACY PROTECTION

Accumulation of large-scale patient data and clinical trials using such data raise the important issue of the protection of personal information. In the case of Korea, numerous limitations on the use of clinical big data have been enacted to address issues related to personal information leaks. Therefore, privacy protection must be mentioned clearly when preparing works on RWD. This practice must include the fact that the name and resident registration number will not be included in the accumulated data, and information directly related to privacy, such as address, telephone number, and e-mail, will be removed. Any combination that can lead to the identification of an individual, such as hospital registration number and date of birth, must be removed immediately prior to the stage of analysis. Further, all files on the extracted information must be encrypted and then stored in an encrypted computer, accessible only to the designated researcher. Additionally, contents of theses must not deal with the personal information of patients; researchers should emphasize that the research has no possibility of causing additional physical harm whatsoever to the subjects. Finally, it must be specified that the advance consent of the patient is not necessary given the anonymity of the subjects through encryption of data and that there is no effect on the rights and welfare of the subjects because past data, collected after having completed treatment, are used (informed consent not required). The statement “the research was conducted after having acquired the approval of IRB” must also be included.

ISSUES TO BE CONSIDERED AT THE TIME OF RESEARCH ANALYSIS

Compared with RCT, EMR-based RWE study poses difficulties in controlling for biases that may affect the study outcomes. Research projects need to be commenced with a definitive design to minimize such biases. Data quality management (DQM) is the most important part of the extracted data in RWE. As extracted data could be deemed byproducts of the process of treating patients, rather than data elaborately organized and structured in accordance with the purposes of the research, the quality of the data can be low. There may even be many errors or erroneous inputting in the course of their collection. Through DQM, comparison with original data can be confirmed, including the mixed use of numbers and letters, erroneous descriptions, and serious errors. This phase may be time consuming, depending on the number or capacity of data. The most serious bias normally occurs at this stage.

As mentioned above, it is important to secure larger samples size to elevate fundamentally the reliability of RWE. Although larger quantities of data do not necessarily elevate accuracy and reliability, using the same would help minimize bias that occurs in the smallest number. Even with plenty of actually extracted data, a researcher may end up excluding a large number once the data are sorted in accordance with the purpose of a research. For example, there are many cases in which the basic baseline results, such as height/weight and blood pressure, end up omitted. Without baseline values, there can be no follow up, and changes in clinical laboratory data cannot be examined. There also are frequent cases in which the actual overall number of research subjects is reduced by more than half, such as when patients for whom follow up is not possible or patients who transferred to another hospital in the middle of the research are excluded. The exclusion of a large number of samples is in itself a significant bias, and it is the aspect of EMR-based clinical research pointed out most frequently by journal reviewers as a hard limitation.

Certainly, not all the data are available in EMR. For one, it is difficult to confirm the compliance of the subjects using EMR, including whether they took drugs as instructed in a drug-related research. Hence, assessment of the effectiveness or efficacy of drugs tends to be difficult in RWE research (This is generally carried out in RCTs). Checking whether subjects took the drugs as instructed by checking the number of pills prescribed and those remaining at the time of the next visit can be implemented as an alternative method for assessment of the compliance of subjects. Failure of the medical staff to make entries in the medical chart is also a problem. Proper research would be possible if the medical staff describe the state of patients accurately and in detail. Further, although patient conditions and treatments can be described, EMR cannot be used to elucidate scientifically causal relationships.

The domain of RWE in addition to EMR data mentioned in this work is extremely broad. It can be collected directly from the patient by means of a diverse range of medical devices and by utilizing a wide range of data. Accordingly, RWD are increasing in volume, whereas RWE is being gradually expanded, with continuous advancement of their respective analytic functions and protocols. In addition to the importance of the quantity of data, it is necessary to concentrate on the means of properly structuring and utilizing rapidly accumulated clinical data. RWE and RCT research are not in a mutually competitive relationship. Presently, RWE research cannot substitute RCT research. However, RCT research, bestowed the highest reliability, and RWE research, perceived to reflect the actual clinical aspect, can be proven to share a mutually supplementary relationship to become the most powerful resource in evidence-based medicine.