Assessing the efficacy of the electronic patient record system EDeR: implementation study--study protocol.

INTRODUCTION: Despite many innovations in information technology, many clinics still rely on paper-based medical records. Critics, however, claim that they are hard to read, because of illegible handwriting, and uncomfortable to use. Moreover, a chronological overview is not always easily possible, content can be destroyed or get lost. There is an overall opinion that electronic medical records (EMRs) should solve these problems and improve physicians' efficiency, patients' safety and reduce the overall costs in practice. However, to date, the evidence supporting this view is sparse. METHODS AND ANALYSIS: In this protocol, we describe a study exploring differences in speed and accuracy when searching clinical information using the paper-based patient record or the Elektronische DateneRfassung (EDeR). Designed as a randomised vignette study, we hypothesise that the EDeR increases efficiency, that is, reduces time on reading the patient history and looking for relevant examination results, helps finding mistakes and missing information quicker and more reliably. In exploratory analyses, we aim at exploring factors associated with a higher performance. ETHICS AND DISSEMINATION: The ethics committee of the Canton Lucerne, Switzerland, approved this study. We presume that the implementation of the EMR software EDeR will have a positive impact on the efficiency of the doctors, which will result in an increase of consultations per day. We believe that the results of our study will provide a valid basis to quantify the added value of an EMR system in an ophthalmological environment.

To compare the gain in

Efficiency

Accuracy

When introducing an electronic patient record system vs paper-based records

The chosen design

Has the potential to validly address the added value of an electronic patient record system.

Allows extrapolating the economic gain of introducing the system.

Study design allows examining the effects

In a realistic clinical setting;

Using relevant clinical problems;

Using data of real patients.

Economic consequences can only be addressed indirectly.

Background

Despite substantial effort to promote electronic data management in patient care, a large proportion of healthcare providers still rely on paper-based patient records. The American Academy of Ophthalmology performed a survey concerning the adoption rate of EMRs within their members and found an adoption rate of only 12%.1

The main problem with paper-based patient records is the handwriting and the missing overview of treatment and patient history. The handwriting is sometimes hard to read for the physician in charge and the missing overview could potentially lead to needless examinations and errors in diagnosis, prescription and treatment,2 because of the absence of consistent information on changes in the state of health, treatment and the parameters measured. As a solution of these problems, there was an idea to support clinical daily routine with the help of computers, as it is reality in many fields of service.

In the last decade, there was enormous development and investment in the field of electronically supported medical records. It seems to be a key strategy to improve healthcare in the different specialties.3 Electronic medical records are supposed to improve physicians’ workflow and to offer a better overview on patient's history. Today, many authors believe that this innovation should improve safety and reduce costs.4 ‘A better understanding of the impact of the EMRs on workflows is vital to understand what the technology really does offer, which is new and unique’.5 6 However, a systematic review published in 2008 and covering the evidence until 2005 concluded that none of the 20 studies provided a formal cost effectiveness analysis of electronic patient record systems in general.7

The cantonal hospital of Lucerne has developed new software addressing the special needs of the ophthalmological clinic. The idea was to create a tool, which offers a complete overview on the patient history once a patient has been referred by another ophthalmologist or a general practitioner. The software includes physician's drawings, imaging, measurement data (eg, visual field testing and electroretinography), numerical data (eg, autorefraction, keratometry and biometry) and ophthalmic image data (eg, fundus photography and optical coherence tomography). These data are essential to support clinical diagnosis to track disease progression and to plan treatment.8

In order to explore the impact on productivity of physicians using electronic health records, it has been shown that there is a statistically significant increase in the average monthly patient visit volume of nine visits per examined provider per month,3 or the reduction of time spent on administration by the nursing staff.9 The collected data also could be used for clinical research and clinical studies.10 In general, there is an existing hypothesis that EMRs should improve patient's safety, but there are neither data nor suggestions available as to how this should be measured. Overall, however, there are only a few studies, which show evidence that the implementation of EMRs has a major impact on the efficiency,4 11 especially in the field of ophthalmology. There seems to be also the demand for more studies with prospective and randomised experimental designs, instead of surveys.11

Possible barriers for a successful use of EMRs are costs and time loss for implementation, issues concerning the security and privacy of the patient's data, lack in training and support of the staff using the EMR software.4 11 The main objective should be the elimination of obstacles for a successful implementation.

In this protocol, we describe a study exploring differences in speed and accuracy when searching clinical information using the paper-based patient record or the Elektronische DateneRfassung (EDeR). Designed as a randomised vignette study, we hypothesise that the EDeR increases efficiency, that is, reduces time on reading the patient history and looking for relevant examination results, helps finding mistakes and missing information quicker and more reliably. In exploratory analyses, we aim at exploring factors associated with a higher performance.

Methods/design

Study design

The study is planned as a prospective simulated study.

Study location

The study is planned to take place at the cantonal hospital of Lucerne in the clinic of ophthalmology. The eye clinic is run by six consultant ophthalmologists, 12 senior physicians and 14 junior doctors.

Study population

The study population will be divided into two groups. One group will consist of senior physicians of the ophthalmological clinic of the cantonal hospital in Lucerne, Switzerland. The other group will be formed by junior doctors, who are usually younger and more computer adepts, but have less clinical experience.

Inclusion criteria

The ophthalmologist of the cantonal hospital Lucerne eye clinic who voluntarily agreed to participate in the study will be included. Each participant will receive a general introduction on the usage of the EDeR of half an hour prior to the assessment. The training will cover aspects of data collection and saving very briefly. The training will be such that the basic functions of EDeR are known.

Primary outcome

Gain in speed when performing the task using the EDeR system.

Secondary outcome

Accuracy of replies given using the EDeR system versus the paper record.

Experimental design

The study is planned as a prospective, randomised study using patient vignettes. The aim is to measure the time needed to solve a given task and the accuracy of the result. For each out of five different ophthalmological subspecialties (neuroophthalmology, corneal, glaucoma, uveitis and orthoptics), we will create five tasks based on a patient sample. Thus, we will create 25 pairs of cases either represented in paper or electronic form. The tasks are chosen on the basis of the relevance in clinical practice.

Types of tasks

We defined the following five task types: (1) to detect the medication dose 3 months after the onset of treatment, (2) decision about the adjustment of a treatment based on the impairment or persistent state of data monitored, (3) question about the acquisition of underlying diseases, (4) question about the detection of implausible data, which could be important for the further treatment and (5) question about the completeness of additional information. Table 1 provides examples of tasks used for neuroophthalmology.

Table 1

Description of the context, task and contrast examined for the example of neuroophthalmology

Task 1
Context	Regular control visit
Task	To detect the medication dose after a certain amount of time after the onset of treatment
Contrast	EdER: 6× daily 3 dragées à 60 mg MestinonPaper record: 15× daily 3 dragées à 60 mg Mestinon
Task 2
Context	Regular control visit
Task	Decision about the adjustment of a treatment based on the impairment or persistent state of data monitored—‘in case of myasthenia gravis, the decision to adjust the steroid dose is made 4 weeks after starting therapy. If there has been a significant improvement in the eye position, recorded with the coordimetry of Hess Weiss, the steroids can be tapered off. Is the eye position unchanged, or even worse, the steroid dose should be increased or an additional medication should be introduced. In the present case: would you increase or decrease the dose of the steroids 4 weeks after the onset of the therapy?’
Contrast	EdER: There has been a significant improvement in the eye position; therefore the steroids can be tapered off.Paper record: the eye position is unchanged; the steroid dose should be increased.
Task 3
Context	Postoperative medication
Task	Question about the detection acquisition of underlying diseases.Question: ‘in case of an allergic reaction to Bactrim the administration of Diamox often also produces allergic signs due to a cross-reaction. Diamox should be avoided in such a situation. Is there a contraindication to administer Diamox after a cataract surgery in the present case?’
Contrast	EdER: the section with the patient history, containing information about allergies and underlying diseases, is blank. Although the patient has a sulfonamide allergy.Paper record: there is a short note about an allergic reaction to Bactrim.
Task 4
Context	Error in documentation
Task	‘Do you find any implausible data regarding visual field recordings during the last 3 years of documentation in the present case of a patient with a pituitary adenoma?’
Contrast	EdER: the series of visual fields will show a constant bitemporal fieldPaper record: the series of visual fields will show one hemianopic visual field within the series of bitemporal fields. This change in just one visual field within an otherwise stable series is implausible and indicates an error of documentation.
Task 5
Context	Completeness of clinical information.
Task	In a patient with an idiopathic intracranial hypertension the diagnosis is made by the fact of (bilateral) papilloedema without visual afferent defects and raised intracranial opening pressure measured by lumbar puncture. Is it correct to make the diagnosis of an idiopathic intracranial hypertension in the present case?’
Contrast	EdER: the necessary information to diagnose idiopathic intracranial hypertension is present.Paper record: no lumbar puncture, diagnosis not possible, because of missing information.

The patient model will come from original paper forms to assure that cases are comparable to the real-life situation. The full patient record, containing a slight modification for the contrast examined in a specific task, will be transferred from the paper into the electronic patient record system. During the experiment, a research assistant will monitor physicians and measure the time required to complete each task. Each physician from each division has to solve all of the tasks. The evaluation will take place at two time points; immediately after implementation of EDeR and 1 year after.

Design matrix summary

This is a two group (EDeR vs paper-based) cross-over (randomly assigning the sequence electronic vs paper-based per task) study. Time (primary endpoint) and accuracy (secondary endpoint) are the two outcome variables. Intervention is the EDeR that is compared with the paper-based patient record. We assess tasks within five clinical subgroups (without examining subgroup effects) and perform the experiment in two groups with different ophthalmological expertise (junior doctors vs senior doctors). Each assessor will complete 50 tasks (ie, 25 tasks based on the electronic patient record and 25 based on the paper patient record). The electronic records are exact representations of an existing paper-based record. Patient records differ only in respect to one contrast that is changed for a specific task. Thus, besides that task-specific detail, the remaining content of each pair of patient records is identical. But, if an assessor checks the same vignette second time (eg, in the electronic version), he or she must read the content of the paper-based record again, because the correct answer, that is, a specific lab value, is different. The tasks were selected based on clinical relevance. For summary, see figure 1.

Figure 1

Study flow and design features.

Detailed research plan

Creating the patient vignettes

Of each specialisation, we will collect five typical patient records on paper.

Checking for plausibility and technical feasibility

After creation of the vignettes, two to three individuals will be selected to evaluate the tasks. They will solve the tasks and afterwards fill out a questionnaire, which consists of questions about the difficulty, comprehensibility and technical feasibility of the task.

Design matrix

We will generate random sequence coding whether the task is started with the paper or the electronic version of the patient record. A second random sequence will be generated to code for the sequence with which tasks will be presented to the participants. These two measures will eliminate order effects.

At the assessment

The participant will be sitting in a consultation room and the assessor will be seated next to him or her to avoid distractions.

We will design a reply form into which each participant enters his or her response to the task. This reply form will be handed over to the assessor after completion. Time will be taken by the assessor after completing each entry into this form. In total, each participant will go through 25 pairs of tasks. In case of a technical problem with the electronic system, the assessment will be suspended and the remaining tasks will be completed after restoration of the system.

There will be no communication between the participant and the assessor.

The assessor will measure the time the participant needs to complete the reply form for each task and each mode (paper based, electronic).

The assessor will take care of a protocol-based procedure.

If any inconveniences of technical or administrative manner occur, the assessor will solve them.

End of the assessment

All forms will be collected and the data will be entered in an anonymised fashion into a spreadsheet for further statistical analysis.

Sample size calculation

By implementing a novel electronic health record system, it is possible to observe a productivity gain of the physician.3 We hypothesise that our productivity gain should be one additional consultation per day assuming that one consultation lasts about 20 min and a physician sees 25 patients daily. Comparing the two methods, paper and EMRs, the overall gain in time should be 60 s on an average when using the EMR software EDeR.

Therefore, we are planning a study of the continuous response variable time from matched pairs of study individuals. We presume that the difference in the response of matched pairs is normally distributed with standard deviation 45. If the true difference in the mean time of matched pairs is 60 s, we will need to study seven pairs of individuals to be able to reject the null hypothesis that this response difference is zero with probability (power) 0.8. The type I error probability associated with this test of this null hypothesis is 0.05.

At the time of writing this protocol, 10 senior physicians and 9 junior doctors were willing to participate in the study.

Data collection

Point of time of data collection

The first data acquisition will take place shortly after the implementation of EDeR. The defined tasks will be available in the paper form and the physicians will fill them out with the help of a medical record, either the paper form or the electronic form.

Additionally, there will be someone from the study personnel, who sits with the physicians in the same room and measure the time they need to complete each task.

Baseline data

Prior to the examination, baseline data will be collected of each participant. Baseline data consist of information about the physicians’ age, extent of previous training in ophthalmology, degree of specialisation and self-reported level of general computing skills.

Follow-up data

At the second time point, 1 year after the first assessment, the experiment will be repeated using the same protocol. In the interval between the two measurements, all physicians will be mainly working with the EDeR system.

Discussion

This paper describes the rationale, methodology and design of a randomised, prospective simulation study using patient vignettes based on real patients. This study will compare the paper-based medical records with EMR software EDeR in terms of efficiency, safety and cost reduction in the daily routine of an ophthalmological clinic. The experiment consists of 25 tasks, which will be solved by two groups of participants: senior and junior doctors.

We presume that the implementation of the EMR software EDeR will have a positive impact on the efficiency of the doctors, which will result in an increase of consultations per day. It can be argued that choosing volunteers for this study could lead to exaggerated results. Nevertheless, we believe that the results of our study will provide a rational basis to quantify the added value of an EMR system in an ophthalmological environment.