Pediatricians' Understanding and Experiences of an Electronic Clinical-Decision-Support-System.

OBJECTIVES: Subsequent dosing errors after implementing an Electronic Medical Record (EMR) at a pediatric hospital in Sweden led to the development, in close collaboration with the clinical profession, of a Clinical Decision Support System (CDSS) with Dose Range Check and Weight Based Dose Calculation integrated directly in the EMR. The aim of this study was to explore the understanding and experiences of the CDSS among Swedish pediatricians after one year of practice.
METHODS: Semi-structured interviews with physicians at different levels of the health care system were performed with seventeen pediatricians working at three different pediatrics wards in Stockholm County Council. The interviews were analysed with a thematic analysis without pre-determined categories.
RESULTS: Six categories and fourteen subcategories emerged from the analysis. The categories included the use, the benefit, the confidence, the situations of disregards, the misgivings/risks and finally the development potential of the implemented CDSS with Weight Based Dose Calculation and Dose Range Check.
CONCLUSIONS: A need for CDSS in the prescribing for children is evident to support the prevention of medication errors. After implementing a CDSS, organized efforts are crucial to understand the need for further development based on the practical knowledge of the clinical profession. Different contextual settings of health care organisations do affect the way how physicians think and act in work. When implementing a CDSS in practice we need to describe and analyse the context where the CDSS should be used as well as the prescribers' needs in work.

Introduction

The most frequent adverse drug events in the pediatric clinical setting is related to erroneous drug orders [1]. The reason for this type of medication errors can be found on a psychological and organizationally level as well as in a poor technical system [2-4]. The latter was a major concern when the mandatory Electronic Medical Record (EMR) of Take Care [5] was introduced in 2008 at the Children’s Hospital of Karolinska University Hospital, Stockholm Sweden with an intent to meet national regulations [6]. Unfortunately, these regulations are written for paper based orders and do not take into considerations the need for added EMR requirement of specific clinical situations. For example, the regulations state that the dose preferably should be written in milliliters. So, when prescribing a drug with the concentration of 10 mg/mL, in mg instead of mL, the risk of a 10-potency error is possible. Since a vial intended for an adult can keep a 10-fold erroneous dose for an infant the risk is increased at pediatric wards. Similar concerns had previously been stated by the American Association of Pediatrics, including the need for Clinical Decision Support Systems (CDSS) such as dosing by weight and dose range checking [7]. Articles have also shown the added benefit of weight based prescribing in pediatrics [8-10], the need to aggressively fix contra productive technology in CDSS [11], the possibility to improve individualized pediatric pharmacotherapy by CDSS [12] and to add manual services for identifying prescribing errors [13].

At our hospital, subsequent dosing errors related to the implemented EMR triggered the initial demands of a pediatric CDSS. Recourses from the Stockholm County and a call for priority changes on the EMR provider made the development of an EMR integrated CDSS possible. In addition, the development of a mandatory dose calculation weight (DCW) was necessary alongside rules for an age- and weight specific dose range check (DRC) provided by the pediatric drug group at the hospital. The CDSS was developed by participatory design with the aid of a 20 pediatrician reference group and released with 1) a dose by weight function (gram/kg, milligram/kg, microgram/kg and units/kg), 2) a soft stop DRC and 3) a mandatory DCW which needs to be updated regularly, based on the age of the child. At the end of the project a list of future development issues regarding the CDSS was handed over to the EMR provider.

In this study, we have used semi-structured interviews with Swedish pediatricians regarding their use of, frustration with, dependence on and risks of a highly anticipated CDSS one year after the development through participatory design. It is a qualitative approach with the overall aim to explore and describe pediatricians’ understanding and experiences of the weight based dose calculation (WBDC) and DRC while prescribing medicines to children.

Methods

Research design

A qualitative approach was chosen since this is a fruitful way to explore people’s experiences, attitudes, thoughts and perceptions of different phenomena [14]. With the aim of getting deeper knowledge of the use and usefulness of the decision support system semi-structured interviews with physicians at different levels of the health care system were performed.

Setting and informants

Seventeen pediatricians (4.9% of the total number of pediatricians) working at three different pediatrics wards in Stockholm County Council were recruited to the interviews (Table 1). The first two informants were selected from a personal register provided by the health authorities. During the interviews it turned up that these informants have used the EMR and the CDSS fairly minimal. For this reason the snow ball sampling strategy were used [14], which means that the participating informants were asked to mention one colleague with experience of the CDSS. When additional data reached saturation, i. e. no more new information was achieved; the recruitment of new informants was stopped. All informants were contacted by the researchers (AV or PBR) explaining the aim and procedure of the study.

Table 1

Participants’ background

Variables				Number (n=17)
Sex
		FemaleMale	9      8
Education		FellowResidentConsultant	1      5      11
Care units	GastroenterologySurgeryNeonatologyEmergencyOncology		6      4      4      2      1

At the beginning of each interview the informant was informed about the purpose of the study, the voluntarily participation, and the confidential data procedure. All informants also received a consent form. The interviews lasted from 25 to 40 minutes. Data was collected in 2012 at the physicians’ workplace.

Data collection

We started each interview broadly with questions related to the difficulties in the drug prescribing process. We continued asking more specific questions about the physicians’ needs for drug information and their experiences about the CDSS. Examples of questions asked were:

To what extent do you currently use computerized decision support systems at point of care? Do you consider the present system to be useful? What additional help should the support system give to you?

Data analysis

The interviews were recorded as audio files, and then transcribed verbatim and processed as texts. A qualitative thematic analysis was performed [14]. The analysis was conducted by two authors experienced in qualitative analysis (AV and PBR) in a stepwise manner as follows:

1) Reading through each transcript individually to get a good grasp of all that had been said.

2) Sections of text and key words in the transcripts, focusing on the research question, were marked and systematically coded. Marked sections with related topics were subsequently grouped into sub-categories.

3) Associations between sub-categories were identified. When opinions differed between research members, such as about meaning or origin, we returned to the transcripts and sought evidence to establish consensus. This iterative process was used throughout the whole analysis, i.e. moving from the whole transcripts to the condensed description and back again.

4) Sub-categories were merged and named from an overall perspective. Quotes were selected to illustrate each category.

5) Quotes were chosen to illustrate and validate the categories.

The software program NVivo 8 was used for sorting the data into different categories [15]. The focus during the analysis was to capture as many different views as possible, not to investigate how many informants described a certain view about the support system.

Ethical considerations

Swedish law [16] did not require approval of this study from the research ethics committee on the basis that our study posed no physical or psychological risk to the participants. However, all participants gave their informed consent prior to involvement in the study.

Results

The results describe the experiences of the informants based on the problems and difficulties they experience when prescribing medicines for children with the CDSS (Table 2).

Table 2

Six categories and fourteen subcategories emerged from the analysis of the interviews

1. Use     • Use is influenced by clinical experience     • Habit leads to increased use     • Good that the CDSS is not compulsory2. Benefit     • Prompts consideration     • Help with calculations reduces errors     • Greatest benefit in emergency care3. Confidence in the WBDC and DRC     • Use of a manual DRC     • Double-checking the dosage4. Situations in which the doctor disregards the WBDC and/or the DRC     • When it is easy to work out the dosage using mental arithmetic     • In case of special indications5. Misgivings/risks     • False security and non-disease specific warnings     • Human error is unavoidable     • Wrong DCW6. Development potential     • Optional or compulsory – registering and signing for weight

Use

Use is influenced by clinical experience

Some informants frequently use the support function as soon as they prescribe a medicine, while others do not prescribe medicines in such large quantities, as a result of which use is sporadic. According to specialists and consultants, it is doctors undergoing specialist training that make more use of and derive the most benefit from the WBDC. Specialists prescribe a limited number of medicines with which they are familiar, while the specialists in training, who as a rule have less experience, go around different departments where they encounter different patient groups, and so they have a greater need for the aid. According to the informants, use of the WBDC increases whenever prescribing something with which they are not familiar. Many of them report finding the WBDC and DRC easy to use and to understand. A lot of them describe it as being good as,

"It's straightforward. There's nothing difficult about it.", "The module works well."

Habit leads to increased use

The informants reported that the more they use the WBDC and DRC the more familiar they become with the aid, the more they rely on the information generated by the aid. One informant also says that the WBDC can be made a routine as it is an important safety factor.

"There is always a false sense of security. Of course there is. Obviously the more you use it, the more you come to rely on it. However I do not believe there is an alternative to not having the DRC, simply because there is a false sense of security."

Good that the CDSS is not compulsory

The informants like the possibility to override the DRC. If it were compulsory, it would lose its role as an aid as ultimately it is always the doctor who has to make the medical decision, and sometimes, due to the patient's status, they have to deviate from the recommendations.

"It has nothing to say about what it thinks I should give instead. It doesn't do that, but just indicates… dose too low or do you really want to prescribe this dose. So it leaves the decision up to me. It doesn't forbid me from using that dose."

Benefit

The benefit for the informants from using the WBDC and DRC is dependent on various factors. In part it depends on the type and number of medicines doctors prescribe and within which department, but also on how much experience they have. Some think they benefit most from the DRC, while others value the WBDC more. Those doctors whom are familiar with the dose levels valued the additional check provided by the DRC. They see this support as an assurance.

The informants say that the WBDC helps them to set doses and the DRC functions as a check on the WBDC, warning and alerting the doctor of under- or over-prescribed doses. According to the informants the DRC is noticed most when it changes color.

"But the DRC is of course one additional safety check that prevents you from making disastrous mistakes. What I mean is that often errors in pediatric care, historically, are that… a prescription can often be made out for ten times the potency or there may be a decimal error. Now of course there is a DRC which ensures that this sort of thing couldn't happen."

Prompts consideration

Those who have received a warning in the DRC think it is a good thing and that it draws attention to the error, causing the doctor to stop and think. A number of informants also say that the doctor needs to consider when a warning has been received about too high or too low a dose, for example when escalating the dose level of prescribed antibiotics or pain medication.

" If I give him one milliliter it is an under-prescribed dose. Exactly, that happened to me the other day when I… instead of prescribing ten milliliters, it ended up as one milliliter, and it (the DRC) said that it was an under-prescribed dose."

Help with calculations reduces errors

All the informants said that the best support was that it prevented mistakes and helped avoid human error in calculations in terms of incorrect decimal places, which have historically occurred in pediatric healthcare.

"But for example, pain medication and antibiotics, where it is known there are a lot of errors. One such case is intravenous paracetamol. As I understand it, it has been given at doses that were ten times too high and things like that. So it's good that an alert is given in that case. Or you could be unsure as to whether it should be fifteen milliliters, one hundred and fifty milliliters or 1.5 milliliters. So it's really good then."

Greatest benefit in emergency care

Although a number of specialists have considerable experience and are familiar with the drugs they prescribe, they say that it is still easy to make a miscalculation, especially in situations in which the prescriber is stressed, tired, on-call or prescribing medication to children outside of his or her own area of specialization. Some informants think that the WBDC and DRC are of greatest benefit within emergency care, where a lot of prescriptions are issued in a short time in high-pressure situations.

"Of course you still have to calculate doses sometimes, when there aren't any WBDC and you have to work it out yourself, and you're tired and stressed or … I usually use drugs that I frequently use. That's when you have it at your fingertips, I'm aware of the dose ranges and I can tell for myself. But sometimes, mainly when you're on-call, there might be a drug that you don't prescribe so often, and that's when it is particularly important of course."

Confidence in the WBDC and DRC

Use of a manual Dose Range Check

Most doctors rely on the information from the DRC, but in the end they perform their own reasonableness assessment of the dosage depending on the medicine in question. Confidence however can be affected by contradictory information, i.e. when the information from the aid does not agree with information from other knowledge bases. One informant reported that in some cases inconsistent recommendations are given in national dosing guidelines and in the DRC and as a result the informant is not sure which source to use.

"I don't know if it's always right, because the national dosing guidelines can give a dose, only for it to come up as an excessive dose. I don't really know how it works… it calculates, not in accordance with the national dosing guidelines then, or I don't know…"

Double-checking the dosage

A number of informants report that when they receive a warning from the DRC or they have the dose level calculated in the WBDC, they still double-check it in the national dosing guidelines or local drug or disease specific instructions and re-calculate it. They might also ask a colleague to check and re-calculate the dose level. Doctors report that they often ask nurses to calculate dose levels and to check that they are reasonable, especially when prescribing infusions where information may be flawed. Doctors find this "double check" reassuring.

"But when I double-check, I repeat the calculation several times, calculating backwards and forwards. I work out how much I want and then I calculate in the opposite direction so that… So I will probably still have found a lot … so I don't just write it out, but I almost always check and double-check."

Situations in which the doctor disregards the Weight Based Dose Calculation and/or the Dose Range Check

When it is easy to work out the dosage using mental arithmetic

The informants say that they automatically work out the dose quickly in their heads when prescribing drugs that are easy to calculate. In these cases they forget about the WBDC or consciously decide not to use it. Some informants describe the routine for working out the dosage as a habit, especially amongst those who prescribe between 10 and 15 drugs once they have learnt the dosages for these.

"It depends on what kind of drug it is. Sometimes I don't use it, if it is something that is easy to work out per kilogram. Then I might issue a prescription directly in TakeCare…yes, without using the WBDC."’

In case of special indications

Informants say that there are drugs that are dosed above or below the reasonable dose level on special indications. The recommendations are also disregarded when prescribing certain infusions or when escalating antibiotics.

"So sometimes we do prescribe dose levels other than what is normal. Definitely, there may well be an indication for which some other dose has to be given. But then you probably know about that yourself. Because…there are of course a few drugs and…then you have to understand why it is flashing yellow."

"There are of course some drugs that are used at above the reasonable dose … yes, I do check whenever it [the DRC] turns yellow, … and in this case there was an escalation so I knew why it was too much. It feels very reassuring."

Misgivings/risks

False security and non-disease specific warnings

A theoretical risk seen by a few of the informants with the WBDC is potential over-reliance and not questioning the number of milliliters that the dose rage check recommends. One informant described this as the risk of being too much of a "checklist person" and no longer thinking independently and making one´s own decisions. Another prescriber however thinks that the CDSS: "only says whether the dose is too low or too high but leaves the decision up to the prescriber. But of course I do have to think about it."

Human error is unavoidable

Some doctors pointed out that there is always a risk of human error when dosing medicines. When a doctor makes a mistake in a prescription, this increases the risk of the nurse giving the patient an incorrect dose. Using the CDSS reduces the risk of errors of this kind. One doctor reports that prior to the implementation of the CDSS it was easy in stressful situations to confuse milliliters with milligrams.

"….and the dose was right, but I didn't see that it said milliliters instead of milligrams… but they managed to give him two incorrect doses…. The child was fine, but of course it could have turned out badly."

Wrong Dose Calculating Weight

One risk pointed out by the informants is entering the wrong DCW in the medicines module from the very beginning. This can happen as a result of the aid allowing doctors to give the DCW in grams or kilograms. In order to avoid errors, some informants suggest inclusion of a function which checks the weight for reasonableness in the same way as the dose.

" …the risk is that you might enter the wrong dosing weight. In TakeCare the weight is in kilograms, while in a lot of other systems it is in grams for neonates … So there ought to be a reasonableness check on dosing weight as well. It ought to do that."

Development potential

In general the informants think that the CDSS is needed, and that it would be a shame if it were lost. One informant says that it is good that it is there, "It makes you wonder why there hasn't been something like this before." Many of the informants however point out that it requires further development and adaptation to meet the needs of the user when setting a dose. Below are a number of suggested developmental measures in which the child's weight is a central parameter for the way in which the system is used.

Optional or compulsory – registering and signing for weight

In order to be able to use the CDSS, the child's DCW must have been entered into the CDSS. All the informants agree that weight is an important parameter when prescribing medicines for children, though there is some disagreement as to whether entering the DCW in the CDSS should be compulsory. The informants say that use of the weight parameter is of decisive importance when prescribing medicines for neonates, but that this becomes less important the older the child is. Some informants believe it is essential to give the current weight when prescribing drugs in pediatric medicine and that it is reasonable to require this. Some find the CDSS insistence on this irritating however. They refer to cases in which weight is of no particular significance or when prescribing drugs in some other context, for example when renewing a prescription or prescribing based on the child's body surface area. In such situations it is difficult to complete a prescription using the current solution. These prescribers want the indication of weight either to be optional or to apply only for infants. One suggestion is that a block is put in place based either on the patient's age or weight, or else the weight parameter can be inactivated.

"Weight might not play any role in that case, but instead they have to carry on with the same dose as they have had before because they're looking at the concentration in the blood for example, that's when you don't have to worry so much about the child's weight."

In order to save time and make prescription easier, the informants want the weight used by an administrating nurse in the medicines module to be automatically transferred from the medical records system to the DCW.

"Having to sign for the weight when the nurse has already taken a weight is an unnecessary step. Because that's something that I can't check… if you want to sign something as a doctor, then it's something you have some control over… something you've done yourself… but if a nurse has weighed a child, then it's that nurse who signs for the weight in her notes, and then it should be there already so that it's just a matter of prescribing and not have to sign for that weight."

Discussion

This study adds the views of pediatricians on Swedish EMR development. The qualitative approach is preferable as it explores the discourse of the users and made it possible in this study to feed back the unfinished parts to the developers. The need for constant development of drug related it-systems is known [2]. Despite this, the suggested changes are not jet been taken into consideration by the EMR provider. Our belief is that the possibility to be prioritized in the developmental process is higher when actual dosing errors occurs. It’s harder to find recourses to optimize a CDSS with a good reputation and functionality but with an apparent need for further risk minimization and better man machine interface, as shown in this article.

A similar method to acquire information from Iranian physicians’ on an EMR was used which found a willingness to use an EMR if these systems provide significant benefits over known problems [17]. And the addition of a CDSS has shown to reduce the number of adverse drug events within pediatric and neonatal settings [18,19].

Early in the process of interviewing we found it necessary for the informants to talk about other issues regarding the EMR and not only the CDSS as many informants experienced both EMR and CDSS as "a unit in the computer". This was allowed and the additional data adds to the list of developmental issues. To further elaborate on the acceptance, perception and use of the system we continued with additional interviews in a student project using an extended technology acceptance model [20]

One interesting part in the current study is the lack of reported warning fatigue by the DRC among the informants. Warning fatigue and non compliance is otherwise a common issue [21-23]. This could be an effect of a list restricted to high risk drugs and a build in real time warning system instead of a warning after completing the whole drug ordering process. The high override rates is otherwise a cause for concern which needs further analysis of the usability of the EMR and CDSS [23]. In addition to the overrides a recent review published the greatest challenges in CDSS [24]. They found primarily a need to improve the human-computer interface which is in line with our findings. The systems need to excel, not letting the health staff just try harder [25]. Likewise it is important to understand the psychological and organizational factors contributing to medications errors. If not engaging into a high safety organization serious events will continue to appear. The technical solutions are part of the advancement but can’t replace e.g. poor education, lack of sleep, work interruptions, intentional deviations from routines and the possibility to question authorities. The informants vocalize a healthy skepticism to the EMR and the CDSS and some of their concern could be addressed by better functionality of the CDSS. A list of suggested future challenges for our EMR and CDSS are published as a supplement (Supplementary Table 1) and the method of using a post development qualitative approach for tracking the need for additional changes has been one of few ways to scientifically communicate between developers and clinicians after the release and after the planned participatory development process. We encourage local development and follow up studies regarding CDSS for WBDC and DRC for EMR in pediatrics. We also see a need to communicate these findings to both CDSS developers and the National Board of Health and Welfare to state requirements of certain tools for the pediatric population when entering an EMR.

Supplementary Table 1

Functions which the informants think should be developed.

Suggested development	Examples of quotations from informants
1. Information about the CDSS	"In the dose range check I know that there's an upper and lower limit that I have to keep in mind….,, but I don't know who has decided on the doses. It's often the case that there are completely different ways of dosing an antibiotic for example; even though it's the same antibiotic, it can be prescribed at different doses depending on what disease the child has. And that's when you don't know what the dose range is based on for the condition the child has."
2. Prescribing also by body surface area	"Sometimes you would rather prescribe by body surface area, and that's when you need more parameters entered in the system. You should be able to set it so that you get body surface area there and not just weight. Children change, so there is a lot from childhood – from a large head and small body to fairly big children with a big body but where the head doesn’t grow that much bigger."
3. Import weight automatically from the medical records system to the CDSS	"Sometimes we have big patients for whom weight plays less of a role, so sometimes you want to temporarily remove or inactivate the dose calculation weight."
4. A flexible system with the option of inactivating the weight parameter	"And sometimes weight doesn't play any role, but I still have to enter it, so I do it just because the software requires it."
5. Also check range for weight	"There is a risk of entering the wrong dosing weight. In TakeCare the weight is in kilograms, while in a lot of other systems it is in grams for neonates … So there ought to be a reasonableness check on dosing weight as well.
6. More dose recommendations attached to the weight based dose calculation	"If there were text in a panel in the weight based dose calculation, this would allow the user to see straight away that the recommended dose is 20 mg/kg x 3 for example, and since it's easier to enter them directly in the weight based dose calculation, the dose will come out right. To make it easier to see the instructions."
7. The weight based dose calculation is used also with infusions, sodium chloride and 'as needed' medication	"We think in milliliters per kilogram and there's no weight based dose calculation for sodium chloride, so you can't write millimoles per kilo - you have to work it out yourself, and there are only solutions available as so many millimoles per milliliter, so you have to work it out yourself and there is a risk of error."
8. Dose range check for bolus doses or continuous infusions	"Some drugs can also be used as a continuous infusion, so you want to give the full daily dose administered. That's when you get into really large amounts and then… if the weight based dose range check could be arranged so that it works for a bolus, or a check for continuous infusion. One example is midazolam, or lidocaine. There are typical drugs like this, and morphine."
9. Selecting prescription in milligrams or micrograms	"For some drugs we don't talk in milligrams when prescribing them, but in micrograms per kilogram… and then you could in some way select the prescription in milligrams or micrograms. There are those situations when you have to sit down and calculate backwards and forward yourself to check that you do in fact get the dose right."
10. Improve user-friendliness	"Sometimes you want to be able to write times four or something like that. Now I have to hit tab and sort of transfer the number, twenty milligrams per kilo, then I have to move twenty, tab, write in twenty again, tab, twenty again. If I could write twenty times four in the weight based dose calculation, for example, then it would be a bit quicker."
11. Nurses should also have access to the CDSS	"Nurses who issue drugs really should be able to use it (the weight based dose calculation with dose range check) as well. Because they also have to consider whether they are giving a reasonable dose."

Conclusion

A need for CDSS in the prescribing for children is evident to support the prevention of medication errors. After implementing a CDSS, organized efforts are crucial to understand the need for further development based on the practical knowledge of the clinical profession. Different contextual settings of health care organisations do affect the way how physicians think and act in work. When implementing a CDSS in practice we need to describe and analyse the context where the CDSS should be used as well as the prescribers’ needs in work. A CDSS adapted to physicians’ specific needs could then be a working tool for taking care of potential medication errors.