What do double-check routines actually detect? An observational assessment and qualitative analysis of identified inconsistencies.

OBJECTIVES: Double checking is used in oncology to detect medication errors before administering chemotherapy. The objectives of the study were to determine the frequency of detected potential medication errors, i.e., mismatching information, and to better understand the nature of these inconsistencies.
DESIGN: In observing checking procedures, field noteswere taken of all inconsistencies that nurses identified during double checking the order against the prepared chemotherapy.
SETTING: Oncological wards and ambulatory infusion centres of three Swiss hospitals. PARTICIPANTS: Nurses' double checking was observed. OUTCOME MEASURES: In a qualitative analysis, (1) a category system for the inconsistencies was developed and (2) independently applied by two researchers.
RESULTS: In 22 (3.2%) of 690 observed double checks, 28 chemotherapy-related inconsistencies were detected. Half of them related to non-matching information between order and drug label, while the other half was identified because the nurses used their own knowledge. 75% of the inconsistencies could be traced back to inappropriate orders, and the inconsistencies led to 33 subsequent or corrective actions.
CONCLUSIONS: In double check situations, the plausibility of the medication is often reviewed. Additionally, they serve as a correction for errors and that are made much earlier in the medication process, during order. Both results open up new opportunities for improving the medication process. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study to investigate what kind of information double checks actually detect.

Using an observational approach and assessing a large number of double checks allows for insights and conclusions that are relevant for clinical practice.

If the potential severity of the detected inconsistencies were differentiated, a more precise potential value could have been derived in terms of prevention of medication errors that the double checks may have had.

Introduction

Chemotherapy medication errors can have severe consequences and are prevalent in oncology care.1 2 In order to make sure that a certain patient gets the right drug in the right dose at the right time via the right route—in short, to detect potential medication errors before a drug is administered, double checking procedures are often introduced in the chemotherapy medication process. They are intended to act as a safety barrier before the administration, as a chemotherapy medication error may have severe or fatal consequences due to the toxicity of the drugs, their narrow therapeutic range and the vulnerability of the patients. A survey study in Switzerland has found that oncology nurses strongly believed that double checks were an effective means to reduce medication error rates and to enhance safety.3 However, as two systematic reviews reported, the evidence supporting double checking as a safety increasing method is weak: the effectiveness of double checking procedures in reducing medication error rates and patient harm, i.e., in increasing medication safety has not yet been demonstrated empirically,4 5 despite positive experiences being reported.6–8

The aim of double checks applied in oncology often is to identify a potential inconsistency of information between two references: the order (as print-out or on a computer screen) and the actual drug (a label on an intravenous bag or a label on a bag or box of pills). An inconsistency means that for example the name, the dose or the day of administration on the drug label does not match the order. The idea behind these checks is that identifying and clarifying inconsistencies helps detecting medication errors before the nurse takes the medication to the patient to administer it, with the ultimate aim to reduce medication errors.

While prior research has investigated the effectiveness of double checks versus single checks,9 10 or the adherence to checking procedures,5 11 no study has assessed what kind of inconsistencies are actually identified during double checks. To be able to identify the potential value of double checks, it is however important to better know what kind of mismatching information double checks detect. In order to address this research gap, we observed double checking procedures in oncology care. The aims of this study were (1) to determine the frequency with which double checking procedures identify an inconsistency, for example, between order and drug (infusion) label; (2) to analyse the inconsistencies identified in these situations in order to develop categories to classify and better understand the inconsistencies. Thus, we did not detect actual administration errors but focused on the inconsistencies that were detected before administration in performing double checks.

While checks are applied throughout the chemotherapy medication process from order to administration,12 we focused in this study on the check i.e., done by nurses after the medication is produced and delivered to the unit and before it is administered to the patient. In many oncology care settings, a double check is applied at this point in the process,10 13 probably, because this is the moment in which the produced chemotherapy enters the nursing medication preparation and administration process. This is the opportunity to intercept any wrong information or error before administration.

Methods

The study was designed as a non-participant observational study. It was part of a larger project on observing double checking procedures.

Setting

We observed medication checking procedures in wards and ambulatory infusion units of three Swiss hospitals (two ambulatory infusion units and the oncological wards of two large university hospitals and one regional hospital). In all of them, the hospital pharmacy produced and labelled the infusion bags. In one hospital, pharmacy also prepared and labelled the oral chemotherapy pills; in another, the nurses prepared the oral drugs. In the third hospital, no oral drugs were dispensed by the ambulatory infusion centre due to regulative restrictions on canton (state) level.

In all three hospitals, the orders were entered in a computer-based system, however, only in one hospital the physician order system directly communicated with the pharmacy production system. In the other systems, pharmacy needed to transfer manually the order into their production system, which was software CATO for all three hospitals. The nurses usually had the order in front of them, either as print-out or on the computer screen. Barcode scanning was not present in any of the hospitals.

The observer was present in the room in which the medication was prepared and observed the double checking procedures for chemotherapy - infusion bags as well as pills that were handed to the patients. As the rooms are small and often busy, the researcher sometimes stood by the door of the room in order not to interfere with the work processes until a check was performed. Most of the double check procedures were assessed during day-long observations in the ambulatory infusion centres or a day clinic. On the wards, we only observed at specific times, when the medication was checked for the current shift, often in the morning.

Checking procedures and inconsistencies

In the observed units, two different checking procedures were applied: two nurses collaborating simultaneously in comparing the order to the actual drug produced, for example, in a read-read back procedure,13 or two nurses checking separately, for example, distant in time and/or space from each other. Due to the ritualistic manner in which the double checks were performed—particularly the collaborative ones—inconsistencies could well be detected by the observers: an inconsistency was defined as a deviation from the usual checking behaviour that may relate to a missing clarity, a question or a remark regarding the information to be checked. Thus, during the medication administration process, an inconsistency is the point at which nurses’ ‘investigation’ is initialised, whether there is an error, or not.

The nurses checked the name and the birthdate of the patient, the current date, the drug, the dose and for infusion bags the kind of solution and the duration of the administration, if already calculated. Sometimes, they also mentioned the date of expiry and the kind of storage that was needed for the drug (cooling, light protection), an information which usually was only displayed on the drug and not in the order. In some places and times, the nurses also conducted the calculation of the infusion duration together after checking these items. As calculating is a cognitively different activity from checking,14 and because it was not a routine part of the checking procedure, we did not integrate calculations in the study.

Data collection procedure

Usually, the nurses conducted a double check right after preparing a set of medication for administration. During the nurses’ medication preparation process, the observer was on stand-by, attentively watching in order to detect any upcoming checking situation. When a double check was conducted, data assessment started. After having observed the complete double check, the observer took a note when an inconsistency was identified during the double check process. This means, the observer wrote down on a sheet of paper when the nurse talked about anything else than the information to check, for example, questions or remarks relating the medication that the nurses talked about in the checking situation. For the collaborative checking procedures (conducted in all units except one ambulatory infusion centre), it was easy to assess an identified inconsistency, as the nurses talked to each other. During the separated procedures, we therefore observed the eye movement and the subsequent actions in order to assess whether during the check, the nurse identified something that needed clarification of the checked set of information. As this usually led to subsequent actions like looking something up, writing something down or asking another person a question, it was possible to assess the inconsistencies in these situations, too. The observer also asked questions in order to fully understand what the issue was that was identified, either right after the nurse(s) finished their task or in the evening after the assessment day. As in the observation situation it was not always easy to determine what is and is not an inconsistency, all upcoming questions and issues related to the medication of the patient during the double check were recorded. The notes were transferred into a digital document by the observer and checked by the other researcher for understandability after each assessment day, so that any missing clarity could be eliminated while the observation was still ‘fresh’ in the mind of the observer. Two observers conducted the assessments, both were trained in patient safety, one was additionally a trained nurse (CZ) and the other a trained psychologist and safety expert (YP). The observation approach was tested in trial assessments prior to the actual study assessments.

Participants

Each observed nurse was informed about the aims of the study, the measures to assure the anonymity of data collection, the expected duration of the study and that participation in the study was completely voluntary and could be waived at any point in time. Their informed consent was documented in signing an agreement.

Sample

We observed N=868 checking situations and assessed upcoming inconsistencies. Of those checking situations, N=512 were related to double checks performed by two persons collaborating simultaneously; N=356 were related to checking situations in which one nurse checked alone as part of a double check, thus these checking situations amount to N=178 double checks. We use the term checking situation to account for the fact that a double check conducted in a collaborative manner usually takes place in one situation, while a double check i.e., performed separately consists in two checking situations. The resulting total of observed double checks was N=690.

Data analysis

The field notes from the observations were first analysed in a qualitative content analysis and subsequently, the category frequencies were counted. The objective of the analysis process was to develop categories describing the kinds of inconsistencies that come up in double checking. Qualitative data analysis involved two broad steps: in the first step, the category system for analysing the field notes was developed. In the second step, the category system was applied to analyse all notes of inconsistencies. One field note contained a description of one observed inconsistency, which means that for the four checks in which more than one inconsistency was identified, the according field note was also split so that one inconsistency was described per note. Three researchers worked in the analysis process, the two observers (CZ and YP) worked as coders and one researcher (DS) as advisor in taking decisions. For the category development, the two coders first coded independently from each other, then discussed and iteratively adjusted the categories developed. CZ coded the whole dataset and YP coded only a subset of the data. The final category system was tested. After discussion with the advisor, the category system was ‘frozen’. One researcher (CZ) coded all data again applying the final category system. After that, the other researcher (YP) also coded the whole dataset independently and blinded to the first researcher’s codings. After each third of the dataset that was coded for the second time, differences between the researcher’s respective codings were discussed in the research team until agreement was achieved. Thus, at the end of this analysis process, the notes of observed inconsistencies that came up during double checking situations and were related to chemotherapy were categorised twice and an agreement was reached between the participating researchers. This procedure made sure that (a) potential misunderstandings and coding errors were identified and (b) differing views were openly discussed, in order to maximise objectivity. The software Atlas.ti was used for the analyses.

Only chemotherapy-related inconsistencies were analysed. Two inconsistencies involving folinic acid, which is used as adjunctive in chemotherapy, were also included. Three inconsistencies related to the determination of flow rates were not included, as the calculation of flow rates was not systematically part of the double check procedure.

Patient and public involvement

The idea to do the larger research project in which this study was conducted evolved from the questions we received from healthcare workers in Switzerland about whether and how to best apply double checking for high-risk drugs. From the time when the project was funded, the public was informed on the website of the Patient Safety Foundation, via the newsletter and in talks on double checking, the research project and its results given at collaborating hospitals and at conferences. We additionally published a recommendation of how and when to use double checking in hospitals which is freely available.

Results

In 22 of 690 double checks (3.2% of double checks), 28 chemotherapy-related inconsistencies were detected; in 4 checks, several (2–3) inconsistencies were identified. During all double checks conducted in a separated procedure, 9 inconsistencies were detected and 19 were detected during the collaborating checking procedures.

Table 1 shows the resulting coding scheme and the frequencies of the categories, along with examples. Analysing the nature of each inconsistency detected, we identified that there were two different kinds of inconsistencies depending on the sources of information or knowledge used:

Table 1

Coding scheme, frequencies and examples

Coding scheme	Frequency	Examples
Type of inconsistency
Disconcordant pair of external information	14 (50%)	A certain drug was ordered to be diluted in 100 mL but was delivered by pharmacy in 250 mL; the duration of the infusion was indicated as being 15 min on the order but was labelled as 30 min by the pharmacy.
Disconcordance between external information and knowledge	14 (50%)	Nurses correcting the infusion duration because it was wrongly ordered or wrongly labelled or both; nurses identifying a wrong ward indicated on the infusion bag; nurses identifying a wrong infusion set.
Total	28 (100%)
Origin of inconsistency
Prescribed infusion duration	12 (43%)	Infusion rate for a first-time administration was ordered, although patient was getting second administration; drug was prescribed to be administered in 15 min, information on drug by pharmacy indicated infusion duration of 30 min.
Wrong quantity of infusion	3 (11%)	The right drug amount was diluted in more solution than prescribed, thus the pharmacy had already corrected an ordererror; the pharmacy having produced 400 mg of a chemotherapeutic drug, while only 390 mg was ordered.
Wrong date on prescription	2 (7%)
Other order-related issues	4 (14%)	Order not yet cleared by the senior physician; pieces of information missing on the order that needed to be filled in; order was changed, but nurses did not know and used old order in checking the produced chemotherapy; carrier solution was ordered to be sodium chloride but was corrected to dextrose by nurse.
Wrong or missing information on the drug label	4 (14%)	The wrong organisational unit on an infusion bag; a missing date on a pack of pills; a wrong duration for taking chemotherapy pills in relation to the number of pills prepared.
Other	3 (11%)	Nurse took wrong prepared chemotherapy infusion bag from refrigerator; two inconsistencies could not be unambiguously categorised.
Total	28 (100%)
Subsequent and corrective actions*
Correcting the order	18 (55%)
Communicating with another person about the inconsistency	8 (24%)	To another nurse (2), to a pharmacist (2); to a physician (4).
Correcting the drug label	3 (9%)	By a nurse (2); by a pharmacist (1).
Calculation repeated	2 (6%)	How long one set of pills were to be taken at home; the infusion rate of an infusion to be administered over 2 days.
Put back wrong infusion bag in refrigerator	1 (3%)
Look something up	1 (3%)	As the organisational unit name was missing on the infusion bag, the nurses looked up in the system whether there were two persons with the same name in the same unit.
Total	33 (100%)
No subsequent action	4 (0)

*The number of actions is higher than the total of inconsistencies, as more than one action may have resulted from an inconsistency. Percentages relate to total number of actions here.

The nurses identified (a) 14 (50%) disconcordant pairs of information between the label of the drug (12 bag labels for intravenous chemotherapy, 2 labels for pill boxes) and the order. They also identified (b) 14 (50%) inconsistencies between information on the drug label or the order and their knowledge. By knowledge, we meant expert knowledge about drugs, and therapies, and situational knowledge about the patient and its condition, the cycle he or she was in, about usual order inadequacies and handling of information by the pharmacy.

Furthermore, in focusing on the origin of the inconsistency, we found that many related to inconsistent infusion durations (12 of the 28). In sum, 21 (75%) inconsistencies could be traced back to information that was already inappropriate or wrong on the order.

We also assessed n=33 subsequent or corrective actions of the nurses in response to the inconsistency detected. While 4 inconsistencies did not lead to any subsequent action, 17 led to one action and 7 inconsistencies to two or three actions (per inconsistency, 1.2 actions were carried out, SD: 0.77). More than half (n=18, 64%) of inconsistencies were acted on in correcting the order. We also recorded eight actions relating to communicating to other persons, which often meant to do a phone call, for example, because the persons were not in the room.

Discussion

This study is the first to explore the immediate inconsistencies detected by double-checking medications. Two results of this study are particularly standing out: first, half of the identified inconsistencies were not identified in checking two sets of information against each other, but in nurses using their own knowledge to evaluate the information on the label or the order. Second, a majority (75%) of the identified inconsistencies originated from the order.

Currently, there is no study quantifying the evidence regarding the relation between nurses’ double checking and actual patient harm.5 However, the potential of double checks to detect administration errors that would have otherwise resulted in actual patient harm is expected to be rather low,5 putting into question the considerable resources and cognitive capacity invested in double checking. The kinds of origins of inconsistencies support this observation, as many of them were related to infusion duration, or missing or wrong information on the order that was often corrected by the nurses. However, the plenty of physician ordering errors or problems that needed to be corrected by the nurses or by pharmacy point to a systemic problem of physician order quality that emerges in the nurses’ double check before administration, i.e., at the very front-end of cancer care. Working on physician order quality in a double check before administration is a misplaced use of human resources and also represents an allocation of responsibility to the nurses that would not be necessary if the quality of the orders was assured earlier in the medication process. Additionally, double checking is known to be a process vulnerable to factors such as human fallibility15 reducing its effectiveness. In line with the approach proposed by Trbovic and Shojania16 of addressing issues at their root cause, we therefore argue that assuring a high physician order quality would be more effective than performing double checks for attaining it. It also would save the time i.e., involved in clarifying inconsistencies, for example, of calling a physician to clarify an order. Our study has shown that each inconsistency entails more than one such action in the mean.

Double checking has been criticised for only catching a part of medication errors due to several reasons, for example, complacency in performing the checks,17 disturbing environmental conditions13 or non-adherence to checking protocols.18 19 The collaborating checking procedure itself has been criticised as a ritualistic chant that reduces attention to detail.10 20 The results of this study open up a completely new perspective on double checking: half of the identified inconsistencies did not result from comparing two sources of information, i.e., the order and the drug label, but from using own knowledge as reference to review information for plausibility.21 Thus, cognitive activities like critical thinking which are different from the mechanistic thought processes applied during the checks, for example, read-read back procedures, were performed during the checks. Consequently, many inconsistencies were identified that cannot be found by checking the concordance or disconcordance of pairs of information: such as flow rate not adapted to the chemotherapy cycle the patient was in, or the wrong ward indicated on an infusion bag. Interestingly, identifying mismatching information in a check goes back to a rather mechanistic cognitive activity,22 which is performed best when all other influences of sense-making are reduced in order to avoid cognitive biases. In contrast, critical thinking and reviewing information using own knowledge works best if all the knowledge that a person may have is actualised. In their analysis of checking procedures, White et al22 proposed that abstract thinking, i.e., critically reviewing a set of drugs i.e., to be administered, is important for medication safety and that this activity should be separated from other mechanistic tasks. Specific research is needed on how to support that kind of activity. Our results thus point to a potentially powerful opportunity to detect errors before drug administration: new ways of integrating critical thinking into the medication process are needed, an argument that Rohde and Domm23 have also brought forward in their review. For example, it may be worthwhile to define specific locations and times for the critical review of the appropriateness of the drugs for a certain patient. A review for plausibility could be conducted before administration and in a space that allows the nurse to distant theirself from the daily business and to very attentively think about the patient and the drugs to be administered at hand. The space may be a booth or an area indicated using duct tape on the floor.24 Moreover, it would be useful to train nurses in doing checks and in doing plausibility reviews, so that they could activate the appropriate mindset for the activity at hand. Specific descriptions of the behaviour expected to be performed within checking procedures are often missing in nursing guidelines, although a prior simulation study has shown that checklists for conducting double checks increased error detection.22 Our results thus point to the need to specifically describe what kind of behaviour is expected to be performed in a check and in a plausibility review, respectively, and to distinctly differentiate them within the medication process in defining specific locations and times.

Hewitt et al25 discussed that double checking also can be regarded as a tool for organisational learning as the double checks had an informal part in which the nurses sometimes would bring up best practices or an opinion on how to accomplish a task. This is only true for double checks that are conducted by two persons working simultaneously. The results of our study show that double checks are a potential source of error detection going back to critical thinking. However, this critical thinking would better be designed to be a separate activity in the medication process for the above-mentioned reasons. Future research should thus not only address the effectiveness of ‘single check’ versus ‘double check’, but also the pair 'double check' versus 'single check plus plausibility review' and its potential to identify relevant medication errors.

Limitations

The inconsistencies detected were not categorised differentiating their severity and probability for harming a patient. This would have allowed to better gauge the potential value of double checking for avoiding medication errors based on our results. We refrained from attributing severity to the identified consistencies, as we could not evaluate how probable it would have been for a specific error to be caught after the check, and because we were not able to reliably evaluate the potentially resulting harm for a patient. It is also possible that the inconsistencies that surfaced in the double check go back to appropriate therapeutic interchanges that are not clinically significant. The study did not assess the clinical significance of the identified inconsistencies. However, if they were not clinically relevant, the inconsistencies still elicit subsequent actions that produce interruptions, take up resources and could potentially be avoided by better process design, i.e., aligning information throughout the medication process.

It is possible for the separated checking procedures that the observers may not have captured all inconsistencies. However, the observers were well-trained and acquainted with the work processes before the actual data gathering and they had the possibility to ask questions to understand whether there was an inconsistency identified. These measures supported a comprehensive assessment of all inconsistencies.

In order to reduce potential subjectivity in the qualitative date analysis of the notes taken, we applied the following means: all the notes were coded twice by two coders working independently from each other, supervised by a third researcher that was involved in discussing non-alignments.

Conclusion

Double checking has long been performed to improve medication safety in catching errors, specifically in the preparation and administration of high-risk drugs such as chemotherapy. However, evidence of its effectiveness to do so has not been established satisfactorily to date. This study analysed what kind of information is actually detected within double check processes in oncology care. Its results point to a function of checks that has not yet been discussed: double checks may work as a moment of critical thinking about the appropriateness of the specific drug administration. This seems to be an important element of check situation, which is not supported by any checking procedure to date. The value of checks considering their costs in terms of human resources has been debated,5 we therefore argue based on the results of this study for integrating moments designed specifically for plausibility reviewing into the high-risk medication nursing process. This would allow nurses to activate an appropriate mindset21 22 to use their own knowledge for the prevention of errors. Additionally, this study showed that errors should be better fixed at their source rather than be allowed to migrate through the system and be discussed and potentially fixed by the nurses during a double check, as was the case with a large amount of identified inconsistencies that went back to order quality.