Designing an electronic blood-borne virus risk alert to improve uptake of testing.

The primary aim of the current study was to test the effect of the presentation design of a test alert system on healthcare workers' (HCWs') decision-making regarding blood-borne virus (BBV) testing. The secondary aim was to determine HCWs' acceptance of the system. An online survey used a within-subjects research design with four design factors as independent variables. The dependent variable was clinical decision. Ten realistic descriptions of hypothetical patients were presented to participants who were asked to decide whether to request BBV testing. The effect of a pre-set course of action to request BBV testing was significant when additional information (cost-effectiveness, date of last BBV test or risk assessment) was not presented, with a 16% increase from 30 to 46% accept decisions. When risk assessment information was presented without a pre-set course of action, the effects of cost-effectiveness (27% increase) and last test date (23% decrease) were significant. The main reason for declining to test was insufficient risk. HCWs' acceptance of the test alert system was high and resistance was low. We make recommendations from the results for the design of a subsequent real-world trial of the test alert system.

Introduction

Background

Late diagnosis of HIV, hepatitis B and hepatitis C, the blood-borne viruses (BBVs), remains common in the UK despite national guidelines and other efforts to increase BBV testing amongst those at risk of infection.[1,2] Increased testing in primary care and certain hospital settings has been prioritized, particularly for HIV.[3,4] However, in many areas of the nation, testing rates have remained low, and late diagnosis is widespread: in around 40% of new HIV diagnoses nationally and over 50% in some parts of north-east England.[5-8] There is often a reluctance amongst general practitioners (GPs), hospital doctors and other healthcare workers (HCWs), especially those less experienced in seeing and managing BBVs, to offer BBV tests as recommended in national guidelines.[9-11] This is due to various reasons, including misconceptions around the process of consent for testing and a lack of appreciation of patients’ risk factors. HIDES-1 and other studies have found that patients presenting with certain indicator conditions have an undiagnosed HIV prevalence > 0.1%, where screening is cost-effective.[11,12] Hence targeted HIV testing amongst patients with identified risk factors and indicator conditions is likely to be cost-effective and lead to earlier diagnosis in those with undiagnosed infections, even in lower-prevalence areas.

Efforts to increase testing for these BBV infections in primary and secondary care using standard or opt-out processes have mostly had limited success or have required significant resources to maintain high testing rates.[13-15] One recent approach in accident-and-emergency departments, the ‘Going Viral’ campaign had some success, but still requires significant staff input to maintain high testing rates.[16] New approaches to improving rates of BBV testing are needed. Electronic patient record (EPR) systems are universally used in primary care and many areas of secondary care. Over recent years, these systems have allowed the development of clinical decision support algorithms to guide clinical management, including the choice of tests and highlighting abnormal results.[16,17] Studies, however, have demonstrated the importance of usability and the involvement of users in their development to ensure the systems work effectively.[18-20] Such systems provide an opportunity to prompt HCWs to perform BBV tests where pathology and EPR data (such as patient demographics and clinical codes) indicate the patient may be at higher risk of undiagnosed BBV infection, ideally at the point of ordering tests during the HCW–patient interaction.

Critical to the success of such an application is the way the alert is displayed and how it interacts with a clinician, particularly during the clinician–patient interaction. Various studies have identified design factors that are important in the design of such systems and alerts to ensure HCWs view the advice, accept it when appropriate and do not succumb to ‘alert fatigue’. Positive predictive factors include quality of display, alert priority level and level of detail of the advice given.[21] Moreover, the position of an alert was a significant predictor of acceptance of the alert advice, with higher acceptance for alerts at the top of the page than those at the bottom.[22] The frequencies of alerts per encounter and repeated reminders may also predict acceptance, although the literature shows a mixed effect, with one study showing reduced acceptance of the alert advice[23] whilst another shows increased acceptance;[21] reduced acceptance may indicate alert fatigue, but increased acceptance may be due to increased familiarity. Furthermore, in drug–drug interaction alerts, trust cues (information that suggests trustworthiness, e.g. endorsement from the head of department) influenced alert acceptance.[24]

Study aims

Given the importance of an alert application’s information display and interaction with clinicians, the primary aim of the current study was to test the effect of the presentation design of a test alert system on HCWs’ decision-making regarding BBV testing. Given the importance of HCWs’ acceptance of new technology in the workplace,[25,26] the secondary aim was to determine their acceptance of BBV alerts. We address the following research questions:

What is the effect of pre-selection of decision on decision-making?

What is the effect of the date of the last BBV test?

What is the effect of cost-effectiveness information?

What is the effect of risk assessment information?

What is the level of acceptance of BBV testing?

How does acceptance vary by HCW characteristics?

Methods

Research design

A within-subjects research design (Table 1) with four independent variables was used: pre-selection of decision (decision to accept the advice to order a BBV test pre-selected or not pre-selected), presentation of the date of the last BBV test (present or absent), presentation of cost-effectiveness information (present or absent) and risk assessment presentation on request (present or absent). The dependent variable was decision (accept/reject).

Table 1.

Research design.

	Risk assessment presentation on request				No risk assessment presented
Pre-selection of decision	P1	P2	P3	P4	P9
No pre-selection	P5	P6	P7	P8	P10
	Date of last BBV test presented		Date not presented
	CEI presented	CEI not presented	CEI presented	CEI not presented

BBV: blood-borne virus; CEI: cost-effectiveness information; P: presentation.

Participants

Participants were recruited through the North East and North Cumbria Primary Care Research Network. They were compensated for their time by a fixed payment of a £5 voucher. Participants were 79 HCWs (60 GPs and 17 nurse practitioners and 2 others), 54 female and 25 male. Mean age was 44 years (SD = 9.22).

Materials, equipment and procedure

Realistic descriptions of ten hypothetical patient scenarios were developed (see online supplementary material), with the patient presenting to an HCW, and each patient representing one cell in the research design (Table 1). The patients had varying presentations that were not clearly related to typical presentations of HIV infection. These were presented as screenshots of the BBV prompt with an EPR system screen in the background.

Based on the description, in an online survey HCWs were asked to decide whether to request BBV testing. Presentations 1–8 allowed an initial choice of accept, reject or request more information. If the choice was ‘request more information’, risk assessment information was presented; the final choice options were then to accept or reject. Presentations 9 and 10 only allowed a final choice of accept or reject, without additional information.

Further questions measured additional consultation time, preference for number of tests included in the prompt, acceptance of and resistance to using the prompt system, preference for duration of prompt suppression after previous rejection by HCW or patient, and preference for prompt presentation. A soft prompt shows a patient’s elevated risk without requiring a HCW to respond, but a hard prompt does require a response. Acceptance and resistance of the prompt system was measured each with a 7-point Likert scale item adapted from previous research.[27,28] In addition, reasons for rejecting the advice to order a BBV test in response to the prompt were recorded.

Data analysis

The design required two data analyses of HCWs’ decision regarding testing for BBVs:

2(pre-selection of decision)×2(risk assessment information) analysis, with date of last BBV test and cost-effectiveness both absent (Presentations 4, 8 and 9–10);

2(pre-selection of decision)×2(last test date)×2(cost-effectiveness) analysis, with additional information present (Presentations 1–8). In Presentations 1–8, both an initial decision (accept, request further information or reject) and a final decision (accept or reject) were available for analysis; if the initial decision was to accept or reject then the final decision and the initial decision were the same. In Presentations 9–10, only a final decision (accept or reject) was available for analysis.

The clinical decision was analysed with generalized mixed-effects binary logistic regression models, including random intercept and participant as a random factor. Other outcomes were analysed with descriptive statistics, confidence intervals and Chi square tests. Open questions were analysed with thematic analysis.

Results

Aim 1: The effect of presentation design on HCWs’ decision-making

Final decision by pre-selection of decision and risk assessment

Pre-selection of decision. The effect of pre-selection of decision (‘accept’ pre-selected or not) was positive (more ‘accept’ decisions), when neither the date of the last BBV test, nor cost-effectiveness information nor risk assessment information was presented (Table 2, Panel A and Figure 1).

Table 2.

Decision analysed by pre-selection of decision and risk assessment information.

Predictor	b	SE	OR	z	p
Panel A: pre-selection of decision
Pre-selection of decision	−1.10	0.47	0.33	−2.34	0.02
Panel B: pre-selection of decision and risk assessment information
Pre-selection of decision	−1.07	0.44	0.34	−2.45	0.01
Risk assessment	−0.73	0.43	0.48	−1.68	0.09
Pre-selection by risk assessment	1.55	0.60	4.71	2.24	0.03

Note: Decision coded as 0 = ‘accept’, 1 = ‘decline’.

Figure 1.

Decision by pre-selection of decision and risk assessment information.

Pre-selection of decision and risk assessment. When risk assessment information was available, the interaction effect of pre-selection and risk assessment was significant (Table 2, Panel B and Figure 1). Specifically, the positive effect of pre-selection was stronger when the information was not available.

Analysis of final decision by pre-selection of decision, last-test date information and cost-effectiveness information

As the initial decision was predominantly ‘request more information’ (77% of responses), the final decision was analysed. This was done by pre-selection of decision (‘accept’ pre-selected or not), date of last BBV test (presented or not) and cost-effectiveness information (presented or not).

Significant were the following effects: the interactions of pre-selection of decision and date of last BBV test and of pre-selection and cost-effectiveness, as well as the main effects of date and cost-effectiveness (Table 3, Panel A and Figure 2).

Table 3.

Decision analysed by pre-selection of decision, date of last BBV test and cost-effectiveness information.

Predictor	b	SE	OR	z	p
Panel A: pre-selection of decision, cost-effectiveness information and date of last BBV test
Pre-selection of decision	0.00	0.44	1.00	0.00	1.00000
Cost-effectiveness information	−1.85	0.47	0.16	−3.97	0.00007
Date of last BBV test	2.22	0.53	9.18	4.21	0.00003
Pre-selection by cost-effectiveness	2.88	0.67	17.87	4.32	0.00002
Pre-selection by date	−2.12	0.69	0.12	−3.09	0.00200
Cost-effectiveness by date	−0.66	0.68	0.52	−0.97	0.33400
Pre-selection by cost-eff. by date	0.13	0.94	1.14	0.14	0.89200
Panel B: cost-effectiveness information and date of last BBV test (without pre-selection)
Cost-effectiveness information	−1.76	0.61	0.17	−2.86	0.00421
Date of last BBV test	1.98	0.73	7.26	2.70	0.00694
Cost-effectiveness by date	−0.15	0.83	0.86	−0.19	0.85343
Panel C: cost-effectiveness information and date of last BBV test (with pre-selection)
Cost-effectiveness information	1.02	0.47	2.77	2.18	0.02910
Date of last BBV test	0.00	0.44	1.00	0.00	1.00000
Cost-effectiveness by date	−1.38	0.73	0.25	−1.89	0.05880

BBV: blood-borne virus.

Note: Decision coded as 0 = ‘accept’, 1 = ‘decline’.

Figure 2.

Decision by pre-selection of decision, cost-effectiveness information and last-test date information.

In simple-effect analysis for the conditions without pre-selection of decision (Table 3, Panel B), the main effect of cost-effectiveness was significant, with more accept decisions when cost-effectiveness information was presented. The main effect of date of the last BBV test was also significant, with fewer accept decisions when the date was presented.

In simple-effect analysis for the conditions with pre-selection of decision (Table 3, Panel C), the effect of cost-effectiveness was significant, with fewer accept decisions when cost-effectiveness information was presented.

Reasons for not requesting a BBV test

On the initial decision, participants gave various reasons for not requesting a BBV test (rejecting the BBV test prompt) at the time of a consultation during which a BBV prompt would appear. The main reason was that insufficient risk had been identified (20 responses). Other reasons included discussing with the patient before test-ordering (eight), a lack of time (seven), obtaining further risk information (five) and doing other tests or testing elsewhere (five).

On the final decision, the following reasons were given (72 responses in total). The main reason was discussing with the patient, with shared decision-making and consent as major considerations (26 responses). Other reasons included doing other tests first (15), obtaining further risk information (8), first asking a GP to review BBV-testing (8 responses, all from nurses), precedence given to the presenting problems or symptoms during the consultation rather than to BBV testing (7) and offering the patient BBV testing (instead of accepting the prompt) (7).

Aim 2: HCWs’ acceptance of the BBV alert system

Additional consultation time

Mean estimated additional consultation time was 3.71 min (see also Table 4). This time was positively correlated with years of work experience (r = 0.31, p = 0.007). Moreover, nurses estimated the additional consultation time higher than GPs (Table 4).

Table 4.

Healthcare workers’ acceptance of the BBV alert system.

		95% confidence interval
Outcome measure	Mean	Lower limit	Upper limit
Additional consultation time
All healthcare workers	3.71	2.92	4.44
Nurse practitioners	5.80	3.80	7.69
General medical practitioners	3.17	2.43	4.02
Acceptance	3.44	3.24	3.66
Resistance	2.41	2.20	2.62

BBV: blood-borne virus.

Note: Bootstrapped bias-corrected accelerated confidence intervals.

Preference for number of tests

A statistically significant 67% of HCWs preferred a single test (for all three BBVs) rather than multiple tests (i.e. several different alerts for individual BBVs), Chi square (1) = 9.23, p = 0.002.

Acceptance and resistance

Overall, acceptance of the BBV alerts was above average (neutral scale value of 3) (Table 4). Overall, resistance to the BBV alerts was below average (neutral scale value of 3) (Table 4).

Preference of prompt presentation

Preference of prompt presentation was predominantly (90%) soft (user can dismiss the prompt).

Duration of prompt suppression after BBV testing has been declined previously

Preference of the duration of prompt suppression after a HCW has declined to order a BBV test was predominantly six months (43% of responses) or one year (48%). Preference for the duration of prompt suppression after a patient has refused a BBV test was predominantly one year (53% of responses), followed by six months (18%) and two years (15%).

Discussion

Presentation design

The effect of pre-selection of decision was positive and significant when additional information (cost-effectiveness, last test date or risk assessment) was not presented, with a 16% increase from 30 to 46% accept decisions and an odds ratio of 1.92. This result can be explained with the concept of nudge,[29] a (deliberate) change in the environment that influences choice behaviour. Specifically, the result is consistent with other evidence of the power of pre-selected options (‘default nudges’) to positively influence behaviour in other domains.[30,31]

When risk assessment information was presented and without pre-selection of decision, the effects of cost-effectiveness (27% increase) and last test date (23% decrease) were significant. In support of our findings, uncertainty about cost-effectiveness has been identified as a barrier to the use of point-of-care tests.[32] However, the effect of cost-effectiveness was negative with pre-selection of decision; this indicates the importance of carefully considering the combination of design factors (such as the presentation of cost-effectiveness information) in alert system design. The negative effect of date of the last BBV test may be because presence of the information suggests that the patient has already been tested. However, this effect potentially indicates a lack of knowledge in HCWs about the importance of regular testing, based on the available data from a patient record and knowledge of their risk factors. Nevertheless, if new patient data after the date of the last previous test trigger the test prompt then BBV testing should be re-considered, as the trigger may not have been present at the date of last testing. Education of HCWs will be needed to improve their knowledge and decision-making in this respect. The predominant reason (71%) for declining to test for BBVs was insufficient risk. In support of this finding, risk perception has been identified as a determinant of decision-making in models of risk decision-making[33] and in protection motivation theory,[34] according to which both people’s threat appraisal (including risk perception) and their coping appraisal (how the threat can be managed) are drivers of their motivation to protect their health. Therefore, as risk perception of BBV infection for a patient decreases, the willingness to order tests for that patient may decrease as well. Another reason for declining to test was that HCWs first wanted to discuss BBV test-ordering with the patient. This highlights the importance that HCWs attach to shared decision-making in their consultations with patients.[35]

HCWs’ acceptance of the test prompt system

The mean estimated additional consultation time was 3.71 min. Given the typical time of 10 min being available for a primary-care consultation, the rated high acceptance of and low resistance to the prompt system seems surprising. This is because responding to the prompt would take up 37% of their consultation time. Perhaps our HCW participants did not take into account the additional consultation time in their ratings. Alternatively, they may have considered the additional time justified in terms of the potential benefit of ordering the test. According to previous research, primary-care HCWs accept two additional minutes of consultation to use a computerized decision-support system in their patient consultations, but not 5 min.[26] Potential reasons for the long additional estimated time include a lack of familiarity with opt-out testing for BBVs, an initial lack of familiarity with the BBV test prompt and stigma associated with HIV testing. Measures to reduce the time to 2 min could include staff training in the use of opt-out HIV testing and the BBV test prompt system. In any case, it is important to establish the actual additional consultation time, if any, to ascertain whether using the prompt system is realistic within the constraints of HCWs’ patient consultations.

The finding that more experienced HCWs gave higher estimates may indicate that they are more realistic in terms of the time required to make a test order decision or that they spend more time communicating information and sharing decision-making with patients. Moreover, the consultation time and potentially also additional time also depends on HCWs’ information-sharing style when using electronic health records.[36]

A majority of HCWs (67%) preferred a single BBV test over separate tests. Furthermore, the predominant preference (90%) for prompt presentation was soft. Both these findings are consistent with the behavioural principle of aversion to complexity in order to reduce cognitive and choice overload.[37] Even though the HCWs preferred a soft prompt, there may be a case for having a hard prompt in patients with substantially higher risks identified. As an alternative to prompts, dynamically annotated visualizations (DAVs) are similar to our soft prompt in that they are continuously displayed. DAVs appear alongside the ordering form, provide real-time graphical decision support for imaging ordering and have been found to be beneficial in reducing clinically inappropriate diagnostic imaging orders.[38] The information is presented without HCWs having to direct their attention to the visualization within the display, does not require a response (thereby reducing workflow interruptions) and only gives explicit recommendations when the algorithm’s confidence is high.

The acceptance of a BBV test alert system was significantly above average and resistance was significantly below average. The finding of high acceptance in HCWs has been explained by factors such as perceived ease of use, perceived usefulness, compatibility, organizational facilitators, subjective norm and habit.[25,39] The findings regarding preferred time interval to re-instate a prompt suggest that HCWs are more likely to be flexible in terms of potentially changing their decision to order a test than patients. This may be because they consider the possibility that the patient may subsequently develop additional risk factors which make testing a higher priority.

Recommendations

The following recommendations are made from the results of the current study for the design of a subsequent real-world trial of the BBV test alert system.

In order to increase the rate of accept decisions, (a) the accept decision should be pre-selected, (b) cost-effectiveness information should be presented if the accept decision is not pre-selected, (c) the date of last BBV test should not be presented and (d) if the accept decision is not pre-selected, then risk information should be presented.

A single combined set of BBV tests should be offered.

Alerts should be presented as a soft prompt, but as a hard prompt in patients with substantially higher risks identified.

Time interval to re-instate a prompt after previous refusal by HCW or patient should be one year. Future research should analyse reasons why specific intervals are preferred and the potential usefulness of a more regular prompt for high-risk individuals.

Conclusion

We have demonstrated the effect of design factors on decision-making for BBV testing. In particular, pre-selection of decision, cost-effectiveness information and date of last BBV test were influential variables. Acceptance of using the BBV test prompt was high and resistance to its use was low. This research provides a basis for the development and real-world testing of the BBV test prompt facility within an EPR system that is the subject of our current research.

Click here for additional data file.

Supplemental material, sj-pdf-1-std-10.1177_0956462420906998 for Designing an electronic blood-borne virus risk alert to improve uptake of testing by Paul van Schaik, Susan Lorrimer and David Chadwick in International Journal of STD & AIDS