Differential Diagnosis Decision Support Systems in Primary and Out-of-Hours Care: A Qualitative Analysis of the Needs of Key Stakeholders in Scotland.

BACKGROUND: Differential Diagnosis Decision Support Systems (DDDSS) and other forms of clinical decision support systems may have a role in supporting clinicians and the public in making decisions about differential diagnosis and accessing health care services. The aim of this qualitative, focus group study was to explore the needs of out-of-hours and primary care clinicians, and members of the public in relation to DDDSS.
METHODS: Twenty-nine participants (comprising 13 advanced nurse practitioners [ANPs], 7 general practitioners [GPs], 2 allied health professional advanced practitioners [AAPs], and 7 members of the public) took part in 1 of 6 focus groups. Data were audio-recorded, transcribed, and analyzed thematically.
RESULTS: Four themes were identified: (1) Current Practice, (2) Attitudes to DDDSS, (3) Implementation Considerations, and (4) Desirable Characteristics of DDDSS. DISCUSSION: There may be a role for differential diagnosis decision support systems in supporting clinicians and members of the public; however, it is important that the needs of these groups are considered when designing or implementing any clinical decision support system in primary or out-of-hours care. More research is needed into how these systems could be used within clinical practice.

Background

There is an increasing strain on primary care services as Scotland’s older population continues to grow,[1] and general practitioner (GP) numbers decline.[2] Efforts are being made by the Scottish Government and Scottish medical schools to address the shortfall by training and recruiting GPs.[3-5] However, this alone is not sufficient to address the growing clinical need. The new General Medical Services contract[6] is refocusing the role of the GP in Scotland, as an “expert medical generalist,” who will lead a team of health care professionals that will include nursing and allied health care professions in advanced practice roles (ANPs and APPs). These advanced practitioners will see a sizable proportion of cases that would traditionally have been seen by a GP, allowing GPs more time to manage the more medically complex patients.

Technology may have a role in supporting clinicians and the public in making health care decisions during this time of change. This article details part of a project that aimed to explore the needs of out-of-hours (OoH) and primary care clinicians, and members of the public, in relation to differential diagnosis.

The term Differential Diagnosis Decision Support System (DDDSS) was coined to refer specifically to the type of technology being discussed as part of the project. DDDSS was identified as a system that would allow an individual to enter clinical findings, history, and demographics, and receive a ranked differential diagnosis (DDx) based on these data. It was also important that these systems linked to some form of evidence base or educational resource for the clinicians or the public. An unpublished literature review and market-research activity, which took place alongside this project, found that there were only a small number of systems that met this description[7] and were commercially available,[8] and that some of the newer systems only had a small amount of published research regarding them.[9] While the systems did appear to have the potential to improve diagnostic practices in clinicians,[7] it was not entirely clear that they were being used effectively when trialed in clinical practice.[9,10] There was also a concerning lack of evidence surrounding patient-facing symptom checkers.[11]

Methods

Six focus groups were held between April and June 2018 with the 3 key stakeholder groups: General Practitioners, Advanced Practitioners (both trainees and qualified), and members of the public. Ethical approval was granted by the University of Glasgow’s College of Medical, Veterinary and Life Sciences Ethics Committee.

Recruitment

Convenience and snowball sampling were employed to recruit clinicians from a clinical decision support study day, an advanced practice teaching event and a GP cluster meeting. Members of the public were reached by email through a variety of health care support and intermediary groups, and by hand-distribution of flyers in a number of libraries in the central belt of Scotland. Both clinicians and the public were encouraged to cascade information to other interested parties.

Data Collection

Prior to the focus group, participants were sent a link to a symptom checker to give them an idea of how the systems worked (https://symptomchecker.Isabelhealthcare.com/). The Isabel Symptom Checker was selected as it is free to access and is very similar to the clinician-facing tool, the Isabel Differential Diagnosis Generator. Participants were shown a brief presentation about DDDSS before the start of each group, so all those present were aware of the different types of system and their key characteristics. A focus group topic guide was used, which included broad questions intended to guide the conversation without restricting discussion and interaction between participants (Table 1).

Table 1.

Example Focus Group Questions for Health Care Professionals.

Focus Group Questions

1. What are your initial thoughts about the use of differential diagnosis decision support systems? What are the “pros and cons?”2. How do you think members of the public would feel about the use of differential diagnosis decision support systems?3. What tools do you use at the moment to assist you in making differential diagnoses?4. What would encourage you to use a differential diagnosis decision support system?5. Considering any applications or clinical decision support tools you use at the moment, are there any barriers that prevent you from using these as often as you would like?6. Do you think there is a need for differential diagnosis decision support systems in primary and out-of-hours care?

Data Analysis

Groups were audio-recorded and transcribed verbatim by a transcribing service. Transcripts were analyzed manually; however, NVivo 11 qualitative data analysis software[12] was used to manage the data. A thematic analysis of the data was undertaken as outlined by Braun and Clarke.[13] Because of time constraints, only 1 researcher (CRP) conducted the analysis; however, emergent themes were discussed and agreed upon with the other members of the team (MAC, BJ), who were also at the focus groups.

Results

Six focus groups were conducted (2 with clinicians and 2 with the public), and 29 participants were recruited (22 clinicians and 7 members of the public). There were 13 ANPs, 7 GPs, and 2 AAPs. Four themes were identified in the data, with associated subthemes (Table 2).

Table 2.

Themes.

Theme 1: Current Practice
Clinician’s current practice:	I’m just gonna double check to see what the utmost up to date guidelines are. Which is quite a different feeling from, basically, putting in the symptoms into a search engine [Julie, GP, HCP Group 4]
Public current practice:	If we aren’t relying on software to support us, then the patients are certainly using software out there to support them [Lindsay, Trainee ANP, HCP Group 3]
Theme 2: Attitudes to DDDSS
Worried about DDDSS:	Inevitably it will come up with a diagnosis of cancer. When the patient walks through the door . . . quite clearly, quickly, we can see this is not a serious diagnosis [Linda, GP, HCP Group 4]
Optimistic about DDDSS:	This seems to be a slightly slicker way of dealing with it . . . I’m not having to write things down. I’m doing it as it’s there on a screen [Lindsay, Trainee ANP, HCP Group 3]
The clinician is the key to the system’s success:	Patients don’t follow flow charts, in my experience [Lydia, GP, HCP Group 4]
Theme 3: Implementation Considerations
Introduction of DDDSS:	It’s maybe another skill set in terms of just, you know, getting used to using the apps, isn’t it. Ask everything we would ask normally. Transfer it on . . . You just need to get really slick at it [Amanda, Trainee ANP, HCP Group 1]
Technology as a barrier to good clinical practice:	I went to a consultant once who did that, and he never looked at me the whole time, until I was going. Which doesn’t make you feel good [Mavis, Public Group 2]
Usage patterns of DDDSS:	Even if you don’t have time in your consultation to look . . . you’ve got the evidence then to be able to filter it through a bit further, without having to do a great big huge search for the documentation. So I think that would be a great help [Kim, ANP, HCP Group 2]
Theme 4: Desirable Characteristics of DDDSS
Provide access to the evidence base:	Perhaps if I had had access at the time to evidence or support that was there, that would have stopped that need for that patient to come back [Lindsay, Trainee ANP, HCP Group 3]
Enables patient involvement:	If I went to the doctor and he was using that, I would rather he told me he was using that . . ., I would rather he turned the computer to me, and we both did it together [Charlie, Public Group 2]
Promotes self-care:	I think anything that makes somebody think twice about using a service, is a good idea [Jordan, Public Group 2]
Workflow integration:	It can’t sit as a separate . . . it just becomes another guideline, another bit of . . . tabs at the bottom of your computer. It needs to be embedded in your every day for everybody. [Amanda, Trainee ANP, HCP Group 1]

Abbreviations: ANP, advanced nurse practitioner; DDDSS, Differential Diagnosis Decision Support Systems; GP, general practitioner; HCP, health care professional.

Theme 1: Current Practice

Clinicians used their own mobile devices for decision support during consultations, and the applications they were using were passive, informational resources. Some felt more comfortable using these resources to confirm their diagnosis, and likened DDDSS to a simple search engine. They were unhappy about using their own devices for applications, and worried that using their devices in front of patients might be viewed negatively. Patients often researched their symptoms online prior to consultations and arrived with a fixed idea of what was causing them. Clinicians felt that the information accessed by the public was not always accurate, and that use of a trusted, validated symptom checker would be an improvement. Clinicians felt that if the public were already using electronic resources to support them, then they should be too.

Theme 2: Attitudes to DDDSS

Clinicians and the public saw the potential for DDDSS to cause anxiety. Members of the public expressed concerns about seeing a long list of differentials and assuming the worst. Some clinicians had already seen this behavior demonstrated in practice. The potential for a lengthy DDx to increase clinician’s diagnostic uncertainty was raised. Some also felt that overriding a DDDSS would make them vulnerable to litigation. GPs were worried about the medicolegal implications of their ANPs potentially reaching broader differentials than them, if the ANPs were using a DDDSS.

In contrast, some were enthusiastic about the potential introduction of DDDSS, particularly trainee ANPs. This group felt that DDDSS could increase their confidence, therefore reducing the supervisory burden placed on their GP supervisor. Some ANPs had also been exposed to a system called Odyssey, which can help clinicians take a thorough history, and allows them to capture data quickly through the use of mouse-clicks and drop-down boxes. Systems that offered benefits such as these were highlighted as a positive, even if they did not offer a dynamic DDx in response to the symptoms entered.

The importance of a skilled clinician to use, and where appropriate, overrule a DDDSS was discussed frequently. “Gut-instinct,” and the ability to deviate from a pathway were seen as important. “Pathways,” “flowcharts,” and “algorithms” were discussed with negative connotations. Some GPs felt that DDDSS might undermine the role of a skilled clinician and were concerned that the next generation of GPs might struggle to develop the same decision-making skills that they had, if they became too reliant on DDDSS.

Theme 3: Implementation Considerations

Education was seen as essential to the successful introduction of DDDSS. Clinicians viewed it as another “tool” that would need to practice using. The public gave examples from their experience of self-management applications that had not been used effectively due to poor promotion and education surrounding them. However, there was a sense that the use of DDDSS was inevitable, and that clinicians and the public would have to adapt to this change.

Technology was seen to have an impact on communication during consultations, and patients discussed times when they had experienced this. Some clinicians echoed this sentiment; however, others felt that technology was already a key part of the consultation, and that it was the responsibility of the clinician to use it appropriately. Interoperability between systems was also raised as a barrier, as clinicians struggled to access notes when traveling outside their area or were having to enter information twice into different systems. Connectivity was also raised as a problem, as some clinicians struggled to update notes on mobile devices during home visits, due to poor signal.

Time was identified as the main issue, which would affect clinician’s DDDSS use. As a result, many said they would use a DDDSS between, not during, consultations. It was identified as common practice for clinicians to use the short gaps between patients to consult the evidence base. Some suggested that DDDSS could be well deployed for this purpose.

Theme 4: Desirable Characteristics of DDDSS

Providing access to the evidence base in a way that was integrated into their workflow was viewed positively by clinicians, however, they would expect the evidence to be from a known source (such as NICE, National Institute for Health and Care Excellence)—and not proprietary to the company that produced the DDDSS. Clinicians gave examples of using DDDSS with patients, to discuss antibiotic prescribing, for example. Members of the public were happy for clinicians to use DDDSS but wanted to be involved. Some systems allow the patient to complete a preassessment questionnaire, which is shared with the clinician. The public were particularly interested in the possibility of them communicating information directly into their electronic health records, potentially when they felt an appointment was not necessary. While some GPs had reservations about preregistration of information creating a bias that would cloud their decision making, the general attitude regarding it was positive.

Clinicians were interested in the potential for DDDSS to reduce their workload by promoting self-care, or redirecting patients to a more appropriate service. Clinicians described appointments where the patient only required reassurance or self-care instructions. Both clinicians and the public were positive about the use of technology to avoid such unnecessary appointments.

Clinicians felt DDDSS had to be integrated into their current systems and workflow in order for them to use it. Some integrated DDDSS created clinical notes, which were then saved in the patient’s record. This was seen as particularly attractive to clinicians, who saw it as a way of saving time as well as improving documentation. Members of the public were interested in systems that generated handouts, with one participant suggesting it could even prevent future appointments if they had access to the handout from a previous appointment.

Discussion

Both clinicians and the public make use of technology to inform their decision making, although the resources used are often more passive than what this study would define as a DDDSS. Given this, the lack of evaluation of online symptom checkers[11] and of DDDSS in general is a concern. There is also a valid concern in that DDDSS could increase anxiety for clinicians, with junior clinicians being particularly susceptible.[14,15] However, trainee ANPs were optimistic about the introduction of DDDSS in the future, particularly if systems could improve history taking and documentation, integrate with their workflow, and provide them with point-of-care access to a trusted evidence base. Given that time was frequently highlighted as a barrier to DDDSS use, perhaps a system that offers time-saving features such as these may be adopted more readily by clinicians. The readiness of the trainee ANPs to adopt systems such as this does raise the question of whether these systems are best introduced to clinicians while training. Some DDDSS have been shown to improve the diagnostic accuracy of medical students.[14-16] Potentially, this may also apply to trainee ANPs.

However, GPs were concerned about the deskilling of future doctors through the use of DDDSS. It is important to note that the accuracy of the DDx generated by a DDDSS has been shown to be improved when the symptoms entered are selected by importance, as opposed to simply entering every piece of information.[17] This evident need for a skilled clinician to operate a system was recognized by both junior and experienced clinicians in the focus groups. Some raised concerns about the potential for DDDSS to affect communication, yet this is another challenge that a skilled clinician would be expected to be able to manage. As some clinicians stated, DDDSS are just another tool, which will require patience and practice to benefit from them.

Education is essential to the successful introduction of a DDDSS, not only for clinicians but also for the public. While clinicians must be confident in their ability to use such a system to support their decision making, members of the public should also feel empowered to make decisions about their health and when to access services through the use of such technologies. Rather than presenting a barrier to communication, DDDSS can be used to promote self-care where appropriate, and facilitate shared decision making between clinicians and the public. Technology can be an effective way of enabling patients to communicate changes or key outcomes with clinician[18,19]; however, it is important that the solution meets the needs of both parties. It is apparent that for a DDDSS to work effectively in the OoH and primary care settings, the needs of these key stakeholders must be taken into consideration when designing and implementing any such system. Further research into the use of DDDSS in primary and OoH care should be conducted to establish enablers and barriers to its use. The research that exists on DDDSS often comes from an experimental setting.[7] More projects exploring their use in clinical practice are needed.

Strengths and Limitations

The key strength of this study is that it examined the attitudes of a variety of stakeholders in relation to DDDSS in primary and OoH care, including GPs, ANPs, AAPs, and members of the public. The main limitations of this study were because of its short timescale. Focus groups were only conducted in one area in the central belt of Scotland, meaning that only individuals able to travel to this particular area participated. The team tried to mitigate this by holding a web conference but received no responses. The short timescale also only allowed time for one researcher to undertake data analysis.

There have not been many studies examining the use of DDDSS in primary or OoH care. The rapid literature review that was undertaken as part of the larger project only found one study which examined this type of technology in this setting.[10] This 2013 pilot study by Henderson and Rubin[10] looked at the utility of the Isabel DDx Generator in general practice and found that the system was not likely to be used in its current format in primary care. Even when systems were well adopted in clinical practice, such as in the study by Barbieri et al[9] of VisualDX’s uptake in an acute setting, it is not clear how the system affects the clinician and patient who are involved in its use. This study is therefore unique and can inform future work on decision making diagnostic software. Our study has also examined the broad concept of DDDSS, and not one individual system. Finally, it has also explored the attitudes of the key stakeholders who would be affected by its adoption in this setting.