Interests and needs of eye care providers in clinical decision support for glaucoma.

OBJECTIVE: To study whether clinicians who treat glaucoma are interested in using clinical decision support (CDS) tools for glaucoma, what glaucoma clinical decisions they feel would benefit from CDS, and what characteristics of CDS design they feel would be important in glaucoma clinical practice. METHODS AND ANALYSIS: Working with the American Glaucoma Society, the Utah Ophthalmology Society and the Utah Optometric Association, we identified a group of clinicians who care for patients with glaucoma. We asked these clinicians about interest in CDS, what glaucoma clinical decisions would benefit from CDS, and what characteristics of CDS tool design would be important in glaucoma clinical practice.
RESULTS: Of the 105 clinicians (31 optometrists, 10 general ophthalmologists and 64 glaucoma specialists), 93 (88.6%) were either 'definitely' or 'probably' interested in using CDS for glaucoma. There were no statistically significant differences in interest between clinical specialties (p=0.12), years in practice (p=0.85) or numbers of patients seen daily (p=0.99). Identifying progression of glaucoma was the clinical decision the largest number of clinicians felt would benefit from CDS (104/105, 99.1%). An easy to use interface was the CDS characteristic the largest number of clinicians felt would be 'very important' (93/105, 88.6%).
CONCLUSION: Of this group of clinicians who treat glaucoma, 88.6% were interested in using CDS for glaucoma and 99.1% felt that identification of glaucomatous progression could benefit from CDS. This level of interest supports future work to develop CDS for glaucoma. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Glaucoma management is complex and could benefit from clinical decision support (CDS). CDS systems are more likely to be successful when clinicians are involved in design and development from an early stage to ensure that the tool meets their needs.

We surveyed 105 clinicians who care for patients with glaucoma to identify what glaucoma clinical decisions they feel would benefit from CDS and what characteristics of CDS tool design they feel would be important in glaucoma clinical practice. Identifying progression of glaucoma was the clinical decision the largest number of clinicians felt would benefit from CDS (104/105, 99.1%). An easy to use interface was the CDS characteristic the largest number of clinicians felt would be ‘very important’ (93/105, 88.6%).

Understanding the perspectives of clinicians who care for patients with glaucoma will guide future research to develop CDS for glaucoma that meets the needs and interests of clinicians.

Introduction

Glaucoma is a complex chronic disease. Clinicians who care for patients with glaucoma must evaluate and integrate data from many sources, from many visits, over long periods of time.1 Incorporating this clinical data with published evidence to make a decision in the midst of a busy clinic schedule is often challenging. The field of predictive modelling, including artificial intelligence, may help relieve this challenge.2–4 The results of these predictive models could be presented to clinicians caring for patients with glaucoma using clinical decision support (CDS) tools.5 CDS tools are computer programs designed to assist clinicians as they make decisions for patient care. These tools can be integrated into electronic health records (EHRs) and can provide information to help with clinical decision making. An example CDS tool is an EHR add-on app that was developed to help with neonatal bilirubin management.6 For neonatal bilirubin management without the add-on app, clinicians must retrieve data that is scattered across the medical record, synthesise the data, and apply guideline algorithms to develop patient-specific treatment plans. The CDS tool gathers the data into one display and provides guideline-based treatment recommendations.

CDS tools are more likely to be successful when clinicians are involved in design and development of the tool from an early stage to ensure that the tool meets their needs.7 8 To be able to develop glaucoma predictive models and CDS tools that meet the needs and interests of practising clinicians, we need to understand to what extent clinicians who care for patients with glaucoma are interested in CDS, what specific glaucoma clinical decisions they believe would benefit from CDS, and what CDS tool design characteristics they believe are important. Without understanding the perspective of the clinicians, it is unlikely that CDS tools will meaningfully influence clinical practice.9

The purpose of this study was to identify a group of clinicians who care for patients with glaucoma to guide future development of CDS tools for glaucoma and to ask if they are interested in using CDS tools for glaucoma, what glaucoma clinical decisions they feel would benefit from CDS and what characteristics of CDS tool design they feel would be important in glaucoma clinical practice. We ensured that the group of clinicians had appropriate representation of clinician age, gender, years in clinical practice, practice type, clinical specialty and number of patients seen per day. The results of this pilot work will guide glaucoma researchers working on predictive modelling and CDS to ensure that models and tools can be developed that meet the needs of practising glaucoma clinicians.

Methods

Study participants

We worked with the American Glaucoma Society, the Utah Ophthalmology Society and the Utah Optometric Association to identify a group of clinicians who care for patients with glaucoma. We sent an email to members of these organisations to identify clinicians who were interested in sharing their opinions about CDS for glaucoma. The email provided background information about CDS tools and informed respondents that participation was for research purposes and responses would be anonymous. The email was sent to 1452 members of the American Glaucoma Society, 146 members of the Utah Ophthalmology Society and 283 members of the Utah Optometric Association. We asked respondents to report age, gender, race/ethnicity, number of years in clinical practice, practice setting (small group private practice (≤3 providers), large group private practice (≥4 providers), multispecialty clinic/health system, academic institution or other), clinical specialty (optometrist, general ophthalmologist or glaucoma specialist ophthalmologist), number of patients seen in a typical clinic day, the proportion of these patients seen for glaucoma, and if scribes, students, residents or fellows are routinely involved in their clinic. Patients and the public were not involved in the design, conduct, reporting, or dissemination plans of our research.

Questionnaire to asses opinions of the clinicians

An online questionnaire was fielded to assess the interest and needs of the clinicians in CDS for glaucoma. The questionnaire asked respondents regarding interest in using a CDS tool for glaucoma, which glaucoma clinical decisions would benefit from a CDS tool (increasing certainty when first diagnosing a patient with glaucoma, determining the optimal time for the next follow-up visual field testing for patients with glaucoma, increasing certainty when identifying progression of glaucoma, increasing certainty when deciding whether or not to intensify glaucoma treatment, and increasing certainty when choosing the next treatment step), and what characteristics of CDS tools would be important in clinical practice (integration into clinic workflow, integration with current EHR, automatic provision of results, automatic data entry, easy to use interface, minimal time required to use tool). The glaucoma clinical decisions included in the questionnaire came from a review of the American Academy of Ophthalmology Preferred Practice Pattern for primary open angle glaucoma.1 The characteristics of CDS tools included in the questionnaire were derived from prior research about successful CDS tool implementation.10–12 There were 21 total questions. The questionnaire was pilot tested for clarity and understanding with five clinicians prior to administration to the full group of clinicians. The full questionnaire and email text are presented in online supplemental material. The online questionnaire was available to the clinicians from 5 March 2020 to 31 March 2020. No incentives were provided for participation. All clinicians who participated in the questionnaire completed it entirely.

Statistical analyses

Descriptive characteristics of the clinicians were calculated using means and measures of central tendencies. We verified that the participating clinicians had representation of a broad range of clinician ages, genders, races, years in clinical practice, practice types, clinical specialties and numbers of patients seen per day. The proportion of clinicians interested in using a CDS tool for glaucoma was calculated and stratified by clinician specialty, number of years in practice and number of patients seen in a day. Differences between the groups were compared with χ2 tests. The number of clinicians who felt each of the five glaucoma clinical decisions would ‘probably’ or ‘definitely’ benefit from CDS was calculated. The number of clinicians who felt that each of the six characteristics of CDS tools would be ‘very important’ was calculated. All statistical analyses were performed using Stata V.16 (StataCorp).

Results

Of the 105 clinicians, 31 were optometrists (29.5%), 10 were general ophthalmologists (9.5%) and 64 were glaucoma specialists (61%). Table 1 presents the characteristics of the clinicians.

Table 1

Characteristics of the study participants

Characteristic	N (%)
Total	105 (100)
Age
≤39 years	33 (31.4)
40–49 years	23 (21.9)
50–59 years	24 (22.9)
≥60 years	25 (23.8)
Gender
Female	26 (24.8)
Male	79 (75.2)
Race
Asian	11 (10.5)
Black	2 (1.9)
Hispanic	3 (2.9)
Other	4 (3.8)
White	86 (81.9)
Years in clinical practice
0–10 years	39 (37.1)
11–20 years	23 (21.9)
≥21 years	43 (41.0)
Practice type
Private practice, ≤3 providers in group	39 (37.1)
Private practice, ≥4 providers in group	30 (28.6)
Health system or multispecialty clinic	9 (8.6)
Academic centre	25 (23.8)
Other	2 (1.9)
Specialty
Optometrist	31 (29.5)
General ophthalmologist	10 (9.5)
Glaucoma specialist	64 (61.0)
Patients seen per day
≤20 patients	25 (23.8)
21–40 patients	53 (50.5)
≥41 patients	27 (25.7)
Percentage of these patients seen for glaucoma
0%–25%	38 (36.2)
26%–50%	8 (7.6)
51%–75%	23 (21.9)
76%–100%	36 (34.3)
Involvement of others in clinical encounter
Scribes	57 (54.3)
Students	23 (21.9)
Residents	23 (21.9)
Fellows	15 (14.3)

Of the 105 clinicians, 49 (46.7%) were ‘definitely’ interested in using a CDS tool for glaucoma, 44 (41.9%) were ‘probably’ interested, 11 (10.5%) were ‘probably not’ interested, and 1 (1.0%) was ‘definitely not’ interested. Table 2 shows those who were interested (either ‘definitely’ or ‘probably’) in using a CDS tool for glaucoma compared with those who were not interested (either ‘probably not’ or ‘definitely not’), stratified by clinician specialty, number of years in practice and number of patients seen in a day. There were no statistically significant differences in interest in using a CDS tool for glaucoma between the different clinical specialties (p=0.12), numbers of years in practice (p=0.85) or numbers of patients seen in a day (p=0.99).

Table 2

Interest of clinicians in using clinical decision support for glaucoma

Characteristic	Interested* N (%)	Not Interested† N (%)	P value‡
Total	93 (88.6)	12 (11.4)
Specialty			0.12
Optometrist	27 (87.1)	4 (12.9)
General ophthalmologist	7 (70.0)	3 (30.0)
Glaucoma specialist	59 (92.2)	5 (7.8)
Years in practice			0.85
0–10 years	34 (87.2)	5 (12.8)
11–20 years	20 (87.0)	3 (13.0)
≥21 years	39 (90.7)	4 (9.3)
No of patients seen per day			0.99
≤20 patients	22 (88.0)	3 (12.0)
21–40 patients	47 (88.7)	6 (11.3)
≥41 patients	24 (88.9)	3 (11.1)

*Clinicians who reported ‘definitely’ or ‘probably’ interested in using a clinical decision support tool for glaucoma.

†Clinicians who reported ‘probably not’ or ‘definitely not’ interested in using a clinical decision support tool for glaucoma.

‡P value from χ2 test.

Table 3 presents the proportion of clinicians who felt that each glaucoma clinical decision would benefit from a CDS tool. Identifying progression of glaucoma was the clinical decision that the largest number of clinicians felt would benefit from a CDS tool (104/105, 99.1%). Table 4 presents the proportion of clinicians who felt that each of the CDS tool characteristics would be important in clinical practice. Having an easy to use interface and minimal time required to use the tool were the CDS tool characteristics that the largest number of clinicians felt would be ‘very important’ (93/105 [88.6%] and 92/105 (87.6%), respectively).

Table 3

Proportion of clinicians who felt that each glaucoma clinical decision would benefit from a clinical decision support tool

Glaucoma clinical decision	Proportion of clinicians who felt the decision would benefit from a clinical decision support tool
	‘Definitely’N (%)	‘Probably’N (%)	CombinedN (%)
First diagnosing a patient with glaucoma	54/105 (51.4)	37/105 (35.2)	91/105 (86.7)
Determining time for next follow-up visual field testing	32/105 (30.5)	58/105 (55.2)	90/105 (85.7)
Identifying progression of glaucoma	83/105 (79.1)	21/105 (20.0)	104/105 (99.1)
Deciding whether or not to intensify glaucoma treatment	70/105 (66.7)	33/105 (31.4)	103/105 (98.1)
Choosing the next treatment step (additional medication, laser, etc)	49/105 (46.7)	45/105 (42.9)	94/105 (89.5)

Table 4

Proportion of clinicians who felt that each of the clinical decision support tool characteristics would be important in clinical practice

Clinical decision support tool characteristic	Proportion of clinicians who felt the characteristic would be ‘very important’ N (%)
Integration in clinical workflow	82/105 (78.1)
Integration with electronic health record	76/105 (72.4)
Automatic provision of results	52/105 (49.5)
Automatic data entry	82/105 (78.1)
Easy to use interface	93/105 (88.6)
Minimal time required to use tool	92/105 (87.6)

Discussion

Nearly 90% of the clinicians in our study who care for patients with glaucoma were either ‘definitely’ or ‘probably’ interested in using a CDS tool for glaucoma. The proportion of clinicians interested in using CDS tools for glaucoma did not vary based on clinical specialty, years in practice, or number of patients seen in a day. For each of the five glaucoma clinical decisions that we asked about, more than 85% of clinicians felt that the decision would benefit from having a CDS tool and 99.1% felt that identification of glaucomatous progression could benefit from CDS. Nearly 90% of clinicians felt that it would be ‘very important’ for the tools to have an easy to use interface and require minimal time to use. The broad interest clinicians in our study showed in CDS tools supports the need for ongoing work in predictive modelling and CDS tool development. Given this interest, it is more likely that these tools will be used clinically. To maximise the chances of successful implementation, these models and tools should be developed with the needs and interests of clinicians in mind.

Though CDS tools often improve care, roughly one-third of published randomised controlled trials evaluating CDS tools showed no effect.11 13 14 A considerable amount of research has been done to understand why some CDS tools are effective and other are not.10 11 One of the important first steps is that providers are interested in and see the need for CDS.7 If the providers do not desire CDS, they are unlikely to use it.9 15 Nearly 90% of clinicians in our study were interested in using a CDS tool for glaucoma. Even when clinicians are initially interested in using a CDS tool, tools are more likely to be successful when clinicians are involved in design and development of the tool from an early stage to ensure that the tool meets their needs.7 8

For each of the five glaucoma clinical decisions that we asked about, more than 85% of clinicians felt that the decision would benefit from having a CDS tool. Clinicians were particularly interested in CDS for identifying progression of glaucoma (104/105 (99.1%) felt that this decision would benefit from CDS) and deciding whether or not to intensify glaucoma treatment (103/105 (98.1%) felt that this decision would benefit from CDS). This is not surprising as identification of glaucomatous progression and the subsequent decision to increase treatment are some of the most difficult decisions in glaucoma management, with only moderate interobserver agreement.16 17

We asked the clinicians regarding their perceived importance of six CDS tool characteristics: integration into clinic workflow, integration with current EHR, automatic provision of results, automatic data entry, easy to use interface and minimal time required to use tool. These six characteristics were chosen because they have consistently been found to be important in the success of CDS tools outside of ophthalmology.10 11 In our study, the two characteristics that the most clinicians felt would be ‘very important’ were an easy to use interface (93/105 (88.6%)) and minimal time required to use the tool (92/105 (87.6%)). Clinicians caring for patients with glaucoma may have prioritised these two characteristics because eye clinics are often busy and EHR efficiency influences clinic volume.18

Future work to develop CDS tools for glaucoma should involve clinicians early in the design process and build off of the findings in this study. Systematic study of glaucoma clinical workflow using cognitive task interviews and ethnographic observations of clinical encounters could help identify the functional requirements important for successful glaucoma CDS implementation.5 Developing any future glaucoma CDS tools using user-centred iterative design principles will increase the likelihood that these tools will be successful.5 In our survey, 90% of clinicians were interested in using a CDS tool for glaucoma and 10% were not. Future qualitative research with clinicians who are not interested in using a CDS tool for glaucoma could help us understand why this disinterest and what potential barriers they foresee for implementation of CDS for glaucoma.

Our study has limitations. The clinicians in our study came from a convenience sample and therefore their opinions may not represent the opinions of the entire eye care community. For example, clinicians who elected to participate in this online survey may be more likely to be interested in CDS for glaucoma. However, the clinicians had adequate representation of various clinician characteristics (clinician age, gender, years in clinical practice, practice type, clinical specialty and number of patients seen per day). Our results should not be interpreted as representative of the entire community of clinicians who care for patients with glaucoma, but instead should be viewed as pilot work to identify the interests and needs of clinicians for CDS for glaucoma. Further work is needed to see if the opinions of our clinicians represent the opinions of the broader eye care community. As another limitation, the questionnaire was hypothetical in nature. It asked clinicians if they would be interested in using a CDS tool and which CDS tool characteristics would be important without actually having the clinicians use one of these tools. However, our findings are important preliminary results. If the clinicians in our study were not interested in CDS tools at this early stage, any CDS intervention would be unlikely to succeed. Additionally, if CDS tools developed in the future don’t have the characteristics noted by clinicians to be important in this hypothetical questionnaire, it is unlikely that the clinicians would engage meaningfully in trying the tool. For example, if a CDS tool for glaucoma is developed with an interface that is difficult to use, our results suggest that glaucoma providers would be sceptical of using the tool from the outset.

In conclusion, we found that nearly 90% of the clinicians in our study were open to the idea of using CDS tools for glaucoma care and 99.1% felt that identification of glaucomatous progression could benefit from CDS. This level of interest in this pilot study supports the need for ongoing work in the fields of predictive modelling and CDS for glaucoma. Our findings also guide future CDS tool development. Close to 90% of clinicians felt that it would be ‘very important’ for the tools to have an easy to use interface and require minimal time to use. It is important that the perspectives and needs of clinicians who care for patients with glaucoma are considered as predictive models and CDS tools are developed to make successful implementation more likely.