Finding the best glaucoma questionnaire: a qualitative and quantitative evaluation of glaucoma knowledge assessments.

OBJECTIVE: The goal of the study was to determine which glaucoma quiz provides the best information about patient glaucoma knowledge to clinicians in clinical practice settings.
METHODS: Four glaucoma quizzes were identified from the literature and national eye education programs and were qualitatively analyzed to categorize questions by topic. Quizzes were assessed by 64 glaucoma specialists using an online survey, with descriptive statistics. Comments about the quizzes were analyzed qualitatively using themes and representative quotations.
RESULTS: Quizzes covered content that was important for glaucoma diagnosis and management. The National Eye Health Education Program (NEHEP) and Prevent Blindness America quizzes covered primarily diagnosis, screening, and risk factors, and the quizzes from the literature primarily covered causes of glaucoma, vision loss, eye drops, and systemic disease. Overall, the NEHEP quiz was ranked best for clinical practice, ranked first by 38%. Ranked second overall, the Gray quiz was ranked first by 34% and last by 34%.
CONCLUSION: The NEHEP quiz was rated most useful for assessing baseline general glaucoma knowledge for a busy clinical practice. The Gray quiz appears to be more useful as part of a comprehensive education program, perhaps in combination with an ophthalmic educator.

Introduction

Education is a key for both the diagnosis and management of glaucoma, as patients need to understand that they have the disease to participate in their treatment. Public awareness campaigns can educate people about the importance of early diagnosis and early treatment to potentially prevent blindness from glaucoma.1 Once glaucoma is diagnosed, it is commonly treated with daily eye drop medications, often for decades in asymptomatic patients.2 Patient education is critical for adherence to glaucoma treatment plans,3 but for many patients there are significant gaps in the understanding of the disease, particularly with regard to medication side effects, symptoms, and progression to blindness.4 The belief that the disease is “not serious enough” is associated with poor adherence to follow-up care.5 However, assessing glaucoma knowledge and providing the most appropriate education in the limited time available in a clinical encounter is difficult.6

Educational interventions do appear to improve eye drop medication adherence, but assessing glaucoma knowledge is not routine.7 Multiple glaucoma quizzes have been developed for glaucoma patients, to assess glaucoma knowledge, both in the setting of research instruments and for community use. No study has compared these quizzes to see which one of these serves best to gather “glaucoma knowledge” information from glaucoma patients in clinical practice.

The purpose of this study was to qualitatively compare four candidate quizzes and to assess which would be most useful to the clinician in a clinical practice setting taking care of glaucoma patients. The overarching goal of this research program is to improve patient education materials, and this study is the first step to determine which knowledge quizzes do practicing clinicians prefer. Subsequently, the quizzes that are preferred by clinicians will be tested in glaucoma patients, to get patient feedback, and for validation in glaucoma patients. Ultimately, a quiz that has been vetted by doctors and patients will be used to assess the effect of educational interventions on glaucoma knowledge. The first step in this process is presented here, addressing the question of which quiz providers like the best.

Methods

Identification and qualitative evaluation of glaucoma knowledge quizzes

Glaucoma knowledge assessments used in research studies were identified from the glaucoma literature and were chosen for further investigation based on the following criteria: development using both patient and expert opinion and feedback, use in glaucoma patient populations, and having general glaucoma knowledge content. The glaucoma knowledge assessment from Gray et al8 is a short-answer format assessment and contains eleven questions. The glaucoma knowledge assessment from Hoevenaars et al9 is a true–false format assessment, and 19 statements about glaucoma were evaluated in this study. Glaucoma quizzes developed by the National Eye Health Education Program ([NEHEP], ten true–false questions) and Prevent Blindness America ([PBA], nine true–false questions), which are available online and are used for public health awareness, were included in the review. All questions from each quiz were categorized based on the topic addressed by the question, to compare the content of each quiz.

Survey of glaucoma specialists

A survey of glaucoma specialists was performed to determine which of the four glaucoma quizzes would be most useful in clinical practice. The study protocol was reviewed by the Duke Institutional Review Board and determined to be exempt. An anonymous online survey (Supplementary materials) using Qualtrics (Qualtrics, Provo, UT, USA) was sent to the membership of the American Glaucoma Society (AGS), after review by and permission from the AGS Research Committee, using the AGS listserve. The survey included the four glaucoma quizzes (presented in random order), as well as questions to rate and rank the quizzes regarding their utility in clinical practice. Five-item scales were used to rate each quiz based on the following: how well the quiz covered information needed by glaucoma patients, how easy it was to understand the language of the quiz, and how willing glaucoma patients would be to complete the quiz. After reviewing and rating each quiz, the respondent ranked the four quizzes. Demographic information was also collected such as age, sex, percent of practice caring for glaucoma, and length of time since glaucoma fellowship training. The survey was active for 2 weeks in August 2014, taken offline after tapering of the response rate, with a total of 84 responses. A total of 64 respondents completed the question ranking all four quizzes. The actual sample size of 64 gives a confidence interval of 12, at the 95% confidence level.

Data analysis of survey responses

Descriptive statistics were used for the survey data. Responses are presented as percentages and means with standard deviations. The provider comments from the survey were analyzed qualitatively, grouping by themes (using word) with representative quotations.

Results

Qualitative assessment of the quizzes and questions

The glaucoma quizzes covered content that is important for the diagnosis and management of glaucoma, with some differences between the quizzes (Table 1). The Eye Q Test from PBA has questions covering: causes and types of glaucoma, diagnosis and screening, clinical course, risk factors, and symptoms; however, eye pressure and treatment, as well as vision loss, are not covered. The Eye Q Test from NEHEP has questions covering the following: eye pressure, diagnosis and screening, vision loss, clinical course, risk factors, and symptoms, but causes and types of glaucoma and treatments are not covered. The assessment from Gray et al has questions covering the following: causes and types of glaucoma, vision loss, clinical course, treatment, and family history as a risk factor, but eye pressure, diagnosis and screening, and symptoms are not covered. Also, there is only one risk factor question in this quiz, compared to three to four risk factor questions in each of the other questionnaires. The assessment from Hoevenaars et al has questions covering the following: causes and types of glaucoma, eye pressure, vision loss, clinical course, and risk factors, but diagnosis and screening and treatment are not covered. This is the only quiz that includes questions about systemic disease, other medications, and diet, and it has about twice as many questions as the other quizzes. Of note, there are an additional 18 questions regarding treatment developed by Hoevenaars et al but they were not included in this study due to the length of the entire assessment.

Table 1

Categorization of four glaucoma quiz questions, by topic

Quiz question	Format
Causes and types of glaucoma
PBA #3	T/F
Gray #1	SA
Gray #2	SA
Hoevenaars #4	T/F
Hoevenaars #7	T/F
Eye pressure
NEHEP #7	T/F
NEHEP #9	T/F
Hoevenaars #5	T/F
Hoevenaars #12	T/F
Hoevenaars #13	T/F
Hoevenaars #18	T/F
Diagnosis and screening
PBA #7	T/F
PBA #9	T/F
NEHEP #10	T/F
Vision loss
NEHEP #8	T/F
Gray #3	SA
Gray #4	SA
Hoevenaars #1	T/F
Hoevenaars #3	T/F
Hoevenaars #6	T/F
Hoevenaars #15	T/F
Clinical course
PBA #8	T/F
NEHEP #6	T/F
Gray #11	SA
Hoevenaars #9	T/F
Testing
Gray #6	SA
Treatment: eye drops
Gray #7	SA
Gray #8	SA
Gray #9	SA
Gray #10	SA
Risk factor: age
PBA #5	T/F
NEHEP #4	T/F
Hoevenaars #2	T/F
Risk factor: black race
PBA #6	T/F
NEHEP #1	T/F
Hoevenaars #16	T/F
Risk factor: prevalence
PBA #4	T/F
Risk factor: family history
PBA #2	T/F
NEHEP #2	T/F
Gray #5	SA
Hoevenaars #11	T/F
Risk factor: refractive error
Hoevenaars #17	T/F
Symptoms
PBA #1	T/F
NEHEP #3	T/F
NEHEP #5	T/F
Hoevenaars #8	T/F
Systemic disease, medications, diet
Hoevenaars #10	T/F
Hoevenaars #14	T/F
Hoevenaars #19	T/F

Notes: Quizzes covered topics important for the diagnosis and management of glaucoma, with some differences. Gray, refers to quiz from reference 8; Hoevenaars, refers to quiz from reference 9; PBA, refers to the Prevent Blindness America Eye Q Test from reference 10 and; NEHEP refers to the National Eye Health Education Program Eye Q Test from reference 11.

Abbreviations: T/F, true/false; SA, short-answer.

Quantitative assessment by glaucoma specialists

The glaucoma specialists who responded to the survey (~ 9% of the AGS listserve) reflected a broad range of age and experience caring for glaucoma patients, with 41% with >20 years since completing their training (Table 2). Most of the respondents spent more than half of their time caring for glaucoma, and over half of the specialists have a practice that is >90% glaucoma. Most of the specialists were not familiar with the quizzes in the survey prior to this study (Table S1).

Table 2

Respondent characteristics

	Number (%)
Age (years)
Younger than 30 years	1 (2%)
30–39	11 (18%)
40–49	12 (20%)
50–59	16 (26%)
60–69	13 (21%)
70 or greater	6 (10%)
Prefer not to answer	2 (3%)
Sex
Male	35 (57%)
Female	23 (38%)
Prefer not to answer	3 (5%)
Percentage of clinical practice glaucoma
<10%	0
10%–40%	2 (3%)
40%–60%	4 (7%)
60%–90%	21 (34%)
>90%	32 (52%)
Prefer not to answer	2 (3%)
Number of years since completion of glaucoma fellowship
<1 year	6 (10%)
1–5 years	5 (8%)
6–9 years	7 (11%)
10–15 years	11 (18%)
16–20 years	5 (8%)
>20 years	25 (41%)
Prefer not to answer	2 (3%)

Notes: Online surveys were completed by 64 glaucoma specialists and members of the American Glaucoma Society. Sixty one provided demographic information. Most respondents, aged 40–70 years old, were male and had a clinical practice of at least 60% glaucoma.

Glaucoma specialists reviewed each of the quizzes and rated each with respect to content, language, and willingness of patients to complete the quiz (Table 3). They rated the quizzes from Gray et al and NEHEP highest for how well the quiz covered everything patients would need to know to adequately care for their glaucoma. The NEHEP quiz was also rated highest for how easy it would be for most glaucoma patients to understand the language. The NEHEP and PBA quizzes were rated highest for how willing most glaucoma patients would be to complete them.

Table 3

Respondent ratings of glaucoma knowledge quizzes

Glaucoma knowledge test
	How well does this test cover everything patients would need to know to adequately care for their glaucoma?
	Not well at all	Not too well	Somewhat well	Very well	Extremely well	Mean (SD)
NEHEP Eye Q Test (n=61)	5 (8%)	11 (18%)	21 (34%)	21 (34%)	3 (5%)	3.10 (1.03)
Gray glaucoma knowledge assessment (n=63)	2 (3%)	9 (14%)	18 (29%)	24 (38%)	10 (16%)	3.49 (1.03)
PBA Eye Q Test (n=66)	5 (8%)	24 (36%)	26 (39%)	8 (12%)	3 (5%)	2.70 (0.94)
Hoevenaars glaucoma knowledge assessment (n=64)	4 (6%)	15 (23%)	29 (45%)	13 (20%)	3 (5%)	2.94 (0.94)
	How easy would it be for most glaucoma patients to understand the language in this test?
	Not easy at all	Not too easy	Somewhat easy	Very easy	Extremely easy	Mean (SD)
NEHEP Eye Q Test (n=59)	2 (3%)	3 (5%)	14 (24%)	33 (56%)	7 (12%)	3.68 (0.88)
Gray glaucoma knowledge assessment (n=63)	4 (6%)	14 (22%)	18 (29%)	20 (32%)	7 (11%)	3.19 (1.11)
PBA Eye Q Test (n=66)	4 (6%)	9 (14%)	26 (39%)	24 (36%)	3 (5%)	3.20 (0.95)
Hoevenaars glaucoma knowledge assessment (n=64)	7 (11%)	28 (44%)	18 (28%)	8 (13%)	3 (5%)	2.56 (1.01)
	How willing would most glaucoma patients be to complete this test?
	Not willing at all	Not too willing	Somewhat willing	Very willing	Extremely willing	Mean (SD)
NEHEP Eye Q Test (n=60)	1 (2%)	3 (5%)	20 (33%)	32 (53%)	4 (7%)	3.58 (0.77)
Gray glaucoma knowledge assessment (n=61)	4 (7%)	14 (23%)	21 (34%)	17 (28%)	5 (8%)	3.08 (1.05)
PBA Eye Q Test (n=65)	0	6 (9%)	26 (40%)	27 (42%)	6 (9%)	3.51 (0.79)
Hoevenaars glaucoma knowledge assessment (n=63)	1 (2%)	17 (27%)	27 (43%)	17 (27%)	1 (2%)	3.00 (0.82)

Notes: The respondents rated each glaucoma knowledge quiz with regard to content, language, and their perception of how willing patients would be to complete the quiz. Gray, refers to quiz from reference 8; Hoevenaars, refers to quiz from reference 9; PBA, refers to the Prevent Blindness America Eye Q Test from reference 10 and; NEHEP refers to the National Eye Health Education Program Eye Q Test from reference 11. The Gray quiz had the highest mean score on how well the quiz covered information needed by glaucoma patients. Response totals with percentages are given, as well as the mean and standard deviation. The NEHEP quiz had the highest mean scores for clarity of language and for willingness of patients to complete the quiz.

Abbreviation: SD, standard deviation.

Overall, the NEHEP quiz was ranked #1 by 38% of respondents, and the Gray et al assessment was ranked #1 by 34% (Table 4). Interestingly, the Gray et al assessment was ranked #4 or last by an equal number of respondents, 34%, suggesting a broad range of opinion regarding this particular test.

Table 4

Overall ranking

Glaucoma knowledge test	Mean rank (SD) (1 is best)	Number ranked as #1 (best) (%)	Number ranked as #4 (last) (%)
NEHEP Eye Q Test	1.94 (0.89)	24 (37.5%)	3 (4.7%)
Gray glaucoma knowledge assessment	2.55 (1.28)	22 (34.4%)	22 (34.4%)
PBA Eye Q Test	2.70 (0.94)	7 (10.9%)	14 (21.9%)
Hoevenaars glaucoma knowledge assessment	2.81 (1.14)	11 (17.2%)	25 (39.1%)

Notes: Gray, refers to quiz from reference 8; Hoevenaars, refers to quiz from reference 9; PBA, refers to the Prevent Blindness America Eye Q Test from reference 10 and; NEHEP refers to the National Eye Health Education Program Eye Q Test from reference 11. The respondents ranked the four quizzes based on how well the quiz would assess glaucoma knowledge in clinical practice. Overall, the NEHEP quiz was ranked best. Interestingly, the Gray quiz was ranked first and last by approximately a third of the respondents, suggesting a dichotomous reaction to this quiz.

Abbreviation: SD, standard deviations.

Qualitative comments from glaucoma specialists

The respondents noted that both the true–false format of the NEHEP and PBA Eye Q Tests was fun, and that shorter was overall better (Table 5). They noted that the short-answer format of the assessment from Gray et al might reveal more about a patient’s knowledge and gaps in their understanding, but that this quiz would take too much time. The assessment from Hoevenaars et al was too long and the language too complex for clinical practice.

Table 5

Summary of qualitative comments from respondents

Glaucoma knowledge test	Positive	Negative
NEHEP Eye Q Test (n=13)	Clear and easy-to-understand“Simple, straightforward, relevant.This one is the best of the four”.	Not enough about treatment“Covers the basic knowledge areas” but not “the knowledge needed tocare for their glaucoma once diagnosed”
Gray glaucoma knowledge assessment (n=22)	Short-answer format likely to revealknowledge and deficitsGeared toward patients on treatment	Too long and too open ended“It covers topics that are very important for patients to understand, butit may slow down clinic flow”
PBA Eye Q Test (n=14)	Fun and short	Wording may be too complexDoes not cover management“Does this list of questions include the most important information for apatient? I don’t think so”
Hoevenaars glaucoma knowledge assessment (n=26)		Too longLanguage too complexToo limited coverage of treatmenta and adherence“This test has too many questions for patients, many of which are beyondthe scope of what they need to know to take care of themselves”

Notes: The survey included a section for comments on each quiz. Gray, refers to quiz from reference 8; Hoevenaars, refers to quiz from reference 9; PBA, refers to the Prevent Blindness America Eye Q Test from reference 10 and; NEHEP refers to the National Eye Health Education Program Eye Q Test from reference 11. The comments were grouped by theme and are summarized in the table, with representative quotations.

The quiz developed by Hoevenaars et al includes 18 additional questions regarding treatment, which were not included in this study due to length.

Discussion

The purpose of this study was to identify a glaucoma quiz that would be most useful in clinical practice, using expert provider opinion. Glaucoma specialists with significant clinical experience caring for glaucoma patients ranked the Eye Q Test from NEHEP as most useful, due to it being clear, easy-to-understand, and covering “the basics”, although a deficiency of this quiz is that the knowledge about glaucoma treatment is not covered. The short-answer format assessment from Gray et al garnered both strong positive and negative opinion, with a third of respondents ranking it first and another third ranking it last. The comments from the respondents as well as the ratings of content, ease, and willingness to complete suggest the following explanation: this assessment covers important content and the open-ended short-answer format is better in identifying gaps in knowledge, however, those same attributes may make this test less appealing to patients and more time consuming in a busy clinical setting.

This spectrum of opinion may also reflect the spectrum of glaucoma patients and their individual educational needs. The NEHEP Eye Q Test was developed as a public health tool and may be best at assessing knowledge in patients with early glaucoma, glaucoma suspect, or risk factors for glaucoma. The assessment from Gray et al, which was developed for patients with glaucoma, is best for patients who know that they have glaucoma and are undergoing treatment. Importantly, based on the provider feedback in our study, this assessment would best be incorporated into a clinical practice as part of a more comprehensive education program, perhaps in combination with an experienced ophthalmic technician or a trained ophthalmic nurse educator.

The limitations of this study include the small sample size, with a response rate of ~9% of the number of individuals on the AGS listserve. The reasons may include the following: an incentive was not offered; the survey was distributed in August when some providers may not have been available; and multiple reminders were not sent, due to the use of the AGS listserve, which is not a research instrument. Also, the survey design did not allow respondents to go back and review the quizzes again when ranking them, which contributed to some dissatisfaction with the survey experience and may have contributed to only 64 respondents completing the ranking question, of the 84 total respondents who completed the survey. Importantly, this study did not include evaluation and feedback from glaucoma patients.

This study describes the first step toward a meaningful assessment of glaucoma knowledge in patients, to be used to evaluate patient education interventions, and their effects on glaucoma knowledge. The next step is to evaluate the quizzes preferred by their doctors in glaucoma patients.

Conclusion

The true–false quiz from NEHEP appears to be most useful for assessing baseline general glaucoma knowledge in the setting of a busy clinical practice, and the short-answer quiz from Gray appears to be more useful as part of a comprehensive education program, perhaps in combination with an ophthalmic educator.

Supplementary materials

Survey message and content

Email message content

Request for your help with a glaucoma patient education project

I am writing to request your participation in this study regarding patient glaucoma knowledge. The purpose of this study is to compare four publicly available “glaucoma knowledge tests” for glaucoma patients, to see which would be best for clinical practice. Glaucoma specialists are best positioned to evaluate what glaucoma patients need to know about glaucoma and what testing they would be willing to do.

This survey should take you approximately 5–10 minutes to complete. After following the attached link, you will be directed to the anonymous online survey. You will be shown each of the four patient’s “glaucoma knowledge tests” (in random order) and asked to rate each quiz by answering several questions. After reviewing and rating all four quizzes, you will be asked to rank them, from most useful in clinical practice to least.

I hope that you will be able to take the time to complete this short survey to give your valuable input. The major benefit of participation in this survey is the identification of a clinically useful knowledge assessment for glaucoma patients. We hope that this study will facilitate the further development of patient-centered education for glaucoma patients. Individual responses will not be linked to a specific person. The use of the American Glaucoma Society membership will be acknowledged in any publications that arise from this project.

Completion of the survey indicates implied consent to participate. Your participation is voluntary. If you wish to decline, simply do not fill in the survey. Please contact me at jullia.rosdahl@duke.edu or the Duke IRB at (919) 668–5111 with any questions or concerns.

Many thanks for your time and interest in glaucoma patient education,

Jullia

Jullia

Rosdahl MD PhD

Duke Eye Center

jullia.rosdahl@duke.edu

Survey content

Please review and rate each of the four “glaucoma knowledge tests”. After you have reviewed and rated each one, please rank them based on which you think would be best for clinical practice.

The four glaucoma knowledge tests will be presented in random order:

– National Eye Health Education Program (NEHEP) Eye Q Test

– Prevent Blindness America (PBA) Eye Q Test

– Gray Glaucoma knowledge assessment, Gray et al. Eye. (2010);24:1777–1789.

– Hoevenaars Glaucoma knowledge assessment, Hoevenaars et al. Euro J Ophthal. (2005);15:32–40.

Please review the following glaucoma knowledge test. Please rate the glaucoma knowledge test in the following areas:

How well does this test cover everything patients would need to know to adequately care for their glaucoma?

– Not well at all

– Not too well

– Somewhat well

– Very well

– Extremely well

How easy would it be for most glaucoma patients to understand the language in this test?

– Not easy at all

– Not too easy

– Somewhat easy

– Very easy

– Extremely easy

How willing would most glaucoma patients be to complete this test?

– Not willing at all

– Not too willing

– Somewhat willing

– Very willing

– Extremely willing

Please use this space to share any comments or concerns about this test.

Please rank all four of the “glaucoma knowledge tests” that you have just reviewed and rated, such that a 1 represents the best test for assessing glaucoma knowledge in clinical practice, and a 4 represents the worst test.

– NEHEP Eye Q Test (ten questions, True/false)

– PBA Eye Q Test (nine questions, True/false)

– Gray Glaucoma knowledge assessment, Gray et al (eleven questions, short answer)

– Hoevenaars Glaucoma knowledge assessment, Hoevenaars et al (19 questions, True/false)

Demographic questions

How old are you? Younger than 30, 30–39, 40–49, 50–59, 60–69, 70 or greater, prefer not to answer What is your sex? male, female, prefer not to answer What percentage of your clinical practice is glaucoma? Less than 10%, 10%–40%, 40%–60%, 60%–90%, greater than 90%, prefer not to answer How many years since you completed your glaucoma fellowship? >1 year, 1–5 years, 6–9 years, 10–15 years, 16–20 years, <20 years, prefer not to answer How familiar are you with the glaucoma knowledge quizzes in this survey?

NEHEP Eye Q Test

○ Extremely familiar

○ Fairly familiar

○ Somewhat familiar

○ Not at all

PBA Eye Q Test

○ Extremely familiar

○ Fairly familiar

○ Somewhat familiar

○ Not at all

Gray Glaucoma knowledge assessment

○ Extremely familiar

○ Fairly familiar

○ Somewhat familiar

○ Not at all

Hoevenaars Glaucoma knowledge assessment

○ Extremely familiar

○ Fairly familiar

○ Somewhat familiar

○ Not at all

Thank you very much for completing this survey. Your time is very much appreciated.

If you would like to be informed of the results of this survey, please email me at jullia.rosdahl@duke.edu.

Table S1

Familiarity with the glaucoma knowledge quizzes

Glaucoma knowledge test	How familiar are you with the glaucoma knowledge quizzes in this survey?
	Extremely familiar	Fairly familiar	Somewhat familiar	Not at all	Mean (SD)
NEHEP Eye Q Test (n=61)	6 (10%)	5 (8%)	15 (25%)	35 (57%)	3.30 (0.99)
Gray glaucoma knowledge assessment (n=60)	3 (5%)	2 (3%)	8 (13%)	47 (78%)	3.65 (0.78)
PBA Eye Q Test (n=61)	3 (5%)	6 (10%)	9 (15%)	43 (70%)	3.51 (0.87)
Hoevenaars glaucoma knowledge assessment (n=60)	2 (3%)	0	8 (13%)	50 (83%)	3.77 (0.62)

Notes: The respondents rated their familiarity with each glaucoma knowledge quizzes prior to the study. Most of the glaucoma specialists who responded were unfamiliar with most or all of the quizzes.

Abbreviations: SD, standard deviation; PBA, Prevent Blindness America; NEHEP, National Eye Health Education Program.