Validation of a New Diabetic Retinopathy Knowledge and Attitudes Questionnaire in People with Diabetic Retinopathy and Diabetic Macular Edema.

Purpose: A validated questionnaire assessing diabetic retinopathy (DR)- and diabetic macular edema (DME)-related knowledge (K) and attitudes (A) is lacking. We developed and validated the Diabetic Retinopathy Knowledge and Attitudes (DRKA) questionnaire and explored the association between K and A and the self-reported difficulty accessing DR-related information (hereafter referred to as Access).
Methods: In this mixed-methods study, eight focus groups with 36 people with DR or DME (mean age, 60.1 ± 8.0 years; 53% male) were conducted to develop content (phase 1). In phase 2, we conducted 10 cognitive interviews to refine item phrasing. In phase 3, we administered 28-item K and nine-item A pilot questionnaires to 200 purposively recruited DR/DME patients (mean age, 59.0 ± 10.6 years; 59% male). The psychometric properties of DRKA were assessed using Rasch and classical methods. The association between K and A and DR-related Access was assessed using univariable linear regression of mean K/A scores against Access.
Results: Following Rasch-guided amendments, the final 22-item K and nine-item A scales demonstrated adequate psychometric properties, although precision remained borderline. The scales displayed excellent discriminant validity, with K/A scores increasing as education level increased. Compared to those with low scores, those with high K/A scores were more likely to report better access to DR-related information, with K scores of 0.99 ± 0.86 for no difficulty; 0.79 ± 1.05 for a little difficulty; and 0.24 ± 0.85 for moderate or worse difficulty (P < 0.001). Conclusions: The psychometrically robust 31-item DRKA questionnaire can measure DR- and DME-related knowledge and attitudes. Translational Relevance: The DRKA questionnaire may be useful for interventions to improve DR-related knowledge and attitudes and, in turn, optimize health behaviors and health literacy. Copyright 2020 The Authors.

Introduction

Diabetic retinopathy (DR) and diabetic macular edema (DME) are common microvascular complications of diabetes which, if left untreated, can cause substantial vision loss, and poor quality of life.– The onset and progression of these two conditions can generally be prevented and controlled if diabetes is optimally managed., Similarly, early detection, surveillance, and timely treatment have been shown to reduce the incidence and progression of DR by up to two-thirds.– These preventative behaviors are influenced by a patient's knowledge (K) of and attitude (A) toward diabetes and DR. However, previous qualitative and quantitative work suggests that patients with DR have a poor understanding of the link among glycated hemoglobin A1c (HbA1c) levels, blood pressure goals, and diabetic eye diseases.– Healthcare professionals have also highlighted erroneous beliefs expressed by some patients regarding laser treatment causing irreversible vision loss, as well as a lack of understanding by patients about the purpose of the HbA1c test and optimal levels to achieve.

The knowledge, attitudes, and practice (KAP) framework is widely used to elucidate the mechanisms of health education necessary for patient behavioral changes via KAP questionnaires. For example, good diabetes- and DR-specific knowledge has been associated with positive preventative practices, such as getting an eye exam by an ophthalmologist,, whereas low levels of practical knowledge about diabetes have been linked to greater likelihood of non-adherence to diabetes vision care guidelines. Importantly, education interventions have been successful in improving knowledge and attitudes in patients with DR.

However, most questionnaires used to assess DR-related KAP are brief,,, lack a sound theoretical framework, have not adequately described the questionnaire development process, or have undergone only rudimentary psychometric testing., As such, a comprehensive understanding of patients’ K and A about their DR cannot be achieved, as no tool has undergone rigorous psychometric evaluation. Without this information, the patient-centered, clinical, and economic impact of suboptimal DR-specific K and A cannot be quantified, nor can appropriate interventions be designed or implemented to improve DR-specific practice, such as adherence to DR screening and accessing related health information.

We have developed the culturally relevant Diabetic Retinopathy Knowledge and Attitudes (DRKA) questionnaire to assess K and A in people with DR in Singapore. This manuscript reports on the processes involved in developing the two scales and describes their psychometric properties using both classical test theory (CTT) and modern psychometric methods. Using the newly validated tool, we explored the association between K and A and self-reported difficulty in accessing DR-related information (hereafter referred to as Access), hypothesizing that those with poorer K and A would report greater difficulty with this activity.

Methods

Development of the DRKA Questionnaire

The DRKA questionnaire was developed using a multiphased approach encompassing qualitative and quantitative methodology. A total of 246 participants across three phases were recruited from retinal clinics at the Singapore National Eye Centre (SNEC) between December 2015 and June 2017. The study included English- and/or Mandarin-speaking participants who were ≥21 years of age and of Chinese, Malay, or Indian ethnicity who had a primary diagnosis of DR and/or DME with type 1 or 2 diabetes. Those with significant hearing or cognitive impairment (as measured by the Six-Item Cognitive Impairment Test) or other ocular comorbidity were ineligible. Patients’ sociodemographic and clinical information was collected via an in-house questionnaire and medical files. Ethical approval from the SingHealth Centralised Institutional Review Board (#2015/2567) was obtained, and all participants provided written informed consent prior to participation. The study was conducted in accordance with the tenets of the Declaration of Helsinki.

Phase 1: Content Development Via Focus Groups and Semi-Structured Interviews

Based on the knowledge, attitudes, practice, and outcomes (KAP-O) framework (Supplementary Materials S1) and relevant related literature,,,,– we developed the moderator's guides for our qualitative phases. Eight focus groups (five English, three Mandarin) comprised of 4 to 6 participants each were conducted among 36 subjects (24 English-speaking; 12 Mandarin-speaking) by facilitators trained in qualitative research (EKF and Ching Siong Tey [TCS]). Three semi-structured interviews were also conducted with retinal specialists from SNEC. The moderator's guide contained open-ended questions to elicit information about patients’ understanding of and attitudes toward DR (e.g., “What things cause diabetic eye problems?”) and its treatment (e.g., “What worries or concerns do you have about your diabetic eye disease and treatment?”; “How do these worries or concerns influence your decisions about managing your diabetic eye disease?”) (Supplementary Materials S2).

Focus groups (65–100 minutes) and semi-structured interviews (30–45 minutes) were conducted at SNEC with a note-taker recording key points and body language. Each session was audiorecorded and professionally transcribed verbatim. After each session, the study group met to summarize the outcomes, and sessions were conducted until no new themes emerged (thematic saturation). The study coordinator (EKF) analyzed the transcripts using an inductive analytical approach based on the constant comparative method., This iterative process involves coding utterances from the transcripts into themes and contrasting the coded themes within and across transcripts. As the aim of the study was to develop questionnaires to measure level of knowledge and attitudes, we focused on the utterances that were incorrect or partially correct, or that demonstrated a lack of understanding or knowledge in the particular area. Two other researchers (TCS and HJTS) reviewed the final coding, and any disagreements were iteratively resolved during five discussion sessions with all three coders by returning to the transcripts, re-reading the relevant point, and agreeing on the final code for that utterance.

Using information from the themes and patient utterances, the study team crafted items to measure DR-specific Knowledge (n = 31) and Attitudes (n = 8) for pretesting; for example, one theme was “unclear about the HbA1c test,” and a supporting quote was “Is it the fasting blood test?” Two multiple choice items on the Knowledge scale were crafted asking what the HbA1c test measures and what the best range for people with diabetes is. The Knowledge items measured the patient's understanding about DR symptomatology, factors associated with the development and progression of DR, and treatment and management regimens and goals. Examples of how the Knowledge items were rated include the following:

Multiple choice—What is the main goal of laser treatment for diabetic eye disease? (A) To improve your vision. (B) To preserve your vision. (C) Both. (D) Not sure.

True/False/Not sure—Vision problems from diabetic eye disease can be improved with new glasses.

Agree/Disagree/Not sure—Which of the following things can increase your risk of developing diabetic eye disease? … High blood pressure.

Participant responses for each item were coded dichotomously as correct (1) or incorrect/not sure (0).

The Attitude items measured patients’ true or false beliefs about diabetic eye disease development and progression, the likelihood of the patient assuming control regarding potential development and progression of diabetic eye disease, and the patient's feelings about DR management experiences in the past. The Attitude items were rated using a four-point Likert scale (“strongly agree” to “strongly disagree”), with higher scores indicating more positive attitudes.

Scores for the K and A scales can simply be summed and averaged as per classical methods to generate raw scores from questionnaires; however, it is recommended that Rasch analysis be applied to the raw data to optimize measurement precision (see Rasch Analysis section below).

Phase 2: Scale Refinement Via Cognitive Interviews

Ten individual cognitive interviews were conducted with DR patients to pretest the clarity of instructions, items, and response options and to highlight any issues by using a combination of the think-aloud and verbal probing techniques. After the first three interviews, feedback was systematically synthesized and subsequently incorporated into a new version of the questionnaires. This process was repeated until no further issues emerged.

Phase 3: Psychometric Analysis in a Large Patient Sample

Study Population

The pilot versions of the DRKA questionnaires were interviewer-administered to 200 patients with DR (134 English-speaking; 66 Mandarin-speaking) who were purposively sampled to ensure a sufficient spread of gender, age, ethnicity, and DR severity (see Supplementary Materials S1 for DR and visual acuity assessment). In addition, 15 retinal specialists and 28 diabetic nurses answered the Knowledge questionnaire via an online survey portal.

Rasch Analysis

We performed Rasch analysis on the DRKA scales using Winsteps 3.91.2 (Winsteps, Chicago, IL) using the dichotomous (Knowledge) and polytomous (Attitudes) Andrich single rating scale model. Rasch analysis provides substantial insight into the psychometric properties of a scale (Supplementary Materials S1), including response category functioning, measurement precision, unidimensionality, item fit, item discrimination, targeting of item difficulty to the person's ability, and differential item functioning (DIF), which reflects item bias for age, gender, and education level. In addition, Rasch analysis converts the ordinal-level raw score data from the item responses into estimates of interval person measures (expressed in logits), which are suitable for parametric testing. We used the person measures (logits) derived from Rasch analyses in our subsequent statistical testing, as using Rasch scores has been shown to improve measurement precision and detection of associations among variables of interest.,

Classical Test Theory Analysis

In order to assess the construct (convergent and divergent) validity of our DRKA scales, participants also answered two generic health literacy questionnaires, namely the Health Literacy Test for Singapore (HLTS) and the Newest Vital Sign (NVS), as well as the Patient Health Questionnaire (PHQ-9), which is a valid measure of depressive symptoms (Supplementary Materials S1). We also assessed the discriminant validity, criterion validity, and temporal reliability of the DRKA questionnaire using standard traditional methods and statistical tests (Supplementary Materials S1).

Statistical Analyses

Patients’ sociodemographic and clinical information for all phases was characterized using means and standard deviations for continuous data and counts and percentages for categorical data using Stata 15 (StataCorp, College Station, TX). We examined the associations of K and A person measures with a single item measuring self-reported difficulty accessing DR-related information (Access): “In the past 12 months, how much difficulty did you have finding information about your diabetic eye disease (e.g., TV, Internet, books, radio, or newspapers?” Response options included no difficulty, a little difficulty, or moderate difficulty or above. We conducted univariable linear regression analysis of K and A separately against the Access item as the independent variable and determined pairwise differences among the three categories (Tukey's test). Linearly increasing/decreasing trends of K/A scores with greater difficulty reported on the Access item were assessed by treating the Access item as a continuous variable taking the values of 1 (no difficulty), 2 (a little difficulty), and 3 (moderate difficulty or above).

Results

Of the 36 participants (mean age, 60.1 ± 8.0 years; 53% male; 50% secondary education), 39% (n = 14), 19% (n = 7), 14% (n = 6), and 22% (n = 8) had mild non-proliferative diabetic retinopathy (NPDR), moderate NPDR, severe NPDR, and proliferative diabetic retinopathy (PDR), respectively; three (8.3%) had clinically significant macular edema (Supplementary Table S1).

Overall, few participants could clearly explain DR. Although most correctly identified that having uncontrolled blood glucose is a risk factor for developing DR, few had heard of “the glycated hemoglobin or HbA1c test” or knew the target value indicating optimal diabetes control (Table 1). Important risk factors such as blood pressure and lipid control were rarely mentioned, and some participants listed factors not known to be associated with DR, including radiation exposure, high eye pressure, processed foods, stress, pollution, and excessive computer usage. Many patients also reported using special diets (e.g., superfoods), Chinese/Indian herbal supplements, acupuncture, or reflexology to reduce DR progression. Most patients were aware of laser treatment and intravitreal injections, but there was a lack of understanding about how they worked and that ongoing sessions may potentially be needed.

Table 1.

Summary of the Focus Group Findings Indicating Knowledge Deficits or Erroneous Attitudes (n = 36 Participants)

Themes	Sample Quotes	Focus Group (Gender of Participant)
Knowledge
Lack of understanding of what diabetic retinopathy is/how diabetes affects the eye	“I think is the nerve … they find out the blood flow is not so good, they asked to go check the heart doctor, could be probably one of the nerve is clot so the blood system is not constantly supplying to the eye.”	FG1 (M)
	“I don't know how blood sugar will affect my eye. All I know is when the doctor tells me. … Even the doctor might not know, they are just guessing”	FG1 (M)
Lack of understanding of what causes diabetic retinopathy/misinformation about causes	“According to the doctor, it is because the diabetes is not controlled so causes the pressure to increase. That is why the thing bleeds.”	FG1 (M)
	“Maybe because of the computer radiation or whatsoever. I talked to the doctor, the doctor says ‘no … nonsense.’ They say this is caused by your diabetes so I say okay, whatever … but it could be caused by that (computer).”	FG2 (M)
	“Partly due to age. … I'd like to assume it is because of old age it deteriorates not because of diabetic.”	FG3 (M)
Lack of understanding of how to stop DR progressing/misinformation about how to stop progression	“Don't see computer so often.”	FG3 (F)
	“Some people advise to take wolfberries.”	FG3 (F)
Unclear about the HbA1c test	“I heard but I don't know what is that.”	FG1 (M)
	“Is it the fasting blood test?”	FG1 (M)
	“They don't elaborate what is this test about. Everything they will say ‘this’ and your reading is ‘this,’ that's all.”	FG2 (M)
	[long pause] “Is that a medicine?”	FG4 (F)
Unclear what the purpose of injections for diabetic macular edema is or how many times they are needed	“Worse come to worse, if doctor finds that there is no clot (in the heart, which will affect the oxygen supply to the eyes) then he will give me this oxygen injection, they just put some oxygen to brighten it. … I don't know what is it, I did not really ask him.”	FG1 (M)
Unclear what the purpose of laser is or how many times needed	“The laser job is to go there and burn and seal off the bleeding … for medical I don't know how it helps with the bleeding.”	FG3 (M)
Uncertain if DR can be asymptomatic	“I do not have vision problem so I did not notice it.”	FG1 (M)
	“My vision was okay, no blood, no nothing so I don't know why she said that it's caused by diabetes.”	FG2 (M)
Attitudes
Conflicting attitudes from traditional/Western medicine relating to managing DR	“So I went to China, I see the Chinese doctor, he gave me Chinese medicine to eat you know, here [Singapore] I ask for medicine to eat, they say no, no medicine to eat … how can it [DR] cure if you keep on drop drop drop [eye drops] there is no internal support, if Chinese can do it, why can't Singapore do it.”	FG2 (M)
	“I go see Chinese sensei my eyes can get better, then they told me don't go too much on injection. The Chinese sensei say don't go for injection, if they inject you the wrong position you go totally blind. That's why I worry.”	FG2 (M)
Injections are painful and not helpful/worry about side effects from treatments	“I don't know how painful the jab was because I have not done before but when I was outside the room, I heard others shouting in pain so they put fear in me.”	FG1 (M)
Getting DR is up to fate/genetics/inevitable when you get old	“I think old people, people who are 60 and above or 65 and above, should not go for injections … because it is very painful as what he [FG2, M] says and it might not work on them also.”	FG2 (F)
	“Probably … I have this condition [DR] is because I have diabetes and my parents have diabetes also, inherited.”	FG4 (F)
The responsibility to manage DR is up to the person with the disease (self)	“I believe that diabetes is not only because of genetics but also because of habits, way of life, diets.”	FG4 (M)
	“At the end of the day, I think is still because of self-disciplined.”	FG4 (F)
The responsibility to manage DR is up to the doctor/government (others)	“We don't seem to have a source where we can connect when I have a question or I need to know/find out—local version to describe local experience about diabetes. … I don't have an area, a source to ask my questions and not sure how to get my answer—all through guesswork.”	FG1 (M)
	“Medical authorities have to play a part … you got to tell me … so if you don't give me any advice, I won't know what to do.”	FG2 (M)

A commonly expressed attitude by participants was that DR is due to aging or genetics and therefore largely inevitable (Table 1). Some participants reported reluctance to receive treatment for DR as it was perceived as being painful or not necessary for the elderly, it could cause blindness or other health problems, or such treatment contravened advice from Traditional Chinese Medicine (TCM) practitioners. Similar themes emerged from the semi-structured interviews with retinal specialists (Supplementary Materials S1).

The mean age of the 10 participants (50% Chinese) was 58 ± 13.5 years, and most were male (n = 8, 80%) (Supplementary Table S2). Following the cognitive interviews, two items were added, five were removed, and 30 changes to item phrasing were made to the K scale (Supplementary Table S3), resulting in a pilot 28-item scale. For the A scale, one item was added and phrasing for 11 items was amended, resulting in a pilot nine-item A scale.

Sociodemographic and Clinical Characteristics

Two hundred participants (mean age, 59.0 ± 10.6 years; 59% male; 69% Chinese; 73% secondary school education or lower) completed the pilot 28-item K scale and the nine-item A scale (Table 2). Most (n = 94, 48.5%) had type 2 diabetes, although nearly half (n = 90, 45%) did not know their diabetes type. Mean duration of diabetes was 16.0 ± 9.8 years; 84 (42%), 52 (26%), 34 (17%), 12 (6%), and 18 (9%) had none to mild NPDR, moderate NPDR, severe NPDR, PDR, or quiescent PDR in the better eye, respectively. Fourteen (7%) had DME. Based on the NVS, 44% (n = 85), 22% (n = 42), and 34% (n = 67) had limited, potentially limited, or adequate health literacy, respectively.

Table 2.

Sociodemographic and Clinical Characteristics of the 200 Participants in Phase 3

Characteristics	n (%a)
Male	118 (59.0)
Diabetes type
Type 1	13 (6.5)
Type 2	97 (48.5)
Unknown	90 (45.0)
At least one other diabetes complication (yes)	79 (39.5)
At least one other comorbidity (yes)	169 (84.5)
Attended diabetes education
Never	95 (47.5)
Yes, in the past few years	42 (21.0)
Yes, at diagnosis of diabetes or soon after	62 (31.0)
Diabetes treatment
Tablets	114 (57.0)
Insulin	79 (39.5)
Ethnicity
Chinese	137 (68.5)
Indian	37 (18.5)
Malay	26 (13.0)
Marital status
Single/never married	28 (14.0)
Married/de facto	150 (75.0)
Divorced/separated/widowed	22 (11.0)
Education level
None or primary	38 (19.0)
Secondary	107 (53.5)
A level, diploma, vocational training, university degree	55 (27.5)
Occupation
Managerial, administrative	36 (18.0)
Production, technical	18 (9.0)
Other	53 (26.5)
Self-employed	13 (6.5)
Not working	80 (40.0)
Monthly household income
<$2000	66 (33.0)
$2000 to <$5000	58 (29.0)
$5000 to <$10,000	34 (17.0)
≥$10,000	15 (7.5)
Housing type
1- or 2-room HDB	9 (4.5)
3-room HDB	41 (20.5)
4-room HDB	69 (34.5)
5-room HDB or executive flat	71 (35.5)
Condominium or landed property	10 (5.0)
Vision impairment (better eye)
No	148 (74.0)
Yes	52 (26.0)
Severity of DR (better eye)
No to mild NPDR	84 (42.0)
Moderate NPDR	52 (26.0)
Severe NPDR	34 (17.0)
PDR	12 (6.0)
Quiescent PDR	18 (9.0)
DME present (better eye), yes	14 (7.0)
NVS category
High likelihood of limited HL (score 0–1)	85 (43.8)
Possibility of limited HL (score 2–3)	42 (21.7)
Almost always indicates adequate HL (score 4–6)	67 (34.5)
Health Literacy Test for Singapore category
Inadequate HL (<3 numeracy; <27 comprehension)	54 (38.3)
Adequate HL (≥3 numeracy; ≥27 comprehension)b	87 (61.7)
Continuous Variables	Mean (SD); Median (Range)
Age (y)	59.0 (10.6); 60.0 (30.0–82.0)
Duration of diabetes (y)	16.0 (9.8); 16.0 (1.0–40.0)
HbA1c (%)	8.0 (1.6); 7.7 (4.4–12.8)
Presenting visual acuity in better eye (logMAR)	0.24 (0.18); 0.20 (0.00–0.90)

Percentages for some variables may not equal 100% due to missing data.

Only 141 answered both the numeracy and comprehension questions.

HDB, Housing Development Board; HL, health literacy.

Rasch Analysis Results

The 28-item K scale displayed good item fit and targeting; however, it had relatively poor precision (Person Separation Index [PSI] < 1.5), evidence of multidimensionality (raw variance explained by the measure 38% and a first contrast eigenvalue of 2.6), and DIF for three items (Table 3). To remedy these issues, we first explored the standardized residual loadings where two items (losing weight and increasing exercise) were loading substantively, suggesting that they were measuring a different construct. Upon removal of these items, the eigenvalue was still greater than 2.0; however, the disattenuated correlations were high (1.0 and 0.84), which suggested that any potential secondary dimensions were likely to be measuring different aspects of DR-specific knowledge. Second, we examined item discrimination to unearth any low information items. The two items are “Getting older” and “Use traditional medicine/alternative therapies” had values of 0.43 and 0.56, respectively. Given that these items were already captured by related items on the A scale, they were deleted. Two other items (stress, attend recommended eye checks) also had poor discrimination (0.36 and 0.66, respectively) and were at the extreme difficult (19.5% correct) and easy (93% correct) end of the knowledge spectrum; as such, we removed these items. The remaining 22-item K scale was unidimensional, with good targeting (Fig. 1) and no DIF, although precision still remained suboptimal (PSI = 1.09).

Table 3.

Fit Parameters of the DRKA Scale Compared to the Rasch Model Before and After Modification (n = 200)

Parameters	Rasch Model	Knowledge (n = 28 Items)	Knowledge Revised (n = 22 Items)a	Attitudes (n = 9 Items)	Attitudes Revised (n = 9 Items)a
Item no.		1–28	1–14, 16, 18–22, 27–28	1–9	1–9
Disordered thresholds	No	No	No	Yes	No
Person separation index	>1.5	1.06	1.09	1.18	1.24
Person reliability	>0.7	0.53	0.54	0.58	0.60
Item fit (infit MnSq)	>0.7 and <1.3	None	None	None	None
Principal components analysis
Variance by the first factor	>50%	37.8%	38.1%	48.2%	46.2%
Eigenvalue for first contrast	<2.0	2.63	2.19	1.71	1.62
Targeting, difference between person and item mean	<1.0	0.82	0.92	0.93	1.32
Differential Item Functioningb	<1.0, P > 0.05	Age (item 10) Gender (item 20) Education (item 11)	None	None	None

Bolded values indicate misfit to the Rasch model.

Columns for which the scales achieved the best fit to the Rasch model.

DIF was tested for age group (≤50 years, >50 years), gender, and education level (none or primary, secondary, or higher).

Figure 1.

Person–item map for the 22-item Diabetic Retinopathy Knowledge questionnaire. To the left of the dashed line are the participants, represented by “#” (signifying 3 participants) and by “•” (signifying 1 to 2 participants); on the right are the items, denoted by their item number and content. Participants with better DR-specific knowledge and the most “difficult” items are near the top of the diagram; participants with worse DR-specific knowledge and the least “difficult” items are near the bottom. This figure shows that items on the Knowledge questionnaire are generally well targeted to participant ability level, with a <1 logit difference between the mean of person knowledge and mean of item difficulty. Four items at the bottom of the figure are too easy for this particular patient sample.

Initial evaluation of the nine-item A scale showed good targeting and unidimensionality with no item misfit or DIF; however, thresholds were disordered, suggesting that the middle response categories of “somewhat agree” and “somewhat disagree” were used interchangeably by participants (Fig. 2). Collapsing these categories resolved the disordering (Fig. 2), which also improved precision but slightly worsened targeting (difference between person/item means > 1 logit) (Fig. 3).

Figure 2.

Category probability curves of the nine-item Diabetic Retinopathy Attitudes questionnaire before (A) and after (B) category collapse. The disordered thresholds on the left demonstrate that the category of “somewhat agree” was at no point the most likely option to be chosen by participants.

Figure 3.

Person–item map for the nine-item Diabetic Retinopathy Attitudes questionnaire. To the left of the dashed line are the participants, represented by “#” (signifying 3 participants) and by “•” (signifying 1 to 2 participants); on the right are the items, denoted by their item number and content. Participants with “better” DR-specific attitudes and the most “difficult” items are near the top of the diagram; participants with worse DR-specific attitudes and the least “difficult” items are near the bottom. This figure shows that items in the Attitudes questionnaire are reasonably well targeted to participant ability level, with a just over 1 logit difference between the mean of person attitude and mean of item difficulty. One item at the bottom of the figure is too “easy” for this particular patient sample, and there is a lack of “difficult” items to challenge those with “better” attitudes.

Classical Test Theory Results

Both the K and A scales displayed excellent discriminant validity, with scores increasing as level of education increased; for example K mean scores (logits) increased from 0.66 ± 0.81 for people with none or primary school to 0.81 ± 0.91 and 1.28 ± 0.80 for people with a secondary or higher education, respectively (Table 4). Those with active PDR (i.e., currently under treatment) had significantly higher K scores than those with milder forms of the disease or quiescent PDR. Compared to people with no vision impairment, those with vision impairment had significantly worse K scores.

Table 4.

Discriminant Validity of the Diabetic Retinopathy Knowledge and Attitudes Questionnaire

	Knowledge (n = 22 Items)		Attitudes (n = 9 Items)
Variable	Mean (SD) (Logits)	P a	Mean (SD) (Logits)	P a
Education level		<0.001 b		<0.001 c
None or primary	0.66 (0.81)		0.98 (1.10)
Secondary	0.81 (0.91)		1.17 (1.26)
A level, diploma, vocational training, university degree	1.28 (0.80)		1.87 (1.19)
DR severity		0.039 d		0.169
None to mild NPDR	0.86 (0.94)		1.33 (1.08)
Moderate NPDR	0.97 (0.66)		1.45 (1.36)
Severe NPDR	0.78 (0.88)		0.97 (1.18)
PDR	1.64 (1.19)		1.95 (1.64)
Quiescent PDR	0.74 (0.86)		1.21 (1.42)
DME status		0.151		0.984
Present	1.24 (0.62)		1.33 (1.23)
Absent	0.89 (0.90)		1.32 (1.26)
Vision impairment (better eye)		0.050		0.242
None (≤0.3 logMAR)	0.98 (0.85)		1.39 (1.23)
Mild (>0.3 logMAR)	0.71 (0.97)		1.15 (1.13)

Bolded values indicate statistically significant results.

Assessed using one-way analysis of variance and pairwise comparison of means using Tukey's method.

Significant difference between none or primary education and A level, diploma, vocational training, university degree, as well as between secondary and A level, diploma, vocational training, university degree.

Significant difference between none or primary education and secondary education, as well as between none or primary education and A level, diploma, vocational training, university degree.

Significant difference between none or mild versus PDR; severe NPDR versus PDR; and quiescent PDR versus PDR

Convergent and divergent validity were excellent for both the DRKA scales, with expected moderate correlations found with the health literacy tests (all r ≈ 0.3), and no to low correlations for the PHQ-9 (Supplementary Table S4). The 22-item K scale demonstrated good concurrent validity, with retinal specialists receiving significantly higher raw scores on average compared to patients (21.5 vs. 13.4; P < 0.001). Finally, both the K (r = 0.77, P < 0.001) and A (r = 0.74, P < 0.001) scales showed acceptable temporal reliability.

Association Between K and A and Access

In univariable analyses, compared to those with lower K scores, those with higher K scores were significantly more likely to report no difficulty accessing information about their DR: 0.99 ± 0.86 for no difficulty (n = 171), 0.79 ± 1.05 for a little difficulty (n = 13), and 0.24 ± 0.85 for moderate or worse difficulty (n = 16) (P trend = 0.001) (Table 5). Results were similar for the association between A scores and self-reported difficulty accessing DR-related information (Table 5).

Table 5.

Comparison of Knowledge and Attitudes Scores by Self-Reported Ability to Access DR-Related Information

	Mean ± SD			Tukey's HSD Test
	No Difficulty (n = 171)	Little Difficulty (n = 13)	Moderate Difficulty or Above (n = 16)	2 vs. 1	3 vs. 1	3 vs. 2	P Trenda
Knowledge	0.99 ± 0.86	0.79 ± 1.05	0.24 ± 0.85	0.708	0.004	0.215	0.001
Attitudes	1.44 ± 1.24	0.94 ± 1.38	0.37 ± 0.79	0.328	0.002	0.414	<0.001

Bolded values indicate statistically significant results.

Test of a linearly increasing or decreasing trend in knowledge/attitudes scores with response to the item, “In the last 12 months, how much difficulty did you have accessing information about your diabetic eye disease (e.g., TV, Internet, books, radio, or newspapers)?”

Discussion

Using a mixed-method study design in a multi-ethnic, clinical population of Singaporeans with DR and DME, we developed and psychometrically assessed the DRKA questionnaire. The 22-item K scale was found to capture patients’ knowledge about the development and progression of DR, treatment and management regimens and goals, and DR symptomatology. The nine-item A scale measures patients’ beliefs about disease development and progression and the onus of responsibility, as well as emotional reactions to DR management experiences. Higher K and A scores were significantly associated with less self-reported difficulty accessing DR-related information. With its strong basis in the KAP framework, the DRKA questionnaire may be useful for measuring the prevalence of suboptimal patient knowledge and attitudes relating to DR, particularly in adults with diabetes who are at risk for developing DR to facilitate early intervention and prevention strategies; for demonstrating the effectiveness of educational interventions to improve patient knowledge/attitudes about DR; and for quantifying the link between knowledge improvements and changes in practice (e.g., attendance rates of recommended eye screening, timely disease management).

Although the psychometric properties of the DRKA questionnaire were promising overall, both scales displayed suboptimal measurement precision (reliability between 0.5 and 0.6), which means they have the capacity to distinguish one or two levels of K/A. Good scale precision depends on a large variance in sample ability, longer test length, higher number of categories per item, and good sample-item targeting. Given that the length and targeting of the DRKA questionnaire were appropriate, the low precision levels are likely due to the dichotomous rating scale (K) and a lack of variance in participant ability. Although we purposively sampled for age, gender, ethnicity, and DR severity, we did not do so for education; had our sample included more patients at the lower or higher ends of the education spectrum, we may have observed more variance in sample ability and better precision. Future work to refine the DRKA questionnaire will aim to rectify this sampling issue.

As hypothesized, the DRKA scale was positively correlated with education. This is important, as it suggests that poor DR-specific knowledge and attitudes may be amenable to improvement via educational interventions. We also observed that people with vision impairment had significantly worse K scores than those without, probably due to the well-established link between low education and literacy levels and vision impairment. We also observed that DRKA scores were substantially higher in those with active PDR compared those with NPDR. This is likely because those with late-stage, active disease frequently attend DR appointments and are regularly educated about their DR by their doctors. Interestingly, those with quiescent PDR had similarly low DRKA scores compared to patients with NPDR, suggesting that knowledge gained during active disease phases may attenuate when the condition is under control. This finding is supported by studies in irritable bowel syndrome, which have found that the type, amount, and source of information desired by patients depend on the course of the disease (i.e., active vs. remission).,

Our inclusion of items on the A scale relating to the inevitability of getting DR due to aging, genetics, or fate is important, as disease-related fatalism has been reported previously in other health fields, such as cancer., Indeed, high fatalism scores have been found to predict non-adherence to cancer screening recommendations, as well as diabetes medication adherence and self-care. As such, our A scale may be useful for predicting non-adherence to follow-up appointments for DR treatment. Similarly, our A scale provides an opportunity for clinicians to enhance patient care by detecting and addressing easily correctable erroneous beliefs, especially as the expertise and authority of health professionals are highly respected in Singapore's hierarchical culture.

We found that higher K and A scores were associated with greater likelihood of reporting no difficulty accessing DR-related information, a proxy measure of patients’ DR-related health literacy. This is important, as limited health literacy has been independently associated with worse glycemic control and higher rates of DR. This suggests that improving DR-related K and A could also enhance DR-specific health literacy, which, in turn, may improve clinical outcomes for people with diabetes and DR. Other studies have also reported a relationship between K and A and DR-related practices in patients with DME and DR, such as getting an eye exam,, although we were not able to assess this relationship in our study. Future studies could use the DRKA questionnaire to assess the associations among K and A, health literacy, and positive DR-related behaviors such as attendance at eye screening and adherence to treatment interventions.

Our study has several strengths, including its mixed-methods design incorporating a strong theoretical framework, as well as evidence-based guidelines including rigorous pretesting and final validity testing using both modern and classical methods. These processes enhance the face, content, and construct validity of the DRKA questionnaire and ensure that the K and A score estimates are robust. Several limitations must also be acknowledged. First, the relatively low measurement precision of the DRKA questionnaire means that results must be interpreted with caution, although this issue is likely sample dependent and not an inherent flaw in the scale itself. Second, we did not include a control group with no DR in our validity testing analyses, which would have strengthened the study design. Future work to assess these additional validity and reliability indices is needed. Third, the DRKA questionnaire may not be generalizable to countries outside of Singapore with different healthcare systems, including the use of TCM, and attitudes toward illness. However, most of the knowledge items are broadly applicable (e.g., understanding laser procedures, intravitreal injections, and HbA1c targets and goals) and could be applied to DR populations more widely. Future work is required to culturally validate the DRKA questionnaire and adapt certain items (e.g., seeing TCM doctors) in Asian and Western populations. Finally, the sensitivity of the DRKA questionnaire to education interventions was not evaluated, and future studies should test the responsiveness of the scale. To facilitate this and make the DRKA questionnaire a more clinically useful tool, a cut-off that defines “good” and “poor” DR-specific knowledge and attitudes is needed; for example, methods that are used to define optimal cutpoints for medical diagnostic tests could be applied by using a clinical outcome relevant to DR-specific knowledge and attitudes as a reference, such as progression of DR severity or non-adherence to treatment appointments.

In conclusion, the 31-item DRKA questionnaire is a psychometrically sound instrument to measure patients’ knowledge and attitudes relating to DR and is associated with patients’ self-reported access to DR-related information. The DRKA questionnaire may be useful to clinicians who wish to quantify their patients’ understanding of their condition, to researchers who intend to design and evaluate educational interventions to improve health behaviors, and to decision-makers who allocate resources to improve DR-related outcomes. Future work is needed to culturally validate the DRKA questionnaire, define cutpoints for poor DR-specific knowledge and attitudes, and test the responsiveness of the instrument to interventions.