A Short Knowledge Assessment Tool Is Valid and Acceptable for Adults with Inflammatory Bowel Disease.

BACKGROUND: For people with inflammatory bowel disease, validated knowledge questionnaires are valuable to identify gaps in understanding and explore the impact on disease variables. AIMS: The aim of this study was to validate the short knowledge questionnaire Inflammatory Bowel Disease Knowledge Inventory Device 2, known as IBD-KID2, for use with adults with inflammatory bowel disease.
METHODS: Concurrent validity of IBD-KID2 was assessed by comparing scores with those achieved on the Crohn's and Colitis Knowledge Score (CCKNOW). IBD-KID2 reliability was assessed with test-retest completion at two time points, generalizability assessed by comparing IBD-KID2 cohort scores at different recruitment centres, and acceptability assessed using participant survey.
RESULTS: Seventy-five adults with inflammatory bowel disease completed the study. The mean percentage scores achieved on the IBD-KID2 and CCKNOW were 72.8% (SD 16.0) and 49.7% (SD 18.2), respectively. There was a significant correlation between IBD-KID2 and CCKNOW scores (R 0.573, P < 0.005), confirming concurrent validity. IBD-KID2 reliability was confirmed as no significant difference was seen between scores at test and retest (mean difference -0.2, P = 0.92). Generalizability was established as no significant score difference was seen between recruitment centres after controlling for population differences. The acceptability survey showed that 49 (69%) participants preferred IBD-KID2 to the CCKNOW, 60 (85%) found the IBD-KID2 easier to complete, and 38 (53%) considered the CCKNOW as most suitable for adults.
CONCLUSIONS: IBD-KID2 is a valid, reliable, and generalizable tool for measuring knowledge in adults with inflammatory bowel disease with good acceptability. IBD-KID2 is easy and quick to complete, hence limiting respondent burden.

Introduction

Inflammatory bowel disease (IBD) is a chronic condition of the gut characterised by phases of active disease and remission. The main clinical sub-types are Crohn’s disease (CD) and ulcerative colitis (UC). There is currently no medical cure, and patients require treatment and specialist monitoring over the course of their life. The complex nature of the disease may necessitate multi-faceted treatment regimens, and outcomes may be improved if the patient is actively involved in managing their condition and receives disease-specific education [1-4].

Improved IBD knowledge has been associated with a number of benefits including enhanced coping [1], improved adherence to medications [5, 6] and disease outcomes [7, 8], and reduced health care costs [3, 7]. Increased knowledge levels have not been shown to enhance health-related quality of life in adults with IBD [3, 9, 10] and may lead to greater levels of anxiety, although the reason for this is unclear [11]. Patients with IBD have demonstrated a need for disease-related information, but this will vary between patients. For example, one study found that patients with CD have a higher need for information on medication, daily life, and pregnancy than patients with UC [12]. Identifying gaps in patients’ existing disease-related knowledge enables healthcare teams to provide tailored information, and validated questionnaires can be a useful tool to formally assess knowledge levels.

Currently, there are a number of IBD knowledge questionnaires available that are used among adults [13-18], but notably all but one of these questionnaires (the Short Knowledge Questionnaire [15]) consists of at least 24 items. The length of a questionnaire is an important factor to consider when used in a time-pressed environment such as clinical practice. The IBD Knowledge Inventory Device (IBD-KID) is a valid and reliable assessment tool originally developed in 2013 to assess disease knowledge in children and adolescents with IBD and their parents [19, 20] and updated in 2019 following a response pattern analysis as IBD-KID2 with a reduced number of items (from 23 to 15) [21]. IBD-KID2 has demonstrated good validity, generalizability, and reliability when tested with children [22, 23] and good sensitivity to detect changes to IBD-related knowledge levels in parents of children with IBD following an education programme [24]. It has also been used to measure knowledge levels of siblings of children with IBD [23], as well as members of the general public [25]. Furthermore, readability of IBD-KID2 was found to be adequate for readers from an age of eight years [22], which is in line with recommended reading levels for public healthcare material [26]. Importantly, neither the wording of IBD-KID2 nor the content is specific to the paediatric IBD population. This fact paired with its favourable properties would make it the ideal tool for assessing disease-related knowledge in adult patients with IBD in a clinical and research setting. The aim of this study was to test the validity and reliability of the IBD-KID2 when used with adult patients with IBD. The generalizability of the IBD-KID2 between two New Zealand centres was also assessed, as well as associations between IBD knowledge and demographic and patient-specific variables.

Methods

Study Centres and Participants

Adults with a confirmed diagnosis of IBD were approached to participate in the study when they attended the outpatient Gastroenterology Service at Christchurch or Dunedin Hospital, New Zealand, between March and June 2020. The inclusion criteria stipulated that the participants had to be 18 years of age or older with a confirmed diagnosis of IBD. Participants were ineligible to part take if they had insufficient English language skills, as they were required to be able to read the participant information sheet and consent form, and complete the surveys.

Study Measures

IBD-KID2

The IBD-KID2 is a self-administered IBD knowledge questionnaire consisting of 15 items (six multiple-choice questions (MCQ) and nine true/false (T/F)) questions. Each correct answer is scored as one, with a maximum total score of 15 (Supplementary Data 1).

CCKNOW

The CCKNOW is a validated self-administered IBD knowledge questionnaire consisting of four T/F and 20 MCQ. Each correct answer is scored with one, with a maximum total score of 24 [13].

Demographic Survey

The demographic survey collected data on participants’ gender, diagnosis, current age, age at diagnosis, time since diagnosis, ethnicity, level of education, and support group membership.

Acceptability Survey

An acceptability survey was devised asking participants to note which one of the two IBD knowledge questionnaires they preferred (CCKNOW vs IBD-KID2), considered easiest to complete and most suitable for adults. This survey also included two open-ended questions, giving participants the opportunity to comment on their preference and note any topics they thought were missing from IBD-KID2.

Study Procedure

Eligible participants were informed about the study by their clinician (Gastroenterologist or IBD Nurse), and permission was sought for the research team to be given their contact details. A portion of the data collection period coincided with the New Zealand COVID-19 2020 lockdown, and hence, participants were approached either face-to-face or via phone during their telehealth appointment. Those who agreed to be contacted were provided with the participant information sheet and a link to an electronic consent form. Participants who completed and signed the online consent form were emailed links to the web-based questionnaires, with one exception where paper format was preferred and the study documents were sent by post. Data collection took place at two time points. At baseline, participants were asked to complete the demographic survey, IBD-KID2 and CCKNOW, and acceptability survey. The order in which the IBD-KID2 and CCKNOW appeared was alternated for each participant to avoid order effects bias. At follow-up, after two weeks, participants were asked to complete IBD-KID2 again. Reminder emails were sent to participants who did not complete their surveys within two weeks.

Validity, Reliability, and Generalizability Testing

Validity

The concurrent validity of IBD-KID2 was assessed to determine whether it is an appropriate tool for use among a population of adults with IBD. Concurrent validity assesses the extent to which a new measure correlates with an already validated one that tests the same or a similar concept. Concurrent validity of IBD-KID2 was assessed against the CCKNOW knowledge questionnaire.

Reliability

Test–retest reliability of IBD-KID2 was assessed to determine the consistency of results across a period of two weeks. The timeframe of two weeks was chosen to avoid memory effects and the possibility for participants to acquire new IBD knowledge as part of their patient journey.

Generalizability

Generalizability of IBD-KID2 was measured by examining the population differences and overall scores between the two study centres.

Statistical Analysis

Demographic and patient-specific information was analysed using descriptive and frequency statistics. Associations between categorical demographic/patient-specific variables and the IBD-KID2/CCKNOW mean percentage scores were assessed using independent samples t-tests, with analysis of variance with Tukey’s post hoc tests for variables with more than two comparator groups, and linear regression analysis (R) for continuous variables.

Concurrent validity of IBD-KID2 was assessed in two ways. The mean percentage scores achieved on the CCKNOW and IBD-KID2 were compared using the paired sample t-test. While a smaller difference between the mean percentage scores of the two questionnaires would usually indicate greater concurrent validity, IBD-KID2 was presumed to be easier to complete as it has been well validated among children and therefore, predicted to achieve higher mean scores. Thus, concurrent validity of IBD-KID2 was also assessed at the individual level using the Pearson Correlation Coefficient, with a score closer to one indicating better concurrent validity.

Reliability of IBD-KID2 was assessed using a paired t-test to compare the baseline and follow-up mean scores. Intra-Class Correlations (ICC) and Cronbach’s alpha were used to establish the degree of internal consistency. The ICC score and 95% confidence interval were calculated based on a two-way mixed effect model, average measures, and absolute agreement [27].

Generalizability was assessed by examining between-centre population differences using independent sample t-tests (linear variables) and Chi-Square Tests (categorical variables), and comparing mean survey scores between the two centres.

The frequency of correct answers was examined to identify gaps in knowledge for those items where the cohort scored less than 50%. The data obtained from the acceptability survey on questionnaire preference, ease of completion, and suitability were analysed using frequency statistics. The data obtained from the two open-ended questions were analysed using manifest content analysis, a type of summative content analysis [28]. The 0.05 level of significance was adopted for this study. Statistical analyses were performed using IBM SPSS for windows, version 25 [29].

Results

Demographic and Patient-Specific Information

While 107 patients were approached to participate, 80 provided consent (response rate 75%), and 75 took part in the study (completion rate 94%) (Table 1). Of the 75 participants who proceeded with the study, all completed the demographic survey, 71 (95%) completed the baseline surveys only, and 67 (89%) completed the baseline and follow-up surveys. Recruitment included 34 (45%) from Christchurch Hospital and 41 (55%) from Dunedin Hospital (Table 1).

Table 1

Demographic and patient-specific details from the overall cohort

Categorical variable	Group	Frequency N (%)
Gender	Male	30 (40)
	Female	45 (60)
Diagnosis	CD	43 (57)
	UC	30 (40)
	Don’t know	2 (3)
Ethnicity	NZ European	72 (96)
	Māori	3 (4)
Educational level	High school	29 (39)
	College	16 (21)
	University	11 (15)
	Post-graduate	19 (25)
Membership of an IBD support group	Yes	20 (27)
	No	55 (73)

CD Crohn’s disease, UC ulcerative colitis, SD standard deviation

Knowledge Survey Scores

Participants achieved an IBD-KID2 mean score of 10.9 (Standard deviation (SD) 2.4, range 5–15) and CCKNOW mean score of 11.9 (SD 4.4, range 3–22), which equate to mean percentage scores of 72.8 (SD 16.0, range 33–100) and 49.7 (SD 18.2 range 13–92), respectively. The association between patient-specific variables and knowledge scores achieved on each survey was examined (Table 2). Significantly higher scores for both CCKNOW and IBD-KID2 were achieved by females and participants with CD. Significantly higher scores on the CCKNOW were also noted for participants who were members of a support group. Level of education was significant overall for CCKNOW scores but not IBD-KID2 scores. When this variable was explored further, participants with a post-secondary education scored significantly higher than those with a high school education on both surveys. No significant association was seen between age at diagnosis and the CCKNOW (R 0.113, P 0.340), or IBD-KID2 score (R 0.197, P 0.098), nor age at the time of study completion for CCKNOW (R 0.15, P 0.20) or IBD-KID2 (R 0.11, P 0.34). A significant association was seen between time since diagnosis on the CCKNOW score (R 0.33, P 0.004), but not IBD-KID2 (R 0.08, P 0.50).

Table 2

Associations between patient-specific characteristics and knowledge survey scores

Categorical variable	Group	CCKNOW % score	P value	IBD-KID2% score	P value
Gender	Male	43.3		67.8
	Female	54.0	0.012	76.0	0.031
Diagnosis*	CD	56.0	0.001	77.8	0.001
	UC	41.1		65.5
Education level	High school	42.1	0.008	67.6	0.190
	College	54.4		76.7
	University	46.7		75.8
	Post-graduate	59.0		75.4
Education level	High school	42.1	0.003	67.6	0.029
	Post-secondary	54.5		75.9
Support group membership	Yes	57.1	0.034	78.0	0.088
	No	47.1		70.1
Order of form set	IBD-KID2 first	54.7	0.015	74.1	0.50
	CCKNOW first	44.6		71.5

*Excluding those who did not know their diagnosis, N = 2

Generalizability

Population differences between the Dunedin and Christchurch cohorts were explored to establish generalizability between the two centres and to examine any differences in knowledge survey scores. No significant population differences were seen between centres for the following variables; age at time of study (Mean difference (MD) −1.2 years, P = 0.75), age at diagnosis (MD −2.3 years, P = 0.48), time since diagnosis (MD 1.0 years, P = 0.71), gender (P = 0.49), or education level (P = 0.80). A significant difference was noted for the distribution of diagnoses between the two centres (P < 0.001), with the Christchurch cohort including 28 (82.4%) participants with CD compared to 15 (38.5%) in Dunedin, and in Christchurch 6 (17.6%) had UC compared with 24 (61.5%) in Dunedin. Christchurch had 16 (47%) participants who belonged to an IBD support group, and Dunedin had four (10%), which represented a significant difference (P < 0.001).

When the knowledge survey percentage scores achieved at each centre were examined, significant differences were found, with Christchurch participants scoring higher on both questionnaires (CCKNOW MD 14.6, P < 0.001; IBD-KID2 MD 13.3, P < 0.001). Due to the difference in distribution of diagnoses between centres, and the significantly lower scores of all participants with UC, the scores achieved on the CCKNOW and IBD-KID2 were controlled for centre and diagnosis. The results showed that there was no longer a significant difference between the centres (CCKNOW P 0.20; IBD-KID2 P 0.76). Due to the difference in distribution of support group membership for all participants, the percentage scores achieved on the CCKNOW and IBD-KID2 were controlled for centre and support group membership. The results showed that there was no longer a significant difference between the centres (CCKNOW score P = 0.31; IBD-KID2 score P = 0.65). Therefore, whilst the populations were significantly different for these two variables, IBD-KID2 was generalizable to the two centres when these were controlled for in the analysis.

Validity Testing

As hypothesized, the difference between the CCKNOW and IBD-KID2 mean percentage scores was significant (P < 0.005). Furthermore, the percentage scores of the CCKNOW and IBD-KID2 were significantly correlated at the individual level (Pearson R = 0.573, P < 0.005) (Fig. 1). This finding confirms concurrent validity of the IBD-KID2.

Fig. 1

Correlation between the CCKNOW and IBD-KID percentage scores

Reliability Testing

The repeat IBD-KID2 surveys were completed between 14 and 28 days after the baseline (mean 16 days), and scores showed no significant difference between the two time points (MD −0.2, P = 0.92) (Fig. 2), indicating that IBD-KID2 has good test–retest reliability in this population group. The ICC between the scores indicated moderate reliability (ICC 0.63, CI 0.4 to −0.77, P < 0.001). Cronbach’s alpha was 0.7 indicating good internal consistency. Notably, Cronbach’s alpha could not be improved by removing any individual items.

Fig. 2

IBD-KID2 test–retest scores at baseline and follow-up completion

Frequency of Correct IBD-KID2 Responses

The frequency of correct answers given to each individual item was examined, with poor knowledge being considered for those items where less than 50% of the cohort scored correctly (Fig. 3).

Fig. 3

Frequency of correct IBD-KID2 scores at baseline completion in 75 subjects with IBD

Acceptability

The results of the acceptability survey showed that 49 (69%) participants preferred IBD-KID2 to the CCKNOW, 60 (85%) found it easier to complete, but 38 (53%) considered the CCKNOW as most suitable for adults with IBD. Of the 75 completing the survey, 53 (71%) provided free text commentary. The most prevalent finding resulting from the free text analysis related to participants noting that IBD-KID was the easier questionnaire (n = 22). Participants also noted that IBD-KID2 uses less medical terms (n = 7), is more relevant (n = 2), appropriate (n = 2), and shorter (n = 2), as well as patient-centred (n = 1), more general (1) and it related to the participant (1). With regard to the CCKNOW, the most noted aspect was that it was considered to be complex and using more complex terms (n = 10). Participants also noted that CCKNOW had more questions/was longer (n = 2), excluded from items (n = 2), goes into greater detail/depth (n = 2), was more relevant (n = 1), relatable (n = 1), challenging (n = 1), interesting (n = 1), ambiguous (n = 1), broader (n = 1), and included more about treatments (n = 1). Five participants noted that they would like to learn the answers, and ten participants reported that they knew less than they realised. Suggested extra topics to be added to IBD-KID2 included nutrition (n = 7), drugs (n = 4), management (n = 3), where to get help (n = 3), symptoms (n = 2), anatomy (n = 2), mental health (n = 2), social aspects (n = 1), and comorbidities (n = 1).

Discussion

This study aimed to establish whether the knowledge assessment tool IBD-KID2 was appropriate for use among adult patients with IBD. Concurrent validity was established using score comparisons with CCKNOW, and reliability and generalizability confirmed using test–retest completion, and between centre comparisons. Associations between IBD knowledge and demographic and patient-specific variables were explored, as well as participant’s views on the acceptability of IBD-KID2. This study successfully established that IBD-KID2 is a valid and acceptable tool for assessing IBD knowledge in adults with CD or UC.

Concurrent validity of IBD-KID2 was confirmed by demonstrating a strong correlation between the CCKNOW and IBD-KID2 percentage scores. Two previous studies used this method when validating their knowledge questionnaires against CCKNOW, finding similar significant levels of correlation as in this study [15, 30].

In terms of frequency of correct answers, all but one item was correctly answered by more than 50% of participants in this study, indicating good overall knowledge. The item that did not reach the 50% threshold was regarding whether IBD in remission can slow down a young person’s growth. A lack of clear knowledge about the potential impact of IBD on growth has been reported for adults with IBD in the past [13, 31]. Of interest is that while the frequency of correct answers for the growth question was also low for children with IBD (<50%) [22], parents of children with IBD demonstrated good levels of knowledge [24]. The highest scoring items in this study were related to colonoscopy, stress, surgery and stopping drugs. Similar results were found as part of a recent study using IBD-KID2 to measure IBD knowledge among the general public where the questions about colonoscopy, stress, and surgery were also high scoring items (>50%) and were assumed to be scored well due to being relatable to other health conditions [25].

Associations between demographic and patient-specific variables in this study showed that participants who identified themselves as female, had a diagnosis of CD (vs UC), and completed post-secondary education, scored higher on both the CCKNOW and IBD-KID2. The finding that female patients have better IBD knowledge has been demonstrated in a number of previous studies [11, 18, 32], as well as their greater desire [33] and need for information [12]. This gender difference may be explained by findings noted by Selinger et al. [11], who reported an association between female gender and higher levels of anxiety, as well as a positive relationship between IBD knowledge and anxiety. A number of studies have shown that patients with CD have a higher level of IBD-specific knowledge than patients with UC [14, 15, 34]. It has been suggested that this difference may be due to the fact that CD can follow a more severe disease course [19] and require more frequent treatment for disease complications, providing the opportunity for more frequent interaction with clinicians, which may facilitate knowledge acquisition [16]. It is well established that patients with IBD with a higher educational status also show higher disease-specific knowledge [14, 17, 18, 34], a finding further supported in this study. When interpreting this result, the role of health literacy should be considered, with the possibility of higher knowledge scores being due to a better understanding of medical terminology [24].

In this study, participants scored significantly higher on the CCKNOW if they were a member of a support group, a trend found in other studies using IBD-KID2. This difference may be due to membership facilitating access to IBD information and resources and more interactions with fellow patients [14, 18]. It has also been suggested that people who join a patient organisation may be more motivated to learn and get actively involved with disease management [14, 31]. Since patient organisation membership is a modifiable factor in increasing patients’ IBD knowledge, IBD healthcare teams should encourage patients to join a support group [17, 30].

Another important aspect of this study was related to the perceived acceptability and feasibility of IBD-KID2. Most participants preferred IBD-KID2 and found it easier to complete than the CCKNOW, however, approximately half of the participants considered the CCKNOW as most suitable for adults with IBD. This finding may be due to the CCKNOW being considered as more difficult [20, 35] and therefore, more appropriate for adults, as well as the fact that the CCKNOW includes items on pregnancy and male fertility. A number of participants noted that they would like to know the answers to the questions and that they knew less than they realised. The desire for more disease-related knowledge has been noted in the past [14, 32], as well as the realisation that knowledge was at a lower level than expected [18]. Interestingly, patients tend to overestimate their knowledge levels, with one study showing that participants’ ratings of their personal knowledge level decreased considerably from 46.4 to 22.3%, after completing a comprehensive IBD knowledge questionnaire [18].

In terms of additional items that should be included in IBD-KID2, the top four mentioned were items on nutrition, drugs, disease management and where to get help. Most of these topics have been highlighted in the past by patients with IBD as being very important to receive information about [33, 36]. One study showed that a significant proportion of patients receive little to no information on certain subjects including long-term prognosis, managing pain and other symptoms, and changes to diet [33]. This finding highlights the importance of providing additional information to patients regarding issues not covered by the questionnaire, if being used to establish gaps in patient’s knowledge. While the internet has gained momentum as a source for disease-related information, especially with younger patients (under the age of 50) and patients with a higher educational level [36], the doctor is still noted as the most important, frequent and preferred source of information [17, 33, 36, 37].

Strengths

This study used a well-established method to confirm validity of IBD-KID2 and utilised a long-standing knowledge questionnaire for comparison. The study had high response and completion rates, and the addition of an acceptability survey provided an extra dimension of data for comparison between the two surveys. Performing the study in two different centres provided a representative sample of the adult IBD population in New Zealand.

Limitations

The main limitation of this study relates to the wider generalizability of IBD-KID2 outside New Zealand for the population of adults with IBD. While ethnicity data on participants were collected, few identified as any group except NZ European, which may have limited generalizability within this study, as well as to the broader population of adults with IBD in other countries. Wider generalizability of IBD-KID2 for use among children with IBD has previously been established [23], and while no additional issues relating to use for adults with IBD are anticipated, further work may address this limitation. The study had a limited sample size, but adds to the growing data being accumulated using IBD-KID2 and may therefore be used for future comparisons among the population of adults, adolescents, and children with IBD. The questionnaires were self-administered and completed by participants without the presence of a researcher, which may have resulted in higher knowledge level scores due to participants looking up relevant information when completing the questionnaire. However, maximum scores on IBD-KID2 were achieved by few participants, and none for CCKNOW, so this appears unlikely. Furthermore, this study did not collect data on variables including family history of IBD, surgical history, medical treatments and IBD hospitalisation history, which have shown an association with IBD knowledge levels in the past [15, 17, 18, 34, 37].

Conclusion

In summary, IBD-KID2 may now be considered for use in clinical or research settings to evaluate IBD knowledge levels of adult patients with IBD as it has been established as a valid and reliable tool for this purpose. It may be used to determine gaps in understanding, which in turn may be used to drive clinical interventions or provide targeted teaching. In terms of future research, using IBD-KID2 to assess knowledge levels of adults with IBD can further explore the role this plays in relation to self-management abilities and patient well-being. It may also be used as a tool to evaluate programs developed to enhance these components.

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 383 KB)