Flare-IBD: development and validation of a questionnaire based on patients' messages on an internet forum for early detection of flare in inflammatory bowel disease: study protocol.

INTRODUCTION: Crohn's disease and ulcerative colitis, the two major forms of inflammatory bowel disease (IBD), are chronic disabling conditions characterised by flares followed by periods of remission. However, patients with IBD are seen every 3-6 months in the outpatient clinic, and the occurrence of a flare between two outpatient visits is not captured. To our knowledge, there is no validated patient-reported outcome (PRO) tool to measure the phenomenon of flare in IBD. This study aimed to use an innovative methodology to collect messages posted by patients in an internet forum for developing and validating a PRO measuring flare in IBD. METHODS AND ANALYSIS: The design involves (1) computer engineering sciences for scraping extraction of messages posted in an internet forum and for identification of messages related to flare; (2) qualitative methods for thematic content analyse of the messages posted, for candidate items generation, for items selection (Delphi process) and for items adjustment ('think-aloud' interviews) and (3) quantitative methods for psychometric validation of the PRO. ETHICS AND DISSEMINATION: Ethical approval was obtained from the Comité de Protection des Personnes (CPP) CPP Nord-Ouest I (19.07.15.44139) in November 2019. The project aims to provide a tool to evaluate IBD flare in current medical practice that is constructed with patients' perspectives. Items generation from a source corresponding to exchanges in an internet forum is an innovative method in this field and provides a wider coverage of qualitative data. If such a forum can result in interesting material, then this could be a new methodological perspective for generating items for questionnaires. Findings will be reported and disseminated widely through international peer-reviewed journal publications, oral and poster presentations at scientific conferences. TRIAL REGISTRATION NUMBER: NCT04180345. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

To our knowledge, there is no validated patient-reported outcome tool to measure the phenomenon of flare in inflammatory bowel disease.

Generating items via a qualitative approach ensures content validity, an innovative methodology to collect perspectives of patients posted in an internet forum seems of interest.

The design involves computer engineering sciences and mixed methods (quantitative and qualitative).

An expert patient is also integrated into the scientific committee and will participate in all decisions adopted by the committee at each stage of the project.

In the absence of a reference methodology, the developed method will be exploratory and aim to reveal a baseline methodology.

Introduction

Crohn’s disease and ulcerative colitis, the two major forms of inflammatory bowel disease (IBD), are chronic disabling conditions characterised by flares followed by periods of remission. In the context of treat-to-target strategies and tight monitoring,1 detecting flare early is the only way to change patients’ lives and disease course. However, patients with IBD are seen every 3–6 months in the outpatient clinic, and the occurrence of a flare between two outpatient visits is not captured.

Patient-reported outcome (PRO) measures developed in the field of IBD include the Inflammatory Bowel Disease Questionnaire (IBDQ)2 3 and its shorter version, the Short IBDQ.4 This tool provides a quality of life measurement for four dimensions: intestinal disorders, systemic symptoms, emotion and impact on social life. However, it is not suitable to measure the specific phenomenon of flare in IBD. The IBD-Control questionnaire, Treatment Satisfaction Questionnaire for Crohn’s disease and Satisfaction for PAtients in Crohn’s diseasE Questionnaire are relevant to evaluate patients’ perceptions concerning proposed treatments.5–7

These questionnaires evaluate the degree of activity at a point in time, without any evaluation of symptomatology evolution or recent exacerbations characteristic of flares. Hence, a questionnaire integrating the patient’s point of view and following US Food and Drug Administration guidance8 is needed. This questionnaire will include a broader perspective than gastrointestinal symptoms to consider elements like, for example, pain, tiredness, physical symptoms other than gastrointestinal, psychological impact and social impact. In 2009, a study of focus groups involving patients with ulcerative colitis indicated that patients reported 15 symptoms usually considered in clinical indicators to evaluate disease evolution but also reported 14 other symptoms not considered in these indicators. Also, in talking about the flare phenomenon, patients did not discuss 11 symptoms included as clinical indicators.9 Consequently, the clinical indicators usually considered are not sufficient to grasp the phenomenon of flare in IBD. To our knowledge, there is no validated PRO tool to measure the phenomenon of flare in IBD.

Generating items via a qualitative approach ensures content validity (item relevance and accurate reflection of patients’ perspectives).10–13 Individual interviews and focus groups are the two predominant methods used to collect qualitative data.14 Most of the time, IBD is diagnosed in young people between 20 and 30 years of age. Thus, an innovative methodology to collect perspectives of patients posted in an internet forum seems of interest, particularly in a young population familiar with current communication media.

This study aimed to use an innovative methodology to collect messages posted by patients in an internet forum for developing and validating a PRO measuring flare in IBD.

Methods and analysis

Design

The design involves computer engineering sciences and mixed methods (quantitative and qualitative) (figure 1).

Figure 1

Prospective dates and general design of the development and validation of the Flare-IBD questionnaire. IBD, inflammatory bowel disease.

Computer engineering sciences

Step 1: scraping extraction of messages posted in an internet forum

Patients’ testimonies will be collected from the Association François Aupetit (AFA) internet forum (https://www.afa.asso.fr/forum/forum.html). The AFA, with 25 000 members and supporters, is a unique French organisation in IBD, recognised for its public utility. The Lorraine Research Laboratory in Computer Science and its Applications (Loria team), specialised in natural language processing and knowledge discovery, will perform a scraping extraction process for messages posted on the AFA internet forum. Scraping is performed using the Scrapy web crawling tool combined with Splash, a JavaScript rendering service, all running under Python. The complete execution depends mainly on the network speed and takes less than 0.5 hour.

Step 2: identification of messages related to flare

Five healthcare providers (HCPs) and 20 patients identified by AFA will each collect 50 randomly assigned extracted messages in a database. Participants will indicate whether the message corresponds to the flare phenomenon in IBD or not. If the message positively matches the flare phenomenon, then participants will highlight excerpts from the text they consider significant flare markers. A total of 1250 messages will be distributed in this step (50 different messages per participant). No guidance will be provided to the participants to let them free to consider all aspects they want from flare in IBD. From these results, the Loria team will develop a supervised learning algorithm to recognise and extract messages about the flare. A test phase will help ensure that retained messages effectively reflect the flare phenomenon in IBD and that messages addressing issues other than flare are eliminated.

Qualitative methods

Step 3: thematic content analysis of messages posted on the AFA website

In a specific interpretative approach, thematic content analysis will involve discovering themes relating to flare in IBD and quantifying their emergence.15 Data analysis will involve using NVivo QSR V.11 to help structure and organise detected themes.

Step 4: candidate items generation

In accordance with the themes found in step 3, items will be generated as close as possible to the spontaneous language used among patients to speak freely about flare on the forum. We assume that this way we will access a natural language used by patients with or without proximity to medical jargon.16.

Step 5: Delphi process for items selection

The expert panel will include HCPs (clinicians, nurses and psychologists) and patients. Participants will be consulted individually and electronically. Thus, individual patients will be able to express their own point of views without being influenced by other participants.17 As recommended in the literature, the experts will evaluate the relevance of items on a 4-point Likert scale.18 Only the most relevant items will be retained. The online survey tool LimeSurvey will be used.

Step 6: ‘think-aloud’ interviews

In individual interviews lasting from 60 to 90 min, patients will be invited to discuss their thoughts about the items as they arise. A complete and a rich dataset will be collected on how the patient reacts to, understands, analyses and answers each item. The key aim during think-aloud interview is to encourage participants’ verbal report on items with typical encouragements such as ‘do not forget to say out loud everything that comes into your head’, ‘keep going’. The interviews will be recorded and transcribed. Content analysis will involve using NVivo QSR V.11 software. Collected qualitative data will be organised not by the addressed theme but by the item discussed by patients. The ‘think-aloud’ aspect represents a debriefing step for the newly developed items and the final possibility to adjust the tool content before psychometric validation.14 19

Quantitative methods

Step 7: psychometric validation

Psychometric properties will be analysed by using classical test theory for dimensionality and item response theories for scale calibration as recommended by COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN).20

For the classical test theory, parameters analysed will be acceptability (amount of missing data, floor and ceiling effect (> or <15%)), construct validity (exploratory factor analysis, discriminant validity by the Kruskal-Wallis test) and internal consistency (Cronbach alpha >0.70).

For item response theories on a Rasch model, parameters analysed will be unidimensionality (principal component analysis of the residuals) for each identified domain, local dependence (residue correlation >0.3), adequacy of items and person, interaction item trait (χ2 tests (expected non-significant for good adequacy to the model)), internal consistency (person separation index >0.85), distribution graph of item thresholds and person, and differential item functioning.

Convergent validity will be calculated from data on C-reactive protein level and/or faecal calprotectin content and/or MRI and/or endoscopy data collected independently of the project until 30 days before or after the Flare-IBD administration. All IBD studies considered that objective signs of disease activity are stable during 60 days.21 22 The Harvey-Bradshaw Index (HBI) for Crohn’s disease and the Simple Clinical Colitis Activity Index (SSCAI) for ulcerative colitis could be added. The HBI is a widely used score assessing clinical activity. A flare will be defined by HBI ≥4 and one objective sign of inflammation (C-reactive protein level >10 mg/L or calprotectin level >250 μg/g or ulcerations seen on MRI) for patients with Crohn’s disease and by SSCAI >2 and (calprotectin >150 μg/g or endoscopy Mayo subscore 0–1) for patients with ulcerative colitis.1

The aim is to test the convergence of Flare-IBD scores with objective biological markers and clinical markers used in routine clinical practice. As Flare-IBD scores will reflect patients’ perspective, the mismatch between Flare-IBD scores and biological/clinical markers could be the result of a poor choice of indicators. But we do not have others to try to establish convergent validity, and no one is available for establishing criterion validity regarding a flare occurred before the medical encounter.

Reproducibility will be tested by a second Flare-IBD administered 8 days later.

All analyses will involve using SAS V.9.4 for Windows (SAS Institute) and RUMM 2030.

Participants

Criteria for inclusion

Every adult patient consulting the gastroenterology unit of the Nancy University Hospital with a confirmed IBD diagnosis, regardless of the patient’s state or treatment, will be considered for inclusion.

Criteria for non-inclusion

We will exclude patients with a diagnosis <3 months and protected persons (minors, adults under guardianship, pregnant or breastfeeding women, people living in public health or social institution, patients in an emergency situation and incarcerated individuals).

Sample size and process of recruitment

Steps 3 and 4 will be conducted by the research team.

Step 5: for the Delphi process, 25 HCPs and 25 patients will be recruited. HCPs will be recruited from the French network of IBD specialists. Patients will be recruited from the health education programme for IBD management at the Nancy University Hospital and among members of the AFA.

Step 6: for the ‘think-aloud’ interviews, depending on the number needed to reach saturation, up to 10 interviews will be conducted with patients,23 24 that is, to obtain sufficient data to account for all aspects of the phenomenon of interest. Saturation is achieved when concepts and subconcepts cannot be further specified with additional interviews. Patients will be recruited from the IBD unit at the Nancy University Hospital.

Step 7: concerning the number of participants, COSMIN recommendations to satisfy the proprieties of a Rasch model are more demanding than are those for structural analysis (principal component analysis and correlation).25 Thus, the requirements level for the Rasch model will be applied and 200 patients will be recruited for step 7.

Reproducibility will be tested by a mailed questionnaire after 8 days. The collection of 60 questionnaires is sufficient to calculate a precise and interpretable intraclass coefficient.

A clinical study technician will be in charge of monitoring clinical nurses who will (1) administer the Flare-IBD in the hepatogastroenterology unit in the Nancy University Hospital and (2) collect biological, endoscopic, medical imaging and clinical data obtained in routine clinical practice. Then, nurses will propose that patients complete the Flare-IBD in the waiting room; this recruitment modality is also interesting to test the portability of the questionnaire in routine clinical practice.

Data gathered from each patient’s medical file will include sex, age, type of IBD and IBD duration. An identification number will be attributed to the patient. A separate database will be created as a correspondence table containing the patient identification number (previously attributed), name, first name and postal address.

Multidisciplinarity in the scientific committee

The Flare-IBD project is multidisciplinary. Indeed, a psychologist, an engineer specialised in natural language processing and knowledge discovery, two epidemiologists and a gastroenterologist are included in the scientific committee.

The Flare-IBD project is the subject of a partnership contract between the three competent supervisory institutions: Nancy University Hospital (the psychologist, two epidemiologists and gastroenterologist), the Loria (the engineer) and the AFA (the expert patient). Moreover, the project benefits from the support of the REsearch in Clinical epidemiology and Public health network, particularly the group ‘Perceived Health Measurement’. This group meets on a regular basis four times a year. The project will be systematically added to the agenda for the group to brainstorm every technical, scientific and ethical aspect.

Patient and public involvement

The project benefits from originality because an expert patient (CD) is also integrated into the scientific committee and will participate in all decisions adopted by the committee at each stage of the project. This patient is a coauthor of this article and will be a coauthor of each paper resulting from the project. Therefore, the patient’s point of view is an essential part of the Flare-IBD development.

Ethics and dissemination

Ethics approval and consent to participate

Patients expressed their experience in the AFA forum. These accessible messages without any identification are under the current public register and thus the content can be exploited to generate scientific knowledge. Apart from the legislative dimension, ethical aspects are also under consideration. Therefore, qualitative data from the forum will be analysed with a high degree of exigency concerning confidentiality and anonymity preservation.26 Also, legal notices in the AFA forum and the information sheet to read before participants post the first message state that the AFA reserves the right for the association and for their partners to use the corpus to ‘show that patients have a great role to play in research and knowledge development in the field of IBD’.

Ethical approval was obtained from the Comité de Protection des Personnes (CPP) CPP Nord-Ouest I (19.07.15.44139) and written informed consent will be obtained from participants before data collection.

Written consent for publication will be obtained from participants at the same time as content to participate (single document).

Dissemination

Practical implementation input

The project aims to provide a tool to evaluate IBD flare in current medical practice that is constructed with patients’ perspectives. Therapeutic intervention that is limited to patients with a flare confirmed by an outpatient visit has failed to alter the natural history of IBD as it can take several weeks before the patient gets an appointment with a gastroenterologist. IBD flares can occur at any time between two outpatient visits and are unpredictable. There is a well-known disconnect between symptoms and intestinal inflammation in patients with IBD.27 28 International guidelines now recommend a tight monitoring of both symptoms and intestinal inflammation in these patients to allow early detection of IBD flares and thus early intervention, with the final aim of preventing disability and disease progression (bowel damage, hospitalisations and surgeries).29 30 However, patients with IBD are seen every 3–6 months in case of active disease and every 6–12 months during the remission phases. Hence, tools allowing tight monitoring of patients with IBD outside these scheduled outpatient visits are eagerly awaited.

Methodological input

Individual interviews and focus groups are the two predominant qualitative methods used to collect the perspectives of patients to generate questionnaire items. Items generation from a source corresponding to exchanges in an internet forum is an innovative method in this field and provides a wider coverage of qualitative data.

Some barriers could be encountered and will be considered (no control of message content and no management of criteria for inclusion and non-inclusion of the author of the messages). Internet forums create an exchange space with no supervision that allows for substantial qualitative data collection probably closer to patients’ concerns. Hence, in the absence of a reference methodology, the developed method will be exploratory and aim to reveal a baseline methodology. If, as we believe, such a forum can result in interesting material, then this could be a new methodological perspective for generating items for questionnaires.

Dissemination plan

Findings will be reported and disseminated widely through international peer-reviewed journal publications, oral and poster presentations at scientific conferences.