Prototype evaluation of a self-management Internet diary for patients with ulcerative colitis.

OBJECTIVE: To evaluate content, navigation, usability, and impact measurability of a prototype Internet-based self-management intervention for patients with ulcerative colitis.
MATERIAL AND METHODS: Analysis of 52 Internet diaries that were used in a six-month test trial. Analysis was done using an evaluation framework for eHealth applications that incorporates goals from theory and empirical studies on living with chronic illness, the software design industry, and health services research.
RESULTS: Content of the diary covered the intended functions of the Internet-based self-management intervention. The evaluation led to several refinement suggestions concerning navigation, usability, and impact measurability of the Internet diary.
CONCLUSION: Psychosocial, medical, and scientific content as well as interface and design are equally important in the development of effective eHealth interventions.

Introduction

In recent years, professionalization of Internet-based health interventions has taken place. Although numerous eHealth interventions have been designed for different chronic diseases,1 we know of only one that was designed for patients with ulcerative colitis (UC).2 UC is a chronic inflammatory illness that is immunologically mediated and of unknown etiology. UC is an inflammatory bowel disease (IBD) and causes intermittent inflammation of the large intestine (the colon). During an inflammation or ‘flare up’, symptoms vary from cramp and abdominal pain to diarrhea and rectal bleeding. Many patients experience long remissions. The functional, psychological, and social impact of UC on the life of patients influences disease course3 and induces an impaired quality of life (QOL)4,5 as well as psychological distress.6

The one Internet-based disease management intervention on IBD that we know of was carried out by Cross and Finkelstein and aimed at enhancing therapy adherence by asking patients to report weekly on disease-related issues to the physician using a simplified laptop that was installed at their house. Patient motivation for therapy adherence and initiative for change of illness behavior derived from a sense of being watched over by a physician.2 We designed a prototype Internet-based intervention for UC patients that, conversely, ensues from a paradigm of self-management to enable patients to watch over themselves.

Objective

The objective of this study was to evaluate and refine a prototype Internet-based self-management diary by systematically analyzing its design in a six-month test trial by 52 UC patients.

Material and methods

Patient selection and characteristics

Gastroenterologists from eight Dutch university medical centers and regional hospitals invited outpatients with UC visiting the hospital to test our prototype Internet-based self-management diary for six months. Seventy-eight patients were included. Fifty-two provided access to their diaries for analysis.

Mean age was 40 years (standard deviation [SD] 10.5), 51% were female, 19% had a low level education (eight years or less formal education), 36% medium level education (9–12 years), and 45% had higher education (13 years or more), reflecting a somewhat skewed distribution towards higher levels of education. Recruiting gastroenterologists professed to having been somewhat biased in selecting patients who were on average higher educated and more comfortable with working with computers.

Generic QOL was assessed with the most widely used QOL scale, the Short-Form 36 (SF-36).7 Disease-specific QOL was assessed with the Inflammatory Bowel Disease Questionnaire (IBDQ).5,8

Evaluation structure

Pagliari developed an evaluation framework for eHealth applications that incorporates goals from both the software designing industry as from health services research.9

We used the first evaluation cycle of this framework for analysis of our Internet-based self-management diary. This cycle is called “evaluation of concepts and prototypes”, and is illustrated in Figure 1.

Figure 1

First cycle from Pagliari’s framework for evaluating eHealth resources at different stages of development and implementation.9

Based on the themes of this cycle, we formulated the following research questions:

What theory preceded the development of the Internet-based self-management diary? (box: Generate ideas, etc; Figure 1),

What are the intended functions and specific requirements of the Internet-based self-management diary? (box: Define, etc; Figure 1),

What are the general features and outlook of the Internet-based self-management diary? (box: Develop, etc; Figure 1),

What are the six-month test results of the Internet-based self-management diary, focusing on its usability? (box: Test, etc; Figure 1).

Following questions 1–4, what suggestions for refinement can be given for further improvement of the Internet-based self-management diary? (box: Refine, etc; Figure 1).

In the following Results section, we attend to research questions 1–4. Because refinement suggestions (research question 5) have the character of implications, they are incorporated into the Discussion.

Results

First research question: Theoretical concepts to be included in the diary

Two gastroenterologists and one medical psychologist agreed on including the following five theoretical concepts in the Internet-based self-management diary.

Self-management

Patients that are chronically ill have to find an answer to two important questions. The first is disease-specific: what medical treatment works for me and what does not? The second takes up life as a whole: how to deal with illness in daily life, at work, at home, in my mind? Achieving optimal treatment and coming to terms with the disease in daily life does not happen overnight, it is a process of trial and error and demands maintenance. It is a process of learning effective self-management. Barlow and colleagues defined self-management as “the individual’s ability to manage the symptoms, treatment, physical and psychosocial consequences and lifestyle changes inherent in living with a chronic condition”. Effective self-management encompasses the “ability to monitor one’s condition and to effect the cognitive, behavioral, and emotional responses necessary to maintain a satisfactory quality of life”.10

Making personal notes

The challenges that chronically ill patients face make keeping notes very helpful. Chronological, regular notes on symptoms, medication and its results, enable patients to figure out what treatment worked and what did not. It also enables patients to be precise and complete in their communication with their physician. Their physician is then able to give optimal advice and help fine tune patients’ self-management. Besides this, the act of writing forces a patient to organize thoughts and clarify emotions which in itself, brings relief. This effect is described in literature under expressive writing.11

Quality of life

Besides making notes, regular assessment of physical, mental, and social elements that influence life, gives patients a framework to make changes and improve the quality of that life. Quality of life assessment is done best by short, generic, and disease-specific questionnaires.

Self-promoting elements

Starting a diary is not difficult, but keeping it regularly and long enough to profit from it is difficult. We should therefore profit from possibilities that the Internet offers to stimulate patients to become and stay active by using techniques such as a regular automatic email reminder and preformatted diary elements that are quick and easy to fill.

Impact measurability

With the Internet-based self-management instrument, we aimed at enhancing self management and by this, enhancing QOL. To measure QOL, two valid and well tried instruments for UC patients are available: the IBDQ and the SF-36. The IBDQ is a disease-specific health status instrument containing 32 questions.8 The SF-36 is a generic health survey used in many patient groups and healthy populations.7 Table 1 depicts the results of these QOL assessments, which reflect scores comparable with reference groups of outpatients with UC.5,7,8

Table 1

Quality of life scores at baseline on Inflammatory Bowel Disease Questionnaire (IBDQ) and Short Form 36 (SF-36) (mean ± standard deviation)

IBDQ
Bowel symptoms	59.7 ± 6.9
Systemic symptoms	26.6 ± 4.2
Emotional function	70.0 ± 6.2
Social function	32.9 ± 3.0
Total	190.1 ± 16.4
SF-36
Physical functioning	86.8 ± 15.4
Role-physical	70.2 ± 40.5
Bodily pain	87.3 ± 18.3
General health	57.8 ± 20.6
Vitality	63.4 ± 15.9
Social functioning	81.9 ± 15.8
Role-emotional	91.4 ± 21.0
Mental health	80.4 ± 13.0

Second research question: intended function and requirements of the diary

Three intended functions that the Internet diary should serve were derived from the minutes of the discussions by research group members when designing the study. The study by Robinson and colleagues guided these discussions.12

The Internet diary must provide self-management possibilities for patients with UC and by this provide opportunities for enhanced QOL

The Internet diary must incorporate self-promoting elements that stimulate patients to use the diary

The clinical impact of the Internet diary must be measurable

Third research question: Features and outlook of the diary

We answer this research question by analyzing the Internet-based self-management diary.

Individual self-management medication plan and internet-based monitoring

The Internet-based self-management intervention encompassed two elements: the distribution of a written, individual, and guided medication plan for each participating UC patient and access to a personal Internet diary for self-monitoring.

The guided medication plan was inspired by a study by Robinson and colleagues.12 In our variant study, patients received an individual self-management medication plan on how to respond to a flare-up of UC. This consisted of various treatment strategies with dosage schemes of medication in increasing doses, representing a ‘step-up’ approach. This was explained to the patients by the gastroenterologist and written down by the physician for the patient on a leaflet to take home.

In addition, password-protected access to a personal environment within a patient empowerment Web site with information on IBD was given, followed by instructions. In the personal environment of the Internet-diary, patients were asked to add disease-specific information in predesigned formats and to answer two brief, electronic questionnaires on generic and disease-specific QOL every two weeks for six months. Scores on the questionnaires were automatically translated into a graph that was visible in the diary. The diary also provided a space for notes. When patients neglected to fill out the diary, a reminder was sent by email. After the six-month study period, the diaries remained accessible for use by the participants.

Schematized impression of the internet-based self-management diary

Figures 2 and 3 provide a precise and schematized overview of the diary’s content, functionalities, and navigation.

Figure 2

Functionalities and navigation of the internet diary.

Figure 3

Simulated screenshot of what a patient sees when he navigates to his ‘Overview of disease-related data’ (= M5 in Figure 2).

Fourth research question: Usability results of the diary after six months

Test results focusing on usability were examined using screenshots of all diary pages of the 52 participants. These were analyzed by systemizing users’ input in categories and where relevant, by making basic addition sums. The most basic usability question is: Did patients really use the diary at all?

Because users were not obliged to biweekly register nonactivity in the exacerbation, medication, and contact with physician fields, we could not check if an empty field was due to nonactivity in these areas or to neglecting to register disease-related developments.

Because we know all patients used medication, a dedicated user should have at least made one note in the medications field. Of the 52 participants, 37 did and 15 did not make any notes on medication use and its results. The diary as constructed was not detailed enough to pick up relapse in the illness condition or self-initiated medication changes.

Frequency of completing the two brief QOL questionnaires was, of all fields, the best measurable field in answering the question if the diary was actually used. Both questionnaires were programmed such that they could only be filled with a minimal interval of two weeks. After six months, therefore, a nonuser would have no entries and a fully dedicated user would have a maximum of 13 data entries in six months (26 weeks). Figure 4 shows this number of completed biweekly sets of QOL questionnaires of all 52 users.

Figure 4

Number of completed bi-weekly sets of quality of life questionnaires.

Note: X-axis, number of completed bi-weekly sets of quality of life questionnaires; Y-axis, number of users.

A notable dichotomy can be observed: a fairly large group (55%) was dedicated to filling both brief questionnaires, filling them 10–13 times, with a notable peak amongst users that never missed a single one. Another opposite and relatively large group (30%) filled little or no questionnaires (two times or less). The remaining group, the ‘average users’ was small in size, filling the diary 3–9 times (15%).

Only three of the 52 participants added a note in the ‘Personal Notes’ space during the six-month study period. However, another diary space not designed for this purpose was used by nine patients to enter personal notes (48 in total). When asked at the end of the study period, some 90% of the users indicated that they found the Internet diary ‘adequate’.

Usability is more than answering the question: “Did participants use the diary?” Another important usability issue is: did the Internet diary, in how the different fields were presented to the user graphically, do what we wanted it to do? Did it actually provide users with self-management possibilities? These questions will be addressed, together with refinement suggestions, which are incorporated into the Discussion because they have the character of implications.

Discussion

Analyzing our prototype diary has led to important suggestions on how Internet-based self-management diaries can be improved in order to offer patients with chronic illness a means to self-manage their illness. The five stages in Figure 1 are used as framework for the Discussion.

Practical refinements

These refinements are a result of what was found and described in Figures 2 and 3.

Login should take one instead of three pages to complete,

Users should remain logged in to the diary until the Web page that accommodates the diary is left,

The personal Internet diary should include a fixed navigation bar,

The ‘Personal Notes’ page should be visible in the ‘View disease-related data’ Web page (M5 in Figure 2) and accessible with a button in a fixed navigation bar. During and after the study period, only three out of 52 users wrote down a personal note in the personal notes field. The fact that nine users individually decided to make personal notes elsewhere in the diary (see comment in Figure 2) indicates that users did feel a need to write down experiences. Therefore, the lack of clear navigation to the personal notes field has been a factor for participants not or hardly using it,

In fixed sized data boxes, the allowable amount of characters and the nature of the theme that the box is used for should be geared to one another (see comment in Figure 3).

Refine the ability to measure how often the diary was used

When measuring clinical impact of the Internet diary, it is important to know whether the Internet diary was really used biweekly. For several fields in the diary, we could not check if an empty field was due to temporal nonactivity in these areas or to neglecting to register disease-related developments. We do know that 29% of the users did not enter any data on their medication use, a core theme within the self-management plan. We also found that only three notes were made in the ‘Personal Notes’ page. A fairly large group (55%) did fill most of the two brief questionnaires, but another well sized group (30%) did not fill the questionnaires more than once or twice in six months. The patient group that was dedicated to use the diary as intended was small. The design and programming of the Internet diary made a clean measure of convenience impossible. The refinement suggestion would be to improve just that. This could be realized by incorporating all diary items into one intelligent, personalized, and user-friendly digital survey that a user is prompted to fill in every two weeks. In this digital survey, a new item should only appear on screen if the previous item is processed correctly. This results in fully completed diaries that would make a measure of convenience possible.

Refinements on usability

To enhance self-management, a diary user will especially be helped if s/he is able to find tendencies and connections such as: ‘Did that exacerbation strike back on my quality of life in that period?’, ‘Have my exacerbations diminished since I switched to that new medication?’, ‘Has the personal medication plan that the gastroenterologist made for me worked?’, etcetera. From the layout of the diary as it is, it was difficult for users to find those tendencies and connections: overview information was spread over three individual Web pages, which hampered finding connections. In addition, information on what happened when was organized for each field individually, either by vertical enumerations of self-reported dates, or horizontally by week numbers. This made it hard to bring chronology into combined information fields to search for or to discover relationships. We suggest creating a central disease overview page on which all fields are organized around one horizontal time line. Besides the central overview page, a few secondary overview pages could be designed to provide the user with calm and focused information per theme.

Refine the ability to measure clinical impact

The two short electronic questionnaires used in the biweekly diary, were designed to provide the user with an indication on QOL, and not for use as scientific data. Replacing the two brief questionnaires with short validated electronic questionnaires would allow use of the outcomes as a clinical impact measurement, as well as a QOL indication to the user. Electronic QOL assessment will only increase in the future; it suits the character of the Internet diary and feasibility is being tested with promising results.13,14 Hjortswang and colleagues evaluated a disease-specific instrument that might be useful, the Short Health Scale.15 An alternative for the generic questionnaire could be the Short Form-12, a condensed version of the SF-36.7 When more elaborate QOL measurements are desired, one could consider building an electronic version of the SF-36 and IBDQ into the Internet diary to appear to the user once every six months with an explanatory comment. If the digital survey that we promoted earlier in this paragraph would also contain incorporated calculable features, the diary would gain even more scientific possibilities as an instrument with several measurable variables.

Refinement suggestions from literature

Anhøj and Nielsen studied an Internet-based monitoring tool for asthma and concluded that using mobile phones instead of computers increases use of the tool.16,17 A refinement suggestion would be to use a handheld computer (personal digital assistant [PDA]) for biweekly data entry, in combination with an Internet environment where users can overview their state-of-affairs at any time. Compliance to the Internet-based disease management intervention for UC patients by Cross and Finkelstein2 (see Introduction) was much higher than ours. We think the involvement of an external professional in their study design triggered patient desire but also a sense of responsibility to be compliant to the system. A final refinement suggestion derives from recent work by Sorbi and colleagues.18 They advise personalizing Internet interventions. By doing so, “one engages the user, which may increase exposure and empowers self-care by tailoring resources, choice and control to the individual”.18 Personalizing the Internet diary could consist of a registered choice in the frequency in which patients enter their disease-related data, ranging from once, twice, three, or four times a month. Because UC patients often experience long remissions, such a degree of freedom might offer patients a more suitable self-check frequency. Another way of personalizing would be to incorporate tools by which users can give their Internet diary a personal look and feel, which is an obvious draw in commercial networking Web sites like MySpace and Facebook. A recent paper in Patient Preference and Adherence illustrates how “full functional capacity can be maintained with good control (in patients with another chronic illness that requires self-management, ie, diabetes).”19 Our study shows that self-management with an Internet diary seems feasible.

Conclusion

A refined version of our Internet-based self-management diary would encompass the following characteristics, which are highly relevant in future research.

The internet diary offers self-management possibilities for patients with UC and this provides opportunities for enhanced QOL

Users login easily to their personal diary on the Internet. The Internet diary has a fixed navigation bar, users know where they are and where they can go at all times. The Web page is used to overview disease-related outcomes, whilst a handheld computer (PDA) is used to enter the disease-related data collected in the diary. In the overview section, information on exacerbations, execution of and changes of the personal medication plan, time and reason for seeing a physician, notes and QOL curve are conveniently arranged on a single summarizing Web page. All items on this page are organized around one horizontal timeline. Besides this central overview, a few Web pages within the Internet diary provide the user with calm and focused information per field. Together, all Web pages make patients recognize relations between illness behavior, treatment and QOL outcomes, and encourage patients to make changes that optimize QOL.

The internet diary incorporates self-promoting elements that encourage patients to use the diary

Ulcerative colitis patients answer a digital survey of disease-related questions in a fixed frequency of once, twice, three, or four times a month. Patients choose the frequency. The digital survey is administered using the handheld computer (PDA). Because the survey is programmed intelligently using information from previous entries, questions are personalized and if possible couched in a multiple choice format. An automatically generated email and/or prompt by PDA reminds users when it is time to complete the next survey.

In the beginning, users receive feedback on their data entries by a health professional. This feedback is aimed at reinforcing patients’ desire to keep using the diary on their own. Feedback frequency decreases in time and eventually stops. Survey outcomes are visible on users’ personal Internet diary. This diary has features that can give the diary a personal look and feel.

The clinical impact of the Internet diary is measurable

Research on the effect of the Internet diary is carried out in a randomized controlled design. The surveys that users complete biweekly serve the patients as well as the researcher. In the process of answering questions by the user, a new question only appears if the previous question was processed correctly. This way users cannot ignore answering a certain survey question. Answers on relevant fields are stored on a central computer as numerals of variables for scientific calculations. The short QOL questionnaires in the digital survey are validated for research and thus useful as well for the patients as in scientific research. More elaborate QOL questionnaires are also administrated using digital technology.