Evaluation of the satisfaction and usefulness of a web-based educational program for breast cancer patients.

The purpose of this study was to evaluate the effectiveness of a web-based breast cancer educational program which consists of special features such as flash animations and online counseling as well as 7 different categories of information on breast cancer. The effectiveness of the program was analyzed in terms of its function and content. A total of 147 women with breast cancer who visited the website for at least 30 minutes and a minimum of 3 visits, participated in the survey.In the satisfaction evaluation of web-based educational program, usefulness of information, system efficiency, adequacy of information and convenience of use all received positive evaluation and showed even distribution of 49.14 (+/-6.05) points out of 64 points total. In the usefulness evaluation, the subcategories had following scores from the highest to the lowest; understanding of breast cancer was 3.34 (+/-0.51), life after treatment (3.21+/-0.58), early detection and examination (3.20+/-0.60), chemotherapy and hormonal therapy (3.18+/-0.55), related factors and prevention (3.16+/-0.59), treatments (3.13+/-0.53), and diagnosis (3.02+/-0.56). Factors affecting the satisfaction of the program were age, religion, income, stage of disease at diagnosis, source of health information, duration of Internet usage, and whether the patient performs breast self-examination. Factors that affect the usefulness of the program were religion, period since diagnosis, source of health information, frequency of Internet usage, recurrence of breast cancer, and family history.Although the program was evaluated as somewhat useful and satisfactory, it should be improved upon by providing in-depth and cutting edge breast health information especially for women from a higher educational and income background.

INTRODUCTION

The number of breast cancer cases in Korea has been increasing significantly, and it has become one of the most common types of cancer in women, making up 16.1% of all cancer incidence in 2001. It is reported that most Women with Breast Cancer (WBC) felt abandoned by health professionals when the treatment was over, and they needed assistance in the transition from being a cancer patient to a cancer survivor [1]. This phenomenon is supported by many Korean research studies [2-4] which indicate that WBC are eager to obtain breast health information, primarily in the areas of prevention of recurrence, etiology, diagnostic tests, and prognosis, as well as symptom management, diets, exercises, and alternative therapy [2, 3]. This finding is consistent with the study from 102 WBC investigating the key educational and counseling needs of WBC in Seoul area [4]. The highest, educational need was to know about the method of recurrence prevention. In the pre-admission period, etiology diagnostic tests, treatment and prevention of the breast cancer were needed the most. During the admission period, symptom management after the operation had the highest score. After discharge, prevention of recurrence had the highest score. The results of this research indicate that an educational program should be focused on the medical and symptom management information based on their treatment stages.

Recently, information and education provided through Internet has been actively used to provide emotional support for the patients and their families, as well as helping them in their decision of choosing a health care provider [5]. The most helpful aspect of the Internet is that the user can seek information at the most convenient time and place for them. Through education via Internet, patients can gain significant medical knowledge, self-care and bio-clinical goals can change positively, and the quality of life improved [6].

Although some studies [7, 8] indicate a negative impact of Internet use, such as decreased rates of family communication, decreased social contact, and greater loneliness, other studies demonstrate that Internet use offers greater social support. Its positive impact was demonstrated in the studies focused on the need of specific types of individuals, for example, parents of children with developmental disabilities [9], individuals with isability [10], caregivers of Alzheimer's disease patients [11], widows experiencing bereavement [12], and elderly individuals residing in assisted and independent living facilities [13].

The effect of Internet use was also demonstrated in cancer patients with various diagnosis [14, 15], and breast cancer patients in particular [16, 17] Fogel et al. [18] indicated that 42% of breast cancer patients use the Internet as a source of information in the United States.

A recent survey in Korea reported the number of Internet user as almost 2.7 million, accounting for 64.1% of the population in 2003 [19]. Among them the number of women was about 1.3 million. The age distribution was highest in their 30s and 40s, and most of them (71.6%) used the Internet to search for information. The mean number of hours spent on the Internet, per week was 12.2 hours. However, well organized Internet sites about health care, breast health in particular, have not been developed to a great extent in Korea. Thus we strongly believe that web-based educational program would be very effective and efficient in supporting WBC in Korea.

Objectives

The primary goal of this study is to develop and test a web-based educational program for WBC in Korea. A web-based program was chosen not only because Korean women are reluctant to discuss breast healthcare publicly [4] but also because Koreans have relatively high access to high-speed Internet at home.

Assess the educational needs of WBC in Korea.

Construct and develop a web-based educational program based on these needs. The program will include a self-learning program using decision tree model.

Evaluate the satisfaction and usefulness of the web-based educational program.

METHODS

1) Development of the Website

To achieve the objective number one, the analysis of the necessity and need of breast cancer related web contents geared towards breast cancer patient was administered from May 2005 to July 2005. Based on this study of general needs, a web-based educational program was developed to achieve the objective number two. The breast cancer related contents of the website were decided to consist of understanding breast cancer, etiology and prevention, early detection and prognosis, symptom management, chemoradiotherapy and hormone treatment, as well as life after full recovery. In order to have distinguishing features compared to already existing websites, and effectively achieve the educational goals, the website developed Flash animation as means of displaying its contents. Prior to uploading them to the website, the rough drafts of the Flash animation were submitted to breast cancer Internet patient community and edited and improved according to their critique. The finished menu of the website consisted of Introduction, Cancer Summary, Breast Cancer Information, Expert Consultations, Patient’s Community, Family’s Community, FAQ, Recent Research Directions, Cancer Survivor’s Stories, and Links (Fig. ). For the Expert Consultations, medical experts from various University hospitals provided their insights. The website premiered on March 19th, 2006 and reopened after edit and improvement on May 1st, 2006, providing information to the patients who signed up through the administration process. Finally on March 19th, 2007, the content became accessible to general public without the login process (http://www. kbccc.org/).

2) Measurement Tools

(1) Items Measuring Satisfaction of the Website

A questionnaire consisting of total 16 questions was developed to evaluate the functions of website, with the highest possible score being 4.0 per item. The subcategories consisted of 2 items of System Efficiency, 3 items of System Convenience, 1 item of Design, 7 item of Adequacy of Information and 3 items of Usefulness of Information. The lowest score possible for each item is 1.0, and highest being 4.0 that makes the total sum of the questionnaire from the lowest score of 16 points to the maximum of 64 points. The reliability score of the measurement tool was Cronbach’s alpha 0.92.

(2) Items Measuring Usefulness of the Website

There are total of 7 questions evaluating the contents of the website and the maximum of 4.0 points indicate the highest satisfaction. The subcategories consisted of cancer chemotherapy and hormonal therapy, and the lowest possible score being 1.0 and the maximum of 4.0 points. The lowest possible score for the evaluation is 7 points with the highest possible score being 28. The reliability score of the measurement tool was Cronbach’s alpha 0.83.

3) Data Collection

Among the people who visited Korea Breast Cancer Cyber Center (KBCCC), the subjects were those who logged in more than 3 times with total visiting period of 30 minutes or longer, and those who consented to participate in the study. It was decided to include the participants who are eligible for the criteria, because the volume of the contents could be understood only by the sufficient frequency and time, namely over 30 minutes through no less than 3 times. There were 497 users according to the access log, however, those who were unmet the criteria were excluded for the study. As it was not sufficient to recruit participants only by contacting e-mails and it was necessary to recruit as many participants as possible, telephone and mail contact were chosen. The subjects were contacted through phone call and gave verbal consent before they were given written consent form along with the questionnaire via postal mail. The total number of people contacted via phone call was 286 and out of them 49 did not answer their phone calls, 74 declined to participate in the study. 163 people agreed to participate in the studies on the phone and out of them 147 people successfully returned their finished questionnaire form.

4) Data Analysis

The collected data were statistically analyzed using the SAS System for Windows Version 8.0 by following methods:

The analysis on the subjects’ specific demographics, satisfaction and usefulness of educational program were done by t-test and ANOVA.

The variables affecting the evaluation of web based educational program was analyzed through multiple regression analysis (Stepwise Regression).

The reliability of instrument was examined by obtaining Cronbach’s alpha value.

RESULTS

1) Demographics and Characteristics Related to Breast Cancer and Internet Utilization

The demographics and characteristics related to breast cancer and Internet utilization are summarized in Table . The average age of subjects is 46.6 years old ranging from 29 to 65. Forty three (29.5%) of the test group replied that they do monthly breast self-examination, and 17.1% percent replied that they never do breast self-examination. Forty one point two percent reported that they participate in breast cancer self-help group and 28.1% replied that they participate at least once every one or two months. Eighty one point nine percent of them replied that they get information of breast cancer through Internet, and 16.0% answered ‘through Internet patient community’ and 6.3% replied that they obtain information through their peers. Also magazines, books and newspaper each served as source of information for 3.5% of the group. Ninety nine point three percent of the patients answered that they use Internet, 65.5% answered that they have been using Internet for more than 3 years, and 22.1% had Internet experience of 1 to 3 years. Sixty percent of them answered that they use Internet more than 5 days a week, and their average time spent on the net was 86.4 minutes ranging from minimum of 12 minutes to maximum of 480 minutes. Eighty six point nine percent answered that they use Internet at home and 95.4% replied that they reference health information available online.

2) Satisfaction of Web-Based Educational Program

‘Usefulness of information’ scored average of 3.17 (±0.41) out of 4.0 scoring the highest, and then came the ‘system efficiency’ scoring at 3.15 (±0.53), ‘adequacy of information’ 3.07 (±0.46), ‘system convenience’ 2.99 (±0.45) and ‘design’ was scored the lowest with 2.90 (±0.52). Total score showed a relative high and even distribution of 49.14 (±6.05) out of 64. The lowest scoring subcategory was ‘adequacy of information’ (2.75±0.67) and the highest scoring subcategory was ‘useful information in decision making related to breast cancer patients’ (3.33±0.51) (Table 2).

Table 2.

Satisfaction of Web-Based Educational Program

Items		n	Mean	SD	Min	Max
System efficiency	Easy to access	146	3.24	0.57	1	4
	Fast to search	145	3.06	0.60	1	4
	Mean	145	3.15	0.53	1.5	4
System convenience	Have contact number	142	2.93	0.57	1	4
	Easy to use the construct of program	143	3.02	0.54	1	4
	Friendly to use	144	3.05	0.57	2	4
	Mean	143	2.99	0.45	1.3	4
System design	Good design	145	2.90	0.52	1	4
Adequacy of information	Correctness	144	3.06	0.56	2	4
	Currency	142	2.92	0.60	1	4
	Necessity	144	3.33	0.51	2	4
	Credibility	144	3.29	0.53	2	4
	Clearness	142	3.20	0.55	1	4
	Sufficiency	143	2.75	0.67	1	4
	Detail	143	2.93	0.64	2	4
	Mean	142	3.07	0.46	2	4
Usefulness of information	Have lots of breast cancer-related information	144	3.26	0.54	2	4
	Useful information in decision making related to breast cancer patients	143	3.10	0.49	2	4
	Easy to understand information given	144	3.17	0.47	2	4
	Mean	143	3.17	0.41	2	4
Total score		143	49.14	6.05	32	64

3) Usefulness of Web-Based Educational Program

In the evaluation of usefulness of web-based educational program, out of 4.0 scale, understanding of breast cancer scored the highest with average of 3.34 (±0.51), then came life after treatment (3.21±0.58), early detection and examination (3.20±0.60), chemotherapy and hormonal therapy (3.18±0.55), related factor and prevention (3.16±0.59), treatment (3.13±0.53), and diagnosis (3.02±0.56) (Table ). Almost each of the categories of the satisfaction and usefulness scores are correlated from 0.163 to 0.699.

4) Influencing Factors on Satisfaction of Web-Based Educational Program

By implementing stepwise regression, the factors that affect the satisfaction of web-based educational program are age, religion, income, stage of disease at diagnosis, source of information, duration of Internet usage and whether the patient performs breast self-examination. It showed that subjects in their 40s, Christian, having income of 4 million won and those who used Internet for 3 years or longer showed positive evaluation. On the other hand, if the source of information about breast cancer was magazine, books or newspaper, the evaluation returned negative. The R2 value of this model was 0.296 (F=6.15, P<0.0001) (Table ).

5) Influencing Factors on Usefulness of Web-Based Educational Program

By implementing stepwise regression, the factors that affect the usefulness of web-based educational program are religion, period since diagnosis, source of information, frequency of Internet usage, recurrence of breast cancer and family history (R). It showed that if the subject is Christian or has other religion, was diagnosed more than one year but less than 3 years ago, rely on Internet as their primary source of information, uses Internet daily, has family history of breast cancer in mother or aunts, then the benefits of web-based educational program were positive. However, in the case of subjects of recurrent breast cancer, the effect was negative (Table ).

The influencing factors are somewhat different between satisfaction and usefulness, besides the religion and the source of information.

DISCUSSION

The purpose of this study was to analyze the acceptance level of an Internet website designed to help those with breast cancer. Recently, through the increased in popularity of Internet websites and its information sharing network and emotional support, there have been many new Internet patient communities with lively activity. Along with the increase in usage of blogs, mini-homepage, online clubs and communities, there have been an increased need for consumer’s participation and common sharing. This sheds light on how important it is to take the health information consumer’s needs into account when developing Internet health information website.

When Internet first became popular it was widely thought that majority of Internet users were male demographic who work in offices or have professional careers. However, data shows that women, especially in their 30s and 40s, use Internet more frequently specifically in terms of searching for Internet health related information [20]. To extrapolate this trend, we can predict that the usage of Internet website for breast cancer patient will only increase with time. In this study we showed that women in their 40s, who have higher income and have been using the Internet for more than 3 years, were the most satisfied by the website. This is because highly educated, high income demographics tend to rely more highly on Internet and use it more frequently, and also this group has higher expectation of quality regarding Internet health information. The recent development of Internet including the more user-friendly web 2.0’s debut has added to the previous functions, resulting in higher reflection of health information consumer’s needs and consideration of functional upgrades.

The study showed that the subjects evaluated the websites beneficially, but agrees that there can be more done in terms of the amount of content. Online patient communities that focus on a specific disease tend to demand the most in-depth information available. There are many websites that have general health information on the web, however in terms of the depth, they do not satisfy the demands of health information consumer. Hence, even those online patient communities that are well received need to keep up their quality control and provide contents that are rich in both quantity and quality.

According to one study [2], there has been frequent usage of Internet health information abroad. However due to language barrier and socio-economical reasons, Koreans living abroad have harder time approaching the medical systems in their residential countries and also if the health information website is in English, it makes it even harder for the Korean speakers which resulted in higher usage of health information website in Korean. Hence, since Internet allows information sharing between all countries, this also allows the Korean immigrants overseas to access the Internet health information websites that are developed within Korea. This shows the usefulness of Korean version of foreign websites and also the websites developed for this study specifically. As the issues around web-based education are not unique to Korean patients, the findings and clinical implication of developing and utilizing the health information websites will be applicable to the users globally. As this website is a sister site to the well-known Breast Cancer website provided in English, it could be extended to the global users by developing the English version of the flash educational materials and posting and linking them to the site.

To conclude, it is highly imperative for the development of educational websites that are sensitive to the needs of health information consumers and especially the online patient communities that are specific to certain conditions.

CONCLUSION

This study aimed to evaluate the effectiveness of the web based educational program by developing a website for breast cancer patients and analyzing the function of educational program and contents by surveying the women with breast cancer and 147 people replied.

Each category of Usefulness of information section showed high scores. The highest scoring subcategory was ‘Useful information in decision making related to breast cancer patients’ (3.33±0.51). Out of total 64 points, it showed a relative even distribution of 49.14 (±6.05) points.

Whether the demographic characteristics influence the educational effectiveness of Web-based educational program was verified. The result showed that all other factors except for the frequency of Internet usage have no relevance, and higher the frequency of Internet usage, the higher the effectiveness of the system scored.

The factors that influence the satisfaction of web-based educational program are age, religion, whether the subjects perform breast self-examination, income, stage of disease at diagnosis, source of health information, and the duration of Internet usage. The R2 value of this model was 0.296 (F=6.15, P<0.0001).

The factors that influence the usefulness of web-based educational program are religion, period since diagnosis, source of information, frequency of Internet usage, recurrence of breast cancer, and family history (R2 = 0.270, F=5.45, P<0.0001).

Through this study, there will be many useful improvements in terms of richness of resources and useful functions in the web-based educational program geared towards breast cancer patient in the future. For example, to improve the satisfaction of the users could be accomplished by developing customized contents by age groups and adopting features that are used more conveniently by age specific cognitive and motor skills. The authors would like to have the follow-up research to test whether women who have used the website are better off by measuring whether they are better informed long-term, or assessing whether they check themselves more effectively and so on. The research design would be post-treatment and long-term outcomes evaluation.

Table 1.

Demographics and Characteristics Related to Breast Cancer and Internet Utilization (N=146)

Variables	Categories	n	(%)	Mean	SD	Min	Max
Age (years)	29-39	20	(13.7)	46.6	6.8	29	65
	40-49	77	(52.7)
	50-59	45	(30.8)
	60-69	4	(2.7)
Marital status	Single	13	(8.9)
	Married	133	(91.1)
Education	Elementary	2	(1.4)
	Middle school	7	(4.8)
	High school	60	(41.1)
	College	61	(41.8)
	Graduate school	16	(11.0)
Religion	Protestant	47	(32.2)
	Buddhism	33	(22.6)
	Catholic	27	(18.5)
	None	37	(25.3)
	Others	2	1.36
Occupation	Homemaker	91	(62.3)
	Commercial/business	1	(0.7)
	Government official	11	(7.5)
	Professionals	12	(8.2)
	Company worker	11	(7.5)
	Others	20	(13.7)
Income	<100	11	(7.5)
	100-199	22	(15.1)
	200-299	29	(19.9)
	300-399	33	(22.6)
	400-499	18	(12.3)
	≥500	33	(22.6)
Residence	Seoul and big city	88	(60.3)
	Small and medium-sized city	51	(34.9)
	Rural area	7	(4.8)
Menarche	Before 12 yrs	9	(6.3)	14.4	1.4	11	19
	After 12 yrs	135	(93.8)
Menopause	20-29 yrs	2	(2.0)	45.5	5.7	22	56
	30-39 yrs	8	(7.8)
	40-49 yrs	71	(69.6)
	50-59 yrs	21	(20.6)
Breast self examination	Once a month	43	(29.5)
	Sometimes	78	(53.4)
	Never	25	(17.1)
	No	92	(63.0)
Methods of breast cancer diagnosis	Routine exam	24	(16.4)
	Occasional exam	21	(14.4)
	Mass palpation	94	(64.4)
	Others	7	(4.8)
Period since Dx (yrs)	<1	40	(27.4)	2.1	2.3	0	13
	1-2	74	(50.7)
	3-4	20	(13.7)
	5-9	9	(6.2)
	≥10	3	(2.1)
Stage of disease at Dx	0	9	(6.2)
	1	51	(35.2)
	2	54	(37.2)
	3	26	(17.9)
	4	0	(0.0)
	Others	5	(3.5)
Type of treatment
Mastectomy	No	70	(48.0)
	Yes	76	(52.1)
Lumpectomy	No	74	(50.7)
	Yes	72	(49.3)
Chemotherapy	No	22	(15.1)
	Yes	124	(84.9)
Radiotherapy	No	63	(43.2)
	Yes	83	(56.9)
Hormonal Tx	No	63	(43.2)
	Yes	83	(56.9)
Breast reconstruction	No	140	(95.9)
	Yes	6	(4.1)
Other diseases	No	116	(79.5)
	Yes	30	(20.6)
Participation to self-help group	1/1-2mon.	41	(28.1)
	1/3-6mon.	8	(5.5)
	1/12mon.	9	(6.2)
	Sometimes	2	(1.4)
	No	86	(58.9)
Source of information
Broadcasting	No	139	(96.5)
	Yes	5	(3.5)
Books, newspaper	No	139	(96.5)
	Yes	5	(3.5)
Internet	No	26	(18.1)
	Yes	118	(81.9)
Self-help group	No	121	(84.0)
	Yes	23	(16.0)
Others	No	135	(93.8)
	Yes	9	(6.3)
Internet utilization	No	1	(0.7)
	Yes	145	(99.3)
Duration of Internet use	<6 mon	7	(4.8)
	6-11 mon	11	(7.6)
	12-35 mon	32	(22.1)
	≥36 mon	95	(65.5)
Frequency of Internet use	Daily	87	(60.0)
	3-4/wk	28	(19.3)
	1-2/wk	21	(14.5)
	1-2/mon	9	(6.2)
Duration of Internet use/day (min)				86.4	81.7	12	480
Place of Internet use	Home	126	(86.9)
	PC room	18	(12.4)
	Workplace	1	(0.7)
Experience of use of health information online	Yes	137	(94.5)
	No	8	(5.5)
Recurrence of breast cancer	Yes	7	(4.8)
	No	137	(94.5)
	Don’t know	1	(0.7)
Metastasis of breast cancer	Yes	10	(6.9)
	No	136	(93.2)
Family History	Mother or mother’s/ father’s sister	5	(3.4)
	Sisters	5	(3.4)
	Cousins	1	(0.7)
	None	132	(90.4)
	Others	3	(2.1)

Table 3.

Usefulness of Web-Based Educational Program

Items	n	Mean	SD	Min	Max
Understanding of breast cancer	145	3.34	0.51	2	4
Related factor and prevention	144	3.16	0.59	1	4
Early detection and examination	144	3.20	0.60	1	4
Diagnosis	143	3.02	0.56	1	4
Treatment	143	3.13	0.53	2	4
Chemotherapy and hormonal therapy	144	3.18	0.55	1	4
Life after treatment	145	3.21	0.58	2	4
Total score	143	22.27	2.77	15	28

Table 4.

The Influencing Factors on Satisfaction of Web-Based Educational Program (N=146)

Factors	B	F	P	95% CI
				LL	UL
Age group: 40-49yrs	2.67899	9.89	0.0021*	0.87318	4.4848
Religion: Christian	3.50845	12.63	0.0005*	1.57456	5.44235
Breast self-exam: sometimes	-2.2156	7.32	0.0077*	-3.99116	-0.44004
Income: 400,000-499,000 W	2.88675	3.96	0.0487*	0.15204	5.62147
Stage of disease at Dx: others	-7.67685	10.16	0.0018*	-12.63088	-2.72282
Type of treatment: radiotherapy	-2.12141	6.21	0.0140*	-4.02414	-0.21869
Source of information: broadcasting	-3.48934	4.62	0.0334*	-6.9034	-0.07527
Duration of Internet usage ≥3yrs	2.42706	4.42	0.0374*	0.43095	4.42316

P < 0.0001.

Table 5.

The Influencing Factors on Usefulness of Web-Based Educational Program (N=146)

Factors	B	F	p	95% CI	Factors
				LL	UL
Religion: Christian	1.41608	9.02	0.0032*	0.51556	2.3166
Religion: Others	6.54765	12.55	0.0005*	2.89931	10.19598
Period since diagnosis: longer than 1 year and shorter than 3 years	1.14509	6.89	0.0097*	0.30356	1.98661
Source of information: Internet	1.8514	11.21	0.0011*	0.77777	2.92504
Frequency of Internet usage: daily	1.03529	5.92	0.0163*	0.16074	1.90984
Recurrence of breast cancer: yes	-2.3431	5	0.0271*	-4.40404	-0.28216
Family history: mother or parent’s sister	2.51074	4.78	0.0305*	0.23668	4.7848

P < 0.0001.