Suitability assessment of health education brochures in Qassim province, Kingdom of Saudi Arabia.

BACKGROUND: Health education is the cornerstone of primary health care. Health education materials distributed to the community should, therefore, be suitable and effective. The purpose of this study was to evaluate the health education brochures, designed and disseminated by Ministry of Health institutions in the Qassim province.
MATERIALS AND METHODS: The study was a cross-sectional review of health education brochures. We used a structured evaluation form, comprising general information on the brochures and a modified Suitability Assessment of Materials (SAM) score sheet. The SAM consisting of 22 criteria in six groups, includes content, literacy demands, graphics, layout/typography, learning stimulation/motivation, and cultural appropriateness. SAM criteria categorize written material into "superior," "adequate" and "not suitable." Two qualified consultant family physicians evaluated the brochures. Data were analyzed using Epi Info version 3.4 statistical package.
RESULTS: We evaluated 110 brochures, the majority of which addressed chronic health conditions such as mental health, diabetes mellitus and hypertension. Seventy-four (67.3%) brochures were evaluated as "adequate," 34 (30.9%) as "not suitable" and 2 (1.8%) as "superior." "Cultural appropriateness" was the highest scoring factor, with 92 (83.6%) brochures falling into either the "superior" or "adequate" category. With regard to "content," 88 (80.0%) brochures fell into either the "superior" or "adequate" category. This was the second highest scoring factor. Graphics was the factor that scored the least. Seventy-five (68.2%) brochures were rated in this factor as "not suitable."
CONCLUSIONS: Although two-thirds of our brochures were considered "adequate," the majority needed improvement to their graphics and learning stimulation factors. We recommend that guidelines for designing health education brochures should be formulated to improve the quality of health education brochures.

INTRODUCTION

Health education is the cornerstone of primary health care. It improves patient's knowledge,[1] compliance to treatment and self management of disease.[2] Health professionals usually educate patients verbally. However, the patients tend to forget this verbal information.[3] Therefore, to increase the effectiveness of health education, information given to patients verbally should be supplemented and reinforced with written materials.[45] Moreover, research has demonstrated that patients appreciate the usefulness of written materials.[67]

Some advantages of written materials are consistency of message, ready availability of the required information, and ability to share this information.[28] These materials can be used to underpin the information obtained verbally from health care providers.[9] However, to influence health behaviors, it is important to ensure that health education materials are suitable for their target audience. To maximize their suitability, a number of factors, such as content, layout, and cultural appropriateness of these materials should be taken into account.[1011] In spite of a range of benefits of written material,[12] the means of acquiring health information are changing in this era of advancing electronics; people rely more on electronic devices such as television and internet. Therefore, printed materials should be made to suit the audience by making them attractive and useful.[13]

The development of effective health education materials requires formative research as well as evaluation, which systematically obtains information to be used to improve these materials. Written education materials in the form of posters and brochures are common in the Kingdom of Saudi Arabia (KSA). Brochures are used as important tools of intervention in health education programs.[14] However, we were unable to find any study conducted to determine the suitability of written health education materials disseminated in KSA in the literature review. Consequently, it is expected that there is a lack of awareness of the factors that make written education material suitable.

A number of health education brochures are developed and disseminated in the Qassim province. However, the evaluation of these brochures for their suitability is not conducted in a systematic manner. The purpose of our study was to formally evaluate the available health education brochures, designed and disseminated by government health care institutions in the Qassim province, KSA.

MATERIALS AND METHODS

This was a cross-sectional review of health education brochures developed by the Ministry of Health (MOH) institutions in the Qassim province. Qassim, a central province of KSA, has 174 health care facilities belonging to Saudi MOH.[15] These facilities provide healthcare to more than a million people. The majority of these health care facilities carry out health education activities, including the distribution of written health education materials.

We invited all health care institutions under the MOH, in the Qassim province to send written health education materials available in their facilities for evaluation. An official letter was issued to all these institutions to this effect. All health education materials received were classified as leaflets, booklets, brochures and posters. Only health education brochures were selected for evaluation. We defined a health education brochure as a single page, two-sided, folded document designed to communicate health-related information to the community or patients. The brochures were further classified as two-fold, three-fold and four-fold. Only brochures in Arabic were included in this study. All the brochures were available to the public and distributed free of charge. A total of 112 brochures were received. Two brochures were excluded because they were not produced by MOH institutions.

Structured evaluation form composed of two sections, was designed and used as a research instrument. The first section of the evaluation form asked for general information while the second section was the Suitability Assessment of Materials (SAM) score sheet. Important information about certain features required for effective written materials was collected.[1617] This information included the title of the brochure and credentials of the authors, publication date, and citation for the sources of the contents.

Suitability Assessment of Materials developed by Doak et al.[18] is considered a comprehensive instrument for the assessment of health education materials.[19] SAM is used by researchers for the evaluation of printed materials,[38] as well as web-based ones.[20] The SAM instrument validation has been conducted with health care workers of various cultures.[18] The SAM uses six factors to assess the suitability of materials: Content, literacy demands, graphics, layout and typography, learning stimulation/motivation and cultural appropriateness.[18] There are 22 criteria within these six factors. For example, the “content” factor includes the criteria: Purpose, behavioral content, scope and summary. Each criterion is scored on a maximum of two points and a minimum of zero point. A score of two points is rated as “superior,” one point as “adequate,” zero point as “not suitable.” If the criterion does not apply to the material, it is labeled as “not applicable” (N/A). Scored items are summed up and the percentage (excluding N/A items) is calculated. The material is then classified according to percentage as either superior (70-100%), adequate (40-69%) or not suitable (0-39%).

Minor modifications were made to the SAM criteria to customize it to Arabic. The SAM criteria use readability formulas to assess the “reading level” of written material for the factor on “literacy demand.’ As readability formulas are not available for Arabic, this item was omitted. Finally, 21 SAM criteria instead of 22 were evaluated and scored. Amendments were also made in the criteria for “typography.” The criteria of ‘uppercase and lowercase serif or sans serif, and “no all caps for long headers or running text” were omitted, as they are not applicable to Arabic typography. Font size of at least 14 points was used as a criterion, instead of size 12 standard set by SAM criteria. Leonard Doak and Cecilia Doak, authors of the original SAM, were consulted through E-mails about the modifications in the SAM instrument.

We prepared written guidelines for the evaluation. The evaluation form was pretested. Two qualified consultant family physicians from the local community who were trained in a workshop arranged for this purpose did the evaluation. Each brochure was independently evaluated and scored by each evaluator.

Data were entered and analyzed using Epi Info version 3.4 statistical package (Centers for Disease Control and Prevention, Atlanta, Georgia, USA). The mean of percentages by the two evaluators, of all six SAM factors and overall SAM percentage were used to designate the categories of “superior,” “adequate” and “not suitable” to each brochure. For individual criterion, the scores assigned by the two evaluators were summed up and the categories re-defined. The criteria which attained four points were labeled as “superior,” while those attained two or three points were labeled, “adequate” and those with zero or one point were labeled as “not suitable.”

RESULTS

A total of 110 brochures were evaluated, out of which 59 were three-fold, 45 were two-fold and six were four-fold. Of the brochures evaluated, 75 (68.2%) were designed and disseminated by hospitals while 35 (31.8%) were made by various departments in the Public Health Administration, Qassim. Table 1 displays the general information on these brochures. The most common subject written on was mental health (21.8%), followed by chronic diseases (12.7%), and the least common addressed topic was ophthalmology (1.8%). Although the institution's name was mentioned in all (100%) brochures, the names of the authors (36.4%) and reviewers (5.5%) were mentioned in only a small proportion of the brochures.

Table 1

Salient features of evaluated health education brochures (n=110)

Seventy-four (67.3%) brochures scored as “adequate,” 34 (30.9%) were “not suitable” and two (1.8%) were rated as “superior” [Figure 1]. Overall, the average SAM score for all brochures was 44.3% (range: 20.0% to 70.0%), which is considered “adequate” based on the SAM ratings. The suitability rating, across the six SAM factors, are displayed in Table 2, while the suitability rating for individual criteria of the SAM factors are shown in Table 3.

Figure 1

Suitability rating of the health education brochures in Qassim (n = 110)

Table 2

Rating of SAM factors for health education brochures in Qassim (n=110)

Table 3

Rating of individual criterion of SAM factors for health education brochures in Qassim (n=110)

The content factor in the majority (74.5%) of the brochures was categorized as “adequate.” A large proportion (92.7%) of the brochures stated their purpose, which warranted the classification as “superior” or “adequate” according to SAM scores. A succinct summary to emphasize important “take home” messages was absent in almost all brochures resulting in the rating of 107 (97.3%) brochures as “not suitable.” Overall, the average SAM score for the “content” factor of all brochures was 48.9% (range: 13.0-75.0%), which placed them in “adequate” category.

The “literacy demand” factor in 54.5% of the brochures was categorized as “adequate;” only one (0.9%) brochure was rated as “superior.” Overall, the average SAM score for the “literacy demand” factor of all brochures was 42.0% (range: 7.0-75.0%), which put them into the “adequate” category. The brochures were largely written in a passive, third-person using medical terms, thus rendering almost half (50.9%) of them “not suitable” with regard to style of writing.

Approximately, two-thirds (68.2%) of the brochures were rated as “not suitable” in graphics. Overall, the average SAM score for the “graphics” factor of all brochures was 25.9% (range: 0.0-75.0%), which fell into the “not suitable” category. Only 3 (2.7%) brochures had covers with graphics that were rated as “superior.” These cover graphics had illustrations that stated the purpose of the brochure clearly. A sizable proportion of brochures (63.7%) had illustrations that made them “not suitable.”

Approximately, two-thirds (64.5%) of the brochures were categorized as “adequate” in “layout and typography.” Overall, the average SAM score for this factor was 48.7% (range: 17.0-92.0%), which fell into the “adequate” category. Most brochures used an appropriate font size (14 point or larger). However, in some brochures pages were text-dense, and watermarks used as background illustrations made the text difficult to read.

More than one-third (41.0%) of the brochures were categorized as “not suitable” in the “learning stimulation and motivation” factor. Overall, the average SAM score for this factor was 39.9% (range: 0.0-75.0%), which fell into the upper limit of the “not suitable” category. None of the brochures used active interaction, and very few used passive interaction such as question and answer format for headings and subheadings. Few brochures focused on practical, behavior-modifying steps for the reader; 11 (10.0%) brochures scored “superior” in this category.

The cultural appropriateness of the majority (82.7%) of the brochures was “adequate.” Overall, the average SAM score for the factor for ‘cultural appropriateness’ of all brochures was 48.3% (range: 25.0-75.0%), which fell into the “adequate” category. On the whole, the brochures were culturally sensitive and did not have any negative cultural image. However, most had no photographs that reflected the target audience.

DISCUSSION

The brochures in this study addressed a variety of topics including mental health, chronic diseases and infectious diseases. Some topics such as mental health and cardiac health were addressed more frequently, as the institutions with these specialties had specific budgets for health promotion activities.

Author's credentials are considered important to assure readers that the information provided was credible since the authors had the appropriate expertise in the field.[21] The majority of our brochures (63.6%) did not give the name of the authors or their credentials. However, all of them displayed the institution's name as well as logo of the MOH. This display of the name of the institution and logo of MOH endorsed the information as credible.

A record of the date of publication is important to assure readers that the information provided was up-to-date. Brochures published within the past 2-3 years are considered up-to-date.[21] However, a study conducted in the United Kingdom found that some general practices were using information leaflets that were 6-13 years old for the education of patients.[22] In our study, only a small proportion (15.5%) of brochures mentioned the date of publication, which made it difficult to determine if they were current.

In this study, the average SAM score for all brochures was 44.3%, which is considered “adequate” on the SAM ratings. In an evaluation of the written education materials for patients with prostate cancer, the mean overall SAM rating was “adequate.” However, average scores were higher (63.3%) in comparison to our study.[23]

Our study found that approximately two-thirds (67.3%) of brochures fell into the “adequate” category. This finding is similar to an evaluation of 31 information leaflets, 64.5% of which were rated as “adequate.”[24] However, in that study, 6 (19.4%) brochures were rated “superior” in contrast to only 2 (1.8%) that were considered “superior” in our study. In another study that evaluated 29 education materials, the researchers found that 22 (75.8%) scored “adequate” for their overall suitability, while SAM suitability scored six as “superior” (20.6%) and one (3.4%) as “not suitable.”[23] In an evaluation of 21 printed educational materials on human papillomavirus (HPV), the average SAM score was 37.4%, which is considered “not suitable.”[25] In another assessment of 35 brochures and eight worldwide web education materials for neurology patients, 14% of the materials were “superior,” 58% were “adequate” and 28% were rated as “not suitable.” In that study, well-known organizations had brochures with an average rating in the “adequate” category.[26]

A concise summary, interactive format, and motivation content help people to adopt the desired behavior.[9] In our study, few brochures (13.6%) were rated “superior” in the criteria for behavioral content. Most of the brochures mainly had scientific information, but no practical guidance for behavior change. Other studies have also found similar limitations in the written health education materials.[23] In our study, 97.3% of brochures had no summary of the contents which is more than was found in another study in which 86% of the materials were without a summary statement.[23] An evaluation of 101 written documents for novel H1N1/09 influenza published on the CDC website, showed that not a single document had a summary.[27]

Many of our brochures needed to be more focused on the information which was in the category of “needed to know,” rather than “nice to know.” Other evaluators also found that the “nice to know” information obscured the “need to know” information in the written materials.[28] Another area of considerable weakness in our brochures was the absence of interactive techniques such as question and answer format. Other researchers also observed this deficiency.[132627]

The vocabulary in the brochures must be simple.[5] In our study, medical jargons were used. These terms, when used should be defined or explained.[29] The use of “jargon” without explanation is a barrier to communication.[30] Evaluation of the use of common vocabulary in the brochures showed that our study had only 2 (1.8%) that could be rated “superior” in comparison to 4 (14%) that were “superior” in another study.[23]

Several studies have found that written materials with graphics are well-received by the target audience[31] because they can help to reinforce messages.[28] In our study, approximately 68% brochures were rated as “not suitable” compared to 82% of the brochures in another study that required improvements to their illustrations.[28] In evaluating the cover graphics for each brochure, only 3 (2.7%) were rated “superior.” This is in contrast to another study in which the researchers rated the majority of cover graphics as “superior.”[23] Captions are helpful in conveying the message of the graphics.[26] In our study, only 10 brochures (9.1%) had captions to explain the information in the graphics, in contrast to another study in which there were captions to the illustrations of 24% of the written materials.[23]

Layout and design involves organizing the brochure in a manner that suits the audience.[28] Though the brochure must be visually appealing and well-formatted,[28] the main message can be obscured with the use of several bright colors and stylish borders.[32] In our study, 76.4% were “adequate” in typography, but in another study,[28] the font used in 51% of the brochures was appropriate. In our study, 25.5% of the brochures needed improvement in layout, while in the study by Arnold et al., only 14% needed such improvements.[28] Many brochures in our study had too much information under subheadings. Subheadings were required in 45.5% of our brochures to shorten the paragraphs and make them easier to understand, compared to 51% in another study that needed shortening of paragraphs.[28] Other researchers also had issue with the paragraphs, text-dense pages and the lack of subdivision of complex information into smaller pieces of written materials.[23262733]

In evaluating content for behavior modeling, 70% of our brochures were rated as “superior” or “adequate” in comparison to 58.6% of the materials which were scored “adequate” in another study.[23] In that same study, evaluation of the materials for motivation showed that 72.3% of the materials were “adequate” or “not suitable,” while all of our brochures (100%) fell into these two categories. In our study, as was also found by Weintraub et al. none of the brochures provided interactive learning stimulation.[23]

Cultural health practices vary among communities. Thus, cultural factors should be considered in designing health education materials.[13] Our brochures were culturally sensitive with no offensive illustrations or examples, as was also noted in other studies.[2328]

Limitations

This study was limited to the evaluation of health education brochures designed by government institutions. The brochures from private health care institutions, and those written in languages other than Arabic, were not evaluated. As the brochures were selected from Qassim province only, the findings of this study may not be generalized to cover those produced and disseminated in other provinces of Saudi Arabia. Moreover, certain criteria of SAM require subjective measurement.[20] Therefore, the evaluator's attitudes, cultural and professional background may have influenced the outcome of this evaluation.[39]

CONCLUSION AND RECOMMENDATIONS

To sum up, this study showed that although two-thirds of the health education brochures were “adequate” in terms of their overall suitability the majority needed improvements to be made to their graphics, learning stimulation and motivation. The lack of a summarized statement, and interactive features, insufficient use of graphics, the absence of instructions to model desired behavior, and a crowded layout were some of the weaknesses also found in other studies that used SAM to evaluate written materials.[813253334]

We recommend that guidelines for designing health education brochures should be established. The brochures should be pilot tested before dissemination. Moreover, health care workers who prepare the health education brochures should be properly trained for the purpose. The health care professionals should strive to ensure that the quality of the written education materials is appropriate for the target audience.