The Annual African American Conference on Diabetes: evolving program evaluation with evolving program implementation.

BACKGROUND: According to 2003 Behavioral Risk Factor Surveillance System data, South Carolina has the fourth highest rate of overall diabetes among the 50 states (9.3%) but the second highest rate among African Americans (15.5%). Nationwide, African Americans are disproportionately affected by diabetes. In addition, 40% of the African American population in South Carolina lives in a rural area, and approximately 26% live at or below the poverty level. Lack of access to health care and diabetes education are additional barriers for people with diabetes and their families. CONTEXT: Since 1997, the South Carolina Diabetes Prevention and Control Program and the Diabetes Today Advisory Council have sponsored the African American Conference on Diabetes, which targets African Americans with diabetes, their families, and their caregivers. This article describes the evolution of the conference and its evaluation.
METHODS: In 2002, we conducted focus groups with 20 African American conference attendees with diabetes to 1) assess the program's effects, 2) determine how to reach more individuals, and 3) improve programming. In 2004, we incorporated the preconference and postconference Diabetes Understanding Scale survey to assess the cognitive impact of the conference on participants. CONSEQUENCES: Focus group results revealed that participants wanted to attend the conference because of the opportunity to increase their knowledge and change their behaviors through 1) education, 2) social support, 3) resources, and 4) logistics. Self-rated understanding increased significantly after the conference for each cognitive understanding item on the Diabetes Understanding Scale.
INTERPRETATION: Focus group results suggested that participants who continue to attend the conference year after year may improve diabetes self-management skills. A quantitative evaluation showed that this 1-day diabetes education conference significantly increased short-term, self-rated cognitive understanding of diabetes behaviors.

Background

Diabetes is a significant public health problem that affects approximately 18 million people (1). The prevalence among adults is expected to double by 2025 (2). Diabetes is more prevalent among older people, and it disproportionately affects people in minority populations (3). Diabetes prevalence also differs geographically. According to 2003 Behavioral Risk Factor Surveillance System data, South Carolina has the fourth highest overall rate of diabetes among the 50 states (9.3%) (4) but the second highest rate among African Americans (15.5%). Nationwide, African Americans are disproportionately affected by diabetes (4).

Self-management is the cornerstone of diabetes care and treatment, yet most people with diabetes do not receive any formal self-management education (5). Diabetes education integrated into comprehensive diabetes care has effectively improved self-management and diabetes clinical outcomes (6). Regardless of race or ethnicity, diabetes and its complications can be controlled through early diagnosis and proper self-management (1). Studies show that intensive glucose control can prevent retinopathy, nephropathy, neuropathy, and microvascular complications among people with diabetes (7,8).

Because African Americans in South Carolina are disproportionately affected by diabetes, the South Carolina Diabetes Prevention and Control Program (SC DPCP) and the Diabetes Today Advisory Council (DTAC) have sponsored the African American Conference on Diabetes (AACD). Since 1997, the AACD has been convened to help educate African Americans with diabetes, their families, and their caregivers. The rigor of evaluation techniques to assess the effects of AACD has increased since 2002. This article describes the AACD's evolution and the simultaneous improvement in its evaluation.

Context

Diabetes in South Carolina

The goal of the Centers for Disease Control and Prevention's (CDC's) National Diabetes Prevention and Control Program is to help people with diabetes have long, healthy, satisfying lives (9). One of the CDC's national objectives is to reduce diabetes-related disparities among high-risk populations (9). The shared mission of the SC DPCP and DTAC is to prevent diabetes and its complications among African Americans through diabetes education and management and to make individuals aware of community resources.

South Carolina is a rural, medically underserved state with a significant diabetes problem among African Americans. In South Carolina, 40% of African Americans live in rural areas, with 26% of them living below the poverty level (10).

Specialized care is primarily available in areas with larger populations but is often inaccessible for poor individuals who live in rural areas (10). Barriers to diabetes self-management include the lack of funds or insurance to cover the cost of ongoing care, medicines, supplies, and diabetes self-management education. Among the 12 counties with a diabetes prevalence that is higher than the state average, two of the counties do not have even one certified diabetes educator (10). Furthermore, six South Carolina counties have a ratio of less than one certified diabetes educator per 10,000 people (10).

Before 1997, SC DPCP's efforts to educate rural African American communities about diabetes through health fairs and presentations in churches had limited exposure. In 1997, the SC DPCP and DTAC hosted the first annual AACD and focused on diabetes self-management. Organizers of the AACD hoped it would be a forum to provide diabetes education and resources to African Americans with diabetes and their families and caregivers across South Carolina. Organizers selected Columbia, the state capital, as the conference site because of its central location and accessibility. Since its inception, conference attendance has continued to increase almost every year; more than 1000 people attended in 2004.

Evolution of the AACD

The AACD is held each November during National Diabetes Awareness Month in Columbia, SC. In 1997, the AACD's first year, the conference was sponsored by two community-based agencies. By 2004, sponsorship funding had increased through educational grants from pharmaceutical companies and start-up money from private businesses. The funding increase allowed attendance to grow steadily from 195 participants in 1997 to 1044 in 2004, with a slight decrease in 2002 (Figure). The 2002 decrease in attendance coincided with the origination of a $5 registration fee; even this nominal fee excluded some people from participating. Since 2003, community partners have assisted people who needed help with the registration cost.

Figure

African American Diabetes Conference attendance, 1997–2004. The 2002 decrease in attendance was attributed to the origination of a $5 registration fee.

The AACD features educational encounter sessions (EESs), which are brief, didactic, skill-building sessions led by experts. AACD sessions educate participants about diabetes care practices (e.g., visiting primary care physicians, having a hemoglobin A1c test, monitoring blood glucose levels regularly) and encourage people with diabetes to adopt diabetes self-management skills and behaviors. At these sessions, participants share and learn not only from experts but also from each other. The SC DPCP and DTAC expanded their partnerships with health professionals and community volunteers to increase the number of people who could receive the AACD's resources.

In 2002, a session titled "Ask the Doctor" was incorporated into the conference. During the session, participants are allowed to question a panel of physician specialists: an internist, a dentist, an optometrist, a podiatrist, and a pharmacist. In a foot care session, a clinician examined participants' feet and then taught them how to examine their own feet. Participants were shown how to use glucometers and products that make blood glucose testing easier and less painful. In addition, they were taught the importance of taking prescribed medication, monitoring their blood glucose levels, and keeping records to share with their health care providers.

Sessions on physical activity focused on having fun while moving to music and burning calories. A nutrition session was designed to teach participants how to prepare healthy soul food that was low in sodium, fat, and sugar. Health and community organizations and exhibitors presented their resources and products.

New formats and topics for future AACD sessions evolve on the basis of participant feedback and interest. For example, the initial AACD format included morning plenary sessions. Because participants were reluctant to voice their concerns and ask questions in the plenary format, the sessions were replaced with the more intimate concurrent group EESs. Attendee feedback indicated a desire for more diverse diabetes-related topics and more EESs. For example, in 2002, a session about depression and diabetes was incorporated. Participants practiced relaxation techniques and were encouraged to talk with health care providers about their mental and emotional health. Also as a result of participant feedback, AACD offered screening stations for blood pressure, cholesterol levels, and kidney disease. The AACD is the only source of diabetes-related education and other resources for many of its participants.

Methods

The AACD's program evaluation became more rigorous as the conference became more complex. Before 2002, only participant registration records and feedback on individual sessions were evaluated. In 2003, a participant questionnaire was added to gauge satisfaction with the AACD programming. Each ensuing year, participant feedback was used formatively to plan the AACD programming.

The evaluation was conducted in two phases: phase 1 in 2002 and phase 2 in 2004. In 2002, program planners realized that the AACD conference had matured and that impact evaluation methods should be used to 1) assess the program's effect, 2) increase the number of people affected by the conference, and 3) improve programming. We selected the focus group method to gain a more comprehensive understanding of participants' thoughts and feelings about diabetes and the AACD.

Data from focus groups can provide insight into the cultural norms that shape diabetes self-management perspectives. A focus group can also create an interactive environment that allows participants to freely discuss issues (11). Qualitative and quantitative methods used in tandem work well for evaluating and planning educational interventions (12). In 2004, the evaluation included a quantitative assessment to determine whether attending the AACD improved participants' basic understanding of diabetes and awareness of important self-management skills.

Phase 1: focus groups

The CDC gave the SC DPCP a nonresearch determination for program evaluation in public health practice, so no Institutional Review Board approval was required for data collection.

Recruitment

Focus group participants were selected from the AACD registration forms, which included a question about participants' previous AACD attendance. Results showed that 70 registrants had attended at least two previous conferences. These registrants were contacted by telephone to confirm their prior attendance and to determine whether they 1) wanted to participate in a focus group and 2) had been diagnosed with diabetes. The majority of registrants were willing to participate; most who were excluded did not meet the disease status requirement (i.e., did not have diabetes). Of the 70 contacted, 28 met the inclusion criteria and were invited to participate. Of the 28 potential participants, two declined to participate because they were not planning to attend the 2002 conference. Later, six additional people dropped out because they were unable to attend the AACD. The remaining 20 participants were assigned to group 1 or group 2 based on the numerical order in which their names were listed on the original list of 28 eligible registrants. Odd numbers were assigned to group 1 (n = 12) and even numbers to group 2 (n = 8).

Facilitation

Two independent, 1-hour focus groups were conducted at the 2002 AACD. Before each focus group, participants wrote on paper their demographic data and diabetes history. Each participant received $20 at the end of the session.

The facilitator told the participants that the focus group was being used as a program evaluation tool to improve the AACD. Participants were assured that their responses would be confidential, told that participation was voluntary, and told that their continued participation would be considered permission to report the aggregate information to stakeholders.

The facilitator was an African American certified health education specialist trained in focus group facilitation and experienced in diabetes prevention and control. The same structured discussion guide (designed to be flexible to allow probing for clarification) was used for each session to ensure that the presentations were consistent. During each session, a staff member wrote the themes of the conversation on a flip chart so that participants could review and validate their responses. Two staff members took notes on the responses and another person audiotaped the sessions.

Analysis

Audiotapes were transcribed verbatim, and the content was analyzed to find recurring themes. The transcription's accuracy was confirmed by comparing it with the field notes. Two individuals coded the transcribed records for themes, one of whom had no previous involvement with the evaluation or program. Both coders were experts in diabetes prevention and control and experienced in qualitative analysis. The coders discussed the themes and came to a consensus before issuing the report. Because no significant demographic differences existed between the two focus groups, we reported aggregated results.

Phase 2: quantitative measurement of diabetes understanding

In 2004, conference planners expanded the evaluation by gathering data on the short-term effect of the AACD educational sessions on participants' understanding of diabetes and its treatment.

Data collection instrument

A modified 13-item Diabetes Understanding Scale was developed from section IV of the Diabetes Care Profile (DCP), an instrument for assessing understanding of diabetes and its treatment (13). The scale addresses topics presented during the AACD. The scale was modified by combining the diet, exercise, and medication items into one item and eliminating a diabetes and pregnancy item because no sessions were held on this topic. Two items on prevention and treatment of high "blood sugar" and low "blood sugar" were presented as the following four items: 1) prevention of high blood sugar, 2) prevention of low blood sugar, 3) treatment of high blood sugar, 4) and treatment of low blood sugar. The item alterations allowed evaluators to consider separately changes in participant understanding of prevention and treatment for high and low blood sugar.

Administration

All participants at the 2004 AACD were invited to participate in the program evaluation by completing the preconference and postconference surveys in their registration packets. Unique identifiers on the forms allowed us to match the participants' preconference forms with their postconference forms. During the presession breakfast, an introduction to the evaluation process was presented. Confidentiality was assured, and participants' completed surveys were considered consent to use their data in an aggregate form.

Participants completed and submitted the preconference diabetes understanding survey and a short demographic questionnaire before the morning sessions began. After the closing session, participants completed the postconference diabetes understanding survey. To encourage participation, participants received free raffle tickets, which they could only turn in with their completed evaluation forms. Of the 1044 attendees, 628 attendees completed at least one of the surveys and provided useable data. Fewer postconference evaluation forms were completed than preconference forms because many participants left the conference before filling out the postconference survey. Using a 5-point Likert scale (with 1 = poor, and 5 = excellent), respondents ranked how well they understood specific diabetes issues. The data were entered into an EpiData version 3.02 database (EpiData Association, Odense, Denmark) and exported as a SAS file (SAS Institute Inc, Cary, NC).

Survey data were analyzed using SAS version 9.1. Descriptive statistics were used to describe the respondent sample by sex, race, age, and diabetes status. The mean survey scores, standard error, and number of respondents for each subscale item preconference and postconference were calculated. Cronbach ? for internal consistency was computed for the preconference subscale items. Proc Ttest was used to examine significant differences in participants' mean survey scores, which had been matched by repeated measures at preconference and postconference (by unique identification numbers).

Consequences

Twenty adults, predominantly African American women, participated in two focus groups (Table 1). Participants freely engaged in discussions, sharing personal and family stories about their diabetes experiences. The devastation caused by diabetes in their families and concern for preventing diabetes and its complications among loved ones dominated the conversations. Table 2 includes direct quotes for each theme.

Table 1

Demographic Characteristics of Focus Groups, 2002 African American Conference on Diabetes, Columbia, SC

Characteristic	Group 1 (n = 12)	Group 2 (n = 8)	Paa	Overall
Race or ethnicity, no. (%)
African American	12 (60.0)	8 (40.0)	NA	20 (100.0)
Sex, no. (%)
Male	2 (16.7)	1 (12.5)	.80	3 (15.0)
Female	10 (83.3)	7 (87.5)		17 (85.0)
Age, y
Mean (SD)	59.5 (10.42)	55.9 (12.55)	.96	58.1 (11.15)
Range	38-78	33-72		33-78
Years with diabetes
Mean (SD)	10.5 (7.39)	7.6 (5.01)	.40	9.4 (6.56)
Range	0.5-27.0	2.0-17.0		0.5-27.0
Education, y
Mean (SD)	14.9 (3.48)	14.8 (2.44)	.94	14.9 (3.03)
Range	8-18	10-18		8-18
Years of conference attendance
Mean (SD)	3.2 (0.94)	3.9 (0.99)	.11	3.5 (1.00)
Range	2-5	3-5		2-5

P values were calculated from z scores.

Table 2

Key Focus Group Themes and Comments, 2002 African American Conference on Diabetes, Columbia, SC

Themes	Participant Comments
Education and learning	"To gain understanding on a misunderstood disease . . ."
	"To find out how to manage my diabetes effectively"
	"I learn to watch my diet better . . . ."
	"Learn how to prepare nutritious foods"
	"Not aware that Medicare and Medicaid would take care of buying shoes and inserts for diabetics"
	"Helped me to realize the dangers of diabetes"
	"I come to get as much new information as I can get."
	"It makes me aware of my blood sugar . . . ."
	"I've learned to ask questions and be involved."
	"I've learned to check my feet . . . every day."
Social support	"To learn how to take care of my daughter from her head to feet."
	"Be a support to others with diabetes"
	"It is good to have family and other people to talk to . . . ."
	"This conference acts like a support group."
	"Reinforcement"
	"For self-control"
	"I have a partnership with my doctor to be responsible for my disease . . . ."
	"I've gained a feeling of empowerment . . . ."
Resources	". . . I have come because of resources . . . ."
	"We like the vendors and the information."
	"Diabetes supplies"
	"The 'Ask the Doctor' sessions were excellent."
	"The literature is in layman's terms . . . I can understand."
Logistics	"Make sure conference is publicized well ahead of time."
	"Provide transportation for disabled."
	"More vendors . . . "
	"Start sessions on time. Eliminate technical problems."
	"Streamline registration . . . ."
	"Provide a better breakfast and lunch."
	"Consider alternating cities."
	". . . give flu shots."

Motivation for participation

When participants were asked, "What inspired you to participate in this conference over the years?" education and learning was the most frequently reported theme among the answers. Participants also frequently cited family and communication issues — a social support theme. They reported that the AACD provided a unique opportunity to meet and talk to others with similar problems.

Logistics

Participants were also asked about which components of the conference were helpful and should be continued in future conferences. Again, the themes of education, social support, and information about resources emerged in their answers. When asked about items that needed improvement, their answers focused on logistics (e.g., the setting, programming issues, exhibitors, conference amenities). Participants were most concerned about the conference center's limited space and getting the registration information early. Participants also expressed a desire to be involved in the conference planning.

Knowledge and behavior change

When asked what they would have done differently as a result of the conference to handle their diabetes, participants said that they were more confident in their ability to manage their diabetes because of what they learned at the conference. They were better able to use the social support systems in their communities and families, making them more effective at performing self-care tasks and more willing to seek professional health care when needed. Access to resources (human and material) was a third theme that emerged. Participants found the information about resources useful.

Phase 2: diabetes understanding

In 2004, 36% of AACD attendees responded to all of the items on the preconference and postconference surveys. Table 3 includes a description of the survey participants. Table 4 presents preconference-to-postconference score changes for each item, the mean number of respondents, and the significance levels for change by item.

Table 3

Characteristics of Participants, Diabetes Understanding Surveys, 2004 African American Conference on Diabetes, Columbia, SCa

Characteristic	No. (%)
Race or ethnicity
African American	556 (88.5)
White	28 (4.4)
Asian	5 (0.8)
American Indian or Alaska Native	5 (0.8)
Other	6 (1.0)
Data missing or unknown	28 (4.4)
Sex
Female	510 (81.2)
Male	87 (13.9)
Data missing or unknown	31 (4.9)
Age, y
<45	115 (18.3)
45-60	235 (37.4)
>60	251 (38.4)
Data missing or unknown	27 (4.3)
Diabetes status
Has diabetes	360 (57.3)
Does not have diabetes	241 (48.3)
Data missing or unknown	27 (4.3)

628 of 1044 conference attendees completed the preconference survey, the postconference survey, or both surveys.

Table 4

Preconference to Postconference Changes in Self-rated Diabetes Understanding Scale Scores, 2004 African American Conference on Diabetes, Columbia, SC

Diabetes Understanding Scale Item	No. Respondents to Item on Both Surveys	Likert Scale Paired Mean Scoresa		Mean Change in Score (SE)	t Test(df)	P
		Preconference Survey	Postconference Survey
Overall diabetes care	390	3.6	4.2	0.6 (0.05)	11.15 (389)	<.001
Coping with stress	347	3.4	4.0	0.6 (0.06)	10.82 (346)	<.001
Diet and blood sugar	366	3.6	4.2	0.6 (0.06)	10.11 (365)	<.001
Exercise and blood sugar	368	3.7	4.3	0.6 (0.05)	9.07 (367)	<.001
Medication and blood sugar	367	3.6	4.2	0.6 (0.05)	8.45 (366)	<.001
Blood sugar monitoring	356	3.7	4.2	0.5 (0.06)	8.25 (355)	<.001
Prevention of high blood sugar	381	3.5	4.1	0.6 (0.05)	10.23 (380)	<.001
Prevention of low blood sugar	360	3.4	4.1	0.7 (0.06)	11.32 (359)	<.001
Treatment of high blood sugar	368	3.6	4.1	0.5 (0.05)	8.53 (367)	<.001
Treatment of low blood sugar	343	3.5	4.1	0.6 (0.06)	10.28 (342)	<.001
Prevention of long-term complications	356	3.4	4.3	0.9 (0.06)	11.87 (355)	<.001
Foot care	363	3.5	4.3	0.8 (0.06)	9.84 (362)	<.001
Benefits of blood sugar control	380	3.7	4.2	0.5 (0.05)	9.84 (379)	<.001

Diabetes Understanding Scale scores were based on a 5-point Likert scale, with 1 = poor understanding and 5 = excellent understanding.

The mean item scores of participants who completed the preconference and postconference surveys indicate a significant increase in self-rated understanding for each item on the scale. Internal consistency analyses of the preconference data revealed that the scale was highly reliable (Cronbach α = 0.96).

Interpretation

In 1997, the AACD was a small conference with fewer than 200 attendees and limited sponsorship, resources, and marketing ability. It has evolved into a program with more than 1000 attendees and greater sponsorship, resources, and marketing. As the AACD has evolved, so too has the quality of its programming, stakeholder expectations, and evaluation rigor. Qualitative and quantitative methods were used to effectively evaluate and plan this educational intervention.

Qualitative findings from focus groups suggest that participants at the AACD were motivated to attend the EESs because they received quality diabetes education, social support, and resources in an inviting, interactive environment. They felt empowered to help others manage diabetes. They also reported that previous AACD attendance improved their ability to adopt effective diabetes self-care practices.

The participants' perspectives were used in planning the logistics of each succeeding AACD. In 2004, the program site was relocated to a larger convention center to provide more space and address the logistical issues identified in 2002. To let more people know about the conference and allow more people to register early, the organizers marketed the conference through multiple media channels. The AACD continued to offer an array of topics on diabetes self-management and expanded the variety of concurrent EESs to provide participants more opportunities to learn about different topics.

The AACD has been providing diabetes education and resources to the community since 1997; however, 2004 was the first year that the cognitive impact of the AACD on participants was evaluated. Quantitative findings from 2004 suggest that the AACD conference format with EESs improved participants' self-reported understanding of diabetes self-management. The survey reliability measures were high and similar to those found in other studies (14,15). Examination of preconference to postconference changes in self-rated understanding suggests that the AACD had a significant effect on diabetes-related understanding, at least in the short term.

Increased conference attendance and increased understanding about diabetes self-management does not necessarily lead to behavior change, which is the primary purpose of the AACD. Although focus groups suggest that the behavior change occurred among multiyear attendees with diabetes, focus groups cannot be used to determine the actual proportion of participants who changed their behavior. Understanding how the AACD benefits stakeholders and participants is important for planners of future programs, as is determining how to transform program evaluation into behavioral research. As the rigor of evaluation techniques increases, new research questions will emerge.

The evolutionary process described in this article is an example of the way that program evaluation not only improves programming but also plays a role in behavioral research. From the AACD evaluation, important research questions have already emerged:

How much of the target audience can we reach using the AACD format?

How can we recruit more attendees?

What is the long-term effect of the AACD on participants' diabetes-related knowledge, attitudes, and behaviors? How can we measure the effects?

As the next step in the evolution of the AACD's program evaluation plan, we are considering using the Behavioral Risk Factor Surveillance System diabetes module to track the behavior of a sample of individuals who attend the conference each year.